Wednesday, February 3, 2016

Using wso2/puppet-modules to deploy WSO2 Products

This article will provide an insight to setup a standalone product using wso2/puppet-modules [1] for the given productIt will provide a sequence for you to follow in testing your WSO2 product of interest for a standalone deployment setup in a given puppet agent instance. 
This can be considered as a guide to setup your own puppet master/agent environment for testing given that you are going to manually setup the server instances.

3 Tasks
  1. Setup two instances
  2. Setup puppet master and puppet agent
  3. Perform a catalog run to setup product instance

Task 1 - Setup two instances
This can be either two EC2 instances, two WSO2 IaaS instances, or two VM instances.
e.g.: 192.168.19.33 - puppet master
        192.168.19.35 - puppet agent

Task 2 - Setup puppet master and puppet agent
Steps to install and configure puppet master and agent are as follows.
Installing puppet
Puppet master
  • Login to puppet master as the super user
  • Issue the following commands.
> ntpdate pool.ntp.org ; apt-get update && sudo apt-get -y install ntp ; service ntp restart
> cd /tmp
> dpkg -i puppetlabs-release-trusty.deb
> apt-get update
> apt-get install puppetmaster
  • Check the puppet version as: puppet -V
Puppet version Should be 3.8.3 or higher
  • Edit etc/hostname file and set property hostname
puppetmaster
  • Edit your puppet master hosts file /etc/hosts as follows and set puppet master hostname
127.0.0.1 localhost
127.0.0.1 puppetmaster

Puppet agent
  • Login to puppet agent as the super user
  • Issue the following commands.
> apt-get update
> apt-get install puppet
  • Edit etc/hostname file and set property hostname
e.g.: qaa-node-1
  • Edit /etc/hosts file of your puppet agent
127.0.0.1 localhost
192.168.19.XXX puppet # puppet master IP address


Configuring puppet
Puppet master
  • Login to puppet master as the super user
  • Create a directory /etc/puppet/environment/production/
  • Modify /etc/puppet/puppet.conf file of puppet master appending the following to [main] and [master] sections accordingly.
[main]
dns_alt_names=puppetmaster,puppet
environmentpath = $confdir/environments
[master]
autosign=true
  • Before restarting the puppet master, clean all certificates, including puppet master’s certificate which is having its old DNS alt names.
> puppet cert clean --all
  • Restart the puppet master for new configuration changes to take effect and to regenerate the certificate with the new dns_alt_names
> service puppetmaster restart

  • Download git repo files
git clone wso2/puppet-modules [1] to a temp folder
  • Prepare puppet modules
    • Copy all modules in ../puppet-modules/modules to /etc/puppet/environment/production/modules/
    • Install java module from puppet forge [2].
> puppet module install 7terminals-java
    • Install stdlib module from puppet forge [3].
> puppet module install puppetlabs-stdlib
  • Prepare hieradata
    • Copy ../puppet-modules/hiera.yaml to /etc/puppet
    • Copy ../puppet-modules/hieradata folder to /etc/puppet/
    • Rename /etc/puppet/hieradata/dev -> /etc/puppet/hieradata/production
  • Prepare site.pp manifest
    • Copy ../puppet-modules/manifests/site.pp to /etc/puppet/environment/production/manifests/
  • Prepare files/packs
    • Copy the pack file (e.g.: wso2am-1.9.1.zip) to ../modules/wso2am/files/
    • Create folder ../modules/wso2base/files
    • Copy jdk installation file (e.g.: jdk-7u79-linux-x64.gz) to ../modules/wso2base/files
    • Add jdk version (home) and filename information to /etc/puppet/hieradata/production/common.yaml

Puppet agent
  • Login to puppet agent as the super user
  • Modify /etc/puppet/puppet.conf file of puppet agent appending the following to the [main] section.
[main]
server = puppet

Task 3 - Set Facter variables and perform a puppet agent run.

Following sample files attached are for APIM 1.9.1. Modify the deployment.conf file according to your product.
  • Login to puppet agent as super user.
  • Copy the file deployment.conf [4] -> /opt/deployment.conf
  • Copy the file setup.sh [5] -> /opt/setup.sh
> chmod 755 setup.sh
  • Run following command to setup product instance.
> ./setup.sh

An instance of a standard WSO2 product of your choice will be installed.
e.g.: 
CARBON_HOME ->  /mnt/10.0.2.89/wso2am-1.9.1
For APIM 1.9.1 the complete deployment time (deployment + server startup) will be approximately 1 minute.

If you are planning to go ahead with Vagrant setup, i.e. to spawn VM instances please refer [6].


[4] Attachment: deployment.conf
product_name=wso2am
product_version=1.9.1
product_profile=default
vm_type=openstack

[5] Attachment: setup.sh
#!/bin/bash
echo "#####################################################"
echo "                   Starting cleanup "
echo "#####################################################"
ps aux | grep -i wso2 | awk {'print $2'} | xargs kill -9
#rm -rf /mnt/*
sed -i '/environment/d' /etc/puppet/puppet.conf
echo "#####################################################"
echo "               Setting up environment "
echo "#####################################################"
rm -f /etc/facter/facts.d/deployment_pattern.txt
mkdir -p /etc/facter/facts.d

while read -r line; do declare  $line; done < deployment.conf

echo product_name=$product_name >> /etc/facter/facts.d/deployment_pattern.txt
echo product_version=$product_version >> /etc/facter/facts.d/deployment_pattern.txt
echo product_profile=$product_profile >> /etc/facter/facts.d/deployment_pattern.txt
echo vm_type=$vm_type >> /etc/facter/facts.d/deployment_pattern.txt

echo "#####################################################"
echo "                    Installing "
echo "#####################################################"

puppet agent --enable
puppet agent -vt
puppet agent --disable