Python load configurations from a file

Say you have a file deployment.cfg with configurations which are nicely grouped in sections.

[environment]

env = openstack

[envconfig]

image = suhan-daf-agentv4-ubuntu14.04

flavor = m2.small

network = qaa-net

instancePassword = suhan

keyPair = mycloudkey

[nodes]

appserver-mgr2

appserver-wkr2

This is how you should read this configuration file in python.

import ConfigParser
import collections

# Global variables
config = ConfigParser.RawConfigParser(allow_no_value=True)

# Get environment
def get_environment():
config.read('deployment.cfg')
return config.get('environment', 'env')

# Load environment configuration
def get_openstack_image():
config.read('deployment.cfg')
return config.get('envconfig', 'image')

.
.
.

# Nova credentials are loaded from OS environmental variables
def load_server_config():
serverList = []
config.read('deployment.cfg')
orderedDic = collections.OrderedDict(config.items('nodes'))
for node, ip in orderedDic.iteritems():
    serverList.append(node)
print serverList
return serverList

Reference:
[1] http://stackoverflow.com/questions/27483096/python-config-parser-array-sort-according-to-file-content-sequence

Puppet fill template using a central configuration file inside a module

Sometimes we need to keep all the configuration information in a central configuration file. It is easy to maintain and gives a clear idea of the scenario.

This article provides guidance on how to keep this kind of a central file and fill template values during a puppet catalog run in agent node.

Arrays of data which will be looped inside template filling process is also demonstrated in this example.

In the current tested scenario, the hostname of the puppet agent node is appserver-wkr

Configuration file is params.pp which resides inside module's manifests folder.

This file is inherited by init.pp of the module which is having a class with the same name as the module's (appserver).

In order to push a configuration in init.pp file we simply include the module in site.pp folder.

When a puppet agent find the master and requests for a catalog run, master will map the hostname with the relevant node configuration in site.pp file and apply it to the agent node (if no match found apply the default node configuration).

To have more clear idea, see the graphical illustration below followed by code content of each relevant file.

Following are the /etc/puppet tree structure in puppet master node and its file contents.

├── manifests

│   └── site.pp

└── modules

   └── appserver

       ├── manifests

       │   ├── init.pp

       │   └── params.pp

       └── templates

           └── polite-file.erb

1. site.pp

import 'appserver'

node 'appserver-wkr' {

       include appserver

}

node default {

}

2. init.pp

class appserver inherits appserver::params{

       file {  "/tmp/$myname":

               ensure  => file,

               content => template('appserver/polite-file.erb'),

       }

}

3. params.pp

class appserver::params ($say_hello_to = 'guys and gals') {

       #$say_hello_to = 'guys and gals'

       $myname = 'welcome file.xml'

       $serverOptions     = '-DworkerNode=true'

       $subDomain         = 'worker'

       $members           = ['10.0.1.196', '10.0.1.198', '10.0.1.200', '10.0.1.202']

}

4. polite-file.erb

<% if @say_hello_to -%>

Hello <%= @say_hello_to %>,

<% end -%>

I'm <%= @myname %>, on a <%= @operatingsystem %> system, nice to meet you.

<members>

         <%- if @members -%>

# loop hostnames

         <%- @members.each do |hostname| -%>

         <member>

          <hostName><%= hostname %></hostName>

            <port>4100</port>

         </member>

         <%- end -%>

         <%- end -%>

</members>

When all the files are in place simple perform a puppet catalog run in the agent node.

> puppet agent -t

Following is the created file (i.e. /tmp/welcome file.xml) contents in agent node.

Hello guys and gals,

I'm welcome file.xml, on a Ubuntu system, nice to meet you.

<members>

<member>

           <hostName>10.0.1.196</hostName>

           <port>4100</port>

</member>

<member>

           <hostName>10.0.1.198</hostName>

           <port>4100</port>

</member>

<member>

           <hostName>10.0.1.200</hostName>

           <port>4100</port>

</member>

<member>

           <hostName>10.0.1.202</hostName>

           <port>4100</port>

</member>

</members>

What if you need to parameterize the port value also?
Here is how to do it...

Re-define members as follows in params.pp file (data in hash form [2])
$members = { '192.168.1.156' => '4100',

             '192.168.1.157' => '4000' }

template file (polite-file.erb) members section is to be modified as follows.
<members>
   <%- if @members -%>
   <%- @members.each_pair do |hostname,port| -%>
      <member>
           <hostName><%= hostname %></hostName>
           <port><%= port %></port>
      </member>
   <%- end -%>
   <%- end -%>

</members>

Created new file contents will be as follows.

Hello guys and gals,

I'm welcome file.xml, on a Ubuntu system, nice to meet you.

        <members>
                <member>
                        <hostName>192.168.1.156</hostName>
                        <port>4100</port>
                </member>
                <member>
                        <hostName>192.168.1.157</hostName>
                        <port>4000</port>
                </member>

        </members>

Related stackoverflow questions.

[1]http://stackoverflow.com/questions/27183036/puppet-fill-templates-with-ip-addresses-of-newly-spawned-instances

[2]http://stackoverflow.com/questions/27222189/puppet-iteration-with-2-dimensional-array-to-fill-template

[3]http://stackoverflow.com/questions/27125371/puppet-catalog-run-error-error-400-on-server-not-authorized-to-call-find-on-p