One of the features offered by Puppet is the ability to break up infrastructure configuration into environments. With environments, you can use a single Puppet master to serve multiple isolated configurations. For instance, you can adopt the development, testing and production series of environments embraced by a number of software development life cycles and by application frameworks such as Ruby on Rails, so that new functionality can be added incrementally without interfering with production systems. Environments can also be used to isolate different sets of machines. A good example of this functionality would be using one environment for web servers and another for databases, so that changes made to the web server environment don’t get applied to machines that don’t need that configuration.

There are two servers that will be used in this guide:

1. puppet.example.com: the puppet master server
2. git.example.com: the GitLab server

NOTE: This will not manage the puppet master configurations directly, but rather the environments associated with each branch of the mail git repository.

Prepare the Local Computer environments¶

• Log into the GitLab web interface and create a new Project for the web service puppetry to be stored
  • This example uses Web Service Puppetry

• Create an repository to hold the configurations:
  

  mkdir -p ~/git/web-service-puppetry && cd ~/git/web-service-puppetry
  

• Initialize the repository:
  

  git init
  

• Create a README, add it to the local repository, then commit the local changes to be pushed to the remote git server
  

  touch README
  git add README
  git commit -m 'first commit'
  

• Add the remote git server location to the local repository
  

  git remote add origin ssh://git@git.altservice.com/SecretUser/web-service-puppetry.git
  

• Push the first commit to the remote git server
  

  git push -u origin master
  

Mapping the Puppet code base against the environments shows the power of this method. People often use a version control system to manage the code, and create a set of branches that each map to an environment.

Creating and using the environments¶

• Show the current branch
  

  git branch
  

• master

Create the environment branches¶

PRODUCTION BRANCH

• Create the production environment branch:
  

  git checkout -b production
  

  • Will result in:
    

    Switched to a new branch 'production'
    

• Populate a bare site.pp for the production environment
  

  mvim site.pp
  git add site.pp
  git commit -m 'Created the initial site.pp for the production environment'
  

  • Will result in:
    

    [production 25a9e1b] Created the initial site.pp for the production environment
    1 files changed, 1 insertions(+), 0 deletions(-)
    

    
    NOTE: I migrated my existing puppet master configurations into to production branch, and then used these files for the testing and development later:
    

    git add --all
    git commit -m 'Migrated existing puppet master configuration'
    

• Push the changes to the git server
  

  git push origin production
  

  • Will result in:
    

    Counting objects: 5, done.
    Writing objects: 100% (3/3), 255 bytes, done.
    Total 3 (delta 0), reused 0 (delta 0)
    Unpacking objects: 100% (3/3), done.
    remote: Creating new environment production
    To git@git.host:deploy.git
    * [new branch]      production -> production
    

DEVELOPMENT BRANCH

• Create the development environment branch:
  

  git checkout -b development
  

  • Will result in:
    

    Switched to a new branch 'development'
    

• Populate a bare site.pp for the development environment
  

  mvim site.pp
  git add site.pp
  git commit -m 'Created the initial site.pp for the development environment'
  

  • Will result in:
    

    [development 25a9e1b] Created the initial site.pp for the development environment
    1 files changed, 1 insertions(+), 0 deletions(-)
    

• Push the changes to the git server
  

  git push origin development
  

  • Will result in:
    

    Counting objects: 5, done.
    Writing objects: 100% (3/3), 255 bytes, done.
    Total 3 (delta 0), reused 0 (delta 0)
    Unpacking objects: 100% (3/3), done.
    remote: Creating new environment development
    To git@git.host:deploy.git
    * [new branch]      development -> development
    

TESTING BRANCH

• Create the testing environment branch:
  

  git checkout -b testing
  

  • Will result in:
    

    Switched to a new branch 'testing'
    

• Populate a bare site.pp for the testing environment
  

  mvim site.pp
  git add site.pp
  git commit -m 'Created the initial site.pp for the testing environment'
  

  • Will result in:
    

    [testing 25a9e1b] Created the initial site.pp for the testing environment
    1 files changed, 1 insertions(+), 0 deletions(-)
    

• Push the changes to the git server
  

  git push origin testing
  

  • Will result in:
    

    Counting objects: 5, done.
    Writing objects: 100% (3/3), 255 bytes, done.
    Total 3 (delta 0), reused 0 (delta 0)
    Unpacking objects: 100% (3/3), done.
    remote: Creating new environment development
    To git@git.host:deploy.git
    * [new branch]      testing -> testing
    

• Now check to see the environment branches:
  

  git branch
  

  • Will result in:
    

      development
      master
      production
    * testing
    

• To switch between each branch just run, adjusting accordingly:
  

  git checkout production
  

And from here on out, you can use the production environment on your hosts, and use git like you would with any code base.

This development model gives us some simple access control. Utilizing access control with a tool like GitLab, we can allow people to generate new environments to test their own code, but deny them access to change the production environment. This allows us to institute some sort of change control, by requiring all code to be reviewed by a merge master before inclusion into production, and allows code to be tested and verified before the request for submission is made.

Prepare the Environments on the Puppet Master Server¶

• Create the environments directory on the puppet master server:
  mkdir /usr/local/etc/puppet/environments

• Adopting the development, testing and production workflow, we can have a puppet.conf that looks something like this:
  

  vi /usr/local/etc/puppet.conf
  

  • And should look something like the following
    

    [main]
      server = puppet.example.com
      environment = production
      confdir = /usr/local/etc/puppet
    [agent]
      report = true
      show_diff = true
    [production]
      manifest = /usr/local/etc/puppet/environments/production/manifests/site.pp
      modulepath = /usr/local/etc/puppet/environments/production/modules
    [testing]
      manifest = /usr/local/etc/puppet/environments/testing/manifests/site.pp
      modulepath = /usr/local/etc/puppet/environments/testing/modules
    [development]
      manifest = /usr/local/etc/puppet/environments/development/manifests/site.pp
      modulepath = /usr/local/etc/puppet/environments/development/modules
    

This will set 3 different environment:

1. production: This is for production ready systems, all bugs and changes should be made in testing or development for environments before being applied to production.
2. testing: This is for testing certain features that may be broken or needs more work before entering the production environment.
3. development: This is for developing features, fixing bugs, and/or modifying themes before entering the testing environment.

With this configuration, we could map three Git branches for these environments and set up a central Git repository with post receive hooks. When changes were pushed to this repository, they would be automatically deployed to the puppet master. The example post-receive hook later in this post will work with this kind of environment setup.

An example will help clarify. Suppose I’m working on a new set of functionality, and don’t want to touch the current set of Puppet modules and inadvertently cause change in production.

I can create a testing branch like so:
git clone git@git.example.com/SecretUser/web-services-puppetry.git

cd puppet 
git branch testing 
git checkout testing

• ... make some edits
  

  git add * 
  git commit -m "made some edits" 
  git push origin testing
  

• and then run my Puppet agent against this new testing code:
  

  puppet agent --test --environment=testing
  

Info: Retrieving plugin
Info: Caching catalog for sun.local
Info: Applying configuration version '1371740640'

As you can see, I set a default environment of production, and then specify paths to the manifest and modulepath directories, using the $environment variable to dynamically populate the path. Production manifest and modulepath paths will end up being $confdir/environments/production/manifests/site.pp and $confdir/environments/production/modules respectively. As new environments are dynamically created, the $environment variable will be substituted as appropriate.

• Next, I moved my existing Puppet module and manifest structure around to suit the new configuration:
  

  mkdir -p /usr/local/etc/puppet/environments/production
  mv /usr/local/etc/puppet/manifests /usr/local/etc/puppet/modules /usr/local/etc/puppet/environments/production
  

• And restart nginx (as I run my puppetmaster under Nginx/Passenger):
  

  service nginx restart
  

• I then ran a couple of agents to ensure everything was still working:
  

  puppet agent --test
  

They defaulted, as expected, to the Production environment.

• Next, install git on the puppet master:
  

  pkg install git
  

• Next, create a local git repository from the existing Puppet configuration:
  

  cd /usr/local/etc/puppet/environments/production
  git init
  git add *
  git commit -m "Initial import of Production Puppet repository" 
  

• Now add the remote GitLab repository:
  

  git remote add ssh://git@git.example.com/SecretUser/web-services-puppetry.git
  

• And push the committed initial configuration to the remote GitLab repository:
  

  git push origin production
  

Continuous Integration of GitLab and Puppet Master¶

After many hours of trial and error, and still not being able to get a functioning post-receive script to work, I decided to modify the above steps to have each environment have its own branch in the web services repo.

• With the production environment being continually updated using a cron script:
  

  crontab -e
  

• And then added the following to the end:
  

  ## Pull latest puppetry from GitLab
  */2     *       *       *       *       ssh-agent bash -c 'ssh-add /root/.ssh/id_ecdsa; cd /usr/local/etc/puppet/environments/production; /usr/local/bin/git fetch --all; /usr/local/bin/git reset --hard production; /usr/local/bin/git pull origin production && service nginx restart'
  

• Add root to the cron allow file
  

  vi /var/cron/allow
  

  • And add the following
    

    root
    

• Then restart the cron service to enable the functionality immediately:
  

  service cron restart
  

(Optional) Adding Continuous Integration to Puppet Master Files¶

I manage many of my services with raw configuration files (until I can learn to manage templates via .erb files), so adding those configurations to either their own branch or even their own repository entirely helps me out.

Resources¶

• http://puppetlabs.com/blog/git-workflow-and-puppet-environments
• https://groups.google.com/forum/#!topic/gitlabhq/WWUox4MLe2U
• https://www.digitalocean.com/community/tutorials/how-to-set-up-automatic-deployment-with-git-with-a-vps
• http://dreamingwell.com/articles/archives/2014/02/using-git-and-puppet-for-agile-server-management.php
• http://krisjordan.com/essays/setting-up-push-to-deploy-with-git
• http://projects.puppetlabs.com/projects/1/wiki/puppet_version_control