Managing a testing environment with Vagrant and Chef-Solo

A few weeks back I wrote about my endeavour to reinstall a private server that I own using the configuration management tool Chef. One of the things that I wanted to revisit was the way I managed a test environment. I had basically manually set up a virtual machine and that I could use as a configuration target for my Chef cookbooks. This worked well, but I already knew at the time I was developing this that I really should have another look at Vagrant, a tool that is specifically designed for the task to manage virtual machines and that integrates easily with Chef Solo. Again, I had some previous knowledge from work projects but it’s been a while. Finally, I found the time to work on this and this post will document what I had to do to tie Vagrant and my Chef cookbooks together.

Installation and Basics

It’s easy to  install Vagrant, just download the installer for your operating system, execute it and you’re ready to go. Now a new Vagrant project has to be initialized:

$ mkdir bigwhoop-vagrant
$ vagrant init ubuntu/trusty64
$ vagrant up

For initialization I have used Ubuntu 14.04 (ubuntu/trusty64), mirroring my target live system. You can choose from other pre-built boxes for example here. The init command will import the selected box and create a new file called Vagrantfile in our project folder, which we’ll customize later. The third command will start a new VM. Vagrant supports a number of providers, I’m going with the default, which is VirtualBox.

Vagrant provides a number of command line parameters, initially we need to know about these:

init - initialize a Vagrant project
up - start a VM
ssh - login to a VM using SSH
halt - stop a VM
destroy - completly destroy a VM
provision - apply the provided configuration to the VM

If you want to, you can login to the VM using the ssh command. After that, since we have not tied the Vagrant project to the cookbooks, yet, we can shutdown the VM by using the halt parameter. Vagrant has a pretty nice documentation which is worth visiting, if you want to learn more.

Matching up Vagrant with Chef Solo

In order to have the VM configured using Chef Solo, we have to make a change to the Vagrantfile. Look for the config.vm.provision statement and change it to use chef_solo:

config.vm.provision :chef_solo do |chef|

  chef.data_bags_path = "../bigwhoop-chef-repo/data_bags"
  chef.cookbooks_path = "../bigwhoop-chef-repo/cookbooks"

  chef.add_recipe "bigwhoop-defaults::default"
  chef.add_recipe "bigwhoop-git"
  chef.add_recipe "bigwhoop-defaults::test"
  chef.add_recipe "bigwhoop-apache2"
  chef.add_recipe "bigwhoop-mysql"
  chef.add_recipe "bigwhoop-bitbucket"
  chef.add_recipe "chef-solo-search"
  chef.add_recipe "users::sysadmins"
  chef.add_recipe "bigwhoop-reinhard.codes"
end

Fortunatly, I already have all the chef configurations available, Vagrant just needs to be pointed to the correct cookbooks and databags folder. Also, we need to supply a runlist that Vagrant passes to Chef Solo. Previously I had to run lists specified as JSON files in my chef-repo folder, one for the live environment and another one for my test VM. The latter can now be removed as the run list in the Vagrantfile takes over that job.

When I first ran the provisioning, the VM stalled after some time. Looking at the system logfile in the VM I found an OutOfMemory error, so I adjusted the memory settings for the VM in the Vagrantfile:

config.vm.provider "virtualbox" do |vb|
  vb.gui = false
  vb.memory = "2048"
end

Accessing the test environment

I’m serving a few websites on my server and – of course – I want to be able to see them served from my test environment as well. Since I have name-based virtual hosts in apache, I’ve added entries to my local /etc/hosts file, which I can comment out after testing. In my previous setup I had my VM configured to a fixed IP address. Vagrant by default uses a private network and the default way of accessing services would be by defining port forwards to your local machine. I wanted to stick with my previous approach, so I changed the network configuration in the Vagrantfile:

config.vm.network "public_network", ip: "192.168.1.125"

TL;DR

Replacing my custom staging solution with a Vagrant-based approach was easy. All of the problems that I had to solve manually (network setup, etc.) Vagrant takes care of better and out-of-the-box.