Actions: | Security

Navigation: Home | Services | Tools | Articles | Other

Chef Cookbook Development

This article aims to be simple distillation of my reading, experimentation and learning about how best to develop Chef cookbooks with more than a nod to the principles of Test Driven Development.


Software tools

First, some software installation so that the example activity/code can be reproducible. Feel free to use your own processes and substitute alternative OS/distro, virtualization software choices. The process of cookbook development is more or less the same across all Linux/Windows/OSX platforms.

  • Chef Development Kit. You'll probably want to ensure that '/opt/chefdk/bin:/opt/chefdk/embedded/bin' are at the beginning of your $PATH. Rubocop and Foodcritic are both included.
  • VirtualBox
  • Vagrant
  • Vagrant plugins: vagrant-berkshelf, vagrant-omnibus, test-kitchen
  • rubygems: kitchen-vagrant

Tool configuration
  • Get a/some boxes, if you like to save time once you are ready to run Test Kitchen (although TK will get them for you, you might not want to wait at that point). Our Test Kitchen setup will by default use an Ubuntu box although Berkshelf's default scaffolding specifies a CentOS box as well.

Getting started


The intention here is to demonstrate a process for creating a new cookbook with example code/commands. For my example, I'm going to start with a small, simple to configure, network service: mini-httpd, a lesser known http server.

The new cookbook's requirements are:

  • install the package: mini-httpd

  • write a configfile: /etc/mini-httpd.conf

  • start the service: mini-httpd

  • some basic level of testing: serverspec

  • The content of the configfile should be:


which will result in the daemon listening on port 8080 on all interfaces for an HTTP request to which it will respond by serving a simple html page which will render to "It works!".


$ berks cookbook mini-httpd --foodcritic

This creates extensive scaffolding for the new cookbook. Use berks help [cookbook] for details.

├── .git
|   <snip>
├── .gitignore
├── .kitchen
│   ├── kitchen-vagrant
│   └── logs
├── .kitchen.yml
├── Berksfile
├── Berksfile.lock
├── Gemfile
├── Thorfile
├── Vagrantfile
├── attributes
├── chefignore
├── files
│   └── default
├── libraries
├── metadata.rb
├── providers
├── recipes
│   └── default.rb
├── resources
├── templates
│   └── default
└── test
    └── integration
        └── default

Commit these files into git:

$ cd mini-httpd
mini-httpd $ git add .
mini-httpd $ git commit -m'initial scaffolding for cookbook: mini-httpd' -a

In order to get a minimal good cookbook, you'll want to edit/verify:

  • .kitchen.yml
  • metadata.rb
  • Berksfile
  • test/environments/kitchen.json
  • recipes/default.db

Edit/Verify .kitchen.yml

Change the provisioner to chef_zero because in constrast to Chef-Solo, Zero means that you don't have to special-case recipes/tests that use search. You'll want to verify that the list of platforms is correct. This is also the file where we specify the chef 'run list' and later might override the cookbook's default attributes for the purposes of testing, but more on that later. For now, we'll just use:

  name: vagrant

  name: chef_zero
  environments_path: test/environments
    environment: kitchen

  - name: ubuntu-12.04

  - name: default
      - recipe[mini-httpd::default]

If you want to be able to customize the VM that kitchen/vagrant makes for testing, you can make modifications to the 'driver' section, for example, specifying the hostname and forwarding ports from host to guest looks like:

  name: vagrant
  vm_hostname: ''
  # Allow access to a web server which is useful when troubleshooting
  - - forwarded_port
    - guest: 80
      host: 8088
      auto_correct: true


Berkshelf creates an example; change it to document your cookbook: what it does, attributes you can pass and generally how to use it.

Edit/Verify metadata.rb

Verify that the Berkshelf-created boilerplate in here is good. This is also where you record any cookbooks that this cookbook depends upon and their version restrictions.

Edit/Verify Berksfile

If you need to tell berkshelf from where it should fetch dependencies.

cookbook 'openstack-common', github: 'stackforge/cookbook-openstack-common', branch: 'stable/grizzly'
cookbook 'ntp-wrapper', git: 'ssh://'

And if you do, you'll want to run berks install or berks update from your cookbook directory.

Edit/Verify test/environments/kitchen.json

This file allows kitchen access to "dummy" environment data, so that tests can be performed with chef-zero which doesn't have access to environment data that is in the real Chef Server.

  "name": "kitchen",
  "default_attributes": {
    "AZ": {
      "name": "kitchen",
      "mcps": [
      "ops": [ ""
  "json_class": "Chef::Environment",
  "description": "AZ specific data for the kitchen AZ",
  "cookbook_versions": {
  "chef_type": "environment"

Edit/Verify recipes/default.rb

This is where you write the code that actually makes something happen. At last!

package 'mini-httpd' do
  action :install
cookbook_file 'mini-httpd_conf' do
  source 'mini-httpd.conf'
  path '/etc/mini-httpd.conf'
  action :create
  mode 0644
  owner 'root'
  group 'root'
  notifies :restart, 'service[mini-httpd]'
cookbook_file 'mini-httpd' do
  source 'mini-httpd'
  path '/etc/default/mini-httpd'
  action :create
  mode 0644
  owner 'root'
  group 'root'
  notifies :restart, 'service[mini-httpd]'
service 'mini-httpd' do
  action [ :enable, :start ]
  supports :restart => true

The cookbook_file resource declared by cookbook_file 'mini-httpd_conf' takes its contents from the COOKBOOK_NAME/files/default sub-directory, the filename is specified by the source attribute. Not really: it searches a series of paths for a filename that matches the source attribute: files/<platform>-<version>/, files/<platform>/, files/default/, for example: files/ubuntu-12.04/, files/ubuntu, files/default. The analogous equivalent is also true for the sources of template resources and integration tests.

In our case, the contents of mini-httpd/files/default/mini-httpd.conf should be:


And the contents of mini-httpd/files/default/mini-httpd should be:

DAEMON_OPTS="-C /etc/mini-httpd.conf"

As we get more sophisticated, using attributes to allow variable configuration, we will remove this resource in favour of a template resource that, by default, will create the same configuration file with the same contents, but allow for alternative values for the mini-httpd directives.

At this point, you might commit your work to git.

mini-httpd $ git add files/
mini-httpd $ git commit -m'configure package, file, service for mini-httpd' -a

Test Kitchen, first run

kitchen test will result in vagrant instantiating a single ubuntu-12.04 VM, installing chef using the omnibus plugin, provisioning with chef-solo, configuring the VM according to our newly created default recipe from the mini-httpd cookbook and running any test suites we have specified in .kitchen.yml.

But at this stage, our process requires a manual verification step to satisfy ourselves that the package, file and service we attempted to manage are actually managed correctly.

$ kitchen create
$ kitchen converge
$ kitchen login

Now, logged in to your test VM, you can poke around manually to determine whether everything has been configured according to your will.

Later on, when we are running automated tests, we will just use:

$ kitchen test

If you are making lots of experimental changes to your cookbook, you can probably accelerate your test-kitchen workflow by separating the steps:

  • kitchen create
  • kitchen converge
  • kitchen setup
  • kitchen verify
  • kitchen login

If you do this, you can edit your cookbook and just run kitchen converge, avoiding the delay of the previous steps, in particular, the creation/provisioning of a new VM every time.

  • Most kitchen subcommands can be run with concurrency greater than zero, specify with -c N
  • Most kitchen subcommands can be given parameters to limit which instances will be affected, either by a list of instances or a regex: e.g. kitchen converge ubuntu
  • You can set the logging level for your kitchen commands with -l [debug | info | warn | error | fatal]

Static Analysis


rubocop will syntax-check and style-check your code based on the Ruby Style Guide. If you use -a, it will auto-correct where it can.

rubocop --show-cops

will show to which rules our check is subjected.


Foodcritic is a style-checking tool for Chef cookbooks, hand it one or more paths to cookbook(s). We're hoping for a single blank line of output.


If, from within the cookbook directory, you run:

mini-httpd $ knife cookbook test mini-httpd -o ..

knife will run a ruby syntax check against all files that end in .rb and .erb (respecting .chefignore).


This is (mostly?) for unit testing, so I'll leave this as a stub for later. Initially, we're going to concentrate our effort on functional testing with Serverspec.

Functional Testing


Create a directory for the Serverspec tests, then we can start to make tests in test/integration/default/serverspec/localhost/mini-httpd_spec.rb:

require 'serverspec'
set :backend, :exec

describe 'mini-httpd service' do
  it 'should be running and enabled' do
    expect(service 'mini-httpd').to be_running
    expect(service 'mini-httpd').to be_enabled

  it 'should be listening on port 8080' do
    expect(port(8080)).to be_listening

For details about what types are available see the serverspec documentation about its resource types.

NOTE Files that contain serverspec tests must be named to match /_spec.rb$/ or test kitchen will not use them.

If you are just modifying your tests, you can accelerate your test-kitchen workflow by separating the steps:

  • kitchen create
  • kitchen converge
  • kitchen setup
  • kitchen verify

If you do this, you can edit your tests and just run kitchen verify, avoiding the delay of the previous steps.


Bats, the Bourne-Again shell Test System.

Create a directory for the bats tests and add a file <cookbook_name>/test/integration/default/bats/<cookbook_name>.bats:

match() {
  local p=$1 v
  shift  for v
    do [[ $v = $p ]] && return
return 1

@test "mini-httpd is listening to port 8080" {
  run nmap -sT -p8080 localhost
  match "80/tcp open  http" "${lines[@]}"
@test "Server accepts HTTP requests" {
  echo "GET / HTTP/1.1" | nc localhost 8080

This brings up another requirement to ensure that certain packages are available during testing, whether or not they're installed for real. Create a file <cookbook_name>/test/integration/default/bats/prepare_recipe.rb

%w{ nc nmap }.each { |pkg| package pkg }

Improvement: templating the config files

Instead of using a static file to provide the content for our configuration files, we can use erb templates. In order to do so, we modify recipes/default.rb using the template resource syntax:

# Install a templated configuration file for mini-httpd
template 'mini-httpd_conf' do
  source 'mini-httpd.conf.erb'
  path '/etc/mini-httpd.conf'
  action :create
  mode 0644
  owner 'root'
  group 'root'
    :port => node['mini-httpd']['port'],
    :user => node['mini-httpd']['user'],
    :data_dir => node['mini-httpd']['data_dir'],
    :log_file => node['mini-httpd']['log_file']
  notifies :restart, 'service[mini-httpd]'

and create

which defines the attributes (keys) that this cookbook recognizes and gives them default values.

default['mini-httpd']['port'] = '8080'
default['mini-httpd']['user'] = 'nobody'
default['mini-httpd']['data_dir'] = '/usr/share/mini-httpd/html'
default['mini-httpd']['log_file'] = '/var/log/mini-httpd.log'

as pointed to by the 'source' attribute of the template resource in our default recipe.

# /etc/mini-httpd.conf
<% if @port %>
  port=<%= @port %>
<% end %>
<% if @user %>
  user=<%= @user %>
<% end %>
<% if @data_dir %>
  data_dir=<%= @data_dir %>
<% end %>
<% if @log_file %>
  logfile=<%= @log_file %>
<% end %>

Modifications for wrapper cookbooks

Wrapper cookbooks are named agb-<cookbook_name> (I'm just using 'agb' for myself - pick your own prefix). Starting is the same as normal:

berks cookbook agb-ntp --foodcritic
cd agb-ntp
git init
git add .
git commit -m'Initial scaffolding for ntp wrapper cookbook' -a

Add the original to your dependency list, with an optional version requirement:

echo "depends 'ntp'" >> metadata.rb
echo "cookbook 'ntp', '~> 1.6.5'" >> Berksfile

For each recipe that you want to make available, create recipes(s) in your agb-<cookbook_name>/recipes/:

echo "include_recipe 'ntp'" >> recipes/default.rb


echo "include_recipe 'graphite::carbon'" >> recipes/carbon.rb
echo "include_recipe 'graphite::web'" >> recipes/web.rb

Add that recipe to your default test suite, by modifying .kitchen.yml:

  name: vagrant

  name: chef_zero

  - name: ubuntu-12.04

  - name: default
      - recipe[agb-ntp::default]

Overriding default attributes

One of the major reasons for using wrapper cookbooks is so that we can easily override attributes set by the cookbook maintainer as defaults to make them more useful for our purposes.

In this example, I set the array of upstream ntp servers to be MI[1-3].node[:domain], or otherwise, allow the cookbook default behaviour, setting default['ntp']['servers'] to various addresses.

Create attributes/default.rb:

unless node[:fqdn].start_with?('MI')
  (1..3).each do |n|
    default['ntp']['servers'].push('MI' + n.to_s + '.' + node[:domain])

Overriding default attributes for testing only

In .kitchen.yml, we can set specific attributes that are appropriate/required for the test run. For example, preventing the default behaviour of using a apt cacher/proxy requires us to set ['apt']['cacher_ipaddress']:

  - name: default
      - recipe[mc-global::default]
        cacher_ipaddress: false


  1. Create:

    berks cookbook <cookbook_name> --foodcritic
    cd <cookbook_name>
    git add .
    git commit -m'initial scaffolding for cookbook: <cookbook_name>' -a
  2. Edit, minimally:

    • metadata.rb
    • recipes/default.rb
    • .kitchen.yml
  3. Test for syntax/style:

    rubocop -a
    foodcritic .
  4. Create functional tests:

    • serverspec tests go in <cookbook_name>/test/integration/default/serverspec/localhost/<cookbook_name>_spec.rb
  5. Commit, submit to review, celebrate!