Deploy OpenStack Swift All-in-one with Puppet

By Curtis Collicutt, Cloud Developer, Edmonton

The Swift object storage system is one of the two original components of OpenStack.

Swift provides object storage, which is a relatively new invention. It'€™s not block storage, and it'€™s not a filesystem, but rather, it'€™s a highly available, durable, distributed, and eventually consistent storage system that is accessed via a REST-ful http API. Swift can store petabytes of unstructured data at a relatively low cost.

DevStack and Swift All-in-one

The OpenStack project has over 1,000 developers from 80 countries. One of the easier ways for them to access an OpenStack environment to test or simply play with it is to use DevStack. This is essentially a shell script that installs all of the standard components of OpenStack into one server (usually a virtual machine) with a single command. When the command is completed, the result is a working OpenStack system (which I should note includes Swift as well).

The Swift project also has something similar to DevStack, but it'€™s not a shell script. Instead it'€™s a set of instructions that can be used to install '€œOpenStack Swift All-in-one'€ (or SAIO).

Using this set of instructions, anyone can start up a virtual machine, follow the clearly laid out steps (which include cutting and pasting commands and configuring files from the documentation), and eventually have a working multi-device, multi-zone, four node '€œcluster'€ of OpenStack Swift.

Because Cybera already has several OpenStack clusters, and is in the midst of deploying a new multi-region, geographically distributed Swift cluster, I thought it would be a good idea for me to set up a learning environment to test out installing, configuring, and using OpenStack Swift.

However, I didn'€™t want to have to cut and paste commands and configuration files every time I wanted a new Swift environment. Of course, I could just do most of the work once and then create a snapshot of the resulting virtual machine, but I also wanted to learn a bit more about the Puppet configuration management system. So I put the two together.

The result is puppet-saio: a Puppet module to deploy OpenStack Swift All-in-one on an Ubuntu Precise 12.04 virtual machine. The repository puppet-saio also comes with a Vagrant configuration file, which is setup to run vagrant up and have a working, automatically installed OpenStack Swift All-in-one instance in just a few minutes.

Using puppet-saio

Before you start, there are a few things you should have set up:

  1. Vagrant
  2. Hypervisor of some sort (likely Virtualbox)
  3. Internet connection
  4. Git

Once those requirements are met, we can start the install.

First, clone the puppet-saio repository. For this demo I'€™m going to use version 0.2.6 of the puppet-saio releases, just because I know it works. (There may be newer versions since the writing of this blog post which could also work.)

curtis$ wget --quiet https://github.com/curtisgithub/puppet-saio/archive/0.2.6.tar.gz -O - | tar -zxv
x puppet-saio-0.2.6/
x puppet-saio-0.2.6/.gitignore
x puppet-saio-0.2.6/README.md
SNIP!
curtis$ cd puppet-saio-0.2.6

Next, run vagrant up. This will create a new Ubuntu Precise virtual machine, install a newer version of Puppet, download the Vcsrepo and stdlib puppet modules from the puppet forge, and then finally use the puppet-saio module to install OpenStack Swift All-in-one. You should note that this process will require downloading many Ubuntu packages from the internet, so I do recommend setting up an apt-cache-ng server and using the package_cache_srv option, if this process is going to be run more than once.

curtis$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
[default] Importing base box 'precise64''€¦
SNIP!
Notice: Compiled catalog for precise64 in environment production in 1.17 seconds
Notice: /Stage[main]/Saio::Install/File[/etc/apt/apt.conf.d/01proxy]/ensure: defined content as '{md5}fd5f652dd5d64dc895c9b342fc368540'
Notice: /Stage[main]/Saio::Install/Exec[apt-get update]/returns: executed successfully
SNIP!
Notice: /Stage[main]/Saio::Start_swift/Exec[startmain]/returns: executed successfully
Notice: /Stage[main]/Saio::Start_swift/Exec[startrest]/returns: executed successfully
SNIP!
curtis$ # Puppet has completed the installed of SAIO!

(I use SNIP! to show where I removed output, for brevity.)

At this point we can ssh into the virtual machine and use Swift. I usually set up an alias to make it a bit simpler to type swift commands:

curtis$ vagrant ssh
vagrant@precise64:~$ alias sw="swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing"
vagrant@precise64:~$ sw list
vagrant@precise64:~$ echo "swift is cool" > swift.txt
vagrant@precise64:~$ sw upload test_container swift.txt
swift.txt
vagrant@precise64:~$ sw list
test_container
vagrant@precise64:~$ sw list test_container
swift.txt

We can also check the version that is running:

vagrant@precise64:~$ curl -s http://localhost:8080/info | python -mjson.tool | grep version
       "version": "1.13.1.rc1.7.g48a2848"

At this point you have a small Swift '€œcluster'€ made up of one virtual machine. You can use this for testing, becoming familiar with OpenStack Swift, or perhaps for learning how to contribute back to the project.