Hosting a website in object storage

By Curtis Collicutt, Cloud Developer, Edmonton

With the support of Cybera, specifically its Amazon Web Services account, I recently set up the CanStack blog to be served from Amazon S3 object storage. Ideally, I would like to serve the blog from a Canadian OpenStack Swift cloud, but I'€™m not aware of any service providers which meet that requirement (please leave a comment if you know of any).

Given that Amazon S3 is, by far, the largest object storage provider, it'€™s worthwhile knowing how to use its system, as most other object storage services emulate the S3 API.

Why object storage?

According to Amazon, object storage is:

"….a simple web services interface that can be used to store and retrieve any amount of data, at any time, from anywhere on the web". '€” Amazon

What are the benefits of object storage from a user perspective?

  1. Capacity '€” In the case of Amazon S3, there is a tremendous amount of storage. An individual user will never be able to fill up S3. For all intents and purposes, it'€™s infinite. Most public OpenStack Swift providers will also have massive capacity.
  2. Durability/Reliability '€” Most object storage systems create multiple copies of each file/object. For example, in OpenStack Swift the default is three copies, but it can be set even higher. The user doesn'€™t have to worry about backing up files '€” that is all taken care of automatically by most object storage systems. With S3, each object is replicated several times across a specific Amazon region.
  3. Availability '€” The S3 service has a 99.9% monthly uptime guarantee, which means downtime of ~8 hours per year. Swift is designed to remain up even if individual nodes are down, and can be upgraded live, but uptime will depend on the service-level agreement (SLA) of the specific Swift service provider.
  4. Scalability '€” Because most public object storage systems are so large, they have the ability to store and serve massive numbers of objects. Many large web and mobile applications are based on frameworks that actually use object storage systems in the background.
  5. Simplicity '€” The default protocol to use S3 and OpenStack Swift is plain HTTP. Also, I was surprised to discover that S3 supports bittorrent and has for years!

Setting up a website on Amazon S3

I feel that using Amazon S3 to host a static website is a good way to get used to the concept of object storage. Hosting a website is not the best use-case for object storage, but it'€™s a good entry point into the technology.

Configuring a bucket

Amazon'€™s documentation on hosting a static website is well done. For the site blog.canstack.ca, I first created a bucket using s3cmd. The bucket name has to match the domain name of the website:

$ s3cmd mb s3://blog.canstack.ca

Unfortunately, there are three steps that need to be performed before uploading the website to S3 that cannot be done via the s3cmd: entering a bucket policy, setting permissions, and turning on static website hosting for the bucket. (At least, I couldn'€™t find any documentation on how to do them from the command line.)

First, set the bucket policy. I copied and pasted the code from the Amazon documentation, only changing it to add the '€œblog.canstack.ca'€ part in the '€œResource'€ section.

Second, I set the permissions.

Finally, I turned on static website hosting.

I also recorded the endpoint for the next step.

dns entry

Next, I asked our DNS administrator to make blog.canstack.ca a CNAME for the bucket endpoint of blog.canstack.ca.s2-website-us-east-1.amazonasw.com.

Syncing the website

The CanStack blog is created by Jekyll: a '€œsimple, blog aware, static website generator.'€ When run, Jekyll places the static website in ./_site.

To copy the entire site up to the S3 bucket all I have to do is run one command:

$ s3cmd sync --delete-removed ./_site s3://blog.canstack.ca/

And that'€™s it. Now I have a website hosted from S3 that will, if my calculations are correct, cost about $0.03-$0.06 a month in AWS fees, depending on how many people access the CanStack blog. If it gets a lot of hits, maybe it'€™ll cost a quarter.

If you have any questions, comments, criticisms, please let me know. 🙂