What you can gain

This will create the neccesary directories, install nginx and gunicorn under supervisord control and upload new configuration files made from templates on all the app-servers. Additionally it deploys the newest codebase from your local git branch.

Set up a complete Django environment on a new server:

fab setup_new_server @app-servers/192.168.0.10

If you desire a database server (might be the same one as before):

fab setup_new_server @db-servers/192.168.0.10
fab syncdb @db-servers

So how to deploy a new commit?

fab deploy

It makes a point to pip your requirements file.

Deploy made everything worse?

fab rollback

Actually even the previous commit was not working particularly well? You can choose a specific previous deploy, from the /releases/ directory.

fab rollback 

The script is available on Github (fabfile.py and fabconfig.yaml).

We use this scripts internally to quickly set up new staging/development servers on VMs.

This has been a huge time saver for us and made it easy to deploy very different projects with one script. And it makes it much easier to react very quickly to unexpected demand and add e.g. another backend node or two in 5 minutes, before any of the users stop thinking the delay is a temporary condition, but rather a permanent issue.

How to make it work

We’ll be using modified by tav which, among other improvements, enables parallel multi-server deployment. It’s not perfect, but the time saved, normally wasted on looking at progress bars alone is worth it.

The script is available on Github (fabfile.py and fabconfig.yaml). In this repository you’ll actually find a whole sample Django project to check how it all works on a e.g. clean VirtualBox machine.

Requirements:

1. Script is made for nodes running Debian (will probably work fine on Ubuntu as well). Tested using recent Debian Squeeze.
2. Set up virtual environment in the desired local directory and pip tav’s fabric fork:

   virtualenv .
   sh bin/activate (OSX users – . bin/activate)
   (OSX might need a brew install libyaml)
   pip install --upgrade -E . \
     -e git://github.com/traviscline/fabric.git@tav#egg=fabric
   pip install --upgrade -E . boto

3. If doing it from a scratch (not our sample project) please just get our .gitignore file and run:

   django-admin.py startproject sample_project
   git init .
   git add .
   git commit -m "initial"

4. Set up SSH public key authentication on your nodes if you require parallel deployment (unless every single one of your machines uses the same root password). If so, after every fab command mentioned add -p .

Modify fabconfig.yaml and settings.py with your hosts and desired database parameters. If you want to have PostgreSQL on the same host as the frontend – just put the same host in app-servers and db-servers section. Modify Django project name as well.

We use this project layout:

• • • …
    • settings.py
  • /conf – configs
  • /static
  • … – virtual environment
  • fabfile.py
  • fabconfig.yaml
  • requirements.txt

And you’re ready to go. Change for your desired IP address.

fab setup_new_server @app-servers/192.168.0.10

And that’s it. Your Django is working, available on port 8080. You can change all the configuration files in /conf (e.g. to enable serving static media files from nginx).

Actually this script is even more powerful, but we’ll talk about it in our next post.

Configuration

fabconfig.yaml contains a few interesting sections:

• default – default settings
  • path – remote path to your project
  • django_project_name – directory name of your Django project
  • database_name, database_user, database_password – self descriptive
• *-servers – server groups (you can create your own)
  • hosts – IPs belonging to this group
  • configs – map of configuration files templates which will be uploaded in form local_file: remote file; you can use variables almost everywhere (including filenames and templates, see the included example file)
  • installation – list of commands to run on nodes first setup
  • post_deploy – commands to be run after deployment
  • cold_start – commands to force-start the servers
  • post_cold_start – commands to be run after force-starting the servers
  • start – commands to start the servers
  • stop – commands to stop the servers
• hostinfo – overrides any of the aforementioned settings on a particular server (which might e.g. run different OS, which requires different configuration files paths)

Next week Tomek Kopczuk () will write about using this script to automatically minify and merge JS and CSS files, optimize images and deploy all the static media to Amazon S3 for a great UX gain.

Follow us on Twitter or grab our !
(–) Marcin Mincer

credits for the linen texture: