[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Fab-user] Experiences working with Fabric

From: Daniel Pope
Subject: [Fab-user] Experiences working with Fabric
Date: Tue, 21 Dec 2010 17:38:43 +0000
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv: Gecko/20101208 Thunderbird/3.1.7

I've been working with Fabric for deployments of perhaps a dozen apps across our 70-odd server network. As I'm now leaving this contract I wanted to submit the experiences I had using Fabric in case the community would find that beneficial.

- Our servers are configured using Puppet (http://www.puppetlabs.com/), and I quickly realised that Puppet classes of the servers are tied very closely to the apps I was deploying to them with Fabric. This allowed me to write sanity checks for our servers such as

assert hasclass('apache::nso')

that have saved embarrassing misadventures when Fabric is misconfigured and head-scratching when Puppet is misconfigured. Implementation:

class_cache = {}

def hasclass(classname):
    """Test whether the current host has the puppet class classname."""
    if env.host_string in class_cache:
        return classname in class_cache[env.host_string]
        with hide('stdout'):
            classes = run("cat /var/lib/puppet/classes.txt")
        class_set = set([l.strip() for l in classes.splitlines()])
        class_cache[env.host_string] = class_set

        return classname in class_set

- Since I was using puppet it was also convenient to use facter, which brings a kind of convenient symmetry:

import yaml

fact_cache = {}

def fact(name):
        facts = fact_cache[env.host_string]
    except KeyError:
        with hide('running', 'stdout'):
            out = run('facter -y')
        facts = yaml.safe_load(out)
        fact_cache[env.host_string] = facts
    return facts[name]

(Going through YAML was merely convenient as it saved me writing a parser for facter's output, and is not a requirement of the approach).

- The fabric sudo() operation had to have pty=True often enough that it might merit a global switch. In the end I reconfigured the sudoers files on all of the machines, but I could only do this because I had sufficient access to the machines.

- Something I wanted time and time again was a remote analogue to the GNU install command, ideally something a little like this (which I suspect is Debian-specific):

def install(local_file, remote_path, owner='root', group='root', mode=0644):
    """Install local_file to remote path setting its ownership and mode."""
    tmp = run('tempfile')
    put(local_file, tmp)
    sudo('install -D -m %04o -o %s -g %s %s %s' % (
        mode, owner, group, tmp, remote_path
    run('rm -f ' + tmp)

- I very quickly found that I wanted meta-tasks to name a list of hosts to run on. Though I didn't like the global side-effect nature of it I found the syntax I liked best to be

for host in each_host(['web1', 'web2']):

implemented as

def each_host(hostlist):
    for h in hostlist:
        with settings(host_string=h):
            yield h

However the implicit side-effects of the above approach meant I avoided it where possible.

- Instead I wrote a @default_role(role) decorator which supplies a host list if no hosts are in the env when the task is called.

Thus I could equally do

$ fab deploy

to use the default role for deployment, or

$ fab -H foo deploy

to deploy to a specific server, perhaps a new one.

This allows me describe meta-tasks that take no hosts (ie decorated with @hosts() or @runs_once) and call those component tasks, eg.

def switch(tag):
    """Switch to a different release."""

with component tasks like

def purgevarnish():
    """Purge the varnish servers"""
    assert hasclass('varnish::nso')
    run('varnishadm -T localhost:6082 url.purge .')

- One of the biggest things was that we have multiple environments: development, staging, production and something ill-defined called "preview". These each have different hosts, different settings, etc.

I wrote a load() task that would populate a global environment. I also wrote a decorator that ensures an environment has been loaded and errors out otherwise. This leads to command lines such as

fab load:nso-staging switch:nso_5.10.00.rc2

I don't know if this could be made general enough for inclusion in Fabric but I have to report I fought with this for quite a while to get everything I needed out of Fabric with a commandline syntax I liked.


I hope the above is useful. I've found Fabric to be an excellent tool; the developers were writing deployment scripts in ant. I'm pleased to report that having converted them to Fabric they are easier to read and maintain, more flexible, and faster.

Mauve Internet
t: 01243 888187
w: www.mauveinternet.co.uk

reply via email to

[Prev in Thread] Current Thread [Next in Thread]