RailsConf 2014 - Deploying Rails is Easier Than it Looks by Ben Dixon

BEN DIXON: OK. So, I'm Ben.

I'm a Rails developer and the author

of a book called Reliably Deploying Rails
Applications.

I spend a lot of people - a lot of people?

A lot of time teaching people Ruby at

MakeItWithCode dot com,

which is an online mentor-driven class

for people who are new to Ruby

but want to get started making things quickly.

I spend the rest of my time working with

the awesome guys at HelloSwimio, where we
make cool

tech for swimmers. So we make things like
the

time tabling system for the Olympic pool in
London,

the speedo swim tracking app, and the first
automatic

lap tracker for the Pebble smart watch.

So, before we get started, could you put your

hands up if you've deployed a Rails app of

some sort to Heroku. Awesome. Now keep your
hand

up if you've deployed a Rails app to a

server that you've set up yourself. Good.
That's less

people.

So if someone came to me and said, I

want to deploy my first Rails app, how should

I do it? I wouldn't think about it, I'd

just say use Heroku. And I'd say use Heroku,

because Heroku makes deploying apps incredibly
simple. You just

commit your code, Heroku create, git push
Heroku, and

magically, a couple of minutes later, your
application is

live, on the internet, for the world to see.

And because Heroku has made this process so
simple

and so easy, it's really easy to take that

server stuff and kind of put it in a

bucket of things we don't look at. Because
it's

complicated and Heroku makes it really easy,
so why

would we bother?

And what I'd like to demonstrate in this talk

is that, in the same way Rails makes building

web applications really, really easy, there
are lots of

great tools out there that make setting up
servers

for them and deploying to those servers really
easy.

And once you get the hang of these tools,

you can do some really cool stuff.

Before we get started, two important bits
of terminology,

which probably you already know. A PaaS, a
platform

as a service, and by that I mean something

like Heroku. So something that just abstracts
all of

your infrastructure away so that you don't
have to

worry about it.

On the other scale, you've got a VPS, a

Virtual Private Server, and for the purposes
of this

talk, I just mean some sort of Linux server

running the cloud. So the type of thing you

get from Linode or RackSpace or DigitalOcean.

So, to get started, I want to take a

little look at what's going on behind the
scenes

when you deploy a typical Rails application.
How do

we go from Rails S on our development machine

to a Rails app on a VPS somewhere serving

our application to the world?

So, here we've got a beautiful picture that
I've

drawn, because I'm new to Keynote. And the
black

box around the outside is our VPS. It's a

single VPS. And this is the first thing that

seemed to surprise lots of people. For a simple

production configuration, you only need one
VPS. In the

same way, in development, you can run your
entire

Rails application on your development machine.
For simple production

application, you can run everything on a single
VPS.

And that VPS will have three main components
on

it for almost all Rails applications. You're
gonna have

a web server, you're gonna have an app server,

and you're gonna have a database server. So,
when

someone types in the address of our website,
a

request comes in. It hits our VPS, and the

first place that request is gonna go is our

web server. Now, if that request was for a

static file, and by a static file, I basically

mean, anything that goes in the Rails asset
pipeline

or anything that you might find in the public

folder, the web server's gonna deal with that
directly.

It's gonna send that file straight back to
the

user for display in their browser and that's
the

end of it.

If, however, that request is for a dynamic
page,

so for a page in our Rails app, essentially,

that's gonna get passed back to an app server.

Now, when we run Rails S in development, what

we're starting is an instance of an app server.

So, the app server here is Rails S running

on our VPS. So the web server passes the

request back. The app server generates the
response and

then passes it back to the web server, which

then returns it to the user.

And in development, we're probably used to
the web,

the app server being something like Thin or
WebRick.

In production it might also be Thin or Webrick.

It might also be something a bit more advanced

like Puma or Unicorn.

Finally, for pretty much all Rails applications,
we're gonna

want some sort of database server. And that's
probably

gonna be PostGres or mySQL or Mongo. And,
again,

we don't need a separate database server.
This all

runs on our single VPS.

So, to recap, a request comes in and it

hits our web server. Probably EngineX or Apache.
If

it's for a static file, the web server deals

with it. If it's for part of our Rails

application, it gets passed back to the app
server,

which is basically Rails S, and the app server

may communicate with the database server,
read some data,

write some data, and then the app server constructs

the response, sends it back to the web server,

which serves it to the user.

So, you'd be forgiven for looking at that
and

thinking, well, that already looks more complicated
than Heroku.

Why on earth would you ever do that? Why

not just use Heroku?

Well, I started doing this, if I'm honest,
because

of cost. I had quite a lot of side

projects, and as you probably already know,
most past

solutions charge by the process. And a lot
of

my personal projects and side projects had
background jobs.

Now when you've got five or six side projects,

all of which would need two Heroku dynos,
suddenly

that bill at the end of the month starts

to get quite substantial.

And so I wanted to save money by deploying

to my own servers. And I did. It's worth

bearing in mind, if that's also your motivation,
that,

as with any new skill, you have to invert.

Invert? You have to put in a certain amount

of time to learn it. If you were to

account for the time I put into learning this

to begin with, at market rates, I'm fairly
sure

there would have been no saving at all. Personally,

they're my own projects. I wasn't paying myself
market

rates. And I was happy to treat it as

an investment which would pay off in the long

term, and it has.

Actually, it turned out, the biggest benefit
of learning

how to do this was nothing to do with

money. It was the infrastructure went from
being this

kind of necessary evil that I have to have

in order to run my Rails apps, to just

another tool that I could use for building
interesting

stuff. And the best example of that, for me,

is Make It With Code. We provide our students

with a cloud IDE that has Ruby pre-installed.
And

that means they can hit the ground running.
They

can start writing Ruby code really quickly.

And the techniques that we use setting those
up

automatically for all of our students are
exactly the

same techniques that I'm gonna cover in this
presentation

for setting up Rails apps. So it means that

our infrastructure has gone from being this
thing that

we have to have in the background, as like

a supporting thing, to something that actually
adds value

to our product in itself. It's a part of

our product.

So if you decide to give this a go,

I really, really urge you not to do it

even slightly like I did it originally. Because
it

was terrible. I started off by getting a VPS.

That bit was good. That bit worked. I Googled,

how do I set up a Rails VPS. I

found all these great tutorials that said,
type in

this command, then that command, then edit
this config

file. And I found loads of things on StackOverflow

that said, here's what to do when that goes

wrong and that goes wrong. And eventually
I had

this working server. And it was great.

And it broke a few times in the first

few weeks. But we fixed that when it happened.

And, by and large, it worked for quite awhile.

And it worked until it didn't work, and when

it didn't work, I was in France, on a

holiday with my family, and it was a fairly

rural bit of France, and we had, we had

a wifi connection. But it was only one wifi

connection for, basically, the village. And
to access that,

we, we could access it by sitting half in

a wardrobe on the top floor of the house,

which was nice.

And, so one morning I woke up and I

was sitting in the wardrobe with a coffee
and

checking my email, and I had a wonderful,
a

lovely email from my host to say, Dear Valued

Customer, your server is dead, you need to
make

a new one, Lots of love, from your hosts.

And I thought, good. Right. So I set up

a little desk in this wardrobe and for the

rest of the day, my family sat by a

river. They drank wine, ate cheese, mocked
me slightly,

and I sat in a roof, in a wardrobe,

in a heat wave, and pretty much followed the

same tutorials I employed the first time.
Made the

same mistakes that I'd made the first time,
and

I got the server back up, but it really

wasn't any easier the second time. And that
seemed

kind of wrong to me. That it seemed really

frustrating.

So I figured there must be a better way

of doing this. If I'm honest, actually what
I

thought was, I can't believe there isn't already
a

better way of doing this, I should invent
it

and then I'll be rich and famous and everyone

will love me. Luckily I took a step back

and thought, OK, this probably already exists.

And, of course, it did. And what I wanted

was a configuration management tool. And a
configuration management

tool is really simple. It allows you to define

and automate the commands that it takes to
set

up the server to do a particular thing. And

what's amazing about a configuration management
tool is once

you've done it once, doing it again and again

and again is completely trivial.

So, that time you put in to start with,

it will probably only be one command to set

up an identical server the next time. And
to

give you an idea of just how simple it

can be, I've got a brief demonstration of
the

process that I actually move, at the moment,
on

a pretty much weekly basis for setting up
new

servers.

At the end of this presentation, I'll link
to

tutorials of how to do this yourself and the

sample code. So please don't worry about following
it

step by step, I just want to give an

idea about how simple it can be.

So, here you can see, on the left, your

left, there is a list of servers that I've

deployed in the past. And I'm using my configuration

management tool of choice, Chef Solo, which
uses JSON

files for defining what should go onto a particular

type of server.

And on the bottom you can see I've got

a couple of web servers for Make It With

Code, which at the moment is running a selection

of Rails and Sinatra apps. And I want to

write a new one. And to do this, I

just create a new JSON file, web3 dot makeitwithcode

dot com. I save it, and then this server,

it's gonna be pretty much the same as web2.

It's just running Rails apps.

So I just copy all the JSON from my

previous server, but it into the new one,
and

the bit I want to draw your attention to

is, if you look at the bottom of that,

you've got this run list with a list of

things that are being called roles. And pretty
much

all configuration management systems have
some sort of analogous

concept to this, which, kind of individual
components that

you can mix and match to define what should

go onto this server that you're setting up.

So I've defined in the past, what goes on

an internet server. What goes on a PostGres
server.

And now if I want that, I can just

drop that role in, and I know that it

will be set up in exactly the way that

I want it. So on this server I want

Redis. I didn't on the previous one. So I

can just uncomment it. Uncomment it, yeah.
And I'll

get exactly the Redis configuration I want.

The good bit, the fun bit, is once I've

done this, I can type on command - knife

solo bootstrap - and then the address of the,

the VPS I've created. Presenter. And then
leave it

for about twenty minutes, go and get a coffee,

and the output that will scroll through, it
will

tell me what commands it's applying. It will
show

me diffs of any configuration files that it's
changing.

And if it goes wrong, which it doesn't very

often, it will tell me exactly what it was

doing when it went wrong, so I can fix

it.

And at the end of this process, I will

have a Rails server which is identical the
last

one I set up, and the one before that,

and I can check that this works. I can

fire up a web browser. I can go to

web3 and the address that I gave it, and

I'll get this wonderful Welcome to EngineX
page, which

says, that server is set up and it's ready

to use.

So, what I'm trying to demonstrate with that
figure

is that configuration management is basically,
don't repeat yourself

for setting up servers. If you think back
to

the first time you used, say, Devise, for
example,

the first time I used Devise, it took me

ages to get all_auth and things like that
working.

And then, six months later, I came back to

it, and didn't really remember any of the
stuff

that I'd done to get it to work. But

I could go back to the code I wrote

the first time, I could look at that code,

copy bits of it, and it was much quicker.

Now, the problem with just copying commands
or entering

commands manually when setting up the server
is there

is no natural audit trail of what did I

do to make this work the first time. So

there's a good chance, the second time you're
gonna

do it, you'll make the same mistakes as the

first time.

With a configuration management utility, you
naturally create this

audit trail as you go along. So that, in

time, that time you invest up front, it goes

much, much further in the long run.

So, hopefully that's shown that the bit where
people

often trip up, getting a server setup, it
doesn't

need to be that difficult, and I'll give you

the exact sample code at the end, which you

can use if you want a head start.

But something like Heroku, it doesn't just
make setting

up a server really easy. It makes deploying
to

one incredibly easy. Git push heroku. It pretty
much

couldn't be easier. And I think it's fair
to

say that the first time I used Heroku, it

was completely revolutionary.

So I came to Rails from Python/Jengo, and
before

that, PHP, and I'd used a whole range of,

kind of, home grown deployment systems, which
ranged from

trying to remember what files you've changed
and FTP

them across to all sync things and custom
shell

scripts. And the one thing they all had in

common was they were flaky and deployment
was something

you didn't want to do, because it would probably

break. So you maybe deployed a couple of times

a week.

So I used Heroku for the first time and

it was amazing. Suddenly I could apploy. Apploy?
Deploy

ten times a day. And it was just fitted

into my normal workflow.

So I was really keen that if I was

deploying to my own servers, it had to be

as simple as that. It had to be one

command to deploy. And luckily, as is often
the

case in the Rails community, loads of very
clever

people wanted exactly the same thing, and
so have

developed some awesome tools, some awesome
gems, to make

it really simple to set that up.

In the Rails community, I think the best known

of these is probably Capistrano, which I'll
talk about

a bit more towards the end of the presentation.

But, first, another brief demo of how simple
it

is to add Capistrano into an existing, in
this

case, Rails 4.1 application. Again, at the
end, I'll

give details of step by, a step by step

tutorial and the ensemble code for doing this.
So

please don't worry about the individual steps.
I just

want to give an idea of what's involved.

So, to get started, Rails 4.1 application.
I just

drop in the Capistrano gem and a few supporting

helper gems. Save that, and then add it to

git and, sorry, bundle and add it to Git.

Under the hood we're still using Git to deploy

in the same way Heroku does, so it's important

to make sure changes like that are committed.

And then running cap install, which just generates
some

basic configuration files. The first one of
these is

a cap file. This is just a rake file

for capistrano. And here, I'm just telling
it that

I want to use those helper gems that I

included in the first step. So I'm telling
it

that I want to use rbenv to manage what

Ruby version I'm using. I want it to install

gems for me, compile assets for me, and apply

migrations. All the normal stuff you do when
setting

up a Rails app.

We've then got our main config file, just
deploy

dot rb. And here I'm just giving the app

a name, telling it which Git repository to
deploy

from, and getting rid of the load of the

boilerplate for the simple application. I'm
not gonna use

it. Uncommenting out a few other bits of boilerplate.

And then finally I'm gonna add an rbenv specific

block that tells it to always use a particular

Ruby version. I think, in this case, 2 point

1 point 1, when deploying this app.

Finally, we have stage configuration files,
so if you

have a production server and a staging server,
and

here I'm just telling it that the production
server

is going to be web3 dot makeitwithcode dot
com,

which is the server we set up in the

first demo. And then that should be made available

to the public at, imaginatively, web3 dot
makeitwithcode dot

com.

Then, firing up a console, running a command
called

setup config, and you can think of this like

heroku create. This is just telling that server,
or

preparing that server for a new application.
It's generating

a few config files, it's creating a new database

yml and telling EngineX that it needs to serve

a new site. Then running another helper task
that

I added this time. Create database. This is
just,

create a new database on that server. Making
sure

there's a user that has permission to access
that

server.

And this is the important command. Cap production
deploy.

And this, if we're using capistrano, is our
equivalent

to git push heroku. And if you look at

the output, it's doing some pretty standard
stuff. It's

applying migrations, compiling assets and
restarting the app server.

And once that completes, we can fire up the

web browser, visit it, and you can see we've

got a simple, if utterly useless Rails application.

And it has access to the database, it can

read and write, and now we can iterate on

that, and every time we want to make a

change, we can just make that change, commit
it,

cap production deploy, and it really is as
simple

as deploying it to heroku.

So one thing that we haven't looked at yet

is, it's very easy to set this stuff up,

but the other great thing about using a path

like heroku is that when it breaks at 2

AM, it kind of isn't our problem. They have

engineers and they get woken up at 2 AM.

And they go and fix it. And our application

just comes back up online again.

And we can't get around that completely. It's
now

our problem when it breaks at 2 AM. But

luckily there are lots of tools out there
which

mean we can minimize the number of times this

will happen. If you think in a typical production

Rails application, you probably got some exception
handling. You

know some exceptions will happen from time
to time,

and so you've got to have your begin and

rescues in that. And then hopefully you've
also got

some sort of exception notification. And that
means that

when an exception that you haven't planned
for happens,

you get an email and you know that you

can go in, you should go in and fix

it.

So, there are an analogous set of tools when

you're setting up your own servers. And these
are

called monitoring tools, and they're really,
really simple. They're

essentially just a list of checks that say,
here's

how something ought to behave. If it's not
behaving,

here's a command that you can run to try

and fix it. If it's still not working, here's

the person to notify and how to notify them

there's a problem.

And that might sound a little bit daunting.
So

every time you set up an application you're
gonna

have to think about all these checks. In practice,

this is where configuration management comes
in. So, if

you remember in that first demonstration,
when I uncommented

that Redis role, as part of that Redis role,

when I originally wrote it, probably a year
or

a couple of years ago, I included the monitoring

configuration for it, which basically says,
is Redis working?

If no, restart it. If it's still not working,

email me.

And now that will get automatically setup
every time

I include that Redis role. So there really
isn't

that much duplication. Configuration management
just does all of

this for us.

So to get a bit more practical, if you

decide to, to go ahead and try this. This

is the stack that I end up using, probably

99% of the time. For a web server, EngineX,

really popular in the Rails community. There's
lots of

really great Rails-specific documentation
out there.

For an app server, Unicorn. This probably
isn't what

you're using in development at the moment.
That's more

likely to be Thin or Webrick. Unicorn's a
little

bit more complicated. But, for that little
bit of

extra complexity, you get some really nice
features. In

particular, it's very easy to set up what's
called

zero-downtime deployment, which you may already
be familiar with.

Essentially what this means is, the simplest
way to

deploy an application is to deploy your new
code,

stop the existing application, and then start
it again

with the new code. The problem is, if you've

got a big Rails application, it can take two

or three minutes to start up. And that means

that, for two or three minutes, every time
you

deploy, your app's offline, which is pretty
bad for

your customers if you're doing this five,
six times

a day.

With Unicorn, you can set it up so that

when you deploy new code, it will start up

your app in the background, and only once
it's

ready will it switch it over and stop the

previous version, so your customers don't
notice an interruption.

For a database, I typ, by default go with

PostGres. Again, you've probably all heard
of it. Very

popular in the Rails community. I recommend
it to

people, because if you're on the Heroku free
plan,

it's probably what you're using already, because
it's their

default database.

The only controversial item on this list,
because all

lists should have at least one controversial
item, is

probably Monit. And it's a bit controversial
because there

are some amazing monitoring tools out there
which are

written using Ruby syntax for defining rules.
I keep

coming back to Monit because it's tiny. Its
syntax

isn't quite as nice to work with as some

of the others, some of the others, but it

uses absolutely no memory, and once you set
it

up, it just stays out of the way and

works, which is pretty much what I look for

in a monitoring tool.

For setting up your server, so your server
provisioning

tool, I recommend getting started with Chef
Solo. Chef

Solo is the tool you saw me using in

the first demonstration. And it's designed
for deploying individual

servers from your development machine. What's
really nice about

Chef Solo is it's built on top of a

much bigger piece of software called Chef
Server. And

Chef Server is designed for when you need
to

deploy lots of servers in complicated configurations.

So, if you need, eventually, to move on from

your simple single-box configuration to much
more complicated ones,

almost everything you've learnt using Chef
Solo will be

applicable to Chef Server.

I mentioned, previously, there were lots of
deployment tools,

automation tools out there for Rails. Probably
the best-known

one is Capistrano. At the moment, there are
two

stable versions of Capistrano out in the wild.
There

is Capistrano 2 and Capistrano 3. Most of
the

documentation and tutorials that you find
will probably be

for Capistrano 2.

Despite that, I recommend starting off with
Capistrano 3,

if you are starting from a blank slate. The

reason for this is that version 3 was a

complete ground-up rewrite of version 2. In
particular, it's

just a rake application. So Capistrano 3 is
just

a, a layer on top of rake that provides

deployment-specific functionality. And that
means that any knowledge you've

already got of how to write rake tasks and

how to interact with rake tasks and how to

debug them is directly applicable to Capistrano
3. And

that just makes the learning curve that little
bit

shallower.

So next steps, if you're gonna give it a

go. I suggest you go and get a VPS.

I suggest starting off with a one gig one.

You can run a Rails app on a 512

meg VPS. You can also spend a huge amount

of time troubleshooting out of memory hours,
which I

think is the least enjoyable thing possible.
You can

get a one gig VPS for about ten dollars

a month at DigitalOcean, I think. They're
also a

sponsor, and they're really generous with
free credit. So

if you corner them in a booth, I imagine

you can get at least a few months of

experimentation for free.

So, the url at the top there, talkingquickly
dot

co dot uk slash deploying_rails. That's a
page specific

to this presentation, and it contains links
to two

key tutorials. The first one goes through
step-by-step how

to set up a Rails server in the way

I did in the first video, and the second

one is how to set up Capistrano with a

new or existing Rails application, as per
the second

video.

They also include the complete sample code
that I'm

actually using in production, pretty much
every day, for

deploying new Rails apps. So, hopefully, the
fairly large

amount of time I put in originally trying
to

work out how to get a working configuration
management

set up, you can take that as a starting

point and get it started very quickly. It
should

take maybe an hour and a half to get

your first server set up and an app deployed

to it.

I mentioned at start, I've also written a
book

about this. Reliably Deploying Rails Applications.

This was basically

the reference that I wish I'd had when I

started doing this. It uses exactly the same
sample

code as those tutorials. So if you start off

using those and find you want to do more

complicated things,

hopefully that will be a useful reference.

And there's a discount code, et cetera, for
RailsConf

attendees on that link.

Finally, I'll be in the AirPad Pit at 4:30

today. So if anyone wants to give this a

go for real and wants a hand setting stuff

up, I'd be very happy to help out. Yeah.

Thank you so much for coming to the talk.

I've really enjoyed doing this. Let's have
some questions.