BEN DIXON: OK. So, I'm Ben.
I'm a Rails developer and the author
of a book called Reliably Deploying Rails Applications.
I spend a lot of people - a lot of people?
A lot of time teaching people Ruby at
MakeItWithCode dot com,
which is an online mentor-driven class
for people who are new to Ruby
but want to get started making things quickly.
I spend the rest of my time working with
the awesome guys at HelloSwimio, where we make cool
tech for swimmers. So we make things like the
time tabling system for the Olympic pool in London,
the speedo swim tracking app, and the first automatic
lap tracker for the Pebble smart watch.
So, before we get started, could you put your
hands up if you've deployed a Rails app of
some sort to Heroku. Awesome. Now keep your hand
up if you've deployed a Rails app to a
server that you've set up yourself. Good. That's less
people.
So if someone came to me and said, I
want to deploy my first Rails app, how should
I do it? I wouldn't think about it, I'd
just say use Heroku. And I'd say use Heroku,
because Heroku makes deploying apps incredibly simple. You just
commit your code, Heroku create, git push Heroku, and
magically, a couple of minutes later, your application is
live, on the internet, for the world to see.
And because Heroku has made this process so simple
and so easy, it's really easy to take that
server stuff and kind of put it in a
bucket of things we don't look at. Because it's
complicated and Heroku makes it really easy, so why
would we bother?
And what I'd like to demonstrate in this talk
is that, in the same way Rails makes building
web applications really, really easy, there are lots of
great tools out there that make setting up servers
for them and deploying to those servers really easy.
And once you get the hang of these tools,
you can do some really cool stuff.
Before we get started, two important bits of terminology,
which probably you already know. A PaaS, a platform
as a service, and by that I mean something
like Heroku. So something that just abstracts all of
your infrastructure away so that you don't have to
worry about it.
On the other scale, you've got a VPS, a
Virtual Private Server, and for the purposes of this
talk, I just mean some sort of Linux server
running the cloud. So the type of thing you
get from Linode or RackSpace or DigitalOcean.
So, to get started, I want to take a
little look at what's going on behind the scenes
when you deploy a typical Rails application. How do
we go from Rails S on our development machine
to a Rails app on a VPS somewhere serving
our application to the world?
So, here we've got a beautiful picture that I've
drawn, because I'm new to Keynote. And the black
box around the outside is our VPS. It's a
single VPS. And this is the first thing that
seemed to surprise lots of people. For a simple
production configuration, you only need one VPS. In the
same way, in development, you can run your entire
Rails application on your development machine. For simple production
application, you can run everything on a single VPS.
And that VPS will have three main components on
it for almost all Rails applications. You're gonna have
a web server, you're gonna have an app server,
and you're gonna have a database server. So, when
someone types in the address of our website, a
request comes in. It hits our VPS, and the
first place that request is gonna go is our
web server. Now, if that request was for a
static file, and by a static file, I basically
mean, anything that goes in the Rails asset pipeline
or anything that you might find in the public
folder, the web server's gonna deal with that directly.
It's gonna send that file straight back to the
user for display in their browser and that's the
end of it.
If, however, that request is for a dynamic page,
so for a page in our Rails app, essentially,
that's gonna get passed back to an app server.
Now, when we run Rails S in development, what
we're starting is an instance of an app server.
So, the app server here is Rails S running
on our VPS. So the web server passes the
request back. The app server generates the response and
then passes it back to the web server, which
then returns it to the user.
And in development, we're probably used to the web,
the app server being something like Thin or WebRick.
In production it might also be Thin or Webrick.
It might also be something a bit more advanced
like Puma or Unicorn.
Finally, for pretty much all Rails applications, we're gonna
want some sort of database server. And that's probably
gonna be PostGres or mySQL or Mongo. And, again,
we don't need a separate database server. This all
runs on our single VPS.
So, to recap, a request comes in and it
hits our web server. Probably EngineX or Apache. If
it's for a static file, the web server deals
with it. If it's for part of our Rails
application, it gets passed back to the app server,
which is basically Rails S, and the app server
may communicate with the database server, read some data,
write some data, and then the app server constructs
the response, sends it back to the web server,
which serves it to the user.
So, you'd be forgiven for looking at that and
thinking, well, that already looks more complicated than Heroku.
Why on earth would you ever do that? Why
not just use Heroku?
Well, I started doing this, if I'm honest, because
of cost. I had quite a lot of side
projects, and as you probably already know, most past
solutions charge by the process. And a lot of
my personal projects and side projects had background jobs.
Now when you've got five or six side projects,
all of which would need two Heroku dynos, suddenly
that bill at the end of the month starts
to get quite substantial.
And so I wanted to save money by deploying
to my own servers. And I did. It's worth
bearing in mind, if that's also your motivation, that,
as with any new skill, you have to invert.
Invert? You have to put in a certain amount
of time to learn it. If you were to
account for the time I put into learning this
to begin with, at market rates, I'm fairly sure
there would have been no saving at all. Personally,
they're my own projects. I wasn't paying myself market
rates. And I was happy to treat it as
an investment which would pay off in the long
term, and it has.
Actually, it turned out, the biggest benefit of learning
how to do this was nothing to do with
money. It was the infrastructure went from being this
kind of necessary evil that I have to have
in order to run my Rails apps, to just
another tool that I could use for building interesting
stuff. And the best example of that, for me,
is Make It With Code. We provide our students
with a cloud IDE that has Ruby pre-installed. And
that means they can hit the ground running. They
can start writing Ruby code really quickly.
And the techniques that we use setting those up
automatically for all of our students are exactly the
same techniques that I'm gonna cover in this presentation
for setting up Rails apps. So it means that
our infrastructure has gone from being this thing that
we have to have in the background, as like
a supporting thing, to something that actually adds value
to our product in itself. It's a part of
our product.
So if you decide to give this a go,
I really, really urge you not to do it
even slightly like I did it originally. Because it
was terrible. I started off by getting a VPS.
That bit was good. That bit worked. I Googled,
how do I set up a Rails VPS. I
found all these great tutorials that said, type in
this command, then that command, then edit this config
file. And I found loads of things on StackOverflow
that said, here's what to do when that goes
wrong and that goes wrong. And eventually I had
this working server. And it was great.
And it broke a few times in the first
few weeks. But we fixed that when it happened.
And, by and large, it worked for quite awhile.
And it worked until it didn't work, and when
it didn't work, I was in France, on a
holiday with my family, and it was a fairly
rural bit of France, and we had, we had
a wifi connection. But it was only one wifi
connection for, basically, the village. And to access that,
we, we could access it by sitting half in
a wardrobe on the top floor of the house,
which was nice.
And, so one morning I woke up and I
was sitting in the wardrobe with a coffee and
checking my email, and I had a wonderful, a
lovely email from my host to say, Dear Valued
Customer, your server is dead, you need to make
a new one, Lots of love, from your hosts.
And I thought, good. Right. So I set up
a little desk in this wardrobe and for the
rest of the day, my family sat by a
river. They drank wine, ate cheese, mocked me slightly,
and I sat in a roof, in a wardrobe,
in a heat wave, and pretty much followed the
same tutorials I employed the first time. Made the
same mistakes that I'd made the first time, and
I got the server back up, but it really
wasn't any easier the second time. And that seemed
kind of wrong to me. That it seemed really
frustrating.
So I figured there must be a better way
of doing this. If I'm honest, actually what I
thought was, I can't believe there isn't already a
better way of doing this, I should invent it
and then I'll be rich and famous and everyone
will love me. Luckily I took a step back
and thought, OK, this probably already exists.
And, of course, it did. And what I wanted
was a configuration management tool. And a configuration management
tool is really simple. It allows you to define
and automate the commands that it takes to set
up the server to do a particular thing. And
what's amazing about a configuration management tool is once
you've done it once, doing it again and again
and again is completely trivial.
So, that time you put in to start with,
it will probably only be one command to set
up an identical server the next time. And to
give you an idea of just how simple it
can be, I've got a brief demonstration of the
process that I actually move, at the moment, on
a pretty much weekly basis for setting up new
servers.
At the end of this presentation, I'll link to
tutorials of how to do this yourself and the
sample code. So please don't worry about following it
step by step, I just want to give an
idea about how simple it can be.
So, here you can see, on the left, your
left, there is a list of servers that I've
deployed in the past. And I'm using my configuration
management tool of choice, Chef Solo, which uses JSON
files for defining what should go onto a particular
type of server.
And on the bottom you can see I've got
a couple of web servers for Make It With
Code, which at the moment is running a selection
of Rails and Sinatra apps. And I want to
write a new one. And to do this, I
just create a new JSON file, web3 dot makeitwithcode
dot com. I save it, and then this server,
it's gonna be pretty much the same as web2.
It's just running Rails apps.
So I just copy all the JSON from my
previous server, but it into the new one, and
the bit I want to draw your attention to
is, if you look at the bottom of that,
you've got this run list with a list of
things that are being called roles. And pretty much
all configuration management systems have some sort of analogous
concept to this, which, kind of individual components that
you can mix and match to define what should
go onto this server that you're setting up.
So I've defined in the past, what goes on
an internet server. What goes on a PostGres server.
And now if I want that, I can just
drop that role in, and I know that it
will be set up in exactly the way that
I want it. So on this server I want
Redis. I didn't on the previous one. So I
can just uncomment it. Uncomment it, yeah. And I'll
get exactly the Redis configuration I want.
The good bit, the fun bit, is once I've
done this, I can type on command - knife
solo bootstrap - and then the address of the,
the VPS I've created. Presenter. And then leave it
for about twenty minutes, go and get a coffee,
and the output that will scroll through, it will
tell me what commands it's applying. It will show
me diffs of any configuration files that it's changing.
And if it goes wrong, which it doesn't very
often, it will tell me exactly what it was
doing when it went wrong, so I can fix
it.
And at the end of this process, I will
have a Rails server which is identical the last
one I set up, and the one before that,
and I can check that this works. I can
fire up a web browser. I can go to
web3 and the address that I gave it, and
I'll get this wonderful Welcome to EngineX page, which
says, that server is set up and it's ready
to use.
So, what I'm trying to demonstrate with that figure
is that configuration management is basically, don't repeat yourself
for setting up servers. If you think back to
the first time you used, say, Devise, for example,
the first time I used Devise, it took me
ages to get all_auth and things like that working.
And then, six months later, I came back to
it, and didn't really remember any of the stuff
that I'd done to get it to work. But
I could go back to the code I wrote
the first time, I could look at that code,
copy bits of it, and it was much quicker.
Now, the problem with just copying commands or entering
commands manually when setting up the server is there
is no natural audit trail of what did I
do to make this work the first time. So
there's a good chance, the second time you're gonna
do it, you'll make the same mistakes as the
first time.
With a configuration management utility, you naturally create this
audit trail as you go along. So that, in
time, that time you invest up front, it goes
much, much further in the long run.
So, hopefully that's shown that the bit where people
often trip up, getting a server setup, it doesn't
need to be that difficult, and I'll give you
the exact sample code at the end, which you
can use if you want a head start.
But something like Heroku, it doesn't just make setting
up a server really easy. It makes deploying to
one incredibly easy. Git push heroku. It pretty much
couldn't be easier. And I think it's fair to
say that the first time I used Heroku, it
was completely revolutionary.
So I came to Rails from Python/Jengo, and before
that, PHP, and I'd used a whole range of,
kind of, home grown deployment systems, which ranged from
trying to remember what files you've changed and FTP
them across to all sync things and custom shell
scripts. And the one thing they all had in
common was they were flaky and deployment was something
you didn't want to do, because it would probably
break. So you maybe deployed a couple of times
a week.
So I used Heroku for the first time and
it was amazing. Suddenly I could apploy. Apploy? Deploy
ten times a day. And it was just fitted
into my normal workflow.
So I was really keen that if I was
deploying to my own servers, it had to be
as simple as that. It had to be one
command to deploy. And luckily, as is often the
case in the Rails community, loads of very clever
people wanted exactly the same thing, and so have
developed some awesome tools, some awesome gems, to make
it really simple to set that up.
In the Rails community, I think the best known
of these is probably Capistrano, which I'll talk about
a bit more towards the end of the presentation.
But, first, another brief demo of how simple it
is to add Capistrano into an existing, in this
case, Rails 4.1 application. Again, at the end, I'll
give details of step by, a step by step
tutorial and the ensemble code for doing this. So
please don't worry about the individual steps. I just
want to give an idea of what's involved.
So, to get started, Rails 4.1 application. I just
drop in the Capistrano gem and a few supporting
helper gems. Save that, and then add it to
git and, sorry, bundle and add it to Git.
Under the hood we're still using Git to deploy
in the same way Heroku does, so it's important
to make sure changes like that are committed.
And then running cap install, which just generates some
basic configuration files. The first one of these is
a cap file. This is just a rake file
for capistrano. And here, I'm just telling it that
I want to use those helper gems that I
included in the first step. So I'm telling it
that I want to use rbenv to manage what
Ruby version I'm using. I want it to install
gems for me, compile assets for me, and apply
migrations. All the normal stuff you do when setting
up a Rails app.
We've then got our main config file, just deploy
dot rb. And here I'm just giving the app
a name, telling it which Git repository to deploy
from, and getting rid of the load of the
boilerplate for the simple application. I'm not gonna use
it. Uncommenting out a few other bits of boilerplate.
And then finally I'm gonna add an rbenv specific
block that tells it to always use a particular
Ruby version. I think, in this case, 2 point
1 point 1, when deploying this app.
Finally, we have stage configuration files, so if you
have a production server and a staging server, and
here I'm just telling it that the production server
is going to be web3 dot makeitwithcode dot com,
which is the server we set up in the
first demo. And then that should be made available
to the public at, imaginatively, web3 dot makeitwithcode dot
com.
Then, firing up a console, running a command called
setup config, and you can think of this like
heroku create. This is just telling that server, or
preparing that server for a new application. It's generating
a few config files, it's creating a new database
yml and telling EngineX that it needs to serve
a new site. Then running another helper task that
I added this time. Create database. This is just,
create a new database on that server. Making sure
there's a user that has permission to access that
server.
And this is the important command. Cap production deploy.
And this, if we're using capistrano, is our equivalent
to git push heroku. And if you look at
the output, it's doing some pretty standard stuff. It's
applying migrations, compiling assets and restarting the app server.
And once that completes, we can fire up the
web browser, visit it, and you can see we've
got a simple, if utterly useless Rails application.
And it has access to the database, it can
read and write, and now we can iterate on
that, and every time we want to make a
change, we can just make that change, commit it,
cap production deploy, and it really is as simple
as deploying it to heroku.
So one thing that we haven't looked at yet
is, it's very easy to set this stuff up,
but the other great thing about using a path
like heroku is that when it breaks at 2
AM, it kind of isn't our problem. They have
engineers and they get woken up at 2 AM.
And they go and fix it. And our application
just comes back up online again.
And we can't get around that completely. It's now
our problem when it breaks at 2 AM. But
luckily there are lots of tools out there which
mean we can minimize the number of times this
will happen. If you think in a typical production
Rails application, you probably got some exception handling. You
know some exceptions will happen from time to time,
and so you've got to have your begin and
rescues in that. And then hopefully you've also got
some sort of exception notification. And that means that
when an exception that you haven't planned for happens,
you get an email and you know that you
can go in, you should go in and fix
it.
So, there are an analogous set of tools when
you're setting up your own servers. And these are
called monitoring tools, and they're really, really simple. They're
essentially just a list of checks that say, here's
how something ought to behave. If it's not behaving,
here's a command that you can run to try
and fix it. If it's still not working, here's
the person to notify and how to notify them
there's a problem.
And that might sound a little bit daunting. So
every time you set up an application you're gonna
have to think about all these checks. In practice,
this is where configuration management comes in. So, if
you remember in that first demonstration, when I uncommented
that Redis role, as part of that Redis role,
when I originally wrote it, probably a year or
a couple of years ago, I included the monitoring
configuration for it, which basically says, is Redis working?
If no, restart it. If it's still not working,
email me.
And now that will get automatically setup every time
I include that Redis role. So there really isn't
that much duplication. Configuration management just does all of
this for us.
So to get a bit more practical, if you
decide to, to go ahead and try this. This
is the stack that I end up using, probably
99% of the time. For a web server, EngineX,
really popular in the Rails community. There's lots of
really great Rails-specific documentation out there.
For an app server, Unicorn. This probably isn't what
you're using in development at the moment. That's more
likely to be Thin or Webrick. Unicorn's a little
bit more complicated. But, for that little bit of
extra complexity, you get some really nice features. In
particular, it's very easy to set up what's called
zero-downtime deployment, which you may already be familiar with.
Essentially what this means is, the simplest way to
deploy an application is to deploy your new code,
stop the existing application, and then start it again
with the new code. The problem is, if you've
got a big Rails application, it can take two
or three minutes to start up. And that means
that, for two or three minutes, every time you
deploy, your app's offline, which is pretty bad for
your customers if you're doing this five, six times
a day.
With Unicorn, you can set it up so that
when you deploy new code, it will start up
your app in the background, and only once it's
ready will it switch it over and stop the
previous version, so your customers don't notice an interruption.
For a database, I typ, by default go with
PostGres. Again, you've probably all heard of it. Very
popular in the Rails community. I recommend it to
people, because if you're on the Heroku free plan,
it's probably what you're using already, because it's their
default database.
The only controversial item on this list, because all
lists should have at least one controversial item, is
probably Monit. And it's a bit controversial because there
are some amazing monitoring tools out there which are
written using Ruby syntax for defining rules. I keep
coming back to Monit because it's tiny. Its syntax
isn't quite as nice to work with as some
of the others, some of the others, but it
uses absolutely no memory, and once you set it
up, it just stays out of the way and
works, which is pretty much what I look for
in a monitoring tool.
For setting up your server, so your server provisioning
tool, I recommend getting started with Chef Solo. Chef
Solo is the tool you saw me using in
the first demonstration. And it's designed for deploying individual
servers from your development machine. What's really nice about
Chef Solo is it's built on top of a
much bigger piece of software called Chef Server. And
Chef Server is designed for when you need to
deploy lots of servers in complicated configurations.
So, if you need, eventually, to move on from
your simple single-box configuration to much more complicated ones,
almost everything you've learnt using Chef Solo will be
applicable to Chef Server.
I mentioned, previously, there were lots of deployment tools,
automation tools out there for Rails. Probably the best-known
one is Capistrano. At the moment, there are two
stable versions of Capistrano out in the wild. There
is Capistrano 2 and Capistrano 3. Most of the
documentation and tutorials that you find will probably be
for Capistrano 2.
Despite that, I recommend starting off with Capistrano 3,
if you are starting from a blank slate. The
reason for this is that version 3 was a
complete ground-up rewrite of version 2. In particular, it's
just a rake application. So Capistrano 3 is just
a, a layer on top of rake that provides
deployment-specific functionality. And that means that any knowledge you've
already got of how to write rake tasks and
how to interact with rake tasks and how to
debug them is directly applicable to Capistrano 3. And
that just makes the learning curve that little bit
shallower.
So next steps, if you're gonna give it a
go. I suggest you go and get a VPS.
I suggest starting off with a one gig one.
You can run a Rails app on a 512
meg VPS. You can also spend a huge amount
of time troubleshooting out of memory hours, which I
think is the least enjoyable thing possible. You can
get a one gig VPS for about ten dollars
a month at DigitalOcean, I think. They're also a
sponsor, and they're really generous with free credit. So
if you corner them in a booth, I imagine
you can get at least a few months of
experimentation for free.
So, the url at the top there, talkingquickly dot
co dot uk slash deploying_rails. That's a page specific
to this presentation, and it contains links to two
key tutorials. The first one goes through step-by-step how
to set up a Rails server in the way
I did in the first video, and the second
one is how to set up Capistrano with a
new or existing Rails application, as per the second
video.
They also include the complete sample code that I'm
actually using in production, pretty much every day, for
deploying new Rails apps. So, hopefully, the fairly large
amount of time I put in originally trying to
work out how to get a working configuration management
set up, you can take that as a starting
point and get it started very quickly. It should
take maybe an hour and a half to get
your first server set up and an app deployed
to it.
I mentioned at start, I've also written a book
about this. Reliably Deploying Rails Applications.
This was basically
the reference that I wish I'd had when I
started doing this. It uses exactly the same sample
code as those tutorials. So if you start off
using those and find you want to do more
complicated things,
hopefully that will be a useful reference.
And there's a discount code, et cetera, for RailsConf
attendees on that link.
Finally, I'll be in the AirPad Pit at 4:30
today. So if anyone wants to give this a
go for real and wants a hand setting stuff
up, I'd be very happy to help out. Yeah.
Thank you so much for coming to the talk.
I've really enjoyed doing this. Let's have some questions.