RICHARD SCHNEEMAN: All right. OK. Hello everyone.
AUDIENCE: Hello.
R.S.: Thank you. Thank you. Welcome to, welcome, let
me be the first to welcome you to RailsConf.
So, our, our talk today is Heroku 2014: A
Year in Review. It is gonna be a play
in six acts, featuring Terrance Lee and Richard Schneeman.
So, of course this is a year in review,
and Heroku does measure their years by RailsConf. So
this is from Portland to Chicago RailsConf year. The
Standard RailsConf Year.
As, as some of you might know, we are
on the Ruby Task Force, and, in fact, that
makes us Ruby Task Force members. And, of course,
this was a big year. We're gonna be talking
a little bit about app performance, some Heroku features,
and community features. So, first up to the stage,
I'm gonna be introducing the one, the only Mister
Terrance Lee. You might have recognized him in some
other roles. He hails from Austin, Texas, which has,
undoubtedly, the best tacos in the entire world. So,
he-
AUDIENCE: [indecipherable]
R.S.: Them's fightin' words, friend. So that he, he's
also sometimes known as the Chief Taco Officer. Or,
or the CTO. And something, something very interesting about
Terrance is, recently, he was inducted into Ruby Core,
so congratulations to, to Terrance. All right.
So, without further ado, Act 1: Deploy Speed.
TERRANCE LEE: Thank you, Richard. So, at the beginning
of the year Rails Standard Year, we focused a
lot on deployment speed. We got a lot of
feedback and realized deployment was not as fast as
it could be. And we wanted to make it
faster. So, the first thing we set out to
do was to actually do a bunch of measurement
and profiling to look at where things were slow,
and how we could make it better, and to
kind of gage, like, the before and after and
know when the good points were to kind of
stop and move on to other things. Cause you
can never make, you can never, you will never
be done with, like, performance improvements.
So, after about six months of work on this,
we managed to cut down the deploy speeds for,
across the platform for Ruby by about forty percent.
So it's a pretty decent speed improvement. And, in
order to do this, we mainly looked at three
various ways to speed this up.
The first thing was running code in parallel, so
running more things, running things, like, more than one
thing at one time. If you cache stuff you
don't have to do it again, and, in general,
just like, cutting out code that doesn't need to
be there.
So, with the parallel code, we worked with the
bundler team on Bundler 1.5. There was a pull
request sent by CookPad that was sent in to
add parallel bundler install for Bundler 1.5. So if
you actually aren't using this yet, I would recommend
upgrading your bundle to at least Bundler 1.5. And
the bundler added this dash j option, which allows
you to specify the number of jobs to run.
And this is, basically, if you're using MRI, it
forks and does a num, these number of sub-processes,
and if you're on JRuby or Rubinius it actually
just uses threads here.
And the benefit of doing this is, when you
actually do bundle install, the dependencies that get installed
get downloaded in parallel, so you're not waiting on
network traffic sequentially anymore, and in addition you're also
install gems in parallel. And this is mostly beneficial,
especially when you're running native extensions. So if you
have something like Nokogiri, that takes a long time.
Oftentimes, you notice, you just like hang and wait
for it to install and then it installs the
next thing, so this allows you to install that,
basically, in the background and then go and install
other gems at the same time.
Also, in Bundler 1.5, Richard actually added this function
that allows people, allows bundler to auto-retry failed commands,
so initially, before this, we would, when we run
bundle install and something would fail because of some
odd network timeout, like during one chance, you would
have to basically repush again, no matter where you
were in the build process. So, by default now,
Bundler actually will retry clones and gem installs for
up to three times by default.
So, it will continue going during the deploy process.
And so is anyone here actually familiar with the
PIGz command? So, just Richard? So, PIGz is Parallel
Gzip, and the build and packaging team at Heroku
worked on this feature, or worked on implementing this
at Heroku using the PIGz command, and in order
to understand the kind of benefit of using something
like this, when you push an app up on
Heroku during the compile process, it actually builds these
things at Heroku that are called slugs. And basically
it's just, like, a tar of your app directory
of everything after the compile face is run.
And, originally, we were just using SquashFS, initially, and
then we moved to kind of just tar files,
and we noticed that one of the slowest point
in the actual build process was actually just going
through and compressing everything in that, that file directory,
and then pushing it up onto S3 after that
was done. And so one of the things that
we looked into was, is there a way we
can make that faster? So, if you ever push
a Heroku app and then you basically, like, wait
when it says, like, compressing and then it goes
to done, like that's the compressing of the actual
slug.
And we managed to use slug, PIGz to now
improve that by sniffing them out. I don't remember
the actual performance improvement, but it was pretty significant,
and the only downside was in certain slugs, the
slug sizes are a little bit bigger. But the
performance trade off was worth it at that time.
The next thing we started doing was looking into
caching. So anyone here using Rails 4? So pretty
good amount of the room. So the one thing
is that we did which differ from Rails 3,
thanks to a bunch of the work that's happened
on the Rails Core team with us is that,
we can now cache assets between deploys. This wasn't
possible in Rails 3 because the cache was, you
couldn't actually reuse the cache. There was times when
the cache would basically be corrupted and then you
would get, like, assets that wouldn't work between deploys.
So the fix there was you actually have to
remove the assets between each deploy in some Rails
3 builds. But it wasn't consistent, so sometimes it
would work and sometimes it didn't. And on Heroku
that's not something we can rely on in an
automated fashion.
But luckily a lot of that stuff has been
fixed for Rails 4, so now we cache assets
between deploys in Rails 4. And so if we
look at Rails 3, I guess this got cut
off, but this is supposed to say about, like,
thirty-two seconds for a Rails 3 deploy, and then
on Rails 4 it got, for the average we,
we measured the steps in the, in the build
process, and on Rails 4, the perk fifty was
about fourteen point something seconds. So a pretty significant
speed improvement there, both due to caching and other
improvements inside of Rails 4 for the asset pipeline.
So the other thing we also looked at was
just, if there's code that is doing extra work,
if we remove that, it will speed up the
build process for everyone who's deploying every day. So
one of the first things that we did was
actually stop downloading bundler more than once. So, initially,
when you, when we do the Ruby version detection,
we actually have to download bundler, and then basically
run that to get the version of Ruby to
install on the application. And then again, we would
then download and install it again because it was
run in a separate process for the actual, like,
installing of your dependencies. And one of the things
we did was to actually just stop doing that
and we would cache the Bundler gem so we
don't have to download that two or three times
during the build process. So, so cutting network IO
and other things.
We also started removing, there was like duplicate checks
between detection of what kind of app you were
using so, and, bin detects. We would use it
to figure out what kind of app you have,
like if it was a Ruby app, a Rack
app, a Rails 3 app, a Rails 4 app,
stuff like that. And then, again, since it was
a separate process in bin compile, we would have
to do it again. So, Richard actually did a
bunch of work to refactor both detect and release,
and so now detect is super simple. It literally
just checks if you have the gemfile file there,
and then all the other work is now deferred
to bin compile. So that means we're only doing
a bunch of these checks once, like examine your
jump file, checking what gems you have. So not
doing that two or more times.
And, if you haven't watched this talk, he gave
this talk at Ancient City Ruby. I don't actually
know if the videos are quite up yet. But
Richard does a talk about testing the untestable, so
if you're interested in learning how we test the
build pack, you should go watch this talk.
So I'd like to introduce Richard, cause he's gonna
present on the next section. So Richard loves Ruby
so much that he got married to her. I
think he got married last, last year.
R.S.: Right before our last RailsConf.
T.L.: Yeah. Right before our last RailsConf. I remember
that. He's also on the Rails Issue Team, and
he's one of the top one hundred Rails contributors,
according to the Rails contributor sites. And you might
also know him for his, this gem called sextant
that he released for Rails 3. Basically, I remember
back in the day, developing Rails apps, when I
wanted to basically verify routes, I would run the
rake routes command, and it would, you know, boot
up the Rails environment and you'd have to wait
a few seconds and then they would print out
all the routes. And then if you wanted to,
like, rerun it using grep, you would keep running
it again.
So, a lot of us, when we're doing development,
already have, like, Rails running in a server while
we're testing things and whatnot. And so what sextant
does is it allows it, supports basically looking at
the routes that are already in memory and just
allowing you to query against the programmatically, and then
that has a view for doing this. And this
was also just merged into Rails 4. So if
you're using Rails 4 or higher, you actually don't
need to sextant gem and it's now built in.
Richard and I both live in Austin, and so
when people come visit, or actually when I'm in
town, which isn't often, we have Ruby meet ups
at Franklin's Barbecue, so if you guys are ever
in town, let us know and we'd be more
than happy to take you to a meet up.
R.S.: All right. So the, for the first part
of this, this act, we're gonna be talking about
app speed, but before we talk about app speed,
we're actually gonna talk about dimensions. So, the, the
document dimensions are, let me see. Here we go.
Were originally written in wide-screen, but the screens here
are standard.
There we go. So. You're actually gonna get to
see all of the slides, as opposed to just
having some of them cut off. So, OK.
On, on app speed. The first thing I want
to talk about is, is tail latencies. Is anybody
familiar with tail latencies? OK. The guys in the
Heroku t-shirts and somebody else.
OK. So this is, this is a normalized distribution.
WE have, on one side, the number of requests.
On the, on the other side, we have the
time to respond. So the further out you go,
the slower it's gonna be. And we can, we
can see this is the distribution of our requests.
So over here, it's super fast. Like you love
to be that customer. You're super happy.
Over here, we have a super slow request, and
you don't want to be that customer and you're
pretty unhappy. So right in the middle is our
average, and I'm sure they talked a ton about
why the average is really misleading in the, in
the last session with Skylight.IO. But, but we're basically
saying that, roughly fifty percent of your, of your
customers, fifty percent of your traffic, is going to
get a response time at this or, or lower.
So, like, this is, this is pretty decent. We
can say, like, fifty percent of the people who
come to our web site get a response before
then. Moving up the distribution, to something like perk
ninety-five, we say ninety-five percent of everyone who visits
our traffic will get a response by now. So
I'm gonna be using those terms, perk fifty, perk
ninety-five, that refers to the percentage of, of requests
that come in that we can respond by.
So this is kind of theorized. This is an
actual application. I, and one thing that you'll notice
is that it's not perfectly normalized. Like, it's not,
like both sides are not symmetrical. We kind of
like, steeply shoot up, and then we have this
really, really long tail, and, and this is kind
of the, what I'm referring to when I'm saying
tail latencies.
So, yes, somebody actually might have gotten a response
in zero milliseconds. You know, I doubt it, but
somebody for sure did get a response in 3000
milliseconds, and that's a really long time to wait
for your request to actually come in and, and
get finished. So even though somebody is getting really
fast responses, and your average isn't bad - your
average is under 250 milliseconds - one customer might
be getting a really slow response and a really
fast response, and, and the net is a bad
experience.
So, the net, it, it just, it's a very
inconsistent experience. So whenever we're talking about application speed,
we have to consider individual request speed and average,
but also consistency. How consistent is each request?
So, how do, how do we do this? What?
How can we, how can we help with this?
Well, one of the things that we launched this
year was PX dynos. So a PX dyno -
a typical dyno only has 512 megabytes of RAM.
It's a shared infrastructure. A PX Dyna has six
gigabytes of RAM and eight CPU cores, which is
a little, a little nicer, a little better, a
little bit more room to play.
And, and it's also real hardware. So, or, it's,
it's not on the same shared infrastructure. So you
can, you can scale with Dynos, you, you can
also scale inside of Dynos. And that's kind of
two, two important parts that we're gonna, gonna have
to cover. So, of course, whenever you have more
requests that you can possibly process, you want to
scale up and say, I'm gonna have more Dynos.
But what happens if, if you're not making the
best use of everything inside of your dyno? Previously
with 512 megabytes of RAM, you could just, you
know, throw a couple Unicorn workers in there and
you're like, oh, I'm probably using most of this.
Like, if you put two unicorn workers in a
PX Dyno, you're not making the most use of
it all.
So, recently, I am super in love with, with
Puma. Evan, this is Evan Phoenix's web server that
was originally written at, to kind of show case
Rubinius. Guess what? It's really nice with MRI as
well. Recently we've gotten some Puma docs, and so
I'm gonna talk about Puma for just, for just
a little bit.
So, if you're, if you're not familiar. I was
totally off on the formatting. So, Puma handles requests
by running multiple processes, or by multiple threads. And
it can actually run in something called a hybrid
mode, where each process has multiple threads. We, we
recommend this, or I recommend this. E- if one
of your processes crash, it doesn't crash your entire
web server. It's kind of nice.
And so the multiple processes is something that we're
pretty familiar with. As Rubyists, we're familiar with forking
processes. We're familiar with Unicorn. But the, the, the
multiple threads is a little bit different.
Even with MRI, even with a, something like a
global interpreter lock, you are still doing enough IO,
you're still hitting your database frequently enough, maybe making
API calls to like, Facebook or GitHub status, being
like, hey, are you still up?
And, and this will give our threads time to
kind of jump around and allow others to do
work. So you, you can get quite an extra
bit of performance with, there.
So, we're actually gonna be using Puma to scale
up inside of our Dynos. So once we give
you that eight gigs of RAM, we want to
make sure that, that you can, you can, you
can make the most use out of it.
In general, with Puma, more processes means more RAM,
and more threads are gonna need more CPU consumption.
So, you want to, you want to maximize your
processes and maximize your threads, kind of without going
over. As soon as you start swapping, as soon
as you go over that RAM limit, your app's
gonna be really slow and that kind of defeats
the purpose of trying to add these resources.
Another issue is that I had kind of never
heard of until I started looking into all of
these multiple web servers, is slow-client. So if somebody's
connecting to your web site via like, a two
G over like a Nokia candy bar phone, uploading
like photos or something like that, like that is
a slow client, and if you're using something like
Unicorn, it can deDOS your, your site, because each
one of those requests takes up an entire Unicorn
worker, whereas Puma has a, has a buffer, and
it buffers those requests as similar to the way
NginX does.
One other thing to consider with Puma is, so
I'm mentioning threads, I'm talking, talking about threads. Ruby,
we're not necessarily known as the most thread-safe culture.
Thread-safe community. And so a lot of apps just
aren't thread-safe. And so some, you might take a
look at Puma and be like, hey, that's not
for me. You can always set your maximum threads
to one, and then now you're behaving just like
Unicorn, except you have the slow-client protection, and whenever
you get that gem that's bad or you, like,
stop mutating your constants are runtime or something, then
you can maybe bump up and try multiple threads.
OK. So, I'm, I'm talking about consistency and I'm
talking a lot about Puma. How does that all
kind of boil down and help? So, does anybody
think that sharing distributed state across multiple machines is
like really fast? Maybe. I, OK. Good.
What about sharing state inside of memory on the
same machine? Is that faster? OK. All right. I
think we're in, in agreement. So, a, a little
bit of a point of controversy. You might have
heard of the, the Heroku router at some point
in time. And this, the router is actually designed,
not randomly, but it is, it is designed to
use a random algorithm. And it basically will try
to deliver requests as fast as humanly possible, or
computerly possible, to individual dynos. So it's like, it
gets the request. It wants to get it to
your dyno as fast as it possibly can.
And adding any sort of additional overhead of distributed
locks or queues is gonna be slowing that down.
Once inside of your, in your, your process, Puma
or Unicorn has, in memory, state of all of
its own processes, and is capable of saying, oh,
hey, this process is busy. This process is not
busy. I can do really intelligent routing and, and
basically, for free.
It's really fast. It, it took a little bit
of convincing for me. So does anybody else need
to be convinced?
AUDIENCE: Yeah.
R.S.: OK. Good. Cause otherwise I could totally just
skip over the next section of slides.
So, this is, this is a graph produced by
the fine developers over at, at RapGenius, and on
one side we will actually see a percentage of
requests queued, and on the bottom we are gonna
be seeing number of dynos. So the goal is
actually to minimize request queuing, like this, this is
time that your customers are waiting that you're not
actually doing anything.
You, so you, you want to minimize that queuing
with the smallest number of resources, so the smallest
number of dynos. This top line, we actually have
is what we've currently got now. The random routing
with a, a single-threaded server. And, like, this is
pretty bad. It, like, starts out bad and it,
like, it doesn't even like trend towards zero. So
this is probably bad. So this is using something
like Webrick in production.
So. Don't use Webrick in production. Or, or like
even, even thin. In single-threaded mode. So, on the,
on the very bottom, we actually have a, like,
mythological, like, if, if we could do all of
that distributed shared state without, and locks and queues,
without having any kind of overhead, we can see
that, basically, it just drops down to zero at
about, you know, in their case, about seventy-five dynos,
and then just, you know, it's straight zero. There's
no queuing.
And it's great. And this would be amazing if
we could have it. But unfortunately there is a
little bit over overhead. What was really interesting to
me is this second one, which is not nearly
as nice as that mythological intelligent router, but it's
kind of not too far off. This is still
our random routing, and, and this was actually done
with Unicorn, and workers set to two. So basically,
once we get the, the request to your operating
system, it's like one of those two workers is
free and can immediately start working on it.
Some, some interesting things known about this is, for
the non-optimal case, for the, we basically don't have
enough dynos to handle this, so that might happen
is, you know, you got on Hacker News or
whatever, slash dotted, Reddited. SnapChatted. Secreted. I don't know.
And it does actually, eventually approach ideal state. So,
it, it gets even better, and unfortunately they kind
of stopped at, at two processes, but it gets
better, the more concurrency that you add. So if
you had three or four workers or, again, if
you're using something like Puma, and each one of
those workers is running, like, four threads, now you
have, like, a massive amount of concurrency that you
could deal with all of these requests coming in.
So, the, the, the, if, again, and we're looking
for consistency. We want that request to get to
our dyno, immediately be able to process it. So,
you can use Puma or Unicorn to maximize that
worker number, and, again, distributed optimi- distributed routing is
slow. In memory, routing is relatively quick.
On, again, just in, in the whole context of
speed, Ruby 2.0 came out and this was awhile
ago. It's got GC, it's optimized for Copy on
Write. In, in Ruby, extra processes, process forks actually
become cheaper. So the first process might take seventy
megabytes, the second one twenty, and ten and seven
and six. So, if you get a larger box,
you can actually run more processes on them. If
you get eight gigs on one box, you can
run more processes than you can if you had
eight gigs across eight boxes.
So, again, more processes mean more concurrency, and more
concurrency means consistency. If you are using workers, you
can, you can also scale out with Resq pool,
and if your application's still slow, we rolled out
a couple of really neat platform features. One of
them is, is called HTTP Request ID. So as
a request comes into our system, we will actually
give it a uuid, and you can see this
in your router log. And then we've got documentation
on how to configure your Rails app so it
will actually pick this up and use that uuid
in tagged logs.
So, like, how is this useful? So, if you
are getting, like, and out of memory error, or
if your request is taking a really long time
and you're like, ah, like, that request is timing
out and, you know, Heroku's returning a response and
we don't even know why. Now, if the request
id is tagged, you can actually follow along between
your two logs and be like, oh, it's hitting
that controller action. Maybe I should be sending that
email in the background as opposed to having to
actually block on it. So you can trace specific
errors.
We also launched Log Runtime Metrics awhile ago, and
this is something that we'll actually put your, your
runtime information directly into your logs. You can check
it out. Liberato will automatically pick it up for
you and make you these, these really nice graphs.
And, again, if you're doing something like Unicorn or,
or Puma, then you want to get as close
to your RAM limit without actually going over.
OK. So, the, the next act in, in our
play, again, introducing Terence, is, we'll be talking about
Ruby on the Heroku stack and in the community.
T.L.: Thank you. So I know we're at RailsConf,
but I've been doing a bunch of work with
Ruby, so I wanted to talk about some Ruby
stuff. So who here is actually using Ruby 1.8.7?
Wow. No one. That's pretty awesome. Oh, wait. One
person. You should probably get off of it.
[laughter]
But. Who is using Ruby 1.9.2? A few more
people. 1.9.3? Good amount of people here.
So, I don't know if you guys were following
along, but Ruby 1.8.7 and 1.9.2 got end-of-lifed at
one point. And then there was a security incidence,
and Zachary Scott and I have volunteered to maintain
security patches till the end of June. So, if
you are on 1.8.7 and 1.9.2, I would recommend
hopefully getting off some time soon, unless you don't
care about security or want to back port your
own patches.
And then Ru- we recently announced that Ruby 1.9.3
is also getting an end of life in February
2015, which is coming up relatively quickly. It's a
little less than a year away now, at this
point. So, please upgrade to at least 2.0.0 or
later.
And, during this past Rails Standard Year, we also
moved the default Ruby on Heroku from 1.9.2 to
2 dot 0 dot 0. We believe people should
be at least using this version of Ruby or
higher.
And, if you don't know yet, you can declare
your Ruby version in the gemfile on Heroku to
get that version. And, we've also are pretty, pretty
serious about supporting the latest versions of Ruby. Basically,
the same day that they come out. So we
did this for 2.0.0, 2.1.0 and 2.1.1, and in
addition we also try to support any of the
preview releases as, whenever they get form release, so
we can, as a community, help, help find bugs
and test things like, put your staging app on
new versions of Ruby. If you find bugs then,
hopefully, we can fix them before they actually make
it to the final release.
And, with regards to security patches, if there are
any security releases that come out, we make sure
to release them that day as well. We take
security pretty seriously.
So, once a security patch has release and we've
patched those Rubies, you have to push your app
again to get that release. And the reason we,
well, a lot of people ask us, like, why
we don't do that. Why we don't just automatically
upgrade peoples' Rubies in place. And the reason it,
the reasoning here is that, not, there might be
a regression to the security patch, or maybe the
patch level is not 100% backwards compatible. There's a
bug that slipped through. But you probably want to
be there when you're actually deploying your application, in
case something does go wrong.
You probably wouldn't want us to deploy something and
then have your site go down and then you're
like, not at your computer at all. You're at
dinner somewhere and it's like super inconvenient to get
paged there.
So, we publish all of this information, all of
the updates to the platform, but also all of
the Ruby updates, including security updates to the devcenter
changelogs. So, if you don't, this is, I think,
devcenter dot heroku dot com slash changelog. And if
you don't subscribe to it, I would recommend subscribing
to it just to keep up to date with
what is happening on Heroku for platform changes in
addition to updates to Ruby specifically on Heroku. And,
there isn't too much traffic. Like, you won't get,
like, a hundred emails a day. So, I highly
recommend subscribing to this to just keep up to
date with things like that here.
So the next thing I would like to talk
about is Matz's Ruby team. So if you didn't
know, back in 2012 we hired three people from
Ruby core. We hired Matz himself, Koichi and Nobu.
And, as I've gone around over the last years
talking and interacting with people, I realize a lot
of people have no idea who, besides Matz, who
Koichi and Nobu are.
So I wanted to take the time to kind
of update people on who these people were and
kind of what they've actually, like, we've been paying
them money, and what they've actually been doing to
kind of move Ruby forward in a positive direction.
So, if you run a git log, since 2012,
since we've hired them, you can see the number
of commits they've made to Ruby itself. So, the,
so Nobu here, who we've hired, has basically more
commits than like the second guy by many, many
commits. And then, Koichi's the third highest committer as
well.
And you're probably wondering why I have six names
here on a list for the top five. And
so there is this, so there isn't actually on
the Ruby Core Team who has the handle svn.
It's not actually a person. So I find out,
the hard way, who this person was. So when
I made my first patch to Ruby after being
on core, I found out that if, all the
date information is done in JST, and I, of
course, did not know that, and but like scumbag
American dates. And so there's basically this bot that
will go through and, like, fix your commits for
you, and like, so he does like another commit,
and it's like, ah, you actually put the wrong
date. Let me fix that for you.
So there's like 710 of those commits. I think
I did this like a month ago. So these
are the number of commits from a month ago.
So, the first person I like to talk about
is Nobuyoshi Nokada. Also known as Nobu. And he's
known, I think on Ruby Core as The Patch
Monster. So, we'll go into why he's known by
this.
So, what do you think the result of Time
dot now equals empty string? I'm sure you thought
it was an infinite loop, right. Or using the
rational, like, so if you're using the rational number
library in standard lib, like, what do you think
the result of doing this operation?
AUDIENCE: Segfault.
T.L.: Yeah. So this is a segfault.
AUDIENCE: [indecipherable - 00:32:40]
T.L.: Thank you. Thank you for reporting the bug.
So these, so Eric Hodel actually reported the other
bug, the time thing, and he found this in
RubyGems I believe. But these are real issues that
are in Ruby itself. So if you actually run
those two things now and you're using later patch
levels, you should not see them. But, they're real
issues, and someone has to go and fix all
them.
And so the person who actually does this is
Nobu. And he actually gets paid full time and,
to basically do bug fixes for Ruby. So all
those two thousand seven hundred and some commits are
bug fixes to Ruby trunk to make Ruby run
better. And, I thanked him when I was just
in Japan last week for all the work he's
done. It's pretty incredible. Like, there's so many times
when things segfault and other things, and he's basically
made it better.
I was at Oweito, and there was actually someone
giving a presentation about, like, thirty tips of, like,
how to use Ruby. And someone was talking about
open uri, and there was code on the screen,
and he found a bug during, like, the guy's
presentation, and during it, he committed a patch to
trunk during that guy's presentation. So, he's pretty awesome.
He doesn't, he doesn't do, he hasn't done any
talks, but I think people should know about the
work he's been doing.
So, this last bug, actually, that I wanted to
talk about, that he fixed was, are any of
you familiar with the regression from, in Ruby 2.1.1,
which regards to hash?
So, I'm sure you're familiar with the fact that,
if you use Ruby 2.1.1 on Rails 4.0.3, it
just doesn't work. Like, there, in Rails, we, we
use this, we fetch, in hashes we use objects
as keys. And if you override the hash and
equal method, and when you fetch you won't get
the right result back. So inside of Rails in
4.0.4, they actually had to work around this bug.
And, Nobu actually was the one who fixed this
inside of Ruby itself.
So, these were just, like, the three most interesting
bugs that I found from within the last year
or two of stuff he's worked on. But, if
you look on site Ruby Core, you can find,
like, hundreds and hundreds of bugs that he's done
within the last year of, just like, rip segfaults
and other things. So he's great to work with.
So the next person I want to talk about
is Koichi Sasada. He's also known as ko1, ko1.
And he doesn't have a nickname in Ruby Core,
so me and Richard spent a good amount of
our talk preparation trying to come up with a
nickname for him. So we came up with the
Performance Pro. And this is a picture of him
giving a talk in Japanese.
So, if you use Ruby 1.9 at all, he
worked on Yarv. So basically the new VM stuff
that made Ruby 1.9, I think it was like
thirty percent faster than 1.8 for like longer-running processes.
More recently he's worked on the RGenGC. And this
was introduced in Ruby 2.1.1, and it allows faster
code execution by having, basically, shorter GC pauses. So
instead of doing full GC every time, like, you
can have these minor ones.
So, just, he spends all of his time thinking
about performance in Ruby, and that's like what he's
paid to work on. So if anyone actually cares
about Ruby performance, you should thank this guy for
the work he's done. If you've looked at the
performance of Ruby since, in the last few years,
like it's improved a lot. A lot due to
this guy's work.
And, I was just, I was talking to him,
and he was telling me that he basically, like,
when he was working on RGenGC, he like, he
was just like, walking around the park and he
had a breakthrough. So he like spends a lot
of his time, even, off of work hours, just
thinking about this stuff.
Other stuff that he's been working on as well
is profiling work. So, if you've used any of
the Man stuff for 2.1.1, with the MemProfiler and
other things, he's been working on, with him to
introduce hooks into the internal API to make stuff
like that work. So we, I think we understand
that profiling, being able to measure your application for
Ruby is super important. So, if you have basically
comments or suggestions on things that you need or
think that you can't improve this thing, like it's
worth talking, reaching out and talking to Koichi about
this.
And some of the stuff he's been working on
in this vein has been, like, the gc_tracer gem.
So, using this to basically get more information about
your garbage collector, an allocation_tracer gem to see how
long live, like, objects are. And then even in
2.2, we're, as a team, we're working on, there
is an incremental GC patch, and then also. Or,
he's working on making the GC better with incremental
GC and there is symbol GC for security things,
which'll be super good for Rails. So we can't
get, like, DOS because of the symbol table being
filled up.
Another, so one of the things, when I was
in Japan, we had a Ruby Core meeting, and
we talked about Ruby releases. And releasing Ruby is
kind of a slow process, and I was, I
wasn't really sure why it took so long. And
so I kind of asked the question, and, and
Naruse, who's the release manager of 2.1 and was
telling me that it requires lots of human and
machine resources. Basically, Ruby has to work on many
configurations, Linux distros, you know, on OS X and
other things. And in order to release, like, the
CI server has to pass and, like, you kind
of have to pass on like various vendors and
what not. And so like, there's a lot of
coordination and like checking to like make an actual
release happen. Which is why things don't release super
fast.
So, some of the stuff that Koichi and my
team and other people on Ruby Core have been
working on is, like, working on infrastructure and services
to help with, basically, testing of Ruby, to kind
of hopefully automate and, like, basically do that per,
either nightly or per commit or something along those
lines.
So hopefully we can get releases that are faster
and are out to users sooner.
If you have ideas for Ruby 2.2, like, I
would love to hear them. We have a meeting
next month in May, about what is gonna go
into Ruby 2.2. So I'd be more than happy
to talk to you about ideas that you have
that you would like to see there. I'm just
gonna skip this stuff since I talked about it
earlier, and we're running short on time. So, here's
Scheems to actually talk about Rails.
R.S.: OK.
Has anybody used Rails? Have we covered that question
yet? OK. Welcome to RailsConf. OK, so Rails 4.1
on Heroku.
A lot of things in a very short amount
of time. We are secure by default. Have you
heard of the secrets dot yml file? OK. So
secrets dot yml file is actually reading out an
environment variable by default, which is great. We love
environment variables. It separates your config from your source.
And, so whenever you push your app, we're gonna
set this environment variable to just, like, literally a
random value. And if, for some reason, you ever
need to like change that, you can do so
by just setting your, the, the secret key base
environment variable to, to whatever you want.
Maybe, you know, like another OpenSSL bug comes out
or something. So, another thing that was worked on
a bunch is the database_url environment variable. This is
something that we have spent a lot of time
looking at. And it's actually, support has been in
Rails for a surprisingly large amount of time, to
just read from the environment variable, but never quite
worked due to some edge cases and random rake
tasks and so on and so forth. So this,
this December, around Christmas time, I spent a lot
of time getting that to work.
So I'd like to happily announce that Rails 4,
4.1 actually does support the database_url environment variable out
of the box. Whoo! And, so, some, to describe
a little, like, the behavior is, bears going over.
If the database_url is present, we're just gonna connect
to that database. It's, that's pretty simple. Makes sense.
If the database.yml is present but there's no environment
variable, then we're gonna use that. That also just
kind of makes sense.
If both are present, then we're gonna merge the
values. Makes sense, right? OK.
So, we, that sounds crazy. Bear with me. But,
a lot of people, you, you want to put
your connection information in your database_url environment variable. But,
there's also other values you can use inside of
your database.yml file to configure ActiveRecord itself. Not your
database. So you can turn off and on prepared
statements. You can change your pool size. All this
kind of thing.
And, we wanted to still enable you to be
able, able to do this. So the, the results
are actually merged, and for, for somebody like Heroku
or, like, if you're using another container, we don't
have to have as much magic. If you, if
you didn't know, database_url, we actually had to over,
whatever your database_url was, we were just writing a
file over top of it. And it's like, forget
that. We're gonna write a custom file.
So people would put stuff in their database_url, or
their database.yml file, and they'd be surprised when it
wasn't there. Like, a different file was there. So,
we no longer, we no longer have to do
that. And Rails plays a little bit nicer with,
with this containerized style environment.
It also means that, you could actually start putting
your ActiveRecord configuration in that file. Another note, if
you were manually setting that, your pool size or
any of those things via a, after reading an
article on our devcenter, go back and revisit that
please, before upgrading to Rails 4.1. Some of the
syntax did change between Rails 4.0 and 4.1. So,
if you can't connect to a database, then maybe,
just like, email Schneemz and be like, I hate
you. What's the link to that thing? And I'll,
I'll help you out.
OK. I think, probably, actually, the last thing that
we have time for, is asset pipeline. Who, like,
if asked in an interview, would say that their
favorite thing in the whole world is Rails asset
pipeline? Oh. Oh.
AUDIENCE: Just Raphael.
R.S.: Just Raphael. We have a bunch of, like,
Rails Core here, by the way. So you should,
you should come and thank them afterwards. For, for
other things. Not for the asset pipeline.
[laughter]
So, the asset pipeline is the number one source
of, of Ruby support tickets at Heroku. Just people
being like, hey, this worked locally, and like, didn't
work in production. And we're like, yeah, that's just
how asset pipeline works. That's not Heroku.
So, so Rails 4.1 added, added a couple things.
It's gonna warn you in development if you're doing
something that's gonna break production. Like, if you've ever
forgotten to add something to your precompile list, well
now, guess what, you get an error. If you
are not properly declaring your asset dependencies, then you're
gonna get an error.
And this is even better, actually, in Rails 4.2.
As some of these checks aren't even needed anymore,
we can just automatically do them for you. But,
unfortunately, those have, are not in Rails 4.1 yet.
So, in general, I have a, a personal belief
that, in programming, or, really in life, the only
thing that should fail silently is. This. This joke.
So. Thank you all very much for, for coming.
We, we have a booth, and later on, what.
What time, three o' clock?
T.L.: Between 3:00 and 4:30.
R.S.: Yeah. From 3:00 to 4:30, we'll actually have
a bunch of Rails contributors coming to, to talk
about. Oh yeah, the slides. Yeah. Yeah.
T.L.: Yeah. 3:00 to 4:30, we'll have community office
hours with some nice people from Rails Core, contrib.
R.S.: Yeah. So come ask.
T.L.: Basically any Rails questions or anything you want.
And then Schneeman will actually be doing a book
signing of his Heroku Up & Running book today
and tomorrow at 2:30. So if you want that.
R.S.: Yeah. So get a, get a free book,
and then come and ask questions and just, like,
hang out. And, any time you stop by the
booth, feel free to ask Heroku questions. And thank
you all very much for coming.