RailsConf 2014 - Heroku 2014: A Year in Review by Terence Lee & Richard Schneeman

RICHARD SCHNEEMAN: All right. OK. Hello everyone.

AUDIENCE: Hello.

R.S.: Thank you. Thank you. Welcome to, welcome,
let

me be the first to welcome you to RailsConf.

So, our, our talk today is Heroku 2014: A

Year in Review. It is gonna be a play

in six acts, featuring Terrance Lee and Richard
Schneeman.

So, of course this is a year in review,

and Heroku does measure their years by RailsConf.
So

this is from Portland to Chicago RailsConf
year. The

Standard RailsConf Year.

As, as some of you might know, we are

on the Ruby Task Force, and, in fact, that

makes us Ruby Task Force members. And, of
course,

this was a big year. We're gonna be talking

a little bit about app performance, some Heroku
features,

and community features. So, first up to the
stage,

I'm gonna be introducing the one, the only
Mister

Terrance Lee. You might have recognized him
in some

other roles. He hails from Austin, Texas,
which has,

undoubtedly, the best tacos in the entire
world. So,

he-

AUDIENCE: [indecipherable]

R.S.: Them's fightin' words, friend. So that
he, he's

also sometimes known as the Chief Taco Officer.
Or,

or the CTO. And something, something very
interesting about

Terrance is, recently, he was inducted into
Ruby Core,

so congratulations to, to Terrance. All right.

So, without further ado, Act 1: Deploy Speed.

TERRANCE LEE: Thank you, Richard. So, at the
beginning

of the year Rails Standard Year, we focused
a

lot on deployment speed. We got a lot of

feedback and realized deployment was not as
fast as

it could be. And we wanted to make it

faster. So, the first thing we set out to

do was to actually do a bunch of measurement

and profiling to look at where things were
slow,

and how we could make it better, and to

kind of gage, like, the before and after and

know when the good points were to kind of

stop and move on to other things. Cause you

can never make, you can never, you will never

be done with, like, performance improvements.

So, after about six months of work on this,

we managed to cut down the deploy speeds for,

across the platform for Ruby by about forty
percent.

So it's a pretty decent speed improvement.
And, in

order to do this, we mainly looked at three

various ways to speed this up.

The first thing was running code in parallel,
so

running more things, running things, like,
more than one

thing at one time. If you cache stuff you

don't have to do it again, and, in general,

just like, cutting out code that doesn't need
to

be there.

So, with the parallel code, we worked with
the

bundler team on Bundler 1.5. There was a pull

request sent by CookPad that was sent in to

add parallel bundler install for Bundler 1.5.
So if

you actually aren't using this yet, I would
recommend

upgrading your bundle to at least Bundler
1.5. And

the bundler added this dash j option, which
allows

you to specify the number of jobs to run.

And this is, basically, if you're using MRI,
it

forks and does a num, these number of sub-processes,

and if you're on JRuby or Rubinius it actually

just uses threads here.

And the benefit of doing this is, when you

actually do bundle install, the dependencies
that get installed

get downloaded in parallel, so you're not
waiting on

network traffic sequentially anymore, and
in addition you're also

install gems in parallel. And this is mostly
beneficial,

especially when you're running native extensions.
So if you

have something like Nokogiri, that takes a
long time.

Oftentimes, you notice, you just like hang
and wait

for it to install and then it installs the

next thing, so this allows you to install
that,

basically, in the background and then go and
install

other gems at the same time.

Also, in Bundler 1.5, Richard actually added
this function

that allows people, allows bundler to auto-retry
failed commands,

so initially, before this, we would, when
we run

bundle install and something would fail because
of some

odd network timeout, like during one chance,
you would

have to basically repush again, no matter
where you

were in the build process. So, by default
now,

Bundler actually will retry clones and gem
installs for

up to three times by default.

So, it will continue going during the deploy
process.

And so is anyone here actually familiar with
the

PIGz command? So, just Richard? So, PIGz is
Parallel

Gzip, and the build and packaging team at
Heroku

worked on this feature, or worked on implementing
this

at Heroku using the PIGz command, and in order

to understand the kind of benefit of using
something

like this, when you push an app up on

Heroku during the compile process, it actually
builds these

things at Heroku that are called slugs. And
basically

it's just, like, a tar of your app directory

of everything after the compile face is run.

And, originally, we were just using SquashFS,
initially, and

then we moved to kind of just tar files,

and we noticed that one of the slowest point

in the actual build process was actually just
going

through and compressing everything in that,
that file directory,

and then pushing it up onto S3 after that

was done. And so one of the things that

we looked into was, is there a way we

can make that faster? So, if you ever push

a Heroku app and then you basically, like,
wait

when it says, like, compressing and then it
goes

to done, like that's the compressing of the
actual

slug.

And we managed to use slug, PIGz to now

improve that by sniffing them out. I don't
remember

the actual performance improvement, but it
was pretty significant,

and the only downside was in certain slugs,
the

slug sizes are a little bit bigger. But the

performance trade off was worth it at that
time.

The next thing we started doing was looking
into

caching. So anyone here using Rails 4? So
pretty

good amount of the room. So the one thing

is that we did which differ from Rails 3,

thanks to a bunch of the work that's happened

on the Rails Core team with us is that,

we can now cache assets between deploys. This
wasn't

possible in Rails 3 because the cache was,
you

couldn't actually reuse the cache. There was
times when

the cache would basically be corrupted and
then you

would get, like, assets that wouldn't work
between deploys.

So the fix there was you actually have to

remove the assets between each deploy in some
Rails

3 builds. But it wasn't consistent, so sometimes
it

would work and sometimes it didn't. And on
Heroku

that's not something we can rely on in an

automated fashion.

But luckily a lot of that stuff has been

fixed for Rails 4, so now we cache assets

between deploys in Rails 4. And so if we

look at Rails 3, I guess this got cut

off, but this is supposed to say about, like,

thirty-two seconds for a Rails 3 deploy, and
then

on Rails 4 it got, for the average we,

we measured the steps in the, in the build

process, and on Rails 4, the perk fifty was

about fourteen point something seconds. So
a pretty significant

speed improvement there, both due to caching
and other

improvements inside of Rails 4 for the asset
pipeline.

So the other thing we also looked at was

just, if there's code that is doing extra
work,

if we remove that, it will speed up the

build process for everyone who's deploying
every day. So

one of the first things that we did was

actually stop downloading bundler more than
once. So, initially,

when you, when we do the Ruby version detection,

we actually have to download bundler, and
then basically

run that to get the version of Ruby to

install on the application. And then again,
we would

then download and install it again because
it was

run in a separate process for the actual,
like,

installing of your dependencies. And one of
the things

we did was to actually just stop doing that

and we would cache the Bundler gem so we

don't have to download that two or three times

during the build process. So, so cutting network
IO

and other things.

We also started removing, there was like duplicate
checks

between detection of what kind of app you
were

using so, and, bin detects. We would use it

to figure out what kind of app you have,

like if it was a Ruby app, a Rack

app, a Rails 3 app, a Rails 4 app,

stuff like that. And then, again, since it
was

a separate process in bin compile, we would
have

to do it again. So, Richard actually did a

bunch of work to refactor both detect and
release,

and so now detect is super simple. It literally

just checks if you have the gemfile file there,

and then all the other work is now deferred

to bin compile. So that means we're only doing

a bunch of these checks once, like examine
your

jump file, checking what gems you have. So
not

doing that two or more times.

And, if you haven't watched this talk, he
gave

this talk at Ancient City Ruby. I don't actually

know if the videos are quite up yet. But

Richard does a talk about testing the untestable,
so

if you're interested in learning how we test
the

build pack, you should go watch this talk.

So I'd like to introduce Richard, cause he's
gonna

present on the next section. So Richard loves
Ruby

so much that he got married to her. I

think he got married last, last year.

R.S.: Right before our last RailsConf.

T.L.: Yeah. Right before our last RailsConf.
I remember

that. He's also on the Rails Issue Team, and

he's one of the top one hundred Rails contributors,

according to the Rails contributor sites.
And you might

also know him for his, this gem called sextant

that he released for Rails 3. Basically, I
remember

back in the day, developing Rails apps, when
I

wanted to basically verify routes, I would
run the

rake routes command, and it would, you know,
boot

up the Rails environment and you'd have to
wait

a few seconds and then they would print out

all the routes. And then if you wanted to,

like, rerun it using grep, you would keep
running

it again.

So, a lot of us, when we're doing development,

already have, like, Rails running in a server
while

we're testing things and whatnot. And so what
sextant

does is it allows it, supports basically looking
at

the routes that are already in memory and
just

allowing you to query against the programmatically,
and then

that has a view for doing this. And this

was also just merged into Rails 4. So if

you're using Rails 4 or higher, you actually
don't

need to sextant gem and it's now built in.

Richard and I both live in Austin, and so

when people come visit, or actually when I'm
in

town, which isn't often, we have Ruby meet
ups

at Franklin's Barbecue, so if you guys are
ever

in town, let us know and we'd be more

than happy to take you to a meet up.

R.S.: All right. So the, for the first part

of this, this act, we're gonna be talking
about

app speed, but before we talk about app speed,

we're actually gonna talk about dimensions.
So, the, the

document dimensions are, let me see. Here
we go.

Were originally written in wide-screen, but
the screens here

are standard.

There we go. So. You're actually gonna get
to

see all of the slides, as opposed to just

having some of them cut off. So, OK.

On, on app speed. The first thing I want

to talk about is, is tail latencies. Is anybody

familiar with tail latencies? OK. The guys
in the

Heroku t-shirts and somebody else.

OK. So this is, this is a normalized distribution.

WE have, on one side, the number of requests.

On the, on the other side, we have the

time to respond. So the further out you go,

the slower it's gonna be. And we can, we

can see this is the distribution of our requests.

So over here, it's super fast. Like you love

to be that customer. You're super happy.

Over here, we have a super slow request, and

you don't want to be that customer and you're

pretty unhappy. So right in the middle is
our

average, and I'm sure they talked a ton about

why the average is really misleading in the,
in

the last session with Skylight.IO. But, but
we're basically

saying that, roughly fifty percent of your,
of your

customers, fifty percent of your traffic,
is going to

get a response time at this or, or lower.

So, like, this is, this is pretty decent.
We

can say, like, fifty percent of the people
who

come to our web site get a response before

then. Moving up the distribution, to something
like perk

ninety-five, we say ninety-five percent of
everyone who visits

our traffic will get a response by now. So

I'm gonna be using those terms, perk fifty,
perk

ninety-five, that refers to the percentage
of, of requests

that come in that we can respond by.

So this is kind of theorized. This is an

actual application. I, and one thing that
you'll notice

is that it's not perfectly normalized. Like,
it's not,

like both sides are not symmetrical. We kind
of

like, steeply shoot up, and then we have this

really, really long tail, and, and this is
kind

of the, what I'm referring to when I'm saying

tail latencies.

So, yes, somebody actually might have gotten
a response

in zero milliseconds. You know, I doubt it,
but

somebody for sure did get a response in 3000

milliseconds, and that's a really long time
to wait

for your request to actually come in and,
and

get finished. So even though somebody is getting
really

fast responses, and your average isn't bad
- your

average is under 250 milliseconds - one customer
might

be getting a really slow response and a really

fast response, and, and the net is a bad

experience.

So, the net, it, it just, it's a very

inconsistent experience. So whenever we're
talking about application speed,

we have to consider individual request speed
and average,

but also consistency. How consistent is each
request?

So, how do, how do we do this? What?

How can we, how can we help with this?

Well, one of the things that we launched this

year was PX dynos. So a PX dyno -

a typical dyno only has 512 megabytes of RAM.

It's a shared infrastructure. A PX Dyna has
six

gigabytes of RAM and eight CPU cores, which
is

a little, a little nicer, a little better,
a

little bit more room to play.

And, and it's also real hardware. So, or,
it's,

it's not on the same shared infrastructure.
So you

can, you can scale with Dynos, you, you can

also scale inside of Dynos. And that's kind
of

two, two important parts that we're gonna,
gonna have

to cover. So, of course, whenever you have
more

requests that you can possibly process, you
want to

scale up and say, I'm gonna have more Dynos.

But what happens if, if you're not making
the

best use of everything inside of your dyno?
Previously

with 512 megabytes of RAM, you could just,
you

know, throw a couple Unicorn workers in there
and

you're like, oh, I'm probably using most of
this.

Like, if you put two unicorn workers in a

PX Dyno, you're not making the most use of

it all.

So, recently, I am super in love with, with

Puma. Evan, this is Evan Phoenix's web server
that

was originally written at, to kind of show
case

Rubinius. Guess what? It's really nice with
MRI as

well. Recently we've gotten some Puma docs,
and so

I'm gonna talk about Puma for just, for just

a little bit.

So, if you're, if you're not familiar. I was

totally off on the formatting. So, Puma handles
requests

by running multiple processes, or by multiple
threads. And

it can actually run in something called a
hybrid

mode, where each process has multiple threads.
We, we

recommend this, or I recommend this. E- if
one

of your processes crash, it doesn't crash
your entire

web server. It's kind of nice.

And so the multiple processes is something
that we're

pretty familiar with. As Rubyists, we're familiar
with forking

processes. We're familiar with Unicorn. But
the, the, the

multiple threads is a little bit different.

Even with MRI, even with a, something like
a

global interpreter lock, you are still doing
enough IO,

you're still hitting your database frequently
enough, maybe making

API calls to like, Facebook or GitHub status,
being

like, hey, are you still up?

And, and this will give our threads time to

kind of jump around and allow others to do

work. So you, you can get quite an extra

bit of performance with, there.

So, we're actually gonna be using Puma to
scale

up inside of our Dynos. So once we give

you that eight gigs of RAM, we want to

make sure that, that you can, you can, you

can make the most use out of it.

In general, with Puma, more processes means
more RAM,

and more threads are gonna need more CPU consumption.

So, you want to, you want to maximize your

processes and maximize your threads, kind
of without going

over. As soon as you start swapping, as soon

as you go over that RAM limit, your app's

gonna be really slow and that kind of defeats

the purpose of trying to add these resources.

Another issue is that I had kind of never

heard of until I started looking into all
of

these multiple web servers, is slow-client.
So if somebody's

connecting to your web site via like, a two

G over like a Nokia candy bar phone, uploading

like photos or something like that, like that
is

a slow client, and if you're using something
like

Unicorn, it can deDOS your, your site, because
each

one of those requests takes up an entire Unicorn

worker, whereas Puma has a, has a buffer,
and

it buffers those requests as similar to the
way

NginX does.

One other thing to consider with Puma is,
so

I'm mentioning threads, I'm talking, talking
about threads. Ruby,

we're not necessarily known as the most thread-safe
culture.

Thread-safe community. And so a lot of apps
just

aren't thread-safe. And so some, you might
take a

look at Puma and be like, hey, that's not

for me. You can always set your maximum threads

to one, and then now you're behaving just
like

Unicorn, except you have the slow-client protection,
and whenever

you get that gem that's bad or you, like,

stop mutating your constants are runtime or
something, then

you can maybe bump up and try multiple threads.

OK. So, I'm, I'm talking about consistency
and I'm

talking a lot about Puma. How does that all

kind of boil down and help? So, does anybody

think that sharing distributed state across
multiple machines is

like really fast? Maybe. I, OK. Good.

What about sharing state inside of memory
on the

same machine? Is that faster? OK. All right.
I

think we're in, in agreement. So, a, a little

bit of a point of controversy. You might have

heard of the, the Heroku router at some point

in time. And this, the router is actually
designed,

not randomly, but it is, it is designed to

use a random algorithm. And it basically will
try

to deliver requests as fast as humanly possible,
or

computerly possible, to individual dynos.
So it's like, it

gets the request. It wants to get it to

your dyno as fast as it possibly can.

And adding any sort of additional overhead
of distributed

locks or queues is gonna be slowing that down.

Once inside of your, in your, your process,
Puma

or Unicorn has, in memory, state of all of

its own processes, and is capable of saying,
oh,

hey, this process is busy. This process is
not

busy. I can do really intelligent routing
and, and

basically, for free.

It's really fast. It, it took a little bit

of convincing for me. So does anybody else
need

to be convinced?

AUDIENCE: Yeah.

R.S.: OK. Good. Cause otherwise I could totally
just

skip over the next section of slides.

So, this is, this is a graph produced by

the fine developers over at, at RapGenius,
and on

one side we will actually see a percentage
of

requests queued, and on the bottom we are
gonna

be seeing number of dynos. So the goal is

actually to minimize request queuing, like
this, this is

time that your customers are waiting that
you're not

actually doing anything.

You, so you, you want to minimize that queuing

with the smallest number of resources, so
the smallest

number of dynos. This top line, we actually
have

is what we've currently got now. The random
routing

with a, a single-threaded server. And, like,
this is

pretty bad. It, like, starts out bad and it,

like, it doesn't even like trend towards zero.
So

this is probably bad. So this is using something

like Webrick in production.

So. Don't use Webrick in production. Or, or
like

even, even thin. In single-threaded mode.
So, on the,

on the very bottom, we actually have a, like,

mythological, like, if, if we could do all
of

that distributed shared state without, and
locks and queues,

without having any kind of overhead, we can
see

that, basically, it just drops down to zero
at

about, you know, in their case, about seventy-five
dynos,

and then just, you know, it's straight zero.
There's

no queuing.

And it's great. And this would be amazing
if

we could have it. But unfortunately there
is a

little bit over overhead. What was really
interesting to

me is this second one, which is not nearly

as nice as that mythological intelligent router,
but it's

kind of not too far off. This is still

our random routing, and, and this was actually
done

with Unicorn, and workers set to two. So basically,

once we get the, the request to your operating

system, it's like one of those two workers
is

free and can immediately start working on
it.

Some, some interesting things known about
this is, for

the non-optimal case, for the, we basically
don't have

enough dynos to handle this, so that might
happen

is, you know, you got on Hacker News or

whatever, slash dotted, Reddited. SnapChatted.
Secreted. I don't know.

And it does actually, eventually approach
ideal state. So,

it, it gets even better, and unfortunately
they kind

of stopped at, at two processes, but it gets

better, the more concurrency that you add.
So if

you had three or four workers or, again, if

you're using something like Puma, and each
one of

those workers is running, like, four threads,
now you

have, like, a massive amount of concurrency
that you

could deal with all of these requests coming
in.

So, the, the, the, if, again, and we're looking

for consistency. We want that request to get
to

our dyno, immediately be able to process it.
So,

you can use Puma or Unicorn to maximize that

worker number, and, again, distributed optimi-
distributed routing is

slow. In memory, routing is relatively quick.

On, again, just in, in the whole context of

speed, Ruby 2.0 came out and this was awhile

ago. It's got GC, it's optimized for Copy
on

Write. In, in Ruby, extra processes, process
forks actually

become cheaper. So the first process might
take seventy

megabytes, the second one twenty, and ten
and seven

and six. So, if you get a larger box,

you can actually run more processes on them.
If

you get eight gigs on one box, you can

run more processes than you can if you had

eight gigs across eight boxes.

So, again, more processes mean more concurrency,
and more

concurrency means consistency. If you are
using workers, you

can, you can also scale out with Resq pool,

and if your application's still slow, we rolled
out

a couple of really neat platform features.
One of

them is, is called HTTP Request ID. So as

a request comes into our system, we will actually

give it a uuid, and you can see this

in your router log. And then we've got documentation

on how to configure your Rails app so it

will actually pick this up and use that uuid

in tagged logs.

So, like, how is this useful? So, if you

are getting, like, and out of memory error,
or

if your request is taking a really long time

and you're like, ah, like, that request is
timing

out and, you know, Heroku's returning a response
and

we don't even know why. Now, if the request

id is tagged, you can actually follow along
between

your two logs and be like, oh, it's hitting

that controller action. Maybe I should be
sending that

email in the background as opposed to having
to

actually block on it. So you can trace specific

errors.

We also launched Log Runtime Metrics awhile
ago, and

this is something that we'll actually put
your, your

runtime information directly into your logs.
You can check

it out. Liberato will automatically pick it
up for

you and make you these, these really nice
graphs.

And, again, if you're doing something like
Unicorn or,

or Puma, then you want to get as close

to your RAM limit without actually going over.

OK. So, the, the next act in, in our

play, again, introducing Terence, is, we'll
be talking about

Ruby on the Heroku stack and in the community.

T.L.: Thank you. So I know we're at RailsConf,

but I've been doing a bunch of work with

Ruby, so I wanted to talk about some Ruby

stuff. So who here is actually using Ruby
1.8.7?

Wow. No one. That's pretty awesome. Oh, wait.
One

person. You should probably get off of it.

[laughter]

But. Who is using Ruby 1.9.2? A few more

people. 1.9.3? Good amount of people here.

So, I don't know if you guys were following

along, but Ruby 1.8.7 and 1.9.2 got end-of-lifed
at

one point. And then there was a security incidence,

and Zachary Scott and I have volunteered to
maintain

security patches till the end of June. So,
if

you are on 1.8.7 and 1.9.2, I would recommend

hopefully getting off some time soon, unless
you don't

care about security or want to back port your

own patches.

And then Ru- we recently announced that Ruby
1.9.3

is also getting an end of life in February

2015, which is coming up relatively quickly.
It's a

little less than a year away now, at this

point. So, please upgrade to at least 2.0.0
or

later.

And, during this past Rails Standard Year,
we also

moved the default Ruby on Heroku from 1.9.2
to

2 dot 0 dot 0. We believe people should

be at least using this version of Ruby or

higher.

And, if you don't know yet, you can declare

your Ruby version in the gemfile on Heroku
to

get that version. And, we've also are pretty,
pretty

serious about supporting the latest versions
of Ruby. Basically,

the same day that they come out. So we

did this for 2.0.0, 2.1.0 and 2.1.1, and in

addition we also try to support any of the

preview releases as, whenever they get form
release, so

we can, as a community, help, help find bugs

and test things like, put your staging app
on

new versions of Ruby. If you find bugs then,

hopefully, we can fix them before they actually
make

it to the final release.

And, with regards to security patches, if
there are

any security releases that come out, we make
sure

to release them that day as well. We take

security pretty seriously.

So, once a security patch has release and
we've

patched those Rubies, you have to push your
app

again to get that release. And the reason
we,

well, a lot of people ask us, like, why

we don't do that. Why we don't just automatically

upgrade peoples' Rubies in place. And the
reason it,

the reasoning here is that, not, there might
be

a regression to the security patch, or maybe
the

patch level is not 100% backwards compatible.
There's a

bug that slipped through. But you probably
want to

be there when you're actually deploying your
application, in

case something does go wrong.

You probably wouldn't want us to deploy something
and

then have your site go down and then you're

like, not at your computer at all. You're
at

dinner somewhere and it's like super inconvenient
to get

paged there.

So, we publish all of this information, all
of

the updates to the platform, but also all
of

the Ruby updates, including security updates
to the devcenter

changelogs. So, if you don't, this is, I think,

devcenter dot heroku dot com slash changelog.
And if

you don't subscribe to it, I would recommend
subscribing

to it just to keep up to date with

what is happening on Heroku for platform changes
in

addition to updates to Ruby specifically on
Heroku. And,

there isn't too much traffic. Like, you won't
get,

like, a hundred emails a day. So, I highly

recommend subscribing to this to just keep
up to

date with things like that here.

So the next thing I would like to talk

about is Matz's Ruby team. So if you didn't

know, back in 2012 we hired three people from

Ruby core. We hired Matz himself, Koichi and
Nobu.

And, as I've gone around over the last years

talking and interacting with people, I realize
a lot

of people have no idea who, besides Matz,
who

Koichi and Nobu are.

So I wanted to take the time to kind

of update people on who these people were
and

kind of what they've actually, like, we've
been paying

them money, and what they've actually been
doing to

kind of move Ruby forward in a positive direction.

So, if you run a git log, since 2012,

since we've hired them, you can see the number

of commits they've made to Ruby itself. So,
the,

so Nobu here, who we've hired, has basically
more

commits than like the second guy by many,
many

commits. And then, Koichi's the third highest
committer as

well.

And you're probably wondering why I have six
names

here on a list for the top five. And

so there is this, so there isn't actually
on

the Ruby Core Team who has the handle svn.

It's not actually a person. So I find out,

the hard way, who this person was. So when

I made my first patch to Ruby after being

on core, I found out that if, all the

date information is done in JST, and I, of

course, did not know that, and but like scumbag

American dates. And so there's basically this
bot that

will go through and, like, fix your commits
for

you, and like, so he does like another commit,

and it's like, ah, you actually put the wrong

date. Let me fix that for you.

So there's like 710 of those commits. I think

I did this like a month ago. So these

are the number of commits from a month ago.

So, the first person I like to talk about

is Nobuyoshi Nokada. Also known as Nobu. And
he's

known, I think on Ruby Core as The Patch

Monster. So, we'll go into why he's known
by

this.

So, what do you think the result of Time

dot now equals empty string? I'm sure you
thought

it was an infinite loop, right. Or using the

rational, like, so if you're using the rational
number

library in standard lib, like, what do you
think

the result of doing this operation?

AUDIENCE: Segfault.

T.L.: Yeah. So this is a segfault.

AUDIENCE: [indecipherable - 00:32:40]

T.L.: Thank you. Thank you for reporting the
bug.

So these, so Eric Hodel actually reported
the other

bug, the time thing, and he found this in

RubyGems I believe. But these are real issues
that

are in Ruby itself. So if you actually run

those two things now and you're using later
patch

levels, you should not see them. But, they're
real

issues, and someone has to go and fix all

them.

And so the person who actually does this is

Nobu. And he actually gets paid full time
and,

to basically do bug fixes for Ruby. So all

those two thousand seven hundred and some
commits are

bug fixes to Ruby trunk to make Ruby run

better. And, I thanked him when I was just

in Japan last week for all the work he's

done. It's pretty incredible. Like, there's
so many times

when things segfault and other things, and
he's basically

made it better.

I was at Oweito, and there was actually someone

giving a presentation about, like, thirty
tips of, like,

how to use Ruby. And someone was talking about

open uri, and there was code on the screen,

and he found a bug during, like, the guy's

presentation, and during it, he committed
a patch to

trunk during that guy's presentation. So,
he's pretty awesome.

He doesn't, he doesn't do, he hasn't done
any

talks, but I think people should know about
the

work he's been doing.

So, this last bug, actually, that I wanted
to

talk about, that he fixed was, are any of

you familiar with the regression from, in
Ruby 2.1.1,

which regards to hash?

So, I'm sure you're familiar with the fact
that,

if you use Ruby 2.1.1 on Rails 4.0.3, it

just doesn't work. Like, there, in Rails,
we, we

use this, we fetch, in hashes we use objects

as keys. And if you override the hash and

equal method, and when you fetch you won't
get

the right result back. So inside of Rails
in

4.0.4, they actually had to work around this
bug.

And, Nobu actually was the one who fixed this

inside of Ruby itself.

So, these were just, like, the three most
interesting

bugs that I found from within the last year

or two of stuff he's worked on. But, if

you look on site Ruby Core, you can find,

like, hundreds and hundreds of bugs that he's
done

within the last year of, just like, rip segfaults

and other things. So he's great to work with.

So the next person I want to talk about

is Koichi Sasada. He's also known as ko1,
ko1.

And he doesn't have a nickname in Ruby Core,

so me and Richard spent a good amount of

our talk preparation trying to come up with
a

nickname for him. So we came up with the

Performance Pro. And this is a picture of
him

giving a talk in Japanese.

So, if you use Ruby 1.9 at all, he

worked on Yarv. So basically the new VM stuff

that made Ruby 1.9, I think it was like

thirty percent faster than 1.8 for like longer-running
processes.

More recently he's worked on the RGenGC. And
this

was introduced in Ruby 2.1.1, and it allows
faster

code execution by having, basically, shorter
GC pauses. So

instead of doing full GC every time, like,
you

can have these minor ones.

So, just, he spends all of his time thinking

about performance in Ruby, and that's like
what he's

paid to work on. So if anyone actually cares

about Ruby performance, you should thank this
guy for

the work he's done. If you've looked at the

performance of Ruby since, in the last few
years,

like it's improved a lot. A lot due to

this guy's work.

And, I was just, I was talking to him,

and he was telling me that he basically, like,

when he was working on RGenGC, he like, he

was just like, walking around the park and
he

had a breakthrough. So he like spends a lot

of his time, even, off of work hours, just

thinking about this stuff.

Other stuff that he's been working on as well

is profiling work. So, if you've used any
of

the Man stuff for 2.1.1, with the MemProfiler
and

other things, he's been working on, with him
to

introduce hooks into the internal API to make
stuff

like that work. So we, I think we understand

that profiling, being able to measure your
application for

Ruby is super important. So, if you have basically

comments or suggestions on things that you
need or

think that you can't improve this thing, like
it's

worth talking, reaching out and talking to
Koichi about

this.

And some of the stuff he's been working on

in this vein has been, like, the gc_tracer
gem.

So, using this to basically get more information
about

your garbage collector, an allocation_tracer
gem to see how

long live, like, objects are. And then even
in

2.2, we're, as a team, we're working on, there

is an incremental GC patch, and then also.
Or,

he's working on making the GC better with
incremental

GC and there is symbol GC for security things,

which'll be super good for Rails. So we can't

get, like, DOS because of the symbol table
being

filled up.

Another, so one of the things, when I was

in Japan, we had a Ruby Core meeting, and

we talked about Ruby releases. And releasing
Ruby is

kind of a slow process, and I was, I

wasn't really sure why it took so long. And

so I kind of asked the question, and, and

Naruse, who's the release manager of 2.1 and
was

telling me that it requires lots of human
and

machine resources. Basically, Ruby has to
work on many

configurations, Linux distros, you know, on
OS X and

other things. And in order to release, like,
the

CI server has to pass and, like, you kind

of have to pass on like various vendors and

what not. And so like, there's a lot of

coordination and like checking to like make
an actual

release happen. Which is why things don't
release super

fast.

So, some of the stuff that Koichi and my

team and other people on Ruby Core have been

working on is, like, working on infrastructure
and services

to help with, basically, testing of Ruby,
to kind

of hopefully automate and, like, basically
do that per,

either nightly or per commit or something
along those

lines.

So hopefully we can get releases that are
faster

and are out to users sooner.

If you have ideas for Ruby 2.2, like, I

would love to hear them. We have a meeting

next month in May, about what is gonna go

into Ruby 2.2. So I'd be more than happy

to talk to you about ideas that you have

that you would like to see there. I'm just

gonna skip this stuff since I talked about
it

earlier, and we're running short on time.
So, here's

Scheems to actually talk about Rails.

R.S.: OK.

Has anybody used Rails? Have we covered that
question

yet? OK. Welcome to RailsConf. OK, so Rails
4.1

on Heroku.

A lot of things in a very short amount

of time. We are secure by default. Have you

heard of the secrets dot yml file? OK. So

secrets dot yml file is actually reading out
an

environment variable by default, which is
great. We love

environment variables. It separates your config
from your source.

And, so whenever you push your app, we're
gonna

set this environment variable to just, like,
literally a

random value. And if, for some reason, you
ever

need to like change that, you can do so

by just setting your, the, the secret key
base

environment variable to, to whatever you want.

Maybe, you know, like another OpenSSL bug
comes out

or something. So, another thing that was worked
on

a bunch is the database_url environment variable.
This is

something that we have spent a lot of time

looking at. And it's actually, support has
been in

Rails for a surprisingly large amount of time,
to

just read from the environment variable, but
never quite

worked due to some edge cases and random rake

tasks and so on and so forth. So this,

this December, around Christmas time, I spent
a lot

of time getting that to work.

So I'd like to happily announce that Rails
4,

4.1 actually does support the database_url
environment variable out

of the box. Whoo! And, so, some, to describe

a little, like, the behavior is, bears going
over.

If the database_url is present, we're just
gonna connect

to that database. It's, that's pretty simple.
Makes sense.

If the database.yml is present but there's
no environment

variable, then we're gonna use that. That
also just

kind of makes sense.

If both are present, then we're gonna merge
the

values. Makes sense, right? OK.

So, we, that sounds crazy. Bear with me. But,

a lot of people, you, you want to put

your connection information in your database_url
environment variable. But,

there's also other values you can use inside
of

your database.yml file to configure ActiveRecord
itself. Not your

database. So you can turn off and on prepared

statements. You can change your pool size.
All this

kind of thing.

And, we wanted to still enable you to be

able, able to do this. So the, the results

are actually merged, and for, for somebody
like Heroku

or, like, if you're using another container,
we don't

have to have as much magic. If you, if

you didn't know, database_url, we actually
had to over,

whatever your database_url was, we were just
writing a

file over top of it. And it's like, forget

that. We're gonna write a custom file.

So people would put stuff in their database_url,
or

their database.yml file, and they'd be surprised
when it

wasn't there. Like, a different file was there.
So,

we no longer, we no longer have to do

that. And Rails plays a little bit nicer with,

with this containerized style environment.

It also means that, you could actually start
putting

your ActiveRecord configuration in that file.
Another note, if

you were manually setting that, your pool
size or

any of those things via a, after reading an

article on our devcenter, go back and revisit
that

please, before upgrading to Rails 4.1. Some
of the

syntax did change between Rails 4.0 and 4.1.
So,

if you can't connect to a database, then maybe,

just like, email Schneemz and be like, I hate

you. What's the link to that thing? And I'll,

I'll help you out.

OK. I think, probably, actually, the last
thing that

we have time for, is asset pipeline. Who,
like,

if asked in an interview, would say that their

favorite thing in the whole world is Rails
asset

pipeline? Oh. Oh.

AUDIENCE: Just Raphael.

R.S.: Just Raphael. We have a bunch of, like,

Rails Core here, by the way. So you should,

you should come and thank them afterwards.
For, for

other things. Not for the asset pipeline.

[laughter]

So, the asset pipeline is the number one source

of, of Ruby support tickets at Heroku. Just
people

being like, hey, this worked locally, and
like, didn't

work in production. And we're like, yeah,
that's just

how asset pipeline works. That's not Heroku.

So, so Rails 4.1 added, added a couple things.

It's gonna warn you in development if you're
doing

something that's gonna break production. Like,
if you've ever

forgotten to add something to your precompile
list, well

now, guess what, you get an error. If you

are not properly declaring your asset dependencies,
then you're

gonna get an error.

And this is even better, actually, in Rails
4.2.

As some of these checks aren't even needed
anymore,

we can just automatically do them for you.
But,

unfortunately, those have, are not in Rails
4.1 yet.

So, in general, I have a, a personal belief

that, in programming, or, really in life,
the only

thing that should fail silently is. This.
This joke.

So. Thank you all very much for, for coming.

We, we have a booth, and later on, what.

What time, three o' clock?

T.L.: Between 3:00 and 4:30.

R.S.: Yeah. From 3:00 to 4:30, we'll actually
have

a bunch of Rails contributors coming to, to
talk

about. Oh yeah, the slides. Yeah. Yeah.

T.L.: Yeah. 3:00 to 4:30, we'll have community
office

hours with some nice people from Rails Core,
contrib.

R.S.: Yeah. So come ask.

T.L.: Basically any Rails questions or anything
you want.

And then Schneeman will actually be doing
a book

signing of his Heroku Up & Running book today

and tomorrow at 2:30. So if you want that.

R.S.: Yeah. So get a, get a free book,

and then come and ask questions and just,
like,

hang out. And, any time you stop by the

booth, feel free to ask Heroku questions.
And thank

you all very much for coming.