??: Hi everyone. So we're gonna do,
we're at Living Social.
We build this consumer marketing platform.
We, you go to LivingSocial dot com and
it looks like it's just deals.
I know it's small and boring.
But there's like, actually a phenomenal
amount of stuff under the water line.
We have, most of our apps are written
in Ruby and Rails. But we also have teams
working in Clojure, Scala. I almost said SmallTalk.
That would be a lie. iOS, like native iOS,
native Android, big data stuff, operations.
All sorts of stuff. So there's tons of interesting,
wei- like, big problems that we come upon.
And we're gonna do a few lightning talks on
those technical things, and then also a
few on how we operate as a distributed team,
and how we just operate as developers.
So, the first person that's gonna speak to you
is Ed Weng. He's a, he's a graduate from
our Hungry Academy training program,
where we pull basically just, like,
thirty random people off the streets.
No. They, they, it was not,
we did not press gang them.
We, they applied, we put them through a,
was it nine weeks? Nine, nine months?
ED WENG: Six months.
??: Six months.
E.W.: I don't know where you got nine weeks.
??: Nine. It was some number of months. It
was six months. They did a really intensive training
program, and are all really amazing developers. And so
he's a Hungry Academy grad. He works on our
mobile consumer apps and if you meet him later,
ask him about his pen pal. So this is
Ed.
E.W.: Hey guys.
[applause]
So, so my name is Ed Weng, and what
I'm gonna talk to you today is about sharing
templates in a service-oriented architecture.
Cool. So, if you've never been to our site
before, if you go to LivingSocial dot com, this
is the page that you land on. And to
give you some context about our architecture, about two
and a half, three years ago, we were still
a monolithic Rails app, right. So everything was inside
this one gigantic Rails app.
Since then, we've been working very hard to kind
of break things out into smaller micro services. So
if you come to this page, this is actually
in an app that we call browse, and browse
is responsible for surfacing the browse experience to our
customers.
So, you can see some of the deals on,
on our site. If you click on, let's say,
the Audible deal, you actually get taken to a
page that looks like this. So, most of our
users won't be able to tell, but they actually
just left the browse app, and now they're in
an app that we call sponsors. So, sponsors is
responsible for surfacing deals that are sponsored. So this
deal is sponsored by Audible dot com, hence its
free. And let's say a user then decides to
click on this tab up on the top. The
shop tab.
They'd get taken to here. And this app, again,
is not the sponsors app anymore. It's an app
that we call products, and products is responsible for
surfacing physical goods that we sell to our customers.
So, if we were to look back at the
last three slides I just showed you, and I
were to ask you, what is the one common
thing that appears on all three slides? You might
say it's the nav bar. And you'd be correct.
One of the things that we kind of discovered
along the way of splitting things out into smaller
services, is things that might be very simple in
a monolithic Rails app can become quite complex when
you move towards a service-oriented architecture. So the big
question that, that we're gonna ask ourselves today is,
well, when it comes to sharing views, how exactly
do we keep ourselves dry, right?
So, I don't need to tell you guys that,
you know, out of the box, in a monolithic
Rails app, you have partials and layouts, right. So,
how much you show the nav-bar in every single
page. Well, you'll throw it into a partial called
nav-bar, and then you'll just go into your application
layout, you'll render the nav-bar, and then it just
shows up everywhere, right. This is great for a
monolithic Rails app. But as soon as you move
to a service-oriented architecture, it starts to break down.
Well, why is that? The first thing is that
it's a lot of copy and paste coding, right,
so any time you want to make a change,
you have to copy that nav-bar into a bunch
of different places. The second thing is that it
requires changes in multiple places, and so it, it
kind of slows your development down. Any time you
want to make a tweak, you have to go
to five, ten, fifteen services and issue a, a
pull request, so that you know, the nav-bar is
updated in that specific application.
What we've noticed is that, things tend to get
out of sync very quickly if you, if you
take this approach. So, it leads to a bad
kind of user experience, in that the nav-bar starts
to change as you go from page to page.
So this doesn't really work for us.
The next thing that you might think of doing
is, well, what about putting everything inside an engine
or a gem, right? And this is a, a
much better solution, right. Why is it much better?
Well, the first thing is that it, it eliminates
that copy and paste coding that we were just
talking about, right. So everything is kind of stored
in this single central repository, and then, and then
any time you want to make a change, you
just update the gem and kind of release it
into the wild.
One of the great things about putting this in
an engine or a gem is that you can
share more than just the mark up, right. You
can share translation files, you can share images, JavaScript,
stylesheets, et cetera, et cetera. So that's great. The
problem with it, though, is that any time you
make an update, you still have to go to
every single app and issue a bundle update.
So, you know, it might work fine if you
have a couple of apps. As soon as you
move to five, ten, fifteen apps, it becomes a
hassle. So this is not quite good enough. And
at LivingSocial, we, we have a saying, that if
services aren't solving all of your problems, you're probably
not using enough services. So, what did we do
to solve our problem?
Of course. We build another service. And the service
we built is called Stepford. It was designed by
a guy named Eric Brody. And Stepford is responsible
for sharing our views across our service-oriented architecture. So,
when we were first building Stepford, we went through
kind of a DDD approach. A desi- a dream-driven
development approach. And we asked ourselves, what do we
really want the interaction with a service to look
like? And this is kind of what we came
up with.
So, we have a client app. We wanted the
client app to be able to ask Stepford and
say, send me the footer, right. Stepford would then
go off, do some stuff behind the scenes, and
then just come back and say, here's your package.
So, let's kind of break that down into three
separate steps.
So the first step, send me the footer. We
tried to come up with the simplest possible thing
that we could think of. And that is just
a GET request to some JSON endpoint, right. So
this is what the request looks like inside the
Stepford app. It's a GET request to, to the
packages controller, and we just put the things that
we want. So, different elements inside the query string.
So, here we're requesting the footer. We can also
pass additional variables. Things like city id, if we
need that footer to be semi-personalized or modified.
So now you're probably asking, well, what's inside a
package, right? So a package is actually made up
of three separate things, right. It's made up of
markup, it's made up of styles, and it's made
up of scripts. And everything kind of revolves around
the markup that, that you're requesting. So you might
imagine that if you're requesting the footer, there's an
associated set of styles with it, and associated set
of scripts with it.
If the styles don't come through, then your footer
looks really ugly. And if the scripts don't' come
through, then your footer probably isn't gonna work the
way that you want it to work. So, we
kind of package all of the three things up,
cause they all kind of have to come together.
And we put it in this concept of a
package.
OK, so the next step. So the next step
is, Stepford gets that request, and what does it
do behind the scenes? So, it's a little bit
complicated, and this is a very simplified high-level version
of it, but in the top-left corner, we see
that the GET request comes into that package's end
point. Stepford then tries to make the package. So
it grabs the mark up, it grabs the styles,
it grabs the scripts, it tries to precompile the
styles, it precompiles the scripts, it throws everything into
the database, and then it actually tries to render
the mark up on the server side. So that's
super interesting. It takes the erb template that is
stored inside Stepford, and it actually renders it into
raw html. It then sends it back via JSON.
So that's the last step of the three-step process.
The here is your package. And I'm about to
show you what a package response looks like. So,
the package response contains the three things that we
talked about, right. The styles, the scripts, and the
mark up. And here, you can see that all
we requested was the footer, right. But you can
imagine that in our, in our architecture, we're, an
app generally requests more than just the footer. It
probably requests the footer, it might request the nav-bar,
it might request button styles, et cetera, et cetera.
And that is kind of all going to show
up in that mark up as raw html.
And the nice thing is that Stepford will kind
of package together all of the styles and all
of the scripts for every single mark up element
that you've requested. So, everything just kind of boils
down into one precompiled stylesheet and one precompiled JavaScript.
Cool, so let's spend a little bit of time
talking about the pros and cons of spinning up
a service like Stepford. The obvious, biggest pro is
that you keep your views DRY, right. I don't
need to tell you the benefit of, of having
all of your views in a, in a single
central repository, right. There's huge benefits. It's easy to
maintain. Et cetera, et cetera.
The second thing is that it's super easy to
release changes. So Stepford is all pull-through, which means
that any time you update a nav-bar or any
time you update the footer, you don't have to
go to every single app and tell them, there's
a new nav-bar. There's a new footer.
The next request that it makes to Stepford, it's
actually going to, Stepford is actually going to send
back the latest version of whatever element there is
on, on the, in the database.
OK. So, there's obviously some cons. The first con
is that it's another service. So one thing that
we've realized by spinning up so many services at
LivingSocial is that, any time you make a service,
you're trading application complexity for network complexity, right.
So that means that your tech-ops team has to
maintain one more application. They have to make sure
that it's up all the time, otherwise, you know,
your styles won't come through. The next thing is
that, because it's another service, it also means that
there's another network request going on. So Stepford does
introduce some amount of latency into your application, right.
There's another HTTP request that goes on on the
server side.
At LivingSocial, the way that we kind of have
gotten around that is we put in a lot
of, a lot of caching, right. So we have
bonus caching around Stepford. We also have Memcache around
it. The reason is is because, a lot of
times, our main UI elements don't change all that
frequently. And so it's OK to put things like
varnish in front of it. It's OK to put
Memcache in front of it.
And the last thing is that changes have a
much larger impact. So when I said it's easy
to release changes, sometimes it's too easy to release
changes. You know, there have been instances where we'll
make the change in JavaScript, and we quite, and
we can't quite anticipate what the effects downstream will
exactly be. And so as such, you really need
very close JavaScript monitoring to make sure that you
understand when errors come through.
So that's it. That's my talk on Stepford. If
you have any questions, please come find me after
the talk. And we, we'll chat.
[applause]
??: Thanks Ed. So, the, these guys get to
watch me use a computer. It's great fun. Over
my shoulder. This is Tyler. Tyler works, also, on
our consumer apps team. He's from Southern California, which
is, his handsome looks, as you can see.
TYLER MONTGOMERY: Northern California.
??: Nor- I botched it. Sorry. But still handsome.
But he's taken ladies. And I just learned from
his website that he wants to learn everything, so
if you see him in the hall, just liter-
tell him anything and it will be the best
part of his day.
T.M.: Howdy. I'm Tyler. I live in Northern California.
I work remote. So, how do you work a
computer? OK. Let's, let's start at the beginning. It's
a good place to start. All right. Ready and
play.
All right. So I'm gonna talk about internal gem
infrastructure. Basically making it easy to write gems, release
gems. For us, you know, sharing is caring. Being
able to share code effectively across your apps makes
things really awesome. It's easy, like we said, to
extract things into services, break down the monolith when
you can share code and you can put it
into a gem really easily. It's, reduces the copy-paste
mayhem that we've had, you know, we put up
a little service and someone writes a little client
that gets copy-pasted fifty places, and then we make
changes. It's really hard to track down who's hitting
what end points, so we've had to go back,
put those things into a gem so we can
share and keep things maintainable.
Building gems and having this, you know, code in
gems really helps build this culture of internal open
source. A gem gets its own repo, gets its
own README. It's really easy for anyone to contribute
to it and people know where things are. You
get all the benefits of pull requests.
It helps with separating concerns. Like, we're told, like,
hey, you shouldn't make a mess, you know. But
where do you put that mess in a Rails
app? It's kind of weird, like, where do I
put all these little tiny objects with single responsibilities?
Where do they live? How do I keep track
of it?
And putting the stuff in a gem is really
nice. It helps you draw boundaries around your code
to find your domain better. Isolating your code is,
is actually really great for deleting it later. It's,
you can track it down. You can find where
it needs to go and get rid of it.
But you don't necessarily have to delete the code.
You don't have to go, where is that one
commit that one time where I did that one
thing? And I gotta go look through all these
old commits. You just go look at the repo,
but you've deleted all that code out of the
other code bases you still have some of that
history around.
Isolating is good too, because it forces you into
how does your code interact with Rails? How do
you interact with the database? How do you do
logging when you're isolated in your own little gem?
Do you even need Rails for this little business
project that you're working on? Does it need to
be part of a Rails app?
This has really helped me, like, try to constrain
my code in the simplest form it needs to
be to get the job done. This helps me
mock and stub boundaries in my tests, cause it's
clearly defined where it needs to define other, other
pieces of our architecture.
This has really helped me with writing good docs,
because I have a README for this thing. It's
really easy. So, a real example, a few weeks
ago I was tasked with automating a daily seatsbe??
(00:15:18) report out of the database, and uploading it
to a third parties FTP site. They then will
return us a plain text email telling us the
results of processing this data. a coworker commented, 1997
wants its architecture back.
So we've all done this. We say, hey man,
I'm just gonna drop this in the model, you
know. This FTP goes to CSV, it uploads. Gets
the job done. You know, fat models, bro. And
a couple weeks later you're like what is, what
is this code? Who put this here? What does
this even do? Or, a year later. Two years
later. Who, who did this?
And, so, you know I was thinking, this is
really gnarly code. It has, just, it has just
kind of a single responsibility. It has just one
thing it needs to do. So, I'll make a
gem. So, the rest of my talk, I'm just
gonna quickly go through building gems, releasing gems, and
then hosting your own gems internally.
So, building a gem. This is another crash course
inside a crash course. This might be review for
some of you, but this is just kind of
the easy way that I see how to do
it. So, it's a, it's another crash course, so.
So building a new gem is this easy. You
already have this installed. You already have bundler installed.
You just type bundle gem your gem name. The
ls on the front there, that's to help us
with namespacing. I'll get into that in a second.
But bundler will do that for you.
This is what it creates. This is your basic
fresh gem. It's the gem file, a license which
is MIT, you get your README, like I said.
You get your Rakefile and then you get your
lib directory with all your program code in there.
The most important file there is the bottom and
that's the gemspec. Without that, you don't have a
gem. There's plenty of documentation on, you know, what
goes in a gemspec. But bundle does a good
job of doing defaults for you.
Namespacing. If you hadn't seen this before, I've done
this with a couple different companies and it's really
helpful. You basically take your company name, shorten it
down to two letters, and you put it in
a module and you put all your classes under
this global kind of module. That's how you use
it there. It's LS for this one, GeminaboxClient, and
you require it. If you see that, like, ls
slash thing, that's how it's, that's how you're doing
namespacing.
This is really helpful. We have some early gems
that didn't do this, and I wasn't sure when
I first started. Is this an open source implementation,
or is this internal? Like, where did this thing
come from? So, maintaining this namespace is, it's, it's
helpful, and knowing where your code is coming from
and that it's an internal project.
So, basic example. This is just kind of an
example that has a few more things in it.
We threw a changes file in there so we
know what happened when the version changes, and then
we've got our, you know, some more program code,
and we have our tests. Like, everything is nicely,
tightly compacted for, for this little project.
A quick review on semantic versioning. You can go
to SemVer dot org. Yeah. Thanks. Some people don't
understand it so I wanted to review it. The
last number in a version number is the bug
fix number. You increment this when things change. Like,
when you fix bugs, but you haven't broken the
public API. You haven't changed or broken anything.
The minor version number is when you added a
new feature. Yay. We have new stuff. But you
haven't removed anything. You haven't broken the public API.
You haven't forced people into changing things.
And then finally, the major version number is, you
broke something. You have changed significant amounts of things
and people beware. Like, this might break your code.
And you can think of this like how hard
it is to upgrade from Rails 2 to 3
or from Rails 3 to 4. That's a big
change so that is a major version number.
If it's stable, it should be 1 point 0.
If it's not stable, it should not be 1
point 0. This is a good signifier that just
says hey, this has been used in production. It's
stable. It's, everyone can use this when you go
to 1 point 0. If it's before then, you
probably see gems that are zero zero twelve-hundred and
thirty-seven. And it's like, I'm not sure if I
should use this. That's a lot of version numbers.
So, that's just quick refresher on building gems and
versioning, namespacing. Now, we need to let the world
behold your awesomeness, or, at least the people at
your company. Releasing should be easy. We say deploying
should be easy. Deploying and releasing gems should be
an easy thing. This is not easy. Remembering the
version number and tagging it and rake build and,
like, this is not easy.
This is easy. Rake release. This is really easy.
When you use bundler you get this rake command
for free. But, don't accidentally open source your code.
This happens, and there are bots that mirror RubyGems
right away. It's very hard to take that back.
So how do go from this to this, but
keeping everything internal?
You can monkey patch, I mean subclass Bundler. Geminabox
will actually tell you on their Wiki page to
monkey patch Bundler. Don't do that. So what we
can, we wrote our own gem that just inherits
from Bundler's gem helper. There's tutorials on this. But
I'm just gonna go through real quick how this
works.
In your, in your Rake file you see this
bundler/gem_tasks at the top. What that actually does is
that gives you those three default rake tasks. The
build, the install, and the release. By default, it's
going to push it to RubyGems, and we don't
want that.
So, when we subclass it, what we want to
do is we want to say, we use geminabox
as our gem server at Living Social. We want
to say, hey, don't push to RubyGems. Push to
geminabox. So we've subclassed Bundler and said hey, don't
push to RubyGems. If you try to, if you
even try to call the private, the protected method
there, you're gonna get an error. So just don't
do it.
And then once we release that gem, it's just
a really small gem, we put ls/gem_tasks in place
of bundler/gem_tasks in all of our other gems. So,
by default this goes to our gem server when
you release. It's really easy.
So, gem servers. You, you know how to get
it out there and now you need some place
to put it. So there's three options. There's geminabox,
stickler, and then gemfury. So if you're hosting your
own, it should be behind the firewall. It's internal
code. You don't want anyone else accessing it.
Geminabox provides a nice web interface for managing your
code. It has authentication. You have to build your
own Rack middleware to do the authentication. It does
pull through mirroring, which is kind of cool. You
can tell it to mirror everything from RubyGems as
you install things, so you don't need to depend
on RubyGems being up when you deploy. Geminabox will
pull down everything for you.
And it has a nice command line client. It's
pretty simple. Patches gem with this inabox command and
allows you to push and setup your own machine
to talk to your server.
The other option for open source thing, gem servers
is stickler. Provides authentication in the same kind of
way. It does selective mirroring, so you can tell
it, you know, mirror ActiveRecord 3 point 2 and
it'll pull it from RubyGems. You can also give
it a whole gem file, gem file lock, and
it'll take everything in there and it'll mirror all
of it for you. Which is handy.
It has a really full-featured command line client. You
can do everything that you can do from their
web interface. And then it's, the author wanted me
to note that it's going to support the bundler
dependency API in the next release coming soon.
For a hosted server that's not behind the firewall,
it's on someone else's machine, there's only one option
I know about. It's gemfury dot com. They provide
authentication obviously because it's not on your machine. They
don't appear to do mirroring, which is, you know,
if you're still rolling on someone else's service you're
gonna probably be relying on RubyGems, too. So, they
don't do mirroring.
They have command line client, and they do support
node.js and Python packages. So that's it. My name's
Tyler Montgomery, and I'm @ubermajestix on Twitter.
??: Thank you, Tyler. Next up we have Dan
Mayer. He's also on our consumer app team. Not
everyone is on the consumer app team. He does
a lot of teaching in the D.C. area, teaching
people Ruby. And I just learned that he can
fix bugs from ski lifts. Over the phone. So
that is a useful parlor trick. And take it
away Dan.
DAN MAYER: All right. Hello. I'm gonna be talking
to you about production code analysis. Obviously as you
build up all these services and split out all
your code, it gets much harder to debug and
start finding out things about your code. Like a
small application is easy to reason about in your
head, but as your systems grow and the architecture
grows it gets more difficult.
Oh. Could somebody fix the projector and get us
all on? Shrink down. Is that one on? Or
is that off as well? If somebody could fix
that one too, that'd be great.
You can't improve if you don't measure. You can't
improve what you don't know. We often focus on
performance and exception monitoring. Some fixate on test code
coverage. There's a lot to learn from what your
code actually does in production. Ruby tools aren't quite
as good as, like, the Java tools, so we
really are still trying, as a community, to improve
those tools. So that's some things we're working on
internally.
And then, I wanted to mention Etsy. They have
some great posts all about measuring production systems. Graphing
everything and actually release Statsd, which we rely on
quite heavily. So I wanted to just say thanks
for that.
All right. So, I like to focus, as we
split things out and kind of forked off code
bases on getting rid of unnecessary code, because I
want to be able to only think about and
reason about the code that we actually care about
and what's actually being used in a system.
That code ends up in production for a whole
lot of reasons. I'd list up a whole bunch
there, but we don't really need to go through
every one. If you've ever worked on a team
as it grows, you eventually find code that you're
like, this was written three years ago and nobody's
using it. Why is it here?
I think it's important to try to track that
down and get rid of it as soon as
possible. Especially as you're adding a lot of code
very quickly to systems.
So there's various ways to find that code. Some's
as simple as just using New Relic, or formally
trace Linux. You can do custom stats instrumentation, which
I'm gonna go, going to talk about a little
bit. There's production code coverage. There's the gem I've
been working on that does code coverage, but not
on your test suite. It's actually live code running
in your system.
And after seeing some of the talks by Tilde,
I also realized I could make it much more
performant in Ruby 2.1.1, so I'll be working on
doing that, cause right now there's a, a high
cost, which I'll talk about a little later.
You can use logs. You should definitely have your
logs all searchable. And we'll go into that. Using
stats in implementation and production code coverage we've deleted
twenty plus thousand lines of app code. And then
hundreds of thousands of lines of code, if you
start including our assets, our JavaScript, test files and
that sort of thing. We've really been able to
shrink down our code bases as we expand out.
So, third-party, real quick. If you use New Relic,
you can go to the transactions. Go look at
the last seven days. Sort by count. The things
at the bottom, likely, are dead, deprecated, or you
can start working on killing them. They have, like,
one request in the last seven days. You might
want to go look at what those are.
It's really easy but it's still useful. If you're
ever finding an end point and you wonder, wait
a minute, what is this? Go check. It might
not be hit in the last seven days. You
can probably kill it. It's really easy. It's actually
even easier, because we have a gem, newrelicroutecheck, which
basically you can download the CSV from New Relic.
This generates your Rails routes file and then compares
your routes versus the output of New Relic, and
then starts telling you, this was in your routes
file but never actually hit. You can go look
at those end points.
Stats instrumentation. We're gonna step through all of these
fairly quickly. But basically there's a lot of different
things you can do with stats to help you
find code and tell what's going on in your
systems. So, background events. You want to know which
jobs are being executed. You want to know if
they're completing successfully. You might want to know how
long they take. This is an example for Resq.
We actually have our own wrapper around most of
our background jobs, so we do it at a
different layer.
Statsd. Before preform. After preform. You can throw in
whatever else you want there. If you want timers.
Anything. But it's really easy to instrument that. Then
you have your nice graphite graphs, where you can
kind of follow and see the, see the charts
and everything. You can also look at graphite and
see if a job's no longer ever being performed.
Emails. Email templates, especially transaction emails. Eventually, you have
a one-off transactional email for some holiday thing. Some
special product. Something that's no longer being triggered. You
can check all your emails. There's no, no reason
to work on updating them or fixing them, especially
when you're doing, say, a Rails update and you
have to change all your ActionMailer code. This will
let you know if it's worth your time.
Views rendered. Eventually your view layer will get very
complicated, with layers and layers of partials. If you
have layers and layers of partials, it's hard to
reason about which ones are still in use, which
ones have been refactored away entirely. You can actually,
really quickly, using ActiveSupport::Notifications, find all your layouts and
all your partials and templates and track them. This
is an example showing it to Statsd, or you
can just render it into your Rails logger and
then you can search it in something like Splunk,
ElasticSearch, see if it exists.
Again, we made a gem for this to make
it easier. You can see flatfoot. Basically, you set
up flatfoot. It does this for you. And then
it gives you some helpers to find used views,
unused views. You can reset it on each deploy.
It makes it really easy to kind of automate
this process and generate reports on it.
One off trackers. Sometimes you just find some code
and you're not quite sure what's going on. It's
very easy, with Statsd, to just throw on one-off
trackers. This is an example of a controller. We're
trying to figure out which of the types, or
paths in the controller are still being hit. You
can go ahead and throw increment on each of
the formats. On that request dot XHR. Now you're
going to be able to say, oh, actually we
stopped using the, you know, JSON format months ago.
Let's get rid of all the related code.
Production performance comparisons. This one I owe thanks to
uberjemef- ubermajestix over here, Tyler. He actually just threw
this out and didn't really make any fanfare of
it on one of our applications. Then I saw
this and I was like, that's really cool. We
had two different options for stripping out some html,
and we wanted to know what's faster. You could
benchmark it locally on some fake data, but the
production data is very different than what you're going
to test with. And it, it will vary a
whole lot over all our applications.
He threw this in there. Splits it fifty percent.
Throws the timer around it. Measures both one implementation
of the algorithm versus the other. Couple days later,
it's incredibly obvious that nokogiri is much faster than
just using the regular strip tags from Rails.
And that, that I, I just thought was so
interesting that I've kind of been carrying around to
point out to everybody I can find that's interested.
Translation usage. This one I owe Chris Morris some
credit. He is another engineer at Living Social. He
works a lot on our payment systems. But he
used to do a lot of our translation and
internationalization work.
And eventually, your translation files get really
large. And they actually take up a lot of
memory in your running Ruby processes. So if you're
trying to shrink down your memory so you can
foot, fit more workers on a box, you might
want to look at your translation files. They tend
to be a pain point for a lot of
companies that have a lot of languages.
This lets us track which keys are actually being
used. What are we still translating in production? Or
what is it we no longer use the translations
for. As you delete all these views, it's easy
to forget about all the translations that were in
the views. This actually makes it very simple to
go and find those.
Production code coverage. This is a little bit more
experimental at the moment. We do run it on
production. I have it on a lot of systems.
I cannot use coverage, which is what, like, simple
cov and things use in your test suite, because
there's a bug in Ruby. If you sample and
turn it on and off, it will crash. Which
luckily I learned prior to production.
So I switched to using set trace func, and
this lets us see each line as it's being
run live, and then we do sampling to cover
the fact that set trace func is extremely slow.
Sometimes four hundred, eight hundred percent slower. So we
can sample a very small rate of requests, but
it still will, over a whole lot of volume,
give you a pretty good idea of what's used
in production.
And, like I said, in 2.1.1, I just learned
that you can actually get the CPU profiler, is
a sampling profiler, and you can get the line-number
data out of it with a very small C
extension. So I'll probably be updating this, and that
will only help 2.1.1. It'll still have to use
set trace func on 1.9.3 and 2.0.0. But you
can get some really interesting data and delete a
lot of code this way.
Here's a quick example of the setup, but really,
go to the gem's home page. You can see.
You can do ignores on things. It'll check your
app path. It tracks in Redis. I use a
startup delay, because Rails is very slow on the
first few requests, because it actually builds a lot
of code in memory at that time. And you
really don't want to be checking your code coverage
as it's building new methods. It's just not a
good use of CPU.
Yeah. And this shows how to set it up
in your Rakefile. And real quick I'll, I'll -
I was gonna have a demo. I think I
missed the slide. But we're, we've got other speakers
so I'm gonna skip ahead real quick.
On logs, I just want to mention. Make sure
to get all your logs from all your applications
in one place. ElasticSearch, Splunk, Cabana, doesn't really matter.
You just want everything in one place so you
can actually see how requests are going across your
system.
I've been working on a gem called imprint. But
also you can look at Zipkin, which is, I
guess, some of like Twitter's work. Mostly they do
Scala, but I guess they have stuff for Ruby
as well. I just learned about that at this
conference. But basically, if you put a trace id
on your requests, you can see as they're coming
in your system, and then as you make multiple
API requests to your backend, you can see how
that fanned out in your system.
You can also drop it to all your events.
So any event that fails, you can track it
to the incoming request. If you have an exception,
it's going to be in the headers. It'll be
on your exception. You can actually take the exception
and go see all the logs for all the,
the requests and debug what was coming in that
was so awkward to cause this exception.
And, I think all these tools can help you
clean up your code and actually understand what's happening
in production, much better. And that's all I've got.
??: Thanks Dan. Right. Are, are you going next?
Or is, K. I don't know.
I, I'm telling Rodrigo to sit down. Nick. Nick
is gonna go next. Nick is a, a Minnesotan.
He's on our Mail tools team. A JRuby contributor.
He, and for one, he's six foot four inches
tall and can help you get things off of
tall shelves. And he's gonna talk to us a
little about meditation.
NICK SIGER: [French]
Which is French for, I think, therefore I am,
of course, by the famous philosopher Renee Descartes. I'd
like us to ponder today a little bit about
whether that's really an appropriate frame of mind. You
can interpret that statement to mean that we identify
ourselves, our entire existence, by the fact that we
can think. And I personally think that's a really
bad idea.
Early in my career, I was obsessed with my
capacity of work done. I wanted to work all
the time. And I was constantly searching for that
sort of state, that elusive state of flow. But,
in fact, what I think I really got stuck
in, most often, was a, an oft more common
state of rat hole.
So, when you think of yourself during the work
day, you know, how often do you find yourself
getting distracted? Running down a rat hole, and then
once you get done with that hole, that hole,
once you finally unwind the stack get all the
way out of that rat hole, how many times
do you find you saying, wow. That felt great.
Like, no. It probably did not. And I'm arguing
that one of the reasons why it doesn't feel
great is because we all too often let our
minds run wild. We take every thought that comes
into our head and we believe it's truth or
we believe there's some element of truth to it.
We might even have perceptions about ourselves and the
way other people perceive ourselves, and we think that
for some reason those are true.
And I'm here to tell you that they're not.
I think that we have, all of us, inside
of us, we have a sort of a silent
watcher-observer of our lives that's behind all the thoughts.
And if we can all take a little bit
of time in our day to access that part
of ourselves, we find that we're gonna be happier.
We're gonna be healthier. And that's kind of a
little bit of what I'm gonna, what I'm gonna
outline today.
So, as far as meditation goes, in terms of
discussion for today, I'm gonna say that meditation is
sort of spending some quiet time, you know, attempting
to find sort of an absence of thought in
your head. And if you've ever tried meditation you
know this is actually extremely hard. People have been
meditating for thousands of thousands of years, and it
is most-certainly a life-long practice. And when you realize
how hard, you think, oh my gosh, how am
I gonna actually- will I ever get to a
state where I feel comfortable with this? Is this
something that's really worth while to do?
And I would advise up front that meditation, like
many other things in our lives that we find
worth while to do, all it requires is just
a little bit of time and making it a
habit to get the most out of it. So
don't make, don't, don't listen to the voices in
your head again. Don't listen to your thoughts that
are saying, oh my gosh, I have no time
to do this. I'm, I'm not gonna do this
right. I don't know how to do this. It's
not gonna be perfect. So therefore I'm not gonna
try. Just ignore all those and just try to
do it.
There's a number of studies out there, meditation does
have actual tangible, scientific health benefits. There's one study
that had people meditate for thirty minutes a day
for eight weeks, and they did brain scans on
the people before and after and found that their
gray matter in brain regions associated with memory and
stress and empathy actually increased in just eight weeks.
So you can actually, you know, even as an
adult, you know, we're long passed puberty, even as
adult you can spend time and change the composition
of your mind and the composition of your body.
Another study at UC Davis foc- said that focusing
on the present, doing activities like meditation, rather than
letting the mind run away, adrift, actually lowered levels
of the stress hormone cortisol in your body. So,
meditation also will allow you to get on top
of your, your hormones, and get a better hormonal
balance. You'll feel a lot better about yourself.
So, unfortunately, a lightning talk is probably a bad
place to actually try to, try to lead a
meditation session, since I think you, to do it
justice, you'd want to probably have just at least
a five minutes quiet time to yourself. But instead,
what I'm gonna do is take just a few
moments and actually ask all of you to, you
don't have to do this yet, but what I'm
gonna ask you to do in a minute here
is close your eyes and just try to get
comfortable, and try to clear, clear your mind of
any thoughts.
And then what I'm gonna ask you to do
is to just spend some time trying to obtain
that, that, that clear head, and watch your mind
for the very next thought that comes out. And
watch your mind like a cat would watch a
mouse hole. So I'm gonna ask you all now
to close your eyes, and watch your mind, and
just try to watch for the very next thought
that comes in. And just try to be focused
and attentive. Don't try to follow it, don't try
to lull yourself to sleep or anything. Just be
focused and attentive and try to look for that
very next thought.
And I'll just give you a few moments. And
when that thought comes in go ahead and open
your eyes. OK. Some of you actually lasted quite
awhile there. That's pretty good. So what you may
be finding with this little exercise is that when
you take the time to be really attentive about
having a clear mind, it actually takes a little
while longer for that thought to come in.
So this is, this is actually the very beginning
of the meditation practice. You're actually trying to, rather
than being completely, completely lifeless or, or almost like
a, well, you, you're actually trying to be very,
very attentive to the immediate present moment.
So, when you're doing this, it's important to not
be self-critical, right. So maybe a thought rushed in
right away, again, and you feel like, oh gosh,
I have no control of it. Well, don't try
to control what's happening. Don't try to be on
top of it. We, you know, we're all control
freaks a lot. We're in technology. We like things
to work. We like to control computers. Don't try
to be in control of the situation. Just observe
where your thoughts are jumping to. And just, just
try to be that passive observer of what's happening
in your mind.
And then, so, once you start to get in
this habit of taking time to at least check
in with yourself, maybe you start a meditation practice
or maybe you just want to take moments out
of your day to just take a break from
things and be a little bit more mindful and
a little more self-aware. Ask yourself throughout the day,
am I at ease at this moment? You know,
ask yourself, how do I feel at this moment?
Am I at ease? Or what's going on inside
me. And just, just pay attention to those feelings
and those, what's going on inside you without judging
or, or trying to make any changes about it.
We're just trying to cultivate self-awareness here.
I also find that doing meditation has lead me
to doing other things really mindfully as well. I
like to, I like to cook. I like to
clean. I like to do all these little kind
of tasks that don't require a huge amount of
mental energy. And I like just focusing on them
intently. And I find that even doing things like
chopping vegetables or, or clean- or doing dishes or
any task which you think normally would be completely,
you know, oh, what a waste of time. I
don't want to do this right now, I really
don't want to do these chores. You can actually
find a little bit of pleasure in just focusing
completely on the task.
And that's another way to just kind of open
yourself up and be, be more at ease. And,
you know, of course, we all have screens on
us at all, at all times of the days
these days, and that's another, you know, where it's
like, looking at a screen is, you're putting your
attention out into the ether, out somewhere else from
where you are right now. So when you're with
someone, even when you're by yourself, maybe every now
and then put the screen down and just focus
on, on just what you're doing.
And I think, for me, my opinion of, of
the, you know, this, this lifelong process of trying
to be more mindful, trying to meditate, trying to
make, make ourselves better through meditation, is actually a
contagious thing and one that we can contribute to
the well-being of the entire planet. So I hope
you will consider, you know, reducing the crazy noise
in your mind and maybe check out what meditation
can do for you and kind of follow up
on some of those studies I, I mentioned.
I will have, I'll Tweet a link later on
it. I was planning on posting it before this
talk but I forgot to do it. I have
to get back to my computer. But I will,
I will Tweet a link later. My, my Twitter
handle is @nicksiegr, N-I-C-K-S-I-E-G-R, and I'll Tweet a link
later with some of the, the articles and links
to thinks that I, that I look, that I
refer to during the course of my talk. So,
meditate. Be happy. Make the world better. Thanks.
RODRIGO FRANCO: Can I go now?
??: Now's your time, Rodrigo. So, Rodrigo is a
developer on our merchant team. And he's an avid
game, both board and video, right?
R.F.: Yes.
??: And you're from Brazil, right?
R.F.: Yes.
??: Definitely south of the Equator.
R.F.: Yes.
??: And, if you see him in the hall,
ask him about his goat.
R.F.: They're gonna, they're gonna see them.
??: All right.
R.F.: But yeah.
??: Take it away.
R.F.: Thank you. Is this one? You're strong. All
right. So, hello everyone. My name is Rodrigo. And
I'm here to give you one trick to boost
your productivity when working from home. My Twitter handle
is @caffo. My wife call me that. So you
can call me that, too. Or Rodrigo. Whatever you
want.
I wrote an article for, or tech blog, about
how I do work from home, and how I
try to focus on things while doing that. The
most important thing that people talk about was the
pet goats, and that they were in the article,
like, briefly. I'm gonna show you some.
Here's me with my pet goat in a hangout
for work. If you want to read the article,
it's available here. It's a sign in link. And
I have been working remotely from home for about
ten years. So, I had to find my way
to focus on stuff. Otherwise I'd just play video
games all the time. And, that's why we have
this unrelated cat picture.
And, this article name is One Weird Trick, because
while I was browsing the internet without ad block,
I saw a lot of these, these like ads,
like. One tip for a flat belly. And, like
DHH said, like, there's no way we can have
like a quick flat belly. So, what, what would
be, like, this one trick that we can get
to have focus? And I think it's boundaries.
Like, I had a couple things I do that
limit my work day and my personal day, because
at home they are very together. And I'm just
gonna briefly show you some of these things, like
everyday I start my day with a startup ritual.
I get presentable, because we do hangouts and I
need to be properly dressed. Not undressed. Whatever. And
then I do what I call a, I open
the cafeteria. I get, like, some fizzy water done.
I put coffee in the press and I grind
the beans. And then I have a very good
cup of coffee.
The other thing that I think it's important, after
all of that done, I get my cup of
coffee. I sit down. I start working. And I
think you really, you really need to eliminate distractions.
You need to find a way to be in
your mind office while in your home. And what
I found out was very good for me was
having a good pair of head phones. The head
phones are nice because they allow you to focus
on stuff, and like people around you, like, my
wife knows when I'm with the headphones, she can't
talk to me. The dog don't care about, a
lot about it, and pet goats just get sad
because he can't get my attention.
And I also have all sorts of gizmos. Like,
I plug the headphones there, so I can have
even more like better sound. And that way my
wife just, like, needs to throw stuff at me.
Like, wake up, get out of here. I need
something. And I call that my focus flag. It's
like a way I can show to the world
and to myself that it's time to work. I'm
not here. I may be here, in my body,
but my mind's doing other things for my company.
So, OK. You got there. You had your startup
ritual. You got your computer. Your headset. Goats away.
So, you can really start working. And then you're
done, because you do a lot of multitasking. And
you can't do that. Like, why we spend all
the time in Hacker News, on Reddit. Because it's
way more fun than do work. Like, you have
that huge list of things to do, but you
want to be on Reddit.
So we need to cut down multitasking. You need
to find ways to not change what you're doing.
So if you're coding, you're coding. You don't have
like, if gmail is loaded, you should not open
a new browser tab. Oh, but I'm just gonna
browse a second. Your day is gone.
So, what I done, for example, I had these,
these tiny Arduino plugged into my computer, so if
someone sends me an IM message and they are
in a white list, I would get a light
that says you need to answer them because it's
your boss or someone from your team. If it's
not, like, you can answer whenever you have time.
So you need to find ways to not multitask.
Another important thing is, don't work all the time.
When I started working remotely, I said like, yeah.
I need to make sure I don't play video
games all the time. And I started working all
the time. And I would never stop. So, you
need to find a good balance between one thing
and the other. You should also spend time with
others. Like, don't spend all the time at the
computer, like just working mindless. Like, find time to
spend time with other people. And also find time
to learn new stuff, like challenge yourself on Code
Wars. Do some Khan Academy stuff. Like, you need
to have a good balance in your life to
make sure your work is, works well, and everything
goes fine. Yeah.
That's it. Thank you so much.