[Script Info]
Title: 
[Events]
Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
Dialogue: 0,0:00:16.60,0:00:17.92,Default,,0000,0000,0000,,TOM DALE: Hey, you guys ready?
Dialogue: 0,0:00:17.94,0:00:19.18,Default,,0000,0000,0000,,Thank you guys so much for coming.
Dialogue: 0,0:00:19.19,0:00:21.77,Default,,0000,0000,0000,,This is awesome. I was really,
Dialogue: 0,0:00:21.77,0:00:23.70,Default,,0000,0000,0000,,I, when they were putting together the schedule,
Dialogue: 0,0:00:23.70,0:00:25.35,Default,,0000,0000,0000,,I said, make sure that you put us down
Dialogue: 0,0:00:25.35,0:00:27.16,Default,,0000,0000,0000,,in the Caves of Moria. So thank you
Dialogue: 0,0:00:27.16,0:00:31.56,Default,,0000,0000,0000,,guys for coming down and making it.
Dialogue: 0,0:00:31.56,0:00:32.91,Default,,0000,0000,0000,,I'm Tom. This is Yehuda.
Dialogue: 0,0:00:32.91,0:00:35.27,Default,,0000,0000,0000,,YEHUDA KATZ: When people told me was signed\Nup
Dialogue: 0,0:00:35.27,0:00:40.52,Default,,0000,0000,0000,,to do a back-to-back talk, I don't know what
Dialogue: 0,0:00:40.52,0:00:40.58,Default,,0000,0000,0000,,I was thinking.
Dialogue: 0,0:00:40.58,0:00:40.82,Default,,0000,0000,0000,,T.D.: Yup. So. We want to talk to you
Dialogue: 0,0:00:40.82,0:00:43.81,Default,,0000,0000,0000,,today about, about Skylight. So, just a little\Nbit
Dialogue: 0,0:00:43.81,0:00:45.73,Default,,0000,0000,0000,,before we talk about that, I want to talk
Dialogue: 0,0:00:45.73,0:00:47.53,Default,,0000,0000,0000,,about us a little bit.
Dialogue: 0,0:00:47.53,0:00:50.42,Default,,0000,0000,0000,,So, in 2011 we started a company called Tilde.
Dialogue: 0,0:00:50.42,0:00:52.76,Default,,0000,0000,0000,,It's this shirt. It may have made me self-conscious,
Dialogue: 0,0:00:52.76,0:00:54.61,Default,,0000,0000,0000,,because this is actually a first edition and\Nit's
Dialogue: 0,0:00:54.61,0:00:57.55,Default,,0000,0000,0000,,printed off-center. Well, either I'm off-center\Nor the shirt's
Dialogue: 0,0:00:57.55,0:01:00.54,Default,,0000,0000,0000,,off-center. One of the two.
Dialogue: 0,0:01:00.54,0:01:02.57,Default,,0000,0000,0000,,So we started Tilde in 2011, and we had
Dialogue: 0,0:01:02.57,0:01:06.03,Default,,0000,0000,0000,,all just left a venture-backed company, and\Nthat was
Dialogue: 0,0:01:06.03,0:01:08.44,Default,,0000,0000,0000,,a pretty traumatic experience for us because\Nwe spent
Dialogue: 0,0:01:08.44,0:01:09.56,Default,,0000,0000,0000,,a lot of time building the company and then
Dialogue: 0,0:01:09.56,0:01:11.11,Default,,0000,0000,0000,,we ran out of money and sold to Facebook
Dialogue: 0,0:01:11.11,0:01:13.50,Default,,0000,0000,0000,,and we really didn't want to repeat that experience.
Dialogue: 0,0:01:13.50,0:01:16.42,Default,,0000,0000,0000,,So, we decided to start Tilde, and when we
Dialogue: 0,0:01:16.42,0:01:19.83,Default,,0000,0000,0000,,did it, we decided to be. DHH and the
Dialogue: 0,0:01:19.83,0:01:21.97,Default,,0000,0000,0000,,other people at Basecamp were talking about,\Nyou know,
Dialogue: 0,0:01:21.97,0:01:23.63,Default,,0000,0000,0000,,being bootstrapped and proud. And that was\Na message
Dialogue: 0,0:01:23.63,0:01:26.25,Default,,0000,0000,0000,,that really resonated with us, and so we wanted
Dialogue: 0,0:01:26.25,0:01:28.36,Default,,0000,0000,0000,,to capture the same thing.
Dialogue: 0,0:01:28.36,0:01:30.25,Default,,0000,0000,0000,,There's only problem with being bootstrapped\Nand proud, and
Dialogue: 0,0:01:30.25,0:01:31.89,Default,,0000,0000,0000,,that is, in order to be both of those
Dialogue: 0,0:01:31.89,0:01:33.18,Default,,0000,0000,0000,,things, you actually need money, it turns\Nout. It's
Dialogue: 0,0:01:33.18,0:01:34.85,Default,,0000,0000,0000,,not like you just say it in a blog
Dialogue: 0,0:01:34.85,0:01:37.66,Default,,0000,0000,0000,,post and then all of the sudden you are
Dialogue: 0,0:01:37.66,0:01:38.56,Default,,0000,0000,0000,,in business.
Dialogue: 0,0:01:38.56,0:01:40.35,Default,,0000,0000,0000,,So, we had to think a lot about, OK,
Dialogue: 0,0:01:40.35,0:01:42.45,Default,,0000,0000,0000,,well, how do we make money? How do we
Dialogue: 0,0:01:42.45,0:01:44.69,Default,,0000,0000,0000,,make money? How do we make a profitable and,
Dialogue: 0,0:01:44.69,0:01:47.29,Default,,0000,0000,0000,,most importantly, sustainable business? Because\Nwe didn't want to
Dialogue: 0,0:01:47.29,0:01:49.38,Default,,0000,0000,0000,,just flip to Facebook in a couple years.
Dialogue: 0,0:01:49.38,0:01:53.13,Default,,0000,0000,0000,,So, looking around, I think the most obvious\Nthing
Dialogue: 0,0:01:53.13,0:01:55.11,Default,,0000,0000,0000,,that people suggested to us is, well, why\Ndon't
Dialogue: 0,0:01:55.11,0:01:58.05,Default,,0000,0000,0000,,you guys just become Ember, Inc.? Raise a\Nfew
Dialogue: 0,0:01:58.05,0:02:01.39,Default,,0000,0000,0000,,million dollars, you know, build a bunch of\Nbusiness
Dialogue: 0,0:02:01.39,0:02:05.73,Default,,0000,0000,0000,,model, mostly prayer. But that's not really\Nhow we
Dialogue: 0,0:02:05.73,0:02:08.96,Default,,0000,0000,0000,,want to think about building open source communities.
Dialogue: 0,0:02:08.96,0:02:10.86,Default,,0000,0000,0000,,We don't really think that that necessarily\Nleads to
Dialogue: 0,0:02:10.86,0:02:13.81,Default,,0000,0000,0000,,the best open source communities. And if you're\Ninterested
Dialogue: 0,0:02:13.81,0:02:16.57,Default,,0000,0000,0000,,more in that, I recommend Leia Silver, who\Nis
Dialogue: 0,0:02:16.57,0:02:19.70,Default,,0000,0000,0000,,one of our co-founders. She's giving a talk\Nthis
Dialogue: 0,0:02:19.70,0:02:22.45,Default,,0000,0000,0000,,afternoon. Oh, sorry. Friday afternoon, about\Nhow to build
Dialogue: 0,0:02:22.45,0:02:25.22,Default,,0000,0000,0000,,a company that is centered on open source.\NSo
Dialogue: 0,0:02:25.22,0:02:26.47,Default,,0000,0000,0000,,if you want to learn more about how we've
Dialogue: 0,0:02:26.47,0:02:28.99,Default,,0000,0000,0000,,done that, I would really suggest you go check
Dialogue: 0,0:02:28.99,0:02:30.06,Default,,0000,0000,0000,,out her talk.
Dialogue: 0,0:02:30.06,0:02:33.69,Default,,0000,0000,0000,,So, no. So, no Ember, Inc. Not allowed.
Dialogue: 0,0:02:33.69,0:02:38.16,Default,,0000,0000,0000,,So, we really want to build something that\Nleveraged
Dialogue: 0,0:02:38.16,0:02:40.25,Default,,0000,0000,0000,,the strengths that we thought that we had.\NOne,
Dialogue: 0,0:02:40.25,0:02:42.68,Default,,0000,0000,0000,,I think most importantly, a really deep knowledge\Nof
Dialogue: 0,0:02:42.68,0:02:44.57,Default,,0000,0000,0000,,open source and a deep knowledge of the Rails
Dialogue: 0,0:02:44.57,0:02:47.09,Default,,0000,0000,0000,,stack, and also Carl, it turns out, is really,
Dialogue: 0,0:02:47.09,0:02:50.69,Default,,0000,0000,0000,,really good at building highly scalable big\Ndata sys-
Dialogue: 0,0:02:50.69,0:02:53.99,Default,,0000,0000,0000,,big data systems. Lots of Hadoop in there.
Dialogue: 0,0:02:53.99,0:02:58.29,Default,,0000,0000,0000,,So, last year at RailsConf, we announced the\Nprivate
Dialogue: 0,0:02:58.29,0:03:00.52,Default,,0000,0000,0000,,beta of Skylight. How many of you have used
Dialogue: 0,0:03:00.52,0:03:01.71,Default,,0000,0000,0000,,Skylight? Can you raise your hand if you have
Dialogue: 0,0:03:01.71,0:03:04.63,Default,,0000,0000,0000,,used it? OK. Many of you. Awesome.
Dialogue: 0,0:03:04.63,0:03:08.13,Default,,0000,0000,0000,,So, so Skylight is a tool for profiling and
Dialogue: 0,0:03:08.13,0:03:11.78,Default,,0000,0000,0000,,measuring the performance of your Rails applications\Nin production.
Dialogue: 0,0:03:11.78,0:03:15.39,Default,,0000,0000,0000,,And, as a product, Skylight, I think, was\Nbuilt
Dialogue: 0,0:03:15.39,0:03:20.35,Default,,0000,0000,0000,,on three really, three key break-throughs.\NThere were key,
Dialogue: 0,0:03:20.35,0:03:22.12,Default,,0000,0000,0000,,three key break-throughs. We didn't want to\Nship a
Dialogue: 0,0:03:22.12,0:03:26.19,Default,,0000,0000,0000,,product that was incrementally better than\Nthe competition. We
Dialogue: 0,0:03:26.19,0:03:28.32,Default,,0000,0000,0000,,wanted to ship a product that was dramatically\Nbetter.
Dialogue: 0,0:03:28.32,0:03:30.08,Default,,0000,0000,0000,,Quantum leap. An order of magnitude better.
Dialogue: 0,0:03:30.08,0:03:32.08,Default,,0000,0000,0000,,And, in order to do that, we spent a
Dialogue: 0,0:03:32.08,0:03:33.89,Default,,0000,0000,0000,,lot of time thinking about it, about how we
Dialogue: 0,0:03:33.89,0:03:36.39,Default,,0000,0000,0000,,could solve most of the problems that we saw
Dialogue: 0,0:03:36.39,0:03:39.31,Default,,0000,0000,0000,,in the existing landscape. And so those, those\Nbreak-throughs
Dialogue: 0,0:03:39.31,0:03:42.30,Default,,0000,0000,0000,,are predicated- sorry, those, delivering a\Nproduct that does
Dialogue: 0,0:03:42.30,0:03:44.92,Default,,0000,0000,0000,,that is predicated on these three break-throughs.
Dialogue: 0,0:03:44.92,0:03:46.87,Default,,0000,0000,0000,,So, the first one I want to talk about
Dialogue: 0,0:03:46.87,0:03:53.87,Default,,0000,0000,0000,,is, honest response times. Honest response\Ntimes. So, DHH
Dialogue: 0,0:03:54.06,0:03:55.80,Default,,0000,0000,0000,,wrote a blog post on what was then the
Dialogue: 0,0:03:55.80,0:03:58.93,Default,,0000,0000,0000,,37Signals blog, now the Basecamp blog, called\NThe problem
Dialogue: 0,0:03:58.93,0:04:01.91,Default,,0000,0000,0000,,with averages. How many of you have read this?
Dialogue: 0,0:04:01.91,0:04:02.46,Default,,0000,0000,0000,,Awesome.
Dialogue: 0,0:04:02.46,0:04:03.78,Default,,0000,0000,0000,,For those of you that have not, how many
Dialogue: 0,0:04:03.78,0:04:08.47,Default,,0000,0000,0000,,of you hate raising your hands at presentations?
Dialogue: 0,0:04:08.47,0:04:10.51,Default,,0000,0000,0000,,So, for those of you that-
Dialogue: 0,0:04:10.51,0:04:11.55,Default,,0000,0000,0000,,Y.K.: Just put a button in every seat. Press
Dialogue: 0,0:04:11.55,0:04:11.78,Default,,0000,0000,0000,,this button-
Dialogue: 0,0:04:11.78,0:04:15.29,Default,,0000,0000,0000,,T.D.: Press the button if you have. Yes. Great.
Dialogue: 0,0:04:15.29,0:04:19.81,Default,,0000,0000,0000,,So, if you read this blog post, the way
Dialogue: 0,0:04:19.81,0:04:22.81,Default,,0000,0000,0000,,it opens is, Our average response time for\NBasecamp
Dialogue: 0,0:04:22.81,0:04:26.77,Default,,0000,0000,0000,,right now is 87ms... That sounds fantastic.\NAnd it
Dialogue: 0,0:04:26.77,0:04:29.95,Default,,0000,0000,0000,,easily leads you to believe that all is well
Dialogue: 0,0:04:29.95,0:04:31.68,Default,,0000,0000,0000,,and that we wouldn't need to spend any more
Dialogue: 0,0:04:31.68,0:04:34.15,Default,,0000,0000,0000,,time optimizing performance.
Dialogue: 0,0:04:34.15,0:04:38.84,Default,,0000,0000,0000,,But that's actually wrong. The average number\Nis completely
Dialogue: 0,0:04:38.84,0:04:42.25,Default,,0000,0000,0000,,skewed by tons of fast responses to feed requests
Dialogue: 0,0:04:42.25,0:04:46.17,Default,,0000,0000,0000,,and other cached replies. If you have 1000\Nrequests
Dialogue: 0,0:04:46.17,0:04:49.23,Default,,0000,0000,0000,,that return in 5ms, and then you can have
Dialogue: 0,0:04:49.23,0:04:53.56,Default,,0000,0000,0000,,200 requests taking 2000ms, or two seconds,\Nyou can
Dialogue: 0,0:04:53.56,0:04:57.51,Default,,0000,0000,0000,,still report an av- a respectable 170ms of\Naverage.
Dialogue: 0,0:04:57.51,0:04:59.82,Default,,0000,0000,0000,,That's useless.
Dialogue: 0,0:04:59.82,0:05:02.52,Default,,0000,0000,0000,,So what does DHH say that we need? DHH
Dialogue: 0,0:05:02.52,0:05:06.57,Default,,0000,0000,0000,,says the solution is histograms. So, for those\Nof
Dialogue: 0,0:05:06.57,0:05:09.01,Default,,0000,0000,0000,,you like me who were sleeping through your\Nstatistics
Dialogue: 0,0:05:09.01,0:05:12.41,Default,,0000,0000,0000,,class in high school, and college, a brief\Nprimer
Dialogue: 0,0:05:12.41,0:05:15.31,Default,,0000,0000,0000,,on histograms. So a histogram is very simple.\NBasically,
Dialogue: 0,0:05:15.31,0:05:17.70,Default,,0000,0000,0000,,you have a, you have a series of numbers
Dialogue: 0,0:05:17.70,0:05:22.39,Default,,0000,0000,0000,,along some axis, and every time you, you're\Nin
Dialogue: 0,0:05:22.39,0:05:24.36,Default,,0000,0000,0000,,that number, you're in that bucket, you basically\Nincrement
Dialogue: 0,0:05:24.36,0:05:25.28,Default,,0000,0000,0000,,that bar by one.
Dialogue: 0,0:05:25.28,0:05:27.67,Default,,0000,0000,0000,,So, this is an example of a histogram of
Dialogue: 0,0:05:27.67,0:05:30.62,Default,,0000,0000,0000,,response times in a Rails application. So\Nyou can
Dialogue: 0,0:05:30.62,0:05:31.98,Default,,0000,0000,0000,,see that there's a big cluster in the middle
Dialogue: 0,0:05:31.98,0:05:35.90,Default,,0000,0000,0000,,around 488ms, 500ms. This isn't a super speedy\Napp
Dialogue: 0,0:05:35.90,0:05:38.74,Default,,0000,0000,0000,,but it's not the worst thing in the world.
Dialogue: 0,0:05:38.74,0:05:39.52,Default,,0000,0000,0000,,And they're all clustered, and then as you\Nkind
Dialogue: 0,0:05:39.52,0:05:40.81,Default,,0000,0000,0000,,of move to the right you can see that
Dialogue: 0,0:05:40.81,0:05:42.17,Default,,0000,0000,0000,,the respond times get longer and longer and\Nlonger,
Dialogue: 0,0:05:42.17,0:05:43.99,Default,,0000,0000,0000,,and as you move to the left, response times
Dialogue: 0,0:05:43.99,0:05:45.72,Default,,0000,0000,0000,,get shorter and shorter and shorter.
Dialogue: 0,0:05:45.72,0:05:47.50,Default,,0000,0000,0000,,So, why do you want a histogram? What's the,
Dialogue: 0,0:05:47.50,0:05:48.60,Default,,0000,0000,0000,,what's the most important thing about a histogram?
Dialogue: 0,0:05:48.60,0:05:52.55,Default,,0000,0000,0000,,Y.K.: Well, I think it's because most requests\Ndon't
Dialogue: 0,0:05:52.55,0:05:53.23,Default,,0000,0000,0000,,actually look like this.
Dialogue: 0,0:05:53.23,0:05:53.51,Default,,0000,0000,0000,,T.D.: Yes.
Dialogue: 0,0:05:53.51,0:05:54.76,Default,,0000,0000,0000,,Y.K.: Most end points don't actually look\Nlike this.
Dialogue: 0,0:05:54.76,0:05:56.42,Default,,0000,0000,0000,,T.D.: Right. If you think about what your\NRails
Dialogue: 0,0:05:56.42,0:05:58.61,Default,,0000,0000,0000,,app is doing, it's a complicated beast, right.\NTurns
Dialogue: 0,0:05:58.61,0:06:02.33,Default,,0000,0000,0000,,out, Ruby frankly, you can, you can do branching
Dialogue: 0,0:06:02.33,0:06:04.36,Default,,0000,0000,0000,,logic. You can do a lot of things.
Dialogue: 0,0:06:04.36,0:06:06.15,Default,,0000,0000,0000,,And so what that means is that one end
Dialogue: 0,0:06:06.15,0:06:09.46,Default,,0000,0000,0000,,point, if you represent that with a single\Nnumber,
Dialogue: 0,0:06:09.46,0:06:11.65,Default,,0000,0000,0000,,you are losing a lot of fidelity, to the
Dialogue: 0,0:06:11.65,0:06:15.19,Default,,0000,0000,0000,,point where it becomes, as DHH said, useless.\NSo,
Dialogue: 0,0:06:15.19,0:06:17.73,Default,,0000,0000,0000,,for example, in a histogram, you can easily\Nsee,
Dialogue: 0,0:06:17.73,0:06:19.81,Default,,0000,0000,0000,,oh, here's a group of requests and response\Ntimes
Dialogue: 0,0:06:19.81,0:06:22.38,Default,,0000,0000,0000,,where I'm hitting the cache, and here's another\Ngroup
Dialogue: 0,0:06:22.38,0:06:24.17,Default,,0000,0000,0000,,where I'm missing it. And you can see that
Dialogue: 0,0:06:24.17,0:06:27.89,Default,,0000,0000,0000,,that cluster is significantly slower than\Nthe faster cache-hitting
Dialogue: 0,0:06:27.89,0:06:28.44,Default,,0000,0000,0000,,cluster.
Dialogue: 0,0:06:28.44,0:06:30.85,Default,,0000,0000,0000,,And the other thing that you get when you
Dialogue: 0,0:06:30.85,0:06:32.80,Default,,0000,0000,0000,,have a, a distribution, when you keep the\Nwhole
Dialogue: 0,0:06:32.80,0:06:35.37,Default,,0000,0000,0000,,distribution in the histogram, is you can\Nlook at
Dialogue: 0,0:06:35.37,0:06:39.47,Default,,0000,0000,0000,,this number at the 95th percentile, right.\NSo the
Dialogue: 0,0:06:39.47,0:06:41.64,Default,,0000,0000,0000,,right, the way to think about the performance\Nof
Dialogue: 0,0:06:41.64,0:06:46.64,Default,,0000,0000,0000,,your web application is not the average, because\Nthe
Dialogue: 0,0:06:46.64,0:06:50.16,Default,,0000,0000,0000,,average doesn't really tell you anything.\NYou want to
Dialogue: 0,0:06:50.16,0:06:52.22,Default,,0000,0000,0000,,think about the 95th percentile, because that's\Nnot the
Dialogue: 0,0:06:52.22,0:06:55.38,Default,,0000,0000,0000,,average response time, that's the average\Nworst response time
Dialogue: 0,0:06:55.38,0:06:57.99,Default,,0000,0000,0000,,that a user is likely to hit.
Dialogue: 0,0:06:57.99,0:06:59.36,Default,,0000,0000,0000,,And the thing to keep in mind is that
Dialogue: 0,0:06:59.36,0:07:01.85,Default,,0000,0000,0000,,it's not as though a customer comes to your
Dialogue: 0,0:07:01.85,0:07:05.10,Default,,0000,0000,0000,,site, they issue one request, and then they're\Ndone,
Dialogue: 0,0:07:05.10,0:07:08.00,Default,,0000,0000,0000,,right. As someone is using your website, they're\Ngonna
Dialogue: 0,0:07:08.00,0:07:10.30,Default,,0000,0000,0000,,be generating a lot of requests. And you need
Dialogue: 0,0:07:10.30,0:07:15.02,Default,,0000,0000,0000,,to look at the 95th percentile, because otherwise\Nevery
Dialogue: 0,0:07:15.02,0:07:17.22,Default,,0000,0000,0000,,request is basically you rolling the dice\Nthat they're
Dialogue: 0,0:07:17.22,0:07:18.61,Default,,0000,0000,0000,,not gonna hit one of those two second, three
Dialogue: 0,0:07:18.61,0:07:21.40,Default,,0000,0000,0000,,second, four second responses, close the tab\Nand go
Dialogue: 0,0:07:21.40,0:07:23.92,Default,,0000,0000,0000,,to your competitor.
Dialogue: 0,0:07:23.92,0:07:25.34,Default,,0000,0000,0000,,So we look at this as, here's the crazy
Dialogue: 0,0:07:25.34,0:07:28.44,Default,,0000,0000,0000,,thing. Here's what I think is crazy. That\Nblog
Dialogue: 0,0:07:28.44,0:07:32.75,Default,,0000,0000,0000,,post that DHH wrote, it's from 2009. It's\Nbeen
Dialogue: 0,0:07:32.75,0:07:35.00,Default,,0000,0000,0000,,five years, and there's still no tool that\Ndoes
Dialogue: 0,0:07:35.00,0:07:36.96,Default,,0000,0000,0000,,what DHH was asking for. So, we, frankly,\Nwe
Dialogue: 0,0:07:36.96,0:07:39.06,Default,,0000,0000,0000,,smelled money. We were like, holy crap.
Dialogue: 0,0:07:39.06,0:07:41.17,Default,,0000,0000,0000,,Y.K.: Yeah, why isn't that slide green?
Dialogue: 0,0:07:41.17,0:07:43.23,Default,,0000,0000,0000,,T.D.: Yeah. It should be green and dollars.\NI
Dialogue: 0,0:07:43.23,0:07:45.27,Default,,0000,0000,0000,,think keynote has the dollars, the make it\Nrain
Dialogue: 0,0:07:45.27,0:07:50.24,Default,,0000,0000,0000,,effect I should have used. So we smelled blood
Dialogue: 0,0:07:50.24,0:07:53.09,Default,,0000,0000,0000,,in the water. We're like, this is awesome.\NThere's
Dialogue: 0,0:07:53.09,0:07:56.61,Default,,0000,0000,0000,,only one problem that we discovered, and that\Nis,
Dialogue: 0,0:07:56.61,0:07:58.33,Default,,0000,0000,0000,,it turns out that building this thing is actually
Dialogue: 0,0:07:58.33,0:08:01.19,Default,,0000,0000,0000,,really, really freaky hard. Really, really\Nhard.
Dialogue: 0,0:08:01.19,0:08:05.78,Default,,0000,0000,0000,,So, we announced the private beta at RailsConf\Nlast
Dialogue: 0,0:08:05.78,0:08:09.14,Default,,0000,0000,0000,,year. Before doing that, we spent a year of
Dialogue: 0,0:08:09.14,0:08:12.79,Default,,0000,0000,0000,,research spiking out prototypes, building\Nprototypes, building out the
Dialogue: 0,0:08:12.79,0:08:16.51,Default,,0000,0000,0000,,beta. We launched at RailsConf, and we realized,\Nwe
Dialogue: 0,0:08:16.51,0:08:18.52,Default,,0000,0000,0000,,made a lot of problems. We made a lot
Dialogue: 0,0:08:18.52,0:08:21.91,Default,,0000,0000,0000,,of errors when we were building this system.
Dialogue: 0,0:08:21.91,0:08:26.27,Default,,0000,0000,0000,,So then, after RailsConf last year, we basically\Ntook
Dialogue: 0,0:08:26.27,0:08:29.69,Default,,0000,0000,0000,,six months to completely rewrite the backend\Nfrom the
Dialogue: 0,0:08:29.69,0:08:32.45,Default,,0000,0000,0000,,ground up. And I think tying into your keynote,
Dialogue: 0,0:08:32.45,0:08:36.28,Default,,0000,0000,0000,,Yehuda, we, we were like, oh. We clearly have
Dialogue: 0,0:08:36.28,0:08:39.12,Default,,0000,0000,0000,,a bespoke problem. No one else is doing this.
Dialogue: 0,0:08:39.12,0:08:42.09,Default,,0000,0000,0000,,So we rewrote our own custom backend. And\Nthen
Dialogue: 0,0:08:42.09,0:08:43.73,Default,,0000,0000,0000,,we had all these problems, and we realized\Nthat
Dialogue: 0,0:08:43.73,0:08:45.39,Default,,0000,0000,0000,,they had actually already all been solved\Nby the
Dialogue: 0,0:08:45.39,0:08:47.77,Default,,0000,0000,0000,,open source community. And so we benefited\Ntremendously by
Dialogue: 0,0:08:47.77,0:08:48.43,Default,,0000,0000,0000,,having a shared solution.
Dialogue: 0,0:08:48.43,0:08:50.71,Default,,0000,0000,0000,,Y.K.: Yeah. So our first release of this was
Dialogue: 0,0:08:50.71,0:08:55.28,Default,,0000,0000,0000,,really very bespoke, and the current release\Nuses a
Dialogue: 0,0:08:55.28,0:08:59.54,Default,,0000,0000,0000,,tremendous amount of very off-the-shelf open\Nsource projects that
Dialogue: 0,0:08:59.54,0:09:04.55,Default,,0000,0000,0000,,just solved the particular problem very effectively,\Nvery well.
Dialogue: 0,0:09:04.55,0:09:05.78,Default,,0000,0000,0000,,None of which are as easy to use as
Dialogue: 0,0:09:05.78,0:09:09.03,Default,,0000,0000,0000,,Rails, but all of which solve really thorny\Nproblems
Dialogue: 0,0:09:09.03,0:09:10.23,Default,,0000,0000,0000,,very effectively.
Dialogue: 0,0:09:10.23,0:09:12.87,Default,,0000,0000,0000,,T.D.: So, so let's just talk, just for your
Dialogue: 0,0:09:12.87,0:09:15.95,Default,,0000,0000,0000,,own understanding, let's talk about how most\Nperformance monitoring
Dialogue: 0,0:09:15.95,0:09:17.67,Default,,0000,0000,0000,,tools work. So the way that most of these
Dialogue: 0,0:09:17.67,0:09:19.93,Default,,0000,0000,0000,,work is that you run your Rails app, and
Dialogue: 0,0:09:19.93,0:09:22.25,Default,,0000,0000,0000,,running inside of your Rails app is some gem,
Dialogue: 0,0:09:22.25,0:09:25.00,Default,,0000,0000,0000,,some agent that you install. And every time\Nthe
Dialogue: 0,0:09:25.00,0:09:28.56,Default,,0000,0000,0000,,Rails app handles a request, it generates\Nevents, and
Dialogue: 0,0:09:28.56,0:09:32.50,Default,,0000,0000,0000,,those events, which include information about\Nperformance data, those
Dialogue: 0,0:09:32.50,0:09:34.93,Default,,0000,0000,0000,,events are passed into the agent.
Dialogue: 0,0:09:34.93,0:09:37.63,Default,,0000,0000,0000,,And then the agent sends that data to some
Dialogue: 0,0:09:37.63,0:09:40.93,Default,,0000,0000,0000,,kind of centralized server. Now, it turns\Nout that
Dialogue: 0,0:09:40.93,0:09:44.14,Default,,0000,0000,0000,,doing a running average is actually really\Nsimple. Which
Dialogue: 0,0:09:44.14,0:09:46.55,Default,,0000,0000,0000,,is why everyone does it. Basically you can\Ndo
Dialogue: 0,0:09:46.55,0:09:48.05,Default,,0000,0000,0000,,it in a single SQL query, right. All you
Dialogue: 0,0:09:48.05,0:09:50.31,Default,,0000,0000,0000,,do is you have three columns in database.\NThe
Dialogue: 0,0:09:50.31,0:09:52.69,Default,,0000,0000,0000,,end point, the running average, and the number\Nof
Dialogue: 0,0:09:52.69,0:09:55.77,Default,,0000,0000,0000,,requests, and then so, you can, those are\Nthe
Dialogue: 0,0:09:55.77,0:09:57.17,Default,,0000,0000,0000,,two things that you need to keep a running
Dialogue: 0,0:09:57.17,0:09:57.31,Default,,0000,0000,0000,,average, right.
Dialogue: 0,0:09:57.31,0:09:58.75,Default,,0000,0000,0000,,So keeping a running average is actually really\Nsimple
Dialogue: 0,0:09:58.75,0:10:00.98,Default,,0000,0000,0000,,from a technical point of view.
Dialogue: 0,0:10:00.98,0:10:03.80,Default,,0000,0000,0000,,Y.K.: I don't think you could even JavaScript\Nthrough
Dialogue: 0,0:10:03.80,0:10:04.60,Default,,0000,0000,0000,,to the lack of integers.
Dialogue: 0,0:10:04.60,0:10:06.05,Default,,0000,0000,0000,,T.D.: Yes. You probably wouldn't want to do\Nany
Dialogue: 0,0:10:06.05,0:10:07.53,Default,,0000,0000,0000,,math in JavaScript, it turns out. So, so we
Dialogue: 0,0:10:07.53,0:10:10.10,Default,,0000,0000,0000,,took a little bit different approach. Yehuda,\Ndo you
Dialogue: 0,0:10:10.10,0:10:12.07,Default,,0000,0000,0000,,want to go over the next section?
Dialogue: 0,0:10:12.07,0:10:15.09,Default,,0000,0000,0000,,Y.K.: Yeah. Sure. So, when we first started,\Nright
Dialogue: 0,0:10:15.09,0:10:17.79,Default,,0000,0000,0000,,at the beginning, we basically did a similar\Nthing
Dialogue: 0,0:10:17.79,0:10:19.79,Default,,0000,0000,0000,,where we had a bunch - your app creates
Dialogue: 0,0:10:19.79,0:10:22.62,Default,,0000,0000,0000,,events. Most of those start off as being ActiveSupport::Notifications,
Dialogue: 0,0:10:22.62,0:10:25.98,Default,,0000,0000,0000,,although it turns out that there's very limited\Nuse
Dialogue: 0,0:10:25.98,0:10:28.49,Default,,0000,0000,0000,,of ActiveSupport::Notifications so we had\Nto do some normalization
Dialogue: 0,0:10:28.49,0:10:30.36,Default,,0000,0000,0000,,work to get them sane, which we're gonna be
Dialogue: 0,0:10:30.36,0:10:32.63,Default,,0000,0000,0000,,upstreaming back into, into Rails.
Dialogue: 0,0:10:32.63,0:10:35.32,Default,,0000,0000,0000,,But, one thing that's kind of unfortunate\Nabout having
Dialogue: 0,0:10:35.32,0:10:37.03,Default,,0000,0000,0000,,every single Rails app have an agent is that
Dialogue: 0,0:10:37.03,0:10:38.65,Default,,0000,0000,0000,,you end up having to do a lot of
Dialogue: 0,0:10:38.65,0:10:40.31,Default,,0000,0000,0000,,the same kind of work over and over again,
Dialogue: 0,0:10:40.31,0:10:42.18,Default,,0000,0000,0000,,and use up a lot of memory. So, for
Dialogue: 0,0:10:42.18,0:10:44.22,Default,,0000,0000,0000,,example, every one of these things is making\NHTTP
Dialogue: 0,0:10:44.22,0:10:46.38,Default,,0000,0000,0000,,requests. So now you have a queue of things
Dialogue: 0,0:10:46.38,0:10:48.81,Default,,0000,0000,0000,,that you're sending over HTTP in every single\None
Dialogue: 0,0:10:48.81,0:10:50.51,Default,,0000,0000,0000,,of your Rails processes. And, of course, you\Nprobably
Dialogue: 0,0:10:50.51,0:10:52.25,Default,,0000,0000,0000,,don't notice this. People are used to Rails\Ntaking
Dialogue: 0,0:10:52.25,0:10:54.09,Default,,0000,0000,0000,,up hundreds and hundreds of megabytes, so\Nyou probably
Dialogue: 0,0:10:54.09,0:10:55.79,Default,,0000,0000,0000,,don't notice if you install some agent and\Nit
Dialogue: 0,0:10:55.79,0:10:59.45,Default,,0000,0000,0000,,suddenly starts taking twenty, thirty, forty,\Nfifty more megabytes.
Dialogue: 0,0:10:59.45,0:11:01.51,Default,,0000,0000,0000,,But we really wanted to keep the actual memory
Dialogue: 0,0:11:01.51,0:11:04.65,Default,,0000,0000,0000,,per process down to a small amount. So one
Dialogue: 0,0:11:04.65,0:11:06.17,Default,,0000,0000,0000,,of the very first things that we did, we
Dialogue: 0,0:11:06.17,0:11:07.91,Default,,0000,0000,0000,,even did it before last year, is that we
Dialogue: 0,0:11:07.91,0:11:09.80,Default,,0000,0000,0000,,pulled out all that shared logic into a, a
Dialogue: 0,0:11:09.80,0:11:13.42,Default,,0000,0000,0000,,separate process called the coordinator. And\Nthe agent is
Dialogue: 0,0:11:13.42,0:11:16.94,Default,,0000,0000,0000,,basically responsible simply for collecting\Nthe, the trace, and
Dialogue: 0,0:11:16.94,0:11:18.90,Default,,0000,0000,0000,,it's not responsible for actually talking\Nto our server
Dialogue: 0,0:11:18.90,0:11:20.71,Default,,0000,0000,0000,,at all. And that means that the coordinator\Nonly
Dialogue: 0,0:11:20.71,0:11:22.72,Default,,0000,0000,0000,,has to do this queue, this keeping a st-
Dialogue: 0,0:11:22.72,0:11:25.55,Default,,0000,0000,0000,,a bunch of stuff of work in one place,
Dialogue: 0,0:11:25.55,0:11:28.50,Default,,0000,0000,0000,,and it doesn't end up using up as much
Dialogue: 0,0:11:28.50,0:11:28.84,Default,,0000,0000,0000,,memory.
Dialogue: 0,0:11:28.84,0:11:31.02,Default,,0000,0000,0000,,And I think this, this ended up being very
Dialogue: 0,0:11:31.02,0:11:31.94,Default,,0000,0000,0000,,effective for us.
Dialogue: 0,0:11:31.94,0:11:33.49,Default,,0000,0000,0000,,T.D.: And I think that low overhead also allows
Dialogue: 0,0:11:33.49,0:11:36.35,Default,,0000,0000,0000,,us to just collect more information, in general.
Dialogue: 0,0:11:36.35,0:11:37.06,Default,,0000,0000,0000,,Y.K.: Yeah.
Dialogue: 0,0:11:37.06,0:11:39.88,Default,,0000,0000,0000,,Now, after our first attempt, we started getting\Na
Dialogue: 0,0:11:39.88,0:11:42.08,Default,,0000,0000,0000,,bunch of customers that were telling us that\Neven
Dialogue: 0,0:11:42.08,0:11:43.92,Default,,0000,0000,0000,,the separate - so the separate coordinator,\Nstarted as
Dialogue: 0,0:11:43.92,0:11:45.26,Default,,0000,0000,0000,,a good thing and a bad thing. On the
Dialogue: 0,0:11:45.26,0:11:47.40,Default,,0000,0000,0000,,one hand, there's only one of them, so it
Dialogue: 0,0:11:47.40,0:11:49.53,Default,,0000,0000,0000,,uses up only one set of memory. On the
Dialogue: 0,0:11:49.53,0:11:51.22,Default,,0000,0000,0000,,other hand, it's really easy for someone to\Ngo
Dialogue: 0,0:11:51.22,0:11:52.84,Default,,0000,0000,0000,,in and PS that process and see how many
Dialogue: 0,0:11:52.84,0:11:54.26,Default,,0000,0000,0000,,megabytes of memory it's using.
Dialogue: 0,0:11:54.26,0:11:56.56,Default,,0000,0000,0000,,So, we got a lot of additional complaints\Nthat
Dialogue: 0,0:11:56.56,0:11:58.10,Default,,0000,0000,0000,,said oh, your process is using a lot of
Dialogue: 0,0:11:58.10,0:12:00.99,Default,,0000,0000,0000,,memory. And, I spent a few weeks, I, I
Dialogue: 0,0:12:00.99,0:12:03.16,Default,,0000,0000,0000,,know Ruby pretty well. I spent a couple of
Dialogue: 0,0:12:03.16,0:12:05.67,Default,,0000,0000,0000,,weeks. I actually wrote a gem called Allocation\NCounter
Dialogue: 0,0:12:05.67,0:12:07.55,Default,,0000,0000,0000,,that basically went in to try to pin point
Dialogue: 0,0:12:07.55,0:12:09.49,Default,,0000,0000,0000,,exactly where the allocations were hap- coming\Nfrom. But
Dialogue: 0,0:12:09.49,0:12:11.80,Default,,0000,0000,0000,,it turns out that it's actually really, really\Nhard
Dialogue: 0,0:12:11.80,0:12:14.02,Default,,0000,0000,0000,,to track down exactly where allocations are\Ncoming from
Dialogue: 0,0:12:14.02,0:12:15.17,Default,,0000,0000,0000,,in Ruby, because something as simple as using\Na
Dialogue: 0,0:12:15.17,0:12:18.63,Default,,0000,0000,0000,,regular expression in Ruby can allocate match\Nobjects that
Dialogue: 0,0:12:18.63,0:12:19.41,Default,,0000,0000,0000,,get put back on the stack.
Dialogue: 0,0:12:19.41,0:12:21.45,Default,,0000,0000,0000,,And so I was able to pair this down
Dialogue: 0,0:12:21.45,0:12:24.22,Default,,0000,0000,0000,,to some degree. But I really discovered quickly\Nthat,
Dialogue: 0,0:12:24.22,0:12:26.98,Default,,0000,0000,0000,,trying to keep a lid on the memory allocation
Dialogue: 0,0:12:26.98,0:12:29.94,Default,,0000,0000,0000,,by doing all the stuff in Ruby, is mostly
Dialogue: 0,0:12:29.94,0:12:31.58,Default,,0000,0000,0000,,fine. But for our specific use case where\Nwe
Dialogue: 0,0:12:31.58,0:12:33.10,Default,,0000,0000,0000,,really wanna, we wanna be telling you, you\Ncan
Dialogue: 0,0:12:33.10,0:12:34.86,Default,,0000,0000,0000,,run the agent on your process, on your box,
Dialogue: 0,0:12:34.86,0:12:36.87,Default,,0000,0000,0000,,and it's not gonna use a lot of memory.
Dialogue: 0,0:12:36.87,0:12:40.08,Default,,0000,0000,0000,,We really needed something more efficient,\Nand our first
Dialogue: 0,0:12:40.08,0:12:42.79,Default,,0000,0000,0000,,thought was, we'll use C++ or C. No problem.
Dialogue: 0,0:12:42.79,0:12:45.22,Default,,0000,0000,0000,,C is, is native. It's great. And Carl did
Dialogue: 0,0:12:45.22,0:12:48.12,Default,,0000,0000,0000,,the work. Carl is very smart. And then he
Dialogue: 0,0:12:48.12,0:12:49.51,Default,,0000,0000,0000,,said, Yehuda. It is now your turn. You need
Dialogue: 0,0:12:49.51,0:12:51.25,Default,,0000,0000,0000,,to start maintaining this. And I said, I don't
Dialogue: 0,0:12:51.25,0:12:53.62,Default,,0000,0000,0000,,trust myself to write C++ code that's running\Nin
Dialogue: 0,0:12:53.62,0:12:56.03,Default,,0000,0000,0000,,all of your guys's boxes, and not seg-fault.\NSo
Dialogue: 0,0:12:56.03,0:12:59.65,Default,,0000,0000,0000,,I don't think that, that doesn't work for\Nme.
Dialogue: 0,0:12:59.65,0:13:01.79,Default,,0000,0000,0000,,And so I, I noticed that rust was coming
Dialogue: 0,0:13:01.79,0:13:03.63,Default,,0000,0000,0000,,along, and what rust really gives you is it
Dialogue: 0,0:13:03.63,0:13:05.90,Default,,0000,0000,0000,,gives you the ability to write low-level code\Na
Dialogue: 0,0:13:05.90,0:13:08.53,Default,,0000,0000,0000,,la C or C++ with magma memory management,\Nthat
Dialogue: 0,0:13:08.53,0:13:11.79,Default,,0000,0000,0000,,keeps your memory allocation low and keeps\Nthings speedy.
Dialogue: 0,0:13:11.79,0:13:14.93,Default,,0000,0000,0000,,Low resource utilization. While also giving\Nyou compile time
Dialogue: 0,0:13:14.93,0:13:17.95,Default,,0000,0000,0000,,guarantees about not seg-faulting. So, again,\Nif your processes
Dialogue: 0,0:13:17.95,0:13:20.32,Default,,0000,0000,0000,,randomly started seg-faulting because you\Ninstalled the agent, I
Dialogue: 0,0:13:20.32,0:13:21.95,Default,,0000,0000,0000,,think you would stop being our customer very\Nquickly.
Dialogue: 0,0:13:21.95,0:13:24.68,Default,,0000,0000,0000,,So having what, pretty much 100% guarantees\Nabout that
Dialogue: 0,0:13:24.68,0:13:26.60,Default,,0000,0000,0000,,was very important to us. And so that's why
Dialogue: 0,0:13:26.60,0:13:28.42,Default,,0000,0000,0000,,we decided to use rust.
Dialogue: 0,0:13:28.42,0:13:29.88,Default,,0000,0000,0000,,I'll just keep going.
Dialogue: 0,0:13:29.88,0:13:30.97,Default,,0000,0000,0000,,T.D.: Keep going.
Dialogue: 0,0:13:30.97,0:13:32.95,Default,,0000,0000,0000,,Y.K.: So, we had this coordinator object.\NAnd basically
Dialogue: 0,0:13:32.95,0:13:36.15,Default,,0000,0000,0000,,the coordinator object is receiving events.\NSo the events
Dialogue: 0,0:13:36.15,0:13:39.75,Default,,0000,0000,0000,,basically end up being these traces that describe\Nwhat's
Dialogue: 0,0:13:39.75,0:13:42.05,Default,,0000,0000,0000,,happening in your application. And the next\Nthing, I
Dialogue: 0,0:13:42.05,0:13:44.42,Default,,0000,0000,0000,,think our initial work on this we used JSON
Dialogue: 0,0:13:44.42,0:13:46.16,Default,,0000,0000,0000,,just to send the pay load to the server,
Dialogue: 0,0:13:46.16,0:13:47.95,Default,,0000,0000,0000,,but we noticed that a lot of people have
Dialogue: 0,0:13:47.95,0:13:49.89,Default,,0000,0000,0000,,really big requests. So you may have a big
Dialogue: 0,0:13:49.89,0:13:51.52,Default,,0000,0000,0000,,request with a big SQL query in it, or
Dialogue: 0,0:13:51.52,0:13:53.32,Default,,0000,0000,0000,,a lot of big SQL queries in it. Some
Dialogue: 0,0:13:53.32,0:13:55.28,Default,,0000,0000,0000,,people have traces that are hundreds and hundreds\Nof
Dialogue: 0,0:13:55.28,0:13:57.50,Default,,0000,0000,0000,,nodes long. And so we really wanted to figure
Dialogue: 0,0:13:57.50,0:14:00.10,Default,,0000,0000,0000,,out how to shrink down the payload size to
Dialogue: 0,0:14:00.10,0:14:02.57,Default,,0000,0000,0000,,something that we could be, you know, pumping\Nout
Dialogue: 0,0:14:02.57,0:14:04.76,Default,,0000,0000,0000,,of your box on a regular basis without tearing
Dialogue: 0,0:14:04.76,0:14:06.85,Default,,0000,0000,0000,,up your bandwidth costs.
Dialogue: 0,0:14:06.85,0:14:09.01,Default,,0000,0000,0000,,So, one of the first things that we did
Dialogue: 0,0:14:09.01,0:14:11.07,Default,,0000,0000,0000,,early on was we switched using protobuf as\Nthe
Dialogue: 0,0:14:11.07,0:14:13.55,Default,,0000,0000,0000,,transport mechanism, and that really shrunk,\Nshrunk down the
Dialogue: 0,0:14:13.55,0:14:17.25,Default,,0000,0000,0000,,payloads a lot. Our earlier prototypes for\Nactually collecting
Dialogue: 0,0:14:17.25,0:14:19.49,Default,,0000,0000,0000,,the data were written in Ruby, but I think
Dialogue: 0,0:14:19.49,0:14:21.37,Default,,0000,0000,0000,,Carl did, like, a weekend hack to just pour
Dialogue: 0,0:14:21.37,0:14:24.18,Default,,0000,0000,0000,,it over the Java and got, like, 200x performance.
Dialogue: 0,0:14:24.18,0:14:26.32,Default,,0000,0000,0000,,And you don't always get 200x performance,\Nif mostly
Dialogue: 0,0:14:26.32,0:14:27.97,Default,,0000,0000,0000,,what you're doing is database queries, you're\Nnot gonna
Dialogue: 0,0:14:27.97,0:14:29.14,Default,,0000,0000,0000,,get a huge performance swing.
Dialogue: 0,0:14:29.14,0:14:31.89,Default,,0000,0000,0000,,But mostly what we're doing is math. And algorithms
Dialogue: 0,0:14:31.89,0:14:34.21,Default,,0000,0000,0000,,and data structures. And for that, Ruby is,\Nit
Dialogue: 0,0:14:34.21,0:14:36.31,Default,,0000,0000,0000,,could, in theory, one day, have a good git
Dialogue: 0,0:14:36.31,0:14:38.57,Default,,0000,0000,0000,,or something, but today, writing that code\Nin Java
Dialogue: 0,0:14:38.57,0:14:40.95,Default,,0000,0000,0000,,didn't end up being significantly more code\Ncause it's
Dialogue: 0,0:14:40.95,0:14:42.54,Default,,0000,0000,0000,,just, you know, algorithms and data structures.
Dialogue: 0,0:14:42.54,0:14:44.42,Default,,0000,0000,0000,,T.D.: And I'll just note something about standardizing\Non
Dialogue: 0,0:14:44.42,0:14:46.78,Default,,0000,0000,0000,,protobufs in our, in our stack, is actually\Na
Dialogue: 0,0:14:46.78,0:14:52.17,Default,,0000,0000,0000,,huge win, because we, we realized, hey, browsers,\Nas
Dialogue: 0,0:14:52.17,0:14:53.35,Default,,0000,0000,0000,,it turns out are pretty powerful these days.\NThey've
Dialogue: 0,0:14:53.35,0:14:54.49,Default,,0000,0000,0000,,got, you know, they can allocate memory, they\Ncan
Dialogue: 0,0:14:54.49,0:14:56.07,Default,,0000,0000,0000,,do all these types of computation. So, and\Nprotobuff's
Dialogue: 0,0:14:56.07,0:14:59.41,Default,,0000,0000,0000,,libraries exist everywhere. So we save ourselves\Na lot
Dialogue: 0,0:14:59.41,0:15:01.59,Default,,0000,0000,0000,,of computation and a lot of time by just
Dialogue: 0,0:15:01.59,0:15:04.26,Default,,0000,0000,0000,,treating protobuff as the canonical serialization\Nform, and then
Dialogue: 0,0:15:04.26,0:15:06.19,Default,,0000,0000,0000,,you can move payloads around the entire stack\Nand
Dialogue: 0,0:15:06.19,0:15:08.56,Default,,0000,0000,0000,,everything speaks the same language, so you've\Nsaved the
Dialogue: 0,0:15:08.56,0:15:09.30,Default,,0000,0000,0000,,serialization and deserialization.
Dialogue: 0,0:15:09.30,0:15:11.99,Default,,0000,0000,0000,,Y.K.: And JavaScript is actually surprisingly\Neffective at des-
Dialogue: 0,0:15:11.99,0:15:13.87,Default,,0000,0000,0000,,at taking protobuffs and converting them to\Nthe format
Dialogue: 0,0:15:13.87,0:15:18.19,Default,,0000,0000,0000,,that we need efficiently. So, so we basically\Ntake
Dialogue: 0,0:15:18.19,0:15:21.03,Default,,0000,0000,0000,,this data. The Java collector is basically\Ncollecting all
Dialogue: 0,0:15:21.03,0:15:23.55,Default,,0000,0000,0000,,these protobuffs, and pretty much it just\Nturns around,
Dialogue: 0,0:15:23.55,0:15:24.94,Default,,0000,0000,0000,,and this is sort of where we got into
Dialogue: 0,0:15:24.94,0:15:28.15,Default,,0000,0000,0000,,bespoke territory before we started rolling\Nour own, but
Dialogue: 0,0:15:28.15,0:15:30.93,Default,,0000,0000,0000,,we realized that when you write a big, distributed,
Dialogue: 0,0:15:30.93,0:15:33.13,Default,,0000,0000,0000,,fault-tolerant system, there's a lot of problems\Nthat you
Dialogue: 0,0:15:33.13,0:15:35.32,Default,,0000,0000,0000,,really just want someone else to have thought\Nabout.
Dialogue: 0,0:15:35.32,0:15:37.75,Default,,0000,0000,0000,,So, what we do is we basically take these,
Dialogue: 0,0:15:37.75,0:15:39.60,Default,,0000,0000,0000,,take these payloads that are coming in. We\Nconvert
Dialogue: 0,0:15:39.60,0:15:41.71,Default,,0000,0000,0000,,them into batches and we send the batches\Ndown
Dialogue: 0,0:15:41.71,0:15:45.26,Default,,0000,0000,0000,,into the Kafka queue. And the, the next thing
Dialogue: 0,0:15:45.26,0:15:47.68,Default,,0000,0000,0000,,that happens, so the Kafka, sorry, Kafka's\Nbasically just
Dialogue: 0,0:15:47.68,0:15:49.91,Default,,0000,0000,0000,,a queue that allows you to throw things into,
Dialogue: 0,0:15:49.91,0:15:53.02,Default,,0000,0000,0000,,I guess, it might be considered similar to\Nlike,
Dialogue: 0,0:15:53.02,0:15:55.43,Default,,0000,0000,0000,,something lime AMQP. It has some nice fault-tolerance\Nproperties
Dialogue: 0,0:15:55.43,0:15:57.95,Default,,0000,0000,0000,,and integrates well with storm. But most important\Nit's
Dialogue: 0,0:15:57.95,0:15:59.67,Default,,0000,0000,0000,,just super, super high through-put.
Dialogue: 0,0:15:59.67,0:16:01.94,Default,,0000,0000,0000,,So basically didn't want to put any barrier\Nbetween
Dialogue: 0,0:16:01.94,0:16:03.56,Default,,0000,0000,0000,,you giving us the data and us getting it
Dialogue: 0,0:16:03.56,0:16:04.48,Default,,0000,0000,0000,,to disc as soon as possible.
Dialogue: 0,0:16:04.48,0:16:05.87,Default,,0000,0000,0000,,T.D.: Yeah. Which we'll, I think, talk about\Nin
Dialogue: 0,0:16:05.87,0:16:06.18,Default,,0000,0000,0000,,a bit.
Dialogue: 0,0:16:06.18,0:16:08.54,Default,,0000,0000,0000,,Y.K.: So we, so the basic Kafka takes the
Dialogue: 0,0:16:08.54,0:16:11.41,Default,,0000,0000,0000,,data and starts sending it into Storm. And\Nif
Dialogue: 0,0:16:11.41,0:16:13.01,Default,,0000,0000,0000,,you think about what has to happen in order
Dialogue: 0,0:16:13.01,0:16:15.09,Default,,0000,0000,0000,,to get some request. So, you have these requests.
Dialogue: 0,0:16:15.09,0:16:18.15,Default,,0000,0000,0000,,There's, you know, maybe traces that have\Na bunch
Dialogue: 0,0:16:18.15,0:16:19.67,Default,,0000,0000,0000,,of SQL queries, and our job is basically to
Dialogue: 0,0:16:19.67,0:16:21.46,Default,,0000,0000,0000,,take all those SQL queries and say, OK, I
Dialogue: 0,0:16:21.46,0:16:22.56,Default,,0000,0000,0000,,can see that in all of your requests, you
Dialogue: 0,0:16:22.56,0:16:24.04,Default,,0000,0000,0000,,had the SQL query and it took around this
Dialogue: 0,0:16:24.04,0:16:25.45,Default,,0000,0000,0000,,amount of time and it happened as a child
Dialogue: 0,0:16:25.45,0:16:27.47,Default,,0000,0000,0000,,of this other node. And the way to think
Dialogue: 0,0:16:27.47,0:16:29.74,Default,,0000,0000,0000,,about that is basically just a processing\Npipeline. Right.
Dialogue: 0,0:16:29.74,0:16:31.48,Default,,0000,0000,0000,,So you have these traces that come in one
Dialogue: 0,0:16:31.48,0:16:33.48,Default,,0000,0000,0000,,side. You start passing them through a bunch\Nof
Dialogue: 0,0:16:33.48,0:16:34.80,Default,,0000,0000,0000,,processing steps, and then you end up on the
Dialogue: 0,0:16:34.80,0:16:36.67,Default,,0000,0000,0000,,other side with the data.
Dialogue: 0,0:16:36.67,0:16:38.88,Default,,0000,0000,0000,,And Storm is actually a way of describing\Nthat
Dialogue: 0,0:16:38.88,0:16:41.93,Default,,0000,0000,0000,,processing pipeline in sort of functional\Nstyle, and then
Dialogue: 0,0:16:41.93,0:16:43.74,Default,,0000,0000,0000,,you tell it, OK. Here's how many servers I
Dialogue: 0,0:16:43.74,0:16:46.84,Default,,0000,0000,0000,,need. Here's how, here's how I'm gonna handle\Nfailures.
Dialogue: 0,0:16:46.84,0:16:50.00,Default,,0000,0000,0000,,And it basically deals with distribution and\Nscaling and
Dialogue: 0,0:16:50.00,0:16:52.38,Default,,0000,0000,0000,,all that stuff for you. And part of that
Dialogue: 0,0:16:52.38,0:16:55.38,Default,,0000,0000,0000,,is because you wrote everything using functional\Nstyle.
Dialogue: 0,0:16:55.38,0:16:57.11,Default,,0000,0000,0000,,And so what happens is Kafka sends the data
Dialogue: 0,0:16:57.11,0:17:00.55,Default,,0000,0000,0000,,into the entry spout, which is sort of terminology
Dialogue: 0,0:17:00.55,0:17:04.04,Default,,0000,0000,0000,,in, terminology in Storm for these streams\Nthat get
Dialogue: 0,0:17:04.04,0:17:06.93,Default,,0000,0000,0000,,created. And they basically go into these\Nprocessing things,
Dialogue: 0,0:17:06.93,0:17:09.84,Default,,0000,0000,0000,,which very clever- cutely are called bolts.\NThis is
Dialogue: 0,0:17:09.84,0:17:12.98,Default,,0000,0000,0000,,definitely not the naming I would have used,\Nbut.
Dialogue: 0,0:17:12.98,0:17:15.13,Default,,0000,0000,0000,,So they're called bolts. And the idea is that
Dialogue: 0,0:17:15.13,0:17:16.97,Default,,0000,0000,0000,,basically every request may have several things.
Dialogue: 0,0:17:16.97,0:17:20.09,Default,,0000,0000,0000,,So, for example, we now automatically detect\Nn +
Dialogue: 0,0:17:20.09,0:17:22.22,Default,,0000,0000,0000,,1 queries and that's sort of a different kind
Dialogue: 0,0:17:22.22,0:17:25.06,Default,,0000,0000,0000,,of processing from just, make a picture of\Nthe
Dialogue: 0,0:17:25.06,0:17:26.98,Default,,0000,0000,0000,,entire request. Or what is the 95th percentile\Nacross
Dialogue: 0,0:17:26.98,0:17:29.09,Default,,0000,0000,0000,,your entire app, right. These are all different\Nkinds
Dialogue: 0,0:17:29.09,0:17:30.94,Default,,0000,0000,0000,,of processing. So we take the data and we
Dialogue: 0,0:17:30.94,0:17:33.58,Default,,0000,0000,0000,,send them into a bunch of bolts, and the
Dialogue: 0,0:17:33.58,0:17:35.75,Default,,0000,0000,0000,,cool thing about bolts is that, again, because\Nthey're
Dialogue: 0,0:17:35.75,0:17:38.89,Default,,0000,0000,0000,,just functional chaining, you can take the\Noutput from
Dialogue: 0,0:17:38.89,0:17:41.13,Default,,0000,0000,0000,,one bolt and feed it into another bolt. And
Dialogue: 0,0:17:41.13,0:17:43.51,Default,,0000,0000,0000,,that works, that works pretty well. And, and\Nyou
Dialogue: 0,0:17:43.51,0:17:44.73,Default,,0000,0000,0000,,don't have to worry about - I mean, you
Dialogue: 0,0:17:44.73,0:17:46.60,Default,,0000,0000,0000,,have to worry a little bit about things like
Dialogue: 0,0:17:46.60,0:17:49.96,Default,,0000,0000,0000,,fault tolerance, failure, item potence. But\Nyou worry about
Dialogue: 0,0:17:49.96,0:17:52.23,Default,,0000,0000,0000,,them at, at the abstraction level, and then\Nthe
Dialogue: 0,0:17:52.23,0:17:54.16,Default,,0000,0000,0000,,operational part is handled for you.
Dialogue: 0,0:17:54.16,0:17:55.74,Default,,0000,0000,0000,,T.D.: So it's just like a very declarative\Nway
Dialogue: 0,0:17:55.74,0:17:58.18,Default,,0000,0000,0000,,of describing how this computation work in,\Nin a
Dialogue: 0,0:17:58.18,0:17:59.23,Default,,0000,0000,0000,,way that's easy to scale.
Dialogue: 0,0:17:59.23,0:18:01.86,Default,,0000,0000,0000,,Y.K.: And Carl actually talked about this\Nat very
Dialogue: 0,0:18:01.86,0:18:04.91,Default,,0000,0000,0000,,high speed yesterday, and you, some of you\Nmay
Dialogue: 0,0:18:04.91,0:18:06.62,Default,,0000,0000,0000,,have been there. I would recommend watching\Nthe video
Dialogue: 0,0:18:06.62,0:18:09.02,Default,,0000,0000,0000,,when it comes out if you want to make
Dialogue: 0,0:18:09.02,0:18:11.16,Default,,0000,0000,0000,,use of this stuff in your own applications.
Dialogue: 0,0:18:11.16,0:18:13.25,Default,,0000,0000,0000,,And then when you're finally done with all\Nthe
Dialogue: 0,0:18:13.25,0:18:14.97,Default,,0000,0000,0000,,processing, you need to actually do something\Nwith it.
Dialogue: 0,0:18:14.97,0:18:16.29,Default,,0000,0000,0000,,You need to put it somewhere so that the
Dialogue: 0,0:18:16.29,0:18:18.36,Default,,0000,0000,0000,,web app can get access to it, and that
Dialogue: 0,0:18:18.36,0:18:21.35,Default,,0000,0000,0000,,is basically, we use Cassandra for this. And\NCassandra
Dialogue: 0,0:18:21.35,0:18:24.93,Default,,0000,0000,0000,,again is mostly, it's a dumb database, but\Nit
Dialogue: 0,0:18:24.93,0:18:27.78,Default,,0000,0000,0000,,has, it's, has high capacity. It has some\Nof
Dialogue: 0,0:18:27.78,0:18:29.08,Default,,0000,0000,0000,,the fault-tolerance capacities that we want.
Dialogue: 0,0:18:29.08,0:18:30.77,Default,,0000,0000,0000,,T.D.: We're very, we're just very, very heavy,\Nright.
Dialogue: 0,0:18:30.77,0:18:32.82,Default,,0000,0000,0000,,Like, we tend to be writing more than we're
Dialogue: 0,0:18:32.82,0:18:33.73,Default,,0000,0000,0000,,ever reading.
Dialogue: 0,0:18:33.73,0:18:36.27,Default,,0000,0000,0000,,Y.K.: Yup. And then when we're done, when\Nwe're
Dialogue: 0,0:18:36.27,0:18:38.78,Default,,0000,0000,0000,,done with a particular batch, Cassandra basically\Nkicks off
Dialogue: 0,0:18:38.78,0:18:40.72,Default,,0000,0000,0000,,the process over again. So we're basically\Ndoing these
Dialogue: 0,0:18:40.72,0:18:41.20,Default,,0000,0000,0000,,things as batches.
Dialogue: 0,0:18:41.20,0:18:42.92,Default,,0000,0000,0000,,T.D.: So these are, these are roll-ups, is\Nwhat's
Dialogue: 0,0:18:42.92,0:18:45.36,Default,,0000,0000,0000,,happening here. So basically every minute,\Nevery ten minutes,
Dialogue: 0,0:18:45.36,0:18:48.14,Default,,0000,0000,0000,,and then at every hour, we reprocess and we
Dialogue: 0,0:18:48.14,0:18:49.97,Default,,0000,0000,0000,,re-aggregate, so that when you query us we\Nknow
Dialogue: 0,0:18:49.97,0:18:51.01,Default,,0000,0000,0000,,exactly what to give you.
Dialogue: 0,0:18:51.01,0:18:52.83,Default,,0000,0000,0000,,Y.K.: Yup. So we sort of have this cycle
Dialogue: 0,0:18:52.83,0:18:55.05,Default,,0000,0000,0000,,where we start off, obviously, in the first\Nfive
Dialogue: 0,0:18:55.05,0:18:56.89,Default,,0000,0000,0000,,second, the first minute, you really want\Nhigh granularity.
Dialogue: 0,0:18:56.89,0:18:59.14,Default,,0000,0000,0000,,You want to see what's happening right now.\NBut,
Dialogue: 0,0:18:59.14,0:19:00.11,Default,,0000,0000,0000,,if you want to go back and look at
Dialogue: 0,0:19:00.11,0:19:02.46,Default,,0000,0000,0000,,data from three months ago, you probably care\Nabout
Dialogue: 0,0:19:02.46,0:19:04.83,Default,,0000,0000,0000,,it, like the day granularity or maybe the\Nhour
Dialogue: 0,0:19:04.83,0:19:07.49,Default,,0000,0000,0000,,granularity. So, we basically do these roll-ups\Nand we
Dialogue: 0,0:19:07.49,0:19:09.20,Default,,0000,0000,0000,,cycle through the process.
Dialogue: 0,0:19:09.20,0:19:11.88,Default,,0000,0000,0000,,T.D.: So this, it turns out, building the\Nsystem
Dialogue: 0,0:19:11.88,0:19:15.17,Default,,0000,0000,0000,,required an intense amount of work. Carl spent\Nprobably
Dialogue: 0,0:19:15.17,0:19:17.49,Default,,0000,0000,0000,,six months reading PHP thesises to find-
Dialogue: 0,0:19:17.49,0:19:18.14,Default,,0000,0000,0000,,Y.K.: Thesis.
Dialogue: 0,0:19:18.14,0:19:23.85,Default,,0000,0000,0000,,T.D.: Thesis. To find, to find data structures\Nand
Dialogue: 0,0:19:23.85,0:19:25.87,Default,,0000,0000,0000,,algorithms that we could use. Because this\Nis a
Dialogue: 0,0:19:25.87,0:19:28.34,Default,,0000,0000,0000,,huge amount of data. Like, I think even a
Dialogue: 0,0:19:28.34,0:19:31.05,Default,,0000,0000,0000,,few months after we were in private data,\Nprivate
Dialogue: 0,0:19:31.05,0:19:34.18,Default,,0000,0000,0000,,beta, we were already handling over a billion\Nrequests
Dialogue: 0,0:19:34.18,0:19:36.20,Default,,0000,0000,0000,,per month. And obviously there's no way that\Nwe-
Dialogue: 0,0:19:36.20,0:19:37.97,Default,,0000,0000,0000,,Y.K.: Basically the number of requests that\Nwe handle
Dialogue: 0,0:19:37.97,0:19:39.77,Default,,0000,0000,0000,,is the sum of all of the requests that
Dialogue: 0,0:19:39.77,0:19:40.01,Default,,0000,0000,0000,,you handle.
Dialogue: 0,0:19:40.01,0:19:40.11,Default,,0000,0000,0000,,T.D.: Right.
Dialogue: 0,0:19:40.11,0:19:41.20,Default,,0000,0000,0000,,Y.K.: And all of our customers handle.
Dialogue: 0,0:19:41.20,0:19:41.91,Default,,0000,0000,0000,,T.D.: Right. Right. So.
Dialogue: 0,0:19:41.91,0:19:43.12,Default,,0000,0000,0000,,Y.K.: So that's a lot of requests.
Dialogue: 0,0:19:43.12,0:19:45.76,Default,,0000,0000,0000,,T.D.: So obviously we can't provide a service,\Nat
Dialogue: 0,0:19:45.76,0:19:48.48,Default,,0000,0000,0000,,least one that's not, we can't provide an\Naffordable
Dialogue: 0,0:19:48.48,0:19:51.13,Default,,0000,0000,0000,,service, an accessible service, if we have\Nto store
Dialogue: 0,0:19:51.13,0:19:53.19,Default,,0000,0000,0000,,terabytes or exabytes of data just to tell\Nyou
Dialogue: 0,0:19:53.19,0:19:53.99,Default,,0000,0000,0000,,how your app is running.
Dialogue: 0,0:19:53.99,0:19:56.63,Default,,0000,0000,0000,,Y.K.: And I think, also a problem, it's problematic
Dialogue: 0,0:19:56.63,0:19:58.43,Default,,0000,0000,0000,,if you store all the data in a database
Dialogue: 0,0:19:58.43,0:19:59.76,Default,,0000,0000,0000,,and then every single time someone wants to\Nlearn
Dialogue: 0,0:19:59.76,0:20:01.59,Default,,0000,0000,0000,,something about that, you have to do a query.
Dialogue: 0,0:20:01.59,0:20:03.16,Default,,0000,0000,0000,,Those queries can take a very long time. They
Dialogue: 0,0:20:03.16,0:20:04.70,Default,,0000,0000,0000,,can take minutes. And I think we really wanted
Dialogue: 0,0:20:04.70,0:20:07.16,Default,,0000,0000,0000,,to have something that would be very, that\Nwould,
Dialogue: 0,0:20:07.16,0:20:09.58,Default,,0000,0000,0000,,where the feedback loop would be fast. So\Nwe
Dialogue: 0,0:20:09.58,0:20:11.86,Default,,0000,0000,0000,,wanted to find algorithms that let us handle\Nthe
Dialogue: 0,0:20:11.86,0:20:13.94,Default,,0000,0000,0000,,data at, at real time, and then provide it
Dialogue: 0,0:20:13.94,0:20:16.02,Default,,0000,0000,0000,,to you at real time instead of these, like,
Dialogue: 0,0:20:16.02,0:20:18.31,Default,,0000,0000,0000,,dump the data somewhere and then do these\Ncomplicated
Dialogue: 0,0:20:18.31,0:20:18.55,Default,,0000,0000,0000,,queries.
Dialogue: 0,0:20:18.55,0:20:20.82,Default,,0000,0000,0000,,T.D.: So, hold on. So this slide was not
Dialogue: 0,0:20:20.82,0:20:24.44,Default,,0000,0000,0000,,supposed to be here. It was supposed to be
Dialogue: 0,0:20:24.44,0:20:27.67,Default,,0000,0000,0000,,a Rails slide. So, whoa. I went too far.
Dialogue: 0,0:20:27.67,0:20:29.97,Default,,0000,0000,0000,,K. We'll watch that again. That's pretty cool.\NSo
Dialogue: 0,0:20:29.97,0:20:32.46,Default,,0000,0000,0000,,then the last thing I want to say is,
Dialogue: 0,0:20:32.46,0:20:34.33,Default,,0000,0000,0000,,perhaps your take away from looking at this\Narchitecture
Dialogue: 0,0:20:34.33,0:20:36.75,Default,,0000,0000,0000,,diagram is, oh my gosh, these Rails guys completely-
Dialogue: 0,0:20:36.75,0:20:38.31,Default,,0000,0000,0000,,Y.K.: They really jumped the shark.
Dialogue: 0,0:20:38.31,0:20:40.87,Default,,0000,0000,0000,,T.D.: They jumped the shark. They ditched\NRails. I
Dialogue: 0,0:20:40.87,0:20:42.33,Default,,0000,0000,0000,,saw, like, three Tweets yesterday - I wasn't\Nhere,
Dialogue: 0,0:20:42.33,0:20:43.45,Default,,0000,0000,0000,,I was in Portland yesterday, but I saw, like,
Dialogue: 0,0:20:43.45,0:20:44.34,Default,,0000,0000,0000,,three Tweets that were like, I'm at RailsConf\Nand
Dialogue: 0,0:20:44.34,0:20:48.81,Default,,0000,0000,0000,,I haven't seen a single talk about, like,\NRails.
Dialogue: 0,0:20:48.81,0:20:51.95,Default,,0000,0000,0000,,So that's true here, too. But, I want to
Dialogue: 0,0:20:51.95,0:20:54.94,Default,,0000,0000,0000,,assure you that we are only using this stack
Dialogue: 0,0:20:54.94,0:20:58.07,Default,,0000,0000,0000,,for the heavy computation. We started in Rails.\NWe
Dialogue: 0,0:20:58.07,0:21:00.73,Default,,0000,0000,0000,,started, we were like, hey, what do we need.
Dialogue: 0,0:21:00.73,0:21:01.93,Default,,0000,0000,0000,,Ah, well, people probably need to authenticate\Nand log
Dialogue: 0,0:21:01.93,0:21:05.09,Default,,0000,0000,0000,,in, and we probably need to do billing. And
Dialogue: 0,0:21:05.09,0:21:06.22,Default,,0000,0000,0000,,those are all things that Rails is really,\Nreally
Dialogue: 0,0:21:06.22,0:21:08.83,Default,,0000,0000,0000,,good at. So we started with Rails as, basically,
Dialogue: 0,0:21:08.83,0:21:11.11,Default,,0000,0000,0000,,the starting point, and then when we realized\Noh
Dialogue: 0,0:21:11.11,0:21:14.04,Default,,0000,0000,0000,,my gosh, computation is really slow. There's\Nno way
Dialogue: 0,0:21:14.04,0:21:15.24,Default,,0000,0000,0000,,we're gonna be able to offer this service.\NOK.
Dialogue: 0,0:21:15.24,0:21:16.06,Default,,0000,0000,0000,,Now let's think about how we can do all
Dialogue: 0,0:21:16.06,0:21:16.27,Default,,0000,0000,0000,,of that.
Dialogue: 0,0:21:16.27,0:21:18.51,Default,,0000,0000,0000,,Y.K.: And I think notably, a lot of people
Dialogue: 0,0:21:18.51,0:21:20.05,Default,,0000,0000,0000,,who look at Rails are like, there's a lot
Dialogue: 0,0:21:20.05,0:21:21.75,Default,,0000,0000,0000,,of companies that have built big stuff on\NRails,
Dialogue: 0,0:21:21.75,0:21:24.09,Default,,0000,0000,0000,,and their attitude is, like, oh, this legacy\Nterrible
Dialogue: 0,0:21:24.09,0:21:25.41,Default,,0000,0000,0000,,Rails app. I really wish we could get rid
Dialogue: 0,0:21:25.41,0:21:26.85,Default,,0000,0000,0000,,of it. If we could just write everything in
Dialogue: 0,0:21:26.85,0:21:30.76,Default,,0000,0000,0000,,Scala or Clojure or Go, everything would be\Namazing.
Dialogue: 0,0:21:30.76,0:21:31.50,Default,,0000,0000,0000,,That is definitely not our attitude. Our attitude\Nis
Dialogue: 0,0:21:31.50,0:21:34.32,Default,,0000,0000,0000,,that Rails is really amazing, at particular,\Nat the
Dialogue: 0,0:21:34.32,0:21:36.74,Default,,0000,0000,0000,,kinds of things that are really common across\Neveryone's
Dialogue: 0,0:21:36.74,0:21:39.90,Default,,0000,0000,0000,,web applications - authentication, billing,\Net cetera. And we
Dialogue: 0,0:21:39.90,0:21:41.43,Default,,0000,0000,0000,,really want to be using Rails for the parts
Dialogue: 0,0:21:41.43,0:21:43.36,Default,,0000,0000,0000,,of our app- even things like error-tracking,\Nwe do
Dialogue: 0,0:21:43.36,0:21:45.04,Default,,0000,0000,0000,,through the Rails app. We want to be using
Dialogue: 0,0:21:45.04,0:21:47.47,Default,,0000,0000,0000,,Rails because it's very productive at doing\Nthose things.
Dialogue: 0,0:21:47.47,0:21:48.79,Default,,0000,0000,0000,,It happens to be very slow with doing data
Dialogue: 0,0:21:48.79,0:21:50.33,Default,,0000,0000,0000,,crunching, so we're gonna use a different\Ntool for
Dialogue: 0,0:21:50.33,0:21:50.54,Default,,0000,0000,0000,,that.
Dialogue: 0,0:21:50.54,0:21:51.91,Default,,0000,0000,0000,,But I don't think you'll ever see me getting
Dialogue: 0,0:21:51.91,0:21:54.21,Default,,0000,0000,0000,,up and saying, ah, I really wish we had
Dialogue: 0,0:21:54.21,0:21:55.08,Default,,0000,0000,0000,,just started writing, you know, the Rails\Napp in
Dialogue: 0,0:21:55.08,0:21:55.16,Default,,0000,0000,0000,,rust.
Dialogue: 0,0:21:55.16,0:21:55.31,Default,,0000,0000,0000,,T.D.: Yeah.
Dialogue: 0,0:21:55.31,0:21:58.09,Default,,0000,0000,0000,,Y.K.: That would be terrible.
Dialogue: 0,0:21:58.09,0:22:02.43,Default,,0000,0000,0000,,T.D.: So that's number one, is, is, honest\Nresponse
Dialogue: 0,0:22:02.43,0:22:04.39,Default,,0000,0000,0000,,times, which we're, which it turns out, seems\Nlike
Dialogue: 0,0:22:04.39,0:22:08.29,Default,,0000,0000,0000,,it should be easy, requires storing insane\Namount of
Dialogue: 0,0:22:08.29,0:22:09.17,Default,,0000,0000,0000,,data.
Dialogue: 0,0:22:09.17,0:22:10.62,Default,,0000,0000,0000,,So the second thing that we realized when\Nwe
Dialogue: 0,0:22:10.62,0:22:12.06,Default,,0000,0000,0000,,were looking at a lot of these tools, is
Dialogue: 0,0:22:12.06,0:22:14.36,Default,,0000,0000,0000,,that most of them focus on data. They focus
Dialogue: 0,0:22:14.36,0:22:16.59,Default,,0000,0000,0000,,on giving you the raw data. But I'm not
Dialogue: 0,0:22:16.59,0:22:19.13,Default,,0000,0000,0000,,a machine. I'm not a computer. I don't enjoy
Dialogue: 0,0:22:19.13,0:22:21.32,Default,,0000,0000,0000,,sifting through data. That's what computers\Nare good for.
Dialogue: 0,0:22:21.32,0:22:23.29,Default,,0000,0000,0000,,I would rather be drinking a beer. It's really
Dialogue: 0,0:22:23.29,0:22:24.83,Default,,0000,0000,0000,,nice in Portland, this time of year.
Dialogue: 0,0:22:24.83,0:22:27.41,Default,,0000,0000,0000,,So, we wanted to think about, if you're trying
Dialogue: 0,0:22:27.41,0:22:31.18,Default,,0000,0000,0000,,to solve the performance problems in your\Napplication, what
Dialogue: 0,0:22:31.18,0:22:32.84,Default,,0000,0000,0000,,are the things that you would suss out with
Dialogue: 0,0:22:32.84,0:22:35.76,Default,,0000,0000,0000,,the existing tools after spending, like, four\Nhours depleting
Dialogue: 0,0:22:35.76,0:22:37.51,Default,,0000,0000,0000,,your ego to get there?
Dialogue: 0,0:22:37.51,0:22:38.93,Default,,0000,0000,0000,,Y.K.: And I think part of this is just
Dialogue: 0,0:22:38.93,0:22:42.26,Default,,0000,0000,0000,,people are actually very, people like to think\Nthat
Dialogue: 0,0:22:42.26,0:22:43.88,Default,,0000,0000,0000,,they're gonna use these tools, but when the\Ntools
Dialogue: 0,0:22:43.88,0:22:45.32,Default,,0000,0000,0000,,require you to dig through a lot of data,
Dialogue: 0,0:22:45.32,0:22:47.09,Default,,0000,0000,0000,,people just don't use them very much. So,\Nthe
Dialogue: 0,0:22:47.09,0:22:48.33,Default,,0000,0000,0000,,goal here was to build a tool that people
Dialogue: 0,0:22:48.33,0:22:50.81,Default,,0000,0000,0000,,actually use and actually like using, and\Nnot to
Dialogue: 0,0:22:50.81,0:22:54.87,Default,,0000,0000,0000,,build a tool that happens to provide a lot
Dialogue: 0,0:22:54.87,0:22:55.04,Default,,0000,0000,0000,,of data you can sift through.
Dialogue: 0,0:22:55.04,0:22:55.06,Default,,0000,0000,0000,,T.D.: Yes.
Dialogue: 0,0:22:55.06,0:22:55.93,Default,,0000,0000,0000,,So, probably the, one of the first things\Nthat
Dialogue: 0,0:22:55.93,0:22:58.53,Default,,0000,0000,0000,,we realized is that we don't want to provide.
Dialogue: 0,0:22:58.53,0:23:00.40,Default,,0000,0000,0000,,This is a trace of a request, you've probably
Dialogue: 0,0:23:00.40,0:23:04.07,Default,,0000,0000,0000,,seen similar UIs using other tools, using,\Nfor example,
Dialogue: 0,0:23:04.07,0:23:07.06,Default,,0000,0000,0000,,the inspector in, in like Chrome or Safari,\Nand
Dialogue: 0,0:23:07.06,0:23:08.70,Default,,0000,0000,0000,,this is just showing basically, it's basically\Na visual
Dialogue: 0,0:23:08.70,0:23:10.83,Default,,0000,0000,0000,,stack trace of where your application is spending\Nits
Dialogue: 0,0:23:10.83,0:23:11.60,Default,,0000,0000,0000,,time.
Dialogue: 0,0:23:11.60,0:23:13.95,Default,,0000,0000,0000,,But I think what was important for us is
Dialogue: 0,0:23:13.95,0:23:17.81,Default,,0000,0000,0000,,showing not just a single request, because\Nyour app
Dialogue: 0,0:23:17.81,0:23:20.57,Default,,0000,0000,0000,,handles, you know, hundreds of thousands of\Nrequests, or
Dialogue: 0,0:23:20.57,0:23:22.68,Default,,0000,0000,0000,,millions of requests. So looking at a single\Nrequest
Dialogue: 0,0:23:22.68,0:23:24.63,Default,,0000,0000,0000,,statistically is complete, it's just noise.
Dialogue: 0,0:23:24.63,0:23:26.50,Default,,0000,0000,0000,,Y.K.: And it's especially bad if it's the\Nworst
Dialogue: 0,0:23:26.50,0:23:28.66,Default,,0000,0000,0000,,request, because the worst request is, is\Nreally noise.
Dialogue: 0,0:23:28.66,0:23:30.85,Default,,0000,0000,0000,,It's like, a hiccup in the network, right.
Dialogue: 0,0:23:30.85,0:23:31.25,Default,,0000,0000,0000,,T.D.: It's the outlier. Yeah.
Dialogue: 0,0:23:31.25,0:23:32.15,Default,,0000,0000,0000,,Y.K.: It's literally the outlier.
Dialogue: 0,0:23:32.15,0:23:35.66,Default,,0000,0000,0000,,T.D.: It's literally the outlier. Yup. So,\Nwhat we
Dialogue: 0,0:23:35.66,0:23:38.70,Default,,0000,0000,0000,,present in Skylight is something a little\Nbit different,
Dialogue: 0,0:23:38.70,0:23:41.77,Default,,0000,0000,0000,,and it's something that we call the aggregate\Ntrace.
Dialogue: 0,0:23:41.77,0:23:46.26,Default,,0000,0000,0000,,So the aggregate trace is basically us taking\Nall
Dialogue: 0,0:23:46.26,0:23:49.56,Default,,0000,0000,0000,,of your requests, averaging them out where\Neach of
Dialogue: 0,0:23:49.56,0:23:51.75,Default,,0000,0000,0000,,these things spends their time, and then showing\Nyou
Dialogue: 0,0:23:51.75,0:23:54.90,Default,,0000,0000,0000,,that. So this is basically like, this is like,
Dialogue: 0,0:23:54.90,0:23:57.93,Default,,0000,0000,0000,,this is like the statue of David. It is
Dialogue: 0,0:23:57.93,0:24:00.88,Default,,0000,0000,0000,,the idealized form of the stack trace of how
Dialogue: 0,0:24:00.88,0:24:02.27,Default,,0000,0000,0000,,your application's behaving.
Dialogue: 0,0:24:02.27,0:24:05.33,Default,,0000,0000,0000,,But, of course, you have the same problem\Nas
Dialogue: 0,0:24:05.33,0:24:07.50,Default,,0000,0000,0000,,before, which is, if this is all that we
Dialogue: 0,0:24:07.50,0:24:10.58,Default,,0000,0000,0000,,were showing you, it would be obscuring a\Nlot
Dialogue: 0,0:24:10.58,0:24:12.87,Default,,0000,0000,0000,,of information. You want to actually be able\Nto
Dialogue: 0,0:24:12.87,0:24:13.99,Default,,0000,0000,0000,,tell the difference between, OK, what's my\Nstack trace
Dialogue: 0,0:24:13.99,0:24:16.42,Default,,0000,0000,0000,,look like for fast requests, and how does\Nthat
Dialogue: 0,0:24:16.42,0:24:18.54,Default,,0000,0000,0000,,differ from requests that are slower.
Dialogue: 0,0:24:18.54,0:24:20.86,Default,,0000,0000,0000,,So what we've got, I've got a little video
Dialogue: 0,0:24:20.86,0:24:22.32,Default,,0000,0000,0000,,here. You can see that when I move the
Dialogue: 0,0:24:22.32,0:24:26.49,Default,,0000,0000,0000,,slider, that this trace below it is actually\Nupdating
Dialogue: 0,0:24:26.49,0:24:29.13,Default,,0000,0000,0000,,in real time. As I move the slider around,
Dialogue: 0,0:24:29.13,0:24:31.77,Default,,0000,0000,0000,,you can see that the aggregate trace actually\Nupdates
Dialogue: 0,0:24:31.77,0:24:34.24,Default,,0000,0000,0000,,with it. And that's because we're collecting\Nall this
Dialogue: 0,0:24:34.24,0:24:36.16,Default,,0000,0000,0000,,information. We're collecting, like I said,\Na lot of
Dialogue: 0,0:24:36.16,0:24:38.67,Default,,0000,0000,0000,,data. We can recompute this aggregate trace\Non the
Dialogue: 0,0:24:38.67,0:24:38.91,Default,,0000,0000,0000,,fly.
Dialogue: 0,0:24:38.91,0:24:41.20,Default,,0000,0000,0000,,Basically, for each bucket, we're storing\Na different trace,
Dialogue: 0,0:24:41.20,0:24:42.88,Default,,0000,0000,0000,,and then on the client we're reassembling\Nthat. We'll
Dialogue: 0,0:24:42.88,0:24:43.90,Default,,0000,0000,0000,,go into that a little bit.
Dialogue: 0,0:24:43.90,0:24:45.80,Default,,0000,0000,0000,,Y.K.: And I think it's really important that\Nyou
Dialogue: 0,0:24:45.80,0:24:48.37,Default,,0000,0000,0000,,be able to do these experiments quickly. If\Nevery
Dialogue: 0,0:24:48.37,0:24:50.06,Default,,0000,0000,0000,,time you think, oh, I wonder what happens\Nif
Dialogue: 0,0:24:50.06,0:24:52.26,Default,,0000,0000,0000,,I add another histogram bucket, if it requires\Na
Dialogue: 0,0:24:52.26,0:24:54.83,Default,,0000,0000,0000,,whole full page refresh. Then that would basically\Nmake
Dialogue: 0,0:24:54.83,0:24:56.31,Default,,0000,0000,0000,,people not want to use the tool. Not able
Dialogue: 0,0:24:56.31,0:24:58.58,Default,,0000,0000,0000,,to use the tool. So, actually building something\Nwhich
Dialogue: 0,0:24:58.58,0:24:59.65,Default,,0000,0000,0000,,is real time and fast, gets the data as
Dialogue: 0,0:24:59.65,0:25:00.11,Default,,0000,0000,0000,,it comes, was really important to us.
Dialogue: 0,0:25:00.11,0:25:01.22,Default,,0000,0000,0000,,T.D.: So that's number one.
Dialogue: 0,0:25:01.22,0:25:04.85,Default,,0000,0000,0000,,And the second thing. So we built that, and
Dialogue: 0,0:25:04.85,0:25:07.93,Default,,0000,0000,0000,,we're like, OK, well what's next? And I think
Dialogue: 0,0:25:07.93,0:25:09.25,Default,,0000,0000,0000,,that the big problem with this is that you
Dialogue: 0,0:25:09.25,0:25:12.02,Default,,0000,0000,0000,,need to know that there's a problem before\Nyou
Dialogue: 0,0:25:12.02,0:25:14.43,Default,,0000,0000,0000,,go look at it, right. So we have been
Dialogue: 0,0:25:14.43,0:25:16.08,Default,,0000,0000,0000,,working for the past few months, and the Storm
Dialogue: 0,0:25:16.08,0:25:18.39,Default,,0000,0000,0000,,infrastructure that we built makes it pretty\Nstraight-forward to
Dialogue: 0,0:25:18.39,0:25:21.15,Default,,0000,0000,0000,,start building more abstractions on top of\Nthe data
Dialogue: 0,0:25:21.15,0:25:21.56,Default,,0000,0000,0000,,that we've already collected.
Dialogue: 0,0:25:21.56,0:25:24.12,Default,,0000,0000,0000,,It's a very declarative system. So we've been\Nworking
Dialogue: 0,0:25:24.12,0:25:26.68,Default,,0000,0000,0000,,on a feature called inspections. And what's\Ncool about
Dialogue: 0,0:25:26.68,0:25:29.28,Default,,0000,0000,0000,,inspections is that we can look at this tremendous
Dialogue: 0,0:25:29.28,0:25:31.27,Default,,0000,0000,0000,,volume of data that we've collected from your\Napp,
Dialogue: 0,0:25:31.27,0:25:33.84,Default,,0000,0000,0000,,and we can automatically tease out what the\Nproblems
Dialogue: 0,0:25:33.84,0:25:35.21,Default,,0000,0000,0000,,are. So the first one that we shipped, this
Dialogue: 0,0:25:35.21,0:25:37.40,Default,,0000,0000,0000,,is in beta right now. It's not, it's not
Dialogue: 0,0:25:37.40,0:25:39.84,Default,,0000,0000,0000,,out and enabled by default, but there, it's\Nbehind
Dialogue: 0,0:25:39.84,0:25:42.44,Default,,0000,0000,0000,,a feature flag that we've had some users turning
Dialogue: 0,0:25:42.44,0:25:42.73,Default,,0000,0000,0000,,on.
Dialogue: 0,0:25:42.73,0:25:44.42,Default,,0000,0000,0000,,And, and trying out. And so what we can
Dialogue: 0,0:25:44.42,0:25:46.45,Default,,0000,0000,0000,,do in this case, is because we have information
Dialogue: 0,0:25:46.45,0:25:48.73,Default,,0000,0000,0000,,about all of the database queries in your\Napp,
Dialogue: 0,0:25:48.73,0:25:50.84,Default,,0000,0000,0000,,we can look and see if you have n
Dialogue: 0,0:25:50.84,0:25:52.39,Default,,0000,0000,0000,,plus one queries. Can you maybe explain what\Nan
Dialogue: 0,0:25:52.39,0:25:53.25,Default,,0000,0000,0000,,n plus one query is?
Dialogue: 0,0:25:53.25,0:25:54.60,Default,,0000,0000,0000,,Y.K.: Yeah. So, I'm, people know, hopefully,\Nwhat n
Dialogue: 0,0:25:54.60,0:25:56.77,Default,,0000,0000,0000,,plus one queries. But the, it's the idea that
Dialogue: 0,0:25:56.77,0:25:59.26,Default,,0000,0000,0000,,you, by accident, for some reason, instead\Nof making
Dialogue: 0,0:25:59.26,0:26:01.97,Default,,0000,0000,0000,,one query, you ask for like all the posts
Dialogue: 0,0:26:01.97,0:26:02.94,Default,,0000,0000,0000,,and then you iterated through all of them\Nand
Dialogue: 0,0:26:02.94,0:26:04.94,Default,,0000,0000,0000,,got all the comments and now you, instead\Nof
Dialogue: 0,0:26:04.94,0:26:08.68,Default,,0000,0000,0000,,having one query, you have one query per post,
Dialogue: 0,0:26:08.68,0:26:10.31,Default,,0000,0000,0000,,right. And you, what I've, what I've like\Nto
Dialogue: 0,0:26:10.31,0:26:12.55,Default,,0000,0000,0000,,do is do eager reloading, where you say include
Dialogue: 0,0:26:12.55,0:26:14.56,Default,,0000,0000,0000,,comments, right. But you have to know that\Nyou
Dialogue: 0,0:26:14.56,0:26:15.04,Default,,0000,0000,0000,,have to do that.
Dialogue: 0,0:26:15.04,0:26:16.90,Default,,0000,0000,0000,,So there's some tools that will run in development
Dialogue: 0,0:26:16.90,0:26:18.38,Default,,0000,0000,0000,,mode, if you happen to catch it, like a
Dialogue: 0,0:26:18.38,0:26:20.46,Default,,0000,0000,0000,,bullet. This is basically a tool that's looking\Nat
Dialogue: 0,0:26:20.46,0:26:22.21,Default,,0000,0000,0000,,every single one of your classes and has some
Dialogue: 0,0:26:22.21,0:26:24.17,Default,,0000,0000,0000,,thresholds that, once we see that a bunch\Nof
Dialogue: 0,0:26:24.17,0:26:27.43,Default,,0000,0000,0000,,your requests have the same exact query, so\Nwe
Dialogue: 0,0:26:27.43,0:26:29.55,Default,,0000,0000,0000,,do some work to pull out binds. So if
Dialogue: 0,0:26:29.55,0:26:32.20,Default,,0000,0000,0000,,it's, like, where something equals one, we\Nwill automatically
Dialogue: 0,0:26:32.20,0:26:34.11,Default,,0000,0000,0000,,pull out the one and replace it with a
Dialogue: 0,0:26:34.11,0:26:34.74,Default,,0000,0000,0000,,question mark.
Dialogue: 0,0:26:34.74,0:26:36.23,Default,,0000,0000,0000,,And then we basically take all those queries,\Nif
Dialogue: 0,0:26:36.23,0:26:39.53,Default,,0000,0000,0000,,they're the exact same query repeated multiple\Ntimes, subject
Dialogue: 0,0:26:39.53,0:26:41.39,Default,,0000,0000,0000,,to some thresholds, we'll start showing you\Nhey, there's
Dialogue: 0,0:26:41.39,0:26:42.45,Default,,0000,0000,0000,,an n plus one query.
Dialogue: 0,0:26:42.45,0:26:43.80,Default,,0000,0000,0000,,And you can imagine this same sort of thing
Dialogue: 0,0:26:43.80,0:26:46.32,Default,,0000,0000,0000,,being done for things, like, are you missing\Nan
Dialogue: 0,0:26:46.32,0:26:49.69,Default,,0000,0000,0000,,index, right. Or, are you using the Ruby version
Dialogue: 0,0:26:49.69,0:26:50.95,Default,,0000,0000,0000,,of JSON when you should be using the native
Dialogue: 0,0:26:50.95,0:26:52.18,Default,,0000,0000,0000,,version of JSON. These are all things that\Nwe
Dialogue: 0,0:26:52.18,0:26:55.14,Default,,0000,0000,0000,,can start detecting just because we're consuming\Nan enormous
Dialogue: 0,0:26:55.14,0:26:57.51,Default,,0000,0000,0000,,amount of information, and we can start writing\Nsome
Dialogue: 0,0:26:57.51,0:26:59.33,Default,,0000,0000,0000,,heuristics for bubbling it up.
Dialogue: 0,0:26:59.33,0:27:02.33,Default,,0000,0000,0000,,So, third and final breakthrough, we realized\Nthat we
Dialogue: 0,0:27:02.33,0:27:05.29,Default,,0000,0000,0000,,really, really needed a lightning fast UI.\NSomething really
Dialogue: 0,0:27:05.29,0:27:08.28,Default,,0000,0000,0000,,responsive. So, in particular, the feedback\Nloop is critical,
Dialogue: 0,0:27:08.28,0:27:09.93,Default,,0000,0000,0000,,right. You can imagine, if the way that you
Dialogue: 0,0:27:09.93,0:27:12.28,Default,,0000,0000,0000,,dug into data was you clicked and you wait
Dialogue: 0,0:27:12.28,0:27:14.32,Default,,0000,0000,0000,,an hour, and then you get your results, no
Dialogue: 0,0:27:14.32,0:27:15.73,Default,,0000,0000,0000,,one would do it. No one would ever do
Dialogue: 0,0:27:15.73,0:27:15.89,Default,,0000,0000,0000,,it.
Dialogue: 0,0:27:15.89,0:27:19.09,Default,,0000,0000,0000,,And the existing tools are OK, but you click
Dialogue: 0,0:27:19.09,0:27:20.47,Default,,0000,0000,0000,,and you wait. You look at it and you're
Dialogue: 0,0:27:20.47,0:27:21.73,Default,,0000,0000,0000,,like, oh, I want a different view, so then
Dialogue: 0,0:27:21.73,0:27:23.24,Default,,0000,0000,0000,,you go edit your query and then you click
Dialogue: 0,0:27:23.24,0:27:25.36,Default,,0000,0000,0000,,and you wait and it's just not a pleasant
Dialogue: 0,0:27:25.36,0:27:26.60,Default,,0000,0000,0000,,experience.
Dialogue: 0,0:27:26.60,0:27:28.85,Default,,0000,0000,0000,,So, so we use Ember, the, the UI that
Dialogue: 0,0:27:28.85,0:27:31.25,Default,,0000,0000,0000,,you're using when you log into Skylight. Even\Nthough
Dialogue: 0,0:27:31.25,0:27:33.29,Default,,0000,0000,0000,,it feels just like a regular website, it doesn't
Dialogue: 0,0:27:33.29,0:27:35.94,Default,,0000,0000,0000,,feel like a native app, is powered, all of
Dialogue: 0,0:27:35.94,0:27:37.68,Default,,0000,0000,0000,,the routing, all of the rendering, all of\Nthe
Dialogue: 0,0:27:37.68,0:27:40.77,Default,,0000,0000,0000,,decision making, is happening in, as an Ember.js\Napp,
Dialogue: 0,0:27:40.77,0:27:43.05,Default,,0000,0000,0000,,and we pair that with D3. So all of
Dialogue: 0,0:27:43.05,0:27:44.83,Default,,0000,0000,0000,,the charts, the charts that you saw there\Nin
Dialogue: 0,0:27:44.83,0:27:48.04,Default,,0000,0000,0000,,the aggregate trace, that is all Ember components\Npowered
Dialogue: 0,0:27:48.04,0:27:48.97,Default,,0000,0000,0000,,by D3.
Dialogue: 0,0:27:48.97,0:27:52.86,Default,,0000,0000,0000,,So, this is actually significantly cleaned\Nup our client-side
Dialogue: 0,0:27:52.86,0:27:55.68,Default,,0000,0000,0000,,code. It makes re-usability really, really\Nawesome. So to
Dialogue: 0,0:27:55.68,0:27:57.04,Default,,0000,0000,0000,,give you an example, this is from our billing
Dialogue: 0,0:27:57.04,0:27:58.79,Default,,0000,0000,0000,,page that I, the designer came and they had,
Dialogue: 0,0:27:58.79,0:28:01.26,Default,,0000,0000,0000,,they had a component that was like, the gate
Dialogue: 0,0:28:01.26,0:28:01.81,Default,,0000,0000,0000,,component.
Dialogue: 0,0:28:01.81,0:28:02.92,Default,,0000,0000,0000,,And, the-
Dialogue: 0,0:28:02.92,0:28:05.90,Default,,0000,0000,0000,,T.D.: It seems really boring at first.
Dialogue: 0,0:28:05.90,0:28:06.80,Default,,0000,0000,0000,,Y.K.: It seemed really boring. But, this is\Nthe
Dialogue: 0,0:28:06.80,0:28:08.95,Default,,0000,0000,0000,,implementation, right. So you could copy and\Npaste this
Dialogue: 0,0:28:08.95,0:28:11.06,Default,,0000,0000,0000,,code over and over again, everywhere you go.\NJust
Dialogue: 0,0:28:11.06,0:28:12.75,Default,,0000,0000,0000,,remember to format it correctly. If you forget\Nto
Dialogue: 0,0:28:12.75,0:28:15.07,Default,,0000,0000,0000,,format it, it's not gonna look the same everywhere.
Dialogue: 0,0:28:15.07,0:28:17.46,Default,,0000,0000,0000,,But I was like, hey, we're using this all
Dialogue: 0,0:28:17.46,0:28:18.01,Default,,0000,0000,0000,,over the place. Why don't we bundle this up
Dialogue: 0,0:28:18.01,0:28:20.07,Default,,0000,0000,0000,,into a component? And so with Ember, it was
Dialogue: 0,0:28:20.07,0:28:22.23,Default,,0000,0000,0000,,super easy. We basically just said, OK, here's\Nnew
Dialogue: 0,0:28:22.23,0:28:24.59,Default,,0000,0000,0000,,calendar date component. It has a property\Non it
Dialogue: 0,0:28:24.59,0:28:26.46,Default,,0000,0000,0000,,called date. Just set that to any JavaScript\Ndata
Dialogue: 0,0:28:26.46,0:28:28.06,Default,,0000,0000,0000,,object. Just set, you don't have to remember\Nabout
Dialogue: 0,0:28:28.06,0:28:30.45,Default,,0000,0000,0000,,converting it or formatting it. Here's the\Ncomponent. Set
Dialogue: 0,0:28:30.45,0:28:31.84,Default,,0000,0000,0000,,the date and it will render the correct thing
Dialogue: 0,0:28:31.84,0:28:32.76,Default,,0000,0000,0000,,automatically.
Dialogue: 0,0:28:32.76,0:28:36.04,Default,,0000,0000,0000,,And, so the architecture of the Ember app\Nlooks
Dialogue: 0,0:28:36.04,0:28:37.64,Default,,0000,0000,0000,,a little bit, something like this, where you\Nhave
Dialogue: 0,0:28:37.64,0:28:39.92,Default,,0000,0000,0000,,many, many different components, most of them\Njust driven
Dialogue: 0,0:28:39.92,0:28:42.37,Default,,0000,0000,0000,,by D3, and then they're plugged into the model
Dialogue: 0,0:28:42.37,0:28:43.48,Default,,0000,0000,0000,,and the controller.
Dialogue: 0,0:28:43.48,0:28:44.91,Default,,0000,0000,0000,,And the Ember app will go fetch those models
Dialogue: 0,0:28:44.91,0:28:46.75,Default,,0000,0000,0000,,from the cloud, and the cloud from the Java
Dialogue: 0,0:28:46.75,0:28:50.19,Default,,0000,0000,0000,,app, which just queries Cassandra, and render\Nthem. And
Dialogue: 0,0:28:50.19,0:28:53.43,Default,,0000,0000,0000,,what's neat about this model is turning on\Nweb
Dialogue: 0,0:28:53.43,0:28:56.36,Default,,0000,0000,0000,,sockets is super easy, right. Because all\Nof these
Dialogue: 0,0:28:56.36,0:28:58.86,Default,,0000,0000,0000,,components are bound to a single place. So\Nwhen
Dialogue: 0,0:28:58.86,0:29:00.89,Default,,0000,0000,0000,,the web socket says, hey, we have updated\Ninformation
Dialogue: 0,0:29:00.89,0:29:02.63,Default,,0000,0000,0000,,for you to show, it just pushes it onto
Dialogue: 0,0:29:02.63,0:29:04.98,Default,,0000,0000,0000,,the model or onto the controller, and the\Nwhole
Dialogue: 0,0:29:04.98,0:29:06.16,Default,,0000,0000,0000,,UI updates automatically.
Dialogue: 0,0:29:06.16,0:29:06.89,Default,,0000,0000,0000,,It's like magic.
Dialogue: 0,0:29:06.89,0:29:07.23,Default,,0000,0000,0000,,And-
Dialogue: 0,0:29:07.23,0:29:08.25,Default,,0000,0000,0000,,Y.K.: Like magic.
Dialogue: 0,0:29:08.25,0:29:09.68,Default,,0000,0000,0000,,T.D.: It's like magic. And, and when debugging,\Nthis
Dialogue: 0,0:29:09.68,0:29:11.56,Default,,0000,0000,0000,,is especially awesome too, because, and I'll\Nmaybe show
Dialogue: 0,0:29:11.56,0:29:15.08,Default,,0000,0000,0000,,a demo of the Ember inspector. It's nice.
Dialogue: 0,0:29:15.08,0:29:17.83,Default,,0000,0000,0000,,So. Yeah. So, lightning fast UI. Reducing\Nthe feedback
Dialogue: 0,0:29:17.83,0:29:19.51,Default,,0000,0000,0000,,loop so that you can quickly play with your
Dialogue: 0,0:29:19.51,0:29:21.88,Default,,0000,0000,0000,,data, makes it go from a chore to something
Dialogue: 0,0:29:21.88,0:29:23.62,Default,,0000,0000,0000,,that actually feels kind of fun.
Dialogue: 0,0:29:23.62,0:29:27.04,Default,,0000,0000,0000,,So, these were the breakthroughs that we had\Nwhen
Dialogue: 0,0:29:27.04,0:29:28.44,Default,,0000,0000,0000,,we were building Skylight. The things that\Nmade us
Dialogue: 0,0:29:28.44,0:29:29.98,Default,,0000,0000,0000,,think, yes, this is actually a product that\Nwe
Dialogue: 0,0:29:29.98,0:29:31.94,Default,,0000,0000,0000,,think deserves to be on the market. So, one,
Dialogue: 0,0:29:31.94,0:29:33.86,Default,,0000,0000,0000,,honest response times. Collect data that no\None else
Dialogue: 0,0:29:33.86,0:29:36.55,Default,,0000,0000,0000,,can collect. Focus on answers instead of just\Ndumping
Dialogue: 0,0:29:36.55,0:29:38.29,Default,,0000,0000,0000,,data, and have a lightning fast UI to do
Dialogue: 0,0:29:38.29,0:29:38.41,Default,,0000,0000,0000,,it.
Dialogue: 0,0:29:38.41,0:29:40.10,Default,,0000,0000,0000,,So we like to think of Skylight as basically
Dialogue: 0,0:29:40.10,0:29:42.69,Default,,0000,0000,0000,,a smart profiler. It's a smart profiler that\Nruns
Dialogue: 0,0:29:42.69,0:29:44.35,Default,,0000,0000,0000,,in production. It's like the profiler that\Nyou run
Dialogue: 0,0:29:44.35,0:29:47.23,Default,,0000,0000,0000,,on your local development machine, but instead\Nof being
Dialogue: 0,0:29:47.23,0:29:49.18,Default,,0000,0000,0000,,on your local dev box which has nothing to
Dialogue: 0,0:29:49.18,0:29:51.61,Default,,0000,0000,0000,,do with the performance characteristics of\Nwhat your users
Dialogue: 0,0:29:51.61,0:29:53.45,Default,,0000,0000,0000,,are experience, we're actually running in\Nproduction.
Dialogue: 0,0:29:53.45,0:29:58.92,Default,,0000,0000,0000,,So, let me just give you guys a quick
Dialogue: 0,0:29:58.92,0:30:00.39,Default,,0000,0000,0000,,demo.
Dialogue: 0,0:30:00.39,0:30:03.12,Default,,0000,0000,0000,,So, this is what the Skylight, this is what
Dialogue: 0,0:30:03.12,0:30:07.61,Default,,0000,0000,0000,,Skylight looks like. What's under this? There\Nwe go.
Dialogue: 0,0:30:07.61,0:30:09.62,Default,,0000,0000,0000,,So, the first thing here is we've got the
Dialogue: 0,0:30:09.62,0:30:12.67,Default,,0000,0000,0000,,app dash board. So this, it's like our, 95th
Dialogue: 0,0:30:12.67,0:30:15.50,Default,,0000,0000,0000,,responsile- 95th percentile response time\Nhas peaked. Maybe you're
Dialogue: 0,0:30:15.50,0:30:17.97,Default,,0000,0000,0000,,all hammering it right now. That would be\Nnice.
Dialogue: 0,0:30:17.97,0:30:19.94,Default,,0000,0000,0000,,So, this is a graph of your response time
Dialogue: 0,0:30:19.94,0:30:22.01,Default,,0000,0000,0000,,over time, and then on the right, this is
Dialogue: 0,0:30:22.01,0:30:24.70,Default,,0000,0000,0000,,the graph of the RPMs, the requests per minute
Dialogue: 0,0:30:24.70,0:30:26.75,Default,,0000,0000,0000,,that your app is handling. So this is app-wide.
Dialogue: 0,0:30:26.75,0:30:29.44,Default,,0000,0000,0000,,And this is live. This updates every minute.
Dialogue: 0,0:30:29.44,0:30:31.04,Default,,0000,0000,0000,,Then down below, you have a list of the
Dialogue: 0,0:30:31.04,0:30:33.73,Default,,0000,0000,0000,,end points in your application. So you can\Nsee,
Dialogue: 0,0:30:33.73,0:30:35.70,Default,,0000,0000,0000,,actually, the top, the slowest ones for us\Nwere,
Dialogue: 0,0:30:35.70,0:30:37.79,Default,,0000,0000,0000,,we have an instrumentation API, and we've\Ngone and
Dialogue: 0,0:30:37.79,0:30:39.93,Default,,0000,0000,0000,,instrumented our background workers. So we\Ncan see them
Dialogue: 0,0:30:39.93,0:30:42.01,Default,,0000,0000,0000,,here, and their response time plays in. So\Nwe
Dialogue: 0,0:30:42.01,0:30:44.22,Default,,0000,0000,0000,,can see that we have this reporting worker\Nthat's
Dialogue: 0,0:30:44.22,0:30:46.90,Default,,0000,0000,0000,,taking 95th percentile, thirteen seconds.
Dialogue: 0,0:30:46.90,0:30:48.88,Default,,0000,0000,0000,,Y.K.: So all that time used to be inside
Dialogue: 0,0:30:48.88,0:30:51.50,Default,,0000,0000,0000,,of some request somewhere, and we discovered\Nthat there
Dialogue: 0,0:30:51.50,0:30:52.84,Default,,0000,0000,0000,,was a lot of time being spent in things
Dialogue: 0,0:30:52.84,0:30:54.68,Default,,0000,0000,0000,,that we could push to the background. We probably
Dialogue: 0,0:30:54.68,0:30:56.79,Default,,0000,0000,0000,,need to update the agony index so that it
Dialogue: 0,0:30:56.79,0:30:59.19,Default,,0000,0000,0000,,doesn't make workers very high, because spending\Nsome time
Dialogue: 0,0:30:59.19,0:31:02.12,Default,,0000,0000,0000,,in your workers is not that big of a
Dialogue: 0,0:31:02.12,0:31:02.13,Default,,0000,0000,0000,,deal.
Dialogue: 0,0:31:02.13,0:31:03.00,Default,,0000,0000,0000,,T.D.: So, so then, if we dive into one
Dialogue: 0,0:31:03.00,0:31:05.30,Default,,0000,0000,0000,,of these, you can see that for this request,
Dialogue: 0,0:31:05.30,0:31:07.00,Default,,0000,0000,0000,,we've got the time explorer up above, and\Nthat
Dialogue: 0,0:31:07.00,0:31:10.43,Default,,0000,0000,0000,,shows a graph of response time at, again,\N95th
Dialogue: 0,0:31:10.43,0:31:11.84,Default,,0000,0000,0000,,percentile, and you can, if you want to go
Dialogue: 0,0:31:11.84,0:31:13.55,Default,,0000,0000,0000,,back and look at historical data, you just\Ndrag
Dialogue: 0,0:31:13.55,0:31:15.25,Default,,0000,0000,0000,,it like this. And this has got a brush,
Dialogue: 0,0:31:15.25,0:31:16.98,Default,,0000,0000,0000,,so you can zoom in and out on different
Dialogue: 0,0:31:16.98,0:31:17.76,Default,,0000,0000,0000,,times.
Dialogue: 0,0:31:17.76,0:31:19.65,Default,,0000,0000,0000,,And every time you change the range, you can
Dialogue: 0,0:31:19.65,0:31:21.36,Default,,0000,0000,0000,,see that it's very responsive. It's never\Nwaiting for
Dialogue: 0,0:31:21.36,0:31:23.04,Default,,0000,0000,0000,,the server. But it is going back and fetching
Dialogue: 0,0:31:23.04,0:31:25.08,Default,,0000,0000,0000,,data from the server and then when the data
Dialogue: 0,0:31:25.08,0:31:29.21,Default,,0000,0000,0000,,comes back, you see the whole UI just updates.
Dialogue: 0,0:31:29.21,0:31:29.25,Default,,0000,0000,0000,,And we get that for free with Ember and
Dialogue: 0,0:31:29.25,0:31:31.19,Default,,0000,0000,0000,,And then down below, as we discussed, you\Nactually
Dialogue: 0,0:31:31.19,0:31:33.76,Default,,0000,0000,0000,,have a real histogram. And this histogram,\Nin this
Dialogue: 0,0:31:33.76,0:31:37.16,Default,,0000,0000,0000,,case, is showing. So this is for fifty-seven\Nrequests.
Dialogue: 0,0:31:37.16,0:31:39.02,Default,,0000,0000,0000,,And if we click and drag, we could just
Dialogue: 0,0:31:39.02,0:31:40.43,Default,,0000,0000,0000,,move this. And you can see that the aggregate
Dialogue: 0,0:31:40.43,0:31:43.36,Default,,0000,0000,0000,,trace below updates in response to us dragging\Nthis.
Dialogue: 0,0:31:43.36,0:31:44.92,Default,,0000,0000,0000,,And if we want to look at the fastest
Dialogue: 0,0:31:44.92,0:31:47.50,Default,,0000,0000,0000,,quartile, we just click faster and we'll just\Nchoose
Dialogue: 0,0:31:47.50,0:31:48.15,Default,,0000,0000,0000,,that range on the histogram.
Dialogue: 0,0:31:48.15,0:31:49.21,Default,,0000,0000,0000,,Y.K.: I think it's the fastest load.
Dialogue: 0,0:31:49.21,0:31:50.90,Default,,0000,0000,0000,,T.D.: The fastest load. And then if you click
Dialogue: 0,0:31:50.90,0:31:52.90,Default,,0000,0000,0000,,on slower, you can see the slower requests.\NSo
Dialogue: 0,0:31:52.90,0:31:54.67,Default,,0000,0000,0000,,this makes it really easy to compare and contrast.
Dialogue: 0,0:31:54.67,0:31:56.71,Default,,0000,0000,0000,,OK. Why are certain requests faster and why\Nare
Dialogue: 0,0:31:56.71,0:31:58.53,Default,,0000,0000,0000,,certain requests slow?
Dialogue: 0,0:31:58.53,0:32:00.78,Default,,0000,0000,0000,,You can see the blue, these blue areas. This
Dialogue: 0,0:32:00.78,0:32:03.56,Default,,0000,0000,0000,,is Ruby code. So, right now it's not super
Dialogue: 0,0:32:03.56,0:32:05.82,Default,,0000,0000,0000,,granular. It would be nice if you could actually
Dialogue: 0,0:32:05.82,0:32:07.94,Default,,0000,0000,0000,,know what was going on here. But, it'll at
Dialogue: 0,0:32:07.94,0:32:09.94,Default,,0000,0000,0000,,least tell you where in your controller action\Nthis
Dialogue: 0,0:32:09.94,0:32:12.69,Default,,0000,0000,0000,,is happening, and then you can actually see\Nwhich
Dialogue: 0,0:32:12.69,0:32:15.92,Default,,0000,0000,0000,,database queries are being executed, and what\Ntheir duration
Dialogue: 0,0:32:15.92,0:32:16.08,Default,,0000,0000,0000,,is.
Dialogue: 0,0:32:16.08,0:32:17.89,Default,,0000,0000,0000,,And you can see that we actually extract the
Dialogue: 0,0:32:17.89,0:32:20.42,Default,,0000,0000,0000,,SQL and we denormalize it so we, so you,
Dialogue: 0,0:32:20.42,0:32:22.16,Default,,0000,0000,0000,,or, we normalize it so you can see exactly
Dialogue: 0,0:32:22.16,0:32:24.02,Default,,0000,0000,0000,,what those requests are even if the values\Nare
Dialogue: 0,0:32:24.02,0:32:24.82,Default,,0000,0000,0000,,totally different between them.
Dialogue: 0,0:32:24.82,0:32:27.65,Default,,0000,0000,0000,,Y.K.: Yeah. So the real query, courtesy of\NRails,
Dialogue: 0,0:32:27.65,0:32:29.73,Default,,0000,0000,0000,,not yet supporting bind extraction is like,\Nwhere id
Dialogue: 0,0:32:29.73,0:32:32.17,Default,,0000,0000,0000,,equals one or, ten or whatever.
Dialogue: 0,0:32:32.17,0:32:33.66,Default,,0000,0000,0000,,T.D.: Yup. So that's pretty cool.
Dialogue: 0,0:32:33.66,0:32:37.43,Default,,0000,0000,0000,,Y.K.: So one, one other thing is, initially,\Nwe
Dialogue: 0,0:32:37.43,0:32:39.27,Default,,0000,0000,0000,,actually just showed the whole trace, but\Nwe discovered
Dialogue: 0,0:32:39.27,0:32:41.66,Default,,0000,0000,0000,,that, obviously when you show whole traces\Nyou have
Dialogue: 0,0:32:41.66,0:32:43.64,Default,,0000,0000,0000,,information that doesn't really matter that\Nmuch. So we
Dialogue: 0,0:32:43.64,0:32:47.34,Default,,0000,0000,0000,,started off by, we've recently basically started\Nto collapse
Dialogue: 0,0:32:47.34,0:32:48.85,Default,,0000,0000,0000,,things that don't matter so much so that you
Dialogue: 0,0:32:48.85,0:32:51.09,Default,,0000,0000,0000,,can basically expand or condense the trace.
Dialogue: 0,0:32:51.09,0:32:52.52,Default,,0000,0000,0000,,And we wanted to make it not, but you
Dialogue: 0,0:32:52.52,0:32:55.69,Default,,0000,0000,0000,,have to think about expanding or condensing\Nindividual areas,
Dialogue: 0,0:32:55.69,0:32:57.96,Default,,0000,0000,0000,,but just, you see what matters the most and
Dialogue: 0,0:32:57.96,0:32:59.10,Default,,0000,0000,0000,,then you can see trivial errors.
Dialogue: 0,0:32:59.10,0:33:02.18,Default,,0000,0000,0000,,T.D.: Yup. So, so that's the demo of Skylight.
Dialogue: 0,0:33:02.18,0:33:04.19,Default,,0000,0000,0000,,We'd really like it if you checked it out.
Dialogue: 0,0:33:04.19,0:33:05.90,Default,,0000,0000,0000,,There is one more thing I want to show
Dialogue: 0,0:33:05.90,0:33:07.72,Default,,0000,0000,0000,,you that is, like, really freaking cool. This\Nis
Dialogue: 0,0:33:07.72,0:33:10.53,Default,,0000,0000,0000,,coming out of Tilde labs. Carl was like, has
Dialogue: 0,0:33:10.53,0:33:13.73,Default,,0000,0000,0000,,been hacking, he's been up until past midnight,\Ngetting
Dialogue: 0,0:33:13.73,0:33:15.77,Default,,0000,0000,0000,,almost no sleep for the past month trying\Nto
Dialogue: 0,0:33:15.77,0:33:16.73,Default,,0000,0000,0000,,have this ready.
Dialogue: 0,0:33:16.73,0:33:19.09,Default,,0000,0000,0000,,I don't know how many of you know this,
Dialogue: 0,0:33:19.09,0:33:23.63,Default,,0000,0000,0000,,but Ruby 2 point 1 has a new, a,
Dialogue: 0,0:33:23.63,0:33:27.95,Default,,0000,0000,0000,,a stack sampling feature. So you can get really
Dialogue: 0,0:33:27.95,0:33:31.15,Default,,0000,0000,0000,,granular information about how your Ruby code\Nis performing.
Dialogue: 0,0:33:31.15,0:33:33.45,Default,,0000,0000,0000,,So I want to show you, I just mentioned
Dialogue: 0,0:33:33.45,0:33:34.57,Default,,0000,0000,0000,,how it would be nice if we could get
Dialogue: 0,0:33:34.57,0:33:36.83,Default,,0000,0000,0000,,more information out of what your Ruby code\Nis
Dialogue: 0,0:33:36.83,0:33:38.76,Default,,0000,0000,0000,,doing. And now we can do that.
Dialogue: 0,0:33:38.76,0:33:42.04,Default,,0000,0000,0000,,Basically, every few milliseconds, this code\Nthat Carl wrote
Dialogue: 0,0:33:42.04,0:33:44.40,Default,,0000,0000,0000,,is going into the, to the Ruby, into MRI,
Dialogue: 0,0:33:44.40,0:33:47.42,Default,,0000,0000,0000,,and it's taking a snap shot of the stack.
Dialogue: 0,0:33:47.42,0:33:50.77,Default,,0000,0000,0000,,And because this is built-in, it's very low-impact.\NIt's
Dialogue: 0,0:33:50.77,0:33:53.57,Default,,0000,0000,0000,,not allocating any new memory. It's very little\Nperformance
Dialogue: 0,0:33:53.57,0:33:55.77,Default,,0000,0000,0000,,hit. Basically you wouldn't even notice it.\NAnd so
Dialogue: 0,0:33:55.77,0:33:58.15,Default,,0000,0000,0000,,every few milliseconds it's sampling, and\Nwe take that
Dialogue: 0,0:33:58.15,0:34:00.26,Default,,0000,0000,0000,,information and we send it up to our servers.
Dialogue: 0,0:34:00.26,0:34:02.26,Default,,0000,0000,0000,,So it's almost like you're running Ruby profiler\Non
Dialogue: 0,0:34:02.26,0:34:05.22,Default,,0000,0000,0000,,your local dev box, where you get extremely\Ngranular
Dialogue: 0,0:34:05.22,0:34:07.16,Default,,0000,0000,0000,,information about where your code is spending\Nits time
Dialogue: 0,0:34:07.16,0:34:09.01,Default,,0000,0000,0000,,in Ruby, per method, per all of these things.
Dialogue: 0,0:34:09.01,0:34:11.91,Default,,0000,0000,0000,,But it's happening in production.
Dialogue: 0,0:34:11.91,0:34:16.41,Default,,0000,0000,0000,,So, this is, so this is a, we enabled
Dialogue: 0,0:34:16.41,0:34:18.40,Default,,0000,0000,0000,,it in staging. You can see that we've got
Dialogue: 0,0:34:18.40,0:34:19.60,Default,,0000,0000,0000,,some rendering bugs. It's still in beta.
Dialogue: 0,0:34:19.60,0:34:21.92,Default,,0000,0000,0000,,Y.K.: Yeah, and we haven't yet collapsed things\Nthat
Dialogue: 0,0:34:21.92,0:34:21.98,Default,,0000,0000,0000,,are not important-
Dialogue: 0,0:34:21.98,0:34:22.02,Default,,0000,0000,0000,,T.D.: Yes.
Dialogue: 0,0:34:22.02,0:34:23.27,Default,,0000,0000,0000,,Y.K.: -for this particular feature.
Dialogue: 0,0:34:23.27,0:34:24.17,Default,,0000,0000,0000,,T.D.: So we want to show, we want to
Dialogue: 0,0:34:24.17,0:34:27.61,Default,,0000,0000,0000,,hide things like, like framework code, obviously.\NBut this
Dialogue: 0,0:34:27.61,0:34:31.07,Default,,0000,0000,0000,,gives you an incredibly, incredibly granular\Nview of what
Dialogue: 0,0:34:31.07,0:34:35.66,Default,,0000,0000,0000,,your app is doing in production. And we think.
Dialogue: 0,0:34:35.66,0:34:39.23,Default,,0000,0000,0000,,This is a, an API that's built into, into
Dialogue: 0,0:34:39.23,0:34:43.16,Default,,0000,0000,0000,,Ruby 2.1.1. Because our agent is running so\Nlow-level,
Dialogue: 0,0:34:43.16,0:34:44.66,Default,,0000,0000,0000,,because we wrote it in Rust, we have the
Dialogue: 0,0:34:44.66,0:34:47.41,Default,,0000,0000,0000,,ability to do things like this, and Carl thinks
Dialogue: 0,0:34:47.41,0:34:48.37,Default,,0000,0000,0000,,that we may be able to actually back port
Dialogue: 0,0:34:48.37,0:34:48.48,Default,,0000,0000,0000,,this to older Rubies, too. So if you're not
Dialogue: 0,0:34:48.48,0:34:50.13,Default,,0000,0000,0000,,on Ruby 2.1, we think that we can actually
Dialogue: 0,0:34:50.13,0:34:52.79,Default,,0000,0000,0000,,bring this. But that's TPD.
Dialogue: 0,0:34:52.79,0:34:55.48,Default,,0000,0000,0000,,Y.K.: Yeah, I- so I think the cool thing
Dialogue: 0,0:34:55.48,0:34:57.94,Default,,0000,0000,0000,,about this, in general, is when you run a
Dialogue: 0,0:34:57.94,0:34:59.43,Default,,0000,0000,0000,,sampling- so this is a sampling profiler,\Nright, we
Dialogue: 0,0:34:59.43,0:35:01.26,Default,,0000,0000,0000,,don't want to be burning every single thing\Nthat
Dialogue: 0,0:35:01.26,0:35:03.79,Default,,0000,0000,0000,,you do in your program with tracing, right.\NThat
Dialogue: 0,0:35:03.79,0:35:05.38,Default,,0000,0000,0000,,would be very slow.
Dialogue: 0,0:35:05.38,0:35:06.92,Default,,0000,0000,0000,,So when you normally run a sampling profiler,\Nyou
Dialogue: 0,0:35:06.92,0:35:08.76,Default,,0000,0000,0000,,have to basically make a loop. You have to
Dialogue: 0,0:35:08.76,0:35:11.09,Default,,0000,0000,0000,,basically create a loop, run this code a million
Dialogue: 0,0:35:11.09,0:35:12.97,Default,,0000,0000,0000,,times and keep sampling it. Eventually we'll\Nget enough
Dialogue: 0,0:35:12.97,0:35:15.03,Default,,0000,0000,0000,,samples to get the information. But it turns\Nout
Dialogue: 0,0:35:15.03,0:35:17.28,Default,,0000,0000,0000,,that your production server is a loop. Your\Nproduction
Dialogue: 0,0:35:17.28,0:35:20.56,Default,,0000,0000,0000,,server is serving tons and tons of requests.\NSo,
Dialogue: 0,0:35:20.56,0:35:22.88,Default,,0000,0000,0000,,by simply tak- you know, taking a few microseconds
Dialogue: 0,0:35:22.88,0:35:25.58,Default,,0000,0000,0000,,out of every request and collecting a couple\Nof
Dialogue: 0,0:35:25.58,0:35:27.21,Default,,0000,0000,0000,,samples, over time we can actually get this\Nreally
Dialogue: 0,0:35:27.21,0:35:29.70,Default,,0000,0000,0000,,high fidelity picture with basically no cost.
Dialogue: 0,0:35:29.70,0:35:31.15,Default,,0000,0000,0000,,And that's pretty mind-blowing. And this is\Nthe kind
Dialogue: 0,0:35:31.15,0:35:34.65,Default,,0000,0000,0000,,of stuff that we can start doing by really
Dialogue: 0,0:35:34.65,0:35:37.25,Default,,0000,0000,0000,,caring about, about both the user experience\Nand the
Dialogue: 0,0:35:37.25,0:35:40.83,Default,,0000,0000,0000,,implementation and getting really scary about\Nit. And I'm
Dialogue: 0,0:35:40.83,0:35:42.70,Default,,0000,0000,0000,,really, like, honestly this is a really exciting\Nfeature
Dialogue: 0,0:35:42.70,0:35:45.33,Default,,0000,0000,0000,,that really shows what we can do as we
Dialogue: 0,0:35:45.33,0:35:46.13,Default,,0000,0000,0000,,start building this out.
Dialogue: 0,0:35:46.13,0:35:47.14,Default,,0000,0000,0000,,T.D.: Once we've got that, once we've got\Nthat
Dialogue: 0,0:35:47.14,0:35:48.38,Default,,0000,0000,0000,,groundwork.
Dialogue: 0,0:35:48.38,0:35:49.82,Default,,0000,0000,0000,,So if you guys want to check it out,
Dialogue: 0,0:35:49.82,0:35:51.76,Default,,0000,0000,0000,,Skylight dot io, it's available today. It's\Nno longer
Dialogue: 0,0:35:51.76,0:35:54.04,Default,,0000,0000,0000,,in private beta. Everyone can sign up. No\Ninvitation
Dialogue: 0,0:35:54.04,0:35:56.63,Default,,0000,0000,0000,,token necessary. And you can get a thirty-day\Nfree
Dialogue: 0,0:35:56.63,0:35:58.24,Default,,0000,0000,0000,,trial if you haven't started one already.\NSo if
Dialogue: 0,0:35:58.24,0:35:59.62,Default,,0000,0000,0000,,you have any questions, please come see us\Nright
Dialogue: 0,0:35:59.62,0:36:00.98,Default,,0000,0000,0000,,now, or we have a booth in the vendor
Dialogue: 0,0:36:00.98,0:36:03.14,Default,,0000,0000,0000,,hall. Thank you guys very much.