[Script Info]
Title: 
[Events]
Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
Dialogue: 0,0:00:16.83,0:00:17.59,Default,,0000,0000,0000,,TOBY HEDE: Good morning everybody.
Dialogue: 0,0:00:17.59,0:00:24.37,Default,,0000,0000,0000,,Friday. Yes. It's been a long week. I'm excited.
Dialogue: 0,0:00:24.37,0:00:29.28,Default,,0000,0000,0000,,I'm highly caffeinated. So without further\Nado,
Dialogue: 0,0:00:29.28,0:00:34.18,Default,,0000,0000,0000,,I present An Ode to 17 Databases in 33 Minutes.
Dialogue: 0,0:00:34.18,0:00:37.87,Default,,0000,0000,0000,,I'm gonna mangle a large number of metaphors.
Dialogue: 0,0:00:37.87,0:00:40.82,Default,,0000,0000,0000,,There'll be a lot of animated gifs.
Dialogue: 0,0:00:40.82,0:00:44.21,Default,,0000,0000,0000,,I've learned that this week, if you see it\Nlike that,
Dialogue: 0,0:00:44.21,0:00:47.91,Default,,0000,0000,0000,,there's Star Wars, Dungeons and Dragons,
Dialogue: 0,0:00:47.91,0:00:49.35,Default,,0000,0000,0000,,and all of that's very, unfortunately, stereotypical.
Dialogue: 0,0:00:49.35,0:00:51.90,Default,,0000,0000,0000,,So a bit of an indictment.
Dialogue: 0,0:00:51.90,0:00:55.84,Default,,0000,0000,0000,,This whole thing started as a joke. Seventeen\Ndatabases.
Dialogue: 0,0:00:55.84,0:00:59.16,Default,,0000,0000,0000,,I actually did in five minutes. Thirty-three\Nminutes is
Dialogue: 0,0:00:59.16,0:01:03.80,Default,,0000,0000,0000,,worse. The whole thing is just a catastrophe,\Nreally.
Dialogue: 0,0:01:03.80,0:01:04.66,Default,,0000,0000,0000,,But anyway.
Dialogue: 0,0:01:04.66,0:01:07.61,Default,,0000,0000,0000,,We're gonna cover a whole bunch of different\Ndatabases
Dialogue: 0,0:01:07.61,0:01:09.89,Default,,0000,0000,0000,,and a little bit of the underlying theory,\Nand
Dialogue: 0,0:01:09.89,0:01:12.67,Default,,0000,0000,0000,,hopefully you'll walk out and you'll understand\Nwhy to
Dialogue: 0,0:01:12.67,0:01:13.73,Default,,0000,0000,0000,,use PostGres.
Dialogue: 0,0:01:13.73,0:01:14.25,Default,,0000,0000,0000,,[laughter]
Dialogue: 0,0:01:14.25,0:01:20.18,Default,,0000,0000,0000,,I'm Toby. You can find me on the internet.
Dialogue: 0,0:01:20.18,0:01:22.26,Default,,0000,0000,0000,,I work at a company called Nine Fold.
Dialogue: 0,0:01:22.26,0:01:26.10,Default,,0000,0000,0000,,V.O.: We're having a problem, there's no screen.
Dialogue: 0,0:01:26.10,0:01:33.10,Default,,0000,0000,0000,,T.H.: Oh. No screens. Is that me?
Dialogue: 0,0:01:35.96,0:01:41.21,Default,,0000,0000,0000,,Before it was, there was no red. So, now
Dialogue: 0,0:01:41.21,0:01:43.86,Default,,0000,0000,0000,,there's no any, anything.
Dialogue: 0,0:01:43.86,0:01:45.18,Default,,0000,0000,0000,,V.O.: Nothing.
Dialogue: 0,0:01:45.18,0:01:46.50,Default,,0000,0000,0000,,T.H.: Hey.
Dialogue: 0,0:01:46.50,0:01:47.82,Default,,0000,0000,0000,,AUDIENCE: Hey!
Dialogue: 0,0:01:47.82,0:01:51.12,Default,,0000,0000,0000,,T.H.: I have no slides.
Dialogue: 0,0:01:51.12,0:01:54.67,Default,,0000,0000,0000,,Well, you missed my beautiful slides. There's.\NYou missed
Dialogue: 0,0:01:54.67,0:01:57.74,Default,,0000,0000,0000,,the first animation. That's a shame. You missed\Nthe
Dialogue: 0,0:01:57.74,0:02:01.50,Default,,0000,0000,0000,,list. It's awesome. You missed me and my excellent
Dialogue: 0,0:02:01.50,0:02:05.07,Default,,0000,0000,0000,,job titles. So yes.
Dialogue: 0,0:02:05.07,0:02:08.34,Default,,0000,0000,0000,,I work at Nine Fold. They have very kindly
Dialogue: 0,0:02:08.34,0:02:12.55,Default,,0000,0000,0000,,flown me over here from Australia, which explains\Nwhy
Dialogue: 0,0:02:12.55,0:02:16.69,Default,,0000,0000,0000,,I sound like I come from the deep south.
Dialogue: 0,0:02:16.69,0:02:18.06,Default,,0000,0000,0000,,Cause I do.
Dialogue: 0,0:02:18.06,0:02:21.26,Default,,0000,0000,0000,,Most of this week, this has been me. So
Dialogue: 0,0:02:21.26,0:02:23.56,Default,,0000,0000,0000,,today I'm finally over the jetlag just in\Ntime
Dialogue: 0,0:02:23.56,0:02:26.67,Default,,0000,0000,0000,,to go home and have it all over again
Dialogue: 0,0:02:26.67,0:02:27.85,Default,,0000,0000,0000,,next week.
Dialogue: 0,0:02:27.85,0:02:32.45,Default,,0000,0000,0000,,So, a couple of quick facts about Straya.\NThere
Dialogue: 0,0:02:32.45,0:02:39.12,Default,,0000,0000,0000,,are much fewer syllables than you're used\Nto using.
Dialogue: 0,0:02:39.12,0:02:43.95,Default,,0000,0000,0000,,This is an, a genuine Australian politician.\NHe's a
Dialogue: 0,0:02:43.95,0:02:48.06,Default,,0000,0000,0000,,mining magnate billionaire and he is currently\Nrunning a
Dialogue: 0,0:02:48.06,0:02:52.88,Default,,0000,0000,0000,,MVP Jurrassic theme park with giant fiberglass\Ndinosaurs. And
Dialogue: 0,0:02:52.88,0:02:56.31,Default,,0000,0000,0000,,I, I for one am for it. So I
Dialogue: 0,0:02:56.31,0:02:58.51,Default,,0000,0000,0000,,realize there wasn't enough Star Wars references\Nso this
Dialogue: 0,0:02:58.51,0:03:00.54,Default,,0000,0000,0000,,is just completely gratuitous.
Dialogue: 0,0:03:00.54,0:03:05.43,Default,,0000,0000,0000,,Anyway. So. The thrust is that distributed\Nsystems are
Dialogue: 0,0:03:05.43,0:03:08.47,Default,,0000,0000,0000,,hard and databases are fun. Pictured here\Nis a
Dialogue: 0,0:03:08.47,0:03:13.53,Default,,0000,0000,0000,,distributed system. You can see there's two\Napp nodes
Dialogue: 0,0:03:13.53,0:03:16.66,Default,,0000,0000,0000,,and then there's two, there's like a master/slave\Nkind
Dialogue: 0,0:03:16.66,0:03:20.92,Default,,0000,0000,0000,,of setup going on here as well. So we're
Dialogue: 0,0:03:20.92,0:03:23.95,Default,,0000,0000,0000,,gonna talk about some of the complexities\Nof running
Dialogue: 0,0:03:23.95,0:03:27.67,Default,,0000,0000,0000,,these types of systems, and it's really fun\Nstuff
Dialogue: 0,0:03:27.67,0:03:29.98,Default,,0000,0000,0000,,once you get under the cover and start thinking
Dialogue: 0,0:03:29.98,0:03:32.25,Default,,0000,0000,0000,,about some of the complexities.
Dialogue: 0,0:03:32.25,0:03:37.03,Default,,0000,0000,0000,,So. NoSQL is a thing. We have NewSQL now.
Dialogue: 0,0:03:37.03,0:03:38.78,Default,,0000,0000,0000,,I'm gonna be covering some of these things.\NWe've
Dialogue: 0,0:03:38.78,0:03:44.00,Default,,0000,0000,0000,,also got PostSQL, Post-Rock Ambient SQL. And\Nthere's a
Dialogue: 0,0:03:44.00,0:03:47.12,Default,,0000,0000,0000,,whole gammit of these things. They all make\Nmy
Dialogue: 0,0:03:47.12,0:03:50.82,Default,,0000,0000,0000,,brain explode and the, I think the trick to
Dialogue: 0,0:03:50.82,0:03:53.07,Default,,0000,0000,0000,,understanding all of this stuff is to actually\Nthink
Dialogue: 0,0:03:53.07,0:03:55.46,Default,,0000,0000,0000,,about some of what's happening underneath.\NAnd you can
Dialogue: 0,0:03:55.46,0:03:59.65,Default,,0000,0000,0000,,make decisions about your databases.
Dialogue: 0,0:03:59.65,0:04:01.70,Default,,0000,0000,0000,,Hopefully you're all familiar with some of\Nthe concepts
Dialogue: 0,0:04:01.70,0:04:07.25,Default,,0000,0000,0000,,of traditional relational databases. We have\NAcid, which provides
Dialogue: 0,0:04:07.25,0:04:10.64,Default,,0000,0000,0000,,certain guarantees about the way that your\Ndata behaves.
Dialogue: 0,0:04:10.64,0:04:13.13,Default,,0000,0000,0000,,You can update data and be sure it was
Dialogue: 0,0:04:13.13,0:04:18.10,Default,,0000,0000,0000,,updated. Things are isolated from each other.\NThings persist
Dialogue: 0,0:04:18.10,0:04:20.97,Default,,0000,0000,0000,,over time.
Dialogue: 0,0:04:20.97,0:04:23.13,Default,,0000,0000,0000,,Another thing that you may have heard of,\Nthis
Dialogue: 0,0:04:23.13,0:04:25.74,Default,,0000,0000,0000,,is a, this is a leap that I need
Dialogue: 0,0:04:25.74,0:04:27.99,Default,,0000,0000,0000,,to another animation, is a thing called the\NCAP
Dialogue: 0,0:04:27.99,0:04:30.88,Default,,0000,0000,0000,,Theorem. So this gets talked about a lot when
Dialogue: 0,0:04:30.88,0:04:34.89,Default,,0000,0000,0000,,we start talking about this new generation\Nof databases.
Dialogue: 0,0:04:34.89,0:04:39.60,Default,,0000,0000,0000,,CAP stands for consistency, availability,\Nand partition tolerance, and
Dialogue: 0,0:04:39.60,0:04:44.43,Default,,0000,0000,0000,,it provides, basically, some strong foundation\Nfor reasoning about
Dialogue: 0,0:04:44.43,0:04:48.05,Default,,0000,0000,0000,,the way distributed systems behave and how\Nthey interoperate
Dialogue: 0,0:04:48.05,0:04:49.68,Default,,0000,0000,0000,,and how they communicate. So I'm gonna give\Nyou
Dialogue: 0,0:04:49.68,0:04:52.62,Default,,0000,0000,0000,,a brief introduction to how that all kind\Nof
Dialogue: 0,0:04:52.62,0:04:52.97,Default,,0000,0000,0000,,works.
Dialogue: 0,0:04:52.97,0:04:57.28,Default,,0000,0000,0000,,So, the original CAP Theorem, as stated, was,\Nis
Dialogue: 0,0:04:57.28,0:04:59.61,Default,,0000,0000,0000,,called Brewer's Conjecture. A guy called Brewer\Njust sort
Dialogue: 0,0:04:59.61,0:05:02.66,Default,,0000,0000,0000,,of had this idea. It's actually on some really
Dialogue: 0,0:05:02.66,0:05:06.68,Default,,0000,0000,0000,,awesomely-designed PowerPoint slides from\Nsome thing he did. And
Dialogue: 0,0:05:06.68,0:05:11.79,Default,,0000,0000,0000,,he was saying that with consistency, availability,\Nand partition
Dialogue: 0,0:05:11.79,0:05:15.21,Default,,0000,0000,0000,,tolerance - so the data can, can only be
Dialogue: 0,0:05:15.21,0:05:17.44,Default,,0000,0000,0000,,two of these things at any one time. So
Dialogue: 0,0:05:17.44,0:05:20.04,Default,,0000,0000,0000,,the data can be consistent or it can be
Dialogue: 0,0:05:20.04,0:05:23.98,Default,,0000,0000,0000,,accessible or it can handle network failures.
Dialogue: 0,0:05:23.98,0:05:28.25,Default,,0000,0000,0000,,So people then took this conjecture and actually\Nmade
Dialogue: 0,0:05:28.25,0:05:31.80,Default,,0000,0000,0000,,a formal kind of proof in, in much more
Dialogue: 0,0:05:31.80,0:05:38.80,Default,,0000,0000,0000,,rigorous computer science terms. And actually\Nsaid, it's impossible,
Dialogue: 0,0:05:38.83,0:05:42.58,Default,,0000,0000,0000,,in an asynchronous network model, to implement\Na read/write
Dialogue: 0,0:05:42.58,0:05:49.21,Default,,0000,0000,0000,,data object that is simultaneously available\Nand is also
Dialogue: 0,0:05:49.21,0:05:50.98,Default,,0000,0000,0000,,atomically consistent.
Dialogue: 0,0:05:50.98,0:05:53.11,Default,,0000,0000,0000,,And so all of this stuff around NewSQL and
Dialogue: 0,0:05:53.11,0:05:56.94,Default,,0000,0000,0000,,NoSQL and bleh, all of that stuff, is about
Dialogue: 0,0:05:56.94,0:06:01.61,Default,,0000,0000,0000,,manipulating these different variables. There's\Nalso a thing called
Dialogue: 0,0:06:01.61,0:06:02.99,Default,,0000,0000,0000,,Base but I'm not gonna talk about it cause
Dialogue: 0,0:06:02.99,0:06:05.79,Default,,0000,0000,0000,,it's actually just a made-up acronym that\Nhas no
Dialogue: 0,0:06:05.79,0:06:06.90,Default,,0000,0000,0000,,relevance to anything.
Dialogue: 0,0:06:06.90,0:06:10.06,Default,,0000,0000,0000,,So, what, what does CAP actually, what, what\Nare
Dialogue: 0,0:06:10.06,0:06:13.65,Default,,0000,0000,0000,,we talking about here? And why is it important?
Dialogue: 0,0:06:13.65,0:06:16.72,Default,,0000,0000,0000,,It's important, actually, because everything\Nis already distributed. What
Dialogue: 0,0:06:16.72,0:06:20.31,Default,,0000,0000,0000,,we do today is inherently a distributed system.\NYou
Dialogue: 0,0:06:20.31,0:06:23.23,Default,,0000,0000,0000,,have a browser talking to a server, an app
Dialogue: 0,0:06:23.23,0:06:25.52,Default,,0000,0000,0000,,server, Rails server - cause we're at RailsConf\N-
Dialogue: 0,0:06:25.52,0:06:29.25,Default,,0000,0000,0000,,and then that's talking to a PostGres database,\Nor
Dialogue: 0,0:06:29.25,0:06:33.81,Default,,0000,0000,0000,,a MySQL database or something even fancier\Nand shinier.
Dialogue: 0,0:06:33.81,0:06:36.11,Default,,0000,0000,0000,,That's a distributed system. And as we move\Ninto
Dialogue: 0,0:06:36.11,0:06:41.46,Default,,0000,0000,0000,,more heavy client-based operations, that distribution\Nis getting much
Dialogue: 0,0:06:41.46,0:06:43.76,Default,,0000,0000,0000,,more front-loaded, so you, you've got state\Nin the
Dialogue: 0,0:06:43.76,0:06:46.59,Default,,0000,0000,0000,,browser that's now synchronizing with state\Non the server.
Dialogue: 0,0:06:46.59,0:06:50.30,Default,,0000,0000,0000,,So we already actually suffer many of these\Nproblems.
Dialogue: 0,0:06:50.30,0:06:55.27,Default,,0000,0000,0000,,This is a handy and completely untrue guide\Nto
Dialogue: 0,0:06:55.27,0:06:59.39,Default,,0000,0000,0000,,NoSQL systems and breaking them into this\Nidea of
Dialogue: 0,0:06:59.39,0:07:03.04,Default,,0000,0000,0000,,some things are available and some things\Nare consistent.
Dialogue: 0,0:07:03.04,0:07:06.99,Default,,0000,0000,0000,,So, all of that is almost but not quite
Dialogue: 0,0:07:06.99,0:07:08.62,Default,,0000,0000,0000,,entirely untrue.
Dialogue: 0,0:07:08.62,0:07:12.80,Default,,0000,0000,0000,,What the actual theorem says is that under\Na
Dialogue: 0,0:07:12.80,0:07:16.49,Default,,0000,0000,0000,,network failure - so you've got multiple nodes\Nand
Dialogue: 0,0:07:16.49,0:07:20.16,Default,,0000,0000,0000,,they now can no longer communicate - you can
Dialogue: 0,0:07:20.16,0:07:23.49,Default,,0000,0000,0000,,choose whether the data is consistent or whether\Nthe
Dialogue: 0,0:07:23.49,0:07:28.85,Default,,0000,0000,0000,,data is available. And I have some demonstrations\Nhere
Dialogue: 0,0:07:28.85,0:07:30.86,Default,,0000,0000,0000,,to just - it actually ends up being very
Dialogue: 0,0:07:30.86,0:07:31.58,Default,,0000,0000,0000,,easy to understand.
Dialogue: 0,0:07:31.58,0:07:36.01,Default,,0000,0000,0000,,So, here we have typical cluster of nodes\Nworking
Dialogue: 0,0:07:36.01,0:07:41.11,Default,,0000,0000,0000,,together. We're gonna model some communication\Nbetween them. So
Dialogue: 0,0:07:41.11,0:07:45.67,Default,,0000,0000,0000,,there's a, there's a write on this system.\NIt
Dialogue: 0,0:07:45.67,0:07:49.45,Default,,0000,0000,0000,,comes in, that gets replicated across, and\Nthen on
Dialogue: 0,0:07:49.45,0:07:51.24,Default,,0000,0000,0000,,the other system we now have that data coming
Dialogue: 0,0:07:51.24,0:07:53.31,Default,,0000,0000,0000,,out. Someone's doing a read. And so this is
Dialogue: 0,0:07:53.31,0:07:57.68,Default,,0000,0000,0000,,the kind of situation that we're talking about.\NSo
Dialogue: 0,0:07:57.68,0:08:01.47,Default,,0000,0000,0000,,whether you're doing master/slave setup in\Na relational database
Dialogue: 0,0:08:01.47,0:08:06.13,Default,,0000,0000,0000,,or something trickier, this is kind of the\Nway
Dialogue: 0,0:08:06.13,0:08:08.30,Default,,0000,0000,0000,,it works. A node gets some data and it
Dialogue: 0,0:08:08.30,0:08:12.02,Default,,0000,0000,0000,,gives it to another node, and they have the
Dialogue: 0,0:08:12.02,0:08:13.86,Default,,0000,0000,0000,,same information.
Dialogue: 0,0:08:13.86,0:08:17.84,Default,,0000,0000,0000,,So when there's a network partition, that,\Nthey no
Dialogue: 0,0:08:17.84,0:08:22.12,Default,,0000,0000,0000,,longer can communicate. So a write comes in,\Nand
Dialogue: 0,0:08:22.12,0:08:25.87,Default,,0000,0000,0000,,now we have to make a decision. And all
Dialogue: 0,0:08:25.87,0:08:27.74,Default,,0000,0000,0000,,of this is actually just science, as you can
Dialogue: 0,0:08:27.74,0:08:30.76,Default,,0000,0000,0000,,tell from this diagram. If those two nodes\Ncan't
Dialogue: 0,0:08:30.76,0:08:33.40,Default,,0000,0000,0000,,communicate, you can talk to the one that\Ngot
Dialogue: 0,0:08:33.40,0:08:36.89,Default,,0000,0000,0000,,the write - that's consistent. It got the\Nwrite.
Dialogue: 0,0:08:36.89,0:08:39.22,Default,,0000,0000,0000,,It can now, can read out that same data.
Dialogue: 0,0:08:39.22,0:08:40.19,Default,,0000,0000,0000,,That's all cool.
Dialogue: 0,0:08:40.19,0:08:43.87,Default,,0000,0000,0000,,Or, you can have both nodes still communicating,\Nand
Dialogue: 0,0:08:43.87,0:08:46.34,Default,,0000,0000,0000,,now you have someone reading data that is\Nno
Dialogue: 0,0:08:46.34,0:08:49.10,Default,,0000,0000,0000,,longer in the write state. So we've got, you
Dialogue: 0,0:08:49.10,0:08:51.65,Default,,0000,0000,0000,,know, we have updated a bank account. It's\Ngot
Dialogue: 0,0:08:51.65,0:08:53.68,Default,,0000,0000,0000,,a hundred dollars in it. It used to have
Dialogue: 0,0:08:53.68,0:08:56.77,Default,,0000,0000,0000,,ten dollars in it. These people are reading\Nten.
Dialogue: 0,0:08:56.77,0:08:59.02,Default,,0000,0000,0000,,These people are reading a hundred. That's\Navailable. The
Dialogue: 0,0:08:59.02,0:09:01.20,Default,,0000,0000,0000,,data is now not consistent. But all of the
Dialogue: 0,0:09:01.20,0:09:03.22,Default,,0000,0000,0000,,nodes can send back that data.
Dialogue: 0,0:09:03.22,0:09:06.66,Default,,0000,0000,0000,,And so all of the discussion about CAP Theorem
Dialogue: 0,0:09:06.66,0:09:11.02,Default,,0000,0000,0000,,and, and you know, people even claiming, we've\Ndefeated
Dialogue: 0,0:09:11.02,0:09:14.04,Default,,0000,0000,0000,,the CAP Theorem in our database at, you know,
Dialogue: 0,0:09:14.04,0:09:19.43,Default,,0000,0000,0000,,low-low prices is incredibly awesome. Just\Nremember this image.
Dialogue: 0,0:09:19.43,0:09:26.00,Default,,0000,0000,0000,,Two things that cannot communicate cannot\Ncommunicate. It's science.
Dialogue: 0,0:09:26.00,0:09:28.25,Default,,0000,0000,0000,,And then when they can communicate, we're\Nback into
Dialogue: 0,0:09:28.25,0:09:30.85,Default,,0000,0000,0000,,the realm of normal operations and things\Nget a
Dialogue: 0,0:09:30.85,0:09:35.09,Default,,0000,0000,0000,,lot easier. If you were interested in any\Nof
Dialogue: 0,0:09:35.09,0:09:39.54,Default,,0000,0000,0000,,the guts of how these things work, definitely\Nhave
Dialogue: 0,0:09:39.54,0:09:42.43,Default,,0000,0000,0000,,a look at a thing called jepsen, which is
Dialogue: 0,0:09:42.43,0:09:47.67,Default,,0000,0000,0000,,this crazy motherfucker who is just analyzing\Nthe network
Dialogue: 0,0:09:47.67,0:09:51.51,Default,,0000,0000,0000,,operations of a whole variety of distributed\Nsystems, and
Dialogue: 0,0:09:51.51,0:09:54.90,Default,,0000,0000,0000,,it will, it's just, it will blow your mind.
Dialogue: 0,0:09:54.90,0:10:01.08,Default,,0000,0000,0000,,OK. Good. That's, that's why. Now I remember.
Dialogue: 0,0:10:01.08,0:10:04.59,Default,,0000,0000,0000,,So, here is our cast. We're about to go
Dialogue: 0,0:10:04.59,0:10:08.85,Default,,0000,0000,0000,,on an adventure through a tortured maze of\Nridiculous
Dialogue: 0,0:10:08.85,0:10:11.62,Default,,0000,0000,0000,,Dungeons and Dragons metaphors. But, first\Nof all, a
Dialogue: 0,0:10:11.62,0:10:14.96,Default,,0000,0000,0000,,shout out to the OwlBear. Yeah. The thing\NI
Dialogue: 0,0:10:14.96,0:10:18.74,Default,,0000,0000,0000,,love about the OwlBear is they've taken the\Nwrong,
Dialogue: 0,0:10:18.74,0:10:22.68,Default,,0000,0000,0000,,the least scary aspects of a bear and an
Dialogue: 0,0:10:22.68,0:10:26.26,Default,,0000,0000,0000,,owl, like if that was an owl with, you
Dialogue: 0,0:10:26.26,0:10:30.12,Default,,0000,0000,0000,,know, if it had a bears head and wings,
Dialogue: 0,0:10:30.12,0:10:33.62,Default,,0000,0000,0000,,that would be way more scary. Anyway.
Dialogue: 0,0:10:33.62,0:10:37.13,Default,,0000,0000,0000,,It's just been bugging me for months. So.
Dialogue: 0,0:10:37.13,0:10:41.67,Default,,0000,0000,0000,,PostGres. As we all know, it's MySQL for hipsters.
Dialogue: 0,0:10:41.67,0:10:45.05,Default,,0000,0000,0000,,It's actually pretty good. So here's its character\Nreference
Dialogue: 0,0:10:45.05,0:10:49.32,Default,,0000,0000,0000,,sheet. We, it's a relational database. It\Nhas a
Dialogue: 0,0:10:49.32,0:10:53.76,Default,,0000,0000,0000,,consistent model. So under conditions in network\Npartition, you
Dialogue: 0,0:10:53.76,0:10:56.65,Default,,0000,0000,0000,,know, your, your slave is not in contact with
Dialogue: 0,0:10:56.65,0:11:00.48,Default,,0000,0000,0000,,the master, it's, it's essentially unavailable.\NThat's the way
Dialogue: 0,0:11:00.48,0:11:01.73,Default,,0000,0000,0000,,we treat it.
Dialogue: 0,0:11:01.73,0:11:05.37,Default,,0000,0000,0000,,PostGres is actually really, really interesting\Ntic, because it
Dialogue: 0,0:11:05.37,0:11:10.29,Default,,0000,0000,0000,,has a bunch of cool stuff hidden underneath\Nit.
Dialogue: 0,0:11:10.29,0:11:13.01,Default,,0000,0000,0000,,So there's a thing called Hstore which is\Na
Dialogue: 0,0:11:13.01,0:11:15.65,Default,,0000,0000,0000,,key-value store that's baked right in. So\Nif you
Dialogue: 0,0:11:15.65,0:11:18.72,Default,,0000,0000,0000,,need a lightweight key-value store and you're\Nalready running
Dialogue: 0,0:11:18.72,0:11:22.70,Default,,0000,0000,0000,,PostGres in production, you, you have one.\NYou don't
Dialogue: 0,0:11:22.70,0:11:25.60,Default,,0000,0000,0000,,need to spin up any other thing. You can
Dialogue: 0,0:11:25.60,0:11:27.45,Default,,0000,0000,0000,,actually do that today.
Dialogue: 0,0:11:27.45,0:11:30.32,Default,,0000,0000,0000,,The really interesting thing about that is,\Nyou can
Dialogue: 0,0:11:30.32,0:11:34.10,Default,,0000,0000,0000,,index those keys. You can do joins across\Nan
Dialogue: 0,0:11:34.10,0:11:38.23,Default,,0000,0000,0000,,Hstore reference into, across multiple tables.\NIt looks and
Dialogue: 0,0:11:38.23,0:11:40.25,Default,,0000,0000,0000,,feels exactly like the kind of thing that\Nyou're
Dialogue: 0,0:11:40.25,0:11:42.06,Default,,0000,0000,0000,,already working with.
Dialogue: 0,0:11:42.06,0:11:46.56,Default,,0000,0000,0000,,We've got, there's some things already baked\Ninto the
Dialogue: 0,0:11:46.56,0:11:49.38,Default,,0000,0000,0000,,Rails ecosystem that make this really easy\Nif you're
Dialogue: 0,0:11:49.38,0:11:52.80,Default,,0000,0000,0000,,doing that kind of information. But the really\Nexciting
Dialogue: 0,0:11:52.80,0:11:55.57,Default,,0000,0000,0000,,thing about what PostGres is up to at the
Dialogue: 0,0:11:55.57,0:12:01.72,Default,,0000,0000,0000,,moment is JSON. And 9.2, 9.3, and upcoming\N9.4
Dialogue: 0,0:12:01.72,0:12:05.81,Default,,0000,0000,0000,,have pretty much a fully baked in JSON document
Dialogue: 0,0:12:05.81,0:12:11.19,Default,,0000,0000,0000,,database. And it is crazy awesome. The new\None
Dialogue: 0,0:12:11.19,0:12:14.35,Default,,0000,0000,0000,,is super high-performance. If you were sort\Nof, it's
Dialogue: 0,0:12:14.35,0:12:17.40,Default,,0000,0000,0000,,the same thing. If you're thinking, ah, you\Nknow,
Dialogue: 0,0:12:17.40,0:12:20.25,Default,,0000,0000,0000,,documents would be easier for this use case,\Nlet's
Dialogue: 0,0:12:20.25,0:12:24.73,Default,,0000,0000,0000,,install something else, we're actually, you\Nalready have one,
Dialogue: 0,0:12:24.73,0:12:26.69,Default,,0000,0000,0000,,and it, it has all of those same properties.
Dialogue: 0,0:12:26.69,0:12:28.61,Default,,0000,0000,0000,,You can index. You can do joins across your
Dialogue: 0,0:12:28.61,0:12:32.76,Default,,0000,0000,0000,,normal table into the documents. It's crazy\Ncool.
Dialogue: 0,0:12:32.76,0:12:36.67,Default,,0000,0000,0000,,MySQL. It's pretty much the same as PostGres,\Nis
Dialogue: 0,0:12:36.67,0:12:42.31,Default,,0000,0000,0000,,my answer. But there's a slight caveat. So,\Nyou
Dialogue: 0,0:12:42.31,0:12:47.90,Default,,0000,0000,0000,,know, I, I recall, they're a company. Many\Nof
Dialogue: 0,0:12:47.90,0:12:50.18,Default,,0000,0000,0000,,the same things apply. Like, this is why,\Nyou
Dialogue: 0,0:12:50.18,0:12:52.38,Default,,0000,0000,0000,,know, they're, they're kind of in the same\Nbucket.
Dialogue: 0,0:12:52.38,0:12:55.95,Default,,0000,0000,0000,,For me, it doesn't particularly matter at\Nthe end
Dialogue: 0,0:12:55.95,0:12:58.19,Default,,0000,0000,0000,,of the day. Whatever you happen to have expertise
Dialogue: 0,0:12:58.19,0:13:01.17,Default,,0000,0000,0000,,in, it's cool. It's got some kind of interesting
Dialogue: 0,0:13:01.17,0:13:02.87,Default,,0000,0000,0000,,things that you can do. You can switch out
Dialogue: 0,0:13:02.87,0:13:07.70,Default,,0000,0000,0000,,storage engines to actually get your different\Nperformance profiles.
Dialogue: 0,0:13:07.70,0:13:11.53,Default,,0000,0000,0000,,It is everywhere. It's got a thing called\NHandler
Dialogue: 0,0:13:11.53,0:13:16.41,Default,,0000,0000,0000,,Socket, which is essentially raw, right. Access\Nthrough a
Dialogue: 0,0:13:16.41,0:13:19.74,Default,,0000,0000,0000,,low-level socket into the table infrastructure.\NThere's some paper
Dialogue: 0,0:13:19.74,0:13:24.44,Default,,0000,0000,0000,,with really high performance kind of things.
Dialogue: 0,0:13:24.44,0:13:26.66,Default,,0000,0000,0000,,You can actually just sort of bypass the whole
Dialogue: 0,0:13:26.66,0:13:29.56,Default,,0000,0000,0000,,SQL engine, which is kind of interesting.\NThe other
Dialogue: 0,0:13:29.56,0:13:31.87,Default,,0000,0000,0000,,thing that's happened since Oracle took over,\Nwhich is
Dialogue: 0,0:13:31.87,0:13:35.34,Default,,0000,0000,0000,,kind of a really good thing, is that there's
Dialogue: 0,0:13:35.34,0:13:40.47,Default,,0000,0000,0000,,some alternatives. So MariaDB is sort of the,\Nthe
Dialogue: 0,0:13:40.47,0:13:44.65,Default,,0000,0000,0000,,more open fork. There's a semi-commercial\Naddition that has
Dialogue: 0,0:13:44.65,0:13:47.90,Default,,0000,0000,0000,,lots of really high-performance features,\Nand they basically run
Dialogue: 0,0:13:47.90,0:13:51.61,Default,,0000,0000,0000,,binary compatible patches, that's Percona.\NAnd they have, like,
Dialogue: 0,0:13:51.61,0:13:55.61,Default,,0000,0000,0000,,huge expertise. And this Toku is quite interesting.\NIt's,
Dialogue: 0,0:13:55.61,0:13:58.40,Default,,0000,0000,0000,,they're doing all of this crazy fractal indexing\Nand
Dialogue: 0,0:13:58.40,0:14:02.25,Default,,0000,0000,0000,,things for particular use cases on very large\Ndatasets.
Dialogue: 0,0:14:02.25,0:14:04.90,Default,,0000,0000,0000,,But it still just looks and behaves in many
Dialogue: 0,0:14:04.90,0:14:07.89,Default,,0000,0000,0000,,ways like the MySQL that you are kind of
Dialogue: 0,0:14:07.89,0:14:08.63,Default,,0000,0000,0000,,used to.
Dialogue: 0,0:14:08.63,0:14:13.13,Default,,0000,0000,0000,,So, there's some interesting things happening\Nthere. So these,
Dialogue: 0,0:14:13.13,0:14:16.64,Default,,0000,0000,0000,,hopefully none of that's a huge surprise.\NThat's databases.
Dialogue: 0,0:14:16.64,0:14:21.32,Default,,0000,0000,0000,,You use it. It comes in the box, and
Dialogue: 0,0:14:21.32,0:14:22.60,Default,,0000,0000,0000,,ActiveRecord talks to it.
Dialogue: 0,0:14:22.60,0:14:24.78,Default,,0000,0000,0000,,So now we're gonna get slightly off the beaten
Dialogue: 0,0:14:24.78,0:14:30.37,Default,,0000,0000,0000,,track. So, a lot of what we know SQL
Dialogue: 0,0:14:30.37,0:14:35.37,Default,,0000,0000,0000,,comes from Dynamo, which was actually a paper\Nthat
Dialogue: 0,0:14:35.37,0:14:40.16,Default,,0000,0000,0000,,Amazon released years ago. I'm not gonna labor\Ntoo
Dialogue: 0,0:14:40.16,0:14:42.46,Default,,0000,0000,0000,,much on this one. The paper's quite interesting.\NIt
Dialogue: 0,0:14:42.46,0:14:48.00,Default,,0000,0000,0000,,talks about how you make a distributed system.
Dialogue: 0,0:14:48.00,0:14:51.93,Default,,0000,0000,0000,,The interesting thing is actually that Riak\Nis essentially
Dialogue: 0,0:14:51.93,0:14:55.30,Default,,0000,0000,0000,,an implementation of the underlying Dynamo\Ntheory. So Riak
Dialogue: 0,0:14:55.30,0:14:58.34,Default,,0000,0000,0000,,is crazy awesome. This is what happens to\Nyou
Dialogue: 0,0:14:58.34,0:15:01.58,Default,,0000,0000,0000,,when you run Riak in production.
Dialogue: 0,0:15:01.58,0:15:02.43,Default,,0000,0000,0000,,[laughter]
Dialogue: 0,0:15:02.43,0:15:05.93,Default,,0000,0000,0000,,I pretty much, like, it's a conversation I,\NI
Dialogue: 0,0:15:05.93,0:15:08.72,Default,,0000,0000,0000,,often have with people is like, wouldn't it\Nbe
Dialogue: 0,0:15:08.72,0:15:12.75,Default,,0000,0000,0000,,awesome to have a problem that needed Riak?\NAnd
Dialogue: 0,0:15:12.75,0:15:13.71,Default,,0000,0000,0000,,it was like, yeah, that would be so cool.
Dialogue: 0,0:15:13.71,0:15:17.97,Default,,0000,0000,0000,,I'd be like the awesomeness engineer.
Dialogue: 0,0:15:17.97,0:15:21.43,Default,,0000,0000,0000,,So Riak is, it's just crazy-well engineered.\NThey're doing
Dialogue: 0,0:15:21.43,0:15:26.87,Default,,0000,0000,0000,,all sorts of interesting stuff. It's inherently,\Nit just
Dialogue: 0,0:15:26.87,0:15:30.68,Default,,0000,0000,0000,,understands clustering. You know, you add\Na new node,
Dialogue: 0,0:15:30.68,0:15:35.26,Default,,0000,0000,0000,,it just, it's there. You know. With, with\Nthose
Dialogue: 0,0:15:35.26,0:15:37.61,Default,,0000,0000,0000,,older kind of databases, it's, it's a pain\Nin
Dialogue: 0,0:15:37.61,0:15:40.17,Default,,0000,0000,0000,,the ass to actually get it working.
Dialogue: 0,0:15:40.17,0:15:45.34,Default,,0000,0000,0000,,So, yeah, they're doing some really interesting\Nthings. It's
Dialogue: 0,0:15:45.34,0:15:47.92,Default,,0000,0000,0000,,got a cloud storage thing so you've got an
Dialogue: 0,0:15:47.92,0:15:50.12,Default,,0000,0000,0000,,S3-compatible API and all of these kind of\Nstuff.
Dialogue: 0,0:15:50.12,0:15:51.36,Default,,0000,0000,0000,,A lot of the magic of the way this
Dialogue: 0,0:15:51.36,0:15:56.66,Default,,0000,0000,0000,,works is through consistent hashing. So, my\Nslides are
Dialogue: 0,0:15:56.66,0:15:58.38,Default,,0000,0000,0000,,all mucked up. But anyway.
Dialogue: 0,0:15:58.38,0:16:00.64,Default,,0000,0000,0000,,So, basically what it does is it just partitions
Dialogue: 0,0:16:00.64,0:16:05.35,Default,,0000,0000,0000,,all of your data into a giant hash ring.
Dialogue: 0,0:16:05.35,0:16:10.45,Default,,0000,0000,0000,,Excuse me. Physical nodes then just own parts\Nof
Dialogue: 0,0:16:10.45,0:16:12.37,Default,,0000,0000,0000,,that hash. You add a new node or take
Dialogue: 0,0:16:12.37,0:16:15.72,Default,,0000,0000,0000,,a node away and it repartitions all the rest
Dialogue: 0,0:16:15.72,0:16:17.94,Default,,0000,0000,0000,,of the data across the remaining nodes. And\Nall
Dialogue: 0,0:16:17.94,0:16:21.48,Default,,0000,0000,0000,,of that is just completely in the background\Nof
Dialogue: 0,0:16:21.48,0:16:24.68,Default,,0000,0000,0000,,how Riak just works operationally.
Dialogue: 0,0:16:24.68,0:16:27.30,Default,,0000,0000,0000,,So for large scale data and, you know, you,
Dialogue: 0,0:16:27.30,0:16:30.71,Default,,0000,0000,0000,,you get away with, it has some really nice
Dialogue: 0,0:16:30.71,0:16:34.17,Default,,0000,0000,0000,,operational characteristics that, that make\Nit quite cool to
Dialogue: 0,0:16:34.17,0:16:34.75,Default,,0000,0000,0000,,manage.
Dialogue: 0,0:16:34.75,0:16:36.53,Default,,0000,0000,0000,,And then the other thing is, it's a very
Dialogue: 0,0:16:36.53,0:16:40.19,Default,,0000,0000,0000,,simple API. It's key-value store, you can\Nstore JSON
Dialogue: 0,0:16:40.19,0:16:42.22,Default,,0000,0000,0000,,documents in it, and it's just a bucket that
Dialogue: 0,0:16:42.22,0:16:45.13,Default,,0000,0000,0000,,has keys, and then it's got other stuff on
Dialogue: 0,0:16:45.13,0:16:49.51,Default,,0000,0000,0000,,top to retrieve data, do secondary indexes\Nand searching
Dialogue: 0,0:16:49.51,0:16:51.13,Default,,0000,0000,0000,,and all of that kind of stuff.
Dialogue: 0,0:16:51.13,0:16:54.22,Default,,0000,0000,0000,,So, it's a very cool piece of tech.
Dialogue: 0,0:16:54.22,0:16:59.08,Default,,0000,0000,0000,,So, the other one we've got is, Google. Fucking
Dialogue: 0,0:16:59.08,0:17:03.81,Default,,0000,0000,0000,,annoying. And you'll see why in a second.\NSo,
Dialogue: 0,0:17:03.81,0:17:06.98,Default,,0000,0000,0000,,Google had this thing called BigTable that,\Nagain, kind
Dialogue: 0,0:17:06.98,0:17:10.47,Default,,0000,0000,0000,,of comes out of the internal research. You\Nhave
Dialogue: 0,0:17:10.47,0:17:14.30,Default,,0000,0000,0000,,access to it through some of their cloud properties.
Dialogue: 0,0:17:14.30,0:17:16.80,Default,,0000,0000,0000,,As you can see, it's got, it's actually a
Dialogue: 0,0:17:16.80,0:17:21.29,Default,,0000,0000,0000,,sparse distributed multidimensional sorted\Nmap, which is good, I
Dialogue: 0,0:17:21.29,0:17:23.62,Default,,0000,0000,0000,,guess. I imagine. It's awesome.
Dialogue: 0,0:17:23.62,0:17:27.72,Default,,0000,0000,0000,,The stuff they're doing with this is crazy.\NSo
Dialogue: 0,0:17:27.72,0:17:30.19,Default,,0000,0000,0000,,this is actually a, all, a couple years old
Dialogue: 0,0:17:30.19,0:17:33.41,Default,,0000,0000,0000,,I think now. Some of these, some of the
Dialogue: 0,0:17:33.41,0:17:37.19,Default,,0000,0000,0000,,information, so. Hundreds of petabytes of\Ndata, you know,
Dialogue: 0,0:17:37.19,0:17:40.58,Default,,0000,0000,0000,,ridiculous numbers of operations a second.\NYou do not
Dialogue: 0,0:17:40.58,0:17:42.65,Default,,0000,0000,0000,,have any of these problems.
Dialogue: 0,0:17:42.65,0:17:46.88,Default,,0000,0000,0000,,So, then they, they took this stuff, they\Nwere
Dialogue: 0,0:17:46.88,0:17:50.21,Default,,0000,0000,0000,,like, ah, we've got BigTable. You know, that\Nwas,
Dialogue: 0,0:17:50.21,0:17:53.50,Default,,0000,0000,0000,,that was fucking easy. Whatever. And so now\Nthey've
Dialogue: 0,0:17:53.50,0:17:55.48,Default,,0000,0000,0000,,got two other things. They've got one called\NSpanner
Dialogue: 0,0:17:55.48,0:18:00.02,Default,,0000,0000,0000,,and one called F-one, where they're basically\Ndoing, you
Dialogue: 0,0:18:00.02,0:18:07.02,Default,,0000,0000,0000,,know, proper, sort of relational looking data\Nacross multiple
Dialogue: 0,0:18:07.35,0:18:10.32,Default,,0000,0000,0000,,data centers and, you know, and. They're kind\Nof
Dialogue: 0,0:18:10.32,0:18:12.59,Default,,0000,0000,0000,,really pushing the boundaries of some of that\NCAP
Dialogue: 0,0:18:12.59,0:18:14.71,Default,,0000,0000,0000,,stuff that's going on.
Dialogue: 0,0:18:14.71,0:18:18.49,Default,,0000,0000,0000,,But all you need is a GPS in every
Dialogue: 0,0:18:18.49,0:18:21.38,Default,,0000,0000,0000,,server, a couple of atomic clocks in each\Ndata
Dialogue: 0,0:18:21.38,0:18:26.83,Default,,0000,0000,0000,,center, and you, great. So, Google's basically\Ntelling everyone
Dialogue: 0,0:18:26.83,0:18:29.72,Default,,0000,0000,0000,,to, you know, just fuck off.
Dialogue: 0,0:18:29.72,0:18:35.17,Default,,0000,0000,0000,,So, another one that I really, I really like,
Dialogue: 0,0:18:35.17,0:18:39.49,Default,,0000,0000,0000,,and have used a long, a long time ago
Dialogue: 0,0:18:39.49,0:18:45.69,Default,,0000,0000,0000,,in, in tech land, tech time, is Cassandra.\NCassandra
Dialogue: 0,0:18:45.69,0:18:50.11,Default,,0000,0000,0000,,is a column-oriented database. Eventually\Nit's awesome. It's really
Dialogue: 0,0:18:50.11,0:18:54.24,Default,,0000,0000,0000,,all about eventual consistency.
Dialogue: 0,0:18:54.24,0:18:57.52,Default,,0000,0000,0000,,And you can see here, this is a man,
Dialogue: 0,0:18:57.52,0:18:59.26,Default,,0000,0000,0000,,he eventually gets it right. So that's well\Ndone
Dialogue: 0,0:18:59.26,0:19:02.36,Default,,0000,0000,0000,,to him there. So Cassandra's a lot like that.
Dialogue: 0,0:19:02.36,0:19:06.02,Default,,0000,0000,0000,,And, again, you know, the cool thing is, it's
Dialogue: 0,0:19:06.02,0:19:10.55,Default,,0000,0000,0000,,a sparse distributor multi dimensional sorted\Nmap. It, when
Dialogue: 0,0:19:10.55,0:19:13.35,Default,,0000,0000,0000,,I was working with it, you, it was, you
Dialogue: 0,0:19:13.35,0:19:16.10,Default,,0000,0000,0000,,had, you described your tables kind of thing\Nin
Dialogue: 0,0:19:16.10,0:19:20.31,Default,,0000,0000,0000,,XML and hated yourself, and then every time\Nsomething
Dialogue: 0,0:19:20.31,0:19:23.46,Default,,0000,0000,0000,,changed you rebooted the server and that took\Nawhile
Dialogue: 0,0:19:23.46,0:19:27.39,Default,,0000,0000,0000,,and, yeah, the whole thing was really difficult.
Dialogue: 0,0:19:27.39,0:19:30.57,Default,,0000,0000,0000,,What it basically does is it takes the availability
Dialogue: 0,0:19:30.57,0:19:33.57,Default,,0000,0000,0000,,side of the question. Like, that's its world\Nmodel.
Dialogue: 0,0:19:33.57,0:19:37.83,Default,,0000,0000,0000,,It has, again, a very simple clustering system.\NNew
Dialogue: 0,0:19:37.83,0:19:41.29,Default,,0000,0000,0000,,nodes, add in, the data gets streamed out.\NIt
Dialogue: 0,0:19:41.29,0:19:46.07,Default,,0000,0000,0000,,has a data model that is really complicated,\Nand
Dialogue: 0,0:19:46.07,0:19:48.47,Default,,0000,0000,0000,,I, even though I've used it, it's really hard
Dialogue: 0,0:19:48.47,0:19:50.91,Default,,0000,0000,0000,,to explain how it actually works.
Dialogue: 0,0:19:50.91,0:19:54.73,Default,,0000,0000,0000,,So column databases basically kind of invert\Nthe, the
Dialogue: 0,0:19:54.73,0:19:56.70,Default,,0000,0000,0000,,whole table structure that you're used to\Nfrom the
Dialogue: 0,0:19:56.70,0:20:01.19,Default,,0000,0000,0000,,relational world. And the advantage is that,\Nfor some
Dialogue: 0,0:20:01.19,0:20:04.16,Default,,0000,0000,0000,,types of data, and for some queries, it is
Dialogue: 0,0:20:04.16,0:20:07.60,Default,,0000,0000,0000,,crazy blazing fast, cause you can just. Time\Nseries
Dialogue: 0,0:20:07.60,0:20:08.62,Default,,0000,0000,0000,,are always a good one, where you can just
Dialogue: 0,0:20:08.62,0:20:10.93,Default,,0000,0000,0000,,have long streams of time series and it will
Dialogue: 0,0:20:10.93,0:20:13.49,Default,,0000,0000,0000,,actually put that on disk or next to each
Dialogue: 0,0:20:13.49,0:20:15.60,Default,,0000,0000,0000,,other and you can just pull it all out.
Dialogue: 0,0:20:15.60,0:20:18.51,Default,,0000,0000,0000,,The cool thing in the new versions of Cassandra
Dialogue: 0,0:20:18.51,0:20:22.30,Default,,0000,0000,0000,,is that they've abstracted all of that out,\Nand
Dialogue: 0,0:20:22.30,0:20:25.57,Default,,0000,0000,0000,,you actually just get tables, so you can create
Dialogue: 0,0:20:25.57,0:20:28.20,Default,,0000,0000,0000,,a table and give it a primary key, and
Dialogue: 0,0:20:28.20,0:20:32.24,Default,,0000,0000,0000,,under the covers, it's setting up rows and\Ncolumn
Dialogue: 0,0:20:32.24,0:20:35.24,Default,,0000,0000,0000,,families and columns and all of, all of these
Dialogue: 0,0:20:35.24,0:20:39.39,Default,,0000,0000,0000,,really abstract concepts, and they've completely\Nmade some of
Dialogue: 0,0:20:39.39,0:20:41.50,Default,,0000,0000,0000,,that go away. Which is really nice.
Dialogue: 0,0:20:41.50,0:20:43.93,Default,,0000,0000,0000,,So you end up with something that looks a
Dialogue: 0,0:20:43.93,0:20:48.74,Default,,0000,0000,0000,,lot like just SQL and, you know, a normal
Dialogue: 0,0:20:48.74,0:20:52.65,Default,,0000,0000,0000,,table kind of structure. It's just clustering\Nout lots
Dialogue: 0,0:20:52.65,0:20:55.10,Default,,0000,0000,0000,,of nodes. It's very tunable, so you can actually
Dialogue: 0,0:20:55.10,0:20:57.99,Default,,0000,0000,0000,,set up, you know, it writes to a node
Dialogue: 0,0:20:57.99,0:21:00.02,Default,,0000,0000,0000,,and you can say, actually write to five nodes
Dialogue: 0,0:21:00.02,0:21:02.02,Default,,0000,0000,0000,,and that's a quorem and now we're cool. So
Dialogue: 0,0:21:02.02,0:21:06.02,Default,,0000,0000,0000,,you can tune how much redundancy you have.
Dialogue: 0,0:21:06.02,0:21:12.59,Default,,0000,0000,0000,,So that's kind of cool. That is a reminder.
Dialogue: 0,0:21:12.59,0:21:17.56,Default,,0000,0000,0000,,That went cold really fast. Thank you.
Dialogue: 0,0:21:17.56,0:21:20.83,Default,,0000,0000,0000,,So, the next one on our list is Memcache.
Dialogue: 0,0:21:20.83,0:21:24.22,Default,,0000,0000,0000,,Memcache, there was, there was a talk earlier\Nin
Dialogue: 0,0:21:24.22,0:21:27.53,Default,,0000,0000,0000,,the week that was describing using Memcache\Nand caching
Dialogue: 0,0:21:27.53,0:21:29.79,Default,,0000,0000,0000,,and it, it had a very interesting observation,\Nwhich
Dialogue: 0,0:21:29.79,0:21:32.67,Default,,0000,0000,0000,,was, it just works. He didn't even know what
Dialogue: 0,0:21:32.67,0:21:36.59,Default,,0000,0000,0000,,version he was running in production, cause\Nneh. Doesn't
Dialogue: 0,0:21:36.59,0:21:38.74,Default,,0000,0000,0000,,matter. That API has been stable for ages.
Dialogue: 0,0:21:38.74,0:21:42.42,Default,,0000,0000,0000,,And I know, I know what you're saying. It's
Dialogue: 0,0:21:42.42,0:21:45.56,Default,,0000,0000,0000,,not a database. It's a cache. Technically\Ntrue. But
Dialogue: 0,0:21:45.56,0:21:48.05,Default,,0000,0000,0000,,it's interesting to think about, because the\Nmoment you
Dialogue: 0,0:21:48.05,0:21:51.38,Default,,0000,0000,0000,,add caching, even if you've been ignoring\Nthe fact
Dialogue: 0,0:21:51.38,0:21:54.78,Default,,0000,0000,0000,,that you had a distributed system before,\Nwith caching
Dialogue: 0,0:21:54.78,0:21:57.33,Default,,0000,0000,0000,,you now really have a distributed system.\NYou've got
Dialogue: 0,0:21:57.33,0:21:59.98,Default,,0000,0000,0000,,data in one thing that may or may not
Dialogue: 0,0:21:59.98,0:22:02.76,Default,,0000,0000,0000,,be fresh, and you've got data in your database
Dialogue: 0,0:22:02.76,0:22:05.12,Default,,0000,0000,0000,,that, you know, you assume is up to date,
Dialogue: 0,0:22:05.12,0:22:07.25,Default,,0000,0000,0000,,and now you've got a synchronization problem.
Dialogue: 0,0:22:07.25,0:22:12.08,Default,,0000,0000,0000,,So, Memcache is actually really, you know,\Nit's, it's
Dialogue: 0,0:22:12.08,0:22:16.66,Default,,0000,0000,0000,,just rock solid, old as the hills technology,\Ncompletely
Dialogue: 0,0:22:16.66,0:22:22.28,Default,,0000,0000,0000,,simple. The API is everywhere. Lots of people\Nactually
Dialogue: 0,0:22:22.28,0:22:26.12,Default,,0000,0000,0000,,have made their, you know, key-value store\Nthey made
Dialogue: 0,0:22:26.12,0:22:28.31,Default,,0000,0000,0000,,in the hacknight, which, you know, is a useful
Dialogue: 0,0:22:28.31,0:22:30.74,Default,,0000,0000,0000,,hobby if you want to annoy everyone.
Dialogue: 0,0:22:30.74,0:22:33.14,Default,,0000,0000,0000,,You have the, their API is actually the Memcached
Dialogue: 0,0:22:33.14,0:22:36.08,Default,,0000,0000,0000,,API. It's got a handful of things. You can
Dialogue: 0,0:22:36.08,0:22:40.13,Default,,0000,0000,0000,,set a key, you can replace one. It does
Dialogue: 0,0:22:40.13,0:22:43.68,Default,,0000,0000,0000,,have something atomic operations so you can\Nincrement and
Dialogue: 0,0:22:43.68,0:22:46.15,Default,,0000,0000,0000,,decrement so that there is some flexibility\Nto actually
Dialogue: 0,0:22:46.15,0:22:51.67,Default,,0000,0000,0000,,do a little bit of data storage in a,
Dialogue: 0,0:22:51.67,0:22:55.78,Default,,0000,0000,0000,,in a more traditional sense.
Dialogue: 0,0:22:55.78,0:22:59.39,Default,,0000,0000,0000,,It's actually a client-server model. Your,\Nyour driver is
Dialogue: 0,0:22:59.39,0:23:02.43,Default,,0000,0000,0000,,responsible for the clustering in a way, so\Nyou
Dialogue: 0,0:23:02.43,0:23:07.05,Default,,0000,0000,0000,,can have multiple Memcache nodes and the,\Nthe hashing
Dialogue: 0,0:23:07.05,0:23:11.28,Default,,0000,0000,0000,,algorithm determines which node, which node\Na particular piece
Dialogue: 0,0:23:11.28,0:23:13.44,Default,,0000,0000,0000,,of data is gonna be on.
Dialogue: 0,0:23:13.44,0:23:15.96,Default,,0000,0000,0000,,That has the property of making it very, very
Dialogue: 0,0:23:15.96,0:23:19.44,Default,,0000,0000,0000,,simple to use. And there's no cluster state.\NThere's
Dialogue: 0,0:23:19.44,0:23:21.89,Default,,0000,0000,0000,,no coordination that nodes have. Like, a lot\Nof
Dialogue: 0,0:23:21.89,0:23:23.52,Default,,0000,0000,0000,,the heavy lifting all of these other things\Nare
Dialogue: 0,0:23:23.52,0:23:27.87,Default,,0000,0000,0000,,doing is about coordinating around all of\Nthat information.
Dialogue: 0,0:23:27.87,0:23:29.75,Default,,0000,0000,0000,,There's a whole bunch of awesome stuff just\Nbaked
Dialogue: 0,0:23:29.75,0:23:34.52,Default,,0000,0000,0000,,into Rails. So you can just easily cache into
Dialogue: 0,0:23:34.52,0:23:38.94,Default,,0000,0000,0000,,Memcache, or your normal Rails fragment mutations.\NAll of
Dialogue: 0,0:23:38.94,0:23:40.87,Default,,0000,0000,0000,,that kind of stuff.
Dialogue: 0,0:23:40.87,0:23:42.41,Default,,0000,0000,0000,,And there's even some things we can, you can
Dialogue: 0,0:23:42.41,0:23:46.29,Default,,0000,0000,0000,,actually put, push that into ActiveRecord\Nand have, have
Dialogue: 0,0:23:46.29,0:23:48.44,Default,,0000,0000,0000,,caching at that level as well.
Dialogue: 0,0:23:48.44,0:23:50.70,Default,,0000,0000,0000,,Redis is an interesting one for the, the Rails
Dialogue: 0,0:23:50.70,0:23:56.58,Default,,0000,0000,0000,,community. Cause it's basically a queue, now.\NEveryone seems
Dialogue: 0,0:23:56.58,0:24:01.37,Default,,0000,0000,0000,,to be running Resq, Sidekiq, and, you know,\NRedis
Dialogue: 0,0:24:01.37,0:24:05.66,Default,,0000,0000,0000,,is, again, one of those just pieces of technology
Dialogue: 0,0:24:05.66,0:24:12.22,Default,,0000,0000,0000,,that is beautifully engineered, incredibly\Nsimple, incredibly robust. The
Dialogue: 0,0:24:12.22,0:24:19.22,Default,,0000,0000,0000,,maintainers are just absolute, you know, scientists,\NI guess.
Dialogue: 0,0:24:19.31,0:24:22.100,Default,,0000,0000,0000,,Just a whole other level of crazy algorithm\Nstuff.
Dialogue: 0,0:24:22.100,0:24:25.30,Default,,0000,0000,0000,,And they make blog posts and, you know, I'm
Dialogue: 0,0:24:25.30,0:24:31.52,Default,,0000,0000,0000,,so stupid. I don't understand what you're\Ntalking about.
Dialogue: 0,0:24:31.52,0:24:35.99,Default,,0000,0000,0000,,It's really fast, it's slightly hard to distribute.\NA
Dialogue: 0,0:24:35.99,0:24:38.71,Default,,0000,0000,0000,,lot of that's in the pipeline with Redis.\NIt's
Dialogue: 0,0:24:38.71,0:24:42.38,Default,,0000,0000,0000,,much more, it's much more simple to, to stick
Dialogue: 0,0:24:42.38,0:24:46.07,Default,,0000,0000,0000,,it on one node and increase the RAM. It's
Dialogue: 0,0:24:46.07,0:24:49.36,Default,,0000,0000,0000,,mu, more complicated then Memcache. It's essentially\Njust an
Dialogue: 0,0:24:49.36,0:24:52.13,Default,,0000,0000,0000,,in-memory cache. It has a bunch of really\Ninteresting
Dialogue: 0,0:24:52.13,0:24:56.68,Default,,0000,0000,0000,,data structures, though. I think if you've\Nbeen confused
Dialogue: 0,0:24:56.68,0:24:59.03,Default,,0000,0000,0000,,all week, now, which country I'm from, whether\NI
Dialogue: 0,0:24:59.03,0:25:01.72,Default,,0000,0000,0000,,say dayta or dahta, so now I just changed
Dialogue: 0,0:25:01.72,0:25:03.71,Default,,0000,0000,0000,,them randomly.
Dialogue: 0,0:25:03.71,0:25:08.07,Default,,0000,0000,0000,,So, you can, you have hashes you have lists,
Dialogue: 0,0:25:08.07,0:25:09.78,Default,,0000,0000,0000,,you have strings. You've got all sorts of\Nother
Dialogue: 0,0:25:09.78,0:25:14.13,Default,,0000,0000,0000,,interesting things. You can do optimistic\Nlocking and have,
Dialogue: 0,0:25:14.13,0:25:17.61,Default,,0000,0000,0000,,you know, a bunch of operations that are essentially
Dialogue: 0,0:25:17.61,0:25:22.37,Default,,0000,0000,0000,,batched. You can do sort of, there's long\Nways
Dialogue: 0,0:25:22.37,0:25:25.44,Default,,0000,0000,0000,,of doing this kind of stuff. It's Resque and
Dialogue: 0,0:25:25.44,0:25:28.69,Default,,0000,0000,0000,,Sidekiq both just make this, make it super\Nsimple
Dialogue: 0,0:25:28.69,0:25:31.14,Default,,0000,0000,0000,,to do background tasks with Rails and install\Nthe
Dialogue: 0,0:25:31.14,0:25:36.96,Default,,0000,0000,0000,,gem, have a worker, and it's all just magic.
Dialogue: 0,0:25:36.96,0:25:39.77,Default,,0000,0000,0000,,It is Lua baked in, which is a whole
Dialogue: 0,0:25:39.77,0:25:41.85,Default,,0000,0000,0000,,other thing. But Lua is a really cool programming
Dialogue: 0,0:25:41.85,0:25:44.94,Default,,0000,0000,0000,,language that is designed for embeddability.\NBut one of
Dialogue: 0,0:25:44.94,0:25:47.21,Default,,0000,0000,0000,,the things that happens if you can actually\Nwrite
Dialogue: 0,0:25:47.21,0:25:51.39,Default,,0000,0000,0000,,little rule, Lua scripts that end up going\Ninto
Dialogue: 0,0:25:51.39,0:25:54.52,Default,,0000,0000,0000,,the Redis server to do more complex operations.\NSo,
Dialogue: 0,0:25:54.52,0:25:57.18,Default,,0000,0000,0000,,in this case, this is a little script that
Dialogue: 0,0:25:57.18,0:26:00.27,Default,,0000,0000,0000,,grabs something off a sorted hash and then\Ndeletes
Dialogue: 0,0:26:00.27,0:26:02.79,Default,,0000,0000,0000,,them and then returns the first thing, like,\Nthen
Dialogue: 0,0:26:02.79,0:26:05.79,Default,,0000,0000,0000,,returns what we had done. But it's, it's an
Dialogue: 0,0:26:05.79,0:26:09.53,Default,,0000,0000,0000,,atomic kind of transactional way.
Dialogue: 0,0:26:09.53,0:26:13.32,Default,,0000,0000,0000,,And, good news everybody! We've just invented\Nstored procedures.
Dialogue: 0,0:26:13.32,0:26:16.41,Default,,0000,0000,0000,,So that's very exciting. Except now they're\Nmuch more
Dialogue: 0,0:26:16.41,0:26:18.64,Default,,0000,0000,0000,,hip, because it's an in-memory database with\Na language
Dialogue: 0,0:26:18.64,0:26:23.33,Default,,0000,0000,0000,,no one's heard of. So. We are rocking it.
Dialogue: 0,0:26:23.33,0:26:28.47,Default,,0000,0000,0000,,Also, maybe use a queue. Just, I know it's
Dialogue: 0,0:26:28.47,0:26:32.87,Default,,0000,0000,0000,,crazy. But, if you're actually queuing, using\NRedis as
Dialogue: 0,0:26:32.87,0:26:36.81,Default,,0000,0000,0000,,your queue, maybe you have a queuing problem\Nand
Dialogue: 0,0:26:36.81,0:26:39.61,Default,,0000,0000,0000,,you have queues. They exist. They're a thing.\NIt's
Dialogue: 0,0:26:39.61,0:26:41.44,Default,,0000,0000,0000,,ridiculous. I know.
Dialogue: 0,0:26:41.44,0:26:46.38,Default,,0000,0000,0000,,So, RabbitMQ is sort of the gold standard,\Nand
Dialogue: 0,0:26:46.38,0:26:49.13,Default,,0000,0000,0000,,Kafka is another one that was talked about\Nearlier
Dialogue: 0,0:26:49.13,0:26:50.91,Default,,0000,0000,0000,,this week, and it is crazy cool.
Dialogue: 0,0:26:50.91,0:26:56.13,Default,,0000,0000,0000,,Where am I? Man. All right. Just gonna stretch.
Dialogue: 0,0:26:56.13,0:26:58.82,Default,,0000,0000,0000,,I've lost count, so I don't know, now I'm
Dialogue: 0,0:26:58.82,0:27:02.02,Default,,0000,0000,0000,,just gonna talk faster. Cool.
Dialogue: 0,0:27:02.02,0:27:08.37,Default,,0000,0000,0000,,Neo4j is really interesting. It's a graph\Ndatabase. That's.
Dialogue: 0,0:27:08.37,0:27:13.35,Default,,0000,0000,0000,,It's slightly hard to explain. But you, the\Nway
Dialogue: 0,0:27:13.35,0:27:15.21,Default,,0000,0000,0000,,I actually think about it, we'll just jump\Nstraight
Dialogue: 0,0:27:15.21,0:27:17.46,Default,,0000,0000,0000,,to here, is it's almost but not quite entirely
Dialogue: 0,0:27:17.46,0:27:22.95,Default,,0000,0000,0000,,unlike a relational database. The difference,\Nessentially, is that
Dialogue: 0,0:27:22.95,0:27:27.41,Default,,0000,0000,0000,,it is optimize for the connections rather\Nthan aggregated
Dialogue: 0,0:27:27.41,0:27:31.71,Default,,0000,0000,0000,,data. So relational database, you, puts things\Nin, in
Dialogue: 0,0:27:31.71,0:27:33.28,Default,,0000,0000,0000,,a way where you can get a sum and
Dialogue: 0,0:27:33.28,0:27:35.18,Default,,0000,0000,0000,,a count and like, that's kind of the heritage
Dialogue: 0,0:27:35.18,0:27:37.03,Default,,0000,0000,0000,,of that kind of world view.
Dialogue: 0,0:27:37.03,0:27:40.34,Default,,0000,0000,0000,,Whereas what the Neo4j people are doing is\Nactually
Dialogue: 0,0:27:40.34,0:27:44.74,Default,,0000,0000,0000,,thinking about connections between pieces\Nof data, and for
Dialogue: 0,0:27:44.74,0:27:49.34,Default,,0000,0000,0000,,some use cases, this is actually really, really\Namazing
Dialogue: 0,0:27:49.34,0:27:52.37,Default,,0000,0000,0000,,stuff. So you have, a graph is basically a
Dialogue: 0,0:27:52.37,0:27:56.85,Default,,0000,0000,0000,,collection of nodes, and those nodes can have\Nrelationships
Dialogue: 0,0:27:56.85,0:27:59.18,Default,,0000,0000,0000,,between each other, and then a node just has
Dialogue: 0,0:27:59.18,0:28:01.33,Default,,0000,0000,0000,,properties.
Dialogue: 0,0:28:01.33,0:28:03.83,Default,,0000,0000,0000,,It's essentially an object database in a way.\NIt's
Dialogue: 0,0:28:03.83,0:28:05.64,Default,,0000,0000,0000,,like very similar to the way that we think
Dialogue: 0,0:28:05.64,0:28:08.11,Default,,0000,0000,0000,,about objects. So it has some really nice\Nproperties
Dialogue: 0,0:28:08.11,0:28:11.86,Default,,0000,0000,0000,,if you're working in a language like Ruby.\NAnd
Dialogue: 0,0:28:11.86,0:28:17.01,Default,,0000,0000,0000,,then it just does stuff that, you know, in
Dialogue: 0,0:28:17.01,0:28:19.09,Default,,0000,0000,0000,,a really intuitive way. So if we've got a
Dialogue: 0,0:28:19.09,0:28:22.16,Default,,0000,0000,0000,,graph of movies and actors, you actually define\Na
Dialogue: 0,0:28:22.16,0:28:26.46,Default,,0000,0000,0000,,relationship by name. Then an actor acts in\Na
Dialogue: 0,0:28:26.46,0:28:28.70,Default,,0000,0000,0000,,movie. And then when you were doing your queries,
Dialogue: 0,0:28:28.70,0:28:32.91,Default,,0000,0000,0000,,this is a language called Cypher, you actually,\Nthat's
Dialogue: 0,0:28:32.91,0:28:34.06,Default,,0000,0000,0000,,a first-class thing.
Dialogue: 0,0:28:34.06,0:28:36.02,Default,,0000,0000,0000,,Whereas in a relational world, you're, you're\Nusing a
Dialogue: 0,0:28:36.02,0:28:39.28,Default,,0000,0000,0000,,foreign key, which has no semantic meaning\Nat all.
Dialogue: 0,0:28:39.28,0:28:41.33,Default,,0000,0000,0000,,You, you just have to remember that, you know,
Dialogue: 0,0:28:41.33,0:28:43.02,Default,,0000,0000,0000,,an actor, you know, there's a table with an
Dialogue: 0,0:28:43.02,0:28:45.73,Default,,0000,0000,0000,,actor id, and a movie id, and we're joining
Dialogue: 0,0:28:45.73,0:28:49.92,Default,,0000,0000,0000,,across somewhere. Whereas Neo4j actually makes\Nthose relationships first
Dialogue: 0,0:28:49.92,0:28:53.36,Default,,0000,0000,0000,,class citizens. So if you've got problems\Nthat are
Dialogue: 0,0:28:53.36,0:29:00.36,Default,,0000,0000,0000,,graph problems, like social network friend\Ncloud stuff, some
Dialogue: 0,0:29:01.55,0:29:04.80,Default,,0000,0000,0000,,of that stuff, Neo4j just makes trivially\Neasy in
Dialogue: 0,0:29:04.80,0:29:06.07,Default,,0000,0000,0000,,a way that you would have had to do
Dialogue: 0,0:29:06.07,0:29:10.12,Default,,0000,0000,0000,,a recursive self-join in PostGres and hate\Nyour life
Dialogue: 0,0:29:10.12,0:29:12.50,Default,,0000,0000,0000,,and, you know.
Dialogue: 0,0:29:12.50,0:29:17.03,Default,,0000,0000,0000,,Couch is cool. I guess. Pretty much that's\Nmy
Dialogue: 0,0:29:17.03,0:29:21.03,Default,,0000,0000,0000,,opinion of it. It's really awesome. But, you\Ncan't
Dialogue: 0,0:29:21.03,0:29:25.66,Default,,0000,0000,0000,,query it. So cool.
Dialogue: 0,0:29:25.66,0:29:28.11,Default,,0000,0000,0000,,That's it. That's a slight disservice to Couch\Nbut,
Dialogue: 0,0:29:28.11,0:29:31.97,Default,,0000,0000,0000,,you know, whatever. MongoDB, as we all know,\Nit
Dialogue: 0,0:29:31.97,0:29:34.56,Default,,0000,0000,0000,,is webscale and that's excellent. If you think\Nof
Dialogue: 0,0:29:34.56,0:29:39.20,Default,,0000,0000,0000,,it as Redis for JSON, that's good. Sixty percent
Dialogue: 0,0:29:39.20,0:29:41.25,Default,,0000,0000,0000,,of the time, it works every time. Everyone's\Nfamiliar
Dialogue: 0,0:29:41.25,0:29:43.17,Default,,0000,0000,0000,,with that.
Dialogue: 0,0:29:43.17,0:29:46.93,Default,,0000,0000,0000,,So, the thing that's really, I mean, Mongo,\Nit
Dialogue: 0,0:29:46.93,0:29:50.92,Default,,0000,0000,0000,,reminds me of My, MySQL. Like, Mongo is kind
Dialogue: 0,0:29:50.92,0:29:54.32,Default,,0000,0000,0000,,of terrible, but MySQL was kind of terrible,\Ntoo.
Dialogue: 0,0:29:54.32,0:29:56.79,Default,,0000,0000,0000,,Like, when that came out, it didn't do transactions,
Dialogue: 0,0:29:56.79,0:30:00.04,Default,,0000,0000,0000,,for example, and I, I was working in enterprise-y
Dialogue: 0,0:30:00.04,0:30:04.42,Default,,0000,0000,0000,,land, and transactions are actually a thing.\NAnd, you're
Dialogue: 0,0:30:04.42,0:30:08.93,Default,,0000,0000,0000,,like, you script kiddies with your database.
Dialogue: 0,0:30:08.93,0:30:10.79,Default,,0000,0000,0000,,So Mongo feels like that, and not, you know,
Dialogue: 0,0:30:10.79,0:30:13.97,Default,,0000,0000,0000,,what we learned is, if you make something\Nthat's
Dialogue: 0,0:30:13.97,0:30:17.54,Default,,0000,0000,0000,,awesome and useful and everywhere and ubiquitous\Nand it
Dialogue: 0,0:30:17.54,0:30:20.75,Default,,0000,0000,0000,,doesn't work, you can make it work. And eventually,
Dialogue: 0,0:30:20.75,0:30:23.31,Default,,0000,0000,0000,,you know, MySQL is a real database. So Mongo
Dialogue: 0,0:30:23.31,0:30:25.47,Default,,0000,0000,0000,,feels a bit like that. It's come a massive
Dialogue: 0,0:30:25.47,0:30:30.69,Default,,0000,0000,0000,,way, right about really early on with very\Nearly
Dialogue: 0,0:30:30.69,0:30:32.31,Default,,0000,0000,0000,,versions.
Dialogue: 0,0:30:32.31,0:30:34.76,Default,,0000,0000,0000,,It stores JSON. Well sort of it. It stores
Dialogue: 0,0:30:34.76,0:30:39.71,Default,,0000,0000,0000,,BSON, anyway. That's just binary JSON basically.\NAnd it's
Dialogue: 0,0:30:39.71,0:30:42.41,Default,,0000,0000,0000,,a, it's a really beautiful model to work with
Dialogue: 0,0:30:42.41,0:30:45.13,Default,,0000,0000,0000,,in a development cycle, which is why think\Nis
Dialogue: 0,0:30:45.13,0:30:47.49,Default,,0000,0000,0000,,why there's, why there's so much appeal. You've\Njust
Dialogue: 0,0:30:47.49,0:30:50.93,Default,,0000,0000,0000,,got kind of, people treat it like an object
Dialogue: 0,0:30:50.93,0:30:53.69,Default,,0000,0000,0000,,database. You've just got an object that's\Nin there,
Dialogue: 0,0:30:53.69,0:30:55.72,Default,,0000,0000,0000,,and you can pull out objects and manipulate\Nthem
Dialogue: 0,0:30:55.72,0:30:59.86,Default,,0000,0000,0000,,and do all of this kind of crazy stuff.
Dialogue: 0,0:30:59.86,0:31:05.22,Default,,0000,0000,0000,,The people who know what they're talking about,\Nthough,
Dialogue: 0,0:31:05.22,0:31:08.45,Default,,0000,0000,0000,,with distributed systems, if the reason you're\Nusing Mongo
Dialogue: 0,0:31:08.45,0:31:10.30,Default,,0000,0000,0000,,is because you think it's a panacea for all
Dialogue: 0,0:31:10.30,0:31:13.70,Default,,0000,0000,0000,,of this, you know, we need to be webscale
Dialogue: 0,0:31:13.70,0:31:17.23,Default,,0000,0000,0000,,and do all of this kind of stuff, that
Dialogue: 0,0:31:17.23,0:31:19.40,Default,,0000,0000,0000,,is not a good reason to use it. Cause
Dialogue: 0,0:31:19.40,0:31:21.92,Default,,0000,0000,0000,,there, there's still a lot of operational\Nproblems and,
Dialogue: 0,0:31:21.92,0:31:23.74,Default,,0000,0000,0000,,and stuff going on.
Dialogue: 0,0:31:23.74,0:31:30.18,Default,,0000,0000,0000,,This, this one is interesting. It's essentially,\NRethinkDB is
Dialogue: 0,0:31:30.18,0:31:33.30,Default,,0000,0000,0000,,coming from the PostGres world view. Cause\NPostGres made,
Dialogue: 0,0:31:33.30,0:31:36.73,Default,,0000,0000,0000,,you know, MySQL was like, whatever, we'll\Nfix it.
Dialogue: 0,0:31:36.73,0:31:39.67,Default,,0000,0000,0000,,PostGres was like, we'll do it right and it,
Dialogue: 0,0:31:39.67,0:31:41.63,Default,,0000,0000,0000,,you can't use it cause it's so slow, but
Dialogue: 0,0:31:41.63,0:31:43.54,Default,,0000,0000,0000,,at least it's correct. And they took lots\Nof
Dialogue: 0,0:31:43.54,0:31:46.54,Default,,0000,0000,0000,,iterations to make it usable. So Rethink is\Nkind
Dialogue: 0,0:31:46.54,0:31:48.34,Default,,0000,0000,0000,,of that school of thought. It's like, we're\Ngonna
Dialogue: 0,0:31:48.34,0:31:50.62,Default,,0000,0000,0000,,make it all correct first, and then we'll\Nmake
Dialogue: 0,0:31:50.62,0:31:55.80,Default,,0000,0000,0000,,it usable. So it's very similar idea. JSON,\Nyou
Dialogue: 0,0:31:55.80,0:31:59.43,Default,,0000,0000,0000,,know, they're trying to make it operationally\Ngreat with
Dialogue: 0,0:31:59.43,0:32:02.98,Default,,0000,0000,0000,,automatic clustering and all this kind of\Nstuff. You
Dialogue: 0,0:32:02.98,0:32:05.15,Default,,0000,0000,0000,,know. Who knows what it is and how it's
Dialogue: 0,0:32:05.15,0:32:07.18,Default,,0000,0000,0000,,actually gonna behave in the real world. It's\Nstill
Dialogue: 0,0:32:07.18,0:32:09.16,Default,,0000,0000,0000,,a very early piece of tech.
Dialogue: 0,0:32:09.16,0:32:11.25,Default,,0000,0000,0000,,And that leads me into, there's a whole world
Dialogue: 0,0:32:11.25,0:32:15.48,Default,,0000,0000,0000,,of databases around what I'm loosely calling\Nthe commercial
Dialogue: 0,0:32:15.48,0:32:20.15,Default,,0000,0000,0000,,fringe. So Couchbase is the Couch guys and\Nsort
Dialogue: 0,0:32:20.15,0:32:24.02,Default,,0000,0000,0000,,of some commercial Memcached guys who got\Ntogether to
Dialogue: 0,0:32:24.02,0:32:28.41,Default,,0000,0000,0000,,make a hybrid something. Aerospike is, their\Nmarketing is
Dialogue: 0,0:32:28.41,0:32:31.52,Default,,0000,0000,0000,,great. That's about the best you can say about
Dialogue: 0,0:32:31.52,0:32:31.87,Default,,0000,0000,0000,,it.
Dialogue: 0,0:32:31.87,0:32:33.29,Default,,0000,0000,0000,,So there's a whole bunch of people trying\Nto
Dialogue: 0,0:32:33.29,0:32:36.80,Default,,0000,0000,0000,,solve these problems in interesting ways.\NBut all of
Dialogue: 0,0:32:36.80,0:32:40.72,Default,,0000,0000,0000,,these ones cost money and, you know, they're,\Nthe
Dialogue: 0,0:32:40.72,0:32:42.20,Default,,0000,0000,0000,,mileage varies and all of that kind of stuff.
Dialogue: 0,0:32:42.20,0:32:43.54,Default,,0000,0000,0000,,The cool thing about open sources ones is\Nyou
Dialogue: 0,0:32:43.54,0:32:45.03,Default,,0000,0000,0000,,get it and you try it and you hate
Dialogue: 0,0:32:45.03,0:32:46.57,Default,,0000,0000,0000,,it and you go back to PostGres so it's
Dialogue: 0,0:32:46.57,0:32:48.19,Default,,0000,0000,0000,,all fine.
Dialogue: 0,0:32:48.19,0:32:53.19,Default,,0000,0000,0000,,So, Hyperdex. This is my favorite. Because\Nthey have
Dialogue: 0,0:32:53.19,0:32:58.38,Default,,0000,0000,0000,,HyperSpace Hashing, and it is so cool. These\Nguys
Dialogue: 0,0:32:58.38,0:33:02.37,Default,,0000,0000,0000,,are making some really broad, amazing claims\Nabout the,
Dialogue: 0,0:33:02.37,0:33:06.55,Default,,0000,0000,0000,,the kind of things that they can do. Crazy
Dialogue: 0,0:33:06.55,0:33:08.69,Default,,0000,0000,0000,,fast. It's, it's a key-value store but it\Nwill
Dialogue: 0,0:33:08.69,0:33:11.60,Default,,0000,0000,0000,,index, you know, it's not just a key but
Dialogue: 0,0:33:11.60,0:33:14.04,Default,,0000,0000,0000,,it will index the properties of a value. So
Dialogue: 0,0:33:14.04,0:33:16.51,Default,,0000,0000,0000,,now you can do que, you know, genuine queries
Dialogue: 0,0:33:16.51,0:33:20.63,Default,,0000,0000,0000,,into the structure of objects that you're\Nstoring.
Dialogue: 0,0:33:20.63,0:33:23.50,Default,,0000,0000,0000,,They've got a whole bunch of papers around\Nwhat
Dialogue: 0,0:33:23.50,0:33:27.30,Default,,0000,0000,0000,,they're doing. So, you can read that as, who
Dialogue: 0,0:33:27.30,0:33:29.68,Default,,0000,0000,0000,,knows what it means. It maps objects to coordinates
Dialogue: 0,0:33:29.68,0:33:34.53,Default,,0000,0000,0000,,in a multi-dimensioned Euclidean space. HyperSpace.\NAnd I'm like.
Dialogue: 0,0:33:34.53,0:33:37.11,Default,,0000,0000,0000,,Take my money!
Dialogue: 0,0:33:37.11,0:33:40.99,Default,,0000,0000,0000,,And there's a, there's a picture of HyperSpace.\NAnd,
Dialogue: 0,0:33:40.99,0:33:43.66,Default,,0000,0000,0000,,like, I've read that like eight times. I don't
Dialogue: 0,0:33:43.66,0:33:49.100,Default,,0000,0000,0000,,understand what's going on. But if, it does\Nseem
Dialogue: 0,0:33:49.100,0:33:52.07,Default,,0000,0000,0000,,to be true. They're trying to solve some of
Dialogue: 0,0:33:52.07,0:33:54.72,Default,,0000,0000,0000,,these problems and, you know, they call themselves\Nlike
Dialogue: 0,0:33:54.72,0:33:59.66,Default,,0000,0000,0000,,a second generation NoSQL thing, in a similar\Nway
Dialogue: 0,0:33:59.66,0:34:01.67,Default,,0000,0000,0000,,to Google, you know, kind of taking all of
Dialogue: 0,0:34:01.67,0:34:05.04,Default,,0000,0000,0000,,this stuff and trying to push the science\Nunderneath
Dialogue: 0,0:34:05.04,0:34:06.100,Default,,0000,0000,0000,,it forward.
Dialogue: 0,0:34:06.100,0:34:09.51,Default,,0000,0000,0000,,So you can, you know, it's got a Ruby
Dialogue: 0,0:34:09.51,0:34:12.96,Default,,0000,0000,0000,,client. You can use it now. It's got, just,
Dialogue: 0,0:34:12.96,0:34:18.43,Default,,0000,0000,0000,,normal key-value. It's got atomic stuff. You\Ncan do
Dialogue: 0,0:34:18.43,0:34:22.97,Default,,0000,0000,0000,,conditional ports, so this is some code that's\Nbasically
Dialogue: 0,0:34:22.97,0:34:26.86,Default,,0000,0000,0000,,is only updating if the, only updating the\Ncurrent
Dialogue: 0,0:34:26.86,0:34:31.97,Default,,0000,0000,0000,,balance if the, updating the balance if the\Ncurrent
Dialogue: 0,0:34:31.97,0:34:34.46,Default,,0000,0000,0000,,balance is what we think it is. Otherwise\Nsome
Dialogue: 0,0:34:34.46,0:34:36.46,Default,,0000,0000,0000,,other thread has updated it.
Dialogue: 0,0:34:36.46,0:34:38.89,Default,,0000,0000,0000,,So there's some really interesting stuff they\Ncan do.
Dialogue: 0,0:34:38.89,0:34:43.65,Default,,0000,0000,0000,,And they're guaranteeing those operations\Nacross the cluster. And
Dialogue: 0,0:34:43.65,0:34:45.62,Default,,0000,0000,0000,,it's also got a transactional engine as well,\Nso
Dialogue: 0,0:34:45.62,0:34:47.25,Default,,0000,0000,0000,,that's really exciting.
Dialogue: 0,0:34:47.25,0:34:51.61,Default,,0000,0000,0000,,Running out of time. HBase and Hadoop. You\Ndon't
Dialogue: 0,0:34:51.61,0:34:54.68,Default,,0000,0000,0000,,have any of these problems. Don't worry about\Nit.
Dialogue: 0,0:34:54.68,0:34:56.22,Default,,0000,0000,0000,,You probably don't want to have any of these
Dialogue: 0,0:34:56.22,0:34:59.84,Default,,0000,0000,0000,,problems. Cause this just ends up, you need\Nto
Dialogue: 0,0:34:59.84,0:35:03.87,Default,,0000,0000,0000,,install every fucking thing the Apache foundation\Nhas ever
Dialogue: 0,0:35:03.87,0:35:08.24,Default,,0000,0000,0000,,made. And this isn't even the full list. This
Dialogue: 0,0:35:08.24,0:35:09.98,Default,,0000,0000,0000,,is like, you probably need those.
Dialogue: 0,0:35:09.98,0:35:12.62,Default,,0000,0000,0000,,I have a friend, he's a bit of a
Dialogue: 0,0:35:12.62,0:35:16.87,Default,,0000,0000,0000,,dick, and he, he calls it, cause he, he
Dialogue: 0,0:35:16.87,0:35:19.71,Default,,0000,0000,0000,,works in an actual big data organization,\Nand he
Dialogue: 0,0:35:19.71,0:35:21.63,Default,,0000,0000,0000,,just, he goes, oh, you people with your small
Dialogue: 0,0:35:21.63,0:35:25.97,Default,,0000,0000,0000,,to medium data. So, yeah, like, most of us,
Dialogue: 0,0:35:25.97,0:35:27.63,Default,,0000,0000,0000,,we don't have big data in any sense of
Dialogue: 0,0:35:27.63,0:35:31.06,Default,,0000,0000,0000,,the word, really. Like, if, if it's got GB
Dialogue: 0,0:35:31.06,0:35:34.60,Default,,0000,0000,0000,,on the end of it, meh. You're not there
Dialogue: 0,0:35:34.60,0:35:35.53,Default,,0000,0000,0000,,yet.
Dialogue: 0,0:35:35.53,0:35:40.73,Default,,0000,0000,0000,,So, again, this is just you know, Facebook\Nis
Dialogue: 0,0:35:40.73,0:35:42.27,Default,,0000,0000,0000,,using the hell out of this stuff, and they're
Dialogue: 0,0:35:42.27,0:35:44.86,Default,,0000,0000,0000,,just like, this is all out of date. They're
Dialogue: 0,0:35:44.86,0:35:49.59,Default,,0000,0000,0000,,like now just, they can't buy hard disks fast
Dialogue: 0,0:35:49.59,0:35:53.93,Default,,0000,0000,0000,,enough. It's crazy. Yeah. There was a punch\Nline
Dialogue: 0,0:35:53.93,0:35:56.38,Default,,0000,0000,0000,,at the end of all of that.
Dialogue: 0,0:35:56.38,0:35:57.92,Default,,0000,0000,0000,,But my friend, the guy who I said was
Dialogue: 0,0:35:57.92,0:36:00.96,Default,,0000,0000,0000,,a bit of a dick, he, he recommends having
Dialogue: 0,0:36:00.96,0:36:04.23,Default,,0000,0000,0000,,a look at this. And this is his quote,
Dialogue: 0,0:36:04.23,0:36:07.09,Default,,0000,0000,0000,,if you want to appear really cool and underground,
Dialogue: 0,0:36:07.09,0:36:09.14,Default,,0000,0000,0000,,then I reckon the next big thing is the
Dialogue: 0,0:36:09.14,0:36:12.28,Default,,0000,0000,0000,,Berkeley Data Analytics Stack. So, there's\Na whole bunch
Dialogue: 0,0:36:12.28,0:36:15.58,Default,,0000,0000,0000,,of people who are looking at that, you know,
Dialogue: 0,0:36:15.58,0:36:18.18,Default,,0000,0000,0000,,crazy big data situation and trying to work\Nout
Dialogue: 0,0:36:18.18,0:36:22.21,Default,,0000,0000,0000,,what that means and what the future is.
Dialogue: 0,0:36:22.21,0:36:24.80,Default,,0000,0000,0000,,And so Apache and Berkeley are kind of in
Dialogue: 0,0:36:24.80,0:36:26.94,Default,,0000,0000,0000,,a cold war for that at the moment. And
Dialogue: 0,0:36:26.94,0:36:29.14,Default,,0000,0000,0000,,then there's heaps of people in the enterprise\Nspace
Dialogue: 0,0:36:29.14,0:36:31.85,Default,,0000,0000,0000,,because you can sell lots of products and\Nor
Dialogue: 0,0:36:31.85,0:36:34.59,Default,,0000,0000,0000,,services to large companies who think they\Nhave a
Dialogue: 0,0:36:34.59,0:36:37.71,Default,,0000,0000,0000,,big data problem. So that's cool.
Dialogue: 0,0:36:37.71,0:36:39.65,Default,,0000,0000,0000,,That's fine. This isn't, this is just a little
Dialogue: 0,0:36:39.65,0:36:44.99,Default,,0000,0000,0000,,thing that's an embeddable document key-value\Nstore that you
Dialogue: 0,0:36:44.99,0:36:47.43,Default,,0000,0000,0000,,can, it's just kind of a fun team and
Dialogue: 0,0:36:47.43,0:36:49.21,Default,,0000,0000,0000,,has an API that looks very similar to the
Dialogue: 0,0:36:49.21,0:36:52.52,Default,,0000,0000,0000,,Mongo one. And it just sits in process.
Dialogue: 0,0:36:52.52,0:36:56.21,Default,,0000,0000,0000,,Oh, ElasticSearch. Every time I use it, I\Nthink,
Dialogue: 0,0:36:56.21,0:37:01.40,Default,,0000,0000,0000,,why can you not be my database? It's awesome.
Dialogue: 0,0:37:01.40,0:37:03.37,Default,,0000,0000,0000,,But it loses a couple of points there because
Dialogue: 0,0:37:03.37,0:37:08.92,Default,,0000,0000,0000,,of its configurationability. It went, it works\Nwhen you
Dialogue: 0,0:37:08.92,0:37:10.83,Default,,0000,0000,0000,,know how to make it works, and it's crazy
Dialogue: 0,0:37:10.83,0:37:12.68,Default,,0000,0000,0000,,complicated sometimes.
Dialogue: 0,0:37:12.68,0:37:19.64,Default,,0000,0000,0000,,So anyway. Thirty. Four minutes over technically,\NI think.
Dialogue: 0,0:37:19.64,0:37:21.95,Default,,0000,0000,0000,,Yeah. So that's good.
Dialogue: 0,0:37:21.95,0:37:28.95,Default,,0000,0000,0000,,That's databases in a nutshell. I'm Toby Hede.\NI'm
Dialogue: 0,0:37:29.16,0:37:31.28,Default,,0000,0000,0000,,around the conference if you want to talk\Nabout
Dialogue: 0,0:37:31.28,0:37:35.34,Default,,0000,0000,0000,,databases. I think of myself as a lapa-, a
Dialogue: 0,0:37:35.34,0:37:39.32,Default,,0000,0000,0000,,lap- a butterfly collector, I guess, is what\NI'm
Dialogue: 0,0:37:39.32,0:37:41.20,Default,,0000,0000,0000,,looking for, of databases.
Dialogue: 0,0:37:41.20,0:37:45.96,Default,,0000,0000,0000,,Yeah. So come and say hi. Cool.