0:00:00.954,0:00:04.537
My colleagues and I are fascinated[br]by the science of moving dots.

0:00:05.114,0:00:05.857
So what are these dots?

0:00:06.101,0:00:07.412
Well, it's all of us.

0:00:07.412,0:00:12.521
And we're moving in our homes,[br]in our offices, as we shop and travel

0:00:12.521,0:00:14.587
throughout our cities [br]and around the world.

0:00:15.168,0:00:18.627
And wouldn't it be great if[br]we could understand all this movement?

0:00:19.184,0:00:21.808
If we could find patterns and meaning[br]and insight in it.

0:00:22.482,0:00:26.104
And luckily for us, we live in a time[br]where we're incredibly good

0:00:26.568,0:00:28.565
at capturing information about ourselves.

0:00:29.006,0:00:32.259
So whether it's through [br]sensors or videos, or apps,

0:00:32.582,0:00:35.391
we can track our movement[br]with incredibly fine detail.

0:00:36.274,0:00:40.607
So it turns out one of the places[br]where we have the best data

0:00:40.607,0:00:42.195
about movement is sports.

0:00:42.682,0:00:47.767
So whether it's basketball or baseball,[br]or football or the other football,

0:00:48.158,0:00:52.465
we're instrumenting our stadiums [br]and our players to track their movements

0:00:52.465,0:00:53.324
every fraction of a second.

0:00:53.928,0:00:59.802
So what we're doing is turning our [br]athletes into -- you probably guessed it

0:01:00.406,0:01:01.404
moving dots.

0:01:02.310,0:01:06.536
So we've got mountains of moving dots[br]and like most raw data,

0:01:07.047,0:01:08.788
it's hard to deal with [br]and not that interesting.

0:01:09.430,0:01:13.223
But there are things that -- for example [br]basketball coaches want to know.

0:01:13.223,0:01:16.949
And the problem is they can't know them[br]because they'd have to watch every second

0:01:16.949,0:01:19.538
of every game, remember it [br]and process it.

0:01:19.980,0:01:23.044
And a person can't do that...[br]but a machine can.

0:01:23.880,0:01:27.038
The problem is a machine can't see[br]the game with the eye of a coach.

0:01:27.572,0:01:29.824
At least they couldn't until now.

0:01:30.451,0:01:32.889
So what have we taught the machine to see?

0:01:33.214,0:01:38.020
So, we started simply.[br]We taught it things like passes,

0:01:38.020,0:01:38.949
shots and rebounds.

0:01:39.437,0:01:43.180
Things that most casual fans would know.[br]And then we moved on to things

0:01:43.180,0:01:44.313
slightly more complicated.

0:01:44.986,0:01:49.212
Events like post-ups,[br]and pick-and-rolls, and isolations.

0:01:49.491,0:01:52.695
And if you don't know them, that's okay.[br]Most casual players probably do.

0:01:54.018,0:01:58.825
Now, we've gotten to a point where today,[br]the machine understands complex events

0:01:59.080,0:02:04.722
like down screens and wide pins.[br]Basically things only professionals know.

0:02:05.001,0:02:09.389
So we have taught a machine to see[br]with the eyes of a coach.

0:02:10.156,0:02:14.637
So how have we been able to do this?[br]If I asked a coach to describe something

0:02:15.008,0:02:17.028
like a pick-and-roll, they would[br]give me a description and

0:02:17.330,0:02:20.186
if I encoded that as an algorithm,[br]it would be terrible.

0:02:21.231,0:02:25.328
The pick-and-roll happens to be the stance [br]in basketball between four players,

0:02:25.328,0:02:27.384
two on offense and two on defense.

0:02:27.605,0:02:29.126
And here's kind of how it goes.

0:02:29.344,0:02:33.282
So there's the guy on offense without [br]the ball and he goes next to the guy

0:02:33.514,0:02:35.650
guarding the guy with the ball,[br]and he kind of stays there

0:02:35.952,0:02:39.110
and they both move and stuff happens, [br]and ta-da, it's a pick-and-roll.

0:02:39.540,0:02:40.635
(Laughter)

0:02:41.779,0:02:44.287
So that is also an example of[br]a terrible algorithm.

0:02:45.100,0:02:49.117
So, if the player who's the interferer[br]-- he's called the screener,

0:02:49.278,0:02:51.926
you know, goes close by,[br]but he doesn't stop.

0:02:52.460,0:02:54.225
It's probably not a pick-and-roll.

0:02:54.736,0:02:58.242
Or if he does stop,[br]but he doesn't stop close enough,

0:02:58.660,0:03:00.494
it's probably not a pick-and-roll.

0:03:01.091,0:03:04.511
Or, if he does go close by [br]and he does stop but they do it

0:03:04.563,0:03:07.227
under the basket,[br]it's probably not a pick-and-roll.

0:03:07.462,0:03:09.526
Or I could be wrong.[br]They could all be pick-and-rolls.

0:03:09.782,0:03:14.077
It really depends on the exact timing,[br]the distances, the locations

0:03:14.356,0:03:15.958
and that 's what makes it hard.[br]

0:03:16.694,0:03:21.484
So, luckily with machine learning[br]we can go beyond our own ability

0:03:21.716,0:03:23.249
to describe the things we know.

0:03:23.314,0:03:25.594
So how does this work?[br]Well, it's by example.

0:03:25.759,0:03:28.589
So we go to the machine and say,[br]"Good morning, machine."

0:03:29.077,0:03:32.141
"Here are some pick-and-rolls,[br]and here are somethings that are not."

0:03:32.490,0:03:34.742
"Please find a way to tell a difference."

0:03:35.076,0:03:38.805
And the key to all of this is to find[br]features that enable it to separate.

0:03:38.807,0:03:41.545
So if I was trying to teach it [br]the difference between an apple and orange,

0:03:41.644,0:03:43.983
I might say, "Why don't you use color,[br]or shape?"

0:03:44.401,0:03:47.511
And the problem that we're solving is,[br]what are those things?

0:03:47.511,0:03:52.249
What are the key features that let a [br]computer navigate the world of moving dots?

0:03:52.505,0:03:57.352
So figuring out all these relationships[br]with relative, absolute, location,

0:03:57.352,0:03:59.261
distance, timing, velocities.

0:03:59.440,0:04:04.300
That's really the key to the science[br]of moving dots, or as we like to call it

0:04:04.486,0:04:07.736
spatiotemporal pattern recognition,[br]in academic vernacular.

0:04:08.108,0:04:09.826
Because the first thing is,[br]you have to make it sound hard

0:04:10.128,0:04:12.125
and... because it is.

0:04:12.682,0:04:15.274
The key thing is for NBA coaches,[br]it's not that they want to know

0:04:15.274,0:04:17.001
whether a pick-and-roll happened or not.

0:04:17.326,0:04:19.485
It's that they want to know how it happened.

0:04:19.521,0:04:22.018
And why is it so important to them?[br]So here's a little insight.

0:04:22.018,0:04:25.414
It turns out in modern basketball, this [br]pick-and-roll is perhaps

0:04:25.414,0:04:26.865
the most important play.

0:04:27.065,0:04:29.171
And knowing how to run it,[br]and knowing how to defend it,

0:04:29.171,0:04:32.348
is basically a key to winning [br]and losing most games.

0:04:32.403,0:04:36.156
So it turns out that the dance has [br]a great many variations

0:04:36.228,0:04:39.848
and identifying the variations are really[br]the things that matter,

0:04:40.080,0:04:42.077
and that's why we need it to be[br]really, really good.

0:04:42.449,0:04:45.096
So, here's an example.[br]There's two offensive players

0:04:45.258,0:04:48.230
getting ready to do the pick-and-roll dance.

0:04:48.393,0:04:51.690
So the guy with ball can either take,[br]or he can reject.

0:04:52.293,0:04:58.005
His teammate can either roll or pop. The [br]guy guarding the ball can go over or under.

0:04:58.121,0:05:02.347
His teammate can either show [br]or play up to touch, or play soft

0:05:02.626,0:05:04.925
and together they can either[br]switch or blitz

0:05:05.273,0:05:09.150
and I didn't know most of the things[br]when I started and it would be[br]

0:05:09.313,0:05:11.890
lovely if everybody moved according to [br]those arrows.

0:05:12.099,0:05:15.884
It would make our lives a lot easier,[br]but it turns out movement is very messy.

0:05:16.047,0:05:21.642
People wiggle a lot and getting these[br]variations identified with very, very

0:05:21.805,0:05:25.706
high accuracy, both in precision and recall[br]is tough because that's what it takes

0:05:25.938,0:05:27.749
to get a professional coach [br]to believe in you.

0:05:28.051,0:05:31.417
And despite all the difficulties with[br]the right spatiotemoporal features

0:05:31.603,0:05:32.834
we have been able to do that.

0:05:33.112,0:05:37.245
Coaches trust are ability of our machine [br]to identify these variations.

0:05:37.478,0:05:42.586
We're at the point where almost every [br]single contender for an NBA championship

0:05:42.772,0:05:46.951
this year is using our software, which is [br]built on a machine that understands

0:05:47.114,0:05:49.366
the moving dots of basketball.

0:05:50.039,0:05:55.124
So, not only that, we have given advice [br]that has changed strategies,

0:05:55.287,0:05:58.282
that have helped teams win [br]very important games

0:05:58.537,0:06:01.231
and it's very exciting because you have[br]coaches who've been in the league for

0:06:01.533,0:06:05.248
30 years, that are willing to take advice [br]from a machine.

0:06:05.573,0:06:08.405
And it's very exciting.[br]It's much more than the pick-and-roll.

0:06:08.614,0:06:11.285
Our computer have started with simple [br]things and learnt more and more

0:06:11.493,0:06:13.839
complex things and now it knows [br]so many things.

0:06:14.048,0:06:19.249
Frankly, I don't understand much of what[br]it does and while it's not special

0:06:19.527,0:06:22.430
to be smarter than me,[br]we were wondering,

0:06:22.685,0:06:27.027
can a machine know more than a coach?[br]Could it know more than person could know?

0:06:27.236,0:06:28.652
Turns out the answer is yes.

0:06:28.861,0:06:32.414
Coaches want players to take good shots.[br]So if I'm standing near the basket

0:06:32.692,0:06:34.527
and there's nobody near me,[br]it's a good shot.

0:06:34.759,0:06:38.822
If I'm standing far away and surrounded[br]by defenders, that's generally a bad shot.

0:06:39.101,0:06:43.977
But we never knew how good "good" was,[br]or how bad "bad" was quantitatively.

0:06:44.209,0:06:45.347
Until now.

0:06:45.974,0:06:48.621
So what we can do, again, [br]using spatiotemporal features.

0:06:48.853,0:06:53.427
We looked at every shot. We can see where [br]is the shot? What's the angle to the basket?

0:06:53.590,0:06:55.795
Where are the defenders standing?[br]What are their distances?

0:06:55.981,0:06:56.887
What are there angles?

0:06:57.096,0:07:01.716
For multiple defenders, we can look at how[br]players move and predict the shot type.

0:07:01.879,0:07:06.081
We can look at all their velocities [br]and we can build a model that predicts

0:07:06.197,0:07:10.029
what is the likelihood that this shot [br]would go in under these circumstances?

0:07:10.354,0:07:15.114
So why is this important?[br]We can take something that was shooting,

0:07:15.299,0:07:17.830
that was one thing before, and turn it [br]into two things.

0:07:18.039,0:07:20.547
The quality of the shot[br]and the quality of the shooter.

0:07:21.917,0:07:25.167
So here's a bubble chart because[br]what's TED without a bubble chart?

0:07:25.492,0:07:28.720
Those are NBA players. [br]The size is the size of the player

0:07:28.929,0:07:30.183
and the color is the position.

0:07:30.368,0:07:34.246
On the x-axis, we've the shot probability.[br]People on the left take difficult shots, [br]

0:07:34.548,0:07:36.220
on the right, they take easy shots.

0:07:36.986,0:07:40.631
On the right is their shooting ability.[br]People who are good at the top,

0:07:40.910,0:07:41.862
bad at the bottom.

0:07:42.071,0:07:45.693
So for example, if there was a player who[br]generally made 47% of their shots

0:07:45.902,0:07:47.458
that's all you knew before.

0:07:47.690,0:07:52.194
But today, I can tell you that player [br]takes shots that an average NBA player

0:07:52.450,0:07:56.095
would make 49% of the time[br]and they were 2% worse.

0:07:56.443,0:08:01.389
And the reason that's important,[br]is that there are lots of 47s out there.

0:08:01.714,0:08:06.938
And so it's really important to know[br]if the 47 that you're considering

0:08:07.101,0:08:10.816
giving 100 million dollars to,[br]is a good shooter who takes bad shots

0:08:11.187,0:08:13.951
or bad shooter who takes good shots.

0:08:15.344,0:08:18.710
Machine understanding doesn't [br]change how we look at players,

0:08:18.896,0:08:21.891
it changes how we look at the game.[br]So there was this very exciting game

0:08:22.100,0:08:23.888
a couple of years ago, in the NBA finals.

0:08:24.144,0:08:27.510
Miami was down by three, [br]there was 20 seconds left.

0:08:27.673,0:08:30.157
They were about to lose the championship.[br]A gentleman named Lebron James

0:08:30.436,0:08:33.013
came up and he took a three to tie.

0:08:33.246,0:08:35.521
He missed.[br]His teammate Chris Bosh got a rebound,

0:08:35.776,0:08:37.634
passed it to another teammate[br]named Ray Allen.

0:08:37.866,0:08:39.700
He sank a three.[br]It went into overtime.

0:08:40.002,0:08:41.790
They won the game.[br]They won the championship.

0:08:42.046,0:08:44.762
It was one of the most exciting [br]games in basketball.

0:08:45.621,0:08:48.918
And our ability to know the shot [br]probability for every player

0:08:49.151,0:08:51.728
at every second, and the likelihood [br]of them getting a rebound at every second

0:08:52.076,0:08:56.813
can illuminate this moment in a way[br]that we never could before.

0:08:57.138,0:09:03.158
Now unfortunately, I can't show you that [br]video, but for you we recreated

0:09:03.158,0:09:07.039
that moment at our weekly basketball game[br]about 3 weeks ago.

0:09:07.340,0:09:08.525
(Laughter)

0:09:10.196,0:09:12.983
And we recreated the tracking [br]that led to the insights.

0:09:13.633,0:09:17.255
So, here is us.[br]This is Chinatown in Los Angeles,

0:09:17.603,0:09:21.272
a park we play every week at and that's us[br]recreating the Ray Allen moment

0:09:21.481,0:09:23.710
and all the tracking that's associated.

0:09:24.615,0:09:28.725
So, here's the shot.[br]I'm going to show you that moment

0:09:28.957,0:09:31.627
and all the insights of that moment.

0:09:31.860,0:09:34.391
The only difference is, instead of the [br]professional players -- it's us

0:09:34.716,0:09:38.082
and instead of a professional [br]announcer, it's me.

0:09:38.222,0:09:39.801
So, bare with me.

0:09:41.472,0:09:46.023
Miami. Down three. 20 seconds left.

0:09:47.625,0:09:52.176
Jeff brings up the ball... Josh catches, [br]puts up a three!

0:10:00.651,0:10:05.806
Won't go! Rebound Noel(??), back to Daria.

0:10:11.030,0:10:14.745
Her 3-pointer -- bang![br]Tied game with five seconds left.

0:10:15.233,0:10:16.533
The crowd goes wild.

0:10:16.858,0:10:18.391
(Laughter)

0:10:18.623,0:10:19.807
That's roughly how it happened.

0:10:20.271,0:10:21.479
(Applause)

0:10:21.641,0:10:21.966
Roughly.

0:10:24.381,0:10:29.350
I'm not going to -- that moment had about[br]a 9% chance of happening in the NBA

0:10:29.629,0:10:31.672
and we know that and a great many other things.

0:10:32.067,0:10:35.735
I'm not going to tell you how many times[br]it took us to make that happen.

0:10:35.944,0:10:37.500
(Laughter)

0:10:37.709,0:10:39.590
Okay, I will![br]It was four, it was four.

0:10:40.193,0:10:41.842
Way to go Doug(??).

0:10:42.353,0:10:48.482
But the important thing about that video [br]and the insights we have for every second

0:10:48.784,0:10:50.502
of every NBA game, It's not that.

0:10:51.501,0:10:54.496
It's the fact you don't have to be a [br]professional team to track movement.

0:10:55.216,0:10:58.211
You do not have to be a professional player[br]to get insights about movement.

0:10:58.443,0:11:02.112
In fact, it doesn't even have to be about [br]sports because we're moving everywhere.

0:11:03.853,0:11:06.222
We're moving in our homes.

0:11:09.635,0:11:11.074
In our offices.

0:11:12.468,0:11:21.918
As we shop and we travel, throughout [br]our cities and around our world.

0:11:23.334,0:11:27.119
What will we know? What will we learn?[br]Perhaps, instead of identifying

0:11:27.421,0:11:31.252
pick-and-rolls, a machine can identify [br]the moment and let me know when

0:11:31.414,0:11:33.156
my daughter takes her first steps.

0:11:33.365,0:11:35.571
Which could literally be happening [br]any second now.

0:11:36.360,0:11:39.587
Perhaps we can learn to better use [br]our buildings, better plan our cities.

0:11:40.191,0:11:44.649
I believe that with the development[br]of the science of moving dots,

0:11:44.881,0:11:48.202
we will move better, we will move smarter,[br]we will move forward.

0:11:48.782,0:11:49.688
Thank you very much.

0:11:50.106,0:11:51.476
(Applause)