0:00:00.954,0:00:04.537
My colleagues and I are fascinated[br]by the science of moving dots.

0:00:04.927,0:00:06.077
So what are these dots?

0:00:06.101,0:00:07.388
Well, it's all of us.

0:00:07.412,0:00:12.497
And we're moving in our homes,[br]in our offices, as we shop and travel

0:00:12.521,0:00:14.587
throughout our cities[br]and around the world.

0:00:14.958,0:00:18.627
And wouldn't it be great[br]if we could understand all this movement?

0:00:18.918,0:00:21.808
If we could find patterns and meaning[br]and insight in it.

0:00:22.259,0:00:24.044
And luckily for us, we live in a time

0:00:24.068,0:00:28.565
where we're incredibly good[br]at capturing information about ourselves.

0:00:28.807,0:00:32.470
So whether it's through[br]sensors or videos, or apps,

0:00:32.494,0:00:35.303
we can track our movement[br]with incredibly fine detail.

0:00:36.092,0:00:41.124
So it turns out one of the places[br]where we have the best data about movement

0:00:41.148,0:00:42.356
is sports.

0:00:42.682,0:00:48.015
So whether it's basketball or baseball,[br]or football or the other football,

0:00:48.039,0:00:52.441
we're instrumenting our stadiums[br]and our players to track their movements

0:00:52.465,0:00:53.778
every fraction of a second.

0:00:53.802,0:00:58.184
So what we're doing[br]is turning our athletes into --

0:00:58.208,0:01:00.167
you probably guessed it --

0:01:00.191,0:01:01.587
moving dots.

0:01:01.946,0:01:06.880
So we've got mountains of moving dots[br]and like most raw data,

0:01:06.904,0:01:09.406
it's hard to deal with[br]and not that interesting.

0:01:09.430,0:01:13.199
But there are things that, for example,[br]basketball coaches want to know.

0:01:13.223,0:01:17.033
And the problem is they can't know them[br]because they'd have to watch every second

0:01:17.057,0:01:19.646
of every game, remember it and process it.

0:01:19.804,0:01:21.734
And a person can't do that,

0:01:21.758,0:01:23.068
but a machine can.

0:01:23.661,0:01:27.071
The problem is a machine can't see[br]the game with the eye of a coach.

0:01:27.363,0:01:29.624
At least they couldn't until now.

0:01:30.228,0:01:32.331
So what have we taught the machine to see?

0:01:33.569,0:01:35.356
So, we started simply.

0:01:35.380,0:01:39.179
We taught it things like passes,[br]shots and rebounds.

0:01:39.203,0:01:41.744
Things that most casual fans would know.

0:01:41.768,0:01:44.600
And then we moved on to things[br]slightly more complicated.

0:01:44.624,0:01:49.212
Events like post-ups,[br]and pick-and-rolls, and isolations.

0:01:49.377,0:01:52.920
And if you don't know them, that's okay.[br]Most casual players probably do.

0:01:53.560,0:01:58.900
Now, we've gotten to a point where today,[br]the machine understands complex events

0:01:58.924,0:02:01.997
like down screens and wide pins.

0:02:02.021,0:02:04.747
Basically things only professionals know.

0:02:04.771,0:02:09.159
So we have taught a machine to see[br]with the eyes of a coach.

0:02:10.009,0:02:11.866
So how have we been able to do this?

0:02:12.511,0:02:15.629
If I asked a coach to describe[br]something like a pick-and-roll,

0:02:15.653,0:02:17.293
they would give me a description,

0:02:17.317,0:02:20.173
and if I encoded that as an algorithm,[br]it would be terrible.

0:02:21.026,0:02:25.304
The pick-and-roll happens to be this dance[br]in basketball between four players,

0:02:25.328,0:02:27.240
two on offense and two on defense.

0:02:27.486,0:02:29.104
And here's kind of how it goes.

0:02:29.128,0:02:31.661
So there's the guy on offense[br]without the ball

0:02:31.685,0:02:34.894
the ball and he goes next to the guy[br]guarding the guy with the ball,

0:02:34.918,0:02:36.175
and he kind of stays there

0:02:36.199,0:02:39.516
and they both move and stuff happens,[br]and ta-da, it's a pick-and-roll.

0:02:39.540,0:02:41.755
(Laughter)

0:02:41.779,0:02:44.287
So that is also an example[br]of a terrible algorithm.

0:02:44.913,0:02:49.117
So, if the player who's the interferer --[br]he's called the screener --

0:02:49.278,0:02:52.150
goes close by, but he doesn't stop,

0:02:52.174,0:02:53.939
it's probably not a pick-and-roll.

0:02:54.560,0:02:58.505
Or if he does stop,[br]but he doesn't stop close enough,

0:02:58.529,0:03:00.290
it's probably not a pick-and-roll.

0:03:00.642,0:03:03.879
Or, if he does go close by[br]and he does stop

0:03:03.903,0:03:07.227
but they do it under the basket,[br]it's probably not a pick-and-roll.

0:03:07.462,0:03:09.986
Or I could be wrong,[br]they could all be pick-and-rolls.

0:03:10.010,0:03:14.578
It really depends on the exact timing,[br]the distances, the locations,

0:03:14.602,0:03:16.097
and that's what makes it hard.

0:03:16.579,0:03:21.523
So, luckily, with machine learning,[br]we can go beyond our own ability

0:03:21.547,0:03:23.290
to describe the things we know.

0:03:23.314,0:03:25.594
So how does this work?[br]Well, it's by example.

0:03:25.759,0:03:28.589
So we go to the machine and say,[br]"Good morning, machine.

0:03:29.077,0:03:32.436
Here are some pick-and-rolls,[br]and here are some things that are not.

0:03:32.720,0:03:34.972
Please find a way to tell the difference."

0:03:35.076,0:03:38.783
And the key to all of this is to find[br]features that enable it to separate.

0:03:38.807,0:03:40.916
So if I was going[br]to teach it the difference

0:03:40.940,0:03:42.321
between an apple and orange,

0:03:42.345,0:03:44.720
I might say, "Why don't you[br]use color or shape?"

0:03:44.744,0:03:47.687
And the problem that we're solving is,[br]what are those things?

0:03:47.711,0:03:48.958
What are the key features

0:03:48.982,0:03:52.481
that let a computer navigate[br]the world of moving dots?

0:03:52.505,0:03:57.328
So figuring out all these relationships[br]with relative and absolute location,

0:03:57.352,0:03:59.261
distance, timing, velocities --

0:03:59.440,0:04:04.368
that's really the key to the science[br]of moving dots, or as we like to call it,

0:04:04.392,0:04:07.736
spatiotemporal pattern recognition,[br]in academic vernacular.

0:04:07.925,0:04:10.823
Because the first thing is,[br]you have to make it sound hard --

0:04:10.847,0:04:12.125
because it is.

0:04:12.410,0:04:15.551
The key thing is, for NBA coaches,[br]it's not that they want to know

0:04:15.575,0:04:17.497
whether a pick-and-roll happened or not.

0:04:17.521,0:04:19.597
It's that they want to know[br]how it happened.

0:04:19.621,0:04:22.607
And why is it so important to them?[br]So here's a little insight.

0:04:22.631,0:04:24.402
It turns out in modern basketball,

0:04:24.426,0:04:26.965
this pick-and-roll is perhaps[br]the most important play.

0:04:27.065,0:04:29.685
And knowing how to run it,[br]and knowing how to defend it,

0:04:29.709,0:04:32.379
is basically a key to winning[br]and losing most games.

0:04:32.403,0:04:36.204
So it turns out that this dance[br]has a great many variations

0:04:36.228,0:04:39.876
and identifying the variations[br]is really the thing that matters,

0:04:39.900,0:04:42.429
and that's why we need this[br]to be really, really good.

0:04:43.228,0:04:44.404
So, here's an example.

0:04:44.428,0:04:46.807
There are two offensive[br]and two defensive players,

0:04:46.831,0:04:48.983
getting ready to do[br]the pick-and-roll dance.

0:04:49.007,0:04:51.690
So the guy with ball[br]can either take, or he can reject.

0:04:52.086,0:04:55.087
His teammate can either roll or pop.

0:04:55.111,0:04:58.097
The guy guarding the ball[br]can either go over or under.

0:04:58.121,0:05:02.686
His teammate can either show[br]or play up to touch, or play soft

0:05:02.710,0:05:05.328
and together they can[br]either switch or blitz

0:05:05.352,0:05:08.011
and I didn't know[br]most of these things when I started

0:05:08.035,0:05:11.955
and it would be lovely if everybody moved[br]according to those arrows.

0:05:11.979,0:05:15.884
It would make our lives a lot easier,[br]but it turns out movement is very messy.

0:05:16.047,0:05:21.531
People wiggle a lot and getting[br]these variations identified

0:05:21.555,0:05:22.858
with very high accuracy,

0:05:22.882,0:05:24.750
both in precision and recall, is tough

0:05:24.774,0:05:28.392
because that's what it takes to get[br]a professional coach to believe in you.

0:05:28.416,0:05:31.796
And despite all the difficulties[br]with the right spatiotemporal features

0:05:31.820,0:05:33.294
we have been able to do that.

0:05:33.318,0:05:37.245
Coaches trust our ability of our machine[br]to identify these variations.

0:05:37.478,0:05:41.011
We're at the point where[br]almost every single contender

0:05:41.035,0:05:42.658
for an NBA championship this year

0:05:42.682,0:05:47.090
is using our software, which is built[br]on a machine that understands

0:05:47.114,0:05:48.748
the moving dots of basketball.

0:05:49.872,0:05:55.025
So not only that, we have given advice[br]that has changed strategies

0:05:55.049,0:05:58.401
that have helped teams win[br]very important games,

0:05:58.425,0:06:02.157
and it's very exciting because you have[br]coaches who've been in the league

0:06:02.181,0:06:05.248
for 30 years that are willing to take[br]advice from a machine.

0:06:05.874,0:06:08.780
And it's very exciting,[br]it's much more than the pick-and-roll.

0:06:08.804,0:06:10.880
Our computer started out[br]with simple things

0:06:10.904,0:06:12.968
and learned more and more complex things

0:06:12.992,0:06:14.553
and now it knows so many things.

0:06:14.577,0:06:17.412
Frankly, I don't understand[br]much of what it does,

0:06:17.436,0:06:21.151
and while it's not that special[br]to be smarter than me,

0:06:21.175,0:06:24.819
we were wondering,[br]can a machine know more than a coach?

0:06:24.843,0:06:26.898
Can it know more than person could know?

0:06:26.922,0:06:28.667
And it turns out the answer is yes.

0:06:28.691,0:06:31.248
The coaches want players[br]to take good shots.

0:06:31.272,0:06:32.923
So if I'm standing near the basket

0:06:32.947,0:06:35.113
and there's nobody near me,[br]it's a good shot.

0:06:35.137,0:06:39.077
If I'm standing far away surrounded[br]by defenders, that's generally a bad shot.

0:06:39.101,0:06:43.977
But we never knew how good "good" was,[br]or how bad "bad" was quantitatively.

0:06:44.209,0:06:45.359
Until now.

0:06:45.771,0:06:48.829
So what we can do, again,[br]using spatiotemporal features,

0:06:48.853,0:06:50.227
we looked at every shot.

0:06:50.251,0:06:53.256
We can see: Where is the shot?[br]What's the angle to the basket?

0:06:53.280,0:06:56.042
Where are the defenders standing?[br]What are their distances?

0:06:56.066,0:06:57.397
What are their angles?

0:06:57.421,0:07:00.398
For multiple defenders, we can look[br]at how the player's moving

0:07:00.422,0:07:01.855
and predict the shot type.

0:07:01.879,0:07:05.953
We can look at all their velocities[br]and we can build a model that predicts

0:07:05.977,0:07:10.029
what is the likelihood that this shot[br]would go in under these circumstances?

0:07:10.188,0:07:11.688
So why is this important?

0:07:12.102,0:07:14.905
We can take something that was shooting,

0:07:14.929,0:07:17.609
which was one thing before,[br]and turn it into two things:

0:07:17.633,0:07:20.284
the quality of the shot[br]and the quality of the shooter.

0:07:21.680,0:07:24.942
So here's a bubble chart,[br]because what's TED without a bubble chart?

0:07:24.966,0:07:25.980
(Laughter)

0:07:26.004,0:07:27.315
Those are NBA players.

0:07:27.339,0:07:30.459
The size is the size of the player[br]and the color is the position.

0:07:30.483,0:07:32.615
On the x-axis,[br]we have the shot probability.

0:07:32.639,0:07:34.592
People on the left take difficult shots,

0:07:34.616,0:07:36.845
on the right, they take easy shots.

0:07:37.194,0:07:39.251
On the [y-axis] is their shooting ability.

0:07:39.275,0:07:41.837
People who are good are at the top,[br]bad at the bottom.

0:07:41.861,0:07:43.621
So for example, if there was a player

0:07:43.621,0:07:45.718
who generally made[br]47 percent of their shots,

0:07:45.718,0:07:47.107
that's all you knew before.

0:07:47.345,0:07:52.195
But today, I can tell you that player[br]takes shots that an average NBA player

0:07:52.219,0:07:54.180
would make 49 percent of the time,

0:07:54.204,0:07:55.888
and they are two percent worse.

0:07:56.266,0:08:00.781
And the reason that's important[br]is that there are lots of 47s out there.

0:08:01.714,0:08:04.263
And so it's really important to know

0:08:04.287,0:08:08.243
if the 47 that you're considering[br]giving 100 million dollars to

0:08:08.267,0:08:11.322
is a good shooter who takes bad shots

0:08:11.346,0:08:13.743
or a bad shooter who takes good shots.

0:08:15.130,0:08:18.463
Machine understanding doesn't just change[br]how we look at players,

0:08:18.487,0:08:20.345
it changes how we look at the game.

0:08:20.369,0:08:24.124
So there was this very exciting game[br]a couple of years ago, in the NBA finals.

0:08:24.148,0:08:27.355
Miami was down by three,[br]there was 20 seconds left.

0:08:27.379,0:08:29.404
They were about to lose the championship.

0:08:29.428,0:08:32.769
A gentleman named LeBron James[br]came up and he took a three to tie.

0:08:32.793,0:08:33.991
He missed.

0:08:34.015,0:08:35.852
His teammate Chris Bosh got a rebound,

0:08:35.876,0:08:38.035
passed it to another teammate[br]named Ray Allen.

0:08:38.059,0:08:39.978
He sank a three. It went into overtime.

0:08:40.002,0:08:42.098
They won the game.[br]They won the championship.

0:08:42.122,0:08:44.566
It was one of the most exciting[br]games in basketball.

0:08:45.438,0:08:48.867
And our ability to know[br]the shot probability for every player

0:08:48.891,0:08:50.079
at every second,

0:08:50.103,0:08:53.059
and the likelihood of them getting[br]a rebound at every second

0:08:53.083,0:08:56.526
can illuminate this moment in a way[br]that we never could before.

0:08:57.618,0:09:00.286
Now unfortunately,[br]I can't show you that video.

0:09:00.310,0:09:04.803
But for you, we recreated that moment

0:09:04.827,0:09:07.163
at our weekly basketball game[br]about 3 weeks ago.

0:09:07.279,0:09:09.446
(Laughter)

0:09:09.573,0:09:12.983
And we recreated the tracking[br]that led to the insights.

0:09:13.199,0:09:17.454
So, here is us.[br]This is Chinatown in Los Angeles,

0:09:17.478,0:09:19.042
a park we play at every week,

0:09:19.066,0:09:21.297
and that's us recreating[br]the Ray Allen moment

0:09:21.321,0:09:23.550
and all the tracking[br]that's associated with it.

0:09:24.772,0:09:26.289
So, here's the shot.

0:09:26.313,0:09:28.829
I'm going to show you that moment

0:09:28.853,0:09:31.440
and all the insights of that moment.

0:09:31.464,0:09:35.194
The only difference is, instead[br]of the professional players, it's us,

0:09:35.218,0:09:37.836
and instead of a professional[br]announcer, it's me.

0:09:37.860,0:09:39.337
So, bear with me.

0:09:41.153,0:09:42.303
Miami.

0:09:42.671,0:09:43.821
Down three.

0:09:44.107,0:09:45.257
Twenty seconds left.

0:09:47.385,0:09:48.583
Jeff brings up the ball.

0:09:50.656,0:09:52.191
Josh catches, puts up a three!

0:09:52.631,0:09:54.480
[Calculating shot probability]

0:09:55.278,0:09:56.428
[Shot quality]

0:09:57.048,0:09:58.833
[Rebound probability]

0:10:00.373,0:10:01.546
Won't go!

0:10:01.570,0:10:03.016
[Rebound probability]

0:10:03.777,0:10:05.033
Rebound, Noel.

0:10:05.057,0:10:06.207
Back to Daria.

0:10:06.509,0:10:09.874
[Shot quality]

0:10:10.676,0:10:12.296
Her three-pointer -- bang!

0:10:12.320,0:10:14.517
Tie game with five seconds left.

0:10:14.880,0:10:16.498
The crowd goes wild.

0:10:16.522,0:10:18.181
(Laughter)

0:10:18.205,0:10:19.752
That's roughly how it happened.

0:10:19.776,0:10:20.927
(Applause)

0:10:20.951,0:10:22.126
Roughly.

0:10:22.150,0:10:23.681
(Applause)

0:10:24.121,0:10:29.605
That moment had about a nine percent[br]chance of happening in the NBA

0:10:29.629,0:10:31.890
and we know that[br]and a great many other things.

0:10:31.914,0:10:35.405
I'm not going to tell you how many times[br]it took us to make that happen.

0:10:35.429,0:10:37.176
(Laughter)

0:10:37.200,0:10:39.072
Okay, I will! It was four.

0:10:39.096,0:10:40.097
(Laughter)

0:10:40.121,0:10:41.286
Way to go, Daria.

0:10:41.647,0:10:45.910
But the important thing about that video

0:10:45.934,0:10:50.502
and the insights we have for every second[br]of every NBA game -- it's not that.

0:10:50.639,0:10:54.568
It's the fact you don't have to be[br]a professional team to track movement.

0:10:55.083,0:10:58.740
You do not have to be a professional[br]player to get insights about movement.

0:10:58.764,0:11:02.622
In fact, it doesn't even have to be about[br]sports because we're moving everywhere.

0:11:03.654,0:11:06.023
We're moving in our homes,

0:11:09.428,0:11:10.633
in our offices,

0:11:12.238,0:11:14.928
as we shop and we travel

0:11:17.318,0:11:18.571
throughout our cities

0:11:20.065,0:11:21.683
and around our world.

0:11:23.270,0:11:25.565
What will we know? What will we learn?

0:11:25.589,0:11:27.894
Perhaps, instead of identifying[br]pick-and-rolls,

0:11:27.918,0:11:30.928
a machine can identify[br]the moment and let me know

0:11:30.952,0:11:33.011
when my daughter takes her first steps.

0:11:33.035,0:11:35.571
Which could literally be happening[br]any second now.

0:11:36.140,0:11:39.837
Perhaps we can learn to better use[br]our buildings, better plan our cities.

0:11:40.362,0:11:44.535
I believe that with the development[br]of the science of moving dots,

0:11:44.559,0:11:48.202
we will move better, we will move smarter,[br]we will move forward.

0:11:48.607,0:11:49.796
Thank you very much.

0:11:49.820,0:11:54.865
(Applause)