WEBVTT

00:00:00.954 --> 00:00:04.537
My colleagues and I are fascinated
by the science of moving dots.

00:00:05.114 --> 00:00:05.857
So what are these dots?

00:00:06.101 --> 00:00:07.412
Well, it's all of us.

00:00:07.412 --> 00:00:12.521
And we're moving in our homes,
in our offices, as we shop and travel

00:00:12.521 --> 00:00:14.587
throughout our cities 
and around the world.

00:00:15.168 --> 00:00:18.627
And wouldn't it be great if
we could understand all this movement?

00:00:19.184 --> 00:00:21.808
If we could find patterns and meaning
and insight in it.

00:00:22.482 --> 00:00:26.104
And luckily for us, we live in a time
where we're incredibly good

00:00:26.568 --> 00:00:28.565
at capturing information about ourselves.

00:00:29.006 --> 00:00:32.259
So whether it's through 
sensors or videos, or apps,

00:00:32.582 --> 00:00:35.391
we can track our movement
with incredibly fine detail.

00:00:36.274 --> 00:00:40.607
So it turns out one of the places
where we have the best data

00:00:40.607 --> 00:00:42.195
about movement is sports.

00:00:42.682 --> 00:00:47.767
So whether it's basketball or baseball,
or football or the other football,

00:00:48.158 --> 00:00:52.465
we're instrumenting our stadiums 
and our players to track their movements

00:00:52.465 --> 00:00:53.324
every fraction of a second.

00:00:53.928 --> 00:00:59.802
So what we're doing is turning our 
athletes into -- you probably guessed it

00:01:00.406 --> 00:01:01.404
moving dots.

00:01:02.310 --> 00:01:06.536
So we've got mountains of moving dots
and like most raw data,

00:01:07.047 --> 00:01:08.788
it's hard to deal with 
and not that interesting.

00:01:09.430 --> 00:01:13.223
But there are things that -- for example 
basketball coaches want to know.

00:01:13.223 --> 00:01:16.949
And the problem is they can't know them
because they'd have to watch every second

00:01:16.949 --> 00:01:19.538
of every game, remember it 
and process it.

00:01:19.980 --> 00:01:23.044
And a person can't do that...
but a machine can.

00:01:23.880 --> 00:01:27.038
The problem is a machine can't see
the game with the eye of a coach.

00:01:27.572 --> 00:01:29.824
At least they couldn't until now.

00:01:30.451 --> 00:01:32.889
So what have we taught the machine to see?

00:01:33.214 --> 00:01:38.020
So, we started simply.
We taught it things like passes,

00:01:38.020 --> 00:01:38.949
shots and rebounds.

00:01:39.437 --> 00:01:43.180
Things that most casual fans would know.
And then we moved on to things

00:01:43.180 --> 00:01:44.313
slightly more complicated.

00:01:44.986 --> 00:01:49.212
Events like post-ups,
and pick-and-rolls, and isolations.

00:01:49.491 --> 00:01:52.695
And if you don't know them, that's okay.
Most casual players probably do.

00:01:54.018 --> 00:01:58.825
Now, we've gotten to a point where today,
the machine understands complex events

00:01:59.080 --> 00:02:04.722
like down screens and wide pins.
Basically things only professionals know.

00:02:05.001 --> 00:02:09.389
So we have taught a machine to see
with the eyes of a coach.

00:02:10.156 --> 00:02:14.637
So how have we been able to do this?
If I asked a coach to describe something

00:02:15.008 --> 00:02:17.028
like a pick-and-roll, they would
give me a description and

00:02:17.330 --> 00:02:20.186
if I encoded that as an algorithm,
it would be terrible.

00:02:21.231 --> 00:02:25.328
The pick-and-roll happens to be the stance 
in basketball between four players,

00:02:25.328 --> 00:02:27.384
two on offense and two on defense.

00:02:27.605 --> 00:02:29.126
And here's kind of how it goes.

00:02:29.344 --> 00:02:33.282
So there's the guy on offense without 
the ball and he goes next to the guy

00:02:33.514 --> 00:02:35.650
guarding the guy with the ball,
and he kind of stays there

00:02:35.952 --> 00:02:39.110
and they both move and stuff happens, 
and ta-da, it's a pick-and-roll.

00:02:39.540 --> 00:02:40.635
(Laughter)

00:02:41.779 --> 00:02:44.287
So that is also an example of
a terrible algorithm.

00:02:45.100 --> 00:02:49.117
So, if the player who's the interferer
-- he's called the screener,

00:02:49.278 --> 00:02:51.926
you know, goes close by,
but he doesn't stop.

00:02:52.460 --> 00:02:54.225
It's probably not a pick-and-roll.

00:02:54.736 --> 00:02:58.242
Or if he does stop,
but he doesn't stop close enough,

00:02:58.660 --> 00:03:00.494
it's probably not a pick-and-roll.

00:03:01.091 --> 00:03:04.511
Or, if he does go close by 
and he does stop but they do it

00:03:04.563 --> 00:03:07.227
under the basket,
it's probably not a pick-and-roll.

00:03:07.462 --> 00:03:09.526
Or I could be wrong.
They could all be pick-and-rolls.

00:03:09.782 --> 00:03:14.077
It really depends on the exact timing,
the distances, the locations

00:03:14.356 --> 00:03:15.958
and that 's what makes it hard.


00:03:16.694 --> 00:03:21.484
So, luckily with machine learning
we can go beyond our own ability

00:03:21.716 --> 00:03:23.249
to describe the things we know.

00:03:23.314 --> 00:03:25.594
So how does this work?
Well, it's by example.

00:03:25.759 --> 00:03:28.589
So we go to the machine and say,
"Good morning, machine."

00:03:29.077 --> 00:03:32.141
"Here are some pick-and-rolls,
and here are somethings that are not."

00:03:32.490 --> 00:03:34.742
"Please find a way to tell a difference."

00:03:35.076 --> 00:03:38.805
And the key to all of this is to find
features that enable it to separate.

00:03:38.807 --> 00:03:41.545
So if I was trying to teach it 
the difference between an apple and orange,

00:03:41.644 --> 00:03:43.983
I might say, "Why don't you use color,
or shape?"

00:03:44.401 --> 00:03:47.511
And the problem that we're solving is,
what are those things?

00:03:47.511 --> 00:03:52.249
What are the key features that let a 
computer navigate the world of moving dots?

00:03:52.505 --> 00:03:57.352
So figuring out all these relationships
with relative, absolute, location,

00:03:57.352 --> 00:03:59.261
distance, timing, velocities.

00:03:59.440 --> 00:04:04.300
That's really the key to the science
of moving dots, or as we like to call it

00:04:04.486 --> 00:04:07.736
spatiotemporal pattern recognition,
in academic vernacular.

00:04:08.108 --> 00:04:09.826
Because the first thing is,
you have to make it sound hard

00:04:10.128 --> 00:04:12.125
and... because it is.

00:04:12.682 --> 00:04:15.274
The key thing is for NBA coaches,
it's not that they want to know

00:04:15.274 --> 00:04:17.001
whether a pick-and-roll happened or not.

00:04:17.326 --> 00:04:19.485
It's that they want to know how it happened.

00:04:19.521 --> 00:04:22.018
And why is it so important to them?
So here's a little insight.

00:04:22.018 --> 00:04:25.414
It turns out in modern basketball, this 
pick-and-roll is perhaps

00:04:25.414 --> 00:04:26.865
the most important play.

00:04:27.065 --> 00:04:29.171
And knowing how to run it,
and knowing how to defend it,

00:04:29.171 --> 00:04:32.348
is basically a key to winning 
and losing most games.

00:04:32.403 --> 00:04:36.156
So it turns out that this dance has 
a great many variations

00:04:36.228 --> 00:04:39.848
and identifying the variations are really
the things that matter,

00:04:39.900 --> 00:04:42.077
and that's why we need it to be
really, really good.

00:04:42.169 --> 00:04:45.096
So, here's an example.
There's two offensive players

00:04:45.258 --> 00:04:47.713
two defensive players, getting ready 
to do the pick-and-roll dance.

00:04:47.713 --> 00:04:51.690
So the guy with ball can either take,
or he can reject.

00:04:52.293 --> 00:04:58.005
His teammate can either roll or pop. The 
guy guarding the ball can go over or under.

00:04:58.121 --> 00:05:02.347
His teammate can either show 
or play up to touch, or play soft

00:05:02.626 --> 00:05:04.925
and together they can either
switch or blitz

00:05:05.273 --> 00:05:09.150
and I didn't know most of the things
when I started and it would be


00:05:09.173 --> 00:05:11.890
lovely if everybody moved according to 
those arrows.

00:05:11.979 --> 00:05:15.884
It would make our lives a lot easier,
but it turns out movement is very messy.

00:05:16.047 --> 00:05:21.555
People wiggle a lot and getting these
variations identified with very, very

00:05:21.555 --> 00:05:25.668
high accuracy, both in precision and recall
is tough because that's what it takes

00:05:25.668 --> 00:05:27.749
to get a professional coach 
to believe in you.

00:05:28.051 --> 00:05:31.203
And despite all the difficulties with
the right spatiotemoporal features

00:05:31.203 --> 00:05:32.812
we have been able to do that.

00:05:32.812 --> 00:05:37.245
Coaches trust our ability of our machine 
to identify these variations.

00:05:37.478 --> 00:05:42.182
We're at the point where almost every 
single contender for an NBA championship

00:05:42.182 --> 00:05:46.951
this year is using our software, which is 
built on a machine that understands

00:05:47.114 --> 00:05:49.366
the moving dots of basketball.

00:05:50.039 --> 00:05:55.124
So, not only that, we have given advice 
that has changed strategies,

00:05:55.287 --> 00:05:58.282
that have helped teams win 
very important games

00:05:58.537 --> 00:06:01.231
and it's very exciting because you have
coaches who've been in the league for

00:06:01.533 --> 00:06:05.248
30 years, that are willing to take advice 
from a machine.

00:06:05.573 --> 00:06:08.204
And it's very exciting.
It's much more than the pick-and-roll.

00:06:08.204 --> 00:06:11.033
Our computer have started with simple 
things and learnt more and more

00:06:11.033 --> 00:06:13.768
complex things and now it knows 
so many things.

00:06:13.768 --> 00:06:19.117
Frankly, I don't understand much of what
it does and while it's not special

00:06:19.117 --> 00:06:22.430
to be smarter than me,
we were wondering,

00:06:22.685 --> 00:06:27.027
can a machine know more than a coach?
Could it know more than person could know?

00:06:27.236 --> 00:06:28.652
Turns out the answer is yes.

00:06:28.861 --> 00:06:32.372
Coaches want players to take good shots.
So if I'm standing near the basket

00:06:32.372 --> 00:06:34.527
and there's nobody near me,
it's a good shot.

00:06:34.759 --> 00:06:38.822
If I'm standing far away and surrounded
by defenders, that's generally a bad shot.

00:06:39.101 --> 00:06:43.977
But we never knew how good "good" was,
or how bad "bad" was quantitatively.

00:06:44.209 --> 00:06:45.347
Until now.

00:06:45.974 --> 00:06:48.621
So what we can do, again, 
using spatiotemporal features.

00:06:48.853 --> 00:06:53.280
We looked at every shot. We can see where 
is the shot? What's the angle to the basket?

00:06:53.280 --> 00:06:55.741
Where are the defenders standing?
What are their distances?

00:06:55.741 --> 00:06:56.887
What are there angles?

00:06:57.096 --> 00:07:01.716
For multiple defenders, we can look at how
players move and predict the shot type.

00:07:01.879 --> 00:07:05.977
We can look at all their velocities 
and we can build a model that predicts

00:07:05.977 --> 00:07:10.029
what is the likelihood that this shot 
would go in under these circumstances?

00:07:10.354 --> 00:07:14.929
So why is this important?
We can take something that was shooting,

00:07:14.929 --> 00:07:17.830
which was one thing before, and turn it 
into two things.

00:07:18.039 --> 00:07:20.547
The quality of the shot
and the quality of the shooter.

00:07:21.917 --> 00:07:25.167
So here's a bubble chart because
what's TED without a bubble chart?

00:07:25.492 --> 00:07:28.720
Those are NBA players. 
The size is the size of the player

00:07:28.929 --> 00:07:30.183
and the color is the position.

00:07:30.278 --> 00:07:34.246
On the x-axis, we've the shot probability.
People on the left take difficult shots, 


00:07:34.548 --> 00:07:36.220
on the right, they take easy shots.

00:07:36.986 --> 00:07:40.550
On the right is their shooting ability.
People who are good at the top,

00:07:40.550 --> 00:07:41.661
bad at the bottom.

00:07:41.661 --> 00:07:45.693
So for example, if there was a player who
generally made 47% of their shots

00:07:45.902 --> 00:07:47.458
that's all you knew before.

00:07:47.690 --> 00:07:52.194
But today, I can tell you that player 
takes shots that an average NBA player

00:07:52.450 --> 00:07:56.095
would make 49% of the time
and they are 2% worse.

00:07:56.443 --> 00:08:01.389
And the reason that's important,
is that there are lots of 47s out there.

00:08:01.714 --> 00:08:06.171
And so it's really important to know
if the 47 that you're considering

00:08:06.171 --> 00:08:10.816
giving 100 million dollars to,
is a good shooter who takes bad shots

00:08:11.187 --> 00:08:13.951
or bad shooter who takes good shots.

00:08:15.344 --> 00:08:18.710
Machine understanding doesn't 
change how we look at players,

00:08:18.896 --> 00:08:21.891
it changes how we look at the game.
So there was this very exciting game

00:08:22.100 --> 00:08:23.888
a couple of years ago, in the NBA finals.

00:08:24.144 --> 00:08:27.510
Miami was down by three, 
there was 20 seconds left.

00:08:27.673 --> 00:08:30.157
They were about to lose the championship.
A gentleman named Lebron James

00:08:30.436 --> 00:08:33.013
came up and he took a three to tie.

00:08:33.246 --> 00:08:35.521
He missed.
His teammate Chris Bosh got a rebound,

00:08:35.776 --> 00:08:37.634
passed it to another teammate
named Ray Allen.

00:08:37.866 --> 00:08:39.700
He sank a three.
It went into overtime.

00:08:40.002 --> 00:08:41.790
They won the game.
They won the championship.

00:08:42.046 --> 00:08:44.762
It was one of the most exciting 
games in basketball.

00:08:45.621 --> 00:08:48.891
And our ability to know the shot 
probability for every player

00:08:48.891 --> 00:08:51.728
at every second, and the likelihood 
of them getting a rebound at every second

00:08:52.076 --> 00:08:56.813
can illuminate this moment in a way
that we never could before.

00:08:57.138 --> 00:09:03.158
Now unfortunately, I can't show you that 
video, but for you we recreated

00:09:03.158 --> 00:09:07.039
that moment at our weekly basketball game
about 3 weeks ago.

00:09:07.340 --> 00:09:08.525
(Laughter)

00:09:10.196 --> 00:09:12.983
And we recreated the tracking 
that led to the insights.

00:09:13.633 --> 00:09:17.255
So, here is us.
This is Chinatown in Los Angeles,

00:09:17.603 --> 00:09:21.272
a park we play every week at and that's us
recreating the Ray Allen moment

00:09:21.481 --> 00:09:23.710
and all the tracking that's associated.

00:09:24.615 --> 00:09:28.725
So, here's the shot.
I'm going to show you that moment

00:09:28.957 --> 00:09:31.627
and all the insights of that moment.

00:09:31.860 --> 00:09:34.391
The only difference is, instead of the 
professional players -- it's us

00:09:34.716 --> 00:09:38.082
and instead of a professional 
announcer, it's me.

00:09:38.222 --> 00:09:39.801
So, bare with me.

00:09:41.472 --> 00:09:46.023
Miami. Down three. 20 seconds left.

00:09:47.625 --> 00:09:52.176
Jeff brings up the ball... Josh catches, 
puts up a three!

00:10:00.651 --> 00:10:05.806
Won't go! Rebound, Noel(??). Back to Daria.

00:10:10.300 --> 00:10:14.745
Her 3-pointer -- bang!
Tied game with five seconds left.

00:10:14.983 --> 00:10:16.533
The crowd goes wild.

00:10:16.858 --> 00:10:18.391
(Laughter)

00:10:18.623 --> 00:10:19.807
That's roughly how it happened.

00:10:20.271 --> 00:10:20.951
(Applause)

00:10:20.951 --> 00:10:21.966
Roughly.

00:10:24.121 --> 00:10:29.350
I'm not going to -- that moment had about
a 9% chance of happening in the NBA

00:10:29.629 --> 00:10:31.672
and we know that and a great many other things.

00:10:32.067 --> 00:10:35.735
I'm not going to tell you how many times
it took us to make that happen.

00:10:35.944 --> 00:10:37.019
(Laughter)

00:10:37.259 --> 00:10:39.830
Okay, I will!
It was four, it was four.

00:10:40.193 --> 00:10:41.842
Way to go Doug(??).

00:10:42.353 --> 00:10:48.084
But the important thing about that video 
and the insights we have for every second

00:10:48.084 --> 00:10:50.502
of every NBA game, it's not that.

00:10:51.091 --> 00:10:54.496
It's the fact you don't have to be a 
professional team to track movement.

00:10:55.216 --> 00:10:58.211
You do not have to be a professional player
to get insights about movement.

00:10:58.443 --> 00:11:02.112
In fact, it doesn't even have to be about 
sports because we're moving everywhere.

00:11:03.853 --> 00:11:06.222
We're moving in our homes.

00:11:09.635 --> 00:11:11.074
In our offices.

00:11:12.238 --> 00:11:21.918
As we shop and we travel, throughout 
our cities and around our world.

00:11:23.334 --> 00:11:27.111
What will we know? What will we learn?
Perhaps, instead of identifying

00:11:27.111 --> 00:11:31.252
pick-and-rolls, a machine can identify 
the moment and let me know when

00:11:31.414 --> 00:11:33.035
my daughter takes her first steps.

00:11:33.035 --> 00:11:35.571
Which could literally be happening 
any second now.

00:11:36.360 --> 00:11:39.587
Perhaps we can learn to better use 
our buildings, better plan our cities.

00:11:40.191 --> 00:11:44.649
I believe that with the development
of the science of moving dots,

00:11:44.681 --> 00:11:48.202
we will move better, we will move smarter,
we will move forward.

00:11:48.782 --> 00:11:49.688
Thank you very much.

00:11:50.106 --> 00:11:51.476
(Applause)