-
My colleagues and I are fascinated
by the science of moving dots.
-
So what are these dots?
-
Well, it's all of us.
-
And we're moving in our homes,
in our offices, as we shop and travel
-
throughout our cities
and around the world.
-
And wouldn't it be great if
we could understand all this movement?
-
If we could find patterns and meaning
and insight in it.
-
And luckily for us, we live in a time
where we're incredibly good
-
at capturing information about ourselves.
-
So whether it's through
sensors or videos, or apps,
-
we can track our movement
with incredibly fine detail.
-
So it turns out one of the places
where we have the best data
-
about movement is sports.
-
So whether it's basketball or baseball,
or football or the other football,
-
we're instrumenting our stadiums
and our players to track their movements
-
every fraction of a second.
-
So what we're doing is turning our
athletes into -- you probably guessed it
-
moving dots.
-
So we've got mountains of moving dots
and like most raw data,
-
it's hard to deal with
and not that interesting.
-
But there are things that -- for example
basketball coaches want to know.
-
And the problem is they can't know them
because they'd have to watch every second
-
of every game, remember it
and process it.
-
And a person can't do that...
but a machine can.
-
The problem is a machine can't see
the game with the eye of a coach.
-
At least they couldn't until now.
-
So what have we taught the machine to see?
-
So, we started simply.
We taught it things like passes,
-
shots and rebounds.
-
Things that most casual fans would know.
And then we moved on to things
-
slightly more complicated.
-
Events like post-ups,
and pick-and-rolls, and isolations.
-
And if you don't know them, that's okay.
Most casual players probably do.
-
Now, we've gotten to a point where today,
the machine understands complex events
-
like down screens and wide pins.
Basically things only professionals know.
-
So we have taught a machine to see
with the eyes of a coach.
-
So how have we been able to do this?
If I asked a coach to describe something
-
like a pick-and-roll, they would
give me a description and
-
if I encoded that as an algorithm,
it would be terrible.
-
The pick-and-roll happens to be the stance
in basketball between four players,
-
two on offense and two on defense.
-
And here's kind of how it goes.
-
So there's the guy on offense without
the ball and he goes next to the guy
-
guarding the guy with the ball,
and he kind of stays there
-
and they both move and stuff happens,
and ta-da, it's a pick-and-roll.
-
(Laughter)
-
So that is also an example of
a terrible algorithm.
-
So, if the player who's the interferer
-- he's called the screener,
-
you know, goes close by,
but he doesn't stop.
-
It's probably not a pick-and-roll.
-
Or if he does stop,
but he doesn't stop close enough,
-
it's probably not a pick-and-roll.
-
Or, if he does go close by
and he does stop but they do it
-
under the basket,
it's probably not a pick-and-roll.
-
Or I could be wrong.
They could all be pick-and-rolls.
-
It really depends on the exact timing,
the distances, the locations
-
and that 's what makes it hard.
-
So, luckily with machine learning
we can go beyond our own ability
-
to describe the things we know.
-
So how does this work?
Well, it's by example.
-
So we go to the machine and say,
"Good morning, machine."
-
"Here are some pick-and-rolls,
and here are somethings that are not."
-
"Please find a way to tell a difference."
-
And the key to all of this is to find
features that enable it to separate.
-
So if I was trying to teach it
the difference between an apple and orange,
-
I might say, "Why don't you use color,
or shape?"
-
And the problem that we're solving is,
what are those things?
-
What are the key features that let a
computer navigate the world of moving dots?
-
So figuring out all these relationships
with relative, absolute, location,
-
distance, timing, velocities.
-
That's really the key to the science
of moving dots, or as we like to call it
-
spatiotemporal pattern recognition,
in academic vernacular.
-
Because the first thing is,
you have to make it sound hard
-
and... because it is.
-
The key thing is for NBA coaches,
it's not that they want to know
-
whether a pick-and-roll happened or not.
-
It's that they want to know how it happened.
-
And why is it so important to them?
So here's a little insight.
-
It turns out in modern basketball, this
pick-and-roll is perhaps
-
the most important play.
-
And knowing how to run it,
and knowing how to defend it,
-
is basically a key to winning
and losing most games.
-
So it turns out that the dance has
a great many variations
-
and identifying the variations are really
the things that matter,
-
and that's why we need it to be
really, really good.
-
So, here's an example.
There's two offensive players
-
getting ready to do the pick-and-roll dance.
-
So the guy with ball can either take,
or he can reject.
-
His teammate can either roll or pop. The
guy guarding the ball can go over or under.
-
His teammate can either show
or play up to touch, or play soft
-
and together they can either
switch or blitz
-
and I didn't know most of the things
when I started and it would be
-
lovely if everybody moved according to
those arrows.
-
It would make our lives a lot easier,
but it turns out movement is very messy.
-
People wiggle a lot and getting these
variations identified with very, very
-
high accuracy, both in precision and recall
is tough because that's what it takes
-
to get a professional coach
to believe in you.
-
And despite all the difficulties with
the right spatiotemoporal features
-
we have been able to do that.
-
Coaches trust are ability of our machine
to identify these variations.
-
We're at the point where almost every
single contender for an NBA championship
-
this year is using our software, which is
built on a machine that understands
-
the moving dots of basketball.
-
So, not only that, we have given advice
that has changed strategies,
-
that have helped teams win
very important games
-
and it's very exciting because you have
coaches who've been in the league for
-
30 years, that are willing to take advice
from a machine.
-
And it's very exciting.
It's much more than the pick-and-roll.
-
Our computer have started with simple
things and learnt more and more
-
complex things and now it knows
so many things.
-
Frankly, I don't understand much of what
it does and while it's not special
-
to be smarter than me,
we were wondering,
-
can a machine know more than a coach?
Could it know more than person could know?
-
Turns out the answer is yes.
-
Coaches want players to take good shots.
So if I'm standing near the basket
-
and there's nobody near me,
it's a good shot.
-
If I'm standing far away and surrounded
by defenders, that's generally a bad shot.
-
But we never knew how good "good" was,
or how bad "bad" was quantitatively.
-
Until now.
-
So what we can do, again,
using spatiotemporal features.
-
We looked at every shot. We can see where
is the shot? What's the angle to the basket?
-
Where are the defenders standing?
What are their distances?
-
What are there angles?
-
For multiple defenders, we can look at how
players move and predict the shot type.
-
We can look at all their velocities
and we can build a model that predicts
-
what is the likelihood that this shot
would go in under these circumstances?
-
So why is this important?
We can take something that was shooting,
-
that was one thing before, and turn it
into two things.
-
The quality of the shot
and the quality of the shooter.
-
So here's a bubble chart because
what's TED without a bubble chart?
-
Those are NBA players.
The size is the size of the player
-
and the color is the position.
-
On the x-axis, we've the shot probability.
People on the left take difficult shots,
-
on the right, they take easy shots.
-
On the right is their shooting ability.
People who are good at the top,
-
bad at the bottom.
-
So for example, if there was a player who
generally made 47% of their shots
-
that's all you knew before.
-
But today, I can tell you that player
takes shots that an average NBA player
-
would make 49% of the time
and they were 2% worse.
-
And the reason that's important,
is that there are lots of 47s out there.
-
And so it's really important to know
if the 47 that you're considering
-
giving 100 million dollars to,
is a good shooter who takes bad shots
-
or bad shooter who takes good shots.
-
Machine understanding doesn't
change how we look at players,
-
it changes how we look at the game.
So there was this very exciting game
-
a couple of years ago, in the NBA finals.
-
Miami was down by three,
there was 20 seconds left.
-
They were about to lose the championship.
A gentleman named Lebron James
-
came up and he took a three to tie.
-
He missed.
His teammate Chris Bosh got a rebound,
-
passed it to another teammate
named Ray Allen.
-
He sank a three.
It went into overtime.
-
They won the game.
They won the championship.
-
It was one of the most exciting
games in basketball.
-
And our ability to know the shot
probability for every player
-
at every second, and the likelihood
of them getting a rebound at every second
-
can illuminate this moment in a way
that we never could before.
-
Now unfortunately, I can't show you that
video, but for you we recreated
-
that moment at our weekly basketball game
about 3 weeks ago.
-
(Laughter)
-
And we recreated the tracking
that led to the insights.
-
So, here is us.
This is Chinatown in Los Angeles,
-
a park we play every week at and that's us
recreating the Ray Allen moment
-
and all the tracking that's associated.
-
So, here's the shot.
I'm going to show you that moment
-
and all the insights of that moment.
-
The only difference is, instead of the
professional players -- it's us
-
and instead of a professional
announcer, it's me.
-
So, bare with me.
-
Miami. Down three. 20 seconds left.
-
Jeff brings up the ball... Josh catches,
puts up a three!
-
Won't go! Rebound Noel(??), back to Daria.
-
Her 3-pointer -- bang!
Tied game with five seconds left.
-
The crowd goes wild.
-
(Laughter)
-
That's roughly how it happened.
-
(Applause)
-
Roughly.
-
I'm not going to -- that moment had about
a 9% chance of happening in the NBA
-
and we know that and a great many other things.
-
I'm not going to tell you how many times
it took us to make that happen.
-
(Laughter)
-
Okay, I will!
It was four, it was four.
-
Way to go Doug(??).
-
But the important thing about that video
and the insights we have for every second
-
of every NBA game, It's not that.
-
It's the fact you don't have to be a
professional team to track movement.
-
You do not have to be a professional player
to get insights about movement.
-
In fact, it doesn't even have to be about
sports because we're moving everywhere.
-
We're moving in our homes.
-
In our offices.
-
As we shop and we travel, throughout
our cities and around our world.
-
What will we know? What will we learn?
Perhaps, instead of identifying
-
pick-and-rolls, a machine can identify
the moment and let me know when
-
my daughter takes her first steps.
-
Which could literally be happening
any second now.
-
Perhaps we can learn to better use
our buildings, better plan our cities.
-
I believe that with the development
of the science of moving dots,
-
we will move better, we will move smarter,
we will move forward.
-
Thank you very much.
-
(Applause)