9:59:59.000,9:59:59.000
My colleagues and I are fascinated[br]by the science of moving dots.

9:59:59.000,9:59:59.000
So what are these dots?

9:59:59.000,9:59:59.000
Well, it's all of us.

9:59:59.000,9:59:59.000
And we're moving in our homes,[br]in our offices, as we shop and travel

9:59:59.000,9:59:59.000
throughout our cities [br]and around the world.

9:59:59.000,9:59:59.000
And wouldn't it be great if[br]we could understand all this movement?

9:59:59.000,9:59:59.000
If we could find patterns and meaning[br]and insight in it.

9:59:59.000,9:59:59.000
And luckily for us, we live in a time[br]where we're incredibly good

9:59:59.000,9:59:59.000
at capturing information about ourselves.

9:59:59.000,9:59:59.000
So whether it's through [br]sensor,s or videos, or apps,

9:59:59.000,9:59:59.000
we can track our movement[br]with incredibly fine detail.

9:59:59.000,9:59:59.000
So it turns out one of the places[br]where we have the best data

9:59:59.000,9:59:59.000
about movement is sports.

9:59:59.000,9:59:59.000
So whether it's basketball or baseball,[br]or football or the other football,

9:59:59.000,9:59:59.000
we're instrumenting our stadiums [br]and our players to track their movements

9:59:59.000,9:59:59.000
every fraction of a second.

9:59:59.000,9:59:59.000
So what we're doing is turning our [br]athletes into -- you probably guessed it

9:59:59.000,9:59:59.000
moving dots.

9:59:59.000,9:59:59.000
So we've got mountains of moving dots[br]and like most raw data,

9:59:59.000,9:59:59.000
it's hard to deal with [br]and not that interesting.

9:59:59.000,9:59:59.000
But there are things that -- for example [br]basketball coaches want to know.

9:59:59.000,9:59:59.000
And the problem is they can't know them[br]because they'd have to watch every second

9:59:59.000,9:59:59.000
of every game, remember it [br]and process it.

9:59:59.000,9:59:59.000
And a person can't do that...[br]but a machine can.

9:59:59.000,9:59:59.000
The problem is a machine can't see[br]the game with the eye of a coach.

9:59:59.000,9:59:59.000
At least they couldn't until now.

9:59:59.000,9:59:59.000
So what have we taught the machine to see?

9:59:59.000,9:59:59.000
So, we started simply.[br]We taught it things like passes,

9:59:59.000,9:59:59.000
shots and rebounds.

9:59:59.000,9:59:59.000
Things that most casual fans would know.[br]And then we moved on to things

9:59:59.000,9:59:59.000
slightly more complicated.

9:59:59.000,9:59:59.000
Events like post-ups,[br]and pick-and-rolls, and isolations.

9:59:59.000,9:59:59.000
And if you don't know them, that's okay.[br]Most casual players probably do.

9:59:59.000,9:59:59.000
Now, we've gotten to a point where today,[br]the machine understands complex events

9:59:59.000,9:59:59.000
like down screens and wide pins.[br]Basically things only professionals know.

9:59:59.000,9:59:59.000
So we have taught a machine to see[br]with the eyes of a coach.

9:59:59.000,9:59:59.000
So how have we been able to do this?[br]If I asked a coach to describe something

9:59:59.000,9:59:59.000
like a pick-and-roll, they would[br]give me a description and

9:59:59.000,9:59:59.000
if I encoded that as an algorithm,[br]it would be terrible.

9:59:59.000,9:59:59.000
The pick-and-roll happens to be the stance [br]in basketball between four players,

9:59:59.000,9:59:59.000
two on offense and two on defense.

9:59:59.000,9:59:59.000
And here's kind of how it goes.

9:59:59.000,9:59:59.000
So there's the guy on offense without [br]the ball and he goes next to the guy

9:59:59.000,9:59:59.000
guarding the guy with the ball,[br]and he kind of stays there

9:59:59.000,9:59:59.000
and they both move and stuff happens, [br]and ta-da, it's a pick-and-roll.

9:59:59.000,9:59:59.000
(Laughter)

9:59:59.000,9:59:59.000
So that is also an example of[br]a terrible algorithm.

9:59:59.000,9:59:59.000
So, if the player who's the interferer[br]-- he's called the screener,

9:59:59.000,9:59:59.000
you know, goes close by,[br]but he doesn't stop.

9:59:59.000,9:59:59.000
It's probably not a pick-and-roll.

9:59:59.000,9:59:59.000
Or if he does stop,[br]but he doesn't stop close enough,

9:59:59.000,9:59:59.000
it's probably not a pick-and-roll.

9:59:59.000,9:59:59.000
Or, if he does go close by [br]and he does stop but they do it

9:59:59.000,9:59:59.000
under the basket,[br]it's probably not a pick-and-roll.

9:59:59.000,9:59:59.000
Or I could be wrong.[br]They could all be pick-and-rolls.

9:59:59.000,9:59:59.000
It really depends on the exact timing,[br]the distances, the locations

9:59:59.000,9:59:59.000
and that 's what makes it hard.[br]

9:59:59.000,9:59:59.000
So, luckily with machine learning[br]we can go beyond our own ability

9:59:59.000,9:59:59.000
to describe the things we know.

9:59:59.000,9:59:59.000
So how does this work?[br]Well, it's by example.

9:59:59.000,9:59:59.000
So we go to the machine and say,[br]"Good morning, machine."

9:59:59.000,9:59:59.000
"Here are some pick-and-rolls,[br]and here are somethings that are not."

9:59:59.000,9:59:59.000
"Please find a way to tell a difference."

9:59:59.000,9:59:59.000
And the key to all of this is to find[br]features that enable it to separate.

9:59:59.000,9:59:59.000
So if I was trying to teach it [br]the difference between an apple and orange,

9:59:59.000,9:59:59.000
I might say, "Why don't you use color,[br]or shape?"

9:59:59.000,9:59:59.000
And the problem that we're solving is,[br]what are those things?

9:59:59.000,9:59:59.000
What are the key features that let a [br]computer navigate the world of moving dots?

9:59:59.000,9:59:59.000
So figuring out all these relationships[br]with relative, absolute, location,

9:59:59.000,9:59:59.000
distance, timing, velocities.

9:59:59.000,9:59:59.000
That's really the key to the science[br]of moving dots, or as we like to call it

9:59:59.000,9:59:59.000
spatiotemporal patter recognition,[br]in academic vernacular.

9:59:59.000,9:59:59.000
Because the first thing is,[br]you have to make it sound hard

9:59:59.000,9:59:59.000
and... because it is.

9:59:59.000,9:59:59.000
The key thing is for NBA coaches,[br]it's not that they want to know

9:59:59.000,9:59:59.000
whether a pick-and-roll happened or not.

9:59:59.000,9:59:59.000
It's that they want to know how it happened.

9:59:59.000,9:59:59.000
And why is it so important to them?[br]So here's a little insight.

9:59:59.000,9:59:59.000
It turns out in modern basketball, this [br]pick-and-roll is perhaps

9:59:59.000,9:59:59.000
the most important play.

9:59:59.000,9:59:59.000
And knowing how to run it,[br]and knowing how to defend it,

9:59:59.000,9:59:59.000
is basically a key to winning [br]and losing most games.

9:59:59.000,9:59:59.000
So it turns out that the stance has [br]a great many variations

9:59:59.000,9:59:59.000
and identifying the variations are really[br]the things that matter,

9:59:59.000,9:59:59.000
and that's why we need it to be[br]really, really good.

9:59:59.000,9:59:59.000
So, here's an example.[br]There's two offensive players

9:59:59.000,9:59:59.000
getting ready to do the pick-and-roll dance.

9:59:59.000,9:59:59.000
So the guy with ball can either take,[br]or he can reject.

9:59:59.000,9:59:59.000
His teammate can either roll or pop. The [br]guy guarding the ball can go over or under.

9:59:59.000,9:59:59.000
His teammate can either show [br]or play up to touch, or play soft

9:59:59.000,9:59:59.000
and together they can either[br]switch or blitz

9:59:59.000,9:59:59.000
and I didn't know most of the things[br]when I started and it would be[br]

9:59:59.000,9:59:59.000
lovely if everybody moved according to [br]those arrows.

9:59:59.000,9:59:59.000
It would make our lives a lot easier,[br]but it turns out movement is very messy.

9:59:59.000,9:59:59.000
People wiggle a lot and getting these[br]variations identified with very, very

9:59:59.000,9:59:59.000
high accuracy, precision and recall[br]is tough because that's what it takes

9:59:59.000,9:59:59.000
to get a professional coach [br]to believe in you.

9:59:59.000,9:59:59.000
And despite t