My colleagues and I are fascinated
by the science of moving dots.

So what are these dots?

Well, it's all of us.

And we're moving in our homes,
in our offices, as we shop and travel

throughout our cities 
and around the world.

And wouldn't it be great if
we could understand all this movement?

If we could find patterns and meaning
and insight in it.

And luckily for us, we live in a time
where we're incredibly good

at capturing information about ourselves.

So whether it's through 
sensor,s or videos, or apps,

we can track our movement
with incredibly fine detail.

So it turns out one of the places
where we have the best data

about movement is sports.

So whether it's basketball or baseball,
or football or the other football,

we're instrumenting our stadiums 
and our players to track their movements

every fraction of a second.

So what we're doing is turning our 
athletes into -- you probably guessed it

moving dots.

So we've got mountains of moving dots
and like most raw data,

it's hard to deal with 
and not that interesting.

But there are things that -- for example 
basketball coaches want to know.

And the problem is they can't know them
because they'd have to watch every second

of every game, remember it 
and process it.

And a person can't do that...
but a machine can.

The problem is a machine can't see
the game with the eye of a coach.

At least they couldn't until now.

So what have we taught the machine to see?

So, we started simply.
We taught it things like passes,

shots and rebounds.

Things that most casual fans would know.
And then we moved on to things

slightly more complicated.

Events like post-ups,
and pick-and-rolls, and isolations.

And if you don't know them, that's okay.
Most casual players probably do.

Now, we've gotten to a point where today,
the machine understands complex events

like down screens and wide pins.
Basically things only professionals know.

So we have taught a machine to see
with the eyes of a coach.

So how have we been able to do this?
If I asked a coach to describe something

like a pick-and-roll, they would
give me a description and

if I encoded that as an algorithm,
it would be terrible.

The pick-and-roll happens to be the stance 
in basketball between four players,

two on offense and two on defense.

And here's kind of how it goes.

So there's the guy on offense without 
the ball and he goes next to the guy

guarding the guy with the ball,
and he kind of stays there

and they both move and stuff happens, 
and ta-da, it's a pick-and-roll.

(Laughter)

So that is also an example of
a terrible algorithm.

So, if the player who's the interferer
-- he's called the screener,

you know, goes close by,
but he doesn't stop.

It's probably not a pick-and-roll.

Or if he does stop,
but he doesn't stop close enough,

it's probably not a pick-and-roll.

Or, if he does go close by 
and he does stop but they do it

under the basket,
it's probably not a pick-and-roll.

Or I could be wrong.
They could all be pick-and-rolls.

It really depends on the exact timing,
the distances, the locations

and that 's what makes it hard.

So, luckily with machine learning
we can go beyond our own ability

to describe the things we know.

So how does this work?
Well, it's by example.