When I was a boy,

I wanted to maximise
my impact on the world,

and I was smart enough
to realise that I am not very smart.

And that I have to build a machine

that learns to become
much smarter than myself,

such that it can solve all the problems
that I cannot solve myself,

and I can retire.

And my first publication
on that dates back 30 years: 1987.

My diploma thesis,

where I already try to solve
the grand problem of AI,

not only build a machine

that learns a little bit here,
learns a little bit there,

but also learns to improve
the learning algorithm itself.

And the way it learns, the way it learns,

and so on recursively, without any limits

except the limits of logics and physics.

And, I'm still working
on the same old thing,

and I'm still pretty much
saying the same thing,

except that now
more people are listening.

Because the learning algorithms

that we have developed
on the way to this goal,

they are now on 3.000 million smartphones.

And all of you have them in your pockets.

What you see here

are the five most valuable companies
of the Western world:

Apple, Google, Facebook,
Microsoft and Amazon.

And all of them are emphasising

that AI, artificial intelligence,

is central to what they are doing.

And all of them are using heavily
the deep learning methods

that my team has developed
since the early nineties,

in Munich and in Switzerland.

Especially something which is called:
"the long short-term memory".

Has anybody in this room ever heard
of the long short-term memory,

or the LSTM?

Hands up, anybody ever heard of that?

Okay.

Has anybody never heard of the LSTM?

Okay.
I see we have a third group in this room:

[those] who didn't
understand the question.

(Laughter)

The LSTM is a little bit like your brain:

it's an artificial neural network
which also has neurons,

and in your brain, you've got
about 100 billion neurons.

And each of them is connected

to roughly 10,000
other neurons on average,

Which means that you have got
a million billion connections.

And each of these connections
has a "strength"

which says how much
does this neuron over here

influence that one over there
at the next time step.

And in the beginning,

all these connections are random
and the system knows nothing;

but then, through a smart
learning algorithm,

it learns from lots of examples
to translate the incoming data,

such as video through the cameras,
or audio through the microphones,

or pain signals through the pain sensors.

It learns to translate that
into output actions,

because some of these neurons
are output neurons,

that control speech muscles
and finger muscles.

And only through experience,

it can learn to solve
all kinds of interesting problems,

such as driving a car

or do the speech recognition
on your smartphone.

Because whenever you take out
your smartphone,

an Android phone, for example,

and you speak to it, and you say:

"Ok Google, show me
the shortest way to Milano."

Then it understands your speech.

Because there is a LSTM in there
which has learned to understand speech.

Every ten milliseconds,
100 times a second,

new inputs are coming from the microphone,

and then are translated, after thinking,

into letters

which are then questioned
to the search engine.

And it has learned to do that

by listening to lots of speech
from women, from men, all kinds of people.

And that's how, since 2015,

Google speech recognition
is now much better than it used to be.

The basic LSTM cell looks like that:

I don't have the time to explain that,

but at least I can list the names

of the brilliant students in my lab
who made that possible.

And what are the big companies
doing with that?

Well, speech recognition
is only one example;

if you are on Facebook -
is anybody on Facebook?

Are you sometimes clicking
at the translate button?

because somebody sent you something
in a foreign language

and then you can translate it.

Is anybody doing that? Yeah.

Whenever you do that,

you are waking up, again,
a long short term memory, an LSTM,

which has learned to translate
text in one language

into translated text.

And Facebook is doing that
four billion times a day,

so every second 50,000 sentences

are being translated

by an LSTM working for Facebook;

and another 50,000 in the second;
then another 50,000.

And to see how much this thing
is now permitting the modern world,

just note that almost 30 percent

of the awesome computational
power for inference

and all these Google Data Centers,

all these data centers of Google,
all over the world,

is used for LSTM.

Almost 30 percent.

If you have an Amazon Echo,

you can ask a question and it answers you.

And the voice that you hear
it's not a recording;

it's an LSTM network

which has learned from training examples

to sound like a female voice.

If you have an iPhone,
and you're using the quick type,

it's trying to predict
what you want to do next

given all the previous context
of what you did so far.

Again, that's an LSTM
which has learned to do that,

so it's on a billion iPhones.

You are a large audience, by my standards:

but when we started this work,
decades ago, in the early '90s,

only few people were interested in that,

because computers were so slow
and you couldn't do so much with it.

And I remember I gave a talk
at a conference,

and there was just
one single person in the audience,

a young lady.

I said, young lady,
it's very embarrassing,

but apparently today
I'm going to give this talk just to you.

And she said,

"OK, but please hurry:
I am the next speaker!"

(Laughter)

Since then, we have
greatly profited from the fact

that every five years
computers are getting ten times cheaper,

which is an old trend that has held
since 1941 at least.

Since this man, Konrad Zuse,

built the first working
program controlled computer in Berlin

and he could do, roughly,
one operation per second.

One!

And then ten years later,
for the same price,

one could do 100 operations:

30 years later,

1 million operations for the same price;

and today, after 75 years, we can do

a million billion times as much
for the same price.

And the trend is not about to stop,

because the physical limits
are much further out there.

Rather soon, and not
so many years or decades,

we will for the first time
have little computational devices

that can compute as much as a human brain;

and that's a trend that doesn't break.

50 years later, there will be
a little computational device,

for the same price,

that can compute as much as all
10 billion human brains taken together.

and there will not only be one,
of those devices, but many many many.

Everything is going to change.

Already in 2011,
computers were fast enough

such that our deep learning methods

for the first time could achieve
a superhuman pattern-recognition result.

It was the first superhuman result
in the history of computer vision.

And back then, computers were
20 times more expensive than today.

So today, for the same price,

we can do 20 times as much.

And just five years ago,

when computers were 10 times
more expensive than today,

we already could win, for the first time,
medical imaging competitions.

What you see behind me
is a slice through the female breast

and the tissue that you see there
has all kinds of cells;

and normally you need a trained doctor,
a trained histologist

who is able to detect
the dangerous cancer cells,

or pre-cancer cells.

Now, our stupid network

knows nothing about cancer,
knows nothing about vision.

It knows nothing in the beginning:

but we can train it to imitate
the human teacher, the doctor.

And it became as good, or better,
than the best competitors.

And very soon,

all of medical diagnosis
is going to be superhuman.

And it's going to be mandatory,

because it's going to be
so much better than the doctors.

After this, all kinds of medical
imaging startups were founded

focusing just on this,
because it's so important.

We can also use LSTM to train robots.

One important thing I want to say is,

that we not only have systems

that slavishly imitate
what humans show them;

no, we also have AIs
that set themselves their own goals.

And like little babies,
invent their own experiment

to explore the world

and to figure out
what you can do in the world.

Without a teacher.

And becoming more and more general
problem solvers in the process,

by learning new skills
on top of old skills.

And this is going to scale:
we call that "Artificial Curiosity".

Or a recent buzzword is "power plane".

Learning to become a more and more
general problem solvers

by learning to invent, like a scientist,
one new interesting goal after another.

And it's going to scale.

And I think,

in not so many years
from now, for the first time,

we are going to have an animal-like AI -

we don't have that yet.

On the level of a little crow,

which already can learn
to use tools, for example,

or a little monkey.

And once we have that,

it may take just a few decades

to do the final step
towards human level intelligence.

Because technological evolution

is about a million times faster
than biological evolution,

and biological evolution
needed 3.5 billion years

to evolve a monkey from scratch.

But then, it took just a few tens
of millions of years afterwards

to evolve human level intelligence.

We have a company
which is called Nnaisense

like birth in [French], "Naissance",
but spelled in a different way,

which is trying to make this a reality

and build the first
true general-purpose AI.

At the moment, almost all research in AI
is very human centric,

and it's all about making human lives
longer and healthier and easier

and making humans
more addicted to their smartphones.

But in the long run, AIs are going to -
especially the smart ones -

are going to set themselves
their own goals.

And I have no doubt, in my mind,

that they are going to become
much smarter than we are.

And what are they going to do?

Of course they are going to realize
what we have realized a long time ago;

namely, that most of the resources,
in the solar system or in general,

are not in our little biosphere.

They are out there in space.

And so, of course,
they are going to emigrate.

And of course they are going to use

trillions of self-replicating
robot factories

to expand in form of a growing AI bubble

which within a few hundred thousand years

is going to cover the entire galaxy

by senders and receivers

such that AIs can travel

the way they are
already traveling in my lab:

by radio, from sender to receiver.

Wireless.

So what we are witnessing now

is much more than just
another Industrial Revolution.

This is something
that transcends humankind,

and even life itself.

The last time something
so important has happened

was maybe 3.5 billion years ago,
when life was invented.

A new type of life is going to emerge
from our little planet

and it's going to colonize
and transform the entire universe.

The universe is still young:
it's only 13.8 billion years old,

it's going to become much older than that,
many times older than that.

So there's plenty of time
to reach all of it,

or all of the visible parts,

totally within the limits
of light speed and physics.

A new type of life is going
to make the universe intelligent.

Now, of course, we are not going to remain
the crown of creation, of course not.

But there is still beauty

in seeing yourself
as part of a grander process

that leads the cosmos

from low complexity
towards higher complexity.

It's a privilege to live at a time

where we can witness
the beginnings of that

and where we can contribute
something to that.

Thank you for your patience.

(Applause)