0:00:06.400,0:00:08.050
When I was a boy,

0:00:10.080,0:00:15.440
I wanted to maximise[br]my impact on the world,

0:00:15.440,0:00:19.460
and I was smart enough[br]to realise that I am not very smart.

0:00:21.280,0:00:24.588
And that I have to build a machine

0:00:24.588,0:00:28.770
that learns to become[br]much smarter than myself,

0:00:29.360,0:00:34.840
such that it can solve all the problems[br]that I cannot solve myself,

0:00:34.840,0:00:36.760
and I can retire.

0:00:38.560,0:00:42.800
And my first publication[br]on that dates back 30 years: 1987.

0:00:42.800,0:00:44.160
My diploma thesis,

0:00:44.160,0:00:48.600
where I already try to solve[br]the grand problem of AI,

0:00:48.600,0:00:50.240
not only build a machine

0:00:50.240,0:00:53.240
that learns a little bit here,[br]learns a little bit there,

0:00:53.240,0:00:58.530
but also learns to improve[br]the learning algorithm itself.

0:00:59.680,0:01:02.880
And the way it learns, the way it learns,

0:01:02.880,0:01:06.230
and so on recursively, without any limits

0:01:06.230,0:01:11.000
except the limits of logics and physics.

0:01:12.480,0:01:16.120
And, I'm still working[br]on the same old thing,

0:01:16.120,0:01:19.800
and I'm still pretty much[br]saying the same thing,

0:01:19.800,0:01:23.510
except that now[br]more people are listening.

0:01:25.160,0:01:28.080
Because the learning algorithms

0:01:28.080,0:01:30.480
that we have developed[br]on the way to this goal,

0:01:30.480,0:01:34.020
they are now on 3.000 million smartphones.

0:01:34.720,0:01:37.340
And all of you have them in your pockets.

0:01:39.950,0:01:40.960
What you see here

0:01:40.960,0:01:45.840
are the five most valuable companies[br]of the Western world:

0:01:45.840,0:01:50.430
Apple, Google, Facebook,[br]Microsoft and Amazon.

0:01:51.360,0:01:53.500
And all of them are emphasising

0:01:55.040,0:01:57.475
that AI, artificial intelligence,

0:01:57.475,0:02:00.270
is central to what they are doing.

0:02:02.000,0:02:07.600
And all of them are using heavily[br]the deep learning methods

0:02:07.600,0:02:11.000
that my team has developed[br]since the early nineties,

0:02:11.000,0:02:14.040
in Munich and in Switzerland.

0:02:14.040,0:02:18.720
Especially something which is called:[br]"the long short-term memory".

0:02:18.720,0:02:24.080
Has anybody in this room ever heard[br]of the long short-term memory,

0:02:24.080,0:02:25.560
or the LSTM?

0:02:25.560,0:02:27.720
Hands up, anybody ever heard of that?

0:02:27.720,0:02:29.000
Okay.

0:02:29.000,0:02:32.500
Has anybody never heard of the LSTM?

0:02:33.990,0:02:39.556
Okay.[br]I see we have a third group in this room:

0:02:43.156,0:02:45.755
[those] who didn't[br]understand the question.

0:02:45.755,0:02:47.625
(Laughter)

0:02:48.420,0:02:51.600
The LSTM is a little bit like your brain:

0:02:52.960,0:02:58.120
it's an artificial neural network[br]which also has neurons,

0:02:58.120,0:03:03.110
and in your brain, you've got[br]about 100 billion neurons.

0:03:04.240,0:03:05.630
And each of them is connected

0:03:05.630,0:03:09.520
to roughly 10,000[br]other neurons on average,

0:03:11.400,0:03:15.020
Which means that you have got[br]a million billion connections.

0:03:16.200,0:03:18.960
And each of these connections[br]has a "strength"

0:03:18.960,0:03:22.040
which says how much[br]does this neuron over here

0:03:22.040,0:03:25.200
influence that one over there[br]at the next time step.

0:03:25.200,0:03:26.320
And in the beginning,

0:03:26.320,0:03:30.160
all these connections are random[br]and the system knows nothing;

0:03:30.160,0:03:33.200
but then, through a smart[br]learning algorithm,

0:03:33.200,0:03:39.440
it learns from lots of examples[br]to translate the incoming data,

0:03:39.440,0:03:46.040
such as video through the cameras,[br]or audio through the microphones,

0:03:46.040,0:03:49.480
or pain signals through the pain sensors.

0:03:49.480,0:03:52.320
It learns to translate that[br]into output actions,

0:03:52.320,0:03:54.650
because some of these neurons[br]are output neurons,

0:03:54.650,0:03:57.650
that control speech muscles[br]and finger muscles.

0:04:00.223,0:04:01.840
And only through experience,

0:04:01.840,0:04:04.680
it can learn to solve[br]all kinds of interesting problems,

0:04:04.680,0:04:07.660
such as driving a car

0:04:10.880,0:04:13.800
or do the speech recognition[br]on your smartphone.

0:04:13.800,0:04:16.720
Because whenever you take out[br]your smartphone,

0:04:16.720,0:04:18.200
an Android phone, for example,

0:04:18.200,0:04:19.786
and you speak to it, and you say:

0:04:19.786,0:04:23.840
"Ok Google, show me[br]the shortest way to Milano."

0:04:23.840,0:04:25.379
Then it understands your speech.

0:04:26.970,0:04:31.760
Because there is a LSTM in there[br]which has learned to understand speech.

0:04:31.760,0:04:35.060
Every ten milliseconds,[br]100 times a second,

0:04:35.060,0:04:37.090
new inputs are coming from the microphone,

0:04:37.090,0:04:42.320
and then are translated, after thinking,

0:04:42.320,0:04:44.080
into letters

0:04:44.080,0:04:47.400
which are then questioned[br]to the search engine.

0:04:48.600,0:04:49.994
And it has learned to do that

0:04:49.994,0:04:54.690
by listening to lots of speech[br]from women, from men, all kinds of people.

0:04:55.390,0:04:57.800
And that's how, since 2015,

0:04:57.800,0:05:00.830
Google speech recognition[br]is now much better than it used to be.

0:05:02.400,0:05:05.360
The basic LSTM cell looks like that:

0:05:05.360,0:05:07.800
I don't have the time to explain that,

0:05:07.800,0:05:11.160
but at least I can list the names

0:05:11.160,0:05:14.320
of the brilliant students in my lab[br]who made that possible.

0:05:15.760,0:05:18.760
And what are the big companies[br]doing with that?

0:05:18.760,0:05:21.600
Well, speech recognition[br]is only one example;

0:05:22.280,0:05:25.170
if you are on Facebook -[br]is anybody on Facebook?

0:05:27.450,0:05:30.426
Are you sometimes clicking[br]at the translate button?

0:05:30.426,0:05:33.120
because somebody sent you something[br]in a foreign language

0:05:33.120,0:05:34.563
and then you can translate it.

0:05:34.563,0:05:37.000
Is anybody doing that? Yeah.

0:05:37.000,0:05:38.160
Whenever you do that,

0:05:38.160,0:05:41.560
you are waking up, again,[br]a long short term memory, an LSTM,

0:05:41.560,0:05:45.120
which has learned to translate[br]text in one language

0:05:45.120,0:05:47.380
into translated text.

0:05:48.880,0:05:53.280
And Facebook is doing that[br]four billion times a day,

0:05:53.280,0:05:59.456
so every second 50,000 sentences

0:05:59.456,0:06:00.880
are being translated

0:06:00.880,0:06:03.160
by an LSTM working for Facebook;

0:06:03.800,0:06:07.440
and another 50,000 in the second;[br]then another 50,000.

0:06:08.360,0:06:13.080
And to see how much this thing[br]is now permitting the modern world,

0:06:13.080,0:06:16.220
just note that almost 30 percent

0:06:16.220,0:06:22.240
of the awesome computational[br]power for inference

0:06:22.240,0:06:24.440
and all these Google Data Centers,

0:06:24.440,0:06:27.240
all these data centers of Google,[br]all over the world,

0:06:27.240,0:06:28.880
is used for LSTM.

0:06:28.880,0:06:30.170
Almost 30 percent.

0:06:30.880,0:06:33.240
If you have an Amazon Echo,

0:06:33.240,0:06:36.840
you can ask a question and it answers you.

0:06:37.440,0:06:40.280
And the voice that you hear[br]it's not a recording;

0:06:40.280,0:06:42.200
it's an LSTM network

0:06:42.200,0:06:44.693
which has learned from training examples

0:06:44.693,0:06:47.650
to sound like a female voice.

0:06:52.050,0:06:54.840
If you have an iPhone,[br]and you're using the quick type,

0:06:55.660,0:06:57.920
it's trying to predict[br]what you want to do next

0:06:57.920,0:07:00.640
given all the previous context[br]of what you did so far.

0:07:01.443,0:07:03.950
Again, that's an LSTM[br]which has learned to do that,

0:07:05.040,0:07:07.100
so it's on a billion iPhones.

0:07:09.920,0:07:12.680
You are a large audience, by my standards:

0:07:13.760,0:07:19.400
but when we started this work,[br]decades ago, in the early '90s,

0:07:19.400,0:07:21.680
only few people were interested in that,

0:07:21.680,0:07:24.900
because computers were so slow[br]and you couldn't do so much with it.

0:07:25.560,0:07:27.720
And I remember I gave a talk[br]at a conference,

0:07:28.898,0:07:31.400
and there was just[br]one single person in the audience,

0:07:32.840,0:07:34.680
a young lady.

0:07:34.680,0:07:38.960
I said, young lady,[br]it's very embarrassing,

0:07:38.960,0:07:42.000
but apparently today[br]I'm going to give this talk just to you.

0:07:42.000,0:07:43.280
And she said,

0:07:44.390,0:07:48.175
"OK, but please hurry:[br]I am the next speaker!"

0:07:48.175,0:07:52.645
(Laughter)

0:07:56.140,0:07:58.940
Since then, we have[br]greatly profited from the fact

0:07:58.940,0:08:02.174
that every five years[br]computers are getting ten times cheaper,

0:08:02.174,0:08:06.360
which is an old trend that has held[br]since 1941 at least.

0:08:06.360,0:08:08.080
Since this man, Konrad Zuse,

0:08:08.080,0:08:12.640
built the first working[br]program controlled computer in Berlin

0:08:12.640,0:08:17.140
and he could do, roughly,[br]one operation per second.

0:08:17.140,0:08:18.270
One!

0:08:19.140,0:08:22.040
And then ten years later,[br]for the same price,

0:08:22.040,0:08:24.520
one could do 100 operations:

0:08:24.520,0:08:25.600
30 years later,

0:08:25.600,0:08:27.960
1 million operations for the same price;

0:08:27.960,0:08:30.480
and today, after 75 years, we can do

0:08:30.480,0:08:33.799
a million billion times as much[br]for the same price.

0:08:33.799,0:08:36.120
And the trend is not about to stop,

0:08:36.120,0:08:39.650
because the physical limits[br]are much further out there.

0:08:42.919,0:08:48.080
Rather soon, and not[br]so many years or decades,

0:08:48.080,0:08:51.280
we will for the first time[br]have little computational devices

0:08:51.280,0:08:54.400
that can compute as much as a human brain;

0:08:55.090,0:08:57.130
and that's a trend that doesn't break.

0:08:57.130,0:09:01.520
50 years later, there will be[br]a little computational device,

0:09:01.520,0:09:02.760
for the same price,

0:09:02.760,0:09:07.800
that can compute as much as all[br]10 billion human brains taken together.

0:09:08.600,0:09:12.600
and there will not only be one,[br]of those devices, but many many many.

0:09:12.600,0:09:14.920
Everything is going to change.

0:09:14.920,0:09:17.720
Already in 2011,[br]computers were fast enough

0:09:17.720,0:09:19.840
such that our deep learning methods

0:09:19.840,0:09:25.480
for the first time could achieve[br]a superhuman pattern-recognition result.

0:09:25.480,0:09:29.960
It was the first superhuman result[br]in the history of computer vision.

0:09:29.960,0:09:34.120
And back then, computers were[br]20 times more expensive than today.

0:09:34.120,0:09:35.680
So today, for the same price,

0:09:35.680,0:09:37.840
we can do 20 times as much.

0:09:37.840,0:09:43.200
And just five years ago,

0:09:43.200,0:09:46.880
when computers were 10 times[br]more expensive than today,

0:09:46.880,0:09:51.440
we already could win, for the first time,[br]medical imaging competitions.

0:09:51.440,0:09:55.960
What you see behind me[br]is a slice through the female breast

0:09:55.960,0:10:00.680
and the tissue that you see there[br]has all kinds of cells;

0:10:00.680,0:10:05.160
and normally you need a trained doctor,[br]a trained histologist

0:10:05.160,0:10:09.560
who is able to detect[br]the dangerous cancer cells,

0:10:09.560,0:10:11.160
or pre-cancer cells.

0:10:11.880,0:10:13.487
Now, our stupid network

0:10:13.487,0:10:16.084
knows nothing about cancer,[br]knows nothing about vision.

0:10:16.084,0:10:17.720
It knows nothing in the beginning:

0:10:17.720,0:10:21.920
but we can train it to imitate[br]the human teacher, the doctor.

0:10:21.920,0:10:26.560
And it became as good, or better,[br]than the best competitors.

0:10:26.560,0:10:28.710
And very soon,

0:10:28.710,0:10:31.880
all of medical diagnosis[br]is going to be superhuman.

0:10:33.690,0:10:35.560
And it's going to be mandatory,

0:10:35.560,0:10:38.253
because it's going to be[br]so much better than the doctors.

0:10:40.440,0:10:45.600
After this, all kinds of medical[br]imaging startups were founded

0:10:45.600,0:10:48.120
focusing just on this,[br]because it's so important.

0:10:49.160,0:10:52.800
We can also use LSTM to train robots.

0:10:52.800,0:10:55.040
One important thing I want to say is,

0:10:55.040,0:10:58.040
that we not only have systems

0:10:58.040,0:11:01.080
that slavishly imitate[br]what humans show them;

0:11:01.080,0:11:05.920
no, we also have AIs[br]that set themselves their own goals.

0:11:07.960,0:11:12.280
And like little babies,[br]invent their own experiment

0:11:12.880,0:11:14.840
to explore the world

0:11:14.840,0:11:17.092
and to figure out[br]what you can do in the world.

0:11:17.560,0:11:19.260
Without a teacher.

0:11:19.260,0:11:23.400
And becoming more and more general[br]problem solvers in the process,

0:11:23.400,0:11:26.680
by learning new skills[br]on top of old skills.

0:11:26.680,0:11:31.120
And this is going to scale:[br]we call that "Artificial Curiosity".

0:11:31.940,0:11:34.200
Or a recent buzzword is "power plane".

0:11:34.720,0:11:38.840
Learning to become a more and more[br]general problem solvers

0:11:38.840,0:11:44.280
by learning to invent, like a scientist,[br]one new interesting goal after another.

0:11:44.840,0:11:47.440
And it's going to scale.

0:11:47.440,0:11:48.450
And I think,

0:11:48.450,0:11:50.790
in not so many years[br]from now, for the first time,

0:11:50.790,0:11:55.520
we are going to have an animal-like AI -

0:11:55.520,0:11:57.720
we don't have that yet.

0:11:58.600,0:12:00.160
On the level of a little crow,

0:12:00.800,0:12:04.040
which already can learn[br]to use tools, for example,

0:12:04.040,0:12:05.360
or a little monkey.

0:12:05.700,0:12:07.360
And once we have that,

0:12:07.360,0:12:09.270
it may take just a few decades

0:12:09.270,0:12:13.400
to do the final step[br]towards human level intelligence.

0:12:14.800,0:12:16.380
Because technological evolution

0:12:16.380,0:12:20.660
is about a million times faster[br]than biological evolution,

0:12:20.660,0:12:27.440
and biological evolution[br]needed 3.5 billion years

0:12:27.440,0:12:31.440
to evolve a monkey from scratch.

0:12:31.440,0:12:35.240
But then, it took just a few tens[br]of millions of years afterwards

0:12:35.240,0:12:37.560
to evolve human level intelligence.

0:12:38.400,0:12:40.680
We have a company[br]which is called Nnaisense

0:12:41.720,0:12:45.120
like birth in [French], "Naissance",[br]but spelled in a different way,

0:12:45.120,0:12:47.826
which is trying to make this a reality

0:12:47.826,0:12:50.960
and build the first[br]true general-purpose AI.

0:12:52.560,0:12:58.120
At the moment, almost all research in AI[br]is very human centric,

0:12:58.120,0:13:04.720
and it's all about making human lives[br]longer and healthier and easier

0:13:04.720,0:13:07.240
and making humans[br]more addicted to their smartphones.

0:13:09.100,0:13:13.320
But in the long run, AIs are going to -[br]especially the smart ones -

0:13:13.320,0:13:16.280
are going to set themselves[br]their own goals.

0:13:16.280,0:13:18.800
And I have no doubt, in my mind,

0:13:18.800,0:13:21.760
that they are going to become[br]much smarter than we are.

0:13:22.480,0:13:24.400
And what are they going to do?

0:13:24.400,0:13:27.960
Of course they are going to realize[br]what we have realized a long time ago;

0:13:27.960,0:13:34.200
namely, that most of the resources,[br]in the solar system or in general,

0:13:34.200,0:13:37.120
are not in our little biosphere.

0:13:37.120,0:13:38.990
They are out there in space.

0:13:40.075,0:13:42.240
And so, of course,[br]they are going to emigrate.

0:13:42.240,0:13:48.920
And of course they are going to use

0:13:48.920,0:13:52.400
trillions of self-replicating[br]robot factories

0:13:52.400,0:13:57.880
to expand in form of a growing AI bubble

0:13:57.880,0:14:00.400
which within a few hundred thousand years

0:14:00.400,0:14:02.560
is going to cover the entire galaxy

0:14:02.560,0:14:04.240
by senders and receivers

0:14:04.240,0:14:06.320
such that AIs can travel

0:14:06.320,0:14:08.920
the way they are[br]already traveling in my lab:

0:14:08.920,0:14:11.160
by radio, from sender to receiver.

0:14:12.200,0:14:13.650
Wireless.

0:14:15.100,0:14:19.000
So what we are witnessing now

0:14:19.000,0:14:24.630
is much more than just[br]another Industrial Revolution.

0:14:24.630,0:14:27.680
This is something[br]that transcends humankind,

0:14:27.680,0:14:29.520
and even life itself.

0:14:29.520,0:14:32.880
The last time something[br]so important has happened

0:14:32.880,0:14:37.240
was maybe 3.5 billion years ago,[br]when life was invented.

0:14:38.430,0:14:42.930
A new type of life is going to emerge[br]from our little planet

0:14:42.930,0:14:48.000
and it's going to colonize[br]and transform the entire universe.

0:14:48.000,0:14:52.000
The universe is still young:[br]it's only 13.8 billion years old,

0:14:52.000,0:14:58.000
it's going to become much older than that,[br]many times older than that.

0:14:58.000,0:15:02.520
So there's plenty of time[br]to reach all of it,

0:15:02.520,0:15:04.240
or all of the visible parts,

0:15:04.240,0:15:07.640
totally within the limits[br]of light speed and physics.

0:15:09.450,0:15:13.780
A new type of life is going[br]to make the universe intelligent.

0:15:13.780,0:15:19.220
Now, of course, we are not going to remain[br]the crown of creation, of course not.

0:15:20.400,0:15:21.880
But there is still beauty

0:15:21.880,0:15:27.200
in seeing yourself[br]as part of a grander process

0:15:27.200,0:15:29.160
that leads the cosmos

0:15:29.160,0:15:32.200
from low complexity[br]towards higher complexity.

0:15:33.640,0:15:36.760
It's a privilege to live at a time

0:15:36.760,0:15:40.080
where we can witness[br]the beginnings of that

0:15:40.080,0:15:43.240
and where we can contribute[br]something to that.

0:15:46.490,0:15:48.300
Thank you for your patience.

0:15:49.160,0:15:54.840
(Applause)