WEBVTT
00:00:00.000 --> 00:00:09.044
preroll music
00:00:09.044 --> 00:00:14.049
Herald: Our next talk is going to be about AI and
it's going to be about proper AI.
00:00:14.049 --> 00:00:17.730
It's not going to be about
deep learning or buzz word bingo.
00:00:17.730 --> 00:00:22.590
It's going to be about actual psychology.
It's going to be about computational metapsychology.
00:00:22.590 --> 00:00:25.750
And now please welcome Joscha!
00:00:25.750 --> 00:00:33.050
applause
00:00:33.050 --> 00:00:35.620
Joscha: Thank you.
00:00:35.620 --> 00:00:37.710
I'm interested in understanding
how the mind works,
00:00:37.710 --> 00:00:42.640
and I believe that the most foolproof perspective
at looking ... of looking at minds is to understand
00:00:42.640 --> 00:00:46.600
that they are systems that if you saw patterns
at them you find meaning.
00:00:46.600 --> 00:00:51.700
And you find meaning in those in very particular
ways and this is what makes us who we are.
00:00:51.700 --> 00:00:55.239
So they way to study and understand who we
are in my understanding is
00:00:55.239 --> 00:01:01.149
to build models of information processing
that constitutes our minds.
00:01:01.149 --> 00:01:05.640
Last year about the same time, I've answered
the four big questions of philosophy:
00:01:05.640 --> 00:01:08.510
"Whats the nature of reality?", "What can
be known?", "Who are we?",
00:01:08.510 --> 00:01:14.650
"What should we do?"
So now, how can I top this?
00:01:14.650 --> 00:01:18.720
applause
00:01:18.720 --> 00:01:22.849
I'm going to give you the drama
that divided a planet.
00:01:22.849 --> 00:01:26.470
Some of a very, very big events,
that happened in the course of last year,
00:01:26.470 --> 00:01:30.080
so I couldn't tell you about it before.
00:01:30.080 --> 00:01:38.489
What color is the dress
laughsapplause
00:01:38.489 --> 00:01:44.720
I mean ahmm... If you have.. do not have any
mental defects you can clearly see it's white
00:01:44.720 --> 00:01:46.550
and gold. Right?
00:01:46.550 --> 00:01:48.720
[voices from audience]
00:01:48.720 --> 00:01:53.009
Turns out, ehmm.. most people seem to have
mental defects and say it is blue and black.
00:01:53.009 --> 00:01:57.500
I have no idea why. Well Ok, I have an idea,
why that is the case.
00:01:57.500 --> 00:02:01.170
Ehmm, I guess that you got too, it has to
do with color renormalization
00:02:01.170 --> 00:02:04.720
and color renormalization happens differently
apparently in different people.
00:02:04.720 --> 00:02:09.000
So we have different wireing to renormalize
the white balance.
00:02:09.000 --> 00:02:12.650
And it seems to work in real world
situations in pretty much the same way,
00:02:12.650 --> 00:02:18.000
but not necessarily for photographs.
Which have only very small fringe around them,
00:02:18.000 --> 00:02:20.600
which gives you hint about the lighting situation.
00:02:20.600 --> 00:02:27.000
And that's why you get this huge divergencies,
which is amazing!
00:02:27.000 --> 00:02:29.660
So what we see that our minds can not know
00:02:29.660 --> 00:02:33.250
objective truths in any way. Outside of mathematics.
00:02:33.250 --> 00:02:36.340
They can generate meaning though.
00:02:36.340 --> 00:02:38.760
How does this work?
00:02:38.760 --> 00:02:42.010
I did robotic soccer for a while,
and there you have the situation,
00:02:42.010 --> 00:02:45.150
that you have a bunch of robots, that are
situated on a playing field.
00:02:45.150 --> 00:02:48.480
And they have a model of what goes on
in the playing field.
00:02:48.480 --> 00:02:52.050
Physics generates data for their sensors.
They read the bits of the sensors.
00:02:52.050 --> 00:02:55.900
And then they use them to.. erghmm update
the world model.
00:02:55.900 --> 00:02:59.020
And sometimes we didn't want
to take the whole playing field along,
00:02:59.020 --> 00:03:03.380
and the physical robots, because they are
expensive and heavy and so on.
00:03:03.380 --> 00:03:06.480
Instead if you just want to improve the learning
and the game play of the robots
00:03:06.480 --> 00:03:07.800
you can use the simulations.
00:03:07.800 --> 00:03:11.200
So we've wrote a computer simulation of the
playing field and the physics, and so on,
00:03:11.200 --> 00:03:15.210
that generates pretty some the same data,
and put the robot mind into the simulator
00:03:15.210 --> 00:03:17.040
robot body, and it works just as well.
00:03:17.040 --> 00:03:20.590
That is, if you the robot, because you can
not know the difference if you are the robot.
00:03:20.590 --> 00:03:24.460
You can not know what's out there. The only
thing that you get to see is what is the structure
00:03:24.460 --> 00:03:27.530
of the data at you system bit interface.
00:03:27.530 --> 00:03:30.090
And then you can derive model from this.
00:03:30.090 --> 00:03:32.960
And this is pretty much the situation
that we are in.
00:03:32.960 --> 00:03:38.180
That is, we are minds that are somehow computational,
00:03:38.180 --> 00:03:40.700
they are able to find regularity in patterns,
00:03:40.700 --> 00:03:44.530
and they are... we.. seem to have access to
something that is full of regularity,
00:03:44.530 --> 00:03:46.630
so we can make sense out of it.
00:03:46.630 --> 00:03:48.930
[ghulp, ghulp]
00:03:48.930 --> 00:03:52.800
Now, if you discover that you are in the same
situation as these robots,
00:03:52.800 --> 00:03:56.180
basically you discover that you are some kind
of apparently biological robot,
00:03:56.180 --> 00:03:58.530
that doesn't have direct access
to the world of concepts.
00:03:58.530 --> 00:04:02.140
That has never actually seen matter
and energy and other people.
00:04:02.140 --> 00:04:04.890
All it got to see was little bits of information,
00:04:04.890 --> 00:04:06.270
that were transmitted through the nerves,
00:04:06.270 --> 00:04:07.870
and the brain had to make sense of them,
00:04:07.870 --> 00:04:10.470
by counting them in elaborate ways.
00:04:10.470 --> 00:04:12.720
What's the best model of the world
that you can have with this?
00:04:12.720 --> 00:04:16.530
What will the state of affairs,
what's the system that you are in?
00:04:16.530 --> 00:04:20.920
And what are the best algorithms that you
should be using, to fix your world model.
00:04:20.920 --> 00:04:23.310
And this question is pretty old.
00:04:23.310 --> 00:04:27.750
And I think that has been answered for the
first time by Ray Solomonoff in the 1960.
00:04:27.750 --> 00:04:30.840
He has discovered an algorithm,
that you can apply when you discover
00:04:30.840 --> 00:04:33.540
that you are an robot,
and all you have is data.
00:04:33.540 --> 00:04:34.870
What is the world like?
00:04:34.870 --> 00:04:40.990
And this algorithm is basically
a combination of induction and Occam's razor.
00:04:40.990 --> 00:04:45.710
And we can mathematically prove that we can
not do better than Solomonoff induction.
00:04:45.710 --> 00:04:51.380
Unfortunately, Solomonoff induction
is not quite computable.
00:04:51.380 --> 00:04:54.450
But everything that we are going to do is
some... is going to be some approximation
00:04:54.450 --> 00:04:55.820
of Salomonoff induction.
00:04:55.820 --> 00:04:59.400
So our concepts can not really refer
to the facts in the world out there.
00:04:59.400 --> 00:05:02.380
We do not get the truth by referring
to stuff out there, in the world.
00:05:02.380 --> 00:05:07.960
We get meaning by suitably encoding
the patterns at our systemic interface.
00:05:07.960 --> 00:05:12.270
And AI has recently made a huge progress in
encoding data at perceptual interfaces.
00:05:12.270 --> 00:05:15.900
Deep learning is about using a stacked hierarchy
of feature detectors.
00:05:15.900 --> 00:05:21.280
That is, we use pattern detectors and we build
them into a networks that are arranged in
00:05:21.280 --> 00:05:23.030
hundreds of layers.
00:05:23.030 --> 00:05:26.500
And then we adjust the links
between these layers.
00:05:26.500 --> 00:05:29.380
Usually some kind of... using
some kind of gradient descent.
00:05:29.380 --> 00:05:33.220
And we can use this to classify
for instance images and parts of speech.
00:05:33.220 --> 00:05:37.950
So, we get to features that are more and more
complex, they started as very, very simple patterns.
00:05:37.950 --> 00:05:41.290
And then get more and more complex,
until we get to object categories.
00:05:41.290 --> 00:05:44.199
And now this systems are able
in image recognition task,
00:05:44.199 --> 00:05:47.480
to approach performance that is very similar
to human performance.
00:05:47.480 --> 00:05:52.040
Also what is nice is that it seems to be somewhat
similar to what the brain seems to be doing
00:05:52.040 --> 00:05:53.740
in visual processing.
00:05:53.740 --> 00:05:57.570
And if you take the activation in different
levels of these networks and you
00:05:57.570 --> 00:06:01.430
erghm... improve the... that... erghmm...
enhance this activation a little bit, what
00:06:01.430 --> 00:06:03.500
you get is stuff that look very psychedelic.
00:06:03.500 --> 00:06:09.620
Which may be similar to what happens, if you
put certain illegal substances into people,
00:06:09.620 --> 00:06:13.650
and enhance the activity on certain layers
of their visual processing.
00:06:13.650 --> 00:06:21.540
[BROKEN AUDIO]If you want to classify the
differences what we do if we want quantify
00:06:21.540 --> 00:06:33.030
this you filter out all the invariences in
the data.
00:06:33.030 --> 00:06:36.360
The pose that she has, the lighting,
the dress that she is on.. has on,
00:06:36.360 --> 00:06:38.020
her facial expression and so on.
00:06:38.020 --> 00:06:42.900
And then we go to only to this things that
is left after we've removed all the nuance data.
00:06:42.900 --> 00:06:47.410
But what if we... erghmm
want to get to something else,
00:06:47.410 --> 00:06:49.850
for instance if we want to understand poses.
00:06:49.850 --> 00:06:53.240
Could be for instance that we have several
dancers and we want to understand what they
00:06:53.240 --> 00:06:54.400
have in common.
00:06:54.400 --> 00:06:58.330
So our best bet is not just to have a single
classification based filtering,
00:06:58.330 --> 00:07:01.199
but instead what we want to have is to take
the low level input
00:07:01.199 --> 00:07:05.180
and get a whole universe of features,
that is interrelated.
00:07:05.180 --> 00:07:07.220
So we have different levels of interrelations.
00:07:07.220 --> 00:07:08.960
At the lowest levels we have percepts.
00:07:08.960 --> 00:07:11.580
On the slightly higher level we have simulations.
00:07:11.580 --> 00:07:16.920
And on even higher level we have concept landscape.
00:07:16.920 --> 00:07:19.300
How does this representation
by simulation work?
00:07:19.300 --> 00:07:22.229
Now imagine you want to understand sound.
00:07:22.229 --> 00:07:23.669
[Ghulp]
00:07:23.669 --> 00:07:26.710
If you are a brain and you want to understand
sound you need to model it.
00:07:26.710 --> 00:07:31.070
Unfortunatly we can not really model sound
with neurons, because sound goes up to 20kHz,
00:07:31.070 --> 00:07:36.660
or if you are old like me maybe to 12 kHz.
20 kHz is what babies could do.
00:07:36.660 --> 00:07:41.240
And... neurons do not want to do 20 kHz.
That's way too fast for them.
00:07:41.240 --> 00:07:43.250
They like something like 20 Hz.
00:07:43.250 --> 00:07:45.590
So what do you do? You need
to make a Fourier transform.
00:07:45.590 --> 00:07:49.650
The Fourier transform measures the amount
of energy at different frequencies.
00:07:49.650 --> 00:07:52.500
And because you can not do it with neurons,
you need to do it in hardware.
00:07:52.500 --> 00:07:54.180
And turns out this is exactly
what we are doing.
00:07:54.180 --> 00:07:59.860
We have this cochlea which is this snail like
thing in our ears,
00:07:59.860 --> 00:08:06.669
and what it does, it transforms energy of
sound in different frequency intervals into
00:08:06.669 --> 00:08:08.009
energy measurments.
00:08:08.009 --> 00:08:10.479
And then gives you something
like what you see here.
00:08:10.479 --> 00:08:12.550
And this is something that the brain can model,
00:08:12.550 --> 00:08:16.210
so we can get a neurosimulator that tries
to recreate this patterns.
00:08:16.210 --> 00:08:21.370
And we can predict the next input from the
cochlea that then understand the sound.
00:08:21.370 --> 00:08:23.410
Of course if you want to understand music,
00:08:23.410 --> 00:08:25.160
we have to go beyond understanding sound.
00:08:25.160 --> 00:08:29.340
We have to understand the transformations
that sound can have if you play it at different pitch.
00:08:29.340 --> 00:08:33.599
We have to arrange the sound in the sequence
that give you rhythms and so on.
00:08:33.599 --> 00:08:35.889
And then we want to identify
some kind of musical grammar
00:08:35.889 --> 00:08:38.799
that we can use to again control the sequencer.
00:08:38.799 --> 00:08:42.529
So we have stucked structures.
That simulate the world.
00:08:42.529 --> 00:08:44.319
And once you've learned this model of music,
00:08:44.319 --> 00:08:47.309
once you've learned the musical grammar,
the sequencer and the sounds.
00:08:47.309 --> 00:08:51.779
You can get to the structure
of the individual piece of music.
00:08:51.779 --> 00:08:54.399
So, if you want to model the world of music.
00:08:54.399 --> 00:08:58.279
You need to have the lowest level of percepts
then we have the higher level of mental simulations.
00:08:58.279 --> 00:09:01.910
And... which give the sequences of the music
and the grammars of music.
00:09:01.910 --> 00:09:05.149
And beyond this you have the conceptual landscape
that you can use
00:09:05.149 --> 00:09:08.249
to describe different styles of music.
00:09:08.249 --> 00:09:12.130
And if you go up in the hierarchy,
you get to more and more abstract models.
00:09:12.130 --> 00:09:13.860
More and more conceptual models.
00:09:13.860 --> 00:09:16.449
And more and more analytic models.
00:09:16.449 --> 00:09:18.160
And this are causal models at some point.
00:09:18.160 --> 00:09:20.999
This causal models can be weakly deterministic,
00:09:20.999 --> 00:09:22.980
basically associative models, which tell you
00:09:22.980 --> 00:09:27.339
if this state happens, it's quite probable
that this one comes afterwords.
00:09:27.339 --> 00:09:29.389
Or you can get to a strongly determined model.
00:09:29.389 --> 00:09:32.730
Strongly determined model is one which tells
you, if you are in this state
00:09:32.730 --> 00:09:33.879
and this condition is met,
00:09:33.879 --> 00:09:35.589
You are are going to go exactly in this state.
00:09:35.589 --> 00:09:40.110
If this condition is not met, or a different
condition is met, you are going to this state.
00:09:40.110 --> 00:09:41.449
And this is what we call an alghorithm.
00:09:41.449 --> 00:09:46.769
it's.. now we are on the domain of computation.
00:09:46.769 --> 00:09:48.730
Computation is slightly different from mathematics.
00:09:48.730 --> 00:09:51.179
It's important to understand this.
00:09:51.179 --> 00:09:54.699
For a long time people have thought that the
universe is written in mathematics.
00:09:54.699 --> 00:09:58.399
Or that.. minds are mathematical,
or anything is mathematical.
00:09:58.399 --> 00:10:00.439
In fact nothing is mathematical.
00:10:00.439 --> 00:10:04.529
Mathematics is just the domain
of formal languages. It doesn't exist.
00:10:04.529 --> 00:10:07.300
Mathematics starts with a void.
00:10:07.300 --> 00:10:11.939
You throw in a few axioms, and if you've chosen
a nice axioms, then you get infinite complexity.
00:10:11.939 --> 00:10:13.679
Most of which is not computable.
00:10:13.679 --> 00:10:16.270
In mathematics you can express arbitrary statements,
00:10:16.270 --> 00:10:18.269
because it's all about formal languages.
00:10:18.269 --> 00:10:20.369
Many of this statements will not make sense.
00:10:20.369 --> 00:10:22.469
Many of these statements will make sense
in some way,
00:10:22.469 --> 00:10:24.429
but you can not test whether they make sense,
00:10:24.429 --> 00:10:26.740
because they're not computable.
00:10:26.740 --> 00:10:29.929
Computation is different.
Computation can exist.
00:10:29.929 --> 00:10:32.459
It's starts with an initial state.
00:10:32.459 --> 00:10:34.739
And then you have a transition function.
You do the work.
00:10:34.739 --> 00:10:38.449
You apply the transition function,
and you get into the next state.
00:10:38.449 --> 00:10:41.249
Computation is always finite.
00:10:41.249 --> 00:10:43.689
Mathematics is the kingdom of specification.
00:10:43.689 --> 00:10:47.290
And computation is the kingdom of implementation.
00:10:47.290 --> 00:10:50.629
It's very important to understand this difference.
00:10:50.629 --> 00:10:55.329
All our access to mathematics of course is
because we do computation.
00:10:55.329 --> 00:10:57.459
We can understand mathematics,
00:10:57.459 --> 00:10:59.939
because our brain can compute
some parts of mathematics.
00:10:59.939 --> 00:11:04.439
Very, very little of it, and to
very constrained complexity.
00:11:04.439 --> 00:11:06.860
But enough, so we can map
some of the infinite complexity
00:11:06.860 --> 00:11:10.410
and noncomputability of mathematics
into computational patterns,
00:11:10.410 --> 00:11:12.279
that we can explore.
00:11:12.279 --> 00:11:14.410
So computation is about doing the work,
00:11:14.410 --> 00:11:16.939
it's about executing the transition function.
00:11:19.730 --> 00:11:22.899
Now we've seen that mental representation
is about concepts,
00:11:22.899 --> 00:11:25.670
mental simulations, conceptual representations
00:11:25.670 --> 00:11:29.110
and this conceptual representations
give us concept spaces.
00:11:29.110 --> 00:11:30.970
And the nice thing
about this concept spaces is
00:11:30.970 --> 00:11:33.399
that they give us an interface
to our mental representations,
00:11:33.399 --> 00:11:36.290
We can use to address and manipulate them.
00:11:36.290 --> 00:11:39.119
And we can share them in cultures.
00:11:39.119 --> 00:11:40.899
And this concepts are compositional.
00:11:40.899 --> 00:11:43.639
We can put them together, to create new concepts.
00:11:43.639 --> 00:11:48.230
And they can be described using
higher dimensional vector spaces.
00:11:48.230 --> 00:11:50.319
They don't do simulation
and prediction and so on,
00:11:50.319 --> 00:11:53.119
but we can capture regularity
in our concept wisdom.
00:11:53.119 --> 00:11:55.220
With this vector space
you can do amazing things.
00:11:55.220 --> 00:11:57.589
For instance, if you take the vector from
"King" to "Queen"
00:11:57.589 --> 00:12:01.009
is pretty much the same vector
as to.. between "Man" and "Woman"
00:12:01.009 --> 00:12:04.110
And because of this properties, because it's
really a high dimentional manifold
00:12:04.110 --> 00:12:07.569
this concepts faces, we can do interesting
things, like machine translation
00:12:07.569 --> 00:12:09.470
without understanding what it means.
00:12:09.470 --> 00:12:13.929
That is without doing any proper mental representation,
that predicts the world.
00:12:13.929 --> 00:12:16.989
So this is a type of meta representation,
that is somewhat incomplete,
00:12:16.989 --> 00:12:21.199
but it captures the landscape that we share
in a culture.
00:12:21.199 --> 00:12:25.089
And then there is another type of meta representation,
that is linguistic protocols.
00:12:25.089 --> 00:12:27.699
Which is basically a formal grammar and vocabulary.
00:12:27.699 --> 00:12:29.619
And we need this linguistic protocols
00:12:29.619 --> 00:12:32.869
to transfer mental representations
between people.
00:12:32.869 --> 00:12:36.019
And we do this by basically
scanning our mental representation,
00:12:36.019 --> 00:12:38.660
disassembling them in some way
or disambiguating them.
00:12:38.660 --> 00:12:43.040
And then we use it as discrete string of symbols
to get it to somebody else,
00:12:43.040 --> 00:12:46.429
and he trains an assembler,
that reverses this process,
00:12:46.429 --> 00:12:51.389
and build something that is pretty similar
to what we intended to convey.
00:12:51.389 --> 00:12:53.569
And if you look at the progression of AI models,
00:12:53.569 --> 00:12:55.600
it pretty much went the opposite direction.
00:12:55.600 --> 00:13:00.279
So AI started with linguistic protocols, which
were expressed in formal grammars.
00:13:00.279 --> 00:13:05.209
And then it got to concepts spaces, and now
it's about to address percepts.
00:13:05.209 --> 00:13:09.689
And at some point in near future it's going
to get better at mental simulations.
00:13:09.689 --> 00:13:11.730
And at some point after that we get to
00:13:11.730 --> 00:13:14.769
attention directed and
motivationally connected systems,
00:13:14.769 --> 00:13:16.600
that make sense of the world.
00:13:16.600 --> 00:13:20.290
that are in some sense able to address meaning.
00:13:20.290 --> 00:13:23.489
This is the hardware that we have can do.
00:13:23.489 --> 00:13:25.629
What kind of hardware do we have?
00:13:25.629 --> 00:13:28.480
That's a very interesting question.
00:13:28.480 --> 00:13:32.230
It could start out with a question:
How difficult is it to define a brain?
00:13:32.230 --> 00:13:35.439
We know that the brain must be
somewhere hidden in the genome.
00:13:35.439 --> 00:13:38.290
The genome fits on a CD ROM.
It's not that complicated.
00:13:38.290 --> 00:13:40.399
It's easier than Microsoft Windows. laughter
00:13:40.399 --> 00:13:45.549
And we also know, that about 2%
of the genome is coding for proteins.
00:13:45.549 --> 00:13:48.429
And maybe about 10% of the genome
has some kind of stuff
00:13:48.429 --> 00:13:51.239
that tells you when to switch protein.
00:13:51.239 --> 00:13:52.829
And the remainder is mostly garbage.
00:13:52.829 --> 00:13:57.170
It's old viruses that are left over and has
never been properly deleted and so on.
00:13:57.170 --> 00:14:01.420
Because there are no real
code revisions in the genome.
00:14:01.420 --> 00:14:08.119
So how much of this 10%
that is 75 MB code for the brain.
00:14:08.119 --> 00:14:09.469
We don't really know.
00:14:09.469 --> 00:14:13.399
What we do know is we share
almost all of this with mice.
00:14:13.399 --> 00:14:15.769
Genetically speaking human
is a pretty big mouse.
00:14:15.769 --> 00:14:21.049
With a few bits changed, so.. to fix some
of the genetic expressions
00:14:21.049 --> 00:14:25.879
And that is most of the stuff there is going
to code for cells and metabolism
00:14:25.879 --> 00:14:27.999
and how your body looks like and so on.
00:14:27.999 --> 00:14:33.679
But if you look at erghmm... how much is expressed
in the brain and only in the brain,
00:14:33.679 --> 00:14:35.170
in terms of proteins and so on.
00:14:35.170 --> 00:14:45.639
We find it's about... well of the 2% it's
about 5%. That is only the 5% of the 2% that
00:14:45.639 --> 00:14:46.799
is only in the brain.
00:14:46.799 --> 00:14:50.199
And another 5% of the 2% is predominantly
in the brain.
00:14:50.199 --> 00:14:52.069
That is more in the brain than anywhere else.
00:14:52.069 --> 00:14:54.249
Which gives you some kind of thing
like a lower bound.
00:14:54.249 --> 00:14:59.379
Which means to encode a brain genetically
base on the hardware that we are using.
00:14:59.379 --> 00:15:03.539
We need something like
at least 500 kB of code.
00:15:03.539 --> 00:15:06.670
Actually ehmm.. this... we very conservative
lower bound.
00:15:06.670 --> 00:15:08.720
It's going to be a little more I guess.
00:15:08.720 --> 00:15:11.449
But it sounds surprisingly little, right?
00:15:11.449 --> 00:15:13.709
But in terms of scientific theories
this is a lot.
00:15:13.709 --> 00:15:16.519
I mean the universe,
according to the core theory
00:15:16.519 --> 00:15:19.420
of the quantum mechanics and so on
is like so much of code.
00:15:19.420 --> 00:15:20.569
It's like half a page of code.
00:15:20.569 --> 00:15:23.100
That's it. That's all you need
to generate the universe.
00:15:23.100 --> 00:15:25.489
And if you want to understand evolution
it's like a paragraph.
00:15:25.489 --> 00:15:29.609
It's couple lines you need to understand
evolutionary process.
00:15:29.609 --> 00:15:32.199
And there is a lots, lots of details, that's
you get afterwards.
00:15:32.199 --> 00:15:34.220
Because this process itself doesn't define
00:15:34.220 --> 00:15:37.259
how the animals are going to look like,
and in similar way is..
00:15:37.259 --> 00:15:41.269
the code of the universe doesn't tell you
what this planet is going to look like.
00:15:41.269 --> 00:15:43.279
And what you guys are going to look like.
00:15:43.279 --> 00:15:45.949
It's just defining the rulebook.
00:15:45.949 --> 00:15:49.209
And in the same sense genome defines the rulebook,
00:15:49.209 --> 00:15:51.569
by which our brain is build.
00:15:51.569 --> 00:15:56.399
erghmmm,.. The brain boots itself
into developer process,
00:15:56.399 --> 00:15:58.119
and this booting takes some time.
00:15:58.119 --> 00:16:01.069
So subliminal learning in which
initial connections are forged
00:16:01.069 --> 00:16:04.910
And basic models are build of the world,
so we can operate in it.
00:16:04.910 --> 00:16:06.999
And how long does this booting take?
00:16:06.999 --> 00:16:09.669
I thing it's about 80 mega seconds.
00:16:09.669 --> 00:16:14.319
That's the time that a child is awake until
it's 2.5 years old.
00:16:14.319 --> 00:16:16.449
By this age you understand Star Wars.
00:16:16.449 --> 00:16:20.029
And I think that everything after
understanding Star Wars is cosmetics.
00:16:20.029 --> 00:16:26.799
laughterapplause
00:16:26.799 --> 00:16:32.820
You are going to be online, if you get to
arrive old age for about 1.5 giga seconds.
00:16:32.820 --> 00:16:37.929
And in this time I think you are going to
get not to watch more than 5 milion concepts.
00:16:37.929 --> 00:16:41.600
Why? I don't know real...
If you look at this child.
00:16:41.600 --> 00:16:45.480
If a child would be able to form a concept
let say every 5 minutes,
00:16:45.480 --> 00:16:48.529
then by the time it's about 4 years old,
it's going to have
00:16:48.529 --> 00:16:51.549
something like 250 thousands concepts.
00:16:51.549 --> 00:16:54.119
And... so... a quarter million.
00:16:54.119 --> 00:16:56.809
And if we extrapolate this into our lifetime,
00:16:56.809 --> 00:16:59.799
at some point it slows down,
because we have enough concepts,
00:16:59.799 --> 00:17:01.230
to describe the world.
00:17:01.230 --> 00:17:04.410
Maybe it's something... It's I think it's
less that 5 million.
00:17:04.410 --> 00:17:07.140
How much storage capacity does the brain has?
00:17:07.140 --> 00:17:12.319
I think that the... the estimates
are pretty divergent,
00:17:12.319 --> 00:17:14.930
The lower bound is something like a 100 GB,
00:17:14.930 --> 00:17:18.569
And the upper bound
is something like 2.5 PB.
00:17:18.569 --> 00:17:21.890
There is even...
even some higher outliers this..
00:17:21.890 --> 00:17:25.630
If you for instance think that we need all
those synaptic vesicle to store information,
00:17:25.630 --> 00:17:27.530
maybe even more fits into this.
00:17:27.530 --> 00:17:31.740
But the 2.5 PB is usually based
on what you need
00:17:31.740 --> 00:17:34.760
to code the information
that is in all the neurons.
00:17:34.760 --> 00:17:36.770
But maybe the neurons
do not really matter so much,
00:17:36.770 --> 00:17:39.930
because if the neuron dies it's not like the
word is changing dramatically.
00:17:39.930 --> 00:17:44.270
The brain is very resilient
against individual neurons failing.
00:17:44.270 --> 00:17:48.930
So the 100 GB capacity is much more
what you actually store in the neurons.
00:17:48.930 --> 00:17:51.380
If you look at all the redundancy
that you need.
00:17:51.380 --> 00:17:54.230
And I think this is much closer to the actual
Ballpark figure.
00:17:54.230 --> 00:17:58.130
Also if you want to store 5 hundred...
5 million concepts,
00:17:58.130 --> 00:18:02.330
and maybe 10 times or 100 times the number
of percepts, on top of this,
00:18:02.330 --> 00:18:05.490
this is roughly the Ballpark figure
that you are going to need.
00:18:05.490 --> 00:18:07.110
So our brain
00:18:07.110 --> 00:18:08.320
is a prediction machine.
00:18:08.320 --> 00:18:11.490
It... What it does is it reduces the entropy
of the environment,
00:18:11.490 --> 00:18:14.610
to solve whatever problems you are encountering,
00:18:14.610 --> 00:18:17.790
if you don't have a... feedback loop, to fix
them.
00:18:17.790 --> 00:18:20.240
So normally if something happens, we have
some kind of feedback loop,
00:18:20.240 --> 00:18:23.440
that regulates our temperature or that makes
problems go away.
00:18:23.440 --> 00:18:26.050
And only when this is not working
we employ recognition.
00:18:26.050 --> 00:18:29.250
And then we start this arbitrary
computational processes,
00:18:29.250 --> 00:18:31.830
that is facilitated by the neural cortex.
00:18:31.830 --> 00:18:34.940
And this.. arhmm.. neural cortex has really
do arbitrary programs.
00:18:34.940 --> 00:18:37.870
But it can do so
with only with very limited complexity,
00:18:37.870 --> 00:18:42.070
because really you just saw,
it's not that complex.
00:18:42.070 --> 00:18:43.900
The modeling of the world is very slow.
00:18:43.900 --> 00:18:46.570
And it's something
that we see in our eye models.
00:18:46.570 --> 00:18:48.150
To learn the basic structure of the world
00:18:48.150 --> 00:18:49.330
takes a very long time.
00:18:49.330 --> 00:18:52.650
To learn basically that we are moving in 3D
and objects are moving,
00:18:52.650 --> 00:18:54.030
and what they look like.
00:18:54.030 --> 00:18:55.130
Once we have this basic model,
00:18:55.130 --> 00:18:59.300
we can get to very, very quick
understanding within this model.
00:18:59.300 --> 00:19:02.110
Basically encoding based
on the structure of the world,
00:19:02.110 --> 00:19:03.610
that we've learned.
00:19:03.610 --> 00:19:07.100
And this is some kind of
data compression, that we are doing.
00:19:07.100 --> 00:19:09.740
We use this model, this grammar of the world,
00:19:09.740 --> 00:19:12.150
this simulation structures that we've learned,
00:19:12.150 --> 00:19:15.190
to encode the world very, very efficently.
00:19:15.190 --> 00:19:17.740
How much data compression do we get?
00:19:17.740 --> 00:19:19.860
Well... if you look at the retina.
00:19:19.860 --> 00:19:24.610
The retina get's data
in the order of about 10Gb/s.
00:19:24.610 --> 00:19:27.500
And the retina already compresses these data,
00:19:27.500 --> 00:19:31.120
and puts them into optic nerve
at the rate of about 1Mb/s
00:19:31.120 --> 00:19:34.030
This is what you get fed into visual cortex.
00:19:34.030 --> 00:19:36.370
And the visual cortex
does some additional compression,
00:19:36.370 --> 00:19:42.110
and by the time it gets to layer four of the
first layer of vision, to V1.
00:19:42.110 --> 00:19:46.880
We are down to something like 1Kb/s.
00:19:46.880 --> 00:19:50.720
So if we extrapolate this, and you get live
to the age of 80 years,
00:19:50.720 --> 00:19:54.140
and you are awake for 2/3 of your lifetime.
00:19:54.140 --> 00:19:56.930
That is you have your eyes open for 2/3 of
your lifetime.
00:19:56.930 --> 00:19:59.040
The stuff that you get into your brain,
00:19:59.040 --> 00:20:03.700
via your visual perception
is going to be only 2TB.
00:20:03.700 --> 00:20:05.370
Only 2TB of visual data.
00:20:05.370 --> 00:20:06.680
Throughout all your lifetime.
00:20:06.680 --> 00:20:09.430
That's all you are going to get ever to see.
00:20:09.430 --> 00:20:11.160
Isn't this depressing?
00:20:11.160 --> 00:20:12.790
laughter
00:20:12.790 --> 00:20:16.540
So I would really like to eghmm..
to tell you,
00:20:16.540 --> 00:20:22.750
choose wisely what you
are going to look at. laughter
00:20:22.750 --> 00:20:26.940
Ok. Let's look at this problem of neural compositionality.
00:20:26.940 --> 00:20:29.250
Our brains has this amazing thing
that they can put
00:20:29.250 --> 00:20:31.510
meta representation together very, very quickly.
00:20:31.510 --> 00:20:33.150
For instance you read a page of code,
00:20:33.150 --> 00:20:35.190
you compile it in you mind
into some kind of program
00:20:35.190 --> 00:20:37.700
it tells you what this page is going to do.
00:20:37.700 --> 00:20:39.110
Isn't that amazing?
00:20:39.110 --> 00:20:40.810
And then you can forget about this,
00:20:40.810 --> 00:20:43.910
disassemble it all, and use the
building blocks for something else.
00:20:43.910 --> 00:20:45.230
It's like legos.
00:20:45.230 --> 00:20:48.000
How you can do this with neurons?
00:20:48.000 --> 00:20:50.160
Legos can do this, because they have
a well defined interface.
00:20:50.160 --> 00:20:52.180
They have all this slots, you know,
that fit together
00:20:52.180 --> 00:20:53.600
in well defined ways.
00:20:53.600 --> 00:20:54.530
How can neurons do this?
00:20:54.530 --> 00:20:57.280
Well, neurons can maybe learn
the interface of other neurons.
00:20:57.280 --> 00:20:59.780
But that's difficult, because every neuron
looks slightly different,
00:20:59.780 --> 00:21:04.830
after all this... some kind of biologically
grown natural stuff.
00:21:04.830 --> 00:21:06.610
laughter
00:21:06.610 --> 00:21:10.620
So what you want to do is,
you want to encapsulate this erhmm...
00:21:10.620 --> 00:21:13.020
diversity of the neurons to make the predictable.
00:21:13.020 --> 00:21:14.820
To give them well defined interface.
00:21:14.820 --> 00:21:16.410
And I think that nature solution to this
00:21:16.410 --> 00:21:19.770
is cortical columns.
00:21:19.770 --> 00:21:24.250
Cortical column is a circuit of
between 100 and 400 neurons.
00:21:24.250 --> 00:21:26.860
And this circuit has some kind of neural network,
00:21:26.860 --> 00:21:28.650
that can learn stuff.
00:21:28.650 --> 00:21:31.070
And after it has learned particular function,
00:21:31.070 --> 00:21:35.320
and in between, it's able to link up these
other cortical columns.
00:21:35.320 --> 00:21:37.120
And we have about 100 million of those.
00:21:37.120 --> 00:21:39.770
Depending on how many neurons
you assume is in there,
00:21:39.770 --> 00:21:41.490
it's... erghmm we guess it's something,
00:21:41.490 --> 00:21:46.500
at least 20 million and maybe
something like a 100 million.
00:21:46.500 --> 00:21:48.330
And this cortical columns, what they can do,
00:21:48.330 --> 00:21:50.280
is they can link up like lego bricks,
00:21:50.280 --> 00:21:54.130
and then perform,
by transmitting information between them,
00:21:54.130 --> 00:21:55.990
pretty much arbitrary computations.
00:21:55.990 --> 00:21:57.540
What kind of computation?
00:21:57.540 --> 00:22:00.130
Well... Solomonoff induction.
00:22:00.130 --> 00:22:03.820
And... they have some short range links,
to their neighbors.
00:22:03.820 --> 00:22:05.690
Which comes almost for free, because erghmm..
00:22:05.690 --> 00:22:08.490
well, they are connected to them,
they are direct neighborhood.
00:22:08.490 --> 00:22:10.050
And they have some long range connectivity,
00:22:10.050 --> 00:22:13.000
so you can combine everything
in your cortex with everything.
00:22:13.000 --> 00:22:14.900
So you need some kind of global switchboard.
00:22:14.900 --> 00:22:17.630
Some grid like architecture
of long range connections.
00:22:17.630 --> 00:22:18.900
They are going to be more expensive,
00:22:18.900 --> 00:22:20.640
they are going to be slower,
00:22:20.640 --> 00:22:23.590
but they are going to be there.
00:22:23.590 --> 00:22:26.070
So how can we optimize
what these guys are doing?
00:22:26.070 --> 00:22:28.270
In some sense it's like an economy.
00:22:28.270 --> 00:22:31.460
It's not enduring based system,
as we often use in machine learning.
00:22:31.460 --> 00:22:32.780
It's really an economy. You have...
00:22:32.780 --> 00:22:35.560
The question is, you have a fixed number of
elements,
00:22:35.560 --> 00:22:37.970
how can you do the most valuable stuff with
them.
00:22:37.970 --> 00:22:41.030
Fixed resources, most valuable stuff, the
problem is economy.
00:22:41.030 --> 00:22:43.320
So you have an economy of information brokers.
00:22:43.320 --> 00:22:45.830
Every one of these guys,
this little cortical columns,
00:22:45.830 --> 00:22:48.150
is very simplistic information broker.
00:22:48.150 --> 00:22:50.950
And they trade rewards against neg entropy,
00:22:50.950 --> 00:22:54.140
Against reducing entropy in the...
in the world.
00:22:54.140 --> 00:22:55.790
And to do this, as we just saw
00:22:55.790 --> 00:22:58.890
that they need some kind of standardized interface.
00:22:58.890 --> 00:23:02.090
And internally, to use this interface
they are going to
00:23:02.090 --> 00:23:03.880
have some kind of state machine.
00:23:03.880 --> 00:23:05.660
And then they are going to pass messages
00:23:05.660 --> 00:23:07.400
between each other.
00:23:07.400 --> 00:23:08.630
And what are these messages?
00:23:08.630 --> 00:23:11.100
Well, it's going to be hard
to discover these messages,
00:23:11.100 --> 00:23:12.800
by looking at brains.
00:23:12.800 --> 00:23:14.800
Because it's very difficult to see in brains,
00:23:14.800 --> 00:23:15.450
what the are actually doing.
00:23:15.450 --> 00:23:17.250
you just see all these neurons.
00:23:17.250 --> 00:23:18.790
And if you would be waiting for neuroscience,
00:23:18.790 --> 00:23:20.970
to discover anything, we wouldn't even have
00:23:20.970 --> 00:23:22.590
gradient descent or anything else.
00:23:22.590 --> 00:23:23.720
We wouldn't have neuron learning.
00:23:23.720 --> 00:23:25.420
We wouldn't have all this advances in AI.
00:23:25.420 --> 00:23:28.230
Jürgen Schmidhuber said that the biggest,
00:23:28.230 --> 00:23:30.010
the last contribution of neuroscience to
00:23:30.010 --> 00:23:32.220
artificial intelligence
was about 50 years ago.
00:23:32.220 --> 00:23:34.280
That's depressing, and it might be
00:23:34.280 --> 00:23:37.870
overemphasizing the unimportance of neuroscience,
00:23:37.870 --> 00:23:39.490
because neuroscience is very important,
00:23:39.490 --> 00:23:41.090
once you know what are you looking for.
00:23:41.090 --> 00:23:42.510
You can actually often find this,
00:23:42.510 --> 00:23:44.320
and see whether you are on the right track.
00:23:44.320 --> 00:23:45.860
But it's very difficult to take neuroscience
00:23:45.860 --> 00:23:47.940
to understand how the brain is working.
00:23:47.940 --> 00:23:49.290
Because it's really like understanding
00:23:49.290 --> 00:23:53.230
flight by looking at birds through a microscope.
00:23:53.230 --> 00:23:55.150
So, what are these messages?
00:23:55.150 --> 00:23:57.850
You are going to need messages,
that tell these cortical columns
00:23:57.850 --> 00:24:00.160
to join themselves into a structure.
00:24:00.160 --> 00:24:01.990
And to unlink again once they're done.
00:24:01.990 --> 00:24:03.690
You need ways that they can request each other
00:24:03.690 --> 00:24:06.040
to perform computations for them.
00:24:06.040 --> 00:24:07.510
You need ways they can inhibit each other
00:24:07.510 --> 00:24:08.320
when they are linked up.
00:24:08.320 --> 00:24:10.990
So they don't do conflicting computations.
00:24:10.990 --> 00:24:12.940
Then they need to tell you whether the computation,
00:24:12.940 --> 00:24:14.110
the result of the computation
00:24:14.110 --> 00:24:16.730
that the are asked to do is probably false.
00:24:16.730 --> 00:24:19.340
Or whether it's probably true,
but you still need to wait for others,
00:24:19.340 --> 00:24:21.990
to tell you whether the details worked out.
00:24:21.990 --> 00:24:24.240
Or whether it's confirmed true that the concepts
00:24:24.240 --> 00:24:26.730
that they stand for is actually the case.
00:24:26.730 --> 00:24:28.150
And then you want to have learning,
00:24:28.150 --> 00:24:29.630
to tell you how well this worked.
00:24:29.630 --> 00:24:31.390
So you will have to announce a bounty,
00:24:31.390 --> 00:24:34.380
that tells them to link up
and kind of reward signal
00:24:34.380 --> 00:24:36.740
that makes do computation in the first place.
00:24:36.740 --> 00:24:38.680
And then you want to have
some kind of reward signal
00:24:38.680 --> 00:24:40.550
once you got the result as an organism.
00:24:40.550 --> 00:24:42.280
But you reach your goal if you made
00:24:42.280 --> 00:24:45.810
the disturbance go away
or what ever you consume the cake.
00:24:45.810 --> 00:24:47.710
And then you will have
some kind of reward signal
00:24:47.710 --> 00:24:49.250
that's you give everybody.
00:24:49.250 --> 00:24:50.650
That was involved in this.
00:24:50.650 --> 00:24:52.720
And this reward signal facilitates learning,
00:24:52.720 --> 00:24:55.230
so the.. difference between the announce reward
00:24:55.230 --> 00:24:57.530
and consumption reward is the learning signal
00:24:57.530 --> 00:24:58.740
for these guys.
00:24:58.740 --> 00:25:00.210
So they can learn how to play together,
00:25:00.210 --> 00:25:02.700
and how to do the Solomonoff induction.
00:25:02.700 --> 00:25:04.660
Now, I've told you that Solomonoff induction
00:25:04.660 --> 00:25:05.280
is not computable.
00:25:05.280 --> 00:25:07.630
And it's mostly because of two things,
00:25:07.630 --> 00:25:09.280
First of all it's needs infinite resources
00:25:09.280 --> 00:25:11.200
to compare all the possible models.
00:25:11.200 --> 00:25:13.530
And the other one is that we do not know
00:25:13.530 --> 00:25:15.440
the priori probability for our Bayesian model.
00:25:15.440 --> 00:25:19.280
If we do not know
how likely unknown stuff is in the world.
00:25:19.280 --> 00:25:22.520
So what we do instead is,
we set some kind of hyperparameter,
00:25:22.520 --> 00:25:25.050
Some kind of default
priori probability for concepts,
00:25:25.050 --> 00:25:28.110
that are encoded by cortical columns.
00:25:28.110 --> 00:25:30.580
And if we set these parameters very low,
00:25:30.580 --> 00:25:32.140
then we are going to end up with inferences
00:25:32.140 --> 00:25:35.250
that are quite probable.
00:25:35.250 --> 00:25:36.480
For unknown things.
00:25:36.480 --> 00:25:37.690
And then we can test for those.
00:25:37.690 --> 00:25:41.350
If we set this parameter higher, we are going
to be very, very creative.
00:25:41.350 --> 00:25:43.670
But we end up with many many theories,
00:25:43.670 --> 00:25:45.140
that are difficult to test.
00:25:45.140 --> 00:25:48.470
Because maybe there are
too many theories to test.
00:25:48.470 --> 00:25:50.650
Basically every of these cortical columns
will now tell you,
00:25:50.650 --> 00:25:52.240
when you ask them if they are true:
00:25:52.240 --> 00:25:54.960
"Yes I'm probably true,
but i still need to ask others,
00:25:54.960 --> 00:25:56.980
to work on the details"
00:25:56.980 --> 00:25:58.670
So these others are going to be get active,
00:25:58.670 --> 00:26:00.640
and they are being asked by the asking element:
00:26:00.640 --> 00:26:01.730
"Are you going to be true?",
00:26:01.730 --> 00:26:04.380
and they say "Yeah, probably yes,
I just have to work on the details"
00:26:04.380 --> 00:26:05.930
and they are going to ask even more.
00:26:05.930 --> 00:26:07.980
So your brain is going to light up like a
christmas tree,
00:26:07.980 --> 00:26:10.240
and do all these amazing computations,
00:26:10.240 --> 00:26:12.450
and you see connections everywhere,
most of them are wrong.
00:26:12.450 --> 00:26:16.310
You are basically in psychotic state
if your hyperparameter is too high.
00:26:16.310 --> 00:26:20.790
You're brain invents more theories
that it can disproof.
00:26:20.790 --> 00:26:24.550
Would it actually sometimes be good
to be in this state?
00:26:24.550 --> 00:26:27.850
You bet. So i think every night our brain
goes in this state.
00:26:27.850 --> 00:26:31.720
We turn up this hyperparameter.
We dream. We get all kinds
00:26:31.720 --> 00:26:34.100
weird connections, and we get to see connections,
00:26:34.100 --> 00:26:36.140
that otherwise we couldn't be seeing.
00:26:36.140 --> 00:26:38.080
Even though... because they are highly improbable.
00:26:38.080 --> 00:26:42.750
But sometimes they hold, and we see... "Oh
my God, DNA is organized in double helix".
00:26:42.750 --> 00:26:44.640
And this is what we remember in the morning.
00:26:44.640 --> 00:26:46.870
All the other stuff is deleted.
00:26:46.870 --> 00:26:48.440
So we usually don't form long term memories
00:26:48.440 --> 00:26:51.480
in dreams, if everything goes well.
00:26:51.480 --> 00:26:56.670
If you accidentally trip this up.. your modulators,
00:26:56.670 --> 00:26:59.100
for instance by consuming illegal substances,
00:26:59.100 --> 00:27:01.690
or because you just gone randomly psychotic
00:27:01.690 --> 00:27:04.600
you was basically entering
a dreaming state I guess.
00:27:04.600 --> 00:27:06.990
You get to a state
when the brain starts inventing more
00:27:06.990 --> 00:27:10.860
concepts that it can disproof.
00:27:10.860 --> 00:27:13.600
So you want to have a state
where this is well balanced.
00:27:13.600 --> 00:27:16.180
And the difference between
highly creative people,
00:27:16.180 --> 00:27:20.070
and very religious people is probably
a different setting of this hyperparameter.
00:27:20.070 --> 00:27:21.890
So I suspect that people that people
that are genius,
00:27:21.890 --> 00:27:23.880
like people like Einstein and so on,
00:27:23.880 --> 00:27:26.600
do not simply have better neurons than others.
00:27:26.600 --> 00:27:29.130
What they mostly have is a slightly hyperparameter,
00:27:29.130 --> 00:27:33.860
that is very finely tuned, so they can get
better balance than other people
00:27:33.860 --> 00:27:43.850
in finding theories that might be true,
but can still be disprooven.
00:27:43.850 --> 00:27:49.480
So inventiveness could be
a hyperparameter in the brain.
00:27:49.480 --> 00:27:54.169
If you want to measure
the quality of belief that we have
00:27:54.169 --> 00:27:56.370
we are going to have to have
some kind of some cost function
00:27:56.370 --> 00:27:58.710
which is based on motivational system.
00:27:58.710 --> 00:28:02.400
And to identify if belief
is good or not we can abstract criteria,
00:28:02.400 --> 00:28:06.440
for instance how well does it predict the
wourld, or how about does it reduce uncertainty
00:28:06.440 --> 00:28:07.590
in the world,
00:28:07.590 --> 00:28:10.020
or is it consistency and sparse.
00:28:10.020 --> 00:28:14.080
And then of course utility, how about does
it help me to satisfy my needs.
00:28:14.080 --> 00:28:18.920
And the motivational system is going
to evaluate all this things by giving a signal.
00:28:18.920 --> 00:28:24.200
And the first signal.. kind of signal
is the possible rewards if we are able to compute
00:28:24.200 --> 00:28:25.020
the task.
00:28:25.020 --> 00:28:27.430
And this is probably done by dopamine.
00:28:27.430 --> 00:28:30.350
So we have a very small area in the brain,
substantia nigra,
00:28:30.350 --> 00:28:33.610
and the ventral tegmental area,
and they produce dopamine.
00:28:33.610 --> 00:28:38.180
And this get fed into lateral frontal cortext
and the frontal lobe,
00:28:38.180 --> 00:28:41.920
which control attention,
and tell you what things to do.
00:28:41.920 --> 00:28:46.020
And if we have successfully done
what you wanted to do,
00:28:46.020 --> 00:28:49.300
we consume the rewards.
00:28:49.300 --> 00:28:51.940
And we do this with another signal
which is serotonine.
00:28:51.940 --> 00:28:53.480
It's also announce to motivational system,
00:28:53.480 --> 00:28:55.870
to this very small are the Raphe nuclei.
00:28:55.870 --> 00:28:58.690
And it feeds into all the areas of the brain
where learning is necessary.
00:28:58.690 --> 00:29:02.160
A connection is strengthen
once you get to result.
00:29:02.160 --> 00:29:07.559
These two substances are emitted
by the motivational system.
00:29:07.559 --> 00:29:09.710
The motivational system is a bunch of needs,
00:29:09.710 --> 00:29:11.510
essentially you regulate it below the cortext.
00:29:11.510 --> 00:29:14.490
They are not part of your mental representations.
00:29:14.490 --> 00:29:16.930
They are part of something
that is more primary than this.
00:29:16.930 --> 00:29:19.360
This is what makes us go,
this is what makes us human.
00:29:19.360 --> 00:29:22.290
This is not our rationality, this is what we want.
00:29:22.290 --> 00:29:27.000
And the needs are physiological,
they are social, they are cognitive.
00:29:27.000 --> 00:29:28.960
And you pretty much born with them.
00:29:28.960 --> 00:29:30.470
They can not be totally adaptive,
00:29:30.470 --> 00:29:33.340
because if we were adaptive,
we wouldn't be doing anything.
00:29:33.340 --> 00:29:35.390
The needs are resistive.
00:29:35.390 --> 00:29:38.290
They are pushing us against the world.
00:29:38.290 --> 00:29:40.170
If you wouldn't have all this needs,
00:29:40.170 --> 00:29:41.740
If you wouldn't have this motivational system,
00:29:41.740 --> 00:29:43.630
you would just be doing what best for you.
00:29:43.630 --> 00:29:45.150
Which means collapse on the ground,
00:29:45.150 --> 00:29:49.010
be a vegetable, rod, give into gravity.
00:29:49.010 --> 00:29:50.270
Instead you do all this unpleasant things,
00:29:50.270 --> 00:29:52.690
to get up in the morning,
you eat, you have sex,
00:29:52.690 --> 00:29:54.120
you do all this crazy things.
00:29:54.120 --> 00:29:58.809
And it's only because the
motivational system forces you to.
00:29:58.809 --> 00:30:00.850
The motivational system
takes this bunch of matter,
00:30:00.850 --> 00:30:02.890
and makes us to do all these strange things,
00:30:02.890 --> 00:30:05.940
just so genomes get replicated and so on.
00:30:05.940 --> 00:30:10.470
And... so to do this, we are going to build
resistance against the world.
00:30:10.470 --> 00:30:13.360
And the motivational system
is in a sense forcing us,
00:30:13.360 --> 00:30:15.470
to do all this things by giving us needs,
00:30:15.470 --> 00:30:18.330
and the need have some kind
of target value and current value.
00:30:18.330 --> 00:30:21.850
If we have a differential
between the target value and current value,
00:30:21.850 --> 00:30:24.590
we perceive some urgency
to do something about the need.
00:30:24.590 --> 00:30:26.680
And when the target value
approaches the current value
00:30:26.680 --> 00:30:28.660
we get the pleasure, which is a learning signal.
00:30:28.660 --> 00:30:30.540
If it gets away from it
we get a displeasure signal,
00:30:30.540 --> 00:30:31.870
which is also a learning signal.
00:30:31.870 --> 00:30:35.370
And we can use this to structure
our understanding of the world.
00:30:35.370 --> 00:30:36.870
To understand what goals are and so on.
00:30:36.870 --> 00:30:40.020
Goals are learned. Needs are not.
00:30:40.020 --> 00:30:42.780
To learn we need success
and failure in the world.
00:30:42.780 --> 00:30:45.940
But to do things we need anticipated reward.
00:30:45.940 --> 00:30:48.120
So it's dopamine that's makes brain go round.
00:30:48.120 --> 00:30:50.560
Dopamine makes you do things.
00:30:50.560 --> 00:30:52.750
But in order to do this in the right way,
00:30:52.750 --> 00:30:54.610
you have to make sure,
that the cells can not
00:30:54.610 --> 00:30:55.880
produce dopamine themselves.
00:30:55.880 --> 00:30:59.100
If they do this they can start
to drive others to work for them.
00:30:59.100 --> 00:31:01.870
You are going to get something like
bureaucracy in your neural cortext,
00:31:01.870 --> 00:31:05.650
where different bosses try
to set up others to they own bidding
00:31:05.650 --> 00:31:07.910
and pitch against other groups in nerual cortext.
00:31:07.910 --> 00:31:09.730
It's going to be horrible.
00:31:09.730 --> 00:31:12.210
So you want to have some kind of central authority,
00:31:12.210 --> 00:31:16.290
that make sure that the cells
do not produce dopamine themselves.
00:31:16.290 --> 00:31:19.679
It's only been produce in
very small area and then given out,
00:31:19.679 --> 00:31:21.059
and pass through the system.
00:31:21.059 --> 00:31:23.350
And after you're done with it's going to be gone,
00:31:23.350 --> 00:31:26.070
so there is no hoarding of the dopamine.
00:31:26.070 --> 00:31:29.770
And in our society the role of dopamine
is played by money.
00:31:29.770 --> 00:31:32.150
Money is not reward in itself.
00:31:32.150 --> 00:31:35.570
It's in some sense way
that you can trade against the reward.
00:31:35.570 --> 00:31:36.850
You can not eat money.
00:31:36.850 --> 00:31:40.500
You can take it later and take
a arbitrary reward for it.
00:31:40.500 --> 00:31:45.400
And in some sense money is the dopamine
that makes organizations
00:31:45.400 --> 00:31:48.410
and society, companies
and many individuals do things.
00:31:48.410 --> 00:31:50.500
They do stuff because of money.
00:31:50.500 --> 00:31:53.309
But money if you compare to dopamine
is pretty broken,
00:31:53.309 --> 00:31:54.850
because you can hoard it.
00:31:54.850 --> 00:31:57.400
So you are going to have this
cortical columns in the real world,
00:31:57.400 --> 00:31:59.670
which are individual people
or individual corporations.
00:31:59.670 --> 00:32:03.250
They are hoarding the dopamine,
they sit on this very big pile of dopamine.
00:32:03.250 --> 00:32:07.890
They are starving the rest
of the society of the dopamine.
00:32:07.890 --> 00:32:10.630
They don't give it away,
and they can make it do it's bidding.
00:32:10.630 --> 00:32:13.970
So for instance they can pitch
substantial part of society
00:32:13.970 --> 00:32:16.130
against understanding of global warming.
00:32:16.130 --> 00:32:20.110
because they profit of global warming
or of technology that leads to global warming,
00:32:20.110 --> 00:32:22.850
which is very bad for all of us. applause
00:32:22.850 --> 00:32:28.850
So our society is a nervous system
that lies to itself.
00:32:28.850 --> 00:32:30.429
How can we overcome this?
00:32:30.429 --> 00:32:32.480
Actually, we don't know.
00:32:32.480 --> 00:32:34.639
To do this we would need
to have some kind of centrialized,
00:32:34.639 --> 00:32:36.660
top-down reward motivational system.
00:32:36.660 --> 00:32:39.010
We have this for instance in the military,
00:32:39.010 --> 00:32:42.520
you have this system of
military rewards that you get.
00:32:42.520 --> 00:32:44.950
And this are completely
controlled from the top.
00:32:44.950 --> 00:32:47.260
Also within working organizations
you have this.
00:32:47.260 --> 00:32:49.600
In corporations you have centralized rewards,
00:32:49.600 --> 00:32:51.850
it's not like rewards flow bottom-up,
00:32:51.850 --> 00:32:55.120
they always flown top-down.
00:32:55.120 --> 00:32:57.850
And there was an attempt
to model society in such a way.
00:32:57.850 --> 00:33:03.380
That was in Chile in the early 1970,
the Allende government had the idea
00:33:03.380 --> 00:33:07.320
to redesign society or economy
in society using cybernetics.
00:33:07.320 --> 00:33:12.590
So Allende invited a bunch of cyberneticians
to redesign the Chilean economy.
00:33:12.590 --> 00:33:14.550
And this was meant to be the control room,
00:33:14.550 --> 00:33:17.460
where Allende and his chief economists
would be sitting,
00:33:17.460 --> 00:33:19.709
to look at what the economy is doing.
00:33:19.709 --> 00:33:23.880
We don't know how this would work out,
because we know how it ended.
00:33:23.880 --> 00:33:27.260
In 1973 there was this big putsch in Chile,
00:33:27.260 --> 00:33:30.290
and this experiment ended among other things.
00:33:30.290 --> 00:33:34.170
Maybe it would have worked, who knows?
Nobody tried it.
00:33:34.170 --> 00:33:38.370
So, there is something else
what is going on in people,
00:33:38.370 --> 00:33:40.030
beyond the motivational system.
00:33:40.030 --> 00:33:43.610
That is: we have social criteria, for learning.
00:33:43.610 --> 00:33:47.670
We also check if our ideas
are normativly acceptable.
00:33:47.670 --> 00:33:50.510
And this is actually a good thing,
because individual may shortcut
00:33:50.510 --> 00:33:52.590
the learning through communication.
00:33:52.590 --> 00:33:55.260
Other people have learned stuff
that we don't need to learn ourselves.
00:33:55.260 --> 00:33:59.800
We can build on this, so we can accelerate
learning by many order of magnitutde,
00:33:59.800 --> 00:34:00.970
which makes culture possible.
00:34:00.970 --> 00:34:04.190
And which makes many anything possible,
because if you were on your own
00:34:04.190 --> 00:34:06.860
you would not be going to find out
very much in your lifetime.
00:34:08.520 --> 00:34:11.270
You know how they say?
Everything that you do,
00:34:11.270 --> 00:34:14.250
you do by standing on the shoulders of giants.
00:34:14.250 --> 00:34:17.779
Or on a big pile of dwarfs
it works either way.
00:34:17.779 --> 00:34:27.089
laughterapplause
00:34:27.089 --> 00:34:30.379
Social learning usually outperforms
individual learning. You can test this.
00:34:30.379 --> 00:34:33.949
But in the case of conflict
between different social truths,
00:34:33.949 --> 00:34:36.659
you need some way to decide who to believe.
00:34:36.659 --> 00:34:39.498
So you have some kind of reputation
estimate for different authority,
00:34:39.498 --> 00:34:42.399
and you use this to check whom you believe.
00:34:42.399 --> 00:34:45.748
And the problem of course is this
in existing society, in real society,
00:34:45.748 --> 00:34:48.389
this reputation system is going
to reflect power structure,
00:34:48.389 --> 00:34:51.699
which may distort your belief systematically.
00:34:51.699 --> 00:34:54.759
Social learning therefore leads groups
to synchronize their opinions.
00:34:54.759 --> 00:34:57.220
And the opinions become ...get another role.
00:34:57.220 --> 00:35:02.180
They become important part
of signalling which group you belong to.
00:35:02.180 --> 00:35:06.630
So opinions start to signal
group loyalty in societies.
00:35:06.630 --> 00:35:11.170
And people in this, and that's the actual world,
they should optimize not for getting the best possible
00:35:11.170 --> 00:35:12.619
opinions in terms of truth.
00:35:12.619 --> 00:35:17.289
They should guess... they should optimize
for doing... having the best possible opinion,
00:35:17.289 --> 00:35:19.799
with respect to agreement with their peers.
00:35:19.799 --> 00:35:22.029
If you have the same opinion
as your peers, you can signal them
00:35:22.029 --> 00:35:24.299
that you are the part of their ingroup,
they are going to like you.
00:35:24.299 --> 00:35:28.160
If you don't do this, chances are
they are not going to like you.
00:35:28.160 --> 00:35:34.049
There is rarely any benefit in life to be
in disagreement with your boss. Right?
00:35:34.049 --> 00:35:39.230
So, if you evolve an opinion forming system
in these curcumstances,
00:35:39.230 --> 00:35:41.220
you should be ending up
with an opinion forming system,
00:35:41.220 --> 00:35:42.980
that leaves you with the most usefull opinion,
00:35:42.980 --> 00:35:45.400
which is the opinion in your environment.
00:35:45.400 --> 00:35:48.400
And it turns out, most people are able
to do this effortlessly.
00:35:48.400 --> 00:35:50.969
laughter
00:35:50.969 --> 00:35:55.529
They have an instinct, that makes them adapt
the dominant opinion in their social environment.
00:35:55.529 --> 00:35:56.599
It's amazing, right?
00:35:56.599 --> 00:36:01.040
And if you are nerd like me,
you don't get this.
00:36:01.040 --> 00:36:08.999
laugingapplause
00:36:08.999 --> 00:36:12.999
So in the world out there,
explanations piggyback on you group allegiance.
00:36:12.999 --> 00:36:15.900
For instance you will find that there is a
substantial group of people that believes
00:36:15.900 --> 00:36:18.380
the minimum wage is good
for the economy and for you
00:36:18.380 --> 00:36:20.549
and another one believes that its bad.
00:36:20.549 --> 00:36:23.470
And its pretty much aligned
with political parties.
00:36:23.470 --> 00:36:25.970
Its not aligned with different
understandings of economy,
00:36:25.970 --> 00:36:30.740
because nobody understands
how the economy works.
00:36:30.740 --> 00:36:36.330
And if you are a nerd you try to understand
the world in terms of what is true and false.
00:36:36.330 --> 00:36:40.680
You try to prove everything by putting it
in some kind of true and false level
00:36:40.680 --> 00:36:43.589
and if you are not a nerd
you try to get to right and wrong
00:36:43.589 --> 00:36:45.609
you try to understand
whether you are in alignment
00:36:45.609 --> 00:36:49.559
with what's objectively right
in your society, right?
00:36:49.559 --> 00:36:55.680
So I guess that nerds are people that have
a defect in there opinion forming system.
00:36:55.680 --> 00:36:57.069
laughing
00:36:57.069 --> 00:37:00.609
And usually that's maladaptive
and under normal circumstances
00:37:00.609 --> 00:37:03.099
nerds would mostly be filtered
from the world,
00:37:03.099 --> 00:37:06.529
because they don't reproduce so well,
because people don't like them so much.
00:37:06.529 --> 00:37:07.960
laughing
00:37:07.960 --> 00:37:11.119
And then something very strange happened.
The computer revolution came along and
00:37:11.119 --> 00:37:14.170
suddenly if you argue with the computer
it doesn't help you if you have the
00:37:14.170 --> 00:37:17.849
normatively correct opinion you need to
be able to understand things in terms of
00:37:17.849 --> 00:37:26.029
true and false, right? applause
00:37:26.029 --> 00:37:29.779
So now we have this strange situation that
the weird people that have this offensive,
00:37:29.779 --> 00:37:33.410
strange opinions and that really don't
mix well with the real normal people
00:37:33.410 --> 00:37:38.119
get all this high paying jobs
and we don't understand how is that happening.
00:37:38.119 --> 00:37:42.599
And it's because suddenly
our maladapting is a benefit.
00:37:42.599 --> 00:37:47.300
But out there there is this world of the
social norms and it's made of paperwalls.
00:37:47.300 --> 00:37:50.349
There are all this things that are true
and false in a society that make
00:37:50.349 --> 00:37:51.549
people behave.
00:37:51.549 --> 00:37:57.390
It's like this japanese wall, there.
They made palaces out of paper basically.
00:37:57.390 --> 00:38:00.339
And these are walls by convention.
00:38:00.339 --> 00:38:04.009
They exist because people agree
that this is a wall.
00:38:04.009 --> 00:38:06.630
And if you are a hypnotist
like Donald Trump
00:38:06.630 --> 00:38:11.109
you can see that these are paper walls
and you can shift them.
00:38:11.109 --> 00:38:14.079
And if you are a nerd like me
you can not see these paperwalls.
00:38:14.079 --> 00:38:20.230
If you pay closely attention you see that
people move and then suddenly middair
00:38:20.230 --> 00:38:22.869
they make a turn. Why would they do this?
00:38:22.869 --> 00:38:24.360
There must be something
that they see there
00:38:24.360 --> 00:38:26.549
and this is basically a normative agreement.
00:38:26.549 --> 00:38:29.690
And you can infer what this is
and then you can manipulate it and understand it.
00:38:29.690 --> 00:38:32.640
Of course you can't fix this, you can
debug yourself in this regard,
00:38:32.640 --> 00:38:34.690
but it's something that is hard
to see for nerds.
00:38:34.690 --> 00:38:38.109
So in some sense they have a superpower:
they can think straight in the presence
00:38:38.109 --> 00:38:39.079
of others.
00:38:39.079 --> 00:38:42.590
But often they end up in their living room
and people are upset.
00:38:42.590 --> 00:38:45.810
laughter
00:38:45.810 --> 00:38:49.789
Learning in a complex domain can not
guarantee that you find the global maximum.
00:38:49.789 --> 00:38:53.970
We know that we can not find truth
because we can not recognize whether we live
00:38:53.970 --> 00:38:57.059
on a plain field or on a
simulated plain field.
00:38:57.059 --> 00:39:00.579
But what we can do is, we can try to
approach a global maximum.
00:39:00.579 --> 00:39:02.339
But we don't know if that
is the global maximum.
00:39:02.339 --> 00:39:05.509
We will always move along
some kind of belief gradient.
00:39:05.509 --> 00:39:09.110
We will take certain elements of
our belief and then give them up
00:39:09.110 --> 00:39:12.650
for new elements of a belief based on
thinking, that this new element
00:39:12.650 --> 00:39:15.049
of belief is better than the one
we give up.
00:39:15.049 --> 00:39:17.079
So we always move along
some kind of gradient.
00:39:17.079 --> 00:39:19.789
and the truth does not matter,
the gradient matters.
00:39:19.789 --> 00:39:23.650
If you think about teaching for a moment,
when I started teaching I often thought:
00:39:23.650 --> 00:39:27.489
Okay, I understand the truth of the
subject, the students don't, so I have to
00:39:27.489 --> 00:39:30.069
give this to them
and at some point I realized:
00:39:30.069 --> 00:39:33.450
Oh, I changed my mind so many times
in the past and I'm probably not going to
00:39:33.450 --> 00:39:35.769
stop changing it in the future.
00:39:35.769 --> 00:39:38.710
I'm always moving along a gradient
and I keep moving along a gradient.
00:39:38.710 --> 00:39:43.099
So I'm not moving to truth,
I'm moving forward.
00:39:43.099 --> 00:39:45.230
And when we teach our kids
we should probably not think about
00:39:45.230 --> 00:39:46.390
how to give them truth.
00:39:46.390 --> 00:39:51.039
We should think about how to put them onto
an interesting gradient, that makes them
00:39:51.039 --> 00:39:55.079
explore the world,
world of possible beliefs.
00:39:55.079 --> 00:40:03.150
applause
00:40:03.150 --> 00:40:05.359
And this possible beliefs
lead us into local minima.
00:40:05.359 --> 00:40:08.150
This is inevitable. This are like valleys
and sometimes this valleys are
00:40:08.150 --> 00:40:11.210
neighbouring and we don't understand
what the people in the neighbouring
00:40:11.210 --> 00:40:15.700
valley are doing unless we are willing to
retrace the steps they have been taken.
00:40:15.700 --> 00:40:19.569
And if you want to get from one valley
into the next, we will have to have some kind
00:40:19.569 --> 00:40:21.789
of energy that moves us over the hill.
00:40:21.789 --> 00:40:27.739
We have to have a trajectory were every
step works by finding reason to give up
00:40:27.739 --> 00:40:30.380
bit of our current belief and adopt a
new belief, because it's somehow
00:40:30.380 --> 00:40:34.739
more useful, more relevant,
more consistent and so on.
00:40:34.739 --> 00:40:38.349
Now the problem is that this is not
monotonous we can not guarantee that
00:40:38.349 --> 00:40:40.499
we're always climbing,
because the problem is, that
00:40:40.499 --> 00:40:44.599
the beliefs themselfs can change
our evaluation of the belief.
00:40:44.599 --> 00:40:50.390
It could be for instance that you start
believing in a religion and this religion
00:40:50.390 --> 00:40:54.299
could tell you: If you give up the belief
in the religion, you're going to face
00:40:54.299 --> 00:40:56.500
eternal damnation in hell.
00:40:56.500 --> 00:40:59.489
As long as you believe in the religion,
it's going to be very expensive for you
00:40:59.489 --> 00:41:02.430
to give up the religion, right?
If you truly belief in it.
00:41:02.430 --> 00:41:05.109
You're now caught
in some kind of attractor.
00:41:05.109 --> 00:41:08.680
Before you believe the religion it is not
very dangerous but once you've gotten
00:41:08.680 --> 00:41:13.019
into the attractor it's very,
very hard to get out.
00:41:13.019 --> 00:41:16.309
So these belief attractors
are actually quite dangerous.
00:41:16.309 --> 00:41:19.920
You can get not only to chaotic behaviour,
where you can not guarantee that your
00:41:19.920 --> 00:41:23.470
current belief is better than the last one
but you can also get into beliefs that are
00:41:23.470 --> 00:41:26.849
almost impossible to change.
00:41:26.849 --> 00:41:33.739
And that makes it possible to program
people to work in societies.
00:41:33.739 --> 00:41:37.529
Social domains are structured by values.
Basically a preference is what makes you
00:41:37.529 --> 00:41:40.769
do things, because you anticipate
pleasure or displeasure,
00:41:40.769 --> 00:41:45.339
and values make you do things
even if you don't anticipate any pleasure.
00:41:45.339 --> 00:41:49.809
These are virtual rewards.
They make us do things, because we believe
00:41:49.809 --> 00:41:51.799
that is stuff
that is more important then us.
00:41:51.799 --> 00:41:55.109
This is what values are about.
00:41:55.109 --> 00:42:00.690
And these values are the source
of what we would call true meaning, deeper meaning.
00:42:00.690 --> 00:42:05.220
There is something that is more important
than us, something that we can serve.
00:42:05.220 --> 00:42:08.769
This is what we usually perceive as
meaningful life, it is one which
00:42:08.769 --> 00:42:12.759
is in the serves of values that are more
important than I myself,
00:42:12.759 --> 00:42:15.749
because after all I'm not that important.
I'm just this machine that runs around
00:42:15.749 --> 00:42:20.789
and tries to optimize its pleasure and
pain, which is kinda boring.
00:42:20.789 --> 00:42:26.329
So my PI has puzzled me, my principle
investigator in the Havard department,
00:42:26.329 --> 00:42:29.349
where I have my desk, Martin Nowak.
00:42:29.349 --> 00:42:33.970
He said, that meaning can not exist without
god; you are either religious,
00:42:33.970 --> 00:42:36.950
or you are a nihilist.
00:42:36.950 --> 00:42:42.789
And this guy is the head of the
department for evolutionary dynamics.
00:42:42.789 --> 00:42:45.769
Also he is a catholic.. chuckling
00:42:45.769 --> 00:42:49.729
So this really puzzled me and I tried
to understand what he meant by this.
00:42:49.729 --> 00:42:53.200
Typically if you are a good atheist
like me,
00:42:53.200 --> 00:42:57.920
you tend to attack gods that are
structured like this, religious gods,
00:42:57.920 --> 00:43:02.940
that are institutional, they are personal,
they are some kind of person.
00:43:02.940 --> 00:43:08.239
They do care about you, they prescribe
norms, for instance don't mastrubate
00:43:08.239 --> 00:43:10.060
it's bad for you.
00:43:10.060 --> 00:43:14.759
Many of this norms are very much aligned
with societal institutions, for instance
00:43:14.759 --> 00:43:20.799
don't questions the authorities,
god wants them to be ruling above you
00:43:20.799 --> 00:43:23.839
and be monogamous and so on and so on.
00:43:23.839 --> 00:43:28.979
So they prescribe norms that do not make
a lot of sense in terms of beings that
00:43:28.979 --> 00:43:31.200
creates world every now and then,
00:43:31.200 --> 00:43:34.619
but they make sense in terms of
what you should be doing to be a
00:43:34.619 --> 00:43:36.730
functioning member of society.
00:43:36.730 --> 00:43:40.799
And this god also does things like it
creates world, they like to manifest as
00:43:40.799 --> 00:43:43.660
burning shrubbery and so on. There are
many books that describe stories that
00:43:43.660 --> 00:43:45.700
these gods have allegedly done.
00:43:45.700 --> 00:43:48.819
And it's very hard to test for all these
features which makes this gods very
00:43:48.819 --> 00:43:54.280
improbable for us. And makes Atheist
very dissatisfied with these gods.
00:43:54.280 --> 00:43:56.569
But then there is a different kind of god.
00:43:56.569 --> 00:43:58.599
This is what we call the spiritual god.
00:43:58.599 --> 00:44:02.410
This spiritual god is independent of
institutions, it still does care about you.
00:44:02.410 --> 00:44:06.489
It's probably conscious. It might not be a
person. There are not that many stories,
00:44:06.489 --> 00:44:10.579
that you can consistently tell about it,
but you might be able to connect to it
00:44:10.579 --> 00:44:15.259
spiritually.
00:44:15.259 --> 00:44:19.470
Then there is a god that is even less
expensive. That is god as a transcendental
00:44:19.470 --> 00:44:23.489
principle and this god is simply the reason
why there is something rather then
00:44:23.489 --> 00:44:28.150
nothing. This god is the question the
universe is the answer to, this is the
00:44:28.150 --> 00:44:29.600
thing that gives meaning.
00:44:29.600 --> 00:44:31.489
Everything else about it is unknowable.
00:44:31.489 --> 00:44:34.190
This is the god of Thomas of Aquinus.
00:44:34.190 --> 00:44:38.089
The God that Thomas of Aquinus discovered
is not the god of Abraham this is not the
00:44:38.089 --> 00:44:39.180
religious god.
00:44:39.180 --> 00:44:43.559
It's a god that is basically a principle
that us ... the universe into existence.
00:44:43.559 --> 00:44:47.140
It's the one that gives
the universe it's purpose.
00:44:47.140 --> 00:44:50.200
And because every other property
is unknowable about this,
00:44:50.200 --> 00:44:52.010
this god is not that expensive.
00:44:52.010 --> 00:44:55.960
Unfortunately it doesn't really work.
I mean Thomas of Aquinus tried to prove
00:44:55.960 --> 00:45:00.049
god. He tried to prove an necessary god,
a god that has to be existing and
00:45:00.049 --> 00:45:02.779
I think we can only prove a possible god.
00:45:02.779 --> 00:45:05.339
So if you try to prove a necessary god,
this god can not exist.
00:45:05.339 --> 00:45:11.650
Which means your god prove is going to
fail. You can only prove possible gods.
00:45:11.650 --> 00:45:13.259
And then there is an even more improper god.
00:45:13.259 --> 00:45:15.890
And that's the god of Aristotle and he said:
00:45:15.890 --> 00:45:20.069
"If there is change in the universe,
something in going to have to change it."
00:45:20.069 --> 00:45:23.640
There must be something that moves it
along from one state to the next.
00:45:23.640 --> 00:45:26.289
So I would say that is the primary
computational transition function
00:45:26.289 --> 00:45:35.079
of the universe.
laughingapplause
00:45:35.079 --> 00:45:38.439
And Aristotle discovered it.
It's amazing isn't it?
00:45:38.439 --> 00:45:41.509
We have to have this because we
can not be conscious in a single state.
00:45:41.509 --> 00:45:43.279
We need to move between states
to be conscious.
00:45:43.279 --> 00:45:45.979
We need to be processes.
00:45:45.979 --> 00:45:50.859
So we can take our gods and sort them by
their metaphysical cost.
00:45:50.859 --> 00:45:53.290
The 1st degree god would be the first mover.
00:45:53.290 --> 00:45:56.069
The 2nd degree god is the god of purpose and meaning.
00:45:56.069 --> 00:45:59.089
3rd degree god is the spiritual god.
And the 4th degree god is this bound to
00:45:59.089 --> 00:46:01.229
religious institutions, right?
00:46:01.229 --> 00:46:03.720
So if you take this statement
from Martin Nowak,
00:46:03.720 --> 00:46:07.759
"You can not have meaning without god!"
I would say: yes! You need at least
00:46:07.759 --> 00:46:14.990
a 2nd degree god to have meaning.
So objective meaning can only exist
00:46:14.990 --> 00:46:19.119
with a 2nd degree god. chuckling
00:46:19.119 --> 00:46:22.269
And subjective meaning can exist as a
function in a cognitive system of course.
00:46:22.269 --> 00:46:24.180
We don't need objective meaning.
00:46:24.180 --> 00:46:27.410
So we can subjectively feel that there is
something more important to us
00:46:27.410 --> 00:46:30.509
and this makes us work in society and
makes us perceive that we have values
00:46:30.509 --> 00:46:34.329
and so on, but we don't need to believe
that there is something outside of the
00:46:34.329 --> 00:46:36.869
universe to have this.
00:46:36.869 --> 00:46:40.650
So the 4th degree god is the one
that is bound to religious institutions,
00:46:40.650 --> 00:46:45.400
it requires a belief attractor and it
enables complex norm prescriptions.
00:46:45.400 --> 00:46:48.430
It my theory is right then it should be
much harder for nerds to believe in
00:46:48.430 --> 00:46:52.039
a 4th degree god then for normal people.
00:46:52.039 --> 00:46:56.489
And what this god does it allows you to
have state building mind viruses.
00:46:56.489 --> 00:47:00.269
Basically religion is a mind virus. And
the amazing thing about these mind viruses
00:47:00.269 --> 00:47:02.489
is that they structure behaviour
in large groups.
00:47:02.489 --> 00:47:06.130
We have evolved to live in small groups
of a few 100 individuals, maybe somthing
00:47:06.130 --> 00:47:07.249
like a 150.
00:47:07.249 --> 00:47:10.059
This is roughly the level
to which reputation works.
00:47:10.059 --> 00:47:15.369
We can keep track of about 150 people and
after this it gets much much worse.
00:47:15.369 --> 00:47:18.290
So in this system where you have
reputation people feel responsible
00:47:18.290 --> 00:47:21.349
for each other and they can
keep track of their doings
00:47:21.349 --> 00:47:23.049
and society kind of sort of works.
00:47:23.049 --> 00:47:27.789
If you want to go beyond this, you have
to right a software that controls people.
00:47:27.789 --> 00:47:32.420
And religions were the first software,
that did this on a very large scale.
00:47:32.420 --> 00:47:35.319
And in order to keep stable they had to be
designed like operating systems
00:47:35.319 --> 00:47:36.039
in some sense.
00:47:36.039 --> 00:47:39.930
They give people different roles
like insects in a hive.
00:47:39.930 --> 00:47:44.529
And they have even as part of this roles is
to update this religion but it has to be
00:47:44.529 --> 00:47:48.380
done very carefully and centrally
because otherwise the religion will split apart
00:47:48.380 --> 00:47:51.719
and fall together into new religions
or be overcome by new ones.
00:47:51.719 --> 00:47:54.259
So there is some kind of
evolutionary dynamics that goes on
00:47:54.259 --> 00:47:55.930
with respect to religion.
00:47:55.930 --> 00:47:58.519
And if you look the religions,
there is actually a veritable evolution
00:47:58.519 --> 00:47:59.739
of religions.
00:47:59.739 --> 00:48:04.789
So we have this Israelic tradition and
the Mesoputanic mythology that gave rise
00:48:04.789 --> 00:48:13.019
to Judaism. applause
00:48:13.019 --> 00:48:16.299
It's kind of cool, right? laughing
00:48:16.299 --> 00:48:36.289
Also history totally repeats itself.
roaring laughterapplause
00:48:36.289 --> 00:48:41.889
Yeah, it totally blew my mind when
I discovered this. laughter
00:48:41.889 --> 00:48:45.039
Of course the real tree of programming
languages is slightly more complicated,
00:48:45.039 --> 00:48:48.599
And the real tree of religion is slightly
more complicated.
00:48:48.599 --> 00:48:51.229
But still its neat.
00:48:51.229 --> 00:48:54.289
So if you want to immunize yourself
against mind viruses,
00:48:54.289 --> 00:48:58.570
first of all you want to check yourself
whether you are infected.
00:48:58.570 --> 00:49:02.809
You should check: Can I let go of my
current beliefs without feeling that
00:49:02.809 --> 00:49:07.670
meaning departures me and I feel very
terrible, when I let go of my beliefs.
00:49:07.670 --> 00:49:11.279
Also you should check: All the other
people around there that don't
00:49:11.279 --> 00:49:17.019
share my belief, are they either stupid,
or crazy, or evil?
00:49:17.019 --> 00:49:19.890
If you think this chances are you are
infected by some kind of mind virus,
00:49:19.890 --> 00:49:23.710
because they are just part
of the out group.
00:49:23.710 --> 00:49:28.059
And does your god have properties that
you know but you did not observe.
00:49:28.059 --> 00:49:32.490
So basically you have a god
of 2nd or 3rd degree or higher.
00:49:32.490 --> 00:49:34.589
In this case you also probably got a mind virus.
00:49:34.589 --> 00:49:37.259
There is nothing wrong
with having a mind virus,
00:49:37.259 --> 00:49:39.920
but if you want to immunize yourself
against this people have invented
00:49:39.920 --> 00:49:44.059
rationalism and enlightenment,
basically to act as immunization against
00:49:44.059 --> 00:49:50.660
mind viruses.
loud applause
00:49:50.660 --> 00:49:53.869
And in some sense its what the mind does
by itself because, if you want to
00:49:53.869 --> 00:49:56.949
understand how you go wrong,
you need to have a mechanism
00:49:56.949 --> 00:49:58.839
that discovers who you are.
00:49:58.839 --> 00:50:03.109
Some kind of auto debugging mechanism,
that makes the mind aware of itself.
00:50:03.109 --> 00:50:04.779
And this is actually the self.
00:50:04.779 --> 00:50:08.339
So according to Robert Kegan:
"The development of ourself is a process,
00:50:08.339 --> 00:50:13.400
in which we learn who we are by making
thing explicit", by making processes that
00:50:13.400 --> 00:50:17.249
are automatic visible to us and by
conceptualize them so we no longer
00:50:17.249 --> 00:50:18.859
identify with them.
00:50:18.859 --> 00:50:22.019
And it starts out with understanding
that there is only pleasure and pain.
00:50:22.019 --> 00:50:25.180
If you are a baby, you have only
pleasure and pain you identify with this.
00:50:25.180 --> 00:50:27.869
And then you turn into a toddler and the
toddler understands that they are not
00:50:27.869 --> 00:50:31.059
their pleasure and pain
but they are their impulses.
00:50:31.059 --> 00:50:34.259
And in the next level if you grow beyond
the toddler age you actually know that
00:50:34.259 --> 00:50:38.880
you have goals and that your needs and
impulses are there to serve goals, but its
00:50:38.880 --> 00:50:40.210
very difficult to let go of the goals,
00:50:40.210 --> 00:50:42.789
if you are a very young child.
00:50:42.789 --> 00:50:46.329
And at some point you realize: Oh, the
goals don't really matter, because
00:50:46.329 --> 00:50:49.509
sometimes you can not reach them, but
we have preferences, we have thing that we
00:50:49.509 --> 00:50:52.950
want to happen and thing that we do not
want to happen. And then at some point
00:50:52.950 --> 00:50:55.869
we realize that other people have
preferences, too.
00:50:55.869 --> 00:50:58.979
And then we start to model the world
as a system where different people have
00:50:58.979 --> 00:51:01.940
different preferences and we have
to navigate this landscape.
00:51:01.940 --> 00:51:06.420
And then we realize that this preferences
also relate to values and we start
00:51:06.420 --> 00:51:09.700
to identify with this values as members of
society.
00:51:09.700 --> 00:51:13.469
And this is basically the stage if you
are an adult being, that you get into.
00:51:13.469 --> 00:51:16.910
And you can get to a stage beyond that,
especially if you have people this, which
00:51:16.910 --> 00:51:20.059
have already done this. And this means
that you understand that people have
00:51:20.059 --> 00:51:23.660
different values and what they do
naturally flows out of them.
00:51:23.660 --> 00:51:26.849
And this values are not necessarily worse
than yours they are just different.
00:51:26.849 --> 00:51:29.450
And you learn that you can hold different
sets of values in your mind at
00:51:29.450 --> 00:51:33.019
the same time, isn't that amazing?
and understand other people, even if
00:51:33.019 --> 00:51:36.660
they are not part of your group.
If you get that, this is really good.
00:51:36.660 --> 00:51:39.269
But I don't think it stops there.
00:51:39.269 --> 00:51:43.019
You can also learn that the stuff that
you perceive is kind of incidental,
00:51:43.019 --> 00:51:45.339
that you can turn it of and you can
manipulate it.
00:51:45.339 --> 00:51:49.940
And at some point you also can realize
that yourself is only incidental that you
00:51:49.940 --> 00:51:52.559
can manipulate it or turn it of.
And that your basically some kind of
00:51:52.559 --> 00:51:57.420
consciousness that happens to run a brain
of some kind of person, that navigates
00:51:57.420 --> 00:52:04.279
the world in terms to get rewards or avoid
displeasure and serve values and so on,
00:52:04.279 --> 00:52:05.130
but it doesn't really matter.
00:52:05.130 --> 00:52:08.119
There is just this consciousness which
understands the world.
00:52:08.119 --> 00:52:11.009
And this is the stage that we typically
call enlightenment.
00:52:11.009 --> 00:52:14.549
In this stage you realize that you are not
your brain, but you are a story that
00:52:14.549 --> 00:52:25.640
your brain tells itself.
applause
00:52:25.640 --> 00:52:29.630
So becoming self aware is a process of
reverse engineering your mind.
00:52:29.630 --> 00:52:32.890
Its a different set of stages in which
to realize what goes on.
00:52:32.890 --> 00:52:33.799
So isn't that amazing.
00:52:33.799 --> 00:52:38.930
AI is a way to get to more self awareness?
00:52:38.930 --> 00:52:41.319
I think that is a good point to stop here.
00:52:41.319 --> 00:52:44.499
The first talk that I gave in this series
was 2 years ago. It was about
00:52:44.499 --> 00:52:45.979
how to build a mind.
00:52:45.979 --> 00:52:49.670
Last year I talked about how to get from
basic computation to consciousness.
00:52:49.670 --> 00:52:53.709
And this year we have talked about
finding meaning using AI.
00:52:53.709 --> 00:52:57.470
I wonder where it goes next.
laughter
00:52:57.470 --> 00:53:22.769
applause
00:53:22.769 --> 00:53:26.489
Herald: Thank you for this amazing talk!
We now have some minutes for Q&A.
00:53:26.489 --> 00:53:31.190
So please line up at the microphones as
always. If you are unable to stand up
00:53:31.190 --> 00:53:36.430
for some reason please very very visibly
rise your hand, we should be able to dispatch
00:53:36.430 --> 00:53:40.099
an audio angle to your location
so you can have a question too.
00:53:40.099 --> 00:53:44.030
And also if you are locationally
disabled, you are not actually in the room
00:53:44.030 --> 00:53:49.069
if you are on the stream, you can use IRC
or twitter to also ask questions.
00:53:49.069 --> 00:53:50.989
We also have a person for that.
00:53:50.989 --> 00:53:53.779
We will start at microphone number 2.
00:53:53.779 --> 00:53:59.940
Q: Wow that's me. Just a guess! What
would you guess, when can you discuss
00:53:59.940 --> 00:54:04.559
your talk with a machine,
in how many years?
00:54:04.559 --> 00:54:07.400
Joscha: I don't know! As a software
engineer I know if I don't have the
00:54:07.400 --> 00:54:12.619
specification all bets are off, until I
have the implementation. laughter
00:54:12.619 --> 00:54:14.509
So it can be of any order of magnitude.
00:54:14.509 --> 00:54:18.249
I have a gut feeling but I also know as a
software engineer that my gut feeling is
00:54:18.249 --> 00:54:23.450
usually wrong, laughter
until I have the specification.
00:54:23.450 --> 00:54:28.200
So the question is if there are silver
bullets? Right now there are some things
00:54:28.200 --> 00:54:30.569
that are not solved yet and it could be
that they are easier to solve
00:54:30.569 --> 00:54:33.469
than we think, but it could be that
they're harder to solve than we think.
00:54:33.469 --> 00:54:36.710
Before I stumbled on this cortical
self organization thing,
00:54:36.710 --> 00:54:40.719
I thought it's going to be something like
maybe 60, 80 years and now I think it's
00:54:40.719 --> 00:54:47.289
way less, but again this is a very
subjective perspective. I don't know.
00:54:47.289 --> 00:54:49.240
Herald: Number 1, please!
00:54:49.240 --> 00:54:55.589
Q: Yes, I wanted to ask a little bit about
metacognition. It seems that you kind of
00:54:55.589 --> 00:55:01.329
end your story saying that it's still
reflecting on input that you get and
00:55:01.329 --> 00:55:04.900
kind of working with your social norms
and this and that, but Colberg
00:55:04.900 --> 00:55:11.839
for instance talks about what he calls a
postconventional universal morality
00:55:11.839 --> 00:55:17.420
for instance, which is thinking about
moral laws without context, basically
00:55:17.420 --> 00:55:23.069
stating that there is something beyond the
relative norm that we have to each other,
00:55:23.069 --> 00:55:29.579
which would only be possible if you can do
kind of, you know, meta cognition,
00:55:29.579 --> 00:55:32.599
thinking about your own thinking
and then modifying that thinking.
00:55:32.599 --> 00:55:37.229
So kind of feeding back your own ideas
into your own mind and coming up with
00:55:37.229 --> 00:55:43.779
stuff that actually can't get ...
well processing external inputs.
00:55:43.779 --> 00:55:48.469
Joscha: Mhm! I think it's very tricky.
This project of defining morality without
00:55:48.469 --> 00:55:53.119
societies exists longer than Kant of
course. And Kant tried to give this
00:55:53.119 --> 00:55:56.869
internal rules and others tried to.
I find this very difficult.
00:55:56.869 --> 00:56:01.069
From my perspective we are just moving
bits of rocks. And this bits of rocks they
00:56:01.069 --> 00:56:07.589
are on some kind of dust mode in a galaxy
out of trillions of galaxies and how can
00:56:07.589 --> 00:56:08.609
there be meaning?
00:56:08.609 --> 00:56:11.180
It's very hard for me to say:
00:56:11.180 --> 00:56:13.969
One chimpanzee species is better than
another chimpanzee species or
00:56:13.969 --> 00:56:16.559
a particular monkey
is better than another monkey.
00:56:16.559 --> 00:56:18.539
This only happens
within a certain framework
00:56:18.539 --> 00:56:20.160
and we have to set this framework.
00:56:20.160 --> 00:56:23.700
And I don't think that we can define this
framework outside of a context of
00:56:23.700 --> 00:56:26.420
social norms, that we have to agree on.
00:56:26.420 --> 00:56:29.650
So objectively I'm not sure
if we can get to ethics.
00:56:29.650 --> 00:56:33.769
I only think that is possible based on
some kind of framework that people
00:56:33.769 --> 00:56:38.339
have to agree on implicitly or explicitly.
00:56:38.339 --> 00:56:40.630
Herald: Microphone number 4, please.
00:56:40.630 --> 00:56:46.559
Q: Hi, thank you, it was a fascinating talk.
I have 2 thought that went through my mind.
00:56:46.559 --> 00:56:51.589
And the first one is that it's so
convincing the models that you present,
00:56:51.589 --> 00:56:56.709
but it's kind of like you present
another metaphor of understanding the
00:56:56.709 --> 00:57:01.670
brain which is still something that we try
to grasp on different levels of science
00:57:01.670 --> 00:57:07.469
basically. And the 2nd one is that your
definition of the nerd who walks
00:57:07.469 --> 00:57:10.950
and doesn't see the walls is kind of
definition... or reminds me
00:57:10.950 --> 00:57:15.229
Richard Rortys definition of the ironist
which is a person who knows that their
00:57:15.229 --> 00:57:20.799
vocabulary is finite and that other people
have also a finite vocabulary and
00:57:20.799 --> 00:57:24.599
then that obviously opens up the whole question
of meaning making which has been
00:57:24.599 --> 00:57:28.979
discussed in so many
other disciplines and fields.
00:57:28.979 --> 00:57:32.930
And I thought about Darridas
deconstruction of ideas and thoughts and
00:57:32.930 --> 00:57:36.300
Butler and then down the rabbit hole to
Nietzsche and I was just wondering,
00:57:36.300 --> 00:57:39.009
if you could maybe
map out other connections
00:57:39.009 --> 00:57:44.430
where basically not AI helping us to
understand the mind, but where
00:57:44.430 --> 00:57:49.819
already existing huge, huge fields of
science, like cognitive process
00:57:49.819 --> 00:57:53.359
coming from the other end could help us
to understand AI.
00:57:53.359 --> 00:57:59.680
Joscha: Thank you, the tradition that you
mentioned Rorty and Butler and so on
00:57:59.680 --> 00:58:02.989
are part of a completely different belief
attractor in my current perspective.
00:58:02.989 --> 00:58:06.209
That is they are mostly
social constructionists.
00:58:06.209 --> 00:58:10.880
They believe that reality at least in the
domains of the mind and sociality
00:58:10.880 --> 00:58:15.359
are social constructs they are part
of social agreement.
00:58:15.359 --> 00:58:17.190
Personally I don't think that
this is the case.
00:58:17.190 --> 00:58:19.630
I think that patterns that we refer to
00:58:19.630 --> 00:58:23.890
are mostly independent of your mind.
The norms are part of social constructs,
00:58:23.890 --> 00:58:28.099
but for instance our motivational
preferences that make us adapt or
00:58:28.099 --> 00:58:32.719
reject norms, are something that builds up
resistance to the environment.
00:58:32.719 --> 00:58:35.660
So they are probably not part
of social agreement.
00:58:35.660 --> 00:58:41.569
And the only thing I can invite you to is
try to retrace both of the different
00:58:41.569 --> 00:58:45.640
belief attractors, try to retrace the
different paths on the landscape.
00:58:45.640 --> 00:58:48.529
All this thing that I tell you, all of
this is of course very speculative.
00:58:48.529 --> 00:58:52.390
These are that seem to be logical
to me at this point in my life.
00:58:52.390 --> 00:58:55.400
And I try to give you the arguments
why I think that is plausible, but don't
00:58:55.400 --> 00:58:59.109
believe in them, question them, challenge
them, see if they work for you!
00:58:59.109 --> 00:59:00.559
I'm not giving you any truth.
00:59:00.559 --> 00:59:05.720
I'm just going to give you suitable encodings
according to my current perspective.
00:59:05.720 --> 00:59:11.739
Q:Thank you!
applause
00:59:11.739 --> 00:59:15.099
Herald: The internet, please!
00:59:19.179 --> 00:59:26.029
Signal angel: So, someone is asking
if in this belief space you're talking about
00:59:26.029 --> 00:59:30.109
how is it possible
to get out of local minima?
00:59:30.109 --> 00:59:33.959
And very related question as well:
00:59:33.959 --> 00:59:38.530
Should we teach some momentum method
to our children,
00:59:38.530 --> 00:59:41.599
so we don't get stuck in a local minima.
00:59:41.599 --> 00:59:44.829
Joscha: I believe at some level it's not
possible to get out of a local minima.
00:59:44.829 --> 00:59:50.329
In an absolute sense, because you only get
to get into some kind of meta minimum,
00:59:50.329 --> 00:59:56.769
but what you can do is to retrace the
path that you took whenever you discover
00:59:56.769 --> 00:59:59.989
that somebody else has a fundamentally
different set of beliefs.
00:59:59.989 --> 01:00:02.769
And if you realize that this person is
basically a smart person that is not
01:00:02.769 --> 01:00:07.359
completely insane but has reasons to
believe in their beliefs and they seem to
01:00:07.359 --> 01:00:10.579
be internally consistent it's usually
worth to retrace what they
01:00:10.579 --> 01:00:12.180
have been thinking and why.
01:00:12.180 --> 01:00:15.930
And this means you have to understand
where their starting point was and
01:00:15.930 --> 01:00:18.279
how they moved from their current point
to their starting point.
01:00:18.279 --> 01:00:22.219
You might not be able to do this
accurately and the important thing is
01:00:22.219 --> 01:00:25.369
also afterwards you discover a second
valley, you haven't discovered
01:00:25.369 --> 01:00:27.059
the landscape inbetween.
01:00:27.059 --> 01:00:30.839
But the only way that we can get an idea
of the lay of the land is that we try to
01:00:30.839 --> 01:00:33.200
retrace as many paths as possible.
01:00:33.200 --> 01:00:36.339
And if we try to teach our children, what
I think what we should be doing is:
01:00:36.339 --> 01:00:38.650
To tell them how to explore
this world on there own.
01:00:38.650 --> 01:00:43.900
It's not that we tell them this is the
valley, basically it's given, it's
01:00:43.900 --> 01:00:47.599
the truth, but instead we have to tell
them: This is the path that we took.
01:00:47.599 --> 01:00:51.239
And these are the things that we saw
inbetween and it is important to be not
01:00:51.239 --> 01:00:54.390
completely naive when we go into this
landscape, but we also have to understand
01:00:54.390 --> 01:00:58.170
that it's always an exploration that
never stops and that might change
01:00:58.170 --> 01:01:01.140
everything that you believe now
at a later point.
01:01:01.140 --> 01:01:05.700
So for me it's about teaching my own
children how to be explorers,
01:01:05.700 --> 01:01:10.950
how to understand that knowledge is always
changing and it's always a moving frontier.
01:01:10.950 --> 01:01:17.230
applause
01:01:17.230 --> 01:01:22.259
Herald: We are unfortunately out of time.
So, please once again thank Joscha!
01:01:22.259 --> 01:01:24.069
applause
Joscha: Thank you!
01:01:24.069 --> 01:01:28.239
applause
01:01:28.239 --> 01:01:32.719
postroll music
01:01:32.719 --> 01:01:40.000
subtitles created by c3subtitles.de
Join, and help us!