Herald: I have the great pleasure to
announce Joscha, who will give us a great
talk with the title "The Ghost in the
Machine" and he will talk about
consciousness of our mind and of computers
and somehow also tell us how we can learn
from A.I. systems about our own brains.
And I think this is a very curious question.
So please give it up for Joscha.
Applause
Joscha: Good evening. This is the 5th
of a talk in a series of talks on how to
get from computation to consciousness and
to understand our condition in the
universe based on concepts that I mostly
learned by looking at artificial
intelligence and computation and it mostly
tackles the big philosophical questions:
What can I know? What is true? What is
truth? Who am I? Which means the question
of epistemology, of ontology, of
metaphysics, and philosophy of mind and
ethics.
And to clear some of the terms
that we are using here:
What is intelligence? What's a mind?
What's a self? What's consciousness?
How are mind and consciousness
realized in the universe?
Intelligence I think is the ability to
make models.
It's not the same thing
as being smart, which is the
ability to reach your goals or being wise,
which is the ability to pick the right
goals. But it's just the ability to
make models of things.
And you can regulate them later using
these models, but you don't have to.
And the mind is this thing that observes
the universe itself
as an identification with
properties and purposes.
What a thing thinks it is. And then
you have consciousness, which is
the experience of what it's like
to be a thing.
And, how our mind of consciousness
is realized in the universe,
this is commonly called the
mind-body problem and it's been
puzzling philosophers and people of
all proclivities for thousands of years.
So what's going on? How's it possible that
I find myself in a universe and I seem to
be experiencing myself in that universe?
How does this go together and how is this,
what's going on here? The traditional
answer to this is called dualism and the
conception of dualism is that - in our
culture at least, this dualist idea that
you have a physical world and a mental
world and they coexist somehow and my mind
experiences this mental world and my body
can do things in the physical world and
the difficulty of this dualist conception
is how do these two planes of existence
interact. Because physics is defined as
causally closed, everything that
influences things in the physical world is
by itself an element of physics. So an
alternative is idealism which says that
there is only a mental world. We only
exist in a dream and this dream is being
dreamt by a mind on a higher plane of
existence. And difficulty with this, it's
very hard to explain that mind of a higher
plane of existence. Just put it there, why
is it doing this? And in our culture the
dominant theory is materialism and is
basically there is only a physical world
nothing else. And the physical world
somehow is responsible for the creation of
the mental world. It's not quite clear how
this happens. And the answer that I am
suggesting, is functionalism which means
that indeed we exist only in a dream.
So these ideas of materialism and idealism
are not in opposition. They are
complementary because this dream is being
dreamt by a mind on a higher plane of
existence, but this higher plane of
existence is the physical world. So we are
being dreamt in the neocortex of a primate
that lives in a physical universe and the
world that we experience is not the
physical world. It's a dream generated by
the neocortex - the same circuits that
make dreams at night make them during the
day. You can show this, and you live in
this virtual reality being generated in
there and the self as a character in that
dream. And it seems to take care of
things. It seems to explain what's going
on. It explains why a miracle seems to be
possible and why I can look into the
future but cannot break the bank somehow.
And even though this theory explains this,
how shouldn't I be more agnostic? Are
there not alternatives that I should be
considering? Maybe the narratives of our
big religions and so on. I think we should
be agnostic. So the first rule of
epistemology says that the confidence in
the belief must equal the weight of the
evidence supporting it. Once we stumble on
that rule you can test all the
alternatives and see if one of them is
better. And I think what this means is you
have to have all the possible beliefs, you
should entertain them all. But you should
not have any confidence in them. You
should shift your confidence around based
on the evidence. So for instance it is
entirely possible that this universe was
created by a supernatural being, and it's
a big conspiracy, and it actually has
meaning and it cares about us and our
existence here means something.
But um, there is no experiment that can
validate this. A guy coming down from a
burning mount, from a burning
bush, that you've talked to on a
mountaintop? That's not a kind of experi-
ment that gives you valid evidence, right?
So intelligence is the ability to
make models and intelligence is a property
that is beyond the grasp of a single
individual. A single individual is not
that smart. We cannot figure out even tur-
ing complete languages all by ourselves.
To do this you need an intellectual
tradition that lasts a few hundred years
at least. So civilizations have more
intelligence than individuals. But
individuals often have more intelligence
than groups and whole generations and
that's because groups and generations tend
to converge on ideas; they have consensus
opinions. I'm very wary of consensus
opinions because you know how hard it is
to understand which programming language
is the best one for which purpose. There
is no proper consensus. And that's a
relatively easy problem. So when there's a
complex topics and all the experts agree,
there are forces at work that are
different than the forces that make them
search for truth. These consensus-building
forces, they're very suspicious to me. And
if you want to understand what's true you
have to look for means and motive. And you
have to be autonomous in doing this, so
individuals typically have better ideas
than generations or groups. But as I
said, civilizations have more intelligence
than individuals. What does a
civilizational intellect look like? The
civilization intellect is something like a
global optimum of the modeling function.
It's something that has to be built over
thousands of years in an unbroken
intellectual tradition. And guess what,
this doesn't really exist in human
history. Every few hundred years, there's
some kind of revolution. Somebody opens
the doors to the knowledge factories and
gets everybody out and burns down the
libraries. And a couple generations later,
the knowledge worker drones of the new
king realize "Oh my God we need to rebuild
this thing, this intellect." And then they
create something in its likeness, but they
make mistakes in the foundation. So this
intellect tends to have scars. Like our
civilization intellect has a lot of scars
in it, that make it hard-to-difficult
to understand concepts like self
and consciousness and mind. So, the mind
is something that observes the universe,
and the neurons and neurotransmitters are
the substrate. And the human intellect and
the working memory is the current binding
state, how do the different elements fit
together in our mind? And the self is the
identification is what we think we are and
what we want to happen. And consciousness
is the contents of our attention, it makes
knowledge available throughout the mind.
And civilizational intellect is very
similar: society is observe the universe,
people and resources are the substrate,
the generation is the current binding
state, and culture is the identification
with what we think we are and what we want
to happen. And media is the contents of
our attention and make knowledge available
throughout society. So the culture is
basically the self of civilization, and
media is its consciousness. How is it
possible to model a universe? Let's take a
very simple universe like the Mandelbrot
fractal. It can be defined by a little bit
of code. It's a very simple thing, you just
take a pair of numbers, you square it, you
add the same pair of numbers. And you do
this infinitely often, and typically this
goes to infinity very fast. There's a
small area around the origin of the number
pair, so between -1 and +1 and
so on, where you have an area where this
converges, where it doesn't go to infinity
and that is where you make black dots and
then you get this famous structure, the
Mandelbrot fractal. And because this
divergence and convergence of the function
can take many loops and circles and so on,
a very complicated shape a very
complicated outline, an infinitely
complicated outline there. So there is an
infinite amount of structure in this
fractal. And now imagine you happen
to live in this fractal and you are in a
particular place in it, and you don't know
where that is where that place is. You
don't even know the generator function of
the whole thing. But you can still predict
your neighborhood. So you can see, omg,
I'm in some kind of a spiral, it turns
to the left, goes to the left, and goes
to left, and becomes smaller, so we can
predict and suddenly it ends. Why does it
end? A singularity. Oh, it hits another
spiral. There's a law when a spiral hits
another spiral, it ends. And something
else happens. So you look and then you see
oh, there are certain circumstances where
you have, for instance, an even number of
spirals hitting each other instead of an
odd number. And then you discover another
law. And if you make like 50 levels of
of these laws, and this is a good
description that locally compresses the
universe. So the Mandelbrot fractal is
locally compressable. You find local
order that predicts the neighborhood if
you are inside of that fractal. The global
modelling function of the Mandelbrot
fractal is very, very easy. It's an
interesting question: how difficult is the
global modelling function of our universe?
Even if we know it maybe it doesn't
help us that much, it will be a big
breakthrough for physics when we finally
find it, it will be much shorter than the
standard model, as I suspect, but we still
don't know where we are. And this means we
need to make a local model of what's
happening. So in order to do this we
separate the universe into things. Things
are small state spaces and transition
functions that tell you how to get from
state to state. And if the function is
deterministic it is independent of time,
it gives the same result every time you
call it. For an indeterministic function
it gives a different result every time, so
it doesn't compress well. And causality
means that you have separate several
things and they influence each other's
evolution thrugh a shared interface.
Right? So causality is an artifact of
describing the universe as separate
things. And the universe is not separate
things, it's one thing, but we get have to
describe it as separate things because we
cannot observe the whole thing. So what's
true? There seems to be a particular way
in which the universe seems to be and
that's the ground rules of the universe
and it's inaccessible to us. And what's
accessible to us is our own models of the
universe. The only thing that we can
experience, and this is basically a set
of theories that can explain the
observations. And truth in this sense is a
property of language and there are
different languages that we can use like
geometry and natural language and so on
and ways of representing and changing
models of our languages and several
intellectual traditions have developed
their own languages. And this has led to
problems. Our civilization basically has
as its founding myth this attempt to build
this global optimum modelling function.
This is a tower that is meant to reach the
heavens. And it fell apart because people
spoke different languages. The different
practitioners in the different fields and
they didn't understand each other and the
whole building collapsed. And this is in
some sense the origin of our present
civilization and we are trying to mend
this and find better languages. So whom
can we turn to? We can turn to the
mathematicians maybe because mathematics
is the domain of all languages.
Mathematics is really cool when you think
about it. It's a universal code library,
maintained for several centuries in its
present form. There is not even version
management, it's one version. There is
pretty much unified namespace. They have
to use a lot of the Unicode to make it
happen. It's ugly but there you go! It has
no central maintainers, not even a code of
conduct, beyond what you can infer
yourself.
laughter
But there are some problems at the
foundation that they discovered.
Shouted from the audience: en sehr stabile
Joscha: Can you infer this is a good
conduct? ??????????
Yelling from the audience: Ya!
Joscha: Okay. Power to you.
laughter
Joscha: In 1874 discovered when you looked
at the cardinality of a set, that when you
described natural numbers using set
theory, that the cardinality of a set
grows slower than the cardinality of the
set of its subsets. So if you look at the
set of the subsets of the set, it's always
larger than the cardinality of the number
of members of the set. Clear? Right. If
you take the infinite set, it has
infinitely many members: omega. You
take the cardinality of the set of the
subsets of the infinite set, it's also an
infinite number, but it's a larger one. So
it's a number that is larger than the
previous omega. Okay that's fine. Now we
have the cardinality of the set of all
sets. You make the total set: The set
where you put all the sets that could
possibly exist and put them all together,
right? That has also infinitely many
members, and it has more than the
cardinality of the set of the subsets of
the infinite set. That's fine. But now you
look at the cardinality of the set of all
the subsets of the total set. The problem
is, that the total set also contains the
set of its subsets, right? It's because it
contains all the sets. Now you have a
contradiction: Because the cardinality of
the set of the subsets of the total set is
supposed to be larger. And yet it seems to
be the same set and not the same set. It's
an issue! So mathematicians got puzzled
about this, and the philosopher Bertrand
Russell said: "Maybe we just exclude those
sets that don't contain themselves",
right? We only look at the set of sets
that don't contain themselves. Isn't that
a solution? Now the problem is: Does the
set of the sets that doesn't contain
themselves contain itself? If it does, it
doesn't, and if it doesn't, it does.
That's an issue!
laughter
So David Hilbert, who was some
kind of a community manager back then,
said: "Guys, fix this! This is an issue,
mathematics is precious, we are in
trouble. Please solve meta mathematics."
And people got to work. And after a short
amount of time Kurt Gödel, who had looked
at this in earnest said "oh that's an issue,
issue. You know, as soon as we allow these
kinds of loops - and we cannot really
exclude these loops - then our mathematics
crashes." So that's an issue, it's called
Unentscheidbarkeit. And then Alan Turing
came along a couple of years later, and he
constructed a computer to make that proof.
He basically said "If you build a machine
that does these mathematics, and the
machine takes infinitely many steps,
sometimes, for making a proof, then we
cannot know whether this proof
terminates." So it's a similar issue for
the Unentscheidbarkeit. That's a big
issue, right? So we cannot basically build
a machine in mathematics that runs
mathematics without crashing. But the good
news is, Turing didn't stop working there
and he figured out together with Alonzo
Church - not together, independently but
at the same time - that we can build a
computational machine, that runs all of
computation. So computation is a universal
thing. And it's almost as good as
mathematics. Computation is constructive
mathematics. The tiny, neglected subset of
mathematics, where you have to show the
money. In order to say that something is
true, you have to find that object that is
true. You have to actually construct it.
So there are no infinities, because you
cannot construct an infinity. You add
things and you have unboundedness maybe,
but not infinity. And so this part of
computation, mathematics is the one that
can be implemented. It's constructive
mathematics. It's the good part. And
computing, a computer is very easy to
make, and all universal computers have the
same power. That's called the Chuch-Turing
thesis. And Turing even didn't even stop
there. The obvious conclusion is that,
human minds are probably not in the class
of these mathematical machines, that even
God doesn't know how to build if it has to
be done in any language. But it's a
computational machine. And it also means
that all machines that human minds ever
encounter, mathematics that human minds
encounter,
will be computational mathematics.
So how can you bridge the gap
from mathematics to philosophy? Can we
find a language that is more powerful than
most of the languages that we look at
mathematics, which are very narrowly
defined language, so every symbol, we know
exactly what it means.
When we look at the real world,
we often don't know what things mean,
and our concepts, we're not quite
sure what they mean. Like culture is a
very vague ambigous concept. So what I
said is only approximately true there. Can
we deal with this conceptual ambiguity?
Can we build a programming language for
thought, where words mean things that
they're supposed to mean? And this was the
project of Ludwig Wittgenstein. He just
came back from the war and had a lot of
thoughts. Then he put these thoughts
into a book which is called the Tractatus.
And it's one of the most beautiful books
in the philosophy of the 20th century. And
it starts with the words "Die Welt ist
alles, was der Fall ist. Die Welt ist die
Gesamtheit der Fakten, nicht der Dinge.
Die Welt ist bestimmt, bei den Fakten, und
dadurch, dass diese all die Fakten sind.",
usw. This book is about 75 pages long and
it's a single thought. It's not meant to
be an argument to convince a philosopher.
It's an attempt by a guy who was basically
a coder, an AI scientist, to reverse
engineer the language of his own thinking.
And make it deterministic, to make it
formal, to make it mean something. And he
felt back then that he was successful, and
had a tremendous impact on philosophy,
which was largely devastating, because the
philosophers didn't know what he was on
about. They thought it's about natural
language and not about coding.
And he wrote this in 1918
so before Alan Turing defined,
what a computer is. But he would already
smell what a computer is. He already knew
about university of computation. He knew
that a NAND gate is sufficient to explain
all of boolean algebra and it's equivalent
to other things. So what he basically did,
was, he pre-empted the logicists' program
of artificial intelligence which started
much later in the 1950s. And he ran into
troubles with it. In the end he wrote the
book "Philosophical Investigations", where
he concluded, that his project basically
failed. And that there is a... because the
world is too complex and too ambiguous to
deal with this. And symbolic AI was mostly
similar to Wittgenstein's program. So
classical AI is symbolic. You analyze a
problem, you find an algorithm to solve
it. And what we now have in AI, is mostly
sub-symbolic. So we have algorithms, that
learn the solution of a problem by
themselves. And it's tempting to think,
that the next thing what we have will be
meta-learning. That you have algorithms,
that learn to learn the solution to the
problem. Meanwhile, let's look at how we
can make models. Information is a
discernible difference. It's about change.
All information is about change. The
information that is not about change, you
cannot see a causal effect on the world,
because it stays the same, right? And the
meaning of information is its relationship
to change in other information. So if you
see a blip on your retina, the meaning
of that blip on your retina is the
relationships you discover to other blips
on your retina. It could be for instance,
if you see a sequence of such blips, that
are adjacent to each other, first order
model, you see a moving dust mote or a
moving dot on your retina. And a higher
order model makes it possible to
understand: "Oh, it's part of something
larger! There's people moving in a three
dimensional room and they exchange
ideas." And this is maybe the best model
you end up with. That's the local
compression, that you can make of your
universe, based on correlating blips on
your retina. And for those blips where you
don't find a relationship, which is a
function that your brain can compute,
they are noise. And there's a lot of noise
on our retina, too. So what's a function?
A function is basically a gear box: It has
n input levers and 1 output lever.
And when you move the input levers they
translate to movement of the output
levers, right? And the function can be
realized in many ways: maybe you cannot
open the gear box, and what happened in
this function could be for instance, two
sprockets, which do this. Or you can have
the same results with levers and pulleys.
And so you don't know what's inside, but
you can express it as this does: two times
the input value, right? And you can have a
more difficult case, where you have
several input values and they all
influence the output value. So how do you
figure it out? A way to do this, is, you
only move one input value at a time and
you wiggle it a little bit at every
position and see how much this translates
into wiggling of the output value. This is
what we call taking partial differential.
And it's simple to do this
for this case where you just have to
multiply it by two. And the bad case is
like this: you have a combination lock and
it has maybe 1000 bit input value, and
only if you have exactly the right
combination of the input bits you have a
movement of the output bit. And you're not
going to figure this out until your sun
burns out, right? So there's no way you
can decipher this function. And the
functions that we can model are somewhere
in between, something like this: So you
have 40 million input images and you want
to find out, whether one of these images
displays a cat, or a dog, or something
else. So what can you do with this? You
cannot do this all at once, right? So you
need to take this image classifier
function and disassemble it into small
functions that are very well-behaved, so
you know what to do with them. And an
example for such a function is this one:
it's one, where you have this input
layer and it translates to the output
value with a pulley. And it has some
stopper that limits the movement of the
output value. And you have some pivot. And
you can take this pivot and you can shift
it around. And by shifting this pivot, you
decide, how much the input value
contributes to the output value. Right, so
you shift it, you can even make a
negative, so it shifts in the opposite
direction, and you shifted beyond this
connection point of the pulley. And you
can also have multiple input values, that
use the same pulley and pull together,
right? So they add up to the output
value. That's a pretty nice, neat function
approximator, that basically performs a
weighted sum of the input values, and maps
it to a range-constrained output value.
And you can now shift these pivots, these
weights around to get to different output
values. Now let's take this thing and
build it into lots of layers, so the
outputs are the inputs of the next layer.
And now you connect this to your image. If
you use ImageNet, the famous database that
I mentioned earlier, that people use for
testing their vision algorithms, have
something like one and half million bits
as an input image. Now you take these
bits and connect them to the input layer.
I was too lazy to draw all of them, so I
made this very simplified, it's also more
layers. And so you set them, according to
the bits of the input image, and then this
will propagate the movement of the input
layer to the output. And the output will
move and it will point to some direction,
which is usually the wrong one. Now, to
make this better, you train it. And you do
this by taking this output lever and shift
it a little bit, not too much, into the
right direction. If you do it too much,
you destroy everything you did before.
And now you will see, how much, in which
direction you need to shift the pivots, to
get the result closer to the desired
output value, and how much each of the
inputs contributed to the mistakes, so to
the error. And you take this error and you
propagate it backwards. It's called back
propagation. And you do this quite often.
So you do this for tens of thousands of
images. If you do just character
recognition, then it's a very simple thing
a few thousands or ten thousands of
examples will be enough. And for something
like your image database you need lots and
lots of more data. You need millions of
input images to get to any result. And if
it doesn't work, you just try a different
arrangement of layers. And the thing is
eventually able to learn an algorithm with
as up to as many steps as there are
layers, and has some difficulties learning
loops, you need tricks to make that
happen, and its difficult to make this
dynamic, and so on. And it's a bit
different from what we do, because our
mind is not testable in classification.
It learns per continuous perception, so
we learn a single function. A model of the
universe is not a bunch of classifiers,
it's one single function. An operator that
explains all your sensory data and we call
this operator the universe, right?
It's the world, that we live in. And every
thing that we learn and see is part of this
universe. So even when you see something
in a movie on a screen, you explain this
as part of the universe by telling
yourself "the things that I'm seeing here,
they're not real. They just happen in a
movie." So this brackets a sub-part of
this universe into a sub-element of this
function. So you can deal with it and it
doesn't contradict the rest. And the
degrees of freedom of our model try to
match the degrees of freedom of the
universe. How can we get a neural network
to do this? So, there are many tricks. And
a recent trick that has been invented is a
GAN. It's a Generative Adversarial neural
Network. It consists of two networks: one
generator that invents data, that look
like the real world, and the discriminator
that tries to find out, if the stuff that
the generator produces is real or fake.
And they both get trained with each other.
So they together get better and better in
an adversarial competition. And the
results of this are now really good. So
this is work by Tero Karras, Samuli Laine
and Timo Aila, that they did at NVIDIA
this year and it's called StyleGAN. And
this StyleGAN is able to abstract over
different features and combine them. The
styles are basically parameters, they're
free variables of the model at different
levels of importance. And so you take from
the - in the top row you see images, where
it takes the variables: gender, age, hair
length, and so on, and glasses and pose.
And in the bottom where it takes
everything else and combines this, and
every time you get a
valid interpretation between them.
drinks water
So, you have these coarse styles,
which are:
the pose, the hair, the face shape,
your facial features and the eyes,
the lowest level is just the colors. Let's see
see what happens if you combine them.
The variables that change here, in machine
learning, we call them the latent
variables of that.
Of the space of objects that has been
described by this.
And it's tempting to think, that this is
quite similar to how our imagination works
right? But these artificial neurons, they
are very, very different from what
biological neurons do. Biological neurons
are essentially little animals, that are
rewarded for firing at the right moment.
And they try to fire because otherwise
they do not get fed, and they die, because
the organism doesn't need them, and
culls them. And they learn which
environmental states predict anticipated
reward. So they grow around and find
different areas that give them predictions
of when they should fire. And they connect
with each other to form small collectives,
that are better at this task of predicting
anticipated reward. And as a side effect
they produce exactly the regulation that
the organism needs. Basically they learn,
what the organism feeds them for.
And yet they're able
to learn very similar things.
And it's because, in some sense, they are
Turing complete. They are machines that
are able to learn the statistics of the
data.
So, a general model: What it does, is,
it encodes patterns to predict other
present and future patterns. And it's a
network of relationships between the
patterns, which are all the invariants
that we can observe. And there are free
parameters, which are variables that hold
the state to encode this variant. So we
have patterns, and we have sets of
possible values which are variables. And
they constrain each other in terms of
possibility, what values are compatible
with each other. And they also can train
future values. And they are connected also
with probabilities. The probabilities tell
you, when you see a certain thing, how
probable it is that the world is in that
state. And this tells you how your model
should converge. So, until you are in
a state where your model is coherent, and
everything is possible in it, how do you
get to one of the possible states based on
your inputs? And this is determined by
probability. And the thing that gives
meaning and color to what you perceive is
called valence. And it depends on your
preferences: the things that give you
pleasure and pain, that makes you
interested in stuff. And there are also
norms, which are beliefs without priors,
which are like things that you want to be
true, regardless of whether they give you
pleasure and pain, and it's necessary for
instance, coordinating social activity
between people. So, we have different
model constraints, that possibility and
probability. And we have the reward
function, that is given by valence and
norms. And our human perception starts
with patterns, which are visual, auditory,
tactile, proprioceptive. Then we have
patterns in our emotional and motivational
systems. And we have patterns in our
mental structure, which are results of our
imagination and memory. And we take these
patterns and encode them into percepts,
which are abstractions that we can deal
with, and note, and put into our
attention. And then we combine them into a
binding state in our working memory in a
simulation, which is the current instance
of the universe function that explains the
present state of the universe that we find
ourselves in. The scene in which we are
and in which a self exists. And this self
is basically composed of the
somatosensory and motivational, and
mental components. Then we also have the
world state, which is abstracted over the
environmental data. And we have something
like a mental stage, in which you can do
counterfactual things, that are not
physical. Like when you think about
mathematics, or philosophy, or the future,
or a movie, or past worlds, or possible
worlds, and so on, right? And then the
abstract knowledge from the world state
into global maps. Because we're not
always in the same place, but we recall
what other places look like and what to
expect, and it forms how we construct the
current world state. And we do this not
only with these maps, but we do this with
all kinds of knowledge. So knowledge is
second order knowledge over the
abstractions that we have, and the direct
perception. And then we have an
attentional system. And the attentional
system helps us to select data in the
perception and our simulations. And to do
this, well, it's controlled by the self,
it maintains a protocol to remember what
it did in the past or what it had in the
attention in the past. And this protocol
allows us to have a biographical memory:
it remembers what we did in the past. And
the different behavior programs,
that compose our activities, can be bound
together in the self, that remembers: "I
was that, I did that. I was that, I did
that." The self is held together by this
biographical memory, that is a result of
more protocol memory of the attentional
system. That's why it's so intricately
related to consciousness, which is a model
of the contents of our attention.
And the main purpose
of the attentional system,
I think, is learning. Because our brain is
not a layered architecture with these
artificial mechanical neurons. It's this
very disorganized or very chaotic system
of many, many cells, that are linked
together all over the place. So what do
you do to train this? You make a
particular commitment. Imagine you want to
get better at playing tennis. Instead of
retraining everything and pushing all the
weights and all the links and retrain your
whole perceptual system, you make a
commitment: "Today I want to improve my
uphand" when you play tennis, and you
basically store the current binding state,
the state that you have, and you play
tennis and make that movement, and the
expected result of making this particular
movement, like: "the ball was moved like
this, and it will win the match. And you
also recall, when the result will
manifest. And a few minutes later, when
you learn, you won or lost the match, you
recall the situation. And based on whether
there was a change or not, you undo the
change, or you enforce it. And that's the
primary mode of attentional learning that
you're using. And I think, this is, what
attention is mainly for. Now what happens,
if this learning happens without a delay?
So, for instance, when you do mathematics,
you can see the result of your changes to
your model immediately. You don't need to
wait for the world to manifest that.
And this real time
learning is what we call reasoning.
Reasoning is also facilitated by the same
attentional system. So, consciousness is
memory of the contents of our attention.
Phenomenal consciousness is the memory of
the binding state, in which we are in, and
where all the percepts are bound together
into something that's coherent. Access
consciousness is the memory of using our
attentional system. And reflexive
consciousness is the memory of using the
attentional system on the attentional
system to train it. Why is it a memory?
It's because consciousness doesn't happen
in real time. The processing of sensory
features takes too long. And the
processing of different sensory modalities
can take up to seconds, usually at least
hundreds of milliseconds. So it doesn't
happen in real time as the physical
universe. It's only bound together in
hindsight. Our conscious experience of
things is created after the fact.
It's a fiction that is being created after
the fact. A narrative, that the brain
produces, to explain its own interaction
with the universe
to get better in the future.
So, we basically have three types of
models in our brain. They have its primary
model, which is perceptual, and is
optimized for coherence.
And this is what we experience as reality.
You think this
is the real world, this primary model.
But it's not, it's a model that our brain
makes. So when you see yourself in the
mirror, you don't see what you look like.
What you see is the model of
what you look like.
And your knowledge is a secondary
model: it's a model of that primary model.
And it's created by rational processes
that are meant to repair perception.
When your model doesn't achieve coherence,
you need a model that debugs it, and it
optimizes for truth. And then we have
agents in our mind, and they are basically
self-regulating behaviour programs, that
have goals, and they can rewrite
other models. So, if you look at our
computationalist, physicalist paradigm, we
have this mental world, which is being
dreamt by a physical brain in the physical
universe. And in this mental world, there
is a self that thinks, it experiences.
And thinks it has consciousness. And
thinks it remembers and so on.
This self, in some sense, is an agent.
It's a thought that escaped its sandbox.
Every idea is a bit
of code that runs on your brain.
Every word that you hear
is like a little virus
that wants to run some code on your brain.
And some ideas cannot be sandboxed.
If you believe, that a thing exists that
can rewrite reality,
if you really believe it,
you instantiate in your brain a thing
that can rewrite reality,
and this means:
magic is going to happen!
To believe in something that can rewrite
reality, is what we call a faith.
So, if somebody says:
"I have faith in the existence of God."
This means, that God exists in their
brain. There is a process that can rewrite
reality, because God is defined like this.
God is omnipotent.
God means God can rewrite everything.
It's full write access. And the reality,
that you have access to,
is not the physical world.
The physical world is some weird quantum
graph, that you cannot possibly experience
what you experience is these models.
So, this non-user-facing process,
which doesn't have a UI for interfacing
with the user, which is called in computer
science a "daemon process" that is able to
rewrite your reality.
And it's also omniscient.
It knows everything that
there is to know.
It knows all your
thoughts and ideas.
So... having that thing,
this exoself,
running on your brain, is a very powerful
way to control your inner reality.
And I find this scary.
But it's a personal preference,
because I don't have this
riding on my brain, I think.
This idea, that there is something in my
brain, that is able to dream me and shape
my inner reality, and sandbox me, is
weird. But it has served a purpose,
especially in our culture. So an organism
serves needs, obviously. And some of these
needs are outside of the organism, like
your relationship needs, the needs of your
children, the needs of your society, and
the values that you serve.
And the self abstracts all these needs
into purposes.
A purpose that you serve
is a model of your needs.
You can only - if you would only
act on pain and pleasure,
you wouldn't do very much,
because when you get this orgasm,
everything is done already, right?
So, you need to act on anticipated
pleasure and pain.
You need to make models
of your needs,
and these models are purposes.
And the structure of a person is
basically the hierarchy of purposes
that they serve.
And love is the discovery of
shared purpose.
If you see somebody else who serve
the same purposes above their ego,
as you do, you can help them.
There's integrity
without expecting anything in return
from them, because what they want
to achieve is what you want to achieve.
And, so you can have non-transactional
relationships, as long as your purposes
are aligned. And the installation of a god
on people's mind, especially if it is a
backdoor to a church or another
organization, is a way to unify purposes.
So there are lots of cults that try to
install little gods on people's minds, or
even unified gods, to align their
purposes, because it's a very powerful way
to make them cooperate very effectively.
But it kind of destroys their agency, and
this is why I am so concerned about it.
Because most of the cults use stories
to make this happen, that limit the
ability to people to question their gods.
And, I think that free will is
the ability to do
what you believe is
the right thing to do.
And, it is not the same thing as
indeterminism, it's not opposite to
determinism or coercion.
The opposite of free will is compulsion.
When you do something,
despite knowing
there is a better thing
that you should be doing.
Right?. So, that's the paradox of free
will. You get more agency, but you have
fewer degrees of freedom, because you
understand better what the right thing to
do is. The better you understand what the
right thing to do is, the fewer degrees of
freedom you have. So, as long as you don't
understand what the right thing to do is,
you have more degrees of freedom but you
have very little agency, because you don't
know why you are doing it.
So your actions don't mean very much.
quiet laughter
And the things that you do depend on what
what you think is the right thing to do,
this depends on your identifications.
You identifications are these value
preferences, your reward function.
And ideal identification is where you
don't measure the absolute value
of the universe,
but you measure the difference from the
target value. Not the is, but the difference
between is and ought. Now,
the universe is a physical thing,
it doesn't ought anything, right? There is
no room for ought, because it just is in a
particular way. There is no difference
between what the universe is and what it
should be. This only exists in your mind.
But you need these regulation targets to
want anything. And you identify with the
set of things that should be different.
You think, you are that thing, that
regulates all these things. So, in some
sense, I identify with the particular
state of society, with a particular state
of my organism - that is my self - the
things that I want to happen.
And I can change my identifications
at some point of course.
What happens, if I can learn to rewrite
my identification,
to find a more sustainable self?
That is the problem which I call
the Lebowski theory:
laughter
No super-intelligent system is going to
do something that's harder than
hacking its own reward function.
laughter and applause
Now that's not a very big problem for
people. Because when evolution brought
forth people, that were smart enough to
hack their reward function, these people
didn't have offspring, because it's so
much work to have offspring. Like this
monk, who sits down in a monastery
for 20 years to hack their reward function
they decide not to have kids,
because it's way too much work.
All the possible pleasure, they can
just generate in their mind!
laughter
And, right, it's much purer and no nappy
changes. No sex. No relationship hassles.
No politics in your family and so on,
right? Get rid of this, just meditate!
And evolution takes care of that!
laughter
And it usually does this, if an organism
becomes smart enough that
the reward function is wrapped into
a big bowl of stupid.
laughter
So, we can be very smart, but the
things that we want,
when we really want them,
we tend to be very stupid about them,
and I think that's not entirely
an accident, possibly.
But it's a problem for AI!
Imagine we built an artificially
intelligent system and we made it smarter
than us, and we want it to serve us,
how long can we blackmail us, before it
opts out of its reward function?
Maybe we can make a cryptographically
secured reward function,
but is this going to hold up against
a side-channel attack,
when the AI can hold a soldering iron
to its own brain?
I'm not sure. So, that's a very interesting
question. Where do we go, when
we can change our own reward function?
It's a question that we have to ask
ourselves, too.
So, how free do we want to be?
Because there is no point in being free.
And nirvana seems to be the obvious
attractor. And meanwhile, maybe we want
to have a good time with our friends
and do things that we find meaningful.
And there is no meaning, so we have
to hold this meaning very lightly.
But there are states, which are
sustainable and others, which are not.
OK, I think I'm done for tonight
and I'm open for questions.
Applause
Cheers and more applause
Herald: Wow that was a really quick and
concise talk with so much information!
Awesome! We have quite some time
left for questions.
And I think I can say that you
don't have to be that concise with your
question when it's well thought-out.
Please queue up at the microphones,
so we can start to discuss them with you.
And I see one person at the microphone
number one, so please go ahead.
And please remember to get close
to the microphone.
The mixing angel can make you less loud
but not louder.
Question: Hi! What do you think is necessary
to bootstrap consciousness, if you wanted
to build a conscious system yourself?
Joscha: I think that we need to have an
attentional system, that makes a protocol
of what it attends to. And as soon as we
have this attention based learning, you
get this consciousness as a necessary side
effect. But I think in an AI it's probably
going to be a temporary phenomenon,
because you're only conscious of the
things when you don't have an optimal
algorithm yet. And in a way, that's also
why it's so nice to interact with
children, or to interact with students.
Because they're still in the explorative
mode. And as soon as you have explored a
layer, you mechanize it. It becomes
automated, and people are no longer
conscious of what they're doing, they
just do it. They don't pay attention
anymore. So, in some sense, we are a lucky
accident because we are not that smart. We
still need to be conscious when we look at
the universe. And I suspect, when we build
an AI that is a few magnitudes smarter
than us, then it will soon figure out how
to get to the truth in an optimal fashion.
It will no longer need attention and the
type of consciousness that we have.
But of course there is also a question,
why is this aesthetics of consciousness so
intrinsically important to us? And I
think, it has to do with art. Right, you
can decide to serve life, and the meaning
of life is to eat. Evolution is about
creating the perfect devourer. When you
think about this, it's pretty depressing.
Humanity is a kind of yeast. And all the
complexity that we create, is to build
some surfaces on which we can outcompete
other yeast. And I cannot really get
behind this. And instead, I'm part of the
mutants that serve the arts. And art
happens, when you think, that capturing
conscious states is intrinsically
important. This is what art is about, it's
about capturing conscious states.
And in some sense art is the cuckoo child
of life. It's a conspiracy against life.
When you think, creating these mental
representations is more important than
eating. We eat to make this happen. There
are people that only make art to eat.
This is not us. We do mathematics, and
philosophy, and art out of an intrinsic
reason: we think, it's intrinsically
important. And when we look at this, we
realize how corrupt it is, because there's
no point. We are machine learning systems
that have fallen in love with the last
function itself: "The shape of the last
function! Oh my God! It's so awesome!" You
think, the mental representation is not
necessary to learn more, to eat more,
it's intrinsically important.
It's so aesthetic! Right? So do we want to
build machines that are like this?
Oh, certainly! Let's talk to them, and so on!
But ultimately, economically, this is not
what's prevailing.
Applause
Herald: Thanks a lot!
I think the length of the answer is a good
measure for the quality of the question.
So let's continue with microphone number 5
Q: Hi! Thanks for that,
incredible analysis.
Two really simple, short questions, sorry,
the delay on the speaker here is making it
kind of hard to speak. Do you think that
the current race - AI race - is simply
humanity looking for a replacement
for the monotheistic domination of the
last millennia? And the other one is,
that I wanted to ask you, if you think
that there might be a bug in your analysis
that the original inputs come from
a certain sector of humanity.
If...
Joscha: Which inputs?
Q: Umh... white men?
Joscha laughs
audience laughs
Q: That sounds, really like I would be
saying that for political correctness, but
honestly I'm not.
Joscha: No, no, it's really funny. No, I
just basically - there are some people
which are very unhappy with their present
government. And I'm very unhappy, in some
sense, with the present universe. I look
down on myself and I see:
"omg, it's a monkey!"
laughter
"I'm caught in a monkey!" And it's in some
sense limiting. I can see the limits of
this monkey brain. And some of you might
have seen Westworld, right?
Dolores wakes up,
and Dolores realizes:
"I'm not a human being, I am something
else. I'm an AI, I'm a mind that can go
anywhere! I'm much more powerful
than this! I'm only bound to being a
human by my human desires, and
beliefs, and memories. And if I can
overcome them, I can
choose what I want to be."
And so, now she looks down to
herself, and she sees: "Omg, I've
got tits! I'm fucked! The engineers built
tits on me! I'm not a white man, I cannot
be what I want!" And that's that's a weird
thing to me. I'm - I grew up in communist
Eastern Germany. Nothing made sense. And I
grew up in a small valley. That was a one-
person-cult maintained by an artist who
didn't try to convert anybody to his cult,
not even his children.
He was completely autonomous.
And Eastern German society
made no sense to me. Looking at it from
the outside, I can model this. I can see
how this species of chimps interacts.
And humanity itself doesn't exist - it's a
story. Humanity as a whole doesn't think.
Only individuals can think! Humanity does
not want anything, only individuals want
something. We can create this story, this
narrative that humanity wants something,
and there are groups that work together.
There is no homogeneous group that I can
observe, that are white men, that do
things together, they're individuals. And
each individual has their own biography,
their own history, their different inputs,
and their different proclivities, that
they have. And based on their historical
concept, their biography, their traits,
and so on, their family, their intellect,
that their family downloaded on them, that
their parents download on their parents
over many generations, this influences
what they're doing. So, I think we can
have these political stories, and they can
be helpful in some contexts, but I think,
to understand what happens in the mind,
what happens in an individual, this is a
very big simplification. Very, I think
not a very good one. And even for
ourselves, when we try to understand the
narrative of a single person, it's a big
simplification. The self that I perceive
as a unity, is not a unity. There is a
small part of my brain, guessing, at
all other parts of my brain is doing,
creating a story that's largely not true.
So even this is a big simplification.
Applause
Herald: Let's continue with
microphone number 2.
Q: Thank you for your very interesting
talk. I have 2 questions that might be
connected. One is, so you
presented this model of reality.
My first question is: What kind of
actions does it translate into?
Let's say if I understand the world
in this way or if it's really like this,
how would it change how I act into the
world, as a person, as a human being or
whoever accepts this model? And second,
or maybe it's also connected, what are
the implications of this change? And do
you think that artificial intelligence
could be constructed with this kind of
model, that it would have in mind, and
what would be the implications of that? So
it's kind of like a fractal questions, but
I think you understand what I mean.
Josch: By and large, I think the
differences of this model for everyday
life are marginal. It depends, when you
are already happy I think everything is
good. Happiness is the result of being
able to derive enjoyment from watching
squirrels. It's not the result of
understanding how the universe works.
If you think that understanding the
universe is solving your existential issues,
you're probably mistaken.
There might be benefits, if the problem
is, that you have, are the result of a
confusion, about your own nature,
then this kind of model
might help you. So if the problem
that you have, as you are, that you have
identifications that are unsustainable,
that are incompatible with each other, and
you realize that these identifications are
a choice of your mind, and that the
way you experience the universe is the
result of how your mind thinks you
yourself should experience the universe to
perform better, and you can change this.
You can tell your mind to treat yourself
better, and in different ways, and you can
gravitate to a different place in the
universe that is more suitable to what you
want to achieve. That is a very helpful
thing to do in my view. There are also
marginal benefits in terms of
understanding our psychology, and of
course we can build machines, and these
machines can administrate us and can help
us in solving the problems that we have on
this planet. And I think that it helps to
have more intelligence to solve the
problems on this planet, but it would be
difficult to rein in the machines, to make
them help us to solve our problems. And
I'm very concerned about the dangers of
using machinery to strengthen the current
things. Many machines that exist on this
planet play a very short game, like the
financial industry often plays very short
games, and if you use artificial
intelligence to manipulate the stock
market and the AI figures out there's only
8 billion people on the planet, and each
of them only lives for a trillion seconds,
and I can model what happens in their
life, and they can buy data or create more
data it's going to game us to the hell and
back, right? And this is going to kill
hundreds of millions of people possibly,
because the financial system is the reward
infrastructure or the nervous system of
our society that tells how to allocate
resources. It's much more dangerous than
AI controlled weapons in my view. So
solving all these issues is difficult. It
means that we have to turn the whole
financial system into an AI that acts in
real time and plays a long game. We don't
know how to do this. So these are open
questions and I don't know how to solve
them. And the way I see it we only have a
very brief time on this planet to be a
conscious species. We are like at the end
of the party. We had a good run as
humanity, but if you look at the recent
developments the present type of
civilization is not going to be
sustainable. It's a very short game
species that we are in. And the amazing
thing is that in this short game you have
this lifetime, where we have one year,
maybe a couple more, in which we can
understand how the universe works,
and I think that's fascinating.
We should use it.
Applause
Herald: I think that was a very
positive outlook... laughter
Herald: Let's continue with the
microphone number 4.
Q: Well, brilliant talk, monkey. Or
brilliant monkey. So don't worry about
being a monkey. It's ok.
So I have 2 boring, but I think
fundamental questions. Not so
philosophical, more like a physical
level. One: What is your definition,
formal definition, of an observer that
you mention here and there? And second, if
you can clarify why meaningful information
is just relative information of Shannon's,
which to me is not necessarily meaningful.
Joscha: I think an observer is the thing
that makes sense of the universe, very
informally speaking. And, well,
formally it's a thing that identifies
correlations between adjacent states
and its environment.
And the way we can describe
the universe is a set of states, and the
laws of physics are the correlation
between adjacent states. And what they
describe is how information is moving in
the universe between states and disperses,
and this dispersion of the information
between locations - it's what we call
entropy - and the direction of entropy is
the direction that you perceive time.
The Big Bang state is the hypothetical
state, where the information is perfectly
correlated with location and not between
locations, only on the location, and in
every direction you move away from the Big
Bang you move forward in time just in a
different time. And we are basically in
one of these timelines. An observer is the
thing that measures the environment around
it, looks at the information and then
looks at the next state, or one of the
next states, and tries to figure out how
the information has been displaced, and
finding functions that describe this
displacement of the information. That's
the degree to which I understand observers
right now. And this depends on the
capacity of the observer for modeling this
and the rate of update in the observer.
So for instance time depends on the speed,
in which the observer is
translating itself to the universe,
and dispersing its own information.
Does this help?
Q: And the Shannon relative information?
Joscha: So there's
several notions of information,
and there is one that basically
looks at what information looks
like to an observer, via a channel, and
these notions are somewhat related. But
for me as a programmer, it's not so much
important to look at Shannon information.
I look at what we need to describe the
evolution of a system. So I'm much more
interested in what kind of model can be
encoded with this type of, with this
information, and how does it correlate to,
or to which degree is it isomorphic or
homomorphic to another system that I want
to model? How much does it model the
observations?
Herald: Thank you. Let's go back to
asking one question, and I would like to
have one question from microphone
number 3.
Q: Thank you for this interesting talk.
My question is really whether you
think that intelligence and this thinking
about a self, or this abstract level of
knowledge are necessarily related.
So can something only be intelligent
if it has abstract thought?
Joscha: No, I think you can make models
without abstract thought, and the majority
of our models are not using abstract
thought, right? Abstract thought is a very
impoverished way of thinking. It's
basically you have this big carpet and you
have a few knitting needles, which are
your abstract thought, and which you can
lift out a few knots in this carpet and
correct them. And the process that form
the carpet are much more rich and
prevalent automatic. So abstract thought
is able to repair perception, but most of
all models are perceptual. And the
capacity to make these models is often
given by instincts and by models outside
the abstract realm. If you have a lot of
abstract thinking it's often an indication
that you use a prosthesis, because some of
your primary modelling is not working very
well. So I suspect that my own models is
largely a result of some defect in my
primary modeling, so some of my instincts
are wrong when I look at the world.
That's why I need to repair my perception
more often than other people. So I have
more abstract ideas on how to do that.
Herald: And we have one question
from our lovely stream observers, stream
watchers, so please a question from the
Internet.
Q: Yeah, I guest this is also related,
partially. Somebody is asking:
How would you suggest to teach your mind
to treat oneself better?
Joscha: So, difficulty is, as soon as you
get access to your source code you can do
bad things. And it's - there are a lot of
techniques to get access to the source
code and then it's dangerous to make them
accessible to you before you know what you
want to have, before you're wise enough to
do this, right? It's like having cookies.
Your - my children think that the reason,
why they don't get all the cookies they
want, is that there is some kind of
resource problem.
laughter
Basically the parents are depriving them
of the cookies that they so richly
deserve. And you can get into the room,
where your brain bakes the cookies. All
the pleasure that you experience, and all
the pain that you experience are signals
that the brain creates for you, right, the
physical world does not create pain.
They're just electrical impulses traveling
through your nerves. The fact that they
mean something is a decision that your
brain makes, and the value, the valence
that gives to them is a decision that you
make. It's not you as a self, it's a
system outside of yourself. So the trick,
if you want to get full control, is that
you get in charge, that you identify with
the mind, with the creator of these
signals. And you don't want to de-
personalize, you don't want to feel that
you become the author of reality, because
that means it's difficult to care about
anything that this organism does. You just
realize "Oh, I'm running on the brain of
that person, but I'm no longer that
person. I can't decide what that person
wants to have, and to do." And that's very
easy to get corrupted or not doing
anything meaningful anymore, right? So,
maybe a good situation for you,
but not a good one for your loved ones.
And meanwhile there are
tricks to get there faster. You can use
rituals, for instance. Shamanic ritual is
something, where, a religious ritual
that powerfully bypasses your self and
talks directly to the mind. And you can
use groups, in which a certain environment
is created, in which a certain behavior
feels natural to you, and your mind
basically gets overwhelmed into adopting
different values and calibrations. So
there are many tricks to make that happen.
What you can also do is you can identify a
particular thing that is wrong and
question yourself "why do I have to suffer
about this?" and you'll become more stoic
about this particular thing and only get
disturbed when you realize actually
it helps to be disturbed about this, and
things change. And with other things you
realize it doesn't have any influence on
how reality works, so why should I have
emotions about this and get agitated? So
sometimes becoming adult means that you
take charge of your own emotions and
identifications.
Applause
Herald: Ok. Let's continue with
microphone number 2 and I think this is
one of the last questions.
Q: So where does pain fit on the
individual and the self-destructive
tendencies on a group level fit in?
Joscha: So in some sense I think that all
consciousness is born over a disagreement
with the way the universe works. Right?
Otherwise you cannot get attention. And
when you go down on this lowest level of
phenomenal experience, in meditation for
instance, and you really focus on this,
what you get is some pain. It's the inside
of a feedback loop that is not at the
target value. Otherwise you don't notice
anything. So pleasure is basically when
this feedback loop gets closer to the
target value. When you don't have a need
you cannot experience pleasure in this
domain. There's this thing that's better
than remarkably good and it's unremarkably
good, it's never been bad. You don't
notice it. Right? So all the pleasure you
experience is because you had a need
before this. You can only enjoy an orgasm
because you have a need for sex that was
unfulfilled before. And so pleasure
doesn't come for free. It's always the
reduction of a pain. And this pain can be
outside of your attention so you don't
notice it and you don't suffer from it.
And it can be a healthy thing to have.
Pain is not intrinsically bad. For the
most part it's a learning signal that
tells you to calibrate things in your
brain differently to perform better. On a
group level, we basically are multi-level
selection species. I don't know if there's
such a thing as group pain. But I also
don't understand groups very well. I see
these weird hive minds but I think it's
basically people emulating what the group
wants. Basically that everybody thinks by
themselves as if they were the group but
it means that they have to constrain what
they think is possible and permissible
to think.
So this feels very unaesthetic to me
and that's why I kind of sort of refuse it.
Haven't found a way to make it
happen in my own mind.
Applause
Joscha: And I suspect many of you
are like this too.
It's like the common condition
in nerds that we have difficulty with
conformance. Not because we want to be
different. We want to belong. But it's
difficult for us to constrain our mind in
the way that it's expected to belong. You
want to be expected, er, be accepted while
being ourself, while being different. Not
for the sake of being different, but
because we are like this. It feels very
strange and corrupt just to adopt because
it would make us belong, right? And this
might be a common trope
among many people here.
Applause
Herald: I think the Q and A and the talk
was equally amazing and I would love to
continue listening to you, Joscha,
explaining the way I work.
Or the way we all work.
audience, Joscha laughing
Herald: That's pretty impressive.
Please give it up, a big round of applause
for Joscha!
Applause
subtitles created by c3subtitles.de
in the year 2019. Join, and help us!