Hello everyone, and a warm welcome to Multimodal Language Processing.
My name is Xaver Funk, and I recently had the chance to really [involve] myself into
this topic, because I am studying neurosciences and this was kind of
something that I had to do. And, yeah, that's what I want to share with you today.
So, what I have been doing recently also, is learning arabic, and a little bit of
mongolian. And mostly what I did was, I had this stream of auditory signals
that maybe came from the ASML audio, and I tried to match those to symbols that
were representing these, right, in the book.
And I kind of had this feeling that this is incomplete.
So there is something missing there.
And while I was on the other hand, studying a lot about multimodal language
processing, which I how gestures influence processing and stuff like that.
I came to the conclusion that, yeah, there is something missing.
In our world today, we are all litterate so we mostly think of languages as
these auditory signals, these mouth noises and the symbols that represent these.
But there is so much more going on, in face to face communication and, yeah,
I want to make this point clear, with a virtual experiment.
So, I want to invite you to first of all, listen to this audio excerpt, from an
"Easy Languages" video. And I give you the subtitles here, with the english
translation as well. So, basically, these are auditory signals in Dutch,
and sequences of symbols in Dutch and English.
And for the people learning Dutch, please just ignore the English, just to make it
a little bit harder. And people who know Dutch, please close your eyes, so that
you don't see it at all. So, let's go.
So, when I was listening to this at first, I was - because I know some Dutch,
I was understanding quite a lot, but, kind of, not everything.
And then, I watched the video that goes with it, it was kind of a different experience.
And that's what we are going to do now. So just watch the video, and if you can,
see how these two women, that are interviewed here, are interacting
with the interviewer, and between each other.
So, I hoped this worked, and you felt a little bit different now.
And even for the people who don't know Dutch, I hope you could kind of follow
what was going on. And even if you didn't, the point I want to make is that
messages are not only auditory, they are always also visual.
We have a lot of non-auditory articulators, like 43 face muscles
for example, and then 2x 34 muscles in the hands, and then even more in
the arms, in our torso. And the people in this video
really knew how to use these. So for example we had a lot of
facial movement going on, like you see on the top, here.
See how she raise her eyebrows, and then, you have this head tilting
at the end, that really put an emphasis on what she's saying.
And then there is a lot of gaze switching as well.That's right in the begining,
when she says (Dutch): Oh genoeg ! Heb je even ?
So, "Oh, there is so much that I want to see! Do you have some time?"
But, she doesn't really say "time", she says "Heb je even", "Do you have a little"
And for me, when I was only listening, I didn't quite get what she was saying,
but when I saw how she adresses the interviewer, I kind of got it,
afterwards.
So then there is of course manual gestures like "hoop op mijn list",
that's that one here, "hoop op mijn list". She says "berglandschap", so that's
a mountain range, and then "lang geleden" "long time ago", right?
So there is a lot of messages that are supported with these manual gestures.
Then there is also stuff like this nose scratching, where we don't even know
is there something to it, or is it just a nose scratching.
Does it carry some information? We don't know.
And then, lastly also arm and torso movements.
And also if you watch at the top here, you have nodding. So you see how
these two kind of nod together, they really give us the impression of
how good friends they are, right. And then if you look at this bottom part
here, that's my favorite part of the video. You really have this complex
orchestration of different gestures, and they are turn-taking.
So, the one on the right says something, and the one on the left answers that
perfectly, and then you have gestures, and then the, putting their hair back,
right, so there is so much going on between them, and it really gives more
than just the auditory message, right.
So, note that there is something that our brain has to achieve here.
Mainly two things : so, it has to segregate all of the stuff that is not
important for the message. That's the segregation problem.
From the important stuff, and then, take all the important stuff, all
the auditory and visuals information and put it together into a coherent message.
That's our binding problem. And all of this, note, all of this is
under a really tight time constraint, when you're turn taking, when you're
having a conversation. And if you say something and
the other person say something, and there is not that much time
between turns. And if you need more time, then that also has
a meaning, right ? If you take time, then that means that you're hesitating
to answer, maybe there is something going on with you emotionally...
So you don't want to have that as well. So, yeah, so basically, this is really
a huge computational problem for your brain.
And well, how did your brain do? Did you feel the video was more difficult
than the audio ? Did you understand more, or did you understand less?
Did you feel more in the scene, maybe? Catching more informations between the lines?
And well, for me, at least as you might guess, for me it was way easier
to follow with the video to interpret these gestures. And this is kind of
a paradox. So, how come that processing more signals simultaneously is easier
that processing speech alone? And this also was shown
in the litterature, so people have made experience with this.
And this is really a surprising facilitation. For example, there are
lot of studies, I'll just give you one example.
So in this study they showed people a "prime", so this was some video
of an action that somebody did, and then they showed the people different videos.
And the videos were either completely congruent, so what was said was the same
as the gesture, and was the same as this prime. So in that case it would be
"chop" and doing the chopping gesture. And then there were different conditions
where either the speech was congruent, incongruent, and the gesture was congruent.
Or the speech was incongruent and the gesture was congruent.
And then they had also weakly congruent stuff, like, this for "chopping",
but this is actually cutting so this is only weakly incongruent.
And then this twisting, which is strongly incongruent.
And then people had to press a button for "yes" if either the speech or
the gesture were related to the prime, and no if neither speech nor gesture
was related to the prime. And what the people found out was that
there were differences in response times, and also in the proportion of errors
that people did, as soon as soon as there were something incongruent.
And from that the authors come to the conclusion that really, speech and
gestures are two sides of the same coin, they mutually interact
to enhance comprehension. And now the big question is, of course,
how does our brain achieve this surprising facilitation?
And we can look back at turn-taking, to maybe get some clues here.
So on average a turn-take takes only about 0 to 200 miliseconds, which is
a fifth of a second. You can see in this video how fast she is responding,
right now, after this, like, this is an instant, right?
And this is quite extraordinary, because producing a single word actually takes
about 600 miliseconds.
So if I just prompt you to say a word, you would take 600 miliseconds to say it.
So there's something going on, it seems like we are predicting already what we are
going to say before the turn of the other person is finished, and we already prepare
our turn.
So there is something that is going on, that has to do with prediction.
Most language use in conversation has to be based on prediction somehow.
And this is quite nice, because prediction is anyways the current hype
in neuroscience nowadays, and it's basically a good candidate for
the overarching function of the brain.
And many people think that what we are doing in our daily lives is basically
constantly computing and updating probability distributions.
And this applies both to action, to perception, and also to language.
So, this will be a rephrasing of the problem we had before,
as a prediction problem. And this become then,
"given the preceding context - so, given all the words that come before - what word
is most likely to come up next?" Right? And to make this more clear,
let me give you a quick example: so, imagine I come to you and I say,
without anymore context "I would like to".
And then, you don't know what I'm going to say next, right?
It could be any of these, for example. I would like to drink, eat, work...
And so on. And now, if I shape my hand
in the form of a "C", and I put it to my mouth, like this, while I say
"I would like to", then your probability distribution over these words changes
in such a way that "drink" is much more likely to be the next word.
And maybe "eat" also a little bit, but the others words probably not,
because, you can associate this gesture with drinking, or a little bit
with eating, because it's also something that you put to your mouth,
but mostly this is commonly understood as "drinking", right.
So, in this way gestures add context to predictions and help this process of
predicting, and that also helps the comprehension.
And, we can actually measure prediction, using neurophysiology.
So this is EEG, and "EEG" stands for "Electro Encephalography", and
it's basically putting electrodes on the scalp, and then measuring
the brain activity that's below. If you do this, well you can measure
brain activity basically. What people usually do, is that
they give people these sentences. So these could be normal sentences,
like this one : "It was his first day at work."
Or it could be so-called garden-path sentences. So these are sentences that are
somehow manipulated artificially to elicit some response. Right?
So this would be : "He spread the warm bread with socks."
So you may have a weird feeling on your head, because nobody spreads
the bread with socks. And this weird feeling, if we would measure you with
an EEG, would constitute this reaction here, that's a so-called N400.
"N" because it is a negative polarity, and it's 400 miliseconds after the word.
So, all of this above here is just electrical activity, right? And you have
this really pronounced peak, when there is a violation of the semantics,
like with "socks". And it's also taken to be a prediction
error. So you did not predict socks, you predicted Nutella, for example,
or honey. But not socks. And this is reflected in this N400 prediction error.
So people are doing this a lot, like, showing these sentences that are somehow manipulated.
We have another example here, this is another topic : if you write in all caps
you have this kind of response for example.
But, what I want to do now with you is bringing you more to the cutting edge
of what is currently done in multimodal processing research.
So the trend is to go away from these artificially constructed sentences, and
more towards naturalistic language comprehension. So, using actual stories,
actual sentences, that are not manipulated in any way. And this will be combined with
computational linguistics - how that works, you will see in a bit.
And also, yeah, with that you can look at multimodal processing if you just add
a video to the audio that you make people listen to.
And what it might look like is like this. So, this is one study
that is currently not published officially yet. It is already on
the archive. And I want to use this to illustrate to you how we might research
naturalistic language comprehension. So the general planners get some
per-word measures - so these would be these ones here. So for each word,
there is some value attached. And then we can use these as regressors
in the big linear regression model. So, using fancy statistics, and with that
we're basically asking our data "how well are you predicted by these regressors?"
And for example, this one here is surprise and this is closely related
to predictions or prediction errors. So this is the negative log probability
of a word, given all of the words that come before it.
So, this is the contexte, basically, and this is some word, "w".
So this is basically telling you how unpredictable is a given word.
And this measure is base on computational language models, so for example,
they would take the whole corpus of a language, and then, see which words
occurs after each other, and thereby get to this value of how unpredictable it is.
And then, they have another thing here. They use the fundamental frequency of each
word as a pitch indicator, to control for prosody, which is also pretty cool.
So they let loose their linear regression models, with these predictors,
so they have a surprisal value for each word, for example, a prosody, ready for
each word, then they indicate where there are meaningful gestures happening,
and, yeah, also, mouth movements.
And, what came out of this, one finding that might be interesting for us,
now, is that for meaningful gestures, the N400 is less negative.
So you can also see this here : for meaningful gesture, this blue line,
you see that it is a lot less negative than the red line where the gestures are
absent. And then there's also, that's why I told you about surprisal, an interesting
interaction between gestures and surprisal. So, the higher the surprisal,
the less unexpected a word, the stronger this facilitating effect of gestures is.
Which is also really interesting. Then there's, this is a similar study,
that I actually got the chance to work on, with a colleague.
So what we did here, we had a measure of entropy. This measures basically the
uncertainty about the next word. So if you think back to the example
we had before, where I was telling you "I would like to", and then something,
but without context, that would be really high entropy, really high uncertainty :
you don't know what's coming next. Right? Then we also had surprisal, we had word
frequency, how often the word came up. And IVC is a measure of,
it's an abbreviation for "instantaneous visual change", so, how much the actor
moved while we were showing this to the people. And then speech envelope,
this is basically a measure of the level of the sound.
And what we found is - and this by the way was an FRMI experiment,
so we can look at wich regions are active during some condition.
And for words where the surprisal was really high, there were these regions
in red active, and for words where entropy was really high,
these regions in blue. And now if we look at interactions with gestures
for the entropy condition, we can see that when there were gestures present, we had
really specific activations compared to when there were no gestures present,
in situations where there is high entropy, so high uncertainty.
So with these tools we try to get into the processes that underlie prediction
in language. So let's take a step back, and have a look
at kind of a more global evolutionnary perspective.
We know from primate research that gesture and gaze are crucial for communication.
You can see it in this video : this ape right here does this gesture,
and this signals to its mother to pick her up. Right?
So these are bonobos, and you can see right now, this "pick me up" gesture.
And Federico Rossano, from the Max Planck Institute for Evolutionnary Institute,
could show that this gesture get more and more ritualized, to the point where
it becomes only a small wrist bend with the arm and one gaze, to instantiate
this carry behavior. Right? So you see that there is also kind of
a prediction involved : the mother has to predict what the child is wanting
to do, right? Going from this, to only this.
Then, building on this, there are some authors that propose that speech and
gesture have a common origin. And the idea here is that, through
these ritualized gestures that we've just seen in those bonobos,
after a time there will be a proto sign language evolving.
Which then at some point will be accompanied by sound as well,
evolving into a proto speech language. And then the proto sign, the proto speech,
will reinforce each other more and more, until language emerges.
And another point, here, or an observation : those of you who have
tried sign language, it kind of feel surprinsingly natural, right?
So, if speech is the true communication medium for humans, why is it
thet sign language feels so real, so natural, right?
And then another point that goes into this theory is that voluntary hand movements
came before voluntary breathing. And you need voluntary breathing to articulate
yourself, right?
So, also, just as complementary to speech,
you can more easily show spatial relations between things.
And then, if you look at child development the same pattern : gesture develop
before speech, and pre-speech turn-taking is faster than later.
So if you're a baby and you gesture, the turn-taking with your mother,
the communication is quite fast, it's almost adult level turn-taking.
Then as you learn language it gets way slower, and only in middle-school it gets
gets back to the adult level turn-taking.
So, what's the point, right? What does all of this mean for language learning?
So for this, let's do another time-travel, back to 1768,
and meet this French Jesuit monk : Claude-François Lizarde de Radonvilliers.
And he wrote this book : (French) "About The Way To Learn Languages"
back in the day, where he reflected on how we should teach people languages.
And interestingly, this is basically the grandfather of the Assimil method,
and also the Méthode Toussaint-Langenscheidt, or, also called
"interlinearversion". So this would be this sheet here.
This was a way people learned languages at the turn of the previous century.
And you can see here that you have the spanish at the top, then some
consideration in the middle, and on the bottom the german.
And this is kind of similar to what Assimil does, right?
So this is really interesting, but that's not the point here.
What he also did in this book is to compare L1 - so first language acquisition -
with second language learning. And he noted that it seems that,
for the first language, parents show their children pictures, and enact words or
concepts, and encourage the children to do the same, like this little boy does here.
But for second language acquisition all we do is give people these vocabulary lists,
and expect them to learn it just like that.
So this is kind of an interesting point, and since then it has been shown,
- and this is actually pretty robust, I was really surprised, that it has been
a really robust finding, that gesture enriched material enhances learning.
So in this study, for example, people tried to teach english-speaking people
japanese words, and they had four different training conditions.
So one, only speech, one repeated speech, one speech plus incongruent gestures
- so gestures that would not match - and then, congruent gestures.
And this is the interesting condition, right?
And then they tested the people after encoding for three different times :
after five minutes, after two days, and after one week.
And also they tested them on forced choice so it's basically multiple choice,
and free recall, so it's prompting the people with the word, and then they
come up themselves with the answer. So these numbers here are basically
the proportion of correct answers that people give.
And you can see that, across the board,
the speech plus congruent gesture condition is
very superior compared to the other ones, which is, yeah, which is interesting, and
so, you would maybe think that the point is "okay, so we just use videos instead
of audios", right? And this is what I would call
Multisensory enrichment. And there is nothing wrong with this,
this is really useful, you have these YouTube channel like Easy Languages
- I'm not sponsored by the way (laugh) - where you have conversations with
real people that from time to time make gestures, and you get the full
conversation thing, right? And you have these one-on-one videos,
like this one from Mandarin Corner. Where they are also a lot of gestures
involved, so the host, Eileen, really tries to integrate a lot of gestures.
But this is actually not the point - I mean, this is cool but I think
you already do that. The point is way deeper.
So, there's another thing going on, not only when you watch gestures,
but when you enact them. This is called the enactment effect.
This was actually coined in 1980, by two germans.
They called it first the "Tu-Effekt", which translates literally to "Do-Effect".
And you can see why people chose to call it the enactment effect, because it sounds
way more fancy (laugh) but I really like the "tu-effekt", it sounds funny.
Anyways, the point is that action words or phrases, this is what they - Engelkamp
et Krumnacker - noticed : that action words and phrases are remembered better
if they're acted out, or accompanied by gestures.
So if you would learn the phrase "chopping garlic", then if you enact it
actually while learning it, you will retain it way better.
And this effect is also really well replicated,
and this was also really surprising to me,
because it is virtually not at all translated into actual teaching.
Nobody does this, nobody tells the students to enact things, right?
Enact words, enact anything.
And it has been well replicated across tasks, across materials and also
across populations : across children, adults, even clinical populations :
People with Alzheimer, people recovering from stroke...
Somehow people made them learn words and then act the words,
and it worked better than without enactment.
And also, this is not only true for action words and concrete words,
but also abstract words. Anything you can somehow find
a representation - with gestures - for, you can use this enactment effect.
And this is way more powerful than multysensory enrichment,
and we can call this "sensorimotor enrichment",
because you use your senses and your "motor".
So, this is also, this ties in with another really interesting development
in neurosciences, called "embodied cognition".
Basically this is the idea that many features of cognition - and these might be
concepts, categories, reasoning or judgement - are shaped by aspects
of the body. And this would be the motor system - so how we move -
the perceptual system - what we see, what we feel, what we hear, and so on.
And also bodily interactions with the environment.
And you might see where I get with this, if you think about concepts and categories
What are words, if not concepts and categories, right?
So we might ask the question, "how are words represented in the brain?"
And there is this really funny study,
they showed people words that had strong olfactory associations, which means
they either stink really hard, or they smell really well.
And, in case you're looking for some inspiration for your spanish poem,
you can go (laugh) to this publication and search through the list of words.
This is also a small - this is only a small sample, there are tons of
really strong smelling words in the study and, yeah.
So basically, what they found is that when they showed people these words,
as compared to words that did not smell that much, some regions in the brain
that are associated with olfaction, so, with smelling, lighted up.
And this kind of has been extended as well to actions.
So, on the left here, these are all the regions that light up
when you move your foot when you move your fingers,
or when you move your tongue. And on the right here, these are
the regions that light up when you read leg-related words, arm-related words,
or face-related words. And you can see that, this more or less,
this is more or less, these activations fit each other, right?
So, in some way, leg-related words are stored where you also move you leg,
arm-related words are stored where you also move your arms, and so on.
So, we can think about words actually as functional networks, like this.
And, note that words are experience-dependent functional networks.
And experience is connected to the body, right?
So for exemple, you surely have, not only read and heard the word "garlic",
you also have smelled garlic, you touched garlic, you tasted garlic
and, really important thing you chopped garlic.
So when you read "garlic", you not only have
the core language areas - in yellow here - activated, but also
subcortical olfactory areas, and some gustatory areas - so, for taste -
action areas, right, and visual areas as well.
So, what I want to tell you here, think about this when you learn languages.
Did you do the same for "knoblauch", for example - the german word for "garlic"?
If you learn german, do you actually get into this huge associated network?
So, and that's the point basically, we are coming to the end,
the point is that language is multimodal, you should use sensory-motor enrichment
when learning languages, and thereby embody your languages.
And if you want to learn more about this, and also for me to give credit,
this is basically where I got most of my input from.
These are four big review articles that discuss all of this stuff.
So yeah, that's basically it. Thanks for listening, and hoping for some cool questions.
I would most certainly guess so. This thing with the phone is also
something that I have experienced quite a lot.
I have lived in Chile for some time, and I've got myself a chilean SIM-card
And I didn't give this number to a lot of people, but somehow, this number got
to people that were, I don't know, trying to sell me something.
And I would get a call from somebody, pick up the phone,
and I would not understand a single word.
Like, Chilean spanish is already really hard, and then it's completely
out of context, I don't know what this person wants from me, and then
it's just [gestures] and I'm like "sorry, I don't understand you"
"I don't understand you", "I don't understand you",
over and over again.
And yeah, I mean, if you're on the phone,
there's also a little bit of noise maybe,
and I really have the feeling that that makes,
especially in a foreign language,
conversing that much harder, because you don't see the mouth movements,
it's not that clear of a sound, you don't see anything else, and yeah.
I would say so. Cool question.
I would guess so, I would guess so. Like, I mean,
especially for autistic people there is a lot of research
on language processing in general,
but I don't know of any studies that are
specifically for multimodal processing,
but I think there are quite a few.
Actually the experiment that I showed you,
the one that I worked on, as well, in the middle of the presentation,
the entropy stuff, we also did this with schizophrenic patients,
but we have not looked at the data yet.
So once this publication is done then,
somebody else will deal with
the clinical data, with the schizophrenic people,
and in general for schizophrenics there's,
there's a lot of,
like, language-related abnormalities,
and I think for autistic people as well.
I'm not sure about ADHD
but yeah, it would be a really interesting thing to,
to look at this for autistic people, for sure.
And maybe people did this. You can, maybe, you can look it up.
I don't have anything in my head right now but yeah.
There should be something.
I think one of the studies that I glanced over actually tried this.
So they had some gestures that were nonsense
I don't know if it was with abstract words or with concrete words,
but they used nonsense gestures,
and they still had an effect, but it was smaller.
So if you try to make this,
to integrate this into your studies,
I would suggest that you try to find an enactment that is as sensical as it gets
I mean, it's not that it's impossible to get enactment for abstract words,
you just have to be a little bit more creative, and I think the more creative
you will be the more effective. Like, similar to mnemonics,
like, the more crazy mnemonic is, the easier it is to remember.
I could see the same effect with enactments as well.
And if you should use signs from sign languages,
I think if you want to that's a cool idea, because then you automatically also learn
the sign language. And when I was preparing this presentation
I actually thought about this. Like why do we learn languages?
Like if I now start learning a language, why do I not learn the sign language
that goes with it, right?
I think it would make things easier, because you actually have
the enactment ready for you,
and it's just a cool thing, right?
You can talk with so many more people.
And also, I think in general, people should learn sign languages regardless
This became really clear to me actually at the Polyglot Gathering in 2019.
In Bratislava we were on some ship where there was a party.
There was loud music, then there were some people who knew sign language
maybe they are listening right now.
So they just started, they were like, on the dance floor,
and instead of screaming into each other's ears, like people usually do,
they just started to sign, and it was so smooth, like
why should we communicate with sound when we can do it with gestures. Right?
I think for many situations it would be a lot easier.
So yeah, if you can use the signs of your target language,
I think that's a cool idea.
Yeah, it does sound like that. Indeed, indeed.
Yeah, I can totally see that.
There is a study, that I came across while reasearching, but I didn't look into it.
If you want the reference, you can reach out to me somehow and I can see
if I can find it and send it to you.
I didn't look deeply into it,
and I think I wouldn't find it now quickly.
So again, there's something done, but I can't recall it from my head right now.
Well so, there's two things
maybe more things but let's start with two
So first of all when you learn a new word,
try to get the whole picture of the word.
Like the garlic example, try to imagine how it smells, how it feels,
how you chop it, try to enact it.
Take a moment, and really try to activate the whole functional network of this word.
And then the other thing was also just to use input with video,
if you're learning with some input.
Look up if you find some interesting channels on YouTube or something.
And then also, that might have not been clear from my presentation,
if you are conversing with people, use signs.
I don't know if people do this naturally in general, I think I kind of do it,
if I talk in my target language, and I'm not sure about a word
I will try to make sure with my hands that somehow,
something gets, like, to the other person.
So what I'm going for is that the other person recognizes
what I'm trying to say, and then gives me the word, right.
Like in the example with the glass of water,
if I don't know "drink" in some language I would,
I would try to [MIMES] Right? "I want to [MIMES]" Right?
And then have the other person gives me the word, because I'm actively reducing
the uncertainty that the other perso has, that is trying to predict
what I'm going to say, by giving gestures. Right?
So that would be my three practical implications for now.
Yeah so this is something that I don't know.
Again I think it's worth trying to do this with the sign language.
I mean, there's a system of really...
There's a system of really fitting gestures that people use,
and it might actually be a good idea to try this out, to use sign language
as you're learning the actual language,
to get this enactment working.
Might be more effective than making up your own gestures.
I mean if you make up your own gestures you have the advantage that
during the process of coming up with the gesture,
you are engaging your brain in a specific way
that's not there if you just get the gesture from somebody.
So there might be an advantage there,
but the other advantage is of course time that you can save,
and the ability to communicate with people that can't hear
So yeah, I think that's open for exploration, for sure.
Well you see it in sign language, right?
People that sign don't really speak.
And they get along pretty nicely.
Another question would be if all of society as a whole can do without verbal.
That's another question, but I think you can restructure society
in a way that everybody can communicate with gestures, for sure.
And according to some people, it was like that before speech developped.
So yeah, there has been some research, not much.
If you are interested in this, make sure to check out my presentation on this topic
from last year's Gathering, and also from last year's conference.
The conference one is not up on YouTube already, but the Gathering one,
and there's, in the end, I show some...
I show a study that was done on polyglots and hyper polyglots
actually only hyper polyglots I think.
And so they put people in a FMRI scanner, and just gave them language material.
And what they found is that the language network
was less active than for monolinguals.
So if you listen to something there's some areas
on the left side of your brain that light up :
you have some typical areas, like Broca's area, Wernicke's area,
and some other ones.
And they found that, for polyglots this "lighting up" is less,
and the interpretation was that the polyglots' language network,
through extensive practice, has become more and more efficient
at dealing with language. And therefore it needs less activation.
So this is one thing that you observe quite often,
when there's some process that you get really good at,
in your brain the activity that you see goes down,
because the network gets more efficient.
So that's why this paper was aptly titled
"The Small And Efficient Network Of Polyglots And Hyperpolyglots".
And they, you can also look this up as well,
they also made them listen to different languages,
and there, the better known the foreign language was
so the first experiment was completed in english, their mother tongue,
and then the second experiment they used their target languages like,
the second best language, third best language and so on.
And there, the lesser known a language, the less active the language network,
and the better known, the more active. So you have kind of the opposite effect.
And they interpreted this as reflecting that the more you know
in a target language, in a foreign language,
the more of the language network gets recruited, the more context you have.
So you have this effect of getting really efficient for your mother tongue,
and getting more of the whole message for,
the better you know a foreign language.
So this was the last question. Alright
Thanks for listening, thanks to the organizers for organizing this,
the streaming works really well, I'm really impressed
Thanks guys!