[Script Info]
Title: 
[Events]
Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
Dialogue: 0,0:00:06.95,0:00:11.19,Default,,0000,0000,0000,,Hello everyone, and a warm welcome to\NMultimodal Language Processing.
Dialogue: 0,0:00:11.45,0:00:17.00,Default,,0000,0000,0000,,My name is Xaver Funk, and I recently had\Nthe chance to really [involve] myself into
Dialogue: 0,0:00:17.16,0:00:20.45,Default,,0000,0000,0000,,this topic, because I am studying\Nneurosciences and this was kind of
Dialogue: 0,0:00:20.45,0:00:26.03,Default,,0000,0000,0000,,something that I had to do. And, yeah,\Nthat's what I want to share with you today.
Dialogue: 0,0:00:26.48,0:00:32.57,Default,,0000,0000,0000,,So, what I have been doing recently also,\Nis learning arabic, and a little bit of
Dialogue: 0,0:00:32.57,0:00:37.30,Default,,0000,0000,0000,,mongolian. And mostly what I did was,\NI had this stream of auditory signals
Dialogue: 0,0:00:37.30,0:00:43.12,Default,,0000,0000,0000,,that maybe came from the ASML audio, and\NI tried to match those to symbols that
Dialogue: 0,0:00:43.12,0:00:46.39,Default,,0000,0000,0000,,were representing these, right, in the\Nbook.
Dialogue: 0,0:00:47.47,0:00:52.00,Default,,0000,0000,0000,,And I kind of had this feeling that this\Nis incomplete.
Dialogue: 0,0:00:52.00,0:00:53.84,Default,,0000,0000,0000,,So there is something missing there.
Dialogue: 0,0:00:53.84,0:00:59.79,Default,,0000,0000,0000,,And while I was on the other hand,\Nstudying a lot about multimodal language
Dialogue: 0,0:00:59.79,0:01:03.71,Default,,0000,0000,0000,,processing, which I how gestures influence\Nprocessing and stuff like that.
Dialogue: 0,0:01:04.57,0:01:07.92,Default,,0000,0000,0000,,I came to the conclusion that, yeah, there\Nis something missing.
Dialogue: 0,0:01:08.60,0:01:12.27,Default,,0000,0000,0000,,In our world today, we are all litterate\Nso we mostly think of languages as
Dialogue: 0,0:01:12.27,0:01:16.48,Default,,0000,0000,0000,,these auditory signals, these mouth noises\Nand the symbols that represent these.
Dialogue: 0,0:01:16.48,0:01:21.58,Default,,0000,0000,0000,,But there is so much more going on, in\Nface to face communication and, yeah,
Dialogue: 0,0:01:21.58,0:01:25.28,Default,,0000,0000,0000,,I want to make this point clear, with a\Nvirtual experiment.
Dialogue: 0,0:01:25.65,0:01:33.14,Default,,0000,0000,0000,,So, I want to invite you to first of all,\Nlisten to this audio excerpt, from an
Dialogue: 0,0:01:33.14,0:01:38.48,Default,,0000,0000,0000,,"Easy Languages" video. And I give you the\Nsubtitles here, with the english
Dialogue: 0,0:01:38.48,0:01:43.78,Default,,0000,0000,0000,,translation as well. So, basically, these\Nare auditory signals in Dutch,
Dialogue: 0,0:01:43.78,0:01:47.20,Default,,0000,0000,0000,,and sequences of symbols\Nin Dutch and English.
Dialogue: 0,0:01:47.20,0:01:51.47,Default,,0000,0000,0000,,And for the people learning Dutch, please\Njust ignore the English, just to make it
Dialogue: 0,0:01:51.47,0:01:55.20,Default,,0000,0000,0000,,a little bit harder. And people who know\NDutch, please close your eyes, so that
Dialogue: 0,0:01:55.20,0:01:59.70,Default,,0000,0000,0000,,you don't see it at all.\NSo, let's go.
Dialogue: 0,0:02:40.56,0:02:46.42,Default,,0000,0000,0000,,So, when I was listening to this at first,\NI was - because I know some Dutch,
Dialogue: 0,0:02:46.42,0:02:50.34,Default,,0000,0000,0000,,I was understanding quite a lot,\Nbut, kind of, not everything.
Dialogue: 0,0:02:50.34,0:02:54.59,Default,,0000,0000,0000,,And then, I watched the video that goes\Nwith it, it was kind of a different experience.
Dialogue: 0,0:02:54.90,0:03:00.90,Default,,0000,0000,0000,,And that's what we are going to do now.\NSo just watch the video, and if you can,
Dialogue: 0,0:03:00.90,0:03:07.15,Default,,0000,0000,0000,,see how these two women, that are\Ninterviewed here, are interacting
Dialogue: 0,0:03:07.15,0:03:10.17,Default,,0000,0000,0000,,with the interviewer, and\Nbetween each other.
Dialogue: 0,0:03:50.93,0:03:54.01,Default,,0000,0000,0000,,So, I hoped this worked, and you felt\Na little bit different now.
Dialogue: 0,0:03:54.01,0:03:57.89,Default,,0000,0000,0000,,And even for the people who don't know\NDutch, I hope you could kind of follow
Dialogue: 0,0:03:57.89,0:04:03.06,Default,,0000,0000,0000,,what was going on. And even if you\Ndidn't, the point I want to make is that
Dialogue: 0,0:04:03.06,0:04:06.29,Default,,0000,0000,0000,,messages are not only auditory,\Nthey are always also visual.
Dialogue: 0,0:04:06.60,0:04:11.19,Default,,0000,0000,0000,,We have a lot of non-auditory\Narticulators, like 43 face muscles
Dialogue: 0,0:04:11.19,0:04:15.92,Default,,0000,0000,0000,,for example, and then 2x 34 muscles\Nin the hands, and then even more in
Dialogue: 0,0:04:15.92,0:04:20.89,Default,,0000,0000,0000,,the arms, in our torso.\NAnd the people in this video
Dialogue: 0,0:04:20.89,0:04:24.78,Default,,0000,0000,0000,,really knew how to use these.\NSo for example we had a lot of
Dialogue: 0,0:04:24.78,0:04:27.99,Default,,0000,0000,0000,,facial movement going on,\Nlike you see on the top, here.
Dialogue: 0,0:04:27.99,0:04:32.19,Default,,0000,0000,0000,,See how she raise her eyebrows,\Nand then, you have this head tilting
Dialogue: 0,0:04:32.19,0:04:35.19,Default,,0000,0000,0000,,at the end, that really put \Nan emphasis on what she's saying.
Dialogue: 0,0:04:35.19,0:04:39.19,Default,,0000,0000,0000,,And then there is a lot of gaze switching\Nas well.That's right in the begining,
Dialogue: 0,0:04:39.19,0:04:42.52,Default,,0000,0000,0000,,when she says \N(Dutch): Oh genoeg ! Heb je even ?
Dialogue: 0,0:04:42.52,0:04:46.34,Default,,0000,0000,0000,,So, "Oh, there is so much that\NI want to see! Do you have some time?"
Dialogue: 0,0:04:46.57,0:04:51.62,Default,,0000,0000,0000,,But, she doesn't really say "time", she\Nsays "Heb je even", "Do you have a little"
Dialogue: 0,0:04:51.62,0:04:55.31,Default,,0000,0000,0000,,And for me, when I was only listening,\NI didn't quite get what she was saying,
Dialogue: 0,0:04:55.31,0:04:58.42,Default,,0000,0000,0000,,but when I saw how she adresses\Nthe interviewer, I kind of got it,
Dialogue: 0,0:04:58.42,0:04:59.42,Default,,0000,0000,0000,,afterwards.
Dialogue: 0,0:05:00.57,0:05:05.92,Default,,0000,0000,0000,,So then there is of course manual gestures\Nlike "hoop op mijn list",
Dialogue: 0,0:05:05.92,0:05:11.67,Default,,0000,0000,0000,,that's that one here, "hoop op mijn list".\NShe says "berglandschap", so that's
Dialogue: 0,0:05:11.67,0:05:16.88,Default,,0000,0000,0000,,a mountain range, and then "lang geleden"\N"long time ago", right?
Dialogue: 0,0:05:16.88,0:05:22.19,Default,,0000,0000,0000,,So there is a lot of messages that are\Nsupported with these manual gestures.
Dialogue: 0,0:05:22.71,0:05:26.59,Default,,0000,0000,0000,,Then there is also stuff like this\Nnose scratching, where we don't even know
Dialogue: 0,0:05:26.59,0:05:30.29,Default,,0000,0000,0000,,is there something to it, or is it just\Na nose scratching.
Dialogue: 0,0:05:30.58,0:05:34.04,Default,,0000,0000,0000,,Does it carry some information?\NWe don't know.
Dialogue: 0,0:05:35.16,0:05:37.81,Default,,0000,0000,0000,,And then, lastly also arm and torso\Nmovements.
Dialogue: 0,0:05:37.81,0:05:41.14,Default,,0000,0000,0000,,And also if you watch at the top here,\Nyou have nodding. So you see how
Dialogue: 0,0:05:41.14,0:05:45.61,Default,,0000,0000,0000,,these two kind of nod together, they\Nreally give us the impression of
Dialogue: 0,0:05:45.61,0:05:51.09,Default,,0000,0000,0000,,how good friends they are, right.\NAnd then if you look at this bottom part
Dialogue: 0,0:05:51.09,0:05:54.59,Default,,0000,0000,0000,,here, that's my favorite part of the\Nvideo. You really have this complex
Dialogue: 0,0:05:54.59,0:06:00.29,Default,,0000,0000,0000,,orchestration of different gestures,\Nand they are turn-taking.
Dialogue: 0,0:06:00.55,0:06:07.07,Default,,0000,0000,0000,,So, the one on the right says something,\Nand the one on the left answers that
Dialogue: 0,0:06:07.07,0:06:12.96,Default,,0000,0000,0000,,perfectly, and then you have gestures,\Nand then the, putting their hair back,
Dialogue: 0,0:06:12.96,0:06:18.47,Default,,0000,0000,0000,,right, so there is so much going on\Nbetween them, and it really gives more
Dialogue: 0,0:06:18.47,0:06:21.99,Default,,0000,0000,0000,,than just the auditory message, right.
Dialogue: 0,0:06:23.15,0:06:27.49,Default,,0000,0000,0000,,So, note that there is something that\Nour brain has to achieve here.
Dialogue: 0,0:06:27.49,0:06:31.49,Default,,0000,0000,0000,,Mainly two things : so, it has to\Nsegregate all of the stuff that is not
Dialogue: 0,0:06:31.49,0:06:34.33,Default,,0000,0000,0000,,important for the message.\NThat's the segregation problem.
Dialogue: 0,0:06:34.50,0:06:37.99,Default,,0000,0000,0000,,From the important stuff, and then,\Ntake all the important stuff, all
Dialogue: 0,0:06:37.99,0:06:41.89,Default,,0000,0000,0000,,the auditory and visuals information and\Nput it together into a coherent message.
Dialogue: 0,0:06:41.95,0:06:45.65,Default,,0000,0000,0000,,That's our binding problem.\NAnd all of this, note, all of this is
Dialogue: 0,0:06:45.65,0:06:49.26,Default,,0000,0000,0000,,under a really tight time constraint,\Nwhen you're turn taking, when you're
Dialogue: 0,0:06:49.26,0:06:51.65,Default,,0000,0000,0000,,having a conversation.\NAnd if you say something and
Dialogue: 0,0:06:51.65,0:06:54.61,Default,,0000,0000,0000,,the other person say something,\Nand there is not that much time
Dialogue: 0,0:06:54.61,0:06:57.42,Default,,0000,0000,0000,,between turns. And if you need\Nmore time, then that also has
Dialogue: 0,0:06:57.42,0:07:01.95,Default,,0000,0000,0000,,a meaning, right ? If you take time, then\Nthat means that you're hesitating
Dialogue: 0,0:07:01.95,0:07:05.15,Default,,0000,0000,0000,,to answer, maybe there is something\Ngoing on with you emotionally...
Dialogue: 0,0:07:05.15,0:07:10.21,Default,,0000,0000,0000,,So you don't want to have that as well.\NSo, yeah, so basically, this is really
Dialogue: 0,0:07:10.21,0:07:12.71,Default,,0000,0000,0000,,a huge computational problem\Nfor your brain.
Dialogue: 0,0:07:13.84,0:07:18.01,Default,,0000,0000,0000,,And well, how did your brain do?\NDid you feel the video was more difficult
Dialogue: 0,0:07:18.01,0:07:21.66,Default,,0000,0000,0000,,than the audio ? Did you understand more,\Nor did you understand less?
Dialogue: 0,0:07:21.81,0:07:25.78,Default,,0000,0000,0000,,Did you feel more in the scene, maybe?\NCatching more informations between the lines?
Dialogue: 0,0:07:26.65,0:07:31.50,Default,,0000,0000,0000,,And well, for me, at least as you might\Nguess, for me it was way easier
Dialogue: 0,0:07:31.50,0:07:34.94,Default,,0000,0000,0000,,to follow with the video to interpret\Nthese gestures. And this is kind of
Dialogue: 0,0:07:34.94,0:07:40.70,Default,,0000,0000,0000,,a paradox. So, how come that processing\Nmore signals simultaneously is easier
Dialogue: 0,0:07:40.70,0:07:45.93,Default,,0000,0000,0000,,that processing speech alone?\NAnd this also was shown
Dialogue: 0,0:07:45.93,0:07:49.23,Default,,0000,0000,0000,,in the litterature, so people have made\Nexperience with this.
Dialogue: 0,0:07:49.73,0:07:54.57,Default,,0000,0000,0000,,And this is really a surprising\Nfacilitation. For example, there are
Dialogue: 0,0:07:54.57,0:07:57.13,Default,,0000,0000,0000,,lot of studies, I'll just give you\None example.
Dialogue: 0,0:07:57.42,0:08:01.12,Default,,0000,0000,0000,,So in this study they showed people\Na "prime", so this was some video
Dialogue: 0,0:08:01.12,0:08:05.36,Default,,0000,0000,0000,,of an action that somebody did, and then\Nthey showed the people different videos.
Dialogue: 0,0:08:05.80,0:08:10.35,Default,,0000,0000,0000,,And the videos were either completely\Ncongruent, so what was said was the same
Dialogue: 0,0:08:10.35,0:08:14.35,Default,,0000,0000,0000,,as the gesture, and was the same as\Nthis prime. So in that case it would be
Dialogue: 0,0:08:14.35,0:08:18.61,Default,,0000,0000,0000,,"chop" and doing the chopping gesture.\NAnd then there were different conditions
Dialogue: 0,0:08:18.61,0:08:26.41,Default,,0000,0000,0000,,where either the speech was congruent,\Nincongruent, and the gesture was congruent.
Dialogue: 0,0:08:26.61,0:08:29.84,Default,,0000,0000,0000,,Or the speech was incongruent and\Nthe gesture was congruent.
Dialogue: 0,0:08:29.95,0:08:34.36,Default,,0000,0000,0000,,And then they had also weakly congruent\Nstuff, like, this for "chopping",
Dialogue: 0,0:08:34.36,0:08:37.75,Default,,0000,0000,0000,,but this is actually cutting so this is\Nonly weakly incongruent.
Dialogue: 0,0:08:37.75,0:08:41.52,Default,,0000,0000,0000,,And then this twisting, which is\Nstrongly incongruent.
Dialogue: 0,0:08:42.48,0:08:47.26,Default,,0000,0000,0000,,And then people had to press a button\Nfor "yes" if either the speech or
Dialogue: 0,0:08:47.26,0:08:52.13,Default,,0000,0000,0000,,the gesture were related to the prime,\Nand no if neither speech nor gesture
Dialogue: 0,0:08:52.13,0:08:55.38,Default,,0000,0000,0000,,was related to the prime.\NAnd what the people found out was that
Dialogue: 0,0:08:55.38,0:08:59.05,Default,,0000,0000,0000,,there were differences in response times,\Nand also in the proportion of errors
Dialogue: 0,0:08:59.05,0:09:02.40,Default,,0000,0000,0000,,that people did, as soon as soon as \Nthere were something incongruent.
Dialogue: 0,0:09:02.40,0:09:07.72,Default,,0000,0000,0000,,And from that the authors come to\Nthe conclusion that really, speech and
Dialogue: 0,0:09:07.72,0:09:11.91,Default,,0000,0000,0000,,gestures are two sides of the\Nsame coin, they mutually interact
Dialogue: 0,0:09:11.91,0:09:16.08,Default,,0000,0000,0000,,to enhance comprehension.\NAnd now the big question is, of course,
Dialogue: 0,0:09:16.08,0:09:19.32,Default,,0000,0000,0000,,how does our brain achieve this surprising\Nfacilitation?
Dialogue: 0,0:09:21.98,0:09:25.80,Default,,0000,0000,0000,,And we can look back at turn-taking,\Nto maybe get some clues here.
Dialogue: 0,0:09:25.80,0:09:32.05,Default,,0000,0000,0000,,So on average a turn-take takes only\Nabout 0 to 200 miliseconds, which is
Dialogue: 0,0:09:32.05,0:09:38.07,Default,,0000,0000,0000,,a fifth of a second. You can see in this\Nvideo how fast she is responding,
Dialogue: 0,0:09:38.07,0:09:43.10,Default,,0000,0000,0000,,right now, after this, like, this is\Nan instant, right?
Dialogue: 0,0:09:43.86,0:09:47.95,Default,,0000,0000,0000,,And this is quite extraordinary, because\Nproducing a single word actually takes
Dialogue: 0,0:09:47.95,0:09:49.37,Default,,0000,0000,0000,,about 600 miliseconds.
Dialogue: 0,0:09:49.69,0:09:54.87,Default,,0000,0000,0000,,So if I just prompt you to say a word,\Nyou would take 600 miliseconds to say it.
Dialogue: 0,0:09:55.85,0:10:00.10,Default,,0000,0000,0000,,So there's something going on, it seems\Nlike we are predicting already what we are
Dialogue: 0,0:10:00.10,0:10:04.15,Default,,0000,0000,0000,,going to say before the turn of the other\Nperson is finished, and we already prepare
Dialogue: 0,0:10:04.15,0:10:05.36,Default,,0000,0000,0000,,our turn.
Dialogue: 0,0:10:05.36,0:10:09.56,Default,,0000,0000,0000,,So there is something that is going on,\Nthat has to do with prediction.
Dialogue: 0,0:10:10.69,0:10:14.99,Default,,0000,0000,0000,,Most language use in conversation\Nhas to be based on prediction somehow.
Dialogue: 0,0:10:14.99,0:10:18.31,Default,,0000,0000,0000,,And this is quite nice, because prediction\Nis anyways the current hype
Dialogue: 0,0:10:18.31,0:10:21.81,Default,,0000,0000,0000,,in neuroscience nowadays, and it's\Nbasically a good candidate for
Dialogue: 0,0:10:21.81,0:10:23.98,Default,,0000,0000,0000,,the overarching function of the brain.
Dialogue: 0,0:10:25.30,0:10:30.63,Default,,0000,0000,0000,,And many people think that what we are\Ndoing in our daily lives is basically
Dialogue: 0,0:10:30.63,0:10:34.38,Default,,0000,0000,0000,,constantly computing and updating\Nprobability distributions.
Dialogue: 0,0:10:35.03,0:10:38.53,Default,,0000,0000,0000,,And this applies both to action,\Nto perception, and also to language.
Dialogue: 0,0:10:39.55,0:10:43.67,Default,,0000,0000,0000,,So, this will be a rephrasing of\Nthe problem we had before,
Dialogue: 0,0:10:44.42,0:10:46.93,Default,,0000,0000,0000,,as a prediction problem.\NAnd this become then,
Dialogue: 0,0:10:46.93,0:10:51.08,Default,,0000,0000,0000,,"given the preceding context - so, given\Nall the words that come before - what word
Dialogue: 0,0:10:51.08,0:10:56.68,Default,,0000,0000,0000,,is most likely to come up next?"\NRight? And to make this more clear,
Dialogue: 0,0:10:56.68,0:11:01.00,Default,,0000,0000,0000,,let me give you a quick example:\Nso, imagine I come to you and I say,
Dialogue: 0,0:11:01.00,0:11:03.95,Default,,0000,0000,0000,,without anymore context "I would like to".
Dialogue: 0,0:11:05.03,0:11:07.97,Default,,0000,0000,0000,,And then, you don't know what I'm going\Nto say next, right?
Dialogue: 0,0:11:07.97,0:11:11.60,Default,,0000,0000,0000,,It could be any of these, for example.\NI would like to drink, eat, work...
Dialogue: 0,0:11:11.60,0:11:15.51,Default,,0000,0000,0000,,And so on.\NAnd now, if I shape my hand
Dialogue: 0,0:11:15.51,0:11:20.96,Default,,0000,0000,0000,,in the form of a "C", and I put it to\Nmy mouth, like this, while I say
Dialogue: 0,0:11:20.96,0:11:26.54,Default,,0000,0000,0000,,"I would like to", then your probability\Ndistribution over these words changes
Dialogue: 0,0:11:26.54,0:11:31.13,Default,,0000,0000,0000,,in such a way that "drink" is much more\Nlikely to be the next word.
Dialogue: 0,0:11:31.13,0:11:35.05,Default,,0000,0000,0000,,And maybe "eat" also a little bit, but\Nthe others words probably not,
Dialogue: 0,0:11:35.05,0:11:39.48,Default,,0000,0000,0000,,because, you can associate this\Ngesture with drinking, or a little bit
Dialogue: 0,0:11:39.48,0:11:43.06,Default,,0000,0000,0000,,with eating, because it's also\Nsomething that you put to your mouth,
Dialogue: 0,0:11:43.06,0:11:47.45,Default,,0000,0000,0000,,but mostly this is commonly understood\Nas "drinking", right.
Dialogue: 0,0:11:48.51,0:11:53.94,Default,,0000,0000,0000,,So, in this way gestures add context to\Npredictions and help this process of
Dialogue: 0,0:11:53.94,0:11:56.66,Default,,0000,0000,0000,,predicting, and that also helps the\Ncomprehension.
Dialogue: 0,0:11:57.92,0:12:01.49,Default,,0000,0000,0000,,And, we can actually measure prediction,\Nusing neurophysiology.
Dialogue: 0,0:12:02.58,0:12:09.76,Default,,0000,0000,0000,,So this is EEG, and "EEG" stands for\N"Electro Encephalography", and
Dialogue: 0,0:12:09.76,0:12:13.34,Default,,0000,0000,0000,,it's basically putting electrodes on the\Nscalp, and then measuring
Dialogue: 0,0:12:13.34,0:12:20.18,Default,,0000,0000,0000,,the brain activity that's below.\NIf you do this, well you can measure
Dialogue: 0,0:12:20.18,0:12:23.91,Default,,0000,0000,0000,,brain activity basically.\NWhat people usually do, is that
Dialogue: 0,0:12:23.91,0:12:27.91,Default,,0000,0000,0000,,they give people these sentences.\NSo these could be normal sentences,
Dialogue: 0,0:12:27.91,0:12:31.46,Default,,0000,0000,0000,,like this one : "It was his first day\Nat work."
Dialogue: 0,0:12:32.11,0:12:37.06,Default,,0000,0000,0000,,Or it could be so-called garden-path\Nsentences. So these are sentences that are
Dialogue: 0,0:12:37.06,0:12:42.77,Default,,0000,0000,0000,,somehow manipulated artificially\Nto elicit some response. Right?
Dialogue: 0,0:12:42.77,0:12:46.50,Default,,0000,0000,0000,,So this would be : "He spread the warm\Nbread with socks."
Dialogue: 0,0:12:46.50,0:12:51.45,Default,,0000,0000,0000,,So you may have a weird feeling on\Nyour head, because nobody spreads
Dialogue: 0,0:12:51.45,0:12:56.88,Default,,0000,0000,0000,,the bread with socks. And this weird\Nfeeling, if we would measure you with
Dialogue: 0,0:12:56.88,0:13:03.76,Default,,0000,0000,0000,,an EEG, would constitute this reaction\Nhere, that's a so-called N400.
Dialogue: 0,0:13:03.76,0:13:09.95,Default,,0000,0000,0000,,"N" because it is a negative polarity,\Nand it's 400 miliseconds after the word.
Dialogue: 0,0:13:10.21,0:13:14.49,Default,,0000,0000,0000,,So, all of this above here is just\Nelectrical activity, right? And you have
Dialogue: 0,0:13:14.49,0:13:18.98,Default,,0000,0000,0000,,this really pronounced peak, when\Nthere is a violation of the semantics,
Dialogue: 0,0:13:18.98,0:13:24.67,Default,,0000,0000,0000,,like with "socks".\NAnd it's also taken to be a prediction
Dialogue: 0,0:13:24.67,0:13:28.65,Default,,0000,0000,0000,,error. So you did not predict socks,\Nyou predicted Nutella, for example,
Dialogue: 0,0:13:28.65,0:13:33.19,Default,,0000,0000,0000,,or honey. But not socks. And this is\Nreflected in this N400 prediction error.
Dialogue: 0,0:13:33.40,0:13:38.22,Default,,0000,0000,0000,,So people are doing this a lot, like, showing\Nthese sentences that are somehow manipulated.
Dialogue: 0,0:13:38.22,0:13:43.12,Default,,0000,0000,0000,,We have another example here, this is\Nanother topic : if you write in all caps
Dialogue: 0,0:13:43.12,0:13:45.93,Default,,0000,0000,0000,,you have this kind of response for\Nexample.
Dialogue: 0,0:13:46.94,0:13:50.56,Default,,0000,0000,0000,,But, what I want to do now with you\Nis bringing you more to the cutting edge
Dialogue: 0,0:13:50.56,0:13:55.81,Default,,0000,0000,0000,,of what is currently done in multimodal\Nprocessing research.
Dialogue: 0,0:13:56.84,0:14:01.04,Default,,0000,0000,0000,,So the trend is to go away from these\Nartificially constructed sentences, and
Dialogue: 0,0:14:01.04,0:14:05.63,Default,,0000,0000,0000,,more towards naturalistic language\Ncomprehension. So, using actual stories,
Dialogue: 0,0:14:05.63,0:14:11.58,Default,,0000,0000,0000,,actual sentences, that are not manipulated\Nin any way. And this will be combined with
Dialogue: 0,0:14:11.58,0:14:15.88,Default,,0000,0000,0000,,computational linguistics - how that\Nworks, you will see in a bit.
Dialogue: 0,0:14:16.32,0:14:21.38,Default,,0000,0000,0000,,And also, yeah, with that you can look at\Nmultimodal processing if you just add
Dialogue: 0,0:14:21.38,0:14:26.02,Default,,0000,0000,0000,,a video to the audio that\Nyou make people listen to.
Dialogue: 0,0:14:27.27,0:14:32.69,Default,,0000,0000,0000,,And what it might look like\Nis like this. So, this is one study
Dialogue: 0,0:14:32.69,0:14:36.27,Default,,0000,0000,0000,,that is currently not published\Nofficially yet. It is already on
Dialogue: 0,0:14:36.27,0:14:43.74,Default,,0000,0000,0000,,the archive. And I want to use this to\Nillustrate to you how we might research
Dialogue: 0,0:14:43.74,0:14:49.47,Default,,0000,0000,0000,,naturalistic language comprehension.\NSo the general planners get some
Dialogue: 0,0:14:49.47,0:14:54.41,Default,,0000,0000,0000,,per-word measures - so these would be\Nthese ones here. So for each word,
Dialogue: 0,0:14:54.41,0:14:59.87,Default,,0000,0000,0000,,there is some value attached.\NAnd then we can use these as regressors
Dialogue: 0,0:14:59.87,0:15:04.74,Default,,0000,0000,0000,,in the big linear regression model.\NSo, using fancy statistics, and with that
Dialogue: 0,0:15:04.74,0:15:12.78,Default,,0000,0000,0000,,we're basically asking our data "how well\Nare you predicted by these regressors?"
Dialogue: 0,0:15:14.40,0:15:19.28,Default,,0000,0000,0000,,And for example, this one here is\Nsurprise and this is closely related
Dialogue: 0,0:15:19.28,0:15:25.16,Default,,0000,0000,0000,,to predictions or prediction errors.\NSo this is the negative log probability
Dialogue: 0,0:15:25.16,0:15:28.63,Default,,0000,0000,0000,,of a word, given all of the words\Nthat come before it.
Dialogue: 0,0:15:28.63,0:15:32.96,Default,,0000,0000,0000,,So, this is the contexte, basically,\Nand this is some word, "w".
Dialogue: 0,0:15:34.11,0:15:37.41,Default,,0000,0000,0000,,So this is basically telling you how\Nunpredictable is a given word.
Dialogue: 0,0:15:38.56,0:15:42.68,Default,,0000,0000,0000,,And this measure is base on computational\Nlanguage models, so for example,
Dialogue: 0,0:15:42.68,0:15:47.61,Default,,0000,0000,0000,,they would take the whole corpus of\Na language, and then, see which words
Dialogue: 0,0:15:47.61,0:15:54.08,Default,,0000,0000,0000,,occurs after each other, and thereby get\Nto this value of how unpredictable it is.
Dialogue: 0,0:15:55.96,0:16:00.78,Default,,0000,0000,0000,,And then, they have another thing here.\NThey use the fundamental frequency of each
Dialogue: 0,0:16:00.78,0:16:05.53,Default,,0000,0000,0000,,word as a pitch indicator, to control\Nfor prosody, which is also pretty cool.
Dialogue: 0,0:16:06.49,0:16:10.66,Default,,0000,0000,0000,,So they let loose their linear\Nregression models, with these predictors,
Dialogue: 0,0:16:10.66,0:16:17.56,Default,,0000,0000,0000,,so they have a surprisal value for each\Nword, for example, a prosody, ready for
Dialogue: 0,0:16:17.56,0:16:21.68,Default,,0000,0000,0000,,each word, then they indicate where\Nthere are meaningful gestures happening,
Dialogue: 0,0:16:21.68,0:16:25.41,Default,,0000,0000,0000,,and, yeah, also, mouth movements.
Dialogue: 0,0:16:26.94,0:16:30.41,Default,,0000,0000,0000,,And, what came out of this, one finding\Nthat might be interesting for us,
Dialogue: 0,0:16:30.41,0:16:36.91,Default,,0000,0000,0000,,now, is that for meaningful gestures,\Nthe N400 is less negative.
Dialogue: 0,0:16:37.81,0:16:42.81,Default,,0000,0000,0000,,So you can also see this here : for\Nmeaningful gesture, this blue line,
Dialogue: 0,0:16:42.81,0:16:47.82,Default,,0000,0000,0000,,you see that it is a lot less negative\Nthan the red line where the gestures are
Dialogue: 0,0:16:47.82,0:16:53.22,Default,,0000,0000,0000,,absent. And then there's also, that's why\NI told you about surprisal, an interesting
Dialogue: 0,0:16:53.22,0:16:57.59,Default,,0000,0000,0000,,interaction between gestures and\Nsurprisal. So, the higher the surprisal,
Dialogue: 0,0:16:57.59,0:17:03.07,Default,,0000,0000,0000,,the less unexpected a word, the stronger\Nthis facilitating effect of gestures is.
Dialogue: 0,0:17:04.05,0:17:09.72,Default,,0000,0000,0000,,Which is also really interesting.\NThen there's, this is a similar study,
Dialogue: 0,0:17:11.61,0:17:14.87,Default,,0000,0000,0000,,that I actually got the chance to work on,\Nwith a colleague.
Dialogue: 0,0:17:16.55,0:17:21.45,Default,,0000,0000,0000,,So what we did here, we had a measure of\Nentropy. This measures basically the
Dialogue: 0,0:17:21.52,0:17:25.60,Default,,0000,0000,0000,,uncertainty about the next word.\NSo if you think back to the example
Dialogue: 0,0:17:25.60,0:17:30.97,Default,,0000,0000,0000,,we had before, where I was telling\Nyou "I would like to", and then something,
Dialogue: 0,0:17:30.97,0:17:34.86,Default,,0000,0000,0000,,but without context, that would be really\Nhigh entropy, really high uncertainty :
Dialogue: 0,0:17:34.86,0:17:39.53,Default,,0000,0000,0000,,you don't know what's coming next. Right?\NThen we also had surprisal, we had word
Dialogue: 0,0:17:39.53,0:17:43.63,Default,,0000,0000,0000,,frequency, how often the word came up.\NAnd IVC is a measure of,
Dialogue: 0,0:17:43.63,0:17:49.89,Default,,0000,0000,0000,,it's an abbreviation for "instantaneous\Nvisual change", so, how much the actor
Dialogue: 0,0:17:49.89,0:17:56.20,Default,,0000,0000,0000,,moved while we were showing this to\Nthe people. And then speech envelope,
Dialogue: 0,0:17:56.20,0:18:00.77,Default,,0000,0000,0000,,this is basically a measure of the level\Nof the sound.
Dialogue: 0,0:18:02.66,0:18:08.80,Default,,0000,0000,0000,,And what we found is - and this\Nby the way was an FRMI experiment,
Dialogue: 0,0:18:08.80,0:18:12.61,Default,,0000,0000,0000,,so we can look at wich regions are active\Nduring some condition.
Dialogue: 0,0:18:13.43,0:18:17.99,Default,,0000,0000,0000,,And for words where the surprisal was\Nreally high, there were these regions
Dialogue: 0,0:18:17.99,0:18:20.99,Default,,0000,0000,0000,,in red active, and for words where\Nentropy was really high,
Dialogue: 0,0:18:20.99,0:18:25.51,Default,,0000,0000,0000,,these regions in blue. And now if\Nwe look at interactions with gestures
Dialogue: 0,0:18:25.51,0:18:31.81,Default,,0000,0000,0000,,for the entropy condition, we can see that\Nwhen there were gestures present, we had
Dialogue: 0,0:18:31.81,0:18:37.79,Default,,0000,0000,0000,,really specific activations compared\Nto when there were no gestures present,
Dialogue: 0,0:18:37.79,0:18:41.28,Default,,0000,0000,0000,,in situations where there is high\Nentropy, so high uncertainty.
Dialogue: 0,0:18:42.59,0:18:50.07,Default,,0000,0000,0000,,So with these tools we try to get into\Nthe processes that underlie prediction
Dialogue: 0,0:18:50.07,0:18:55.90,Default,,0000,0000,0000,,in language.\NSo let's take a step back, and have a look
Dialogue: 0,0:18:55.90,0:19:00.07,Default,,0000,0000,0000,,at kind of a more global\Nevolutionnary perspective.
Dialogue: 0,0:19:01.67,0:19:06.39,Default,,0000,0000,0000,,We know from primate research that gesture\Nand gaze are crucial for communication.
Dialogue: 0,0:19:06.77,0:19:10.03,Default,,0000,0000,0000,,You can see it in this video : this ape\Nright here does this gesture,
Dialogue: 0,0:19:10.46,0:19:14.88,Default,,0000,0000,0000,,and this signals to its mother\Nto pick her up. Right?
Dialogue: 0,0:19:15.48,0:19:21.97,Default,,0000,0000,0000,,So these are bonobos, and you can see\Nright now, this "pick me up" gesture.
Dialogue: 0,0:19:23.52,0:19:29.34,Default,,0000,0000,0000,,And Federico Rossano, from the Max Planck\NInstitute for Evolutionnary Institute,
Dialogue: 0,0:19:30.19,0:19:35.94,Default,,0000,0000,0000,,could show that this gesture get more\Nand more ritualized, to the point where
Dialogue: 0,0:19:35.94,0:19:45.99,Default,,0000,0000,0000,,it becomes only a small wrist bend with\Nthe arm and one gaze, to instantiate
Dialogue: 0,0:19:45.99,0:19:50.19,Default,,0000,0000,0000,,this carry behavior. Right?\NSo you see that there is also kind of
Dialogue: 0,0:19:50.19,0:19:56.50,Default,,0000,0000,0000,,a prediction involved : the mother has\Nto predict what the child is wanting
Dialogue: 0,0:19:56.50,0:20:00.09,Default,,0000,0000,0000,,to do, right? \NGoing from this, to only this.
Dialogue: 0,0:20:01.17,0:20:06.99,Default,,0000,0000,0000,,Then, building on this, there are some\Nauthors that propose that speech and
Dialogue: 0,0:20:06.99,0:20:11.69,Default,,0000,0000,0000,,gesture have a common origin.\NAnd the idea here is that, through
Dialogue: 0,0:20:11.69,0:20:15.20,Default,,0000,0000,0000,,these ritualized gestures that\Nwe've just seen in those bonobos,
Dialogue: 0,0:20:15.20,0:20:18.65,Default,,0000,0000,0000,,after a time there will be a proto\Nsign language evolving.
Dialogue: 0,0:20:19.00,0:20:22.14,Default,,0000,0000,0000,,Which then at some point will be\Naccompanied by sound as well,
Dialogue: 0,0:20:22.14,0:20:27.50,Default,,0000,0000,0000,,evolving into a proto speech language.\NAnd then the proto sign, the proto speech,
Dialogue: 0,0:20:27.50,0:20:32.85,Default,,0000,0000,0000,,will reinforce each other more and more,\Nuntil language emerges.
Dialogue: 0,0:20:34.38,0:20:40.47,Default,,0000,0000,0000,,And another point, here,\Nor an observation : those of you who have
Dialogue: 0,0:20:40.47,0:20:44.14,Default,,0000,0000,0000,,tried sign language, it kind of feel\Nsurprinsingly natural, right?
Dialogue: 0,0:20:44.14,0:20:50.72,Default,,0000,0000,0000,,So, if speech is the true communication\Nmedium for humans, why is it
Dialogue: 0,0:20:50.72,0:20:56.29,Default,,0000,0000,0000,,thet sign language feels so real,\Nso natural, right?
Dialogue: 0,0:20:58.20,0:21:03.44,Default,,0000,0000,0000,,And then another point that goes into this\Ntheory is that voluntary hand movements
Dialogue: 0,0:21:03.44,0:21:07.55,Default,,0000,0000,0000,,came before voluntary breathing. And you\Nneed voluntary breathing to articulate
Dialogue: 0,0:21:07.55,0:21:08.82,Default,,0000,0000,0000,,yourself, right?
Dialogue: 0,0:21:09.68,0:21:13.90,Default,,0000,0000,0000,,So, also, just as complementary to speech,
Dialogue: 0,0:21:13.90,0:21:18.06,Default,,0000,0000,0000,,you can more easily show spatial\Nrelations between things.
Dialogue: 0,0:21:18.97,0:21:22.62,Default,,0000,0000,0000,,And then, if you look at child development\Nthe same pattern : gesture develop
Dialogue: 0,0:21:22.62,0:21:29.88,Default,,0000,0000,0000,,before speech, and pre-speech turn-taking\Nis faster than later.
Dialogue: 0,0:21:29.88,0:21:34.58,Default,,0000,0000,0000,,So if you're a baby and you gesture,\Nthe turn-taking with your mother,
Dialogue: 0,0:21:34.58,0:21:40.15,Default,,0000,0000,0000,,the communication is quite fast,\Nit's almost adult level turn-taking.
Dialogue: 0,0:21:40.73,0:21:44.92,Default,,0000,0000,0000,,Then as you learn language it gets way\Nslower, and only in middle-school it gets
Dialogue: 0,0:21:44.92,0:21:47.95,Default,,0000,0000,0000,,gets back to the adult level turn-taking.
Dialogue: 0,0:21:48.83,0:21:52.87,Default,,0000,0000,0000,,So, what's the point, right? What does all\Nof this mean for language learning?
Dialogue: 0,0:21:54.08,0:21:58.78,Default,,0000,0000,0000,,So for this, let's do another\Ntime-travel, back to 1768,
Dialogue: 0,0:21:58.78,0:22:07.33,Default,,0000,0000,0000,,and meet this French Jesuit monk :\NClaude-François Lizarde de Radonvilliers.
Dialogue: 0,0:22:07.72,0:22:11.61,Default,,0000,0000,0000,,And he wrote this book :\N(French) "About The Way To Learn Languages"
Dialogue: 0,0:22:12.41,0:22:17.67,Default,,0000,0000,0000,,back in the day, where he reflected on how\Nwe should teach people languages.
Dialogue: 0,0:22:18.24,0:22:23.67,Default,,0000,0000,0000,,And interestingly, this is basically\Nthe grandfather of the Assimil method,
Dialogue: 0,0:22:24.04,0:22:29.94,Default,,0000,0000,0000,,and also the Méthode\NToussaint-Langenscheidt, or, also called
Dialogue: 0,0:22:29.94,0:22:34.19,Default,,0000,0000,0000,,"interlinearversion". So this would be\Nthis sheet here.
Dialogue: 0,0:22:34.19,0:22:38.52,Default,,0000,0000,0000,,This was a way people learned languages\Nat the turn of the previous century.
Dialogue: 0,0:22:39.47,0:22:44.86,Default,,0000,0000,0000,,And you can see here that you have\Nthe spanish at the top, then some
Dialogue: 0,0:22:44.86,0:22:50.09,Default,,0000,0000,0000,,consideration in the middle,\Nand on the bottom the german.
Dialogue: 0,0:22:50.91,0:22:53.96,Default,,0000,0000,0000,,And this is kind of similar to what\NAssimil does, right?
Dialogue: 0,0:22:55.18,0:22:58.26,Default,,0000,0000,0000,,So this is really interesting,\Nbut that's not the point here.
Dialogue: 0,0:22:58.99,0:23:03.94,Default,,0000,0000,0000,,What he also did in this book is to compare\NL1 - so first language acquisition -
Dialogue: 0,0:23:03.94,0:23:09.24,Default,,0000,0000,0000,,with second language learning.\NAnd he noted that it seems that,
Dialogue: 0,0:23:09.24,0:23:15.74,Default,,0000,0000,0000,,for the first language, parents show their\Nchildren pictures, and enact words or
Dialogue: 0,0:23:15.74,0:23:20.09,Default,,0000,0000,0000,,concepts, and encourage the children to do\Nthe same, like this little boy does here.
Dialogue: 0,0:23:20.09,0:23:24.79,Default,,0000,0000,0000,,But for second language acquisition all we\Ndo is give people these vocabulary lists,
Dialogue: 0,0:23:24.79,0:23:27.29,Default,,0000,0000,0000,,and expect them to learn it\Njust like that.
Dialogue: 0,0:23:28.14,0:23:33.17,Default,,0000,0000,0000,,So this is kind of an interesting point,\Nand since then it has been shown,
Dialogue: 0,0:23:33.17,0:23:37.42,Default,,0000,0000,0000,,- and this is actually pretty robust,\NI was really surprised, that it has been
Dialogue: 0,0:23:37.42,0:23:42.80,Default,,0000,0000,0000,,a really robust finding, that gesture\Nenriched material enhances learning.
Dialogue: 0,0:23:43.43,0:23:51.88,Default,,0000,0000,0000,,So in this study, for example, people\Ntried to teach english-speaking people
Dialogue: 0,0:23:51.88,0:23:56.46,Default,,0000,0000,0000,,japanese words, and they had four\Ndifferent training conditions.
Dialogue: 0,0:23:56.98,0:24:03.06,Default,,0000,0000,0000,,So one, only speech, one repeated speech,\None speech plus incongruent gestures
Dialogue: 0,0:24:03.45,0:24:07.37,Default,,0000,0000,0000,,- so gestures that would not match -\Nand then, congruent gestures.
Dialogue: 0,0:24:08.10,0:24:10.26,Default,,0000,0000,0000,,And this is the interesting condition,\Nright?
Dialogue: 0,0:24:11.17,0:24:15.60,Default,,0000,0000,0000,,And then they tested the people after\Nencoding for three different times :
Dialogue: 0,0:24:15.60,0:24:18.43,Default,,0000,0000,0000,,after five minutes, after two days,\Nand after one week.
Dialogue: 0,0:24:19.32,0:24:24.90,Default,,0000,0000,0000,,And also they tested them on forced choice\Nso it's basically multiple choice,
Dialogue: 0,0:24:25.63,0:24:29.22,Default,,0000,0000,0000,,and free recall, so it's prompting \Nthe people with the word, and then they
Dialogue: 0,0:24:29.22,0:24:34.71,Default,,0000,0000,0000,,come up themselves with the answer.\NSo these numbers here are basically
Dialogue: 0,0:24:34.71,0:24:37.50,Default,,0000,0000,0000,,the proportion of correct \Nanswers that people give.
Dialogue: 0,0:24:37.98,0:24:40.35,Default,,0000,0000,0000,,And you can see that,\Nacross the board,
Dialogue: 0,0:24:40.35,0:24:43.48,Default,,0000,0000,0000,,the speech plus congruent\Ngesture condition is
Dialogue: 0,0:24:43.48,0:24:50.56,Default,,0000,0000,0000,,very superior compared to the other ones,\Nwhich is, yeah, which is interesting, and
Dialogue: 0,0:24:50.56,0:24:57.40,Default,,0000,0000,0000,,so, you would maybe think that the point\Nis "okay, so we just use videos instead
Dialogue: 0,0:24:57.40,0:24:59.84,Default,,0000,0000,0000,,of audios", right?\NAnd this is what I would call
Dialogue: 0,0:24:59.84,0:25:03.36,Default,,0000,0000,0000,,Multisensory enrichment.\NAnd there is nothing wrong with this,
Dialogue: 0,0:25:03.36,0:25:10.66,Default,,0000,0000,0000,,this is really useful, you have\Nthese YouTube channel like Easy Languages
Dialogue: 0,0:25:10.66,0:25:15.08,Default,,0000,0000,0000,,- I'm not sponsored by the way (laugh) -\Nwhere you have conversations with
Dialogue: 0,0:25:15.08,0:25:18.70,Default,,0000,0000,0000,,real people that from time to time\Nmake gestures, and you get the full
Dialogue: 0,0:25:18.70,0:25:23.30,Default,,0000,0000,0000,,conversation thing, right?\NAnd you have these one-on-one videos,
Dialogue: 0,0:25:23.30,0:25:30.42,Default,,0000,0000,0000,,like this one from Mandarin Corner.\NWhere they are also a lot of gestures
Dialogue: 0,0:25:30.42,0:25:34.10,Default,,0000,0000,0000,,involved, so the host, Eileen, really\Ntries to integrate a lot of gestures.
Dialogue: 0,0:25:35.15,0:25:38.55,Default,,0000,0000,0000,,But this is actually not the point\N- I mean, this is cool but I think
Dialogue: 0,0:25:38.55,0:25:43.61,Default,,0000,0000,0000,,you already do that.\NThe point is way deeper.
Dialogue: 0,0:25:43.98,0:25:49.81,Default,,0000,0000,0000,,So, there's another thing going on,\Nnot only when you watch gestures,
Dialogue: 0,0:25:49.81,0:25:54.06,Default,,0000,0000,0000,,but when you enact them.\NThis is called the enactment effect.
Dialogue: 0,0:25:55.11,0:25:58.01,Default,,0000,0000,0000,,This was actually coined in 1980,\Nby two germans.
Dialogue: 0,0:25:58.32,0:26:03.70,Default,,0000,0000,0000,,They called it first the "Tu-Effekt",\Nwhich translates literally to "Do-Effect".
Dialogue: 0,0:26:04.89,0:26:09.10,Default,,0000,0000,0000,,And you can see why people chose to call\Nit the enactment effect, because it sounds
Dialogue: 0,0:26:09.10,0:26:13.15,Default,,0000,0000,0000,,way more fancy (laugh) but I really like\Nthe "tu-effekt", it sounds funny.
Dialogue: 0,0:26:14.07,0:26:19.52,Default,,0000,0000,0000,,Anyways, the point is that action words\Nor phrases, this is what they - Engelkamp
Dialogue: 0,0:26:19.52,0:26:23.29,Default,,0000,0000,0000,,et Krumnacker - noticed : that action\Nwords and phrases are remembered better
Dialogue: 0,0:26:23.29,0:26:27.24,Default,,0000,0000,0000,,if they're acted out,\Nor accompanied by gestures.
Dialogue: 0,0:26:27.54,0:26:33.32,Default,,0000,0000,0000,,So if you would learn the phrase\N"chopping garlic", then if you enact it
Dialogue: 0,0:26:33.32,0:26:36.62,Default,,0000,0000,0000,,actually while learning it,\Nyou will retain it way better.
Dialogue: 0,0:26:37.28,0:26:40.25,Default,,0000,0000,0000,,And this effect is also\Nreally well replicated,
Dialogue: 0,0:26:40.25,0:26:42.48,Default,,0000,0000,0000,,and this was also\Nreally surprising to me,
Dialogue: 0,0:26:42.48,0:26:48.90,Default,,0000,0000,0000,,because it is virtually not at all\Ntranslated into actual teaching.
Dialogue: 0,0:26:49.19,0:26:53.15,Default,,0000,0000,0000,,Nobody does this, nobody tells\Nthe students to enact things, right?
Dialogue: 0,0:26:53.32,0:26:55.72,Default,,0000,0000,0000,,Enact words, enact anything.
Dialogue: 0,0:26:55.72,0:27:01.34,Default,,0000,0000,0000,,And it has been well replicated\Nacross tasks, across materials and also
Dialogue: 0,0:27:01.34,0:27:05.42,Default,,0000,0000,0000,,across populations : across children,\Nadults, even clinical populations :
Dialogue: 0,0:27:05.42,0:27:08.45,Default,,0000,0000,0000,,People with Alzheimer, people recovering\Nfrom stroke...
Dialogue: 0,0:27:08.45,0:27:13.05,Default,,0000,0000,0000,,Somehow people made them\Nlearn words and then act the words,
Dialogue: 0,0:27:13.05,0:27:18.24,Default,,0000,0000,0000,,and it worked better\Nthan without enactment.
Dialogue: 0,0:27:19.16,0:27:23.47,Default,,0000,0000,0000,,And also, this is not only true for\Naction words and concrete words,
Dialogue: 0,0:27:23.47,0:27:26.46,Default,,0000,0000,0000,,but also abstract words.\NAnything you can somehow find
Dialogue: 0,0:27:26.46,0:27:31.68,Default,,0000,0000,0000,,a representation - with gestures - for,\Nyou can use this enactment effect.
Dialogue: 0,0:27:32.58,0:27:36.34,Default,,0000,0000,0000,,And this is way more powerful than\Nmultysensory enrichment,
Dialogue: 0,0:27:36.34,0:27:39.94,Default,,0000,0000,0000,,and we can call this \N"sensorimotor enrichment",
Dialogue: 0,0:27:39.96,0:27:42.50,Default,,0000,0000,0000,,because you use\Nyour senses and your "motor".
Dialogue: 0,0:27:44.40,0:27:50.57,Default,,0000,0000,0000,,So, this is also, this ties in with\Nanother really interesting development
Dialogue: 0,0:27:50.57,0:27:53.02,Default,,0000,0000,0000,,in neurosciences, called\N"embodied cognition".
Dialogue: 0,0:27:53.57,0:27:58.28,Default,,0000,0000,0000,,Basically this is the idea that many\Nfeatures of cognition - and these might be
Dialogue: 0,0:27:58.28,0:28:01.94,Default,,0000,0000,0000,,concepts, categories, reasoning or\Njudgement - are shaped by aspects
Dialogue: 0,0:28:01.94,0:28:06.40,Default,,0000,0000,0000,,of the body. And this would be\Nthe motor system - so how we move -
Dialogue: 0,0:28:06.40,0:28:11.27,Default,,0000,0000,0000,,the perceptual system - what we see, what\Nwe feel, what we hear, and so on.
Dialogue: 0,0:28:11.27,0:28:14.52,Default,,0000,0000,0000,,And also bodily interactions with\Nthe environment.
Dialogue: 0,0:28:15.34,0:28:19.34,Default,,0000,0000,0000,,And you might see where I get with this,\Nif you think about concepts and categories
Dialogue: 0,0:28:19.80,0:28:23.55,Default,,0000,0000,0000,,What are words, if not concepts and\Ncategories, right?
Dialogue: 0,0:28:23.91,0:28:27.58,Default,,0000,0000,0000,,So we might ask the question, "how are\Nwords represented in the brain?"
Dialogue: 0,0:28:28.51,0:28:31.05,Default,,0000,0000,0000,,And there is this really funny study,
Dialogue: 0,0:28:31.60,0:28:37.76,Default,,0000,0000,0000,,they showed people words that had strong\Nolfactory associations, which means
Dialogue: 0,0:28:37.81,0:28:42.40,Default,,0000,0000,0000,,they either stink really hard, or\Nthey smell really well.
Dialogue: 0,0:28:43.11,0:28:49.10,Default,,0000,0000,0000,,And, in case you're looking for some\Ninspiration for your spanish poem,
Dialogue: 0,0:28:49.10,0:28:52.69,Default,,0000,0000,0000,,you can go (laugh) to this publication and\Nsearch through the list of words.
Dialogue: 0,0:28:52.69,0:28:57.25,Default,,0000,0000,0000,,This is also a small - this is only\Na small sample, there are tons of
Dialogue: 0,0:28:57.25,0:29:02.20,Default,,0000,0000,0000,,really strong smelling words\Nin the study and, yeah.
Dialogue: 0,0:29:02.53,0:29:06.48,Default,,0000,0000,0000,,So basically, what they found is that\Nwhen they showed people these words,
Dialogue: 0,0:29:06.48,0:29:11.80,Default,,0000,0000,0000,,as compared to words that did not smell\Nthat much, some regions in the brain
Dialogue: 0,0:29:11.80,0:29:16.42,Default,,0000,0000,0000,,that are associated with olfaction,\Nso, with smelling, lighted up.
Dialogue: 0,0:29:17.70,0:29:24.08,Default,,0000,0000,0000,,And this kind of has been\Nextended as well to actions.
Dialogue: 0,0:29:24.36,0:29:29.34,Default,,0000,0000,0000,,So, on the left here, these are\Nall the regions that light up
Dialogue: 0,0:29:29.34,0:29:32.47,Default,,0000,0000,0000,,when you move your foot\Nwhen you move your fingers,
Dialogue: 0,0:29:32.47,0:29:36.09,Default,,0000,0000,0000,,or when you move your tongue.\NAnd on the right here, these are
Dialogue: 0,0:29:36.09,0:29:42.21,Default,,0000,0000,0000,,the regions that light up when you read\Nleg-related words, arm-related words,
Dialogue: 0,0:29:42.21,0:29:46.21,Default,,0000,0000,0000,,or face-related words.\NAnd you can see that, this more or less,
Dialogue: 0,0:29:46.21,0:29:50.76,Default,,0000,0000,0000,,this is more or less,\Nthese activations fit each other, right?
Dialogue: 0,0:29:51.10,0:29:55.65,Default,,0000,0000,0000,,So, in some way, leg-related words\Nare stored where you also move you leg,
Dialogue: 0,0:29:55.65,0:29:59.30,Default,,0000,0000,0000,,arm-related words are stored where\Nyou also move your arms, and so on.
Dialogue: 0,0:30:00.07,0:30:07.14,Default,,0000,0000,0000,,So, we can think about words actually\Nas functional networks, like this.
Dialogue: 0,0:30:08.39,0:30:12.87,Default,,0000,0000,0000,,And, note that words are\Nexperience-dependent functional networks.
Dialogue: 0,0:30:13.89,0:30:17.14,Default,,0000,0000,0000,,And experience is connected\Nto the body, right?
Dialogue: 0,0:30:18.25,0:30:25.50,Default,,0000,0000,0000,,So for exemple, you surely have, not only\Nread and heard the word "garlic",
Dialogue: 0,0:30:25.50,0:30:29.29,Default,,0000,0000,0000,,you also have smelled garlic, \Nyou touched garlic, you tasted garlic
Dialogue: 0,0:30:29.29,0:30:31.54,Default,,0000,0000,0000,,and, really important thing\Nyou chopped garlic.
Dialogue: 0,0:30:31.54,0:30:34.52,Default,,0000,0000,0000,,So when you read "garlic", \Nyou not only have
Dialogue: 0,0:30:34.52,0:30:38.64,Default,,0000,0000,0000,,the core language areas - in yellow here -\Nactivated, but also
Dialogue: 0,0:30:38.64,0:30:44.94,Default,,0000,0000,0000,,subcortical olfactory areas, and some\Ngustatory areas - so, for taste -
Dialogue: 0,0:30:44.94,0:30:49.30,Default,,0000,0000,0000,,action areas, right, and visual areas\Nas well.
Dialogue: 0,0:30:49.60,0:30:54.42,Default,,0000,0000,0000,,So, what I want to tell you here,\Nthink about this when you learn languages.
Dialogue: 0,0:30:54.82,0:31:02.43,Default,,0000,0000,0000,,Did you do the same for "knoblauch", for\Nexample - the german word for "garlic"?
Dialogue: 0,0:31:02.69,0:31:09.02,Default,,0000,0000,0000,,If you learn german, do you actually\Nget into this huge associated network?
Dialogue: 0,0:31:09.98,0:31:13.15,Default,,0000,0000,0000,,So, and that's the point basically,\Nwe are coming to the end,
Dialogue: 0,0:31:13.65,0:31:18.17,Default,,0000,0000,0000,,the point is that language is multimodal,\Nyou should use sensory-motor enrichment
Dialogue: 0,0:31:18.17,0:31:22.35,Default,,0000,0000,0000,,when learning languages, and thereby\Nembody your languages.
Dialogue: 0,0:31:23.78,0:31:27.41,Default,,0000,0000,0000,,And if you want to learn more about this,\Nand also for me to give credit,
Dialogue: 0,0:31:27.41,0:31:31.06,Default,,0000,0000,0000,,this is basically where I got most\Nof my input from.
Dialogue: 0,0:31:31.68,0:31:36.98,Default,,0000,0000,0000,,These are four big review articles that\Ndiscuss all of this stuff.
Dialogue: 0,0:31:37.72,0:31:43.73,Default,,0000,0000,0000,,So yeah, that's basically it.\NThanks for listening, and hoping for some cool questions.
Dialogue: 0,0:31:53.43,0:31:57.70,Default,,0000,0000,0000,,I would most certainly guess so.\NThis thing with the phone is also
Dialogue: 0,0:31:57.70,0:32:01.01,Default,,0000,0000,0000,,something that I have\Nexperienced quite a lot.
Dialogue: 0,0:32:03.93,0:32:09.37,Default,,0000,0000,0000,,I have lived in Chile for some time,\Nand I've got myself a chilean SIM-card
Dialogue: 0,0:32:09.86,0:32:14.74,Default,,0000,0000,0000,,And I didn't give this number to a lot\Nof people, but somehow, this number got
Dialogue: 0,0:32:14.74,0:32:18.48,Default,,0000,0000,0000,,to people that were, I don't know,\Ntrying to sell me something.
Dialogue: 0,0:32:19.38,0:32:22.16,Default,,0000,0000,0000,,And I would get a call from\Nsomebody, pick up the phone,
Dialogue: 0,0:32:22.16,0:32:24.20,Default,,0000,0000,0000,,and I would not understand\Na single word.
Dialogue: 0,0:32:24.20,0:32:28.11,Default,,0000,0000,0000,,Like, Chilean spanish is already\Nreally hard, and then it's completely
Dialogue: 0,0:32:28.11,0:32:31.34,Default,,0000,0000,0000,,out of context, I don't know what\Nthis person wants from me, and then
Dialogue: 0,0:32:31.34,0:32:34.39,Default,,0000,0000,0000,,it's just [gestures] and I'm like\N"sorry, I don't understand you"
Dialogue: 0,0:32:34.39,0:32:36.00,Default,,0000,0000,0000,,"I don't understand you",\N"I don't understand you",
Dialogue: 0,0:32:36.00,0:32:37.80,Default,,0000,0000,0000,,over and over again.
Dialogue: 0,0:32:37.81,0:32:40.100,Default,,0000,0000,0000,,And yeah, I mean, if you're on the phone,
Dialogue: 0,0:32:42.51,0:32:45.87,Default,,0000,0000,0000,,there's also a little bit of noise maybe,
Dialogue: 0,0:32:46.18,0:32:50.26,Default,,0000,0000,0000,,and I really have the feeling that\Nthat makes,
Dialogue: 0,0:32:50.26,0:32:52.62,Default,,0000,0000,0000,,especially in a foreign language,
Dialogue: 0,0:32:52.62,0:32:56.22,Default,,0000,0000,0000,,conversing that much harder, because\Nyou don't see the mouth movements,
Dialogue: 0,0:32:56.82,0:33:01.56,Default,,0000,0000,0000,,it's not that clear of a sound,\Nyou don't see anything else, and yeah.
Dialogue: 0,0:33:01.56,0:33:04.17,Default,,0000,0000,0000,,I would say so.\NCool question.
Dialogue: 0,0:33:17.91,0:33:21.11,Default,,0000,0000,0000,,I would guess so, I would guess so.\NLike, I mean,
Dialogue: 0,0:33:22.78,0:33:25.55,Default,,0000,0000,0000,,especially for autistic people\Nthere is a lot of research
Dialogue: 0,0:33:25.80,0:33:28.18,Default,,0000,0000,0000,,on language processing in general,
Dialogue: 0,0:33:29.58,0:33:34.10,Default,,0000,0000,0000,,but I don't know of any studies that are
Dialogue: 0,0:33:34.89,0:33:38.42,Default,,0000,0000,0000,,specifically for multimodal processing,
Dialogue: 0,0:33:38.42,0:33:40.58,Default,,0000,0000,0000,,but I think there are quite a few.
Dialogue: 0,0:33:41.24,0:33:43.84,Default,,0000,0000,0000,,Actually the experiment that I showed you,
Dialogue: 0,0:33:47.28,0:33:50.82,Default,,0000,0000,0000,,the one that I worked on, as well,\Nin the middle of the presentation,
Dialogue: 0,0:33:50.82,0:33:54.99,Default,,0000,0000,0000,,the entropy stuff, we also did this\Nwith schizophrenic patients,
Dialogue: 0,0:33:55.95,0:33:58.02,Default,,0000,0000,0000,,but we have not looked at the data yet.
Dialogue: 0,0:33:58.02,0:34:00.23,Default,,0000,0000,0000,,So once this publication is done then,
Dialogue: 0,0:34:01.47,0:34:05.36,Default,,0000,0000,0000,,somebody else will deal with
Dialogue: 0,0:34:05.36,0:34:07.79,Default,,0000,0000,0000,,the clinical data,\Nwith the schizophrenic people,
Dialogue: 0,0:34:08.15,0:34:10.68,Default,,0000,0000,0000,,and in general for schizophrenics there's,
Dialogue: 0,0:34:11.28,0:34:12.53,Default,,0000,0000,0000,,there's a lot of,
Dialogue: 0,0:34:14.53,0:34:19.42,Default,,0000,0000,0000,,like, language-related abnormalities,
Dialogue: 0,0:34:20.51,0:34:22.83,Default,,0000,0000,0000,,and I think for autistic people as well.
Dialogue: 0,0:34:23.37,0:34:24.81,Default,,0000,0000,0000,,I'm not sure about ADHD
Dialogue: 0,0:34:28.36,0:34:31.35,Default,,0000,0000,0000,,but yeah, it would be\Na really interesting thing to,
Dialogue: 0,0:34:33.18,0:34:36.29,Default,,0000,0000,0000,,to look at this for autistic people, for sure.
Dialogue: 0,0:34:36.29,0:34:40.27,Default,,0000,0000,0000,,And maybe people did this.\NYou can, maybe, you can look it up.
Dialogue: 0,0:34:41.30,0:34:45.94,Default,,0000,0000,0000,,I don't have anything in my head right now\Nbut yeah.
Dialogue: 0,0:34:47.19,0:34:49.70,Default,,0000,0000,0000,,There should be something.
Dialogue: 0,0:34:59.25,0:35:04.16,Default,,0000,0000,0000,,I think one of the studies that\NI glanced over actually tried this.
Dialogue: 0,0:35:04.16,0:35:07.53,Default,,0000,0000,0000,,So they had some gestures that were nonsense
Dialogue: 0,0:35:08.30,0:35:11.47,Default,,0000,0000,0000,,I don't know if it was with abstract words\Nor with concrete words,
Dialogue: 0,0:35:11.85,0:35:13.96,Default,,0000,0000,0000,,but they used nonsense gestures,
Dialogue: 0,0:35:13.96,0:35:16.90,Default,,0000,0000,0000,,and they still had an effect,\Nbut it was smaller.
Dialogue: 0,0:35:17.89,0:35:20.65,Default,,0000,0000,0000,,So if you try to make this,
Dialogue: 0,0:35:21.10,0:35:22.92,Default,,0000,0000,0000,,to integrate this into your studies,
Dialogue: 0,0:35:23.55,0:35:28.98,Default,,0000,0000,0000,,I would suggest that you try to find\Nan enactment that is as sensical as it gets
Dialogue: 0,0:35:29.76,0:35:33.90,Default,,0000,0000,0000,,I mean, it's not that it's impossible\Nto get enactment for abstract words,
Dialogue: 0,0:35:34.71,0:35:38.38,Default,,0000,0000,0000,,you just have to be a little bit more\Ncreative, and I think the more creative
Dialogue: 0,0:35:38.38,0:35:42.93,Default,,0000,0000,0000,,you will be the more effective.\NLike, similar to mnemonics,
Dialogue: 0,0:35:42.93,0:35:48.24,Default,,0000,0000,0000,,like, the more crazy mnemonic is,\Nthe easier it is to remember.
Dialogue: 0,0:35:48.86,0:35:53.01,Default,,0000,0000,0000,,I could see the same effect with\Nenactments as well.
Dialogue: 0,0:35:53.72,0:35:58.42,Default,,0000,0000,0000,,And if you should use signs from\Nsign languages,
Dialogue: 0,0:35:59.24,0:36:03.56,Default,,0000,0000,0000,,I think if you want to that's a cool idea,\Nbecause then you automatically also learn
Dialogue: 0,0:36:03.56,0:36:07.48,Default,,0000,0000,0000,,the sign language.\NAnd when I was preparing this presentation
Dialogue: 0,0:36:07.48,0:36:11.32,Default,,0000,0000,0000,,I actually thought about this.\NLike why do we learn languages?
Dialogue: 0,0:36:13.04,0:36:17.32,Default,,0000,0000,0000,,Like if I now start learning a language,\Nwhy do I not learn the sign language
Dialogue: 0,0:36:17.32,0:36:19.31,Default,,0000,0000,0000,,that goes with it, right?
Dialogue: 0,0:36:19.31,0:36:22.55,Default,,0000,0000,0000,,I think it would make things easier,\Nbecause you actually have
Dialogue: 0,0:36:22.55,0:36:24.23,Default,,0000,0000,0000,,the enactment ready for you,
Dialogue: 0,0:36:25.04,0:36:27.91,Default,,0000,0000,0000,,and it's just a cool thing, right?
Dialogue: 0,0:36:27.91,0:36:31.96,Default,,0000,0000,0000,,You can talk with so many more people.
Dialogue: 0,0:36:33.30,0:36:40.17,Default,,0000,0000,0000,,And also, I think in general, people\Nshould learn sign languages regardless
Dialogue: 0,0:36:41.95,0:36:47.42,Default,,0000,0000,0000,,This became really clear to me actually\Nat the Polyglot Gathering in 2019.
Dialogue: 0,0:36:47.42,0:36:53.21,Default,,0000,0000,0000,,In Bratislava we were on some ship where\Nthere was a party.
Dialogue: 0,0:36:55.07,0:36:58.52,Default,,0000,0000,0000,,There was loud music, then there were some\Npeople who knew sign language
Dialogue: 0,0:36:58.52,0:37:00.47,Default,,0000,0000,0000,,maybe they are listening right now.
Dialogue: 0,0:37:01.36,0:37:04.41,Default,,0000,0000,0000,,So they just started, they were like,\Non the dance floor,
Dialogue: 0,0:37:04.42,0:37:07.81,Default,,0000,0000,0000,,and instead of screaming into\Neach other's ears, like people usually do,
Dialogue: 0,0:37:07.81,0:37:10.48,Default,,0000,0000,0000,,they just started to sign, \Nand it was so smooth, like
Dialogue: 0,0:37:11.37,0:37:15.15,Default,,0000,0000,0000,,why should we communicate with sound\Nwhen we can do it with gestures. Right?
Dialogue: 0,0:37:15.92,0:37:19.26,Default,,0000,0000,0000,,I think for many situations\Nit would be a lot easier.
Dialogue: 0,0:37:20.47,0:37:23.81,Default,,0000,0000,0000,,So yeah, if you can use the signs\Nof your target language,
Dialogue: 0,0:37:24.73,0:37:26.70,Default,,0000,0000,0000,,I think that's a cool idea.
Dialogue: 0,0:37:31.38,0:37:34.98,Default,,0000,0000,0000,,Yeah, it does sound like that.\NIndeed, indeed.
Dialogue: 0,0:37:37.20,0:37:39.76,Default,,0000,0000,0000,,Yeah, I can totally see that.
Dialogue: 0,0:37:50.58,0:37:55.07,Default,,0000,0000,0000,,There is a study, that I came across while\Nreasearching, but I didn't look into it.
Dialogue: 0,0:37:55.92,0:38:00.47,Default,,0000,0000,0000,,If you want the reference, you can\Nreach out to me somehow and I can see
Dialogue: 0,0:38:00.47,0:38:03.01,Default,,0000,0000,0000,,if I can find it and send it to you.
Dialogue: 0,0:38:05.29,0:38:07.100,Default,,0000,0000,0000,,I didn't look deeply into it,
Dialogue: 0,0:38:09.51,0:38:13.75,Default,,0000,0000,0000,,and I think I wouldn't find it now quickly.
Dialogue: 0,0:38:14.98,0:38:19.67,Default,,0000,0000,0000,,So again, there's something done, but\NI can't recall it from my head right now.
Dialogue: 0,0:38:23.97,0:38:27.02,Default,,0000,0000,0000,,Well so, there's two things
Dialogue: 0,0:38:27.70,0:38:30.04,Default,,0000,0000,0000,,maybe more things but let's start with two
Dialogue: 0,0:38:30.26,0:38:32.84,Default,,0000,0000,0000,,So first of all when you learn a new word,
Dialogue: 0,0:38:33.66,0:38:37.35,Default,,0000,0000,0000,,try to get the whole picture of the word.
Dialogue: 0,0:38:38.03,0:38:44.47,Default,,0000,0000,0000,,Like the garlic example, try to imagine\Nhow it smells, how it feels,
Dialogue: 0,0:38:44.47,0:38:46.91,Default,,0000,0000,0000,,how you chop it, try to enact it.
Dialogue: 0,0:38:47.80,0:38:54.33,Default,,0000,0000,0000,,Take a moment, and really try to activate\Nthe whole functional network of this word.
Dialogue: 0,0:38:55.13,0:39:02.47,Default,,0000,0000,0000,,And then the other thing was also just\Nto use input with video,
Dialogue: 0,0:39:02.47,0:39:04.47,Default,,0000,0000,0000,,if you're learning with some input.
Dialogue: 0,0:39:05.12,0:39:08.58,Default,,0000,0000,0000,,Look up if you find some interesting\Nchannels on YouTube or something.
Dialogue: 0,0:39:09.12,0:39:12.84,Default,,0000,0000,0000,,And then also, that might have not\Nbeen clear from my presentation,
Dialogue: 0,0:39:12.84,0:39:16.41,Default,,0000,0000,0000,,if you are conversing with people, use signs.
Dialogue: 0,0:39:20.16,0:39:25.09,Default,,0000,0000,0000,,I don't know if people do this naturally\Nin general, I think I kind of do it,
Dialogue: 0,0:39:25.09,0:39:29.39,Default,,0000,0000,0000,,if I talk in my target language,\Nand I'm not sure about a word
Dialogue: 0,0:39:29.39,0:39:32.88,Default,,0000,0000,0000,,I will try to make sure with my hands\Nthat somehow,
Dialogue: 0,0:39:32.88,0:39:36.99,Default,,0000,0000,0000,,something gets, like, to the other person.
Dialogue: 0,0:39:37.34,0:39:40.13,Default,,0000,0000,0000,,So what I'm going for is that the other\Nperson recognizes
Dialogue: 0,0:39:40.13,0:39:43.00,Default,,0000,0000,0000,,what I'm trying to say, and then\Ngives me the word, right.
Dialogue: 0,0:39:43.58,0:39:46.01,Default,,0000,0000,0000,,Like in the example with the glass of water,
Dialogue: 0,0:39:46.51,0:39:48.82,Default,,0000,0000,0000,,if I don't know "drink" in some language\NI would,
Dialogue: 0,0:39:48.82,0:39:52.20,Default,,0000,0000,0000,,I would try to [MIMES]\NRight? "I want to [MIMES]" Right?
Dialogue: 0,0:39:52.24,0:39:57.78,Default,,0000,0000,0000,,And then have the other person gives me\Nthe word, because I'm actively reducing
Dialogue: 0,0:39:57.78,0:40:01.13,Default,,0000,0000,0000,,the uncertainty that the other perso has,\Nthat is trying to predict
Dialogue: 0,0:40:01.13,0:40:04.04,Default,,0000,0000,0000,,what I'm going to say,\Nby giving gestures. Right?
Dialogue: 0,0:40:05.03,0:40:08.91,Default,,0000,0000,0000,,So that would be my three\Npractical implications for now.
Dialogue: 0,0:40:18.54,0:40:21.12,Default,,0000,0000,0000,,Yeah so this is something that\NI don't know.
Dialogue: 0,0:40:21.90,0:40:27.31,Default,,0000,0000,0000,,Again I think it's worth trying to do this\Nwith the sign language.
Dialogue: 0,0:40:27.31,0:40:31.78,Default,,0000,0000,0000,,I mean, there's a system of really...
Dialogue: 0,0:40:34.90,0:40:39.08,Default,,0000,0000,0000,,There's a system of really fitting\Ngestures that people use,
Dialogue: 0,0:40:39.95,0:40:47.10,Default,,0000,0000,0000,,and it might actually be a good idea\Nto try this out, to use sign language
Dialogue: 0,0:40:47.10,0:40:49.97,Default,,0000,0000,0000,,as you're learning the actual language,
Dialogue: 0,0:40:51.84,0:40:54.54,Default,,0000,0000,0000,,to get this enactment working.
Dialogue: 0,0:40:56.03,0:40:59.95,Default,,0000,0000,0000,,Might be more effective than\Nmaking up your own gestures.
Dialogue: 0,0:41:01.20,0:41:06.12,Default,,0000,0000,0000,,I mean if you make up your own gestures\Nyou have the advantage that
Dialogue: 0,0:41:09.58,0:41:12.11,Default,,0000,0000,0000,,during the process of\Ncoming up with the gesture,
Dialogue: 0,0:41:12.11,0:41:15.38,Default,,0000,0000,0000,,you are engaging your brain\Nin a specific way
Dialogue: 0,0:41:15.42,0:41:18.75,Default,,0000,0000,0000,,that's not there if you just get\Nthe gesture from somebody.
Dialogue: 0,0:41:19.62,0:41:21.67,Default,,0000,0000,0000,,So there might be an advantage there,
Dialogue: 0,0:41:21.67,0:41:27.75,Default,,0000,0000,0000,,but the other advantage is of course\Ntime that you can save,
Dialogue: 0,0:41:27.75,0:41:32.91,Default,,0000,0000,0000,,and the ability to communicate\Nwith people that can't hear
Dialogue: 0,0:41:34.33,0:41:38.15,Default,,0000,0000,0000,,So yeah, I think that's open for\Nexploration, for sure.
Dialogue: 0,0:41:44.22,0:41:46.34,Default,,0000,0000,0000,,Well you see it in sign language, right?
Dialogue: 0,0:41:47.94,0:41:51.37,Default,,0000,0000,0000,,People that sign don't really speak.
Dialogue: 0,0:41:52.39,0:41:55.02,Default,,0000,0000,0000,,And they get along pretty nicely.
Dialogue: 0,0:41:56.20,0:42:01.00,Default,,0000,0000,0000,,Another question would be if all of society\Nas a whole can do without verbal.
Dialogue: 0,0:42:01.96,0:42:06.45,Default,,0000,0000,0000,,That's another question, but I think\Nyou can restructure society
Dialogue: 0,0:42:06.45,0:42:10.09,Default,,0000,0000,0000,,in a way that everybody can communicate\Nwith gestures, for sure.
Dialogue: 0,0:42:11.68,0:42:16.89,Default,,0000,0000,0000,,And according to some people, it was like\Nthat before speech developped.
Dialogue: 0,0:42:26.32,0:42:28.82,Default,,0000,0000,0000,,So yeah, there has been some research,\Nnot much.
Dialogue: 0,0:42:29.51,0:42:34.45,Default,,0000,0000,0000,,If you are interested in this, make sure\Nto check out my presentation on this topic
Dialogue: 0,0:42:34.45,0:42:39.76,Default,,0000,0000,0000,,from last year's Gathering,\Nand also from last year's conference.
Dialogue: 0,0:42:40.71,0:42:44.51,Default,,0000,0000,0000,,The conference one is not up on YouTube\Nalready, but the Gathering one,
Dialogue: 0,0:42:44.51,0:42:48.93,Default,,0000,0000,0000,,and there's, in the end, I show some...
Dialogue: 0,0:42:51.71,0:42:55.62,Default,,0000,0000,0000,,I show a study that was done on\Npolyglots and hyper polyglots
Dialogue: 0,0:42:56.74,0:42:58.67,Default,,0000,0000,0000,,actually only hyper polyglots I think.
Dialogue: 0,0:43:00.29,0:43:04.84,Default,,0000,0000,0000,,And so they put people in a FMRI scanner,\Nand just gave them language material.
Dialogue: 0,0:43:06.12,0:43:08.91,Default,,0000,0000,0000,,And what they found is that\Nthe language network
Dialogue: 0,0:43:08.91,0:43:15.69,Default,,0000,0000,0000,,was less active than for monolinguals.
Dialogue: 0,0:43:16.35,0:43:19.79,Default,,0000,0000,0000,,So if you listen to something\Nthere's some areas
Dialogue: 0,0:43:19.79,0:43:22.08,Default,,0000,0000,0000,,on the left side of your brain\Nthat light up :
Dialogue: 0,0:43:22.08,0:43:26.81,Default,,0000,0000,0000,,you have some typical areas,\Nlike Broca's area, Wernicke's area,
Dialogue: 0,0:43:27.01,0:43:28.58,Default,,0000,0000,0000,,and some other ones.
Dialogue: 0,0:43:29.00,0:43:34.16,Default,,0000,0000,0000,,And they found that, for polyglots\Nthis "lighting up" is less,
Dialogue: 0,0:43:34.16,0:43:38.82,Default,,0000,0000,0000,,and the interpretation was that\Nthe polyglots' language network,
Dialogue: 0,0:43:38.82,0:43:42.91,Default,,0000,0000,0000,,through extensive practice, has become\Nmore and more efficient
Dialogue: 0,0:43:42.91,0:43:47.58,Default,,0000,0000,0000,,at dealing with language.\NAnd therefore it needs less activation.
Dialogue: 0,0:43:47.98,0:43:53.22,Default,,0000,0000,0000,,So this is one thing that\Nyou observe quite often,
Dialogue: 0,0:43:59.29,0:44:01.91,Default,,0000,0000,0000,,when there's some process that\Nyou get really good at,
Dialogue: 0,0:44:02.67,0:44:05.100,Default,,0000,0000,0000,,in your brain the activity\Nthat you see goes down,
Dialogue: 0,0:44:05.100,0:44:08.33,Default,,0000,0000,0000,,because the network gets more efficient.
Dialogue: 0,0:44:08.67,0:44:11.35,Default,,0000,0000,0000,,So that's why this paper was aptly titled
Dialogue: 0,0:44:11.35,0:44:14.96,Default,,0000,0000,0000,,"The Small And Efficient Network\NOf Polyglots And Hyperpolyglots".
Dialogue: 0,0:44:15.63,0:44:18.04,Default,,0000,0000,0000,,And they, you can also look this up as well,
Dialogue: 0,0:44:18.56,0:44:22.38,Default,,0000,0000,0000,,they also made them listen to\Ndifferent languages,
Dialogue: 0,0:44:23.07,0:44:27.53,Default,,0000,0000,0000,,and there, the better known\Nthe foreign language was
Dialogue: 0,0:44:28.10,0:44:32.39,Default,,0000,0000,0000,,so the first experiment was completed\Nin english, their mother tongue,
Dialogue: 0,0:44:32.68,0:44:37.11,Default,,0000,0000,0000,,and then the second experiment they used\Ntheir target languages like,
Dialogue: 0,0:44:37.11,0:44:39.85,Default,,0000,0000,0000,,the second best language, third best\Nlanguage and so on.
Dialogue: 0,0:44:40.57,0:44:44.81,Default,,0000,0000,0000,,And there, the lesser known a language,\Nthe less active the language network,
Dialogue: 0,0:44:45.17,0:44:49.04,Default,,0000,0000,0000,,and the better known, the more active.\NSo you have kind of the opposite effect.
Dialogue: 0,0:44:49.29,0:44:52.47,Default,,0000,0000,0000,,And they interpreted this as reflecting\Nthat the more you know
Dialogue: 0,0:44:52.47,0:44:54.64,Default,,0000,0000,0000,,in a target language,\Nin a foreign language,
Dialogue: 0,0:44:54.77,0:45:00.34,Default,,0000,0000,0000,,the more of the language network\Ngets recruited, the more context you have.
Dialogue: 0,0:45:01.71,0:45:05.30,Default,,0000,0000,0000,,So you have this effect of getting really\Nefficient for your mother tongue,
Dialogue: 0,0:45:05.81,0:45:09.84,Default,,0000,0000,0000,,and getting more of the whole message for,
Dialogue: 0,0:45:11.58,0:45:13.46,Default,,0000,0000,0000,,the better you know a foreign language.
Dialogue: 0,0:45:16.01,0:45:18.01,Default,,0000,0000,0000,,So this was the last question.\NAlright
Dialogue: 0,0:45:18.77,0:45:23.76,Default,,0000,0000,0000,,Thanks for listening, thanks to\Nthe organizers for organizing this,
Dialogue: 0,0:45:24.11,0:45:27.39,Default,,0000,0000,0000,,the streaming works really well,\NI'm really impressed
Dialogue: 0,0:45:27.91,0:45:29.08,Default,,0000,0000,0000,,Thanks guys!