Robots with "soul"
-
0:01 - 0:03My job is to design, build and study
-
0:03 - 0:05robots that communicate with people.
-
0:05 - 0:07But this story doesn't start with robotics at all,
-
0:07 - 0:09it starts with animation.
-
0:09 - 0:11When I first saw Pixar's "Luxo Jr.,"
-
0:11 - 0:13I was amazed by how much emotion
-
0:13 - 0:15they could put into something
-
0:15 - 0:17as trivial as a desk lamp.
-
0:17 - 0:19I mean, look at them -- at the end of this movie,
-
0:19 - 0:22you actually feel something for two pieces of furniture.
-
0:22 - 0:24(Laughter)
-
0:24 - 0:26And I said, I have to learn how to do this.
-
0:26 - 0:29So I made a really bad career decision.
-
0:29 - 0:32And that's what my mom was like when I did it.
-
0:32 - 0:34(Laughter)
-
0:34 - 0:36I left a very cozy tech job in Israel
-
0:36 - 0:38at a nice software company and I moved to New York
-
0:38 - 0:39to study animation.
-
0:39 - 0:41And there I lived
-
0:41 - 0:44in a collapsing apartment building
in Harlem with roommates. -
0:44 - 0:45I'm not using this phrase metaphorically,
-
0:45 - 0:47the ceiling actually collapsed one day
-
0:47 - 0:48in our living room.
-
0:48 - 0:51Whenever they did those news stories
about building violations in New York, -
0:51 - 0:53they would put the report in front of our building.
-
0:53 - 0:57As kind of like a backdrop
to show how bad things are. -
0:57 - 0:59Anyway, during the day I went to school and at night
-
0:59 - 1:02I would sit and draw frame by frame
of pencil animation. -
1:02 - 1:05And I learned two surprising lessons --
-
1:05 - 1:07one of them was that
-
1:07 - 1:09when you want to arouse emotions,
-
1:09 - 1:11it doesn't matter so much how something looks,
-
1:11 - 1:13it's all in the motion -- it's in the timing
-
1:13 - 1:15of how the thing moves.
-
1:15 - 1:18And the second, was something
one of our teachers told us. -
1:18 - 1:20He actually did the weasel in Ice Age.
-
1:20 - 1:22And he said:
-
1:22 - 1:25"As an animator you are not
a director, you're an actor." -
1:25 - 1:28So, if you want to find the
right motion for a character, -
1:28 - 1:30don't think about it, go use your body to find it --
-
1:30 - 1:32stand in front of a mirror, act it out
-
1:32 - 1:34in front of a camera -- whatever you need.
-
1:34 - 1:36And then put it back in your character.
-
1:36 - 1:39A year later I found myself at MIT
-
1:39 - 1:41in the robotic life group, it was one of the first groups
-
1:41 - 1:43researching the relationships
between humans and robots. -
1:43 - 1:45And I still had this dream to make
-
1:45 - 1:48an actual, physical Luxo Jr. lamp.
-
1:48 - 1:50But I found that robots didn't move at all
-
1:50 - 1:51in this engaging way that I was used to
-
1:51 - 1:53for my animation studies.
-
1:53 - 1:55Instead, they were all --
-
1:55 - 1:57how should I put it, they were all kind of robotic.
-
1:57 - 1:59(Laughter)
-
1:59 - 2:03And I thought, what if I took whatever
I learned in animation school, -
2:03 - 2:05and used that to design my robotic desk lamp.
-
2:05 - 2:08So I went and designed frame by frame
-
2:08 - 2:09to try to make this robot
-
2:09 - 2:12as graceful and engaging as possible.
-
2:12 - 2:14And here when you see the robot interacting with me
-
2:14 - 2:16on a desktop.
-
2:16 - 2:18And I'm actually redesigning the robot so,
-
2:18 - 2:20unbeknownst to itself,
-
2:20 - 2:22it's kind of digging its own grave by helping me.
-
2:22 - 2:24(Laughter)
-
2:24 - 2:26I wanted it to be less of a mechanical structure
-
2:26 - 2:28giving me light,
-
2:28 - 2:31and more of a helpful, kind of quiet apprentice
-
2:31 - 2:34that's always there when you need
it and doesn't really interfere. -
2:34 - 2:36And when, for example, I'm looking for a battery
-
2:36 - 2:37that I can't find,
-
2:37 - 2:42in a subtle way, it will show me where the battery is.
-
2:42 - 2:44So you can see my confusion here.
-
2:44 - 2:49I'm not an actor.
-
2:49 - 2:50And I want you to notice how the same
-
2:50 - 2:52mechanical structure can at one point,
-
2:52 - 2:55just by the way it moves seem gentle and caring --
-
2:55 - 2:58and in the other case, seem
violent and confrontational. -
2:58 - 3:02And it's the same structure,
just the motion is different. -
3:07 - 3:13Actor: "You want to know something?
Well, you want to know something? -
3:13 - 3:14He was already dead!
-
3:14 - 3:18Just laying there, eyes glazed over!"
-
3:18 - 3:19(Laughter)
-
3:19 - 3:23But, moving in graceful ways is just one
building block of this whole structure -
3:23 - 3:24called human-robot interaction.
-
3:24 - 3:26I was at the time doing my Ph.D.,
-
3:26 - 3:28I was working on human robot teamwork;
-
3:28 - 3:30teams of humans and robots working together.
-
3:30 - 3:31I was studying the engineering,
-
3:31 - 3:34the psychology, the philosophy of teamwork.
-
3:34 - 3:36And at the same time I found myself
-
3:36 - 3:37in my own kind of teamwork situation
-
3:37 - 3:40with a good friend of mine who is actually here.
-
3:40 - 3:42And in that situation we can easily imagine robots
-
3:42 - 3:44in the near future being there with us.
-
3:44 - 3:46It was after a Passover seder.
-
3:46 - 3:48We were folding up a lot of folding chairs,
-
3:48 - 3:51and I was amazed at how quickly
we found our own rhythm. -
3:51 - 3:53Everybody did their own part.
-
3:53 - 3:54We didn't have to divide our tasks.
-
3:54 - 3:56We didn't have to communicate verbally about this.
-
3:56 - 3:58It all just happened.
-
3:58 - 3:59And I thought,
-
3:59 - 4:01humans and robots don't look at all like this.
-
4:01 - 4:02When humans and robots interact,
-
4:02 - 4:03it's much more like a chess game.
-
4:03 - 4:05The human does a thing,
-
4:05 - 4:07the robot analyzes whatever the human did,
-
4:07 - 4:08then the robot decides what to do next,
-
4:08 - 4:09plans it and does it.
-
4:09 - 4:11And then the human waits, until it's their turn again.
-
4:11 - 4:13So, it's much more like a chess game
-
4:13 - 4:15and that makes sense because chess is great
-
4:15 - 4:16for mathematicians and computer scientists.
-
4:16 - 4:19It's all about information analysis,
-
4:19 - 4:22decision making and planning.
-
4:22 - 4:25But I wanted my robot to be less of a chess player,
-
4:25 - 4:27and more like a doer
-
4:27 - 4:29that just clicks and works together.
-
4:29 - 4:33So I made my second horrible career choice:
-
4:33 - 4:35I decided to study acting for a semester.
-
4:35 - 4:38I took off from a Ph.D. I went to acting classes.
-
4:38 - 4:41I actually participated in a play,
-
4:41 - 4:43I hope theres no video of that around still.
-
4:43 - 4:46And I got every book I could find about acting,
-
4:46 - 4:48including one from the 19th century
-
4:48 - 4:49that I got from the library.
-
4:49 - 4:52And I was really amazed because my
name was the second name on the list -- -
4:52 - 4:55the previous name was in 1889. (Laughter)
-
4:55 - 4:57And this book was kind of waiting for 100 years
-
4:57 - 5:00to be rediscovered for robotics.
-
5:00 - 5:02And this book shows actors
-
5:02 - 5:04how to move every muscle in the body
-
5:04 - 5:07to match every kind of emotion
that they want to express. -
5:07 - 5:09But the real revelation was
-
5:09 - 5:10when I learned about method acting.
-
5:10 - 5:12It became very popular in the 20th century.
-
5:12 - 5:15And method acting said, you don't have
to plan every muscle in your body. -
5:15 - 5:18Instead you have to use your body
to find the right movement. -
5:18 - 5:20You have to use your sense memory
-
5:20 - 5:22to reconstruct the emotions and kind of
-
5:22 - 5:24think with your body to find the right expression.
-
5:24 - 5:26Improvise, play off yor scene partner.
-
5:26 - 5:30And this came at the same time
as I was reading about this trend -
5:30 - 5:33in cognitive psychology called embodied cognition.
-
5:33 - 5:34Which also talks about the same ideas --
-
5:34 - 5:36We use our bodies to think,
-
5:36 - 5:38we don't just think with our brains
and use our bodies to move. -
5:38 - 5:41but our bodies feed back into our brain
-
5:41 - 5:43to generate the way that we behave.
-
5:43 - 5:44And it was like a lightning bolt.
-
5:44 - 5:46I went back to my office.
-
5:46 - 5:48I wrote this paper -- which I never really published
-
5:48 - 5:51called "Acting Lessons for Artificial Intelligence."
-
5:51 - 5:52And I even took another month
-
5:52 - 5:55to do what was then the first theater play
-
5:55 - 5:57with a human and a robot acting together.
-
5:57 - 6:00That's what you saw before with the actors.
-
6:00 - 6:02And I thought:
-
6:02 - 6:05How can we make an artificial intelligence model --
-
6:05 - 6:06computer, computational model --
-
6:06 - 6:09that will model some of these ideas of improvisation,
-
6:09 - 6:11of taking risks, of taking chances,
-
6:11 - 6:13even of making mistakes.
-
6:13 - 6:15Maybe it can make for better robotic teammates.
-
6:15 - 6:18So I worked for quite a long time on these models
-
6:18 - 6:20and I implemented them on a number of robots.
-
6:20 - 6:22Here you can see a very early example
-
6:22 - 6:26with the robots trying to use this
embodied artificial intelligence, -
6:26 - 6:29to try to match my movements
as closely as possible, -
6:29 - 6:30sort of like a game.
-
6:30 - 6:32Let's look at it.
-
6:36 - 6:40You can see when I psych it out, it gets fooled.
-
6:40 - 6:42And it's a little bit like what you might see actors do
-
6:42 - 6:44when they try to mirror each other
-
6:44 - 6:46to find the right synchrony between them.
-
6:46 - 6:48And then, I did another experiment,
-
6:48 - 6:52and I got people off the street
to use the robotic desk lamp, -
6:52 - 6:56and try out this idea of embodied
artificial intelligence. -
6:56 - 7:01So, I actually used two kinds
of brains for the same robot. -
7:01 - 7:02The robot is the same lamp that you saw,
-
7:02 - 7:04and I put in it two brains.
-
7:04 - 7:06For one half of the people,
-
7:06 - 7:08I put in a brain that's kind of the traditional,
-
7:08 - 7:10calculated robotic brain.
-
7:10 - 7:12It waits for its turn, it analyzes everything, it plans.
-
7:12 - 7:14Let's call it the calculated brain.
-
7:14 - 7:18The other got more the stage actor, risk taker brain.
-
7:18 - 7:20Let's call it the adventurous brain.
-
7:20 - 7:23It sometimes acts without knowing
everything it has to know. -
7:23 - 7:25It sometimes makes mistakes and corrects them.
-
7:25 - 7:27And I had them do this very tedious task
-
7:27 - 7:29that took almost 20 minutes
-
7:29 - 7:30and they had to work together.
-
7:30 - 7:33Somehow simulating like a factory job
-
7:33 - 7:35of repetitively doing the same thing.
-
7:35 - 7:37And what I found was that people actually loved
-
7:37 - 7:39the adventurous robot.
-
7:39 - 7:40And they thought it was more intelligent,
-
7:40 - 7:42more committed, a better member of the team,
-
7:42 - 7:44contributed to the success of the team more.
-
7:44 - 7:46They even called it 'he' and 'she,'
-
7:46 - 7:49whereas people with the calculated brain called it 'it.'
-
7:49 - 7:52And nobody ever called it 'he' or 'she'.
-
7:52 - 7:53When they talked about it after the task
-
7:53 - 7:55with the adventurous brain,
-
7:55 - 7:59they said, "By the end, we were good
friends and high-fived mentally." -
7:59 - 8:01Whatever that means.
-
8:01 - 8:04(Laughter) Sounds painful.
-
8:04 - 8:07Whereas the people with the calculated brain
-
8:07 - 8:09said it was just like a lazy apprentice.
-
8:09 - 8:12It only did what it was supposed
to do and nothing more. -
8:12 - 8:14Which is almost what people expect robots to do,
-
8:14 - 8:17so I was surprised that people
had higher expectations -
8:17 - 8:22of robots, than what anybody in robotics
thought robots should be doing. -
8:22 - 8:24And in a way, I thought, maybe it's time --
-
8:24 - 8:27just like method acting changed the way
-
8:27 - 8:28people thought about acting in the 19th century,
-
8:28 - 8:30from going from the very calculated,
-
8:30 - 8:32planned way of behaving,
-
8:32 - 8:35to a more intuitive, risk-taking,
embodied way of behaving. -
8:35 - 8:37Maybe it's time for robots
-
8:37 - 8:40to have the same kind of revolution.
-
8:40 - 8:41A few years later,
-
8:41 - 8:43I was at my next research job
at Georgia Tech in Atlanta, -
8:43 - 8:45and I was working in a group
-
8:45 - 8:46dealing with robotic musicians.
-
8:46 - 8:49And I thought, music, that's the perfect place
-
8:49 - 8:51to look at teamwork, coordination,
-
8:51 - 8:53timing, improvisation --
-
8:53 - 8:55and we just got this robot playing marimba.
-
8:55 - 8:57Marimba, for everybody who was like me,
-
8:57 - 9:00it was this huge, wooden xylophone.
-
9:00 - 9:03And, when I was looking at this,
-
9:03 - 9:06I looked at other works in
human-robot improvisation -- -
9:06 - 9:08yes, there are other works in
human-robot improvisation -- -
9:08 - 9:10and they were also a little bit like a chess game.
-
9:10 - 9:11The human would play,
-
9:11 - 9:14the robot would analyze what was played,
-
9:14 - 9:16would improvise their own part.
-
9:16 - 9:18So, this is what musicians called
-
9:18 - 9:19a call and response interaction,
-
9:19 - 9:23and it also fits very well, robots
and artificial intelligence. -
9:23 - 9:25But I thought, if I use the same ideas I used
-
9:25 - 9:28in the theater play and in the teamwork studies,
-
9:28 - 9:31maybe I can make the robots jam together
-
9:31 - 9:32like a band.
-
9:32 - 9:36Everybody's riffing off each other,
nobody is stopping it for a moment. -
9:36 - 9:39And so, I tried to do the same
things, this time with music, -
9:39 - 9:40where the robot doesn't really know
-
9:40 - 9:41what it's about to play.
-
9:41 - 9:43It just sort of moves its body
-
9:43 - 9:45and uses opportunities to play,
-
9:45 - 9:47And does what my jazz teacher
when I was 17 taught me. -
9:47 - 9:49She said, when you improvise,
-
9:49 - 9:50sometimes you don't know what you're doing
-
9:50 - 9:51and you're still doing it.
-
9:51 - 9:53And so I tried to make a robot that doesn't actually
-
9:53 - 9:55know what it's doing, but it's still doing it.
-
9:55 - 9:58So let's look at a few seconds
from this performance. -
9:58 - 10:01Where the robot listens to the human musician
-
10:01 - 10:02and improvises.
-
10:02 - 10:05And then, look at how the human musician also
-
10:05 - 10:07responds to what the robot is doing, and picking up
-
10:07 - 10:09from its behavior.
-
10:09 - 10:14And at some point can even be surprised
by what the robot came up with. -
10:14 - 11:00(Music)
-
11:00 - 11:05(Applause)
-
11:05 - 11:07Being a musician is not just about making notes,
-
11:07 - 11:09otherwise nobody would ever go see a live show.
-
11:09 - 11:11Musicians also communicate with their bodies,
-
11:11 - 11:13with other band members, with the audience,
-
11:13 - 11:15they use their bodies to express the music.
-
11:15 - 11:18And I thought, we already have
a robot musician on stage, -
11:18 - 11:21why not make it be a full-fledged musician.
-
11:21 - 11:23And I started designing a socially expressive head
-
11:23 - 11:25for the robot.
-
11:25 - 11:27The head does't actually touch the marimba,
-
11:27 - 11:28it just expresses what the music is like.
-
11:28 - 11:31These are some napkin sketches
from a bar in Atlanta, -
11:31 - 11:34that was dangerously located exactly halfway
-
11:34 - 11:36between my lab and my home. (Laughter)
-
11:36 - 11:37So I spent, I would say on average,
-
11:37 - 11:40three to four hours a day there.
-
11:40 - 11:43I think. (Laughter)
-
11:43 - 11:46And I went back to my animation
tools and tried to figure out -
11:46 - 11:48not just what a robotic musician would look like,
-
11:48 - 11:51but especially what a robotic
musician would move like. -
11:51 - 11:54To sort of show that it doesn't like
what the other person is playing -- -
11:54 - 11:56and maybe show whatever beat it's feeling
-
11:56 - 11:58at the moment.
-
11:58 - 12:03So we ended up actually getting the money
to build this robot, which was nice. -
12:03 - 12:05I'm going to show you now the
same kind of performance, -
12:05 - 12:07this time with a socially expressive head.
-
12:07 - 12:09And notice one thing --
-
12:09 - 12:11how the robot is really showing us
-
12:11 - 12:13the beat it's picking up from the human.
-
12:13 - 12:17We're also giving the human a sense
that the robot knows what it's doing. -
12:17 - 12:18And also how it changes the way it moves
-
12:18 - 12:21as soon as it starts its own solo.
-
12:21 - 12:25(Music)
-
12:25 - 12:28Now it's looking at me to make sure I'm listening.
-
12:28 - 12:49(Music)
-
12:49 - 12:52And now look at the final chord of the piece again,
-
12:52 - 12:55and this time the robot communicates with its body
-
12:55 - 12:57when it's busy doing its own thing.
-
12:57 - 12:59And when it's ready
-
12:59 - 13:02to coordinate the final chord with me.
-
13:02 - 13:15(Music)
-
13:15 - 13:21(Applause)
-
13:21 - 13:25Thanks. I hope you see how much this totally not --
-
13:25 - 13:28how much this part of the body
that doesn't touch the instrument -
13:28 - 13:31actually helps with the musical performance.
-
13:31 - 13:35And at some point, we are in Atlanta,
so obviously some rapper -
13:35 - 13:36will come into our lab at some point.
-
13:36 - 13:39And we had this rapper come in
-
13:39 - 13:41and do a little jam with the robot.
-
13:41 - 13:44And here you can see the robot
-
13:44 - 13:45basically responding to the beat and --
-
13:45 - 13:48notice two things. One, how irresistible it is
-
13:48 - 13:51to join the robot while it's moving its head.
-
13:50 - 13:52and you kind of want to move
your own head when it does it. -
13:52 - 13:56And second, even though the rapper
is really focused on his iPhone, -
13:56 - 13:59as soon as the robot turns to him, he turns back.
-
13:59 - 14:01So even though it's just in
the periphery of his vision -- -
14:01 - 14:04it's just in the corner of his eye --
it's very powerful. -
14:04 - 14:06And the reason is that we can't ignore
-
14:06 - 14:08physical things moving in our environment.
-
14:08 - 14:09We are wired for that.
-
14:09 - 14:13So, if you have a problem with maybe your partners
-
14:13 - 14:15looking at the iPhone too much
or their smartphone too much, -
14:15 - 14:17you might want to have a robot there
-
14:17 - 14:19to get their attention. (Laughter)
-
14:19 - 14:38(Music)
-
14:38 - 14:45(Applause)
-
14:45 - 14:47Just to introduce the last robot
-
14:47 - 14:50that we've worked on,
-
14:50 - 14:52that came out of something kind
of surprising that we found: -
14:52 - 14:55At some point people didn't care anymore
about the robot being so intelligent, -
14:55 - 14:56and can improvise and listen,
-
14:56 - 15:01and do all these embodied intelligence
things that I spent years on developing. -
15:01 - 15:04They really liked that the robot was
enjoying the music. (Laughter) -
15:04 - 15:07And they didn't say that the
robot was moving to the music, -
15:07 - 15:08they said that the robot was enjoying the music.
-
15:08 - 15:11And we thought, why don't we take this idea,
-
15:11 - 15:14and I designed a new piece of furniture.
-
15:14 - 15:16This time it wasn't a desk
lamp; it was a speaker dock. -
15:16 - 15:19It was one of those things you
plug your smartphone in. -
15:19 - 15:21And I thought, what would happen
-
15:21 - 15:23if your speaker dock didn't
just play the music for you, -
15:23 - 15:26but it would actually enjoy it too. (Laughter)
-
15:26 - 15:27And so again, here are some animation tests
-
15:27 - 15:32from an early stage. (Laughter)
-
15:32 - 15:36And this is what the final product looked like.
-
15:47 - 16:09("Drop It Like It's Hot")
-
16:09 - 16:12So, a lot of bobbing head.
-
16:12 - 16:15(Applause)
-
16:15 - 16:17A lot of bobbing heads in the audience,
-
16:17 - 16:20so we can still see robots influence people.
-
16:20 - 16:23And it's not just fun and games.
-
16:23 - 16:25I think one of the reasons I care so much
-
16:25 - 16:27about robots that use their body to communicate
-
16:27 - 16:29and use their body to move --
-
16:29 - 16:33and I'm going to let you in on a little
secret we roboticists are hiding -- -
16:33 - 16:35is that every one of you is
going to be living with a robot -
16:35 - 16:37at some point in their life.
-
16:37 - 16:40Somewhere in your future there's
going to be a robot in your life. -
16:40 - 16:42And if not in yours, then in your children's lives.
-
16:42 - 16:43And I want these robots to be --
-
16:43 - 16:47to be more fluent, more engaging, more graceful
-
16:47 - 16:49than currently they seem to be.
-
16:49 - 16:51And for that I think that maybe robots
-
16:51 - 16:52need to be less like chess players
-
16:52 - 16:55and more like stage actors and more like musicians.
-
16:55 - 16:58Maybe they should be able to
take chances and improvise. -
16:58 - 17:00And maybe they should be able to
anticipate what you're about to do. -
17:00 - 17:03And maybe they need to be able to make mistakes
-
17:03 - 17:04and correct them,
-
17:04 - 17:06because in the end we are human.
-
17:06 - 17:09And maybe as humans, robots
that are a little less than perfect -
17:09 - 17:11are just perfect for us.
-
17:11 - 17:13Thank you.
-
17:13 - 17:16(Applause)
- Title:
- Robots with "soul"
- Speaker:
- Guy Hoffman
- Description:
-
What kind of robots does an animator / jazz musician / roboticist make? Playful, reactive, curious ones. Guy Hoffman shows demo film of his family of unusual robots -- including two musical bots that like to jam with humans. (Filmed at TEDxJaffa.)
- Video Language:
- English
- Team:
- closed TED
- Project:
- TEDTalks
- Duration:
- 17:38
Krystian Aparta edited English subtitles for Robots with "soul" | ||
Krystian Aparta commented on English subtitles for Robots with "soul" | ||
Krystian Aparta edited English subtitles for Robots with "soul" | ||
Krystian Aparta edited English subtitles for Robots with "soul" | ||
Morton Bast edited English subtitles for Robots with "soul" | ||
Morton Bast edited English subtitles for Robots with "soul" | ||
Amirpouya Ghaemian commented on English subtitles for Robots with "soul" | ||
janet dragojevic edited English subtitles for Robots with "soul" |
Amirpouya Ghaemian
Hi,
5:07 - 5:09
But the real relevation was ...
Is that a "revelation" ?
because I hear so ...
Krystian Aparta
The English transcript was updated on 12/7/2015.