Robots with "soul"
-
0:01 - 0:05My job is to design, build and study
robots that communicate with people. -
0:05 - 0:09But this story doesn't start with robotics
at all, it starts with animation. -
0:09 - 0:11When I first saw Pixar's "Luxo Jr.,"
-
0:11 - 0:14I was amazed by how much
emotion they could put -
0:14 - 0:17into something as trivial as a desk lamp.
-
0:17 - 0:20I mean, look at them --
at the end of this movie, -
0:20 - 0:23you actually feel something
for two pieces of furniture. -
0:23 - 0:24(Laughter)
-
0:24 - 0:26And I said, I have
to learn how to do this. -
0:26 - 0:28So I made a really bad career decision.
-
0:28 - 0:29(Laughter)
-
0:29 - 0:32And that's what my mom
was like when I did it. -
0:32 - 0:34(Laughter)
-
0:34 - 0:37I left a very cozy tech job in Israel
at a nice software company -
0:37 - 0:40and I moved to New York
to study animation. -
0:40 - 0:41And there I lived
-
0:41 - 0:44in a collapsing apartment building
in Harlem with roommates. -
0:44 - 0:46I'm not using this phrase
metaphorically -- -
0:46 - 0:48the ceiling actually collapsed
one day in our living room. -
0:48 - 0:52Whenever they did news stories
about building violations in New York, -
0:52 - 0:54they would put the report
in front of our building, -
0:54 - 0:57as kind of, like, a backdrop
to show how bad things are. -
0:57 - 0:59Anyway, during the day, I went to school
-
0:59 - 1:02and at night I would sit and draw
frame by frame of pencil animation. -
1:02 - 1:04And I learned two surprising lessons.
-
1:05 - 1:09One of them was that
when you want to arouse emotions, -
1:09 - 1:11it doesn't matter so much
how something looks; -
1:11 - 1:14it's all in the motion, in the timing
of how the thing moves. -
1:15 - 1:18And the second was something
one of our teachers told us. -
1:18 - 1:20He actually did the weasel in "Ice Age."
-
1:21 - 1:25And he said, "As an animator,
you're not a director -- you're an actor." -
1:25 - 1:28So, if you want to find
the right motion for a character, -
1:28 - 1:30don't think about it --
go use your body to find it. -
1:30 - 1:33Stand in front of a mirror,
act it out in front of a camera -- -
1:33 - 1:36whatever you need -- and then
put it back in your character. -
1:37 - 1:40A year later I found myself at MIT
in the Robotic Life Group. -
1:40 - 1:43It was one of the first groups
researching the relationships -
1:43 - 1:44between humans and robots.
-
1:44 - 1:48And I still had this dream
to make an actual, physical Luxo Jr. lamp. -
1:48 - 1:51But I found that robots didn't move
at all in this engaging way -
1:51 - 1:53that I was used to
from my animation studies. -
1:53 - 1:56Instead, they were all --
how should I put it -- -
1:56 - 1:57they were all kind of robotic.
-
1:57 - 1:59(Laughter)
-
1:59 - 2:03And I thought, what if I took
whatever I learned in animation school, -
2:03 - 2:06and used that to design
my robotic desk lamp. -
2:06 - 2:08So I went and designed frame by frame
-
2:08 - 2:12to try to make this robot as graceful
and engaging as possible. -
2:12 - 2:16And here when you see the robot
interacting with me on a desktop -- -
2:16 - 2:18and I'm actually redesigning the robot,
-
2:18 - 2:20so, unbeknownst to itself,
-
2:20 - 2:22it's kind of digging its own
grave by helping me. -
2:22 - 2:24(Laughter)
-
2:24 - 2:28I wanted it to be less of a mechanical
structure giving me light, -
2:28 - 2:31and more of a helpful,
kind of quiet apprentice -
2:31 - 2:34that's always there when you need it
and doesn't really interfere. -
2:34 - 2:37And when, for example, I'm looking
for a battery that I can't find, -
2:37 - 2:40in a subtle way, it'll show me
where the battery is. -
2:42 - 2:44So you can see my confusion here.
-
2:44 - 2:46I'm not an actor.
-
2:49 - 2:52And I want you to notice
how the same mechanical structure -
2:52 - 2:54can, at one point,
just by the way it moves, -
2:54 - 2:56seem gentle and caring
and in the other case, -
2:56 - 2:58seem violent and confrontational.
-
2:58 - 3:01And it's the same structure,
just the motion is different. -
3:07 - 3:12Actor: "You want to know something?
Well, you want to know something? -
3:12 - 3:14He was already dead!
-
3:14 - 3:18Just laying there, eyes glazed over!"
-
3:18 - 3:19(Laughter)
-
3:19 - 3:22But, moving in a graceful way
is just one building block -
3:22 - 3:25of this whole structure
called human-robot interaction. -
3:25 - 3:28I was, at the time, doing my PhD,
I was working on human-robot teamwork, -
3:28 - 3:30teams of humans and robots
working together. -
3:30 - 3:32I was studying the engineering,
-
3:32 - 3:34the psychology,
the philosophy of teamwork, -
3:34 - 3:36and at the same time,
-
3:36 - 3:38I found myself in my own kind
of teamwork situation, -
3:38 - 3:40with a good friend of mine,
who's actually here. -
3:40 - 3:43And in that situation,
we can easily imagine robots -
3:43 - 3:45in the near future being there with us.
-
3:45 - 3:46It was after a Passover Seder.
-
3:46 - 3:48We were folding up
a lot of folding chairs, -
3:48 - 3:51and I was amazed at how quickly
we found our own rhythm. -
3:51 - 3:54Everybody did their own part,
we didn't have to divide our tasks. -
3:54 - 3:57We didn't have to communicate
verbally about this -- -
3:57 - 3:58it all just happened.
-
3:58 - 4:01And I thought, humans and robots
don't look at all like this. -
4:01 - 4:04When humans and robots interact,
it's much more like a chess game: -
4:04 - 4:07the human does a thing, the robot
analyzes whatever the human did, -
4:07 - 4:10the robot decides what to do next,
plans it and does it. -
4:10 - 4:12Then the human waits,
until it's their turn again. -
4:12 - 4:15So it's much more like a chess game,
and that makes sense, -
4:15 - 4:18because chess is great for mathematicians
and computer scientists. -
4:18 - 4:22It's all about information, analysis,
decision-making and planning. -
4:22 - 4:25But I wanted my robot
to be less of a chess player, -
4:25 - 4:27and more like a doer
-
4:27 - 4:29that just clicks and works together.
-
4:29 - 4:33So I made my second
horrible career choice: -
4:33 - 4:35I decided to study acting for a semester.
-
4:35 - 4:38I took off from the PhD,
I went to acting classes. -
4:38 - 4:40I actually participated in a play --
-
4:40 - 4:42I hope there’s no video
of that around still. -
4:42 - 4:44(Laughter)
-
4:44 - 4:46And I got every book
I could find about acting, -
4:46 - 4:49including one from the 19th century
that I got from the library. -
4:49 - 4:53And I was really amazed, because my name
was the second name on the list -- -
4:53 - 4:55the previous name was in 1889.
-
4:55 - 4:56(Laughter)
-
4:56 - 4:58And this book was
kind of waiting for 100 years -
4:58 - 5:00to be rediscovered for robotics.
-
5:00 - 5:02And this book shows actors
-
5:02 - 5:04how to move every muscle in the body
-
5:04 - 5:07to match every kind of emotion
that they want to express. -
5:07 - 5:10But the real revelation was
when I learned about method acting. -
5:10 - 5:12It became very popular
in the 20th century. -
5:12 - 5:13And method acting said
-
5:13 - 5:16you don't have to plan
every muscle in your body; -
5:16 - 5:19instead, you have to use your body
to find the right movement. -
5:19 - 5:22You have to use your sense memory
to reconstruct the emotions -
5:22 - 5:25and kind of think with your body
to find the right expression -- -
5:25 - 5:27improvise, play off your scene partner.
-
5:27 - 5:28And this came at the same time
-
5:28 - 5:31as I was reading about this trend
in cognitive psychology, -
5:31 - 5:34called embodied cognition,
which also talks about the same ideas. -
5:34 - 5:36We use our bodies to think;
-
5:36 - 5:39we don't just think with our brains
and use our bodies to move, -
5:39 - 5:41but our bodies feed back into our brain
-
5:41 - 5:43to generate the way that we behave.
-
5:43 - 5:45And it was like a lightning bolt.
-
5:45 - 5:46I went back to my office,
-
5:46 - 5:48I wrote this paper,
which I never really published, -
5:48 - 5:51called "Acting Lessons
for Artificial Intelligence." -
5:51 - 5:52And I even took another month
-
5:52 - 5:55to do what was then the first theater play
-
5:55 - 5:57with a human and a robot acting together.
-
5:57 - 5:59That's what you saw
before with the actors. -
6:01 - 6:02And I thought:
-
6:02 - 6:05How can we make an artificial
intelligence model -- -
6:05 - 6:07a computer, computational model --
-
6:07 - 6:09that will model some of these
ideas of improvisation, -
6:09 - 6:11of taking risks, of taking chances,
-
6:11 - 6:13even of making mistakes?
-
6:13 - 6:15Maybe it can make for better
robotic teammates. -
6:15 - 6:18So I worked for quite
a long time on these models -
6:18 - 6:20and I implemented them
on a number of robots. -
6:20 - 6:23Here you can see a very early example
-
6:23 - 6:26with the robots trying to use
this embodied artificial intelligence -
6:26 - 6:29to try to match my movements
as closely as possible. -
6:29 - 6:30It's sort of like a game.
-
6:31 - 6:32Let's look at it.
-
6:36 - 6:39You can see when I psych
it out, it gets fooled. -
6:40 - 6:42And it's a little bit
like what you might see actors do -
6:42 - 6:44when they try to mirror each other
-
6:44 - 6:46to find the right synchrony between them.
-
6:46 - 6:48And then, I did another experiment,
-
6:48 - 6:52and I got people off the street
to use the robotic desk lamp, -
6:52 - 6:55and try out this idea
of embodied artificial intelligence. -
6:56 - 7:01So, I actually used two kinds
of brains for the same robot. -
7:01 - 7:02The robot is the same lamp that you saw,
-
7:03 - 7:04and I put two brains in it.
-
7:04 - 7:06For one half of the people,
-
7:06 - 7:09I put in a brain
that's kind of the traditional, -
7:09 - 7:10calculated robotic brain.
-
7:10 - 7:12It waits for its turn,
it analyzes everything, it plans. -
7:13 - 7:14Let's call it the calculated brain.
-
7:14 - 7:17The other got more the stage
actor, risk-taker brain. -
7:17 - 7:20Let's call it the adventurous brain.
-
7:20 - 7:23It sometimes acts without knowing
everything it has to know. -
7:23 - 7:25It sometimes makes mistakes
and corrects them. -
7:25 - 7:29And I had them do this very tedious task
that took almost 20 minutes, -
7:29 - 7:30and they had to work together,
-
7:30 - 7:33somehow simulating, like, a factory job
-
7:33 - 7:35of repetitively doing the same thing.
-
7:35 - 7:39What I found is that people
actually loved the adventurous robot. -
7:39 - 7:41They thought it was more intelligent,
-
7:41 - 7:43more committed,
a better member of the team, -
7:43 - 7:45contributed to the success
of the team more. -
7:45 - 7:47They even called it "he" and "she,"
-
7:47 - 7:49whereas people with the calculated brain
-
7:49 - 7:51called it "it," and nobody
ever called it "he" or "she." -
7:52 - 7:55When they talked about it after the task,
with the adventurous brain, -
7:55 - 7:59they said, "By the end, we were good
friends and high-fived mentally." -
7:59 - 8:01Whatever that means.
-
8:01 - 8:03(Laughter)
-
8:03 - 8:04Sounds painful.
-
8:04 - 8:07Whereas the people
with the calculated brain -
8:07 - 8:09said it was just like a lazy apprentice.
-
8:09 - 8:12It only did what it was supposed
to do and nothing more, -
8:12 - 8:14which is almost what people
expect robots to do, -
8:14 - 8:18so I was surprised that people
had higher expectations of robots -
8:18 - 8:21than what anybody in robotics
thought robots should be doing. -
8:22 - 8:24And in a way, I thought,
maybe it's time -- -
8:24 - 8:27just like method acting
changed the way people thought -
8:27 - 8:29about acting in the 19th century,
-
8:29 - 8:32from going from the very calculated,
planned way of behaving, -
8:32 - 8:35to a more intuitive, risk-taking,
embodied way of behaving -- -
8:35 - 8:38maybe it's time for robots
to have the same kind of revolution. -
8:40 - 8:44A few years later, I was at my next
research job at Georgia Tech in Atlanta, -
8:44 - 8:46and I was working in a group
dealing with robotic musicians. -
8:46 - 8:49And I thought, music:
that's the perfect place -
8:49 - 8:53to look at teamwork, coordination,
timing, improvisation -- -
8:53 - 8:55and we just got this robot
playing marimba. -
8:55 - 8:58And the marimba, for everybody like me,
-
8:58 - 9:01it was this huge, wooden xylophone.
-
9:01 - 9:03And when I was looking at this,
-
9:03 - 9:06I looked at other works
in human-robot improvisation -- -
9:06 - 9:08yes, there are other works
in human-robot improvisation -- -
9:08 - 9:11and they were also a little bit
like a chess game. -
9:11 - 9:12The human would play,
-
9:12 - 9:14the robot analyzed what was played,
-
9:14 - 9:16and would improvise their own part.
-
9:16 - 9:19So, this is what musicians called
a call-and-response interaction, -
9:19 - 9:23and it also fits very well
robots and artificial intelligence. -
9:23 - 9:26But I thought, if I use the same ideas
I used in the theater play -
9:26 - 9:28and in the teamwork studies,
-
9:28 - 9:32maybe I can make the robots
jam together like a band. -
9:32 - 9:36Everybody's riffing off each other,
nobody is stopping for a moment. -
9:36 - 9:39And so I tried to do the same
things, this time with music, -
9:39 - 9:42where the robot doesn't really know
what it's about to play, -
9:42 - 9:45it just sort of moves its body
and uses opportunities to play, -
9:45 - 9:48and does what my jazz teacher
when I was 17 taught me. -
9:48 - 9:49She said, when you improvise,
-
9:49 - 9:52sometimes you don't know
what you're doing, and you still do it. -
9:52 - 9:55So I tried to make a robot that doesn't
actually know what it's doing, -
9:55 - 9:57but is still doing it.
-
9:57 - 9:59So let's look at a few seconds
from this performance, -
9:59 - 10:01where the robot listens
to the human musician -
10:01 - 10:02and improvises.
-
10:02 - 10:05And then, look how the human
musician also responds -
10:05 - 10:07to what the robot is doing
-
10:07 - 10:09and picking up from its behavior,
-
10:09 - 10:13and at some point can even be surprised
by what the robot came up with. -
10:13 - 10:16(Music)
-
10:56 - 10:58(Music ends)
-
10:59 - 11:05(Applause)
-
11:05 - 11:07Being a musician
is not just about making notes, -
11:07 - 11:10otherwise nobody
would ever go see a live show. -
11:10 - 11:12Musicians also communicate
with their bodies, -
11:12 - 11:14with other band members,
with the audience, -
11:14 - 11:16they use their bodies
to express the music. -
11:16 - 11:18And I thought, we already have
a robot musician on stage, -
11:18 - 11:21why not make it be
a full-fledged musician? -
11:21 - 11:23And I started designing
a socially expressive head -
11:24 - 11:25for the robot.
-
11:25 - 11:27The head doesn’t actually
touch the marimba, -
11:27 - 11:29it just expresses what the music is like.
-
11:29 - 11:31These are some napkin sketches
from a bar in Atlanta -
11:31 - 11:34that was dangerously
located exactly halfway -
11:34 - 11:36between my lab and my home.
-
11:36 - 11:40So I spent, I would say, on average,
three to four hours a day there. -
11:40 - 11:41I think.
-
11:41 - 11:43(Laughter)
-
11:43 - 11:46And I went back to my animation
tools and tried to figure out -
11:46 - 11:48not just what a robotic
musician would look like, -
11:48 - 11:51but especially what a robotic
musician would move like, -
11:51 - 11:54to sort of show that it doesn't like
what the other person is playing -- -
11:54 - 11:58and maybe show whatever beat
it's feeling at the moment. -
11:58 - 12:03So we ended up actually getting the money
to build this robot, which was nice. -
12:03 - 12:05I'm going to show you now
the same kind of performance, -
12:05 - 12:07this time with a socially expressive head.
-
12:07 - 12:09And notice one thing --
-
12:09 - 12:11how the robot is really showing us
-
12:11 - 12:13the beat it's picking up from the human,
-
12:13 - 12:17while also giving the human a sense
that the robot knows what it's doing. -
12:17 - 12:19And also how it changes the way it moves
-
12:19 - 12:21as soon as it starts its own solo.
-
12:21 - 12:25(Music)
-
12:25 - 12:28Now it's looking at me,
showing that it's listening. -
12:28 - 12:30(Music)
-
12:49 - 12:52Now look at the final chord
of the piece again. -
12:52 - 12:55And this time the robot
communicates with its body -
12:55 - 12:57when it's busy doing its own thing,
-
12:57 - 13:02and when it's ready to coordinate
the final chord with me. -
13:02 - 13:05(Music)
-
13:10 - 13:12(Music ending)
-
13:14 - 13:15(Final chord)
-
13:15 - 13:21(Applause)
-
13:21 - 13:22Thanks.
-
13:22 - 13:24I hope you see
-
13:24 - 13:28how much this part of the body
that doesn't touch the instrument -
13:28 - 13:31actually helps
with the musical performance. -
13:31 - 13:33And at some point -- we are in Atlanta,
-
13:33 - 13:36so obviously some rapper
will come into our lab at some point -- -
13:36 - 13:41and we had this rapper come in and do
a little jam with the robot. -
13:41 - 13:45Here you can see the robot
basically responding to the beat. -
13:45 - 13:47Notice two things:
-
13:47 - 13:50one, how irresistible it is to join
the robot while it's moving its head. -
13:50 - 13:53You kind of want to move
your own head when it does it. -
13:53 - 13:56And second, even though the rapper
is really focused on his iPhone, -
13:56 - 14:00as soon as the robot turns
to him, he turns back. -
14:00 - 14:02So even though it's just
in the periphery of his vision, -
14:02 - 14:04in the corner of his eye,
it's very powerful. -
14:04 - 14:06And the reason is that we can't ignore
-
14:06 - 14:08physical things moving in our environment.
-
14:08 - 14:09We are wired for that.
-
14:09 - 14:11So if you have a problem --
-
14:11 - 14:16maybe your partner is looking
at their iPhone or smartphone too much -- -
14:16 - 14:19you might want to have a robot
there to get their attention. -
14:19 - 14:20(Laughter)
-
14:20 - 14:22(Music)
-
14:34 - 14:36(Music ends)
-
14:38 - 14:45(Applause)
-
14:46 - 14:49Just to introduce the last robot
that we've worked on, -
14:49 - 14:52it came out of something
surprising that we found: -
14:52 - 14:55Some point people didn't care
about the robot being intelligent, -
14:55 - 14:56able to improvise and listen,
-
14:56 - 15:01and do all these embodied intelligence
things that I spent years developing. -
15:01 - 15:04They really liked that the robot
was enjoying the music. -
15:04 - 15:05(Laughter)
-
15:05 - 15:07And they didn't say the robot
was moving to the music, -
15:07 - 15:09they said "enjoying" the music.
-
15:09 - 15:11And we thought,
why don't we take this idea, -
15:11 - 15:14and I designed a new piece of furniture.
-
15:14 - 15:16This time it wasn't a desk lamp,
it was a speaker dock, -
15:16 - 15:19one of those things
you plug your smartphone in. -
15:19 - 15:20And I thought,
-
15:20 - 15:24what would happen if your speaker dock
didn't just play the music for you, -
15:24 - 15:26but would actually enjoy it, too?
-
15:26 - 15:30And so again, here are some
animation tests from an early stage. -
15:30 - 15:31(Laughter)
-
15:32 - 15:35And this is what the final
product looked like. -
15:46 - 15:48(Music)
-
16:07 - 16:09(Music ends)
-
16:09 - 16:12So, a lot of bobbing heads.
-
16:12 - 16:16(Applause)
-
16:16 - 16:18A lot of bobbing heads in the audience,
-
16:18 - 16:20so we can still see
robots influence people. -
16:20 - 16:23And it's not just fun and games.
-
16:23 - 16:25I think one of the reasons I care so much
-
16:25 - 16:28about robots that use
their body to communicate -
16:28 - 16:29and use their body to move is --
-
16:29 - 16:33I'm going to let you in on a little
secret we roboticists are hiding -- -
16:33 - 16:36is that every one of you
is going to be living with a robot -
16:36 - 16:37at some point in your life.
-
16:37 - 16:40Somewhere in your future,
there will be a robot in your life. -
16:40 - 16:42If not in yours, your children's lives.
-
16:42 - 16:47And I want these robots to be more fluent,
more engaging, more graceful -
16:47 - 16:49than currently they seem to be.
-
16:49 - 16:52And for that I think maybe robots
need to be less like chess players -
16:52 - 16:55and more like stage actors
and more like musicians. -
16:55 - 16:58Maybe they should be able
to take chances and improvise. -
16:58 - 17:01Maybe they should be able
to anticipate what you're about to do. -
17:01 - 17:04Maybe they even need to be able
to make mistakes and correct them, -
17:04 - 17:06because in the end, we are human.
-
17:06 - 17:10And maybe as humans,
robots that are a little less than perfect -
17:10 - 17:11are just perfect for us.
-
17:11 - 17:13Thank you.
-
17:13 - 17:19(Applause)
- Title:
- Robots with "soul"
- Speaker:
- Guy Hoffman
- Description:
-
What kind of robots does an animator / jazz musician / roboticist make? Playful, reactive, curious ones. Guy Hoffman shows demo film of his family of unusual robots -- including two musical bots that like to jam with humans. (Filmed at TEDxJaffa.)
- Video Language:
- English
- Team:
- closed TED
- Project:
- TEDTalks
- Duration:
- 17:38
Krystian Aparta edited English subtitles for Robots with "soul" | ||
Krystian Aparta commented on English subtitles for Robots with "soul" | ||
Krystian Aparta edited English subtitles for Robots with "soul" | ||
Krystian Aparta edited English subtitles for Robots with "soul" | ||
Morton Bast edited English subtitles for Robots with "soul" | ||
Morton Bast edited English subtitles for Robots with "soul" | ||
Amirpouya Ghaemian commented on English subtitles for Robots with "soul" | ||
janet dragojevic edited English subtitles for Robots with "soul" |
Amirpouya Ghaemian
Hi,
5:07 - 5:09
But the real relevation was ...
Is that a "revelation" ?
because I hear so ...
Krystian Aparta
The English transcript was updated on 12/7/2015.