A big world of small motions | Michael Rubinstein | TEDxYouth@BeaconStreet
-
0:15 - 0:21So over the past few centuries,
microscopes have revolutionized our world. -
0:23 - 0:28They revealed to us a tiny world
of objects, life and structures -
0:28 - 0:31that are too small for us
to see with our naked eyes. -
0:31 - 0:34They are a tremendous contribution
to science and technology. -
0:34 - 0:38Today I'd like to introduce you
to a new type of microscope, -
0:38 - 0:40a microscope for changes.
-
0:40 - 0:43It doesn't use optics
like a regular microscope -
0:43 - 0:45to make small objects bigger,
-
0:45 - 0:50but instead it uses a video camera
and image processing -
0:50 - 0:54to reveal to us the tiniest motions
and color changes in objects and people, -
0:55 - 0:58changes that are impossible
for us to see with our naked eyes. -
0:59 - 1:03And it lets us look at our world
in a completely new way. -
1:03 - 1:06So what do I mean by color changes?
-
1:07 - 1:10Our skin, for example,
changes its color very slightly -
1:10 - 1:12when the blood flows under it.
-
1:12 - 1:14That change is incredibly subtle,
-
1:14 - 1:17which is why, when you look
at other people, -
1:17 - 1:19when you look at the person
sitting next to you, -
1:19 - 1:22you don't see their skin
or their face changing color. -
1:22 - 1:27When we look at this video of Steve here,
it appears to us like a static picture, -
1:28 - 1:31but once we look at this video
through our new, special microscope, -
1:31 - 1:35suddenly we see
a completely different image. -
1:35 - 1:39What you see here are small changes
in the color of Steve's skin, -
1:39 - 1:43magnified 100 times
so that they become visible. -
1:44 - 1:46We can actually see a human pulse.
-
1:47 - 1:50We can see how fast
Steve's heart is beating, -
1:50 - 1:54but we can also see the actual way
that the blood flows in his face. -
1:55 - 1:58And we can do that not just
to visualize the pulse, -
1:58 - 2:01but also to actually recover
our heart rates, -
2:01 - 2:04and measure our heart rates.
-
2:04 - 2:08And we can do it with regular cameras
and without touching the patients. -
2:08 - 2:13So here you see the pulse and heart rate
we extracted from a neonatal baby -
2:13 - 2:16from a video we took
with a regular DSLR camera, -
2:16 - 2:18and the heart rate measurement we get
-
2:18 - 2:23is as accurate as the one you'd get
with a standard monitor in a hospital. -
2:23 - 2:26And it doesn't even have to be
a video we recorded. -
2:26 - 2:29We can do it essentially
with other videos as well. -
2:29 - 2:33So I just took a short clip
from "Batman Begins" here -
2:33 - 2:35just to show Christian Bale's pulse.
-
2:35 - 2:37(Laughter)
-
2:37 - 2:39And you know, presumably
he's wearing makeup, -
2:39 - 2:41the lighting here is kind of challenging,
-
2:41 - 2:44but still, just from the video,
we're able to extract his pulse -
2:44 - 2:46and show it quite well.
-
2:46 - 2:48So how do we do all that?
-
2:48 - 2:52We basically analyze the changes
in the light that are recorded -
2:52 - 2:55at every pixel in the video over time,
-
2:55 - 2:57and then we crank up those changes.
-
2:57 - 2:59We make them bigger
so that we can see them. -
2:59 - 3:02The tricky part is that those signals,
-
3:02 - 3:04those changes that we're after,
are extremely subtle, -
3:04 - 3:07so we have to be very careful
when you try to separate them -
3:07 - 3:10from noise that always exists in videos.
-
3:10 - 3:14So we use some clever
image processing techniques -
3:14 - 3:18to get a very accurate measurement
of the color at each pixel in the video, -
3:18 - 3:21and then the way
the color changes over time, -
3:21 - 3:23and then we amplify those changes.
-
3:23 - 3:27We make them bigger to create those types
of enhanced videos, or magnified videos, -
3:27 - 3:30that actually show us those changes.
-
3:32 - 3:36But it turns out we can do that
not just to show tiny changes in color, -
3:36 - 3:38but also tiny motions,
-
3:38 - 3:42and that's because the light
that gets recorded in our cameras -
3:42 - 3:45will change not only if the color
of the object changes, -
3:45 - 3:47but also if the object moves.
-
3:48 - 3:53So this is my daughter
when she was about two months old. -
3:56 - 3:59It's a video I recorded
about three years ago. -
3:59 - 4:03And as new parents, we all want
to make sure our babies are healthy, -
4:03 - 4:05that they're breathing,
that they're alive, of course. -
4:05 - 4:07So I too got one of those baby monitors
-
4:07 - 4:10so that I could see my daughter
when she was asleep. -
4:10 - 4:14And this is pretty much what you'll see
with a standard baby monitor. -
4:14 - 4:16You can see the baby's sleeping,
-
4:16 - 4:18but there's not too much information
there. -
4:18 - 4:20There's not too much we can see.
-
4:20 - 4:22Wouldn't it be better,
or more informative, or more useful, -
4:22 - 4:25if instead we could look
at the view like this. -
4:25 - 4:30So here I took the motions
and I magnified them 30 times, -
4:31 - 4:34and then I could clearly see
that my daughter -
4:34 - 4:35was indeed alive and breathing.
-
4:35 - 4:38(Laughter)
-
4:38 - 4:40Here is a side-by-side comparison.
-
4:40 - 4:42So again, in the source video,
in the original video, -
4:42 - 4:44there's not too much we can see,
-
4:44 - 4:48but once we magnify the motions,
the breathing becomes much more visible. -
4:48 - 4:51And it turns out,
there's a lot of phenomena -
4:51 - 4:54we can reveal and magnify
with our new motion microscope. -
4:54 - 4:59We can see how our veins and arteries
are pulsing in our bodies. -
5:00 - 5:03We can see that our eyes
are constantly moving -
5:03 - 5:05in this wobbly motion.
-
5:05 - 5:06And that's actually my eye,
-
5:06 - 5:09and again this video was taken
right after my daughter was born, -
5:09 - 5:13so you can see I wasn't getting
too much sleep. (Laughter) -
5:14 - 5:16Even when a person is sitting still,
-
5:16 - 5:19there's a lot of information
we can extract -
5:19 - 5:22about their breathing patterns,
small facial expressions. -
5:23 - 5:25Maybe we could use those motions
-
5:25 - 5:27to tell us something about
our thoughts or our emotions. -
5:29 - 5:32We can also magnify
small mechanical movements, -
5:32 - 5:34like vibrations in engines,
-
5:34 - 5:38that can help engineers detect
and diagnose machinery problems, -
5:40 - 5:46or see how our buildings and structures
sway in the wind and react to forces. -
5:46 - 5:50Those are all things that our society
knows how to measure in various ways, -
5:50 - 5:53but measuring those motions is one thing,
-
5:53 - 5:55and actually seeing those motions
as they happen -
5:55 - 5:58is a whole different thing.
-
5:58 - 6:02And ever since we discovered
this new technology, -
6:02 - 6:04we made our code available online
-
6:04 - 6:06so that others could use
and experiment with it. -
6:08 - 6:10It's very simple to use.
-
6:10 - 6:12It can work on your own videos.
-
6:12 - 6:15Our collaborators at Quantum Research
even created this nice website -
6:15 - 6:18where you can upload your videos
and process them online, -
6:18 - 6:22so even if you don't have any experience
in computer science or programming, -
6:22 - 6:25you can still very easily experiment
with this new microscope. -
6:25 - 6:27And I'd like to show you
just a couple of examples -
6:27 - 6:29of what others have done with it.
-
6:32 - 6:37So this video was made
by a YouTube user called Tamez85. -
6:37 - 6:39I don't know who that user is,
-
6:39 - 6:41but he, or she, used our code
-
6:41 - 6:44to magnify small belly movements
during pregnancy. -
6:45 - 6:46It's kind of creepy.
-
6:46 - 6:49(Laughter)
-
6:49 - 6:53People have used it to magnify
pulsing veins in their hands. -
6:54 - 6:57And you know it's not real science
unless you use guinea pigs, -
6:58 - 7:01and apparently this guinea pig
is called Tiffany, -
7:01 - 7:04and this YouTube user claims
it is the first rodent on Earth -
7:04 - 7:06that was motion-magnified.
-
7:07 - 7:09You can also do some art with it.
-
7:09 - 7:12So this video was sent to me
by a design student at Yale. -
7:12 - 7:15She wanted to see
if there's any difference -
7:15 - 7:16in the way her classmates move.
-
7:16 - 7:20She made them all stand still,
and then magnified their motions. -
7:20 - 7:23It's like seeing still pictures
come to life. -
7:24 - 7:26And the nice thing with all those examples
-
7:26 - 7:28is that we had nothing to do with them.
-
7:28 - 7:32We just provided this new tool,
a new way to look at the world, -
7:32 - 7:37and then people find other interesting,
new and creative ways of using it. -
7:38 - 7:40But we didn't stop there.
-
7:41 - 7:45This tool not only allows us
to look at the world in a new way, -
7:45 - 7:47it also redefines what we can do
-
7:47 - 7:50and pushes the limits
of what we can do with our cameras. -
7:50 - 7:53So as scientists, we started wondering,
-
7:53 - 7:56what other types of physical phenomena
produce tiny motions -
7:56 - 7:59that we could now use
our cameras to measure? -
7:59 - 8:03And one such phenomenon
that we focused on recently is sound. -
8:04 - 8:06Sound, as we all know,
is basically changes -
8:06 - 8:08in air pressure
that travel through the air. -
8:08 - 8:12Those pressure waves hit objects
and they create small vibrations in them, -
8:12 - 8:15which is how we hear
and how we record sound. -
8:15 - 8:18But it turns out that sound
also produces visual motions. -
8:19 - 8:21Those are motions
that are not visible to us -
8:21 - 8:24but are visible to a camera
with the right processing. -
8:24 - 8:26So here are two examples.
-
8:26 - 8:29This is me demonstrating
my great singing skills. -
8:31 - 8:34(Singing)
-
8:34 - 8:35(Laughter)
-
8:35 - 8:38And I took a high-speed video
of my throat while I was humming. -
8:38 - 8:39Again, if you stare at that video,
-
8:39 - 8:41there's not too much
you'll be able to see, -
8:41 - 8:46but once we magnify the motions 100 times,
we can see all the motions and ripples -
8:46 - 8:49in the neck that are involved
in producing the sound. -
8:49 - 8:52That signal is there in that video.
-
8:52 - 8:54We also know that singers
can break a wine glass -
8:54 - 8:56if they hit the correct note.
-
8:56 - 8:58So here, we're going to play a note
-
8:58 - 9:01that's in the resonance frequency
of that glass -
9:01 - 9:03through a loudspeaker that's next to it.
-
9:03 - 9:08Once we play that note
and magnify the motions 250 times, -
9:08 - 9:11we can very clearly see
how the glass vibrates -
9:11 - 9:14and resonates in response to the sound.
-
9:14 - 9:17It's not something you're used to seeing
every day. -
9:17 - 9:19And we actually have the demo
right outside set up, -
9:19 - 9:21so I encourage you to stop by,
-
9:21 - 9:24and just play with it yourself,
you can actually see it live. -
9:25 - 9:28But this made us think.
It gave us this crazy idea. -
9:28 - 9:33Can we actually invert this process
and recover sound from video -
9:33 - 9:38by analyzing the tiny vibrations
that sound waves create in objects, -
9:38 - 9:42and essentially convert those
back into the sounds that produced them. -
9:43 - 9:46In this way, we can turn
everyday objects into microphones. -
9:48 - 9:50So that's exactly what we did.
-
9:50 - 9:52So here's an empty bag of chips
that was lying on a table, -
9:52 - 9:55and we're going to turn that bag of chips
into a microphone -
9:55 - 9:57by filming it with a video camera
-
9:57 - 10:01and analyzing the tiny motions
that sound waves create in it. -
10:01 - 10:04So here's the sound
that we played in the room. -
10:04 - 10:08(Music: "Mary Had a Little Lamb")
-
10:12 - 10:15And this is a high-speed video
we recorded of that bag of chips. -
10:15 - 10:17Again it's playing.
-
10:17 - 10:20There's no chance you'll be able
to see anything going on in that video -
10:20 - 10:21just by looking at it,
-
10:21 - 10:24but here's the sound we were able
to recover just by analyzing -
10:24 - 10:26the tiny motions in that video.
-
10:27 - 10:30(Music: "Mary Had a Little Lamb")
-
10:45 - 10:46I call it -- Thank you.
-
10:46 - 10:49(Applause)
-
10:54 - 10:56I call it the visual microphone.
-
10:56 - 10:59We actually extract audio signals
from video signals. -
10:59 - 11:02And just to give you a sense
of the scale of the motions here, -
11:02 - 11:07a pretty loud sound will cause
that bag of chips -
11:07 - 11:09to move less than a micrometer.
-
11:10 - 11:12That's one thousandth of a millimeter.
-
11:12 - 11:16That's how tiny the motions are
that we are now able to pull out -
11:16 - 11:19just by observing how light
bounces off objects -
11:19 - 11:22and gets recorded by our cameras.
-
11:22 - 11:25We can recover sounds
from other objects, like plants. -
11:26 - 11:29(Music: "Mary Had a Little Lamb")
-
11:34 - 11:36And we can recover speech as well.
-
11:36 - 11:39So here's a person speaking in a room.
-
11:39 - 11:44Voice: Mary had a little lamb
whose fleece was white as snow, -
11:44 - 11:48and everywhere that Mary went,
that lamb was sure to go. -
11:49 - 11:51Michael Rubinstein: And here's
that speech again recovered -
11:51 - 11:54just from this video
of that same bag of chips. -
11:54 - 11:59Voice: Mary had a little lamb
whose fleece was white as snow, -
11:59 - 12:04and everywhere that Mary went,
that lamb was sure to go. -
12:04 - 12:07MR: We used "Mary Had a Little Lamb"
-
12:07 - 12:09because those are said to be
the first words -
12:09 - 12:13that Thomas Edison spoke
into his phonograph in 1877. -
12:13 - 12:17It was one of the first
sound recording devices in history. -
12:17 - 12:20It basically directed the sounds
onto a diaphragm -
12:20 - 12:24that vibrated a needle that essentially
engraved the sound on tinfoil -
12:24 - 12:27that was wrapped around the cylinder.
-
12:27 - 12:30Here's a demonstration of recording
-
12:30 - 12:32and replaying sound
with Edison's phonograph. -
12:34 - 12:36(Video) Voice: Testing, testing,
one two three. -
12:36 - 12:40Mary had a little lamb
whose fleece was white as snow, -
12:40 - 12:43and everywhere that Mary went,
the lamb was sure to go. -
12:43 - 12:46Testing, testing, one two three.
-
12:46 - 12:50Mary had a little lamb
whose fleece was white as snow, -
12:50 - 12:54and everywhere that Mary went,
the lamb was sure to go. -
12:56 - 12:59MR: And now, 137 years later,
-
13:00 - 13:03we're able to get sound
in pretty much similar quality -
13:03 - 13:08but by just watching objects
vibrate to sound with cameras, -
13:08 - 13:10and we can even do that when the camera
-
13:10 - 13:14is 15 feet away from the object,
behind soundproof glass. -
13:14 - 13:17So this is the sound that we were able
to recover in that case. -
13:17 - 13:22Voice: Mary had a little lamb
whose fleece was white as snow, -
13:22 - 13:27and everywhere that Mary went,
the lamb was sure to go. -
13:28 - 13:32MR: And of course, surveillance is
the first application that comes to mind. -
13:32 - 13:34(Laughter)
-
13:34 - 13:38But it might actually be useful
for other things as well. -
13:38 - 13:41Maybe in the future,
we'll be able to use it, for example, -
13:41 - 13:44to recover sound across space,
-
13:44 - 13:47because sound can't travel
in space, but light can. -
13:47 - 13:50We've only just begun exploring
-
13:50 - 13:53other possible uses
for this new technology. -
13:53 - 13:55It lets us see physical processes
that we know are there -
13:55 - 14:00but that we've never been able
to see with our own eyes until now. -
14:01 - 14:02This is our team.
-
14:02 - 14:05Everything I showed you today
is a result of a collaboration -
14:05 - 14:07with this great group
of people you see here, -
14:07 - 14:10and I encourage you and welcome you
to check out our website, -
14:10 - 14:12try it out yourself,
-
14:12 - 14:15and join us in exploring
this world of tiny motions. -
14:15 - 14:17Thank you.
-
14:17 - 14:19(Applause)
- Title:
- A big world of small motions | Michael Rubinstein | TEDxYouth@BeaconStreet
- Description:
-
This talk was given at a local TEDx event, produced independently of the TED Conferences. Meet the “motion microscope,” a video-processing tool that plays up tiny changes in motion and color impossible to see with the naked eye. Video researcher Michael Rubinstein plays us clip after jaw-dropping clip showing how this tech can track an individual’s pulse and heartbeat simply from a piece of footage. Watch him recreate a conversation by amplifying the movements from sound waves bouncing off a bag of chips. The wow-inspiring and sinister applications of this tech you have to see to believe.
- Video Language:
- English
- Team:
- closed TED
- Project:
- TEDxTalks
- Duration:
- 14:24
Ivana Korom edited English subtitles for A big world of small motions | Michael Rubinstein | TEDxYouth@BeaconStreet | ||
Ivana Korom edited English subtitles for A big world of small motions | Michael Rubinstein | TEDxYouth@BeaconStreet | ||
TED Translators admin approved English subtitles for A big world of small motions | Michael Rubinstein | TEDxYouth@BeaconStreet | ||
TED Translators admin edited English subtitles for A big world of small motions | Michael Rubinstein | TEDxYouth@BeaconStreet | ||
TED Translators admin edited English subtitles for A big world of small motions | Michael Rubinstein | TEDxYouth@BeaconStreet | ||
TED Translators admin edited English subtitles for A big world of small motions | Michael Rubinstein | TEDxYouth@BeaconStreet | ||
Ivana Korom accepted English subtitles for A big world of small motions | Michael Rubinstein | TEDxYouth@BeaconStreet | ||
TED Translators admin edited English subtitles for A big world of small motions | Michael Rubinstein | TEDxYouth@BeaconStreet |