YouTube

Got a YouTube account?

New: enable viewer-created translations and captions on your YouTube channel!

English subtitles

← A big world of small motions | Michael Rubinstein | TEDxYouth@BeaconStreet

Get Embed Code
12 Languages

Showing Revision 8 created 12/24/2014 by TED Translators admin.

  1. So over the past few centuries,
    microscopes have revolutionized our world.
  2. They revealed to us a tiny world
    of objects, life and structures
  3. that are too small for us
    to see with our naked eyes.
  4. They are a tremendous contribution
    to science and technology.
  5. Today I'd like to introduce you
    to a new type of microscope,
  6. a microscope for changes.
  7. It doesn't use optics
    like a regular microscope
  8. to make small objects bigger,
  9. but instead it uses a video camera
    and image processing
  10. to reveal to us the tiniest motions
    and color changes in objects and people,
  11. changes that are impossible
    for us to see with our naked eyes.
  12. And it lets us look at our world
    in a completely new way.
  13. So what do I mean by color changes?
  14. Our skin, for example,
    changes its color very slightly
  15. when the blood flows under it.
  16. That change is incredibly subtle,
  17. which is why, when you look
    at other people,
  18. when you look at the person
    sitting next to you,
  19. you don't see their skin
    or their face changing color.
  20. When we look at this video of Steve here,
    it appears to us like a static picture,
  21. but once we look at this video
    through our new, special microscope,
  22. suddenly we see
    a completely different image.
  23. What you see here are small changes
    in the color of Steve's skin,
  24. magnified 100 times
    so that they become visible.
  25. We can actually see a human pulse.
  26. We can see how fast
    Steve's heart is beating,
  27. but we can also see the actual way
    that the blood flows in his face.
  28. And we can do that not just
    to visualize the pulse,
  29. but also to actually recover
    our heart rates,
  30. and measure our heart rates.
  31. And we can do it with regular cameras
    and without touching the patients.
  32. So here you see the pulse and heart rate
    we extracted from a neonatal baby
  33. from a video we took
    with a regular DSLR camera,
  34. and the heart rate measurement we get
  35. is as accurate as the one you'd get
    with a standard monitor in a hospital.
  36. And it doesn't even have to be
    a video we recorded.
  37. We can do it essentially
    with other videos as well.
  38. So I just took a short clip
    from "Batman Begins" here
  39. just to show Christian Bale's pulse.
  40. (Laughter)
  41. And you know, presumably
    he's wearing makeup,
  42. the lighting here is kind of challenging,
  43. but still, just from the video,
    we're able to extract his pulse
  44. and show it quite well.
  45. So how do we do all that?
  46. We basically analyze the changes
    in the light that are recorded
  47. at every pixel in the video over time,
  48. and then we crank up those changes.
  49. We make them bigger
    so that we can see them.
  50. The tricky part is that those signals,
  51. those changes that we're after,
    are extremely subtle,
  52. so we have to be very careful
    when you try to separate them
  53. from noise that always exists in videos.
  54. So we use some clever
    image processing techniques
  55. to get a very accurate measurement
    of the color at each pixel in the video,
  56. and then the way
    the color changes over time,
  57. and then we amplify those changes.
  58. We make them bigger to create those types
    of enhanced videos, or magnified videos,
  59. that actually show us those changes.
  60. But it turns out we can do that
    not just to show tiny changes in color,
  61. but also tiny motions,
  62. and that's because the light
    that gets recorded in our cameras
  63. will change not only if the color
    of the object changes,
  64. but also if the object moves.
  65. So this is my daughter
    when she was about two months old.
  66. It's a video I recorded
    about three years ago.
  67. And as new parents, we all want
    to make sure our babies are healthy,
  68. that they're breathing,
    that they're alive, of course.
  69. So I too got one of those baby monitors
  70. so that I could see my daughter
    when she was asleep.
  71. And this is pretty much what you'll see
    with a standard baby monitor.
  72. You can see the baby's sleeping,
  73. but there's not too much information
    there.
  74. There's not too much we can see.
  75. Wouldn't it be better,
    or more informative, or more useful,
  76. if instead we could look
    at the view like this.
  77. So here I took the motions
    and I magnified them 30 times,
  78. and then I could clearly see
    that my daughter
  79. was indeed alive and breathing.
  80. (Laughter)
  81. Here is a side-by-side comparison.
  82. So again, in the source video,
    in the original video,
  83. there's not too much we can see,
  84. but once we magnify the motions,
    the breathing becomes much more visible.
  85. And it turns out,
    there's a lot of phenomena
  86. we can reveal and magnify
    with our new motion microscope.
  87. We can see how our veins and arteries
    are pulsing in our bodies.
  88. We can see that our eyes
    are constantly moving
  89. in this wobbly motion.
  90. And that's actually my eye,
  91. and again this video was taken
    right after my daughter was born,
  92. so you can see I wasn't getting
    too much sleep. (Laughter)
  93. Even when a person is sitting still,
  94. there's a lot of information
    we can extract
  95. about their breathing patterns,
    small facial expressions.
  96. Maybe we could use those motions
  97. to tell us something about
    our thoughts or our emotions.
  98. We can also magnify
    small mechanical movements,
  99. like vibrations in engines,
  100. that can help engineers detect
    and diagnose machinery problems,
  101. or see how our buildings and structures
    sway in the wind and react to forces.
  102. Those are all things that our society
    knows how to measure in various ways,
  103. but measuring those motions is one thing,
  104. and actually seeing those motions
    as they happen
  105. is a whole different thing.
  106. And ever since we discovered
    this new technology,
  107. we made our code available online
  108. so that others could use
    and experiment with it.
  109. It's very simple to use.
  110. It can work on your own videos.
  111. Our collaborators at Quantum Research
    even created this nice website
  112. where you can upload your videos
    and process them online,
  113. so even if you don't have any experience
    in computer science or programming,
  114. you can still very easily experiment
    with this new microscope.
  115. And I'd like to show you
    just a couple of examples
  116. of what others have done with it.
  117. So this video was made
    by a YouTube user called Tamez85.
  118. I don't know who that user is,
  119. but he, or she, used our code
  120. to magnify small belly movements
    during pregnancy.
  121. It's kind of creepy.
  122. (Laughter)
  123. People have used it to magnify
    pulsing veins in their hands.
  124. And you know it's not real science
    unless you use guinea pigs,
  125. and apparently this guinea pig
    is called Tiffany,
  126. and this YouTube user claims
    it is the first rodent on Earth
  127. that was motion-magnified.
  128. You can also do some art with it.
  129. So this video was sent to me
    by a design student at Yale.
  130. She wanted to see
    if there's any difference
  131. in the way her classmates move.
  132. She made them all stand still,
    and then magnified their motions.
  133. It's like seeing still pictures
    come to life.
  134. And the nice thing with all those examples
  135. is that we had nothing to do with them.
  136. We just provided this new tool,
    a new way to look at the world,
  137. and then people find other interesting,
    new and creative ways of using it.
  138. But we didn't stop there.
  139. This tool not only allows us
    to look at the world in a new way,
  140. it also redefines what we can do
  141. and pushes the limits
    of what we can do with our cameras.
  142. So as scientists, we started wondering,
  143. what other types of physical phenomena
    produce tiny motions
  144. that we could now use
    our cameras to measure?
  145. And one such phenomenon
    that we focused on recently is sound.
  146. Sound, as we all know,
    is basically changes
  147. in air pressure
    that travel through the air.
  148. Those pressure waves hit objects
    and they create small vibrations in them,
  149. which is how we hear
    and how we record sound.
  150. But it turns out that sound
    also produces visual motions.
  151. Those are motions
    that are not visible to us
  152. but are visible to a camera
    with the right processing.
  153. So here are two examples.
  154. This is me demonstrating
    my great singing skills.
  155. (Singing)
  156. (Laughter)
  157. And I took a high-speed video
    of my throat while I was humming.
  158. Again, if you stare at that video,
  159. there's not too much
    you'll be able to see,
  160. but once we magnify the motions 100 times,
    we can see all the motions and ripples
  161. in the neck that are involved
    in producing the sound.
  162. That signal is there in that video.
  163. We also know that singers
    can break a wine glass
  164. if they hit the correct note.
  165. So here, we're going to play a note
  166. that's in the resonance frequency
    of that glass
  167. through a loudspeaker that's next to it.
  168. Once we play that note
    and magnify the motions 250 times,
  169. we can very clearly see
    how the glass vibrates
  170. and resonates in response to the sound.
  171. It's not something you're used to seeing
    every day.
  172. And we actually have the demo
    right outside set up,
  173. so I encourage you to stop by,
  174. and just play with it yourself,
    you can actually see it live.
  175. But this made us think.
    It gave us this crazy idea.
  176. Can we actually invert this process
    and recover sound from video
  177. by analyzing the tiny vibrations
    that sound waves create in objects,
  178. and essentially convert those
    back into the sounds that produced them.
  179. In this way, we can turn
    everyday objects into microphones.
  180. So that's exactly what we did.
  181. So here's an empty bag of chips
    that was lying on a table,
  182. and we're going to turn that bag of chips
    into a microphone
  183. by filming it with a video camera
  184. and analyzing the tiny motions
    that sound waves create in it.
  185. So here's the sound
    that we played in the room.
  186. (Music: "Mary Had a Little Lamb")
  187. And this is a high-speed video
    we recorded of that bag of chips.
  188. Again it's playing.
  189. There's no chance you'll be able
    to see anything going on in that video
  190. just by looking at it,
  191. but here's the sound we were able
    to recover just by analyzing
  192. the tiny motions in that video.
  193. (Music: "Mary Had a Little Lamb")
  194. I call it -- Thank you.
  195. (Applause)
  196. I call it the visual microphone.
  197. We actually extract audio signals
    from video signals.
  198. And just to give you a sense
    of the scale of the motions here,
  199. a pretty loud sound will cause
    that bag of chips
  200. to move less than a micrometer.
  201. That's one thousandth of a millimeter.
  202. That's how tiny the motions are
    that we are now able to pull out
  203. just by observing how light
    bounces off objects
  204. and gets recorded by our cameras.
  205. We can recover sounds
    from other objects, like plants.
  206. (Music: "Mary Had a Little Lamb")
  207. And we can recover speech as well.
  208. So here's a person speaking in a room.
  209. Voice: Mary had a little lamb
    whose fleece was white as snow,
  210. and everywhere that Mary went,
    that lamb was sure to go.
  211. Michael Rubinstein: And here's
    that speech again recovered
  212. just from this video
    of that same bag of chips.
  213. Voice: Mary had a little lamb
    whose fleece was white as snow,
  214. and everywhere that Mary went,
    that lamb was sure to go.
  215. MR: We used "Mary Had a Little Lamb"
  216. because those are said to be
    the first words
  217. that Thomas Edison spoke
    into his phonograph in 1877.
  218. It was one of the first
    sound recording devices in history.
  219. It basically directed the sounds
    onto a diaphragm
  220. that vibrated a needle that essentially
    engraved the sound on tinfoil
  221. that was wrapped around the cylinder.
  222. Here's a demonstration of recording
  223. and replaying sound
    with Edison's phonograph.
  224. (Video) Voice: Testing, testing,
    one two three.
  225. Mary had a little lamb
    whose fleece was white as snow,
  226. and everywhere that Mary went,
    the lamb was sure to go.
  227. Testing, testing, one two three.
  228. Mary had a little lamb
    whose fleece was white as snow,
  229. and everywhere that Mary went,
    the lamb was sure to go.
  230. MR: And now, 137 years later,
  231. we're able to get sound
    in pretty much similar quality
  232. but by just watching objects
    vibrate to sound with cameras,
  233. and we can even do that when the camera
  234. is 15 feet away from the object,
    behind soundproof glass.
  235. So this is the sound that we were able
    to recover in that case.
  236. Voice: Mary had a little lamb
    whose fleece was white as snow,
  237. and everywhere that Mary went,
    the lamb was sure to go.
  238. MR: And of course, surveillance is
    the first application that comes to mind.
  239. (Laughter)
  240. But it might actually be useful
    for other things as well.
  241. Maybe in the future,
    we'll be able to use it, for example,
  242. to recover sound across space,
  243. because sound can't travel
    in space, but light can.
  244. We've only just begun exploring
  245. other possible uses
    for this new technology.
  246. It lets us see physical processes
    that we know are there
  247. but that we've never been able
    to see with our own eyes until now.
  248. This is our team.
  249. Everything I showed you today
    is a result of a collaboration
  250. with this great group
    of people you see here,
  251. and I encourage you and welcome you
    to check out our website,
  252. try it out yourself,
  253. and join us in exploring
    this world of tiny motions.
  254. Thank you.
  255. (Applause)