English subtitles

← If the blind could see | Alberto Rizzoli & Marita Cheng | TEDxMelbourne

Get Embed Code
9 Languages

Showing Revision 17 created 04/16/2017 by Denise RQ.

  1. Marita Cheng: When I was growing up,
  2. I had a family friend
    who became blind in his 20s.
  3. When we went out as a family,
    he would say to me,
  4. "Rita hold my hand, hold my arm,
    and tell me what you see."
  5. So I'd say, "There's
    some flowers here to the left.
  6. There's a gate here to the right.
  7. There's a mountain over in the distance."
  8. And he would say,
    "What color are those flowers?
  9. Can I use my hand,
    reach out, and touch them?
  10. Could you lead my hand to that?"
  11. I'd say, "Oh, they're pink, they're blue."
  12. And he'd say, "Tell me more,
    tell me more about what you can see.
  13. Share it with me."
  14. About eight months ago,
    Alberto and I decided to create an app
  15. to enable blind people
    to recognize their surroundings.
  16. We used something called
    convolutional neural networks,
  17. which is a computer system
    that's been trained on millions of images.
  18. It learns the features of a dog.
    It learns what a flower looks like.
  19. It learns a fork, a knife,
    everyday objects.
  20. Using this system,
    we created something called "Aipoly"
  21. that recognizes
    over 1,000 everyday objects.
  22. So, a blind person just needs
    to walk around with their phone,
  23. and put it over various objects,
    and it will say the name of the object.
  24. Using voice over, the phone can relay
    the word on the screen
  25. to that blind person,
  26. so they know exactly
    what's in front of them.
  27. (Applause)
  28. Since we released
    our application in January,
  29. we've had over a 100,000 downloads
    around the world.
  30. The app has been so popular
    we've translated it into seven languages.
  31. Alberto Rizzoli: After experiencing
    the technology the first time,
  32. our users kept asking us for more.
  33. We asked them
  34. to think of our technology
    as a superpower for a moment,
  35. something they could effortlessly
    evoke at any time,
  36. and gain understanding
    of what was in front of them.
  37. And surprisingly,
  38. nobody really wants
    X-ray vision or telescopic goggles,
  39. but what everyone wants
    is more information.
  40. It's not surprising
  41. because 60% of the information
    that we perceive comes through sight.
  42. It is the main tool that we use
  43. to understand our surroundings
    and often, to make decisions.
  44. If you're blind you must rely
    on other senses like touch or hearing,
  45. and you miss out
    on the lightning-fast identification
  46. that our brain and eyes do every second
    of every day, if you're a sighted person.
  47. We went
  48. to the Santa Clara Valley Blind Center,
  49. and we tried to build this superpower.
  50. We tried to see what kind
    of information people wanted,
  51. and it's simple things
    like whether a dish is clean or not,
  52. whether you can cross the street,
    what product am I looking at?
  53. Things that can lead to a decision,
  54. from a simple gaze, to an understanding
    of the situation in front of you.
  55. We asked what form factor
    people preferred, and we built it.
  56. So we put together
  57. some jawbone conductive headphones,
    a pair of sunglasses, and a tiny camera,
  58. and we asked our friends
    to think of a common situation
  59. in which they had to make
    many small decisions,
  60. and we told them
    we will be giving them the prototype,
  61. and taking them
    in the middle of that situation.
  62. Let's see how it went. (Video starts)
  63. [We asked blind individuals
  64. [what are the hardest things to do
    when visually impaired]
  65. I mean it takes me forever
    to go grocery-shopping.
  66. Even with someone helping me
    that has known me for years.
  67. I'll say, "What's in that cabinet?"
  68. Or develop a system,
    right to left, top to bottom.
  69. [So we took them grocery shopping
    with our technology]
  70. Computer: Oranges.
  71. Man: This is great. I'm really liking it.
  72. Apples, grapes, carrots.
  73. I'm looking, I'm looking.
  74. Computer: Lilies.
    Man: Lilies.
  75. Computer: Bouquet.
    Man: A bouquet, ahh!
  76. Computer: Roses, flowers.
  77. Man: Can I take these home?
  78. This is great.
  79. Computer: Roses.
    Man: Roses.
  80. Computer: Bouquet, tulips.
    Man: Tulips.
  81. Computer: Pineapple.
    Woman: It's a pineapple.
  82. Computer: Mango.
    Woman: Mango.
  83. Computer: M&Ms.
    Woman: M&Ms.
  84. Computer: Tic Tac.
    Woman: It just said, "Tic Tac."
  85. Computer: Tic Tac.
    Woman: Tic Tac.
  86. Computer: Paper note, calendar.
  87. Woman: Calendar, you got it.
  88. Wow, I didn't know what that was at all.
  89. Computer: Pretzels.
    Woman: Pretzels.
  90. Computer: Pretzels.
    Woman: It said, "Pretzels."
  91. Computer: Lipton tea.
    Woman 2: Lipton?
  92. Tea?
  93. Computer: Lipton teabags.
  94. Woman 2: It's like I'm seeing it,
    but I'm not, it's seeing it for me.
  95. Computer: Coffee mate.
    Woman 2: Mate; coffee mate.
  96. It didn't say "coffee,"
    but it kept saying "mate."
  97. Computer: Mate, mate.
  98. Man 2: I put on the glasses,
  99. and right away, it told me
    there was an apple, there were oranges,
  100. and there was this, and there was that,
    and it's like, "This is great!"
  101. Instant love.
  102. (Video ends)
  103. (Applause)
  104. AR: That little pair of glasses
    connected to their phones
  105. could identify four
    to 5,000 objects in real time.
  106. That's about the capacity
    of a five-year-old child.
  107. A simple accessory
    can now expand a person's perception
  108. to thousands of new possibilities.
  109. This is the power of marrying
    artificial and human intelligence,
  110. and the potential
    is still vastly untapped.
  111. This isn't going to be a revolution
    just because GPUs are getting faster,
  112. or the research is getting more open,
  113. but because the barriers of entry
    to impacting millions of lives
  114. for artificial intelligence
    are getting lower and lower.
  115. The Paralympic games
    are starting in a few weeks,
  116. an event where sheer force of will,
    training, and technology
  117. turn people with a disability
    into super humans.
  118. and so, too, will all ability to think,
    perceive, make decisions, and learn
  119. increase exponentially.
  120. You will be building
    the tools to make this happen.
  121. So tomorrow, with your morning coffee,
  122. take 40 minutes
    and try out a tutorial on deep learning.
  123. Build yourself a small superpower.
  124. All it takes is your laptop,
    and a bunch of data,
  125. like your holiday pictures.
  126. Superpower engineer -
    that's a great dream job.
  127. The good news is that the world
    needs many, many more of them
  128. so we can't wait to see
    what you will be building next.
  129. Thank you.
  130. (Applause)