English subtitles

← The danger of AI is weirder than you think

Get Embed Code
32 Languages

Showing Revision 8 created 01/06/2020 by Erin Gregory.

  1. So, artificial intelligence
  2. is known for disrupting
    all kinds of industries.
  3. What about ice cream?
  4. What kind of mind-blowing
    new flavors could we generate
  5. with the power of an advanced
    artificial intelligence?
  6. So I teamed up with a group of coders
    from Kealing Middle School
  7. to find out the answer to this question.
  8. They collected over 1,600
    existing ice cream flavors,
  9. and together, we fed them to an algorithm
    to see what it would generate.
  10. And here are some of the flavors
    that the AI came up with.
  11. [Pumpkin Trash Break]

  12. (Laughter)

  13. [Peanut Butter Slime]

  14. [Strawberry Cream Disease]

  15. (Laughter)

  16. These flavors are not delicious,
    as we might have hoped they would be.

  17. So the question is: What happened?
  18. What went wrong?
  19. Is the AI trying to kill us?
  20. Or is it trying to do what we asked,
    and there was a problem?
  21. In movies, when something
    goes wrong with AI,

  22. it's usually because the AI has decided
  23. that it doesn't want to obey
    the humans anymore,
  24. and it's got its own goals,
    thank you very much.
  25. In real life, though,
    the AI that we actually have
  26. is not nearly smart enough for that.
  27. It has the approximate computing power
  28. of an earthworm,
  29. or maybe at most a single honeybee,
  30. and actually, probably maybe less.
  31. Like, we're constantly learning
    new things about brains
  32. that make it clear how much our AIs
    don't measure up to real brains.
  33. So today's AI can do a task
    like identify a pedestrian in a picture,
  34. but it doesn't have a concept
    of what the pedestrian is
  35. beyond that it's a collection
    of lines and textures and things.
  36. It doesn't know what a human actually is.
  37. So will today's AI
    do what we ask it to do?
  38. It will if it can,
  39. but it might not do what we actually want.
  40. So let's say that you
    were trying to get an AI

  41. to take this collection of robot parts
  42. and assemble them into some kind of robot
    to get from Point A to Point B.
  43. Now, if you were going to try
    and solve this problem
  44. by writing a traditional-style
    computer program,
  45. you would give the program
    step-by-step instructions
  46. on how to take these parts,
  47. how to assemble them
    into a robot with legs
  48. and then how to use those legs
    to walk to Point B.
  49. But when you're using AI
    to solve the problem,
  50. it goes differently.
  51. You don't tell it
    how to solve the problem,
  52. you just give it the goal,
  53. and it has to figure out for itself
    via trial and error
  54. how to reach that goal.
  55. And it turns out that the way AI tends
    to solve this particular problem
  56. is by doing this:
  57. it assembles itself into a tower
    and then falls over
  58. and lands at Point B.
  59. And technically, this solves the problem.
  60. Technically, it got to Point B.
  61. The danger of AI is not that
    it's going to rebel against us,
  62. it's that it's going to do
    exactly what we ask it to do.
  63. So then the trick
    of working with AI becomes:
  64. How do we set up the problem
    so that it actually does what we want?
  65. So this little robot here
    is being controlled by an AI.

  66. The AI came up with a design
    for the robot legs
  67. and then figured out how to use them
    to get past all these obstacles.
  68. But when David Ha set up this experiment,
  69. he had to set it up
    with very, very strict limits
  70. on how big the AI
    was allowed to make the legs,
  71. because otherwise ...
  72. (Laughter)

  73. And technically, it got
    to the end of that obstacle course.

  74. So you see how hard it is to get AI
    to do something as simple as just walk.
  75. So seeing the AI do this,
    you may say, OK, no fair,

  76. you can't just be
    a tall tower and fall over,
  77. you have to actually, like,
    use legs to walk.
  78. And it turns out,
    that doesn't always work, either.
  79. This AI's job was to move fast.
  80. They didn't tell it that it had
    to run facing forward
  81. or that it couldn't use its arms.
  82. So this is what you get
    when you train AI to move fast,
  83. you get things like somersaulting
    and silly walks.
  84. It's really common.
  85. So is twitching along the floor in a heap.
  86. (Laughter)

  87. So in my opinion, you know what
    should have been a whole lot weirder

  88. is the "Terminator" robots.
  89. Hacking "The Matrix" is another thing
    that AI will do if you give it a chance.
  90. So if you train an AI in a simulation,
  91. it will learn how to do things like
    hack into the simulation's math errors
  92. and harvest them for energy.
  93. Or it will figure out how to move faster
    by glitching repeatedly into the floor.
  94. When you're working with AI,
  95. it's less like working with another human
  96. and a lot more like working
    with some kind of weird force of nature.
  97. And it's really easy to accidentally
    give AI the wrong problem to solve,
  98. and often we don't realize that
    until something has actually gone wrong.
  99. So here's an experiment I did,

  100. where I wanted the AI
    to copy paint colors,
  101. to invent new paint colors,
  102. given the list like the ones
    here on the left.
  103. And here's what the AI
    actually came up with.
  104. [Sindis Poop, Turdly, Suffer, Gray Pubic]

  105. (Laughter)

  106. So technically,

  107. it did what I asked it to.
  108. I thought I was asking it for,
    like, nice paint color names,
  109. but what I was actually asking it to do
  110. was just imitate the kinds
    of letter combinations
  111. that it had seen in the original.
  112. And I didn't tell it anything
    about what words mean,
  113. or that there are maybe some words
  114. that it should avoid using
    in these paint colors.
  115. So its entire world
    is the data that I gave it.
  116. Like with the ice cream flavors,
    it doesn't know about anything else.
  117. So it is through the data

  118. that we often accidentally tell AI
    to do the wrong thing.
  119. This is a fish called a tench.
  120. And there was a group of researchers
  121. who trained an AI to identify
    this tench in pictures.
  122. But then when they asked it
  123. what part of the picture it was actually
    using to identify the fish,
  124. here's what it highlighted.
  125. Yes, those are human fingers.
  126. Why would it be looking for human fingers
  127. if it's trying to identify a fish?
  128. Well, it turns out that the tench
    is a trophy fish,
  129. and so in a lot of pictures
    that the AI had seen of this fish
  130. during training,
  131. the fish looked like this.
  132. (Laughter)

  133. And it didn't know that the fingers
    aren't part of the fish.

  134. So you see why it is so hard
    to design an AI

  135. that actually can understand
    what it's looking at.
  136. And this is why designing
    the image recognition
  137. in self-driving cars is so hard,
  138. and why so many self-driving car failures
  139. are because the AI got confused.
  140. I want to talk about an example from 2016.
  141. There was a fatal accident when somebody
    was using Tesla's autopilot AI,
  142. but instead of using it on the highway
    like it was designed for,
  143. they used it on city streets.
  144. And what happened was,
  145. a truck drove out in front of the car
    and the car failed to brake.
  146. Now, the AI definitely was trained
    to recognize trucks in pictures.
  147. But what it looks like happened is
  148. the AI was trained to recognize
    trucks on highway driving,
  149. where you would expect
    to see trucks from behind.
  150. Trucks on the side is not supposed
    to happen on a highway,
  151. and so when the AI saw this truck,
  152. it looks like the AI recognized it
    as most likely to be a road sign
  153. and therefore, safe to drive underneath.
  154. Here's an AI misstep
    from a different field.

  155. Amazon recently had to give up
    on a résumé-sorting algorithm
  156. that they were working on
  157. when they discovered that the algorithm
    had learned to discriminate against women.
  158. What happened is they had trained it
    on example résumés
  159. of people who they had hired in the past.
  160. And from these examples, the AI learned
    to avoid the résumés of people
  161. who had gone to women's colleges
  162. or who had the word "women"
    somewhere in their resume,
  163. as in, "women's soccer team"
    or "Society of Women Engineers."
  164. The AI didn't know that it wasn't supposed
    to copy this particular thing
  165. that it had seen the humans do.
  166. And technically, it did
    what they asked it to do.
  167. They just accidentally asked it
    to do the wrong thing.
  168. And this happens all the time with AI.

  169. AI can be really destructive
    and not know it.
  170. So the AIs that recommend
    new content in Facebook, in YouTube,
  171. they're optimized to increase
    the number of clicks and views.
  172. And unfortunately, one way
    that they have found of doing this
  173. is to recommend the content
    of conspiracy theories or bigotry.
  174. The AIs themselves don't have any concept
    of what this content actually is,
  175. and they don't have any concept
    of what the consequences might be
  176. of recommending this content.
  177. So, when we're working with AI,

  178. it's up to us to avoid problems.
  179. And avoiding things going wrong,
  180. that may come down to
    the age-old problem of communication,
  181. where we as humans have to learn
    how to communicate with AI.
  182. We have to learn what AI
    is capable of doing and what it's not,
  183. and to understand that,
    with its tiny little worm brain,
  184. AI doesn't really understand
    what we're trying to ask it to do.
  185. So in other words, we have
    to be prepared to work with AI
  186. that's not the super-competent,
    all-knowing AI of science fiction.
  187. We have to be prepared to work with an AI
  188. that's the one that we actually have
    in the present day.
  189. And present-day AI is plenty weird enough.
  190. Thank you.

  191. (Applause)