[Script Info]
Title: 
[Events]
Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,After 13.8 billion years\Nof cosmic history,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,our universe has woken up
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and become aware of itself.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,From a small blue planet,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,tiny, concious parts of our universe\Nhave begun gazing out into the cosmos
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,with telescopes,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,discovering something humbling.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,We've discovered that our universe\Nis vastly grander
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,than our ancestor's imagined,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and that life seems to be an almost\Nimprectibly small [protobation]
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,on an otherwise dead universe.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,But we've also discovered\Nsomething inspiring,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,which is that the technology\Nwe're developing has the potential
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,to help life flourish like never before,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,not just for centuries\Nbut for billions of years,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and not just on Earth but throughout\Nmuch of this amazing cosmos.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,I think of the earliest life as life 1.0
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,because it was really dumb,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,like bacteria unable to learn\Nanything during its lifetime.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,I think of us humans as Life 2.0\Nbecause we can learn,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,which in nerdy geek speak
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,might think of as installing\Nnew software into our brains,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,like languages and job skills.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Life 3.0, which can design not only\Nits software but also its hardware
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,of course doesn't exist yet.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,But perhaps our technology\Nhas already made us life 2.1,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,with our artificial knees, pacemakers\Nand cochlear implants.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,So let's take a closer look\Nat our relationship with technology, OK?
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,As an example,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,the Apollo 11 Moon Mission was both\Nsuccessful and inspiring,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,showing that when we humans\Nuse technology wisely,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,we can accomplish things\Nthat our ancestors could only dream of.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,But there's an even more inspiring journey
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,propelled by something\Nmore powerful than rocket engines ...
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,with passengers who\Naren't just three astronauts
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,but all of humanity.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Let's talk about our collective\Njourney into the future
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,with artificial intelligence.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,My friend [Yan Tallan] likes to point out\Nthat just as with rocketry,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,it's not enough to make\Nour technology powerful.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,We also have to figure out,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,if we're going to be really ambitious,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,how to steer it and where\Nwe want to go with it.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,So let's talk about all three\Nartificial intelligences:
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,the power, the steering\Nand the destination.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Let's start with the power.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,I define intelligence very inclusively --
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,simply as our ability\Nto accomplish complex goals
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,because I want to include both\Nbiological and artificial intelligence
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and I want to avoid the silly\N[carbon]-chauvenism idea
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,that you can only be smart\Nif you're made of meat.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,It's really amazing how the power\Nof AI has grown recently.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Just think about it.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Not long ago,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,robots couldn't walk.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Now, they can do backflips.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Not long ago,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,we didn't have self-driving cars.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Now, we have self-flying rockets.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Not long ago,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,AI couldn't do face recognition.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Now, AI can generate fake faces
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and simulate your face saying stuff\Nthat you never said.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Not long ago,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,AI couldn't beat us at the game of Go.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Then, Google DeepMind's Alpha Zero AI\Ntook 3,000 years of human Go games
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and Go wisdom,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,ignored it all and became the world's best\Nplayer by just playing against itself.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,And the most impressive feat here\Nwasn't that it crushed human gamers,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,but that it crushed human AI researchers
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,who had spent decades hand-crafting\Ngameplaying software.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,And Alphazero crushed human AI researchers\Nnot just in GO but even at chess,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,which we have been working on since 1950.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,So all this amazing recent progress in AI\Nreally begs the question:
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,how far will it go?
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,I like to think about this question
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,in terms of this abstract\Nlandscape of tasks,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,where the elevation represents\Nhow hard it is for AI to do each task
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,at human level,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and the sea level represents\Nwhat AI can do today.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,The seal level is rising\Nas the AI improves,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,so there's a kind of global warming\Ngoing on here in the task landscape.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,And the obvious takeaway is to avoid\Ncareers at the waterfront --
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,(Laughter)
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,which will soon be\Nautomated and disrupted.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,But there's a much\Nbigger question as well.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,How high will the water end up rising?
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Will it eventually rise\Nto flood everything?
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Imagine human intelligence at all tasks.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,This is the definition\Nof artificial general intelligence --
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,AGI,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,which has been the holy grail\Nof AI research since its inception,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,but this definition,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,people will say, "Ah,\Nthere will always be jobs
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,that humans can do better than machines,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,are simply saying\Nthat we'll never get AGI.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Sure, we might still choose to have\Nsome human jobs
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,or to give humans income\Nand purpose with our jobs,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,but AGI will in any case transform\Nlife as we know it
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,with humans no longer being\Nthe most intelligent.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Now if the water level does reach AGI,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,then further AI progress will be driven\Nmainly not by humans but by AI,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,which means that there's a possiblity\Nthat further AI progress
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,could be way faster than the typical\Nhuman research and development
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,time scale of years,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,raising the controversial possibility\Nof an intelligence explosion
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,where recursively self-improving AI
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,rapidly leaves human\Nintelligence far behind,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,creating what's known\Nas super intelligence.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,All right, reality check:
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,are we going to get AGI any time soon?
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Some famous AI researchers\Nlike Rodney Brooks think
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,it won't happen for hundreds of years.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,But others, like Google DeepMind\Nfounder Demis Hassabis,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,are more optimistic
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and are working to try to make\Nit happen much sooner.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,And recent surveys have shown\Nthat most AI researchers
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,have actually shared Demis's optimism,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,expecting that we will get AGI\Nwithin decades,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,so within the lifetime of many of us,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,which begs the question --
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and then what?
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,What do we want the role of humans to be
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,if machines can do everything better\Nand cheaper than us?
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,The way I see it, we face a choice.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,One option is to be complacent.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,We can say, "Oh, let's just build machines\Nthat can do everything we can do
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and not worry about the consequences.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Come on, if we build technology\Nthat makes all humans obsolete,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,what could possibly go wrong?"
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,(Laughter)
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,But I think that would be\Nembarrassingly lame.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,I think we should be more ambitious --
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,in the spirit of TED.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Let's envision the truly inspiring\Nhigh-tech future
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and try to steer towards it.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,This brings us to the second part\Nof our rocket metaphor:
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,the steering.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,We're making AI more powerful,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,but how can we steer towards the future
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,where AI helps humanity flourish\Nrather than flounder?
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,To help with this,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,I co-founded the Future Life Institute.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,It's a small non-profit promoting\Nbeneficial technology use
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and our goal is simply\Nfor the future of life to exist
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and be as inspiring as possible.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,You know, I love technology.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Technology is why today is better\Nthan the stoneage.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,And I'm optimistic that we can create\Na really inspiring high-tech future,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,if --
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and this is a big if --
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,if we win the wisdom race --
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,the race between the growing\Npower of our technology
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and the growing wisdom\Nwith which we manage it.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,But this is going to require\Na change of strategy
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,because our old strategy has been\Nlearning from mistakes.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,We invented fire,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,screwed up a bunch of times,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,invented the fire extinguisher.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,(Laughter)
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,We invented the car,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,screwed up a bunch of times,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,invented the traffic light,\Nthe seatbelt and the airbag,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,but more powerful technology\Nlike nuclear weapons and AGI --
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,learning from mistakes is lousy strategy,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,don't you think?
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,(Laughter)
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,It's much better to be proactive\Nrather than be reactive;
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,plan ahead and get things\Nright the first time
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,because that might be\Nthe only time we'll get.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,But it is funny because\Nsometimes people tell me,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,"Max,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,ssshhhh,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,don't talk like that.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,That's Luddite scare-mongering."
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,But it's not scare-mongering.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,It's what we at MIT\Ncall safety engineering.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Think about it:
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,before NASA launched\Nthe Apollo 11 Mission,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,they systematically thought through\Neverything that could go wrong
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,when you put people on top of\Nexplosive fuel tanks
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and launched them somewhere\Nwhere no one could help them.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,And there was a lot that could go wrong.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Was that scare-mongering?
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,No.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,That's was precisely\Nthe safety engineering
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,that insured the success of the mission,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and that is precisely the strategy\NI think we should take with AGI.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Think through what can go wrong\Nto make sure it goes right.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,So in this spirit,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,we've organized conferences,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,bringing together leading AI researchers\Nand other thinkers
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,who discuss how to grow this wisdom\Nwe need to keep AI beneficial.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Our last conference\Nwas in Asilomar, California last year
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and produced this list of 23 principles
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,which have since been signed\Nby over 1,000 AI researchers
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and key industry leaders.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,And I want to tell you\Nabout three of these principles.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,One is that we should avoid an arms race\Nand lethal autonomous weapons.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,The idea here is that any science\Ncan be used for new ways of helping people
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,of new ways of harming people.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,For example, biology and chemistry\Nare much more likely to be used
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,for new medicines or new cures\Nthan for new ways of killing people,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,because biologist and chemists\Npushed hard --
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and successfully --
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,for bans on biological\Nand chemical weapons.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,And in the same spirit,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,most AI researchers want to stigmatize\Nand ban lethal autonomous weapons.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Another Asilomar AI principle
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,is that we should mitigate\NAI-fueled income inequality.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,I think that if we can grow\Nthe economic pie dramatically with AI,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and we still can't figure out how\Nto divide this pie
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,so that everyone gets better off,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,then shame on us.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,(Applause)
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,All right, now raise your hand\Nif your computer has ever crashed.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,(Laughter)
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Wow, that's a lot of hands.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Well, then you'll appreciate\Nthis principle
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,that we should invest much more\Nin the AI safety research,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,because as we put AI in charge\Nof more decisions and infrastructure,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,we need to figure out how to transform\Ntoday's buggy and hackable computers
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,into robust AI systems\Nthat we can really trust,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,because otherwise,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,all this awesome new technology\Ncan malfunction and harm us
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,or get hacked and be turned against us.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,And this AI safety work has to include\Nwork on AI value alignment,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,because the real threat\Nfrom AGI isn't malice,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,like in silly Hollywood movies,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,but competence.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,AGI accomplishing goals that just\Naren't aligned with ours.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,For example,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,when we humans drove\Nthe West African Black Rhino extinct,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,we didn't do it because we're a bunch\Nof evil rhinocerous haters,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,did we?
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,We did it because we were\Nsmarter than them
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and our goals weren't aligned with theirs.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,But AGI is by definition smarter than us,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,so to make sure that we don't put\Nourselves in the position of those rhinos
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,if we create AGI,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,we need to figure out how to make machines\Nunderstand our goals,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,adopt our goals
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and retain our goals.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,And whose goals should these be, anyway?
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Which goals should they be?
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,This brings us to the third part\Nof our rocket metaphor:
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,the destination.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,We're making AI more powerful,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,trying to figure out how to steer it,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,but where do we want to go with it?
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,This is the elephant in the room\Nthat almost nobody talks about --
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,not even here at TED --
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,because we're so fixated\Non short-term AI challenges.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Look, our species is trying\Nto build AGI,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,motivated by curiosity and economics,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,but what sort of future society\Nare we hoping for if we succeed?
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,We did an opinion poll on this recently,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and I was struck to see
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,that most people actually want us\Nto build super-intelligence:
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,AI that's vastly smarter\Nthan us in all ways.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,What there was the greatest agreement on\Nwas that we should be ambitious
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and help life spread into the cosmos,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,but there was much less agreement\Nabout who or what should be in charge.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,And I was actually quite amused
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,to see that there's some some people\Nwho want it to be just the machines.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,(Laughter)
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,And there was total disagreement\Nabout what the role of humans should be,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,even at the most basic level,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,so let's take a closer look\Nat possible futures
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,that we might choose to steer towards.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,So don't get be wrong here;
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,I'm not talking about space travel,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,merely about humanity's\Nmetaphorical journey into the future.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,So one option that some\Nof my AI colleagues like
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,is to build super-intelligence\Nand keep it under human control,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,like an enslaved god,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,disconnected from the internet
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and used to create unimaginable\Ntechnology and wealth
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,for whoever controls it.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,But [Lord Acton] warned us\Nthat power corrupts,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and absolute power corrupts absolutely,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,so you might worry that maybe\Nwe humans just aren't smart enough
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,or wise enough rather,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,to handle this much power.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Also, aside from any moral\Nqualms you might have
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,about enslaving superior minds,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,you might worry that maybe\Nthe super intelligence could outsmart us,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,break out
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and take over.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,But I also have colleauges who are fine\Nwith AI taking over
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and even causing human extinction,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,as long as we feel the the AIs\Nare our worthy descendants,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,like our children.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,But how would we know that they AIs\Nhave adopted our best values,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and aren't just unconscious zombies\Ntricking us into anthropomorphizing them?
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Also, shouldn't those people who don't\Nwant human extiction
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,have a say in the matter, too?
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Now, if you didn't like either\Nof those two high-tech options,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,it's important to remember\Nthat low-tech is suicide
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,from a cosmic perspective,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,because if we don't go far beyond\Ntoday's technology,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,the question isn't whether humanity\Nis going to go extinct,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,merely whether we're going to be\Ntaken out by the next killer asteroid,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,super volcano
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,or some other problem that better\Ntechnology could have solved.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,So, how about having\Nour cake and eating it ...
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,with AGI that's not enslaved
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,but treats us well because its values\Nare aligned with ours?
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,This is the gist of what Eleazer Yukowski\Nhas called "Friendly AI,"
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and if we can do this,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,it could be awesome.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,It could not only eliminate negative\Nexperiences like disease, poverty,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,crime and other suffering,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,but it could also give us\Nthe freedom to choose
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,from a fantastic new diversity\Nof positive experiences --
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,basically making us the masters\Nof our own destiny.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,So in summary,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,our situation with technology\Nis complicated,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,but the big picture is rather simple.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Most AI researchers expect AGI\Nwithin decades,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and if we just bumble\Ninto this unprepared,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,it will probably be the biggest\Nmistake in human history --
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,let's face it.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,It could enable brutal,\Nglobal dictatorship
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,with unprecedented inequality,\Nsurveillance and suffering,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and maybe even human extinction.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,But if we steer carefully,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,we could end up in a fantastic future,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,where everybody's better off:
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,the poor are richer,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,the rich are richer,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,everybody is healthy and free\Nto live out their dreams.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Now, hang on.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Do you folks want the future\Nthat's politically right or left?
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Do you want the pious society\Nwith strict moral rules,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,or do you an hedonistic free-for-all,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,more like Burning Man 24/7?
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Do you want beautiful beaches,\Nforests and lakes
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,or would you prefer to rearrange\Nsome of those atoms
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,with the computers and they\Ncan be vitual experiences?
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,With friendly AI,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,we could simply build\Nall of these societies
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and give people the freedom to choose\Nwhich one they want to live in,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,because we would no longer\Nbe limited by our intelligence,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,merely by the laws of physics.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,So the resources and space for this\Nwould be astronomical --
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,literally.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,So here's our choice.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,We can either be complacent\Nabout our future,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,taking as an article of blind faith
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,that any new technology\Nis guaranteed to be beneficial,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and just repeat that to ourselves\Nas a mantra over and over and over again
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,as we drift like a rudderless ship\Ntowards our own obsolesence.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Or we can be ambitious --
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,thinking hard about how\Nto steer our technology
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and where we want to go with it
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,to create the age of amazement.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,We're all here to celebrate\Nthe age of amazement,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and I feel that its essence should lie\Nin becoming not overpowered
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,but empowered by our technology.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Thank you.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,(Applause)