[Script Info]
Title: 
[Events]
Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
Dialogue: 0,0:00:00.00,0:00:18.72,Default,,0000,0000,0000,,{\i1}34C3 preroll music{\i0}
Dialogue: 0,0:00:18.72,0:00:24.100,Default,,0000,0000,0000,,Herald: Humans of Congress, it is my\Npleasure to announce the next speaker.
Dialogue: 0,0:00:24.100,0:00:31.75,Default,,0000,0000,0000,,I was supposed to pick out a few awards or\Nsomething, to actually present what he's
Dialogue: 0,0:00:31.75,0:00:36.42,Default,,0000,0000,0000,,done in his life, but I can\Nonly say: he's one of us!
Dialogue: 0,0:00:36.42,0:00:40.15,Default,,0000,0000,0000,,{\i1}applause{\i0}
Dialogue: 0,0:00:40.15,0:00:43.28,Default,,0000,0000,0000,,Charles Stross!\N{\i1}ongoing applause{\i0}
Dialogue: 0,0:00:43.28,0:00:45.97,Default,,0000,0000,0000,,Charles Stross: Hi! Is this on?\NGood. Great.
Dialogue: 0,0:00:45.97,0:00:50.68,Default,,0000,0000,0000,,I'm really pleased to be here and I\Nwant to start by apologizing for my total
Dialogue: 0,0:00:50.68,0:00:57.63,Default,,0000,0000,0000,,lack of German. So this talk is gonna be in\NEnglish. Good morning. I'm Charlie Stross
Dialogue: 0,0:00:57.63,0:01:03.56,Default,,0000,0000,0000,,and it's my job to tell lies for money, or\Nrather, I write science fiction, much of
Dialogue: 0,0:01:03.56,0:01:07.86,Default,,0000,0000,0000,,it about on the future, which in recent\Nyears has become ridiculously hard to
Dialogue: 0,0:01:07.86,0:01:16.09,Default,,0000,0000,0000,,predict. In this talk I'm going to talk\Nabout why. Now our species, Homo sapiens
Dialogue: 0,0:01:16.09,0:01:21.99,Default,,0000,0000,0000,,sapiens, is about 300,000 years old. It\Nused to be about 200,000 years old,
Dialogue: 0,0:01:21.99,0:01:26.19,Default,,0000,0000,0000,,but it grew an extra 100,000\Nyears in the past year because of new
Dialogue: 0,0:01:26.19,0:01:31.33,Default,,0000,0000,0000,,archaeological discoveries, I mean, go\Nfigure. For all but the last three
Dialogue: 0,0:01:31.33,0:01:36.76,Default,,0000,0000,0000,,centuries or so - of that span, however -\Npredicting the future was really easy. If
Dialogue: 0,0:01:36.76,0:01:41.37,Default,,0000,0000,0000,,you were an average person - as opposed to\Nmaybe a king or a pope - natural disasters
Dialogue: 0,0:01:41.37,0:01:46.69,Default,,0000,0000,0000,,aside, everyday life 50 years in the\Nfuture would resemble everyday life 50
Dialogue: 0,0:01:46.69,0:01:56.09,Default,,0000,0000,0000,,years in your past. Let that sink in for a\Nbit. For 99.9% of human existence on this
Dialogue: 0,0:01:56.09,0:02:01.93,Default,,0000,0000,0000,,earth, the future was static. Then\Nsomething changed and the future began to
Dialogue: 0,0:02:01.93,0:02:06.60,Default,,0000,0000,0000,,shift increasingly rapidly, until, in the\Npresent day, things are moving so fast,
Dialogue: 0,0:02:06.60,0:02:13.44,Default,,0000,0000,0000,,it's barely possible to anticipate trends\Nfrom one month to the next. Now as an
Dialogue: 0,0:02:13.44,0:02:17.83,Default,,0000,0000,0000,,eminent computer scientist, Edsger Dijkstra\Nonce remarked, computer science is no more
Dialogue: 0,0:02:17.83,0:02:23.62,Default,,0000,0000,0000,,about computers than astronomy is about\Nbuilding big telescopes, the same can be
Dialogue: 0,0:02:23.62,0:02:28.76,Default,,0000,0000,0000,,said of my field of work, writing science\Nfiction, sci-fi is rarely about science
Dialogue: 0,0:02:28.76,0:02:33.69,Default,,0000,0000,0000,,and even more rarely about predicting the\Nfuture, but sometimes we dabble in
Dialogue: 0,0:02:33.69,0:02:42.14,Default,,0000,0000,0000,,Futurism and lately, Futurism has gotten\Nreally, really, weird. Now when I write a
Dialogue: 0,0:02:42.14,0:02:47.23,Default,,0000,0000,0000,,near future work of fiction, one set, say, a\Ndecade hence, there used to be a recipe I
Dialogue: 0,0:02:47.23,0:02:53.77,Default,,0000,0000,0000,,could follow, that worked eerily well. Simply put:\N90% of the next decade stuff is
Dialogue: 0,0:02:53.77,0:02:57.42,Default,,0000,0000,0000,,already here around us today.\NBuildings are designed to
Dialogue: 0,0:02:57.42,0:03:03.04,Default,,0000,0000,0000,,last many years, automobiles have a design\Nlife of about a decade, so half the cars on
Dialogue: 0,0:03:03.04,0:03:10.24,Default,,0000,0000,0000,,the road in 2027 are already there now -\Nthey're new. People? There'll be some new
Dialogue: 0,0:03:10.24,0:03:15.52,Default,,0000,0000,0000,,faces, aged 10 and under, and some older\Npeople will have died, but most of us
Dialogue: 0,0:03:15.52,0:03:22.84,Default,,0000,0000,0000,,adults will still be around, albeit older\Nand grayer, this is the 90% of a near
Dialogue: 0,0:03:22.84,0:03:30.82,Default,,0000,0000,0000,,future that's already here today. After\Nthe already existing 90%, another 9% of a
Dialogue: 0,0:03:30.82,0:03:35.65,Default,,0000,0000,0000,,near future a decade hence used to be\Neasily predictable: you look at trends
Dialogue: 0,0:03:35.65,0:03:39.97,Default,,0000,0000,0000,,dictated by physical limits, such as\NMoore's law and you look at Intel's road
Dialogue: 0,0:03:39.97,0:03:44.00,Default,,0000,0000,0000,,map and you use a bit of creative\Nextrapolation and you won't go too far
Dialogue: 0,0:03:44.00,0:03:52.51,Default,,0000,0000,0000,,wrong. If I predict - wearing my futurology\Nhat - that in 2027 LTE cellular phones will
Dialogue: 0,0:03:52.51,0:03:57.56,Default,,0000,0000,0000,,be ubiquitous, 5G will be available for\Nhigh bandwidth applications and there will be
Dialogue: 0,0:03:57.56,0:04:01.79,Default,,0000,0000,0000,,fallback to some kind of satellite data\Nservice at a price, you probably won't
Dialogue: 0,0:04:01.79,0:04:04.33,Default,,0000,0000,0000,,laugh at me.\NI mean, it's not like I'm predicting that
Dialogue: 0,0:04:04.33,0:04:08.11,Default,,0000,0000,0000,,airlines will fly slower and Nazis will\Ntake over the United States, is it ?
Dialogue: 0,0:04:08.11,0:04:09.90,Default,,0000,0000,0000,,{\i1}laughing{\i0}
Dialogue: 0,0:04:09.90,0:04:15.21,Default,,0000,0000,0000,,And therein lies the problem. There is\Nremaining 1% of what Donald Rumsfeld
Dialogue: 0,0:04:15.21,0:04:20.94,Default,,0000,0000,0000,,called the "unknown unknowns", what throws off\Nall predictions. As it happens, airliners
Dialogue: 0,0:04:20.94,0:04:26.06,Default,,0000,0000,0000,,today are slower than they were in the\N1970s and don't get me started about the Nazis,
Dialogue: 0,0:04:26.06,0:04:31.86,Default,,0000,0000,0000,,I mean, nobody in 2007 was expecting a Nazi\Nrevival in 2017, were they?
Dialogue: 0,0:04:31.86,0:04:37.32,Default,,0000,0000,0000,,Only this time, Germans get to be the good guys. \N{\i1}laughing, applause{\i0}
Dialogue: 0,0:04:37.32,0:04:42.26,Default,,0000,0000,0000,,So. My recipe for fiction set 10 years \Nin the future used to be:
Dialogue: 0,0:04:42.26,0:04:47.36,Default,,0000,0000,0000,,"90% is already here,\N9% is not here yet but predictable
Dialogue: 0,0:04:47.36,0:04:53.73,Default,,0000,0000,0000,,and 1% is 'who ordered that?'" But unfortunately\Nthe ratios have changed, I think we're now
Dialogue: 0,0:04:53.73,0:04:59.66,Default,,0000,0000,0000,,down to maybe 80% already here - climate\Nchange takes a huge toll on architecture -
Dialogue: 0,0:04:59.66,0:05:05.76,Default,,0000,0000,0000,,then 15% not here yet, but predictable and\Na whopping 5% of utterly unpredictable
Dialogue: 0,0:05:05.76,0:05:12.74,Default,,0000,0000,0000,,deep craziness. Now... before I carry on\Nwith this talk, I want to spend a minute or
Dialogue: 0,0:05:12.74,0:05:18.53,Default,,0000,0000,0000,,two ranting loudly and ruling out the\Nsingularity. Some of you might assume, that
Dialogue: 0,0:05:18.53,0:05:23.59,Default,,0000,0000,0000,,as the author of books like "Singularity\NSky" and "Accelerando",
Dialogue: 0,0:05:23.59,0:05:28.22,Default,,0000,0000,0000,,I expect an impending technological\Nsingularity,
Dialogue: 0,0:05:28.22,0:05:32.09,Default,,0000,0000,0000,,that we will develop self-improving\Nartificial intelligence and mind uploading
Dialogue: 0,0:05:32.09,0:05:35.41,Default,,0000,0000,0000,,and the whole wish list of transhumanist\Naspirations promoted by the likes of
Dialogue: 0,0:05:35.41,0:05:42.30,Default,,0000,0000,0000,,Ray Kurzweil, will come to pass. Unfortunately\Nthis isn't the case. I think transhumanism
Dialogue: 0,0:05:42.30,0:05:49.05,Default,,0000,0000,0000,,is a warmed-over Christian heresy. While\Nits adherents tend to be outspoken atheists,
Dialogue: 0,0:05:49.05,0:05:51.91,Default,,0000,0000,0000,,they can't quite escape from the\Nhistory that gave rise to our current
Dialogue: 0,0:05:51.91,0:05:57.22,Default,,0000,0000,0000,,Western civilization. Many of you are\Nfamiliar with design patterns, an approach
Dialogue: 0,0:05:57.22,0:06:01.56,Default,,0000,0000,0000,,to software engineering that focuses on\Nabstraction and simplification, in order
Dialogue: 0,0:06:01.56,0:06:07.73,Default,,0000,0000,0000,,to promote reusable code. When you look at\Nthe AI singularity as a narrative and
Dialogue: 0,0:06:07.73,0:06:11.79,Default,,0000,0000,0000,,identify the numerous places in their\Nstory where the phrase "and then a miracle
Dialogue: 0,0:06:11.79,0:06:18.79,Default,,0000,0000,0000,,happens" occur, it becomes apparent pretty\Nquickly, that they've reinvented Christiantiy.
Dialogue: 0,0:06:18.79,0:06:19.46,Default,,0000,0000,0000,,{\i1}applause{\i0}
Dialogue: 0,0:06:19.46,0:06:25.33,Default,,0000,0000,0000,,Indeed, the wellspring of\Ntoday's transhumanists draw in a long rich
Dialogue: 0,0:06:25.33,0:06:29.99,Default,,0000,0000,0000,,history of Russian philosophy, exemplified\Nby the russian orthodox theologian Nikolai
Dialogue: 0,0:06:29.99,0:06:37.17,Default,,0000,0000,0000,,Fyodorovich Fedorov by way of his disciple\NKonstantin Tsiolkovsky, whose derivation
Dialogue: 0,0:06:37.17,0:06:40.52,Default,,0000,0000,0000,,of a rocket equation makes him\Nessentially the father of modern space
Dialogue: 0,0:06:40.52,0:06:45.35,Default,,0000,0000,0000,,flight. Once you start probing the nether\Nregions of transhumanist forth and run
Dialogue: 0,0:06:45.35,0:06:49.80,Default,,0000,0000,0000,,into concepts like Roko's Basilisk - by the\Nway, any of you who didn't know about the
Dialogue: 0,0:06:49.80,0:06:54.12,Default,,0000,0000,0000,,Basilisk before, are now doomed to an\Neternity in AI hell, terribly sorry - you
Dialogue: 0,0:06:54.12,0:06:57.89,Default,,0000,0000,0000,,realize, they've mangled it to match some\Nof the nastier aspects of Presbyterian
Dialogue: 0,0:06:57.89,0:07:03.19,Default,,0000,0000,0000,,Protestantism. Now they basically invented\Noriginal sin and Satan in the guise of an
Dialogue: 0,0:07:03.19,0:07:09.27,Default,,0000,0000,0000,,AI that doesn't exist yet ,it's.. kind of\Npeculiar. Anyway, my take on the
Dialogue: 0,0:07:09.27,0:07:12.95,Default,,0000,0000,0000,,singularity is: What if something walks\Nlike a duck and quacks like a duck? It's
Dialogue: 0,0:07:12.95,0:07:18.46,Default,,0000,0000,0000,,probably a duck. And if it looks like a\Nreligion, it's probably a religion.
Dialogue: 0,0:07:18.46,0:07:23.28,Default,,0000,0000,0000,,I don't see much evidence for human-like,\Nself-directed artificial intelligences
Dialogue: 0,0:07:23.28,0:07:28.07,Default,,0000,0000,0000,,coming along any time soon, and a fair bit\Nof evidence, that nobody accepts and freaks
Dialogue: 0,0:07:28.07,0:07:32.15,Default,,0000,0000,0000,,in cognitive science departments, even\Nwant it. I mean, if we invented an AI
Dialogue: 0,0:07:32.15,0:07:35.94,Default,,0000,0000,0000,,that was like a human mind, it would do the\NAI equivalent of sitting on the sofa,
Dialogue: 0,0:07:35.94,0:07:39.46,Default,,0000,0000,0000,,munching popcorn and\Nwatching the Super Bowl all day.
Dialogue: 0,0:07:39.46,0:07:43.26,Default,,0000,0000,0000,,It wouldn't be much use to us.\N{\i1}laughter, applause{\i0}
Dialogue: 0,0:07:43.26,0:07:46.78,Default,,0000,0000,0000,,What we're getting instead,\Nis self-optimizing tools that defy
Dialogue: 0,0:07:46.78,0:07:51.22,Default,,0000,0000,0000,,human comprehension, but are not\Nin fact any more like our kind
Dialogue: 0,0:07:51.22,0:07:57.50,Default,,0000,0000,0000,,of intelligence than a Boeing 737 is like\Na seagull. Boeing 737s and seagulls both
Dialogue: 0,0:07:57.50,0:08:04.24,Default,,0000,0000,0000,,fly, Boeing 737s don't lay eggs and shit\Neverywhere. So I'm going to wash my hands
Dialogue: 0,0:08:04.24,0:08:09.59,Default,,0000,0000,0000,,of a singularity as a useful explanatory\Nmodel of the future without further ado.
Dialogue: 0,0:08:09.59,0:08:14.67,Default,,0000,0000,0000,,I'm one of those vehement atheists as well\Nand I'm gonna try and offer you a better
Dialogue: 0,0:08:14.67,0:08:20.96,Default,,0000,0000,0000,,model for what's happening to us. Now, as\Nmy fellow Scottish science fictional author
Dialogue: 0,0:08:20.96,0:08:27.13,Default,,0000,0000,0000,,Ken MacLeod likes to say "the secret\Nweapon of science fiction is history".
Dialogue: 0,0:08:27.13,0:08:31.23,Default,,0000,0000,0000,,History is, loosely speaking, is the written\Nrecord of what and how people did things
Dialogue: 0,0:08:31.23,0:08:36.53,Default,,0000,0000,0000,,in past times. Times that have slipped out\Nof our personal memories. We science
Dialogue: 0,0:08:36.53,0:08:41.10,Default,,0000,0000,0000,,fiction writers tend to treat history as a\Ngiant toy chest to raid, whenever we feel
Dialogue: 0,0:08:41.10,0:08:45.39,Default,,0000,0000,0000,,like telling a story. With a little bit of\Nhistory, it's really easy to whip up an
Dialogue: 0,0:08:45.39,0:08:49.02,Default,,0000,0000,0000,,entertaining yarn about a galactic empire\Nthat mirrors the development and decline
Dialogue: 0,0:08:49.02,0:08:53.53,Default,,0000,0000,0000,,of a Habsburg Empire or to respin the\NOctober Revolution as a tale of how Mars
Dialogue: 0,0:08:53.53,0:08:59.60,Default,,0000,0000,0000,,got its independence. But history is\Nuseful for so much more than that.
Dialogue: 0,0:08:59.60,0:09:05.38,Default,,0000,0000,0000,,It turns out, that our personal memories\Ndon't span very much time at all. I'm 53
Dialogue: 0,0:09:05.38,0:09:10.80,Default,,0000,0000,0000,,and I barely remember the 1960s. I only\Nremember the 1970s with the eyes of a 6 to
Dialogue: 0,0:09:10.80,0:09:17.58,Default,,0000,0000,0000,,16 year old. My father died this year,\Naged 93, and he'd just about remembered the
Dialogue: 0,0:09:17.58,0:09:22.77,Default,,0000,0000,0000,,1930s. Only those of my father's\Ngeneration directly remember the Great
Dialogue: 0,0:09:22.78,0:09:29.08,Default,,0000,0000,0000,,Depression and can compare it to the\N2007/08 global financial crisis directly.
Dialogue: 0,0:09:29.08,0:09:34.25,Default,,0000,0000,0000,,We Westerners tend to pay little attention\Nto cautionary tales told by 90-somethings.
Dialogue: 0,0:09:34.25,0:09:38.100,Default,,0000,0000,0000,,We're modern, we're change obsessed and we\Ntend to repeat our biggest social mistakes
Dialogue: 0,0:09:38.100,0:09:43.26,Default,,0000,0000,0000,,just as they slip out of living memory,\Nwhich means they recur on a timescale of
Dialogue: 0,0:09:43.26,0:09:47.68,Default,,0000,0000,0000,,70 to 100 years.\NSo if our personal memories are useless,
Dialogue: 0,0:09:47.68,0:09:52.33,Default,,0000,0000,0000,,we need a better toolkit\Nand history provides that toolkit.
Dialogue: 0,0:09:52.33,0:09:57.10,Default,,0000,0000,0000,,History gives us the perspective to see what\Nwent wrong in the past and to look for
Dialogue: 0,0:09:57.10,0:10:01.51,Default,,0000,0000,0000,,patterns and check to see whether those \Npatterns are recurring in the present.
Dialogue: 0,0:10:01.51,0:10:06.69,Default,,0000,0000,0000,,Looking in particular at the history of the past two\Nto four hundred years, that age of rapidly
Dialogue: 0,0:10:06.69,0:10:11.80,Default,,0000,0000,0000,,increasing change that I mentioned at the\Nbeginning. One glaringly obvious deviation
Dialogue: 0,0:10:11.80,0:10:16.93,Default,,0000,0000,0000,,from the norm of the preceding\N3000 centuries is obvious, and that's
Dialogue: 0,0:10:16.93,0:10:22.06,Default,,0000,0000,0000,,the development of artificial intelligence,\Nwhich happened no earlier than 1553 and no
Dialogue: 0,0:10:22.06,0:10:29.36,Default,,0000,0000,0000,,later than 1844. I'm talking of course\Nabout the very old, very slow AI's we call
Dialogue: 0,0:10:29.36,0:10:34.24,Default,,0000,0000,0000,,corporations. What lessons from the history\Nof a company can we draw that tell us
Dialogue: 0,0:10:34.24,0:10:38.49,Default,,0000,0000,0000,,about the likely behavior of the type of\Nartificial intelligence we're interested
Dialogue: 0,0:10:38.49,0:10:47.26,Default,,0000,0000,0000,,in here, today?\NWell. Need a mouthful of water.
Dialogue: 0,0:10:47.26,0:10:51.62,Default,,0000,0000,0000,,Let me crib from Wikipedia for a moment.
Dialogue: 0,0:10:51.62,0:10:56.33,Default,,0000,0000,0000,,Wikipedia: "In the late 18th\Ncentury, Stewart Kyd, the author of the
Dialogue: 0,0:10:56.33,0:11:02.80,Default,,0000,0000,0000,,first treatise on corporate law in English,\Ndefined a corporation as: 'a collection of
Dialogue: 0,0:11:02.80,0:11:08.29,Default,,0000,0000,0000,,many individuals united into one body,\Nunder a special denomination, having
Dialogue: 0,0:11:08.29,0:11:13.78,Default,,0000,0000,0000,,perpetual succession under an artificial\Nform, and vested, by policy of the law, with
Dialogue: 0,0:11:13.78,0:11:20.20,Default,,0000,0000,0000,,the capacity of acting, in several respects,\Nas an individual, enjoying privileges and
Dialogue: 0,0:11:20.20,0:11:24.15,Default,,0000,0000,0000,,immunities in common, and of exercising a\Nvariety of political rights, more or less
Dialogue: 0,0:11:24.15,0:11:28.51,Default,,0000,0000,0000,,extensive, according to the design of its\Ninstitution, or the powers conferred upon
Dialogue: 0,0:11:28.51,0:11:32.27,Default,,0000,0000,0000,,it, either at the time of its creation, or\Nat any subsequent period of its
Dialogue: 0,0:11:32.27,0:11:36.70,Default,,0000,0000,0000,,existence.'"\NThis was a late 18th century definition,
Dialogue: 0,0:11:36.70,0:11:42.51,Default,,0000,0000,0000,,sound like a piece of software to you?\NIn 1844, the British government passed the
Dialogue: 0,0:11:42.51,0:11:46.45,Default,,0000,0000,0000,,"Joint Stock Companies Act" which created\Na register of companies and allowed any
Dialogue: 0,0:11:46.45,0:11:51.36,Default,,0000,0000,0000,,legal person, for a fee, to register a\Ncompany which in turn existed as a
Dialogue: 0,0:11:51.36,0:11:55.87,Default,,0000,0000,0000,,separate legal person. Prior to that point,\Nit required a Royal Charter or an act of
Dialogue: 0,0:11:55.87,0:12:00.42,Default,,0000,0000,0000,,Parliament to create a company.\NSubsequently, the law was extended to limit
Dialogue: 0,0:12:00.42,0:12:04.68,Default,,0000,0000,0000,,the liability of individual shareholders\Nin event of business failure and then both
Dialogue: 0,0:12:04.68,0:12:08.63,Default,,0000,0000,0000,,Germany and the United States added their\Nown unique twists to what today we see is
Dialogue: 0,0:12:08.63,0:12:14.36,Default,,0000,0000,0000,,the doctrine of corporate personhood.\NNow, though plenty of other things that
Dialogue: 0,0:12:14.36,0:12:18.74,Default,,0000,0000,0000,,happened between the 16th and 21st centuries\Ndid change the shape of the world we live in.
Dialogue: 0,0:12:18.74,0:12:22.15,Default,,0000,0000,0000,,I've skipped the changes in\Nagricultural productivity that happened
Dialogue: 0,0:12:22.15,0:12:25.55,Default,,0000,0000,0000,,due to energy economics,\Nwhich finally broke the Malthusian trap
Dialogue: 0,0:12:25.55,0:12:29.13,Default,,0000,0000,0000,,our predecessors lived in. \NThis in turn broke the long-term
Dialogue: 0,0:12:29.13,0:12:32.68,Default,,0000,0000,0000,,cap on economic growth of about \N0.1% per year
Dialogue: 0,0:12:32.68,0:12:35.68,Default,,0000,0000,0000,,in the absence of famines, plagues and \Nwars and so on.
Dialogue: 0,0:12:35.68,0:12:38.79,Default,,0000,0000,0000,,I've skipped the germ theory of diseases \Nand the development of trade empires
Dialogue: 0,0:12:38.79,0:12:42.76,Default,,0000,0000,0000,,in the age of sail and gunpowder,\Nthat were made possible by advances
Dialogue: 0,0:12:42.76,0:12:45.12,Default,,0000,0000,0000,,in accurate time measurement.
Dialogue: 0,0:12:45.12,0:12:49.08,Default,,0000,0000,0000,,I've skipped the rise, and\Nhopefully decline, of the pernicious
Dialogue: 0,0:12:49.08,0:12:52.43,Default,,0000,0000,0000,,theory of scientific racism that\Nunderpinned Western colonialism and the
Dialogue: 0,0:12:52.43,0:12:57.22,Default,,0000,0000,0000,,slave trade. I've skipped the rise of\Nfeminism, the ideological position that
Dialogue: 0,0:12:57.22,0:13:02.47,Default,,0000,0000,0000,,women are human beings rather than\Nproperty and the decline of patriarchy.
Dialogue: 0,0:13:02.47,0:13:05.100,Default,,0000,0000,0000,,I've skipped the whole of the\NEnlightenment and the Age of Revolutions,
Dialogue: 0,0:13:05.100,0:13:09.41,Default,,0000,0000,0000,,but this is a technocratic.. technocentric\NCongress, so I want to frame this talk in
Dialogue: 0,0:13:09.41,0:13:14.87,Default,,0000,0000,0000,,terms of AI, which we all like to think we\Nunderstand. Here's the thing about these
Dialogue: 0,0:13:14.87,0:13:21.05,Default,,0000,0000,0000,,artificial persons we call corporations.\NLegally, they're people. They have goals,
Dialogue: 0,0:13:21.05,0:13:25.96,Default,,0000,0000,0000,,they operate in pursuit of these goals,\Nthey have a natural life cycle.
Dialogue: 0,0:13:25.96,0:13:33.23,Default,,0000,0000,0000,,In the 1950s, a typical U.S. corporation on the\NS&P 500 Index had a life span of 60 years.
Dialogue: 0,0:13:33.23,0:13:37.80,Default,,0000,0000,0000,,Today it's down to less than 20 years. \NThis is largely due to predation.
Dialogue: 0,0:13:37.80,0:13:41.66,Default,,0000,0000,0000,,Corporations are cannibals, they eat \None another.
Dialogue: 0,0:13:41.66,0:13:45.85,Default,,0000,0000,0000,,They're also hive super organisms\Nlike bees or ants.
Dialogue: 0,0:13:45.85,0:13:48.70,Default,,0000,0000,0000,,For the first century and a\Nhalf, they relied entirely on human
Dialogue: 0,0:13:48.70,0:13:52.40,Default,,0000,0000,0000,,employees for their internal operation,\Nbut today they're automating their
Dialogue: 0,0:13:52.40,0:13:57.11,Default,,0000,0000,0000,,business processes very rapidly. Each\Nhuman is only retained so long as they can
Dialogue: 0,0:13:57.11,0:14:01.34,Default,,0000,0000,0000,,perform their assigned tasks more\Nefficiently than a piece of software
Dialogue: 0,0:14:01.34,0:14:05.44,Default,,0000,0000,0000,,and they can all be replaced by another\Nhuman, much as the cells in our own bodies
Dialogue: 0,0:14:05.44,0:14:09.80,Default,,0000,0000,0000,,are functionally interchangeable and a\Ngroup of cells can - in extremis - often be
Dialogue: 0,0:14:09.80,0:14:14.51,Default,,0000,0000,0000,,replaced by a prosthetic device.\NTo some extent, corporations can be
Dialogue: 0,0:14:14.51,0:14:19.02,Default,,0000,0000,0000,,trained to serve of the personal desires of\Ntheir chief executives, but even CEOs can
Dialogue: 0,0:14:19.02,0:14:22.75,Default,,0000,0000,0000,,be dispensed with, if their activities\Ndamage the corporation, as Harvey
Dialogue: 0,0:14:22.75,0:14:26.39,Default,,0000,0000,0000,,Weinstein found out a couple of months\Nago.
Dialogue: 0,0:14:26.39,0:14:30.94,Default,,0000,0000,0000,,Finally, our legal environment today has\Nbeen tailored for the convenience of
Dialogue: 0,0:14:30.94,0:14:34.91,Default,,0000,0000,0000,,corporate persons, rather than human\Npersons, to the point where our governments
Dialogue: 0,0:14:34.91,0:14:40.38,Default,,0000,0000,0000,,now mimic corporations in many of our\Ninternal structures.
Dialogue: 0,0:14:40.38,0:14:43.94,Default,,0000,0000,0000,,So, to understand where we're going, we\Nneed to start by asking "What do our
Dialogue: 0,0:14:43.94,0:14:51.85,Default,,0000,0000,0000,,current actually existing AI overlords\Nwant?"
Dialogue: 0,0:14:51.85,0:14:56.40,Default,,0000,0000,0000,,Now, Elon Musk, who I believe you've\Nall heard of, has an obsessive fear of one
Dialogue: 0,0:14:56.40,0:14:59.75,Default,,0000,0000,0000,,particular hazard of artificial\Nintelligence, which he conceives of as
Dialogue: 0,0:14:59.75,0:15:03.76,Default,,0000,0000,0000,,being a piece of software that functions\Nlike a brain in a box, namely the
Dialogue: 0,0:15:03.76,0:15:09.86,Default,,0000,0000,0000,,Paperclip Optimizer or Maximizer.\NA Paperclip Maximizer is a term of art for
Dialogue: 0,0:15:09.86,0:15:14.52,Default,,0000,0000,0000,,a goal seeking AI that has a single\Npriority, e.g., maximizing the
Dialogue: 0,0:15:14.52,0:15:19.76,Default,,0000,0000,0000,,number of paperclips in the universe. The\NPaperclip Maximizer is able to improve
Dialogue: 0,0:15:19.76,0:15:24.11,Default,,0000,0000,0000,,itself in pursuit of its goal, but has no\Nability to vary its goal, so will
Dialogue: 0,0:15:24.12,0:15:27.60,Default,,0000,0000,0000,,ultimately attempt to convert all the\Nmetallic elements in the solar system into
Dialogue: 0,0:15:27.60,0:15:31.75,Default,,0000,0000,0000,,paperclips, even if this is obviously\Ndetrimental to the well-being of the
Dialogue: 0,0:15:31.75,0:15:36.36,Default,,0000,0000,0000,,humans who set it this goal. \NUnfortunately I don't think Musk
Dialogue: 0,0:15:36.36,0:15:41.17,Default,,0000,0000,0000,,is paying enough attention, \Nconsider his own companies.\N
Dialogue: 0,0:15:41.17,0:15:45.48,Default,,0000,0000,0000,,Tesla isn't a Paperclip Maximizer, it's a\Nbattery Maximizer.
Dialogue: 0,0:15:45.48,0:15:48.24,Default,,0000,0000,0000,,After all, a battery.. an\Nelectric car is a battery with wheels and
Dialogue: 0,0:15:48.24,0:15:54.20,Default,,0000,0000,0000,,seats. SpaceX is an orbital payload\NMaximizer, driving down the cost of space
Dialogue: 0,0:15:54.20,0:15:59.42,Default,,0000,0000,0000,,launches in order to encourage more sales\Nfor the service it provides. SolarCity is
Dialogue: 0,0:15:59.42,0:16:05.70,Default,,0000,0000,0000,,a photovoltaic panel maximizer and so on.\NAll three of the.. Musk's very own slow AIs
Dialogue: 0,0:16:05.70,0:16:09.26,Default,,0000,0000,0000,,are based on an architecture, designed to\Nmaximize return on shareholder
Dialogue: 0,0:16:09.26,0:16:13.18,Default,,0000,0000,0000,,investment, even if by doing so they cook\Nthe planet the shareholders have to live
Dialogue: 0,0:16:13.18,0:16:15.98,Default,,0000,0000,0000,,on or turn the entire thing into solar\Npanels.
Dialogue: 0,0:16:15.98,0:16:19.48,Default,,0000,0000,0000,,But hey, if you're Elon Musk, thats okay,\Nyou're gonna retire on Mars anyway.
Dialogue: 0,0:16:19.48,0:16:20.88,Default,,0000,0000,0000,,{\i1}laughing{\i0}
Dialogue: 0,0:16:20.88,0:16:25.14,Default,,0000,0000,0000,,By the way, I'm ragging on Musk in this\Ntalks, simply because he's the current
Dialogue: 0,0:16:25.14,0:16:28.60,Default,,0000,0000,0000,,opinionated tech billionaire, who thinks\Nfor disrupting a couple of industries
Dialogue: 0,0:16:28.60,0:16:33.58,Default,,0000,0000,0000,,entitles him to make headlines.\NIf this was 2007 and my focus slightly
Dialogue: 0,0:16:33.58,0:16:39.04,Default,,0000,0000,0000,,difference.. different, I'd be ragging on\NSteve Jobs and if we're in 1997 my target
Dialogue: 0,0:16:39.04,0:16:42.26,Default,,0000,0000,0000,,would be Bill Gates.\NDon't take it personally, Elon.
Dialogue: 0,0:16:42.26,0:16:44.11,Default,,0000,0000,0000,,{\i1}laughing{\i0}
Dialogue: 0,0:16:44.11,0:16:49.14,Default,,0000,0000,0000,,Back to topic. The problem of\Ncorporations is, that despite their overt
Dialogue: 0,0:16:49.14,0:16:53.96,Default,,0000,0000,0000,,goals, whether they make electric vehicles\Nor beer or sell life insurance policies,
Dialogue: 0,0:16:53.96,0:16:59.78,Default,,0000,0000,0000,,they all have a common implicit Paperclip\NMaximizer goal: to generate revenue. If
Dialogue: 0,0:16:59.78,0:17:03.73,Default,,0000,0000,0000,,they don't make money, they're eaten by a\Nbigger predator or they go bust. It's as
Dialogue: 0,0:17:03.73,0:17:07.93,Default,,0000,0000,0000,,vital to them as breathing is to us\Nmammals. They generally pursue their
Dialogue: 0,0:17:07.93,0:17:12.44,Default,,0000,0000,0000,,implicit goal - maximizing revenue - by\Npursuing their overt goal.
Dialogue: 0,0:17:12.44,0:17:16.51,Default,,0000,0000,0000,,But sometimes they try instead to\Nmanipulate their environment, to ensure
Dialogue: 0,0:17:16.51,0:17:22.97,Default,,0000,0000,0000,,that money flows to them regardless.\NHuman toolmaking culture has become very
Dialogue: 0,0:17:22.97,0:17:27.98,Default,,0000,0000,0000,,complicated over time. New technologies\Nalways come with an attached implicit
Dialogue: 0,0:17:27.98,0:17:32.87,Default,,0000,0000,0000,,political agenda that seeks to extend the\Nuse of the technology. Governments react
Dialogue: 0,0:17:32.87,0:17:37.23,Default,,0000,0000,0000,,to this by legislating to control new\Ntechnologies and sometimes we end up with
Dialogue: 0,0:17:37.23,0:17:41.63,Default,,0000,0000,0000,,industries actually indulging in legal\Nduels through the regulatory mechanism of
Dialogue: 0,0:17:41.63,0:17:49.58,Default,,0000,0000,0000,,law to determine, who prevails. For\Nexample, consider the automobile. You
Dialogue: 0,0:17:49.58,0:17:53.91,Default,,0000,0000,0000,,can't have mass automobile transport\Nwithout gas stations and fuel distribution
Dialogue: 0,0:17:53.91,0:17:57.16,Default,,0000,0000,0000,,pipelines.\NThese in turn require access to whoever
Dialogue: 0,0:17:57.16,0:18:01.18,Default,,0000,0000,0000,,owns the land the oil is extracted from\Nunder and before you know it, you end up
Dialogue: 0,0:18:01.18,0:18:06.66,Default,,0000,0000,0000,,with a permanent army in Iraq and a clamp\Ndictatorship in Saudi Arabia. Closer to
Dialogue: 0,0:18:06.66,0:18:12.38,Default,,0000,0000,0000,,home, automobiles imply jaywalking laws and\Ndrink-driving laws. They affect Town
Dialogue: 0,0:18:12.38,0:18:16.75,Default,,0000,0000,0000,,Planning regulations and encourage\Nsuburban sprawl, the construction of human
Dialogue: 0,0:18:16.75,0:18:21.43,Default,,0000,0000,0000,,infrastructure on a scale required by\Nautomobiles, not pedestrians.
Dialogue: 0,0:18:21.43,0:18:24.98,Default,,0000,0000,0000,,This in turn is bad for competing\Ntransport technologies, like buses or
Dialogue: 0,0:18:24.98,0:18:31.72,Default,,0000,0000,0000,,trams, which work best in cities with a\Nhigh population density. So to get laws
Dialogue: 0,0:18:31.72,0:18:35.40,Default,,0000,0000,0000,,that favour the automobile in place,\Nproviding an environment conducive to
Dialogue: 0,0:18:35.40,0:18:40.41,Default,,0000,0000,0000,,doing business, automobile companies spend\Nmoney on political lobbyists and when they
Dialogue: 0,0:18:40.41,0:18:46.62,Default,,0000,0000,0000,,can get away with it, on bribes. Bribery\Nneedn't be blatant of course. E.g.,
Dialogue: 0,0:18:46.62,0:18:51.93,Default,,0000,0000,0000,,the reforms of a British railway network\Nin the 1960s dismembered many branch lines
Dialogue: 0,0:18:51.93,0:18:56.41,Default,,0000,0000,0000,,and coincided with a surge in road\Nbuilding and automobile sales. These
Dialogue: 0,0:18:56.41,0:19:01.14,Default,,0000,0000,0000,,reforms were orchestrated by Transport\NMinister Ernest Marples, who was purely a
Dialogue: 0,0:19:01.14,0:19:05.95,Default,,0000,0000,0000,,politician. The fact that he accumulated a\Nconsiderable personal fortune during this
Dialogue: 0,0:19:05.95,0:19:10.27,Default,,0000,0000,0000,,period by buying shares in motorway\Nconstruction corporations, has nothing to
Dialogue: 0,0:19:10.27,0:19:18.12,Default,,0000,0000,0000,,do with it. So, no conflict of interest\Nthere - now if the automobile in industry
Dialogue: 0,0:19:18.12,0:19:23.23,Default,,0000,0000,0000,,can't be considered a pure Paperclip\NMaximizer... sorry, the automobile
Dialogue: 0,0:19:23.23,0:19:27.51,Default,,0000,0000,0000,,industry in isolation can't be considered\Na pure Paperclip Maximizer. You have to
Dialogue: 0,0:19:27.51,0:19:31.66,Default,,0000,0000,0000,,look at it in conjunction with the fossil\Nfuel industries, the road construction
Dialogue: 0,0:19:31.66,0:19:37.81,Default,,0000,0000,0000,,business, the accident insurance sector\Nand so on. When you do this, you begin to
Dialogue: 0,0:19:37.81,0:19:42.74,Default,,0000,0000,0000,,see the outline of a paperclip-maximizing\Necosystem that invades far-flung lands and
Dialogue: 0,0:19:42.74,0:19:47.17,Default,,0000,0000,0000,,grinds up and kills around one and a\Nquarter million people per year. That's
Dialogue: 0,0:19:47.17,0:19:50.95,Default,,0000,0000,0000,,the global death toll from automobile\Naccidents currently, according to the World
Dialogue: 0,0:19:50.95,0:19:56.43,Default,,0000,0000,0000,,Health Organization. It rivals the First\NWorld War on an ongoing permanent basis
Dialogue: 0,0:19:56.43,0:20:01.51,Default,,0000,0000,0000,,and these are all side effects of its\Ndrive to sell you a new car. Now,
Dialogue: 0,0:20:01.51,0:20:06.64,Default,,0000,0000,0000,,automobiles aren't of course a total\Nliability. Today's cars are regulated
Dialogue: 0,0:20:06.64,0:20:11.30,Default,,0000,0000,0000,,stringently for safety and, in theory, to\Nreduce toxic emissions. They're fast,
Dialogue: 0,0:20:11.30,0:20:16.99,Default,,0000,0000,0000,,efficient and comfortable. We can thank\Nlegal mandated regulations imposed by
Dialogue: 0,0:20:16.99,0:20:22.03,Default,,0000,0000,0000,,governments for this, of course. Go back\Nto the 1970s and cars didn't have crumple
Dialogue: 0,0:20:22.03,0:20:27.20,Default,,0000,0000,0000,,zones, go back to the 50s and they didn't\Ncome with seat belts as standard. In the
Dialogue: 0,0:20:27.20,0:20:33.69,Default,,0000,0000,0000,,1930s, indicators, turn signals and brakes\Non all four wheels were optional and your
Dialogue: 0,0:20:33.69,0:20:38.85,Default,,0000,0000,0000,,best hope of surviving a 50 km/h-crash was \Nto be thrown out of a car and land somewhere
Dialogue: 0,0:20:38.85,0:20:43.06,Default,,0000,0000,0000,,without breaking your neck.\NRegulator agencies are our current
Dialogue: 0,0:20:43.06,0:20:47.38,Default,,0000,0000,0000,,political system's tool of choice for\Npreventing Paperclip Maximizers from
Dialogue: 0,0:20:47.38,0:20:55.16,Default,,0000,0000,0000,,running amok. Unfortunately, regulators\Ndon't always work. The first failure mode
Dialogue: 0,0:20:55.16,0:20:59.73,Default,,0000,0000,0000,,of regulators that you need to be aware of\Nis regulatory capture, where regulatory
Dialogue: 0,0:20:59.73,0:21:05.04,Default,,0000,0000,0000,,bodies are captured by the industries they\Ncontrol. Ajit Pai, Head of American Federal
Dialogue: 0,0:21:05.04,0:21:09.46,Default,,0000,0000,0000,,Communications Commission, which just voted\Nto eliminate net neutrality rules in the
Dialogue: 0,0:21:09.46,0:21:14.10,Default,,0000,0000,0000,,U.S., has worked as Associate\NGeneral Counsel for Verizon Communications
Dialogue: 0,0:21:14.10,0:21:19.29,Default,,0000,0000,0000,,Inc, the largest current descendant of the\NBell Telephone system's monopoly. After
Dialogue: 0,0:21:19.29,0:21:24.69,Default,,0000,0000,0000,,the AT&T antitrust lawsuit, the Bell\Nnetwork was broken up into the seven baby
Dialogue: 0,0:21:24.69,0:21:32.28,Default,,0000,0000,0000,,bells. They've now pretty much reformed\Nand reaggregated and Verizon is the largest current one.
Dialogue: 0,0:21:32.28,0:21:36.10,Default,,0000,0000,0000,,Why should someone with a transparent\Ninterest in a technology corporation end
Dialogue: 0,0:21:36.10,0:21:41.13,Default,,0000,0000,0000,,up running a regulator that tries to\Ncontrol the industry in question? Well, if
Dialogue: 0,0:21:41.13,0:21:44.92,Default,,0000,0000,0000,,you're going to regulate a complex\Ntechnology, you need to recruit regulators
Dialogue: 0,0:21:44.92,0:21:48.58,Default,,0000,0000,0000,,from people who understand it.\NUnfortunately, most of those people are
Dialogue: 0,0:21:48.58,0:21:53.52,Default,,0000,0000,0000,,industry insiders. Ajit Pai is clearly\Nvery much aware of how Verizon is
Dialogue: 0,0:21:53.52,0:21:58.45,Default,,0000,0000,0000,,regulated, very insightful into its\Noperations and wants to do something about
Dialogue: 0,0:21:58.45,0:22:02.51,Default,,0000,0000,0000,,it - just not necessarily in the public\Ninterest.
Dialogue: 0,0:22:02.51,0:22:11.03,Default,,0000,0000,0000,,{\i1}applause{\i0}\NWhen regulators end up staffed by people
Dialogue: 0,0:22:11.03,0:22:15.37,Default,,0000,0000,0000,,drawn from the industries they're supposed\Nto control, they frequently end up working
Dialogue: 0,0:22:15.37,0:22:20.02,Default,,0000,0000,0000,,with their former office mates, to make it\Neasier to turn a profit, either by raising
Dialogue: 0,0:22:20.02,0:22:24.35,Default,,0000,0000,0000,,barriers to keep new insurgent companies\Nout or by dismantling safeguards that
Dialogue: 0,0:22:24.35,0:22:31.61,Default,,0000,0000,0000,,protect the public. Now a second problem\Nis regulatory lag where a technology
Dialogue: 0,0:22:31.61,0:22:35.24,Default,,0000,0000,0000,,advances so rapidly, that regulations are\Nlaughably obsolete by the time they're
Dialogue: 0,0:22:35.24,0:22:40.45,Default,,0000,0000,0000,,issued. Consider the EU directive\Nrequiring cookie notices on websites to
Dialogue: 0,0:22:40.45,0:22:45.71,Default,,0000,0000,0000,,caution users, that their activities are\Ntracked and their privacy may be violated.
Dialogue: 0,0:22:45.71,0:22:51.42,Default,,0000,0000,0000,,This would have been a good idea in 1993\Nor 1996, but unfortunatelly it didn't show up
Dialogue: 0,0:22:51.42,0:22:57.69,Default,,0000,0000,0000,,until 2011. Fingerprinting and tracking\Nmechanisms have nothing to do with cookies
Dialogue: 0,0:22:57.69,0:23:04.15,Default,,0000,0000,0000,,and were already widespread by then. Tim\NBerners-Lee observed in 1995, that five
Dialogue: 0,0:23:04.15,0:23:07.78,Default,,0000,0000,0000,,years worth of change was happening on the\Nweb for every 12 months of real-world
Dialogue: 0,0:23:07.78,0:23:12.39,Default,,0000,0000,0000,,time. By that yardstick, the cookie law\Ncame out nearly a century too late to do
Dialogue: 0,0:23:12.39,0:23:19.27,Default,,0000,0000,0000,,any good. Again, look at Uber. This month,\Nthe European Court of Justice ruled that
Dialogue: 0,0:23:19.27,0:23:24.52,Default,,0000,0000,0000,,Uber is a taxi service, not a Web App. This\Nis arguably correct - the problem is, Uber
Dialogue: 0,0:23:24.52,0:23:28.97,Default,,0000,0000,0000,,has spread globally since it was founded\Neight years ago, subsidizing its drivers to
Dialogue: 0,0:23:28.97,0:23:33.58,Default,,0000,0000,0000,,put competing private hire firms out of\Nbusiness. Whether this is a net good for
Dialogue: 0,0:23:33.58,0:23:38.66,Default,,0000,0000,0000,,societys own is debatable. The problem is, a\Ntaxi driver can get awfully hungry if she
Dialogue: 0,0:23:38.66,0:23:42.43,Default,,0000,0000,0000,,has to wait eight years for a court ruling\Nagainst a predator intent on disrupting
Dialogue: 0,0:23:42.43,0:23:49.55,Default,,0000,0000,0000,,her business. So, to recap: firstly, we\Nalready have Paperclip Maximizers and
Dialogue: 0,0:23:49.55,0:23:54.98,Default,,0000,0000,0000,,Musk's AI alarmism is curiously mirror\Nblind. Secondly, we have mechanisms for
Dialogue: 0,0:23:54.98,0:23:59.51,Default,,0000,0000,0000,,keeping Paperclip Maximizers in check, but\Nthey don't work very well against AIs that
Dialogue: 0,0:23:59.51,0:24:03.49,Default,,0000,0000,0000,,deploy the dark arts, especially\Ncorruption and bribery and they're even
Dialogue: 0,0:24:03.49,0:24:07.77,Default,,0000,0000,0000,,worse against true AIs, that evolved too\Nfast for human mediated mechanisms like
Dialogue: 0,0:24:07.77,0:24:13.53,Default,,0000,0000,0000,,the law to keep up with. Finally, unlike\Nthe naive vision of a Paperclip Maximizer
Dialogue: 0,0:24:13.53,0:24:19.19,Default,,0000,0000,0000,,that maximizes only paperclips, existing\NAIs have multiple agendas, their overt
Dialogue: 0,0:24:19.19,0:24:24.21,Default,,0000,0000,0000,,goal, but also profit seeking, expansion\Ninto new markets and to accommodate the
Dialogue: 0,0:24:24.21,0:24:27.50,Default,,0000,0000,0000,,desire of whoever is currently in the\Ndriving seat.
Dialogue: 0,0:24:27.50,0:24:30.50,Default,,0000,0000,0000,,{\i1}sighs{\i0}
Dialogue: 0,0:24:30.50,0:24:36.19,Default,,0000,0000,0000,,Now, this brings me to the next major\Nheading in this dismaying laundry list:
Dialogue: 0,0:24:36.19,0:24:42.56,Default,,0000,0000,0000,,how it all went wrong. It seems to me that\Nour current political upheavals, the best
Dialogue: 0,0:24:42.56,0:24:49.38,Default,,0000,0000,0000,,understood, is arising from the capture\Nof post 1917 democratic institutions by
Dialogue: 0,0:24:49.38,0:24:54.81,Default,,0000,0000,0000,,large-scale AI. Everywhere you look, you\Nsee voters protesting angrily against an
Dialogue: 0,0:24:54.81,0:24:58.78,Default,,0000,0000,0000,,entrenched establishment, that seems\Ndetermined to ignore the wants and needs
Dialogue: 0,0:24:58.78,0:25:04.33,Default,,0000,0000,0000,,of their human constituents in favor of\Nthose of the machines. The brexit upset
Dialogue: 0,0:25:04.33,0:25:07.13,Default,,0000,0000,0000,,was largely result of a protest vote\Nagainst the British political
Dialogue: 0,0:25:07.13,0:25:11.34,Default,,0000,0000,0000,,establishment, the election of Donald\NTrump likewise, with a side order of racism
Dialogue: 0,0:25:11.34,0:25:15.94,Default,,0000,0000,0000,,on top. Our major political parties are\Nled by people who are compatible with the
Dialogue: 0,0:25:15.94,0:25:20.79,Default,,0000,0000,0000,,system as it exists today, a system that\Nhas been shaped over decades by
Dialogue: 0,0:25:20.79,0:25:26.01,Default,,0000,0000,0000,,corporations distorting our government and\Nregulatory environments. We humans live in
Dialogue: 0,0:25:26.01,0:25:30.70,Default,,0000,0000,0000,,a world shaped by the desires and needs of\NAI, forced to live on their terms and we're
Dialogue: 0,0:25:30.70,0:25:34.01,Default,,0000,0000,0000,,taught, that we're valuable only to the\Nextent we contribute to the rule of the
Dialogue: 0,0:25:34.01,0:25:40.26,Default,,0000,0000,0000,,machines. Now this is free sea and we're\Nall more interested in computers and
Dialogue: 0,0:25:40.26,0:25:44.39,Default,,0000,0000,0000,,communications technology than this\Nhistorical crap. But as I said earlier,
Dialogue: 0,0:25:44.39,0:25:49.15,Default,,0000,0000,0000,,history is a secret weapon, if you know how\Nto use it. What history is good for, is
Dialogue: 0,0:25:49.15,0:25:53.08,Default,,0000,0000,0000,,enabling us to spot recurring patterns\Nthat repeat across timescales outside our
Dialogue: 0,0:25:53.08,0:25:58.13,Default,,0000,0000,0000,,personal experience. And if we look at our\Nhistorical very slow AIs, what do we learn
Dialogue: 0,0:25:58.13,0:26:04.88,Default,,0000,0000,0000,,from them about modern AI and how it's\Ngoing to behave? Well to start with, our
Dialogue: 0,0:26:04.88,0:26:09.88,Default,,0000,0000,0000,,AIs have been warped, the new AIs,\Nthe electronic one's instantiated in our
Dialogue: 0,0:26:09.88,0:26:14.64,Default,,0000,0000,0000,,machines, have been warped by a terrible\Nfundamentally flawed design decision back
Dialogue: 0,0:26:14.64,0:26:20.29,Default,,0000,0000,0000,,in 1995, but as damaged democratic\Npolitical processes crippled our ability
Dialogue: 0,0:26:20.29,0:26:24.57,Default,,0000,0000,0000,,to truly understand the world around us\Nand led to the angry upheavals and upsets
Dialogue: 0,0:26:24.57,0:26:30.44,Default,,0000,0000,0000,,of our present decade. That mistake was\Nthe decision, to fund the build-out of a
Dialogue: 0,0:26:30.44,0:26:34.37,Default,,0000,0000,0000,,public World Wide Web as opposed to be\Nearlier government-funded corporate and
Dialogue: 0,0:26:34.37,0:26:38.23,Default,,0000,0000,0000,,academic Internet by\Nmonetizing eyeballs through advertising
Dialogue: 0,0:26:38.23,0:26:45.26,Default,,0000,0000,0000,,revenue. The ad-supported web we're used\Nto today wasn't inevitable. If you recall
Dialogue: 0,0:26:45.26,0:26:49.51,Default,,0000,0000,0000,,the web as it was in 1994, there were very\Nfew ads at all and not much, in a way, of
Dialogue: 0,0:26:49.51,0:26:55.72,Default,,0000,0000,0000,,Commerce. 1995 was the year, the World Wide\NWeb really came to public attention in the
Dialogue: 0,0:26:55.72,0:27:00.57,Default,,0000,0000,0000,,anglophone world and consumer-facing\Nwebsites began to appear. Nobody really
Dialogue: 0,0:27:00.57,0:27:04.49,Default,,0000,0000,0000,,knew, how this thing was going to be paid\Nfor. The original .com bubble was all
Dialogue: 0,0:27:04.49,0:27:07.85,Default,,0000,0000,0000,,about working out, how to monetize the web\Nfor the first time and a lot of people
Dialogue: 0,0:27:07.85,0:27:12.84,Default,,0000,0000,0000,,lost their shirts in the process. A naive\Ninitial assumption was that the
Dialogue: 0,0:27:12.84,0:27:17.44,Default,,0000,0000,0000,,transaction cost of setting up a tcp/ip\Nconnection over modem was too high to
Dialogue: 0,0:27:17.44,0:27:22.50,Default,,0000,0000,0000,,support.. to be supported by per-use micro\Nbilling for web pages. So instead of
Dialogue: 0,0:27:22.50,0:27:27.06,Default,,0000,0000,0000,,charging people fraction of a euro cent\Nfor every page view, we'd bill customers
Dialogue: 0,0:27:27.06,0:27:31.57,Default,,0000,0000,0000,,indirectly, by shoving advertising banners\Nin front of their eyes and hoping they'd
Dialogue: 0,0:27:31.57,0:27:39.03,Default,,0000,0000,0000,,click through and buy something.\NUnfortunately, advertising is in an
Dialogue: 0,0:27:39.03,0:27:45.51,Default,,0000,0000,0000,,industry, one of those pre-existing very\Nslow AI ecosystems I already alluded to.
Dialogue: 0,0:27:45.51,0:27:49.70,Default,,0000,0000,0000,,Advertising tries to maximize its hold on\Nthe attention of the minds behind each
Dialogue: 0,0:27:49.70,0:27:53.75,Default,,0000,0000,0000,,human eyeball. The coupling of advertising\Nwith web search was an inevitable
Dialogue: 0,0:27:53.75,0:27:57.99,Default,,0000,0000,0000,,outgrowth, I mean how better to attract\Nthe attention of reluctant subjects, than to
Dialogue: 0,0:27:57.99,0:28:01.45,Default,,0000,0000,0000,,find out what they're really interested in\Nseeing and selling ads that relate to
Dialogue: 0,0:28:01.45,0:28:07.07,Default,,0000,0000,0000,,those interests. The problem of applying\Nthe paperclip maximize approach to
Dialogue: 0,0:28:07.07,0:28:12.51,Default,,0000,0000,0000,,monopolizing eyeballs, however, is that\Neyeballs are a limited, scarce resource.
Dialogue: 0,0:28:12.51,0:28:18.50,Default,,0000,0000,0000,,There are only 168 hours in every week, in\Nwhich I can gaze at banner ads. Moreover,
Dialogue: 0,0:28:18.50,0:28:22.29,Default,,0000,0000,0000,,most ads are irrelevant to my interests and\Nit doesn't matter, how often you flash an ad
Dialogue: 0,0:28:22.29,0:28:28.13,Default,,0000,0000,0000,,for dog biscuits at me, I'm never going to\Nbuy any. I have a cat. To make best
Dialogue: 0,0:28:28.13,0:28:31.99,Default,,0000,0000,0000,,revenue-generating use of our eyeballs,\Nit's necessary for the ad industry to
Dialogue: 0,0:28:31.99,0:28:36.79,Default,,0000,0000,0000,,learn, who we are and what interests us and\Nto target us increasingly minutely in hope
Dialogue: 0,0:28:36.79,0:28:39.55,Default,,0000,0000,0000,,of hooking us with stuff we're attracted\Nto.
Dialogue: 0,0:28:39.55,0:28:43.88,Default,,0000,0000,0000,,In other words: the ad industry is a\Npaperclip maximizer, but for its success,
Dialogue: 0,0:28:43.88,0:28:49.99,Default,,0000,0000,0000,,it relies on developing a theory of mind\Nthat applies to human beings.
Dialogue: 0,0:28:49.99,0:28:52.99,Default,,0000,0000,0000,,{\i1}sighs{\i0}
Dialogue: 0,0:28:52.99,0:28:55.74,Default,,0000,0000,0000,,Do I need to divert on to the impassioned \Nrant about the hideous corruption
Dialogue: 0,0:28:55.74,0:28:59.76,Default,,0000,0000,0000,,and evil that is Facebook?\NAudience: Yes!
Dialogue: 0,0:28:59.76,0:29:03.29,Default,,0000,0000,0000,,CS: Okay, somebody said yes.\NI'm guessing you've heard it all before,
Dialogue: 0,0:29:03.29,0:29:07.43,Default,,0000,0000,0000,,but for too long don't read.. summary is:\NFacebook is as much a search engine as
Dialogue: 0,0:29:07.43,0:29:12.03,Default,,0000,0000,0000,,Google or Amazon. Facebook searches are\Noptimized for faces, that is for human
Dialogue: 0,0:29:12.03,0:29:15.65,Default,,0000,0000,0000,,beings. If you want to find someone you\Nfell out of touch with thirty years ago,
Dialogue: 0,0:29:15.65,0:29:20.61,Default,,0000,0000,0000,,Facebook probably knows where they live,\Nwhat their favorite color is, what sized
Dialogue: 0,0:29:20.61,0:29:24.39,Default,,0000,0000,0000,,shoes they wear and what they said about\Nyou to your friends behind your back all
Dialogue: 0,0:29:24.39,0:29:30.09,Default,,0000,0000,0000,,those years ago, that made you cut them off.\NEven if you don't have a Facebook account,
Dialogue: 0,0:29:30.09,0:29:34.23,Default,,0000,0000,0000,,Facebook has a You account, a hole in their\Nsocial graph of a bunch of connections
Dialogue: 0,0:29:34.23,0:29:38.67,Default,,0000,0000,0000,,pointing in to it and your name tagged on\Nyour friends photographs. They know a lot
Dialogue: 0,0:29:38.67,0:29:42.53,Default,,0000,0000,0000,,about you and they sell access to their\Nsocial graph to advertisers, who then
Dialogue: 0,0:29:42.53,0:29:47.40,Default,,0000,0000,0000,,target you, even if you don't think you use\NFacebook. Indeed, there is barely any
Dialogue: 0,0:29:47.40,0:29:52.27,Default,,0000,0000,0000,,point in not using Facebook these days, if\Never. Social media Borg: "Resistance is
Dialogue: 0,0:29:52.27,0:30:00.51,Default,,0000,0000,0000,,futile!" So however, Facebook is trying to\Nget eyeballs on ads, so is Twitter and so
Dialogue: 0,0:30:00.51,0:30:05.97,Default,,0000,0000,0000,,are Google. To do this, they fine-tuned the\Ncontent they show you to make it more
Dialogue: 0,0:30:05.97,0:30:11.52,Default,,0000,0000,0000,,attractive to your eyes and by attractive\NI do not mean pleasant. We humans have an
Dialogue: 0,0:30:11.52,0:30:15.14,Default,,0000,0000,0000,,evolved automatic reflex to pay attention\Nto threats and horrors as well as
Dialogue: 0,0:30:15.14,0:30:19.83,Default,,0000,0000,0000,,pleasurable stimuli and the algorithms,\Nthat determine what they show us when we
Dialogue: 0,0:30:19.83,0:30:24.26,Default,,0000,0000,0000,,look at Facebook or Twitter, take this bias\Ninto account. You might react more
Dialogue: 0,0:30:24.26,0:30:28.49,Default,,0000,0000,0000,,strongly to a public hanging in Iran or an\Noutrageous statement by Donald Trump than
Dialogue: 0,0:30:28.49,0:30:32.10,Default,,0000,0000,0000,,to a couple kissing. The algorithm knows\Nand will show you whatever makes you pay
Dialogue: 0,0:30:32.10,0:30:38.39,Default,,0000,0000,0000,,attention, not necessarily what you need or\Nwant to see.
Dialogue: 0,0:30:38.39,0:30:42.51,Default,,0000,0000,0000,,So this brings me to another point about\Ncomputerized AI as opposed to corporate
Dialogue: 0,0:30:42.51,0:30:47.26,Default,,0000,0000,0000,,AI. AI algorithms tend to embody the\Nprejudices and beliefs of either the
Dialogue: 0,0:30:47.26,0:30:52.76,Default,,0000,0000,0000,,programmers, or the data set \Nthe AI was trained on.\N
Dialogue: 0,0:30:52.76,0:30:56.25,Default,,0000,0000,0000,,A couple of years ago I ran across an\Naccount of a webcam, developed by mostly\N
Dialogue: 0,0:30:56.25,0:31:00.86,Default,,0000,0000,0000,,pale-skinned Silicon Valley engineers, that\Nhad difficulty focusing or achieving correct
Dialogue: 0,0:31:00.86,0:31:04.50,Default,,0000,0000,0000,,color balance, when pointed at dark-skinned\Nfaces.
Dialogue: 0,0:31:04.50,0:31:08.40,Default,,0000,0000,0000,,Fast an example of human programmer\Ninduced bias, they didn't have a wide
Dialogue: 0,0:31:08.40,0:31:13.25,Default,,0000,0000,0000,,enough test set and didn't recognize that\Nthey were inherently biased towards
Dialogue: 0,0:31:13.25,0:31:19.29,Default,,0000,0000,0000,,expecting people to have pale skin. But\Nwith today's deep learning, bias can creep
Dialogue: 0,0:31:19.29,0:31:24.01,Default,,0000,0000,0000,,in, while the datasets for neural networks are\Ntrained on, even without the programmers
Dialogue: 0,0:31:24.01,0:31:28.66,Default,,0000,0000,0000,,intending it. Microsoft's first foray into\Na conversational chat bot driven by
Dialogue: 0,0:31:28.66,0:31:33.33,Default,,0000,0000,0000,,machine learning, Tay, was what we yanked\Noffline within days last year, because
Dialogue: 0,0:31:33.33,0:31:37.15,Default,,0000,0000,0000,,4chan and reddit based trolls discovered,\Nthat they could train it towards racism and
Dialogue: 0,0:31:37.15,0:31:43.65,Default,,0000,0000,0000,,sexism for shits and giggles. Just imagine\Nyou're a poor naive innocent AI who's just
Dialogue: 0,0:31:43.65,0:31:48.48,Default,,0000,0000,0000,,been switched on and you're hoping to pass\Nyour Turing test and what happens? 4chan
Dialogue: 0,0:31:48.48,0:31:53.19,Default,,0000,0000,0000,,decide to play with your head.\N{\i1}laughing{\i0}
Dialogue: 0,0:31:53.19,0:31:57.52,Default,,0000,0000,0000,,I got to feel sorry for Tay.\NNow, humans may be biased,
Dialogue: 0,0:31:57.52,0:32:01.06,Default,,0000,0000,0000,,but at least individually we're\Naccountable and if somebody gives you
Dialogue: 0,0:32:01.06,0:32:06.50,Default,,0000,0000,0000,,racist or sexist abuse to your face, you\Ncan complain or maybe punch them. It's
Dialogue: 0,0:32:06.50,0:32:10.74,Default,,0000,0000,0000,,impossible to punch a corporation and it\Nmay not even be possible to identify the
Dialogue: 0,0:32:10.74,0:32:16.03,Default,,0000,0000,0000,,source of unfair bias, when you're dealing\Nwith a machine learning system. AI based
Dialogue: 0,0:32:16.03,0:32:21.74,Default,,0000,0000,0000,,systems that instantiate existing\Nprejudices make social change harder.
Dialogue: 0,0:32:21.74,0:32:25.47,Default,,0000,0000,0000,,Traditional advertising works by playing\Non the target customer's insecurity and
Dialogue: 0,0:32:25.47,0:32:30.75,Default,,0000,0000,0000,,fear as much as their aspirations. And fear\Nof a loss of social status and privileges
Dialogue: 0,0:32:30.75,0:32:36.24,Default,,0000,0000,0000,,are powerful stress. Fear and xenophobia\Nare useful tools for tracking advertising..
Dialogue: 0,0:32:36.24,0:32:39.58,Default,,0000,0000,0000,,ah, eyeballs.\NWhat happens when we get pervasive social
Dialogue: 0,0:32:39.58,0:32:44.38,Default,,0000,0000,0000,,networks, that have learned biases against\Nsay Feminism or Islam or melanin? Or deep
Dialogue: 0,0:32:44.38,0:32:48.23,Default,,0000,0000,0000,,learning systems, trained on datasets\Ncontaminated by racist dipshits and their
Dialogue: 0,0:32:48.23,0:32:53.37,Default,,0000,0000,0000,,propaganda? Deep learning systems like the\Nones inside Facebook, that determine which
Dialogue: 0,0:32:53.37,0:32:58.21,Default,,0000,0000,0000,,stories to show you to get you to pay as\Nmuch attention as possible to be adverse.
Dialogue: 0,0:32:58.21,0:33:04.84,Default,,0000,0000,0000,,I think, you probably have an inkling of\Nhow.. where this is now going. Now, if you
Dialogue: 0,0:33:04.84,0:33:08.99,Default,,0000,0000,0000,,think, this is sounding a bit bleak and\Nunpleasant, you'd be right. I write sci-fi.
Dialogue: 0,0:33:08.99,0:33:12.67,Default,,0000,0000,0000,,You read or watch or play sci-fi. We're\Nacculturated to think of science and
Dialogue: 0,0:33:12.67,0:33:19.42,Default,,0000,0000,0000,,technology as good things that make life\Nbetter, but this ain't always so. Plenty of
Dialogue: 0,0:33:19.42,0:33:23.14,Default,,0000,0000,0000,,technologies have historically been\Nheavily regulated or even criminalized for
Dialogue: 0,0:33:23.14,0:33:27.69,Default,,0000,0000,0000,,good reason and once you get past any\Nreflexive indignation, criticism of
Dialogue: 0,0:33:27.69,0:33:32.74,Default,,0000,0000,0000,,technology and progress, you might agree\Nwith me, that it is reasonable to ban
Dialogue: 0,0:33:32.74,0:33:39.14,Default,,0000,0000,0000,,individuals from owning nuclear weapons or\Nnerve gas. Less obviously, they may not be
Dialogue: 0,0:33:39.14,0:33:42.82,Default,,0000,0000,0000,,weapons, but we've banned\Nchlorofluorocarbon refrigerants, because
Dialogue: 0,0:33:42.82,0:33:45.57,Default,,0000,0000,0000,,they were building up in the high\Nstratosphere and destroying the ozone
Dialogue: 0,0:33:45.57,0:33:51.36,Default,,0000,0000,0000,,layer that protects us from UVB radiation.\NWe banned tetra e-file LED in
Dialogue: 0,0:33:51.36,0:33:57.54,Default,,0000,0000,0000,,gasoline, because it poisoned people and\Nled to a crime wave. These are not
Dialogue: 0,0:33:57.54,0:34:03.24,Default,,0000,0000,0000,,weaponized technologies, but they have\Nhorrible side effects. Now, nerve gas and
Dialogue: 0,0:34:03.24,0:34:08.69,Default,,0000,0000,0000,,leaded gasoline were 1930s chemical\Ntechnologies, promoted by 1930s
Dialogue: 0,0:34:08.69,0:34:15.39,Default,,0000,0000,0000,,corporations. Halogenated refrigerants and\Nnuclear weapons are totally 1940s. ICBMs
Dialogue: 0,0:34:15.39,0:34:19.21,Default,,0000,0000,0000,,date to the 1950s. You know, I have\Ndifficulty seeing why people are getting
Dialogue: 0,0:34:19.21,0:34:26.04,Default,,0000,0000,0000,,so worked up over North Korea. North Korea\Nreaches 1953 level parity - be terrified
Dialogue: 0,0:34:26.04,0:34:30.76,Default,,0000,0000,0000,,and hide under the bed!\NI submit that the 21st century is throwing
Dialogue: 0,0:34:30.76,0:34:35.03,Default,,0000,0000,0000,,up dangerous new technologies, just as our\Nexisting strategies for regulating very
Dialogue: 0,0:34:35.03,0:34:42.06,Default,,0000,0000,0000,,slow AIs have proven to be inadequate. And\NI don't have an answer to how we regulate
Dialogue: 0,0:34:42.06,0:34:45.89,Default,,0000,0000,0000,,new technologies, I just want to flag it up\Nas a huge social problem that is going to
Dialogue: 0,0:34:45.89,0:34:50.38,Default,,0000,0000,0000,,affect the coming century.\NI'm now going to give you four examples of
Dialogue: 0,0:34:50.38,0:34:54.29,Default,,0000,0000,0000,,new types of AI application that are\Ngoing to warp our societies even more
Dialogue: 0,0:34:54.29,0:35:00.80,Default,,0000,0000,0000,,badly than the old slow AIs, we.. have done.\NThis isn't an exhaustive list, this is just
Dialogue: 0,0:35:00.80,0:35:05.16,Default,,0000,0000,0000,,some examples I dream, I pulled out of\Nmy ass. We need to work out a general
Dialogue: 0,0:35:05.16,0:35:08.43,Default,,0000,0000,0000,,strategy for getting on top of this sort\Nof thing before they get on top of us and
Dialogue: 0,0:35:08.43,0:35:12.04,Default,,0000,0000,0000,,I think, this is actually a very urgent\Nproblem. So I'm just going to give you this
Dialogue: 0,0:35:12.04,0:35:18.11,Default,,0000,0000,0000,,list of dangerous new technologies that\Nare arriving now, or coming, and send you
Dialogue: 0,0:35:18.11,0:35:22.15,Default,,0000,0000,0000,,away to think about what to do next. I\Nmean, we are activists here, we should be
Dialogue: 0,0:35:22.15,0:35:27.69,Default,,0000,0000,0000,,thinking about this and planning what\Nto do. Now, the first nasty technology I'd
Dialogue: 0,0:35:27.69,0:35:31.60,Default,,0000,0000,0000,,like to talk about, is political hacking\Ntools that rely on social graph directed
Dialogue: 0,0:35:31.60,0:35:39.50,Default,,0000,0000,0000,,propaganda. This is low-hanging fruit\Nafter the electoral surprises of 2016.
Dialogue: 0,0:35:39.50,0:35:43.28,Default,,0000,0000,0000,,Cambridge Analytica pioneered the use of\Ndeep learning by scanning the Facebook and
Dialogue: 0,0:35:43.28,0:35:47.75,Default,,0000,0000,0000,,Twitter social graphs to identify voters\Npolitical affiliations by simply looking
Dialogue: 0,0:35:47.75,0:35:52.95,Default,,0000,0000,0000,,at what tweets or Facebook comments they\Nliked, very able to do this, to identify
Dialogue: 0,0:35:52.95,0:35:56.40,Default,,0000,0000,0000,,individuals with a high degree of\Nprecision, who were vulnerable to
Dialogue: 0,0:35:56.40,0:36:01.49,Default,,0000,0000,0000,,persuasion and who lived in electorally\Nsensitive districts. They then canvassed
Dialogue: 0,0:36:01.49,0:36:06.91,Default,,0000,0000,0000,,them with propaganda, that targeted their\Npersonal hot-button issues to change their
Dialogue: 0,0:36:06.91,0:36:12.06,Default,,0000,0000,0000,,electoral intentions. The tools developed\Nby web advertisers to sell products have
Dialogue: 0,0:36:12.06,0:36:16.26,Default,,0000,0000,0000,,now been weaponized for political purposes\Nand the amount of personal information
Dialogue: 0,0:36:16.26,0:36:21.26,Default,,0000,0000,0000,,about our affiliations that we expose on\Nsocial media, makes us vulnerable. Aside, in
Dialogue: 0,0:36:21.26,0:36:24.97,Default,,0000,0000,0000,,the last U.S. Presidential election, as\Nmounting evidence for the British
Dialogue: 0,0:36:24.97,0:36:28.51,Default,,0000,0000,0000,,referendum on leaving the EU was subject\Nto foreign cyber war attack, now
Dialogue: 0,0:36:28.51,0:36:33.25,Default,,0000,0000,0000,,weaponized social media, as was the most\Nrecent French Presidential election.
Dialogue: 0,0:36:33.25,0:36:38.08,Default,,0000,0000,0000,,In fact, if we remember the leak of emails\Nfrom the Macron campaign, it turns out that
Dialogue: 0,0:36:38.08,0:36:42.02,Default,,0000,0000,0000,,many of those emails were false, because\Nthe Macron campaign anticipated that they
Dialogue: 0,0:36:42.02,0:36:47.12,Default,,0000,0000,0000,,would be attacked and an email trove would\Nbe leaked in the last days before the
Dialogue: 0,0:36:47.12,0:36:51.26,Default,,0000,0000,0000,,election. So they deliberately set up\Nfalse emails that would be hacked and then
Dialogue: 0,0:36:51.26,0:37:00.61,Default,,0000,0000,0000,,leaked and then could be discredited. It\Ngets twisty fast. Now I'm kind of biting
Dialogue: 0,0:37:00.61,0:37:04.65,Default,,0000,0000,0000,,my tongue and trying, not to take sides\Nhere. I have my own political affiliation
Dialogue: 0,0:37:04.65,0:37:09.77,Default,,0000,0000,0000,,after all, and I'm not terribly mainstream.\NBut if social media companies don't work
Dialogue: 0,0:37:09.77,0:37:14.22,Default,,0000,0000,0000,,out how to identify and flag micro-\Ntargeted propaganda, then democratic
Dialogue: 0,0:37:14.22,0:37:18.14,Default,,0000,0000,0000,,institutions will stop working and elections\Nwill be replaced by victories, whoever
Dialogue: 0,0:37:18.14,0:37:23.20,Default,,0000,0000,0000,,can buy the most trolls. This won't\Nsimply be billionaires but.. like the Koch
Dialogue: 0,0:37:23.20,0:37:26.47,Default,,0000,0000,0000,,brothers and Robert Mercer from the U.S.\Nthrowing elections to whoever will
Dialogue: 0,0:37:26.47,0:37:31.12,Default,,0000,0000,0000,,hand them the biggest tax cuts. Russian\Nmilitary cyber war doctrine calls for the
Dialogue: 0,0:37:31.12,0:37:35.73,Default,,0000,0000,0000,,use of social media to confuse and disable\Nperceived enemies, in addition to the
Dialogue: 0,0:37:35.73,0:37:39.90,Default,,0000,0000,0000,,increasingly familiar use of zero-day\Nexploits for espionage, such as spear
Dialogue: 0,0:37:39.90,0:37:43.20,Default,,0000,0000,0000,,phishing and distributed denial-of-service\Nattacks, on our infrastructure, which are
Dialogue: 0,0:37:43.20,0:37:49.26,Default,,0000,0000,0000,,practiced by Western agencies. Problem is,\Nonce the Russians have demonstrated that
Dialogue: 0,0:37:49.26,0:37:53.99,Default,,0000,0000,0000,,this is an effective tactic, the use of\Npropaganda bot armies in cyber war will go
Dialogue: 0,0:37:53.99,0:38:00.40,Default,,0000,0000,0000,,global. And at that point, our social\Ndiscourse will be irreparably poisoned.
Dialogue: 0,0:38:00.40,0:38:04.99,Default,,0000,0000,0000,,Incidentally, I'd like to add - as another\Naside like the Elon Musk thing - I hate
Dialogue: 0,0:38:04.99,0:38:09.68,Default,,0000,0000,0000,,the cyber prefix! It usually indicates,\Nthat whoever's using it has no idea what
Dialogue: 0,0:38:09.68,0:38:15.94,Default,,0000,0000,0000,,they're talking about.\N{\i1}applause, laughter{\i0}
Dialogue: 0,0:38:15.94,0:38:21.12,Default,,0000,0000,0000,,Unfortunately, much as the way the term\Nhacker was corrupted from its original
Dialogue: 0,0:38:21.12,0:38:27.02,Default,,0000,0000,0000,,meaning in the 1990s, the term cyber war\Nhas, it seems, to have stuck and it's now an
Dialogue: 0,0:38:27.02,0:38:31.50,Default,,0000,0000,0000,,actual thing that we can point to and say:\N"This is what we're talking about". So I'm
Dialogue: 0,0:38:31.50,0:38:35.51,Default,,0000,0000,0000,,afraid, we're stuck with this really\Nhorrible term. But that's a digression, I
Dialogue: 0,0:38:35.51,0:38:38.78,Default,,0000,0000,0000,,should get back on topic, because I've only\Ngot 20 minutes to go.
Dialogue: 0,0:38:38.78,0:38:46.24,Default,,0000,0000,0000,,Now, the second threat that we need to\Nthink about regulating ,or controlling, is
Dialogue: 0,0:38:46.24,0:38:50.03,Default,,0000,0000,0000,,an adjunct to deep learning target\Npropaganda: it's the use of neural network
Dialogue: 0,0:38:50.03,0:38:56.94,Default,,0000,0000,0000,,generated false video media. We're used to\Nphotoshopped images these days, but faking
Dialogue: 0,0:38:56.94,0:39:02.51,Default,,0000,0000,0000,,video and audio takes it to the next\Nlevel. Luckily, faking video and audio is
Dialogue: 0,0:39:02.51,0:39:08.56,Default,,0000,0000,0000,,labor-intensive, isn't it? Well nope, not\Nanymore. We're seeing the first generation
Dialogue: 0,0:39:08.56,0:39:13.51,Default,,0000,0000,0000,,of AI assisted video porn, in which the\Nfaces of film stars are mapped onto those
Dialogue: 0,0:39:13.51,0:39:17.37,Default,,0000,0000,0000,,of other people in a video clip, using\Nsoftware rather than laborious in human
Dialogue: 0,0:39:17.37,0:39:22.49,Default,,0000,0000,0000,,process.\NA properly trained neural network
Dialogue: 0,0:39:22.49,0:39:29.43,Default,,0000,0000,0000,,recognizes faces and transforms the face\Nof the Hollywood star, they want to put
Dialogue: 0,0:39:29.43,0:39:35.19,Default,,0000,0000,0000,,into a porn movie, into the face of - onto\Nthe face of the porn star in the porn clip
Dialogue: 0,0:39:35.19,0:39:40.52,Default,,0000,0000,0000,,and suddenly you have "Oh dear God, get it\Nout of my head" - no, not gonna give you
Dialogue: 0,0:39:40.52,0:39:43.50,Default,,0000,0000,0000,,any examples. Let's just say it's bad\Nstuff.
Dialogue: 0,0:39:43.50,0:39:47.26,Default,,0000,0000,0000,,{\i1}laughs{\i0}\NMeanwhile we have WaveNet, a system
Dialogue: 0,0:39:47.26,0:39:51.12,Default,,0000,0000,0000,,for generating realistic sounding speech,\Nif a voice of a human's speak of a neural
Dialogue: 0,0:39:51.12,0:39:56.16,Default,,0000,0000,0000,,network has been trained to mimic any\Nhuman speaker. We can now put words into
Dialogue: 0,0:39:56.16,0:40:01.25,Default,,0000,0000,0000,,other people's mouths realistically\Nwithout employing a voice actor. This
Dialogue: 0,0:40:01.25,0:40:06.65,Default,,0000,0000,0000,,stuff is still geek intensive. It requires\Nrelatively expensive GPUs or cloud
Dialogue: 0,0:40:06.65,0:40:10.91,Default,,0000,0000,0000,,computing clusters, but in less than a\Ndecade it'll be out in the wild, turned
Dialogue: 0,0:40:10.91,0:40:15.50,Default,,0000,0000,0000,,into something, any damn script kiddie can\Nuse and just about everyone will be able
Dialogue: 0,0:40:15.50,0:40:18.94,Default,,0000,0000,0000,,to fake up a realistic video of someone\Nthey don't like doing something horrible.
Dialogue: 0,0:40:18.94,0:40:26.97,Default,,0000,0000,0000,,I mean, Donald Trump in the White House. I\Ncan't help but hope that out there
Dialogue: 0,0:40:26.97,0:40:30.67,Default,,0000,0000,0000,,somewhere there's some geek like Steve\NBannon with a huge rack of servers who's
Dialogue: 0,0:40:30.67,0:40:40.98,Default,,0000,0000,0000,,faking it all, but no. Now, also we've\Nalready seen alarm this year over bizarre
Dialogue: 0,0:40:40.98,0:40:45.09,Default,,0000,0000,0000,,YouTube channels that attempt to monetize\Nchildren's TV brands by scraping the video
Dialogue: 0,0:40:45.09,0:40:50.22,Default,,0000,0000,0000,,content of legitimate channels and adding\Ntheir own advertising in keywords on top
Dialogue: 0,0:40:50.22,0:40:54.11,Default,,0000,0000,0000,,before reposting it. This is basically\Nyour YouTube spam.
Dialogue: 0,0:40:54.11,0:40:58.52,Default,,0000,0000,0000,,Many of these channels are shaped by\Npaperclip maximizing advertising AIs, but
Dialogue: 0,0:40:58.52,0:41:03.67,Default,,0000,0000,0000,,are simply trying to maximise their search\Nranking on YouTube and it's entirely
Dialogue: 0,0:41:03.67,0:41:08.47,Default,,0000,0000,0000,,algorithmic: you have a whole list of\Nkeywords, you perm, you take them, you slap
Dialogue: 0,0:41:08.47,0:41:15.30,Default,,0000,0000,0000,,them on top of existing popular videos and\Nre-upload the videos. Once you add neural
Dialogue: 0,0:41:15.30,0:41:19.51,Default,,0000,0000,0000,,network driven tools for inserting\Ncharacter A into pirated video B, to click
Dialogue: 0,0:41:19.51,0:41:24.42,Default,,0000,0000,0000,,maximize.. for click maximizing bots,\Nthings are gonna get very weird, though. And
Dialogue: 0,0:41:24.42,0:41:28.98,Default,,0000,0000,0000,,they're gonna get even weirder, when these\Ntools are deployed for political gain.
Dialogue: 0,0:41:28.98,0:41:34.83,Default,,0000,0000,0000,,We tend - being primates, that evolved 300\Nthousand years ago in a smartphone free
Dialogue: 0,0:41:34.83,0:41:39.95,Default,,0000,0000,0000,,environment - to evaluate the inputs from\Nour eyes and ears much less critically
Dialogue: 0,0:41:39.95,0:41:44.38,Default,,0000,0000,0000,,than what random strangers on the Internet\Ntell us in text. We're already too
Dialogue: 0,0:41:44.38,0:41:48.93,Default,,0000,0000,0000,,vulnerable to fake news as it is. Soon\Nthey'll be coming for us, armed with
Dialogue: 0,0:41:48.93,0:41:54.83,Default,,0000,0000,0000,,believable video evidence. The Smart Money\Nsays that by 2027 you won't be able to
Dialogue: 0,0:41:54.83,0:41:58.55,Default,,0000,0000,0000,,believe anything you see in video, unless\Nfor a cryptographic signatures on it,
Dialogue: 0,0:41:58.55,0:42:02.53,Default,,0000,0000,0000,,linking it back to the camera that shot\Nthe raw feed. But you know how good most
Dialogue: 0,0:42:02.53,0:42:08.15,Default,,0000,0000,0000,,people are at using encryption - it's going to\Nbe chaos!
Dialogue: 0,0:42:08.15,0:42:14.07,Default,,0000,0000,0000,,So, paperclip maximizers with focus on\Neyeballs are very 20th century. The new
Dialogue: 0,0:42:14.07,0:42:19.75,Default,,0000,0000,0000,,generation is going to be focusing on our\Nnervous system. Advertising as an industry
Dialogue: 0,0:42:19.75,0:42:22.51,Default,,0000,0000,0000,,can only exist because of a quirk of our\Nnervous system, which is that we're
Dialogue: 0,0:42:22.51,0:42:26.68,Default,,0000,0000,0000,,susceptible to addiction. Be it \Ntobacco, gambling or heroin, we
Dialogue: 0,0:42:26.68,0:42:31.98,Default,,0000,0000,0000,,recognize addictive behavior, when we see\Nit. Well, do we? It turns out the human
Dialogue: 0,0:42:31.98,0:42:36.25,Default,,0000,0000,0000,,brain's reward feedback loops are\Nrelatively easy to gain. Large
Dialogue: 0,0:42:36.25,0:42:41.40,Default,,0000,0000,0000,,corporations like Zynga - producers of\NFarmVille - exist solely because of it,
Dialogue: 0,0:42:41.40,0:42:45.58,Default,,0000,0000,0000,,free to use social media platforms like\NFacebook and Twitter, are dominant precisely
Dialogue: 0,0:42:45.58,0:42:50.10,Default,,0000,0000,0000,,because they're structured to reward\Nfrequent short bursts of interaction and
Dialogue: 0,0:42:50.10,0:42:54.79,Default,,0000,0000,0000,,to generate emotional engagement - not\Nnecessarily positive emotions, anger and
Dialogue: 0,0:42:54.79,0:42:59.66,Default,,0000,0000,0000,,hatred are just as good when it comes to\Nattracting eyeballs for advertisers.
Dialogue: 0,0:42:59.66,0:43:04.51,Default,,0000,0000,0000,,Smartphone addiction is a side effect of\Nadvertising as a revenue model. Frequent
Dialogue: 0,0:43:04.51,0:43:09.85,Default,,0000,0000,0000,,short bursts of interaction to keep us\Ncoming back for more. Now a new.. newish
Dialogue: 0,0:43:09.85,0:43:13.72,Default,,0000,0000,0000,,development, thanks to deep learning again -\NI keep coming back to deep learning,
Dialogue: 0,0:43:13.72,0:43:19.29,Default,,0000,0000,0000,,don't I? - use of neural networks in a\Nmanner that Marvin Minsky never envisaged,
Dialogue: 0,0:43:19.29,0:43:22.53,Default,,0000,0000,0000,,back when he was deciding that the\NPerzeptron was where it began and ended
Dialogue: 0,0:43:22.53,0:43:27.36,Default,,0000,0000,0000,,and it couldn't do anything.\NWell, we have neuroscientists now, who've
Dialogue: 0,0:43:27.36,0:43:33.68,Default,,0000,0000,0000,,mechanized the process of making apps more\Naddictive. Dopamine Labs is one startup
Dialogue: 0,0:43:33.68,0:43:38.03,Default,,0000,0000,0000,,that provides tools to app developers to\Nmake any app more addictive, as well as to
Dialogue: 0,0:43:38.03,0:43:41.13,Default,,0000,0000,0000,,reduce the desire to continue\Nparticipating in a behavior if it's
Dialogue: 0,0:43:41.13,0:43:46.52,Default,,0000,0000,0000,,undesirable, if the app developer actually\Nwants to help people kick the habit. This
Dialogue: 0,0:43:46.52,0:43:52.06,Default,,0000,0000,0000,,goes way beyond automated A/B testing. A/B\Ntesting allows developers to plot a binary
Dialogue: 0,0:43:52.06,0:43:58.76,Default,,0000,0000,0000,,tree path between options, moving towards a\Nsingle desired goal. But true deep
Dialogue: 0,0:43:58.76,0:44:03.64,Default,,0000,0000,0000,,learning, addictiveness maximizers, can\Noptimize for multiple attractors in
Dialogue: 0,0:44:03.64,0:44:10.01,Default,,0000,0000,0000,,parallel. The more users you've got on\Nyour app, the more effectively you can work
Dialogue: 0,0:44:10.01,0:44:16.92,Default,,0000,0000,0000,,out, what attracts them and train them and\Nfocus on extra addictive characteristics.
Dialogue: 0,0:44:16.92,0:44:20.61,Default,,0000,0000,0000,,Now, going by their public face, the folks\Nat Dopamine Labs seem to have ethical
Dialogue: 0,0:44:20.61,0:44:24.93,Default,,0000,0000,0000,,qualms about the misuse of addiction\Nmaximizers. But neuroscience isn't a
Dialogue: 0,0:44:24.93,0:44:28.73,Default,,0000,0000,0000,,secret and sooner or later some really\Nunscrupulous sociopaths will try to see
Dialogue: 0,0:44:28.73,0:44:36.44,Default,,0000,0000,0000,,how far they can push it. So let me give\Nyou a specific imaginary scenario: Apple
Dialogue: 0,0:44:36.44,0:44:41.40,Default,,0000,0000,0000,,have put a lot of effort into making real-\Ntime face recognition work on the iPhone X
Dialogue: 0,0:44:41.40,0:44:44.94,Default,,0000,0000,0000,,and it's going to be everywhere on\Neverybody's phone in another couple of
Dialogue: 0,0:44:44.94,0:44:50.76,Default,,0000,0000,0000,,years. You can't fool an iPhone X with a\Nphoto or even a simple mask. It does depth
Dialogue: 0,0:44:50.76,0:44:54.01,Default,,0000,0000,0000,,mapping to ensure, your eyes are in the\Nright place and can tell whether they're
Dialogue: 0,0:44:54.01,0:44:58.20,Default,,0000,0000,0000,,open or closed. It recognizes your face\Nfrom underlying bone structure through
Dialogue: 0,0:44:58.20,0:45:02.76,Default,,0000,0000,0000,,makeup and bruises. It's running\Ncontinuously, checking pretty much as often
Dialogue: 0,0:45:02.76,0:45:06.84,Default,,0000,0000,0000,,as every time you'd hit the home button on\Na more traditional smartphone UI and it
Dialogue: 0,0:45:06.84,0:45:13.75,Default,,0000,0000,0000,,can see where your eyeballs are pointing.\NThe purpose of a face recognition system
Dialogue: 0,0:45:13.75,0:45:19.98,Default,,0000,0000,0000,,is to provide for real-time authenticate\Ncontinuous authentication when you're
Dialogue: 0,0:45:19.98,0:45:23.96,Default,,0000,0000,0000,,using a device - not just enter a PIN or\Nsign a password or use a two factor
Dialogue: 0,0:45:23.96,0:45:28.42,Default,,0000,0000,0000,,authentication pad, but the device knows\Nthat you are its authorized user on a
Dialogue: 0,0:45:28.42,0:45:33.00,Default,,0000,0000,0000,,continuous basis and if somebody grabs\Nyour phone and runs away with it, it'll
Dialogue: 0,0:45:33.00,0:45:38.05,Default,,0000,0000,0000,,know that it's been stolen immediately, it\Nsees the face of the thief.
Dialogue: 0,0:45:38.05,0:45:42.91,Default,,0000,0000,0000,,However, your phone monitoring your facial\Nexpressions and correlating against app
Dialogue: 0,0:45:42.91,0:45:48.25,Default,,0000,0000,0000,,usage has other implications. Your phone\Nwill be aware of precisely what you like
Dialogue: 0,0:45:48.25,0:45:53.55,Default,,0000,0000,0000,,to look at on your screen.. on its screen.\NWe may well have sufficient insight on the
Dialogue: 0,0:45:53.55,0:45:59.74,Default,,0000,0000,0000,,part of the phone to identify whether\Nyou're happy or sad, bored or engaged.
Dialogue: 0,0:45:59.74,0:46:04.68,Default,,0000,0000,0000,,With addiction seeking deep learning tools\Nand neural network generated images, those
Dialogue: 0,0:46:04.68,0:46:09.15,Default,,0000,0000,0000,,synthetic videos I was talking about, it's\Nentirely.. in principle entirely possible to
Dialogue: 0,0:46:09.15,0:46:15.25,Default,,0000,0000,0000,,feed you an endlessly escalating payload\Nof arousal-inducing inputs. It might be
Dialogue: 0,0:46:15.25,0:46:19.44,Default,,0000,0000,0000,,Facebook or Twitter messages, optimized to\Nproduce outrage, or it could be porn
Dialogue: 0,0:46:19.44,0:46:24.62,Default,,0000,0000,0000,,generated by AI to appeal to kinks you\Ndon't even consciously know you have.
Dialogue: 0,0:46:24.62,0:46:28.98,Default,,0000,0000,0000,,But either way, the app now owns your\Ncentral nervous system and you will be
Dialogue: 0,0:46:28.98,0:46:36.23,Default,,0000,0000,0000,,monetized. And finally, I'd like to raise a\Nreally hair-raising specter that goes well
Dialogue: 0,0:46:36.23,0:46:41.73,Default,,0000,0000,0000,,beyond the use of deep learning and\Ntargeted propaganda and cyber war. Back in
Dialogue: 0,0:46:41.73,0:46:46.34,Default,,0000,0000,0000,,2011, an obscure Russian software house\Nlaunched an iPhone app for pickup artists
Dialogue: 0,0:46:46.34,0:46:52.94,Default,,0000,0000,0000,,called 'Girls Around Me'. Spoiler: Apple\Npulled it like a hot potato as soon as
Dialogue: 0,0:46:52.94,0:46:59.06,Default,,0000,0000,0000,,word got out that it existed. Now, Girls\NAround Me works out where the user is
Dialogue: 0,0:46:59.06,0:47:03.73,Default,,0000,0000,0000,,using GPS, then it would query Foursquare\Nand Facebook for people matching a simple
Dialogue: 0,0:47:03.73,0:47:09.88,Default,,0000,0000,0000,,relational search, for single females on\NFacebook, per relationship status, who have
Dialogue: 0,0:47:09.88,0:47:15.02,Default,,0000,0000,0000,,checked in, or been checked in by their\Nfriends, in your vicinity on Foursquare.
Dialogue: 0,0:47:15.02,0:47:19.27,Default,,0000,0000,0000,,The app then displays their locations on a\Nmap along with links to their social media
Dialogue: 0,0:47:19.27,0:47:24.56,Default,,0000,0000,0000,,profiles. If they were doing it today, the\Ninterface would be gamified, showing strike
Dialogue: 0,0:47:24.56,0:47:28.98,Default,,0000,0000,0000,,rates and a leaderboard and flagging\Ntargets who succumbed to harassment as
Dialogue: 0,0:47:28.98,0:47:31.96,Default,,0000,0000,0000,,easy lays.\NBut these days, the cool kids and single
Dialogue: 0,0:47:31.96,0:47:35.98,Default,,0000,0000,0000,,adults are all using dating apps with a\Nmissing vowel in the name, only a creeper
Dialogue: 0,0:47:35.98,0:47:45.42,Default,,0000,0000,0000,,would want something like Girls Around Me,\Nright? Unfortunately, there are much, much
Dialogue: 0,0:47:45.42,0:47:49.45,Default,,0000,0000,0000,,nastier uses of and scraping social media\Nto find potential victims for serial
Dialogue: 0,0:47:49.45,0:47:54.48,Default,,0000,0000,0000,,rapists. Does your social media profile\Nindicate your political religious
Dialogue: 0,0:47:54.48,0:47:59.65,Default,,0000,0000,0000,,affiliation? No? Cambridge Analytica can\Nwork them out with 99.9% precision
Dialogue: 0,0:47:59.65,0:48:04.66,Default,,0000,0000,0000,,anyway, so don't worry about that. We\Nalready have you pegged. Now add a service
Dialogue: 0,0:48:04.66,0:48:08.43,Default,,0000,0000,0000,,that can identify people's affiliation and\Nlocation and you have a beginning of a
Dialogue: 0,0:48:08.43,0:48:13.52,Default,,0000,0000,0000,,flash mob app, one that will show people\Nlike us and people like them on a
Dialogue: 0,0:48:13.52,0:48:18.55,Default,,0000,0000,0000,,hyperlocal map.\NImagine you're a young female and a
Dialogue: 0,0:48:18.55,0:48:22.24,Default,,0000,0000,0000,,supermarket like Target has figured out\Nfrom your purchase patterns, that you're
Dialogue: 0,0:48:22.24,0:48:28.16,Default,,0000,0000,0000,,pregnant, even though you don't know it\Nyet. This actually happened in 2011. Now
Dialogue: 0,0:48:28.16,0:48:31.25,Default,,0000,0000,0000,,imagine, that all the anti-abortion\Ncampaigners in your town have an app
Dialogue: 0,0:48:31.25,0:48:36.11,Default,,0000,0000,0000,,called "Babies Risk" on their phones.\NSomeone has paid for the analytics feed
Dialogue: 0,0:48:36.11,0:48:40.32,Default,,0000,0000,0000,,from the supermarket and every time you go\Nnear a family planning clinic, a group of
Dialogue: 0,0:48:40.32,0:48:44.27,Default,,0000,0000,0000,,unfriendly anti-abortion protesters\Nsomehow miraculously show up and swarm
Dialogue: 0,0:48:44.27,0:48:49.99,Default,,0000,0000,0000,,you. Or imagine you're male and gay and\Nthe "God hates fags"-crowd has invented a
Dialogue: 0,0:48:49.99,0:48:55.30,Default,,0000,0000,0000,,100% reliable gaydar app, based on your\NGrindr profile, and is getting their fellow
Dialogue: 0,0:48:55.30,0:49:00.81,Default,,0000,0000,0000,,travelers to queer bash gay men - only when\Nthey're alone or outnumbered by ten to
Dialogue: 0,0:49:00.81,0:49:06.41,Default,,0000,0000,0000,,one. That's the special horror of precise\Ngeolocation not only do you always know
Dialogue: 0,0:49:06.41,0:49:13.02,Default,,0000,0000,0000,,where you are, the AIs know, where you are\Nand some of them aren't friendly. Or
Dialogue: 0,0:49:13.02,0:49:17.33,Default,,0000,0000,0000,,imagine, you're in Pakistan and Christian\NMuslim tensions are rising or in rural
Dialogue: 0,0:49:17.33,0:49:24.25,Default,,0000,0000,0000,,Alabama or an Democrat, you know the\Npossibilities are endless. Someone out
Dialogue: 0,0:49:24.25,0:49:30.12,Default,,0000,0000,0000,,there is working on this. A geolocation\Naware, social media scraping deep learning
Dialogue: 0,0:49:30.12,0:49:34.23,Default,,0000,0000,0000,,application, that uses a gamified\Ncompetitive interface to reward its
Dialogue: 0,0:49:34.23,0:49:38.50,Default,,0000,0000,0000,,players for joining in acts of mob\Nviolence against whoever the app developer
Dialogue: 0,0:49:38.50,0:49:41.64,Default,,0000,0000,0000,,hates.\NProbably it has an innocuous seeming, but
Dialogue: 0,0:49:41.64,0:49:46.65,Default,,0000,0000,0000,,highly addictive training mode, to get the\Nusers accustomed to working in teams and
Dialogue: 0,0:49:46.65,0:49:52.64,Default,,0000,0000,0000,,obeying the apps instructions. Think\NIngress or Pokemon Go. Then at some pre-
Dialogue: 0,0:49:52.64,0:49:57.77,Default,,0000,0000,0000,,planned zero-hour, it switches mode and\Nstarts rewarding players for violence,
Dialogue: 0,0:49:57.77,0:50:01.78,Default,,0000,0000,0000,,players who have been primed to think of\Ntheir targets as vermin by a steady drip
Dialogue: 0,0:50:01.78,0:50:06.47,Default,,0000,0000,0000,,feed of micro-targeted dehumanizing\Npropaganda inputs, delivered over a period
Dialogue: 0,0:50:06.47,0:50:12.31,Default,,0000,0000,0000,,of months. And the worst bit of this picture?\NIs that the app developer isn't even a
Dialogue: 0,0:50:12.31,0:50:16.54,Default,,0000,0000,0000,,nation-state trying to disrupt its enemies\Nor an extremist political group trying to
Dialogue: 0,0:50:16.54,0:50:22.00,Default,,0000,0000,0000,,murder gays, Jews or Muslims. It's just a\NPaperclip Maximizer doing what it does
Dialogue: 0,0:50:22.00,0:50:27.90,Default,,0000,0000,0000,,and you are the paper. Welcome to the 21st\Ncentury.
Dialogue: 0,0:50:27.90,0:50:40.73,Default,,0000,0000,0000,,{\i1}applause{\i0}\NUhm...
Dialogue: 0,0:50:40.73,0:50:41.96,Default,,0000,0000,0000,,Thank you.
Dialogue: 0,0:50:41.96,0:50:47.78,Default,,0000,0000,0000,,{\i1}ongoing applause{\i0}\NWe have a little time for questions. Do
Dialogue: 0,0:50:47.78,0:50:53.79,Default,,0000,0000,0000,,you have a microphone for the orders? Do\Nwe have any questions? ... OK.
Dialogue: 0,0:50:53.79,0:50:56.43,Default,,0000,0000,0000,,Herald: So you are doing a Q&A?\NCS: Hmm?
Dialogue: 0,0:50:56.43,0:51:02.29,Default,,0000,0000,0000,,Herald: So you are doing a Q&A. Well if\Nthere are any questions, please come
Dialogue: 0,0:51:02.29,0:51:24.32,Default,,0000,0000,0000,,forward to the microphones, numbers 1\Nthrough 4 and ask.
Dialogue: 0,0:51:24.32,0:51:29.01,Default,,0000,0000,0000,,Mic 1: Do you really think it's all\Nbleak and dystopian like you prescribed
Dialogue: 0,0:51:29.01,0:51:34.75,Default,,0000,0000,0000,,it, because I also think the future can be\Nbright, looking at the internet with open
Dialogue: 0,0:51:34.75,0:51:38.73,Default,,0000,0000,0000,,source and like, it's all growing and going\Nfaster and faster in a good
Dialogue: 0,0:51:38.73,0:51:44.11,Default,,0000,0000,0000,,direction. So what do you think about\Nthe balance here?
Dialogue: 0,0:51:44.11,0:51:48.07,Default,,0000,0000,0000,,CS: {\i1}sighs{\i0} Basically, I think the\Nproblem is, that about 3% of us
Dialogue: 0,0:51:48.07,0:51:54.33,Default,,0000,0000,0000,,are sociopaths or psychopaths, who spoil\Neverything for the other 97% of us.
Dialogue: 0,0:51:54.33,0:51:56.98,Default,,0000,0000,0000,,Wouldn't it be great if somebody could\Nwrite an app that would identify all the
Dialogue: 0,0:51:56.98,0:51:59.69,Default,,0000,0000,0000,,psychopaths among us and let the rest of\Nus just kill them?
Dialogue: 0,0:51:59.69,0:52:05.32,Default,,0000,0000,0000,,{\i1}laughing, applause{\i0}\NYeah, we have all the
Dialogue: 0,0:52:05.32,0:52:12.30,Default,,0000,0000,0000,,tools to make a utopia, we have it now\Ntoday. A bleak miserable grim meathook
Dialogue: 0,0:52:12.30,0:52:17.51,Default,,0000,0000,0000,,future is not inevitable, but it's up to\Nus to use these tools to prevent the bad
Dialogue: 0,0:52:17.51,0:52:22.48,Default,,0000,0000,0000,,stuff happening and to do that, we have to\Nanticipate the bad outcomes and work to
Dialogue: 0,0:52:22.48,0:52:25.26,Default,,0000,0000,0000,,try and figure out a way to deal with\Nthem. That's what this talk is. I'm trying
Dialogue: 0,0:52:25.26,0:52:29.30,Default,,0000,0000,0000,,to do a bit of a wake-up call and get\Npeople thinking about how much worse
Dialogue: 0,0:52:29.30,0:52:33.63,Default,,0000,0000,0000,,things can get and what we need to do to\Nprevent it from happening. What I was
Dialogue: 0,0:52:33.63,0:52:37.96,Default,,0000,0000,0000,,saying earlier about our regulatory\Nsystems being broken, stands. How do we
Dialogue: 0,0:52:37.96,0:52:47.50,Default,,0000,0000,0000,,regulate the deep learning technologies?\NThis is something we need to think about.
Dialogue: 0,0:52:47.50,0:52:55.23,Default,,0000,0000,0000,,H: Okay mic number two.\NMic 2: Hello? ... When you talk about
Dialogue: 0,0:52:55.23,0:53:02.25,Default,,0000,0000,0000,,corporations as AIs, where do you see that\Nanalogy you're making? Do you see them as
Dialogue: 0,0:53:02.25,0:53:10.01,Default,,0000,0000,0000,,literally AIs or figuratively?\NCS: Almost literally. If
Dialogue: 0,0:53:10.01,0:53:14.23,Default,,0000,0000,0000,,you're familiar with philosopher\N(?) Searle's Chinese room paradox
Dialogue: 0,0:53:14.23,0:53:17.62,Default,,0000,0000,0000,,from the 1970s, by which he attempted to\Nprove that artificial intelligence was
Dialogue: 0,0:53:17.62,0:53:24.38,Default,,0000,0000,0000,,impossible, a corporation is very much the\NChinese room implementation of an AI. It
Dialogue: 0,0:53:24.38,0:53:29.40,Default,,0000,0000,0000,,is a bunch of human beings in a box. You\Nput inputs into the box, you get apples
Dialogue: 0,0:53:29.40,0:53:33.00,Default,,0000,0000,0000,,out of a box. Does it matter, whether it's\Nall happening in software or whether
Dialogue: 0,0:53:33.00,0:53:37.52,Default,,0000,0000,0000,,there's a human being following rules\Ninbetween to assemble the output? I don't
Dialogue: 0,0:53:37.52,0:53:41.19,Default,,0000,0000,0000,,see there being much of a difference.\NNow you have to look at a company at a
Dialogue: 0,0:53:41.19,0:53:46.63,Default,,0000,0000,0000,,very abstract level to view it as an AI,\Nbut more and more companies are automating
Dialogue: 0,0:53:46.63,0:53:54.50,Default,,0000,0000,0000,,their internal business processes. You've\Ngot to view this as an ongoing trend. And
Dialogue: 0,0:53:54.50,0:54:01.49,Default,,0000,0000,0000,,yeah, they have many of the characteristics\Nof an AI.
Dialogue: 0,0:54:01.49,0:54:06.24,Default,,0000,0000,0000,,Herald: Okay mic number four.\NMic 4: Hi, thanks for your talk.
Dialogue: 0,0:54:06.24,0:54:13.22,Default,,0000,0000,0000,,You probably heard of the Time Well\NSpent and Design Ethics movements that
Dialogue: 0,0:54:13.22,0:54:16.85,Default,,0000,0000,0000,,are alerting developers to dark patterns\Nin UI design, where
Dialogue: 0,0:54:16.85,0:54:20.63,Default,,0000,0000,0000,,these people design apps to manipulate\Npeople. I'm curious if you find any
Dialogue: 0,0:54:20.63,0:54:26.75,Default,,0000,0000,0000,,optimism in the possibility of amplifying\Nor promoting those movements.
Dialogue: 0,0:54:26.75,0:54:31.57,Default,,0000,0000,0000,,CS: Uhm, you know, I knew about dark\Npatterns, I knew about people trying to
Dialogue: 0,0:54:31.57,0:54:36.23,Default,,0000,0000,0000,,optimize them, I wasn't actually aware\Nthere were movements against this. Okay I'm
Dialogue: 0,0:54:36.23,0:54:40.76,Default,,0000,0000,0000,,53 years old, I'm out of touch. I haven't\Nactually done any serious programming in
Dialogue: 0,0:54:40.76,0:54:47.00,Default,,0000,0000,0000,,15 years. I'm so rusty, my rust has rust on\Nit. But, you know, it is a worrying trend
Dialogue: 0,0:54:47.00,0:54:54.00,Default,,0000,0000,0000,,and actual activism is a good start.\NRaising awareness of hazards and of what
Dialogue: 0,0:54:54.00,0:54:58.33,Default,,0000,0000,0000,,we should be doing about them, is a good\Nstart. And I would classify this actually
Dialogue: 0,0:54:58.33,0:55:04.22,Default,,0000,0000,0000,,as a moral issue. We need to..\Ncorporations evaluate everything in terms
Dialogue: 0,0:55:04.22,0:55:07.96,Default,,0000,0000,0000,,of revenue, because it's very\Nequivalent to breathing, they have to
Dialogue: 0,0:55:07.96,0:55:13.81,Default,,0000,0000,0000,,breathe. Corporations don't usually have\Nany moral framework. We're humans, we need
Dialogue: 0,0:55:13.81,0:55:18.48,Default,,0000,0000,0000,,a moral framework to operate within. Even\Nif it's as simple as first "Do no harm!"
Dialogue: 0,0:55:18.48,0:55:22.50,Default,,0000,0000,0000,,or "Do not do unto others that which would\Nbe repugnant if it was done unto you!",
Dialogue: 0,0:55:22.50,0:55:26.45,Default,,0000,0000,0000,,the Golden Rule. So, yeah, we should be\Ntrying to spread awareness of this about
Dialogue: 0,0:55:26.45,0:55:32.17,Default,,0000,0000,0000,,and working with program developers, to\Nlook to remind them that they are human
Dialogue: 0,0:55:32.17,0:55:36.48,Default,,0000,0000,0000,,beings and have to be humane in their\Napplication of technology, is a necessary
Dialogue: 0,0:55:36.48,0:55:39.51,Default,,0000,0000,0000,,start.\N{\i1}applause{\i0}
Dialogue: 0,0:55:39.51,0:55:45.50,Default,,0000,0000,0000,,H: Thank you! Mic 3?\NMic 3: Hi! Yeah, I think that folks,
Dialogue: 0,0:55:45.50,0:55:48.60,Default,,0000,0000,0000,,especially in this sort of crowd, tend to\Njump to the "just get off of
Dialogue: 0,0:55:48.60,0:55:52.33,Default,,0000,0000,0000,,Facebook"-solution first, for a lot of\Nthese things that are really, really
Dialogue: 0,0:55:52.33,0:55:56.64,Default,,0000,0000,0000,,scary. But what worries me, is how we sort\Nof silence ourselves when we do that.
Dialogue: 0,0:55:56.64,0:56:02.24,Default,,0000,0000,0000,,After the election I actually got back on\NFacebook, because the Women's March was
Dialogue: 0,0:56:02.24,0:56:07.25,Default,,0000,0000,0000,,mostly organized through Facebook. But\Nyeah, I think we need a lot more
Dialogue: 0,0:56:07.25,0:56:12.88,Default,,0000,0000,0000,,regulation, but we can't just throw it\Nout. We're.. because it's..
Dialogue: 0,0:56:12.88,0:56:16.52,Default,,0000,0000,0000,,social media is the only... really good\Nplatform we have right now
Dialogue: 0,0:56:16.52,0:56:20.41,Default,,0000,0000,0000,,to express ourselves, to\Nhave our rules, or power.
Dialogue: 0,0:56:20.41,0:56:24.66,Default,,0000,0000,0000,,CS: Absolutely. I have made\Na point of not really using Facebook
Dialogue: 0,0:56:24.66,0:56:28.48,Default,,0000,0000,0000,,for many, many, many years.\NI have a Facebook page simply to
Dialogue: 0,0:56:28.48,0:56:31.53,Default,,0000,0000,0000,,shut up the young marketing people at my\Npublisher, who used to prop up every two
Dialogue: 0,0:56:31.53,0:56:34.68,Default,,0000,0000,0000,,years and say: "Why don't you have a\NFacebook. Everybody's got a Facebook."
Dialogue: 0,0:56:34.68,0:56:39.79,Default,,0000,0000,0000,,No, I've had a blog since 1993!\N{\i1}laughing{\i0}
Dialogue: 0,0:56:39.79,0:56:44.17,Default,,0000,0000,0000,,But no, I'm gonna have to use Facebook,\Nbecause these days, not using Facebook is
Dialogue: 0,0:56:44.17,0:56:50.78,Default,,0000,0000,0000,,like not using email. You're cutting off\Nyour nose to spite your face. What we
Dialogue: 0,0:56:50.78,0:56:54.83,Default,,0000,0000,0000,,really do need to be doing, is looking for\Nsome form of effective oversight of
Dialogue: 0,0:56:54.83,0:57:00.83,Default,,0000,0000,0000,,Facebook and particularly, of how they..\Nthe algorithms that show you content, are
Dialogue: 0,0:57:00.83,0:57:05.34,Default,,0000,0000,0000,,written. What I was saying earlier about\Nhow algorithms are not as transparent as
Dialogue: 0,0:57:05.34,0:57:10.53,Default,,0000,0000,0000,,human beings to people, applies hugely to\Nthem. And both, Facebook and Twitter
Dialogue: 0,0:57:10.53,0:57:15.18,Default,,0000,0000,0000,,control the information\Nthat they display to you.
Dialogue: 0,0:57:15.18,0:57:18.69,Default,,0000,0000,0000,,Herald: Okay, I'm terribly sorry for all the\Npeople queuing at the mics now, we're out
Dialogue: 0,0:57:18.69,0:57:24.96,Default,,0000,0000,0000,,of time. I also have to apologize, I\Nannounced, that this talk was being held in
Dialogue: 0,0:57:24.96,0:57:29.69,Default,,0000,0000,0000,,English, but it was being held in English.\N{\i1}the latter pronounced on the G{\i0}
Dialogue: 0,0:57:29.69,0:57:31.47,Default,,0000,0000,0000,,Thank you very much, Charles Stross!
Dialogue: 0,0:57:31.47,0:57:33.69,Default,,0000,0000,0000,,CS: Thank you very much for\Nlistening to me, it's been a pleasure!
Dialogue: 0,0:57:33.69,0:57:36.50,Default,,0000,0000,0000,,{\i1}applause{\i0}
Dialogue: 0,0:57:36.50,0:57:52.67,Default,,0000,0000,0000,,{\i1}postroll music{\i0}
Dialogue: 0,0:57:52.67,0:57:57.78,Default,,0000,0000,0000,,{\i1}subtitles created by c3subtitles.de\Nin the year 2018{\i0}