[Script Info]
Title: 
[Events]
Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
Dialogue: 0,0:00:00.00,0:00:08.90,Default,,0000,0000,0000,,{\i1}Musik{\i0}
Dialogue: 0,0:00:08.90,0:00:20.04,Default,,0000,0000,0000,,Herald: Who of you is using Facebook? Twitter? \NDiaspora?
Dialogue: 0,0:00:20.04,0:00:27.63,Default,,0000,0000,0000,,{\i1}concerned noise{\i0} And all of that data\Nyou enter there
Dialogue: 0,0:00:27.63,0:00:34.24,Default,,0000,0000,0000,,gets to server, gets into the hand of somebody\Nwho's using it
Dialogue: 0,0:00:34.24,0:00:38.52,Default,,0000,0000,0000,,and the next talk\Nis especially about that,
Dialogue: 0,0:00:38.52,0:00:43.88,Default,,0000,0000,0000,,because there's also intelligent machines\Nand intelligent algorithms
Dialogue: 0,0:00:43.88,0:00:47.49,Default,,0000,0000,0000,,that try to make something\Nout of that data.
Dialogue: 0,0:00:47.49,0:00:50.92,Default,,0000,0000,0000,,So the post-doc researcher Jennifer Helsby
Dialogue: 0,0:00:50.92,0:00:55.84,Default,,0000,0000,0000,,of the University of Chicago,\Nwhich works in this
Dialogue: 0,0:00:55.84,0:00:59.37,Default,,0000,0000,0000,,intersection between policy and \Ntechnology,
Dialogue: 0,0:00:59.37,0:01:04.71,Default,,0000,0000,0000,,will now ask you the question:\NTo who would we give that power?
Dialogue: 0,0:01:04.71,0:01:12.86,Default,,0000,0000,0000,,Dr. Helsby: Thanks.\N{\i1}applause{\i0}
Dialogue: 0,0:01:12.86,0:01:17.09,Default,,0000,0000,0000,,Okay, so, today I'm gonna do a brief tour\Nof intelligent systems
Dialogue: 0,0:01:17.09,0:01:18.64,Default,,0000,0000,0000,,and how they're currently used
Dialogue: 0,0:01:18.64,0:01:21.76,Default,,0000,0000,0000,,and then we're gonna look at some examples\Nwith respect
Dialogue: 0,0:01:21.76,0:01:23.71,Default,,0000,0000,0000,,to the properties that we might care about
Dialogue: 0,0:01:23.71,0:01:26.00,Default,,0000,0000,0000,,these systems having,\Nand I'll talk a little bit about
Dialogue: 0,0:01:26.00,0:01:27.94,Default,,0000,0000,0000,,some of the work that's been done in academia
Dialogue: 0,0:01:27.94,0:01:28.68,Default,,0000,0000,0000,,on these topics.
Dialogue: 0,0:01:28.68,0:01:31.78,Default,,0000,0000,0000,,And then we'll talk about some\Npromising paths forward.
Dialogue: 0,0:01:31.78,0:01:37.04,Default,,0000,0000,0000,,So, I wanna start with this:\NKranzberg's First Law of Technology
Dialogue: 0,0:01:37.04,0:01:40.42,Default,,0000,0000,0000,,So, it's not good or bad,\Nbut it also isn't neutral.
Dialogue: 0,0:01:40.42,0:01:42.98,Default,,0000,0000,0000,,Technology shapes our world,\Nand it can act as
Dialogue: 0,0:01:42.98,0:01:46.14,Default,,0000,0000,0000,,a liberating force-- or an oppressive and\Ncontrolling force.
Dialogue: 0,0:01:46.14,0:01:49.73,Default,,0000,0000,0000,,So, in this talk, I'm gonna go\Ntowards some of the aspects
Dialogue: 0,0:01:49.73,0:01:53.83,Default,,0000,0000,0000,,of intelligent systems that might be more\Ncontrolling in nature.
Dialogue: 0,0:01:53.83,0:01:56.06,Default,,0000,0000,0000,,So, as we all know,
Dialogue: 0,0:01:56.06,0:01:59.77,Default,,0000,0000,0000,,because of the rapidly decreasing cost\Nof storage and computation,
Dialogue: 0,0:01:59.77,0:02:02.17,Default,,0000,0000,0000,,along with the rise of new sensor technologies,
Dialogue: 0,0:02:02.17,0:02:05.51,Default,,0000,0000,0000,,data collection devices\Nare being pushed into every
Dialogue: 0,0:02:05.51,0:02:08.33,Default,,0000,0000,0000,,aspect of our lives: in our homes, our cars,
Dialogue: 0,0:02:08.33,0:02:10.47,Default,,0000,0000,0000,,in our pockets, on our wrists.
Dialogue: 0,0:02:10.47,0:02:13.28,Default,,0000,0000,0000,,And data collection systems act as intermediaries
Dialogue: 0,0:02:13.28,0:02:15.23,Default,,0000,0000,0000,,for a huge amount of human communication.
Dialogue: 0,0:02:15.23,0:02:17.90,Default,,0000,0000,0000,,And much of this data sits in government
Dialogue: 0,0:02:17.90,0:02:19.86,Default,,0000,0000,0000,,and corporate databases.
Dialogue: 0,0:02:19.86,0:02:23.09,Default,,0000,0000,0000,,So, in order to make use of this data,
Dialogue: 0,0:02:23.09,0:02:27.28,Default,,0000,0000,0000,,we need to be able to make some inferences.
Dialogue: 0,0:02:27.28,0:02:30.28,Default,,0000,0000,0000,,So, one way of approaching this is I can hire
Dialogue: 0,0:02:30.28,0:02:32.31,Default,,0000,0000,0000,,a lot of humans, and I can have these humans
Dialogue: 0,0:02:32.31,0:02:34.99,Default,,0000,0000,0000,,manually examine the data, and they can acquire
Dialogue: 0,0:02:34.99,0:02:36.90,Default,,0000,0000,0000,,expert knowledge of the domain, and then
Dialogue: 0,0:02:36.90,0:02:38.51,Default,,0000,0000,0000,,perhaps they can make some decisions
Dialogue: 0,0:02:38.51,0:02:40.83,Default,,0000,0000,0000,,or at least some recommendations\Nbased on it.
Dialogue: 0,0:02:40.83,0:02:43.03,Default,,0000,0000,0000,,However, there's some problems with this.
Dialogue: 0,0:02:43.03,0:02:45.81,Default,,0000,0000,0000,,One is that it's slow, and thus expensive.
Dialogue: 0,0:02:45.81,0:02:48.06,Default,,0000,0000,0000,,It's also biased. We know that humans have
Dialogue: 0,0:02:48.06,0:02:50.70,Default,,0000,0000,0000,,all sorts of biases, both conscious and unconscious,
Dialogue: 0,0:02:50.70,0:02:53.39,Default,,0000,0000,0000,,and it would be nice to have a system\Nthat did not have
Dialogue: 0,0:02:53.39,0:02:54.96,Default,,0000,0000,0000,,these inaccuracies.
Dialogue: 0,0:02:54.96,0:02:57.07,Default,,0000,0000,0000,,It's also not very transparent: I might
Dialogue: 0,0:02:57.07,0:02:58.91,Default,,0000,0000,0000,,not really know the factors that led to
Dialogue: 0,0:02:58.91,0:03:00.93,Default,,0000,0000,0000,,some decisions being made.
Dialogue: 0,0:03:00.93,0:03:03.36,Default,,0000,0000,0000,,Even humans themselves\Noften don't really understand
Dialogue: 0,0:03:03.36,0:03:05.36,Default,,0000,0000,0000,,why they came to a given decision, because
Dialogue: 0,0:03:05.36,0:03:08.13,Default,,0000,0000,0000,,of their being emotional in nature.
Dialogue: 0,0:03:08.13,0:03:11.53,Default,,0000,0000,0000,,And, thus, these human decision making systems
Dialogue: 0,0:03:11.53,0:03:13.17,Default,,0000,0000,0000,,are often difficult to audit.
Dialogue: 0,0:03:13.17,0:03:15.82,Default,,0000,0000,0000,,So, another way to proceed is maybe instead
Dialogue: 0,0:03:15.82,0:03:18.00,Default,,0000,0000,0000,,I study the system and the data carefully
Dialogue: 0,0:03:18.00,0:03:20.52,Default,,0000,0000,0000,,and I write down the best rules\Nfor making a decision
Dialogue: 0,0:03:20.52,0:03:23.28,Default,,0000,0000,0000,,or, I can have a machine\Ndynamically figure out
Dialogue: 0,0:03:23.28,0:03:25.46,Default,,0000,0000,0000,,the best rules, as in machine learning.
Dialogue: 0,0:03:25.46,0:03:28.64,Default,,0000,0000,0000,,So, maybe this is a better approach.
Dialogue: 0,0:03:28.64,0:03:32.23,Default,,0000,0000,0000,,It's certainly fast, and thus cheap.
Dialogue: 0,0:03:32.23,0:03:34.29,Default,,0000,0000,0000,,And maybe I can construct\Nthe system in such a way
Dialogue: 0,0:03:34.29,0:03:37.09,Default,,0000,0000,0000,,that it doesn't have the biases that are inherent
Dialogue: 0,0:03:37.09,0:03:39.21,Default,,0000,0000,0000,,in human decision making.
Dialogue: 0,0:03:39.21,0:03:41.56,Default,,0000,0000,0000,,And, since I've written these rules down,
Dialogue: 0,0:03:41.56,0:03:42.82,Default,,0000,0000,0000,,or a computer has learned these rules,
Dialogue: 0,0:03:42.82,0:03:45.14,Default,,0000,0000,0000,,then I can just show them to somebody, right?
Dialogue: 0,0:03:45.14,0:03:46.82,Default,,0000,0000,0000,,And then they can audit it.
Dialogue: 0,0:03:46.82,0:03:49.02,Default,,0000,0000,0000,,So, more and more decision making is being
Dialogue: 0,0:03:49.02,0:03:50.75,Default,,0000,0000,0000,,done in this way.
Dialogue: 0,0:03:50.75,0:03:53.17,Default,,0000,0000,0000,,And so, in this model, we take data
Dialogue: 0,0:03:53.17,0:03:55.71,Default,,0000,0000,0000,,we make an inference based on that data
Dialogue: 0,0:03:55.71,0:03:58.12,Default,,0000,0000,0000,,using these algorithms, and then
Dialogue: 0,0:03:58.12,0:03:59.42,Default,,0000,0000,0000,,we can take actions.
Dialogue: 0,0:03:59.42,0:04:01.86,Default,,0000,0000,0000,,And, when we take this more scientific approach
Dialogue: 0,0:04:01.86,0:04:04.20,Default,,0000,0000,0000,,to making decisions and optimizing for
Dialogue: 0,0:04:04.20,0:04:07.31,Default,,0000,0000,0000,,a desired outcome,\Nwe can take an experimental approach
Dialogue: 0,0:04:07.31,0:04:10.08,Default,,0000,0000,0000,,so we can determine\Nwhich actions are most effective
Dialogue: 0,0:04:10.08,0:04:12.31,Default,,0000,0000,0000,,in achieving a desired outcome.
Dialogue: 0,0:04:12.31,0:04:14.01,Default,,0000,0000,0000,,Maybe there are some types of communication
Dialogue: 0,0:04:14.01,0:04:16.75,Default,,0000,0000,0000,,styles that are most effective\Nwith certain people.
Dialogue: 0,0:04:16.75,0:04:19.51,Default,,0000,0000,0000,,I can perhaps deploy some individualized incentives
Dialogue: 0,0:04:19.51,0:04:22.06,Default,,0000,0000,0000,,to get the outcome that I desire.
Dialogue: 0,0:04:22.06,0:04:25.99,Default,,0000,0000,0000,,And, maybe even if I carefully design an experiment
Dialogue: 0,0:04:25.99,0:04:27.81,Default,,0000,0000,0000,,with the environment in which people make
Dialogue: 0,0:04:27.81,0:04:30.70,Default,,0000,0000,0000,,these decisions, perhaps even very small changes
Dialogue: 0,0:04:30.70,0:04:34.25,Default,,0000,0000,0000,,can introduce significant changes\Nin peoples' behavior.
Dialogue: 0,0:04:34.25,0:04:37.32,Default,,0000,0000,0000,,So, through these mechanisms,\Nand this experimental approach,
Dialogue: 0,0:04:37.32,0:04:39.84,Default,,0000,0000,0000,,I can maximize the probability\Nthat humans do
Dialogue: 0,0:04:39.84,0:04:42.02,Default,,0000,0000,0000,,what I want.
Dialogue: 0,0:04:42.02,0:04:45.38,Default,,0000,0000,0000,,So, algorithmic decision making is being used
Dialogue: 0,0:04:45.38,0:04:47.27,Default,,0000,0000,0000,,in industry, and is used\Nin lots of other areas,
Dialogue: 0,0:04:47.27,0:04:49.53,Default,,0000,0000,0000,,from astrophysics to medicine, and is now
Dialogue: 0,0:04:49.53,0:04:52.20,Default,,0000,0000,0000,,moving into new domains, including
Dialogue: 0,0:04:52.20,0:04:53.99,Default,,0000,0000,0000,,government applications.
Dialogue: 0,0:04:53.99,0:04:58.56,Default,,0000,0000,0000,,So, we have recommendation engines like\NNetflix, Yelp, SoundCloud,
Dialogue: 0,0:04:58.56,0:05:00.70,Default,,0000,0000,0000,,that direct our attention to what we should
Dialogue: 0,0:05:00.70,0:05:03.51,Default,,0000,0000,0000,,watch and listen to.
Dialogue: 0,0:05:03.51,0:05:07.92,Default,,0000,0000,0000,,Since 2009, Google uses\Npersonalized searched results,
Dialogue: 0,0:05:07.92,0:05:12.84,Default,,0000,0000,0000,,including if you're not logged in\Ninto your Google account.
Dialogue: 0,0:05:12.84,0:05:15.39,Default,,0000,0000,0000,,And we also have algorithm curation and filtering,
Dialogue: 0,0:05:15.39,0:05:17.53,Default,,0000,0000,0000,,as in the case of Facebook News Feed,
Dialogue: 0,0:05:17.53,0:05:19.87,Default,,0000,0000,0000,,Google News, Yahoo News,
Dialogue: 0,0:05:19.87,0:05:22.84,Default,,0000,0000,0000,,which shows you what news articles, for example,
Dialogue: 0,0:05:22.84,0:05:24.33,Default,,0000,0000,0000,,you should be looking at.
Dialogue: 0,0:05:24.33,0:05:25.65,Default,,0000,0000,0000,,And this is important, because a lot of people
Dialogue: 0,0:05:25.65,0:05:29.41,Default,,0000,0000,0000,,get news from these media.
Dialogue: 0,0:05:29.41,0:05:31.52,Default,,0000,0000,0000,,We even have algorithmic journalists!
Dialogue: 0,0:05:31.52,0:05:35.24,Default,,0000,0000,0000,,So, automatic systems generate articles
Dialogue: 0,0:05:35.24,0:05:36.88,Default,,0000,0000,0000,,about weather, traffic, or sports
Dialogue: 0,0:05:36.88,0:05:38.73,Default,,0000,0000,0000,,instead of a human.
Dialogue: 0,0:05:38.73,0:05:41.95,Default,,0000,0000,0000,,And, another application that's more recent
Dialogue: 0,0:05:41.95,0:05:43.57,Default,,0000,0000,0000,,is the use of predictive systems
Dialogue: 0,0:05:43.57,0:05:45.18,Default,,0000,0000,0000,,in political campaigns.
Dialogue: 0,0:05:45.18,0:05:47.37,Default,,0000,0000,0000,,So, political campaigns also now take this
Dialogue: 0,0:05:47.37,0:05:50.34,Default,,0000,0000,0000,,approach to predict on an individual basis
Dialogue: 0,0:05:50.34,0:05:53.30,Default,,0000,0000,0000,,which candidate voters\Nare likely to vote for.
Dialogue: 0,0:05:53.30,0:05:55.50,Default,,0000,0000,0000,,And then they can target,\Non an individual basis,
Dialogue: 0,0:05:55.50,0:05:58.20,Default,,0000,0000,0000,,those that can be persuaded otherwise.
Dialogue: 0,0:05:58.20,0:06:00.83,Default,,0000,0000,0000,,And, finally, in the public sector,
Dialogue: 0,0:06:00.83,0:06:02.71,Default,,0000,0000,0000,,we're starting to use predictive systems
Dialogue: 0,0:06:02.71,0:06:06.32,Default,,0000,0000,0000,,in areas from policing, to health,\Nto education and energy.
Dialogue: 0,0:06:06.32,0:06:08.98,Default,,0000,0000,0000,,So, there are some advantages to this.
Dialogue: 0,0:06:08.98,0:06:12.79,Default,,0000,0000,0000,,So, one thing is that we can automate
Dialogue: 0,0:06:12.79,0:06:15.76,Default,,0000,0000,0000,,aspects of our lives\Nthat we consider to be mundane
Dialogue: 0,0:06:15.76,0:06:17.62,Default,,0000,0000,0000,,using systems that are intelligent
Dialogue: 0,0:06:17.62,0:06:19.58,Default,,0000,0000,0000,,and adaptive enough.
Dialogue: 0,0:06:19.58,0:06:21.68,Default,,0000,0000,0000,,We can make use of all the data
Dialogue: 0,0:06:21.68,0:06:23.99,Default,,0000,0000,0000,,and really get the pieces of information we
Dialogue: 0,0:06:23.99,0:06:25.83,Default,,0000,0000,0000,,really care about.
Dialogue: 0,0:06:25.83,0:06:29.65,Default,,0000,0000,0000,,We can spend money in the most effective way,
Dialogue: 0,0:06:29.65,0:06:32.11,Default,,0000,0000,0000,,and we can do this with this experimental
Dialogue: 0,0:06:32.11,0:06:34.21,Default,,0000,0000,0000,,approach to optimize actions to produce
Dialogue: 0,0:06:34.21,0:06:35.19,Default,,0000,0000,0000,,desired outcomes.
Dialogue: 0,0:06:35.19,0:06:37.30,Default,,0000,0000,0000,,So, we can embed intelligence
Dialogue: 0,0:06:37.30,0:06:39.52,Default,,0000,0000,0000,,into all of these mundane objects
Dialogue: 0,0:06:39.52,0:06:41.18,Default,,0000,0000,0000,,and enable them to make decisions for us,
Dialogue: 0,0:06:41.18,0:06:42.86,Default,,0000,0000,0000,,and so that's what we're doing more and more,
Dialogue: 0,0:06:42.86,0:06:45.21,Default,,0000,0000,0000,,and we can have an object\Nthat decides for us
Dialogue: 0,0:06:45.21,0:06:46.84,Default,,0000,0000,0000,,what temperature we should set our house,
Dialogue: 0,0:06:46.84,0:06:49.01,Default,,0000,0000,0000,,what we should be doing, etc.
Dialogue: 0,0:06:49.01,0:06:52.40,Default,,0000,0000,0000,,So, there might be some implications here.
Dialogue: 0,0:06:52.40,0:06:55.68,Default,,0000,0000,0000,,We want these systems\Nthat do work on this data
Dialogue: 0,0:06:55.68,0:06:58.04,Default,,0000,0000,0000,,to increase the opportunities\Navailable to us.
Dialogue: 0,0:06:58.04,0:07:00.26,Default,,0000,0000,0000,,But it might be that there are some implications
Dialogue: 0,0:07:00.26,0:07:01.78,Default,,0000,0000,0000,,that we have not carefully thought through.
Dialogue: 0,0:07:01.78,0:07:03.43,Default,,0000,0000,0000,,This is a new area, and people are only
Dialogue: 0,0:07:03.43,0:07:05.94,Default,,0000,0000,0000,,starting to scratch the surface of what the
Dialogue: 0,0:07:05.94,0:07:07.29,Default,,0000,0000,0000,,problems might be.
Dialogue: 0,0:07:07.29,0:07:09.60,Default,,0000,0000,0000,,In some cases, they might narrow the options
Dialogue: 0,0:07:09.60,0:07:10.99,Default,,0000,0000,0000,,available to people,
Dialogue: 0,0:07:10.99,0:07:13.20,Default,,0000,0000,0000,,and this approach subjects people to
Dialogue: 0,0:07:13.20,0:07:15.62,Default,,0000,0000,0000,,suggestive messaging intended to nudge them
Dialogue: 0,0:07:15.62,0:07:17.17,Default,,0000,0000,0000,,to a desired outcome.
Dialogue: 0,0:07:17.17,0:07:19.32,Default,,0000,0000,0000,,Some people may have a problem with that.
Dialogue: 0,0:07:19.32,0:07:20.65,Default,,0000,0000,0000,,Values we care about are not gonna be
Dialogue: 0,0:07:20.65,0:07:23.86,Default,,0000,0000,0000,,baked into these systems by default.
Dialogue: 0,0:07:23.86,0:07:25.96,Default,,0000,0000,0000,,It's also the case that some algorithmic systems
Dialogue: 0,0:07:25.96,0:07:28.30,Default,,0000,0000,0000,,facilitate work that we do not like.
Dialogue: 0,0:07:28.30,0:07:30.20,Default,,0000,0000,0000,,For example, in the case of mass surveillance.
Dialogue: 0,0:07:30.20,0:07:32.13,Default,,0000,0000,0000,,And even the same systems,
Dialogue: 0,0:07:32.13,0:07:34.04,Default,,0000,0000,0000,,used by different people or organizations,
Dialogue: 0,0:07:34.04,0:07:36.11,Default,,0000,0000,0000,,have very different consequences.
Dialogue: 0,0:07:36.11,0:07:37.32,Default,,0000,0000,0000,,For example, if I can predict
Dialogue: 0,0:07:37.32,0:07:40.02,Default,,0000,0000,0000,,with high accuracy, based on say search queries,
Dialogue: 0,0:07:40.02,0:07:42.05,Default,,0000,0000,0000,,who's gonna be admitted to a hospital,
Dialogue: 0,0:07:42.05,0:07:43.75,Default,,0000,0000,0000,,some people would be interested\Nin knowing that.
Dialogue: 0,0:07:43.75,0:07:46.12,Default,,0000,0000,0000,,You might be interested\Nin having your doctor know that.
Dialogue: 0,0:07:46.12,0:07:47.92,Default,,0000,0000,0000,,But that same predictive model\Nin the hands of
Dialogue: 0,0:07:47.92,0:07:50.57,Default,,0000,0000,0000,,an insurance company\Nhas a very different implication.
Dialogue: 0,0:07:50.57,0:07:53.39,Default,,0000,0000,0000,,So, the point here is that these systems
Dialogue: 0,0:07:53.39,0:07:55.86,Default,,0000,0000,0000,,structure and influence how humans interact
Dialogue: 0,0:07:55.86,0:07:58.36,Default,,0000,0000,0000,,with each other, how they interact with society,
Dialogue: 0,0:07:58.36,0:07:59.85,Default,,0000,0000,0000,,and how they interact with government.
Dialogue: 0,0:07:59.85,0:08:03.08,Default,,0000,0000,0000,,And if they constrain what people can do,
Dialogue: 0,0:08:03.08,0:08:05.07,Default,,0000,0000,0000,,we should really care about this.
Dialogue: 0,0:08:05.07,0:08:08.27,Default,,0000,0000,0000,,So now I'm gonna go to\Nsort of an extreme case,
Dialogue: 0,0:08:08.27,0:08:11.93,Default,,0000,0000,0000,,just as an example, and that's this\NChinese Social Credit System.
Dialogue: 0,0:08:11.93,0:08:14.17,Default,,0000,0000,0000,,And so this is probably one of the more
Dialogue: 0,0:08:14.17,0:08:17.26,Default,,0000,0000,0000,,ambitious uses of data,
Dialogue: 0,0:08:17.26,0:08:18.88,Default,,0000,0000,0000,,that is used to rank each citizen
Dialogue: 0,0:08:18.88,0:08:21.19,Default,,0000,0000,0000,,based on their behavior, in China.
Dialogue: 0,0:08:21.19,0:08:24.21,Default,,0000,0000,0000,,So right now, there are various pilot systems
Dialogue: 0,0:08:24.21,0:08:27.66,Default,,0000,0000,0000,,deployed by various companies doing this in\NChina.
Dialogue: 0,0:08:27.66,0:08:30.73,Default,,0000,0000,0000,,They're currently voluntary, and by 2020
Dialogue: 0,0:08:30.73,0:08:32.63,Default,,0000,0000,0000,,this system is gonna be decided on,
Dialogue: 0,0:08:32.63,0:08:34.68,Default,,0000,0000,0000,,or a combination of the systems,
Dialogue: 0,0:08:34.68,0:08:37.41,Default,,0000,0000,0000,,that is gonna be mandatory for everyone.
Dialogue: 0,0:08:37.41,0:08:40.95,Default,,0000,0000,0000,,And so, in this system, there are some citizens,
Dialogue: 0,0:08:40.95,0:08:44.38,Default,,0000,0000,0000,,and a huge range of data sources are used.
Dialogue: 0,0:08:44.38,0:08:46.82,Default,,0000,0000,0000,,So, some of the data sources are
Dialogue: 0,0:08:46.82,0:08:48.36,Default,,0000,0000,0000,,your financial data,
Dialogue: 0,0:08:48.36,0:08:50.02,Default,,0000,0000,0000,,your criminal history,
Dialogue: 0,0:08:50.02,0:08:52.32,Default,,0000,0000,0000,,how many points you have\Non your driver's license,
Dialogue: 0,0:08:52.32,0:08:55.36,Default,,0000,0000,0000,,medical information-- for example,\Nif you take birth control pills,
Dialogue: 0,0:08:55.36,0:08:56.81,Default,,0000,0000,0000,,that's incorporated.
Dialogue: 0,0:08:56.81,0:08:59.83,Default,,0000,0000,0000,,Your purchase history-- for example,\Nif you purchase games,
Dialogue: 0,0:08:59.83,0:09:02.43,Default,,0000,0000,0000,,you are down-ranked in the system.
Dialogue: 0,0:09:02.43,0:09:04.49,Default,,0000,0000,0000,,Some of the systems, not all of them,
Dialogue: 0,0:09:04.49,0:09:07.26,Default,,0000,0000,0000,,incorporate social media monitoring,
Dialogue: 0,0:09:07.26,0:09:09.20,Default,,0000,0000,0000,,which makes sense if you're a state like China,
Dialogue: 0,0:09:09.20,0:09:11.27,Default,,0000,0000,0000,,you probably want to know about
Dialogue: 0,0:09:11.27,0:09:14.90,Default,,0000,0000,0000,,political statements that people\Nare saying on social media.
Dialogue: 0,0:09:14.90,0:09:18.02,Default,,0000,0000,0000,,And, one of the more interesting parts is
Dialogue: 0,0:09:18.02,0:09:22.16,Default,,0000,0000,0000,,social network analysis:\Nlooking at the relationships between people.
Dialogue: 0,0:09:22.16,0:09:24.27,Default,,0000,0000,0000,,So, if you have a close relationship with\Nsomebody
Dialogue: 0,0:09:24.27,0:09:26.18,Default,,0000,0000,0000,,and they have a low credit score,
Dialogue: 0,0:09:26.18,0:09:29.13,Default,,0000,0000,0000,,that can have implications on your credit\Nscore.
Dialogue: 0,0:09:29.13,0:09:34.44,Default,,0000,0000,0000,,So, the way that these scores\Nare generated is secret.
Dialogue: 0,0:09:34.44,0:09:38.14,Default,,0000,0000,0000,,And, according to the call for these systems
Dialogue: 0,0:09:38.14,0:09:39.27,Default,,0000,0000,0000,,put out by the government,
Dialogue: 0,0:09:39.27,0:09:42.81,Default,,0000,0000,0000,,the goal is to\N"carry forward the sincerity and
Dialogue: 0,0:09:42.81,0:09:45.76,Default,,0000,0000,0000,,traditional virtues" and\Nestablish the idea of a
Dialogue: 0,0:09:45.76,0:09:47.52,Default,,0000,0000,0000,,"sincerity culture."
Dialogue: 0,0:09:47.52,0:09:49.44,Default,,0000,0000,0000,,But wait, it gets better:
Dialogue: 0,0:09:49.44,0:09:52.45,Default,,0000,0000,0000,,so, there's a portal that enables citizens
Dialogue: 0,0:09:52.45,0:09:55.04,Default,,0000,0000,0000,,to look up the citizen score of anyone.
Dialogue: 0,0:09:55.04,0:09:56.52,Default,,0000,0000,0000,,And many people like this system,
Dialogue: 0,0:09:56.52,0:09:58.32,Default,,0000,0000,0000,,they think it's a fun game.
Dialogue: 0,0:09:58.32,0:10:00.70,Default,,0000,0000,0000,,They boast about it on social media,
Dialogue: 0,0:10:00.70,0:10:03.61,Default,,0000,0000,0000,,they put their score in their dating profile,
Dialogue: 0,0:10:03.61,0:10:04.76,Default,,0000,0000,0000,,because if you're ranked highly you're
Dialogue: 0,0:10:04.76,0:10:06.59,Default,,0000,0000,0000,,part of an exclusive club.
Dialogue: 0,0:10:06.59,0:10:10.06,Default,,0000,0000,0000,,You can get VIP treatment\Nat hotels and other companies.
Dialogue: 0,0:10:10.06,0:10:11.88,Default,,0000,0000,0000,,But the downside is that, if you're excluded
Dialogue: 0,0:10:11.88,0:10:15.54,Default,,0000,0000,0000,,from that club, your weak score\Nmay have other implications,
Dialogue: 0,0:10:15.54,0:10:20.12,Default,,0000,0000,0000,,like being unable to get access\Nto credit, housing, jobs.
Dialogue: 0,0:10:20.12,0:10:23.40,Default,,0000,0000,0000,,There is some reporting that even travel visas
Dialogue: 0,0:10:23.40,0:10:27.00,Default,,0000,0000,0000,,might be restricted\Nif your score is particularly low.
Dialogue: 0,0:10:27.00,0:10:31.16,Default,,0000,0000,0000,,So, a system like this, for a state, is really
Dialogue: 0,0:10:31.16,0:10:34.69,Default,,0000,0000,0000,,the optimal solution\Nto the problem of the public.
Dialogue: 0,0:10:34.69,0:10:37.13,Default,,0000,0000,0000,,It constitutes a very subtle and insiduous
Dialogue: 0,0:10:37.13,0:10:39.35,Default,,0000,0000,0000,,mechanism of social control.
Dialogue: 0,0:10:39.35,0:10:41.21,Default,,0000,0000,0000,,You don't need to spend a lot of money on
Dialogue: 0,0:10:41.21,0:10:43.80,Default,,0000,0000,0000,,police or prisons if you can set up a system
Dialogue: 0,0:10:43.80,0:10:45.82,Default,,0000,0000,0000,,where people discourage one another from
Dialogue: 0,0:10:45.82,0:10:48.93,Default,,0000,0000,0000,,anti-social acts like political action\Nin exchange for
Dialogue: 0,0:10:48.93,0:10:51.43,Default,,0000,0000,0000,,a coupon for a free Uber ride.
Dialogue: 0,0:10:51.43,0:10:55.27,Default,,0000,0000,0000,,So, there are a lot of\Nlegitimate questions here:
Dialogue: 0,0:10:55.27,0:10:58.37,Default,,0000,0000,0000,,What protections does\Nuser data have in this scheme?
Dialogue: 0,0:10:58.37,0:11:01.28,Default,,0000,0000,0000,,Do any safeguards exist to prevent tampering?
Dialogue: 0,0:11:01.28,0:11:04.31,Default,,0000,0000,0000,,What mechanism, if any, is there to prevent
Dialogue: 0,0:11:04.31,0:11:08.81,Default,,0000,0000,0000,,false input data from creating erroneous inferences?
Dialogue: 0,0:11:08.81,0:11:10.42,Default,,0000,0000,0000,,Is there any way that people can fix
Dialogue: 0,0:11:10.42,0:11:12.54,Default,,0000,0000,0000,,their score once they're ranked poorly?
Dialogue: 0,0:11:12.54,0:11:13.90,Default,,0000,0000,0000,,Or does it end up becoming a
Dialogue: 0,0:11:13.90,0:11:15.72,Default,,0000,0000,0000,,self-fulfilling prophecy?
Dialogue: 0,0:11:15.72,0:11:17.85,Default,,0000,0000,0000,,Your weak score means you have less access
Dialogue: 0,0:11:17.85,0:11:21.62,Default,,0000,0000,0000,,to jobs and credit, and now you will have
Dialogue: 0,0:11:21.62,0:11:24.71,Default,,0000,0000,0000,,limited access to opportunity.
Dialogue: 0,0:11:24.71,0:11:27.11,Default,,0000,0000,0000,,So, let's take a step back.
Dialogue: 0,0:11:27.11,0:11:28.47,Default,,0000,0000,0000,,So, what do we want?
Dialogue: 0,0:11:28.47,0:11:31.54,Default,,0000,0000,0000,,So, we probably don't want that,
Dialogue: 0,0:11:31.54,0:11:33.57,Default,,0000,0000,0000,,but as advocates we really wanna
Dialogue: 0,0:11:33.57,0:11:36.13,Default,,0000,0000,0000,,understand what questions we should be asking
Dialogue: 0,0:11:36.13,0:11:37.51,Default,,0000,0000,0000,,of these systems. Right now there's
Dialogue: 0,0:11:37.51,0:11:39.57,Default,,0000,0000,0000,,very little oversight,
Dialogue: 0,0:11:39.57,0:11:41.42,Default,,0000,0000,0000,,and we wanna make sure that we don't
Dialogue: 0,0:11:41.42,0:11:44.03,Default,,0000,0000,0000,,sort of sleepwalk our way to a situation
Dialogue: 0,0:11:44.03,0:11:46.65,Default,,0000,0000,0000,,where we've lost even more power
Dialogue: 0,0:11:46.65,0:11:49.74,Default,,0000,0000,0000,,to these centralized systems of control.
Dialogue: 0,0:11:49.74,0:11:52.21,Default,,0000,0000,0000,,And if you're an implementer, we wanna understand
Dialogue: 0,0:11:52.21,0:11:53.71,Default,,0000,0000,0000,,what can we be doing better.
Dialogue: 0,0:11:53.71,0:11:56.02,Default,,0000,0000,0000,,Are there better ways that we can be implementing
Dialogue: 0,0:11:56.02,0:11:57.64,Default,,0000,0000,0000,,these systems?
Dialogue: 0,0:11:57.64,0:11:59.43,Default,,0000,0000,0000,,Are there values that, as humans,
Dialogue: 0,0:11:59.43,0:12:01.06,Default,,0000,0000,0000,,we care about that we should make sure
Dialogue: 0,0:12:01.06,0:12:02.42,Default,,0000,0000,0000,,these systems have?
Dialogue: 0,0:12:02.42,0:12:05.55,Default,,0000,0000,0000,,So, the first thing\Nthat most people in the room
Dialogue: 0,0:12:05.55,0:12:07.82,Default,,0000,0000,0000,,might think about is privacy.
Dialogue: 0,0:12:07.82,0:12:10.51,Default,,0000,0000,0000,,Which is, of course, of the utmost importance.
Dialogue: 0,0:12:10.51,0:12:12.92,Default,,0000,0000,0000,,We need privacy, and there is a good discussion
Dialogue: 0,0:12:12.92,0:12:15.68,Default,,0000,0000,0000,,on the importance of protecting\Nuser data where possible.
Dialogue: 0,0:12:15.68,0:12:18.42,Default,,0000,0000,0000,,So, in this talk, I'm gonna focus\Non the other aspects of
Dialogue: 0,0:12:18.42,0:12:19.47,Default,,0000,0000,0000,,algorithmic decision making,
Dialogue: 0,0:12:19.47,0:12:21.19,Default,,0000,0000,0000,,that I think have got less attention.
Dialogue: 0,0:12:21.19,0:12:25.14,Default,,0000,0000,0000,,Because it's not just privacy\Nthat we need to worry about here.
Dialogue: 0,0:12:25.14,0:12:28.52,Default,,0000,0000,0000,,We also want systems that are fair and equitable.
Dialogue: 0,0:12:28.52,0:12:30.24,Default,,0000,0000,0000,,We want transparent systems,
Dialogue: 0,0:12:30.24,0:12:35.11,Default,,0000,0000,0000,,we don't want opaque decisions\Nto be made about us,
Dialogue: 0,0:12:35.11,0:12:36.51,Default,,0000,0000,0000,,decisions that might have serious impacts
Dialogue: 0,0:12:36.51,0:12:37.78,Default,,0000,0000,0000,,on our lives.
Dialogue: 0,0:12:37.78,0:12:40.49,Default,,0000,0000,0000,,And we need some accountability mechanisms.
Dialogue: 0,0:12:40.49,0:12:41.89,Default,,0000,0000,0000,,So, for the rest of this talk
Dialogue: 0,0:12:41.89,0:12:43.23,Default,,0000,0000,0000,,we're gonna go through each one of these things
Dialogue: 0,0:12:43.23,0:12:45.23,Default,,0000,0000,0000,,and look at some examples.
Dialogue: 0,0:12:45.23,0:12:47.71,Default,,0000,0000,0000,,So, the first thing is fairness.
Dialogue: 0,0:12:47.71,0:12:50.45,Default,,0000,0000,0000,,And so, as I said in the beginning,\Nthis is one area
Dialogue: 0,0:12:50.45,0:12:52.69,Default,,0000,0000,0000,,where there might be an advantage
Dialogue: 0,0:12:52.69,0:12:55.08,Default,,0000,0000,0000,,to making decisions by machine,
Dialogue: 0,0:12:55.08,0:12:56.74,Default,,0000,0000,0000,,especially in areas where there have
Dialogue: 0,0:12:56.74,0:12:59.41,Default,,0000,0000,0000,,historically been fairness issues with
Dialogue: 0,0:12:59.41,0:13:02.35,Default,,0000,0000,0000,,decision making, such as law enforcement.
Dialogue: 0,0:13:02.35,0:13:05.84,Default,,0000,0000,0000,,So, this is one way that police departments
Dialogue: 0,0:13:05.84,0:13:08.36,Default,,0000,0000,0000,,use predictive models.
Dialogue: 0,0:13:08.36,0:13:10.54,Default,,0000,0000,0000,,The idea here is police would like to
Dialogue: 0,0:13:10.54,0:13:13.45,Default,,0000,0000,0000,,allocate resources in a more effective way,
Dialogue: 0,0:13:13.45,0:13:15.05,Default,,0000,0000,0000,,and they would also like to enable
Dialogue: 0,0:13:15.05,0:13:16.64,Default,,0000,0000,0000,,proactive policing.
Dialogue: 0,0:13:16.64,0:13:20.11,Default,,0000,0000,0000,,So, if you can predict where crimes\Nare going to occur,
Dialogue: 0,0:13:20.11,0:13:22.15,Default,,0000,0000,0000,,or who is going to commit crimes,
Dialogue: 0,0:13:22.15,0:13:24.87,Default,,0000,0000,0000,,then you can put cops in those places,
Dialogue: 0,0:13:24.87,0:13:27.77,Default,,0000,0000,0000,,or perhaps following these people,
Dialogue: 0,0:13:27.77,0:13:29.30,Default,,0000,0000,0000,,and then the crimes will not occur.
Dialogue: 0,0:13:29.30,0:13:31.37,Default,,0000,0000,0000,,So, it's sort of the pre-crime approach.
Dialogue: 0,0:13:31.37,0:13:34.65,Default,,0000,0000,0000,,So, there are a few ways of going about this.
Dialogue: 0,0:13:34.65,0:13:37.92,Default,,0000,0000,0000,,One way is doing this individual-level prediction.
Dialogue: 0,0:13:37.92,0:13:41.09,Default,,0000,0000,0000,,So you take each citizen\Nand estimate the risk
Dialogue: 0,0:13:41.09,0:13:43.77,Default,,0000,0000,0000,,that each citizen will participate,\Nsay, in violence
Dialogue: 0,0:13:43.77,0:13:45.28,Default,,0000,0000,0000,,based on some data.
Dialogue: 0,0:13:45.28,0:13:46.78,Default,,0000,0000,0000,,And then you can flag those people that are
Dialogue: 0,0:13:46.78,0:13:49.20,Default,,0000,0000,0000,,considered particularly violent.
Dialogue: 0,0:13:49.20,0:13:51.52,Default,,0000,0000,0000,,So, this is currently done.
Dialogue: 0,0:13:51.52,0:13:52.59,Default,,0000,0000,0000,,This is done in the U.S.
Dialogue: 0,0:13:52.59,0:13:56.12,Default,,0000,0000,0000,,It's done in Chicago,\Nby the Chicago Police Department.
Dialogue: 0,0:13:56.12,0:13:58.35,Default,,0000,0000,0000,,And they maintain a heat list of individuals
Dialogue: 0,0:13:58.35,0:14:00.79,Default,,0000,0000,0000,,that are considered most likely to commit,
Dialogue: 0,0:14:00.79,0:14:03.53,Default,,0000,0000,0000,,or be the victim of, violence.
Dialogue: 0,0:14:03.53,0:14:06.70,Default,,0000,0000,0000,,And this is done using data\Nthat the police maintain.
Dialogue: 0,0:14:06.70,0:14:09.59,Default,,0000,0000,0000,,So, the features that are used\Nin this predictive model
Dialogue: 0,0:14:09.59,0:14:12.21,Default,,0000,0000,0000,,include things that are derived from
Dialogue: 0,0:14:12.21,0:14:14.61,Default,,0000,0000,0000,,individuals' criminal history.
Dialogue: 0,0:14:14.61,0:14:16.81,Default,,0000,0000,0000,,So, for example, have they been involved in
Dialogue: 0,0:14:16.81,0:14:18.35,Default,,0000,0000,0000,,gun violence in the past?
Dialogue: 0,0:14:18.35,0:14:21.45,Default,,0000,0000,0000,,Do they have narcotics arrests? And so on.
Dialogue: 0,0:14:21.45,0:14:22.86,Default,,0000,0000,0000,,But another thing that's incorporated
Dialogue: 0,0:14:22.86,0:14:25.06,Default,,0000,0000,0000,,in the Chicago Police Department model is
Dialogue: 0,0:14:25.06,0:14:28.30,Default,,0000,0000,0000,,information derived from\Nsocial media network analysis.
Dialogue: 0,0:14:28.30,0:14:30.63,Default,,0000,0000,0000,,So, who you interact with,
Dialogue: 0,0:14:30.63,0:14:32.28,Default,,0000,0000,0000,,as noted in police data.
Dialogue: 0,0:14:32.28,0:14:34.90,Default,,0000,0000,0000,,So, for example, your co-arrestees.
Dialogue: 0,0:14:34.90,0:14:36.44,Default,,0000,0000,0000,,When officers conduct field interviews,
Dialogue: 0,0:14:36.44,0:14:38.24,Default,,0000,0000,0000,,who are people interacting with?
Dialogue: 0,0:14:38.24,0:14:42.94,Default,,0000,0000,0000,,And then this is all incorporated\Ninto this risk score.
Dialogue: 0,0:14:42.94,0:14:44.64,Default,,0000,0000,0000,,So another way to proceed,
Dialogue: 0,0:14:44.64,0:14:47.07,Default,,0000,0000,0000,,which is the method that most companies
Dialogue: 0,0:14:47.07,0:14:49.58,Default,,0000,0000,0000,,that sell products like this\Nto the police have taken,
Dialogue: 0,0:14:49.58,0:14:51.46,Default,,0000,0000,0000,,is instead predicting which areas
Dialogue: 0,0:14:51.46,0:14:53.81,Default,,0000,0000,0000,,are likely to have crimes committed in them.
Dialogue: 0,0:14:53.81,0:14:56.69,Default,,0000,0000,0000,,So, take my city, I put a grid down,
Dialogue: 0,0:14:56.69,0:14:58.18,Default,,0000,0000,0000,,and then I use crime statistics
Dialogue: 0,0:14:58.18,0:15:00.43,Default,,0000,0000,0000,,and maybe some ancillary data sources,
Dialogue: 0,0:15:00.43,0:15:01.79,Default,,0000,0000,0000,,to determine which areas have
Dialogue: 0,0:15:01.79,0:15:04.71,Default,,0000,0000,0000,,the highest risk of crimes occurring in them,
Dialogue: 0,0:15:04.71,0:15:06.33,Default,,0000,0000,0000,,and I can flag those areas and send
Dialogue: 0,0:15:06.33,0:15:08.47,Default,,0000,0000,0000,,police officers to them.
Dialogue: 0,0:15:08.47,0:15:10.95,Default,,0000,0000,0000,,So now, let's look at some of the tools
Dialogue: 0,0:15:10.95,0:15:14.01,Default,,0000,0000,0000,,that are used for this geographic-level prediction.
Dialogue: 0,0:15:14.01,0:15:19.04,Default,,0000,0000,0000,,So, here are 3 companies that sell these
Dialogue: 0,0:15:19.04,0:15:22.91,Default,,0000,0000,0000,,geographic-level predictive policing systems.
Dialogue: 0,0:15:22.91,0:15:25.64,Default,,0000,0000,0000,,So, PredPol has a system that uses
Dialogue: 0,0:15:25.64,0:15:27.20,Default,,0000,0000,0000,,primarily crime statistics:
Dialogue: 0,0:15:27.20,0:15:30.21,Default,,0000,0000,0000,,only the time, place, and type of crime
Dialogue: 0,0:15:30.21,0:15:33.04,Default,,0000,0000,0000,,to predict where crimes will occur.
Dialogue: 0,0:15:33.04,0:15:35.97,Default,,0000,0000,0000,,HunchLab uses a wider range of data sources
Dialogue: 0,0:15:35.97,0:15:37.26,Default,,0000,0000,0000,,including, for example, weather
Dialogue: 0,0:15:37.26,0:15:39.72,Default,,0000,0000,0000,,and then Hitachi is a newer system
Dialogue: 0,0:15:39.72,0:15:42.10,Default,,0000,0000,0000,,that has a predictive crime analytics tool
Dialogue: 0,0:15:42.10,0:15:44.78,Default,,0000,0000,0000,,that also incorporates social media.
Dialogue: 0,0:15:44.78,0:15:47.85,Default,,0000,0000,0000,,The first one, to my knowledge, to do so.
Dialogue: 0,0:15:47.85,0:15:49.40,Default,,0000,0000,0000,,And these systems are in use
Dialogue: 0,0:15:49.40,0:15:52.82,Default,,0000,0000,0000,,in 50+ cities in the U.S.
Dialogue: 0,0:15:52.82,0:15:56.54,Default,,0000,0000,0000,,So, why do police departments buy this?
Dialogue: 0,0:15:56.54,0:15:57.76,Default,,0000,0000,0000,,Some police departments are interesting in
Dialogue: 0,0:15:57.76,0:16:00.50,Default,,0000,0000,0000,,buying systems like this, because they're marketed
Dialogue: 0,0:16:00.50,0:16:02.66,Default,,0000,0000,0000,,as impartial systems,
Dialogue: 0,0:16:02.66,0:16:06.20,Default,,0000,0000,0000,,so it's a way to police in an unbiased way.
Dialogue: 0,0:16:06.20,0:16:08.04,Default,,0000,0000,0000,,And so, these companies make
Dialogue: 0,0:16:08.04,0:16:08.67,Default,,0000,0000,0000,,statements like this--
Dialogue: 0,0:16:08.67,0:16:10.80,Default,,0000,0000,0000,,by the way, the references\Nwill all be at the end,
Dialogue: 0,0:16:10.80,0:16:12.56,Default,,0000,0000,0000,,and they'll be on the slides--
Dialogue: 0,0:16:12.56,0:16:13.37,Default,,0000,0000,0000,,So, for example
Dialogue: 0,0:16:13.37,0:16:16.11,Default,,0000,0000,0000,,the predictive crime analytics from Hitachi
Dialogue: 0,0:16:16.11,0:16:17.61,Default,,0000,0000,0000,,claims that the system is anonymous,
Dialogue: 0,0:16:17.61,0:16:19.35,Default,,0000,0000,0000,,because it shows you an area,
Dialogue: 0,0:16:19.35,0:16:23.06,Default,,0000,0000,0000,,it doesn't show you\Nto look for a particular person.
Dialogue: 0,0:16:23.06,0:16:25.70,Default,,0000,0000,0000,,and PredPol reassures people that
Dialogue: 0,0:16:25.70,0:16:29.56,Default,,0000,0000,0000,,it eliminates any liberties or profiling concerns.
Dialogue: 0,0:16:29.56,0:16:32.27,Default,,0000,0000,0000,,And HunchLab notes that the system
Dialogue: 0,0:16:32.27,0:16:35.17,Default,,0000,0000,0000,,fairly represents priorities for public safety
Dialogue: 0,0:16:35.17,0:16:38.77,Default,,0000,0000,0000,,and is unbiased by race\Nor ethnicity, for example.
Dialogue: 0,0:16:38.77,0:16:43.53,Default,,0000,0000,0000,,So, let's take a minute\Nto describe in more detail
Dialogue: 0,0:16:43.53,0:16:48.10,Default,,0000,0000,0000,,what we mean when we talk about fairness.
Dialogue: 0,0:16:48.10,0:16:51.30,Default,,0000,0000,0000,,So, when we talk about fairness,
Dialogue: 0,0:16:51.30,0:16:52.74,Default,,0000,0000,0000,,we mean a few things.
Dialogue: 0,0:16:52.74,0:16:56.07,Default,,0000,0000,0000,,So, one is fairness with respect to individuals:
Dialogue: 0,0:16:56.07,0:16:58.04,Default,,0000,0000,0000,,so if I'm very similar to somebody
Dialogue: 0,0:16:58.04,0:17:00.17,Default,,0000,0000,0000,,and we go through some process
Dialogue: 0,0:17:00.17,0:17:03.43,Default,,0000,0000,0000,,and there is two very different\Noutcomes to that process
Dialogue: 0,0:17:03.43,0:17:05.68,Default,,0000,0000,0000,,we would consider that to be unfair.
Dialogue: 0,0:17:05.68,0:17:07.93,Default,,0000,0000,0000,,So, we want similar people to be treated
Dialogue: 0,0:17:07.93,0:17:09.54,Default,,0000,0000,0000,,in a similar way.
Dialogue: 0,0:17:09.54,0:17:13.08,Default,,0000,0000,0000,,But, there are certain protected attributes
Dialogue: 0,0:17:13.08,0:17:15.20,Default,,0000,0000,0000,,that we wouldn't want someone
Dialogue: 0,0:17:15.20,0:17:17.10,Default,,0000,0000,0000,,to discriminate based on.
Dialogue: 0,0:17:17.10,0:17:20.07,Default,,0000,0000,0000,,And so, there's this other property,\NGroup Fairness.
Dialogue: 0,0:17:20.07,0:17:22.25,Default,,0000,0000,0000,,So, we can look at the statistical parity
Dialogue: 0,0:17:22.25,0:17:25.44,Default,,0000,0000,0000,,between groups, based on gender, race, etc.
Dialogue: 0,0:17:25.44,0:17:28.05,Default,,0000,0000,0000,,and see if they're treated in a similar way.
Dialogue: 0,0:17:28.05,0:17:30.41,Default,,0000,0000,0000,,And we might not expect that in some cases,
Dialogue: 0,0:17:30.41,0:17:32.43,Default,,0000,0000,0000,,for example if the base rates in each group
Dialogue: 0,0:17:32.43,0:17:34.66,Default,,0000,0000,0000,,are very different.
Dialogue: 0,0:17:34.66,0:17:36.89,Default,,0000,0000,0000,,And then there's also Fairness in Errors.
Dialogue: 0,0:17:36.89,0:17:40.08,Default,,0000,0000,0000,,All predictive systems are gonna make errors,
Dialogue: 0,0:17:40.08,0:17:42.99,Default,,0000,0000,0000,,and if the errors are concentrated,
Dialogue: 0,0:17:42.99,0:17:46.40,Default,,0000,0000,0000,,then that may also represent unfairness.
Dialogue: 0,0:17:46.40,0:17:50.15,Default,,0000,0000,0000,,And so this concern arose recently with Facebook
Dialogue: 0,0:17:50.15,0:17:52.29,Default,,0000,0000,0000,,because people with Native American names
Dialogue: 0,0:17:52.29,0:17:54.39,Default,,0000,0000,0000,,had their profiles flagged as fraudulent
Dialogue: 0,0:17:54.39,0:17:58.76,Default,,0000,0000,0000,,far more often than those\Nwith White American names.
Dialogue: 0,0:17:58.76,0:18:00.56,Default,,0000,0000,0000,,So these are the sorts of things\Nthat we worry about
Dialogue: 0,0:18:00.56,0:18:02.19,Default,,0000,0000,0000,,and each of these are metrics,
Dialogue: 0,0:18:02.19,0:18:04.24,Default,,0000,0000,0000,,and if you're interested more you should
Dialogue: 0,0:18:04.24,0:18:06.16,Default,,0000,0000,0000,,check those 2 papers out.
Dialogue: 0,0:18:06.16,0:18:10.64,Default,,0000,0000,0000,,So, how can potential issues\Nwith predictive policing
Dialogue: 0,0:18:10.64,0:18:13.85,Default,,0000,0000,0000,,have implications for these principles?
Dialogue: 0,0:18:13.85,0:18:18.56,Default,,0000,0000,0000,,So, one problem is\Nthe training data that's used.
Dialogue: 0,0:18:18.56,0:18:21.06,Default,,0000,0000,0000,,Some of these systems only use crime statistics,
Dialogue: 0,0:18:21.06,0:18:23.60,Default,,0000,0000,0000,,other systems-- all of them use crime statistics
Dialogue: 0,0:18:23.60,0:18:25.62,Default,,0000,0000,0000,,in some way.
Dialogue: 0,0:18:25.62,0:18:31.42,Default,,0000,0000,0000,,So, one problem is that crime databases
Dialogue: 0,0:18:31.42,0:18:34.83,Default,,0000,0000,0000,,contain only crimes that've been detected.
Dialogue: 0,0:18:34.83,0:18:38.63,Default,,0000,0000,0000,,Right? So, the police are only gonna detect
Dialogue: 0,0:18:38.63,0:18:41.01,Default,,0000,0000,0000,,crimes that they know are happening,
Dialogue: 0,0:18:41.01,0:18:44.11,Default,,0000,0000,0000,,either through patrol and their own investigation
Dialogue: 0,0:18:44.11,0:18:46.32,Default,,0000,0000,0000,,or because they've been alerted to crime,
Dialogue: 0,0:18:46.32,0:18:48.79,Default,,0000,0000,0000,,for example by a citizen calling the police.
Dialogue: 0,0:18:48.79,0:18:52.18,Default,,0000,0000,0000,,So, a citizen has to feel like\Nthey {\i1}can{\i0} call the police,
Dialogue: 0,0:18:52.18,0:18:54.02,Default,,0000,0000,0000,,like that's a good idea.
Dialogue: 0,0:18:54.02,0:18:58.79,Default,,0000,0000,0000,,So, some crimes suffer\Nfrom this problem less than others:
Dialogue: 0,0:18:58.79,0:19:02.25,Default,,0000,0000,0000,,for example, gun violence\Nis much easier to detect
Dialogue: 0,0:19:02.25,0:19:03.64,Default,,0000,0000,0000,,relative to fraud, for example,
Dialogue: 0,0:19:03.64,0:19:07.51,Default,,0000,0000,0000,,which is very difficult to detect.
Dialogue: 0,0:19:07.51,0:19:11.94,Default,,0000,0000,0000,,Now the racial profiling aspect\Nof this might come in
Dialogue: 0,0:19:11.94,0:19:15.59,Default,,0000,0000,0000,,because of biased policing in the past.
Dialogue: 0,0:19:15.59,0:19:19.100,Default,,0000,0000,0000,,So, for example, for marijuana arrests,
Dialogue: 0,0:19:19.100,0:19:22.62,Default,,0000,0000,0000,,black people are arrested in the U.S. at rates
Dialogue: 0,0:19:22.62,0:19:25.12,Default,,0000,0000,0000,,4 times that of white people,
Dialogue: 0,0:19:25.12,0:19:27.96,Default,,0000,0000,0000,,even though there is statistical parity
Dialogue: 0,0:19:27.96,0:19:31.39,Default,,0000,0000,0000,,with these 2 groups, to within a few percent.
Dialogue: 0,0:19:31.39,0:19:35.82,Default,,0000,0000,0000,,So, this is where problems can arise.
Dialogue: 0,0:19:35.82,0:19:37.16,Default,,0000,0000,0000,,So, let's go back to this
Dialogue: 0,0:19:37.16,0:19:38.75,Default,,0000,0000,0000,,geographic-level predictive policing.
Dialogue: 0,0:19:38.75,0:19:42.46,Default,,0000,0000,0000,,So the danger here is that, unless this system
Dialogue: 0,0:19:42.46,0:19:44.30,Default,,0000,0000,0000,,is very carefully constructed,
Dialogue: 0,0:19:44.30,0:19:47.09,Default,,0000,0000,0000,,this sort of crime area ranking might
Dialogue: 0,0:19:47.09,0:19:49.02,Default,,0000,0000,0000,,again become a self-fulling prophecy.
Dialogue: 0,0:19:49.02,0:19:51.46,Default,,0000,0000,0000,,If you send police officers to these areas,
Dialogue: 0,0:19:51.46,0:19:53.22,Default,,0000,0000,0000,,you further scrutinize them,
Dialogue: 0,0:19:53.22,0:19:55.66,Default,,0000,0000,0000,,and then again you're only detecting a subset
Dialogue: 0,0:19:55.66,0:19:57.98,Default,,0000,0000,0000,,of crimes, and the cycle continues.
Dialogue: 0,0:19:57.98,0:20:02.14,Default,,0000,0000,0000,,So, one obvious issue is that
Dialogue: 0,0:20:02.14,0:20:07.60,Default,,0000,0000,0000,,this statement about geographic-based\Ncrime prediction
Dialogue: 0,0:20:07.60,0:20:10.23,Default,,0000,0000,0000,,being anonymous is not true,
Dialogue: 0,0:20:10.23,0:20:13.16,Default,,0000,0000,0000,,because race and location are very strongly
Dialogue: 0,0:20:13.16,0:20:14.84,Default,,0000,0000,0000,,correlated in the U.S.
Dialogue: 0,0:20:14.84,0:20:16.61,Default,,0000,0000,0000,,And this is something that machine-learning\Nsystems
Dialogue: 0,0:20:16.61,0:20:20.05,Default,,0000,0000,0000,,can potentially learn.
Dialogue: 0,0:20:20.05,0:20:23.04,Default,,0000,0000,0000,,Another issue is that, for example,
Dialogue: 0,0:20:23.04,0:20:25.58,Default,,0000,0000,0000,,for individual fairness, one of my homes
Dialogue: 0,0:20:25.58,0:20:27.60,Default,,0000,0000,0000,,sits within one of these boxes.
Dialogue: 0,0:20:27.60,0:20:29.95,Default,,0000,0000,0000,,Some of these boxes\Nin these systems are very small,
Dialogue: 0,0:20:29.95,0:20:33.40,Default,,0000,0000,0000,,for example PredPol is 500ft x 500ft,
Dialogue: 0,0:20:33.40,0:20:36.35,Default,,0000,0000,0000,,so it's maybe only a few houses.
Dialogue: 0,0:20:36.35,0:20:39.15,Default,,0000,0000,0000,,So, the implications of this system are that
Dialogue: 0,0:20:39.15,0:20:40.85,Default,,0000,0000,0000,,you have police officers maybe sitting
Dialogue: 0,0:20:40.85,0:20:42.98,Default,,0000,0000,0000,,in a police cruiser outside your home
Dialogue: 0,0:20:42.98,0:20:45.45,Default,,0000,0000,0000,,and a few doors down someone
Dialogue: 0,0:20:45.45,0:20:46.80,Default,,0000,0000,0000,,may not be within that box,
Dialogue: 0,0:20:46.80,0:20:48.16,Default,,0000,0000,0000,,and doesn't have this.
Dialogue: 0,0:20:48.16,0:20:51.40,Default,,0000,0000,0000,,So, that may represent unfairness.
Dialogue: 0,0:20:51.40,0:20:54.93,Default,,0000,0000,0000,,So, there are real questions here,
Dialogue: 0,0:20:54.93,0:20:57.72,Default,,0000,0000,0000,,especially because there's no opt-out.
Dialogue: 0,0:20:57.72,0:21:00.06,Default,,0000,0000,0000,,There's no way to opt-out of this system:
Dialogue: 0,0:21:00.06,0:21:02.24,Default,,0000,0000,0000,,if you live in a city that has this,
Dialogue: 0,0:21:02.24,0:21:04.91,Default,,0000,0000,0000,,then you have to deal with it.
Dialogue: 0,0:21:04.91,0:21:07.23,Default,,0000,0000,0000,,So, it's quite difficult to find out
Dialogue: 0,0:21:07.23,0:21:09.88,Default,,0000,0000,0000,,what's really going on
Dialogue: 0,0:21:09.88,0:21:11.17,Default,,0000,0000,0000,,because the algorithm is secret.
Dialogue: 0,0:21:11.17,0:21:13.05,Default,,0000,0000,0000,,And, in most cases, we don't know
Dialogue: 0,0:21:13.05,0:21:14.79,Default,,0000,0000,0000,,the full details of the inputs.
Dialogue: 0,0:21:14.79,0:21:16.68,Default,,0000,0000,0000,,We have some idea\Nabout what features are used,
Dialogue: 0,0:21:16.68,0:21:17.97,Default,,0000,0000,0000,,but that's about it.
Dialogue: 0,0:21:17.97,0:21:19.51,Default,,0000,0000,0000,,We also don't know the output.
Dialogue: 0,0:21:19.51,0:21:21.90,Default,,0000,0000,0000,,That would be knowing police allocation,
Dialogue: 0,0:21:21.90,0:21:23.18,Default,,0000,0000,0000,,police strategies,
Dialogue: 0,0:21:23.18,0:21:26.30,Default,,0000,0000,0000,,and in order to nail down\Nwhat's really going on here
Dialogue: 0,0:21:26.30,0:21:28.61,Default,,0000,0000,0000,,in order to verify the validity of
Dialogue: 0,0:21:28.61,0:21:30.01,Default,,0000,0000,0000,,these companies' claims,
Dialogue: 0,0:21:30.01,0:21:33.80,Default,,0000,0000,0000,,it may be necessary\Nto have a 3rd party come in,
Dialogue: 0,0:21:33.80,0:21:35.63,Default,,0000,0000,0000,,examine the inputs and outputs of the system,
Dialogue: 0,0:21:35.63,0:21:37.59,Default,,0000,0000,0000,,and say concretely what's going on.
Dialogue: 0,0:21:37.59,0:21:39.46,Default,,0000,0000,0000,,And if everything is fine and dandy
Dialogue: 0,0:21:39.46,0:21:40.93,Default,,0000,0000,0000,,then this shouldn't be a problem.
Dialogue: 0,0:21:40.93,0:21:43.62,Default,,0000,0000,0000,,So, that's potentially one role that
Dialogue: 0,0:21:43.62,0:21:44.77,Default,,0000,0000,0000,,advocates can play.
Dialogue: 0,0:21:44.77,0:21:46.72,Default,,0000,0000,0000,,Maybe we should start pushing for audits
Dialogue: 0,0:21:46.72,0:21:48.82,Default,,0000,0000,0000,,of systems that are used in this way.
Dialogue: 0,0:21:48.82,0:21:50.97,Default,,0000,0000,0000,,These could have serious implications
Dialogue: 0,0:21:50.97,0:21:52.68,Default,,0000,0000,0000,,for peoples' lives.
Dialogue: 0,0:21:52.68,0:21:55.25,Default,,0000,0000,0000,,So, we'll return\Nto this idea a little bit later,
Dialogue: 0,0:21:55.25,0:21:58.21,Default,,0000,0000,0000,,but for now this leads us\Nnicely to Transparency.
Dialogue: 0,0:21:58.21,0:21:59.42,Default,,0000,0000,0000,,So, we wanna know
Dialogue: 0,0:21:59.42,0:22:01.93,Default,,0000,0000,0000,,what these systems are doing.
Dialogue: 0,0:22:01.93,0:22:04.73,Default,,0000,0000,0000,,But it's very hard,\Nfor the reasons described earlier,
Dialogue: 0,0:22:04.73,0:22:06.14,Default,,0000,0000,0000,,but even in the case of something like
Dialogue: 0,0:22:06.14,0:22:09.85,Default,,0000,0000,0000,,trying to understand Google's search algorithm,
Dialogue: 0,0:22:09.85,0:22:11.68,Default,,0000,0000,0000,,it's difficult because it's personalized.
Dialogue: 0,0:22:11.68,0:22:13.53,Default,,0000,0000,0000,,So, by construction, each user is
Dialogue: 0,0:22:13.53,0:22:15.32,Default,,0000,0000,0000,,only seeing one endpoint.
Dialogue: 0,0:22:15.32,0:22:18.17,Default,,0000,0000,0000,,So, it's a very isolating system.
Dialogue: 0,0:22:18.17,0:22:20.35,Default,,0000,0000,0000,,What do other people see?
Dialogue: 0,0:22:20.35,0:22:22.41,Default,,0000,0000,0000,,And one reason it's difficult to make
Dialogue: 0,0:22:22.41,0:22:24.10,Default,,0000,0000,0000,,some of these systems transparent
Dialogue: 0,0:22:24.10,0:22:26.68,Default,,0000,0000,0000,,is because of, simply, the complexity
Dialogue: 0,0:22:26.68,0:22:27.95,Default,,0000,0000,0000,,of the algorithms.
Dialogue: 0,0:22:27.95,0:22:30.31,Default,,0000,0000,0000,,So, an algorithm can become so complex that
Dialogue: 0,0:22:30.31,0:22:31.67,Default,,0000,0000,0000,,it's difficult to comprehend,
Dialogue: 0,0:22:31.67,0:22:33.29,Default,,0000,0000,0000,,even for the designer of the system,
Dialogue: 0,0:22:33.29,0:22:35.51,Default,,0000,0000,0000,,or the implementer of the system.
Dialogue: 0,0:22:35.51,0:22:38.42,Default,,0000,0000,0000,,The designed might know that this algorithm
Dialogue: 0,0:22:38.42,0:22:42.89,Default,,0000,0000,0000,,maximizes some metric-- say, accuracy,
Dialogue: 0,0:22:42.89,0:22:44.57,Default,,0000,0000,0000,,but they may not always have a solid
Dialogue: 0,0:22:44.57,0:22:46.78,Default,,0000,0000,0000,,understanding of what the algorithm is doing
Dialogue: 0,0:22:46.78,0:22:48.33,Default,,0000,0000,0000,,for all inputs.
Dialogue: 0,0:22:48.33,0:22:50.97,Default,,0000,0000,0000,,Certainly with respect to fairness.
Dialogue: 0,0:22:50.97,0:22:55.76,Default,,0000,0000,0000,,So, in some cases,\Nit might not be appropriate to use
Dialogue: 0,0:22:55.76,0:22:57.38,Default,,0000,0000,0000,,an extremely complex model.
Dialogue: 0,0:22:57.38,0:22:59.53,Default,,0000,0000,0000,,It might be better to use a simpler system
Dialogue: 0,0:22:59.53,0:23:02.91,Default,,0000,0000,0000,,with human-interpretable features.
Dialogue: 0,0:23:02.91,0:23:04.75,Default,,0000,0000,0000,,Another issue that arises
Dialogue: 0,0:23:04.75,0:23:07.56,Default,,0000,0000,0000,,from the opacity of these systems
Dialogue: 0,0:23:07.56,0:23:09.41,Default,,0000,0000,0000,,and the centralized control
Dialogue: 0,0:23:09.41,0:23:11.86,Default,,0000,0000,0000,,is that it makes them very influential.
Dialogue: 0,0:23:11.86,0:23:13.95,Default,,0000,0000,0000,,And thus, an excellent target
Dialogue: 0,0:23:13.95,0:23:16.21,Default,,0000,0000,0000,,for manipulation or tampering.
Dialogue: 0,0:23:16.21,0:23:18.48,Default,,0000,0000,0000,,So, this might be tampering that is done
Dialogue: 0,0:23:18.48,0:23:21.95,Default,,0000,0000,0000,,from an organization that controls the system,
Dialogue: 0,0:23:21.95,0:23:23.77,Default,,0000,0000,0000,,or an insider at one of the organizations,
Dialogue: 0,0:23:23.77,0:23:27.14,Default,,0000,0000,0000,,or anyone who's able to compromise their security.
Dialogue: 0,0:23:27.14,0:23:30.25,Default,,0000,0000,0000,,So, this is an interesting academic work
Dialogue: 0,0:23:30.25,0:23:32.10,Default,,0000,0000,0000,,that looked at the possibility of
Dialogue: 0,0:23:32.10,0:23:34.16,Default,,0000,0000,0000,,slightly modifying search rankings
Dialogue: 0,0:23:34.16,0:23:36.62,Default,,0000,0000,0000,,to shift people's political views.
Dialogue: 0,0:23:36.62,0:23:39.01,Default,,0000,0000,0000,,So, since people are most likely to
Dialogue: 0,0:23:39.01,0:23:41.33,Default,,0000,0000,0000,,click on the top search results,
Dialogue: 0,0:23:41.33,0:23:44.43,Default,,0000,0000,0000,,so 90% of clicks go to the\Nfirst page of search results,
Dialogue: 0,0:23:44.43,0:23:46.72,Default,,0000,0000,0000,,then perhaps by reshuffling\Nthings a little bit,
Dialogue: 0,0:23:46.72,0:23:48.73,Default,,0000,0000,0000,,or maybe dropping some search results,
Dialogue: 0,0:23:48.73,0:23:50.27,Default,,0000,0000,0000,,you can influence people's views
Dialogue: 0,0:23:50.27,0:23:51.68,Default,,0000,0000,0000,,in a coherent way,
Dialogue: 0,0:23:51.68,0:23:53.09,Default,,0000,0000,0000,,and maybe you can make it so subtle
Dialogue: 0,0:23:53.09,0:23:55.75,Default,,0000,0000,0000,,that no one is able to notice.
Dialogue: 0,0:23:55.75,0:23:57.25,Default,,0000,0000,0000,,So in this academic study,
Dialogue: 0,0:23:57.25,0:24:00.35,Default,,0000,0000,0000,,they did an experiment
Dialogue: 0,0:24:00.35,0:24:02.07,Default,,0000,0000,0000,,in the 2014 Indian election.
Dialogue: 0,0:24:02.07,0:24:04.22,Default,,0000,0000,0000,,So they used real voters,
Dialogue: 0,0:24:04.22,0:24:06.45,Default,,0000,0000,0000,,and they kept the size\Nof the experiment small enough
Dialogue: 0,0:24:06.45,0:24:08.19,Default,,0000,0000,0000,,that it was not going to influence the outcome
Dialogue: 0,0:24:08.19,0:24:10.09,Default,,0000,0000,0000,,of the election.
Dialogue: 0,0:24:10.09,0:24:12.14,Default,,0000,0000,0000,,So the researchers took people,
Dialogue: 0,0:24:12.14,0:24:14.23,Default,,0000,0000,0000,,they determined their political leaning,
Dialogue: 0,0:24:14.23,0:24:17.43,Default,,0000,0000,0000,,and they segmented them into\Ncontrol and treatment groups,
Dialogue: 0,0:24:17.43,0:24:19.27,Default,,0000,0000,0000,,where the treatment was manipulation
Dialogue: 0,0:24:19.27,0:24:21.21,Default,,0000,0000,0000,,of the search ranking results,
Dialogue: 0,0:24:21.21,0:24:24.41,Default,,0000,0000,0000,,And then they had these people\Nbrowse the web.
Dialogue: 0,0:24:24.41,0:24:25.97,Default,,0000,0000,0000,,And what they found, is that
Dialogue: 0,0:24:25.97,0:24:28.23,Default,,0000,0000,0000,,this mechanism is very effective at shifting
Dialogue: 0,0:24:28.23,0:24:30.43,Default,,0000,0000,0000,,people's voter preferences.
Dialogue: 0,0:24:30.43,0:24:33.65,Default,,0000,0000,0000,,So, in this study, they were able to introduce
Dialogue: 0,0:24:33.65,0:24:36.85,Default,,0000,0000,0000,,a 20% shift in voter preferences.
Dialogue: 0,0:24:36.85,0:24:39.30,Default,,0000,0000,0000,,Even alerting users to the fact that this
Dialogue: 0,0:24:39.30,0:24:41.73,Default,,0000,0000,0000,,was going to be done, telling them
Dialogue: 0,0:24:41.73,0:24:44.05,Default,,0000,0000,0000,,"we are going to manipulate your search results,"
Dialogue: 0,0:24:44.05,0:24:45.73,Default,,0000,0000,0000,,"really pay attention,"
Dialogue: 0,0:24:45.73,0:24:49.10,Default,,0000,0000,0000,,they were totally unable to decrease
Dialogue: 0,0:24:49.10,0:24:50.86,Default,,0000,0000,0000,,the magnitude of the effect.
Dialogue: 0,0:24:50.86,0:24:55.11,Default,,0000,0000,0000,,So, the margins of error in many elections
Dialogue: 0,0:24:55.11,0:24:57.67,Default,,0000,0000,0000,,is incredibly small,
Dialogue: 0,0:24:57.67,0:24:59.93,Default,,0000,0000,0000,,and the authors estimate that this shift
Dialogue: 0,0:24:59.93,0:25:02.01,Default,,0000,0000,0000,,could change the outcome of about
Dialogue: 0,0:25:02.01,0:25:07.11,Default,,0000,0000,0000,,25% of elections worldwide, if this were done.
Dialogue: 0,0:25:07.11,0:25:10.92,Default,,0000,0000,0000,,And the bias is so small that no one can tell.
Dialogue: 0,0:25:10.92,0:25:14.28,Default,,0000,0000,0000,,So, all humans, no matter how smart
Dialogue: 0,0:25:14.28,0:25:17.11,Default,,0000,0000,0000,,and resistant to manipulation\Nwe think we are,
Dialogue: 0,0:25:17.11,0:25:21.91,Default,,0000,0000,0000,,all of us are subject to this sort of manipulation,
Dialogue: 0,0:25:21.91,0:25:24.32,Default,,0000,0000,0000,,and we really can't tell.
Dialogue: 0,0:25:24.32,0:25:27.13,Default,,0000,0000,0000,,So, I'm not saying that this is occurring,
Dialogue: 0,0:25:27.13,0:25:31.39,Default,,0000,0000,0000,,but right now there is no\Nregulation to stop this,
Dialogue: 0,0:25:31.39,0:25:34.41,Default,,0000,0000,0000,,there is no way we could reliably detect this,
Dialogue: 0,0:25:34.41,0:25:37.21,Default,,0000,0000,0000,,so there's a huge amount of power here.
Dialogue: 0,0:25:37.21,0:25:39.78,Default,,0000,0000,0000,,So, something to think about.
Dialogue: 0,0:25:39.78,0:25:42.71,Default,,0000,0000,0000,,But it's not only corporations that are interested
Dialogue: 0,0:25:42.71,0:25:47.27,Default,,0000,0000,0000,,in this sort of behavioral manipulation.
Dialogue: 0,0:25:47.27,0:25:51.12,Default,,0000,0000,0000,,In 2010, UK Prime Minister David Cameron
Dialogue: 0,0:25:51.12,0:25:54.97,Default,,0000,0000,0000,,created this UK Behavioural Insights Team,
Dialogue: 0,0:25:54.97,0:25:57.27,Default,,0000,0000,0000,,which is informally called the Nudge Unit.
Dialogue: 0,0:25:57.27,0:26:01.49,Default,,0000,0000,0000,,And so what they do is\Nthey use behavioral science
Dialogue: 0,0:26:01.49,0:26:04.77,Default,,0000,0000,0000,,and this predictive analytics approach,
Dialogue: 0,0:26:04.77,0:26:06.12,Default,,0000,0000,0000,,with experimentation,
Dialogue: 0,0:26:06.12,0:26:07.94,Default,,0000,0000,0000,,to have people make better decisions
Dialogue: 0,0:26:07.94,0:26:09.69,Default,,0000,0000,0000,,for themselves and society--
Dialogue: 0,0:26:09.69,0:26:11.99,Default,,0000,0000,0000,,as determined by the UK government.
Dialogue: 0,0:26:11.99,0:26:14.27,Default,,0000,0000,0000,,And as of a few months ago,
Dialogue: 0,0:26:14.27,0:26:16.85,Default,,0000,0000,0000,,after an executive order signed by Obama
Dialogue: 0,0:26:16.85,0:26:19.35,Default,,0000,0000,0000,,in September, the United States now has
Dialogue: 0,0:26:19.35,0:26:21.43,Default,,0000,0000,0000,,its own Nudge Unit.
Dialogue: 0,0:26:21.43,0:26:24.01,Default,,0000,0000,0000,,So, to be clear, I don't think that this is
Dialogue: 0,0:26:24.01,0:26:25.54,Default,,0000,0000,0000,,some sort of malicious plot.
Dialogue: 0,0:26:25.54,0:26:27.44,Default,,0000,0000,0000,,I think that there {\i1}can{\i0} be huge value
Dialogue: 0,0:26:27.44,0:26:29.49,Default,,0000,0000,0000,,in these sorts of initiatives,
Dialogue: 0,0:26:29.49,0:26:31.33,Default,,0000,0000,0000,,positively impacting people's lives,
Dialogue: 0,0:26:31.33,0:26:34.18,Default,,0000,0000,0000,,but when this sort of behavioral manipulation
Dialogue: 0,0:26:34.18,0:26:37.29,Default,,0000,0000,0000,,is being done, in part openly,
Dialogue: 0,0:26:37.29,0:26:39.46,Default,,0000,0000,0000,,oversight is pretty important,
Dialogue: 0,0:26:39.46,0:26:41.70,Default,,0000,0000,0000,,and we really need to consider
Dialogue: 0,0:26:41.70,0:26:46.09,Default,,0000,0000,0000,,what these systems are optimizing for.
Dialogue: 0,0:26:46.09,0:26:47.85,Default,,0000,0000,0000,,And that's something that we might
Dialogue: 0,0:26:47.85,0:26:52.09,Default,,0000,0000,0000,,not always know, or at least understand,
Dialogue: 0,0:26:52.09,0:26:54.45,Default,,0000,0000,0000,,so for example, for industry,
Dialogue: 0,0:26:54.45,0:26:57.68,Default,,0000,0000,0000,,we do have a pretty good understanding there:
Dialogue: 0,0:26:57.68,0:26:59.81,Default,,0000,0000,0000,,industry cares about optimizing for
Dialogue: 0,0:26:59.81,0:27:01.96,Default,,0000,0000,0000,,the time spent on the website,
Dialogue: 0,0:27:01.96,0:27:04.93,Default,,0000,0000,0000,,Facebook wants you to spend more time on Facebook,
Dialogue: 0,0:27:04.93,0:27:06.95,Default,,0000,0000,0000,,they want you to click on ads,
Dialogue: 0,0:27:06.95,0:27:09.11,Default,,0000,0000,0000,,click on newsfeed items,
Dialogue: 0,0:27:09.11,0:27:11.30,Default,,0000,0000,0000,,they want you to like things.
Dialogue: 0,0:27:11.30,0:27:14.31,Default,,0000,0000,0000,,And, fundamentally: profit.
Dialogue: 0,0:27:14.31,0:27:17.60,Default,,0000,0000,0000,,So, already this has some serious implications,
Dialogue: 0,0:27:17.60,0:27:19.69,Default,,0000,0000,0000,,and this had pretty serious implications
Dialogue: 0,0:27:19.69,0:27:22.19,Default,,0000,0000,0000,,in the last 10 years, in media for example.
Dialogue: 0,0:27:22.19,0:27:25.12,Default,,0000,0000,0000,,The optimizing for click-through rate in journalism
Dialogue: 0,0:27:25.12,0:27:26.63,Default,,0000,0000,0000,,has produced a race to the bottom
Dialogue: 0,0:27:26.63,0:27:28.04,Default,,0000,0000,0000,,in terms of quality.
Dialogue: 0,0:27:28.04,0:27:30.92,Default,,0000,0000,0000,,And another issue is that optimizing
Dialogue: 0,0:27:30.92,0:27:34.59,Default,,0000,0000,0000,,for what people like might not always be
Dialogue: 0,0:27:34.59,0:27:35.84,Default,,0000,0000,0000,,the best approach.
Dialogue: 0,0:27:35.84,0:27:38.86,Default,,0000,0000,0000,,So, Facebook officials have said publicly
Dialogue: 0,0:27:38.86,0:27:41.28,Default,,0000,0000,0000,,about how Facebook's goal is to make you happy,
Dialogue: 0,0:27:41.28,0:27:43.15,Default,,0000,0000,0000,,they want you to open that newsfeed
Dialogue: 0,0:27:43.15,0:27:45.08,Default,,0000,0000,0000,,and just feel great.
Dialogue: 0,0:27:45.08,0:27:47.38,Default,,0000,0000,0000,,But, there's an issue there, right?
Dialogue: 0,0:27:47.38,0:27:50.17,Default,,0000,0000,0000,,Because people get their news,
Dialogue: 0,0:27:50.17,0:27:52.37,Default,,0000,0000,0000,,like 40% of people according to Pew Research,
Dialogue: 0,0:27:52.37,0:27:54.60,Default,,0000,0000,0000,,get their news from Facebook.
Dialogue: 0,0:27:54.60,0:27:58.46,Default,,0000,0000,0000,,So, if people don't want to see
Dialogue: 0,0:27:58.46,0:28:01.24,Default,,0000,0000,0000,,war and corpses,\Nbecause it makes them feel sad,
Dialogue: 0,0:28:01.24,0:28:04.18,Default,,0000,0000,0000,,so this is not a system that is gonna optimize
Dialogue: 0,0:28:04.18,0:28:07.15,Default,,0000,0000,0000,,for an informed population.
Dialogue: 0,0:28:07.15,0:28:09.36,Default,,0000,0000,0000,,It's not gonna produce a population that is
Dialogue: 0,0:28:09.36,0:28:11.47,Default,,0000,0000,0000,,ready to engage in civic life.
Dialogue: 0,0:28:11.47,0:28:13.06,Default,,0000,0000,0000,,It's gonna produce an amused populations
Dialogue: 0,0:28:13.06,0:28:16.81,Default,,0000,0000,0000,,whose time is occupied by cat pictures.
Dialogue: 0,0:28:16.81,0:28:19.16,Default,,0000,0000,0000,,So, in politics, we have a similar
Dialogue: 0,0:28:19.16,0:28:21.27,Default,,0000,0000,0000,,optimization problem that's occurring.
Dialogue: 0,0:28:21.27,0:28:23.77,Default,,0000,0000,0000,,So, these political campaigns that use
Dialogue: 0,0:28:23.77,0:28:26.77,Default,,0000,0000,0000,,these predictive systems,
Dialogue: 0,0:28:26.77,0:28:28.67,Default,,0000,0000,0000,,are optimizing for votes for the desired candidate,
Dialogue: 0,0:28:28.67,0:28:30.20,Default,,0000,0000,0000,,of course.
Dialogue: 0,0:28:30.20,0:28:33.50,Default,,0000,0000,0000,,So, instead of a political campaign being
Dialogue: 0,0:28:33.50,0:28:36.14,Default,,0000,0000,0000,,--well, maybe this is a naive view, but--
Dialogue: 0,0:28:36.14,0:28:38.07,Default,,0000,0000,0000,,being an open discussion of the issues
Dialogue: 0,0:28:38.07,0:28:39.83,Default,,0000,0000,0000,,facing the country,
Dialogue: 0,0:28:39.83,0:28:43.20,Default,,0000,0000,0000,,it becomes this micro-targeted\Npersuasion game,
Dialogue: 0,0:28:43.20,0:28:44.67,Default,,0000,0000,0000,,and the people that get targeted
Dialogue: 0,0:28:44.67,0:28:47.35,Default,,0000,0000,0000,,are a very small subset of all people,
Dialogue: 0,0:28:47.35,0:28:49.40,Default,,0000,0000,0000,,and it's only gonna be people that are
Dialogue: 0,0:28:49.40,0:28:51.41,Default,,0000,0000,0000,,you know, on the edge, maybe disinterested,
Dialogue: 0,0:28:51.41,0:28:54.40,Default,,0000,0000,0000,,those are the people that are gonna get attention
Dialogue: 0,0:28:54.40,0:28:58.84,Default,,0000,0000,0000,,from political candidates.
Dialogue: 0,0:28:58.84,0:29:01.87,Default,,0000,0000,0000,,In policy, as with these Nudge Units,
Dialogue: 0,0:29:01.87,0:29:03.54,Default,,0000,0000,0000,,they're being used to enable
Dialogue: 0,0:29:03.54,0:29:06.11,Default,,0000,0000,0000,,better use of government services.
Dialogue: 0,0:29:06.11,0:29:07.42,Default,,0000,0000,0000,,There are some good projects that have
Dialogue: 0,0:29:07.42,0:29:09.42,Default,,0000,0000,0000,,come out of this:
Dialogue: 0,0:29:09.42,0:29:11.41,Default,,0000,0000,0000,,increasing voter registration,
Dialogue: 0,0:29:11.41,0:29:12.74,Default,,0000,0000,0000,,improving health outcomes,
Dialogue: 0,0:29:12.74,0:29:14.42,Default,,0000,0000,0000,,improving education outcomes.
Dialogue: 0,0:29:14.42,0:29:16.42,Default,,0000,0000,0000,,But some of these predictive systems
Dialogue: 0,0:29:16.42,0:29:18.23,Default,,0000,0000,0000,,that we're starting to see in government
Dialogue: 0,0:29:18.23,0:29:20.70,Default,,0000,0000,0000,,are optimizing for compliance,
Dialogue: 0,0:29:20.70,0:29:23.67,Default,,0000,0000,0000,,as is the case with predictive policing.
Dialogue: 0,0:29:23.67,0:29:25.46,Default,,0000,0000,0000,,So this is something that we need to
Dialogue: 0,0:29:25.46,0:29:28.65,Default,,0000,0000,0000,,watch carefully.
Dialogue: 0,0:29:28.65,0:29:30.12,Default,,0000,0000,0000,,I think this is a nice quote that
Dialogue: 0,0:29:30.12,0:29:33.34,Default,,0000,0000,0000,,sort of describes the problem.
Dialogue: 0,0:29:33.34,0:29:35.20,Default,,0000,0000,0000,,In some ways me might be narrowing
Dialogue: 0,0:29:35.20,0:29:38.26,Default,,0000,0000,0000,,our horizon, and the danger is that
Dialogue: 0,0:29:38.26,0:29:41.99,Default,,0000,0000,0000,,these tools are separating people.
Dialogue: 0,0:29:41.99,0:29:43.57,Default,,0000,0000,0000,,And this is particularly bad
Dialogue: 0,0:29:43.57,0:29:45.94,Default,,0000,0000,0000,,for political action, because political action
Dialogue: 0,0:29:45.94,0:29:49.88,Default,,0000,0000,0000,,requires people to have shared experience,
Dialogue: 0,0:29:49.88,0:29:53.80,Default,,0000,0000,0000,,and thus are able to collectively act
Dialogue: 0,0:29:53.80,0:29:57.63,Default,,0000,0000,0000,,to exert pressure to fix problems.
Dialogue: 0,0:29:57.63,0:30:00.81,Default,,0000,0000,0000,,So, finally: accountability.
Dialogue: 0,0:30:00.81,0:30:03.40,Default,,0000,0000,0000,,So, we need some oversight mechanisms.
Dialogue: 0,0:30:03.40,0:30:06.52,Default,,0000,0000,0000,,For example, in the case of errors--
Dialogue: 0,0:30:06.52,0:30:08.22,Default,,0000,0000,0000,,so this is particularly important for
Dialogue: 0,0:30:08.22,0:30:10.85,Default,,0000,0000,0000,,civil or bureaucratic systems.
Dialogue: 0,0:30:10.85,0:30:14.33,Default,,0000,0000,0000,,So, when an algorithm produces some decision,
Dialogue: 0,0:30:14.33,0:30:16.55,Default,,0000,0000,0000,,we don't always want humans to just
Dialogue: 0,0:30:16.55,0:30:18.04,Default,,0000,0000,0000,,defer to the machine,
Dialogue: 0,0:30:18.04,0:30:21.86,Default,,0000,0000,0000,,and that might represent one of the problems.
Dialogue: 0,0:30:21.86,0:30:25.42,Default,,0000,0000,0000,,So, there are starting to be some cases
Dialogue: 0,0:30:25.42,0:30:28.04,Default,,0000,0000,0000,,of computer algorithms yielding a decision,
Dialogue: 0,0:30:28.04,0:30:30.41,Default,,0000,0000,0000,,and then humans being unable to correct
Dialogue: 0,0:30:30.41,0:30:31.80,Default,,0000,0000,0000,,an obvious error.
Dialogue: 0,0:30:31.80,0:30:35.19,Default,,0000,0000,0000,,So there's this case in Georgia,\Nin the United States,
Dialogue: 0,0:30:35.19,0:30:37.26,Default,,0000,0000,0000,,where 2 young people went to
Dialogue: 0,0:30:37.26,0:30:38.53,Default,,0000,0000,0000,,the Department of Motor Vehicles,
Dialogue: 0,0:30:38.53,0:30:39.75,Default,,0000,0000,0000,,they're twins, and they went
Dialogue: 0,0:30:39.75,0:30:42.10,Default,,0000,0000,0000,,to get their driver's license.
Dialogue: 0,0:30:42.10,0:30:44.98,Default,,0000,0000,0000,,However, they were both flagged by
Dialogue: 0,0:30:44.98,0:30:47.49,Default,,0000,0000,0000,,a fraud algorithm that uses facial recognition
Dialogue: 0,0:30:47.49,0:30:48.81,Default,,0000,0000,0000,,to look for similar faces,
Dialogue: 0,0:30:48.81,0:30:50.92,Default,,0000,0000,0000,,and I guess the people that designed the system
Dialogue: 0,0:30:50.92,0:30:54.55,Default,,0000,0000,0000,,didn't think of the possibility of twins.
Dialogue: 0,0:30:54.55,0:30:58.49,Default,,0000,0000,0000,,Yeah.\NSo, they just left
Dialogue: 0,0:30:58.49,0:30:59.89,Default,,0000,0000,0000,,without their driver's licenses.
Dialogue: 0,0:30:59.89,0:31:01.89,Default,,0000,0000,0000,,The people in the Department of Motor Vehicles
Dialogue: 0,0:31:01.89,0:31:03.81,Default,,0000,0000,0000,,were unable to correct this.
Dialogue: 0,0:31:03.81,0:31:06.82,Default,,0000,0000,0000,,So, this is one implication--
Dialogue: 0,0:31:06.82,0:31:08.58,Default,,0000,0000,0000,,it's like something out of Kafka.
Dialogue: 0,0:31:08.58,0:31:11.53,Default,,0000,0000,0000,,But there are also cases of errors being made,
Dialogue: 0,0:31:11.53,0:31:13.88,Default,,0000,0000,0000,,and people not noticing until
Dialogue: 0,0:31:13.88,0:31:15.91,Default,,0000,0000,0000,,after actions have been taken,
Dialogue: 0,0:31:15.91,0:31:17.57,Default,,0000,0000,0000,,some of them very serious--
Dialogue: 0,0:31:17.57,0:31:19.13,Default,,0000,0000,0000,,because people simply deferred
Dialogue: 0,0:31:19.13,0:31:20.62,Default,,0000,0000,0000,,to the machine.
Dialogue: 0,0:31:20.62,0:31:23.31,Default,,0000,0000,0000,,So, this is an example from San Francisco.
Dialogue: 0,0:31:23.31,0:31:26.68,Default,,0000,0000,0000,,So, an ALPR-- an Automated License Plate Reader--
Dialogue: 0,0:31:26.68,0:31:29.43,Default,,0000,0000,0000,,is a device that uses image recognition
Dialogue: 0,0:31:29.43,0:31:32.10,Default,,0000,0000,0000,,to detect and read license plates,
Dialogue: 0,0:31:32.10,0:31:34.34,Default,,0000,0000,0000,,and usually to compare license plates
Dialogue: 0,0:31:34.34,0:31:37.16,Default,,0000,0000,0000,,with a known list of plates of interest.
Dialogue: 0,0:31:37.16,0:31:39.80,Default,,0000,0000,0000,,And, so, San Francisco uses these
Dialogue: 0,0:31:39.80,0:31:42.18,Default,,0000,0000,0000,,and they're mounted on police cars.
Dialogue: 0,0:31:42.18,0:31:46.66,Default,,0000,0000,0000,,So, in this case, San Francisco ALPR
Dialogue: 0,0:31:46.66,0:31:48.88,Default,,0000,0000,0000,,got a hit on a car,
Dialogue: 0,0:31:48.88,0:31:53.03,Default,,0000,0000,0000,,and it was the car of a 47-year-old woman,
Dialogue: 0,0:31:53.03,0:31:54.84,Default,,0000,0000,0000,,with no criminal history.
Dialogue: 0,0:31:54.84,0:31:56.03,Default,,0000,0000,0000,,And so it was a false hit
Dialogue: 0,0:31:56.03,0:31:58.10,Default,,0000,0000,0000,,because it was a blurry image,
Dialogue: 0,0:31:58.10,0:31:59.71,Default,,0000,0000,0000,,and it matched erroneously with
Dialogue: 0,0:31:59.71,0:32:00.91,Default,,0000,0000,0000,,one of the plates of interest
Dialogue: 0,0:32:00.91,0:32:03.48,Default,,0000,0000,0000,,that happened to be a stolen vehicle.
Dialogue: 0,0:32:03.48,0:32:06.87,Default,,0000,0000,0000,,So, they conducted a traffic stop on her,
Dialogue: 0,0:32:06.87,0:32:09.33,Default,,0000,0000,0000,,and they take her out of the vehicle,
Dialogue: 0,0:32:09.33,0:32:11.05,Default,,0000,0000,0000,,they search her and the vehicle,
Dialogue: 0,0:32:11.05,0:32:12.66,Default,,0000,0000,0000,,she gets a pat-down,
Dialogue: 0,0:32:12.66,0:32:14.85,Default,,0000,0000,0000,,and they have her kneel
Dialogue: 0,0:32:14.85,0:32:17.78,Default,,0000,0000,0000,,at gunpoint, in the street.
Dialogue: 0,0:32:17.78,0:32:20.99,Default,,0000,0000,0000,,So, how much oversight should be present
Dialogue: 0,0:32:20.99,0:32:23.100,Default,,0000,0000,0000,,depends on the implications of the system.
Dialogue: 0,0:32:23.100,0:32:25.28,Default,,0000,0000,0000,,It's certainly the case that
Dialogue: 0,0:32:25.28,0:32:26.91,Default,,0000,0000,0000,,for some of these decision-making systems,
Dialogue: 0,0:32:26.91,0:32:29.22,Default,,0000,0000,0000,,an error might not be that important,
Dialogue: 0,0:32:29.22,0:32:31.15,Default,,0000,0000,0000,,it could be relatively harmless,
Dialogue: 0,0:32:31.15,0:32:33.56,Default,,0000,0000,0000,,but in this case,\Nan error in this algorithmic decision
Dialogue: 0,0:32:33.56,0:32:36.26,Default,,0000,0000,0000,,led to this totally innocent person
Dialogue: 0,0:32:36.26,0:32:40.02,Default,,0000,0000,0000,,literally having a gun pointed at her.
Dialogue: 0,0:32:40.02,0:32:44.02,Default,,0000,0000,0000,,So, that brings us to: we need some way of
Dialogue: 0,0:32:44.02,0:32:45.42,Default,,0000,0000,0000,,getting some information about
Dialogue: 0,0:32:45.42,0:32:47.25,Default,,0000,0000,0000,,what is going on here.
Dialogue: 0,0:32:47.25,0:32:50.18,Default,,0000,0000,0000,,We don't wanna have to wait for these events
Dialogue: 0,0:32:50.18,0:32:52.58,Default,,0000,0000,0000,,before we are able to determine
Dialogue: 0,0:32:52.58,0:32:54.41,Default,,0000,0000,0000,,some information about the system.
Dialogue: 0,0:32:54.41,0:32:56.14,Default,,0000,0000,0000,,So, auditing is one option:
Dialogue: 0,0:32:56.14,0:32:58.11,Default,,0000,0000,0000,,to independently verify the statements
Dialogue: 0,0:32:58.11,0:33:00.81,Default,,0000,0000,0000,,of companies, in situations where we have
Dialogue: 0,0:33:00.81,0:33:02.94,Default,,0000,0000,0000,,inputs and outputs.
Dialogue: 0,0:33:02.94,0:33:05.20,Default,,0000,0000,0000,,So, for example, this could be done with
Dialogue: 0,0:33:05.20,0:33:07.49,Default,,0000,0000,0000,,Google, Facebook.
Dialogue: 0,0:33:07.49,0:33:09.19,Default,,0000,0000,0000,,If you have the inputs of a system,
Dialogue: 0,0:33:09.19,0:33:10.65,Default,,0000,0000,0000,,say you have test accounts,
Dialogue: 0,0:33:10.65,0:33:11.73,Default,,0000,0000,0000,,or real accounts,
Dialogue: 0,0:33:11.73,0:33:14.36,Default,,0000,0000,0000,,maybe you can collect\Npeople's information together.
Dialogue: 0,0:33:14.36,0:33:15.83,Default,,0000,0000,0000,,So that was something that was done
Dialogue: 0,0:33:15.83,0:33:18.76,Default,,0000,0000,0000,,during the 2012 Obama campaign
Dialogue: 0,0:33:18.76,0:33:20.25,Default,,0000,0000,0000,,by ProPublica.
Dialogue: 0,0:33:20.25,0:33:21.27,Default,,0000,0000,0000,,People noticed that they were getting
Dialogue: 0,0:33:21.27,0:33:24.74,Default,,0000,0000,0000,,different emails from the Obama campaign,
Dialogue: 0,0:33:24.74,0:33:26.01,Default,,0000,0000,0000,,and were interested to see
Dialogue: 0,0:33:26.01,0:33:28.21,Default,,0000,0000,0000,,based on what factors
Dialogue: 0,0:33:28.21,0:33:29.75,Default,,0000,0000,0000,,the emails were changing.
Dialogue: 0,0:33:29.75,0:33:32.66,Default,,0000,0000,0000,,So, I think about 200 people submitted emails
Dialogue: 0,0:33:32.66,0:33:34.94,Default,,0000,0000,0000,,and they were able to determine some information
Dialogue: 0,0:33:34.94,0:33:38.81,Default,,0000,0000,0000,,about what the emails\Nwere being varied based on.
Dialogue: 0,0:33:38.81,0:33:40.86,Default,,0000,0000,0000,,So there have been some successful
Dialogue: 0,0:33:40.86,0:33:43.08,Default,,0000,0000,0000,,attempts at this.
Dialogue: 0,0:33:43.08,0:33:45.92,Default,,0000,0000,0000,,So, compare inputs and then look at
Dialogue: 0,0:33:45.92,0:33:48.71,Default,,0000,0000,0000,,why one item was shown to one user
Dialogue: 0,0:33:48.71,0:33:50.29,Default,,0000,0000,0000,,and not another, and see if there's
Dialogue: 0,0:33:50.29,0:33:51.88,Default,,0000,0000,0000,,any statistical differences.
Dialogue: 0,0:33:51.88,0:33:56.28,Default,,0000,0000,0000,,So, there's some potential legal issues
Dialogue: 0,0:33:56.28,0:33:57.75,Default,,0000,0000,0000,,with the test accounts, so that's something
Dialogue: 0,0:33:57.75,0:34:01.50,Default,,0000,0000,0000,,to think about-- I'm not a lawyer.
Dialogue: 0,0:34:01.50,0:34:03.92,Default,,0000,0000,0000,,So, for example, if you wanna examine
Dialogue: 0,0:34:03.92,0:34:06.27,Default,,0000,0000,0000,,ad-targeting algorithms,
Dialogue: 0,0:34:06.27,0:34:07.97,Default,,0000,0000,0000,,one way to proceed is to construct
Dialogue: 0,0:34:07.97,0:34:10.59,Default,,0000,0000,0000,,a browsing profile, and then examine
Dialogue: 0,0:34:10.59,0:34:12.99,Default,,0000,0000,0000,,what ads are served back to you.
Dialogue: 0,0:34:12.99,0:34:14.12,Default,,0000,0000,0000,,And so this is something that
Dialogue: 0,0:34:14.12,0:34:16.25,Default,,0000,0000,0000,,academic researchers have looked at,
Dialogue: 0,0:34:16.25,0:34:17.49,Default,,0000,0000,0000,,because, at the time at least,
Dialogue: 0,0:34:17.49,0:34:20.88,Default,,0000,0000,0000,,you didn't need to make an account to do this.
Dialogue: 0,0:34:20.88,0:34:24.77,Default,,0000,0000,0000,,So, this was a study that was presented at
Dialogue: 0,0:34:24.77,0:34:27.80,Default,,0000,0000,0000,,Privacy Enhancing Technologies last year,
Dialogue: 0,0:34:27.80,0:34:31.15,Default,,0000,0000,0000,,and in this study, the researchers
Dialogue: 0,0:34:31.15,0:34:33.18,Default,,0000,0000,0000,,generate some browsing profiles
Dialogue: 0,0:34:33.18,0:34:35.91,Default,,0000,0000,0000,,that differ only by one characteristic,
Dialogue: 0,0:34:35.91,0:34:37.69,Default,,0000,0000,0000,,so they're basically identical in every way
Dialogue: 0,0:34:37.69,0:34:39.05,Default,,0000,0000,0000,,except for one thing.
Dialogue: 0,0:34:39.05,0:34:42.36,Default,,0000,0000,0000,,And that is denoted by Treatment 1 and 2.
Dialogue: 0,0:34:42.36,0:34:44.46,Default,,0000,0000,0000,,So this is a randomized, controlled trial,
Dialogue: 0,0:34:44.46,0:34:46.39,Default,,0000,0000,0000,,but I left out the randomization part
Dialogue: 0,0:34:46.39,0:34:48.22,Default,,0000,0000,0000,,for simplicity.
Dialogue: 0,0:34:48.22,0:34:54.80,Default,,0000,0000,0000,,So, in one study,\Nthey applied a treatment of gender.
Dialogue: 0,0:34:54.80,0:34:56.80,Default,,0000,0000,0000,,So, they had the browsing profiles
Dialogue: 0,0:34:56.80,0:34:59.32,Default,,0000,0000,0000,,in Treatment 1 be male browsing profiles,
Dialogue: 0,0:34:59.32,0:35:02.03,Default,,0000,0000,0000,,and the browsing profiles in Treatment 2\Nbe female.
Dialogue: 0,0:35:02.03,0:35:04.43,Default,,0000,0000,0000,,And they wanted to see: is there any difference
Dialogue: 0,0:35:04.43,0:35:06.08,Default,,0000,0000,0000,,in the way that ads are targeted
Dialogue: 0,0:35:06.08,0:35:08.71,Default,,0000,0000,0000,,if browsing profiles are effectively identical
Dialogue: 0,0:35:08.71,0:35:11.02,Default,,0000,0000,0000,,except for gender?
Dialogue: 0,0:35:11.02,0:35:14.71,Default,,0000,0000,0000,,So, it turns out that there {\i1}was{\i0}.
Dialogue: 0,0:35:14.71,0:35:19.18,Default,,0000,0000,0000,,So, a 3rd-party site was showing Google ads
Dialogue: 0,0:35:19.18,0:35:21.29,Default,,0000,0000,0000,,for senior executive positions
Dialogue: 0,0:35:21.29,0:35:23.98,Default,,0000,0000,0000,,at a rate 6 times higher to the fake men
Dialogue: 0,0:35:23.98,0:35:27.06,Default,,0000,0000,0000,,than for the fake women in this study.
Dialogue: 0,0:35:27.06,0:35:30.11,Default,,0000,0000,0000,,So, this sort of auditing is not going to
Dialogue: 0,0:35:30.11,0:35:32.78,Default,,0000,0000,0000,,be able to determine everything
Dialogue: 0,0:35:32.78,0:35:34.93,Default,,0000,0000,0000,,that algorithms are doing, but they can
Dialogue: 0,0:35:34.93,0:35:36.52,Default,,0000,0000,0000,,sometimes uncover interesting,
Dialogue: 0,0:35:36.52,0:35:40.90,Default,,0000,0000,0000,,at least statistical differences.
Dialogue: 0,0:35:40.90,0:35:47.10,Default,,0000,0000,0000,,So, this leads us to the fundamental issue:
Dialogue: 0,0:35:47.10,0:35:49.18,Default,,0000,0000,0000,,Right now, we're really not in control
Dialogue: 0,0:35:49.18,0:35:50.51,Default,,0000,0000,0000,,of some of these systems,
Dialogue: 0,0:35:50.51,0:35:54.48,Default,,0000,0000,0000,,and we really need these predictive systems
Dialogue: 0,0:35:54.48,0:35:56.12,Default,,0000,0000,0000,,to be controlled by us,
Dialogue: 0,0:35:56.12,0:35:57.82,Default,,0000,0000,0000,,in order for them not to be used
Dialogue: 0,0:35:57.82,0:36:00.11,Default,,0000,0000,0000,,as a system of control.
Dialogue: 0,0:36:00.11,0:36:03.22,Default,,0000,0000,0000,,So there are some technologies that I'd like
Dialogue: 0,0:36:03.22,0:36:06.89,Default,,0000,0000,0000,,to point you all to.
Dialogue: 0,0:36:06.89,0:36:08.32,Default,,0000,0000,0000,,We need tools in the digital commons
Dialogue: 0,0:36:08.32,0:36:11.16,Default,,0000,0000,0000,,that can help address some of these concerns.
Dialogue: 0,0:36:11.16,0:36:13.35,Default,,0000,0000,0000,,So, the first thing is that of course
Dialogue: 0,0:36:13.35,0:36:14.73,Default,,0000,0000,0000,,we known that minimizing the amount of
Dialogue: 0,0:36:14.73,0:36:17.07,Default,,0000,0000,0000,,data available can help in some contexts,
Dialogue: 0,0:36:17.07,0:36:18.98,Default,,0000,0000,0000,,which we can do by making systems
Dialogue: 0,0:36:18.98,0:36:22.78,Default,,0000,0000,0000,,that are private by design, and by default.
Dialogue: 0,0:36:22.78,0:36:24.55,Default,,0000,0000,0000,,Another thing is that these audit tools
Dialogue: 0,0:36:24.55,0:36:25.89,Default,,0000,0000,0000,,might be useful.
Dialogue: 0,0:36:25.89,0:36:30.72,Default,,0000,0000,0000,,And, so, these 2 nice examples in academia...
Dialogue: 0,0:36:30.72,0:36:34.36,Default,,0000,0000,0000,,the ad experiment that I just showed was done
Dialogue: 0,0:36:34.36,0:36:36.12,Default,,0000,0000,0000,,using AdFisher.
Dialogue: 0,0:36:36.12,0:36:38.20,Default,,0000,0000,0000,,So, these are 2 toolkits that you can use
Dialogue: 0,0:36:38.20,0:36:41.44,Default,,0000,0000,0000,,to start doing this sort of auditing.
Dialogue: 0,0:36:41.44,0:36:44.58,Default,,0000,0000,0000,,Another technology that is generally useful,
Dialogue: 0,0:36:44.58,0:36:46.70,Default,,0000,0000,0000,,but particularly in the case of prediction
Dialogue: 0,0:36:46.70,0:36:48.79,Default,,0000,0000,0000,,it's useful to maintain access to
Dialogue: 0,0:36:48.79,0:36:50.29,Default,,0000,0000,0000,,as many sites as possible,
Dialogue: 0,0:36:50.29,0:36:52.59,Default,,0000,0000,0000,,through anonymity systems like Tor,
Dialogue: 0,0:36:52.59,0:36:54.32,Default,,0000,0000,0000,,because it's impossible to personalize
Dialogue: 0,0:36:54.32,0:36:55.65,Default,,0000,0000,0000,,when everyone looks the same.
Dialogue: 0,0:36:55.65,0:36:59.13,Default,,0000,0000,0000,,So this is a very important technology.
Dialogue: 0,0:36:59.13,0:37:01.52,Default,,0000,0000,0000,,Something that doesn't really exist,
Dialogue: 0,0:37:01.52,0:37:03.63,Default,,0000,0000,0000,,but that I think is pretty important,
Dialogue: 0,0:37:03.63,0:37:05.83,Default,,0000,0000,0000,,is having some tool to view the landscape.
Dialogue: 0,0:37:05.83,0:37:08.16,Default,,0000,0000,0000,,So, as we know from these few studies
Dialogue: 0,0:37:08.16,0:37:10.44,Default,,0000,0000,0000,,that have been done,
Dialogue: 0,0:37:10.44,0:37:12.06,Default,,0000,0000,0000,,different people are not seeing the internet
Dialogue: 0,0:37:12.06,0:37:12.95,Default,,0000,0000,0000,,in the same way.
Dialogue: 0,0:37:12.95,0:37:15.73,Default,,0000,0000,0000,,This is one reason why we don't like censorship.
Dialogue: 0,0:37:15.73,0:37:17.88,Default,,0000,0000,0000,,But, rich and poor people,
Dialogue: 0,0:37:17.88,0:37:19.66,Default,,0000,0000,0000,,from academic research we know that
Dialogue: 0,0:37:19.66,0:37:23.79,Default,,0000,0000,0000,,there is widespread price discrimination\Non the internet,
Dialogue: 0,0:37:23.79,0:37:25.65,Default,,0000,0000,0000,,so rich and poor people see a different view
Dialogue: 0,0:37:25.65,0:37:26.97,Default,,0000,0000,0000,,of the Internet,
Dialogue: 0,0:37:26.97,0:37:28.40,Default,,0000,0000,0000,,men and women see a different view
Dialogue: 0,0:37:28.40,0:37:29.94,Default,,0000,0000,0000,,of the Internet.
Dialogue: 0,0:37:29.94,0:37:31.20,Default,,0000,0000,0000,,We wanna know how different people
Dialogue: 0,0:37:31.20,0:37:32.45,Default,,0000,0000,0000,,see the same site,
Dialogue: 0,0:37:32.45,0:37:34.33,Default,,0000,0000,0000,,and this could be the beginning of
Dialogue: 0,0:37:34.33,0:37:36.33,Default,,0000,0000,0000,,a defense system for this sort of
Dialogue: 0,0:37:36.33,0:37:41.73,Default,,0000,0000,0000,,manipulation/tampering that I showed earlier.
Dialogue: 0,0:37:41.73,0:37:45.55,Default,,0000,0000,0000,,Another interesting approach is obfuscation:
Dialogue: 0,0:37:45.55,0:37:46.98,Default,,0000,0000,0000,,injecting noise into the system.
Dialogue: 0,0:37:46.98,0:37:49.19,Default,,0000,0000,0000,,So there's an interesting browser extension
Dialogue: 0,0:37:49.19,0:37:51.72,Default,,0000,0000,0000,,called Adnauseum, that's for Firefox,
Dialogue: 0,0:37:51.72,0:37:54.58,Default,,0000,0000,0000,,which clicks on every single ad you're served,
Dialogue: 0,0:37:54.58,0:37:55.68,Default,,0000,0000,0000,,to inject noise.
Dialogue: 0,0:37:55.68,0:37:57.02,Default,,0000,0000,0000,,So that's, I think, an interesting approach
Dialogue: 0,0:37:57.02,0:38:00.17,Default,,0000,0000,0000,,that people haven't looked at too much.
Dialogue: 0,0:38:00.17,0:38:03.78,Default,,0000,0000,0000,,So in terms of policy,
Dialogue: 0,0:38:03.78,0:38:06.53,Default,,0000,0000,0000,,Facebook and Google, these internet giants,
Dialogue: 0,0:38:06.53,0:38:08.83,Default,,0000,0000,0000,,have billions of users,
Dialogue: 0,0:38:08.83,0:38:12.22,Default,,0000,0000,0000,,and sometimes they like to call themselves
Dialogue: 0,0:38:12.22,0:38:13.77,Default,,0000,0000,0000,,new public utilities,
Dialogue: 0,0:38:13.77,0:38:15.00,Default,,0000,0000,0000,,and if that's the case then
Dialogue: 0,0:38:15.00,0:38:17.55,Default,,0000,0000,0000,,it might be necessary to subject them
Dialogue: 0,0:38:17.55,0:38:20.54,Default,,0000,0000,0000,,to additional regulation.
Dialogue: 0,0:38:20.54,0:38:21.99,Default,,0000,0000,0000,,Another problem that's come up,
Dialogue: 0,0:38:21.99,0:38:23.54,Default,,0000,0000,0000,,for example with some of the studies
Dialogue: 0,0:38:23.54,0:38:24.90,Default,,0000,0000,0000,,that Facebook has done,
Dialogue: 0,0:38:24.90,0:38:29.04,Default,,0000,0000,0000,,is sometimes a lack of ethics review.
Dialogue: 0,0:38:29.04,0:38:31.06,Default,,0000,0000,0000,,So, for example, in academia,
Dialogue: 0,0:38:31.06,0:38:33.86,Default,,0000,0000,0000,,if you're gonna do research involving humans,
Dialogue: 0,0:38:33.86,0:38:35.39,Default,,0000,0000,0000,,there's an Institutional Review Board
Dialogue: 0,0:38:35.39,0:38:36.97,Default,,0000,0000,0000,,that you go to that verifies that
Dialogue: 0,0:38:36.97,0:38:39.14,Default,,0000,0000,0000,,you're doing things in an ethical manner.
Dialogue: 0,0:38:39.14,0:38:40.91,Default,,0000,0000,0000,,And some companies do have internal
Dialogue: 0,0:38:40.91,0:38:43.03,Default,,0000,0000,0000,,review processes like this, but it might
Dialogue: 0,0:38:43.03,0:38:45.12,Default,,0000,0000,0000,,be important to have an independent
Dialogue: 0,0:38:45.12,0:38:48.20,Default,,0000,0000,0000,,ethics board that does this sort of thing.
Dialogue: 0,0:38:48.20,0:38:50.85,Default,,0000,0000,0000,,And we {\i1}really{\i0} need 3rd-party auditing.
Dialogue: 0,0:38:50.85,0:38:54.52,Default,,0000,0000,0000,,So, for example, some companies
Dialogue: 0,0:38:54.52,0:38:56.22,Default,,0000,0000,0000,,don't want auditing to be done
Dialogue: 0,0:38:56.22,0:38:59.19,Default,,0000,0000,0000,,because of IP concerns,
Dialogue: 0,0:38:59.19,0:39:00.58,Default,,0000,0000,0000,,and if that's the concern
Dialogue: 0,0:39:00.58,0:39:03.18,Default,,0000,0000,0000,,maybe having a set of people
Dialogue: 0,0:39:03.18,0:39:05.68,Default,,0000,0000,0000,,that are not paid by the company
Dialogue: 0,0:39:05.68,0:39:07.20,Default,,0000,0000,0000,,to check how some of these systems
Dialogue: 0,0:39:07.20,0:39:08.64,Default,,0000,0000,0000,,are being implemented,
Dialogue: 0,0:39:08.64,0:39:11.24,Default,,0000,0000,0000,,could help give us confidence that
Dialogue: 0,0:39:11.24,0:39:16.98,Default,,0000,0000,0000,,things are being done in a reasonable way.
Dialogue: 0,0:39:16.98,0:39:20.27,Default,,0000,0000,0000,,So, in closing,
Dialogue: 0,0:39:20.27,0:39:23.18,Default,,0000,0000,0000,,algorithmic decision making is here,
Dialogue: 0,0:39:23.18,0:39:26.14,Default,,0000,0000,0000,,and it's barreling forward\Nat a very fast rate,
Dialogue: 0,0:39:26.14,0:39:27.89,Default,,0000,0000,0000,,and we need to figure out what
Dialogue: 0,0:39:27.89,0:39:30.41,Default,,0000,0000,0000,,the guide rails should be,
Dialogue: 0,0:39:30.41,0:39:31.38,Default,,0000,0000,0000,,and how to install them
Dialogue: 0,0:39:31.38,0:39:33.12,Default,,0000,0000,0000,,to handle some of the potential threats.
Dialogue: 0,0:39:33.12,0:39:35.47,Default,,0000,0000,0000,,There's a huge amount of power here.
Dialogue: 0,0:39:35.47,0:39:37.91,Default,,0000,0000,0000,,We need more openness in these systems.
Dialogue: 0,0:39:37.91,0:39:39.59,Default,,0000,0000,0000,,And, right now,
Dialogue: 0,0:39:39.59,0:39:41.56,Default,,0000,0000,0000,,with the intelligent systems that do exist,
Dialogue: 0,0:39:41.56,0:39:43.92,Default,,0000,0000,0000,,we don't know what's occurring really,
Dialogue: 0,0:39:43.92,0:39:46.51,Default,,0000,0000,0000,,and we need to watch carefully
Dialogue: 0,0:39:46.51,0:39:49.10,Default,,0000,0000,0000,,where and how these systems are being used.
Dialogue: 0,0:39:49.10,0:39:50.69,Default,,0000,0000,0000,,And I think this community has
Dialogue: 0,0:39:50.69,0:39:53.94,Default,,0000,0000,0000,,an important role to play in this fight,
Dialogue: 0,0:39:53.94,0:39:55.73,Default,,0000,0000,0000,,to study what's being done,
Dialogue: 0,0:39:55.73,0:39:57.16,Default,,0000,0000,0000,,to show people what's being done,
Dialogue: 0,0:39:57.16,0:39:58.67,Default,,0000,0000,0000,,to raise the debate and advocate,
Dialogue: 0,0:39:58.67,0:40:01.20,Default,,0000,0000,0000,,and, where necessary, to resist.
Dialogue: 0,0:40:01.20,0:40:03.34,Default,,0000,0000,0000,,Thanks.
Dialogue: 0,0:40:03.34,0:40:13.13,Default,,0000,0000,0000,,{\i1}applause{\i0}
Dialogue: 0,0:40:13.13,0:40:17.52,Default,,0000,0000,0000,,Herald: So, let's have a question and answer.
Dialogue: 0,0:40:17.52,0:40:19.08,Default,,0000,0000,0000,,Microphone 2, please.
Dialogue: 0,0:40:19.08,0:40:20.20,Default,,0000,0000,0000,,Mic 2: Hi there.
Dialogue: 0,0:40:20.20,0:40:23.26,Default,,0000,0000,0000,,Thanks for the talk.
Dialogue: 0,0:40:23.26,0:40:26.23,Default,,0000,0000,0000,,Since these pre-crime softwares also
Dialogue: 0,0:40:26.23,0:40:27.36,Default,,0000,0000,0000,,arrived here in Germany
Dialogue: 0,0:40:27.36,0:40:29.68,Default,,0000,0000,0000,,with the start of the so-called CopWatch system
Dialogue: 0,0:40:29.68,0:40:32.78,Default,,0000,0000,0000,,in southern Germany,\Nand Bavaria and Nuremberg especially,
Dialogue: 0,0:40:32.78,0:40:35.42,Default,,0000,0000,0000,,where they try to predict burglary crime
Dialogue: 0,0:40:35.42,0:40:37.46,Default,,0000,0000,0000,,using that criminal record
Dialogue: 0,0:40:37.46,0:40:40.17,Default,,0000,0000,0000,,geographical analysis, like you explained,
Dialogue: 0,0:40:40.17,0:40:43.38,Default,,0000,0000,0000,,leads me to a 2-fold question:
Dialogue: 0,0:40:43.38,0:40:47.90,Default,,0000,0000,0000,,first, have you heard of any research
Dialogue: 0,0:40:47.90,0:40:49.76,Default,,0000,0000,0000,,that measures the effectiveness
Dialogue: 0,0:40:49.76,0:40:53.69,Default,,0000,0000,0000,,of such measures, at all?
Dialogue: 0,0:40:53.69,0:40:57.04,Default,,0000,0000,0000,,And, second:
Dialogue: 0,0:40:57.04,0:41:00.60,Default,,0000,0000,0000,,What do you think of the game theory
Dialogue: 0,0:41:00.60,0:41:02.69,Default,,0000,0000,0000,,if the thieves or the bad guys
Dialogue: 0,0:41:02.69,0:41:07.62,Default,,0000,0000,0000,,know the system, and when they\Ngame the system,
Dialogue: 0,0:41:07.62,0:41:09.98,Default,,0000,0000,0000,,they will probably win,
Dialogue: 0,0:41:09.98,0:41:11.64,Default,,0000,0000,0000,,since one police officer in an interview said
Dialogue: 0,0:41:11.64,0:41:14.02,Default,,0000,0000,0000,,this system is used to reduce
Dialogue: 0,0:41:14.02,0:41:16.46,Default,,0000,0000,0000,,the personal costs of policing,
Dialogue: 0,0:41:16.46,0:41:19.46,Default,,0000,0000,0000,,so they just send the guys\Nwhere the red flags are,
Dialogue: 0,0:41:19.46,0:41:22.29,Default,,0000,0000,0000,,and the others take the day off.
Dialogue: 0,0:41:22.29,0:41:24.36,Default,,0000,0000,0000,,Dr. Helsby: Yup.
Dialogue: 0,0:41:24.36,0:41:27.15,Default,,0000,0000,0000,,Um, so, with respect to
Dialogue: 0,0:41:27.15,0:41:30.99,Default,,0000,0000,0000,,testing the effectiveness of predictive policing,
Dialogue: 0,0:41:30.99,0:41:31.99,Default,,0000,0000,0000,,the companies,
Dialogue: 0,0:41:31.99,0:41:33.91,Default,,0000,0000,0000,,some of them do randomized, controlled trials
Dialogue: 0,0:41:33.91,0:41:35.24,Default,,0000,0000,0000,,and claim a reduction in policing.
Dialogue: 0,0:41:35.24,0:41:38.35,Default,,0000,0000,0000,,The best independent study that I've seen
Dialogue: 0,0:41:38.35,0:41:40.68,Default,,0000,0000,0000,,is by this RAND Corporation
Dialogue: 0,0:41:40.68,0:41:43.12,Default,,0000,0000,0000,,that did a study in, I think,
Dialogue: 0,0:41:43.12,0:41:44.92,Default,,0000,0000,0000,,Shreveport, Louisiana,
Dialogue: 0,0:41:44.92,0:41:47.59,Default,,0000,0000,0000,,and in their report they claim
Dialogue: 0,0:41:47.59,0:41:50.19,Default,,0000,0000,0000,,that there was no statistically significant
Dialogue: 0,0:41:50.19,0:41:52.90,Default,,0000,0000,0000,,difference, they didn't find any reduction.
Dialogue: 0,0:41:52.90,0:41:54.10,Default,,0000,0000,0000,,And it {\i1}was{\i0} specifically looking at
Dialogue: 0,0:41:54.10,0:41:56.73,Default,,0000,0000,0000,,property crime, which I think you mentioned.
Dialogue: 0,0:41:56.73,0:41:59.48,Default,,0000,0000,0000,,So, I think right now there's sort of
Dialogue: 0,0:41:59.48,0:42:01.07,Default,,0000,0000,0000,,conflicting reports between
Dialogue: 0,0:42:01.07,0:42:06.18,Default,,0000,0000,0000,,the independent auditors\Nand these company claims.
Dialogue: 0,0:42:06.18,0:42:09.29,Default,,0000,0000,0000,,So there definitely needs to be more study.
Dialogue: 0,0:42:09.29,0:42:12.24,Default,,0000,0000,0000,,And then, the 2nd thing...sorry,\Nremind me what it was?
Dialogue: 0,0:42:12.24,0:42:15.19,Default,,0000,0000,0000,,Mic 2: What about the guys gaming the system?
Dialogue: 0,0:42:15.19,0:42:16.95,Default,,0000,0000,0000,,Dr. Helsby: Oh, yeah.
Dialogue: 0,0:42:16.95,0:42:18.90,Default,,0000,0000,0000,,I think it's a legitimate concern.
Dialogue: 0,0:42:18.90,0:42:22.48,Default,,0000,0000,0000,,Like, if all the outputs\Nwere just immediately public,
Dialogue: 0,0:42:22.48,0:42:24.60,Default,,0000,0000,0000,,then, yes, everyone knows the location
Dialogue: 0,0:42:24.60,0:42:26.55,Default,,0000,0000,0000,,of all police officers,
Dialogue: 0,0:42:26.55,0:42:29.01,Default,,0000,0000,0000,,and I imagine that people would have
Dialogue: 0,0:42:29.01,0:42:30.78,Default,,0000,0000,0000,,a problem with that.
Dialogue: 0,0:42:30.78,0:42:32.68,Default,,0000,0000,0000,,Yup.
Dialogue: 0,0:42:32.68,0:42:35.99,Default,,0000,0000,0000,,Heraldl: Microphone #4, please.
Dialogue: 0,0:42:35.99,0:42:39.37,Default,,0000,0000,0000,,Mic 4: Yeah, this is not actually a question,
Dialogue: 0,0:42:39.37,0:42:40.78,Default,,0000,0000,0000,,but just a comment.
Dialogue: 0,0:42:40.78,0:42:42.97,Default,,0000,0000,0000,,I've enjoyed your talk very much,
Dialogue: 0,0:42:42.97,0:42:47.79,Default,,0000,0000,0000,,in particular after watching
Dialogue: 0,0:42:47.79,0:42:52.27,Default,,0000,0000,0000,,the talk in Hall 1 earlier in the afternoon.
Dialogue: 0,0:42:52.27,0:42:55.73,Default,,0000,0000,0000,,The "Say Hi to Your New Boss", about
Dialogue: 0,0:42:55.73,0:42:59.61,Default,,0000,0000,0000,,algorithms that are trained with big data,
Dialogue: 0,0:42:59.61,0:43:02.39,Default,,0000,0000,0000,,and finally make decisions.
Dialogue: 0,0:43:02.39,0:43:08.21,Default,,0000,0000,0000,,And I think these 2 talks are kind of complementary,
Dialogue: 0,0:43:08.21,0:43:11.31,Default,,0000,0000,0000,,and if people are interested in the topic
Dialogue: 0,0:43:11.31,0:43:14.71,Default,,0000,0000,0000,,they might want to check out the other talk
Dialogue: 0,0:43:14.71,0:43:16.26,Default,,0000,0000,0000,,and watch it later, because these
Dialogue: 0,0:43:16.26,0:43:17.32,Default,,0000,0000,0000,,fit very well together.
Dialogue: 0,0:43:17.32,0:43:19.59,Default,,0000,0000,0000,,Dr. Helsby: Yeah, it was a great talk.
Dialogue: 0,0:43:19.59,0:43:22.13,Default,,0000,0000,0000,,Herald: Microphone #2, please.
Dialogue: 0,0:43:22.13,0:43:25.05,Default,,0000,0000,0000,,Mic 2: Um, yeah, you mentioned
Dialogue: 0,0:43:25.05,0:43:27.32,Default,,0000,0000,0000,,the need to have some kind of 3rd-party auditing
Dialogue: 0,0:43:27.32,0:43:30.90,Default,,0000,0000,0000,,or some kind of way to
Dialogue: 0,0:43:30.90,0:43:31.93,Default,,0000,0000,0000,,peek into these algorithms
Dialogue: 0,0:43:31.93,0:43:33.08,Default,,0000,0000,0000,,and to see what they're doing,
Dialogue: 0,0:43:33.08,0:43:34.42,Default,,0000,0000,0000,,and to see if they're being fair.
Dialogue: 0,0:43:34.42,0:43:36.20,Default,,0000,0000,0000,,Can you talk a little bit more about that?
Dialogue: 0,0:43:36.20,0:43:38.06,Default,,0000,0000,0000,,Like, going forward,
Dialogue: 0,0:43:38.06,0:43:40.69,Default,,0000,0000,0000,,some kind of regulatory structures
Dialogue: 0,0:43:40.69,0:43:44.20,Default,,0000,0000,0000,,would probably have to emerge
Dialogue: 0,0:43:44.20,0:43:47.20,Default,,0000,0000,0000,,to analyze and to look at
Dialogue: 0,0:43:47.20,0:43:49.34,Default,,0000,0000,0000,,these black boxes that are just sort of
Dialogue: 0,0:43:49.34,0:43:51.31,Default,,0000,0000,0000,,popping up everywhere and, you know,
Dialogue: 0,0:43:51.31,0:43:52.94,Default,,0000,0000,0000,,controlling more and more of the things
Dialogue: 0,0:43:52.94,0:43:56.15,Default,,0000,0000,0000,,in our lives, and important decisions.
Dialogue: 0,0:43:56.15,0:43:58.54,Default,,0000,0000,0000,,So, just, what kind of discussions
Dialogue: 0,0:43:58.54,0:43:59.46,Default,,0000,0000,0000,,are there for that?
Dialogue: 0,0:43:59.46,0:44:01.81,Default,,0000,0000,0000,,And what kind of possibility\Nis there for that?
Dialogue: 0,0:44:01.81,0:44:04.90,Default,,0000,0000,0000,,And, I'm sure that companies would be
Dialogue: 0,0:44:04.90,0:44:08.00,Default,,0000,0000,0000,,very, very resistant to
Dialogue: 0,0:44:08.00,0:44:09.89,Default,,0000,0000,0000,,any kind of attempt to look into
Dialogue: 0,0:44:09.89,0:44:13.89,Default,,0000,0000,0000,,algorithms, and to...
Dialogue: 0,0:44:13.89,0:44:15.07,Default,,0000,0000,0000,,Dr. Helsby: Yeah, I mean, definitely
Dialogue: 0,0:44:15.07,0:44:18.07,Default,,0000,0000,0000,,companies would be very resistant to
Dialogue: 0,0:44:18.07,0:44:19.67,Default,,0000,0000,0000,,having people look into their algorithms.
Dialogue: 0,0:44:19.67,0:44:22.19,Default,,0000,0000,0000,,So, if you wanna do a very rigorous
Dialogue: 0,0:44:22.19,0:44:23.34,Default,,0000,0000,0000,,audit of what's going on
Dialogue: 0,0:44:23.34,0:44:25.66,Default,,0000,0000,0000,,then it's probably necessary to have
Dialogue: 0,0:44:25.66,0:44:26.59,Default,,0000,0000,0000,,a few people come in
Dialogue: 0,0:44:26.59,0:44:28.90,Default,,0000,0000,0000,,and sign NDAs, and then
Dialogue: 0,0:44:28.90,0:44:31.04,Default,,0000,0000,0000,,look through the systems.
Dialogue: 0,0:44:31.04,0:44:33.14,Default,,0000,0000,0000,,So, that's one way to proceed.
Dialogue: 0,0:44:33.14,0:44:35.05,Default,,0000,0000,0000,,But, another way to proceed that--
Dialogue: 0,0:44:35.05,0:44:38.72,Default,,0000,0000,0000,,so, these academic researchers have done
Dialogue: 0,0:44:38.72,0:44:40.01,Default,,0000,0000,0000,,a few experiments
Dialogue: 0,0:44:40.01,0:44:42.81,Default,,0000,0000,0000,,and found some interesting things,
Dialogue: 0,0:44:42.81,0:44:45.50,Default,,0000,0000,0000,,and that's sort all the attempts at auditing
Dialogue: 0,0:44:45.50,0:44:46.45,Default,,0000,0000,0000,,that we've seen:
Dialogue: 0,0:44:46.45,0:44:48.49,Default,,0000,0000,0000,,there was 1 attempt in 2012\Nfor the Obama campaign,
Dialogue: 0,0:44:48.49,0:44:49.91,Default,,0000,0000,0000,,but there's really not been any
Dialogue: 0,0:44:49.91,0:44:51.50,Default,,0000,0000,0000,,sort of systematic attempt--
Dialogue: 0,0:44:51.50,0:44:52.59,Default,,0000,0000,0000,,you know, like, in censorship
Dialogue: 0,0:44:52.59,0:44:54.54,Default,,0000,0000,0000,,we see a systematic attempt to
Dialogue: 0,0:44:54.54,0:44:56.78,Default,,0000,0000,0000,,do measurement as often as possible,
Dialogue: 0,0:44:56.78,0:44:58.24,Default,,0000,0000,0000,,check what's going on,
Dialogue: 0,0:44:58.24,0:44:59.34,Default,,0000,0000,0000,,and that itself, you know,
Dialogue: 0,0:44:59.34,0:45:00.90,Default,,0000,0000,0000,,can act as an oversight mechanism.
Dialogue: 0,0:45:00.90,0:45:01.88,Default,,0000,0000,0000,,But, right now,
Dialogue: 0,0:45:01.88,0:45:03.90,Default,,0000,0000,0000,,I think many of these companies
Dialogue: 0,0:45:03.90,0:45:05.26,Default,,0000,0000,0000,,realize no one is watching,
Dialogue: 0,0:45:05.26,0:45:07.16,Default,,0000,0000,0000,,so there's no real push to have
Dialogue: 0,0:45:07.16,0:45:10.44,Default,,0000,0000,0000,,people verify: are you being fair when you
Dialogue: 0,0:45:10.44,0:45:11.54,Default,,0000,0000,0000,,implement this system?
Dialogue: 0,0:45:11.54,0:45:12.97,Default,,0000,0000,0000,,Because no one's really checking.
Dialogue: 0,0:45:12.97,0:45:13.98,Default,,0000,0000,0000,,Mic 2: Do you think that,
Dialogue: 0,0:45:13.98,0:45:15.34,Default,,0000,0000,0000,,at some point, it would be like
Dialogue: 0,0:45:15.34,0:45:19.06,Default,,0000,0000,0000,,an FDA or SEC, to give some American examples...
Dialogue: 0,0:45:19.06,0:45:21.49,Default,,0000,0000,0000,,an actual government regulatory agency
Dialogue: 0,0:45:21.49,0:45:24.96,Default,,0000,0000,0000,,that has the power and ability to
Dialogue: 0,0:45:24.96,0:45:27.93,Default,,0000,0000,0000,,not just sort of look and try to
Dialogue: 0,0:45:27.93,0:45:31.71,Default,,0000,0000,0000,,reverse engineer some of these algorithms,
Dialogue: 0,0:45:31.71,0:45:33.92,Default,,0000,0000,0000,,but actually peek in there and make sure
Dialogue: 0,0:45:33.92,0:45:36.42,Default,,0000,0000,0000,,that things are fair, because it seems like
Dialogue: 0,0:45:36.42,0:45:38.24,Default,,0000,0000,0000,,there's just-- it's so important now
Dialogue: 0,0:45:38.24,0:45:41.77,Default,,0000,0000,0000,,that, again, it could be the difference between
Dialogue: 0,0:45:41.77,0:45:42.93,Default,,0000,0000,0000,,life and death, between
Dialogue: 0,0:45:42.93,0:45:44.59,Default,,0000,0000,0000,,getting a job, not getting a job,
Dialogue: 0,0:45:44.59,0:45:46.13,Default,,0000,0000,0000,,being pulled over,\Nnot being pulled over,
Dialogue: 0,0:45:46.13,0:45:48.07,Default,,0000,0000,0000,,being racially profiled,\Nnot racially profiled,
Dialogue: 0,0:45:48.07,0:45:49.41,Default,,0000,0000,0000,,things like that.\NDr. Helsby: Right.
Dialogue: 0,0:45:49.41,0:45:50.43,Default,,0000,0000,0000,,Mic 2: Is it moving in that direction?
Dialogue: 0,0:45:50.43,0:45:52.25,Default,,0000,0000,0000,,Or is it way too early for it?
Dialogue: 0,0:45:52.25,0:45:55.11,Default,,0000,0000,0000,,Dr. Helsby: I mean, so some people have...
Dialogue: 0,0:45:55.11,0:45:56.86,Default,,0000,0000,0000,,someone has called for, like,
Dialogue: 0,0:45:56.86,0:45:59.08,Default,,0000,0000,0000,,a Federal Search Commission,
Dialogue: 0,0:45:59.08,0:46:00.93,Default,,0000,0000,0000,,or like a Federal Algorithms Commission,
Dialogue: 0,0:46:00.93,0:46:03.20,Default,,0000,0000,0000,,that would do this sort of oversight work,
Dialogue: 0,0:46:03.20,0:46:06.13,Default,,0000,0000,0000,,but it's in such early stages right now
Dialogue: 0,0:46:06.13,0:46:09.97,Default,,0000,0000,0000,,that there's no real push for that.
Dialogue: 0,0:46:09.97,0:46:13.33,Default,,0000,0000,0000,,But I think it's a good idea.
Dialogue: 0,0:46:13.33,0:46:15.73,Default,,0000,0000,0000,,Herald: And again, #2 please.
Dialogue: 0,0:46:15.73,0:46:17.06,Default,,0000,0000,0000,,Mic 2: Thank you again for your talk.
Dialogue: 0,0:46:17.06,0:46:19.31,Default,,0000,0000,0000,,I was just curious if you can point
Dialogue: 0,0:46:19.31,0:46:20.44,Default,,0000,0000,0000,,to any examples of
Dialogue: 0,0:46:20.44,0:46:22.62,Default,,0000,0000,0000,,either current producers or consumers
Dialogue: 0,0:46:22.62,0:46:24.03,Default,,0000,0000,0000,,of these algorithmic systems
Dialogue: 0,0:46:24.03,0:46:26.39,Default,,0000,0000,0000,,who are actively and publicly trying
Dialogue: 0,0:46:26.39,0:46:27.72,Default,,0000,0000,0000,,to do so in a responsible manner
Dialogue: 0,0:46:27.72,0:46:29.72,Default,,0000,0000,0000,,by describing what they're trying to do
Dialogue: 0,0:46:29.72,0:46:31.38,Default,,0000,0000,0000,,and how they're going about it?
Dialogue: 0,0:46:31.38,0:46:37.21,Default,,0000,0000,0000,,Dr. Helsby: So, yeah, there are some companies,
Dialogue: 0,0:46:37.21,0:46:39.00,Default,,0000,0000,0000,,for example, like DataKind,
Dialogue: 0,0:46:39.00,0:46:42.71,Default,,0000,0000,0000,,that try to deploy algorithmic systems
Dialogue: 0,0:46:42.71,0:46:44.64,Default,,0000,0000,0000,,in as responsible a way as possible,
Dialogue: 0,0:46:44.64,0:46:47.25,Default,,0000,0000,0000,,for like public policy.
Dialogue: 0,0:46:47.25,0:46:49.55,Default,,0000,0000,0000,,Like, I actually also implement systems
Dialogue: 0,0:46:49.55,0:46:51.75,Default,,0000,0000,0000,,for public policy in a transparent way.
Dialogue: 0,0:46:51.75,0:46:54.33,Default,,0000,0000,0000,,Like, all the code is in GitHub, etc.
Dialogue: 0,0:46:54.33,0:47:00.02,Default,,0000,0000,0000,,And it is also the case to give credit to
Dialogue: 0,0:47:00.02,0:47:01.99,Default,,0000,0000,0000,,Google, and these giants,
Dialogue: 0,0:47:01.99,0:47:06.11,Default,,0000,0000,0000,,they're trying to implement transparency systems
Dialogue: 0,0:47:06.11,0:47:08.17,Default,,0000,0000,0000,,that help you understand.
Dialogue: 0,0:47:08.17,0:47:09.29,Default,,0000,0000,0000,,This has been done with respect to
Dialogue: 0,0:47:09.29,0:47:12.33,Default,,0000,0000,0000,,how your data is being collected,
Dialogue: 0,0:47:12.33,0:47:14.58,Default,,0000,0000,0000,,but for example if you go on Amazon.com
Dialogue: 0,0:47:14.58,0:47:17.89,Default,,0000,0000,0000,,you can see a recommendation has been made,
Dialogue: 0,0:47:17.89,0:47:19.42,Default,,0000,0000,0000,,and that is pretty transparent.
Dialogue: 0,0:47:19.42,0:47:21.48,Default,,0000,0000,0000,,You can see "this item\Nwas recommended to me,"
Dialogue: 0,0:47:21.48,0:47:25.04,Default,,0000,0000,0000,,so you know that prediction\Nis being used in this case,
Dialogue: 0,0:47:25.04,0:47:27.09,Default,,0000,0000,0000,,and it will say why prediction is being used:
Dialogue: 0,0:47:27.09,0:47:29.23,Default,,0000,0000,0000,,because you purchased some item.
Dialogue: 0,0:47:29.23,0:47:30.38,Default,,0000,0000,0000,,And Google has a similar thing,
Dialogue: 0,0:47:30.38,0:47:32.42,Default,,0000,0000,0000,,if you go to like Google Ad Settings,
Dialogue: 0,0:47:32.42,0:47:35.25,Default,,0000,0000,0000,,you can even turn off personalization of ads
Dialogue: 0,0:47:35.25,0:47:36.38,Default,,0000,0000,0000,,if you want,
Dialogue: 0,0:47:36.38,0:47:38.12,Default,,0000,0000,0000,,and you can also see some of the inferences
Dialogue: 0,0:47:38.12,0:47:39.40,Default,,0000,0000,0000,,that have been learned about you.
Dialogue: 0,0:47:39.40,0:47:40.82,Default,,0000,0000,0000,,A subset of the inferences that have been
Dialogue: 0,0:47:40.82,0:47:41.70,Default,,0000,0000,0000,,learned about you.
Dialogue: 0,0:47:41.70,0:47:43.94,Default,,0000,0000,0000,,So, like, what interests...
Dialogue: 0,0:47:43.94,0:47:47.87,Default,,0000,0000,0000,,Herald: A question from the internet, please?
Dialogue: 0,0:47:47.87,0:47:50.93,Default,,0000,0000,0000,,Signal Angel: Yes, billetQ is asking
Dialogue: 0,0:47:50.93,0:47:54.48,Default,,0000,0000,0000,,how do you avoid biases in machine learning?
Dialogue: 0,0:47:54.48,0:47:57.38,Default,,0000,0000,0000,,I asume analysis system, for example,
Dialogue: 0,0:47:57.38,0:48:00.42,Default,,0000,0000,0000,,could be biased against women and minorities,
Dialogue: 0,0:48:00.42,0:48:04.96,Default,,0000,0000,0000,,if used for hiring decisions\Nbased on known data.
Dialogue: 0,0:48:04.96,0:48:06.50,Default,,0000,0000,0000,,Dr. Helsby: Yeah, so one thing is to
Dialogue: 0,0:48:06.50,0:48:08.53,Default,,0000,0000,0000,,just explicitly check.
Dialogue: 0,0:48:08.53,0:48:12.20,Default,,0000,0000,0000,,So, you can check to see how
Dialogue: 0,0:48:12.20,0:48:14.31,Default,,0000,0000,0000,,positive outcomes are being distributed
Dialogue: 0,0:48:14.31,0:48:16.78,Default,,0000,0000,0000,,among those protected classes.
Dialogue: 0,0:48:16.78,0:48:19.21,Default,,0000,0000,0000,,You could also incorporate these sort of
Dialogue: 0,0:48:19.21,0:48:21.44,Default,,0000,0000,0000,,fairness constraints in the function
Dialogue: 0,0:48:21.44,0:48:24.07,Default,,0000,0000,0000,,that you optimize when you train the system,
Dialogue: 0,0:48:24.07,0:48:25.95,Default,,0000,0000,0000,,and so, if you're interested in reading more
Dialogue: 0,0:48:25.95,0:48:28.96,Default,,0000,0000,0000,,about this, the 2 papers--
Dialogue: 0,0:48:28.96,0:48:31.91,Default,,0000,0000,0000,,let me go to References--
Dialogue: 0,0:48:31.91,0:48:32.73,Default,,0000,0000,0000,,there's a good paper called
Dialogue: 0,0:48:32.73,0:48:35.34,Default,,0000,0000,0000,,Fairness Through Awareness that describes
Dialogue: 0,0:48:35.34,0:48:37.50,Default,,0000,0000,0000,,how to go about doing this,
Dialogue: 0,0:48:37.50,0:48:39.58,Default,,0000,0000,0000,,so I recommend this person read that.
Dialogue: 0,0:48:39.58,0:48:40.97,Default,,0000,0000,0000,,It's good.
Dialogue: 0,0:48:40.97,0:48:43.40,Default,,0000,0000,0000,,Herald: Microphone 2, please.
Dialogue: 0,0:48:43.40,0:48:45.40,Default,,0000,0000,0000,,Mic2: Thanks again for your talk.
Dialogue: 0,0:48:45.40,0:48:49.65,Default,,0000,0000,0000,,Umm, hello?
Dialogue: 0,0:48:49.65,0:48:50.100,Default,,0000,0000,0000,,Okay.
Dialogue: 0,0:48:50.100,0:48:52.96,Default,,0000,0000,0000,,Umm, I see of course a problem with
Dialogue: 0,0:48:52.96,0:48:54.62,Default,,0000,0000,0000,,all the black boxes that you describe
Dialogue: 0,0:48:54.62,0:48:57.07,Default,,0000,0000,0000,,with regards for the crime systems,
Dialogue: 0,0:48:57.07,0:48:59.57,Default,,0000,0000,0000,,but when we look at the advertising systems
Dialogue: 0,0:48:59.57,0:49:02.17,Default,,0000,0000,0000,,in many cases they are very networked.
Dialogue: 0,0:49:02.17,0:49:04.16,Default,,0000,0000,0000,,There are many different systems collaborating
Dialogue: 0,0:49:04.16,0:49:07.11,Default,,0000,0000,0000,,and exchanging data via open APIs:
Dialogue: 0,0:49:07.11,0:49:08.72,Default,,0000,0000,0000,,RESTful APIs, and various
Dialogue: 0,0:49:08.72,0:49:11.72,Default,,0000,0000,0000,,demand-side platforms\Nand audience-exchange platforms,
Dialogue: 0,0:49:11.72,0:49:12.54,Default,,0000,0000,0000,,and everything.
Dialogue: 0,0:49:12.54,0:49:15.42,Default,,0000,0000,0000,,So, can that help to at least
Dialogue: 0,0:49:15.42,0:49:22.16,Default,,0000,0000,0000,,increase awareness on where targeting, personalization
Dialogue: 0,0:49:22.16,0:49:23.68,Default,,0000,0000,0000,,might be happening?
Dialogue: 0,0:49:23.68,0:49:26.19,Default,,0000,0000,0000,,I mean, I'm looking at systems like
Dialogue: 0,0:49:26.19,0:49:29.54,Default,,0000,0000,0000,,BuiltWith, that surface what kind of
Dialogue: 0,0:49:29.54,0:49:31.38,Default,,0000,0000,0000,,JavaScript libraries are used elsewhere.
Dialogue: 0,0:49:31.38,0:49:32.100,Default,,0000,0000,0000,,So, is that something that could help
Dialogue: 0,0:49:32.100,0:49:35.67,Default,,0000,0000,0000,,at least to give a better awareness
Dialogue: 0,0:49:35.67,0:49:38.69,Default,,0000,0000,0000,,and listing all the points where
Dialogue: 0,0:49:38.69,0:49:41.41,Default,,0000,0000,0000,,you might be targeted...
Dialogue: 0,0:49:41.41,0:49:43.07,Default,,0000,0000,0000,,Dr. Helsby: So, like, with respect to
Dialogue: 0,0:49:43.07,0:49:46.46,Default,,0000,0000,0000,,advertising, the fact that\Nthere is behind the scenes
Dialogue: 0,0:49:46.46,0:49:48.45,Default,,0000,0000,0000,,this like complicated auction process
Dialogue: 0,0:49:48.45,0:49:50.65,Default,,0000,0000,0000,,that's occurring, just makes things
Dialogue: 0,0:49:50.65,0:49:51.82,Default,,0000,0000,0000,,a lot more complicated.
Dialogue: 0,0:49:51.82,0:49:54.17,Default,,0000,0000,0000,,So, for example, I said briefly
Dialogue: 0,0:49:54.17,0:49:57.27,Default,,0000,0000,0000,,that they found that there's this\Nstatistical difference
Dialogue: 0,0:49:57.27,0:49:59.10,Default,,0000,0000,0000,,between how men and women are treated,
Dialogue: 0,0:49:59.10,0:50:01.34,Default,,0000,0000,0000,,but it doesn't necessarily mean that
Dialogue: 0,0:50:01.34,0:50:03.64,Default,,0000,0000,0000,,"Oh, the algorithm is definitely biased."
Dialogue: 0,0:50:03.64,0:50:06.37,Default,,0000,0000,0000,,It could be because of this auction process,
Dialogue: 0,0:50:06.37,0:50:10.57,Default,,0000,0000,0000,,it could be that women are considered
Dialogue: 0,0:50:10.57,0:50:12.63,Default,,0000,0000,0000,,more valuable when it comes to advertising,
Dialogue: 0,0:50:12.63,0:50:15.10,Default,,0000,0000,0000,,and so these executive ads are getting
Dialogue: 0,0:50:15.10,0:50:17.16,Default,,0000,0000,0000,,outbid by some other ads,
Dialogue: 0,0:50:17.16,0:50:18.89,Default,,0000,0000,0000,,and so there's a lot of potential
Dialogue: 0,0:50:18.89,0:50:20.49,Default,,0000,0000,0000,,causes for that.
Dialogue: 0,0:50:20.49,0:50:22.83,Default,,0000,0000,0000,,So, I think it just makes things\Na lot more complicated.
Dialogue: 0,0:50:22.83,0:50:25.91,Default,,0000,0000,0000,,I don't know if it helps\Nwith the bias at all.
Dialogue: 0,0:50:25.91,0:50:27.41,Default,,0000,0000,0000,,Mic 2: Well, the question was more
Dialogue: 0,0:50:27.41,0:50:30.30,Default,,0000,0000,0000,,a direction... can it help to surface
Dialogue: 0,0:50:30.30,0:50:32.50,Default,,0000,0000,0000,,and make people aware of that fact?
Dialogue: 0,0:50:32.50,0:50:34.93,Default,,0000,0000,0000,,I mean, I can talk to my kids probably,
Dialogue: 0,0:50:34.93,0:50:36.26,Default,,0000,0000,0000,,and they will probably understand,
Dialogue: 0,0:50:36.26,0:50:38.42,Default,,0000,0000,0000,,but I can't explain that to my grandma,
Dialogue: 0,0:50:38.42,0:50:43.15,Default,,0000,0000,0000,,who's also, umm, looking at an iPad.
Dialogue: 0,0:50:43.15,0:50:44.29,Default,,0000,0000,0000,,Dr. Helsby: So, the fact that
Dialogue: 0,0:50:44.29,0:50:45.69,Default,,0000,0000,0000,,the systems are...
Dialogue: 0,0:50:45.69,0:50:48.51,Default,,0000,0000,0000,,I don't know if I understand.
Dialogue: 0,0:50:48.51,0:50:50.53,Default,,0000,0000,0000,,Mic 2: OK. I think that the main problem
Dialogue: 0,0:50:50.53,0:50:53.71,Default,,0000,0000,0000,,is that we are behind the industry efforts
Dialogue: 0,0:50:53.71,0:50:57.18,Default,,0000,0000,0000,,to being targeted at, and many people
Dialogue: 0,0:50:57.18,0:51:00.58,Default,,0000,0000,0000,,do know, but a lot more people don't know,
Dialogue: 0,0:51:00.58,0:51:03.16,Default,,0000,0000,0000,,and making them aware of the fact
Dialogue: 0,0:51:03.16,0:51:07.27,Default,,0000,0000,0000,,that they are a target, in a way,
Dialogue: 0,0:51:07.27,0:51:10.99,Default,,0000,0000,0000,,is something that can only be shown
Dialogue: 0,0:51:10.99,0:51:14.78,Default,,0000,0000,0000,,by a 3rd party that disposed that data,
Dialogue: 0,0:51:14.78,0:51:16.34,Default,,0000,0000,0000,,and make audits in a way--
Dialogue: 0,0:51:16.34,0:51:17.93,Default,,0000,0000,0000,,maybe in an automated way.
Dialogue: 0,0:51:17.93,0:51:19.17,Default,,0000,0000,0000,,Dr. Helsby: Right.
Dialogue: 0,0:51:19.17,0:51:21.41,Default,,0000,0000,0000,,Yeah, I think it certainly\Ncould help with advocacy
Dialogue: 0,0:51:21.41,0:51:23.06,Default,,0000,0000,0000,,if that's the point, yeah.
Dialogue: 0,0:51:23.06,0:51:26.08,Default,,0000,0000,0000,,Herald: Another question\Nfrom the internet, please.
Dialogue: 0,0:51:26.08,0:51:29.32,Default,,0000,0000,0000,,Signal Angel: Yes, on IRC they are asking
Dialogue: 0,0:51:29.32,0:51:31.44,Default,,0000,0000,0000,,if we know that prediction in some cases
Dialogue: 0,0:51:31.44,0:51:34.46,Default,,0000,0000,0000,,provides an influence that cannot be controlled.
Dialogue: 0,0:51:34.46,0:51:38.48,Default,,0000,0000,0000,,So, r4v5 would like to know from you
Dialogue: 0,0:51:38.48,0:51:41.52,Default,,0000,0000,0000,,if there are some cases or areas where
Dialogue: 0,0:51:41.52,0:51:45.06,Default,,0000,0000,0000,,machine learning simply shouldn't go?
Dialogue: 0,0:51:45.06,0:51:48.35,Default,,0000,0000,0000,,Dr. Helsby: Umm, so I think...
Dialogue: 0,0:51:48.35,0:51:52.56,Default,,0000,0000,0000,,I mean, yes, I think that it is the case
Dialogue: 0,0:51:52.56,0:51:54.65,Default,,0000,0000,0000,,that in some cases machine learning
Dialogue: 0,0:51:54.65,0:51:56.18,Default,,0000,0000,0000,,might not be appropriate.
Dialogue: 0,0:51:56.18,0:51:58.36,Default,,0000,0000,0000,,For example, if you use machine learning
Dialogue: 0,0:51:58.36,0:52:00.97,Default,,0000,0000,0000,,to decide who should be searched.
Dialogue: 0,0:52:00.97,0:52:02.62,Default,,0000,0000,0000,,I don't think it should be the case that
Dialogue: 0,0:52:02.62,0:52:03.81,Default,,0000,0000,0000,,machine learning algorithms should
Dialogue: 0,0:52:03.81,0:52:05.44,Default,,0000,0000,0000,,ever be used to determine
Dialogue: 0,0:52:05.44,0:52:08.43,Default,,0000,0000,0000,,probable cause, or something like that.
Dialogue: 0,0:52:08.43,0:52:12.34,Default,,0000,0000,0000,,So, if it's just one piece of evidence
Dialogue: 0,0:52:12.34,0:52:13.30,Default,,0000,0000,0000,,that you consider,
Dialogue: 0,0:52:13.30,0:52:14.99,Default,,0000,0000,0000,,and there's human oversight always,
Dialogue: 0,0:52:14.99,0:52:18.52,Default,,0000,0000,0000,,{\i1}maybe{\i0} it's fine, but
Dialogue: 0,0:52:18.52,0:52:20.84,Default,,0000,0000,0000,,we should be very suspicious and hesitant
Dialogue: 0,0:52:20.84,0:52:22.12,Default,,0000,0000,0000,,in certain contexts where
Dialogue: 0,0:52:22.12,0:52:24.53,Default,,0000,0000,0000,,the ramifications are very serious.
Dialogue: 0,0:52:24.53,0:52:27.26,Default,,0000,0000,0000,,Like the No Fly List, and so on.
Dialogue: 0,0:52:27.26,0:52:29.20,Default,,0000,0000,0000,,Herald: And #2 again.
Dialogue: 0,0:52:29.20,0:52:30.81,Default,,0000,0000,0000,,Mic 2: A second question
Dialogue: 0,0:52:30.81,0:52:33.51,Default,,0000,0000,0000,,that just occurred to me, if you don't mind.
Dialogue: 0,0:52:33.51,0:52:35.34,Default,,0000,0000,0000,,Umm, until the advent of
Dialogue: 0,0:52:35.34,0:52:36.56,Default,,0000,0000,0000,,algorithmic systems,
Dialogue: 0,0:52:36.56,0:52:40.47,Default,,0000,0000,0000,,when there've been cases of serious harm
Dialogue: 0,0:52:40.47,0:52:42.80,Default,,0000,0000,0000,,that's been resulted in individuals or groups,
Dialogue: 0,0:52:42.80,0:52:44.58,Default,,0000,0000,0000,,and it's been demonstrated that
Dialogue: 0,0:52:44.58,0:52:46.03,Default,,0000,0000,0000,,it's occurred because of
Dialogue: 0,0:52:46.03,0:52:49.40,Default,,0000,0000,0000,,an individual or a system of people
Dialogue: 0,0:52:49.40,0:52:53.02,Default,,0000,0000,0000,,being systematically biased, then often
Dialogue: 0,0:52:53.02,0:52:55.13,Default,,0000,0000,0000,,one of the actions that's taken is
Dialogue: 0,0:52:55.13,0:52:56.87,Default,,0000,0000,0000,,pressure's applied, and then
Dialogue: 0,0:52:56.87,0:52:59.66,Default,,0000,0000,0000,,people are required to change,
Dialogue: 0,0:52:59.66,0:53:01.05,Default,,0000,0000,0000,,and hopely be held responsible,
Dialogue: 0,0:53:01.05,0:53:02.91,Default,,0000,0000,0000,,and then change the way that they do things
Dialogue: 0,0:53:02.91,0:53:06.40,Default,,0000,0000,0000,,to try to remove bias from that system.
Dialogue: 0,0:53:06.40,0:53:07.84,Default,,0000,0000,0000,,What's the current thinking about
Dialogue: 0,0:53:07.84,0:53:10.30,Default,,0000,0000,0000,,how we can go about doing that
Dialogue: 0,0:53:10.30,0:53:12.60,Default,,0000,0000,0000,,when the systems that are doing that
Dialogue: 0,0:53:12.60,0:53:13.65,Default,,0000,0000,0000,,are algorithmic?
Dialogue: 0,0:53:13.65,0:53:15.100,Default,,0000,0000,0000,,Is it just going to be human oversight,
Dialogue: 0,0:53:15.100,0:53:16.91,Default,,0000,0000,0000,,and humans are gonna have to be
Dialogue: 0,0:53:16.91,0:53:18.38,Default,,0000,0000,0000,,held responsible for the oversight?
Dialogue: 0,0:53:18.38,0:53:20.89,Default,,0000,0000,0000,,Dr. Helsby: So, in terms of bias,
Dialogue: 0,0:53:20.89,0:53:22.57,Default,,0000,0000,0000,,if we're concerned about bias towards
Dialogue: 0,0:53:22.57,0:53:24.02,Default,,0000,0000,0000,,particular types of people,
Dialogue: 0,0:53:24.02,0:53:25.71,Default,,0000,0000,0000,,that's something that we can optimize for.
Dialogue: 0,0:53:25.71,0:53:28.84,Default,,0000,0000,0000,,So, we can train systems that are unbiased
Dialogue: 0,0:53:28.84,0:53:30.02,Default,,0000,0000,0000,,in this way.
Dialogue: 0,0:53:30.02,0:53:32.11,Default,,0000,0000,0000,,So that's one way to deal with it.
Dialogue: 0,0:53:32.11,0:53:34.04,Default,,0000,0000,0000,,But there's always gonna be errors,
Dialogue: 0,0:53:34.04,0:53:35.42,Default,,0000,0000,0000,,so that's sort of a separate issue
Dialogue: 0,0:53:35.42,0:53:37.51,Default,,0000,0000,0000,,from the bias, and in the case
Dialogue: 0,0:53:37.51,0:53:39.18,Default,,0000,0000,0000,,where there are errors,
Dialogue: 0,0:53:39.18,0:53:40.54,Default,,0000,0000,0000,,there must be oversight.
Dialogue: 0,0:53:40.54,0:53:45.08,Default,,0000,0000,0000,,So, one way that one could improve
Dialogue: 0,0:53:45.08,0:53:46.41,Default,,0000,0000,0000,,the way that this is done
Dialogue: 0,0:53:46.41,0:53:48.16,Default,,0000,0000,0000,,is by making sure that you're
Dialogue: 0,0:53:48.16,0:53:50.80,Default,,0000,0000,0000,,keeping track of confidence of decisions.
Dialogue: 0,0:53:50.80,0:53:54.04,Default,,0000,0000,0000,,So, if you have a low confidence prediction,
Dialogue: 0,0:53:54.04,0:53:56.26,Default,,0000,0000,0000,,then maybe a human\Nshould come in and check things.
Dialogue: 0,0:53:56.26,0:53:58.81,Default,,0000,0000,0000,,So, that might be one way to proceed.
Dialogue: 0,0:54:02.10,0:54:03.99,Default,,0000,0000,0000,,Herald: So, there's no more question.
Dialogue: 0,0:54:03.99,0:54:06.20,Default,,0000,0000,0000,,I close this talk now,
Dialogue: 0,0:54:06.20,0:54:08.24,Default,,0000,0000,0000,,and thank you very much
Dialogue: 0,0:54:08.24,0:54:09.41,Default,,0000,0000,0000,,and a big applause to
Dialogue: 0,0:54:09.41,0:54:11.78,Default,,0000,0000,0000,,Jennifer Helsby!
Dialogue: 0,0:54:11.78,0:54:16.31,Default,,0000,0000,0000,,{\i1}roaring applause{\i0}
Dialogue: 0,0:54:16.31,0:54:28.00,Default,,0000,0000,0000,,subtitles created by c3subtitles.de\NJoin, and help us!