32c3 preroll music Angel: I introduce Whitney Merrill. She is an attorney in the US and she just recently, actually last week, graduated to her CS masters in Illinois. applause Angel: Without further ado: ‘Predicting Crime In A Big Data World’. cautious applause Whitney Merrill: Hi everyone. Thank you so much for coming. I know it´s been a exhausting Congress, so I appreciate you guys coming to hear me talk about Big Data and Crime Prediction. This is kind of a hobby of mine, I, in my last semester at Illinois, decided to poke around what´s currently happening, how these algorithms are being used and kind of figure out what kind of information can be gathered. So, I have about 30 minutes with you guys. I´m gonna do a broad overview of the types of programs. I´m gonna talk about what Predictive Policing is, the data used, similar systems in other areas where predictive algorithms are trying to better society, current uses in policing. I´m gonna talk a little bit about their effectiveness and then give you some final thoughts. So, imagine, in the very near future a Police officer is walking down the street wearing a camera on her collar. In her ear is a feed of information about the people and cars she passes alerting her to individuals and cars that might fit a particular crime or profile for a criminal. Early in the day she examined a map highlighting hotspots for crime. In the area she´s been set to patrol the predictive policing software indicates that there is an 82% chance of burglary at 2 pm, and it´s currently 2:10 pm. As she passes one individual her camera captures the individual´s face, runs it through a coordinated Police database - all of the Police departments that use this database are sharing information. Facial recognition software indicates that the person is Bobby Burglar who was previously convicted of burglary, was recently released and is now currently on patrole. The voice in her ear whispers: 50 percent likely to commit a crime. Can she stop and search him? Should she chat him up? Should see how he acts? Does she need additional information to stop and detain him? And does it matter that he´s carrying a large duffle bag? Did the algorithm take this into account or did it just look at his face? What information was being collected at the time the algorithm chose to say 50% to provide the final analysis? So, another thought I´m gonna have you guys think about as I go through this presentation, is this quote that is more favorable towards Police algorithms, which is: “As people become data plots and probability scores, law enforcement officials and politicians alike can point and say: ‘Technology is void of the racist, profiling bias of humans.’” Is that true? Well, they probably will point and say that, but is it actually void of racist, profiling humans? And I´m gonna talk about that as well. So, Predictive Policing explained. Who and what? First of all, Predictive Policing actually isn´t new. All we´re doing is adding technology, doing better, faster aggregation of data. Analysts in Police departments have been doing this by hand for decades. These techniques are used to create profiles that accurately match likely offenders with specific past crimes. So, there´s individual targeting and then we have location-based targeting. The location-based, the goal is to help Police forces deploy their resources in a correct manner, in an efficient manner. They can be as simple as recommending that general crime may happen in a particular area, or specifically, what type of crime will happen in a one-block-radius. They take into account the time of day, the recent data collected and when in the year it´s happening as well as weather etc. So, another really quick thing worth going over, cause not everyone is familiar with machine learning. This is a very basic breakdown of training an algorithm on a data set. You collect it from many different sources, you put it all together, you clean it up, you split it into 3 sets: a training set, a validation set and a test set. The training set is what is going to develop the rules in which it´s going to kind of determine the final outcome. You´re gonna use a validation set to optimize it and finally apply this to establish a confidence level. There you´ll set a support level where you say you need a certain amount of data to determine whether or not the algorithm has enough information to kind of make a prediction. So, rules with a low support level are less likely to be statistically significant and the confidence level in the end is basically if there´s an 85% confidence level that means there´s an 85% chance that the suspect, e.g. meeting the rule in question, is engaged in criminal conduct. So, what does this mean? Well, it encourages collection and hoarding of data about crimes and individuals. Because you want as much information as possible so that you detect even the less likely scenarios. Information sharing is also encouraged because it´s easier, it´s done by third parties, or even what are called fourth parties and shared amongst departments. And here, your criminal data again was being done by analysts in Police departments for decades, but the information sharing and the amount of information they could aggregate was just significantly more difficult. So, what are these Predictive Policing algorithms and software… what are they doing? Are they determining guilt and innocence? And, unlike a thoughtcrime, they are not saying this person is guilty, this person is innocent. It´s creating a probability of whether or not the person has likely committed a crime or will likely commit a crime. And it can only say something to the future and the past. This here is a picture from one particular piece of software provided by HunchLab; and patterns emerge here from past crimes that can profile criminal types and associations, detect crime patterns etc. Generally in this types of algorithms they are using unsupervised data, that means someone is not going through and saying true-false, good-bad, good-bad. There´s just 1) too much information and 2) they´re trying to do clustering, determine the things that are similar. So, really quickly, I´m also gonna talk about the data that´s used. There are several different types: Personal characteristics, demographic information, activities of individuals, scientific data etc. This comes from all sorts of sources, one that really shocked me, was, and I´ll talk about it a little bit in the future, but, is the radiation detectors on New York City Police are constantly taking in data and it´s so sensitive, it can detect if you´ve had a recent medical treatment that involves radiation. Facial recognition and biometrics are clear here and the third-party doctrine – which basically says in the United States that you have no reasonable expectation of privacy in data you share with third parties – facilitates easy collection for Police officers and Government officials because they can go and ask for the information without any sort of warrant. For a really great overview: a friend of mine, Dia, did a talk here at CCC on “The architecture of a street level panopticon”. Does a really great overview of how this type of data is collected on the streets. Worth checking out ´cause I´m gonna gloss over kind of the types of data. There is in the United States what they call Multistate Anti-Terrorism Information Exchange Program which uses everything from credit history, your concealed weapons permits, aircraft pilot licenses, fishing licences etc. that´s searchable and shared amongst Police departments and Government officials and this is just more information. So, if they can collect it, they will aggregate it into a data base. So, what are the current uses? There are many, many different companies currently making software and marketing it to Police departments. All of them are slightly different, have different features, but currently it´s a competition to get clients, Police departments etc. The more Police departments you have the more data sharing you can sell, saying: “Oh, by enrolling you’ll now have x,y and z Police departments’ data to access” etc. These here are Hitachi and HunchLab, they both are hotspot targeting, it´s not individual targeting, those are a lot rarer. And it´s actually being used in my home town, which I´ll talk about in a little bit. Here, the appropriate tactics are automatically displayed for officers when they´re entering mission areas. So HunchLab will tell an officer: “Hey, you´re entering an area where there´s gonna be burglary that you should keep an eye out, be aware”. And this is updating in live time and they´re hoping it mitigates crime. Here are 2 other ones, the Domain Awareness System was created in New York City after 9/11 in conjunction with Microsoft. New York City actually makes money selling it to other cities to use this. CCTV-cameras are collected, they can… If they say there´s a man wearing a red shirt, the software will look for people wearing red shirts and at least alert Police departments to people that meet this description walking in public in New York City. The other one is by IBM and there are quite a few, you know, it´s just generally another hotspot targeting, each have a few different features. Worth mentioning, too, is the Heat List. This targeted individuals. I’m from the city of Chicago. I grew up in the city. There are currently 420 names, when this came out about a year ago, of individuals who are 500 times more likely than average to be involved in violence. Individual names, passed around to each Police officer in Chicago. They consider the rap sheet, disturbance calls, social network etc. But one of the main things they considered in placing mainly young black individuals on this list were known acquaintances and their arrest histories. So if kids went to school or young teenagers went to school with several people in a gang – and that individual may not even be involved in a gang – they’re more likely to appear on the list. The list has been heavily criticized for being racist, for not giving these children or young individuals on the list a chance to change their history because it’s being decided for them. They’re being told: “You are likely to be a criminal, and we’re gonna watch you”. Officers in Chicago visited these individuals would do knock and announce with a knock on the door and say: “Hi, I’m here, like just checking up what are you up to”. Which you don’t need any special suspicion to do. But it’s, you know, kind of a harassment that might cause a feedback, back into the data collected. This is PRECOBS. It’s currently used here in Hamburg. They actually went to Chicago and visited the Chicago Police Department to learn about Predictive Policing tactics in Chicago to implement it throughout Germany, Hamburg and Berlin. It’s used to generally forecast repeat-offenses. Again, when training data sets you need enough data points to predict crime. So crimes that are less likely to happen or happen very rarely: much harder to predict. Crimes that aren’t reported: much harder to predict. So a lot of these software… like pieces of software rely on algorithms that are hoping that there’s a same sort of picture, that they can predict: where and when and what type of crime will happen. PRECOBS is actually a term with a plan – the movie ‘Minority Report’, if you’re familiar with it, it’s the 3 psychics who predict crimes before they happen. So there’re other, similar systems in the world that are being used to predict whether or not something will happen. The first one is ‘Disease and Diagnosis’. They found that algorithms are actually more likely than doctors to predict what disease an individual has. It’s kind of shocking. The other is ‘Security Clearance’ in the US. It allows access to classified documents. There’s no automatic access in the US. So every person who wants to see some sort of secret cleared document must go through this process. And it’s vetting individuals. So it’s an opt-in process. But here they’re trying to predict who will disclose information, who will break the clearance system; and predict there… Here, the error rate, they’re probably much more comfortable with a high error rate. Because they have so many people competing for a particular job, to get clearance, that if they’re wrong, that somebody probably won’t disclose information, they don’t care, they just rather eliminate them than take the risk. So I’m an attorney in the US. I have this urge to talk about US law. It also seems to impact a lot of people internationally. Here we’re talking about the targeting of individuals, not hotspots. So targeting of individuals is not as widespread, currently. However it’s happening in Chicago; and other cities are considering implementing programs and there are grants right now to encourage Police departments to figure out target lists. So in the US suspicion is based on the totality of the circumstances. That’s the whole picture. The Police officer, the individual must look at the whole picture of what’s happening before they can detain an individual. It’s supposed to be a balanced assessment of relative weights, meaning – you know – if you know that the person is a pastor maybe then pacing in front of a liquor store, is not as suspicious as somebody who’s been convicted of 3 burglaries. It has to be ‘based on specific and articulable facts’. And the Police officers can use experience and common sense to determine whether or not their suspicion… Large amounts of networked data generally can provide individualized suspicion. The principal components here… the events leading up to the stop-and-search – what is the person doing right before they’re detained as well as the use of historical facts known about that individual, the crime, the area in which it’s happening etc. So it can rely on both things. No court in the US has really put out a percentage as what Probable Cause and Reasonable Suspicion. So ‘Probable Cause’ you need to get a warrant to search and seize an individual. ‘Reasonable Suspicion’ is needed to do stop-and-frisk in the US – stop an individual and question them. And this is a little bit different than what they call ‘Consensual Encounters’, where a Police officer goes up to you and chats you up. ‘Reasonable Suspicion’ – you’re actually detained. But I had a law professor who basically said: “30%..45% seem like a really good number just to show how low it really is”.You don’t even need to be 50% sure that somebody has committed a crime. So, officers can draw from their own experience to determine ‘Probable Cause’. And the UK has a similar ‘Reasonable Suspicion’ standard which depend on the circumstances of each case. So, I’m not as familiar with UK law but I believe even that some of the analysis-run ‘Reasonable Suspicion’ is similar. Is this like a black box? So, I threw this slide in for those who are interested in comparing this US law. Generally a dog sniff in the US falls under a particular set of legal history which is: a dog can go up, sniff for dogs, alert and that is completely okay. And the Police officers can use that data to detain and further search an individual. So is an algorithm similar to the dog which is kind of a black box? Information goes out, it’s processed, information comes out and a prediction is made. Police rely on the ‘Good Faith’ in ‘Totality of the Circumstances’ to make their decision. So there’s really no… if they’re relying on the algorithm and think in that situation that everything’s okay we might reach a level of ‘Reasonable Suspicion’ where the individual can now pat down the person he’s decided on the street or the algorithm has alerted to. So, the big question is, you know, “Could the officer consult predictive software apps in any individual analysis. Could he say: ‘60% likely to commit a crime’”. In my hypo: Does that mean that the person without looking at anything else detain that individual. And the answer is “Probably not”. One: predictive Policing algorithms just can not take in the Totality of the Circumstances. They have to be frequently updated, there are things that are happening that the algorithm possibly could not have taken into account. The problem here is that the algorithm itself, the prediction itself becomes part of Totality of the Circumstances, which I’m going to talk about a little bit more later. But officers have to have Reasonable Suspicion before the stop occurs. Retroactive justification is not sufficient. So, the algorithm can’t just say: “60% likely, you detain the individual and then figure out why you’ve detained the person”. It has to be before the detention actually happens. And the suspicion must relate to current criminal activity. The person must be doing something to indicate criminal activity. Just the fact that an algorithm says, based on these facts: “60%”, or even without articulating why the algorithm has chosen that, isn’t enough. Maybe you can see a gun shaped bulge in the pocket etc. So, effectiveness… the Totality of the Circumstances, can the algorithms keep up? Generally, probably not. Missing data, not capable of processing this data in real time. There’s no idea… the algorithm doesn’t know, and the Police officer probably doesn’t know the all of the facts. So the Police officer can take the algorithm into consideration but the problem here is: Did the algorithm know that the individual was active in the community, or was a politician, or that was a personal friend of the officer etc. It can’t just be relied upon. What if the algorithm did take into account that the individual was a Pastor? Now that information is counted twice and the balancing for the Totality of the Circumstances is off. Humans here must be the final decider. What are the problems? Well, there’s bad underlying data, there’s no transparency into what kind of data is being used, how it was collected, how old it is, how often it’s been updated, whether or not it’s been verified. There could just be noise in the training data. Honestly, the data is biased. It was collected by individuals in the US; generally there’ve been several studies done that black, young individuals are stopped more often than whites. And this is going to cause a collection bias. It’s gonna be drastically disproportionate to the makeup of the population of cities; and as more data has been collected on minorities, refugees in poor neighborhoods it’s gonna feed back in and of course only have data on those groups and provide feedback and say: “More crime is likely to happen because that’s where the data was collected”. So, what’s an acceptable error rate, well, depends on the burden of proof. Harm is different for an opt-in system. You know, what’s my harm if I don’t get clearance, or I don’t get the job; but I’m opting in, I’m asking to being considered for employment. In the US, what’s an error? If you search and find nothing, if you think you have Reasonable Suspicion based on good faith, both on the algorithm and what you witness, the US says that it’s no 4th Amendment violation, even if nothing has happened. It’s very low error false-positive rate here. In Big Data, generally, and machine-learning it’s great! Like 1% error is fantastic! But that’s pretty large for the number of individuals stopped each day. Or who might be subject to these algorithms. Because even though there’re only 400 individuals on the list in Chicago those individuals have been listed basically as targets by the Chicago Police Department. Other problems include database errors. Exclusion of evidence in the US only happens when there’s gross negligence or systematic misconduct. That’s very difficult to prove, especially when a lot of people view these algorithms as a big box. Data goes in, predictions come out, everyone’s happy. You rely and trust on the quality of IBM, HunchLab etc. to provide good software. Finally, some more concerns I have include feedback loop auditing and access to data and algorithms and the prediction thresholds. How certain must a prediction be – before it’s reported to the Police – that the person might commit a crime. Or that crime might happen in the individual area. If Reasonable Suspicion is as low as 35%, and reasonable Suspicion in the US has been held at: That guy drives a car that drug dealers like to drive, and he’s in the DEA database as a possible drug dealer. That was enough to stop and search him. So, are there Positives? Well, PredPol, which is one of the services that provides Predictive Policing software, says: “Since these cities have implemented there’s been dropping crime”. In L.A. 13% reduction in crime, in one division. There was even one day where they had no crime reported. Santa Cruz – 25..29% reduction, -9% in assaults etc. One: these are Police departments self-reporting these successes for… you know, take it for what it is and reiterated by the people selling the software. But perhaps it is actually reducing crime. It’s kind of hard to tell because there’s a feedback loop. Do we know that crime is really being reduced? Will it affect the data that is collected in the future? It’s really hard to know. Because if you send the Police officers into a community it’s more likely that they’re going to affect that community and that data collection. Will more crimes happen because they feel like the Police are harassing them? It’s very likely and it’s a problem here. So, some final thoughts. Predictive Policing programs are not going anywhere. They’re only in their wheelstart. And I think that more analysis, more transparency, more access to data needs to happen around these algorithms. There needs to be regulation. Currently, a very successful way in which these companies get data is they buy from Third Party sources and then sell it to Police departments. So perhaps PredPol might get information from Google, Facebook, Social Media accounts; aggregate data themselves, and then turn around and sell it to Police departments or provide access to Police departments. And generally, the Courts are gonna have to begin to work out how to handle this type of data. There’s not case law, at least in the US, that really knows how to handle predictive algorithms in determining what the analysis says. And so there really needs to be a lot more research and thought put into this. And one of the big things in order for this to actually be useful: if this is a tactic that had been used by Police departments for decades, we need to eliminate the bias in the data sets. Because right now all that it’s doing is facilitating and continuing bias, set in the database. And it’s incredibly difficult. It’s data collected by humans. And it causes initial selection bias. Which is gonna have to stop for it to be successful. And perhaps these systems can cause implicit bias or confirmation bias, e.g. Police are going to believe what they’ve been told. So if a Police officer goes on duty to an area and an algorithm says: “You’re 70% likely to find a burglar in this area”. Are they gonna find a burglar because they’ve been told: “You might find a burglar”? And finally the US border. There is no 4th Amendment protection at the US border. It’s an exception to the warrant requirement. This means no suspicion is needed to commit a search. So this data is gonna go into a way to examine you when you cross the border. And aggregate data can be used to refuse you entry into the US etc. And I think that’s pretty much it. And so a few minutes for questions. applause Thank you! Herald: Thanks a lot for your talk, Whitney. We have about 4 minutes left for questions. So please line up at the microphones and remember to make short and easy questions. Microphone No.2, please. Question: Just a comment: if I want to run a crime organization, like, I would target the PRECOBS here in Hamburg, maybe. So I can take the crime to the scenes where the PRECOBS doesn’t suspect. Whitney: Possibly. And I think this is a big problem in getting availability of data; in that there’s a good argument for Police departments to say: “We don’t want to tell you what our tactics are for Policing, because it might move crime”. Herald: Do we have questions from the internet? Yes, then please, one question from the internet. Signal Angel: Is there evidence that data like the use of encrypted messaging systems, encrypted emails, VPN, TOR, with automated request to the ISP, are used to obtain real names and collected to contribute to the scoring? Whitney: I’m not sure if that’s being taken into account by Predictive Policing algorithms, or by the software being used. I know that Police departments do take those things into consideration. And considering that in the US Totality of the Circumstances is how you evaluate suspicion. They are gonna take all of those things into account and they actually kind of have to take into account. Herald: Okay, microphone No.1, please. Question: In your example you mentioned disease tracking, e.g. Google Flu Trends is a good example of preventive Predictive Policing. Are there any examples where – instead of increasing Policing in the lives of communities – where sociologists or social workers are called to use predictive tools, instead of more criminalization? Whitney: I’m not aware if that’s… if Police departments are sending social workers instead of Police officers. But that wouldn’t surprise me because algorithms are being used to suspect child abuse. And in the US they’re gonna send a social worker in regard. So I would not be surprised if that’s also being considered. Since that’s part of the resources. Herald: OK, so if you have a really short question, then microphone No.2, please. Last question. Question: Okay, thank you for the talk. This talk as well as few others brought the thought in the debate about the fine-tuning that is required between false positives and preventing crimes or terror. Now, it’s a different situation if the Policeman is predicting, or a system is predicting somebody’s stealing a paper from someone; or someone is creating a terror attack. And the justification to prevent it under the expense of false positive is different in these cases. How do you make sure that the decision or the fine-tuning is not going to be deep down in the algorithm and by the programmers, but rather by the customer – the Policemen or the authorities? Whitney: I can imagine that Police officers are using common sense in that, and their knowledge about the situation and even what they’re being told by the algorithm. You hope that they’re gonna take… they probably are gonna take terrorism to a different level than a common burglary or a stealing of a piece of paper or a non-violent crime. And that fine-tuning is probably on a Police department by Police department basis. Herald: Thank you! This was Whitney Merrill, give a warm round of applause, please!! Whitney: Thank you! applause postroll music Subtitles created by c3subtitles.de in the year 2016. Join and help us!