WEBVTT 00:00:05.012 --> 00:00:17.227 rC3 preroll music 00:00:17.227 --> 00:00:24.052 Herald: Yeah, and the next talk I'm very proud to announce, we have a speaker who's 00:00:24.932 --> 00:00:31.332 coming in from sunny California and he's an attorney, he's working for Harvard. 00:00:31.332 --> 00:00:36.932 He's doing so many things and he's fighting for our digital rights. Very 00:00:36.932 --> 00:00:41.892 happy to say hi. Welcome. Kurt Opsahl: Thank you! 00:00:46.612 --> 00:00:58.212 Herald: And "Spot the Surveillance" is the topic. We'll see what we haven't seen 00:00:58.212 --> 00:01:04.908 before. And I'm very happy that you're here and... Kurt Opsahl. Please, let us 00:01:04.908 --> 00:01:09.492 know what's up. Thank you. 00:01:09.492 --> 00:01:11.880 Kurt: Thank you. Hello, everybody, my name is Kurt, also I'm the deputy executive 00:01:11.880 --> 00:01:16.767 director general counsel of the Electronic Frontier Foundation. I'm here to talk to 00:01:16.767 --> 00:01:21.534 you about observing police surveillance of protests. So why do we want to observe 00:01:21.534 --> 00:01:26.780 police at protests? Well, because protests are political expression. And the Council 00:01:26.780 --> 00:01:31.023 of Europe put it the right of individuals to gather with other people and make their 00:01:31.023 --> 00:01:35.175 collective voice heard is fundamental to a properly functioning democracy. And this 00:01:35.175 --> 00:01:38.522 is a right which is protected by the European Convention on Human Rights and 00:01:38.522 --> 00:01:43.937 other international rights treaties. But surveillance - consumer's rights. So 00:01:43.937 --> 00:01:48.377 knowing what technologies are used can help you understand the threats to your 00:01:48.377 --> 00:01:52.808 privacy and security, as well as provide tools to advocate for limits on police use 00:01:52.808 --> 00:01:55.941 of surveillance; surveillance that may chill people's right to express themselves 00:01:55.941 --> 00:02:00.532 on these public issues. Just as an analog surveillance historically has been used as 00:02:00.532 --> 00:02:04.512 a tool of repression. Nowadays, policymakers and the public have to 00:02:04.512 --> 00:02:10.471 understand that the threat posed by emerging technologies is a danger to human 00:02:10.471 --> 00:02:14.149 rights, and they need to understand that to successfully defend human rights in the 00:02:14.149 --> 00:02:18.989 digital age. So journalists who are reporting on protest action should know 00:02:18.989 --> 00:02:24.171 the surveillance that is in use. Activists who are advocating for limitations on 00:02:24.171 --> 00:02:27.991 police use of surveillance need to know what surveillance is being used to 00:02:27.991 --> 00:02:31.947 effectively advocate. And legal observers may need to document the use of 00:02:31.947 --> 00:02:35.764 surveillance at protests in order to challenge the police actions at the 00:02:35.764 --> 00:02:39.412 protest or challenge the police policies that are being used after the protest with 00:02:39.412 --> 00:02:44.047 the footage they've obtained. So where are we going today? We're going to provide a 00:02:44.047 --> 00:02:48.513 lot of information about various types of surveillance technologies in use by police 00:02:48.513 --> 00:02:55.211 around the world. We're going to look at what the appearance is, how it works, what 00:02:55.211 --> 00:03:00.279 kind of data they collect and how they're used by police. And at the end a few other 00:03:00.279 --> 00:03:05.057 resources available for those who want to dive a little bit deeper on the topic. So 00:03:05.057 --> 00:03:09.966 police surveillance technology is everywhere. It's on the police themselves, 00:03:09.966 --> 00:03:16.997 on their vehicles, on the roadways. It could be above you in the air, surrounding 00:03:16.997 --> 00:03:21.632 you in the environment, can be a lot of different places and you need to know 00:03:21.632 --> 00:03:26.432 where. The police officers themselves often find in the form of either body worn 00:03:26.432 --> 00:03:30.515 cameras or additional devices that they're using, which are basically mobile 00:03:30.515 --> 00:03:37.186 biometric sensors. Body-Worn cameras or technology that's come out become more 00:03:37.186 --> 00:03:40.929 popular over the last decade or so. Originally was something that was being 00:03:40.929 --> 00:03:46.973 used as a way to provide police accountability that give a record of their 00:03:46.973 --> 00:03:50.628 interactions with the public. And maybe, for example, could you show police 00:03:50.628 --> 00:03:54.554 brutality or maybe deter police brutality? But there are two-way streets: these are 00:03:54.554 --> 00:03:59.423 often used to surveil protesters, and the footage may later be used to support 00:03:59.423 --> 00:04:09.085 arrests and charges. For example, we use this NPR story where after a rally weeks 00:04:09.085 --> 00:04:15.929 later, the police went identifying people through body cam footage and brought 00:04:15.929 --> 00:04:20.296 action against them for obstructing the roadway, part of the civil disobedience of 00:04:20.296 --> 00:04:27.587 the protest, based on finding them on the body camera footage. They can be in a all 00:04:27.587 --> 00:04:32.523 variety of places, so if you're looking for Body-Worn cameras, you got to look at 00:04:32.523 --> 00:04:37.214 different places to see where they might be. So a couple of places they might see 00:04:37.214 --> 00:04:40.707 them: on the head - head mounted camera. So it might be on the glasses on the side, 00:04:40.707 --> 00:04:47.240 could be a lens right in the center. The center one is pretty hard to find, but the 00:04:47.240 --> 00:04:52.075 ones on the side or might be part of the glasses or the helmet they're wearing are 00:04:52.075 --> 00:04:56.363 generally pretty obvious. These ones, they're not particularly common, but they 00:04:56.363 --> 00:05:03.737 do happen. Shoulder mounted cameras also a little bit less common, but they have an 00:05:03.737 --> 00:05:07.760 interesting feature: in this case they're using the Warrior 360 from Blue Line 00:05:07.760 --> 00:05:14.993 Innovations as an example, and it is a dome camera that looks all directions, 360 00:05:14.993 --> 00:05:20.515 degrees off the officer's left shoulder. All most cameras like a front facing 00:05:20.515 --> 00:05:28.254 camera will capture only 180 degrees. Chest mounted cameras are the most common 00:05:28.254 --> 00:05:34.579 and these are being used very, very widely. We give some examples here from 00:05:34.579 --> 00:05:44.173 Amsterdam, Middelburg and from West Midlands Police in the EU... or soon to be 00:05:44.173 --> 00:05:50.134 not in EU for I guess Britain. And there are several known types: Excelon, Wolfcom 00:05:50.134 --> 00:05:55.610 and Watchguard are very common. They operate in similar manners, though, with 00:05:55.610 --> 00:06:03.097 some differences. And you can take a look at some of the examples that are available 00:06:03.097 --> 00:06:06.632 to those companies webpages where they will explain the products they have an 00:06:06.632 --> 00:06:10.883 offer and see what matches up for your jurisdiction. Or you can also look in for 00:06:10.883 --> 00:06:14.972 news article. Oftentimes there's a news article about what may the first policy to 00:06:14.972 --> 00:06:20.866 bring Body_Worn cameras is introduced in a particular police department. The're also 00:06:20.866 --> 00:06:25.582 smartphone based cameras, and these are kind of the low end, it's basically just 00:06:25.582 --> 00:06:30.161 an Android cell phone using its internal camera with an app that does recording 00:06:30.161 --> 00:06:35.595 placed in a pocket. So the camera is a little bit above the clock and you can see 00:06:35.595 --> 00:06:39.946 forward. But it's also a very subtle technique. And it could be easily confused 00:06:39.946 --> 00:06:44.024 if you were looking for someone just starting their phone to their pocket. It 00:06:44.024 --> 00:06:47.952 also might be clipped somewhere on the on their uniform. But if you see anything 00:06:47.952 --> 00:06:52.546 where the camera is facing outward and it's attached to the officer, there's a 00:06:52.546 --> 00:06:57.757 good chance that that is a Body-Worn camera and the setup is on play. Last of 00:06:57.757 --> 00:07:01.482 the Body-Worn cameras will talk about the semi-obscured cams. So this is an example 00:07:01.482 --> 00:07:06.987 for coverable Body-Worn, Body-Worn Utility, and it is 00:07:06.987 --> 00:07:14.163 partially concealed, basically looks like a button on someone's uniform that if 00:07:14.163 --> 00:07:18.514 you're not looking closely, you might not notice. But if you see, if you know, it 00:07:18.514 --> 00:07:23.091 appears where you would expect actually not to have a button, it's slightly 00:07:23.091 --> 00:07:25.858 larger, looks a little bit different. It looks like a camera if you look closely. 00:07:25.858 --> 00:07:28.806 But if you're looking at a distance, you're not particularly paying attention. 00:07:28.806 --> 00:07:34.835 You might not see it at all. In addition to Body-Worn cameras, there are often used 00:07:34.835 --> 00:07:43.568 mobile biometric devices, so these can be handheld scanners, could be a tablet or a 00:07:43.568 --> 00:07:50.552 camera phone, and in some cases it just is a camera, which had an applicable app on 00:07:50.552 --> 00:07:57.661 it. But we'll see that... so how to tell whether they're using a phone or whether 00:07:57.661 --> 00:08:03.694 using as a biometric scanner? By the body language. So, for example, if the police 00:08:03.694 --> 00:08:08.941 officer is holding up the phone trying to capture someone's face, that is most 00:08:08.941 --> 00:08:13.873 likely because they have a capturing of photo and they may be connecting that to a 00:08:13.873 --> 00:08:21.820 facial recognition application. And you also will see mobile fingerprinting. So as 00:08:21.820 --> 00:08:26.444 an example, in the United Kingdom, they have an app on the officer's phone 00:08:26.444 --> 00:08:31.826 combined with a fingerprint scanning device and takes the people's fingerprints 00:08:31.826 --> 00:08:36.502 and checks them against some databases. One is a database of everyone that police 00:08:36.502 --> 00:08:40.459 have detained, putting them into the database and then checking against it for 00:08:40.459 --> 00:08:45.611 new people. And the other one is a database for immigration, collected at the 00:08:45.611 --> 00:08:51.754 border when someone comes into the UK. And this allows the police to do a very rapid 00:08:51.754 --> 00:08:58.790 check of their records on somebody in the field. Some of these devices are 00:08:58.790 --> 00:09:04.432 multimodal. We'll do both of them to be able to do fingerprints and take photos 00:09:04.432 --> 00:09:10.733 for facial recognition. This here is the Dataworks Plus Evolution - does it both, 00:09:10.733 --> 00:09:19.092 and that can be convenient for the officers, but it's a little bit more 00:09:19.092 --> 00:09:24.592 dangerous to civil liberties. And some of the Body-Worn cameras, in this example 00:09:24.592 --> 00:09:33.212 Wolfcam, has a biometric capability built in facial recognition so it can use its 00:09:33.212 --> 00:09:36.922 regular camera functions and of course, all of them to take the picture. That 00:09:36.922 --> 00:09:40.932 picture could be uploaded to a database and facial recognition will be done later. 00:09:40.932 --> 00:09:47.102 But this one is designed to streamline that process. So take a moment, as an 00:09:47.102 --> 00:09:52.211 aside, to talk about facial recognition in Europe per Algorithmwatch. The 00:09:52.211 --> 00:09:56.452 organization says that there are at least 11 police agencies in Europe who use 00:09:56.452 --> 00:10:02.664 facial recognition. I show them on the map here. The UK Court of Appeal found that 00:10:02.664 --> 00:10:07.634 automatic facial recognition technology used by the South Wales police was not 00:10:07.634 --> 00:10:13.783 lawful. However, elsewhere in the UK, they are still using it. The Metropolitan 00:10:13.783 --> 00:10:21.155 Police in London is doing a life facial recognition throughout the city of London, 00:10:21.155 --> 00:10:25.392 and it contends that the situation is distinguishable from South Wales. So that 00:10:25.392 --> 00:10:30.592 doesn't apply to them. We'll see how that turns out. There's also been some pressure 00:10:30.592 --> 00:10:35.785 on the European Commission to put a ban in place or put restrictions on facial 00:10:35.785 --> 00:10:42.611 recognition. And in September, there was a quote from the commissioners saying that 00:10:42.611 --> 00:10:45.488 they were considering whether we need additional safeguards or whether we need 00:10:45.488 --> 00:10:50.700 to go further and not allow facial recognition cases in certain areas or even 00:10:50.700 --> 00:10:54.582 temporarily, which is not a particularly strong statement. But at least they are 00:10:54.582 --> 00:10:59.899 considering the idea and something that one can advocate for the United States, a 00:10:59.899 --> 00:11:05.146 number of jurisdictions at the local level, cities have put restrictions on 00:11:05.146 --> 00:11:09.345 their police departments so they cannot use facial recognition. It's a growing 00:11:09.345 --> 00:11:15.317 movement. And while a national or international law that would limit police 00:11:15.317 --> 00:11:18.801 use of facial recognition would be best for civil liberties, you can also start at 00:11:18.801 --> 00:11:24.107 your local level. All right, what we wouldn't be on the police officers 00:11:24.107 --> 00:11:28.633 ourselves, whereelse - vehicles and roadways and this can come up for the 00:11:28.633 --> 00:11:32.930 vehicles, roadways adjacent to the protests and within the protests 00:11:32.930 --> 00:11:38.832 themselves. So adjacent to the protests is looking at the exits and entrances to the 00:11:38.832 --> 00:11:45.479 protest areas. And they may use existing ANPR or place new ANPR, or ALPR - 00:11:45.479 --> 00:11:50.450 automated numberplate reserve called ALPR in the United States. These are cameras. 00:11:50.450 --> 00:11:55.140 And they pointed towards a roadway to where cars will be. They are designed to 00:11:55.140 --> 00:12:01.601 take a picture, determine what the license plate / number plate is, optical character 00:12:01.601 --> 00:12:05.686 resolution. They will eventually... recognition that will eventually be able 00:12:05.686 --> 00:12:12.196 to see what it is, check the database and find out who registered for that car. And 00:12:12.196 --> 00:12:17.788 it can be uploaded to a central server for police to search, can add vehicles to a 00:12:17.788 --> 00:12:24.302 watch list. It is a very powerful tool because many people are using cars to get 00:12:24.302 --> 00:12:30.192 to and from protests. And even if they're going in in a group, at least one member 00:12:30.192 --> 00:12:36.576 of the group would have to have the car. And it has been used to go after someone 00:12:36.576 --> 00:12:41.464 after protest. So in this case, it was from 11 of years back. A citizen of the UK 00:12:41.464 --> 00:12:47.640 went to a protest and was later pulled over because they had captured the license 00:12:47.640 --> 00:12:55.736 plate while the protest added to database and then used that to pull them on. So if 00:12:55.736 --> 00:13:00.626 there is a protest, the police might come in and use a portable number plate reader. 00:13:00.626 --> 00:13:07.693 So here's some examples of what they might look like either on a tripod or on a 00:13:07.693 --> 00:13:12.758 trailer, and they can set these up basically anywhere. They would often be 00:13:12.758 --> 00:13:17.677 used at the entrance or exit to the zone in which the protesters expected, to see 00:13:17.677 --> 00:13:24.866 who's coming in, coming out during the protest time period and try to capture the 00:13:24.866 --> 00:13:31.753 crowd through their license plates. It also now becoming more and more common on 00:13:31.753 --> 00:13:37.391 police cars. You can see a couple of examples we have here. One - it shows a 00:13:37.391 --> 00:13:43.921 rather obvious in the top one, it is a UK police car. And see, the camera sticks 00:13:43.921 --> 00:13:50.083 out, fairly obvious that they have a camera on the light bar; the lower from 00:13:50.083 --> 00:13:54.604 the French police - less obvious, it looks like an ordinary light bar. You might be 00:13:54.604 --> 00:13:59.797 able to tell that it's a little bit different than some other ones, because it 00:13:59.797 --> 00:14:04.942 has sort of a funny thing in the center, but it's a pretty subtle approach. So 00:14:04.942 --> 00:14:08.613 there's all kinds, they might also be mounted on the hood of a trunk and may be 00:14:08.613 --> 00:14:14.474 more or less obvious. But take a look at... close and also take a look at what 00:14:14.474 --> 00:14:17.676 the police cars behavior. If they are driving, for example, slowly down the 00:14:17.676 --> 00:14:21.958 street next to a whole bunch of parked car, as it may be that they are doing 00:14:21.958 --> 00:14:25.054 "gridding", a practice known as "gridding", where they are looking for 00:14:25.054 --> 00:14:32.332 capturing every parked car's license plate in a particular zone. Slow and steady in 00:14:32.332 --> 00:14:39.282 order to do that. And then there are the fixed number plate readers, these are 00:14:39.282 --> 00:14:46.523 often at traffic lights and intersections on the highways, any sort of high speed 00:14:46.523 --> 00:14:52.159 toll road will have them. They also are... here, they're used for other purposes, 00:14:52.159 --> 00:15:00.317 like to establish fines, to check border crossings. They are very common fixtures 00:15:00.317 --> 00:15:06.113 on on roadways. So a protest happens in a zone that already has them. The police 00:15:06.113 --> 00:15:10.782 will be able to access that information and know who entered or exited that area 00:15:10.782 --> 00:15:19.994 to look around. All right, and then within the protest itself. There may be adding 00:15:19.994 --> 00:15:26.096 additional surveillance capacities, so in this example, we have a... the Santa Fe 00:15:26.096 --> 00:15:34.832 Police Department knew about a protest that was protesting a statue and some 00:15:34.832 --> 00:15:38.487 people would take action to remove the statue. So in order to capture that 00:15:38.487 --> 00:15:42.977 through surveillance, they placed this trailer, which has a number of camera and 00:15:42.977 --> 00:15:49.822 audio capabilities and just have rolled it in right next to the statue to capture the 00:15:49.822 --> 00:15:57.938 protest action. And these cameras can come in a variety of forms. In this case we got 00:15:57.938 --> 00:16:06.963 watch towers. Personal controling cameras can be in the watchtower or they can be 00:16:06.963 --> 00:16:12.265 operating remotely. As you can see, they are using a scissor jack to raise it about 00:16:12.265 --> 00:16:17.493 that van, the other one is an assembly. It's not easy for someone to get in and 00:16:17.493 --> 00:16:21.962 out of there. So it may have a person, but somehow inconvenient to actually have a 00:16:21.962 --> 00:16:26.554 person inside these watchtowers, but it's much more convenient to use their built-In 00:16:26.554 --> 00:16:31.839 surveillance capabilities and remotely observe the area around the watchtower 00:16:31.839 --> 00:16:38.139 with those cameras. And then there also pure surveillance units. This is an 00:16:38.139 --> 00:16:48.131 example here showing four cameras raised pole and just adding surveillance 00:16:48.131 --> 00:16:58.155 capability basically to anywhere. Some of them are much more complex: thermal 00:16:58.155 --> 00:17:04.890 imaging cameras. Thermal imaging often come from the leading company is FLIR, it 00:17:04.890 --> 00:17:09.121 stands for "Forward-looking Infrared". FLIR system makes a lot of these devices 00:17:09.121 --> 00:17:13.580 and makes them available to police departments. Thermal imaging cameras allow 00:17:13.580 --> 00:17:17.853 the police to be able to conduct surveillance after dark where the lighting 00:17:17.853 --> 00:17:23.217 is poor, where they might not be able to identify individuals very easily. Instead, 00:17:23.217 --> 00:17:28.877 they can use their heat signature and be able to continue to monitor the protest 00:17:28.877 --> 00:17:32.434 when the lighting conditions are less and a lot of things on protest will happen at 00:17:32.434 --> 00:17:37.785 night, candlelight vigils were commonplace. So police will be looking to 00:17:37.785 --> 00:17:42.108 thermal imaging to make sure that they have strong surveillance capabilities 00:17:42.108 --> 00:17:49.589 after dark. Another thing you might see at a protest is an emergency command vehicle. 00:17:49.589 --> 00:17:54.581 These are often massive bus sized vehicles and they do have some surveillance 00:17:54.581 --> 00:17:59.273 capabilities. They might have some cameras, but more often they are command 00:17:59.273 --> 00:18:03.101 and control. So they are the places where somebody would be receiving footage from 00:18:03.101 --> 00:18:10.817 cameras and operating cameras remotely, like a communications with another people 00:18:10.817 --> 00:18:16.075 in the field. They also may have some built in capabilities and they may provide 00:18:16.075 --> 00:18:20.130 the focal point, were a local connection, they're getting information from local 00:18:20.130 --> 00:18:27.002 devices and then they have the uplink in the command center. One thing I wanted to 00:18:27.002 --> 00:18:31.078 point out, it's a common misconception or something that comes up a lot where people 00:18:31.078 --> 00:18:35.710 are concerned about police surveillance, as we'll see an unmarked vehicle or a van 00:18:35.710 --> 00:18:41.920 with no windows. It may even have some antennas or satellite. And well that is 00:18:41.920 --> 00:18:46.676 possibly an undercover police vehicle, you shouldn't assume that vehicle belongs to 00:18:46.676 --> 00:18:53.229 law enforcement. That could very easily be a news media vehicle. News media also goes 00:18:53.229 --> 00:18:58.510 to protests. They also have satellite uplinks and antennas, look very similar. 00:18:58.510 --> 00:19:05.428 And in some cases, the media has a security situation there. They're worried 00:19:05.428 --> 00:19:11.871 that there may be theft of equipment and they have unmarked vans. So it is worth 00:19:11.871 --> 00:19:15.704 noting that there is an unmarked vehicle, but you shouldn't necessarily assume that 00:19:15.704 --> 00:19:21.015 it is a police unmarked vehicle. Also, sometimes people see especially they see 00:19:21.015 --> 00:19:26.452 some antennas or satellite dish vehicles that maybe that's were "stingrays". This 00:19:26.452 --> 00:19:33.304 is a misconception. Stingrays are pretty small and they don't require an external 00:19:33.304 --> 00:19:40.011 antenna. You could put a stingray inside a trunk of a car, maybe it will be a 00:19:40.011 --> 00:19:47.172 briefcase sized. So it would be unlikely that if you're going to use a stingray or 00:19:47.172 --> 00:19:52.259 similar, IMSI-catcher that you would want to put it in the vehicle. I don't 00:19:52.259 --> 00:19:58.569 need to put in a vehicle that has its own antenna. There have been not very much 00:19:58.569 --> 00:20:03.810 documentation of these technologies being used in the US domestic protests. They 00:20:03.810 --> 00:20:08.549 have been used, we know it, in some protests and more authoritarian countries. 00:20:08.549 --> 00:20:16.501 So it's unclear how often they will are being used and they are very dangerous in 00:20:16.501 --> 00:20:20.705 the surveillance. So It's aim is capturing, it is able to determine what 00:20:20.705 --> 00:20:25.610 cell phones are nearby, get a unique identifier with that cell phone, and in 00:20:25.610 --> 00:20:29.862 many cases be able to use that information to determine while individuals president 00:20:29.862 --> 00:20:35.577 protests. That information has been used after some protests in Ukraine, for 00:20:35.577 --> 00:20:40.293 example, to send a text message to people telling them: we're onto you, we know you 00:20:40.293 --> 00:20:45.276 were there, which can be very intimidating to individuals. But the challenge is that 00:20:45.276 --> 00:20:49.226 you're trying to observe police surveillance in the protest, it is hard 00:20:49.226 --> 00:20:54.103 for you to observe it because they are often hidden. You may be able to find out 00:20:54.103 --> 00:20:57.822 more information later through investigative journalism or public record 00:20:57.822 --> 00:21:02.442 requests, or news reports. If somebody is prosecuted using that information, it may 00:21:02.442 --> 00:21:09.405 become obvious. It is difficult to see it at the protest itself. So next category, 00:21:09.405 --> 00:21:16.086 look it up in the sky. There are lots of forms of aerial surveillance or for the 00:21:16.086 --> 00:21:21.394 agencies to surveil protests from above using traditional aircraft, on board 00:21:21.394 --> 00:21:27.412 pilots and as well as remotely operated aerial systems - drones. Law enforcement 00:21:27.412 --> 00:21:30.859 may also use these aerial vehicles to communicate with the crowd, to use the 00:21:30.859 --> 00:21:36.422 loudspeakers to send a message to the crowd. Order them to disperse. And we've 00:21:36.422 --> 00:21:40.194 seen this actually drones with loudspeakers being used by the German 00:21:40.194 --> 00:21:45.792 police in order to tell people to stay apart - Corona That same technology can be 00:21:45.792 --> 00:21:51.842 used for protests. And these planes and drones will often be equipped with high 00:21:51.842 --> 00:21:56.432 definition cameras, capable of either extremely wide angle to get the whole 00:21:56.432 --> 00:22:01.640 scene or an extreme zoom where they might be able to zoom in on a particular person, 00:22:01.640 --> 00:22:07.473 particular license plate, and then use that data later, partnering that the 00:22:07.473 --> 00:22:12.699 aircraft with license plate recognition, facial recognition, video analytics and 00:22:12.699 --> 00:22:21.149 even a cell-site simulator inside the aircraft. And we know this has happened in 00:22:21.149 --> 00:22:31.005 a recent protest in Texas. A Texas police drone got some footage of a protester 00:22:31.005 --> 00:22:36.272 allegedly throwing a water bottle. They took that video, they took the picture, 00:22:36.272 --> 00:22:41.404 put it out, offered a cash reward, an anonymous tip to turn the kid in and 00:22:41.404 --> 00:22:48.332 protester was prosecuted. So police are definitely using these things to gather 00:22:48.332 --> 00:22:55.118 information at protesters. So a common method for a larger police departments is 00:22:55.118 --> 00:22:59.852 fixed-wing aircraft; for smaller ones - they may use private contractors to 00:22:59.852 --> 00:23:05.162 provide these fixed-wing aircraft. So this is an example of the kind of plane you use 00:23:05.162 --> 00:23:08.912 by a company called Persistent Surveillance Systems. It rents out a plane 00:23:08.912 --> 00:23:12.392 like this, well not this exact one, you look up that tail number, it's going to be 00:23:12.392 --> 00:23:16.896 to a different company. But the same model of the Cessna 207-A. And these will circle 00:23:16.896 --> 00:23:23.342 around the protest using their cameras to observe protesters below. And the 00:23:23.342 --> 00:23:28.580 advantage of planes is they can often circle for quite a long time and provide a 00:23:28.580 --> 00:23:36.402 wide view on the area. Also, helicopters, helicopters will often be seen hovering 00:23:36.402 --> 00:23:41.102 over a protest, they are a little bit easier to maneuver and be able to go 00:23:41.102 --> 00:23:47.343 backwards and forwards over the protest that are used by police to continually 00:23:47.343 --> 00:23:52.270 observe. And we'll use two examples here. One of them from the Oakland Police 00:23:52.270 --> 00:23:56.280 Department, the other one from the Rineland Police Department. In both cases, 00:23:56.280 --> 00:24:01.735 they have a FLIR attached to the helicopter, a forward looking infrared 00:24:01.735 --> 00:24:06.362 that would allow them, in addition to regular camera capabilities, to use 00:24:06.362 --> 00:24:10.622 thermal imaging to follow someone at a protest or follow what's going on after 00:24:10.622 --> 00:24:18.252 dark. We can also see that some helicopters will have spotlights, so that 00:24:18.252 --> 00:24:24.081 they can signal to officers on the ground who to follow, who do pay attention to. 00:24:24.081 --> 00:24:31.292 Another thing for both fixed wing and helicopters: they look for the tail 00:24:31.292 --> 00:24:35.352 number. In most jurisdictions, they're required to have a tail number visible, 00:24:35.352 --> 00:24:40.232 and then you could look up that tail number on services like Flight Aware and 00:24:40.232 --> 00:24:44.362 be able to find out further information about what that plane has been doing, what 00:24:44.362 --> 00:24:51.737 the helicopter has been doing, as well as the ownership. Finally, drones. Drones are 00:24:51.737 --> 00:24:56.543 becoming very commonplace because they're getting cheaper all the time and having 00:24:56.543 --> 00:25:01.501 additional capacities; drones are also known as unmanned aerial vehicles - UAVs; 00:25:01.501 --> 00:25:08.694 UAS - unmanned aerial systems and a lot of police departments are getting them for 00:25:08.694 --> 00:25:14.552 their capabilities using what's called a quadrotor. And they can be controlled by 00:25:14.552 --> 00:25:20.795 remote control, have a camera built into this and be useful for getting above the 00:25:20.795 --> 00:25:27.472 scene view. So one way to spot it... well, first of all - just listen to it, they 00:25:27.472 --> 00:25:30.947 make kind of have a distinctive noise, sometimes are really marked as police. 00:25:30.947 --> 00:25:38.436 Also look for the pilots operating nearby. So oftentimes a patrol out there, 00:25:38.436 --> 00:25:42.222 sometimes they are labeled like in the upper left there, it says "police drone 00:25:42.222 --> 00:25:47.254 operator" - pretty easy to identify. All the times they might have like "drone", 00:25:47.254 --> 00:25:54.107 "UAV", "aviation unit" on their uniform, or a nearby police vehicle. The other 00:25:54.107 --> 00:25:59.574 thing is that if you identify a drone, there're often within line of sight is 00:25:59.574 --> 00:26:03.577 going to be the operator. So when do you see the drone look around and see, if 00:26:03.577 --> 00:26:07.800 someone has the controls in their hand, is looking up the drone, you can probably 00:26:07.800 --> 00:26:13.256 identify the operator and go look for information that might have uniform about 00:26:13.256 --> 00:26:19.310 who is operating that particular drone. But also, keep in mind, both for drones 00:26:19.310 --> 00:26:23.962 and other aircraft, that it's not necessarily the police. Journalists and 00:26:23.962 --> 00:26:29.629 activists will often fly drones over protest, news helicopters for a large 00:26:29.629 --> 00:26:34.629 protests are going to be more common than police helicopters. And many times they 00:26:34.629 --> 00:26:40.222 are labeled with picture of the BBC News Copter. But this means that just as you 00:26:40.222 --> 00:26:47.127 see a helicopter that has both a camera and is flying over the protest, that does 00:26:47.127 --> 00:26:53.984 not necessarily mean that it's a police helicopter. Also, the technology, which is 00:26:53.984 --> 00:27:00.383 actually not very commonplace outside of protests in war zones, but - the 00:27:00.383 --> 00:27:04.687 "Dronekiller" technology, which is basically a real gun that knocks drones 00:27:04.687 --> 00:27:11.321 out of the sky, sending radio signals to interfere with the drones operation and 00:27:11.321 --> 00:27:18.812 cause it to fall and crash. These have been used in Iraq and Afghanistan and the 00:27:18.812 --> 00:27:23.310 technology could be certainly used, but we really haven't seen it used more freely. 00:27:23.310 --> 00:27:27.792 Just wanted to tell you about it because... oh, my God, Dronekillers. All 00:27:27.792 --> 00:27:32.787 right, last place to look for police technology - in the environment around 00:27:32.787 --> 00:27:38.986 you. There will be in many places camera networks. So a lot of the cameras will see 00:27:38.986 --> 00:27:46.348 a neighborhood will be private cameras, will be police cameras, will be cameras 00:27:46.348 --> 00:27:52.678 being used by city, non police agencies. That can be a lot of cameras. This also 00:27:52.678 --> 00:27:56.095 means that you're trying to observe what cameras are going on. There's too much 00:27:56.095 --> 00:27:59.776 information. There'll be so many cameras in many areas that you could spend all 00:27:59.776 --> 00:28:03.746 your time documenting, observing the cameras and these other things. So you 00:28:03.746 --> 00:28:07.307 might not want to spend all your time paying attention to that, because you can 00:28:07.307 --> 00:28:11.544 go back later at any point and see the fixed cameras. But there are a couple of 00:28:11.544 --> 00:28:17.317 things that... first, identifying them for two different brands to identify. But 00:28:17.317 --> 00:28:21.858 here's some information about the kinds of cameras that are available. Bullet cameras 00:28:21.858 --> 00:28:25.542 are directional, so you can sort of see which way it's pointing and what it would 00:28:25.542 --> 00:28:30.258 be covering from that. Then you have dome cameras, which are designed, so you can't 00:28:30.258 --> 00:28:34.529 see which way it's pointing, or at least you can see maybe some of this area one 00:28:34.529 --> 00:28:38.874 hundred and eighty degrees. But the exact direction is pointing is obscured by the 00:28:38.874 --> 00:28:46.272 dome. Pan-Tilt-Zoom cameras can change which way they're pointing. They can 00:28:46.272 --> 00:28:51.893 sometimes be coupled with a dome camera so that the dome camera can both change the 00:28:51.893 --> 00:28:58.876 way it's looking and obscure which way its looking. Thermal imaging cameras and ALPR 00:28:58.876 --> 00:29:04.448 cameras are also becoming a fixed locations and we are all having to do with 00:29:04.448 --> 00:29:10.220 traffic control. Thermal is actually not as common. That is mostly used as a 00:29:10.220 --> 00:29:20.106 technology that is vehicles, is kind of expensive. But in this case, the picture 00:29:20.106 --> 00:29:25.324 shown is a thermal imaging camera. So sometimes people will go to that 00:29:25.324 --> 00:29:31.833 additional expense. And one subcategory of all the cameras that are environment are 00:29:31.833 --> 00:29:37.147 going to be police observation devices, so the category of sets of sensors which are 00:29:37.147 --> 00:29:42.804 operated by the police and they may included multiple cameras, gunshot 00:29:42.804 --> 00:29:49.015 detection, facial recognition. For example, in the United Kingdom, say in the 00:29:49.015 --> 00:29:53.498 city of London, is doing live facial recognition as a police observation 00:29:53.498 --> 00:29:59.271 devices are a collection of these cameras in one location. Sometimes they're marked 00:29:59.271 --> 00:30:07.856 as police, sometimes they are not. And there is the way you would suspect that 00:30:07.856 --> 00:30:12.796 it's a police observation device is - if it has a lot of different sensors in one 00:30:12.796 --> 00:30:17.760 location trying to cover the whole ground around, then that is the kind of thing you 00:30:17.760 --> 00:30:23.518 would see most frequently from a police office image. And then finally, Smart 00:30:23.518 --> 00:30:28.287 Street Lights. The Smart Street Lights have a number of wonderful applications. 00:30:28.287 --> 00:30:33.847 Some initiatives like in the US Smart Cities and the EU E-Street Initiative are 00:30:33.847 --> 00:30:38.085 exploiting cities to use more Smart Street Lights, because they could turn down the 00:30:38.085 --> 00:30:44.273 power usage when the light is less needed. And there's some advance towards a project 00:30:44.273 --> 00:30:50.035 by the Arnold University of Applied Sciences has a technology, which will use 00:30:50.035 --> 00:30:55.309 motion detection, sound detection. Being able to tell that there are people walking 00:30:55.309 --> 00:31:00.414 nearby and brighten their path. Sounds great. But the same kind of technology 00:31:00.414 --> 00:31:06.174 being able to detect motion, being able to have audio signals, video signals - can be 00:31:06.174 --> 00:31:10.391 used for surveillance. So here on the slide we show the smart lighting 00:31:10.391 --> 00:31:16.353 capabilities being advertised by Intel. In addition to some things that you might 00:31:16.353 --> 00:31:23.389 expect, like being able to adjusted for traffic patterns, provide when needed. 00:31:23.389 --> 00:31:26.013 They talk about crime investigations, monitor parking 00:31:26.013 --> 00:31:31.358 violations, safety announcements that are coming from the smart cameras. So all of 00:31:31.358 --> 00:31:36.904 these technologies are possible and hopefully this will not become a 00:31:36.904 --> 00:31:40.441 commonplace use. But if it is, it would mean that a surveillance device is 00:31:40.441 --> 00:31:45.296 everywhere along every street where they're putting these devices. Wich 00:31:45.296 --> 00:31:49.482 lightning the city you are lightning the city with surveillance. So has it been 00:31:49.482 --> 00:31:56.476 used? Yes. The city of San Diego had a number of protests surrounding the protest 00:31:56.476 --> 00:32:02.744 around George Floyd, and they used these thirty five times. They searched 00:32:02.744 --> 00:32:07.631 information gathered through the Smart Street Light Network for evidence in 00:32:07.631 --> 00:32:16.354 criminal cases coming out of that protest. So what additional resources are there? 00:32:16.354 --> 00:32:19.445 There's plenty of additional resources, if you wanted to try to read more - and I 00:32:19.445 --> 00:32:22.830 encourage you to take this only as a starting point for a lot more to learn. So 00:32:22.830 --> 00:32:25.856 we'll start out with a very important resource. If you're someone who's going to 00:32:25.856 --> 00:32:30.262 go, whether as an activist, as a protester, as a journalist, you should 00:32:30.262 --> 00:32:38.142 prepare yourself for some surveillance self defense: at ssd.eff.org we have an 00:32:38.142 --> 00:32:43.220 "Attending a Protest" guide. Go there and learn important tips on protecting 00:32:43.220 --> 00:32:48.202 yourself when you're going to protests. Put your device with fulldesk encryption: 00:32:48.202 --> 00:32:53.242 a strong unique password, turning off the biometric unlock, use end-to-end 00:32:53.242 --> 00:32:58.962 encryption for messages and calls, walking or taking a bicycle to get to the protest, 00:32:58.962 --> 00:33:04.485 sort of a vehicle, which couldn't be subjected to a ANPR / ALPR device. Wear a 00:33:04.485 --> 00:33:11.170 mask, you should wear a mask for Covid anyway. But if you wear a mask, get a big: 00:33:11.170 --> 00:33:16.530 the larger the mask, the more it protects you. There is also recently a study that 00:33:16.530 --> 00:33:24.798 showed that they're making efforts to try to make facial recognition continue on 00:33:24.798 --> 00:33:30.269 despite people's use of masks. And there was a study that showed that red and black 00:33:30.269 --> 00:33:37.733 masks were harder for the AI to be able to determine who was behind the mask. So wear 00:33:37.733 --> 00:33:42.879 a red or black mask. If you get one that covers more of your face like a bandana, 00:33:42.879 --> 00:33:47.020 it's going to be harder for the facial recognition algorithms. So do some things 00:33:47.020 --> 00:33:53.066 to protect yourself, both from Covid and from surveillance. If you want to also 00:33:53.066 --> 00:33:59.328 just practice it out, you can go to our Spot the Surveillance. This is an online 00:33:59.328 --> 00:34:05.792 program, you can do the desktop version or virtual reality version, where it places 00:34:05.792 --> 00:34:11.424 you in a virtual street corner with some surveillance devices nearby. And you can 00:34:11.424 --> 00:34:15.648 look around and try to identify all of the surveillance devices that you see. It 00:34:15.648 --> 00:34:21.302 takes just a few minutes to go through the exercise, but it's a good way to practice 00:34:21.302 --> 00:34:27.950 your skills and identify what surveillance might be around on the street. And if you 00:34:27.950 --> 00:34:34.376 want to get a lot more information about any of these devices, go to EFF Street- 00:34:34.376 --> 00:34:40.219 Level Surveillance project: eff.org/sls - street level surveillance. This will 00:34:40.219 --> 00:34:46.076 provide more detailed information about various technologies that are in use. That 00:34:46.076 --> 00:34:50.396 can be a good starting point, especially if you found out what is being used in 00:34:50.396 --> 00:34:55.912 your jurisdiction. Go there and find out more about it. And you can also find out 00:34:55.912 --> 00:35:02.316 just what is going on more generally with these kinds of technologies: eff.org/sls. 00:35:02.316 --> 00:35:06.858 All right. Well, thank you. That comes to the close of my talk. Thank you for 00:35:06.858 --> 00:35:19.175 turning in. And now let me turn it over to my future self for Q&A. Thanks you. 00:35:19.175 --> 00:35:29.022 Herald: Welcome back. Thanks so much Kurt. Now we have some time for questions and 00:35:29.022 --> 00:35:34.383 it's getting more and more. I'm just hurrying up. OK, are there devices, apps 00:35:34.383 --> 00:35:39.057 or services developed or run by private companies and who makes sure the data is 00:35:39.057 --> 00:35:46.052 not directly sold to third parties? Kurt: So, yes, there are private networks. 00:35:46.052 --> 00:35:49.899 And one of the things we talked about just now is there's a lot of private camera 00:35:49.899 --> 00:35:54.341 networks that are providing information to the police, sometimes private networks 00:35:54.341 --> 00:35:59.157 going through a registry where police from organizers ask people to volunteer, put 00:35:59.157 --> 00:36:02.733 their information into a registry. So they are sort of explicitly saying they're 00:36:02.733 --> 00:36:06.589 going to turn over their information to the police or other things like Amazon's 00:36:06.589 --> 00:36:11.702 ring camera. They have been promoting it as a antitheft tool, trying to stop 00:36:11.702 --> 00:36:16.461 package stuff stolen from doors. But this also is creating a panopticon of 00:36:16.461 --> 00:36:22.722 everyone's doorbell camera. If they're all using ring will be provided... get the 00:36:22.722 --> 00:36:28.262 video and will provide it to the police. So and many of these organizations, you 00:36:28.262 --> 00:36:32.057 know, if they're larger than they will have some some privacy practices, probably 00:36:32.057 --> 00:36:37.326 policies. But by and large, they will talk about the privacy of the person who owns 00:36:37.326 --> 00:36:43.585 the marketplace and not really consider the bystanders, the people walking by. So 00:36:43.585 --> 00:36:48.402 if you have a doorbell camera at your front door that can hear audio, so maybe 00:36:48.402 --> 00:36:52.262 someone could ring your bell and say hello. It will also capture people walking 00:36:52.262 --> 00:37:00.077 by and those people's privacy is important to be considered. 00:37:00.077 --> 00:37:06.389 Herald: All right, then we have: what hope do we have against all this, which best 00:37:06.389 --> 00:37:11.332 case legal countermeasures do we have by been attending protests? And the another 00:37:11.332 --> 00:37:16.758 one, which I would connect directly: is it possible to intervene against surveillance 00:37:16.758 --> 00:37:21.440 based on laws or presumption of innocence? Kurt: Yes. 00:37:21.440 --> 00:37:26.064 Herald: I don't know if German laws are meant, but maybe you still can say 00:37:26.064 --> 00:37:28.062 something? Kurt: Well, I mean, so there's many 00:37:28.062 --> 00:37:32.582 different laws that might be an issue. I mean, we have an international audience 00:37:32.582 --> 00:37:35.924 here, but I think there are some also basic human rights principles that apply 00:37:35.924 --> 00:37:41.014 to many jurisdictions. But I would say actually one of the most effective tools 00:37:41.014 --> 00:37:44.883 to push back against this kind of police surveillance is working locally with the, 00:37:44.883 --> 00:37:52.266 like, a city, the mayor, the city council and a number of locations have passed 00:37:52.266 --> 00:37:56.183 rules about what their police can do against their citizens. So putting 00:37:56.183 --> 00:38:01.452 limitations on what police can do at the local level, where your activities in the 00:38:01.452 --> 00:38:06.552 city, which you live, taking things to your representative government and saying 00:38:06.552 --> 00:38:10.974 we need to have some limitations on this, we need to have it within civilian 00:38:10.974 --> 00:38:14.875 controls for the police themselves are not deciding what technologies to use, but it 00:38:14.875 --> 00:38:17.762 has to pass through an elected representative. And I think that is 00:38:17.762 --> 00:38:23.332 probably one of the most effective ways to at least start change where you live. But 00:38:23.332 --> 00:38:27.462 you can also try and promote that to your national legislature, state legislatures, 00:38:27.462 --> 00:38:32.673 go up several levels. And one of the things that I hope comes out of this 00:38:32.673 --> 00:38:34.902 guide, where people getting more information about what kind of 00:38:34.902 --> 00:38:38.573 surveillance is available, so that they can go to their representatives, go 00:38:38.573 --> 00:38:42.778 through the political process with the information of what tool use something 00:38:42.778 --> 00:38:48.212 that they have drawn a line about. Go to your representative and say we need to 00:38:48.212 --> 00:38:52.166 make sure that the information that they're gathered is being used in a manner 00:38:52.166 --> 00:38:56.299 consistent with human rights principles. And we need civilian control from the 00:38:56.299 --> 00:39:00.552 local government on how to do. Herald: Who's controlling the controler 00:39:00.552 --> 00:39:13.972 instances, yes. We have more questions. OK, so the police operate equipment like 00:39:13.972 --> 00:39:17.682 ANPR-reader IMSI-Catcher etc. , that get information that they could get in a 00:39:17.682 --> 00:39:26.022 cheaper way, like reading traffic signs or license plates, or cell info from 00:39:26.022 --> 00:39:33.712 operators. And is there a reason for that? Especially concerning EU, because US 00:39:33.712 --> 00:39:42.282 differs a lot. And another question: has police in EU, US been known to use illegal 00:39:42.282 --> 00:39:49.801 or questionable tech for surveillance? Kurt: So I think on the first question 00:39:49.801 --> 00:39:57.190 about using things like ANPR to determine license plates, this technology is common 00:39:57.190 --> 00:40:03.212 in the European Union, though by and large it is being put in place for other 00:40:03.212 --> 00:40:08.370 reasons, not to get the protesters necessarily. They are looking for, you 00:40:08.370 --> 00:40:12.409 know, making sure that people are paying a toll or might be a speed trap on the 00:40:12.409 --> 00:40:17.431 autobahn, where it takes a picture of the license plate of anyone traveling over a 00:40:17.431 --> 00:40:22.654 speed limit - in the places that have speed limits on the part of the autobahn. 00:40:22.654 --> 00:40:28.773 And I think also it's being used for enforcement of things like traffic 00:40:28.773 --> 00:40:33.916 citations: your cars parked on location too long. They know who to send the bill 00:40:33.916 --> 00:40:41.123 to. And I think these technologies could be repurposed for surveillance. And that's 00:40:41.123 --> 00:40:45.217 what we really need, is policies that are ensuring that if these things are being 00:40:45.217 --> 00:40:49.739 used for a purpose, that the sort of the citizenship agrees with in that 00:40:49.739 --> 00:40:54.100 jurisdiction to enforce parking, for example, that is not also being repurposed 00:40:54.100 --> 00:40:59.288 against political activities and being used at a wider scale than it was 00:40:59.288 --> 00:41:03.814 envisioned. Also, maybe, you know, it's not a good thing to have perfect parking 00:41:03.814 --> 00:41:08.293 enforcement, you know, a lot of parking fines were based on the notion that, like, 00:41:08.293 --> 00:41:12.646 you might not get caught every time and when you change it, a system where 00:41:12.646 --> 00:41:17.395 previously the fines were set with the notion that a lot of people would get away 00:41:17.395 --> 00:41:20.720 with it.. to I do, like, to make an example of those who didn't. And then you 00:41:20.720 --> 00:41:24.600 changed that to perfect enforcement because the computer, the ANPR system, 00:41:24.600 --> 00:41:30.256 surveillance knows exactly the minute that a fine is due and then assesses that fine. 00:41:30.256 --> 00:41:33.932 That actually changes the dynamic of power between citizenship and the state 00:41:33.932 --> 00:41:38.630 significantly. And it will all be freeze and forms of well, we're just trying to 00:41:38.630 --> 00:41:42.377 enforce the existing laws. How could you be against that? But really, it changes 00:41:42.377 --> 00:41:45.885 the dynamic. And it's something that for those who want to be an activist on this, 00:41:45.885 --> 00:41:49.851 again, talk to your local jurisdiction and try and make sure that these things have 00:41:49.851 --> 00:41:52.794 safe and sane policies that respect human rights. 00:41:52.794 --> 00:42:01.147 Herald: So I would interpret that like prevention of... don't come on the idea 00:42:01.147 --> 00:42:05.635 that you need to to protect your data, right? Fun. 00:42:05.635 --> 00:42:09.650 Kurt: And just turning to the another one. Do we have information about whether 00:42:09.650 --> 00:42:14.618 police are misusing these technologies? So, I mean, there's some isolated examples 00:42:14.618 --> 00:42:19.460 where people have misused their technologies. And I use a couple of them 00:42:19.460 --> 00:42:23.331 in the slides. And there was someone who went to a political protest. Their car was 00:42:23.331 --> 00:42:29.565 put into a database to get pulled over later. And then also in South Wales, the 00:42:29.565 --> 00:42:36.839 court found that there the police use of facial recognition was in violation of UK 00:42:36.839 --> 00:42:42.027 law, though, as I noted, not the Metropolitan Police in London don't agree 00:42:42.027 --> 00:42:45.446 with that. They say it doesn't apply to them. And I think actually use of facial 00:42:45.446 --> 00:42:49.851 recognition technologies is a very tempting thing by police. They want to use 00:42:49.851 --> 00:42:56.297 it as much as possible, make it easy for them. And I think you will see that. But 00:42:56.297 --> 00:43:01.312 the other piece of this is unless there are rules that say here are limitations on 00:43:01.312 --> 00:43:05.507 how you can use these technologies and they can use them without having to risk 00:43:05.507 --> 00:43:11.432 violating them. So we need to have those rules in place. I hope that the Council of 00:43:11.432 --> 00:43:17.271 Europe puts at least a moratorium on facial recognition for use for police. And 00:43:17.271 --> 00:43:22.844 until we can figure out how to use this technology safely, it's kind of cool that 00:43:22.844 --> 00:43:25.680 you can unlock your phone in your face without having to type in a password, 00:43:25.680 --> 00:43:30.092 but... we want to make sure that technology is used properly. 00:43:30.092 --> 00:43:35.242 Herald: OK, so I think you're going to be around in the 2D world. You're going to 00:43:35.242 --> 00:43:41.252 explore that, you've told me before. Yeah, there's more questions. I hope maybe you 00:43:41.252 --> 00:43:47.462 find him in the 2D world and you just ask him there. Thanks so much. Thanks so much. 00:43:47.462 --> 00:43:52.698 Very nice having you. Bye Kurt. Kurt: Bye. 00:43:52.698 --> 00:43:52.754 rC3 postroll music 00:43:52.754 --> 00:44:36.012 Subtitles created by c3subtitles.de in the year 2021. Join, and help us!