0:00:05.012,0:00:17.227
rC3 preroll music
0:00:17.227,0:00:24.052
Herald: Yeah, and the next talk I'm very[br]proud to announce, we have a speaker who's
0:00:24.932,0:00:31.332
coming in from sunny California and he's[br]an attorney, he's working for Harvard.
0:00:31.332,0:00:36.932
He's doing so many things and he's[br]fighting for our digital rights. Very
0:00:36.932,0:00:41.892
happy to say hi. Welcome.[br]Kurt Opsahl: Thank you!
0:00:46.612,0:00:58.212
Herald: And "Spot the Surveillance" is the[br]topic. We'll see what we haven't seen
0:00:58.212,0:01:04.908
before. And I'm very happy that you're[br]here and... Kurt Opsahl. Please, let us
0:01:04.908,0:01:09.492
know what's up. Thank you.
0:01:09.492,0:01:11.880
Kurt: Thank you. Hello, everybody, my name[br]is Kurt, also I'm the deputy executive
0:01:11.880,0:01:16.767
director general counsel of the Electronic[br]Frontier Foundation. I'm here to talk to
0:01:16.767,0:01:21.534
you about observing police surveillance of[br]protests. So why do we want to observe
0:01:21.534,0:01:26.780
police at protests? Well, because protests[br]are political expression. And the Council
0:01:26.780,0:01:31.023
of Europe put it the right of individuals[br]to gather with other people and make their
0:01:31.023,0:01:35.175
collective voice heard is fundamental to a[br]properly functioning democracy. And this
0:01:35.175,0:01:38.522
is a right which is protected by the[br]European Convention on Human Rights and
0:01:38.522,0:01:43.937
other international rights treaties. But[br]surveillance - consumer's rights. So
0:01:43.937,0:01:48.377
knowing what technologies are used can[br]help you understand the threats to your
0:01:48.377,0:01:52.808
privacy and security, as well as provide[br]tools to advocate for limits on police use
0:01:52.808,0:01:55.941
of surveillance; surveillance that may[br]chill people's right to express themselves
0:01:55.941,0:02:00.532
on these public issues. Just as an analog[br]surveillance historically has been used as
0:02:00.532,0:02:04.512
a tool of repression. Nowadays,[br]policymakers and the public have to
0:02:04.512,0:02:10.471
understand that the threat posed by[br]emerging technologies is a danger to human
0:02:10.471,0:02:14.149
rights, and they need to understand that[br]to successfully defend human rights in the
0:02:14.149,0:02:18.989
digital age. So journalists who are[br]reporting on protest action should know
0:02:18.989,0:02:24.171
the surveillance that is in use. Activists[br]who are advocating for limitations on
0:02:24.171,0:02:27.991
police use of surveillance need to know[br]what surveillance is being used to
0:02:27.991,0:02:31.947
effectively advocate. And legal observers[br]may need to document the use of
0:02:31.947,0:02:35.764
surveillance at protests in order to[br]challenge the police actions at the
0:02:35.764,0:02:39.412
protest or challenge the police policies[br]that are being used after the protest with
0:02:39.412,0:02:44.047
the footage they've obtained. So where are[br]we going today? We're going to provide a
0:02:44.047,0:02:48.513
lot of information about various types of[br]surveillance technologies in use by police
0:02:48.513,0:02:55.211
around the world. We're going to look at[br]what the appearance is, how it works, what
0:02:55.211,0:03:00.279
kind of data they collect and how they're[br]used by police. And at the end a few other
0:03:00.279,0:03:05.057
resources available for those who want to[br]dive a little bit deeper on the topic. So
0:03:05.057,0:03:09.966
police surveillance technology is[br]everywhere. It's on the police themselves,
0:03:09.966,0:03:16.997
on their vehicles, on the roadways. It[br]could be above you in the air, surrounding
0:03:16.997,0:03:21.632
you in the environment, can be a lot of[br]different places and you need to know
0:03:21.632,0:03:26.432
where. The police officers themselves[br]often find in the form of either body worn
0:03:26.432,0:03:30.515
cameras or additional devices that they're[br]using, which are basically mobile
0:03:30.515,0:03:37.186
biometric sensors. Body-Worn cameras or[br]technology that's come out become more
0:03:37.186,0:03:40.929
popular over the last decade or so.[br]Originally was something that was being
0:03:40.929,0:03:46.973
used as a way to provide police[br]accountability that give a record of their
0:03:46.973,0:03:50.628
interactions with the public. And maybe,[br]for example, could you show police
0:03:50.628,0:03:54.554
brutality or maybe deter police brutality?[br]But there are two-way streets: these are
0:03:54.554,0:03:59.423
often used to surveil protesters, and the[br]footage may later be used to support
0:03:59.423,0:04:09.085
arrests and charges. For example, we use[br]this NPR story where after a rally weeks
0:04:09.085,0:04:15.929
later, the police went identifying people[br]through body cam footage and brought
0:04:15.929,0:04:20.296
action against them for obstructing the[br]roadway, part of the civil disobedience of
0:04:20.296,0:04:27.587
the protest, based on finding them on the[br]body camera footage. They can be in a all
0:04:27.587,0:04:32.523
variety of places, so if you're looking[br]for Body-Worn cameras, you got to look at
0:04:32.523,0:04:37.214
different places to see where they might[br]be. So a couple of places they might see
0:04:37.214,0:04:40.707
them: on the head - head mounted camera.[br]So it might be on the glasses on the side,
0:04:40.707,0:04:47.240
could be a lens right in the center. The[br]center one is pretty hard to find, but the
0:04:47.240,0:04:52.075
ones on the side or might be part of the[br]glasses or the helmet they're wearing are
0:04:52.075,0:04:56.363
generally pretty obvious. These ones,[br]they're not particularly common, but they
0:04:56.363,0:05:03.737
do happen. Shoulder mounted cameras also a[br]little bit less common, but they have an
0:05:03.737,0:05:07.760
interesting feature: in this case they're[br]using the Warrior 360 from Blue Line
0:05:07.760,0:05:14.993
Innovations as an example, and it is a[br]dome camera that looks all directions, 360
0:05:14.993,0:05:20.515
degrees off the officer's left shoulder.[br]All most cameras like a front facing
0:05:20.515,0:05:28.254
camera will capture only 180 degrees.[br]Chest mounted cameras are the most common
0:05:28.254,0:05:34.579
and these are being used very, very[br]widely. We give some examples here from
0:05:34.579,0:05:44.173
Amsterdam, Middelburg and from West[br]Midlands Police in the EU... or soon to be
0:05:44.173,0:05:50.134
not in EU for I guess Britain. And there[br]are several known types: Excelon, Wolfcom
0:05:50.134,0:05:55.610
and Watchguard are very common. They[br]operate in similar manners, though, with
0:05:55.610,0:06:03.097
some differences. And you can take a look[br]at some of the examples that are available
0:06:03.097,0:06:06.632
to those companies webpages where they[br]will explain the products they have an
0:06:06.632,0:06:10.883
offer and see what matches up for your[br]jurisdiction. Or you can also look in for
0:06:10.883,0:06:14.972
news article. Oftentimes there's a news[br]article about what may the first policy to
0:06:14.972,0:06:20.866
bring Body_Worn cameras is introduced in a[br]particular police department. The're also
0:06:20.866,0:06:25.582
smartphone based cameras, and these are[br]kind of the low end, it's basically just
0:06:25.582,0:06:30.161
an Android cell phone using its internal[br]camera with an app that does recording
0:06:30.161,0:06:35.595
placed in a pocket. So the camera is a[br]little bit above the clock and you can see
0:06:35.595,0:06:39.946
forward. But it's also a very subtle[br]technique. And it could be easily confused
0:06:39.946,0:06:44.024
if you were looking for someone just[br]starting their phone to their pocket. It
0:06:44.024,0:06:47.952
also might be clipped somewhere on the on[br]their uniform. But if you see anything
0:06:47.952,0:06:52.546
where the camera is facing outward and[br]it's attached to the officer, there's a
0:06:52.546,0:06:57.757
good chance that that is a Body-Worn[br]camera and the setup is on play. Last of
0:06:57.757,0:07:01.482
the Body-Worn cameras will talk about the[br]semi-obscured cams. So this is an example
0:07:01.482,0:07:06.987
for coverable Body-Worn, [br]Body-Worn Utility, and it is
0:07:06.987,0:07:14.163
partially concealed, basically looks like[br]a button on someone's uniform that if
0:07:14.163,0:07:18.514
you're not looking closely, you might not[br]notice. But if you see, if you know, it
0:07:18.514,0:07:23.091
appears where you would expect actually[br]not to have a button, it's slightly
0:07:23.091,0:07:25.858
larger, looks a little bit different. It[br]looks like a camera if you look closely.
0:07:25.858,0:07:28.806
But if you're looking at a distance,[br]you're not particularly paying attention.
0:07:28.806,0:07:34.835
You might not see it at all. In addition[br]to Body-Worn cameras, there are often used
0:07:34.835,0:07:43.568
mobile biometric devices, so these can be[br]handheld scanners, could be a tablet or a
0:07:43.568,0:07:50.552
camera phone, and in some cases it just is[br]a camera, which had an applicable app on
0:07:50.552,0:07:57.661
it. But we'll see that... so how to tell[br]whether they're using a phone or whether
0:07:57.661,0:08:03.694
using as a biometric scanner? By the body[br]language. So, for example, if the police
0:08:03.694,0:08:08.941
officer is holding up the phone trying to[br]capture someone's face, that is most
0:08:08.941,0:08:13.873
likely because they have a capturing of[br]photo and they may be connecting that to a
0:08:13.873,0:08:21.820
facial recognition application. And you[br]also will see mobile fingerprinting. So as
0:08:21.820,0:08:26.444
an example, in the United Kingdom, they[br]have an app on the officer's phone
0:08:26.444,0:08:31.826
combined with a fingerprint scanning[br]device and takes the people's fingerprints
0:08:31.826,0:08:36.502
and checks them against some databases.[br]One is a database of everyone that police
0:08:36.502,0:08:40.459
have detained, putting them into the[br]database and then checking against it for
0:08:40.459,0:08:45.611
new people. And the other one is a[br]database for immigration, collected at the
0:08:45.611,0:08:51.754
border when someone comes into the UK. And[br]this allows the police to do a very rapid
0:08:51.754,0:08:58.790
check of their records on somebody in the[br]field. Some of these devices are
0:08:58.790,0:09:04.432
multimodal. We'll do both of them to be[br]able to do fingerprints and take photos
0:09:04.432,0:09:10.733
for facial recognition. This here is the[br]Dataworks Plus Evolution - does it both,
0:09:10.733,0:09:19.092
and that can be convenient for the[br]officers, but it's a little bit more
0:09:19.092,0:09:24.592
dangerous to civil liberties. And some of[br]the Body-Worn cameras, in this example
0:09:24.592,0:09:33.212
Wolfcam, has a biometric capability built[br]in facial recognition so it can use its
0:09:33.212,0:09:36.922
regular camera functions and of course,[br]all of them to take the picture. That
0:09:36.922,0:09:40.932
picture could be uploaded to a database[br]and facial recognition will be done later.
0:09:40.932,0:09:47.102
But this one is designed to streamline[br]that process. So take a moment, as an
0:09:47.102,0:09:52.211
aside, to talk about facial recognition in[br]Europe per Algorithmwatch. The
0:09:52.211,0:09:56.452
organization says that there are at least[br]11 police agencies in Europe who use
0:09:56.452,0:10:02.664
facial recognition. I show them on the map[br]here. The UK Court of Appeal found that
0:10:02.664,0:10:07.634
automatic facial recognition technology[br]used by the South Wales police was not
0:10:07.634,0:10:13.783
lawful. However, elsewhere in the UK, they[br]are still using it. The Metropolitan
0:10:13.783,0:10:21.155
Police in London is doing a life facial[br]recognition throughout the city of London,
0:10:21.155,0:10:25.392
and it contends that the situation is[br]distinguishable from South Wales. So that
0:10:25.392,0:10:30.592
doesn't apply to them. We'll see how that[br]turns out. There's also been some pressure
0:10:30.592,0:10:35.785
on the European Commission to put a ban in[br]place or put restrictions on facial
0:10:35.785,0:10:42.611
recognition. And in September, there was a[br]quote from the commissioners saying that
0:10:42.611,0:10:45.488
they were considering whether we need[br]additional safeguards or whether we need
0:10:45.488,0:10:50.700
to go further and not allow facial[br]recognition cases in certain areas or even
0:10:50.700,0:10:54.582
temporarily, which is not a particularly[br]strong statement. But at least they are
0:10:54.582,0:10:59.899
considering the idea and something that[br]one can advocate for the United States, a
0:10:59.899,0:11:05.146
number of jurisdictions at the local[br]level, cities have put restrictions on
0:11:05.146,0:11:09.345
their police departments so they cannot[br]use facial recognition. It's a growing
0:11:09.345,0:11:15.317
movement. And while a national or[br]international law that would limit police
0:11:15.317,0:11:18.801
use of facial recognition would be best[br]for civil liberties, you can also start at
0:11:18.801,0:11:24.107
your local level. All right, what we[br]wouldn't be on the police officers
0:11:24.107,0:11:28.633
ourselves, whereelse - vehicles and[br]roadways and this can come up for the
0:11:28.633,0:11:32.930
vehicles, roadways adjacent to the[br]protests and within the protests
0:11:32.930,0:11:38.832
themselves. So adjacent to the protests is[br]looking at the exits and entrances to the
0:11:38.832,0:11:45.479
protest areas. And they may use existing[br]ANPR or place new ANPR, or ALPR -
0:11:45.479,0:11:50.450
automated numberplate reserve called ALPR[br]in the United States. These are cameras.
0:11:50.450,0:11:55.140
And they pointed towards a roadway to[br]where cars will be. They are designed to
0:11:55.140,0:12:01.601
take a picture, determine what the license[br]plate / number plate is, optical character
0:12:01.601,0:12:05.686
resolution. They will eventually...[br]recognition that will eventually be able
0:12:05.686,0:12:12.196
to see what it is, check the database and[br]find out who registered for that car. And
0:12:12.196,0:12:17.788
it can be uploaded to a central server for[br]police to search, can add vehicles to a
0:12:17.788,0:12:24.302
watch list. It is a very powerful tool[br]because many people are using cars to get
0:12:24.302,0:12:30.192
to and from protests. And even if they're[br]going in in a group, at least one member
0:12:30.192,0:12:36.576
of the group would have to have the car.[br]And it has been used to go after someone
0:12:36.576,0:12:41.464
after protest. So in this case, it was[br]from 11 of years back. A citizen of the UK
0:12:41.464,0:12:47.640
went to a protest and was later pulled[br]over because they had captured the license
0:12:47.640,0:12:55.736
plate while the protest added to database[br]and then used that to pull them on. So if
0:12:55.736,0:13:00.626
there is a protest, the police might come[br]in and use a portable number plate reader.
0:13:00.626,0:13:07.693
So here's some examples of what they might[br]look like either on a tripod or on a
0:13:07.693,0:13:12.758
trailer, and they can set these up[br]basically anywhere. They would often be
0:13:12.758,0:13:17.677
used at the entrance or exit to the zone[br]in which the protesters expected, to see
0:13:17.677,0:13:24.866
who's coming in, coming out during the[br]protest time period and try to capture the
0:13:24.866,0:13:31.753
crowd through their license plates. It[br]also now becoming more and more common on
0:13:31.753,0:13:37.391
police cars. You can see a couple of[br]examples we have here. One - it shows a
0:13:37.391,0:13:43.921
rather obvious in the top one, it is a UK[br]police car. And see, the camera sticks
0:13:43.921,0:13:50.083
out, fairly obvious that they have a[br]camera on the light bar; the lower from
0:13:50.083,0:13:54.604
the French police - less obvious, it looks[br]like an ordinary light bar. You might be
0:13:54.604,0:13:59.797
able to tell that it's a little bit[br]different than some other ones, because it
0:13:59.797,0:14:04.942
has sort of a funny thing in the center,[br]but it's a pretty subtle approach. So
0:14:04.942,0:14:08.613
there's all kinds, they might also be[br]mounted on the hood of a trunk and may be
0:14:08.613,0:14:14.474
more or less obvious. But take a look[br]at... close and also take a look at what
0:14:14.474,0:14:17.676
the police cars behavior. If they are[br]driving, for example, slowly down the
0:14:17.676,0:14:21.958
street next to a whole bunch of parked[br]car, as it may be that they are doing
0:14:21.958,0:14:25.054
"gridding", a practice known as[br]"gridding", where they are looking for
0:14:25.054,0:14:32.332
capturing every parked car's license plate[br]in a particular zone. Slow and steady in
0:14:32.332,0:14:39.282
order to do that. And then there are the[br]fixed number plate readers, these are
0:14:39.282,0:14:46.523
often at traffic lights and intersections[br]on the highways, any sort of high speed
0:14:46.523,0:14:52.159
toll road will have them. They also are...[br]here, they're used for other purposes,
0:14:52.159,0:15:00.317
like to establish fines, to check border[br]crossings. They are very common fixtures
0:15:00.317,0:15:06.113
on on roadways. So a protest happens in a[br]zone that already has them. The police
0:15:06.113,0:15:10.782
will be able to access that information[br]and know who entered or exited that area
0:15:10.782,0:15:19.994
to look around. All right, and then within[br]the protest itself. There may be adding
0:15:19.994,0:15:26.096
additional surveillance capacities, so in[br]this example, we have a... the Santa Fe
0:15:26.096,0:15:34.832
Police Department knew about a protest[br]that was protesting a statue and some
0:15:34.832,0:15:38.487
people would take action to remove the[br]statue. So in order to capture that
0:15:38.487,0:15:42.977
through surveillance, they placed this[br]trailer, which has a number of camera and
0:15:42.977,0:15:49.822
audio capabilities and just have rolled it[br]in right next to the statue to capture the
0:15:49.822,0:15:57.938
protest action. And these cameras can come[br]in a variety of forms. In this case we got
0:15:57.938,0:16:06.963
watch towers. Personal controling cameras[br]can be in the watchtower or they can be
0:16:06.963,0:16:12.265
operating remotely. As you can see, they[br]are using a scissor jack to raise it about
0:16:12.265,0:16:17.493
that van, the other one is an assembly.[br]It's not easy for someone to get in and
0:16:17.493,0:16:21.962
out of there. So it may have a person, but[br]somehow inconvenient to actually have a
0:16:21.962,0:16:26.554
person inside these watchtowers, but it's[br]much more convenient to use their built-In
0:16:26.554,0:16:31.839
surveillance capabilities and remotely[br]observe the area around the watchtower
0:16:31.839,0:16:38.139
with those cameras. And then there also[br]pure surveillance units. This is an
0:16:38.139,0:16:48.131
example here showing four cameras raised[br]pole and just adding surveillance
0:16:48.131,0:16:58.155
capability basically to anywhere. Some of[br]them are much more complex: thermal
0:16:58.155,0:17:04.890
imaging cameras. Thermal imaging often[br]come from the leading company is FLIR, it
0:17:04.890,0:17:09.121
stands for "Forward-looking Infrared".[br]FLIR system makes a lot of these devices
0:17:09.121,0:17:13.580
and makes them available to police[br]departments. Thermal imaging cameras allow
0:17:13.580,0:17:17.853
the police to be able to conduct[br]surveillance after dark where the lighting
0:17:17.853,0:17:23.217
is poor, where they might not be able to[br]identify individuals very easily. Instead,
0:17:23.217,0:17:28.877
they can use their heat signature and be[br]able to continue to monitor the protest
0:17:28.877,0:17:32.434
when the lighting conditions are less and[br]a lot of things on protest will happen at
0:17:32.434,0:17:37.785
night, candlelight vigils were[br]commonplace. So police will be looking to
0:17:37.785,0:17:42.108
thermal imaging to make sure that they[br]have strong surveillance capabilities
0:17:42.108,0:17:49.589
after dark. Another thing you might see at[br]a protest is an emergency command vehicle.
0:17:49.589,0:17:54.581
These are often massive bus sized vehicles[br]and they do have some surveillance
0:17:54.581,0:17:59.273
capabilities. They might have some[br]cameras, but more often they are command
0:17:59.273,0:18:03.101
and control. So they are the places where[br]somebody would be receiving footage from
0:18:03.101,0:18:10.817
cameras and operating cameras remotely,[br]like a communications with another people
0:18:10.817,0:18:16.075
in the field. They also may have some[br]built in capabilities and they may provide
0:18:16.075,0:18:20.130
the focal point, were a local connection,[br]they're getting information from local
0:18:20.130,0:18:27.002
devices and then they have the uplink in[br]the command center. One thing I wanted to
0:18:27.002,0:18:31.078
point out, it's a common misconception or[br]something that comes up a lot where people
0:18:31.078,0:18:35.710
are concerned about police surveillance,[br]as we'll see an unmarked vehicle or a van
0:18:35.710,0:18:41.920
with no windows. It may even have some[br]antennas or satellite. And well that is
0:18:41.920,0:18:46.676
possibly an undercover police vehicle, you[br]shouldn't assume that vehicle belongs to
0:18:46.676,0:18:53.229
law enforcement. That could very easily be[br]a news media vehicle. News media also goes
0:18:53.229,0:18:58.510
to protests. They also have satellite[br]uplinks and antennas, look very similar.
0:18:58.510,0:19:05.428
And in some cases, the media has a[br]security situation there. They're worried
0:19:05.428,0:19:11.871
that there may be theft of equipment and[br]they have unmarked vans. So it is worth
0:19:11.871,0:19:15.704
noting that there is an unmarked vehicle,[br]but you shouldn't necessarily assume that
0:19:15.704,0:19:21.015
it is a police unmarked vehicle. Also,[br]sometimes people see especially they see
0:19:21.015,0:19:26.452
some antennas or satellite dish vehicles[br]that maybe that's were "stingrays". This
0:19:26.452,0:19:33.304
is a misconception. Stingrays are pretty[br]small and they don't require an external
0:19:33.304,0:19:40.011
antenna. You could put a stingray inside a[br]trunk of a car, maybe it will be a
0:19:40.011,0:19:47.172
briefcase sized. So it would be unlikely[br]that if you're going to use a stingray or
0:19:47.172,0:19:52.259
similar, IMSI-catcher that you would[br]want to put it in the vehicle. I don't
0:19:52.259,0:19:58.569
need to put in a vehicle that has its own[br]antenna. There have been not very much
0:19:58.569,0:20:03.810
documentation of these technologies being[br]used in the US domestic protests. They
0:20:03.810,0:20:08.549
have been used, we know it, in some[br]protests and more authoritarian countries.
0:20:08.549,0:20:16.501
So it's unclear how often they will are[br]being used and they are very dangerous in
0:20:16.501,0:20:20.705
the surveillance. So It's aim is[br]capturing, it is able to determine what
0:20:20.705,0:20:25.610
cell phones are nearby, get a unique[br]identifier with that cell phone, and in
0:20:25.610,0:20:29.862
many cases be able to use that information[br]to determine while individuals president
0:20:29.862,0:20:35.577
protests. That information has been used[br]after some protests in Ukraine, for
0:20:35.577,0:20:40.293
example, to send a text message to people[br]telling them: we're onto you, we know you
0:20:40.293,0:20:45.276
were there, which can be very intimidating[br]to individuals. But the challenge is that
0:20:45.276,0:20:49.226
you're trying to observe police[br]surveillance in the protest, it is hard
0:20:49.226,0:20:54.103
for you to observe it because they are[br]often hidden. You may be able to find out
0:20:54.103,0:20:57.822
more information later through[br]investigative journalism or public record
0:20:57.822,0:21:02.442
requests, or news reports. If somebody is[br]prosecuted using that information, it may
0:21:02.442,0:21:09.405
become obvious. It is difficult to see it[br]at the protest itself. So next category,
0:21:09.405,0:21:16.086
look it up in the sky. There are lots of[br]forms of aerial surveillance or for the
0:21:16.086,0:21:21.394
agencies to surveil protests from above[br]using traditional aircraft, on board
0:21:21.394,0:21:27.412
pilots and as well as remotely operated[br]aerial systems - drones. Law enforcement
0:21:27.412,0:21:30.859
may also use these aerial vehicles to[br]communicate with the crowd, to use the
0:21:30.859,0:21:36.422
loudspeakers to send a message to the[br]crowd. Order them to disperse. And we've
0:21:36.422,0:21:40.194
seen this actually drones with[br]loudspeakers being used by the German
0:21:40.194,0:21:45.792
police in order to tell people to stay[br]apart - Corona That same technology can be
0:21:45.792,0:21:51.842
used for protests. And these planes and[br]drones will often be equipped with high
0:21:51.842,0:21:56.432
definition cameras, capable of either[br]extremely wide angle to get the whole
0:21:56.432,0:22:01.640
scene or an extreme zoom where they might[br]be able to zoom in on a particular person,
0:22:01.640,0:22:07.473
particular license plate, and then use[br]that data later, partnering that the
0:22:07.473,0:22:12.699
aircraft with license plate recognition,[br]facial recognition, video analytics and
0:22:12.699,0:22:21.149
even a cell-site simulator inside the[br]aircraft. And we know this has happened in
0:22:21.149,0:22:31.005
a recent protest in Texas. A Texas police[br]drone got some footage of a protester
0:22:31.005,0:22:36.272
allegedly throwing a water bottle. They[br]took that video, they took the picture,
0:22:36.272,0:22:41.404
put it out, offered a cash reward, an[br]anonymous tip to turn the kid in and
0:22:41.404,0:22:48.332
protester was prosecuted. So police are[br]definitely using these things to gather
0:22:48.332,0:22:55.118
information at protesters. So a common[br]method for a larger police departments is
0:22:55.118,0:22:59.852
fixed-wing aircraft; for smaller ones -[br]they may use private contractors to
0:22:59.852,0:23:05.162
provide these fixed-wing aircraft. So this[br]is an example of the kind of plane you use
0:23:05.162,0:23:08.912
by a company called Persistent[br]Surveillance Systems. It rents out a plane
0:23:08.912,0:23:12.392
like this, well not this exact one, you[br]look up that tail number, it's going to be
0:23:12.392,0:23:16.896
to a different company. But the same model[br]of the Cessna 207-A. And these will circle
0:23:16.896,0:23:23.342
around the protest using their cameras to[br]observe protesters below. And the
0:23:23.342,0:23:28.580
advantage of planes is they can often[br]circle for quite a long time and provide a
0:23:28.580,0:23:36.402
wide view on the area. Also, helicopters,[br]helicopters will often be seen hovering
0:23:36.402,0:23:41.102
over a protest, they are a little bit[br]easier to maneuver and be able to go
0:23:41.102,0:23:47.343
backwards and forwards over the protest[br]that are used by police to continually
0:23:47.343,0:23:52.270
observe. And we'll use two examples here.[br]One of them from the Oakland Police
0:23:52.270,0:23:56.280
Department, the other one from the[br]Rineland Police Department. In both cases,
0:23:56.280,0:24:01.735
they have a FLIR attached to the[br]helicopter, a forward looking infrared
0:24:01.735,0:24:06.362
that would allow them, in addition to[br]regular camera capabilities, to use
0:24:06.362,0:24:10.622
thermal imaging to follow someone at a[br]protest or follow what's going on after
0:24:10.622,0:24:18.252
dark. We can also see that some[br]helicopters will have spotlights, so that
0:24:18.252,0:24:24.081
they can signal to officers on the ground[br]who to follow, who do pay attention to.
0:24:24.081,0:24:31.292
Another thing for both fixed wing and[br]helicopters: they look for the tail
0:24:31.292,0:24:35.352
number. In most jurisdictions, they're[br]required to have a tail number visible,
0:24:35.352,0:24:40.232
and then you could look up that tail[br]number on services like Flight Aware and
0:24:40.232,0:24:44.362
be able to find out further information[br]about what that plane has been doing, what
0:24:44.362,0:24:51.737
the helicopter has been doing, as well as[br]the ownership. Finally, drones. Drones are
0:24:51.737,0:24:56.543
becoming very commonplace because they're[br]getting cheaper all the time and having
0:24:56.543,0:25:01.501
additional capacities; drones are also[br]known as unmanned aerial vehicles - UAVs;
0:25:01.501,0:25:08.694
UAS - unmanned aerial systems and a lot of[br]police departments are getting them for
0:25:08.694,0:25:14.552
their capabilities using what's called a[br]quadrotor. And they can be controlled by
0:25:14.552,0:25:20.795
remote control, have a camera built into[br]this and be useful for getting above the
0:25:20.795,0:25:27.472
scene view. So one way to spot it... well,[br]first of all - just listen to it, they
0:25:27.472,0:25:30.947
make kind of have a distinctive noise,[br]sometimes are really marked as police.
0:25:30.947,0:25:38.436
Also look for the pilots operating nearby.[br]So oftentimes a patrol out there,
0:25:38.436,0:25:42.222
sometimes they are labeled like in the[br]upper left there, it says "police drone
0:25:42.222,0:25:47.254
operator" - pretty easy to identify. All[br]the times they might have like "drone",
0:25:47.254,0:25:54.107
"UAV", "aviation unit" on their uniform,[br]or a nearby police vehicle. The other
0:25:54.107,0:25:59.574
thing is that if you identify a drone,[br]there're often within line of sight is
0:25:59.574,0:26:03.577
going to be the operator. So when do you[br]see the drone look around and see, if
0:26:03.577,0:26:07.800
someone has the controls in their hand, is[br]looking up the drone, you can probably
0:26:07.800,0:26:13.256
identify the operator and go look for[br]information that might have uniform about
0:26:13.256,0:26:19.310
who is operating that particular drone.[br]But also, keep in mind, both for drones
0:26:19.310,0:26:23.962
and other aircraft, that it's not[br]necessarily the police. Journalists and
0:26:23.962,0:26:29.629
activists will often fly drones over[br]protest, news helicopters for a large
0:26:29.629,0:26:34.629
protests are going to be more common than[br]police helicopters. And many times they
0:26:34.629,0:26:40.222
are labeled with picture of the BBC News[br]Copter. But this means that just as you
0:26:40.222,0:26:47.127
see a helicopter that has both a camera[br]and is flying over the protest, that does
0:26:47.127,0:26:53.984
not necessarily mean that it's a police[br]helicopter. Also, the technology, which is
0:26:53.984,0:27:00.383
actually not very commonplace outside of[br]protests in war zones, but - the
0:27:00.383,0:27:04.687
"Dronekiller" technology, which is[br]basically a real gun that knocks drones
0:27:04.687,0:27:11.321
out of the sky, sending radio signals to[br]interfere with the drones operation and
0:27:11.321,0:27:18.812
cause it to fall and crash. These have[br]been used in Iraq and Afghanistan and the
0:27:18.812,0:27:23.310
technology could be certainly used, but we[br]really haven't seen it used more freely.
0:27:23.310,0:27:27.792
Just wanted to tell you about it[br]because... oh, my God, Dronekillers. All
0:27:27.792,0:27:32.787
right, last place to look for police[br]technology - in the environment around
0:27:32.787,0:27:38.986
you. There will be in many places camera[br]networks. So a lot of the cameras will see
0:27:38.986,0:27:46.348
a neighborhood will be private cameras,[br]will be police cameras, will be cameras
0:27:46.348,0:27:52.678
being used by city, non police agencies.[br]That can be a lot of cameras. This also
0:27:52.678,0:27:56.095
means that you're trying to observe what[br]cameras are going on. There's too much
0:27:56.095,0:27:59.776
information. There'll be so many cameras[br]in many areas that you could spend all
0:27:59.776,0:28:03.746
your time documenting, observing the[br]cameras and these other things. So you
0:28:03.746,0:28:07.307
might not want to spend all your time[br]paying attention to that, because you can
0:28:07.307,0:28:11.544
go back later at any point and see the[br]fixed cameras. But there are a couple of
0:28:11.544,0:28:17.317
things that... first, identifying them for[br]two different brands to identify. But
0:28:17.317,0:28:21.858
here's some information about the kinds of[br]cameras that are available. Bullet cameras
0:28:21.858,0:28:25.542
are directional, so you can sort of see[br]which way it's pointing and what it would
0:28:25.542,0:28:30.258
be covering from that. Then you have dome[br]cameras, which are designed, so you can't
0:28:30.258,0:28:34.529
see which way it's pointing, or at least[br]you can see maybe some of this area one
0:28:34.529,0:28:38.874
hundred and eighty degrees. But the exact[br]direction is pointing is obscured by the
0:28:38.874,0:28:46.272
dome. Pan-Tilt-Zoom cameras can change[br]which way they're pointing. They can
0:28:46.272,0:28:51.893
sometimes be coupled with a dome camera so[br]that the dome camera can both change the
0:28:51.893,0:28:58.876
way it's looking and obscure which way its[br]looking. Thermal imaging cameras and ALPR
0:28:58.876,0:29:04.448
cameras are also becoming a fixed[br]locations and we are all having to do with
0:29:04.448,0:29:10.220
traffic control. Thermal is actually not[br]as common. That is mostly used as a
0:29:10.220,0:29:20.106
technology that is vehicles, is kind of[br]expensive. But in this case, the picture
0:29:20.106,0:29:25.324
shown is a thermal imaging camera. So[br]sometimes people will go to that
0:29:25.324,0:29:31.833
additional expense. And one subcategory of[br]all the cameras that are environment are
0:29:31.833,0:29:37.147
going to be police observation devices, so[br]the category of sets of sensors which are
0:29:37.147,0:29:42.804
operated by the police and they may[br]included multiple cameras, gunshot
0:29:42.804,0:29:49.015
detection, facial recognition. For[br]example, in the United Kingdom, say in the
0:29:49.015,0:29:53.498
city of London, is doing live facial[br]recognition as a police observation
0:29:53.498,0:29:59.271
devices are a collection of these cameras[br]in one location. Sometimes they're marked
0:29:59.271,0:30:07.856
as police, sometimes they are not. And[br]there is the way you would suspect that
0:30:07.856,0:30:12.796
it's a police observation device is - if[br]it has a lot of different sensors in one
0:30:12.796,0:30:17.760
location trying to cover the whole ground[br]around, then that is the kind of thing you
0:30:17.760,0:30:23.518
would see most frequently from a police[br]office image. And then finally, Smart
0:30:23.518,0:30:28.287
Street Lights. The Smart Street Lights[br]have a number of wonderful applications.
0:30:28.287,0:30:33.847
Some initiatives like in the US Smart[br]Cities and the EU E-Street Initiative are
0:30:33.847,0:30:38.085
exploiting cities to use more Smart Street[br]Lights, because they could turn down the
0:30:38.085,0:30:44.273
power usage when the light is less needed.[br]And there's some advance towards a project
0:30:44.273,0:30:50.035
by the Arnold University of Applied[br]Sciences has a technology, which will use
0:30:50.035,0:30:55.309
motion detection, sound detection. Being[br]able to tell that there are people walking
0:30:55.309,0:31:00.414
nearby and brighten their path. Sounds[br]great. But the same kind of technology
0:31:00.414,0:31:06.174
being able to detect motion, being able to[br]have audio signals, video signals - can be
0:31:06.174,0:31:10.391
used for surveillance. So here on the[br]slide we show the smart lighting
0:31:10.391,0:31:16.353
capabilities being advertised by Intel. In[br]addition to some things that you might
0:31:16.353,0:31:23.389
expect, like being able to adjusted for[br]traffic patterns, provide when needed.
0:31:23.389,0:31:26.013
They talk about crime[br]investigations, monitor parking
0:31:26.013,0:31:31.358
violations, safety announcements that are[br]coming from the smart cameras. So all of
0:31:31.358,0:31:36.904
these technologies are possible and[br]hopefully this will not become a
0:31:36.904,0:31:40.441
commonplace use. But if it is, it would[br]mean that a surveillance device is
0:31:40.441,0:31:45.296
everywhere along every street where[br]they're putting these devices. Wich
0:31:45.296,0:31:49.482
lightning the city you are lightning the[br]city with surveillance. So has it been
0:31:49.482,0:31:56.476
used? Yes. The city of San Diego had a[br]number of protests surrounding the protest
0:31:56.476,0:32:02.744
around George Floyd, and they used these[br]thirty five times. They searched
0:32:02.744,0:32:07.631
information gathered through the Smart[br]Street Light Network for evidence in
0:32:07.631,0:32:16.354
criminal cases coming out of that protest.[br]So what additional resources are there?
0:32:16.354,0:32:19.445
There's plenty of additional resources, if[br]you wanted to try to read more - and I
0:32:19.445,0:32:22.830
encourage you to take this only as a[br]starting point for a lot more to learn. So
0:32:22.830,0:32:25.856
we'll start out with a very important[br]resource. If you're someone who's going to
0:32:25.856,0:32:30.262
go, whether as an activist, as a[br]protester, as a journalist, you should
0:32:30.262,0:32:38.142
prepare yourself for some surveillance[br]self defense: at ssd.eff.org we have an
0:32:38.142,0:32:43.220
"Attending a Protest" guide. Go there and[br]learn important tips on protecting
0:32:43.220,0:32:48.202
yourself when you're going to protests.[br]Put your device with fulldesk encryption:
0:32:48.202,0:32:53.242
a strong unique password, turning off the[br]biometric unlock, use end-to-end
0:32:53.242,0:32:58.962
encryption for messages and calls, walking[br]or taking a bicycle to get to the protest,
0:32:58.962,0:33:04.485
sort of a vehicle, which couldn't be[br]subjected to a ANPR / ALPR device. Wear a
0:33:04.485,0:33:11.170
mask, you should wear a mask for Covid[br]anyway. But if you wear a mask, get a big:
0:33:11.170,0:33:16.530
the larger the mask, the more it protects[br]you. There is also recently a study that
0:33:16.530,0:33:24.798
showed that they're making efforts to try[br]to make facial recognition continue on
0:33:24.798,0:33:30.269
despite people's use of masks. And there[br]was a study that showed that red and black
0:33:30.269,0:33:37.733
masks were harder for the AI to be able to[br]determine who was behind the mask. So wear
0:33:37.733,0:33:42.879
a red or black mask. If you get one that[br]covers more of your face like a bandana,
0:33:42.879,0:33:47.020
it's going to be harder for the facial[br]recognition algorithms. So do some things
0:33:47.020,0:33:53.066
to protect yourself, both from Covid and[br]from surveillance. If you want to also
0:33:53.066,0:33:59.328
just practice it out, you can go to our[br]Spot the Surveillance. This is an online
0:33:59.328,0:34:05.792
program, you can do the desktop version or[br]virtual reality version, where it places
0:34:05.792,0:34:11.424
you in a virtual street corner with some[br]surveillance devices nearby. And you can
0:34:11.424,0:34:15.648
look around and try to identify all of the[br]surveillance devices that you see. It
0:34:15.648,0:34:21.302
takes just a few minutes to go through the[br]exercise, but it's a good way to practice
0:34:21.302,0:34:27.950
your skills and identify what surveillance[br]might be around on the street. And if you
0:34:27.950,0:34:34.376
want to get a lot more information about[br]any of these devices, go to EFF Street-
0:34:34.376,0:34:40.219
Level Surveillance project: eff.org/sls -[br]street level surveillance. This will
0:34:40.219,0:34:46.076
provide more detailed information about[br]various technologies that are in use. That
0:34:46.076,0:34:50.396
can be a good starting point, especially[br]if you found out what is being used in
0:34:50.396,0:34:55.912
your jurisdiction. Go there and find out[br]more about it. And you can also find out
0:34:55.912,0:35:02.316
just what is going on more generally with[br]these kinds of technologies: eff.org/sls.
0:35:02.316,0:35:06.858
All right. Well, thank you. That comes to[br]the close of my talk. Thank you for
0:35:06.858,0:35:19.175
turning in. And now let me turn it over to[br]my future self for Q&A. Thanks you.
0:35:19.175,0:35:29.022
Herald: Welcome back. Thanks so much Kurt.[br]Now we have some time for questions and
0:35:29.022,0:35:34.383
it's getting more and more. I'm just[br]hurrying up. OK, are there devices, apps
0:35:34.383,0:35:39.057
or services developed or run by private[br]companies and who makes sure the data is
0:35:39.057,0:35:46.052
not directly sold to third parties?[br]Kurt: So, yes, there are private networks.
0:35:46.052,0:35:49.899
And one of the things we talked about just[br]now is there's a lot of private camera
0:35:49.899,0:35:54.341
networks that are providing information to[br]the police, sometimes private networks
0:35:54.341,0:35:59.157
going through a registry where police from[br]organizers ask people to volunteer, put
0:35:59.157,0:36:02.733
their information into a registry. So they[br]are sort of explicitly saying they're
0:36:02.733,0:36:06.589
going to turn over their information to[br]the police or other things like Amazon's
0:36:06.589,0:36:11.702
ring camera. They have been promoting it[br]as a antitheft tool, trying to stop
0:36:11.702,0:36:16.461
package stuff stolen from doors. But this[br]also is creating a panopticon of
0:36:16.461,0:36:22.722
everyone's doorbell camera. If they're all[br]using ring will be provided... get the
0:36:22.722,0:36:28.262
video and will provide it to the police.[br]So and many of these organizations, you
0:36:28.262,0:36:32.057
know, if they're larger than they will[br]have some some privacy practices, probably
0:36:32.057,0:36:37.326
policies. But by and large, they will talk[br]about the privacy of the person who owns
0:36:37.326,0:36:43.585
the marketplace and not really consider[br]the bystanders, the people walking by. So
0:36:43.585,0:36:48.402
if you have a doorbell camera at your[br]front door that can hear audio, so maybe
0:36:48.402,0:36:52.262
someone could ring your bell and say[br]hello. It will also capture people walking
0:36:52.262,0:37:00.077
by and those people's privacy is important[br]to be considered.
0:37:00.077,0:37:06.389
Herald: All right, then we have: what hope[br]do we have against all this, which best
0:37:06.389,0:37:11.332
case legal countermeasures do we have by[br]been attending protests? And the another
0:37:11.332,0:37:16.758
one, which I would connect directly: is it[br]possible to intervene against surveillance
0:37:16.758,0:37:21.440
based on laws or presumption of innocence?[br]Kurt: Yes.
0:37:21.440,0:37:26.064
Herald: I don't know if German laws are[br]meant, but maybe you still can say
0:37:26.064,0:37:28.062
something?[br]Kurt: Well, I mean, so there's many
0:37:28.062,0:37:32.582
different laws that might be an issue. I[br]mean, we have an international audience
0:37:32.582,0:37:35.924
here, but I think there are some also[br]basic human rights principles that apply
0:37:35.924,0:37:41.014
to many jurisdictions. But I would say[br]actually one of the most effective tools
0:37:41.014,0:37:44.883
to push back against this kind of police[br]surveillance is working locally with the,
0:37:44.883,0:37:52.266
like, a city, the mayor, the city council[br]and a number of locations have passed
0:37:52.266,0:37:56.183
rules about what their police can do[br]against their citizens. So putting
0:37:56.183,0:38:01.452
limitations on what police can do at the[br]local level, where your activities in the
0:38:01.452,0:38:06.552
city, which you live, taking things to[br]your representative government and saying
0:38:06.552,0:38:10.974
we need to have some limitations on this,[br]we need to have it within civilian
0:38:10.974,0:38:14.875
controls for the police themselves are not[br]deciding what technologies to use, but it
0:38:14.875,0:38:17.762
has to pass through an elected[br]representative. And I think that is
0:38:17.762,0:38:23.332
probably one of the most effective ways to[br]at least start change where you live. But
0:38:23.332,0:38:27.462
you can also try and promote that to your[br]national legislature, state legislatures,
0:38:27.462,0:38:32.673
go up several levels. And one of the[br]things that I hope comes out of this
0:38:32.673,0:38:34.902
guide, where people getting more[br]information about what kind of
0:38:34.902,0:38:38.573
surveillance is available, so that they[br]can go to their representatives, go
0:38:38.573,0:38:42.778
through the political process with the[br]information of what tool use something
0:38:42.778,0:38:48.212
that they have drawn a line about. Go to[br]your representative and say we need to
0:38:48.212,0:38:52.166
make sure that the information that[br]they're gathered is being used in a manner
0:38:52.166,0:38:56.299
consistent with human rights principles.[br]And we need civilian control from the
0:38:56.299,0:39:00.552
local government on how to do.[br]Herald: Who's controlling the controler
0:39:00.552,0:39:13.972
instances, yes. We have more questions.[br]OK, so the police operate equipment like
0:39:13.972,0:39:17.682
ANPR-reader IMSI-Catcher etc. , that get[br]information that they could get in a
0:39:17.682,0:39:26.022
cheaper way, like reading traffic signs or[br]license plates, or cell info from
0:39:26.022,0:39:33.712
operators. And is there a reason for that?[br]Especially concerning EU, because US
0:39:33.712,0:39:42.282
differs a lot. And another question: has[br]police in EU, US been known to use illegal
0:39:42.282,0:39:49.801
or questionable tech for surveillance?[br]Kurt: So I think on the first question
0:39:49.801,0:39:57.190
about using things like ANPR to determine[br]license plates, this technology is common
0:39:57.190,0:40:03.212
in the European Union, though by and large[br]it is being put in place for other
0:40:03.212,0:40:08.370
reasons, not to get the protesters[br]necessarily. They are looking for, you
0:40:08.370,0:40:12.409
know, making sure that people are paying a[br]toll or might be a speed trap on the
0:40:12.409,0:40:17.431
autobahn, where it takes a picture of the[br]license plate of anyone traveling over a
0:40:17.431,0:40:22.654
speed limit - in the places that have[br]speed limits on the part of the autobahn.
0:40:22.654,0:40:28.773
And I think also it's being used for[br]enforcement of things like traffic
0:40:28.773,0:40:33.916
citations: your cars parked on location[br]too long. They know who to send the bill
0:40:33.916,0:40:41.123
to. And I think these technologies could[br]be repurposed for surveillance. And that's
0:40:41.123,0:40:45.217
what we really need, is policies that are[br]ensuring that if these things are being
0:40:45.217,0:40:49.739
used for a purpose, that the sort of the[br]citizenship agrees with in that
0:40:49.739,0:40:54.100
jurisdiction to enforce parking, for[br]example, that is not also being repurposed
0:40:54.100,0:40:59.288
against political activities and being[br]used at a wider scale than it was
0:40:59.288,0:41:03.814
envisioned. Also, maybe, you know, it's[br]not a good thing to have perfect parking
0:41:03.814,0:41:08.293
enforcement, you know, a lot of parking[br]fines were based on the notion that, like,
0:41:08.293,0:41:12.646
you might not get caught every time and[br]when you change it, a system where
0:41:12.646,0:41:17.395
previously the fines were set with the[br]notion that a lot of people would get away
0:41:17.395,0:41:20.720
with it.. to I do, like, to make an[br]example of those who didn't. And then you
0:41:20.720,0:41:24.600
changed that to perfect enforcement[br]because the computer, the ANPR system,
0:41:24.600,0:41:30.256
surveillance knows exactly the minute that[br]a fine is due and then assesses that fine.
0:41:30.256,0:41:33.932
That actually changes the dynamic of power[br]between citizenship and the state
0:41:33.932,0:41:38.630
significantly. And it will all be freeze[br]and forms of well, we're just trying to
0:41:38.630,0:41:42.377
enforce the existing laws. How could you[br]be against that? But really, it changes
0:41:42.377,0:41:45.885
the dynamic. And it's something that for[br]those who want to be an activist on this,
0:41:45.885,0:41:49.851
again, talk to your local jurisdiction and[br]try and make sure that these things have
0:41:49.851,0:41:52.794
safe and sane policies that respect human[br]rights.
0:41:52.794,0:42:01.147
Herald: So I would interpret that like[br]prevention of... don't come on the idea
0:42:01.147,0:42:05.635
that you need to to protect your data,[br]right? Fun.
0:42:05.635,0:42:09.650
Kurt: And just turning to the another one.[br]Do we have information about whether
0:42:09.650,0:42:14.618
police are misusing these technologies?[br]So, I mean, there's some isolated examples
0:42:14.618,0:42:19.460
where people have misused their[br]technologies. And I use a couple of them
0:42:19.460,0:42:23.331
in the slides. And there was someone who[br]went to a political protest. Their car was
0:42:23.331,0:42:29.565
put into a database to get pulled over[br]later. And then also in South Wales, the
0:42:29.565,0:42:36.839
court found that there the police use of[br]facial recognition was in violation of UK
0:42:36.839,0:42:42.027
law, though, as I noted, not the[br]Metropolitan Police in London don't agree
0:42:42.027,0:42:45.446
with that. They say it doesn't apply to[br]them. And I think actually use of facial
0:42:45.446,0:42:49.851
recognition technologies is a very[br]tempting thing by police. They want to use
0:42:49.851,0:42:56.297
it as much as possible, make it easy for[br]them. And I think you will see that. But
0:42:56.297,0:43:01.312
the other piece of this is unless there[br]are rules that say here are limitations on
0:43:01.312,0:43:05.507
how you can use these technologies and[br]they can use them without having to risk
0:43:05.507,0:43:11.432
violating them. So we need to have those[br]rules in place. I hope that the Council of
0:43:11.432,0:43:17.271
Europe puts at least a moratorium on[br]facial recognition for use for police. And
0:43:17.271,0:43:22.844
until we can figure out how to use this[br]technology safely, it's kind of cool that
0:43:22.844,0:43:25.680
you can unlock your phone in your face[br]without having to type in a password,
0:43:25.680,0:43:30.092
but... we want to make sure that[br]technology is used properly.
0:43:30.092,0:43:35.242
Herald: OK, so I think you're going to be[br]around in the 2D world. You're going to
0:43:35.242,0:43:41.252
explore that, you've told me before. Yeah,[br]there's more questions. I hope maybe you
0:43:41.252,0:43:47.462
find him in the 2D world and you just ask[br]him there. Thanks so much. Thanks so much.
0:43:47.462,0:43:52.698
Very nice having you. Bye Kurt.[br]Kurt: Bye.
0:43:52.698,0:43:52.754
rC3 postroll music
0:43:52.754,0:44:36.012
Subtitles created by c3subtitles.de[br]in the year 2021. Join, and help us!