0:00:00.000,0:00:13.850
preroll music
0:00:13.850,0:00:18.720
Vasilios: Hello, everyone, thanks for coming[br]today. I'm going to introduce the ultrasound
0:00:18.720,0:00:24.500
ecosystem, which is an exotic and kind of[br]little known ecosystem. So I would like to
0:00:24.500,0:00:29.820
start with a short story about the[br]product, which is also our motivation for
0:00:29.820,0:00:39.870
this work. So some time ago, there was a[br]product that worked in the ultrasound
0:00:39.870,0:00:44.450
spectrum that cannot be perceived by[br]humans. And the product was actually an
0:00:44.450,0:00:50.700
interesting idea. It was very promising[br]and everything, but it also had a fatal
0:00:50.700,0:00:56.730
flaw. So now that I've done this[br]introduction, I would like to tell you
0:00:56.730,0:01:01.350
more about the story of the product and[br]how it came to be and what was it? What
0:01:01.350,0:01:08.150
was its lifecycle. So 2012, a company[br]called SilverPush was a startup in
0:01:08.150,0:01:14.810
India. It was founded there and they had[br]this ultrasound device tracking product.
0:01:14.810,0:01:18.770
I'll go more into the technical details[br]later. So for a couple of years, they were
0:01:18.770,0:01:26.789
working on that product. And it wasn't[br]until 2014 that they basically got some
0:01:26.789,0:01:32.769
serious funding from a venture center or[br]other angel investors for millions. So in
0:01:32.769,0:01:38.840
2014, they also got a few months after[br]they got funded. They also got some press
0:01:38.840,0:01:44.940
coverage about the product and they got[br]some pretty good reviews on their
0:01:44.940,0:01:49.260
newspapers and articles about what the[br]product could do. And at the same time,
0:01:49.260,0:01:52.950
they were doing what most of the companies[br]are doing, like publishing patents about
0:01:52.950,0:02:00.029
their technology and everything. So things[br]later started to go like year after year
0:02:00.029,0:02:06.499
and half maybe started to go not that well[br]for them. The security community noticed
0:02:06.499,0:02:10.759
and there was some press coverage about[br]the product that was not so positive
0:02:10.759,0:02:19.250
anymore. So this is one of the very first[br]emails that appear on the Web regarding
0:02:19.250,0:02:24.709
the product. So it's from a W3C[br]working group. So a researcher there is
0:02:24.709,0:02:31.220
basically. Notifying the other members of[br]the group that, OK, there is this product,
0:02:31.220,0:02:36.349
maybe there are transparency issues, and[br]certainly the users are not aware of
0:02:36.349,0:02:42.489
what exactly is going on there. So let's[br]keep an eye on it. And so this was a very
0:02:42.489,0:02:48.370
one of the very first things published[br]about the product from the privacy and
0:02:48.370,0:02:54.470
security perspective. So what happened[br]then was the press took notice and they
0:02:54.470,0:03:01.379
got all those headlines urging users to be[br]very careful. And, oh, this is a this is
0:03:01.379,0:03:08.769
evil, take care. People are eavesdropping[br]on you and stuff. So, of course, this led
0:03:08.769,0:03:14.950
also on the FTC to take action. They[br]organized a workshop on cross device tracking
0:03:14.950,0:03:22.060
in general, I think, and they made specific[br]mentions for ultrasound cross device tracking
0:03:22.060,0:03:28.170
don't worry if you're not familiar with this terms,[br]I'm going to define everything later. So
0:03:28.170,0:03:33.390
what basically they were saying is[br]transparency issues. How do how do we
0:03:33.390,0:03:39.860
protect ourselves? How is that thing[br]working? So, then the users, of course,
0:03:39.860,0:03:43.689
started to react. And there were like many[br]people who were unhappy, they were
0:03:43.689,0:03:48.430
complaining, what is this? I don't want[br]that thing. So people were actually
0:03:48.430,0:03:53.479
suggesting solutions and the solutions[br]that were making sense. And of course, you
0:03:53.479,0:04:01.260
have always the users that are completely[br]immune to what you have there. So what
0:04:01.260,0:04:10.549
happened then is like five months after[br]the FTC took much more serious action
0:04:10.549,0:04:16.000
regarding this specific product. So it[br]sent a letter to all the developers. And
0:04:16.000,0:04:19.780
the letter was essentially saying, you[br]know, you're using this framework in
0:04:19.780,0:04:27.160
Europe. We've seen it in Google Play[br]store. It's not enough that you are asking
0:04:27.160,0:04:31.380
for the microphone permission. You should[br]let the users know that you are tracking
0:04:31.380,0:04:36.330
them if you are doing so. Otherwise, you[br]are violating rule X, Y, Z, and you're not
0:04:36.330,0:04:42.590
allowed to do that. So this was pretty[br]serious, I would say. And what happened
0:04:42.590,0:04:46.610
next is basically the company withdrew[br]from the US market and said, you know, we
0:04:46.610,0:04:51.040
have nothing to do with the U.S. market[br]and this product is not active there. You
0:04:51.040,0:04:57.320
shouldn't be concerned. So end of story[br]like the product is not out there in the
0:04:57.320,0:05:03.660
US at least anymore. Are we safe? So it[br]seemed to us that it was assumed that this
0:05:03.660,0:05:10.030
was an isolated security incident. And to[br]be fair, very little became known about
0:05:10.030,0:05:15.090
the technology. At this point. The press[br]moved on to other hot topics at the time,
0:05:15.090,0:05:20.950
people went quiet, like if people are not[br]using it, it's fine. So everyone
0:05:20.950,0:05:25.840
seemed happy. But we're curious people. So[br]we had lots of questions that were not
0:05:25.840,0:05:35.040
answered. So our main questions was like[br]why they were using ultrasounds. We'll see
0:05:35.040,0:05:41.620
that what they are doing, you can do with[br]our technologies, how such frameworks
0:05:41.620,0:05:47.120
work. We had no idea there was no coverage[br]or nothing about it. The technical,
0:05:47.120,0:05:53.340
technically speaking, out there, are there[br]other such products there? Because we were
0:05:53.340,0:05:59.410
aware of one. Everyone on all the articles[br]was referring to that one product, but we
0:05:59.410,0:06:03.210
were not sure if there are others doing[br]the same thing. And of course, we were
0:06:03.210,0:06:07.870
looking for a report about the whole[br]ecosystem and how it works. And there was
0:06:07.870,0:06:13.290
nothing. So what do you do then if if[br]there are no technical resources?
0:06:13.290,0:06:19.740
Basically, we decided to do our own[br]research and come up with this report that
0:06:19.740,0:06:24.980
we were lacking. So we're done with[br]motivation so far. We were pretty pumped
0:06:24.980,0:06:29.891
up about looking into it. OK, what's[br]there? The rest of the presentation will
0:06:29.891,0:06:34.010
go as follows. Like first I'm going to[br]introduce ultrasound tracking and other
0:06:34.010,0:06:40.620
terminology, then I'm going to go on with[br]the attack details. And indeed, we have an
0:06:40.620,0:06:47.610
attack again against the Tor browser. Then[br]we're doing a formal security analysis of
0:06:47.610,0:06:53.350
the ecosystem and try to pinpoint the[br]things that went wrong. And then we'll try
0:06:53.350,0:07:00.470
to introduce our countermeasures and[br]advocate for proper practices. So to begin
0:07:00.470,0:07:06.940
with, I'm Vasilis. I've done this work[br]with other curious people. These are
0:07:06.940,0:07:13.440
showing how Yanick Fratantonio, Christopher[br]Kruegel and Giovanni Vigna from UCSB and also
0:07:13.440,0:07:19.280
Federico Maggi from Polytechnical[br]Damilola. Let's now start with the
0:07:19.280,0:07:26.340
ecosystem, so apparently ultrasounds are[br]used in a lot of places and they can be
0:07:26.340,0:07:30.920
utilized for different purposes, some of[br]them are cross device tracking that are
0:07:30.920,0:07:36.770
referred already to audience analytics,[br]synchronized content, proximity, marketing
0:07:36.770,0:07:41.460
and device pairing. You can do some other[br]things, but you will see them later. So to
0:07:41.460,0:07:48.920
begin with what cross device tracking is,[br]cross device tracking is basically the holy
0:07:48.920,0:07:53.630
grail for marketers right now because[br]you're using your multiple devices
0:07:53.630,0:07:57.990
smartphone, laptop, computer, maybe your[br]TV and to them, your appear as different
0:07:57.990,0:08:02.330
people. And they all want to be able to[br]link to link those devices to know that
0:08:02.330,0:08:06.920
you're the same person so that they can[br]build their profiles more accurately. So,
0:08:06.920,0:08:13.300
for instance, if you're watching an ad on[br]the TV, they want to be able to know that
0:08:13.300,0:08:19.720
it's you so that they can push relevant[br]ads from your smartphone or follow up ads.
0:08:19.720,0:08:27.860
Um. So this is employed by major[br]advertising networks, and there are two
0:08:27.860,0:08:32.780
ways to do it, either deterministically or[br]probabilistically, that deterministic
0:08:32.780,0:08:38.780
approach is much more reliable. You get[br]100 percent accuracy and works as follows.
0:08:38.780,0:08:43.200
If you are Facebook, the users are heavily[br]incentivized to log in from all their
0:08:43.200,0:08:49.220
devices. So what happens is that. You can[br]immediately know that, OK, this user has
0:08:49.220,0:08:55.191
these three devices and I can put relevant[br]content to all of them. However, if you
0:08:55.191,0:08:58.910
are not Facebook or Google you, it's much[br]more unlikely that the users would want to
0:08:58.910,0:09:03.580
log into your platform from their[br]different devices. So you have to look for
0:09:03.580,0:09:09.970
alternatives. And one tool to come up with[br]those alternatives are ultrasound beacons.
0:09:09.970,0:09:18.480
So, um, ultrasound tracking products are[br]using ultrasound because they may sound
0:09:18.480,0:09:22.590
exotic, but basically there they are. What[br]they are doing is they are encoding a
0:09:22.590,0:09:29.460
sequence of symbols, um, in a very high[br]frequency that it's inaudible by humans.
0:09:29.460,0:09:35.520
That's the first key feature. The second one[br]is they can be emitted by most commercial
0:09:35.520,0:09:39.400
speakers and they can be captured by most[br]commercial microphones, for instance,
0:09:39.400,0:09:48.620
found on your smartphone. So the technical[br]details are the following. I know there
0:09:48.620,0:09:54.380
are a lot of experts in these kinds of[br]things here, so I'm averaging out what how
0:09:54.380,0:09:57.590
the companies are doing it right now. I'm[br]not saying that this is the best way to do
0:09:57.590,0:10:01.520
it, but this is more or less what they're[br]doing. Of course, they have patents, so
0:10:01.520,0:10:06.230
each one of them is doing a slightly[br]different thing so they don't overlap.
0:10:06.230,0:10:13.330
They're using the near ultrasound spectrum[br]between the eight eight kilohertz and 20
0:10:13.330,0:10:18.940
kilohertz, which is inaudible by usually[br]by adults. They divide it in smaller
0:10:18.940,0:10:27.380
chunks. So if you divide it in chunks that[br]have size of 75 Hertz, you get 26, about 26
0:10:27.380,0:10:33.920
chunks, and then you can assign letter of[br]the alphabet on each one of them. And then
0:10:33.920,0:10:38.109
what they are doing is usually within four[br]to five seconds. They emit sequences of
0:10:38.109,0:10:45.410
characters. Usually they contain for four[br]to six characters in there, and they use
0:10:45.410,0:10:51.670
it to incorporate a unique ID[br]corresponding to their source, they attach
0:10:51.670,0:10:56.710
the beacon to. So there is no ultrasound[br]beacon standard, as I said previously, but
0:10:56.710,0:11:00.450
there are lots of patents, so each one of[br]them is doing a slightly different thing.
0:11:00.450,0:11:06.350
But this is a basic principle. We did some[br]experiments and we found out that within
0:11:06.350,0:11:13.880
seven meters, you get pretty good accuracy[br]in low error rate. So of course, this depends
0:11:13.880,0:11:20.250
exactly how you encode things. But with[br]applications found on Google Play, this
0:11:20.250,0:11:24.990
worked up to seven meters. Um, we couldn't[br]find computer speakers that were not able
0:11:24.990,0:11:33.310
to emit near ultrasound frequencies and[br]work with this technology and.. we this is
0:11:33.310,0:11:36.590
pretty known for this kind of frequencies,[br]they cannot penetrate through physical
0:11:36.590,0:11:41.000
objects, but this is not a problem for[br]their purposes. And we did some
0:11:41.000,0:11:46.720
experiments with our research assistant[br]and we can say that they are audible by
0:11:46.720,0:11:54.420
animals. So if you combine cross device[br]tracking and ultrasound because you get
0:11:54.420,0:12:02.350
ultrasound cross device tracking. So now what[br]you can do with this and this is this is a
0:12:02.350,0:12:07.320
pretty good idea, actually, because it[br]offers high accuracy, you don't ask the
0:12:07.320,0:12:16.720
users to log in, which is very high, very[br]demanding thing to ask for. You can embed
0:12:16.720,0:12:22.860
those beacons in websites or TV ads, and[br]this technology, however, requires some
0:12:22.860,0:12:26.210
sort of sophisticated backend[br]infrastructure. We're going to see more
0:12:26.210,0:12:30.260
about it later. And you also need the[br]network of publishers who are willing to
0:12:30.260,0:12:36.740
incorporate incorporate beacons in their[br]content, whatever this content is. And
0:12:36.740,0:12:41.250
then, of course, you need an ultrasound[br]cross device tracking framework that is going
0:12:41.250,0:12:47.080
to run on the user's mobile device, a[br]smartphone. So these frameworks are
0:12:47.080,0:12:52.680
essentially and as the advertising SDK is the[br]key that the developers can use to display
0:12:52.680,0:12:57.060
ads on their free apps. So it's not that[br]the developers are going to incorporate
0:12:57.060,0:13:04.491
the ultrasound framework is going to[br]incorporate an advertising SDK with
0:13:04.491,0:13:09.610
varying degrees of understanding of what[br]it does. So here is how ultrasound cross device
0:13:09.610,0:13:15.740
tracking works. On step one, basically, we[br]have the advertising client. He just wants
0:13:15.740,0:13:20.200
to advertise, advertises his products. He[br]goes to the ultrasound cross device
0:13:20.200,0:13:25.250
tracking provider who has the[br]infrastructure set up, set up a campaign,
0:13:25.250,0:13:31.610
and they provide their associates a unique[br]ultrasound because with this campaign and
0:13:31.610,0:13:37.660
then pushes this become to content[br]publishers to incorporate them
0:13:37.660,0:13:43.860
incorporated into their content, depending[br]on what the advertiser advertising client
0:13:43.860,0:13:49.270
is trying to achieve. So this is step[br]three or step for a user is basically
0:13:49.270,0:13:56.950
accessing all of those content providers[br]either. This is a TV ad or a website on
0:13:56.950,0:14:03.030
the Internet and one this once this[br]content is loaded or displayed by your TV.
0:14:03.030,0:14:08.010
At the same time, the device, the devices[br]speakers are emitting the ultrasounds. And
0:14:08.010,0:14:13.580
if you have the ultrasound cross device tracking[br]framework on your phone, which is usually
0:14:13.580,0:14:18.910
listening on the background, then it picks[br]up the ultrasound and on step six, it
0:14:18.910,0:14:25.060
submits it back to the service provider,[br]which now knows that, OK, this guy has
0:14:25.060,0:14:31.700
watched this DVR or whatever it is, and[br]I'm going to add this to his profile and
0:14:31.700,0:14:38.220
push our target dates back to his device.[br]So, of course, by doing this, they're just
0:14:38.220,0:14:45.900
trying to improve, improve their[br]conversion rate and get more customers.
0:14:45.900,0:14:52.970
Another use of ultrasounds currently in[br]practice is proximity marketing, so venues
0:14:52.970,0:14:59.380
basically set up multiple, multiple[br]ultrasound meters. This is kind of fancy
0:14:59.380,0:15:05.350
name for speakers and this is kind of the[br]nice thing about the ultrasound. You just
0:15:05.350,0:15:11.470
need speakers. So they put this in[br]multiple locations in their venue, either
0:15:11.470,0:15:18.320
a supermarket or a stadium, for instance,[br]and then there is a customer up. If you're
0:15:18.320,0:15:23.310
a supermarket, there is a supermarket up.[br]If you're an NBA team, which will see
0:15:23.310,0:15:29.730
later, you have this fun application that[br]the fans of your team can download
0:15:29.730,0:15:35.080
and install on their smartphones. And then[br]once this app, this happens, listing on
0:15:35.080,0:15:40.550
the background and it picks up the[br]ultrasound and submits them back to the
0:15:40.550,0:15:47.660
company. So the main purpose of using is[br]this is basically to study in user
0:15:47.660,0:15:55.220
behavior, in user behavior, provide real[br]time notifications like, OK, you are in
0:15:55.220,0:15:59.610
this aisle on the supermarket, but if you[br]just walk two meters down, you're going to
0:15:59.610,0:16:06.170
see this product in discount. Or the third[br]point, which kind of incentivizes the
0:16:06.170,0:16:11.240
users more, is basically you're offering[br]reward points for users visiting your
0:16:11.240,0:16:17.600
store. And actually there is a product[br]doing exactly that on the market. So some
0:16:17.600,0:16:23.830
other uses are device pairing. And this[br]basically relies on the fact that
0:16:23.830,0:16:29.029
ultrasounds do not penetrate through[br]objects. So if you have a small TV, say,
0:16:29.029,0:16:36.810
with or Chromecast, for instance, they can[br]emit random PIN through ultrasound. Your
0:16:36.810,0:16:40.700
device picks it up and submits it back to[br]the device through the Internet. And now
0:16:40.700,0:16:44.410
you've proved that you are on the same[br]physical location with the with Chromecast
0:16:44.410,0:16:51.320
or whatever your TV is. Also, Google[br]recently acquired sleek login. They are
0:16:51.320,0:16:55.770
also using ultrasounds for authentication.[br]It's not entirely clear what their product
0:16:55.770,0:17:00.120
is about, though. And also you have[br]audience measurement and analytics. So
0:17:00.120,0:17:07.240
what they are doing is basically if you're[br]if you incorporate multiple beacons in the
0:17:07.240,0:17:12.169
night, then you can basically track the[br]reactions and the behavior of the users of
0:17:12.169,0:17:17.559
it, of the audience in the sense that[br]first, you know, how many people have
0:17:17.559,0:17:21.470
watched your ad a second, you know what[br]happened. So if they show it's Sanderlin
0:17:21.470,0:17:26.709
between and this, so they submit only the[br]first beacon of the two, if you have two,
0:17:26.709,0:17:34.389
then you also track their behavior. OK, so[br]we've seen all these technologies and then
0:17:34.389,0:17:40.779
we started wondering how secure is that[br]thing? Like, OK, what security measures
0:17:40.779,0:17:46.470
are there applied by companies and[br]everything? So I'm going to immediately
0:17:46.470,0:17:51.369
start with the exploitation of the[br]technology. So to do that, we just need
0:17:51.369,0:17:59.470
the computer with speakers and the Tor browser[br]and the smartphone with an ultrasound app
0:17:59.470,0:18:03.000
and a state level advisory. I'm going to[br]say more about the state level advisory
0:18:03.000,0:18:11.679
later, but just keep in mind that it's on[br]the Tor threat model, so. I have a
0:18:11.679,0:18:15.059
video of the attack. I'm going to stop it,[br]I'm going to pose it in different places
0:18:15.059,0:18:24.669
to explain some more stuff. Yeah, OK, so[br]I'm going to set up the scene before that.
0:18:24.669,0:18:28.019
So let's make the assumption that we have[br]a whistle blower that wants to leak some
0:18:28.019,0:18:34.189
documents to a journalist, but he doesn't[br]know that the journalist is working with
0:18:34.189,0:18:38.699
the government and his main intent is[br]basically to deanonymize him. So the
0:18:38.699,0:18:42.559
journalist does the following, asks the[br]whistleblower to upload the documents to a
0:18:42.559,0:18:48.360
Tor hidden service or a website that he owns.[br]And the whistleblower basically thinking
0:18:48.360,0:18:55.340
that he's safe to do that through Tor[br]loads the page. So now I'm having I have the
0:18:55.340,0:19:07.279
demo, which is exactly that implements[br]exactly that scenario. So the whistle
0:19:07.279,0:19:12.970
blower opens the Tor browser, so the setup is[br]the following, we have the phone next to
0:19:12.970,0:19:16.780
the computer. This can be up to seven[br]meters away, but for practical purposes,
0:19:16.780,0:19:21.169
it has to be next to the computer. So we[br]have the Tor browser. What are we going to do
0:19:21.169,0:19:28.749
first? For the purpose of the demo, we use[br]them smart for listening framework that's
0:19:28.749,0:19:36.529
visible to the.. to the user. This is[br]basically the demo(?). Those apps, ultrasound
0:19:36.529,0:19:40.929
cross device tracking apps run in the background,[br]so now we're setting set it on listening
0:19:40.929,0:19:46.280
mode so that it starts listening. Of[br]course, in normal framework, the user
0:19:46.280,0:19:52.570
doesn't have to do that part. But we want[br]to show that. We want to show that what's
0:19:52.570,0:20:02.570
happening. So now the whistleblower is[br]going to load the innocuous were paid,
0:20:02.570,0:20:12.980
suggested by the journalist and see what[br]happens to. OK, now we've loaded the page
0:20:12.980,0:20:20.319
and the phone is listening in reality in[br]the background, so let's see what happens.
0:20:30.589,0:20:36.320
OK, this is looks pretty bad. We have lots[br]of information about the user visiting our
0:20:36.320,0:20:43.700
hidden service. I assume you already have some[br]clues about how this happened, what the
0:20:43.700,0:20:54.850
information that we have is the following.[br]First of all. We have his IP address,
0:20:54.850,0:21:02.070
phone number. Don't call this phone[br]number, because this isn't right. The ID
0:21:02.070,0:21:10.859
is he may end his Google account email. So[br]this is enough to say and his location, of
0:21:10.859,0:21:15.389
course, and this is enough to say that we[br]essentially deanonymized him, even if we
0:21:15.389,0:21:22.340
had the IP address, that would have been[br]enough. So before I explain exactly how
0:21:22.340,0:21:26.169
the attacked work, I'm going to introduce[br]some tools that the attackers have at
0:21:26.169,0:21:32.320
their disposal. The first one is a Bitcoin[br]injection. So what you can essentially do
0:21:32.320,0:21:37.229
is basically craft your own ultrasound[br]beacons and push them to devices,
0:21:37.229,0:21:40.809
listening for beacons, and then their[br]devices are going to treat them like valid
0:21:40.809,0:21:45.160
beacons and submit them back to the[br]company's backend. And then the same
0:21:45.160,0:21:49.989
things. Basically, you can also replace[br]ultrasound beacons, meaning that you can
0:21:49.989,0:21:55.320
capture them from virus location. And this[br]is actually happening on the wild at a
0:21:55.320,0:22:04.309
large scale for a specific application.[br]And then once you capture those beacons,
0:22:04.309,0:22:11.429
you can replace them back to the company's[br]back end through the user's devices to
0:22:11.429,0:22:16.990
give you a clue. There is a company that[br]incentivizes users to visit stores by
0:22:16.990,0:22:22.720
providing them offers and end points when[br]they are visiting stores and people are
0:22:22.720,0:22:27.660
capturing the beacons and are replaying them[br]back to their devices from home. So they
0:22:27.660,0:22:30.580
are selling the beacons through the[br]Internet so that they don't have to go to
0:22:30.580,0:22:39.559
the actual stores. OK, the problem here is[br]basically that the framework is handling
0:22:39.559,0:22:43.000
every beacon. It doesn't have a way to[br]distinguish between the valid and
0:22:43.000,0:22:48.050
maliciously crafted beacons. And my favorite[br]tool for the attackers is basically a beacon
0:22:48.050,0:22:55.350
trap, which is a code snippet that[br]once you loaded, you basically reproduce
0:22:55.350,0:23:01.679
one or more inaudible beacons that the[br]attacker chose to. So this can happen in
0:23:01.679,0:23:06.759
lots of ways on the demo. I use the first[br]one. So you build a website and you have
0:23:06.759,0:23:12.189
some JavaScript there just playing the[br]ultrasounds from the back. What else you can
0:23:12.189,0:23:17.769
do is basically start crosseyed scripting[br]vulnerability. Just exploit it on any
0:23:17.769,0:23:22.929
random website and then you can inject[br]beacons to the visitors of this website
0:23:22.929,0:23:30.249
or a man-in-the-middle attacks just[br]adding or javascript snippet on that
0:23:30.249,0:23:37.779
user's traffic or they send an audio[br]message to the to the victim. So how did
0:23:37.779,0:23:41.830
Tor deanonymization attack work? It's the[br]following. So first the adversary needs to
0:23:41.830,0:23:50.049
set up, set up a campaign, and then once[br]he captures the the beacon associated with
0:23:50.049,0:23:55.540
that campaign, he builds a beacon trap and[br]essentially on step three lures, the user
0:23:55.540,0:24:00.789
to visit it. This is what the journalist[br]basically did for the whistleblower on our
0:24:00.789,0:24:05.840
scenario. Then the user loads the[br]resource. He has no idea this is possible.
0:24:05.840,0:24:12.440
And she slapped him amidst the ultrasound,[br]beacon. If you if your smartphone has such a
0:24:12.440,0:24:17.460
framework, it's going to pick it up and[br]submit it back to the provider and I don't
0:24:17.460,0:24:22.179
know about you, but when I'm using Tor,[br]I'm not connecting my phone through to the
0:24:22.179,0:24:25.509
Internet, through the Tor network. My[br]phone is connected through my normal Wi-
0:24:25.509,0:24:34.039
Fi. So now the ultrasound service provider[br]knows that the you know, this smartphone
0:24:34.039,0:24:37.690
device omitted that specific beacon. And[br]then I step seven, basically the
0:24:37.690,0:24:42.809
adversary, which is state level adversary.[br]Can simply subpoena the provider for the
0:24:42.809,0:24:48.509
AP or other identifiers, which from what[br]we've seen, they collect plenty of them.
0:24:48.509,0:24:54.519
OK, so the first two elements, we have[br]them already like the Tor browser
0:24:54.519,0:25:02.919
computer, which biggest fine smartphone[br]with ultrasound tracking enabled
0:25:02.919,0:25:08.330
framework. Fine. What about the state[br]level adversity? So we didn't have a state
0:25:08.330,0:25:13.090
level adversity handy. So what we did is[br]basically we redirected the
0:25:13.090,0:25:18.540
traffic from step six to the advertized[br]backend. And I want to stress a point
0:25:18.540,0:25:28.600
here. This is not. A long, long shot[br]assumption. So what we've seen in October
0:25:28.600,0:25:33.179
is the following. I don't know how many of[br]you realize, but AT&T was running a spy
0:25:33.179,0:25:41.029
program, a thing called Hammesfahr, and it[br]was providing paid access to governments
0:25:41.029,0:25:45.269
only with an administrative subpoena,[br]which is not doesn't even need to be
0:25:45.269,0:25:50.789
obtained by it's ads. So it's pretty easy[br]for them to get access to this kind of
0:25:50.789,0:25:55.520
data. Especially we're talking about an IP[br]address. It's not it's very easy for them
0:25:55.520,0:26:01.570
to get it. So we also came up with some[br]more attacks. First one is profile,
0:26:01.570,0:26:07.710
corruption. Advertisers really like to[br]build profiles about you, your interests
0:26:07.710,0:26:15.210
and your behavior. So what you are[br]basically doing is you can inject beacons
0:26:15.210,0:26:21.209
to other people or even to your own phone[br]and then you can malform their profile.
0:26:21.209,0:26:28.309
Exactly. The impact of this attack depends[br]on how the backend of the advertising
0:26:28.309,0:26:33.039
company and the infrastructure works, but[br]the attack is definitely possible. And
0:26:33.039,0:26:40.169
then there is information leakage attack[br]were works under a similar assumption. You
0:26:40.169,0:26:44.510
can replay Beacon's eavesdrop and replay[br]because your own phone to make your
0:26:44.510,0:26:49.519
profile similar to that of the victims.[br]And then based on how recommendation
0:26:49.519,0:26:56.299
systems work, you're very likely to get[br]similar arts and similar content with that
0:26:56.299,0:27:01.019
of the victims. So of course, this also[br]depends about exactly how the
0:27:01.019,0:27:07.369
recommendation system is implemented, but[br]it's definitely possible. OK, so we've
0:27:07.369,0:27:11.539
seen certain things that makes us think[br]that, OK, the ecosystem is not very
0:27:11.539,0:27:19.409
secure. Um, we try to find out exactly why[br]this happened. So we did a security
0:27:19.409,0:27:24.580
evaluation or we came up with four points.[br]The first one is that we came up with we
0:27:24.580,0:27:31.749
realized that the threat model is[br]inaccurate, that ultrasound, because none
0:27:31.749,0:27:39.130
of the implementations we've seen had any[br]security features. Um, they also violated
0:27:39.130,0:27:44.149
the fundamental security principle and[br]they lacked transparency when it comes
0:27:44.149,0:27:49.269
when it came to user interface. So let's[br]go through them one by one. So inaccurate
0:27:49.269,0:27:52.999
and model. Basically what they do is[br]basically they rely on the fact that
0:27:52.999,0:27:58.360
ultrasounds cannot penetrate the walls and[br]they travel up to seven meters reliably.
0:27:58.360,0:28:05.559
However, as I said, as a matter of fact,[br]they also assume that you cannot capture
0:28:05.559,0:28:10.870
and replay because because of that, that's[br]the reason, um, what what's happening in
0:28:10.870,0:28:15.029
practice, that you can get really close[br]using beacon traps. So their assumption
0:28:15.029,0:28:21.800
is not that accurate. Um, also, the[br]security capabilities of beacons are
0:28:21.800,0:28:30.129
heavily constrained by the low bandwidth[br]the channel is has the limited time that
0:28:30.129,0:28:33.580
you have to reach the users. So if someone[br]is in a supermarket, he's not going to
0:28:33.580,0:28:37.170
stop somewhere for a very long time. So[br]you have limited time and a noisy
0:28:37.170,0:28:42.440
environment. So you want a very low error[br]rate. And adding crypto to the beacons
0:28:42.440,0:28:49.139
it may not be a good idea, but it also[br]results. This also results in replay in
0:28:49.139,0:28:54.259
injection attacks being possible. Um, we[br]also hear the violation of the privilege
0:28:54.259,0:28:59.849
of, uh, sorry, the principle of least privilege.[br]So what happens is basically all these
0:28:59.849,0:29:05.110
apps need full access to the microphone.[br]And based on the way it works, it's
0:29:05.110,0:29:10.489
completely unnecessary for them to gain[br]access to the audible frequencies.
0:29:10.489,0:29:14.669
However, even if they want to, there's no[br]way to gain access only to the ultrasound
0:29:14.669,0:29:20.530
spectrum, both in Android and iOS. You[br]have to gain either access to the whole
0:29:20.530,0:29:26.629
spectrum or no access at all. So this, of[br]course, results in the first malicious
0:29:26.629,0:29:32.229
developers can at any time start using[br]their access to the microphone. And of
0:29:32.229,0:29:38.520
course, all the benign ultrasound enabled[br]apps are perceived by as malicious by the
0:29:38.520,0:29:45.399
users. And this actually will say more[br]about it later. So lack of transparency is
0:29:45.399,0:29:51.259
inclose. This is a bad combination with[br]what exactly we've seen previously,
0:29:51.259,0:29:55.919
because it that we've observed large[br]discrepancies between apps when it comes
0:29:55.919,0:30:00.879
to informing the users and also lots of[br]discrepancies when it comes to providing
0:30:00.879,0:30:06.110
opt out options. And there is a conflict[br]of interest there, because if you're a
0:30:06.110,0:30:12.600
framework developer, developer, you want[br]to advise for proper practices for your
0:30:12.600,0:30:17.960
customers, but you are not you're not[br]going to enforce them or kind of blackmail
0:30:17.960,0:30:22.499
them. Either you do it properly or you're[br]not using my framework. So there is a
0:30:22.499,0:30:27.190
conflict of interest there. So what[br]happened because of a lack of
0:30:27.190,0:30:33.289
transparency is the following. Signals 360 is[br]one of those frameworks. An NBA team
0:30:33.289,0:30:39.500
started using this in May. And then a few[br]months after there is a sue and someone
0:30:39.500,0:30:43.839
claims, you know, that thing is listening[br]in the background. And what's interesting
0:30:43.839,0:30:49.220
is on the claim, what they are saying is,[br]OK, I gave permission through the Android
0:30:49.220,0:30:54.119
permission system for them to access the[br]microphone, but it was not explained to me
0:30:54.119,0:30:58.840
exactly what they were doing. And this is[br]in close ties with what the FTC was saying
0:30:58.840,0:31:08.740
in the letter a few months ago. Also,[br]again, the same story, um, football team
0:31:08.740,0:31:14.340
starts using such a framework a few months[br]after people are complaining that they are
0:31:14.340,0:31:21.679
being eavesdropped on. Um, I think what[br]happened here is that. When the team was
0:31:21.679,0:31:25.749
playing a match, the application started[br]listening for ultrasounds, but not all
0:31:25.749,0:31:29.560
your fans are going to be in the stadium,[br]so you end up listening for ultrasounds in
0:31:29.560,0:31:37.029
a church and other places. So, yeah,[br]people were also pissed. Um, OK, just to
0:31:37.029,0:31:41.989
put it into perspective how prevalent[br]these technologies are, the ecosystem is
0:31:41.989,0:31:48.000
growing. Even though that one company[br]withdrew. There are other companies in the
0:31:48.000,0:31:54.899
ecosystem are coming up with new products[br]as well. So the number of users is
0:31:54.899,0:32:00.110
relatively low, but it's also very hard to[br]estimate right now. We could find around
0:32:00.110,0:32:05.269
10 companies offering ultrasound related[br]products and the majority of them is
0:32:05.269,0:32:09.780
gathered around proximity marketing. There[br]was only one company doing ultrasound
0:32:09.780,0:32:16.590
cross device tracking. At least we found[br]one. Um, and this is mainly due to
0:32:16.590,0:32:21.290
infrastructure complexity. It's not easy[br]to do all those things. And secondly, I
0:32:21.290,0:32:26.140
also believe that the whole backslash from[br]the security community is incentivized
0:32:26.140,0:32:32.599
other companies from joining because they[br]don't want a tarnished reputation. OK, so
0:32:32.599,0:32:36.919
we have this situation right now.[br]Companies are using ultrasound. What are
0:32:36.919,0:32:48.340
we going to do? So this was our initial[br]idea. This is what we thought first. But
0:32:48.340,0:32:54.950
we want to fix things. So we tried to come[br]up with certain steps that we need to take
0:32:54.950,0:33:02.019
to actually fix that thing and make it[br]usable, but not dangerous. Um, so we
0:33:02.019,0:33:07.240
listed what's wrong with it. We did it[br]already. We we developed some quick fixes
0:33:07.240,0:33:12.330
that I'm going to present later and medium[br]term solutions as well. And then we
0:33:12.330,0:33:16.830
started advocating for a long term changes[br]that are going to make the ecosystem
0:33:16.830,0:33:23.650
reliable. And we need the involvement from[br]the community there. Definitely. So. We
0:33:23.650,0:33:29.519
developed some short and medium term[br]solutions, um, the first one is a browser
0:33:29.519,0:33:37.890
extension, our browser extension basically[br]does the following is based on HTML5, the
0:33:37.890,0:33:45.899
Web audio API. Um, it filters all audio[br]sources and places a filter between the
0:33:45.899,0:33:51.280
audio source and the destination on the[br]Web page and filters out ultrasounds. To
0:33:51.280,0:33:55.489
do that, we use a heisel filter that[br]attenuates all frequencies above 18kHz
0:33:55.489,0:34:04.539
and it works pretty reliably. And[br]we leave all audible frequencies, intact.
0:34:04.539,0:34:10.060
But it's not going to work with[br]obsolete legacy technologies such as
0:34:10.060,0:34:16.789
flash. OK, we also have an adroit[br]permission, I think this somewhat more
0:34:16.789,0:34:22.980
medium term solution, what we did is we[br]developed a unique developed parts for the
0:34:22.980,0:34:28.810
Android permission system. This allows for[br]fine grained control over the audio channel,
0:34:28.810,0:34:35.099
basically separates the permission needed[br]for listening to audible sound and the
0:34:35.099,0:34:39.750
permission needed for listening to the[br]ultrasound spectrum. So at least we force the
0:34:39.750,0:34:44.559
applications to specifically declare that[br]they are going to listen to four
0:34:44.559,0:34:49.399
ultrasounds. And of course, users can, on[br]the latest Android versions, can also
0:34:49.399,0:34:54.369
disable this permission and it can act as[br]an opt out option if the app is not
0:34:54.369,0:35:02.899
providing it. We also initiated discussion[br]on the Turbo Tracker, but, um, we have,
0:35:02.899,0:35:09.380
um, we are advocating for some long term[br]solutions, so we really need some
0:35:09.380,0:35:15.650
standardization here. Um, let's agree on[br]ultrasound to confirm that and decide what
0:35:15.650,0:35:20.440
security features can be there. I mean, we[br]need to figure out what's technically
0:35:20.440,0:35:25.410
possible there because it's not clear. And[br]then once we have a standard, we can start
0:35:25.410,0:35:32.109
building some APIs. And the APIs are very[br]nice idea because, um, they will work as
0:35:32.109,0:35:37.250
the Bluetooth APIs work, meaning that they[br]will provide some methods to discover,
0:35:37.250,0:35:42.240
process, generate and emit the sound[br]beacons through a new API related
0:35:42.240,0:35:48.809
permission. And this means that we will[br]stop having overprivileged apps. We won't
0:35:48.809,0:35:54.310
need access to the microphone anymore,[br]which is a huge problem right now. And of
0:35:54.310,0:35:58.700
course, the applications will not be[br]considered spying anymore. And there is
0:35:58.700,0:36:03.630
also another problem that we found out[br]while we were playing with those shops.
0:36:03.630,0:36:08.240
Um, if you have a framework listening[br]through the microphone, other apps cannot
0:36:08.240,0:36:12.289
access it. So we are trying to open the[br]camera app to record the video on the app.
0:36:12.289,0:36:17.320
Camera app was crashing because the framework[br]was locking the access to the
0:36:17.320,0:36:22.349
microphone. Now we may have some[br]developers from frameworks saying, you
0:36:22.349,0:36:26.020
know, I'm not going to use your API. I'm[br]going to keep asking for access to the
0:36:26.020,0:36:34.090
microphone. But we can force them to use[br]this API if we somehow, um, by default
0:36:34.090,0:36:38.750
filter out the ultrasound frequencies[br]from the microphone and
0:36:38.750,0:36:44.640
provide the way to the user to enable them[br]on a pure application basis from his
0:36:44.640,0:36:56.200
phone. OK, so. Here's what we did, um, we[br]analyzed them, multiple ultrasound
0:36:56.200,0:37:00.329
tracking technologies, we saw what what's[br]out there in the real world and reverse
0:37:00.329,0:37:08.500
engineered such frameworks. We identified,[br]um, quite a few security shortcomings. We
0:37:08.500,0:37:16.150
introduced our attacks and proposed some,[br]um, usable countermeasures. Um, and
0:37:16.150,0:37:21.580
hopefully we initiated the discussion[br]about standardizing ultrasound because,
0:37:21.580,0:37:27.539
um, but there are still things left to do.[br]So for the application developers, please,
0:37:27.539,0:37:32.880
um, explicitly notify the users about what[br]your app is doing. Many of them would
0:37:32.880,0:37:41.150
appreciate to know that. Um, also, we need[br]to improve transparency in the data
0:37:41.150,0:37:47.150
collection process because they collecting[br]lots of data and very few information were
0:37:47.150,0:37:52.010
available about what kind of data they[br]framework's collect. Um, we also think
0:37:52.010,0:37:57.010
it's a good idea to have an opt in option[br]if it's not too much to ask, at least an
0:37:57.010,0:38:07.910
opt out and standard security practices,[br]um, as always. So framework providers
0:38:07.910,0:38:13.730
basically need to make sure that the[br]developers inform the users and also make
0:38:13.730,0:38:21.030
sure that the users consent regularly to[br]listening for because like it's not enough
0:38:21.030,0:38:25.809
if you consent once and then a month after[br]the app is still listening for ultrasound beacons
0:38:25.809,0:38:33.170
you have to periodically ask the user if it's[br]still okay to do that. Um. Ideally, every time
0:38:33.170,0:38:39.619
you are going to listen and then, of[br]course, we need to work on standardizing
0:38:39.619,0:38:43.930
ultrasound because this is going to be a[br]long process and then building the
0:38:43.930,0:38:48.430
specialized, specialized API. Hopefully[br]this is going to be easier once we have a
0:38:48.430,0:38:56.960
standard and see what kind of[br]authentication mechanisms can we have in
0:38:56.960,0:39:03.989
this kind of constrained transmission[br]channel. So..
0:39:03.989,0:39:17.149
applause
0:39:17.149,0:39:21.229
Herald: Thank you Vasilios. If you have any[br]questions, please do line up at the four
0:39:21.229,0:39:26.679
microphones here in the walkways and the[br]first question will be the front
0:39:26.679,0:39:30.959
microphone here.[br]Mic: Hello and thank you for your
0:39:30.959,0:39:35.240
presentation. And I have a couple of[br]questions to ask that are technical and
0:39:35.240,0:39:41.070
they are very related. First of all, do[br]you think that blocking out in our system
0:39:41.070,0:39:47.799
level the high frequencies for either[br]microphone or the speakers as well, a
0:39:47.799,0:39:53.070
something that is technically feasible and[br]will not put a very high latency in the
0:39:53.070,0:39:56.750
processing?[br]Vasilios: So we did that through the
0:39:56.750,0:39:59.350
permission. You are talking[br]about the smartphone right?
0:39:59.350,0:40:03.850
Mic: Yeah, basically, because you have to[br]have a real time sound and microphone
0:40:03.850,0:40:06.769
feedback.[br]Vasilios: So we did that with the
0:40:06.769,0:40:14.179
permission. And I think it's not it's not[br]to resource demanding, if that's
0:40:14.179,0:40:17.219
your question. So it's[br]definitely possible to do that.
0:40:17.219,0:40:21.820
Mic: And the second part is, so[br]there is a new market maybe for some
0:40:21.820,0:40:28.170
companies producing and microphones and[br]speakers that explicitly block out
0:40:28.170,0:40:33.860
ultrasounds, right?[br]Vasilios: Possibly. Possibly. Um, I'm not
0:40:33.860,0:40:38.690
sure if you can do this from the[br]application level. We developed parts for
0:40:38.690,0:40:43.869
the Android system. I think our first[br]approach back then was basically try to
0:40:43.869,0:40:48.130
build an app to do that from the[br]application, from the user land. And
0:40:48.130,0:40:53.100
basically, I'm not sure if you can I doubt[br]actually an Android if you can filter out
0:40:53.100,0:40:58.569
ultrasounds. But from a browser, we have[br]our extension. It works on Chrome. You can
0:40:58.569,0:41:04.250
easily use our code to do the[br]same thing on the Firefox.
0:41:04.250,0:41:06.600
Mic: Thanks.[br]Herald: The next question is from the
0:41:06.600,0:41:10.460
front right microphone.[br]Mic: Thank you for your talk. I have a
0:41:10.460,0:41:15.220
question about the attack requirements[br]against the whistleblower using Tor.
0:41:15.220,0:41:23.730
I'm curious, the attacker has access to[br]the app on the smartphone and also access
0:41:23.730,0:41:32.790
to the smartphone microphone. Wouldn't the[br]attacker then be able to just listen in on
0:41:32.790,0:41:37.340
the conversation of the whistleblower and[br]thereby identify him?
0:41:37.340,0:41:40.670
Vasilios: Yeah, absolutely. Absolutely.[br]It's a major problem. The problem is that
0:41:40.670,0:41:47.760
they have access to the microphone. So[br]this is very this is very real and it's
0:41:47.760,0:41:52.870
not going to be resolved even if we had[br]access only to the ultrasound spectrum.
0:41:52.870,0:41:57.359
What we're saying is basically, if we only[br]had access to the ultrasound spectrum,
0:41:57.359,0:42:04.820
you're still uhm you are still vulnerable[br]to these attacks unless you incorporate
0:42:04.820,0:42:10.420
some crypto mechanisms that prevent these[br]things from happening. Is this your
0:42:10.420,0:42:15.900
question or?[br]Mic: Um, well, I can still pull off the
0:42:15.900,0:42:19.350
same attack if I don't[br]use ultrasound right?
0:42:19.350,0:42:21.540
Vasilios: Through the audible spectrum?[br]Mic: Yes,
0:42:21.540,0:42:28.990
Vasilios: You can absolutely do. There is[br]one company doing tracking in the audible
0:42:28.990,0:42:35.560
spectrum. This is much harder to mitigate.[br]We're looking into it about ways, but
0:42:35.560,0:42:39.109
there are so many ways to incorporate[br]beacons in the audible spectrum. The thing
0:42:39.109,0:42:47.240
is that there is not much of an ecosystem[br]in this area right now that so you don't
0:42:47.240,0:42:52.640
have lots of frameworks are there as many[br]as you have for ultrasounds.
0:42:52.640,0:42:56.219
Mic: Thank you.[br]Herald: Our next question will be from
0:42:56.219,0:43:01.349
the Internet via our signal angel[br]Signal Angel: $Username is asking, have
0:43:01.349,0:43:08.170
you heard about exploiting parricide[br]ultrasound emiters like IC component's?
0:43:08.170,0:43:10.230
Vasilios: Can you please[br]repeat the question?
0:43:10.230,0:43:14.600
Signal Angel: Yes, sure. The question is,[br]can you use other components on the main
0:43:14.600,0:43:23.740
board or maybe the hard disk to emit[br]ultrasounds and then broadcast the beacon
0:43:23.740,0:43:28.960
via this?[br]Vailios: Uh. So that's a very that's a
0:43:28.960,0:43:35.450
very good question. The answer is I don't[br]know, possibly, and it's very scary. Um,
0:43:35.450,0:43:42.489
hopefully not, but I doubt it. I think[br]there should be a way to do it. Um, maybe
0:43:42.489,0:43:47.200
the problem is that you cannot do this[br]completely in a completely inaudible way.
0:43:47.200,0:43:51.760
Like you may be able to meet ultrasounds,[br]but you will also emit some sort of sound
0:43:51.760,0:43:58.010
in the audible spectrum so that the user[br]will know that something is going on.
0:43:58.010,0:44:02.520
Herald: The next question[br]from the left microphone.
0:44:02.520,0:44:06.559
Mic: Thank you for your talk and[br]especially thanks for the research. So,
0:44:06.559,0:44:12.919
uh, do you know of any framework's or, uh,[br]STKs that cash the beacon's they find?
0:44:12.919,0:44:17.760
Because for my use case, I my phone was[br]mostly offline. I just make it online when
0:44:17.760,0:44:21.950
I have to check[br]something. So I'm not that concerned. But
0:44:21.950,0:44:24.660
you do you know, if they like cash the[br]beacons and and submit them later
0:44:24.660,0:44:32.250
something like this. Of course they do.[br]I'm not surprised, unfortunately. Yeah.
0:44:32.250,0:44:39.450
Thanks. Next question from the rear.[br]Right. Oh, what is the data rate? You can
0:44:39.450,0:44:44.119
send in the ultrasound. Very good[br]question. And it's totally relevant to the
0:44:44.119,0:44:51.250
cryptographic mechanisms we want to[br]incorporate from our experiments. Um, in
0:44:51.250,0:44:58.480
four seconds you can basically send like[br]five to six alphabet characters if you're
0:44:58.480,0:45:04.500
willing to kind of reduce the range a lot[br]less in less than seven meters, you may be
0:45:04.500,0:45:11.970
able to send more. But the standard is not[br]very robust in this sense. But these
0:45:11.970,0:45:16.260
experiments were done with this kind of[br]naive encoding that most of the companies
0:45:16.260,0:45:22.930
are using. So if you do the encoding in a[br]very smart way, possibly you can increase
0:45:22.930,0:45:29.329
that. And a small second part, what's the[br]energy consumption on the phone if that is
0:45:29.329,0:45:35.110
running all the time? Wouldn't I detect[br]that? So it's not, uh, it's not good. We
0:45:35.110,0:45:38.890
saw that it was draining the battery and[br]actually in the comments, I don't know if
0:45:38.890,0:45:44.500
I had that comment here. Some people were[br]complaining that, um, I tried and it was
0:45:44.500,0:45:53.029
draining my battery. And, um, there is an[br]impact. Absolutely. Amazon and Google Nest
0:45:53.029,0:45:57.710
and all the other parts, aren't you more[br]worried about that? You know, the always
0:45:57.710,0:46:02.400
listening thing from Google and Amazon and[br]everyone is coming up with some something
0:46:02.400,0:46:10.130
like that that's always on. And so that[br]it's kind of strange because a user's
0:46:10.130,0:46:18.369
consent. But at the same time, they don't[br]completely understand. So there is a gray
0:46:18.369,0:46:22.670
line there, like you can say that the[br]users, OK, you consented to that up,
0:46:22.670,0:46:28.549
starting with your with your phone and[br]listening on the background. But at the
0:46:28.549,0:46:34.869
same time, the users don't have the best[br]understanding. Always. Thank you. Next
0:46:34.869,0:46:39.430
question from the front left microphone[br]first. Thank you for the talk. I would be
0:46:39.430,0:46:43.809
interested in how you selected your real[br]world applications and how many you found
0:46:43.809,0:46:51.119
that already use such a framework. So what[br]was the first part of the question, how
0:46:51.119,0:46:56.790
you selected your real world applications[br]from the marketplace staff if you had any.
0:46:56.790,0:47:04.109
So we're trying to do a systematic scan of[br]the whole market, but it's not easy. So we
0:47:04.109,0:47:09.440
not able to do that. There are resources[br]on the Internet. Luckily, the companies
0:47:09.440,0:47:15.710
need to advertise their product. So they[br]basically publish press releases saying,
0:47:15.710,0:47:22.000
you know, this NBA team started using our[br]product. We did some sort of scanning
0:47:22.000,0:47:27.890
through alternative datasets, but[br]definitely we don't have an exhaustive
0:47:27.890,0:47:33.049
list of applications. What I can say,[br]though, is that there are applications
0:47:33.049,0:47:40.250
with. Using such frameworks with nearly up[br]to, if I remember correctly, up to one
0:47:40.250,0:47:49.160
million installations. One notable[br]example, OK, I'm not entirely sure what I
0:47:49.160,0:47:55.380
wanted, but up to a million we definitely[br]saw. OK, thanks. Do we have more questions
0:47:55.380,0:48:02.500
from the Internet? Yes, E.F. is asking, is[br]he aware of or are you aware sorry? Are
0:48:02.500,0:48:05.569
you aware of any framework available by[br]Google or Apple? In other words, how do we
0:48:05.569,0:48:11.960
know that it's not, for instance,[br]seriously snitching on us? How do we know
0:48:11.960,0:48:19.910
that it's not true? It's not serious. Some[br]maybe Aleksa snitching on us. We don't. I
0:48:19.910,0:48:24.160
think that's a that's a very large[br]discussion. Right. So is the same problem
0:48:24.160,0:48:34.059
that these companies are having? Because[br]if I go back here, basically the users are
0:48:34.059,0:48:43.690
accusing them of eavesdropping. Especially[br]here from reverse engineering those
0:48:43.690,0:48:49.869
frameworks, we couldn't find any such[br]activity, but again, it's very hard to
0:48:49.869,0:48:54.259
convince the users that you are listening[br]to the ultrasound spectrum. You if you're
0:48:54.259,0:48:59.769
accessing the whole audible frequencies[br]through the microphone, you're going to or
0:48:59.769,0:49:04.119
you will always find yourself in this[br]position. So I guess it's the same problem
0:49:04.119,0:49:09.339
that Alexa has from Amazon. But in this[br]case, you can actually solve it by
0:49:09.339,0:49:15.410
constraining the spectrum that you gain[br]access to. Next question from the front
0:49:15.410,0:49:21.069
left microphone, please. Has anybody done[br]an audible demonstration off these beacons
0:49:21.069,0:49:26.230
bypassed by transposing them down an[br]octave or two, I think might be useful for
0:49:26.230,0:49:34.089
for or your talk to something like that.[br]So you mean a demo, but using audible
0:49:34.089,0:49:40.630
frequencies? Essentially, there is this[br]one company, but they are being pretty to
0:49:40.630,0:49:44.869
all of these companies are being pretty[br]secretive with their technology. So they
0:49:44.869,0:49:51.430
publish what's needed for marketing[br]purposes like accuracy sometimes remains
0:49:51.430,0:49:57.390
very limited technical details. But apart[br]from these, you have to get your hands on
0:49:57.390,0:50:04.829
the framework somehow and analyze it[br]yourself. So in this kind of overview we
0:50:04.829,0:50:08.130
need for the ecosystem, we had to do[br]everything by ourselves. There was no
0:50:08.130,0:50:15.789
resources out there were very limited, um,[br]or recording it and playing it down and
0:50:15.789,0:50:23.290
transposing it yourself, if you know where[br]as a beacon of. Possibly I'm not I'm not
0:50:23.290,0:50:31.779
entirely sure you could. Yeah. Another[br]question from our signal, angel mestas,
0:50:31.779,0:50:37.789
again asking, um, would it be possible,[br]even if you have a low pass filter to use,
0:50:37.789,0:50:44.810
uh, for instance, the cost effect and high[br]cost effect to transmit the beacon via
0:50:44.810,0:50:53.900
ultrasound, but in a regime which is as[br]free for the app? So it's basically the
0:50:53.900,0:50:59.799
question, can I somehow, via Aliasing USA[br]address on signal to make a normal signal
0:50:59.799,0:51:08.319
out of it? Possibly, I don't know. I think[br]you are much more creative than I am, so
0:51:08.319,0:51:16.819
maybe I should add more bullet points on[br]this controversialist here. Apparently,
0:51:16.819,0:51:23.150
there are many more ways to do this,[br]possibly like hardware missions. This one
0:51:23.150,0:51:29.619
sounds like a good idea, too. So next[br]question from the real right microphone. I
0:51:29.619,0:51:33.559
apologize if you explain the story they[br]didn't understand, but is is sort of
0:51:33.559,0:51:38.819
drowning out the signals, like jamming.[br]They just broadcasting white noise in that
0:51:38.819,0:51:43.810
spectrum, an effective countermeasure. And[br]as a follow up, if it is, would it
0:51:43.810,0:51:56.750
terrorize my dog? So absolutely, it's[br]effective. I mean, this it works up to
0:51:56.750,0:52:01.770
seven meters, but we're not saying it's[br]not fragile, so you can do that, but it's
0:52:01.770,0:52:05.829
noise pollution. And my dog, I don't think[br]it was happy. I did it for a very limited
0:52:05.829,0:52:10.280
time. I could see her ears moving, but I[br]don't think she would appreciate it if I
0:52:10.280,0:52:16.720
had the device at home doing this all the[br]time. Do we have any more questions from
0:52:16.720,0:52:22.460
the Internet? Yes, EULEX is asking to what[br]extent could we use these for our own
0:52:22.460,0:52:26.559
needs? For example, people in repressive[br]situations, for example, activists could
0:52:26.559,0:52:30.630
use it to transmit secret encrypted[br]messages. Are there any efforts in this
0:52:30.630,0:52:40.829
area? Yes, there are. People are[br]developing ultrasound modems. I think
0:52:40.829,0:52:51.030
there is even a tag on it. And yes, of[br]course there is. So I would say, yes, I'm
0:52:51.030,0:52:57.029
not entirely sure about the capabilities[br]of this channel in terms of bandwidth, but
0:52:57.029,0:53:01.890
this is why we we are not advocating to[br]kill the technology just to make it secure
0:53:01.890,0:53:06.900
and know its limitations. So you can do[br]good stuff with it. And this is what we
0:53:06.900,0:53:13.720
want. Next question from the Rio, right?[br]Yeah, I'm wondering if you could transfer
0:53:13.720,0:53:19.859
that technique from the ultrasound range[br]also to the Audible Range, for example, by
0:53:19.859,0:53:26.550
using watermarks, audio, watermarks, and[br]then, well, your permission thingy with
0:53:26.550,0:53:31.740
the ultrasound permissions would be[br]ineffective and you could also track the
0:53:31.740,0:53:37.810
user. How about this? Is it possible audio[br]watermarks in the audible spectrum? Yeah,
0:53:37.810,0:53:42.900
it's absolutely possible. Um, our[br]countermeasures are not effective against
0:53:42.900,0:53:50.490
this. Um, it's just that there is from our[br]research, just one company doing this. Uh,
0:53:50.490,0:53:57.119
so this one, um, I think technically it's[br]a bit more challenging to do that.
0:53:57.119,0:54:02.809
Instead, they're just admitting they are[br]doing it in a very basic way. So
0:54:02.809,0:54:08.480
hopefully, um, if there is a clear way to[br]do it through ultrasounds, they are not
0:54:08.480,0:54:15.400
going to reside reside in the audible[br]spectrum. But our countermeasures are not
0:54:15.400,0:54:22.640
effective against the audible. Um.[br]Watermarks. Yeah, thanks, next question
0:54:22.640,0:54:28.960
from the front left microphone. I've heard[br]that I don't think it's very credible, but
0:54:28.960,0:54:34.079
I've heard that there is some sound on[br]this sub sound spectrum. There were some
0:54:34.079,0:54:40.700
experiments showing that they can[br]influence our mood, the mood of humans. Is
0:54:40.700,0:54:47.900
there any relevant information about how[br]ultrasounds could affect us? So without
0:54:47.900,0:54:54.580
being an expert in this particular area?[br]I've read similar articles when I was
0:54:54.580,0:54:59.190
looking into it. I can tell you it's very[br]annoying, especially if you're listening
0:54:59.190,0:55:05.680
to it through headphones. You cannot[br]really hear the sound, but you can if
0:55:05.680,0:55:11.599
you're using headphones, you can feel the[br]pressure. So if I don't know what kind of
0:55:11.599,0:55:19.809
medical condition you may develop, but you[br]won't be very sane after. Do we have any
0:55:19.809,0:55:27.289
more questions? Yes. One further question,[br]um, would it be possible to, um, use a
0:55:27.289,0:55:33.999
charming solution to get rid of the[br]signals? Yes, but you you're going to
0:55:33.999,0:55:38.450
follow the you know, it's going to result[br]in noise pollution, but if you are being
0:55:38.450,0:55:46.690
paranoid about it, yes, it's and it's, I[br]think, a straightforward thing to do. Any
0:55:46.690,0:55:53.330
more questions? One more on the front left[br]microphone. Know, you said that physical
0:55:53.330,0:55:59.049
objects will block the ultrasound. How[br]solid do the physical objects need to be?
0:55:59.049,0:56:04.680
So, for example, does my pocket block the[br]ultrasound and thus prevent my phone to
0:56:04.680,0:56:11.579
call the environment and vice versa? OK,[br]well, that's a good question. I don't
0:56:11.579,0:56:16.529
think that clothes can actually do that[br]unless it's very thick. Thin girls
0:56:16.529,0:56:27.190
definitely block it. Um. Thick glass, I[br]would say it reduce the transmission rate,
0:56:27.190,0:56:35.559
the signal to noise ratio by a lot, but it[br]could go through it, so. You need
0:56:35.559,0:56:42.690
something quite concrete, metal. I don't[br]think it goes through it. So are there any
0:56:42.690,0:56:48.160
more? Doesn't look like it, maybe, maybe[br]one more sorry. Oh, good signal, good bye.
0:56:48.160,0:57:02.350
Kitty is asking, could you name or compile[br]a list of tracking programs and apps? So.
0:57:02.350,0:57:07.410
That's a good question. We're trying to[br]make an exhaustive list and try to resolve
0:57:07.410,0:57:16.529
this in a systematic way. I've already[br]listed two Macenta frameworks. One is the
0:57:16.529,0:57:20.160
Silverbush one three actually. One is the[br]Silver Paswan. There is another one used
0:57:20.160,0:57:32.940
by single 360. So developed the signal[br]360, and then there is a listener one.
0:57:32.940,0:57:39.609
These are very popular. Um, and then its[br]developer is incorporating them into their
0:57:39.609,0:57:48.749
applications in different ways, offering[br]varying levels of transparency for the
0:57:48.749,0:57:54.339
users. So it's better if you start knowing[br]what the frameworks are and then trying to
0:57:54.339,0:57:59.039
find the applications using them, because[br]you know what? You're looking in the code
0:57:59.039,0:58:06.280
and you can develop some queries and[br]enabling you to access an ability to to
0:58:06.280,0:58:13.509
track which applications are using them.[br]What what we observed for Silverbush is
0:58:13.509,0:58:18.820
basically after the company announced that[br]they are moving out of the US and because
0:58:18.820,0:58:24.390
of the whole backslash, maybe even before[br]that, um, companies started to drop the
0:58:24.390,0:58:30.109
framework. So all their versions had the[br]framework, but they are not using it
0:58:30.109,0:58:52.549
anymore. I think that's it. Thank you very[br]much, Vasilios Lovelady's.
0:58:52.549,0:59:02.932
Subtitles created by c3subtitles.de[br]in the year 2021. Join, and help us!