WEBVTT
00:00:00.000 --> 00:00:13.850
preroll music
00:00:13.850 --> 00:00:18.720
Vasilios: Hello, everyone, thanks for coming
today. I'm going to introduce the ultrasound
00:00:18.720 --> 00:00:24.500
ecosystem, which is an exotic and kind of
little known ecosystem. So I would like to
00:00:24.500 --> 00:00:29.820
start with a short story about the
product, which is also our motivation for
00:00:29.820 --> 00:00:39.870
this work. So some time ago, there was a
product that worked in the ultrasound
00:00:39.870 --> 00:00:44.450
spectrum that cannot be perceived by
humans. And the product was actually an
00:00:44.450 --> 00:00:50.700
interesting idea. It was very promising
and everything, but it also had a fatal
00:00:50.700 --> 00:00:56.730
flaw. So now that I've done this
introduction, I would like to tell you
00:00:56.730 --> 00:01:01.350
more about the story of the product and
how it came to be and what was it? What
00:01:01.350 --> 00:01:08.150
was its lifecycle. So 2012, a company
called SilverPush was a startup in
00:01:08.150 --> 00:01:14.810
India. It was founded there and they had
this ultrasound device tracking product.
00:01:14.810 --> 00:01:18.770
I'll go more into the technical details
later. So for a couple of years, they were
00:01:18.770 --> 00:01:26.789
working on that product. And it wasn't
until 2014 that they basically got some
00:01:26.789 --> 00:01:32.769
serious funding from a venture center or
other angel investors for millions. So in
00:01:32.769 --> 00:01:38.840
2014, they also got a few months after
they got funded. They also got some press
00:01:38.840 --> 00:01:44.940
coverage about the product and they got
some pretty good reviews on their
00:01:44.940 --> 00:01:49.260
newspapers and articles about what the
product could do. And at the same time,
00:01:49.260 --> 00:01:52.950
they were doing what most of the companies
are doing, like publishing patents about
00:01:52.950 --> 00:02:00.029
their technology and everything. So things
later started to go like year after year
00:02:00.029 --> 00:02:06.499
and half maybe started to go not that well
for them. The security community noticed
00:02:06.499 --> 00:02:10.759
and there was some press coverage about
the product that was not so positive
00:02:10.759 --> 00:02:19.250
anymore. So this is one of the very first
emails that appear on the Web regarding
00:02:19.250 --> 00:02:24.709
the product. So it's from a W3C
working group. So a researcher there is
00:02:24.709 --> 00:02:31.220
basically. Notifying the other members of
the group that, OK, there is this product,
00:02:31.220 --> 00:02:36.349
maybe there are transparency issues, and
certainly the users are not aware of
00:02:36.349 --> 00:02:42.489
what exactly is going on there. So let's
keep an eye on it. And so this was a very
00:02:42.489 --> 00:02:48.370
one of the very first things published
about the product from the privacy and
00:02:48.370 --> 00:02:54.470
security perspective. So what happened
then was the press took notice and they
00:02:54.470 --> 00:03:01.379
got all those headlines urging users to be
very careful. And, oh, this is a this is
00:03:01.379 --> 00:03:08.769
evil, take care. People are eavesdropping
on you and stuff. So, of course, this led
00:03:08.769 --> 00:03:14.950
also on the FTC to take action. They
organized a workshop on cross device tracking
00:03:14.950 --> 00:03:22.060
in general, I think, and they made specific
mentions for ultrasound cross device tracking
00:03:22.060 --> 00:03:28.170
don't worry if you're not familiar with this terms,
I'm going to define everything later. So
00:03:28.170 --> 00:03:33.390
what basically they were saying is
transparency issues. How do how do we
00:03:33.390 --> 00:03:39.860
protect ourselves? How is that thing
working? So, then the users, of course,
00:03:39.860 --> 00:03:43.689
started to react. And there were like many
people who were unhappy, they were
00:03:43.689 --> 00:03:48.430
complaining, what is this? I don't want
that thing. So people were actually
00:03:48.430 --> 00:03:53.479
suggesting solutions and the solutions
that were making sense. And of course, you
00:03:53.479 --> 00:04:01.260
have always the users that are completely
immune to what you have there. So what
00:04:01.260 --> 00:04:10.549
happened then is like five months after
the FTC took much more serious action
00:04:10.549 --> 00:04:16.000
regarding this specific product. So it
sent a letter to all the developers. And
00:04:16.000 --> 00:04:19.780
the letter was essentially saying, you
know, you're using this framework in
00:04:19.780 --> 00:04:27.160
Europe. We've seen it in Google Play
store. It's not enough that you are asking
00:04:27.160 --> 00:04:31.380
for the microphone permission. You should
let the users know that you are tracking
00:04:31.380 --> 00:04:36.330
them if you are doing so. Otherwise, you
are violating rule X, Y, Z, and you're not
00:04:36.330 --> 00:04:42.590
allowed to do that. So this was pretty
serious, I would say. And what happened
00:04:42.590 --> 00:04:46.610
next is basically the company withdrew
from the US market and said, you know, we
00:04:46.610 --> 00:04:51.040
have nothing to do with the U.S. market
and this product is not active there. You
00:04:51.040 --> 00:04:57.320
shouldn't be concerned. So end of story
like the product is not out there in the
00:04:57.320 --> 00:05:03.660
US at least anymore. Are we safe? So it
seemed to us that it was assumed that this
00:05:03.660 --> 00:05:10.030
was an isolated security incident. And to
be fair, very little became known about
00:05:10.030 --> 00:05:15.090
the technology. At this point. The press
moved on to other hot topics at the time,
00:05:15.090 --> 00:05:20.950
people went quiet, like if people are not
using it, it's fine. So everyone
00:05:20.950 --> 00:05:25.840
seemed happy. But we're curious people. So
we had lots of questions that were not
00:05:25.840 --> 00:05:35.040
answered. So our main questions was like
why they were using ultrasounds. We'll see
00:05:35.040 --> 00:05:41.620
that what they are doing, you can do with
our technologies, how such frameworks
00:05:41.620 --> 00:05:47.120
work. We had no idea there was no coverage
or nothing about it. The technical,
00:05:47.120 --> 00:05:53.340
technically speaking, out there, are there
other such products there? Because we were
00:05:53.340 --> 00:05:59.410
aware of one. Everyone on all the articles
was referring to that one product, but we
00:05:59.410 --> 00:06:03.210
were not sure if there are others doing
the same thing. And of course, we were
00:06:03.210 --> 00:06:07.870
looking for a report about the whole
ecosystem and how it works. And there was
00:06:07.870 --> 00:06:13.290
nothing. So what do you do then if if
there are no technical resources?
00:06:13.290 --> 00:06:19.740
Basically, we decided to do our own
research and come up with this report that
00:06:19.740 --> 00:06:24.980
we were lacking. So we're done with
motivation so far. We were pretty pumped
00:06:24.980 --> 00:06:29.891
up about looking into it. OK, what's
there? The rest of the presentation will
00:06:29.891 --> 00:06:34.010
go as follows. Like first I'm going to
introduce ultrasound tracking and other
00:06:34.010 --> 00:06:40.620
terminology, then I'm going to go on with
the attack details. And indeed, we have an
00:06:40.620 --> 00:06:47.610
attack again against the Tor browser. Then
we're doing a formal security analysis of
00:06:47.610 --> 00:06:53.350
the ecosystem and try to pinpoint the
things that went wrong. And then we'll try
00:06:53.350 --> 00:07:00.470
to introduce our countermeasures and
advocate for proper practices. So to begin
00:07:00.470 --> 00:07:06.940
with, I'm Vasilis. I've done this work
with other curious people. These are
00:07:06.940 --> 00:07:13.440 line:1
showing how Yanick Fratantonio, Christopher
Kruegel and Giovanni Vigna from UCSB and also
00:07:13.440 --> 00:07:19.280 line:1
Federico Maggi from Polytechnical
Damilola. Let's now start with the
00:07:19.280 --> 00:07:26.340
ecosystem, so apparently ultrasounds are
used in a lot of places and they can be
00:07:26.340 --> 00:07:30.920
utilized for different purposes, some of
them are cross device tracking that are
00:07:30.920 --> 00:07:36.770
referred already to audience analytics,
synchronized content, proximity, marketing
00:07:36.770 --> 00:07:41.460
and device pairing. You can do some other
things, but you will see them later. So to
00:07:41.460 --> 00:07:48.920
begin with what cross device tracking is,
cross device tracking is basically the holy
00:07:48.920 --> 00:07:53.630
grail for marketers right now because
you're using your multiple devices
00:07:53.630 --> 00:07:57.990
smartphone, laptop, computer, maybe your
TV and to them, your appear as different
00:07:57.990 --> 00:08:02.330
people. And they all want to be able to
link to link those devices to know that
00:08:02.330 --> 00:08:06.920
you're the same person so that they can
build their profiles more accurately. So,
00:08:06.920 --> 00:08:13.300
for instance, if you're watching an ad on
the TV, they want to be able to know that
00:08:13.300 --> 00:08:19.720
it's you so that they can push relevant
ads from your smartphone or follow up ads.
00:08:19.720 --> 00:08:27.860
Um. So this is employed by major
advertising networks, and there are two
00:08:27.860 --> 00:08:32.780
ways to do it, either deterministically or
probabilistically, that deterministic
00:08:32.780 --> 00:08:38.780
approach is much more reliable. You get
100 percent accuracy and works as follows.
00:08:38.780 --> 00:08:43.200
If you are Facebook, the users are heavily
incentivized to log in from all their
00:08:43.200 --> 00:08:49.220
devices. So what happens is that. You can
immediately know that, OK, this user has
00:08:49.220 --> 00:08:55.191
these three devices and I can put relevant
content to all of them. However, if you
00:08:55.191 --> 00:08:58.910
are not Facebook or Google you, it's much
more unlikely that the users would want to
00:08:58.910 --> 00:09:03.580
log into your platform from their
different devices. So you have to look for
00:09:03.580 --> 00:09:09.970
alternatives. And one tool to come up with
those alternatives are ultrasound beacons.
00:09:09.970 --> 00:09:18.480
So, um, ultrasound tracking products are
using ultrasound because they may sound
00:09:18.480 --> 00:09:22.590
exotic, but basically there they are. What
they are doing is they are encoding a
00:09:22.590 --> 00:09:29.460
sequence of symbols, um, in a very high
frequency that it's inaudible by humans.
00:09:29.460 --> 00:09:35.520
That's the first key feature. The second one
is they can be emitted by most commercial
00:09:35.520 --> 00:09:39.400
speakers and they can be captured by most
commercial microphones, for instance,
00:09:39.400 --> 00:09:48.620
found on your smartphone. So the technical
details are the following. I know there
00:09:48.620 --> 00:09:54.380
are a lot of experts in these kinds of
things here, so I'm averaging out what how
00:09:54.380 --> 00:09:57.590
the companies are doing it right now. I'm
not saying that this is the best way to do
00:09:57.590 --> 00:10:01.520
it, but this is more or less what they're
doing. Of course, they have patents, so
00:10:01.520 --> 00:10:06.230
each one of them is doing a slightly
different thing so they don't overlap.
00:10:06.230 --> 00:10:13.330
They're using the near ultrasound spectrum
between the eight eight kilohertz and 20
00:10:13.330 --> 00:10:18.940
kilohertz, which is inaudible by usually
by adults. They divide it in smaller
00:10:18.940 --> 00:10:27.380
chunks. So if you divide it in chunks that
have size of 75 Hertz, you get 26, about 26
00:10:27.380 --> 00:10:33.920
chunks, and then you can assign letter of
the alphabet on each one of them. And then
00:10:33.920 --> 00:10:38.109
what they are doing is usually within four
to five seconds. They emit sequences of
00:10:38.109 --> 00:10:45.410
characters. Usually they contain for four
to six characters in there, and they use
00:10:45.410 --> 00:10:51.670
it to incorporate a unique ID
corresponding to their source, they attach
00:10:51.670 --> 00:10:56.710
the beacon to. So there is no ultrasound
beacon standard, as I said previously, but
00:10:56.710 --> 00:11:00.450
there are lots of patents, so each one of
them is doing a slightly different thing.
00:11:00.450 --> 00:11:06.350
But this is a basic principle. We did some
experiments and we found out that within
00:11:06.350 --> 00:11:13.880
seven meters, you get pretty good accuracy
in low error rate. So of course, this depends
00:11:13.880 --> 00:11:20.250
exactly how you encode things. But with
applications found on Google Play, this
00:11:20.250 --> 00:11:24.990
worked up to seven meters. Um, we couldn't
find computer speakers that were not able
00:11:24.990 --> 00:11:33.310
to emit near ultrasound frequencies and
work with this technology and.. we this is
00:11:33.310 --> 00:11:36.590
pretty known for this kind of frequencies,
they cannot penetrate through physical
00:11:36.590 --> 00:11:41.000
objects, but this is not a problem for
their purposes. And we did some
00:11:41.000 --> 00:11:46.720
experiments with our research assistant
and we can say that they are audible by
00:11:46.720 --> 00:11:54.420
animals. So if you combine cross device
tracking and ultrasound because you get
00:11:54.420 --> 00:12:02.350
ultrasound cross device tracking. So now what
you can do with this and this is this is a
00:12:02.350 --> 00:12:07.320
pretty good idea, actually, because it
offers high accuracy, you don't ask the
00:12:07.320 --> 00:12:16.720
users to log in, which is very high, very
demanding thing to ask for. You can embed
00:12:16.720 --> 00:12:22.860
those beacons in websites or TV ads, and
this technology, however, requires some
00:12:22.860 --> 00:12:26.210
sort of sophisticated backend
infrastructure. We're going to see more
00:12:26.210 --> 00:12:30.260
about it later. And you also need the
network of publishers who are willing to
00:12:30.260 --> 00:12:36.740
incorporate incorporate beacons in their
content, whatever this content is. And
00:12:36.740 --> 00:12:41.250
then, of course, you need an ultrasound
cross device tracking framework that is going
00:12:41.250 --> 00:12:47.080
to run on the user's mobile device, a
smartphone. So these frameworks are
00:12:47.080 --> 00:12:52.680
essentially and as the advertising SDK is the
key that the developers can use to display
00:12:52.680 --> 00:12:57.060
ads on their free apps. So it's not that
the developers are going to incorporate
00:12:57.060 --> 00:13:04.491
the ultrasound framework is going to
incorporate an advertising SDK with
00:13:04.491 --> 00:13:09.610
varying degrees of understanding of what
it does. So here is how ultrasound cross device
00:13:09.610 --> 00:13:15.740
tracking works. On step one, basically, we
have the advertising client. He just wants
00:13:15.740 --> 00:13:20.200
to advertise, advertises his products. He
goes to the ultrasound cross device
00:13:20.200 --> 00:13:25.250
tracking provider who has the
infrastructure set up, set up a campaign,
00:13:25.250 --> 00:13:31.610
and they provide their associates a unique
ultrasound because with this campaign and
00:13:31.610 --> 00:13:37.660
then pushes this become to content
publishers to incorporate them
00:13:37.660 --> 00:13:43.860
incorporated into their content, depending
on what the advertiser advertising client
00:13:43.860 --> 00:13:49.270
is trying to achieve. So this is step
three or step for a user is basically
00:13:49.270 --> 00:13:56.950
accessing all of those content providers
either. This is a TV ad or a website on
00:13:56.950 --> 00:14:03.030
the Internet and one this once this
content is loaded or displayed by your TV.
00:14:03.030 --> 00:14:08.010
At the same time, the device, the devices
speakers are emitting the ultrasounds. And
00:14:08.010 --> 00:14:13.580
if you have the ultrasound cross device tracking
framework on your phone, which is usually
00:14:13.580 --> 00:14:18.910
listening on the background, then it picks
up the ultrasound and on step six, it
00:14:18.910 --> 00:14:25.060
submits it back to the service provider,
which now knows that, OK, this guy has
00:14:25.060 --> 00:14:31.700
watched this DVR or whatever it is, and
I'm going to add this to his profile and
00:14:31.700 --> 00:14:38.220
push our target dates back to his device.
So, of course, by doing this, they're just
00:14:38.220 --> 00:14:45.900
trying to improve, improve their
conversion rate and get more customers.
00:14:45.900 --> 00:14:52.970
Another use of ultrasounds currently in
practice is proximity marketing, so venues
00:14:52.970 --> 00:14:59.380
basically set up multiple, multiple
ultrasound meters. This is kind of fancy
00:14:59.380 --> 00:15:05.350
name for speakers and this is kind of the
nice thing about the ultrasound. You just
00:15:05.350 --> 00:15:11.470
need speakers. So they put this in
multiple locations in their venue, either
00:15:11.470 --> 00:15:18.320
a supermarket or a stadium, for instance,
and then there is a customer up. If you're
00:15:18.320 --> 00:15:23.310
a supermarket, there is a supermarket up.
If you're an NBA team, which will see
00:15:23.310 --> 00:15:29.730
later, you have this fun application that
the fans of your team can download
00:15:29.730 --> 00:15:35.080
and install on their smartphones. And then
once this app, this happens, listing on
00:15:35.080 --> 00:15:40.550
the background and it picks up the
ultrasound and submits them back to the
00:15:40.550 --> 00:15:47.660
company. So the main purpose of using is
this is basically to study in user
00:15:47.660 --> 00:15:55.220
behavior, in user behavior, provide real
time notifications like, OK, you are in
00:15:55.220 --> 00:15:59.610
this aisle on the supermarket, but if you
just walk two meters down, you're going to
00:15:59.610 --> 00:16:06.170
see this product in discount. Or the third
point, which kind of incentivizes the
00:16:06.170 --> 00:16:11.240
users more, is basically you're offering
reward points for users visiting your
00:16:11.240 --> 00:16:17.600
store. And actually there is a product
doing exactly that on the market. So some
00:16:17.600 --> 00:16:23.830
other uses are device pairing. And this
basically relies on the fact that
00:16:23.830 --> 00:16:29.029
ultrasounds do not penetrate through
objects. So if you have a small TV, say,
00:16:29.029 --> 00:16:36.810
with or Chromecast, for instance, they can
emit random PIN through ultrasound. Your
00:16:36.810 --> 00:16:40.700
device picks it up and submits it back to
the device through the Internet. And now
00:16:40.700 --> 00:16:44.410
you've proved that you are on the same
physical location with the with Chromecast
00:16:44.410 --> 00:16:51.320
or whatever your TV is. Also, Google
recently acquired sleek login. They are
00:16:51.320 --> 00:16:55.770
also using ultrasounds for authentication.
It's not entirely clear what their product
00:16:55.770 --> 00:17:00.120
is about, though. And also you have
audience measurement and analytics. So
00:17:00.120 --> 00:17:07.240
what they are doing is basically if you're
if you incorporate multiple beacons in the
00:17:07.240 --> 00:17:12.169
night, then you can basically track the
reactions and the behavior of the users of
00:17:12.169 --> 00:17:17.559
it, of the audience in the sense that
first, you know, how many people have
00:17:17.559 --> 00:17:21.470
watched your ad a second, you know what
happened. So if they show it's Sanderlin
00:17:21.470 --> 00:17:26.709
between and this, so they submit only the
first beacon of the two, if you have two,
00:17:26.709 --> 00:17:34.389
then you also track their behavior. OK, so
we've seen all these technologies and then
00:17:34.389 --> 00:17:40.779
we started wondering how secure is that
thing? Like, OK, what security measures
00:17:40.779 --> 00:17:46.470
are there applied by companies and
everything? So I'm going to immediately
00:17:46.470 --> 00:17:51.369
start with the exploitation of the
technology. So to do that, we just need
00:17:51.369 --> 00:17:59.470
the computer with speakers and the Tor browser
and the smartphone with an ultrasound app
00:17:59.470 --> 00:18:03.000
and a state level advisory. I'm going to
say more about the state level advisory
00:18:03.000 --> 00:18:11.679
later, but just keep in mind that it's on
the Tor threat model, so. I have a
00:18:11.679 --> 00:18:15.059
video of the attack. I'm going to stop it,
I'm going to pose it in different places
00:18:15.059 --> 00:18:24.669
to explain some more stuff. Yeah, OK, so
I'm going to set up the scene before that.
00:18:24.669 --> 00:18:28.019
So let's make the assumption that we have
a whistle blower that wants to leak some
00:18:28.019 --> 00:18:34.189
documents to a journalist, but he doesn't
know that the journalist is working with
00:18:34.189 --> 00:18:38.699
the government and his main intent is
basically to deanonymize him. So the
00:18:38.699 --> 00:18:42.559
journalist does the following, asks the
whistleblower to upload the documents to a
00:18:42.559 --> 00:18:48.360
Tor hidden service or a website that he owns.
And the whistleblower basically thinking
00:18:48.360 --> 00:18:55.340
that he's safe to do that through Tor
loads the page. So now I'm having I have the
00:18:55.340 --> 00:19:07.279
demo, which is exactly that implements
exactly that scenario. So the whistle
00:19:07.279 --> 00:19:12.970
blower opens the Tor browser, so the setup is
the following, we have the phone next to
00:19:12.970 --> 00:19:16.780
the computer. This can be up to seven
meters away, but for practical purposes,
00:19:16.780 --> 00:19:21.169
it has to be next to the computer. So we
have the Tor browser. What are we going to do
00:19:21.169 --> 00:19:28.749
first? For the purpose of the demo, we use
them smart for listening framework that's
00:19:28.749 --> 00:19:36.529
visible to the.. to the user. This is
basically the demo(?). Those apps, ultrasound
00:19:36.529 --> 00:19:40.929
cross device tracking apps run in the background,
so now we're setting set it on listening
00:19:40.929 --> 00:19:46.280
mode so that it starts listening. Of
course, in normal framework, the user
00:19:46.280 --> 00:19:52.570
doesn't have to do that part. But we want
to show that. We want to show that what's
00:19:52.570 --> 00:20:02.570
happening. So now the whistleblower is
going to load the innocuous were paid,
00:20:02.570 --> 00:20:12.980
suggested by the journalist and see what
happens to. OK, now we've loaded the page
00:20:12.980 --> 00:20:20.319
and the phone is listening in reality in
the background, so let's see what happens.
00:20:30.589 --> 00:20:36.320
OK, this is looks pretty bad. We have lots
of information about the user visiting our
00:20:36.320 --> 00:20:43.700
hidden service. I assume you already have some
clues about how this happened, what the
00:20:43.700 --> 00:20:54.850
information that we have is the following.
First of all. We have his IP address,
00:20:54.850 --> 00:21:02.070
phone number. Don't call this phone
number, because this isn't right. The ID
00:21:02.070 --> 00:21:10.859
is he may end his Google account email. So
this is enough to say and his location, of
00:21:10.859 --> 00:21:15.389
course, and this is enough to say that we
essentially deanonymized him, even if we
00:21:15.389 --> 00:21:22.340
had the IP address, that would have been
enough. So before I explain exactly how
00:21:22.340 --> 00:21:26.169
the attacked work, I'm going to introduce
some tools that the attackers have at
00:21:26.169 --> 00:21:32.320
their disposal. The first one is a Bitcoin
injection. So what you can essentially do
00:21:32.320 --> 00:21:37.229
is basically craft your own ultrasound
beacons and push them to devices,
00:21:37.229 --> 00:21:40.809
listening for beacons, and then their
devices are going to treat them like valid
00:21:40.809 --> 00:21:45.160
beacons and submit them back to the
company's backend. And then the same
00:21:45.160 --> 00:21:49.989
things. Basically, you can also replace
ultrasound beacons, meaning that you can
00:21:49.989 --> 00:21:55.320
capture them from virus location. And this
is actually happening on the wild at a
00:21:55.320 --> 00:22:04.309
large scale for a specific application.
And then once you capture those beacons,
00:22:04.309 --> 00:22:11.429
you can replace them back to the company's
back end through the user's devices to
00:22:11.429 --> 00:22:16.990
give you a clue. There is a company that
incentivizes users to visit stores by
00:22:16.990 --> 00:22:22.720
providing them offers and end points when
they are visiting stores and people are
00:22:22.720 --> 00:22:27.660
capturing the beacons and are replaying them
back to their devices from home. So they
00:22:27.660 --> 00:22:30.580
are selling the beacons through the
Internet so that they don't have to go to
00:22:30.580 --> 00:22:39.559
the actual stores. OK, the problem here is
basically that the framework is handling
00:22:39.559 --> 00:22:43.000
every beacon. It doesn't have a way to
distinguish between the valid and
00:22:43.000 --> 00:22:48.050
maliciously crafted beacons. And my favorite
tool for the attackers is basically a beacon
00:22:48.050 --> 00:22:55.350 line:1
trap, which is a code snippet that
once you loaded, you basically reproduce
00:22:55.350 --> 00:23:01.679
one or more inaudible beacons that the
attacker chose to. So this can happen in
00:23:01.679 --> 00:23:06.759
lots of ways on the demo. I use the first
one. So you build a website and you have
00:23:06.759 --> 00:23:12.189
some JavaScript there just playing the
ultrasounds from the back. What else you can
00:23:12.189 --> 00:23:17.769
do is basically start crosseyed scripting
vulnerability. Just exploit it on any
00:23:17.769 --> 00:23:22.929
random website and then you can inject
beacons to the visitors of this website
00:23:22.929 --> 00:23:30.249
or a man-in-the-middle attacks just
adding or javascript snippet on that
00:23:30.249 --> 00:23:37.779
user's traffic or they send an audio
message to the to the victim. So how did
00:23:37.779 --> 00:23:41.830
Tor deanonymization attack work? It's the
following. So first the adversary needs to
00:23:41.830 --> 00:23:50.049
set up, set up a campaign, and then once
he captures the the beacon associated with
00:23:50.049 --> 00:23:55.540
that campaign, he builds a beacon trap and
essentially on step three lures, the user
00:23:55.540 --> 00:24:00.789
to visit it. This is what the journalist
basically did for the whistleblower on our
00:24:00.789 --> 00:24:05.840
scenario. Then the user loads the
resource. He has no idea this is possible.
00:24:05.840 --> 00:24:12.440
And she slapped him amidst the ultrasound,
beacon. If you if your smartphone has such a
00:24:12.440 --> 00:24:17.460
framework, it's going to pick it up and
submit it back to the provider and I don't
00:24:17.460 --> 00:24:22.179
know about you, but when I'm using Tor,
I'm not connecting my phone through to the
00:24:22.179 --> 00:24:25.509
Internet, through the Tor network. My
phone is connected through my normal Wi-
00:24:25.509 --> 00:24:34.039
Fi. So now the ultrasound service provider
knows that the you know, this smartphone
00:24:34.039 --> 00:24:37.690
device omitted that specific beacon. And
then I step seven, basically the
00:24:37.690 --> 00:24:42.809 line:1
adversary, which is state level adversary.
Can simply subpoena the provider for the
00:24:42.809 --> 00:24:48.509
AP or other identifiers, which from what
we've seen, they collect plenty of them.
00:24:48.509 --> 00:24:54.519
OK, so the first two elements, we have
them already like the Tor browser
00:24:54.519 --> 00:25:02.919
computer, which biggest fine smartphone
with ultrasound tracking enabled
00:25:02.919 --> 00:25:08.330
framework. Fine. What about the state
level adversity? So we didn't have a state
00:25:08.330 --> 00:25:13.090
level adversity handy. So what we did is
basically we redirected the
00:25:13.090 --> 00:25:18.540
traffic from step six to the advertized
backend. And I want to stress a point
00:25:18.540 --> 00:25:28.600
here. This is not. A long, long shot
assumption. So what we've seen in October
00:25:28.600 --> 00:25:33.179
is the following. I don't know how many of
you realize, but AT&T was running a spy
00:25:33.179 --> 00:25:41.029 line:1
program, a thing called Hammesfahr, and it
was providing paid access to governments
00:25:41.029 --> 00:25:45.269
only with an administrative subpoena,
which is not doesn't even need to be
00:25:45.269 --> 00:25:50.789
obtained by it's ads. So it's pretty easy
for them to get access to this kind of
00:25:50.789 --> 00:25:55.520
data. Especially we're talking about an IP
address. It's not it's very easy for them
00:25:55.520 --> 00:26:01.570
to get it. So we also came up with some
more attacks. First one is profile,
00:26:01.570 --> 00:26:07.710
corruption. Advertisers really like to
build profiles about you, your interests
00:26:07.710 --> 00:26:15.210
and your behavior. So what you are
basically doing is you can inject beacons
00:26:15.210 --> 00:26:21.209
to other people or even to your own phone
and then you can malform their profile.
00:26:21.209 --> 00:26:28.309
Exactly. The impact of this attack depends
on how the backend of the advertising
00:26:28.309 --> 00:26:33.039
company and the infrastructure works, but
the attack is definitely possible. And
00:26:33.039 --> 00:26:40.169
then there is information leakage attack
were works under a similar assumption. You
00:26:40.169 --> 00:26:44.510
can replay Beacon's eavesdrop and replay
because your own phone to make your
00:26:44.510 --> 00:26:49.519
profile similar to that of the victims.
And then based on how recommendation
00:26:49.519 --> 00:26:56.299
systems work, you're very likely to get
similar arts and similar content with that
00:26:56.299 --> 00:27:01.019
of the victims. So of course, this also
depends about exactly how the
00:27:01.019 --> 00:27:07.369
recommendation system is implemented, but
it's definitely possible. OK, so we've
00:27:07.369 --> 00:27:11.539
seen certain things that makes us think
that, OK, the ecosystem is not very
00:27:11.539 --> 00:27:19.409
secure. Um, we try to find out exactly why
this happened. So we did a security
00:27:19.409 --> 00:27:24.580
evaluation or we came up with four points.
The first one is that we came up with we
00:27:24.580 --> 00:27:31.749
realized that the threat model is
inaccurate, that ultrasound, because none
00:27:31.749 --> 00:27:39.130
of the implementations we've seen had any
security features. Um, they also violated
00:27:39.130 --> 00:27:44.149
the fundamental security principle and
they lacked transparency when it comes
00:27:44.149 --> 00:27:49.269
when it came to user interface. So let's
go through them one by one. So inaccurate
00:27:49.269 --> 00:27:52.999
and model. Basically what they do is
basically they rely on the fact that
00:27:52.999 --> 00:27:58.360
ultrasounds cannot penetrate the walls and
they travel up to seven meters reliably.
00:27:58.360 --> 00:28:05.559
However, as I said, as a matter of fact,
they also assume that you cannot capture
00:28:05.559 --> 00:28:10.870
and replay because because of that, that's
the reason, um, what what's happening in
00:28:10.870 --> 00:28:15.029
practice, that you can get really close
using beacon traps. So their assumption
00:28:15.029 --> 00:28:21.800
is not that accurate. Um, also, the
security capabilities of beacons are
00:28:21.800 --> 00:28:30.129
heavily constrained by the low bandwidth
the channel is has the limited time that
00:28:30.129 --> 00:28:33.580
you have to reach the users. So if someone
is in a supermarket, he's not going to
00:28:33.580 --> 00:28:37.170
stop somewhere for a very long time. So
you have limited time and a noisy
00:28:37.170 --> 00:28:42.440
environment. So you want a very low error
rate. And adding crypto to the beacons
00:28:42.440 --> 00:28:49.139
it may not be a good idea, but it also
results. This also results in replay in
00:28:49.139 --> 00:28:54.259
injection attacks being possible. Um, we
also hear the violation of the privilege
00:28:54.259 --> 00:28:59.849
of, uh, sorry, the principle of least privilege.
So what happens is basically all these
00:28:59.849 --> 00:29:05.110
apps need full access to the microphone.
And based on the way it works, it's
00:29:05.110 --> 00:29:10.489
completely unnecessary for them to gain
access to the audible frequencies.
00:29:10.489 --> 00:29:14.669
However, even if they want to, there's no
way to gain access only to the ultrasound
00:29:14.669 --> 00:29:20.530
spectrum, both in Android and iOS. You
have to gain either access to the whole
00:29:20.530 --> 00:29:26.629
spectrum or no access at all. So this, of
course, results in the first malicious
00:29:26.629 --> 00:29:32.229
developers can at any time start using
their access to the microphone. And of
00:29:32.229 --> 00:29:38.520
course, all the benign ultrasound enabled
apps are perceived by as malicious by the
00:29:38.520 --> 00:29:45.399
users. And this actually will say more
about it later. So lack of transparency is
00:29:45.399 --> 00:29:51.259
inclose. This is a bad combination with
what exactly we've seen previously,
00:29:51.259 --> 00:29:55.919
because it that we've observed large
discrepancies between apps when it comes
00:29:55.919 --> 00:30:00.879
to informing the users and also lots of
discrepancies when it comes to providing
00:30:00.879 --> 00:30:06.110
opt out options. And there is a conflict
of interest there, because if you're a
00:30:06.110 --> 00:30:12.600
framework developer, developer, you want
to advise for proper practices for your
00:30:12.600 --> 00:30:17.960
customers, but you are not you're not
going to enforce them or kind of blackmail
00:30:17.960 --> 00:30:22.499
them. Either you do it properly or you're
not using my framework. So there is a
00:30:22.499 --> 00:30:27.190
conflict of interest there. So what
happened because of a lack of
00:30:27.190 --> 00:30:33.289
transparency is the following. Signals 360 is
one of those frameworks. An NBA team
00:30:33.289 --> 00:30:39.500
started using this in May. And then a few
months after there is a sue and someone
00:30:39.500 --> 00:30:43.839
claims, you know, that thing is listening
in the background. And what's interesting
00:30:43.839 --> 00:30:49.220
is on the claim, what they are saying is,
OK, I gave permission through the Android
00:30:49.220 --> 00:30:54.119
permission system for them to access the
microphone, but it was not explained to me
00:30:54.119 --> 00:30:58.840
exactly what they were doing. And this is
in close ties with what the FTC was saying
00:30:58.840 --> 00:31:08.740
in the letter a few months ago. Also,
again, the same story, um, football team
00:31:08.740 --> 00:31:14.340
starts using such a framework a few months
after people are complaining that they are
00:31:14.340 --> 00:31:21.679
being eavesdropped on. Um, I think what
happened here is that. When the team was
00:31:21.679 --> 00:31:25.749
playing a match, the application started
listening for ultrasounds, but not all
00:31:25.749 --> 00:31:29.560
your fans are going to be in the stadium,
so you end up listening for ultrasounds in
00:31:29.560 --> 00:31:37.029
a church and other places. So, yeah,
people were also pissed. Um, OK, just to
00:31:37.029 --> 00:31:41.989
put it into perspective how prevalent
these technologies are, the ecosystem is
00:31:41.989 --> 00:31:48.000
growing. Even though that one company
withdrew. There are other companies in the
00:31:48.000 --> 00:31:54.899
ecosystem are coming up with new products
as well. So the number of users is
00:31:54.899 --> 00:32:00.110
relatively low, but it's also very hard to
estimate right now. We could find around
00:32:00.110 --> 00:32:05.269
10 companies offering ultrasound related
products and the majority of them is
00:32:05.269 --> 00:32:09.780
gathered around proximity marketing. There
was only one company doing ultrasound
00:32:09.780 --> 00:32:16.590
cross device tracking. At least we found
one. Um, and this is mainly due to
00:32:16.590 --> 00:32:21.290
infrastructure complexity. It's not easy
to do all those things. And secondly, I
00:32:21.290 --> 00:32:26.140
also believe that the whole backslash from
the security community is incentivized
00:32:26.140 --> 00:32:32.599
other companies from joining because they
don't want a tarnished reputation. OK, so
00:32:32.599 --> 00:32:36.919
we have this situation right now.
Companies are using ultrasound. What are
00:32:36.919 --> 00:32:48.340
we going to do? So this was our initial
idea. This is what we thought first. But
00:32:48.340 --> 00:32:54.950
we want to fix things. So we tried to come
up with certain steps that we need to take
00:32:54.950 --> 00:33:02.019
to actually fix that thing and make it
usable, but not dangerous. Um, so we
00:33:02.019 --> 00:33:07.240
listed what's wrong with it. We did it
already. We we developed some quick fixes
00:33:07.240 --> 00:33:12.330
that I'm going to present later and medium
term solutions as well. And then we
00:33:12.330 --> 00:33:16.830
started advocating for a long term changes
that are going to make the ecosystem
00:33:16.830 --> 00:33:23.650
reliable. And we need the involvement from
the community there. Definitely. So. We
00:33:23.650 --> 00:33:29.519
developed some short and medium term
solutions, um, the first one is a browser
00:33:29.519 --> 00:33:37.890
extension, our browser extension basically
does the following is based on HTML5, the
00:33:37.890 --> 00:33:45.899 line:1
Web audio API. Um, it filters all audio
sources and places a filter between the
00:33:45.899 --> 00:33:51.280
audio source and the destination on the
Web page and filters out ultrasounds. To
00:33:51.280 --> 00:33:55.489
do that, we use a heisel filter that
attenuates all frequencies above 18kHz
00:33:55.489 --> 00:34:04.539 line:1
and it works pretty reliably. And
we leave all audible frequencies, intact.
00:34:04.539 --> 00:34:10.060
But it's not going to work with
obsolete legacy technologies such as
00:34:10.060 --> 00:34:16.789
flash. OK, we also have an adroit
permission, I think this somewhat more
00:34:16.789 --> 00:34:22.980
medium term solution, what we did is we
developed a unique developed parts for the
00:34:22.980 --> 00:34:28.810
Android permission system. This allows for
fine grained control over the audio channel,
00:34:28.810 --> 00:34:35.099
basically separates the permission needed
for listening to audible sound and the
00:34:35.099 --> 00:34:39.750
permission needed for listening to the
ultrasound spectrum. So at least we force the
00:34:39.750 --> 00:34:44.559
applications to specifically declare that
they are going to listen to four
00:34:44.559 --> 00:34:49.399
ultrasounds. And of course, users can, on
the latest Android versions, can also
00:34:49.399 --> 00:34:54.369
disable this permission and it can act as
an opt out option if the app is not
00:34:54.369 --> 00:35:02.899
providing it. We also initiated discussion
on the Turbo Tracker, but, um, we have,
00:35:02.899 --> 00:35:09.380
um, we are advocating for some long term
solutions, so we really need some
00:35:09.380 --> 00:35:15.650
standardization here. Um, let's agree on
ultrasound to confirm that and decide what
00:35:15.650 --> 00:35:20.440
security features can be there. I mean, we
need to figure out what's technically
00:35:20.440 --> 00:35:25.410
possible there because it's not clear. And
then once we have a standard, we can start
00:35:25.410 --> 00:35:32.109
building some APIs. And the APIs are very
nice idea because, um, they will work as
00:35:32.109 --> 00:35:37.250
the Bluetooth APIs work, meaning that they
will provide some methods to discover,
00:35:37.250 --> 00:35:42.240
process, generate and emit the sound
beacons through a new API related
00:35:42.240 --> 00:35:48.809
permission. And this means that we will
stop having overprivileged apps. We won't
00:35:48.809 --> 00:35:54.310
need access to the microphone anymore,
which is a huge problem right now. And of
00:35:54.310 --> 00:35:58.700
course, the applications will not be
considered spying anymore. And there is
00:35:58.700 --> 00:36:03.630
also another problem that we found out
while we were playing with those shops.
00:36:03.630 --> 00:36:08.240
Um, if you have a framework listening
through the microphone, other apps cannot
00:36:08.240 --> 00:36:12.289
access it. So we are trying to open the
camera app to record the video on the app.
00:36:12.289 --> 00:36:17.320
Camera app was crashing because the framework
was locking the access to the
00:36:17.320 --> 00:36:22.349
microphone. Now we may have some
developers from frameworks saying, you
00:36:22.349 --> 00:36:26.020
know, I'm not going to use your API. I'm
going to keep asking for access to the
00:36:26.020 --> 00:36:34.090
microphone. But we can force them to use
this API if we somehow, um, by default
00:36:34.090 --> 00:36:38.750
filter out the ultrasound frequencies
from the microphone and
00:36:38.750 --> 00:36:44.640
provide the way to the user to enable them
on a pure application basis from his
00:36:44.640 --> 00:36:56.200
phone. OK, so. Here's what we did, um, we
analyzed them, multiple ultrasound
00:36:56.200 --> 00:37:00.329
tracking technologies, we saw what what's
out there in the real world and reverse
00:37:00.329 --> 00:37:08.500
engineered such frameworks. We identified,
um, quite a few security shortcomings. We
00:37:08.500 --> 00:37:16.150
introduced our attacks and proposed some,
um, usable countermeasures. Um, and
00:37:16.150 --> 00:37:21.580
hopefully we initiated the discussion
about standardizing ultrasound because,
00:37:21.580 --> 00:37:27.539
um, but there are still things left to do.
So for the application developers, please,
00:37:27.539 --> 00:37:32.880
um, explicitly notify the users about what
your app is doing. Many of them would
00:37:32.880 --> 00:37:41.150
appreciate to know that. Um, also, we need
to improve transparency in the data
00:37:41.150 --> 00:37:47.150
collection process because they collecting
lots of data and very few information were
00:37:47.150 --> 00:37:52.010
available about what kind of data they
framework's collect. Um, we also think
00:37:52.010 --> 00:37:57.010
it's a good idea to have an opt in option
if it's not too much to ask, at least an
00:37:57.010 --> 00:38:07.910
opt out and standard security practices,
um, as always. So framework providers
00:38:07.910 --> 00:38:13.730
basically need to make sure that the
developers inform the users and also make
00:38:13.730 --> 00:38:21.030
sure that the users consent regularly to
listening for because like it's not enough
00:38:21.030 --> 00:38:25.809
if you consent once and then a month after
the app is still listening for ultrasound beacons
00:38:25.809 --> 00:38:33.170
you have to periodically ask the user if it's
still okay to do that. Um. Ideally, every time
00:38:33.170 --> 00:38:39.619
you are going to listen and then, of
course, we need to work on standardizing
00:38:39.619 --> 00:38:43.930
ultrasound because this is going to be a
long process and then building the
00:38:43.930 --> 00:38:48.430
specialized, specialized API. Hopefully
this is going to be easier once we have a
00:38:48.430 --> 00:38:56.960
standard and see what kind of
authentication mechanisms can we have in
00:38:56.960 --> 00:39:03.989
this kind of constrained transmission
channel. So..
00:39:03.989 --> 00:39:17.149
applause
00:39:17.149 --> 00:39:21.229
Herald: Thank you Vasilios. If you have any
questions, please do line up at the four
00:39:21.229 --> 00:39:26.679
microphones here in the walkways and the
first question will be the front
00:39:26.679 --> 00:39:30.959
microphone here.
Mic: Hello and thank you for your
00:39:30.959 --> 00:39:35.240
presentation. And I have a couple of
questions to ask that are technical and
00:39:35.240 --> 00:39:41.070
they are very related. First of all, do
you think that blocking out in our system
00:39:41.070 --> 00:39:47.799
level the high frequencies for either
microphone or the speakers as well, a
00:39:47.799 --> 00:39:53.070
something that is technically feasible and
will not put a very high latency in the
00:39:53.070 --> 00:39:56.750
processing?
Vasilios: So we did that through the
00:39:56.750 --> 00:39:59.350
permission. You are talking
about the smartphone right?
00:39:59.350 --> 00:40:03.850
Mic: Yeah, basically, because you have to
have a real time sound and microphone
00:40:03.850 --> 00:40:06.769
feedback.
Vasilios: So we did that with the
00:40:06.769 --> 00:40:14.179
permission. And I think it's not it's not
to resource demanding, if that's
00:40:14.179 --> 00:40:17.219
your question. So it's
definitely possible to do that.
00:40:17.219 --> 00:40:21.820
Mic: And the second part is, so
there is a new market maybe for some
00:40:21.820 --> 00:40:28.170
companies producing and microphones and
speakers that explicitly block out
00:40:28.170 --> 00:40:33.860
ultrasounds, right?
Vasilios: Possibly. Possibly. Um, I'm not
00:40:33.860 --> 00:40:38.690
sure if you can do this from the
application level. We developed parts for
00:40:38.690 --> 00:40:43.869
the Android system. I think our first
approach back then was basically try to
00:40:43.869 --> 00:40:48.130
build an app to do that from the
application, from the user land. And
00:40:48.130 --> 00:40:53.100
basically, I'm not sure if you can I doubt
actually an Android if you can filter out
00:40:53.100 --> 00:40:58.569
ultrasounds. But from a browser, we have
our extension. It works on Chrome. You can
00:40:58.569 --> 00:41:04.250
easily use our code to do the
same thing on the Firefox.
00:41:04.250 --> 00:41:06.600
Mic: Thanks.
Herald: The next question is from the
00:41:06.600 --> 00:41:10.460
front right microphone.
Mic: Thank you for your talk. I have a
00:41:10.460 --> 00:41:15.220
question about the attack requirements
against the whistleblower using Tor.
00:41:15.220 --> 00:41:23.730
I'm curious, the attacker has access to
the app on the smartphone and also access
00:41:23.730 --> 00:41:32.790
to the smartphone microphone. Wouldn't the
attacker then be able to just listen in on
00:41:32.790 --> 00:41:37.340
the conversation of the whistleblower and
thereby identify him?
00:41:37.340 --> 00:41:40.670
Vasilios: Yeah, absolutely. Absolutely.
It's a major problem. The problem is that
00:41:40.670 --> 00:41:47.760
they have access to the microphone. So
this is very this is very real and it's
00:41:47.760 --> 00:41:52.870
not going to be resolved even if we had
access only to the ultrasound spectrum.
00:41:52.870 --> 00:41:57.359
What we're saying is basically, if we only
had access to the ultrasound spectrum,
00:41:57.359 --> 00:42:04.820
you're still uhm you are still vulnerable
to these attacks unless you incorporate
00:42:04.820 --> 00:42:10.420
some crypto mechanisms that prevent these
things from happening. Is this your
00:42:10.420 --> 00:42:15.900
question or?
Mic: Um, well, I can still pull off the
00:42:15.900 --> 00:42:19.350
same attack if I don't
use ultrasound right?
00:42:19.350 --> 00:42:21.540
Vasilios: Through the audible spectrum?
Mic: Yes,
00:42:21.540 --> 00:42:28.990
Vasilios: You can absolutely do. There is
one company doing tracking in the audible
00:42:28.990 --> 00:42:35.560
spectrum. This is much harder to mitigate.
We're looking into it about ways, but
00:42:35.560 --> 00:42:39.109
there are so many ways to incorporate
beacons in the audible spectrum. The thing
00:42:39.109 --> 00:42:47.240
is that there is not much of an ecosystem
in this area right now that so you don't
00:42:47.240 --> 00:42:52.640
have lots of frameworks are there as many
as you have for ultrasounds.
00:42:52.640 --> 00:42:56.219
Mic: Thank you.
Herald: Our next question will be from
00:42:56.219 --> 00:43:01.349
the Internet via our signal angel
Signal Angel: $Username is asking, have
00:43:01.349 --> 00:43:08.170
you heard about exploiting parricide
ultrasound emiters like IC component's?
00:43:08.170 --> 00:43:10.230
Vasilios: Can you please
repeat the question?
00:43:10.230 --> 00:43:14.600
Signal Angel: Yes, sure. The question is,
can you use other components on the main
00:43:14.600 --> 00:43:23.740
board or maybe the hard disk to emit
ultrasounds and then broadcast the beacon
00:43:23.740 --> 00:43:28.960
via this?
Vailios: Uh. So that's a very that's a
00:43:28.960 --> 00:43:35.450
very good question. The answer is I don't
know, possibly, and it's very scary. Um,
00:43:35.450 --> 00:43:42.489
hopefully not, but I doubt it. I think
there should be a way to do it. Um, maybe
00:43:42.489 --> 00:43:47.200
the problem is that you cannot do this
completely in a completely inaudible way.
00:43:47.200 --> 00:43:51.760
Like you may be able to meet ultrasounds,
but you will also emit some sort of sound
00:43:51.760 --> 00:43:58.010
in the audible spectrum so that the user
will know that something is going on.
00:43:58.010 --> 00:44:02.520
Herald: The next question
from the left microphone.
00:44:02.520 --> 00:44:06.559
Mic: Thank you for your talk and
especially thanks for the research. So,
00:44:06.559 --> 00:44:12.919
uh, do you know of any framework's or, uh,
STKs that cash the beacon's they find?
00:44:12.919 --> 00:44:17.760
Because for my use case, I my phone was
mostly offline. I just make it online when
00:44:17.760 --> 00:44:21.950
I have to check
something. So I'm not that concerned. But
00:44:21.950 --> 00:44:24.660
you do you know, if they like cash the
beacons and and submit them later
00:44:24.660 --> 00:44:32.250
something like this. Of course they do.
I'm not surprised, unfortunately. Yeah.
00:44:32.250 --> 00:44:39.450
Thanks. Next question from the rear.
Right. Oh, what is the data rate? You can
00:44:39.450 --> 00:44:44.119
send in the ultrasound. Very good
question. And it's totally relevant to the
00:44:44.119 --> 00:44:51.250
cryptographic mechanisms we want to
incorporate from our experiments. Um, in
00:44:51.250 --> 00:44:58.480
four seconds you can basically send like
five to six alphabet characters if you're
00:44:58.480 --> 00:45:04.500
willing to kind of reduce the range a lot
less in less than seven meters, you may be
00:45:04.500 --> 00:45:11.970
able to send more. But the standard is not
very robust in this sense. But these
00:45:11.970 --> 00:45:16.260
experiments were done with this kind of
naive encoding that most of the companies
00:45:16.260 --> 00:45:22.930
are using. So if you do the encoding in a
very smart way, possibly you can increase
00:45:22.930 --> 00:45:29.329
that. And a small second part, what's the
energy consumption on the phone if that is
00:45:29.329 --> 00:45:35.110
running all the time? Wouldn't I detect
that? So it's not, uh, it's not good. We
00:45:35.110 --> 00:45:38.890
saw that it was draining the battery and
actually in the comments, I don't know if
00:45:38.890 --> 00:45:44.500
I had that comment here. Some people were
complaining that, um, I tried and it was
00:45:44.500 --> 00:45:53.029
draining my battery. And, um, there is an
impact. Absolutely. Amazon and Google Nest
00:45:53.029 --> 00:45:57.710
and all the other parts, aren't you more
worried about that? You know, the always
00:45:57.710 --> 00:46:02.400
listening thing from Google and Amazon and
everyone is coming up with some something
00:46:02.400 --> 00:46:10.130
like that that's always on. And so that
it's kind of strange because a user's
00:46:10.130 --> 00:46:18.369
consent. But at the same time, they don't
completely understand. So there is a gray
00:46:18.369 --> 00:46:22.670
line there, like you can say that the
users, OK, you consented to that up,
00:46:22.670 --> 00:46:28.549
starting with your with your phone and
listening on the background. But at the
00:46:28.549 --> 00:46:34.869
same time, the users don't have the best
understanding. Always. Thank you. Next
00:46:34.869 --> 00:46:39.430
question from the front left microphone
first. Thank you for the talk. I would be
00:46:39.430 --> 00:46:43.809
interested in how you selected your real
world applications and how many you found
00:46:43.809 --> 00:46:51.119
that already use such a framework. So what
was the first part of the question, how
00:46:51.119 --> 00:46:56.790
you selected your real world applications
from the marketplace staff if you had any.
00:46:56.790 --> 00:47:04.109
So we're trying to do a systematic scan of
the whole market, but it's not easy. So we
00:47:04.109 --> 00:47:09.440
not able to do that. There are resources
on the Internet. Luckily, the companies
00:47:09.440 --> 00:47:15.710
need to advertise their product. So they
basically publish press releases saying,
00:47:15.710 --> 00:47:22.000
you know, this NBA team started using our
product. We did some sort of scanning
00:47:22.000 --> 00:47:27.890
through alternative datasets, but
definitely we don't have an exhaustive
00:47:27.890 --> 00:47:33.049
list of applications. What I can say,
though, is that there are applications
00:47:33.049 --> 00:47:40.250
with. Using such frameworks with nearly up
to, if I remember correctly, up to one
00:47:40.250 --> 00:47:49.160
million installations. One notable
example, OK, I'm not entirely sure what I
00:47:49.160 --> 00:47:55.380
wanted, but up to a million we definitely
saw. OK, thanks. Do we have more questions
00:47:55.380 --> 00:48:02.500
from the Internet? Yes, E.F. is asking, is
he aware of or are you aware sorry? Are
00:48:02.500 --> 00:48:05.569
you aware of any framework available by
Google or Apple? In other words, how do we
00:48:05.569 --> 00:48:11.960
know that it's not, for instance,
seriously snitching on us? How do we know
00:48:11.960 --> 00:48:19.910
that it's not true? It's not serious. Some
maybe Aleksa snitching on us. We don't. I
00:48:19.910 --> 00:48:24.160
think that's a that's a very large
discussion. Right. So is the same problem
00:48:24.160 --> 00:48:34.059
that these companies are having? Because
if I go back here, basically the users are
00:48:34.059 --> 00:48:43.690
accusing them of eavesdropping. Especially
here from reverse engineering those
00:48:43.690 --> 00:48:49.869
frameworks, we couldn't find any such
activity, but again, it's very hard to
00:48:49.869 --> 00:48:54.259
convince the users that you are listening
to the ultrasound spectrum. You if you're
00:48:54.259 --> 00:48:59.769
accessing the whole audible frequencies
through the microphone, you're going to or
00:48:59.769 --> 00:49:04.119
you will always find yourself in this
position. So I guess it's the same problem
00:49:04.119 --> 00:49:09.339
that Alexa has from Amazon. But in this
case, you can actually solve it by
00:49:09.339 --> 00:49:15.410
constraining the spectrum that you gain
access to. Next question from the front
00:49:15.410 --> 00:49:21.069
left microphone, please. Has anybody done
an audible demonstration off these beacons
00:49:21.069 --> 00:49:26.230
bypassed by transposing them down an
octave or two, I think might be useful for
00:49:26.230 --> 00:49:34.089
for or your talk to something like that.
So you mean a demo, but using audible
00:49:34.089 --> 00:49:40.630
frequencies? Essentially, there is this
one company, but they are being pretty to
00:49:40.630 --> 00:49:44.869
all of these companies are being pretty
secretive with their technology. So they
00:49:44.869 --> 00:49:51.430
publish what's needed for marketing
purposes like accuracy sometimes remains
00:49:51.430 --> 00:49:57.390
very limited technical details. But apart
from these, you have to get your hands on
00:49:57.390 --> 00:50:04.829
the framework somehow and analyze it
yourself. So in this kind of overview we
00:50:04.829 --> 00:50:08.130
need for the ecosystem, we had to do
everything by ourselves. There was no
00:50:08.130 --> 00:50:15.789
resources out there were very limited, um,
or recording it and playing it down and
00:50:15.789 --> 00:50:23.290
transposing it yourself, if you know where
as a beacon of. Possibly I'm not I'm not
00:50:23.290 --> 00:50:31.779
entirely sure you could. Yeah. Another
question from our signal, angel mestas,
00:50:31.779 --> 00:50:37.789
again asking, um, would it be possible,
even if you have a low pass filter to use,
00:50:37.789 --> 00:50:44.810
uh, for instance, the cost effect and high
cost effect to transmit the beacon via
00:50:44.810 --> 00:50:53.900
ultrasound, but in a regime which is as
free for the app? So it's basically the
00:50:53.900 --> 00:50:59.799
question, can I somehow, via Aliasing USA
address on signal to make a normal signal
00:50:59.799 --> 00:51:08.319
out of it? Possibly, I don't know. I think
you are much more creative than I am, so
00:51:08.319 --> 00:51:16.819
maybe I should add more bullet points on
this controversialist here. Apparently,
00:51:16.819 --> 00:51:23.150
there are many more ways to do this,
possibly like hardware missions. This one
00:51:23.150 --> 00:51:29.619
sounds like a good idea, too. So next
question from the real right microphone. I
00:51:29.619 --> 00:51:33.559
apologize if you explain the story they
didn't understand, but is is sort of
00:51:33.559 --> 00:51:38.819
drowning out the signals, like jamming.
They just broadcasting white noise in that
00:51:38.819 --> 00:51:43.810
spectrum, an effective countermeasure. And
as a follow up, if it is, would it
00:51:43.810 --> 00:51:56.750
terrorize my dog? So absolutely, it's
effective. I mean, this it works up to
00:51:56.750 --> 00:52:01.770
seven meters, but we're not saying it's
not fragile, so you can do that, but it's
00:52:01.770 --> 00:52:05.829
noise pollution. And my dog, I don't think
it was happy. I did it for a very limited
00:52:05.829 --> 00:52:10.280
time. I could see her ears moving, but I
don't think she would appreciate it if I
00:52:10.280 --> 00:52:16.720
had the device at home doing this all the
time. Do we have any more questions from
00:52:16.720 --> 00:52:22.460
the Internet? Yes, EULEX is asking to what
extent could we use these for our own
00:52:22.460 --> 00:52:26.559
needs? For example, people in repressive
situations, for example, activists could
00:52:26.559 --> 00:52:30.630
use it to transmit secret encrypted
messages. Are there any efforts in this
00:52:30.630 --> 00:52:40.829
area? Yes, there are. People are
developing ultrasound modems. I think
00:52:40.829 --> 00:52:51.030
there is even a tag on it. And yes, of
course there is. So I would say, yes, I'm
00:52:51.030 --> 00:52:57.029
not entirely sure about the capabilities
of this channel in terms of bandwidth, but
00:52:57.029 --> 00:53:01.890
this is why we we are not advocating to
kill the technology just to make it secure
00:53:01.890 --> 00:53:06.900
and know its limitations. So you can do
good stuff with it. And this is what we
00:53:06.900 --> 00:53:13.720
want. Next question from the Rio, right?
Yeah, I'm wondering if you could transfer
00:53:13.720 --> 00:53:19.859
that technique from the ultrasound range
also to the Audible Range, for example, by
00:53:19.859 --> 00:53:26.550
using watermarks, audio, watermarks, and
then, well, your permission thingy with
00:53:26.550 --> 00:53:31.740
the ultrasound permissions would be
ineffective and you could also track the
00:53:31.740 --> 00:53:37.810
user. How about this? Is it possible audio
watermarks in the audible spectrum? Yeah,
00:53:37.810 --> 00:53:42.900
it's absolutely possible. Um, our
countermeasures are not effective against
00:53:42.900 --> 00:53:50.490
this. Um, it's just that there is from our
research, just one company doing this. Uh,
00:53:50.490 --> 00:53:57.119
so this one, um, I think technically it's
a bit more challenging to do that.
00:53:57.119 --> 00:54:02.809
Instead, they're just admitting they are
doing it in a very basic way. So
00:54:02.809 --> 00:54:08.480
hopefully, um, if there is a clear way to
do it through ultrasounds, they are not
00:54:08.480 --> 00:54:15.400
going to reside reside in the audible
spectrum. But our countermeasures are not
00:54:15.400 --> 00:54:22.640
effective against the audible. Um.
Watermarks. Yeah, thanks, next question
00:54:22.640 --> 00:54:28.960
from the front left microphone. I've heard
that I don't think it's very credible, but
00:54:28.960 --> 00:54:34.079
I've heard that there is some sound on
this sub sound spectrum. There were some
00:54:34.079 --> 00:54:40.700
experiments showing that they can
influence our mood, the mood of humans. Is
00:54:40.700 --> 00:54:47.900
there any relevant information about how
ultrasounds could affect us? So without
00:54:47.900 --> 00:54:54.580
being an expert in this particular area?
I've read similar articles when I was
00:54:54.580 --> 00:54:59.190
looking into it. I can tell you it's very
annoying, especially if you're listening
00:54:59.190 --> 00:55:05.680
to it through headphones. You cannot
really hear the sound, but you can if
00:55:05.680 --> 00:55:11.599
you're using headphones, you can feel the
pressure. So if I don't know what kind of
00:55:11.599 --> 00:55:19.809
medical condition you may develop, but you
won't be very sane after. Do we have any
00:55:19.809 --> 00:55:27.289
more questions? Yes. One further question,
um, would it be possible to, um, use a
00:55:27.289 --> 00:55:33.999
charming solution to get rid of the
signals? Yes, but you you're going to
00:55:33.999 --> 00:55:38.450
follow the you know, it's going to result
in noise pollution, but if you are being
00:55:38.450 --> 00:55:46.690
paranoid about it, yes, it's and it's, I
think, a straightforward thing to do. Any
00:55:46.690 --> 00:55:53.330
more questions? One more on the front left
microphone. Know, you said that physical
00:55:53.330 --> 00:55:59.049
objects will block the ultrasound. How
solid do the physical objects need to be?
00:55:59.049 --> 00:56:04.680
So, for example, does my pocket block the
ultrasound and thus prevent my phone to
00:56:04.680 --> 00:56:11.579
call the environment and vice versa? OK,
well, that's a good question. I don't
00:56:11.579 --> 00:56:16.529
think that clothes can actually do that
unless it's very thick. Thin girls
00:56:16.529 --> 00:56:27.190
definitely block it. Um. Thick glass, I
would say it reduce the transmission rate,
00:56:27.190 --> 00:56:35.559
the signal to noise ratio by a lot, but it
could go through it, so. You need
00:56:35.559 --> 00:56:42.690
something quite concrete, metal. I don't
think it goes through it. So are there any
00:56:42.690 --> 00:56:48.160
more? Doesn't look like it, maybe, maybe
one more sorry. Oh, good signal, good bye.
00:56:48.160 --> 00:57:02.350
Kitty is asking, could you name or compile
a list of tracking programs and apps? So.
00:57:02.350 --> 00:57:07.410
That's a good question. We're trying to
make an exhaustive list and try to resolve
00:57:07.410 --> 00:57:16.529
this in a systematic way. I've already
listed two Macenta frameworks. One is the
00:57:16.529 --> 00:57:20.160
Silverbush one three actually. One is the
Silver Paswan. There is another one used
00:57:20.160 --> 00:57:32.940
by single 360. So developed the signal
360, and then there is a listener one.
00:57:32.940 --> 00:57:39.609
These are very popular. Um, and then its
developer is incorporating them into their
00:57:39.609 --> 00:57:48.749
applications in different ways, offering
varying levels of transparency for the
00:57:48.749 --> 00:57:54.339
users. So it's better if you start knowing
what the frameworks are and then trying to
00:57:54.339 --> 00:57:59.039
find the applications using them, because
you know what? You're looking in the code
00:57:59.039 --> 00:58:06.280
and you can develop some queries and
enabling you to access an ability to to
00:58:06.280 --> 00:58:13.509
track which applications are using them.
What what we observed for Silverbush is
00:58:13.509 --> 00:58:18.820
basically after the company announced that
they are moving out of the US and because
00:58:18.820 --> 00:58:24.390
of the whole backslash, maybe even before
that, um, companies started to drop the
00:58:24.390 --> 00:58:30.109
framework. So all their versions had the
framework, but they are not using it
00:58:30.109 --> 00:58:52.549
anymore. I think that's it. Thank you very
much, Vasilios Lovelady's.
00:58:52.549 --> 00:59:02.932
Subtitles created by c3subtitles.de
in the year 2021. Join, and help us!