1
00:00:00,000 --> 00:00:13,850
preroll music
2
00:00:13,850 --> 00:00:18,720
Vasilios: Hello, everyone, thanks for coming
today. I'm going to introduce the ultrasound
3
00:00:18,720 --> 00:00:24,500
ecosystem, which is an exotic and kind of
little known ecosystem. So I would like to
4
00:00:24,500 --> 00:00:29,820
start with a short story about the
product, which is also our motivation for
5
00:00:29,820 --> 00:00:39,870
this work. So some time ago, there was a
product that worked in the ultrasound
6
00:00:39,870 --> 00:00:44,450
spectrum that cannot be perceived by
humans. And the product was actually an
7
00:00:44,450 --> 00:00:50,700
interesting idea. It was very promising
and everything, but it also had a fatal
8
00:00:50,700 --> 00:00:56,730
flaw. So now that I've done this
introduction, I would like to tell you
9
00:00:56,730 --> 00:01:01,350
more about the story of the product and
how it came to be and what was it? What
10
00:01:01,350 --> 00:01:08,150
was its lifecycle. So 2012, a company
called SilverPush was a startup in
11
00:01:08,150 --> 00:01:14,810
India. It was founded there and they had
this ultrasound device tracking product.
12
00:01:14,810 --> 00:01:18,770
I'll go more into the technical details
later. So for a couple of years, they were
13
00:01:18,770 --> 00:01:26,789
working on that product. And it wasn't
until 2014 that they basically got some
14
00:01:26,789 --> 00:01:32,769
serious funding from a venture center or
other angel investors for millions. So in
15
00:01:32,769 --> 00:01:38,840
2014, they also got a few months after
they got funded. They also got some press
16
00:01:38,840 --> 00:01:44,940
coverage about the product and they got
some pretty good reviews on their
17
00:01:44,940 --> 00:01:49,260
newspapers and articles about what the
product could do. And at the same time,
18
00:01:49,260 --> 00:01:52,950
they were doing what most of the companies
are doing, like publishing patents about
19
00:01:52,950 --> 00:02:00,029
their technology and everything. So things
later started to go like year after year
20
00:02:00,029 --> 00:02:06,499
and half maybe started to go not that well
for them. The security community noticed
21
00:02:06,499 --> 00:02:10,759
and there was some press coverage about
the product that was not so positive
22
00:02:10,759 --> 00:02:19,250
anymore. So this is one of the very first
emails that appear on the Web regarding
23
00:02:19,250 --> 00:02:24,709
the product. So it's from a W3C
working group. So a researcher there is
24
00:02:24,709 --> 00:02:31,220
basically. Notifying the other members of
the group that, OK, there is this product,
25
00:02:31,220 --> 00:02:36,349
maybe there are transparency issues, and
certainly the users are not aware of
26
00:02:36,349 --> 00:02:42,489
what exactly is going on there. So let's
keep an eye on it. And so this was a very
27
00:02:42,489 --> 00:02:48,370
one of the very first things published
about the product from the privacy and
28
00:02:48,370 --> 00:02:54,470
security perspective. So what happened
then was the press took notice and they
29
00:02:54,470 --> 00:03:01,379
got all those headlines urging users to be
very careful. And, oh, this is a this is
30
00:03:01,379 --> 00:03:08,769
evil, take care. People are eavesdropping
on you and stuff. So, of course, this led
31
00:03:08,769 --> 00:03:14,950
also on the FTC to take action. They
organized a workshop on cross device tracking
32
00:03:14,950 --> 00:03:22,060
in general, I think, and they made specific
mentions for ultrasound cross device tracking
33
00:03:22,060 --> 00:03:28,170
don't worry if you're not familiar with this terms,
I'm going to define everything later. So
34
00:03:28,170 --> 00:03:33,390
what basically they were saying is
transparency issues. How do how do we
35
00:03:33,390 --> 00:03:39,860
protect ourselves? How is that thing
working? So, then the users, of course,
36
00:03:39,860 --> 00:03:43,689
started to react. And there were like many
people who were unhappy, they were
37
00:03:43,689 --> 00:03:48,430
complaining, what is this? I don't want
that thing. So people were actually
38
00:03:48,430 --> 00:03:53,479
suggesting solutions and the solutions
that were making sense. And of course, you
39
00:03:53,479 --> 00:04:01,260
have always the users that are completely
immune to what you have there. So what
40
00:04:01,260 --> 00:04:10,549
happened then is like five months after
the FTC took much more serious action
41
00:04:10,549 --> 00:04:16,000
regarding this specific product. So it
sent a letter to all the developers. And
42
00:04:16,000 --> 00:04:19,780
the letter was essentially saying, you
know, you're using this framework in
43
00:04:19,780 --> 00:04:27,160
Europe. We've seen it in Google Play
store. It's not enough that you are asking
44
00:04:27,160 --> 00:04:31,380
for the microphone permission. You should
let the users know that you are tracking
45
00:04:31,380 --> 00:04:36,330
them if you are doing so. Otherwise, you
are violating rule X, Y, Z, and you're not
46
00:04:36,330 --> 00:04:42,590
allowed to do that. So this was pretty
serious, I would say. And what happened
47
00:04:42,590 --> 00:04:46,610
next is basically the company withdrew
from the US market and said, you know, we
48
00:04:46,610 --> 00:04:51,040
have nothing to do with the U.S. market
and this product is not active there. You
49
00:04:51,040 --> 00:04:57,320
shouldn't be concerned. So end of story
like the product is not out there in the
50
00:04:57,320 --> 00:05:03,660
US at least anymore. Are we safe? So it
seemed to us that it was assumed that this
51
00:05:03,660 --> 00:05:10,030
was an isolated security incident. And to
be fair, very little became known about
52
00:05:10,030 --> 00:05:15,090
the technology. At this point. The press
moved on to other hot topics at the time,
53
00:05:15,090 --> 00:05:20,950
people went quiet, like if people are not
using it, it's fine. So everyone
54
00:05:20,950 --> 00:05:25,840
seemed happy. But we're curious people. So
we had lots of questions that were not
55
00:05:25,840 --> 00:05:35,040
answered. So our main questions was like
why they were using ultrasounds. We'll see
56
00:05:35,040 --> 00:05:41,620
that what they are doing, you can do with
our technologies, how such frameworks
57
00:05:41,620 --> 00:05:47,120
work. We had no idea there was no coverage
or nothing about it. The technical,
58
00:05:47,120 --> 00:05:53,340
technically speaking, out there, are there
other such products there? Because we were
59
00:05:53,340 --> 00:05:59,410
aware of one. Everyone on all the articles
was referring to that one product, but we
60
00:05:59,410 --> 00:06:03,210
were not sure if there are others doing
the same thing. And of course, we were
61
00:06:03,210 --> 00:06:07,870
looking for a report about the whole
ecosystem and how it works. And there was
62
00:06:07,870 --> 00:06:13,290
nothing. So what do you do then if if
there are no technical resources?
63
00:06:13,290 --> 00:06:19,740
Basically, we decided to do our own
research and come up with this report that
64
00:06:19,740 --> 00:06:24,980
we were lacking. So we're done with
motivation so far. We were pretty pumped
65
00:06:24,980 --> 00:06:29,891
up about looking into it. OK, what's
there? The rest of the presentation will
66
00:06:29,891 --> 00:06:34,010
go as follows. Like first I'm going to
introduce ultrasound tracking and other
67
00:06:34,010 --> 00:06:40,620
terminology, then I'm going to go on with
the attack details. And indeed, we have an
68
00:06:40,620 --> 00:06:47,610
attack again against the Tor browser. Then
we're doing a formal security analysis of
69
00:06:47,610 --> 00:06:53,350
the ecosystem and try to pinpoint the
things that went wrong. And then we'll try
70
00:06:53,350 --> 00:07:00,470
to introduce our countermeasures and
advocate for proper practices. So to begin
71
00:07:00,470 --> 00:07:06,940
with, I'm Vasilis. I've done this work
with other curious people. These are
72
00:07:06,940 --> 00:07:13,440
showing how Yanick Fratantonio, Christopher
Kruegel and Giovanni Vigna from UCSB and also
73
00:07:13,440 --> 00:07:19,280
Federico Maggi from Polytechnical
Damilola. Let's now start with the
74
00:07:19,280 --> 00:07:26,340
ecosystem, so apparently ultrasounds are
used in a lot of places and they can be
75
00:07:26,340 --> 00:07:30,920
utilized for different purposes, some of
them are cross device tracking that are
76
00:07:30,920 --> 00:07:36,770
referred already to audience analytics,
synchronized content, proximity, marketing
77
00:07:36,770 --> 00:07:41,460
and device pairing. You can do some other
things, but you will see them later. So to
78
00:07:41,460 --> 00:07:48,920
begin with what cross device tracking is,
cross device tracking is basically the holy
79
00:07:48,920 --> 00:07:53,630
grail for marketers right now because
you're using your multiple devices
80
00:07:53,630 --> 00:07:57,990
smartphone, laptop, computer, maybe your
TV and to them, your appear as different
81
00:07:57,990 --> 00:08:02,330
people. And they all want to be able to
link to link those devices to know that
82
00:08:02,330 --> 00:08:06,920
you're the same person so that they can
build their profiles more accurately. So,
83
00:08:06,920 --> 00:08:13,300
for instance, if you're watching an ad on
the TV, they want to be able to know that
84
00:08:13,300 --> 00:08:19,720
it's you so that they can push relevant
ads from your smartphone or follow up ads.
85
00:08:19,720 --> 00:08:27,860
Um. So this is employed by major
advertising networks, and there are two
86
00:08:27,860 --> 00:08:32,780
ways to do it, either deterministically or
probabilistically, that deterministic
87
00:08:32,780 --> 00:08:38,780
approach is much more reliable. You get
100 percent accuracy and works as follows.
88
00:08:38,780 --> 00:08:43,200
If you are Facebook, the users are heavily
incentivized to log in from all their
89
00:08:43,200 --> 00:08:49,220
devices. So what happens is that. You can
immediately know that, OK, this user has
90
00:08:49,220 --> 00:08:55,191
these three devices and I can put relevant
content to all of them. However, if you
91
00:08:55,191 --> 00:08:58,910
are not Facebook or Google you, it's much
more unlikely that the users would want to
92
00:08:58,910 --> 00:09:03,580
log into your platform from their
different devices. So you have to look for
93
00:09:03,580 --> 00:09:09,970
alternatives. And one tool to come up with
those alternatives are ultrasound beacons.
94
00:09:09,970 --> 00:09:18,480
So, um, ultrasound tracking products are
using ultrasound because they may sound
95
00:09:18,480 --> 00:09:22,590
exotic, but basically there they are. What
they are doing is they are encoding a
96
00:09:22,590 --> 00:09:29,460
sequence of symbols, um, in a very high
frequency that it's inaudible by humans.
97
00:09:29,460 --> 00:09:35,520
That's the first key feature. The second one
is they can be emitted by most commercial
98
00:09:35,520 --> 00:09:39,400
speakers and they can be captured by most
commercial microphones, for instance,
99
00:09:39,400 --> 00:09:48,620
found on your smartphone. So the technical
details are the following. I know there
100
00:09:48,620 --> 00:09:54,380
are a lot of experts in these kinds of
things here, so I'm averaging out what how
101
00:09:54,380 --> 00:09:57,590
the companies are doing it right now. I'm
not saying that this is the best way to do
102
00:09:57,590 --> 00:10:01,520
it, but this is more or less what they're
doing. Of course, they have patents, so
103
00:10:01,520 --> 00:10:06,230
each one of them is doing a slightly
different thing so they don't overlap.
104
00:10:06,230 --> 00:10:13,330
They're using the near ultrasound spectrum
between the eight eight kilohertz and 20
105
00:10:13,330 --> 00:10:18,940
kilohertz, which is inaudible by usually
by adults. They divide it in smaller
106
00:10:18,940 --> 00:10:27,380
chunks. So if you divide it in chunks that
have size of 75 Hertz, you get 26, about 26
107
00:10:27,380 --> 00:10:33,920
chunks, and then you can assign letter of
the alphabet on each one of them. And then
108
00:10:33,920 --> 00:10:38,109
what they are doing is usually within four
to five seconds. They emit sequences of
109
00:10:38,109 --> 00:10:45,410
characters. Usually they contain for four
to six characters in there, and they use
110
00:10:45,410 --> 00:10:51,670
it to incorporate a unique ID
corresponding to their source, they attach
111
00:10:51,670 --> 00:10:56,710
the beacon to. So there is no ultrasound
beacon standard, as I said previously, but
112
00:10:56,710 --> 00:11:00,450
there are lots of patents, so each one of
them is doing a slightly different thing.
113
00:11:00,450 --> 00:11:06,350
But this is a basic principle. We did some
experiments and we found out that within
114
00:11:06,350 --> 00:11:13,880
seven meters, you get pretty good accuracy
in low error rate. So of course, this depends
115
00:11:13,880 --> 00:11:20,250
exactly how you encode things. But with
applications found on Google Play, this
116
00:11:20,250 --> 00:11:24,990
worked up to seven meters. Um, we couldn't
find computer speakers that were not able
117
00:11:24,990 --> 00:11:33,310
to emit near ultrasound frequencies and
work with this technology and.. we this is
118
00:11:33,310 --> 00:11:36,590
pretty known for this kind of frequencies,
they cannot penetrate through physical
119
00:11:36,590 --> 00:11:41,000
objects, but this is not a problem for
their purposes. And we did some
120
00:11:41,000 --> 00:11:46,720
experiments with our research assistant
and we can say that they are audible by
121
00:11:46,720 --> 00:11:54,420
animals. So if you combine cross device
tracking and ultrasound because you get
122
00:11:54,420 --> 00:12:02,350
ultrasound cross device tracking. So now what
you can do with this and this is this is a
123
00:12:02,350 --> 00:12:07,320
pretty good idea, actually, because it
offers high accuracy, you don't ask the
124
00:12:07,320 --> 00:12:16,720
users to log in, which is very high, very
demanding thing to ask for. You can embed
125
00:12:16,720 --> 00:12:22,860
those beacons in websites or TV ads, and
this technology, however, requires some
126
00:12:22,860 --> 00:12:26,210
sort of sophisticated backend
infrastructure. We're going to see more
127
00:12:26,210 --> 00:12:30,260
about it later. And you also need the
network of publishers who are willing to
128
00:12:30,260 --> 00:12:36,740
incorporate incorporate beacons in their
content, whatever this content is. And
129
00:12:36,740 --> 00:12:41,250
then, of course, you need an ultrasound
cross device tracking framework that is going
130
00:12:41,250 --> 00:12:47,080
to run on the user's mobile device, a
smartphone. So these frameworks are
131
00:12:47,080 --> 00:12:52,680
essentially and as the advertising SDK is the
key that the developers can use to display
132
00:12:52,680 --> 00:12:57,060
ads on their free apps. So it's not that
the developers are going to incorporate
133
00:12:57,060 --> 00:13:04,491
the ultrasound framework is going to
incorporate an advertising SDK with
134
00:13:04,491 --> 00:13:09,610
varying degrees of understanding of what
it does. So here is how ultrasound cross device
135
00:13:09,610 --> 00:13:15,740
tracking works. On step one, basically, we
have the advertising client. He just wants
136
00:13:15,740 --> 00:13:20,200
to advertise, advertises his products. He
goes to the ultrasound cross device
137
00:13:20,200 --> 00:13:25,250
tracking provider who has the
infrastructure set up, set up a campaign,
138
00:13:25,250 --> 00:13:31,610
and they provide their associates a unique
ultrasound because with this campaign and
139
00:13:31,610 --> 00:13:37,660
then pushes this become to content
publishers to incorporate them
140
00:13:37,660 --> 00:13:43,860
incorporated into their content, depending
on what the advertiser advertising client
141
00:13:43,860 --> 00:13:49,270
is trying to achieve. So this is step
three or step for a user is basically
142
00:13:49,270 --> 00:13:56,950
accessing all of those content providers
either. This is a TV ad or a website on
143
00:13:56,950 --> 00:14:03,030
the Internet and one this once this
content is loaded or displayed by your TV.
144
00:14:03,030 --> 00:14:08,010
At the same time, the device, the devices
speakers are emitting the ultrasounds. And
145
00:14:08,010 --> 00:14:13,580
if you have the ultrasound cross device tracking
framework on your phone, which is usually
146
00:14:13,580 --> 00:14:18,910
listening on the background, then it picks
up the ultrasound and on step six, it
147
00:14:18,910 --> 00:14:25,060
submits it back to the service provider,
which now knows that, OK, this guy has
148
00:14:25,060 --> 00:14:31,700
watched this DVR or whatever it is, and
I'm going to add this to his profile and
149
00:14:31,700 --> 00:14:38,220
push our target dates back to his device.
So, of course, by doing this, they're just
150
00:14:38,220 --> 00:14:45,900
trying to improve, improve their
conversion rate and get more customers.
151
00:14:45,900 --> 00:14:52,970
Another use of ultrasounds currently in
practice is proximity marketing, so venues
152
00:14:52,970 --> 00:14:59,380
basically set up multiple, multiple
ultrasound meters. This is kind of fancy
153
00:14:59,380 --> 00:15:05,350
name for speakers and this is kind of the
nice thing about the ultrasound. You just
154
00:15:05,350 --> 00:15:11,470
need speakers. So they put this in
multiple locations in their venue, either
155
00:15:11,470 --> 00:15:18,320
a supermarket or a stadium, for instance,
and then there is a customer up. If you're
156
00:15:18,320 --> 00:15:23,310
a supermarket, there is a supermarket up.
If you're an NBA team, which will see
157
00:15:23,310 --> 00:15:29,730
later, you have this fun application that
the fans of your team can download
158
00:15:29,730 --> 00:15:35,080
and install on their smartphones. And then
once this app, this happens, listing on
159
00:15:35,080 --> 00:15:40,550
the background and it picks up the
ultrasound and submits them back to the
160
00:15:40,550 --> 00:15:47,660
company. So the main purpose of using is
this is basically to study in user
161
00:15:47,660 --> 00:15:55,220
behavior, in user behavior, provide real
time notifications like, OK, you are in
162
00:15:55,220 --> 00:15:59,610
this aisle on the supermarket, but if you
just walk two meters down, you're going to
163
00:15:59,610 --> 00:16:06,170
see this product in discount. Or the third
point, which kind of incentivizes the
164
00:16:06,170 --> 00:16:11,240
users more, is basically you're offering
reward points for users visiting your
165
00:16:11,240 --> 00:16:17,600
store. And actually there is a product
doing exactly that on the market. So some
166
00:16:17,600 --> 00:16:23,830
other uses are device pairing. And this
basically relies on the fact that
167
00:16:23,830 --> 00:16:29,029
ultrasounds do not penetrate through
objects. So if you have a small TV, say,
168
00:16:29,029 --> 00:16:36,810
with or Chromecast, for instance, they can
emit random PIN through ultrasound. Your
169
00:16:36,810 --> 00:16:40,700
device picks it up and submits it back to
the device through the Internet. And now
170
00:16:40,700 --> 00:16:44,410
you've proved that you are on the same
physical location with the with Chromecast
171
00:16:44,410 --> 00:16:51,320
or whatever your TV is. Also, Google
recently acquired sleek login. They are
172
00:16:51,320 --> 00:16:55,770
also using ultrasounds for authentication.
It's not entirely clear what their product
173
00:16:55,770 --> 00:17:00,120
is about, though. And also you have
audience measurement and analytics. So
174
00:17:00,120 --> 00:17:07,240
what they are doing is basically if you're
if you incorporate multiple beacons in the
175
00:17:07,240 --> 00:17:12,169
night, then you can basically track the
reactions and the behavior of the users of
176
00:17:12,169 --> 00:17:17,559
it, of the audience in the sense that
first, you know, how many people have
177
00:17:17,559 --> 00:17:21,470
watched your ad a second, you know what
happened. So if they show it's Sanderlin
178
00:17:21,470 --> 00:17:26,709
between and this, so they submit only the
first beacon of the two, if you have two,
179
00:17:26,709 --> 00:17:34,389
then you also track their behavior. OK, so
we've seen all these technologies and then
180
00:17:34,389 --> 00:17:40,779
we started wondering how secure is that
thing? Like, OK, what security measures
181
00:17:40,779 --> 00:17:46,470
are there applied by companies and
everything? So I'm going to immediately
182
00:17:46,470 --> 00:17:51,369
start with the exploitation of the
technology. So to do that, we just need
183
00:17:51,369 --> 00:17:59,470
the computer with speakers and the Tor browser
and the smartphone with an ultrasound app
184
00:17:59,470 --> 00:18:03,000
and a state level advisory. I'm going to
say more about the state level advisory
185
00:18:03,000 --> 00:18:11,679
later, but just keep in mind that it's on
the Tor threat model, so. I have a
186
00:18:11,679 --> 00:18:15,059
video of the attack. I'm going to stop it,
I'm going to pose it in different places
187
00:18:15,059 --> 00:18:24,669
to explain some more stuff. Yeah, OK, so
I'm going to set up the scene before that.
188
00:18:24,669 --> 00:18:28,019
So let's make the assumption that we have
a whistle blower that wants to leak some
189
00:18:28,019 --> 00:18:34,189
documents to a journalist, but he doesn't
know that the journalist is working with
190
00:18:34,189 --> 00:18:38,699
the government and his main intent is
basically to deanonymize him. So the
191
00:18:38,699 --> 00:18:42,559
journalist does the following, asks the
whistleblower to upload the documents to a
192
00:18:42,559 --> 00:18:48,360
Tor hidden service or a website that he owns.
And the whistleblower basically thinking
193
00:18:48,360 --> 00:18:55,340
that he's safe to do that through Tor
loads the page. So now I'm having I have the
194
00:18:55,340 --> 00:19:07,279
demo, which is exactly that implements
exactly that scenario. So the whistle
195
00:19:07,279 --> 00:19:12,970
blower opens the Tor browser, so the setup is
the following, we have the phone next to
196
00:19:12,970 --> 00:19:16,780
the computer. This can be up to seven
meters away, but for practical purposes,
197
00:19:16,780 --> 00:19:21,169
it has to be next to the computer. So we
have the Tor browser. What are we going to do
198
00:19:21,169 --> 00:19:28,749
first? For the purpose of the demo, we use
them smart for listening framework that's
199
00:19:28,749 --> 00:19:36,529
visible to the.. to the user. This is
basically the demo(?). Those apps, ultrasound
200
00:19:36,529 --> 00:19:40,929
cross device tracking apps run in the background,
so now we're setting set it on listening
201
00:19:40,929 --> 00:19:46,280
mode so that it starts listening. Of
course, in normal framework, the user
202
00:19:46,280 --> 00:19:52,570
doesn't have to do that part. But we want
to show that. We want to show that what's
203
00:19:52,570 --> 00:20:02,570
happening. So now the whistleblower is
going to load the innocuous were paid,
204
00:20:02,570 --> 00:20:12,980
suggested by the journalist and see what
happens to. OK, now we've loaded the page
205
00:20:12,980 --> 00:20:20,319
and the phone is listening in reality in
the background, so let's see what happens.
206
00:20:30,589 --> 00:20:36,320
OK, this is looks pretty bad. We have lots
of information about the user visiting our
207
00:20:36,320 --> 00:20:43,700
hidden service. I assume you already have some
clues about how this happened, what the
208
00:20:43,700 --> 00:20:54,850
information that we have is the following.
First of all. We have his IP address,
209
00:20:54,850 --> 00:21:02,070
phone number. Don't call this phone
number, because this isn't right. The ID
210
00:21:02,070 --> 00:21:10,859
is he may end his Google account email. So
this is enough to say and his location, of
211
00:21:10,859 --> 00:21:15,389
course, and this is enough to say that we
essentially deanonymized him, even if we
212
00:21:15,389 --> 00:21:22,340
had the IP address, that would have been
enough. So before I explain exactly how
213
00:21:22,340 --> 00:21:26,169
the attacked work, I'm going to introduce
some tools that the attackers have at
214
00:21:26,169 --> 00:21:32,320
their disposal. The first one is a Bitcoin
injection. So what you can essentially do
215
00:21:32,320 --> 00:21:37,229
is basically craft your own ultrasound
beacons and push them to devices,
216
00:21:37,229 --> 00:21:40,809
listening for beacons, and then their
devices are going to treat them like valid
217
00:21:40,809 --> 00:21:45,160
beacons and submit them back to the
company's backend. And then the same
218
00:21:45,160 --> 00:21:49,989
things. Basically, you can also replace
ultrasound beacons, meaning that you can
219
00:21:49,989 --> 00:21:55,320
capture them from virus location. And this
is actually happening on the wild at a
220
00:21:55,320 --> 00:22:04,309
large scale for a specific application.
And then once you capture those beacons,
221
00:22:04,309 --> 00:22:11,429
you can replace them back to the company's
back end through the user's devices to
222
00:22:11,429 --> 00:22:16,990
give you a clue. There is a company that
incentivizes users to visit stores by
223
00:22:16,990 --> 00:22:22,720
providing them offers and end points when
they are visiting stores and people are
224
00:22:22,720 --> 00:22:27,660
capturing the beacons and are replaying them
back to their devices from home. So they
225
00:22:27,660 --> 00:22:30,580
are selling the beacons through the
Internet so that they don't have to go to
226
00:22:30,580 --> 00:22:39,559
the actual stores. OK, the problem here is
basically that the framework is handling
227
00:22:39,559 --> 00:22:43,000
every beacon. It doesn't have a way to
distinguish between the valid and
228
00:22:43,000 --> 00:22:48,050
maliciously crafted beacons. And my favorite
tool for the attackers is basically a beacon
229
00:22:48,050 --> 00:22:55,350
trap, which is a code snippet that
once you loaded, you basically reproduce
230
00:22:55,350 --> 00:23:01,679
one or more inaudible beacons that the
attacker chose to. So this can happen in
231
00:23:01,679 --> 00:23:06,759
lots of ways on the demo. I use the first
one. So you build a website and you have
232
00:23:06,759 --> 00:23:12,189
some JavaScript there just playing the
ultrasounds from the back. What else you can
233
00:23:12,189 --> 00:23:17,769
do is basically start crosseyed scripting
vulnerability. Just exploit it on any
234
00:23:17,769 --> 00:23:22,929
random website and then you can inject
beacons to the visitors of this website
235
00:23:22,929 --> 00:23:30,249
or a man-in-the-middle attacks just
adding or javascript snippet on that
236
00:23:30,249 --> 00:23:37,779
user's traffic or they send an audio
message to the to the victim. So how did
237
00:23:37,779 --> 00:23:41,830
Tor deanonymization attack work? It's the
following. So first the adversary needs to
238
00:23:41,830 --> 00:23:50,049
set up, set up a campaign, and then once
he captures the the beacon associated with
239
00:23:50,049 --> 00:23:55,540
that campaign, he builds a beacon trap and
essentially on step three lures, the user
240
00:23:55,540 --> 00:24:00,789
to visit it. This is what the journalist
basically did for the whistleblower on our
241
00:24:00,789 --> 00:24:05,840
scenario. Then the user loads the
resource. He has no idea this is possible.
242
00:24:05,840 --> 00:24:12,440
And she slapped him amidst the ultrasound,
beacon. If you if your smartphone has such a
243
00:24:12,440 --> 00:24:17,460
framework, it's going to pick it up and
submit it back to the provider and I don't
244
00:24:17,460 --> 00:24:22,179
know about you, but when I'm using Tor,
I'm not connecting my phone through to the
245
00:24:22,179 --> 00:24:25,509
Internet, through the Tor network. My
phone is connected through my normal Wi-
246
00:24:25,509 --> 00:24:34,039
Fi. So now the ultrasound service provider
knows that the you know, this smartphone
247
00:24:34,039 --> 00:24:37,690
device omitted that specific beacon. And
then I step seven, basically the
248
00:24:37,690 --> 00:24:42,809
adversary, which is state level adversary.
Can simply subpoena the provider for the
249
00:24:42,809 --> 00:24:48,509
AP or other identifiers, which from what
we've seen, they collect plenty of them.
250
00:24:48,509 --> 00:24:54,519
OK, so the first two elements, we have
them already like the Tor browser
251
00:24:54,519 --> 00:25:02,919
computer, which biggest fine smartphone
with ultrasound tracking enabled
252
00:25:02,919 --> 00:25:08,330
framework. Fine. What about the state
level adversity? So we didn't have a state
253
00:25:08,330 --> 00:25:13,090
level adversity handy. So what we did is
basically we redirected the
254
00:25:13,090 --> 00:25:18,540
traffic from step six to the advertized
backend. And I want to stress a point
255
00:25:18,540 --> 00:25:28,600
here. This is not. A long, long shot
assumption. So what we've seen in October
256
00:25:28,600 --> 00:25:33,179
is the following. I don't know how many of
you realize, but AT&T was running a spy
257
00:25:33,179 --> 00:25:41,029
program, a thing called Hammesfahr, and it
was providing paid access to governments
258
00:25:41,029 --> 00:25:45,269
only with an administrative subpoena,
which is not doesn't even need to be
259
00:25:45,269 --> 00:25:50,789
obtained by it's ads. So it's pretty easy
for them to get access to this kind of
260
00:25:50,789 --> 00:25:55,520
data. Especially we're talking about an IP
address. It's not it's very easy for them
261
00:25:55,520 --> 00:26:01,570
to get it. So we also came up with some
more attacks. First one is profile,
262
00:26:01,570 --> 00:26:07,710
corruption. Advertisers really like to
build profiles about you, your interests
263
00:26:07,710 --> 00:26:15,210
and your behavior. So what you are
basically doing is you can inject beacons
264
00:26:15,210 --> 00:26:21,209
to other people or even to your own phone
and then you can malform their profile.
265
00:26:21,209 --> 00:26:28,309
Exactly. The impact of this attack depends
on how the backend of the advertising
266
00:26:28,309 --> 00:26:33,039
company and the infrastructure works, but
the attack is definitely possible. And
267
00:26:33,039 --> 00:26:40,169
then there is information leakage attack
were works under a similar assumption. You
268
00:26:40,169 --> 00:26:44,510
can replay Beacon's eavesdrop and replay
because your own phone to make your
269
00:26:44,510 --> 00:26:49,519
profile similar to that of the victims.
And then based on how recommendation
270
00:26:49,519 --> 00:26:56,299
systems work, you're very likely to get
similar arts and similar content with that
271
00:26:56,299 --> 00:27:01,019
of the victims. So of course, this also
depends about exactly how the
272
00:27:01,019 --> 00:27:07,369
recommendation system is implemented, but
it's definitely possible. OK, so we've
273
00:27:07,369 --> 00:27:11,539
seen certain things that makes us think
that, OK, the ecosystem is not very
274
00:27:11,539 --> 00:27:19,409
secure. Um, we try to find out exactly why
this happened. So we did a security
275
00:27:19,409 --> 00:27:24,580
evaluation or we came up with four points.
The first one is that we came up with we
276
00:27:24,580 --> 00:27:31,749
realized that the threat model is
inaccurate, that ultrasound, because none
277
00:27:31,749 --> 00:27:39,130
of the implementations we've seen had any
security features. Um, they also violated
278
00:27:39,130 --> 00:27:44,149
the fundamental security principle and
they lacked transparency when it comes
279
00:27:44,149 --> 00:27:49,269
when it came to user interface. So let's
go through them one by one. So inaccurate
280
00:27:49,269 --> 00:27:52,999
and model. Basically what they do is
basically they rely on the fact that
281
00:27:52,999 --> 00:27:58,360
ultrasounds cannot penetrate the walls and
they travel up to seven meters reliably.
282
00:27:58,360 --> 00:28:05,559
However, as I said, as a matter of fact,
they also assume that you cannot capture
283
00:28:05,559 --> 00:28:10,870
and replay because because of that, that's
the reason, um, what what's happening in
284
00:28:10,870 --> 00:28:15,029
practice, that you can get really close
using beacon traps. So their assumption
285
00:28:15,029 --> 00:28:21,800
is not that accurate. Um, also, the
security capabilities of beacons are
286
00:28:21,800 --> 00:28:30,129
heavily constrained by the low bandwidth
the channel is has the limited time that
287
00:28:30,129 --> 00:28:33,580
you have to reach the users. So if someone
is in a supermarket, he's not going to
288
00:28:33,580 --> 00:28:37,170
stop somewhere for a very long time. So
you have limited time and a noisy
289
00:28:37,170 --> 00:28:42,440
environment. So you want a very low error
rate. And adding crypto to the beacons
290
00:28:42,440 --> 00:28:49,139
it may not be a good idea, but it also
results. This also results in replay in
291
00:28:49,139 --> 00:28:54,259
injection attacks being possible. Um, we
also hear the violation of the privilege
292
00:28:54,259 --> 00:28:59,849
of, uh, sorry, the principle of least privilege.
So what happens is basically all these
293
00:28:59,849 --> 00:29:05,110
apps need full access to the microphone.
And based on the way it works, it's
294
00:29:05,110 --> 00:29:10,489
completely unnecessary for them to gain
access to the audible frequencies.
295
00:29:10,489 --> 00:29:14,669
However, even if they want to, there's no
way to gain access only to the ultrasound
296
00:29:14,669 --> 00:29:20,530
spectrum, both in Android and iOS. You
have to gain either access to the whole
297
00:29:20,530 --> 00:29:26,629
spectrum or no access at all. So this, of
course, results in the first malicious
298
00:29:26,629 --> 00:29:32,229
developers can at any time start using
their access to the microphone. And of
299
00:29:32,229 --> 00:29:38,520
course, all the benign ultrasound enabled
apps are perceived by as malicious by the
300
00:29:38,520 --> 00:29:45,399
users. And this actually will say more
about it later. So lack of transparency is
301
00:29:45,399 --> 00:29:51,259
inclose. This is a bad combination with
what exactly we've seen previously,
302
00:29:51,259 --> 00:29:55,919
because it that we've observed large
discrepancies between apps when it comes
303
00:29:55,919 --> 00:30:00,879
to informing the users and also lots of
discrepancies when it comes to providing
304
00:30:00,879 --> 00:30:06,110
opt out options. And there is a conflict
of interest there, because if you're a
305
00:30:06,110 --> 00:30:12,600
framework developer, developer, you want
to advise for proper practices for your
306
00:30:12,600 --> 00:30:17,960
customers, but you are not you're not
going to enforce them or kind of blackmail
307
00:30:17,960 --> 00:30:22,499
them. Either you do it properly or you're
not using my framework. So there is a
308
00:30:22,499 --> 00:30:27,190
conflict of interest there. So what
happened because of a lack of
309
00:30:27,190 --> 00:30:33,289
transparency is the following. Signals 360 is
one of those frameworks. An NBA team
310
00:30:33,289 --> 00:30:39,500
started using this in May. And then a few
months after there is a sue and someone
311
00:30:39,500 --> 00:30:43,839
claims, you know, that thing is listening
in the background. And what's interesting
312
00:30:43,839 --> 00:30:49,220
is on the claim, what they are saying is,
OK, I gave permission through the Android
313
00:30:49,220 --> 00:30:54,119
permission system for them to access the
microphone, but it was not explained to me
314
00:30:54,119 --> 00:30:58,840
exactly what they were doing. And this is
in close ties with what the FTC was saying
315
00:30:58,840 --> 00:31:08,740
in the letter a few months ago. Also,
again, the same story, um, football team
316
00:31:08,740 --> 00:31:14,340
starts using such a framework a few months
after people are complaining that they are
317
00:31:14,340 --> 00:31:21,679
being eavesdropped on. Um, I think what
happened here is that. When the team was
318
00:31:21,679 --> 00:31:25,749
playing a match, the application started
listening for ultrasounds, but not all
319
00:31:25,749 --> 00:31:29,560
your fans are going to be in the stadium,
so you end up listening for ultrasounds in
320
00:31:29,560 --> 00:31:37,029
a church and other places. So, yeah,
people were also pissed. Um, OK, just to
321
00:31:37,029 --> 00:31:41,989
put it into perspective how prevalent
these technologies are, the ecosystem is
322
00:31:41,989 --> 00:31:48,000
growing. Even though that one company
withdrew. There are other companies in the
323
00:31:48,000 --> 00:31:54,899
ecosystem are coming up with new products
as well. So the number of users is
324
00:31:54,899 --> 00:32:00,110
relatively low, but it's also very hard to
estimate right now. We could find around
325
00:32:00,110 --> 00:32:05,269
10 companies offering ultrasound related
products and the majority of them is
326
00:32:05,269 --> 00:32:09,780
gathered around proximity marketing. There
was only one company doing ultrasound
327
00:32:09,780 --> 00:32:16,590
cross device tracking. At least we found
one. Um, and this is mainly due to
328
00:32:16,590 --> 00:32:21,290
infrastructure complexity. It's not easy
to do all those things. And secondly, I
329
00:32:21,290 --> 00:32:26,140
also believe that the whole backslash from
the security community is incentivized
330
00:32:26,140 --> 00:32:32,599
other companies from joining because they
don't want a tarnished reputation. OK, so
331
00:32:32,599 --> 00:32:36,919
we have this situation right now.
Companies are using ultrasound. What are
332
00:32:36,919 --> 00:32:48,340
we going to do? So this was our initial
idea. This is what we thought first. But
333
00:32:48,340 --> 00:32:54,950
we want to fix things. So we tried to come
up with certain steps that we need to take
334
00:32:54,950 --> 00:33:02,019
to actually fix that thing and make it
usable, but not dangerous. Um, so we
335
00:33:02,019 --> 00:33:07,240
listed what's wrong with it. We did it
already. We we developed some quick fixes
336
00:33:07,240 --> 00:33:12,330
that I'm going to present later and medium
term solutions as well. And then we
337
00:33:12,330 --> 00:33:16,830
started advocating for a long term changes
that are going to make the ecosystem
338
00:33:16,830 --> 00:33:23,650
reliable. And we need the involvement from
the community there. Definitely. So. We
339
00:33:23,650 --> 00:33:29,519
developed some short and medium term
solutions, um, the first one is a browser
340
00:33:29,519 --> 00:33:37,890
extension, our browser extension basically
does the following is based on HTML5, the
341
00:33:37,890 --> 00:33:45,899
Web audio API. Um, it filters all audio
sources and places a filter between the
342
00:33:45,899 --> 00:33:51,280
audio source and the destination on the
Web page and filters out ultrasounds. To
343
00:33:51,280 --> 00:33:55,489
do that, we use a heisel filter that
attenuates all frequencies above 18kHz
344
00:33:55,489 --> 00:34:04,539
and it works pretty reliably. And
we leave all audible frequencies, intact.
345
00:34:04,539 --> 00:34:10,060
But it's not going to work with
obsolete legacy technologies such as
346
00:34:10,060 --> 00:34:16,789
flash. OK, we also have an adroit
permission, I think this somewhat more
347
00:34:16,789 --> 00:34:22,980
medium term solution, what we did is we
developed a unique developed parts for the
348
00:34:22,980 --> 00:34:28,810
Android permission system. This allows for
fine grained control over the audio channel,
349
00:34:28,810 --> 00:34:35,099
basically separates the permission needed
for listening to audible sound and the
350
00:34:35,099 --> 00:34:39,750
permission needed for listening to the
ultrasound spectrum. So at least we force the
351
00:34:39,750 --> 00:34:44,559
applications to specifically declare that
they are going to listen to four
352
00:34:44,559 --> 00:34:49,399
ultrasounds. And of course, users can, on
the latest Android versions, can also
353
00:34:49,399 --> 00:34:54,369
disable this permission and it can act as
an opt out option if the app is not
354
00:34:54,369 --> 00:35:02,899
providing it. We also initiated discussion
on the Turbo Tracker, but, um, we have,
355
00:35:02,899 --> 00:35:09,380
um, we are advocating for some long term
solutions, so we really need some
356
00:35:09,380 --> 00:35:15,650
standardization here. Um, let's agree on
ultrasound to confirm that and decide what
357
00:35:15,650 --> 00:35:20,440
security features can be there. I mean, we
need to figure out what's technically
358
00:35:20,440 --> 00:35:25,410
possible there because it's not clear. And
then once we have a standard, we can start
359
00:35:25,410 --> 00:35:32,109
building some APIs. And the APIs are very
nice idea because, um, they will work as
360
00:35:32,109 --> 00:35:37,250
the Bluetooth APIs work, meaning that they
will provide some methods to discover,
361
00:35:37,250 --> 00:35:42,240
process, generate and emit the sound
beacons through a new API related
362
00:35:42,240 --> 00:35:48,809
permission. And this means that we will
stop having overprivileged apps. We won't
363
00:35:48,809 --> 00:35:54,310
need access to the microphone anymore,
which is a huge problem right now. And of
364
00:35:54,310 --> 00:35:58,700
course, the applications will not be
considered spying anymore. And there is
365
00:35:58,700 --> 00:36:03,630
also another problem that we found out
while we were playing with those shops.
366
00:36:03,630 --> 00:36:08,240
Um, if you have a framework listening
through the microphone, other apps cannot
367
00:36:08,240 --> 00:36:12,289
access it. So we are trying to open the
camera app to record the video on the app.
368
00:36:12,289 --> 00:36:17,320
Camera app was crashing because the framework
was locking the access to the
369
00:36:17,320 --> 00:36:22,349
microphone. Now we may have some
developers from frameworks saying, you
370
00:36:22,349 --> 00:36:26,020
know, I'm not going to use your API. I'm
going to keep asking for access to the
371
00:36:26,020 --> 00:36:34,090
microphone. But we can force them to use
this API if we somehow, um, by default
372
00:36:34,090 --> 00:36:38,750
filter out the ultrasound frequencies
from the microphone and
373
00:36:38,750 --> 00:36:44,640
provide the way to the user to enable them
on a pure application basis from his
374
00:36:44,640 --> 00:36:56,200
phone. OK, so. Here's what we did, um, we
analyzed them, multiple ultrasound
375
00:36:56,200 --> 00:37:00,329
tracking technologies, we saw what what's
out there in the real world and reverse
376
00:37:00,329 --> 00:37:08,500
engineered such frameworks. We identified,
um, quite a few security shortcomings. We
377
00:37:08,500 --> 00:37:16,150
introduced our attacks and proposed some,
um, usable countermeasures. Um, and
378
00:37:16,150 --> 00:37:21,580
hopefully we initiated the discussion
about standardizing ultrasound because,
379
00:37:21,580 --> 00:37:27,539
um, but there are still things left to do.
So for the application developers, please,
380
00:37:27,539 --> 00:37:32,880
um, explicitly notify the users about what
your app is doing. Many of them would
381
00:37:32,880 --> 00:37:41,150
appreciate to know that. Um, also, we need
to improve transparency in the data
382
00:37:41,150 --> 00:37:47,150
collection process because they collecting
lots of data and very few information were
383
00:37:47,150 --> 00:37:52,010
available about what kind of data they
framework's collect. Um, we also think
384
00:37:52,010 --> 00:37:57,010
it's a good idea to have an opt in option
if it's not too much to ask, at least an
385
00:37:57,010 --> 00:38:07,910
opt out and standard security practices,
um, as always. So framework providers
386
00:38:07,910 --> 00:38:13,730
basically need to make sure that the
developers inform the users and also make
387
00:38:13,730 --> 00:38:21,030
sure that the users consent regularly to
listening for because like it's not enough
388
00:38:21,030 --> 00:38:25,809
if you consent once and then a month after
the app is still listening for ultrasound beacons
389
00:38:25,809 --> 00:38:33,170
you have to periodically ask the user if it's
still okay to do that. Um. Ideally, every time
390
00:38:33,170 --> 00:38:39,619
you are going to listen and then, of
course, we need to work on standardizing
391
00:38:39,619 --> 00:38:43,930
ultrasound because this is going to be a
long process and then building the
392
00:38:43,930 --> 00:38:48,430
specialized, specialized API. Hopefully
this is going to be easier once we have a
393
00:38:48,430 --> 00:38:56,960
standard and see what kind of
authentication mechanisms can we have in
394
00:38:56,960 --> 00:39:03,989
this kind of constrained transmission
channel. So..
395
00:39:03,989 --> 00:39:17,149
applause
396
00:39:17,149 --> 00:39:21,229
Herald: Thank you Vasilios. If you have any
questions, please do line up at the four
397
00:39:21,229 --> 00:39:26,679
microphones here in the walkways and the
first question will be the front
398
00:39:26,679 --> 00:39:30,959
microphone here.
Mic: Hello and thank you for your
399
00:39:30,959 --> 00:39:35,240
presentation. And I have a couple of
questions to ask that are technical and
400
00:39:35,240 --> 00:39:41,070
they are very related. First of all, do
you think that blocking out in our system
401
00:39:41,070 --> 00:39:47,799
level the high frequencies for either
microphone or the speakers as well, a
402
00:39:47,799 --> 00:39:53,070
something that is technically feasible and
will not put a very high latency in the
403
00:39:53,070 --> 00:39:56,750
processing?
Vasilios: So we did that through the
404
00:39:56,750 --> 00:39:59,350
permission. You are talking
about the smartphone right?
405
00:39:59,350 --> 00:40:03,850
Mic: Yeah, basically, because you have to
have a real time sound and microphone
406
00:40:03,850 --> 00:40:06,769
feedback.
Vasilios: So we did that with the
407
00:40:06,769 --> 00:40:14,179
permission. And I think it's not it's not
to resource demanding, if that's
408
00:40:14,179 --> 00:40:17,219
your question. So it's
definitely possible to do that.
409
00:40:17,219 --> 00:40:21,820
Mic: And the second part is, so
there is a new market maybe for some
410
00:40:21,820 --> 00:40:28,170
companies producing and microphones and
speakers that explicitly block out
411
00:40:28,170 --> 00:40:33,860
ultrasounds, right?
Vasilios: Possibly. Possibly. Um, I'm not
412
00:40:33,860 --> 00:40:38,690
sure if you can do this from the
application level. We developed parts for
413
00:40:38,690 --> 00:40:43,869
the Android system. I think our first
approach back then was basically try to
414
00:40:43,869 --> 00:40:48,130
build an app to do that from the
application, from the user land. And
415
00:40:48,130 --> 00:40:53,100
basically, I'm not sure if you can I doubt
actually an Android if you can filter out
416
00:40:53,100 --> 00:40:58,569
ultrasounds. But from a browser, we have
our extension. It works on Chrome. You can
417
00:40:58,569 --> 00:41:04,250
easily use our code to do the
same thing on the Firefox.
418
00:41:04,250 --> 00:41:06,600
Mic: Thanks.
Herald: The next question is from the
419
00:41:06,600 --> 00:41:10,460
front right microphone.
Mic: Thank you for your talk. I have a
420
00:41:10,460 --> 00:41:15,220
question about the attack requirements
against the whistleblower using Tor.
421
00:41:15,220 --> 00:41:23,730
I'm curious, the attacker has access to
the app on the smartphone and also access
422
00:41:23,730 --> 00:41:32,790
to the smartphone microphone. Wouldn't the
attacker then be able to just listen in on
423
00:41:32,790 --> 00:41:37,340
the conversation of the whistleblower and
thereby identify him?
424
00:41:37,340 --> 00:41:40,670
Vasilios: Yeah, absolutely. Absolutely.
It's a major problem. The problem is that
425
00:41:40,670 --> 00:41:47,760
they have access to the microphone. So
this is very this is very real and it's
426
00:41:47,760 --> 00:41:52,870
not going to be resolved even if we had
access only to the ultrasound spectrum.
427
00:41:52,870 --> 00:41:57,359
What we're saying is basically, if we only
had access to the ultrasound spectrum,
428
00:41:57,359 --> 00:42:04,820
you're still uhm you are still vulnerable
to these attacks unless you incorporate
429
00:42:04,820 --> 00:42:10,420
some crypto mechanisms that prevent these
things from happening. Is this your
430
00:42:10,420 --> 00:42:15,900
question or?
Mic: Um, well, I can still pull off the
431
00:42:15,900 --> 00:42:19,350
same attack if I don't
use ultrasound right?
432
00:42:19,350 --> 00:42:21,540
Vasilios: Through the audible spectrum?
Mic: Yes,
433
00:42:21,540 --> 00:42:28,990
Vasilios: You can absolutely do. There is
one company doing tracking in the audible
434
00:42:28,990 --> 00:42:35,560
spectrum. This is much harder to mitigate.
We're looking into it about ways, but
435
00:42:35,560 --> 00:42:39,109
there are so many ways to incorporate
beacons in the audible spectrum. The thing
436
00:42:39,109 --> 00:42:47,240
is that there is not much of an ecosystem
in this area right now that so you don't
437
00:42:47,240 --> 00:42:52,640
have lots of frameworks are there as many
as you have for ultrasounds.
438
00:42:52,640 --> 00:42:56,219
Mic: Thank you.
Herald: Our next question will be from
439
00:42:56,219 --> 00:43:01,349
the Internet via our signal angel
Signal Angel: $Username is asking, have
440
00:43:01,349 --> 00:43:08,170
you heard about exploiting parricide
ultrasound emiters like IC component's?
441
00:43:08,170 --> 00:43:10,230
Vasilios: Can you please
repeat the question?
442
00:43:10,230 --> 00:43:14,600
Signal Angel: Yes, sure. The question is,
can you use other components on the main
443
00:43:14,600 --> 00:43:23,740
board or maybe the hard disk to emit
ultrasounds and then broadcast the beacon
444
00:43:23,740 --> 00:43:28,960
via this?
Vailios: Uh. So that's a very that's a
445
00:43:28,960 --> 00:43:35,450
very good question. The answer is I don't
know, possibly, and it's very scary. Um,
446
00:43:35,450 --> 00:43:42,489
hopefully not, but I doubt it. I think
there should be a way to do it. Um, maybe
447
00:43:42,489 --> 00:43:47,200
the problem is that you cannot do this
completely in a completely inaudible way.
448
00:43:47,200 --> 00:43:51,760
Like you may be able to meet ultrasounds,
but you will also emit some sort of sound
449
00:43:51,760 --> 00:43:58,010
in the audible spectrum so that the user
will know that something is going on.
450
00:43:58,010 --> 00:44:02,520
Herald: The next question
from the left microphone.
451
00:44:02,520 --> 00:44:06,559
Mic: Thank you for your talk and
especially thanks for the research. So,
452
00:44:06,559 --> 00:44:12,919
uh, do you know of any framework's or, uh,
STKs that cash the beacon's they find?
453
00:44:12,919 --> 00:44:17,760
Because for my use case, I my phone was
mostly offline. I just make it online when
454
00:44:17,760 --> 00:44:21,950
I have to check
something. So I'm not that concerned. But
455
00:44:21,950 --> 00:44:24,660
you do you know, if they like cash the
beacons and and submit them later
456
00:44:24,660 --> 00:44:32,250
something like this. Of course they do.
I'm not surprised, unfortunately. Yeah.
457
00:44:32,250 --> 00:44:39,450
Thanks. Next question from the rear.
Right. Oh, what is the data rate? You can
458
00:44:39,450 --> 00:44:44,119
send in the ultrasound. Very good
question. And it's totally relevant to the
459
00:44:44,119 --> 00:44:51,250
cryptographic mechanisms we want to
incorporate from our experiments. Um, in
460
00:44:51,250 --> 00:44:58,480
four seconds you can basically send like
five to six alphabet characters if you're
461
00:44:58,480 --> 00:45:04,500
willing to kind of reduce the range a lot
less in less than seven meters, you may be
462
00:45:04,500 --> 00:45:11,970
able to send more. But the standard is not
very robust in this sense. But these
463
00:45:11,970 --> 00:45:16,260
experiments were done with this kind of
naive encoding that most of the companies
464
00:45:16,260 --> 00:45:22,930
are using. So if you do the encoding in a
very smart way, possibly you can increase
465
00:45:22,930 --> 00:45:29,329
that. And a small second part, what's the
energy consumption on the phone if that is
466
00:45:29,329 --> 00:45:35,110
running all the time? Wouldn't I detect
that? So it's not, uh, it's not good. We
467
00:45:35,110 --> 00:45:38,890
saw that it was draining the battery and
actually in the comments, I don't know if
468
00:45:38,890 --> 00:45:44,500
I had that comment here. Some people were
complaining that, um, I tried and it was
469
00:45:44,500 --> 00:45:53,029
draining my battery. And, um, there is an
impact. Absolutely. Amazon and Google Nest
470
00:45:53,029 --> 00:45:57,710
and all the other parts, aren't you more
worried about that? You know, the always
471
00:45:57,710 --> 00:46:02,400
listening thing from Google and Amazon and
everyone is coming up with some something
472
00:46:02,400 --> 00:46:10,130
like that that's always on. And so that
it's kind of strange because a user's
473
00:46:10,130 --> 00:46:18,369
consent. But at the same time, they don't
completely understand. So there is a gray
474
00:46:18,369 --> 00:46:22,670
line there, like you can say that the
users, OK, you consented to that up,
475
00:46:22,670 --> 00:46:28,549
starting with your with your phone and
listening on the background. But at the
476
00:46:28,549 --> 00:46:34,869
same time, the users don't have the best
understanding. Always. Thank you. Next
477
00:46:34,869 --> 00:46:39,430
question from the front left microphone
first. Thank you for the talk. I would be
478
00:46:39,430 --> 00:46:43,809
interested in how you selected your real
world applications and how many you found
479
00:46:43,809 --> 00:46:51,119
that already use such a framework. So what
was the first part of the question, how
480
00:46:51,119 --> 00:46:56,790
you selected your real world applications
from the marketplace staff if you had any.
481
00:46:56,790 --> 00:47:04,109
So we're trying to do a systematic scan of
the whole market, but it's not easy. So we
482
00:47:04,109 --> 00:47:09,440
not able to do that. There are resources
on the Internet. Luckily, the companies
483
00:47:09,440 --> 00:47:15,710
need to advertise their product. So they
basically publish press releases saying,
484
00:47:15,710 --> 00:47:22,000
you know, this NBA team started using our
product. We did some sort of scanning
485
00:47:22,000 --> 00:47:27,890
through alternative datasets, but
definitely we don't have an exhaustive
486
00:47:27,890 --> 00:47:33,049
list of applications. What I can say,
though, is that there are applications
487
00:47:33,049 --> 00:47:40,250
with. Using such frameworks with nearly up
to, if I remember correctly, up to one
488
00:47:40,250 --> 00:47:49,160
million installations. One notable
example, OK, I'm not entirely sure what I
489
00:47:49,160 --> 00:47:55,380
wanted, but up to a million we definitely
saw. OK, thanks. Do we have more questions
490
00:47:55,380 --> 00:48:02,500
from the Internet? Yes, E.F. is asking, is
he aware of or are you aware sorry? Are
491
00:48:02,500 --> 00:48:05,569
you aware of any framework available by
Google or Apple? In other words, how do we
492
00:48:05,569 --> 00:48:11,960
know that it's not, for instance,
seriously snitching on us? How do we know
493
00:48:11,960 --> 00:48:19,910
that it's not true? It's not serious. Some
maybe Aleksa snitching on us. We don't. I
494
00:48:19,910 --> 00:48:24,160
think that's a that's a very large
discussion. Right. So is the same problem
495
00:48:24,160 --> 00:48:34,059
that these companies are having? Because
if I go back here, basically the users are
496
00:48:34,059 --> 00:48:43,690
accusing them of eavesdropping. Especially
here from reverse engineering those
497
00:48:43,690 --> 00:48:49,869
frameworks, we couldn't find any such
activity, but again, it's very hard to
498
00:48:49,869 --> 00:48:54,259
convince the users that you are listening
to the ultrasound spectrum. You if you're
499
00:48:54,259 --> 00:48:59,769
accessing the whole audible frequencies
through the microphone, you're going to or
500
00:48:59,769 --> 00:49:04,119
you will always find yourself in this
position. So I guess it's the same problem
501
00:49:04,119 --> 00:49:09,339
that Alexa has from Amazon. But in this
case, you can actually solve it by
502
00:49:09,339 --> 00:49:15,410
constraining the spectrum that you gain
access to. Next question from the front
503
00:49:15,410 --> 00:49:21,069
left microphone, please. Has anybody done
an audible demonstration off these beacons
504
00:49:21,069 --> 00:49:26,230
bypassed by transposing them down an
octave or two, I think might be useful for
505
00:49:26,230 --> 00:49:34,089
for or your talk to something like that.
So you mean a demo, but using audible
506
00:49:34,089 --> 00:49:40,630
frequencies? Essentially, there is this
one company, but they are being pretty to
507
00:49:40,630 --> 00:49:44,869
all of these companies are being pretty
secretive with their technology. So they
508
00:49:44,869 --> 00:49:51,430
publish what's needed for marketing
purposes like accuracy sometimes remains
509
00:49:51,430 --> 00:49:57,390
very limited technical details. But apart
from these, you have to get your hands on
510
00:49:57,390 --> 00:50:04,829
the framework somehow and analyze it
yourself. So in this kind of overview we
511
00:50:04,829 --> 00:50:08,130
need for the ecosystem, we had to do
everything by ourselves. There was no
512
00:50:08,130 --> 00:50:15,789
resources out there were very limited, um,
or recording it and playing it down and
513
00:50:15,789 --> 00:50:23,290
transposing it yourself, if you know where
as a beacon of. Possibly I'm not I'm not
514
00:50:23,290 --> 00:50:31,779
entirely sure you could. Yeah. Another
question from our signal, angel mestas,
515
00:50:31,779 --> 00:50:37,789
again asking, um, would it be possible,
even if you have a low pass filter to use,
516
00:50:37,789 --> 00:50:44,810
uh, for instance, the cost effect and high
cost effect to transmit the beacon via
517
00:50:44,810 --> 00:50:53,900
ultrasound, but in a regime which is as
free for the app? So it's basically the
518
00:50:53,900 --> 00:50:59,799
question, can I somehow, via Aliasing USA
address on signal to make a normal signal
519
00:50:59,799 --> 00:51:08,319
out of it? Possibly, I don't know. I think
you are much more creative than I am, so
520
00:51:08,319 --> 00:51:16,819
maybe I should add more bullet points on
this controversialist here. Apparently,
521
00:51:16,819 --> 00:51:23,150
there are many more ways to do this,
possibly like hardware missions. This one
522
00:51:23,150 --> 00:51:29,619
sounds like a good idea, too. So next
question from the real right microphone. I
523
00:51:29,619 --> 00:51:33,559
apologize if you explain the story they
didn't understand, but is is sort of
524
00:51:33,559 --> 00:51:38,819
drowning out the signals, like jamming.
They just broadcasting white noise in that
525
00:51:38,819 --> 00:51:43,810
spectrum, an effective countermeasure. And
as a follow up, if it is, would it
526
00:51:43,810 --> 00:51:56,750
terrorize my dog? So absolutely, it's
effective. I mean, this it works up to
527
00:51:56,750 --> 00:52:01,770
seven meters, but we're not saying it's
not fragile, so you can do that, but it's
528
00:52:01,770 --> 00:52:05,829
noise pollution. And my dog, I don't think
it was happy. I did it for a very limited
529
00:52:05,829 --> 00:52:10,280
time. I could see her ears moving, but I
don't think she would appreciate it if I
530
00:52:10,280 --> 00:52:16,720
had the device at home doing this all the
time. Do we have any more questions from
531
00:52:16,720 --> 00:52:22,460
the Internet? Yes, EULEX is asking to what
extent could we use these for our own
532
00:52:22,460 --> 00:52:26,559
needs? For example, people in repressive
situations, for example, activists could
533
00:52:26,559 --> 00:52:30,630
use it to transmit secret encrypted
messages. Are there any efforts in this
534
00:52:30,630 --> 00:52:40,829
area? Yes, there are. People are
developing ultrasound modems. I think
535
00:52:40,829 --> 00:52:51,030
there is even a tag on it. And yes, of
course there is. So I would say, yes, I'm
536
00:52:51,030 --> 00:52:57,029
not entirely sure about the capabilities
of this channel in terms of bandwidth, but
537
00:52:57,029 --> 00:53:01,890
this is why we we are not advocating to
kill the technology just to make it secure
538
00:53:01,890 --> 00:53:06,900
and know its limitations. So you can do
good stuff with it. And this is what we
539
00:53:06,900 --> 00:53:13,720
want. Next question from the Rio, right?
Yeah, I'm wondering if you could transfer
540
00:53:13,720 --> 00:53:19,859
that technique from the ultrasound range
also to the Audible Range, for example, by
541
00:53:19,859 --> 00:53:26,550
using watermarks, audio, watermarks, and
then, well, your permission thingy with
542
00:53:26,550 --> 00:53:31,740
the ultrasound permissions would be
ineffective and you could also track the
543
00:53:31,740 --> 00:53:37,810
user. How about this? Is it possible audio
watermarks in the audible spectrum? Yeah,
544
00:53:37,810 --> 00:53:42,900
it's absolutely possible. Um, our
countermeasures are not effective against
545
00:53:42,900 --> 00:53:50,490
this. Um, it's just that there is from our
research, just one company doing this. Uh,
546
00:53:50,490 --> 00:53:57,119
so this one, um, I think technically it's
a bit more challenging to do that.
547
00:53:57,119 --> 00:54:02,809
Instead, they're just admitting they are
doing it in a very basic way. So
548
00:54:02,809 --> 00:54:08,480
hopefully, um, if there is a clear way to
do it through ultrasounds, they are not
549
00:54:08,480 --> 00:54:15,400
going to reside reside in the audible
spectrum. But our countermeasures are not
550
00:54:15,400 --> 00:54:22,640
effective against the audible. Um.
Watermarks. Yeah, thanks, next question
551
00:54:22,640 --> 00:54:28,960
from the front left microphone. I've heard
that I don't think it's very credible, but
552
00:54:28,960 --> 00:54:34,079
I've heard that there is some sound on
this sub sound spectrum. There were some
553
00:54:34,079 --> 00:54:40,700
experiments showing that they can
influence our mood, the mood of humans. Is
554
00:54:40,700 --> 00:54:47,900
there any relevant information about how
ultrasounds could affect us? So without
555
00:54:47,900 --> 00:54:54,580
being an expert in this particular area?
I've read similar articles when I was
556
00:54:54,580 --> 00:54:59,190
looking into it. I can tell you it's very
annoying, especially if you're listening
557
00:54:59,190 --> 00:55:05,680
to it through headphones. You cannot
really hear the sound, but you can if
558
00:55:05,680 --> 00:55:11,599
you're using headphones, you can feel the
pressure. So if I don't know what kind of
559
00:55:11,599 --> 00:55:19,809
medical condition you may develop, but you
won't be very sane after. Do we have any
560
00:55:19,809 --> 00:55:27,289
more questions? Yes. One further question,
um, would it be possible to, um, use a
561
00:55:27,289 --> 00:55:33,999
charming solution to get rid of the
signals? Yes, but you you're going to
562
00:55:33,999 --> 00:55:38,450
follow the you know, it's going to result
in noise pollution, but if you are being
563
00:55:38,450 --> 00:55:46,690
paranoid about it, yes, it's and it's, I
think, a straightforward thing to do. Any
564
00:55:46,690 --> 00:55:53,330
more questions? One more on the front left
microphone. Know, you said that physical
565
00:55:53,330 --> 00:55:59,049
objects will block the ultrasound. How
solid do the physical objects need to be?
566
00:55:59,049 --> 00:56:04,680
So, for example, does my pocket block the
ultrasound and thus prevent my phone to
567
00:56:04,680 --> 00:56:11,579
call the environment and vice versa? OK,
well, that's a good question. I don't
568
00:56:11,579 --> 00:56:16,529
think that clothes can actually do that
unless it's very thick. Thin girls
569
00:56:16,529 --> 00:56:27,190
definitely block it. Um. Thick glass, I
would say it reduce the transmission rate,
570
00:56:27,190 --> 00:56:35,559
the signal to noise ratio by a lot, but it
could go through it, so. You need
571
00:56:35,559 --> 00:56:42,690
something quite concrete, metal. I don't
think it goes through it. So are there any
572
00:56:42,690 --> 00:56:48,160
more? Doesn't look like it, maybe, maybe
one more sorry. Oh, good signal, good bye.
573
00:56:48,160 --> 00:57:02,350
Kitty is asking, could you name or compile
a list of tracking programs and apps? So.
574
00:57:02,350 --> 00:57:07,410
That's a good question. We're trying to
make an exhaustive list and try to resolve
575
00:57:07,410 --> 00:57:16,529
this in a systematic way. I've already
listed two Macenta frameworks. One is the
576
00:57:16,529 --> 00:57:20,160
Silverbush one three actually. One is the
Silver Paswan. There is another one used
577
00:57:20,160 --> 00:57:32,940
by single 360. So developed the signal
360, and then there is a listener one.
578
00:57:32,940 --> 00:57:39,609
These are very popular. Um, and then its
developer is incorporating them into their
579
00:57:39,609 --> 00:57:48,749
applications in different ways, offering
varying levels of transparency for the
580
00:57:48,749 --> 00:57:54,339
users. So it's better if you start knowing
what the frameworks are and then trying to
581
00:57:54,339 --> 00:57:59,039
find the applications using them, because
you know what? You're looking in the code
582
00:57:59,039 --> 00:58:06,280
and you can develop some queries and
enabling you to access an ability to to
583
00:58:06,280 --> 00:58:13,509
track which applications are using them.
What what we observed for Silverbush is
584
00:58:13,509 --> 00:58:18,820
basically after the company announced that
they are moving out of the US and because
585
00:58:18,820 --> 00:58:24,390
of the whole backslash, maybe even before
that, um, companies started to drop the
586
00:58:24,390 --> 00:58:30,109
framework. So all their versions had the
framework, but they are not using it
587
00:58:30,109 --> 00:58:52,549
anymore. I think that's it. Thank you very
much, Vasilios Lovelady's.
588
00:58:52,549 --> 00:59:02,932
Subtitles created by c3subtitles.de
in the year 2021. Join, and help us!