Talking Behind Your Back (33c3)

preroll music

Vasilios: Hello, everyone, thanks for coming
today. I'm going to introduce the ultrasound

ecosystem, which is an exotic and kind of
little known ecosystem. So I would like to

start with a short story about the
product, which is also our motivation for

this work. So some time ago, there was a
product that worked in the ultrasound

spectrum that cannot be perceived by
humans. And the product was actually an

interesting idea. It was very promising
and everything, but it also had a fatal

flaw. So now that I've done this
introduction, I would like to tell you

more about the story of the product and
how it came to be and what was it? What

was its lifecycle. So 2012, a company
called SilverPush was a startup in

India. It was founded there and they had
this ultrasound device tracking product.

I'll go more into the technical details
later. So for a couple of years, they were

working on that product. And it wasn't
until 2014 that they basically got some

serious funding from a venture center or
other angel investors for millions. So in

2014, they also got a few months after
they got funded. They also got some press

coverage about the product and they got
some pretty good reviews on their

newspapers and articles about what the
product could do. And at the same time,

they were doing what most of the companies
are doing, like publishing patents about

their technology and everything. So things
later started to go like year after year

and half maybe started to go not that well
for them. The security community noticed

and there was some press coverage about
the product that was not so positive

anymore. So this is one of the very first
emails that appear on the Web regarding

the product. So it's from a W3C
working group. So a researcher there is

basically. Notifying the other members of
the group that, OK, there is this product,

maybe there are transparency issues, and
certainly the users are not aware of

what exactly is going on there. So let's
keep an eye on it. And so this was a very

one of the very first things published
about the product from the privacy and

security perspective. So what happened
then was the press took notice and they

got all those headlines urging users to be
very careful. And, oh, this is a this is

evil, take care. People are eavesdropping
on you and stuff. So, of course, this led

also on the FTC to take action. They
organized a workshop on cross device tracking

in general, I think, and they made specific
mentions for ultrasound cross device tracking

don't worry if you're not familiar with this terms,
I'm going to define everything later. So

what basically they were saying is
transparency issues. How do how do we

protect ourselves? How is that thing
working? So, then the users, of course,

started to react. And there were like many
people who were unhappy, they were

complaining, what is this? I don't want
that thing. So people were actually

suggesting solutions and the solutions
that were making sense. And of course, you

have always the users that are completely
immune to what you have there. So what

happened then is like five months after
the FTC took much more serious action

regarding this specific product. So it
sent a letter to all the developers. And

the letter was essentially saying, you
know, you're using this framework in

Europe. We've seen it in Google Play
store. It's not enough that you are asking

for the microphone permission. You should
let the users know that you are tracking

them if you are doing so. Otherwise, you
are violating rule X, Y, Z, and you're not

allowed to do that. So this was pretty
serious, I would say. And what happened

next is basically the company withdrew
from the US market and said, you know, we

have nothing to do with the U.S. market
and this product is not active there. You

shouldn't be concerned. So end of story
like the product is not out there in the

US at least anymore. Are we safe? So it
seemed to us that it was assumed that this

was an isolated security incident. And to
be fair, very little became known about

the technology. At this point. The press
moved on to other hot topics at the time,

people went quiet, like if people are not
using it, it's fine. So everyone

seemed happy. But we're curious people. So
we had lots of questions that were not

answered. So our main questions was like
why they were using ultrasounds. We'll see

that what they are doing, you can do with
our technologies, how such frameworks

work. We had no idea there was no coverage
or nothing about it. The technical,

technically speaking, out there, are there
other such products there? Because we were

aware of one. Everyone on all the articles
was referring to that one product, but we

were not sure if there are others doing
the same thing. And of course, we were

looking for a report about the whole
ecosystem and how it works. And there was

nothing. So what do you do then if if
there are no technical resources?

Basically, we decided to do our own
research and come up with this report that

we were lacking. So we're done with
motivation so far. We were pretty pumped

up about looking into it. OK, what's
there? The rest of the presentation will

go as follows. Like first I'm going to
introduce ultrasound tracking and other

terminology, then I'm going to go on with
the attack details. And indeed, we have an

attack again against the Tor browser. Then
we're doing a formal security analysis of

the ecosystem and try to pinpoint the
things that went wrong. And then we'll try

to introduce our countermeasures and
advocate for proper practices. So to begin

with, I'm Vasilis. I've done this work
with other curious people. These are

showing how Yanick Fratantonio, Christopher
Kruegel and Giovanni Vigna from UCSB and also

Federico Maggi from Polytechnical
Damilola. Let's now start with the

ecosystem, so apparently ultrasounds are
used in a lot of places and they can be

utilized for different purposes, some of
them are cross device tracking that are

referred already to audience analytics,
synchronized content, proximity, marketing

and device pairing. You can do some other
things, but you will see them later. So to

begin with what cross device tracking is,
cross device tracking is basically the holy

grail for marketers right now because
you're using your multiple devices

smartphone, laptop, computer, maybe your
TV and to them, your appear as different

people. And they all want to be able to
link to link those devices to know that

you're the same person so that they can
build their profiles more accurately. So,

for instance, if you're watching an ad on
the TV, they want to be able to know that

it's you so that they can push relevant
ads from your smartphone or follow up ads.

Um. So this is employed by major
advertising networks, and there are two

ways to do it, either deterministically or
probabilistically, that deterministic

approach is much more reliable. You get
100 percent accuracy and works as follows.

If you are Facebook, the users are heavily
incentivized to log in from all their

devices. So what happens is that. You can
immediately know that, OK, this user has

these three devices and I can put relevant
content to all of them. However, if you

are not Facebook or Google you, it's much
more unlikely that the users would want to

log into your platform from their
different devices. So you have to look for

alternatives. And one tool to come up with
those alternatives are ultrasound beacons.

So, um, ultrasound tracking products are
using ultrasound because they may sound

exotic, but basically there they are. What
they are doing is they are encoding a

sequence of symbols, um, in a very high
frequency that it's inaudible by humans.

That's the first key feature. The second one
is they can be emitted by most commercial

speakers and they can be captured by most
commercial microphones, for instance,

found on your smartphone. So the technical
details are the following. I know there

are a lot of experts in these kinds of
things here, so I'm averaging out what how

the companies are doing it right now. I'm
not saying that this is the best way to do

it, but this is more or less what they're
doing. Of course, they have patents, so

each one of them is doing a slightly
different thing so they don't overlap.

They're using the near ultrasound spectrum
between the eight eight kilohertz and 20

kilohertz, which is inaudible by usually
by adults. They divide it in smaller

chunks. So if you divide it in chunks that
have size of 75 Hertz, you get 26, about 26

chunks, and then you can assign letter of
the alphabet on each one of them. And then

what they are doing is usually within four
to five seconds. They emit sequences of

characters. Usually they contain for four
to six characters in there, and they use

it to incorporate a unique ID
corresponding to their source, they attach

the beacon to. So there is no ultrasound
beacon standard, as I said previously, but

there are lots of patents, so each one of
them is doing a slightly different thing.

But this is a basic principle. We did some
experiments and we found out that within

seven meters, you get pretty good accuracy
in low error rate. So of course, this depends

exactly how you encode things. But with
applications found on Google Play, this

worked up to seven meters. Um, we couldn't
find computer speakers that were not able

to emit near ultrasound frequencies and
work with this technology and.. we this is

pretty known for this kind of frequencies,
they cannot penetrate through physical

objects, but this is not a problem for
their purposes. And we did some

experiments with our research assistant
and we can say that they are audible by

animals. So if you combine cross device
tracking and ultrasound because you get

ultrasound cross device tracking. So now what
you can do with this and this is this is a

pretty good idea, actually, because it
offers high accuracy, you don't ask the

users to log in, which is very high, very
demanding thing to ask for. You can embed

those beacons in websites or TV ads, and
this technology, however, requires some

sort of sophisticated backend
infrastructure. We're going to see more

about it later. And you also need the
network of publishers who are willing to

incorporate incorporate beacons in their
content, whatever this content is. And

then, of course, you need an ultrasound
cross device tracking framework that is going

to run on the user's mobile device, a
smartphone. So these frameworks are

essentially and as the advertising SDK is the
key that the developers can use to display

ads on their free apps. So it's not that
the developers are going to incorporate

the ultrasound framework is going to
incorporate an advertising SDK with

varying degrees of understanding of what
it does. So here is how ultrasound cross device

tracking works. On step one, basically, we
have the advertising client. He just wants

to advertise, advertises his products. He
goes to the ultrasound cross device

tracking provider who has the
infrastructure set up, set up a campaign,

and they provide their associates a unique
ultrasound because with this campaign and

then pushes this become to content
publishers to incorporate them

incorporated into their content, depending
on what the advertiser advertising client

is trying to achieve. So this is step
three or step for a user is basically

accessing all of those content providers
either. This is a TV ad or a website on

the Internet and one this once this
content is loaded or displayed by your TV.

At the same time, the device, the devices
speakers are emitting the ultrasounds. And

if you have the ultrasound cross device tracking
framework on your phone, which is usually

listening on the background, then it picks
up the ultrasound and on step six, it

submits it back to the service provider,
which now knows that, OK, this guy has

watched this DVR or whatever it is, and
I'm going to add this to his profile and

push our target dates back to his device.
So, of course, by doing this, they're just

trying to improve, improve their
conversion rate and get more customers.

Another use of ultrasounds currently in
practice is proximity marketing, so venues

basically set up multiple, multiple
ultrasound meters. This is kind of fancy

name for speakers and this is kind of the
nice thing about the ultrasound. You just

need speakers. So they put this in
multiple locations in their venue, either

a supermarket or a stadium, for instance,
and then there is a customer up. If you're

a supermarket, there is a supermarket up.
If you're an NBA team, which will see

later, you have this fun application that
the fans of your team can download

and install on their smartphones. And then
once this app, this happens, listing on

the background and it picks up the
ultrasound and submits them back to the

company. So the main purpose of using is
this is basically to study in user

behavior, in user behavior, provide real
time notifications like, OK, you are in

this aisle on the supermarket, but if you
just walk two meters down, you're going to

see this product in discount. Or the third
point, which kind of incentivizes the

users more, is basically you're offering
reward points for users visiting your

store. And actually there is a product
doing exactly that on the market. So some

other uses are device pairing. And this
basically relies on the fact that

ultrasounds do not penetrate through
objects. So if you have a small TV, say,

with or Chromecast, for instance, they can
emit random PIN through ultrasound. Your

device picks it up and submits it back to
the device through the Internet. And now

you've proved that you are on the same
physical location with the with Chromecast

or whatever your TV is. Also, Google
recently acquired sleek login. They are

also using ultrasounds for authentication.
It's not entirely clear what their product

is about, though. And also you have
audience measurement and analytics. So

what they are doing is basically if you're
if you incorporate multiple beacons in the

night, then you can basically track the
reactions and the behavior of the users of

it, of the audience in the sense that
first, you know, how many people have

watched your ad a second, you know what
happened. So if they show it's Sanderlin

between and this, so they submit only the
first beacon of the two, if you have two,

then you also track their behavior. OK, so
we've seen all these technologies and then

we started wondering how secure is that
thing? Like, OK, what security measures

are there applied by companies and
everything? So I'm going to immediately

start with the exploitation of the
technology. So to do that, we just need

the computer with speakers and the Tor browser
and the smartphone with an ultrasound app

and a state level advisory. I'm going to
say more about the state level advisory

later, but just keep in mind that it's on
the Tor threat model, so. I have a

video of the attack. I'm going to stop it,
I'm going to pose it in different places

to explain some more stuff. Yeah, OK, so
I'm going to set up the scene before that.

So let's make the assumption that we have
a whistle blower that wants to leak some

documents to a journalist, but he doesn't
know that the journalist is working with

the government and his main intent is
basically to deanonymize him. So the

journalist does the following, asks the
whistleblower to upload the documents to a

Tor hidden service or a website that he owns.
And the whistleblower basically thinking

that he's safe to do that through Tor
loads the page. So now I'm having I have the

demo, which is exactly that implements
exactly that scenario. So the whistle

blower opens the Tor browser, so the setup is
the following, we have the phone next to

the computer. This can be up to seven
meters away, but for practical purposes,

it has to be next to the computer. So we
have the Tor browser. What are we going to do

first? For the purpose of the demo, we use
them smart for listening framework that's

visible to the.. to the user. This is
basically the demo(?). Those apps, ultrasound

cross device tracking apps run in the background,
so now we're setting set it on listening

mode so that it starts listening. Of
course, in normal framework, the user

doesn't have to do that part. But we want
to show that. We want to show that what's

happening. So now the whistleblower is
going to load the innocuous were paid,

suggested by the journalist and see what
happens to. OK, now we've loaded the page

and the phone is listening in reality in
the background, so let's see what happens.

OK, this is looks pretty bad. We have lots
of information about the user visiting our

hidden service. I assume you already have some
clues about how this happened, what the

information that we have is the following.
First of all. We have his IP address,

phone number. Don't call this phone
number, because this isn't right. The ID

is he may end his Google account email. So
this is enough to say and his location, of

course, and this is enough to say that we
essentially deanonymized him, even if we

had the IP address, that would have been
enough. So before I explain exactly how

the attacked work, I'm going to introduce
some tools that the attackers have at

their disposal. The first one is a Bitcoin
injection. So what you can essentially do

is basically craft your own ultrasound
beacons and push them to devices,

listening for beacons, and then their
devices are going to treat them like valid

beacons and submit them back to the
company's backend. And then the same

things. Basically, you can also replace
ultrasound beacons, meaning that you can

capture them from virus location. And this
is actually happening on the wild at a

large scale for a specific application.
And then once you capture those beacons,

you can replace them back to the company's
back end through the user's devices to

give you a clue. There is a company that
incentivizes users to visit stores by

providing them offers and end points when
they are visiting stores and people are

capturing the beacons and are replaying them
back to their devices from home. So they

are selling the beacons through the
Internet so that they don't have to go to

the actual stores. OK, the problem here is
basically that the framework is handling

every beacon. It doesn't have a way to
distinguish between the valid and

maliciously crafted beacons. And my favorite
tool for the attackers is basically a beacon

trap, which is a code snippet that
once you loaded, you basically reproduce

one or more inaudible beacons that the
attacker chose to. So this can happen in

lots of ways on the demo. I use the first
one. So you build a website and you have

some JavaScript there just playing the
ultrasounds from the back. What else you can

do is basically start crosseyed scripting
vulnerability. Just exploit it on any

random website and then you can inject
beacons to the visitors of this website

or a man-in-the-middle attacks just
adding or javascript snippet on that

user's traffic or they send an audio
message to the to the victim. So how did

Tor deanonymization attack work? It's the
following. So first the adversary needs to

set up, set up a campaign, and then once
he captures the the beacon associated with

that campaign, he builds a beacon trap and
essentially on step three lures, the user

to visit it. This is what the journalist
basically did for the whistleblower on our

scenario. Then the user loads the
resource. He has no idea this is possible.

And she slapped him amidst the ultrasound,
beacon. If you if your smartphone has such a

framework, it's going to pick it up and
submit it back to the provider and I don't

know about you, but when I'm using Tor,
I'm not connecting my phone through to the

Internet, through the Tor network. My
phone is connected through my normal Wi-

Fi. So now the ultrasound service provider
knows that the you know, this smartphone

device omitted that specific beacon. And
then I step seven, basically the

adversary, which is state level adversary.
Can simply subpoena the provider for the

AP or other identifiers, which from what
we've seen, they collect plenty of them.

OK, so the first two elements, we have
them already like the Tor browser

computer, which biggest fine smartphone
with ultrasound tracking enabled

framework. Fine. What about the state
level adversity? So we didn't have a state

level adversity handy. So what we did is
basically we redirected the

traffic from step six to the advertized
backend. And I want to stress a point

here. This is not. A long, long shot
assumption. So what we've seen in October

is the following. I don't know how many of
you realize, but AT&T was running a spy

program, a thing called Hammesfahr, and it
was providing paid access to governments

only with an administrative subpoena,
which is not doesn't even need to be

obtained by it's ads. So it's pretty easy
for them to get access to this kind of

data. Especially we're talking about an IP
address. It's not it's very easy for them

to get it. So we also came up with some
more attacks. First one is profile,

corruption. Advertisers really like to
build profiles about you, your interests

and your behavior. So what you are
basically doing is you can inject beacons

to other people or even to your own phone
and then you can malform their profile.

Exactly. The impact of this attack depends
on how the backend of the advertising

company and the infrastructure works, but
the attack is definitely possible. And

then there is information leakage attack
were works under a similar assumption. You

can replay Beacon's eavesdrop and replay
because your own phone to make your

profile similar to that of the victims.
And then based on how recommendation

systems work, you're very likely to get
similar arts and similar content with that

of the victims. So of course, this also
depends about exactly how the

recommendation system is implemented, but
it's definitely possible. OK, so we've

seen certain things that makes us think
that, OK, the ecosystem is not very

secure. Um, we try to find out exactly why
this happened. So we did a security

evaluation or we came up with four points.
The first one is that we came up with we

realized that the threat model is
inaccurate, that ultrasound, because none

of the implementations we've seen had any
security features. Um, they also violated

the fundamental security principle and
they lacked transparency when it comes

when it came to user interface. So let's
go through them one by one. So inaccurate

and model. Basically what they do is
basically they rely on the fact that

ultrasounds cannot penetrate the walls and
they travel up to seven meters reliably.

However, as I said, as a matter of fact,
they also assume that you cannot capture

and replay because because of that, that's
the reason, um, what what's happening in

practice, that you can get really close
using beacon traps. So their assumption

is not that accurate. Um, also, the
security capabilities of beacons are

heavily constrained by the low bandwidth
the channel is has the limited time that

you have to reach the users. So if someone
is in a supermarket, he's not going to

stop somewhere for a very long time. So
you have limited time and a noisy

environment. So you want a very low error
rate. And adding crypto to the beacons

it may not be a good idea, but it also
results. This also results in replay in

injection attacks being possible. Um, we
also hear the violation of the privilege

of, uh, sorry, the principle of least privilege.
So what happens is basically all these

apps need full access to the microphone.
And based on the way it works, it's

completely unnecessary for them to gain
access to the audible frequencies.

However, even if they want to, there's no
way to gain access only to the ultrasound

spectrum, both in Android and iOS. You
have to gain either access to the whole

spectrum or no access at all. So this, of
course, results in the first malicious

developers can at any time start using
their access to the microphone. And of

course, all the benign ultrasound enabled
apps are perceived by as malicious by the

users. And this actually will say more
about it later. So lack of transparency is

inclose. This is a bad combination with
what exactly we've seen previously,

because it that we've observed large
discrepancies between apps when it comes

to informing the users and also lots of
discrepancies when it comes to providing

opt out options. And there is a conflict
of interest there, because if you're a

framework developer, developer, you want
to advise for proper practices for your

customers, but you are not you're not
going to enforce them or kind of blackmail

them. Either you do it properly or you're
not using my framework. So there is a

conflict of interest there. So what
happened because of a lack of

transparency is the following. Signals 360 is
one of those frameworks. An NBA team

started using this in May. And then a few
months after there is a sue and someone

claims, you know, that thing is listening
in the background. And what's interesting

is on the claim, what they are saying is,
OK, I gave permission through the Android

permission system for them to access the
microphone, but it was not explained to me

exactly what they were doing. And this is
in close ties with what the FTC was saying

in the letter a few months ago. Also,
again, the same story, um, football team

starts using such a framework a few months
after people are complaining that they are

being eavesdropped on. Um, I think what
happened here is that. When the team was

playing a match, the application started
listening for ultrasounds, but not all

your fans are going to be in the stadium,
so you end up listening for ultrasounds in

a church and other places. So, yeah,
people were also pissed. Um, OK, just to

put it into perspective how prevalent
these technologies are, the ecosystem is

growing. Even though that one company
withdrew. There are other companies in the

ecosystem are coming up with new products
as well. So the number of users is

relatively low, but it's also very hard to
estimate right now. We could find around

10 companies offering ultrasound related
products and the majority of them is

gathered around proximity marketing. There
was only one company doing ultrasound

cross device tracking. At least we found
one. Um, and this is mainly due to

infrastructure complexity. It's not easy
to do all those things. And secondly, I

also believe that the whole backslash from
the security community is incentivized

other companies from joining because they
don't want a tarnished reputation. OK, so

we have this situation right now.
Companies are using ultrasound. What are

we going to do? So this was our initial
idea. This is what we thought first. But

we want to fix things. So we tried to come
up with certain steps that we need to take

to actually fix that thing and make it
usable, but not dangerous. Um, so we

listed what's wrong with it. We did it
already. We we developed some quick fixes

that I'm going to present later and medium
term solutions as well. And then we

started advocating for a long term changes
that are going to make the ecosystem

reliable. And we need the involvement from
the community there. Definitely. So. We

developed some short and medium term
solutions, um, the first one is a browser

extension, our browser extension basically
does the following is based on HTML5, the

Web audio API. Um, it filters all audio
sources and places a filter between the

audio source and the destination on the
Web page and filters out ultrasounds. To

do that, we use a heisel filter that
attenuates all frequencies above 18kHz

and it works pretty reliably. And
we leave all audible frequencies, intact.

But it's not going to work with
obsolete legacy technologies such as

flash. OK, we also have an adroit
permission, I think this somewhat more

medium term solution, what we did is we
developed a unique developed parts for the

Android permission system. This allows for
fine grained control over the audio channel,

basically separates the permission needed
for listening to audible sound and the

permission needed for listening to the
ultrasound spectrum. So at least we force the

applications to specifically declare that
they are going to listen to four

ultrasounds. And of course, users can, on
the latest Android versions, can also

disable this permission and it can act as
an opt out option if the app is not

providing it. We also initiated discussion
on the Turbo Tracker, but, um, we have,

um, we are advocating for some long term
solutions, so we really need some

standardization here. Um, let's agree on
ultrasound to confirm that and decide what

security features can be there. I mean, we
need to figure out what's technically

possible there because it's not clear. And
then once we have a standard, we can start

building some APIs. And the APIs are very
nice idea because, um, they will work as

the Bluetooth APIs work, meaning that they
will provide some methods to discover,

process, generate and emit the sound
beacons through a new API related

permission. And this means that we will
stop having overprivileged apps. We won't

need access to the microphone anymore,
which is a huge problem right now. And of

course, the applications will not be
considered spying anymore. And there is

also another problem that we found out
while we were playing with those shops.

Um, if you have a framework listening
through the microphone, other apps cannot

access it. So we are trying to open the
camera app to record the video on the app.

Camera app was crashing because the framework
was locking the access to the

microphone. Now we may have some
developers from frameworks saying, you

know, I'm not going to use your API. I'm
going to keep asking for access to the

microphone. But we can force them to use
this API if we somehow, um, by default

filter out the ultrasound frequencies
from the microphone and

provide the way to the user to enable them
on a pure application basis from his

phone. OK, so. Here's what we did, um, we
analyzed them, multiple ultrasound

tracking technologies, we saw what what's
out there in the real world and reverse

engineered such frameworks. We identified,
um, quite a few security shortcomings. We

introduced our attacks and proposed some,
um, usable countermeasures. Um, and

hopefully we initiated the discussion
about standardizing ultrasound because,

um, but there are still things left to do.
So for the application developers, please,

um, explicitly notify the users about what
your app is doing. Many of them would

appreciate to know that. Um, also, we need
to improve transparency in the data

collection process because they collecting
lots of data and very few information were

available about what kind of data they
framework's collect. Um, we also think

it's a good idea to have an opt in option
if it's not too much to ask, at least an

opt out and standard security practices,
um, as always. So framework providers

basically need to make sure that the
developers inform the users and also make

sure that the users consent regularly to
listening for because like it's not enough

if you consent once and then a month after
the app is still listening for ultrasound beacons

you have to periodically ask the user if it's
still okay to do that. Um. Ideally, every time

you are going to listen and then, of
course, we need to work on standardizing

ultrasound because this is going to be a
long process and then building the

specialized, specialized API. Hopefully
this is going to be easier once we have a

standard and see what kind of
authentication mechanisms can we have in

this kind of constrained transmission
channel. So..

applause

Herald: Thank you Vasilios. If you have any
questions, please do line up at the four

microphones here in the walkways and the
first question will be the front

microphone here.
Mic: Hello and thank you for your

presentation. And I have a couple of
questions to ask that are technical and

they are very related. First of all, do
you think that blocking out in our system

level the high frequencies for either
microphone or the speakers as well, a

something that is technically feasible and
will not put a very high latency in the

processing?
Vasilios: So we did that through the

permission. You are talking
about the smartphone right?

Mic: Yeah, basically, because you have to
have a real time sound and microphone

feedback.
Vasilios: So we did that with the

permission. And I think it's not it's not
to resource demanding, if that's

your question. So it's
definitely possible to do that.

Mic: And the second part is, so
there is a new market maybe for some

companies producing and microphones and
speakers that explicitly block out

ultrasounds, right?
Vasilios: Possibly. Possibly. Um, I'm not

sure if you can do this from the
application level. We developed parts for

the Android system. I think our first
approach back then was basically try to

build an app to do that from the
application, from the user land. And

basically, I'm not sure if you can I doubt
actually an Android if you can filter out

ultrasounds. But from a browser, we have
our extension. It works on Chrome. You can

easily use our code to do the
same thing on the Firefox.

Mic: Thanks.
Herald: The next question is from the

front right microphone.
Mic: Thank you for your talk. I have a

question about the attack requirements
against the whistleblower using Tor.

I'm curious, the attacker has access to
the app on the smartphone and also access

to the smartphone microphone. Wouldn't the
attacker then be able to just listen in on

the conversation of the whistleblower and
thereby identify him?

Vasilios: Yeah, absolutely. Absolutely.
It's a major problem. The problem is that

they have access to the microphone. So
this is very this is very real and it's

not going to be resolved even if we had
access only to the ultrasound spectrum.

What we're saying is basically, if we only
had access to the ultrasound spectrum,

you're still uhm you are still vulnerable
to these attacks unless you incorporate

some crypto mechanisms that prevent these
things from happening. Is this your

question or?
Mic: Um, well, I can still pull off the

same attack if I don't
use ultrasound right?

Vasilios: Through the audible spectrum?
Mic: Yes,

Vasilios: You can absolutely do. There is
one company doing tracking in the audible

spectrum. This is much harder to mitigate.
We're looking into it about ways, but

there are so many ways to incorporate
beacons in the audible spectrum. The thing

is that there is not much of an ecosystem
in this area right now that so you don't

have lots of frameworks are there as many
as you have for ultrasounds.

Mic: Thank you.
Herald: Our next question will be from

the Internet via our signal angel
Signal Angel: $Username is asking, have

you heard about exploiting parricide
ultrasound emiters like IC component's?

Vasilios: Can you please
repeat the question?

Signal Angel: Yes, sure. The question is,
can you use other components on the main

board or maybe the hard disk to emit
ultrasounds and then broadcast the beacon

via this?
Vailios: Uh. So that's a very that's a

very good question. The answer is I don't
know, possibly, and it's very scary. Um,

hopefully not, but I doubt it. I think
there should be a way to do it. Um, maybe

the problem is that you cannot do this
completely in a completely inaudible way.

Like you may be able to meet ultrasounds,
but you will also emit some sort of sound

in the audible spectrum so that the user
will know that something is going on.

Herald: The next question
from the left microphone.

Mic: Thank you for your talk and
especially thanks for the research. So,

uh, do you know of any framework's or, uh,
STKs that cash the beacon's they find?

Because for my use case, I my phone was
mostly offline. I just make it online when

I have to check
something. So I'm not that concerned. But

you do you know, if they like cash the
beacons and and submit them later

something like this. Of course they do.
I'm not surprised, unfortunately. Yeah.

Thanks. Next question from the rear.
Right. Oh, what is the data rate? You can

send in the ultrasound. Very good
question. And it's totally relevant to the

cryptographic mechanisms we want to
incorporate from our experiments. Um, in

four seconds you can basically send like
five to six alphabet characters if you're

willing to kind of reduce the range a lot
less in less than seven meters, you may be

able to send more. But the standard is not
very robust in this sense. But these

experiments were done with this kind of
naive encoding that most of the companies

are using. So if you do the encoding in a
very smart way, possibly you can increase

that. And a small second part, what's the
energy consumption on the phone if that is

running all the time? Wouldn't I detect
that? So it's not, uh, it's not good. We

saw that it was draining the battery and
actually in the comments, I don't know if

I had that comment here. Some people were
complaining that, um, I tried and it was

draining my battery. And, um, there is an
impact. Absolutely. Amazon and Google Nest

and all the other parts, aren't you more
worried about that? You know, the always

listening thing from Google and Amazon and
everyone is coming up with some something

like that that's always on. And so that
it's kind of strange because a user's

consent. But at the same time, they don't
completely understand. So there is a gray

line there, like you can say that the
users, OK, you consented to that up,

starting with your with your phone and
listening on the background. But at the

same time, the users don't have the best
understanding. Always. Thank you. Next

question from the front left microphone
first. Thank you for the talk. I would be

interested in how you selected your real
world applications and how many you found

that already use such a framework. So what
was the first part of the question, how

you selected your real world applications
from the marketplace staff if you had any.

So we're trying to do a systematic scan of
the whole market, but it's not easy. So we

not able to do that. There are resources
on the Internet. Luckily, the companies

need to advertise their product. So they
basically publish press releases saying,

you know, this NBA team started using our
product. We did some sort of scanning

through alternative datasets, but
definitely we don't have an exhaustive

list of applications. What I can say,
though, is that there are applications

with. Using such frameworks with nearly up
to, if I remember correctly, up to one

million installations. One notable
example, OK, I'm not entirely sure what I

wanted, but up to a million we definitely
saw. OK, thanks. Do we have more questions

from the Internet? Yes, E.F. is asking, is
he aware of or are you aware sorry? Are

you aware of any framework available by
Google or Apple? In other words, how do we

know that it's not, for instance,
seriously snitching on us? How do we know

that it's not true? It's not serious. Some
maybe Aleksa snitching on us. We don't. I

think that's a that's a very large
discussion. Right. So is the same problem

that these companies are having? Because
if I go back here, basically the users are

accusing them of eavesdropping. Especially
here from reverse engineering those

frameworks, we couldn't find any such
activity, but again, it's very hard to

convince the users that you are listening
to the ultrasound spectrum. You if you're

accessing the whole audible frequencies
through the microphone, you're going to or

you will always find yourself in this
position. So I guess it's the same problem

that Alexa has from Amazon. But in this
case, you can actually solve it by

constraining the spectrum that you gain
access to. Next question from the front

left microphone, please. Has anybody done
an audible demonstration off these beacons

bypassed by transposing them down an
octave or two, I think might be useful for

for or your talk to something like that.
So you mean a demo, but using audible

frequencies? Essentially, there is this
one company, but they are being pretty to

all of these companies are being pretty
secretive with their technology. So they

publish what's needed for marketing
purposes like accuracy sometimes remains

very limited technical details. But apart
from these, you have to get your hands on

the framework somehow and analyze it
yourself. So in this kind of overview we

need for the ecosystem, we had to do
everything by ourselves. There was no

resources out there were very limited, um,
or recording it and playing it down and

transposing it yourself, if you know where
as a beacon of. Possibly I'm not I'm not

entirely sure you could. Yeah. Another
question from our signal, angel mestas,

again asking, um, would it be possible,
even if you have a low pass filter to use,

uh, for instance, the cost effect and high
cost effect to transmit the beacon via

ultrasound, but in a regime which is as
free for the app? So it's basically the

question, can I somehow, via Aliasing USA
address on signal to make a normal signal

out of it? Possibly, I don't know. I think
you are much more creative than I am, so

maybe I should add more bullet points on
this controversialist here. Apparently,

there are many more ways to do this,
possibly like hardware missions. This one

sounds like a good idea, too. So next
question from the real right microphone. I

apologize if you explain the story they
didn't understand, but is is sort of

drowning out the signals, like jamming.
They just broadcasting white noise in that

spectrum, an effective countermeasure. And
as a follow up, if it is, would it

terrorize my dog? So absolutely, it's
effective. I mean, this it works up to

seven meters, but we're not saying it's
not fragile, so you can do that, but it's

noise pollution. And my dog, I don't think
it was happy. I did it for a very limited

time. I could see her ears moving, but I
don't think she would appreciate it if I

had the device at home doing this all the
time. Do we have any more questions from

the Internet? Yes, EULEX is asking to what
extent could we use these for our own

needs? For example, people in repressive
situations, for example, activists could

use it to transmit secret encrypted
messages. Are there any efforts in this

area? Yes, there are. People are
developing ultrasound modems. I think

there is even a tag on it. And yes, of
course there is. So I would say, yes, I'm

not entirely sure about the capabilities
of this channel in terms of bandwidth, but

this is why we we are not advocating to
kill the technology just to make it secure

and know its limitations. So you can do
good stuff with it. And this is what we

want. Next question from the Rio, right?
Yeah, I'm wondering if you could transfer

that technique from the ultrasound range
also to the Audible Range, for example, by

using watermarks, audio, watermarks, and
then, well, your permission thingy with

the ultrasound permissions would be
ineffective and you could also track the

user. How about this? Is it possible audio
watermarks in the audible spectrum? Yeah,

it's absolutely possible. Um, our
countermeasures are not effective against

this. Um, it's just that there is from our
research, just one company doing this. Uh,

so this one, um, I think technically it's
a bit more challenging to do that.

Instead, they're just admitting they are
doing it in a very basic way. So

hopefully, um, if there is a clear way to
do it through ultrasounds, they are not

going to reside reside in the audible
spectrum. But our countermeasures are not

effective against the audible. Um.
Watermarks. Yeah, thanks, next question

from the front left microphone. I've heard
that I don't think it's very credible, but

I've heard that there is some sound on
this sub sound spectrum. There were some

experiments showing that they can
influence our mood, the mood of humans. Is

there any relevant information about how
ultrasounds could affect us? So without

being an expert in this particular area?
I've read similar articles when I was

looking into it. I can tell you it's very
annoying, especially if you're listening

to it through headphones. You cannot
really hear the sound, but you can if

you're using headphones, you can feel the
pressure. So if I don't know what kind of

medical condition you may develop, but you
won't be very sane after. Do we have any

more questions? Yes. One further question,
um, would it be possible to, um, use a

charming solution to get rid of the
signals? Yes, but you you're going to

follow the you know, it's going to result
in noise pollution, but if you are being

paranoid about it, yes, it's and it's, I
think, a straightforward thing to do. Any

more questions? One more on the front left
microphone. Know, you said that physical

objects will block the ultrasound. How
solid do the physical objects need to be?

So, for example, does my pocket block the
ultrasound and thus prevent my phone to

call the environment and vice versa? OK,
well, that's a good question. I don't

think that clothes can actually do that
unless it's very thick. Thin girls

definitely block it. Um. Thick glass, I
would say it reduce the transmission rate,

the signal to noise ratio by a lot, but it
could go through it, so. You need

something quite concrete, metal. I don't
think it goes through it. So are there any

more? Doesn't look like it, maybe, maybe
one more sorry. Oh, good signal, good bye.

Kitty is asking, could you name or compile
a list of tracking programs and apps? So.

That's a good question. We're trying to
make an exhaustive list and try to resolve

this in a systematic way. I've already
listed two Macenta frameworks. One is the

Silverbush one three actually. One is the
Silver Paswan. There is another one used

by single 360. So developed the signal
360, and then there is a listener one.

These are very popular. Um, and then its
developer is incorporating them into their

applications in different ways, offering
varying levels of transparency for the

users. So it's better if you start knowing
what the frameworks are and then trying to

find the applications using them, because
you know what? You're looking in the code

and you can develop some queries and
enabling you to access an ability to to

track which applications are using them.
What what we observed for Silverbush is

basically after the company announced that
they are moving out of the US and because

of the whole backslash, maybe even before
that, um, companies started to drop the

framework. So all their versions had the
framework, but they are not using it

anymore. I think that's it. Thank you very
much, Vasilios Lovelady's.

Subtitles created by c3subtitles.de
in the year 2021. Join, and help us!