1
00:00:00,000 --> 00:00:19,909
36C3 preroll music
2
00:00:19,909 --> 00:00:26,720
Herald Angel: Our next speaker is jiska.
jiska is attending this conference since
3
00:00:26,720 --> 00:00:32,209
ages like a decade? even more?
jiska: 20, 22 C3
4
00:00:32,209 --> 00:00:36,941
Herald Angel: Long, long time. Sometimes
she is also doing some talks here. The
5
00:00:36,941 --> 00:00:42,010
last one last year was about Bluetooth.
There she was in depth. This time it will
6
00:00:42,010 --> 00:00:49,510
be a more general talk about wireless
protocols NFC, LTE, Wi-Fi, and, of course,
7
00:00:49,510 --> 00:00:56,460
Bluetooth. So she will tell us what is
broken in all those protocols. So have fun
8
00:00:56,460 --> 00:01:00,590
and enjoy the talk "All wireless
communications stacks are equally broken"
9
00:01:00,590 --> 00:01:02,820
by jiska.
10
00:01:02,820 --> 00:01:09,230
applause
11
00:01:09,230 --> 00:01:14,251
jiska: So welcome to my talk. I thought it
first to be a foundation talk, but it will
12
00:01:14,251 --> 00:01:18,570
also have new topics about everything that
is kind of fundamentally broken in
13
00:01:18,570 --> 00:01:23,999
wireless communication and it will cover
anything in your smartphone soul like NFC,
14
00:01:23,999 --> 00:01:30,450
Bluetooth, Wi-Fi, LTE. You could order
them like by communication range or by
15
00:01:30,450 --> 00:01:36,880
specification length or lines of code. But
the thing is, so the specification length
16
00:01:36,880 --> 00:01:41,200
and line of code also mean increased
complexity. And if there is increased
17
00:01:41,200 --> 00:01:47,549
complexity, you might have issues with
security in it, in the very end. And then
18
00:01:47,549 --> 00:01:52,570
there is something that is even worse than
LTE, which is vendor specific additions
19
00:01:52,570 --> 00:01:56,840
that would be when you open like five
instances of IDA and like tried to
20
00:01:56,840 --> 00:02:04,210
analyze where the wireless message is
going and what it is doing. So most of
21
00:02:04,210 --> 00:02:08,810
this in this talk will be about wireless
exploitation and the new stuff will be
22
00:02:08,810 --> 00:02:14,700
fuzzing techniques and a new escalation
target. But everything else is more like a
23
00:02:14,700 --> 00:02:21,320
general view on wireless exploitation. So
first, to understand what the wireless
24
00:02:21,320 --> 00:02:27,380
exploit does is to separate it in
different layers. So there is the lowest
25
00:02:27,380 --> 00:02:31,459
layer, which is some hybrid chip which
runs a firmware, let's say bluetooth
26
00:02:31,459 --> 00:02:35,889
firmware, which is then attached to a
driver. Then there's some privileged
27
00:02:35,889 --> 00:02:40,430
stuff, it depends a bit on what kind of
system you're on. And in the end, there
28
00:02:40,430 --> 00:02:45,040
will be applications and no matter where
you exploit is on that layer that you're
29
00:02:45,040 --> 00:02:50,060
exploiting, some security measures become
ineffective. So, for example, if there is
30
00:02:50,060 --> 00:02:56,589
encryption and you have an exploit for
that layer, it would become ineffective.
31
00:02:56,589 --> 00:03:02,840
And it depends, so the higher you are, the
higher also the exploit prices get. So for
32
00:03:02,840 --> 00:03:08,159
the Wi-Fi RCE, you would be at 100K, for
baseband RCE with local privilege
33
00:03:08,159 --> 00:03:12,859
escalation, it gets already 200 K, and if
it's just a messenger or something, then
34
00:03:12,859 --> 00:03:17,739
it's like really, really high in the
price. So the question is like, why is
35
00:03:17,739 --> 00:03:27,499
this wireless stuff a bit cheaper? So
well, you need a certain distance. And so
36
00:03:27,499 --> 00:03:31,980
that's probably a thing. And then also,
maybe they are just too easy to find. I
37
00:03:31,980 --> 00:03:37,709
don't know. Like at least, maybe for me, I
don't know for normal people. Or maybe the
38
00:03:37,709 --> 00:03:42,629
market demand is not that high for them.
Or they are not privileged enough. I don't
39
00:03:42,629 --> 00:03:49,549
know. But actually they'd need like only
like non or less interaction. So. Yeah.
40
00:03:49,549 --> 00:03:55,959
Still a thing I would say. So within the
group I am working at we had a lot of
41
00:03:55,959 --> 00:04:00,950
wireless research and also tools that we
released. And the first one I think that
42
00:04:00,950 --> 00:04:07,010
was running on a mobile phone is NFCGate,
which is currently managed by Max. Then
43
00:04:07,010 --> 00:04:11,870
there is nexmon, which is our largest
project, which is patching of Broadcom
44
00:04:11,870 --> 00:04:17,030
Wi-Fi. And Matthias, who did that, reached
his goal by just saying, like, I now have
45
00:04:17,030 --> 00:04:24,900
kind of a software defined radio in a
commodity like Broadcom Wi-Fi chip. And so
46
00:04:24,900 --> 00:04:29,140
he was a bit bored and kicked off two new
projects before he left, which he then
47
00:04:29,140 --> 00:04:33,889
handed over. The first one is Qualcomm LTE
and the second one was Broadcom Bluetooth,
48
00:04:33,889 --> 00:04:39,540
which I ended up with. And then we have
someone else who is Milan and he is doing
49
00:04:39,540 --> 00:04:44,600
stuff that comes more like from the
application layer. So he implemented an
50
00:04:44,600 --> 00:04:51,880
open source solution for Apple Airdrop
that you can run on your Raspberry Pi. And
51
00:04:51,880 --> 00:04:57,680
well hackers gonna hack, so this stuff has
been used a lot for exploitation, not by
52
00:04:57,680 --> 00:05:01,760
us, but by others. So there were three
groups using nexmon for Wi-Fi
53
00:05:01,760 --> 00:05:06,420
exploitation, at least like what is
publicly known and like the bigger ones,
54
00:05:06,420 --> 00:05:10,980
maybe I forget someone, but so there is a
lot of exploitation going on there. Then
55
00:05:10,980 --> 00:05:15,510
internal blue has been used to demonstrate
an attack against a key negotiation of
56
00:05:15,510 --> 00:05:22,470
Bluetooth just this august. And the open
Airdrop implementation was used for some
57
00:05:22,470 --> 00:05:28,150
honeypots at Black Hat and AirDos. And
like a lot of stuff is going on there. And
58
00:05:28,150 --> 00:05:31,940
then you might ask yourself, like, yeah,
so, if everybody is using it for
59
00:05:31,940 --> 00:05:39,060
exploitation why don't we just do it
ourselves. And we actually did that and we
60
00:05:39,060 --> 00:05:44,800
even did that for this very first project,
this NFC project. And the most important
61
00:05:44,800 --> 00:05:49,930
thing you need to know about NFC is that
the near field is not really a near field.
62
00:05:49,930 --> 00:05:54,300
So there's -- it's just communication, but
it's not near field communication, which
63
00:05:54,300 --> 00:06:00,110
means so if you are able to forward the
communication, so for example, you have
64
00:06:00,110 --> 00:06:03,800
like your credit card and then a
smartphone with NFC, you could forward it
65
00:06:03,800 --> 00:06:10,090
over the cloud or some server and then to
another smartphone and then to the payment
66
00:06:10,090 --> 00:06:15,300
terminal. And usually there's no time
constraint or distance pounding that would
67
00:06:15,300 --> 00:06:19,400
prevent this. So you can at least forward
and relay messages and you might even be
68
00:06:19,400 --> 00:06:26,060
able to modify them on the way. And some
students of us did like some testing in
69
00:06:26,060 --> 00:06:32,920
some systems of some third parties who
then politely asked them to please stop
70
00:06:32,920 --> 00:06:40,700
the testing. So it was not really a cool
thing overall. Like, not not good to
71
00:06:40,700 --> 00:06:45,510
publish and so on. And the more happy I am
about that there's other researchers who
72
00:06:45,510 --> 00:06:52,830
actually used some other tooling to look
into NFC. So just this month there has
73
00:06:52,830 --> 00:06:57,530
been a talk at Black Hat, so not by us,
but by others about the visa credit cards.
74
00:06:57,530 --> 00:07:05,780
And it's just all broken and it's cool
that some people like just did it anyway.
75
00:07:05,780 --> 00:07:10,180
Yeah. So this is more about -- the NFC
stuff is more about forwarding and the
76
00:07:10,180 --> 00:07:16,160
actual specification, but something that
is also cool is -- If you get code
77
00:07:16,160 --> 00:07:21,830
execution within a chip and this is very
different attack scenario. And for
78
00:07:21,830 --> 00:07:27,180
Bluetooth, I think it's especially bad
because of how everything is designed. And
79
00:07:27,180 --> 00:07:34,250
the first design issue in Bluetooth is
that the encryption key is stored in a way
80
00:07:34,250 --> 00:07:38,170
that the chip can always ask for
encryption keys here at the host, or it's
81
00:07:38,170 --> 00:07:43,860
even already on the chip. And there is no
kind of security there. So whenever you
82
00:07:43,860 --> 00:07:47,800
have code execution on the chip, it means
you can get all the encryption keys, not
83
00:07:47,800 --> 00:07:52,280
of just the active connection, and then
break everything that is kind of a trusted
84
00:07:52,280 --> 00:07:56,630
connection by this key. And that even
breaks features like the Android Smart
85
00:07:56,630 --> 00:08:02,230
Lock. And Android Smart Lock is a thing
you can unlock your Android smartphone, if
86
00:08:02,230 --> 00:08:06,300
you have a trusted device and if you'd
like add this, you might do this for your
87
00:08:06,300 --> 00:08:12,790
car because it's nice in your car when you
have like your audio and your navigation
88
00:08:12,790 --> 00:08:16,940
and everything without a locked
smartphone. But the question is like, how
89
00:08:16,940 --> 00:08:21,480
secure is the Bluetooth of your car? Would
you trust that one to unlock your
90
00:08:21,480 --> 00:08:28,620
smartphone? I don't know. And the next
thing is, so if you have code execution on
91
00:08:28,620 --> 00:08:33,899
a Bluetooth chip, it also means that you
might be able to escalate into some other
92
00:08:33,899 --> 00:08:42,050
components so that you go up all the
layers. The next question, is the exploit
93
00:08:42,050 --> 00:08:46,589
persistent? So let's say I have something
that is running on the chip and I don't
94
00:08:46,589 --> 00:08:52,700
know, extracting encryption keys or doing
whatever. You might ask yourself, so how
95
00:08:52,700 --> 00:08:56,320
long will it be on the chip? I mean, it's
just a Bluetooth chip you switch Bluetooth
96
00:08:56,320 --> 00:09:02,379
off sometimes and then the specifications.
So it's just a page like almost 1000. So
97
00:09:02,379 --> 00:09:06,819
just like the first third of the
specification it says not the HCI_Reset
98
00:09:06,819 --> 00:09:12,500
command will not necessarily perform a
hardware reset. This is implementation
99
00:09:12,500 --> 00:09:17,339
defined. Then I looked into the Cypress
and Broadcom chips and saw, yeah, so if
100
00:09:17,339 --> 00:09:21,880
you do each day reset, it's obviously not
a full hardware reset, it's just flashing
101
00:09:21,880 --> 00:09:26,600
some queues connection stuff here and
there. So there is definitely a memory
102
00:09:26,600 --> 00:09:33,689
area where you could put your exploit and
it would be persistent. So then you might
103
00:09:33,689 --> 00:09:39,329
say, yeah, OK, so what do I do? I don't
know. I put my smartphone into flight mode
104
00:09:39,329 --> 00:09:46,010
for a hard reset. That usually doesn't
work. You might also like reboot your
105
00:09:46,010 --> 00:09:50,110
phone. In most cases, this works. For some
other coexistence stuff, I had the
106
00:09:50,110 --> 00:09:54,409
impression that sometimes so it's a bit
strange, it might not necessarily like,
107
00:09:54,409 --> 00:10:02,889
reset. Turning off for a while might hard-
reset the chip I think. Or you just put
108
00:10:02,889 --> 00:10:08,240
your smartphone in a blender and then.
Yeah. So that might turn off the Bluetooth
109
00:10:08,240 --> 00:10:16,019
chip finally. Yeah. So the next issue is
so let's say we have an exploit running
110
00:10:16,019 --> 00:10:20,889
there, but we first need an exploit. So
the very first step is still missing as a
111
00:10:20,889 --> 00:10:26,480
building block. And after the talk last
year, I did some stuff with the unicorn
112
00:10:26,480 --> 00:10:31,519
and fuzzing on the chip and it was super
slow. And then suddenly Jan showed up and
113
00:10:31,519 --> 00:10:38,809
Jan said, Hey, I want to build a fully
emulated chip for superfast fuzzing and
114
00:10:38,809 --> 00:10:42,889
attach it to Linux and everything should
like run on the real -- like as on a real
115
00:10:42,889 --> 00:10:47,430
system, just the over the air input will
be fast. And I was like, you cannot do
116
00:10:47,430 --> 00:10:51,480
this for a master thesis. And then he was
building that thing within three months
117
00:10:51,480 --> 00:10:57,350
and the remaining three months he was
writing a thesis and e-mails to vendors.
118
00:10:57,350 --> 00:11:02,819
So here we go. What does Frankenstein do?
So it's running on an evaluation board of
119
00:11:02,819 --> 00:11:07,659
that, yeah, it's just a normal Bluetooth
Bord that's connected to a Linux host over
120
00:11:07,659 --> 00:11:14,999
UART and a modem over the air and then you
would snapshot that thing and emulate it
121
00:11:14,999 --> 00:11:20,540
and give it fast input attached to the
real host. And that means that if you find
122
00:11:20,540 --> 00:11:24,839
some vulnerability, it might be within all
the components or it might also be under
123
00:11:24,839 --> 00:11:30,209
Linux host or it might be something that
is full stack. So you have something that
124
00:11:30,209 --> 00:11:34,630
starts on the chip, gets to the host, the
host requests further things and then it
125
00:11:34,630 --> 00:11:38,690
goes back to the chip. So you could build
like quite complex stuff. And for this I
126
00:11:38,690 --> 00:11:45,420
have a short demo video. So the reason why
I do this as a video is that it might
127
00:11:45,420 --> 00:11:49,640
happen that it finds heap overflows
otherwise. And then also it's not super
128
00:11:49,640 --> 00:11:54,550
stable at the moment. So you can see it's
scanning for LE devices and then Wireshark
129
00:11:54,550 --> 00:11:58,369
most of the time would get malformed
packets, but sometimes it could also get
130
00:11:58,369 --> 00:12:08,690
normal packets and like some mesh stuff,
whatever. So this is Frankenstein running.
131
00:12:08,690 --> 00:12:16,140
Ja, so what Jan focused on is early
connection states. That means stuff
132
00:12:16,140 --> 00:12:21,980
where you don't need a pairing. And then
he found like heap overflows there in very
133
00:12:21,980 --> 00:12:28,741
basic package types. So quite interesting.
And then the stuff was fixed, I think, or
134
00:12:28,741 --> 00:12:33,220
hope, whatever. So at least like in very
recent devices and then the iPhone 11 came
135
00:12:33,220 --> 00:12:37,870
out and in contrast to the specification,
over the air, the iPhone 11 says, hey, I'm
136
00:12:37,870 --> 00:12:42,899
Bluetooth 5.1. I was like, wow, first
consumer device, Blueotooth 5.1. And I was
137
00:12:42,899 --> 00:12:48,019
like, I don't really mind my way of the
exploitation as long as I can get code
138
00:12:48,019 --> 00:12:53,139
execution on chip. So if it is with user
interaction and a pairing and whatever, I
139
00:12:53,139 --> 00:12:57,990
don't care as long as I get code execution
on it. And then I was like, okay, let's
140
00:12:57,990 --> 00:13:02,870
add some fuzz cases to Frankenstein,
continue fuzzing. And then I found that
141
00:13:02,870 --> 00:13:08,259
specific evaluation board that Jan was
building this for has a problem with the
142
00:13:08,259 --> 00:13:14,849
heap configuration for certain packet
types. And so if you change that, you
143
00:13:14,849 --> 00:13:19,490
would hard-brick the device. I mean, I
bricked two evaluation boards trying to
144
00:13:19,490 --> 00:13:25,139
fix stuff. So, yeah, it's just
bricked. And so that means for me to
145
00:13:25,139 --> 00:13:30,420
continue fuzzing to write like to port
something like 200 handwritten hooks to
146
00:13:30,420 --> 00:13:35,769
another evaluation board. It's almost
running. So there's just some stuff with
147
00:13:35,769 --> 00:13:40,309
thread switching that's not super smooth
yet, but like it's almost on the next
148
00:13:40,309 --> 00:13:46,069
board. And further plans are to add more
hardware. So we're also working on the
149
00:13:46,069 --> 00:13:52,839
Samsung Galaxy S 10 and probably a MacBook
to get it in there. So then it would not
150
00:13:52,839 --> 00:13:57,460
just be Linux, but at least macOS, maybe
Android. I don't know yet. And another
151
00:13:57,460 --> 00:14:01,069
thing that would be cool and also we
didn't build yet, but it might be feasible
152
00:14:01,069 --> 00:14:07,999
with some USRP X310 over PCI Express and
with FPGA and all the fancy stuff to get
153
00:14:07,999 --> 00:14:12,790
real over the air input, which then would
mean that you would have a full queue like
154
00:14:12,790 --> 00:14:19,879
from over the air real Bluetooth packets
going all the way up and then to a host
155
00:14:19,879 --> 00:14:23,839
and the way back. And you could also use
that just to test your new emulation
156
00:14:23,839 --> 00:14:31,200
scheme or whatever you want to change. So
not just security. Ja, so the next thing
157
00:14:31,200 --> 00:14:36,679
is, so if you have code execution, what do
you do with it? And the normal approach is
158
00:14:36,679 --> 00:14:43,360
to try to go all the layers up. But there
might also be some chip level escalation
159
00:14:43,360 --> 00:14:50,269
and you might immediately see it on the
next picture. So this is from a Broadcom
160
00:14:50,269 --> 00:14:55,519
chip, but that's something that you would
also see in many other chips, which is
161
00:14:55,519 --> 00:14:59,960
that you have a shared antenna. And you
could also have two antennas, but they are
162
00:14:59,960 --> 00:15:04,179
both on 2.5GHz and it's in the very same
smartphone super close next to each other
163
00:15:04,179 --> 00:15:07,970
and you would get interference. So usually
you just have the same antenna and do some
164
00:15:07,970 --> 00:15:12,899
coordination like when it's Bluetooth
sending, when it's Wi-Fi sending so that
165
00:15:12,899 --> 00:15:19,230
they don't interfere. And this feature is
called coexistence and there is tons of
166
00:15:19,230 --> 00:15:24,020
coexistence interfaces. So this is just
the one from Broadcom. And when I saw it,
167
00:15:24,020 --> 00:15:29,290
I was like, oh, Francesco, let's look into
this. You know, all the Wi-Fi stuff, I
168
00:15:29,290 --> 00:15:32,920
know all the Bluetooth stuff let's do
something. And he was like, no, it's just
169
00:15:32,920 --> 00:15:37,499
it's just a marketing feature so that it
can like sell one chip for the price of
170
00:15:37,499 --> 00:15:41,620
two chips or something. And I was like,
no, no, no, it must be an exploitation
171
00:15:41,620 --> 00:15:47,999
feature. So, and then to end this
discussion, I went to Italy for eating
172
00:15:47,999 --> 00:15:53,259
some ice cream and saw reality somewhere
in between. It's more like it's hardcoded
173
00:15:53,259 --> 00:15:58,430
blacklisting for wireless channels and
stuff. It's traffic classes for different
174
00:15:58,430 --> 00:16:02,889
types of traffic for Bluetooth and Wi-Fi.
And you can look it up in tons of patents
175
00:16:02,889 --> 00:16:08,949
and it's like super, super proprietary.
And so we let's say we played a game which
176
00:16:08,949 --> 00:16:14,311
was like I tried to steal his antenna and
he tried to steal my antenna. And so it
177
00:16:14,311 --> 00:16:18,779
turned out, if you do that, yeah, you can
turn off Wi-Fi via Bluetooth, Bluetooth
178
00:16:18,779 --> 00:16:24,709
via Wi-Fi. And then also like on most
phones, you need to reboot them and some
179
00:16:24,709 --> 00:16:28,009
of them even reboot them themselves. So
this is just like a speed accelerated
180
00:16:28,009 --> 00:16:33,879
thing with an Samsung Galaxy S 8 that is
not up to date. For iPhones, you would
181
00:16:33,879 --> 00:16:38,999
just immediately see a reboot without any
interaction of things going off and on. So
182
00:16:38,999 --> 00:16:42,689
Broadcom is still in the process of fixing
it. I don't know if they can fix it, but
183
00:16:42,689 --> 00:16:46,899
they said they could fix it. But something
you should definitely fix is like the
184
00:16:46,899 --> 00:16:50,339
driver itself so that the smartphone
reboots and so on. So I don't know. I
185
00:16:50,339 --> 00:16:55,560
thought it would be fixed, actually, in
iOS 13 because it mentions Francesco and
186
00:16:55,560 --> 00:17:00,689
me, but still on 13.3 I don't know, it,
it's still - um - you can
187
00:17:00,689 --> 00:17:04,059
still crash the iPhone that way. But
188
00:17:04,059 --> 00:17:08,430
it's just some resource blocking so it's
like not super dangerous thing, I would
189
00:17:08,430 --> 00:17:13,850
say. And you still need a Bluetooth RCE
before you could do it and so on. But
190
00:17:13,850 --> 00:17:21,370
still not cool that it's still not fixed.
Yeah, so what about the other stacks and
191
00:17:21,370 --> 00:17:26,820
the escalations? So there is tons of
different Bluetooth stacks, so it's really
192
00:17:26,820 --> 00:17:32,910
a mess. And obviously because of
Frankenstein, we had this first Linux
193
00:17:32,910 --> 00:17:39,480
Bluetooth stack attached and so on. But,
yeah. So what has been there for a
194
00:17:39,480 --> 00:17:43,780
wireless 2017, these BlueBorne attacks,
you might have heard of them. And they
195
00:17:43,780 --> 00:17:47,491
found escalations like on Android,
Windows, Linux, iOS, whatever. And then
196
00:17:47,491 --> 00:17:52,530
you might say, like in security, you often
say, so someone looked into it. It must be
197
00:17:52,530 --> 00:17:57,760
secure now. And then there's so many
features coming. So there's all these IoT
198
00:17:57,760 --> 00:18:01,750
devices. So everybody nowadays has
wireless headphones and fitness trackers
199
00:18:01,750 --> 00:18:06,650
and Bluetooth always on. And in the Apple
ecosystem, it's really a mess. So if you
200
00:18:06,650 --> 00:18:11,080
have more than one Apple device, you would
have Bluetooth enabled all the time.
201
00:18:11,080 --> 00:18:14,150
Otherwise you couldn't use a lot of
features. And then there is like stuff
202
00:18:14,150 --> 00:18:18,450
like Web Bluetooth so Bluetooth LE
support from within the browser. So it's
203
00:18:18,450 --> 00:18:23,450
like a lot of new attack surfaces that
arised since then. I think -- So that's
204
00:18:23,450 --> 00:18:31,570
more my personal estimation is, 2020 might
be more BlueBorne like attacks. The
205
00:18:31,570 --> 00:18:34,980
saddest Bluetooth stack somehow is the
Linux Bluetooth stack. So I don't want to
206
00:18:34,980 --> 00:18:39,000
blame the developers there. I mean, it's
not their fault, but it's not enough
207
00:18:39,000 --> 00:18:43,720
people contributing to that project. And
if you would try to to analyze something
208
00:18:43,720 --> 00:18:48,050
that is going on in the stack and you
don't really know what is going on, you
209
00:18:48,050 --> 00:18:52,090
would do git blame or whatever and you
would always see the same guy as the
210
00:18:52,090 --> 00:18:56,710
commiter. So at least if you're on a
specific problem, then there's only one
211
00:18:56,710 --> 00:19:01,620
person committing there. And so the
picture down there actually has the same
212
00:19:01,620 --> 00:19:06,820
guy thrice. So this is also a bit of a pun
here intended. We did some fuzzing in
213
00:19:06,820 --> 00:19:12,030
there. We still need to evaluate some of
the results. So yeah, but I also feel like
214
00:19:12,030 --> 00:19:16,880
nobody's really using it and it's kind of
sad. I mean, there's some Linux users, I
215
00:19:16,880 --> 00:19:23,110
guess, but ... Yeah. Then there is the
weirdest stack I would say. So there's the
216
00:19:23,110 --> 00:19:27,852
Apple Bluetooth stack and this one is
actually three. So there is there is the
217
00:19:27,852 --> 00:19:31,380
MacOS Bluetooth stack. There's an iOS
Bluetooth stack. They are definitely
218
00:19:31,380 --> 00:19:34,750
different. And then there is a third
embedded one, for example, for the
219
00:19:34,750 --> 00:19:43,300
AirPods. They are all running different
different things. So, yeah, whatever. And
220
00:19:43,300 --> 00:19:47,850
then they also have tons of proprietary
protocols on top of their Bluetooth stuff
221
00:19:47,850 --> 00:19:52,170
that they're also very special. And I had
like at least two students, just one
222
00:19:52,170 --> 00:19:58,240
porting it to iOS one to MacOS. And then
we also have students working on the other
223
00:19:58,240 --> 00:20:02,711
protocols that are on top of Bluetooth.
And if you look into this, it's like, what
224
00:20:02,711 --> 00:20:06,560
the hell? It's really hard to reverse
engineer because you have like three
225
00:20:06,560 --> 00:20:10,651
different implementations and then
sometimes you're like: "yeah, okay. Maybe
226
00:20:10,651 --> 00:20:15,610
it's also just bad code." And in the end,
so from what I saw so far, I would say
227
00:20:15,610 --> 00:20:23,640
that it's kind of both. And then there is
the stack that I played also lots around
228
00:20:23,640 --> 00:20:29,060
with, which is the Android Bluetooth
Stack. And they did a lot for the security
229
00:20:29,060 --> 00:20:32,780
in the recent releases. And it annoys me
so much that when I want to get internal
230
00:20:32,780 --> 00:20:36,770
blue running on it, I just echo to the
serial port instead so I bypass everything
231
00:20:36,770 --> 00:20:42,860
that the operating system does. And so
something that Android cannot do, which
232
00:20:42,860 --> 00:20:46,030
Apple does, is so Apple has all the
proprietary protocols. And if something
233
00:20:46,030 --> 00:20:50,320
goes wrong, they immediately cut the
connection. But Android doesn't because of
234
00:20:50,320 --> 00:20:54,580
compatibility and stuff. So you could just
send garbage for like two minutes and try
235
00:20:54,580 --> 00:20:57,840
and see what happens. And it would
continue listening and asking and
236
00:20:57,840 --> 00:21:04,980
confirming. But that's kind of an
overall design issue, I think. And yet
237
00:21:04,980 --> 00:21:10,630
there's Windows and I couldn't find any
students to work on Windows. laughter
238
00:21:10,630 --> 00:21:19,140
If someone wants to do this, that would be
great. And so another stack that's kind of
239
00:21:19,140 --> 00:21:26,440
missing here is LTE. And I would call this
like the long term exploitation plan. So
240
00:21:26,440 --> 00:21:30,620
it's not. I think if the long term
evaluation, evolution, whatever, but
241
00:21:30,620 --> 00:21:37,320
exploitation I think is the best thing for
the E. So we have like tons of wireless
242
00:21:37,320 --> 00:21:42,540
stuff where we are working on and I mean
like even PowerPC. And then there is
243
00:21:42,540 --> 00:21:48,380
Qualcomm and they have they have this
Qualcomm hexagon DSP. I hate it so much.
244
00:21:48,380 --> 00:21:53,420
So there's even source code leaks for
their LTE stuff. But it's just such a pain
245
00:21:53,420 --> 00:21:58,900
to work on it. So you might have noticed
is that ARM has this LTE project with
246
00:21:58,900 --> 00:22:05,430
Qualcomm, but it's just not fun. But other
people were doing a lot in this area and
247
00:22:05,430 --> 00:22:12,410
they've already presented here today and
yesterday. So the first thing is the SIM
248
00:22:12,410 --> 00:22:18,310
card in the phone. So the SIM cards should
be a thing like. From from my perspective,
249
00:22:18,310 --> 00:22:23,540
that should be secure because it protects
your key material. And then it runs tons
250
00:22:23,540 --> 00:22:27,040
of applications. I don't know. And then
you can exploit them and get the victim's
251
00:22:27,040 --> 00:22:31,400
location, dial premium numbers and launch
a browser. And then I didn't really
252
00:22:31,400 --> 00:22:38,891
understand, like there's just this WIB set
browser whatever, and then there's launch
253
00:22:38,891 --> 00:22:42,150
browser what, is it? And I think it even
launches a browser then on the smartphone,
254
00:22:42,150 --> 00:22:48,340
whatever. It's just crazy. And then I was
trying to call Deutsche Telekom and I'm a
255
00:22:48,340 --> 00:22:52,120
business customer. So it's just like
three minutes for a call for me. So giving
256
00:22:52,120 --> 00:22:57,700
a call there is nice. And then the first thing
they told me is: "You are secure. We know
257
00:22:57,700 --> 00:23:03,080
you have three SIM cards and they are all
up to date." So I have to say, one of them
258
00:23:03,080 --> 00:23:08,130
is more than 10 years old, but maybe it's
up to date. And my answer is like, what
259
00:23:08,130 --> 00:23:12,520
exactly is running on my SIM card? They of
course not answered. So yeah, something is
260
00:23:12,520 --> 00:23:16,750
running there. If you want to know more
about SIM cards, there has been talk
261
00:23:16,750 --> 00:23:22,570
already yesterday evening and it's already
online. And then there's also a lot of
262
00:23:22,570 --> 00:23:27,170
people looking into LTE. And I think the
most popular one is to work by Yongdae
263
00:23:27,170 --> 00:23:33,220
Kim. He did even some LTE fuzzing
framework that he didn't release publicly
264
00:23:33,220 --> 00:23:39,220
so far, because of the findings. So it's
like, should you publish? Should you not
265
00:23:39,220 --> 00:23:44,200
publish? But so the findings are super
interesting. And he also has students here
266
00:23:44,200 --> 00:23:51,750
who just did a talk this morning.
Responsible disclosure. So that's the
267
00:23:51,750 --> 00:23:57,560
thing. When you find stuff you need to do
is responsible disclosure. And so I said
268
00:23:57,560 --> 00:24:02,360
Jan was writing a lot of e-mails. And one
of the first that he wrote was to ThreadX,
269
00:24:02,360 --> 00:24:08,060
because ThreadX is the operating system
that runs on the Broadcom Bluetooth
270
00:24:08,060 --> 00:24:14,470
chip. And so he said, like, your
heap is a bit broken and does not have any
271
00:24:14,470 --> 00:24:18,640
checks. You could implement the following
checks, which are pretty cheap and it
272
00:24:18,640 --> 00:24:22,440
should be cool. And then I could not
attack it anymore. And then ThreadX was
273
00:24:22,440 --> 00:24:28,040
answering, which was a bit unexpected,
that they already knew about this
274
00:24:28,040 --> 00:24:33,380
exploitation technique and that it is up
to the application to not be vulnerable to
275
00:24:33,380 --> 00:24:37,830
memory corruption, so not to cause any
memory corruption. So it's the programmers
276
00:24:37,830 --> 00:24:42,020
fault if they do something and it's not
the operating system that has to take care
277
00:24:42,020 --> 00:24:51,640
of the heap. Okay. Yeah. Next issue. So
the bin diffing and the testing if a
278
00:24:51,640 --> 00:24:56,590
vulnerability is still there. So you might
not always get feedback from all the
279
00:24:56,590 --> 00:25:01,460
vendors. If they fix it, they may just fix
it at a certain point in time and then you
280
00:25:01,460 --> 00:25:04,621
tell them all we tested the next release
and it's still vulnerable. And then they
281
00:25:04,621 --> 00:25:08,380
would say, like for Samsung said, Yeah, we
cannot send your patches in advance
282
00:25:08,380 --> 00:25:12,090
without an NDA because Broadcom, blah,
blah, blah and so on and so forth. And
283
00:25:12,090 --> 00:25:17,250
then Broadcom offered to send us patches
in advance. And I said, Yeah. Nice. And I
284
00:25:17,250 --> 00:25:21,500
also sent them a device list because they
already knew it from the previous process.
285
00:25:21,500 --> 00:25:24,610
So if you tell them the following 10
devices have an issue, then you would
286
00:25:24,610 --> 00:25:28,770
already know that we can test those
devices anyway. And so and after I sent
287
00:25:28,770 --> 00:25:33,520
them the list, they said: "Oh, wait, but
you need an NDA." So no, I mean, we are
288
00:25:33,520 --> 00:25:40,560
doing this for free anyway. And signing an
NDA, I wouldn't do that. Yeah. So overall,
289
00:25:40,560 --> 00:25:44,550
it also did Broadcom Product Security
Incident Response Team is a bit strange so
290
00:25:44,550 --> 00:25:49,550
they wouldn't hand out any CVEs. And what
I mean what I do is like I first get a CVE
291
00:25:49,550 --> 00:25:53,060
and then informed them and the other
customers because I also don't get any
292
00:25:53,060 --> 00:25:56,500
incident number or something. So if I
wouldn't do this, I wouldn't have any
293
00:25:56,500 --> 00:26:03,140
number to refer a vulnerability to. And
well, at least they are also not doing
294
00:26:03,140 --> 00:26:07,480
that much legal trouble. But it's just.
Yeah, not really something happening
295
00:26:07,480 --> 00:26:13,860
there. But some of their customers were
nice at least to my students, so they paid.
296
00:26:13,860 --> 00:26:17,950
So one customer, they don't want to be
named here, but they payed to fly to DefCon
297
00:26:17,950 --> 00:26:22,160
for one of my students and Samsung gave a
bounty off one thousand dollar. I mean,
298
00:26:22,160 --> 00:26:26,630
still I mean we are in the range of of
very very more expensive exploits if it
299
00:26:26,630 --> 00:26:31,040
would be on the black market, but for
students it's definitely nice. Yeah.
300
00:26:31,040 --> 00:26:34,860
Responsible disclosure timelines. So this
is something that I thought like maybe
301
00:26:34,860 --> 00:26:38,500
some of this responsible disclosure
timeline is just because of how I
302
00:26:38,500 --> 00:26:42,350
communicate with the vendor. And sometimes
I'm writing e-mails like a five year old
303
00:26:42,350 --> 00:26:49,400
or something - I don't know. But actually.
So this is a timeline of Quarkslab, who
304
00:26:49,400 --> 00:26:54,490
also found just this year vulnerabilities
in Broadcom Wi-Fi chips. And so they've
305
00:26:54,490 --> 00:27:00,840
also asked about NDA and then also their
exploit timeline. It's a bit fun because
306
00:27:00,840 --> 00:27:04,651
they had similar exploitation strategies
as the very first exploit that you could
307
00:27:04,651 --> 00:27:10,710
see by Google Project Zero and then, yeah,
more distorted timeline, whatever. And in
308
00:27:10,710 --> 00:27:18,810
the end, well. So it's just taking time.
And again, no CVE id issued and so on and
309
00:27:18,810 --> 00:27:26,110
so forth. So it's the very same stuff for
others, which is pretty sad. And then so
310
00:27:26,110 --> 00:27:31,230
for Cyprus, which is partially having
source code of Broadcom, it also
311
00:27:31,230 --> 00:27:36,590
manufactures chips. It's also very slow
for the response of disclosure. And then I
312
00:27:36,590 --> 00:27:40,220
got told by other people, like, yeah, if
you would disclose something to Qualcomm,
313
00:27:40,220 --> 00:27:46,101
it also takes very long. And luckily we
didn't find something in an Intel CPU. But
314
00:27:46,101 --> 00:27:49,710
yeah, there is ... so on the wireless
market, there still so many other vendors
315
00:27:49,710 --> 00:27:56,310
to become friends with. So yeah. So
practical solutions to end my talk. What
316
00:27:56,310 --> 00:28:01,110
could you do to defend yourself if you
don't have a tinfoil hat? Other things I
317
00:28:01,110 --> 00:28:06,620
can recommend is the secure Wi-Fi set up.
So don't use antennas, just use antenna
318
00:28:06,620 --> 00:28:13,390
cables. If you do that in our lap a lot.
So this is a setup by Felix. And so when I
319
00:28:13,390 --> 00:28:17,710
was sending my slides to Francesca in
advance she just said "cool, I have the
320
00:28:17,710 --> 00:28:23,960
same one right now at my desktop". So it's
a very common setup. Maybe not at your
321
00:28:23,960 --> 00:28:30,450
home, but for us it is. Or you'd just go
to the air-gapped device. So this is my
322
00:28:30,450 --> 00:28:37,400
Powerbook 170, that's a really great
device. Almost impossible to get it online
323
00:28:37,400 --> 00:28:45,310
and it has Word and Excel.
So ask all the questions.
324
00:28:45,310 --> 00:28:54,150
Applause
325
00:28:54,150 --> 00:28:58,250
Herald Angel: Thank you very much to
jiska. We still have several minutes left.
326
00:28:58,250 --> 00:29:03,000
You will find eight microphones in the
room. Please line up in behind the
327
00:29:03,000 --> 00:29:08,190
microphones to ask a question. And the
first question goes to the Internet.
328
00:29:08,190 --> 00:29:12,520
Signal Angel: So at jiska, the question
is, are the Bluetooth issues you are
329
00:29:12,520 --> 00:29:17,740
talking about also present in Bluetooth
Low Energy IoT devices.
330
00:29:17,740 --> 00:29:22,830
jiska: Yes. So, I mean, there is IoT
devices, I cannot tell the vendor, but
331
00:29:22,830 --> 00:29:28,620
there is also some popular devices that
have like Cypress Broadcom chips and then
332
00:29:28,620 --> 00:29:33,380
it's even worse because they don't have a
separate stack, but often they have an
333
00:29:33,380 --> 00:29:37,110
application running on the same arm core
and then you don't even need any
334
00:29:37,110 --> 00:29:40,070
escalation.
Herald: All right. We have another
335
00:29:40,070 --> 00:29:42,720
question at the microphone number one,
please.
336
00:29:42,720 --> 00:29:47,390
Microphone 1: Thank you for the talk. My
question is, could you like did you
337
00:29:47,390 --> 00:29:53,850
actually when you fuzzed the Bluetooth low
energy chip, did you when you managed to
338
00:29:53,850 --> 00:29:57,710
get code execution, did you actually
climb up the protocol?
339
00:29:57,710 --> 00:30:01,070
Did you like access Bluetooth profiles or
something like this?
340
00:30:01,070 --> 00:30:06,810
jiska: Ah, so we, for example with the
link key extraction, we were building some
341
00:30:06,810 --> 00:30:14,800
proof of concepts. But so it depends. We
don't currently have like a fully exploit
342
00:30:14,800 --> 00:30:18,930
chain in terms of first on the chip and
then on the host. We have something that
343
00:30:18,930 --> 00:30:25,970
goes directly on some hosts, but yeah,
there's tons of things there to do. Sorry?
344
00:30:25,970 --> 00:30:30,170
Microphone 1: Yeah. And when you fuzz the
... how did you actually fuzz the chip
345
00:30:30,170 --> 00:30:33,610
itself? How did you extract the
firmware from the chip?
346
00:30:33,610 --> 00:30:37,201
jiska: So there is ... so Broadcom and
Cyprus are very nice because they have a
347
00:30:37,201 --> 00:30:41,890
read RAM command so you don't need any
secure bypass or something. And for the
348
00:30:41,890 --> 00:30:50,710
evaluation kits, there is even symbols
that we found in it. So symbols only means
349
00:30:50,710 --> 00:30:55,700
like function names and global variable
names, that's it. But that's something to
350
00:30:55,700 --> 00:30:59,320
work with.
Herald: Thank you. Another question from
351
00:30:59,320 --> 00:31:02,740
the Internet, please.
Signal: Would you like the return of
352
00:31:02,740 --> 00:31:06,480
physical switches for
the network controller?
353
00:31:06,480 --> 00:31:11,380
jiska: Yeah, so that would be nice to like
physically switch it off. Actually, I
354
00:31:11,380 --> 00:31:14,730
don't know where Paul is, but he is
building ... There is Paul. He is building
355
00:31:14,730 --> 00:31:22,440
such a device. When is your talk at 10
o'clock. Paul is giving a talk about a
356
00:31:22,440 --> 00:31:25,880
device where you have a physical
controller to switch off your wireless
357
00:31:25,880 --> 00:31:28,890
stuff.
Herald: OK. The next question is again
358
00:31:28,890 --> 00:31:33,560
microphone number one, please.
Microphone 1: Yes. Thank you. We just
359
00:31:33,560 --> 00:31:38,980
bought a new car and by because
connecting the Bluetooth of my phone to
360
00:31:38,980 --> 00:31:45,620
the car's system was very, very hard. And
I had to reboot the radio several times.
361
00:31:45,620 --> 00:31:51,550
And then I found a message that the radio
must be directly connected to the CAN-bus
362
00:31:51,550 --> 00:31:57,090
of the car. So you have a blueooth stack
connected directly to a CAN-bus. It was a
363
00:31:57,090 --> 00:32:02,250
very cheap car. But if you
have an idea of what this means then...
364
00:32:02,250 --> 00:32:08,221
jiska: Can you can you borrow me that car?
Microphone 1: It's a Toyota Aygo, you can
365
00:32:08,221 --> 00:32:13,820
have it everywhere.
jiska: Well, that shouldn't be.
366
00:32:13,820 --> 00:32:17,240
Herald: Alright, do we have a question at
microphone number eight, please?
367
00:32:17,240 --> 00:32:21,590
Microphone 8: Hi and thank you for the
talk first of all. Uh well, if I
368
00:32:21,590 --> 00:32:26,730
understood correctly, you said that the
vendors didn't mention if they fixed it or
369
00:32:26,730 --> 00:32:32,090
not or they don't know if they fixed it.
jiska: Umm, yeah. So it depends. I know
370
00:32:32,090 --> 00:32:36,650
like so if you look into the Android
security updates, so for example, August 5
371
00:32:36,650 --> 00:32:40,920
has some Broadcom issue that was fixed and
I know which one that was and so on and so
372
00:32:40,920 --> 00:32:46,990
forth. But so then it also means I like to
get that one onto a Samsung device. I
373
00:32:46,990 --> 00:32:50,390
would need ... so they wouldn't build it
in the August update, but only in the
374
00:32:50,390 --> 00:32:55,160
September update and then release it to
Europe, which is like mid or end of
375
00:32:55,160 --> 00:32:59,370
September. And then I could download it to
my phone and test it over the air if it's
376
00:32:59,370 --> 00:33:06,630
like really fixed and so on and so forth.
So it's ... there is like the first thing
377
00:33:06,630 --> 00:33:10,190
is like that is listed publicly that it is
fixed. And then the next thing is that it
378
00:33:10,190 --> 00:33:14,890
is actually fixed and it's really hard.
And for the communication with Apple, I
379
00:33:14,890 --> 00:33:18,059
don't know. So sometimes they fix it
silently without mentioning us. And then
380
00:33:18,059 --> 00:33:24,440
there's this iOS 13 thing where they
mentioned us but didn't fix it. So, yeah.
381
00:33:24,440 --> 00:33:27,720
Microphone 8: Were there any issues like
that they found and they didn't know if
382
00:33:27,720 --> 00:33:31,200
they fixed it or not. And you did like
patch-diffing or something like that?
383
00:33:31,200 --> 00:33:35,059
jiska: Yeah, I did a lot of patch-diffing
and I currently have a student who is
384
00:33:35,059 --> 00:33:40,210
doing nothing else than developing diffing
tools for the particular issues that I
385
00:33:40,210 --> 00:33:42,830
have.
Microphone 8: And did you find that they
386
00:33:42,830 --> 00:33:45,929
fixed it or not?
jiska: So it's first of all ... so we are
387
00:33:45,929 --> 00:33:50,750
... so first of all, it's currently about
speed and stuff and I gave him some some
388
00:33:50,750 --> 00:33:54,330
iPhone stuff for the next task.
But yes, it's work in progress. So most of
389
00:33:54,330 --> 00:33:59,700
the other stuff I did by hand, so I also
have a good idea about like what a changed
390
00:33:59,700 --> 00:34:05,200
within each kind of chip generation.
Microphone 8: Okay. Thank you very much.
391
00:34:05,200 --> 00:34:09,180
Herald: All right. We had another question
from the Internet.
392
00:34:09,180 --> 00:34:13,820
Signal: Yes. So from Mastodon, how exactly
was the snapshot of the Samsung Bluetooth
393
00:34:13,820 --> 00:34:18,200
stack extracted for the fuzzing process?
jiska: The Samsung is ... so for Samsung
394
00:34:18,200 --> 00:34:24,150
we have snap shotting, but for Samsung, we
don't have the rest of Frankenstein. The
395
00:34:24,150 --> 00:34:30,860
other stuff is running on an evaluation
board. So the first part is mapping all
396
00:34:30,860 --> 00:34:34,680
the hardware registers. So this is the
first trip that runs and tries to find
397
00:34:34,680 --> 00:34:40,619
like all the memory regions. And once that
is done, there is a snapshotting hook that
398
00:34:40,619 --> 00:34:44,129
you set to the function. So let's say you
want to look into a device scanning so you
399
00:34:44,129 --> 00:34:48,760
would set the function into device
scanning. And once that it's called by the
400
00:34:48,760 --> 00:34:53,530
Linux stack, he would freeze the whole chip
and disable like other interrupt stuff,
401
00:34:53,530 --> 00:34:57,580
whatever that would kill it otherwise and
then copy everything that is in the
402
00:34:57,580 --> 00:35:02,680
registers ... so that is kind of the snap
shotting. And once you have a snapshot,
403
00:35:02,680 --> 00:35:08,119
then you can try to find everything that
kills your emulation like interrupts again
404
00:35:08,119 --> 00:35:12,740
and thread switches and so on.
Herald: All right. We have one more
405
00:35:12,740 --> 00:35:15,810
question from microphone, number one,
please.
406
00:35:15,810 --> 00:35:21,380
Microphone 1: OK. So do you think that
open source, the driver or that open
407
00:35:21,380 --> 00:35:25,500
hardware design will improve the
situation?
408
00:35:25,500 --> 00:35:30,950
jiska: So open source? I think it would
improve the situation. But also one thing.
409
00:35:30,950 --> 00:35:36,630
So I had to talk at mrmcd in September
this year. Another thing which is not
410
00:35:36,630 --> 00:35:41,190
about open source is that the patching
capabilities of the Broadcom bluetooth
411
00:35:41,190 --> 00:35:47,980
chips are very limited in terms of how
much can be fixed. So just open sourcing
412
00:35:47,980 --> 00:35:54,120
wouldn't help Broadcom, for example.
Microphone 1: Like you mean like the
413
00:35:54,120 --> 00:35:59,510
firmware is burnt into the chip and it's
limited to ...
414
00:35:59,510 --> 00:36:01,180
jiska: Yeah
Microphone 1: Yeah, it's limited, right?
415
00:36:01,180 --> 00:36:06,430
jiska: Yes, it's in the ROM. And then you
have patch ROM slots and you have like 128
416
00:36:06,430 --> 00:36:10,900
patch ROM slot and each patch ROM slot is
a four byte override breakpoint thingy
417
00:36:10,900 --> 00:36:14,880
that branches then somewhere else into
RAM. And then RAM is also limited.
418
00:36:14,880 --> 00:36:21,430
So you couldn't like branch into large
chunks of RAM all the time. Yeah.
419
00:36:21,430 --> 00:36:25,280
Microphone 1: Thank you.
Herald: All right. If there are not any
420
00:36:25,280 --> 00:36:28,190
more questions?
jiska: Internet!
421
00:36:28,190 --> 00:36:31,869
Herald: Internet? Oh, more Internet
questions. Then, please go ahead.
422
00:36:31,869 --> 00:36:36,240
Signal: Yes. So winfreak on Twitter asks
what stack was tested when mentioning
423
00:36:36,240 --> 00:36:40,700
Android? There are several and Google is
convinced rewriting it every year is a
424
00:36:40,700 --> 00:36:45,220
good idea.
jiska: Ah, yeah. So this stuff that's like
425
00:36:45,220 --> 00:36:51,140
the standard stack that runs on a Samsung
phone for example. So I think, like for
426
00:36:51,140 --> 00:36:55,000
the main entry, there's only one ... I
know that there's like legacy stacks, but
427
00:36:55,000 --> 00:37:02,340
they switch to only one.
Herald: So Signal Angel, do you have more
428
00:37:02,340 --> 00:37:10,200
for us?
Signal: Yes. What is your hat made of?
429
00:37:10,200 --> 00:37:18,440
jiska: My hat? So it's like aluminum foil.
And then there is the cyber cyber thingy.
430
00:37:18,440 --> 00:37:26,270
So that's also important. Yeah. So but as
I said, it doesn't really help. It would
431
00:37:26,270 --> 00:37:31,870
more help to put the smartphone in a
blender, for example.
432
00:37:31,870 --> 00:37:35,950
Herald: Alright. Thank you very much for
this awesome talk. Give her a huge round
433
00:37:35,950 --> 00:37:37,740
of applause to jiska.
434
00:37:37,740 --> 00:37:40,985
applause
435
00:37:40,985 --> 00:37:44,290
36c3 postroll
436
00:37:44,290 --> 00:38:08,000
subtitles created by c3subtitles.de
in the year 2020. Join, and help us!