1
00:00:00,000 --> 00:00:13,070
33c3 intro music
2
00:00:13,070 --> 00:00:15,790
Herald: Let me present Felix Domke with
3
00:00:15,790 --> 00:00:19,631
the Software Defined Emissions: A Hacker's
Review of Dieselgate.
4
00:00:19,631 --> 00:00:26,000
applause
5
00:00:26,000 --> 00:00:29,441
Felix Domke: Yeah, hey everyone. Thank you
6
00:00:29,441 --> 00:00:34,090
for coming here. I saw there are a lot of
interesting talks at the same time in the
7
00:00:34,090 --> 00:00:39,270
other rooms, so thank you for coming here
and listening to me about software-defined
8
00:00:39,270 --> 00:00:46,450
emissions. "A Hacker's Review of
Dieselgate" is the subtitle. I'm Felix
9
00:00:46,450 --> 00:00:52,620
Domke. I usually do embedded software,
mainly security. I'm definitely not on
10
00:00:52,620 --> 00:00:58,809
cars and definitely not on things that
have combustion thingies, so I only got
11
00:00:58,809 --> 00:01:03,969
dragged into car software last year when
my own Volkswagen car was accused of
12
00:01:03,969 --> 00:01:10,940
cheating and I wanted to know what exactly
was going on. I held the talk last year
13
00:01:10,940 --> 00:01:15,900
about some of the details of the
Volkswagen Sharan defeat device. For the
14
00:01:15,900 --> 00:01:20,750
details you can take a look at that talk.
This time I want to look more at the
15
00:01:20,750 --> 00:01:26,780
process of finding or analyzing car
software. I want to look at whether this
16
00:01:26,780 --> 00:01:35,850
process scales to more cars. The first
step when having a piece of software that
17
00:01:35,850 --> 00:01:40,659
does not always do what people think it
does is, well obtain a firmware image,
18
00:01:40,659 --> 00:01:46,200
obtain a binary image of the firmware, and
in the case of my car I knew it was a
19
00:01:46,200 --> 00:01:52,090
Bosch EDC17, which is a Bosch ECU that a
lot of cars use, including my Volkswagen
20
00:01:52,090 --> 00:01:57,610
car. So, I didn't know anything about
ECUs, dumping software and so on, so I
21
00:01:57,610 --> 00:02:02,189
asked Google "Hey, what do I need to do to
dump an EDC 17," and Google had a lot of
22
00:02:02,189 --> 00:02:07,520
answers for this, but usually those were
people that wanted to sell me some device.
23
00:02:07,520 --> 00:02:13,480
Those were chip tuners that built their
own devices where you can plug in the ECU
24
00:02:13,480 --> 00:02:18,767
and then it extracts the image, usually by
exploiting some bugs in the software. But
25
00:02:18,767 --> 00:02:22,669
I didn't really want to buy something and
it takes like a lot of time until I get in
26
00:02:22,669 --> 00:02:28,320
my hands. I wanted to start. So I was
looking to do this on my own. What these
27
00:02:28,320 --> 00:02:35,180
sites usually tell you without paying is
how you wire up your given ECU for their
28
00:02:35,180 --> 00:02:41,699
device, so they tell you where to connect
12 volt, where to connect the CAN bus,
29
00:02:41,699 --> 00:02:46,910
which is the serial communication bus that
the ECU uses to communicate with the rest
30
00:02:46,910 --> 00:02:53,690
of the car
devices. Usually it's pretty easy, so...
31
00:02:53,690 --> 00:02:57,810
when analyzing ECU it makes a lot of sense
to reproduce the scenario on your desk and
32
00:02:57,810 --> 00:03:04,700
not in your car, so in order to make an
ECU boot all you need is ground, 12 volts,
33
00:03:04,700 --> 00:03:09,630
there's usually an ignition pin that you
also have to supply 12 volt to it, and
34
00:03:09,630 --> 00:03:16,090
then it boots. So on my desk it looked
something like this, and then once we have
35
00:03:16,090 --> 00:03:23,190
the setup we can boot the ECU, we can use
Python to talk to the ECU, which is great,
36
00:03:23,190 --> 00:03:27,910
and then we can use socket CAN, which is
the Linux CAN support that's really great,
37
00:03:27,910 --> 00:03:32,450
and we can even use MicroPython if we
want to have a smaller device that we can
38
00:03:32,450 --> 00:03:38,540
put in a car. So we can talk with the ECU.
Talking with the ECU in modern cars,
39
00:03:38,540 --> 00:03:45,290
there's a protocol called UDS. Basically,
I simplified this slightly, you can ask
40
00:03:45,290 --> 00:03:50,400
the ECU "Hey, I want to read memory by
address," you give it an address and you
41
00:03:50,400 --> 00:03:54,629
ask it to read four bytes in this case of
that address, and then it returns it to
42
00:03:54,629 --> 00:04:00,540
you. So I thought "Hey, maybe I can use
this to dump the software." In my case the
43
00:04:00,540 --> 00:04:05,459
device responds with a Security Access
Denied, so I looked into what I need to
44
00:04:05,459 --> 00:04:10,310
do. You actually have to do a Security
Access command. You send a command that's
45
00:04:10,310 --> 00:04:15,840
called Request Seed. You get back
basically a 32 bit random number and then
46
00:04:15,840 --> 00:04:20,100
what you have to do is to, have to
process this seed through a super-secret
47
00:04:20,100 --> 00:04:25,450
function and then return it in a response
call. The question is how do we know this
48
00:04:25,450 --> 00:04:29,860
super secret function. There are multiple
methods. We can look at the ECU software
49
00:04:29,860 --> 00:04:34,090
itself, if the algorithms in there, to
verify it. We can reverse diagnostic
50
00:04:34,090 --> 00:04:39,570
software that uses this mechanism. So for
example the Volkswagen software they use
51
00:04:39,570 --> 00:04:43,230
for car shops, or maybe someone else
already reversed this and put it in their
52
00:04:43,230 --> 00:04:48,600
own tools which may be easier to get it
from in terms of third-party diagnostic
53
00:04:48,600 --> 00:04:55,210
software. And in the case of my Bosch ECU,
the super secret function was this. I
54
00:04:55,210 --> 00:04:59,530
basically had to add this number to it.
The mechanism is called pin code.
55
00:04:59,530 --> 00:05:06,280
It's in... I mean it's not super
secret. Anyway, once I know this...
56
00:05:06,280 --> 00:05:13,759
applause
Yeah, thank you. I mean, it... once you
57
00:05:13,759 --> 00:05:19,481
do this, you send back the result and hey,
then you can read... you can send the read
58
00:05:19,481 --> 00:05:24,280
command again and, hey you're getting back
data, so this is great, right? We can read
59
00:05:24,280 --> 00:05:29,250
memory at runtime of the ECU, and we can
even do this while the car is operating.
60
00:05:29,250 --> 00:05:34,140
However it turns out that for the Bosch
ECU you can only dump specific regions. You
61
00:05:34,140 --> 00:05:39,430
can dump most of memory, some memory areas
are excluded, but most of the interesting
62
00:05:39,430 --> 00:05:44,220
stuff you can read. But you can't read any
code. You cannot read anything in flash.
63
00:05:44,220 --> 00:05:50,200
But we are hackers, of course, so we find
a way. The CPU used in these ECUs is a
64
00:05:50,200 --> 00:05:57,440
Infineon TriCore CPU, and it's used, at
least this particular one, was used in the
65
00:05:57,440 --> 00:06:03,870
ECU I cared about, and the security model
for this chip is that you can always enter
66
00:06:03,870 --> 00:06:07,760
a specific bootloader mode and execute
your own code, so you can strap a few
67
00:06:07,760 --> 00:06:11,389
lines they... the chip tuners tell you
that, right, they tell you what's high and
68
00:06:11,389 --> 00:06:16,870
low, which pins you have to connect your
ground and 3.3 volt, and then it enters
69
00:06:16,870 --> 00:06:21,200
this bootloader mode, you can upload some
piece of code. However, you can't read the
70
00:06:21,200 --> 00:06:25,690
flash, because the flash is locked. When
you start in bootloader mode, the flash is
71
00:06:25,690 --> 00:06:32,610
not readable until you write a specific
password to a register. That was not so
72
00:06:32,610 --> 00:06:37,460
great, so I looked into what else I could
do. The datasheet is very specific on how
73
00:06:37,460 --> 00:06:43,210
to operate this chip. For example there's
this one flash supply pin. So, even though
74
00:06:43,210 --> 00:06:47,260
the flash is in the same package as the
rest of the CPU, it has a dedicated supply
75
00:06:47,260 --> 00:06:52,720
pin and it tells you which parameters not
to exceed to ensure correct operation, but
76
00:06:52,720 --> 00:06:57,250
I really don't want the correct operation
which is in this case preventing me from
77
00:06:57,250 --> 00:07:02,530
dumping the flash. So, what can we do? We
can violate the requirements. The
78
00:07:02,530 --> 00:07:06,430
requirements for 3.3 volts. Let's see what
happens outside of that range. And turns
79
00:07:06,430 --> 00:07:11,700
out, down to a certain voltage level,
roughly 1.6 volt, everything just works as
80
00:07:11,700 --> 00:07:17,510
normal. That doesn't help us.
And below that voltage the device hangs in
81
00:07:17,510 --> 00:07:21,400
the bootloader, so that doesn't help us
either. The interesting parts happened
82
00:07:21,400 --> 00:07:26,151
when you are at the very specific voltage
level, and this is a little bit
83
00:07:26,151 --> 00:07:31,480
unscientific, because it's really just the
voltage level I tried, and then most of
84
00:07:31,480 --> 00:07:35,340
the time the device comes up and flash is
protected, and then the remaining times
85
00:07:35,340 --> 00:07:39,430
the device comes up and hangs in the
bootloader. But one in 10 times something
86
00:07:39,430 --> 00:07:45,490
interesting happened. The device came up
and the flash was not protected, so I
87
00:07:45,490 --> 00:07:54,195
could dump it out.
applause
88
00:07:54,195 --> 00:07:58,310
So having the image now in my hands, I
could start in actually reversing the
89
00:07:58,310 --> 00:08:03,041
defeat device and what I found was - I
don't want to duplicate a lot of what I
90
00:08:03,041 --> 00:08:07,311
talked about in the last talk - I found a
function called "acoustic function," or
91
00:08:07,311 --> 00:08:12,520
"Akustikfunktion" in German. It's a
function that senses vehicle speed, the
92
00:08:12,520 --> 00:08:17,139
duration of the engine operation, and some
other things, and then controls emission
93
00:08:17,139 --> 00:08:21,060
related functionality, or in short you can
say that this is the test cycle detection
94
00:08:21,060 --> 00:08:26,310
that enables the defeat device. And I
verified it to exist on my Sharan device
95
00:08:26,310 --> 00:08:31,870
by driving through the test cycle and
logging data. And during the last year I
96
00:08:31,870 --> 00:08:36,890
verified that it's actually the same
defeat device, more or less, that exists
97
00:08:36,890 --> 00:08:41,139
on a lot of other Volkswagen cars. All
these Volkswagen defeat devices that we
98
00:08:41,139 --> 00:08:46,420
talked about for the Euro 5 cars, they use
more or less the same acoustic function.
99
00:08:46,420 --> 00:08:51,120
Basically, to remind you, there are a few
curves stored in the software that look
100
00:08:51,120 --> 00:08:59,760
like this. This is the NEDC. This is the
test cycle you have to drive a car
101
00:08:59,760 --> 00:09:05,360
through. They exactly define how fast you
have to drive for a given time in seconds,
102
00:09:05,360 --> 00:09:10,260
so it's speed over time. If we draw this
as distance over time it looks like this.
103
00:09:10,260 --> 00:09:16,560
So this is the distance you got. You're
not really moving the car, because you're
104
00:09:16,560 --> 00:09:22,490
doing this in a lab on a dynamometer, but
what the car thinks it has moved to, and
105
00:09:22,490 --> 00:09:25,740
if we overlay this with the curves we
found in a software there's a perfect
106
00:09:25,740 --> 00:09:31,290
match. So this is the way how they
describe the test cycle. So this was for
107
00:09:31,290 --> 00:09:36,790
my Sharan. So I looked into, what do the
other cars do, especially what do the
108
00:09:36,790 --> 00:09:41,940
cars in North America do, because they're
not using the NEDC. And I found something
109
00:09:41,940 --> 00:09:48,480
interesting, or some someone sent an
interesting document to me, that was this.
110
00:09:48,480 --> 00:09:54,190
It was an emission service action. It
basically describes how there was a recall
111
00:09:54,190 --> 00:09:57,750
for some vehicles, that required a
software update in the shop.
112
00:09:57,750 --> 00:10:03,040
So this is basically the document that
informs the car shop what they have to do,
113
00:10:03,040 --> 00:10:08,940
and it had something very interesting in
it. By the way, this was in December 2014,
114
00:10:08,940 --> 00:10:13,450
so this was way before the whole
Dieselgate was public, but this was
115
00:10:13,450 --> 00:10:17,380
already while the EPA was already talking
with Volkswagen, already demanding
116
00:10:17,380 --> 00:10:22,370
explanations. All that investigation was
already proceeding. Volkswagen knew about
117
00:10:22,370 --> 00:10:26,950
this, that people figured out about the
defeat device, and it had something very
118
00:10:26,950 --> 00:10:30,990
interesting in it, that said "in addition,
the vehicle's engine management software
119
00:10:30,990 --> 00:10:34,830
had been improved to ensure the vehicle's
tailpipe emissions are optimized and
120
00:10:34,830 --> 00:10:40,890
operating efficiently." That sounds really
fishy to me, so I was curious, what
121
00:10:40,890 --> 00:10:46,130
exactly did they change in the software
update? And luckily they tell you the old
122
00:10:46,130 --> 00:10:53,120
and the new software versions, and you can
then go and look them up on a firmware
123
00:10:53,120 --> 00:10:58,210
DVD, that you can download on the
Volkswagen website, and it turned out that
124
00:10:58,210 --> 00:11:03,210
it's an ECU software similar to the the
Bosch ECU software I looked at before. So
125
00:11:03,210 --> 00:11:07,620
there's an acoustic function again there,
and the curve stored there, they match the
126
00:11:07,620 --> 00:11:14,820
US test cycles. This is one of them. There
are many more test cycles in the US, so
127
00:11:14,820 --> 00:11:18,410
there's another curve that matches this,
and this is the curve stored in the
128
00:11:18,410 --> 00:11:27,260
software and this is the corresponding
test cycle. And there are a lot of them.
129
00:11:27,260 --> 00:11:32,820
But I noticed something really
interesting, and some of the curves...
130
00:11:32,820 --> 00:11:41,580
they were much wider open than the other
ones. So for example this one... there's
131
00:11:41,580 --> 00:11:46,240
really a nonzero probability that if you
just have your morning commute, through, I
132
00:11:46,240 --> 00:11:50,200
don't know, some streets or something,
that you accidentally match this driving
133
00:11:50,200 --> 00:11:55,490
cycle every time you start driving in the
morning, so the car would, every time you
134
00:11:55,490 --> 00:11:59,751
drive this, think it is in test cycle
mode, and would operate with the
135
00:11:59,751 --> 00:12:06,399
optimized... in the optimized emission
mode and apparently this caused problems
136
00:12:06,399 --> 00:12:12,480
and what I saw, what Volkswagen added in
the software that was part of this recall,
137
00:12:12,480 --> 00:12:16,313
was this function. So, this is from it
from a disassembly. In pseudocode, this is
138
00:12:16,313 --> 00:12:22,090
this. So they started looking at the
steering wheel angle, and if they figured
139
00:12:22,090 --> 00:12:28,040
out that you move the steering wheel
angle, then they ignored the curves, the
140
00:12:28,040 --> 00:12:31,900
more open curves. So the idea is "Yeah, if
you move the steering wheel, you're
141
00:12:31,900 --> 00:12:38,930
definitely not in a test cycle, so at that
point we do not try to operate in this
142
00:12:38,930 --> 00:12:43,450
emission optimized mode. And it's a
little bit of speculation, but it matches
143
00:12:43,450 --> 00:12:47,650
up pretty well with with all the facts
that I read, is that because those cars
144
00:12:47,650 --> 00:12:52,860
operated in the test cycle mode too often,
that eventually caused the the particulate
145
00:12:52,860 --> 00:12:59,500
filters to clog, and their solution for
Volkswagen, and again, this was while they
146
00:12:59,500 --> 00:13:04,713
were already investigated by the EPA, was
to add the steering wheel angle detection.
147
00:13:04,713 --> 00:13:08,529
For more details, I worked with this with
the NDR, and they produced a feature on
148
00:13:08,529 --> 00:13:13,750
that, so there are some more details.
So this is Volkswagen, but there are more
149
00:13:13,750 --> 00:13:20,779
cars, and if we look at this... this is a
meta-study based on something that the the
150
00:13:20,779 --> 00:13:25,810
Ministry of Transport... they tested a lot
of diesel cars and what they found was
151
00:13:25,810 --> 00:13:31,899
this. This is actually a representation by
the ICCT. So the orange line is the
152
00:13:31,899 --> 00:13:38,770
emission limit, and the bars have an upper
and lower end, and the lower end is how
153
00:13:38,770 --> 00:13:44,420
much emissions the cars have. This is just
for nitrogen oxides, for NOx emissions,
154
00:13:44,420 --> 00:13:48,640
what they had in the lab, when you're
driving the test cycle. So and you can see
155
00:13:48,640 --> 00:13:53,100
all of these cars managed to stay under
the orange line, so they get their
156
00:13:53,100 --> 00:13:57,580
certification, but when driving them on a
real street, they produce the emissions
157
00:13:57,580 --> 00:14:01,709
corresponding to the upper end of that
bar, which is for some cars significantly
158
00:14:01,709 --> 00:14:09,070
higher. It's off by a factor of 10 and
more. When you're driving the car on a
159
00:14:09,070 --> 00:14:13,470
street. And this is interesting, because
the cars, they can meet the emission
160
00:14:13,470 --> 00:14:16,580
goals. The question is why don't they
always meet the emission goals?
161
00:14:16,580 --> 00:14:23,360
Why do they operate so differently in the
test cycle than on the street? And I try
162
00:14:23,360 --> 00:14:29,300
to give you a partial answer. And let's
look at how a car can optimize emissions.
163
00:14:29,300 --> 00:14:33,250
The first thing they do... so this is a
very simplified diesel engine. So fresh
164
00:14:33,250 --> 00:14:38,850
air goes in, fuel goes out, and there's an
exhaust pipe, right? And a lot of nitrogen
165
00:14:38,850 --> 00:14:44,750
oxide, a lot of NOx, goes out as well, and
we don't want that. So we added an EGR
166
00:14:44,750 --> 00:14:49,440
valve, which is basically a valve that
causes a part... a fraction of the
167
00:14:49,440 --> 00:14:54,229
outgoing air to recirculate again through
the engine and burn again. And what this
168
00:14:54,229 --> 00:14:59,100
causes is that the flame temperature goes
down, and if we look at the relationship,
169
00:14:59,100 --> 00:15:04,720
it's very simplified here, but with a
lower flame temperature you get fewer NOx
170
00:15:04,720 --> 00:15:11,230
concentrations, so you improve emissions
by lowering the flame temperature, however
171
00:15:11,230 --> 00:15:18,010
at the same time you're increasing the
soot level, or the particulate matter, and
172
00:15:18,010 --> 00:15:22,840
there is this trade-off - if you do too
much of EGR, too much of the exhaust gas
173
00:15:22,840 --> 00:15:27,899
recirculation, you're getting too much
soot, and the other hand if you do it too
174
00:15:27,899 --> 00:15:32,399
few, you get too much NOx, so you can
argue that the green area isn't really
175
00:15:32,399 --> 00:15:37,330
great, because there's no point where both
of them are great. And here we see the
176
00:15:37,330 --> 00:15:45,930
result of a clogged EGR valve. If there's
too much soot it will clog. EGR, as the
177
00:15:45,930 --> 00:15:54,019
conclusion, is the least cost solution. It
doesn't really work at higher loads. It
178
00:15:54,019 --> 00:15:57,850
works at low loads, and it does not
require exhaust... high exhaust
179
00:15:57,850 --> 00:16:03,480
temperatures, which is great, but
excessive use of that clogs particulate
180
00:16:03,480 --> 00:16:07,220
filters, affects the combustion, the
drivability goes down, and there are
181
00:16:07,220 --> 00:16:11,329
trade-offs with this. It's also not very
useful for higher engine loads, for
182
00:16:11,329 --> 00:16:16,560
example when you're accelerating you have
to disable EGR at high speeds. So a better
183
00:16:16,560 --> 00:16:22,170
method, that was added on top of this, is
called "selective catalytic reduction". I
184
00:16:22,170 --> 00:16:27,450
am... so, basically the idea is you have
an SCR catalyst in your exhaust pipe...
185
00:16:27,450 --> 00:16:33,990
there are more catalysts there, but let's
talk about NOx, and in there, this
186
00:16:33,990 --> 00:16:39,640
happens. We can simplify this, somehow,
and say if you put ammonia into this
187
00:16:39,640 --> 00:16:43,759
catalyst, the NOx is converted to nitrogen
and water.
188
00:16:43,759 --> 00:16:48,480
And Nitrogen and water is great, it's
harmless. It's already part of the air.
189
00:16:48,480 --> 00:16:53,110
The only issue is ammonia is this and this
is not something you want the driver to
190
00:16:53,110 --> 00:16:58,550
refill in your car. So instead this
solution is we can create ammonia in the
191
00:16:58,550 --> 00:17:03,450
car from using from something that's less
dangerous, and we have the reaction there.
192
00:17:03,450 --> 00:17:09,500
We can simplify this again and say we take
urea - Harnstoff auf Deutsch - and heat,
193
00:17:09,500 --> 00:17:15,869
and we create ammonia. Urea or urea
solution is this. It's called AdBlue or
194
00:17:15,869 --> 00:17:21,209
DEF - diesel exhaust fluid - it's not
dangerous. You can buy it, you can
195
00:17:21,209 --> 00:17:26,439
transport it... it's relatively cheap. The
idea is, we have this reaction, it
196
00:17:26,439 --> 00:17:32,519
requires ammonia in the catalyst, and we
put AdBlue into it, or urea, and using the
197
00:17:32,519 --> 00:17:36,350
heat that we have from the exhaust pipe,
we create the ammonia that we need to
198
00:17:36,350 --> 00:17:44,419
reduce the nitrogen oxides back to
nitrogen and water. There's a great
199
00:17:44,419 --> 00:17:48,390
property of this, that some of the ammonia
that's produced in the catalyst stays
200
00:17:48,390 --> 00:17:53,469
there until it's used up, so
there's some storage there. So the the
201
00:17:53,469 --> 00:17:56,990
requirement for creating ammonia is heat,
and if you don't have heat, but for
202
00:17:56,990 --> 00:18:00,940
example because you just started up your
engine... if there is still ammonia from
203
00:18:00,940 --> 00:18:05,759
the last usage in your catalyst you can
still use that, and use that up, and by
204
00:18:05,759 --> 00:18:09,859
the time you have used it up, maybe the
heat is enough to supply more AdBlue and
205
00:18:09,859 --> 00:18:17,770
then fill up that storage. The downside is
you need a pump to dose the AdBlue, and
206
00:18:17,770 --> 00:18:21,600
you need lots of software to control this
process. And you need a heater because the
207
00:18:21,600 --> 00:18:26,020
AdBlue freezes at some point, and it's an
expensive solution, it adds roughly $500
208
00:18:26,020 --> 00:18:32,080
to a car, which can be significant amount
of money for a small car, and it requires
209
00:18:32,080 --> 00:18:37,700
a large AdBlue tank for long service
intervals, so you don't have to refill it
210
00:18:37,700 --> 00:18:42,410
every few thousand kilometers or
something. The great thing about SCR is
211
00:18:42,410 --> 00:18:48,000
that it's efficient at higher loads.
There's a third method called LNT, Lean
212
00:18:48,000 --> 00:18:52,130
NOx Trap, it's cheaper than SCR for
smaller engine, it doesn't require
213
00:18:52,130 --> 00:18:54,320
anything,
however the bad thing is it requires
214
00:18:54,320 --> 00:18:59,319
frequent re-generation, which decreases
fuel efficiency, so it's kind of a stop-
215
00:18:59,319 --> 00:19:03,299
gap solution. And it's not efficient for
continuous high engine load, for example
216
00:19:03,299 --> 00:19:06,949
if you're driving on the German Autobahn
at full speed, then LNT is not going to
217
00:19:06,949 --> 00:19:13,110
help you much. For the sake of this talk,
let's keep in mind that EGR is exhaust gas
218
00:19:13,110 --> 00:19:16,500
recirculation, that's the thing that
operates within the engine and then we
219
00:19:16,500 --> 00:19:20,970
have the SCR, the selective catalytic
reduction, that uses AdBlue and is after
220
00:19:20,970 --> 00:19:27,470
the engine. We also saw that all these
technologies have significant trade-offs
221
00:19:27,470 --> 00:19:32,169
for NOx compliance, so we can kind of see
the motivation for a defeat device here,
222
00:19:32,169 --> 00:19:36,559
because it would be the solution to all of
these trade-offs. You get no downsides
223
00:19:36,559 --> 00:19:42,700
during regular driving, because nobody can
measure your emissions and while
224
00:19:42,700 --> 00:19:46,960
maintaining conformance because during a
test cycle you have perfect emissions.
225
00:19:46,960 --> 00:19:53,880
That kind of explains why there are defeat
devices. OK, let's get back to the bigger
226
00:19:53,880 --> 00:20:01,919
picture and see what other cars do. So
this is an Opel car, it's a Zafira car,
227
00:20:01,919 --> 00:20:07,399
it's a Euro 6 car, it's a pretty modern
car, it has an SCR catalyst. In theory it
228
00:20:07,399 --> 00:20:13,769
should have really great, low emissions,
especially at higher speeds because that's
229
00:20:13,769 --> 00:20:18,450
where SCR is good at. But quite
surprisingly it doesn't. If we look again
230
00:20:18,450 --> 00:20:24,499
at this report, we can see that this
Zafira exceeds the limit by up to 12x
231
00:20:24,499 --> 00:20:28,860
compared to the Euro 6 limit. This is
especially interesting because there's
232
00:20:28,860 --> 00:20:34,619
this Opel advertisement where they
advertise their diesel technology applying
233
00:20:34,619 --> 00:20:39,710
to the Insignia and the Zafira and they
say a lot of diesel fun without regrets,
234
00:20:39,710 --> 00:20:44,899
the new diesel generation of Opel achieves
best emission values and gasoline levels.
235
00:20:44,899 --> 00:20:48,950
Yeah, after they got sued for this they
had to change it slightly and they had to
236
00:20:48,950 --> 00:20:53,502
add this to the sentence.
scattered laughter
237
00:20:55,680 --> 00:20:59,070
So, during this testing, they had
this 12x emission
238
00:20:59,070 --> 00:21:03,389
limits. For example, one particular test
was to drive the test cycle in the same
239
00:21:03,389 --> 00:21:05,869
way, but at a different temperature,
at 10°C,
240
00:21:05,869 --> 00:21:10,200
and the car exceeded the
values by a factor of 6 even though the
241
00:21:10,200 --> 00:21:15,330
car would be operated in the very same
way. It was just that the ambient air
242
00:21:15,330 --> 00:21:21,840
temperature was 10°C instead
of 25. So they asked Opel why this was the
243
00:21:21,840 --> 00:21:26,659
case during their investigation, and Opel
responded, saying that the EGR and the SCR
244
00:21:26,659 --> 00:21:32,653
injection, they work to the full extent in
the temperature range of 20 to 30°C.
245
00:21:32,653 --> 00:21:42,190
It's what they call "normal use".
So our question was, is it really just the
246
00:21:42,190 --> 00:21:49,549
temperature window? So we got a car and
investigated. The ECU in that car is a
247
00:21:49,549 --> 00:21:56,350
General Motors ECU. It's developed in-
house, Opel is a GM daughter. It uses an
248
00:21:56,350 --> 00:22:06,350
automotive PowerPC, yay PowerPC! It uses
somewhat obscure variable length
249
00:22:06,350 --> 00:22:12,629
instruction extension to PowerPC. So how
do we start? Again, we need a firmware
250
00:22:12,629 --> 00:22:19,109
image. So let's ask the Internet, "How do
I dump this ECU?" Luckily, someone in some
251
00:22:19,109 --> 00:22:23,710
chip tuning forum already uploaded their
stock ECU, which means they're what they
252
00:22:23,710 --> 00:22:30,019
dumped from their ECU using some chip
tuning tool. It's not the same ECU, but
253
00:22:30,019 --> 00:22:34,018
it's very similar and I hope they shared
some code so I can analyze the software,
254
00:22:34,018 --> 00:22:40,799
maybe find a way to dump it. So the dump
was made with this tool and the tool did
255
00:22:40,799 --> 00:22:45,789
not let me export the binary. And the tool
is free, however to use any of the
256
00:22:45,789 --> 00:22:50,720
features in that software, you have to buy
their expensive hardware, that then
257
00:22:50,720 --> 00:22:55,590
connects to the car, which I didn't want.
This is how their software looks like. I
258
00:22:55,590 --> 00:22:59,429
loaded the image I found on the internet,
and I couldn't save it or anything without
259
00:22:59,429 --> 00:23:05,419
having the device attached. However, I can
just use a memory debugger and just dump
260
00:23:05,419 --> 00:23:10,249
it from the address space, and that gave
me a first firmware image to start with. I
261
00:23:10,249 --> 00:23:14,220
threw it in a disassembler and I found the
UDS function and, yeah, it implements read
262
00:23:14,220 --> 00:23:19,370
memory by address, so that's good. Most of
the RAM was readable without a security
263
00:23:19,370 --> 00:23:24,570
challenge. That is good, so I didn't even
need a security challenge to read RAM.
264
00:23:24,570 --> 00:23:30,489
However, the flash, it is readable, but
only with the security challenge. So let's
265
00:23:30,489 --> 00:23:36,669
take a look at the security challenge.
Maybe it's as simple as the Bosch one. So
266
00:23:36,669 --> 00:23:40,880
their way of doing this is, they store
16-bit input and output value in the
267
00:23:40,880 --> 00:23:45,340
firmware, and it's different for every
device, and they don't store the algorithm
268
00:23:45,340 --> 00:23:50,619
to compute the output from the input,
instead they just store the pair, and,
269
00:23:50,619 --> 00:23:55,789
well, it's just 16 bit, right, so let's
brute force it. The issue is, you can only
270
00:23:55,789 --> 00:24:03,139
try every 15 seconds, so it's kind of
lame. The question is, how do the GM
271
00:24:03,139 --> 00:24:08,069
tools, the original factory tools, get
access to that? Luckily, chip tuners had
272
00:24:08,069 --> 00:24:12,851
reversed that and then obfuscated it into
their own tools. But that can be de-
273
00:24:12,851 --> 00:24:18,580
obfuscated and eventually it's just a
little bit amount of bit shifting and so
274
00:24:18,580 --> 00:24:26,339
that was easy to fix. And also the the GM
repair manuals tell you how to wire up the
275
00:24:26,339 --> 00:24:30,620
ECU. They tell you where to put 12 volts,
the CAN bus again, and ground, and the
276
00:24:30,620 --> 00:24:35,569
ignition pin, and with all of that in
place, I can do the security challenge. I
277
00:24:35,569 --> 00:24:41,869
can now read all of flash memory and read
the four megabytes of PowerPC code, which
278
00:24:41,869 --> 00:24:46,600
mostly consists of mathematical functions.
There are no strings or anything, it's
279
00:24:46,600 --> 00:24:52,039
really hard to find what a function does,
what... There are thousands of variables.
280
00:24:52,039 --> 00:24:55,740
It's really hard to find what they mean,
right, so I need to know some entry
281
00:24:55,740 --> 00:25:01,369
points, some known data values, and then I
could refer to... one thing I could find
282
00:25:01,369 --> 00:25:06,019
are real-world constants, for example
there's the density of diesel fuel stored,
283
00:25:06,019 --> 00:25:10,549
which allows me to understand that this is
something related to fuel, an amount of
284
00:25:10,549 --> 00:25:17,050
fuel. Or more useful are the OBD2 calls.
So there are some standardized things you
285
00:25:17,050 --> 00:25:22,420
can ask an ECU, that's engine rpm, vehicle
speed, and things like that, and I would
286
00:25:22,420 --> 00:25:27,959
find that table in the firmware and then I
had a first start of things like RPM,
287
00:25:27,959 --> 00:25:33,220
speed, and so on. That was a good start.
It's not much more than what you can see
288
00:25:33,220 --> 00:25:37,349
here. So there's a lot of stuff not
included in these.
289
00:25:37,349 --> 00:25:39,820
The next thing I did was, I drove the car
290
00:25:39,820 --> 00:25:44,139
for a few weeks and I let a device
attached that would constantly log all
291
00:25:44,139 --> 00:25:50,009
memory using the rebuy address thing, and
every few minutes I would get one memory
292
00:25:50,009 --> 00:25:55,179
dump, basically. It's a few hundred
kilobytes of RAM, and then I put this into
293
00:25:55,179 --> 00:26:01,389
my disassembly, and that allowed me to
understand more of what individual
294
00:26:01,389 --> 00:26:07,080
variables do there. And I found some
interesting things. So the first thing,
295
00:26:07,080 --> 00:26:10,799
one of the first things I found, was
basically something that resembles this.
296
00:26:10,799 --> 00:26:16,309
So there was something that look at the
ambient temperature, and this basically
297
00:26:16,309 --> 00:26:23,390
checks for range, right, and it did...
this was for controlling the SCR systems,
298
00:26:23,390 --> 00:26:27,588
and it's interesting to know that the NEDC
requires the temperature between 20
299
00:26:27,588 --> 00:26:31,799
and 30°C, and this is right
centered around this, when they check from
300
00:26:31,799 --> 00:26:38,940
17.5 to 33°C. But this was of
course nothing new. I found something
301
00:26:38,940 --> 00:26:44,039
similar, however, another temperature
check. And as you can see it's written in
302
00:26:44,039 --> 00:26:47,989
a different way. It effectively achieves
the same thing, but it's a separate piece
303
00:26:47,989 --> 00:26:52,559
of code, and this time it was for the EGR
system. So we have these two exhaust
304
00:26:52,559 --> 00:26:58,270
treatment or optimization mechanisms
there, EGR and SCR, and they don't share
305
00:26:58,270 --> 00:27:01,899
code. They have their own temperature
window. So we found the temperature
306
00:27:01,899 --> 00:27:06,219
window, which was known to exist. The
question was, is there more? And one thing
307
00:27:06,219 --> 00:27:11,529
we found was this: It's basically reading
the vehicle speed and comparing to a fixed
308
00:27:11,529 --> 00:27:17,649
number, and it turned out it's something
like this. So it would check the
309
00:27:17,649 --> 00:27:23,860
vehicle speed and if it's above 145 km/h
it would set a flag and then under 140 it
310
00:27:23,860 --> 00:27:27,009
would clear. Keep in mind that the NEDC
maximum speed
311
00:27:27,009 --> 00:27:34,179
120 km/h, so during a test cycle that
would never happen. So let's see if this,
312
00:27:34,179 --> 00:27:38,849
what we found in software, if this really
translate something the car does in the
313
00:27:38,849 --> 00:27:44,610
real world, and it's getting slightly
technical here, I apologize, but we need
314
00:27:44,610 --> 00:27:52,320
to log some variables and a useful value
to to know is, how much NOx is there after
315
00:27:52,320 --> 00:27:56,769
the engine, and after the SCR catalyst,
and luckily there are 2 NOx sensors in the
316
00:27:56,769 --> 00:28:00,879
car. One before and one after the
catalyst, and they give you basically the
317
00:28:00,879 --> 00:28:05,440
NOx concentration in ppm. So we log that,
and we also log the signal of how much
318
00:28:05,440 --> 00:28:09,790
AdBlue is dosed into the system, and we
log the catalyst temperature. And one
319
00:28:09,790 --> 00:28:14,120
thing to keep in mind is that there's also
this amount of ammonia that's stored in
320
00:28:14,120 --> 00:28:18,619
the catalyst. We don't have this as a
value, but just keep this in mind. And
321
00:28:18,619 --> 00:28:26,519
this is how we've driven the car. The
blue line is the vehicle speed. You can
322
00:28:26,519 --> 00:28:33,739
see that it goes from 0 to 150 km/h, and
the critical point here is the 145 km/h
323
00:28:33,739 --> 00:28:39,380
that we found in the firmware. The green
one is the catalyst temperature, which we
324
00:28:39,380 --> 00:28:45,519
see between ambient level, and then up to
380 degrees. The critical point here is
325
00:28:45,519 --> 00:28:53,400
200 degrees Celsius, where this urea to
ammonia process starts to work. We logged
326
00:28:53,400 --> 00:28:58,399
something that is the SCR strategy. So it
turns out there are multiple ways how the
327
00:28:58,399 --> 00:29:04,961
ECU computes how much AdBlue to dose and I
call them strategy. So 0 means off, no
328
00:29:04,961 --> 00:29:09,300
AdBlue is dosed. 1 means the regular way
that keeps into account the storage
329
00:29:09,300 --> 00:29:15,369
mechanism, and then 2 is a special reduced
way. And then also we log the actual
330
00:29:15,369 --> 00:29:24,200
dosing value. And then we also had the
sensor data from between the engine and
331
00:29:24,200 --> 00:29:32,159
the catalyst, and between the catalyst
and the exhaust. The first thing that
332
00:29:32,159 --> 00:29:37,950
happens... or, actually nothing happens
until the point where we reach 200°C
333
00:29:37,950 --> 00:29:46,779
at the catalyst. You can...
until that point, as I said the required
334
00:29:46,779 --> 00:29:51,429
temperature is not... does not allow
AdBlue dosing, and then it starts dosing
335
00:29:51,429 --> 00:29:59,229
quite a few amount of AdBlue. But then,
when we cross the 145 km/h, the SCR
336
00:29:59,229 --> 00:30:07,049
strategy changes, and no further AdBlue is
dosed until basically this point, and this
337
00:30:07,049 --> 00:30:15,440
point is exactly 120 seconds after we go
lower than 140 km/h. So this matches what
338
00:30:15,440 --> 00:30:17,019
we found
in the software, right, this was what we
339
00:30:17,019 --> 00:30:19,549
found in the
software. So we can see that this was
340
00:30:19,549 --> 00:30:27,749
actually true. This is real behavior of
the ECU. And to look at the effect of
341
00:30:27,749 --> 00:30:35,109
this, you have to check the difference
between the blue and the green line in the
342
00:30:35,109 --> 00:30:40,930
lower diagram, between basically the
amount of NOx that is removed by the SCR
343
00:30:40,930 --> 00:30:46,090
catalyst. And you can see during the
regular operation quite a lot of NOx is
344
00:30:46,090 --> 00:30:49,759
removed. The blue line goes up because
we're driving faster and faster, and the
345
00:30:49,759 --> 00:30:56,609
green line goes down almost to zero, and
this this works for quite a while, and it
346
00:30:56,609 --> 00:31:03,309
even keeps working a while until the
catalyst runs out of ammonia, and then it
347
00:31:03,309 --> 00:31:07,429
would need more AdBlue to operate, but
because we're in the reduced mode it does
348
00:31:07,429 --> 00:31:13,559
not put any more AdBlue into the system.
So the SCR basically stops working, and
349
00:31:13,559 --> 00:31:19,279
the emission levels reach the engine
emissions, so no further... The SCR system
350
00:31:19,279 --> 00:31:24,789
does not work in this red area. And here
we see this again, so here are the
351
00:31:24,789 --> 00:31:28,099
sequences of active dosing. Here we see
where the catalyst temperature is too low
352
00:31:28,099 --> 00:31:33,639
for dosing. We see the regular operation.
We see where it's still working,
353
00:31:33,639 --> 00:31:38,049
because there's still ammonia stored and
then until we run out of ammonia and no
354
00:31:38,049 --> 00:31:45,940
refill happens, until exactly 120 seconds
after going below 140 kilometers an hour.
355
00:31:45,940 --> 00:31:52,669
So our conclusion after this is that the
SCR is programmed to stop working at 145
356
00:31:52,669 --> 00:31:58,089
km/h. The efficiency goes to 0. Opel
offered a hand-waving explanation, one of
357
00:31:58,089 --> 00:32:01,999
the press releases why this is necessary.
They argued with some physical details,
358
00:32:01,999 --> 00:32:07,549
and we presented these physical details to
some experts, professors that work with
359
00:32:07,549 --> 00:32:14,219
combustion engines for a long time. Most
of them disagree with these explanations,
360
00:32:14,219 --> 00:32:20,250
I mean, yeah. But more importantly, other
cars including my Volkswagen Sharan
361
00:32:20,250 --> 00:32:26,759
device, which is Euro 5 car, so one
generation older, and it's known to have a
362
00:32:26,759 --> 00:32:34,320
defeat device, and it performs
significantly better than this car. OK, we
363
00:32:34,320 --> 00:32:39,579
continue to look. We found something here
that is a... that looks like this. It
364
00:32:39,579 --> 00:32:44,749
takes a... there's a
barometric pressure sensor that sends us
365
00:32:44,749 --> 00:32:49,239
the pressure of the air and compares it
with a value and if we look at how
366
00:32:49,239 --> 00:32:55,470
pressure is related to height we see that
what they check with is 91.5 kilopascal
367
00:32:55,470 --> 00:33:00,080
and which corresponds to 850 meter. And
apparently Europe's highest test center's
368
00:33:00,080 --> 00:33:05,599
at 800 meter, which may be a coincidence
or not. But above that point they reduce
369
00:33:05,599 --> 00:33:10,989
their CR system as well. Now, the
interesting thing is... yeah barometric
370
00:33:10,989 --> 00:33:14,690
pressure is something very important to
know for an ECU. There's a good reason to
371
00:33:14,690 --> 00:33:18,659
have the sensor for all of the combustion
process. You need to know how much air
372
00:33:18,659 --> 00:33:24,440
there is. So for EGR it makes a lot of
sense to have this, but for SCR, which is
373
00:33:24,440 --> 00:33:29,809
the system after the engine, no combustion
is happening. We are not aware of any
374
00:33:29,809 --> 00:33:35,889
effect that the outside air pressure has
on the SCR system, and also other SC
375
00:33:35,889 --> 00:33:41,210
equipped cars don't have this mechanism,
so... for us it does not make
376
00:33:41,210 --> 00:33:47,004
physically... it doesn't seem to be
physically required. So far we looked at
377
00:33:47,004 --> 00:33:54,130
SCR. Let's look at EGR. What we saw was,
when we drove the car during the test
378
00:33:54,130 --> 00:33:56,358
cycle, so we put it in the lab and drive
the
379
00:33:56,358 --> 00:34:01,789
test cycle, we consistently saw much
higher EGR values, much higher than
380
00:34:01,789 --> 00:34:05,350
compared to driving on the street,
compared to all kind of scenarios that we
381
00:34:05,350 --> 00:34:09,399
drove on the street. So higher EGR value
here means, that the EGR valve, that I
382
00:34:09,399 --> 00:34:13,919
showed you earlier, is more open, more
exhaust gas recirculates to the engine. It
383
00:34:13,919 --> 00:34:20,730
causes lower NOx emissions before the SCR
catalyst, and we really... we're curious
384
00:34:20,730 --> 00:34:26,260
why did the car behave so differently when
running on a street than running in a test
385
00:34:26,260 --> 00:34:29,070
cycle. And we already took into account
temperature, so the temperature was not
386
00:34:29,070 --> 00:34:34,739
the issue anymore. And thankfully the car,
when it computes the reason for reducing
387
00:34:34,739 --> 00:34:41,870
EGR, it stores a reason in some variable
that we can log, and it looks like this.
388
00:34:41,870 --> 00:34:46,739
There is a number of things that can
happen, that causes the ECU to switch to
389
00:34:46,739 --> 00:34:51,909
some low EGR mode, and a few of them
make sense, for example if something is
390
00:34:51,909 --> 00:34:56,230
broken, fault flags are set, or if the, I
don't know, the coolant temperature is out
391
00:34:56,230 --> 00:35:02,930
of range, it makes sense to just keep the
device running at all cost. But when none
392
00:35:02,930 --> 00:35:08,430
of these reasons apply, the value stored
is 2, and 2 basically means, that the full
393
00:35:08,430 --> 00:35:16,840
EGR operation is used, so it's basically
the NOx-optimized mode with the fewest
394
00:35:16,840 --> 00:35:21,730
emissions. And then we looked at some
real-world driving, you can see this in
395
00:35:21,730 --> 00:35:26,010
the background - the vehicle speed is in
the background - and we saw that... the
396
00:35:26,010 --> 00:35:31,070
red graph shows you the reason to go to
this limited EGR mode and what we saw is
397
00:35:31,070 --> 00:35:37,710
that most of the time the reason is 13 and
only a few times it's 2, which means that
398
00:35:37,710 --> 00:35:43,080
it's not limited. And looking into this
more details, we see it sometimes drops
399
00:35:43,080 --> 00:35:49,090
back to 2, to the unlimited mode, to the
optimized... emission optimized mode, but
400
00:35:49,090 --> 00:35:53,510
any acceleration, or almost any
acceleration switches it back to 13, and
401
00:35:53,510 --> 00:35:59,650
then it stays there for a long time. And
13, if we look it up, is what I call load
402
00:35:59,650 --> 00:36:06,840
limit. And then, interestingly, if we run
it through the NEDC, we never saw a 13. So
403
00:36:06,840 --> 00:36:11,470
the engine stays in mode 2 all the time,
and 16 just means that the engine is off.
404
00:36:11,470 --> 00:36:16,950
But we never see 13. So this explains why
the EGR values were so different in a test
405
00:36:16,950 --> 00:36:23,330
cycle. So, let's look into this load limit
function that we found. It's basically
406
00:36:23,330 --> 00:36:30,160
defined by curves, by five curves. For
every gear there's a curve, or for a
407
00:36:30,160 --> 00:36:36,230
bucket of gears. It's basically that they
look up RPM, they get a value for that
408
00:36:36,230 --> 00:36:45,380
curve, and if you exceed that value, they
switch to the reduced EGR mode. What they
409
00:36:45,380 --> 00:36:49,780
compare this threshold with is the amount
of fuel injected per cylinder per
410
00:36:49,780 --> 00:36:53,300
revolution, but you can also say this is
torque, just with a
411
00:36:53,300 --> 00:36:59,030
constant factor. And then once you are
outside of one of these curves, it
412
00:36:59,030 --> 00:37:05,090
switches to the non-optimized mode where
it emits a lot more emissions, and then
413
00:37:05,090 --> 00:37:10,350
you have to go back into the green area to
switch back to the optimized mode. So
414
00:37:10,350 --> 00:37:12,221
let's see what this means in practice. So
here we
415
00:37:12,221 --> 00:37:18,910
have a car, and the traffic light is red,
so the car stops, and then the traffic
416
00:37:18,910 --> 00:37:26,350
light goes green and the car accelerates,
and accelerates, and accelerates, gets
417
00:37:26,350 --> 00:37:33,870
faster and faster, and then it's at the
highest speed here, and drives for a
418
00:37:33,870 --> 00:37:39,140
while. And this is a typical city cycle,
this is there to... how you drive in a
419
00:37:39,140 --> 00:37:45,220
city, and then the next traffic light
turns red and the car brakes and stops in
420
00:37:45,220 --> 00:37:52,900
front of the traffic light. Let's take a
look at this again with one more variable,
421
00:37:52,900 --> 00:38:01,925
the RPM. We can see that when the car
starts moving, the RPM goes up. And then
422
00:38:01,925 --> 00:38:05,500
at some point there is a drop in RPM, and
this is because it's a manual transmission
423
00:38:05,500 --> 00:38:10,280
and the driver switched to the next gear.
Now it's switched to again the next gear,
424
00:38:10,280 --> 00:38:15,500
and this causes the RPM to drop, but the
speed to remain almost
425
00:38:15,500 --> 00:38:19,920
constant, and it drives for a long time in
the same gear, and then the traffic light
426
00:38:19,920 --> 00:38:24,530
goes red, the driver presses the clutch,
the engine goes back to idle state, there
427
00:38:24,530 --> 00:38:28,600
is no connection anymore to the wheels,
between the engines and the wheels, and
428
00:38:28,600 --> 00:38:37,120
the car gets slower. OK, one more
variable. It's the last one, I promise. It
429
00:38:37,120 --> 00:38:44,290
is torque. The engine power in kilowatt or
something is not just a function of RPM,
430
00:38:44,290 --> 00:38:48,460
it's a function of RPM and torque. so RPM
and torque together are very useful to
431
00:38:48,460 --> 00:38:53,830
characterize engine behavior. And a very
good way to do this is to have a graph
432
00:38:53,830 --> 00:39:00,230
where we put RPM on the one axis and
torque we put on the other axis, and then
433
00:39:00,230 --> 00:39:05,890
we draw this in two dimensions, and so we
get this, basically. This is the operating
434
00:39:05,890 --> 00:39:14,050
points we go through when driving the
cycle we saw. So the green dot here
435
00:39:14,050 --> 00:39:21,660
indicates where we are. And so we restart
the car, the car accelerates, sorry, the
436
00:39:21,660 --> 00:39:27,590
car idles for a while, so the green dot
stays there. It idles at around 800 RPM,
437
00:39:27,590 --> 00:39:32,570
almost no torque, because there's nothing
to move, and then the driver accelerates
438
00:39:32,570 --> 00:39:38,850
and the torque goes up, the RPM goes up
more slowly, and then at some point, the
439
00:39:38,850 --> 00:39:43,760
driver presses the clutch, which
disconnects the engine, the
440
00:39:43,760 --> 00:39:48,960
torque goes down, the RPM adjusts to the
speed of the next gear, and then the
441
00:39:48,960 --> 00:39:53,940
driver releases the clutch and now the
engine again has to move the car, so the
442
00:39:53,940 --> 00:39:59,330
torque goes up until reaching the the
highest RPM value and then that the driver
443
00:39:59,330 --> 00:40:03,760
again switches to the next gear, so the
whole thing repeats, and then while the
444
00:40:03,760 --> 00:40:09,110
car is driving, the majority of this the
cycle, the engine spends in this one
445
00:40:09,110 --> 00:40:14,950
operating point. We're currently at 1800
RPM or something, and 80 Newton meter or
446
00:40:14,950 --> 00:40:19,970
so torque. And then at some point the
driver presses the clutch, the engine goes
447
00:40:19,970 --> 00:40:26,779
back to idle and stays there, basically.
So this is how you read this diagram. And
448
00:40:26,779 --> 00:40:34,480
now what we found in the firmware was that
overlaid basically on this representation
449
00:40:34,480 --> 00:40:42,490
we see a mask, or a limit. If we go over
this curve, those are the same curves that
450
00:40:42,490 --> 00:40:50,500
I showed you earlier, just laid on top of
this. If we go over this curve,
451
00:40:50,500 --> 00:40:59,230
then we switch to the worse emission mode,
we switch to the mode where the EGR value
452
00:40:59,230 --> 00:41:06,600
is limited. So we can see in our driving
that this happens basically at this point,
453
00:41:06,600 --> 00:41:10,240
the point where the driver
accelerates above a certain point, that
454
00:41:10,240 --> 00:41:18,270
causes it to go over the load limit and
the engine basically switches or
455
00:41:18,270 --> 00:41:22,510
significantly reduces EGR. And that's fine
because EGR doesn't work when you need a
456
00:41:22,510 --> 00:41:27,590
lot of engine power, so it make sense that
that's at that point, and what we would
457
00:41:27,590 --> 00:41:32,740
think is that it switches back once we
leave this load envelope, once we go below
458
00:41:32,740 --> 00:41:36,840
the limit again, once we are inside the
limit, we would expect the ECU to switch
459
00:41:36,840 --> 00:41:42,670
back to the full EGR operation. But what
we see instead is that this does not
460
00:41:42,670 --> 00:41:50,120
happen, and the reason is that you don't
have to go under the maximum, the load
461
00:41:50,120 --> 00:41:54,770
limit, you have to go into this green
area. You have to go back to idling at a
462
00:41:54,770 --> 00:42:00,450
very low RPM to switch back to the full
EGR mode and this only happens at the very
463
00:42:00,450 --> 00:42:04,810
end. When the driving cycle is almost
done, when the driver presses the clutch
464
00:42:04,810 --> 00:42:12,080
and lets the engine idle. So especially
this long sequence where the driver... the
465
00:42:12,080 --> 00:42:18,150
car was driving at the same speed, we were
technically in ... within the load limit,
466
00:42:18,150 --> 00:42:21,970
where we're not exceeding the load limit,
but because we previously exceeded the
467
00:42:21,970 --> 00:42:26,960
load limit and it doesn't matter for how
long you exceeded it, and we did not go to
468
00:42:26,960 --> 00:42:33,300
the green area before, we were still in
this low EGR, high emission mode, even
469
00:42:33,300 --> 00:42:41,010
though we're still within the load limit
imposed by the software. So let's take a
470
00:42:41,010 --> 00:42:47,800
look at how often this actually happens in
real-world data. So here's us driving
471
00:42:47,800 --> 00:42:53,320
through a city, and we can see we
constantly exceed these load limits. And
472
00:42:53,320 --> 00:43:00,840
this is driving on the Autobahn, and yeah
we constantly exceed those. But they look
473
00:43:00,840 --> 00:43:05,940
interesting. They look as if they had been
designed according to something, right,
474
00:43:05,940 --> 00:43:12,670
they have the specific form and it's not
just... yeah, I... I don't know... and it
475
00:43:12,670 --> 00:43:16,480
turns out if you do something really
strange, you can stay within these limits,
476
00:43:16,480 --> 00:43:20,760
so we tried that and we managed to stay
within the limit by doing something, and
477
00:43:20,760 --> 00:43:24,900
we... it was reproducible, we could do
this a lot of time it would always stay in
478
00:43:24,900 --> 00:43:31,690
this limit and the answer is: If you drive
the test cycle you're staying in this
479
00:43:31,690 --> 00:43:41,430
limit.
applause
480
00:43:41,430 --> 00:43:48,260
So yeah, these curves basically defined...
they closely correlate to the limits that
481
00:43:48,260 --> 00:43:55,650
you need to pass the NEDC. Okay, to be
clear it is fully acceptable that the EGR
482
00:43:55,650 --> 00:44:00,520
rate is reduced when... for higher engine
loads. It's natural, you have to do this.
483
00:44:00,520 --> 00:44:06,930
For example, when you accelerate the EGR
rate will decrease up to zero probably,
484
00:44:06,930 --> 00:44:10,770
when you do it ... when you're running at
high speeds, all of that is great. So this
485
00:44:10,770 --> 00:44:14,570
method of having a load limit ... well,
you can argue if really having the load
486
00:44:14,570 --> 00:44:18,830
limit exactly where the NEDC is makes
sense, but having a load limit is okay,
487
00:44:18,830 --> 00:44:22,610
right? However, what we think is not okay
is that, if you only exceeded the limit
488
00:44:22,610 --> 00:44:28,930
once ... um... you would stay in this high
emissions mode for potentially a long time
489
00:44:28,930 --> 00:44:35,270
until you get back to low speed idle the
next time. And we think that is the
490
00:44:35,270 --> 00:44:40,170
problem. We ... so far this was all based
on what we saw in the software, so let's
491
00:44:40,170 --> 00:44:46,610
see if this translates to something that
happens in reality. So to repro this we...
492
00:44:46,610 --> 00:44:51,690
the car... drive at constantly... or we
let it idle, then we accelerate it to
493
00:44:51,690 --> 00:44:56,260
2,000 RPM, we let it drive there for a
while and then we quickly exceeded the
494
00:44:56,260 --> 00:45:02,710
load limit by going to 3,000 and then
going back and then after doing that we
495
00:45:02,710 --> 00:45:07,490
would again stay at 2,000 RPM. So it looks
like this and we would naturally expect
496
00:45:07,490 --> 00:45:12,750
the engine to operate in the same way on
the left and on the right side because the
497
00:45:12,750 --> 00:45:16,920
engine is doing the same thing there, it's
the same torque level, it's the same RPM,
498
00:45:16,920 --> 00:45:22,050
everything is the same. So we would expect
the same emissions, right, um ... and it
499
00:45:22,050 --> 00:45:26,250
turns out it isn't. And ... this is a
slightly convoluted diagram. So if you
500
00:45:26,250 --> 00:45:32,110
look at the green and red bars in the
middle you can see what happens before and
501
00:45:32,110 --> 00:45:36,090
after exceeding the limit for just once.
And in the middle you can see the EGR
502
00:45:36,090 --> 00:45:40,790
position, the EGR valve position, and you
can see that we get pretty high values
503
00:45:40,790 --> 00:45:50,390
between... 6... maybe 65 percent or
something before exceeding the load limit
504
00:45:50,390 --> 00:45:52,920
once.
And after we exceeded it once even though
505
00:45:52,920 --> 00:45:57,970
the engine again is operating in the same
exact operating point, we see much lower
506
00:45:57,970 --> 00:46:03,570
EGR valve positions, around 50% or
something. And if we look at the bottom we
507
00:46:03,570 --> 00:46:08,760
see what the engine NOx emissions and we
see that they are significantly higher on
508
00:46:08,760 --> 00:46:13,320
the right side than they are on the left
side. So this... for me, this does not
509
00:46:13,320 --> 00:46:17,890
sound like this is truly optimized for
emissions because the engine is doing the
510
00:46:17,890 --> 00:46:25,550
same thing, in both cases the emissions
should be low. So going back to this quote
511
00:46:25,550 --> 00:46:31,280
that it works, the EGR and SCR injection
work to the full extent in a temperature
512
00:46:31,280 --> 00:46:36,980
range of 20 to 30°C. Okay,
but what about the EGR load limit and what
513
00:46:36,980 --> 00:46:41,180
about the the barometric pressure limit
for SCR and what about the SCR speed
514
00:46:41,180 --> 00:46:46,290
limit? That would not be "to the full
extent", right? And the Opel answer is
515
00:46:46,290 --> 00:46:50,050
really interesting. Of course, they denied
doing a test cycle detection, they say
516
00:46:50,050 --> 00:46:57,450
they don't do that. And what they said is,
when asked whether they lied to the KBA
517
00:46:57,450 --> 00:47:00,830
when saying that it works to the full
extent they said "The statement 'fully'
518
00:47:00,830 --> 00:47:11,570
was really related to the NEDC test
schedule, right, which... it went on and
519
00:47:11,570 --> 00:47:17,190
further... the Opel CEO had to say this.
He said: "The recent
520
00:47:17,190 --> 00:47:21,460
accusations based on the findings of
hacker Mr. Felix Domke" - hey, that's me -
521
00:47:21,460 --> 00:47:26,030
"are misleading oversimplifications and
misinterpretations of the complicated
522
00:47:26,030 --> 00:47:30,580
interrelationships of a modern emission
control system of a diesel engine.
523
00:47:30,580 --> 00:47:34,880
Emission control devices are highly
sophisticated integrated systems which
524
00:47:34,880 --> 00:47:39,400
cannot be broken down into isolated
parameters." Especially not by a hacker,
525
00:47:39,400 --> 00:47:50,460
right?
applause
526
00:47:50,460 --> 00:47:54,630
That was kind of funny. There was another
funny thing. Sorry, I only have a German
527
00:47:54,630 --> 00:47:57,990
quote and I didn't want to translate it,
but when Opel basically ... they
528
00:47:57,990 --> 00:48:02,270
repeatedly say they don't have a cycle
detection, right, and they say it's not a
529
00:48:02,270 --> 00:48:08,660
cycle detection because, if you use the
car on the street in the same way as you
530
00:48:08,660 --> 00:48:12,870
would do them during the test cycle, the
car would behave in the same way, so it's
531
00:48:12,870 --> 00:48:23,930
not...,right? applause ... and ... okay.
But what is with Volkswagen, right, they
532
00:48:23,930 --> 00:48:28,330
have the same thing, if you drive the NEDC
on the street the car will go to test mode
533
00:48:28,330 --> 00:48:36,410
they have the same thing. I don't see how
this does not represent a cycle detection.
534
00:48:36,410 --> 00:48:42,370
That was a lot of things to say about
Opel, but on the bright side, they also
535
00:48:42,370 --> 00:48:46,770
said that they will - even though all that
was incorrect, what we found - they said
536
00:48:46,770 --> 00:48:50,780
"We will further improve the efficiency of
emissions after treatment of our SCR
537
00:48:50,780 --> 00:48:55,990
diesel engines and so on as far as the
laws of physics allow. This includes a
538
00:48:55,990 --> 00:48:59,680
voluntary service action" - and this
basically means a software update for your
539
00:48:59,680 --> 00:49:04,990
car - "for the cars that are already on
the road starting in June." So that is
540
00:49:04,990 --> 00:49:08,780
great. They're actually improving
something. Question's in which year,
541
00:49:08,780 --> 00:49:17,000
because this statement is from May 2016
and it's not out yet, but... Opel actually
542
00:49:17,000 --> 00:49:21,230
provided a new software already in July
and I think they already worked on this
543
00:49:21,230 --> 00:49:27,010
for quite a while and in July 16 the German
KBA, the Kraftfahrtbundesamt, the Federal
544
00:49:27,010 --> 00:49:31,730
Motor Transport Authority, they are pretty
nice actually, and they do know about what
545
00:49:31,730 --> 00:49:36,210
they do, they are bit limited by the
resources they have, and by the manpower
546
00:49:36,210 --> 00:49:41,630
they have, but they know about cars and
they know how to do these investigations.
547
00:49:41,630 --> 00:49:44,760
I mean, they're a little bit bound, but
what they should do and what they should
548
00:49:44,760 --> 00:49:49,200
not do, but they asked me to review a new
ECU software that was given to them by
549
00:49:49,200 --> 00:49:57,360
Opel for the Zafira in question and
Insignia, which had a similar ECU and I
550
00:49:57,360 --> 00:50:01,350
looked at that software and I dumped the
firmware and I looked at basically all the
551
00:50:01,350 --> 00:50:05,460
code sequences that I looked at before and
I was positively surprised because they
552
00:50:05,460 --> 00:50:10,950
removed... they addressed each of our
concerns. All of them, within the physical
553
00:50:10,950 --> 00:50:14,530
limitations of course. So they improved
the temperature window and everything, so
554
00:50:14,530 --> 00:50:19,090
there was a significant improvement. They
were able to improve the software and they
555
00:50:19,090 --> 00:50:25,690
let the DUH, which is the German
Environmental Aid, they used a PEMS system
556
00:50:25,690 --> 00:50:28,910
- PEMS is a portable
emissions measurement system. It's
557
00:50:28,910 --> 00:50:33,460
something you put on the exhaust pipe on
your car and then you can measure the
558
00:50:33,460 --> 00:50:40,700
exhaust during real-world driving, and
Opel gave them a car with the new ECU
559
00:50:40,700 --> 00:50:45,520
software. Otherwise the car was identical
to the old software, and the results are
560
00:50:45,520 --> 00:50:49,680
this, right, so on the left side you see
the old software, that has all these
561
00:50:49,680 --> 00:50:55,040
things that we criticized, and on the
right side you see the same car with a new
562
00:50:55,040 --> 00:51:00,500
ECU software and it's significantly
better. It's only slightly above the
563
00:51:00,500 --> 00:51:05,770
limit, right, but it's much better than
before and to put this in relation,
564
00:51:05,770 --> 00:51:11,260
before they were on the list pretty bad -
so this is sorted by worst to best - so
565
00:51:11,260 --> 00:51:15,720
they are in the, well, upper half at
least, and now they are almost one of the
566
00:51:15,720 --> 00:51:21,840
best cars, just by switching the ECU
software. And I mean this is great news,
567
00:51:21,840 --> 00:51:27,690
right, they actually improved their cars.
Let's just hope they get this out to the
568
00:51:27,690 --> 00:51:31,460
cars soon. Let's just hope it doesn't have
side effects and something, but I'm sure
569
00:51:31,460 --> 00:51:37,440
Opel knows how to test for this. Going
back to these, we worked on the Opel
570
00:51:37,440 --> 00:51:44,290
thing... I think the Opel case, it....
once they actually upgrade the cars, and
571
00:51:44,290 --> 00:51:48,940
once the cars really show these great
values that the preliminary software
572
00:51:48,940 --> 00:51:53,240
showed, I think we can close the Opel
case, but there's a lot of other cars
573
00:51:53,240 --> 00:51:59,940
still to look at, and really, I mean...
the effort to do this does not scale to so
574
00:51:59,940 --> 00:52:05,960
many cars, so we need to do something more
fundamentally to improve the situation.
575
00:52:05,960 --> 00:52:12,240
What I found out is that digital control
systems, they are black boxes. The
576
00:52:12,240 --> 00:52:17,590
manufacturers have designed them to be
black boxes. They even boast to you that
577
00:52:17,590 --> 00:52:22,060
they are 7,000 parameter in there and no
hacker can understand this and it's a very
578
00:52:22,060 --> 00:52:26,910
sophisticated problem. They are designed
to be black box, and this is not just true
579
00:52:26,910 --> 00:52:31,650
for Opel, this is true for all car
manufacturers. Nobody wants anyone to look
580
00:52:31,650 --> 00:52:37,930
into their ECUs, and people seem to be ok
with that. Like they think "Oh this is so
581
00:52:37,930 --> 00:52:40,850
complicated, there are so many German
engineers working on
582
00:52:40,850 --> 00:52:46,040
this problem, they must have found a great
solution." So we are trusting these black
583
00:52:46,040 --> 00:52:53,120
boxes and we are not able to review the
black boxes that we put into our cars and
584
00:52:53,120 --> 00:52:59,130
we have to trust the manufacturer to do
the right thing and currently, the
585
00:52:59,130 --> 00:53:03,851
investigation to do this without
assistance from the manufacturer, it does
586
00:53:03,851 --> 00:53:11,940
not scale. We can do it but... the
manufacturers can put more security on
587
00:53:11,940 --> 00:53:18,370
their ECUs... it probably can be broken,
but it takes a lot more time, so it simply
588
00:53:18,370 --> 00:53:23,580
does not scale sufficiently. The issue is
black boxes are really powerful, right.
589
00:53:23,580 --> 00:53:27,600
Black boxes can hurt people with, for
example, excessive emissions. They can
590
00:53:27,600 --> 00:53:33,160
kill people if we think think about
autonomous cars that do mistakes. So what
591
00:53:33,160 --> 00:53:40,060
we do need, I think, is more transparency.
A system that can kill people needs to be
592
00:53:40,060 --> 00:53:43,880
reviewable by the people. I think this is
a very important thing.
593
00:53:43,880 --> 00:53:52,755
applause
594
00:53:52,755 --> 00:53:56,730
So, to have a system that can kill
595
00:53:56,730 --> 00:54:02,050
people... to have it reviewable by the
people, we need to do things. For example,
596
00:54:02,050 --> 00:54:06,550
we need... we want access to source code
for reviews. It doesn't necessarily mean
597
00:54:06,550 --> 00:54:11,210
we want open source, but we don't ask at
all the car manufacturers to open source
598
00:54:11,210 --> 00:54:15,140
all the software. That's not what I'm
talking about. What we need is... think
599
00:54:15,140 --> 00:54:18,870
about how Microsoft is sharing source code
of Windows with universities or other
600
00:54:18,870 --> 00:54:25,530
countries. We need experts to look at the
source code, and we want control software
601
00:54:25,530 --> 00:54:30,130
that is reviewable by design, that has a
lot of documentation, that has good
602
00:54:30,130 --> 00:54:34,240
comments, that is human readable code. I
don't want to see a disassembly, I want to
603
00:54:34,240 --> 00:54:38,460
see the source, the MATLAB, or whatever
they are using to define the functionality
604
00:54:38,460 --> 00:54:44,300
source, and read that. And I want to
understand why did they choose that
605
00:54:44,300 --> 00:54:49,150
curve of that map in this way? What was
the design criteria? That needs to be
606
00:54:49,150 --> 00:54:55,040
reviewed. And we need transparency for
control software decisions, which means
607
00:54:55,040 --> 00:55:02,710
that if a car operates in a certain way,
if I'm driving that car, I want to choose
608
00:55:02,710 --> 00:55:07,300
that I can log what the car is doing, for
example by putting
609
00:55:07,300 --> 00:55:10,740
in, I don't know, a USB stick or something
if it's my car, and then the car will log
610
00:55:10,740 --> 00:55:17,070
all the data to that. That is... in the
end that allows me to reconstruct any
611
00:55:17,070 --> 00:55:21,780
decision that the software does. I think
this is required to have the necessary
612
00:55:21,780 --> 00:55:31,070
transparency, that allows us to un-
blackbox these devices. All right.
613
00:55:31,070 --> 00:56:00,940
Thank you very much.
applause
614
00:56:00,940 --> 00:56:04,180
Okay, I actually finished five minutes
early. I didn't think this would happen,
615
00:56:04,180 --> 00:56:06,050
so...
Herald: I'm so surprised.
616
00:56:06,050 --> 00:56:07,703
F: I am surprised too.
Herald: You are on time. You have five
617
00:56:07,703 --> 00:56:09,600
minutes left
F: Wow, what do I do with these five
618
00:56:09,600 --> 00:56:12,160
minutes
Herald: We can walk around the stage or...
619
00:56:12,160 --> 00:56:15,390
Maybe people have some questions?
F: I think so!
620
00:56:15,390 --> 00:56:24,019
Herald: Well, let's ask the Internet! Is
the Internet ready?
621
00:56:24,019 --> 00:56:28,850
Signal Angel: Yes. Our first question:
What dou you think is the responsibility
622
00:56:28,850 --> 00:56:32,930
of Bosch as a supplier for having their
software and hardware used for this?
623
00:56:32,930 --> 00:56:37,350
F: So the question was: What's the
responsibility for Bosch, who built the
624
00:56:37,350 --> 00:56:42,770
software for Volkswagen? It's a good
question and I have to be careful in what
625
00:56:42,770 --> 00:56:47,940
I answer. My personal opinion, and let's
take this aside from Volkswagen and Bosch,
626
00:56:47,940 --> 00:56:52,280
is that if you build software that you
know is used to be illegally it should...
627
00:56:52,280 --> 00:56:57,390
it must be your responsibility to not do
that. And I'm not sure if this is
628
00:56:57,390 --> 00:57:03,880
something that is legally enforceable, but
it should be something that's enforceable
629
00:57:03,880 --> 00:57:10,540
ethically or for all of us programmers,
that we don't build software that is
630
00:57:10,540 --> 00:57:17,340
designed to break the law.
applause
631
00:57:17,340 --> 00:57:20,840
Herald: We quickly hop over to microphone
1 please.
632
00:57:20,840 --> 00:57:25,440
Microphone: Thank you for a wonderful
talk. I'm just wondering if you're aware
633
00:57:25,440 --> 00:57:30,720
of some cases of Volkswagen cars in
Australia, which was suffering from sudden
634
00:57:30,720 --> 00:57:35,740
and rapid power loss. This was happening
about five years ago and there was a case
635
00:57:35,740 --> 00:57:41,510
where a Volkswagen suffered rapid power
loss on a motorway. The driver was Mrs.
636
00:57:41,510 --> 00:57:47,990
Melissa Ryan and she was rear-ended by a
truck and killed. So when you say that
637
00:57:47,990 --> 00:57:52,980
these things can cause death, were you...
are you aware that any sort of Volkswagen
638
00:57:52,980 --> 00:57:58,150
software has been leading to power loss in
the vehicles and affecting
639
00:57:58,150 --> 00:58:02,010
performance on the road, now I don't know
whether Australian driving conditions are
640
00:58:02,010 --> 00:58:06,200
different to European driving conditions,
and how that might affect that. Have you
641
00:58:06,200 --> 00:58:10,690
done any tests that might indicate that
could be happening in normal driving?
642
00:58:10,690 --> 00:58:14,900
F: Yeah, so... the question was whether
I'm aware of, I think an Australian
643
00:58:14,900 --> 00:58:17,290
incident, right, where...
M1: Can I...
644
00:58:17,290 --> 00:58:20,920
F: Yeah.
M1: There were many reported cases. One of
645
00:58:20,920 --> 00:58:23,650
them was fatal, but there were many
reported cases of that happening.
646
00:58:23,650 --> 00:58:28,000
F: Of a sudden power loss, is that right?
M1: Sudden and rapid power loss in the
647
00:58:28,000 --> 00:58:31,000
engine.
F: Yeah, of the engine. I'm not aware of
648
00:58:31,000 --> 00:58:38,850
these incidents and I what I do know
and... is that the the personal safety is
649
00:58:38,850 --> 00:58:43,320
the number one design criteria for ECUs.
That does not mean that they are perfect,
650
00:58:43,320 --> 00:58:47,210
of course, that could mean that rare
bugs... that there could be malfunctions.
651
00:58:47,210 --> 00:58:53,010
I don't know about this, but at least it's
the first design principle to provide the
652
00:58:53,010 --> 00:58:58,050
safety for the people driving the car,
which i think is a good thing, right. It's
653
00:58:58,050 --> 00:59:02,040
not the profit or anything, or at least we
can hope so. I'm not aware of this
654
00:59:02,040 --> 00:59:07,510
particular incidence, and so I can't
really say anything more about this. It
655
00:59:07,510 --> 00:59:11,690
would be great if... Are you aware of any
additional details that were found in the
656
00:59:11,690 --> 00:59:16,290
investigation, please sent them to me.
M1: Volkswagen was claiming that this was
657
00:59:16,290 --> 00:59:20,660
a gearbox problem on automatic cars, but
then it started happening on manual cars
658
00:59:20,660 --> 00:59:24,590
as well, so that excuse went out of the
window.
659
00:59:24,590 --> 00:59:27,430
F: The issue with the problems is that
most of them are very complex, so they
660
00:59:27,430 --> 00:59:32,580
probably involve more than just the engine
ECU, so they're very... but it's a
661
00:59:32,580 --> 00:59:37,450
good example of where we need to
understand exactly what is happening, and
662
00:59:37,450 --> 00:59:42,200
where we may not want to rely on
Volkswagen or any other manufacturer alone
663
00:59:42,200 --> 00:59:48,790
to assist in figuring out what happens. We
need more transparency there so that we
664
00:59:48,790 --> 00:59:54,040
can have definitely neutral accident
investigations.
665
00:59:54,040 --> 00:59:58,410
Herald: This was a long question and
really detailed answer. Thank you very
666
00:59:58,410 --> 01:00:00,410
much.
F: Sorry, I will be short
667
01:00:00,410 --> 01:00:06,601
Herald: Felix, that's your applause
applause
668
01:00:06,601 --> 01:00:18,610
music
669
01:00:18,610 --> 01:00:31,000
subtitles created by c3subtitles.de
in the year 2018. Join, and help us!