WEBVTT
00:00:00.000 --> 00:00:18.751
36c3 Intro Music
00:00:19.571 --> 00:00:23.300
Herald: ...now with the talk "The useful-
harmless spectrum". As I said,
00:00:23.300 --> 00:00:26.625
he needs no introduction: Fefe.
00:00:26.625 --> 00:00:30.482
Applause
00:00:30.482 --> 00:00:32.472
Tapping on the microphone
00:00:37.082 --> 00:00:40.110
Fefe: Good morning, I am happy that there
are so many people here.
00:00:40.110 --> 00:00:44.340
Thankfully this is not Hall 1. That would
be bad, with so many people.
00:00:44.340 --> 00:00:46.770
I have to manage your expectations
before I start,
00:00:46.770 --> 00:00:50.820
I actually submitted a different talk
last year about TCB-minimization,
00:00:50.820 --> 00:00:54.240
which would have been a bit technical,
about what you can do
00:00:54.240 --> 00:00:59.670
as a programmer. It was not accepted,
I don't know why - schedule was full.
00:00:59.670 --> 00:01:02.520
I submitted it again this year,
but I didn't want it to look
00:01:02.520 --> 00:01:05.520
like I want to bother them, so I
submitted another talk.
00:01:05.520 --> 00:01:10.800
...of course they accepted that one.
Which meant, I had to
00:01:10.800 --> 00:01:12.810
quickly prepare it now.
Audience laughs
00:01:13.320 --> 00:01:19.290
Well, the problem is, this is more of a
thought process than a structured
00:01:19.290 --> 00:01:23.490
presentation. I hope that it'll be
helpful none the less. But it's
00:01:23.490 --> 00:01:27.720
not as structured as my usual talks.
I will just start. So, there are multiple
00:01:27.720 --> 00:01:32.310
approaches, that basically result in
the same result, and I will
00:01:32.310 --> 00:01:36.540
just let you listen. Relatively early in
my career, I decided the following:
00:01:36.540 --> 00:01:40.980
I will never write software on which
people's lives may depend,
00:01:40.980 --> 00:01:45.780
like medical devices, nuclear reactors;
that was my idea.
00:01:45.780 --> 00:01:51.120
Of course not military either. And then
I met somebody that writes code for
00:01:51.120 --> 00:01:54.450
nuclear reactors. And it was the kind of
guy that says "That's super easy"
00:01:54.450 --> 00:02:00.300
So when those that know their
limits don't do it, then
00:02:00.300 --> 00:02:03.660
the other people will.
Audience laughs
00:02:04.630 --> 00:02:08.980
I don't want to generalize though.
I also met another guy that
00:02:08.980 --> 00:02:12.160
was not like this, but I mean,
this type of person exist.
00:02:12.160 --> 00:02:18.220
I believe that the problem here
is that you learn programming
00:02:18.220 --> 00:02:23.350
exploratively: It's not like a set path,
on which you walk, but rather you
00:02:23.350 --> 00:02:28.690
are just walking around and finding
your limits. But by definition this also
00:02:28.690 --> 00:02:33.160
means, that you don't know your limits
yet, because you are looking for them.
00:02:33.160 --> 00:02:38.260
This also means that you are always
working at your limit though. When people
00:02:38.260 --> 00:02:41.200
write software, then they go
just as far as they believe they
00:02:41.200 --> 00:02:47.320
can just barely go. In turn, this also
means that the technology that
00:02:47.320 --> 00:02:50.530
is being rolled out out there is mainly
not tried and tested
00:02:50.530 --> 00:02:55.240
or well understood, but rather it is the
technology, that the programmer
00:02:55.240 --> 00:03:01.450
just barely still understood. This is a
bit of a problem, which is further
00:03:01.450 --> 00:03:04.660
amplified by today's modularization and
dependency wave, where
00:03:04.660 --> 00:03:09.460
people just pull in modules from elsewhere
00:03:09.460 --> 00:03:16.540
and just assume that the writer of that
module must know what they are doing,
00:03:16.540 --> 00:03:20.650
though without any foundation in reality.
And it is often not the case. Instead,
00:03:20.650 --> 00:03:25.480
they are people like you and me, that
also worked exploratively.
00:03:25.480 --> 00:03:30.070
You can also do a little thought
experiment and get to this
00:03:30.070 --> 00:03:34.210
conclusion yourself; you could even
observe it happening. Let's assume
00:03:34.210 --> 00:03:37.840
that somebody finds a better way to
deal with complexity. For example
00:03:37.840 --> 00:03:41.200
modularization, or object-oriented
programming, when this was new.
00:03:41.200 --> 00:03:44.770
So then you would hope that we would
improve the software that we
00:03:44.770 --> 00:03:47.560
had written before, because we now
have it more under control.
00:03:47.560 --> 00:03:51.100
But this does not happen.
Instead, we now write bigger
00:03:51.100 --> 00:03:57.220
software and work at our limit
again. I think this is not
00:03:57.220 --> 00:04:00.400
a problem of software development or
programming, but generally
00:04:00.400 --> 00:04:03.790
a problem of humans. Evolution
made us this way, and we have to
00:04:03.790 --> 00:04:07.820
learn to deal with it. Let me illustrate
this: I have a theory,
00:04:07.820 --> 00:04:14.870
which I call the gradient-theory. The
thesis is, that humans treat their
00:04:14.870 --> 00:04:18.110
environment like a process of optimization
in mathematics. This means you
00:04:18.110 --> 00:04:22.850
have a terrain and you are looking for
the highest or lowest point - that is an
00:04:22.850 --> 00:04:29.360
optimization problem. And you can't
directly aim for it, because you don't
00:04:29.360 --> 00:04:34.280
know the terrain. Instead, you have to
make assumptions, and you can observe
00:04:34.280 --> 00:04:37.490
this on yourself. If it's too cold, then
you go to the radiator and
00:04:37.490 --> 00:04:41.510
you don't set it to the perfect heat,
you set it to "hot", then you wait
00:04:41.510 --> 00:04:44.390
until it's too hot, then you
turn it down again.
00:04:44.390 --> 00:04:47.510
So we interact with our environment in
a process of approximation.
00:04:47.510 --> 00:04:50.030
And not just with heaters, but also when
driving a car, when we have a map.
00:04:50.030 --> 00:04:53.840
We look, "where is the limit? Where do
we have to turn?", and
00:04:53.840 --> 00:04:58.730
we ignore the journey to the turn,
even if it is nice.
00:04:58.730 --> 00:05:03.410
Many things that we do, also including
our choice of speed, is such a gradient.
00:05:03.410 --> 00:05:06.320
We accelerate until we feel unwell,
then we slow down again.
00:05:06.320 --> 00:05:11.390
Or when searching for something in
a telephone book or dictionary,
00:05:11.390 --> 00:05:15.785
we make an assumption of where
it will be. And when it is
00:05:15.785 --> 00:05:19.070
too far, we go back again. The essence
of it is: We make an assumption
00:05:19.070 --> 00:05:22.580
about what the terrain looks like.
We have smooth transitions here,
00:05:22.580 --> 00:05:26.480
so this technique works well.
This is called gradient descent
00:05:26.480 --> 00:05:29.930
by the way, when you try to follow
gravity to find the lowest point.
00:05:29.930 --> 00:05:34.490
But it does not work well
in two scenarios:
00:05:34.490 --> 00:05:38.090
Firstly, when there is a cliff where I
can't go back once I have walked
00:05:38.090 --> 00:05:41.930
over it. It also doesn't go well when
you don't notice that you have gone
00:05:41.930 --> 00:05:46.400
too far. Well it is similar to the cliff,
and the second problem is
00:05:46.400 --> 00:05:49.970
when you can't roll back
for other reasons.
00:05:49.970 --> 00:05:53.810
This happens frequently in software
development, and it turns out, that
00:05:53.810 --> 00:05:58.340
this is exactly the kind of problem that
human have. For example,
00:05:58.340 --> 00:06:03.430
when we have a trial subscription for two
weeks, people forget to cancel it again,
00:06:04.030 --> 00:06:09.580
or drug addiction is a classic, or
gambling addiction. And in software
00:06:09.580 --> 00:06:12.370
development or project management
in general this is common:
00:06:12.370 --> 00:06:17.260
We have already invested so much that
we can't go back. Security is not
00:06:17.260 --> 00:06:22.240
a gradient. It may look like one, but it
isn't. I think this is
00:06:22.240 --> 00:06:26.800
a fundamental issue in IT security.
You don't notice when you
00:06:26.800 --> 00:06:30.640
have gone too far. You only notice
when you get hacked. And then
00:06:30.640 --> 00:06:35.020
you can no longer go back, all the data
is already gone. Complexity is also
00:06:35.020 --> 00:06:38.260
not a gradient, similarly to security,
but it feels like one. I think
00:06:38.260 --> 00:06:42.130
this is the reason why we deal with
it so badly. It feels
00:06:42.130 --> 00:06:45.130
as if we have everything under
control. And when we notice,
00:06:45.130 --> 00:06:50.140
that we don't, we can't go back.
By the way, giving out data to
00:06:50.140 --> 00:06:54.820
facebook is also such a "pseudo-gradient".
00:06:54.820 --> 00:07:00.550
When you notice that you gave away too
much, it is too late.
00:07:00.550 --> 00:07:05.650
So the conclusion is:
Complexity is evil. We notice it too
00:07:05.650 --> 00:07:09.610
late and we get into it too easily.
So we have to counteract that somehow.
00:07:09.610 --> 00:07:14.680
If this is our job, we are externalizing
the costs to our customers,
00:07:14.680 --> 00:07:19.480
to our users, and to our future self.
00:07:19.480 --> 00:07:24.700
This is why you rarely find older software
developers that are happy.
00:07:24.700 --> 00:07:28.901
Audience laughs
So, this was the first train of thought,
00:07:28.901 --> 00:07:32.786
that led me in this direction. The second
train of thought: Let me just show you
00:07:32.786 --> 00:07:35.854
the GNU manifesto, as a representative.
This is not GNU-bashing,
00:07:35.854 --> 00:07:39.484
but you can show this pretty well with
the example of the GNU manifesto.
00:07:39.484 --> 00:07:43.647
This is the original announcement of the
GNU project by Richard Stallman. He wrote:
00:07:43.647 --> 00:07:47.939
"GNU will be able to run Unix programs, but
will not be identical to Unix. We will make
00:07:47.939 --> 00:07:53.041
all improvements that are convenient".
This is a very bad sentence.
00:07:53.041 --> 00:07:58.473
What does "convenient" mean? For who?
00:07:58.473 --> 00:08:03.258
But this is the approach that a lot of
programmers have:
00:08:03.258 --> 00:08:07.281
"Oh we can just add this quickly."
We are lacking a corrective, that
00:08:07.281 --> 00:08:11.304
we think in advance "what legacy am I
hanging to my leg right now?"
00:08:11.304 --> 00:08:15.766
I think this "convenience" thought when
extending software is our "original sin"
00:08:15.766 --> 00:08:20.010
- to get a bit catholic here -
in software development.
00:08:20.010 --> 00:08:24.252
Everyone has done it before, and you
just can't correct it after the fact.
00:08:24.252 --> 00:08:27.256
So the only way of getting rid of it
is to throw away
00:08:27.256 --> 00:08:31.626
the whole software or module and
start over again. But software doesn't die.
00:08:31.626 --> 00:08:36.592
Only when dealing with software, I learned
that it is good that people die,
00:08:36.592 --> 00:08:40.508
because it is a corrective that is needed.
If a system is supposed to improve,
00:08:40.508 --> 00:08:44.026
the old stuff has to be able to die at
some point. And this does not
00:08:44.026 --> 00:08:49.584
happen with software. It is a feature
that things don't last forever.
00:08:49.584 --> 00:08:55.269
In general, you can observe that when
somebody is extending their software and
00:08:55.269 --> 00:08:58.484
they have a choice between "We do
something to solve our specific problem"
00:08:58.484 --> 00:09:01.905
or "We do something to solve a more
general problem", people will
00:09:01.905 --> 00:09:06.636
always try to solve the
more general problem.
00:09:06.636 --> 00:09:12.057
"The more danger, the more honor."
And you can see this across the board.
00:09:12.057 --> 00:09:16.859
There are very few exceptions to this. And
I had my "aha-moment" when I opened
00:09:16.859 --> 00:09:21.215
'gdb' on a project one day. I took '/tmp'
here, but that project was
00:09:21.215 --> 00:09:26.135
some checkout.
In my webserver, I have a '.gdbinit' file.
00:09:26.135 --> 00:09:30.507
It's a configuration file for the GNU-
debugger, where you can for example say
00:09:30.507 --> 00:09:33.405
"Open this application that I want to
00:09:33.405 --> 00:09:36.808
debug with these arguments!"
And in there, I write "Don't use Port 80,
00:09:36.808 --> 00:09:41.393
that doesn't work, instead use port
8005" or something, to debug it on
00:09:41.393 --> 00:09:46.097
localhost. And one day, gdb started
saying "no, I don't accept this
00:09:46.097 --> 00:09:50.553
.gdbinit file because it is in a directory
00:09:50.553 --> 00:09:56.000
that you have not specifically allowed."
This was exactly such an attempt to fix
00:09:56.000 --> 00:10:01.097
an issue after shipping, after the fact.
gdb noticed: "Our config-file has become
00:10:01.097 --> 00:10:05.810
so powerful, that it is a security issue",
00:10:05.810 --> 00:10:11.038
and then retroactively nailed down the
whole config. And this broke more
00:10:11.038 --> 00:10:15.686
than it needed to - perhaps, I don't
know for sure - but it was very annoying
00:10:15.686 --> 00:10:19.270
for me. You can put an auto path in here,
but that is when I noticed it
00:10:19.270 --> 00:10:22.218
for the first time. This was a few years
00:10:22.218 --> 00:10:25.942
ago. I don't know, when exactly that was.
There was a similar case like this
00:10:25.942 --> 00:10:30.041
again: With Vim, the editor, that I like
to use. You can do things like
00:10:30.041 --> 00:10:33.882
in a comment in the file that is being
edited, you can put some configuration
00:10:33.882 --> 00:10:37.028
settings in the first or last three lines.
00:10:37.028 --> 00:10:41.870
It is supposed to be used for "I use
tabstop=4 here", or something.
00:10:41.870 --> 00:10:46.160
But the parser for this had
a security bug, which made it
00:10:46.160 --> 00:10:50.512
possible to create a file that
executes code, when it is
00:10:50.512 --> 00:10:55.564
opened in vim, which was of course
not intended. But it is the same
00:10:55.564 --> 00:10:59.847
issue. I think you can generalize this
00:10:59.847 --> 00:11:03.135
a bit - though earlier I argued
against generalizations, but
00:11:03.135 --> 00:11:06.535
in analysis it is good, in software
it is usually bad. let me illustrate
00:11:06.535 --> 00:11:10.777
with an example:
Let's assume that we have a CSV file
00:11:10.777 --> 00:11:16.194
with some trouble tickets. Field 4
is the one, that we are interested in.
00:11:16.194 --> 00:11:21.511
Let's assume it looks like this. It's CSV.
So, now I would like to have the sum
00:11:21.511 --> 00:11:26.285
of the four fields. So first I use
cut, we are in Unix here.
00:11:26.285 --> 00:11:31.012
Then the first line has to go,
00:11:31.012 --> 00:11:34.193
so I use tail. Now the first line
is gone, now I just have to
00:11:34.193 --> 00:11:37.746
calculate the sum. There is an
application for this too: paste. that is
00:11:37.746 --> 00:11:43.442
how you do it in Unix. Then I have to
calculate it. There we go! But what if
00:11:43.442 --> 00:11:49.381
it doesn't say 1 here, but instead "fred"?
We notice: cut does not have a problem,
00:11:49.381 --> 00:11:54.442
tail does not have a problem, paste is
fine, but bc falls on its face.
00:11:54.442 --> 00:12:01.973
Even worse, bc is programmable.
There could be the
00:12:01.973 --> 00:12:05.214
Ackermann-function here and
your computer would be gone
00:12:05.214 --> 00:12:09.772
for an hour, while it is trying to
solve some recursion. And I think it
00:12:09.772 --> 00:12:14.823
is useful to introduce a concept here
to say: cut, tail and paste are harmless,
00:12:14.823 --> 00:12:18.817
bc is not. This is one of the thoughts
where I thought "okay, you can make
00:12:18.817 --> 00:12:22.152
a talk about this".
But this is not enough.
00:12:22.152 --> 00:12:27.235
There are different kinds of harmless.
But I think this simple idea
00:12:27.235 --> 00:12:31.405
already helps us a bit.
Let's make it into a sentence:
00:12:31.405 --> 00:12:35.204
Software is harmless, when unexpected
input don't produce unexpected
00:12:35.204 --> 00:12:38.868
behavior or unexpected kinds of output.
For example, an SHA-checksum is always
00:12:38.868 --> 00:12:43.166
harmless. Regardless of
what data I put in, the output
00:12:43.166 --> 00:12:47.742
has a known format. Or word
count (wc) is also one of those.
00:12:47.742 --> 00:12:52.104
Now you could say: "Okay, just use
awk!" And in awk I don't have a problem
00:12:52.104 --> 00:12:55.955
when it says "fred" instead of "4"
and the interpreter also does not
00:12:55.955 --> 00:13:00.541
interpret any functions.
It looks better, but
00:13:00.541 --> 00:13:05.397
is it really harmless?
It turns out, awk is a different kind of
00:13:05.397 --> 00:13:09.385
not harmless, because you can write
in the filesystem with it. So I don't have
00:13:09.385 --> 00:13:13.548
to worry about the input, but I have to
worry about the code, that I hand to it
00:13:13.548 --> 00:13:17.275
on the command line. So that is
another distinction you can make.
00:13:17.275 --> 00:13:21.812
This is a big problem in the game
industry by the way:
00:13:21.812 --> 00:13:25.862
The game development industry
has started putting interpreters
00:13:25.862 --> 00:13:30.856
into their games, to be able to write
their business logic - not the AI,
00:13:30.856 --> 00:13:36.820
but small scripts - in a scripting
language. One of the most
00:13:36.820 --> 00:13:41.132
popular script-interpreters for this
purpose is Lua. And Lua is primarily
00:13:41.132 --> 00:13:45.091
used because it can't do anything,
if you don't specifically allow it.
00:13:45.091 --> 00:13:48.926
So It can't open files or sockets.
You can enable this manually though,
00:13:48.926 --> 00:13:53.190
and then you have a problem again
of course. But this is a real issue.
00:13:53.190 --> 00:13:57.149
Many open-source people don't think
about this, because they think "Well,
00:13:57.149 --> 00:14:00.358
I will ship it and the rest is no longer
my issue." But I think,
00:14:00.358 --> 00:14:03.335
that we generally have to think
about this, and preferably
00:14:03.335 --> 00:14:06.771
before shipping, optimally already while
programming. So, this is
00:14:06.771 --> 00:14:11.226
a different kind of harmlessness.
The first kind was "Can bad input
00:14:11.226 --> 00:14:15.014
cause bad output?" And now: "Can the
application itself do bad things?"
00:14:15.014 --> 00:14:19.322
This is a very modern thought,
because we work a lot more with
00:14:19.322 --> 00:14:23.874
sandboxing today. In sandboxing, the goal
is to prevent a program from
00:14:23.874 --> 00:14:28.024
accidentally or deliberately doing bad
things. And there are again different
00:14:28.024 --> 00:14:32.605
things that a program can do.
bc can eat processing time. awk can
00:14:32.605 --> 00:14:37.095
read and write in your filesystem, and
this goes on and on. Let's get back
00:14:37.095 --> 00:14:41.740
to the GNU manifesto: GNU awk is a special
version of awk and it can open sockets,
00:14:41.740 --> 00:14:45.652
without any need. This means, if we
just use awk and thing "Well, awk can
00:14:45.652 --> 00:14:49.086
write in the filesystem, but I mounted
that read-only, so nothing
00:14:49.086 --> 00:14:53.457
can happen". But then if GNU awk
is being used, it is suddenly
00:14:53.457 --> 00:14:57.802
no longer harmless. Bash
can open sockets too by the way!
00:14:57.802 --> 00:15:02.788
I don't know, how many people knew that?
This goes on of course: after awk
00:15:02.788 --> 00:15:06.446
came Perl. It's even worse, and
Perl can do eval(), which in my
00:15:06.446 --> 00:15:11.425
opinion is the worse evil that you can
have in a programming language.
00:15:11.425 --> 00:15:15.985
A bit closer to the end-user you can also
observe this in browsers. Let's look at
00:15:15.985 --> 00:15:20.523
Netscape for example:
Several times, Netscape had the choice
00:15:20.523 --> 00:15:24.977
between "useful" and "harmless" and always
chose "useful". It started with
00:15:24.977 --> 00:15:29.442
the plugins. I don't know, who
of you still remembers the Flash-plugin,
00:15:29.442 --> 00:15:33.755
or before that we all had the RealPlayer,
and there was also an Acrobat-plugin -
00:15:33.755 --> 00:15:37.641
And all of it was shit, because the
plugins were native code: they could do
00:15:37.641 --> 00:15:41.829
everything, that their operating system
allowed. This means that it was very
00:15:41.829 --> 00:15:45.635
useful, but also very dangerous.
And it was a conscious choice of
00:15:45.635 --> 00:15:49.579
the browsers, to allow this.
The actual goal of this talk is
00:15:49.579 --> 00:15:54.202
to give the programmers among you a
bit of awareness that you don't just
00:15:54.202 --> 00:15:58.933
add a plugin interface that
can do everything.
00:15:58.933 --> 00:16:04.564
The next iteration was:
We'll do everything in JavaScript.
00:16:04.564 --> 00:16:09.562
At first it looked better, but this
JavaScript eventually also ran with
00:16:09.562 --> 00:16:13.861
enough privileges do do bad things
in the system, or at least in the browser.
00:16:13.861 --> 00:16:17.610
It turns out: People now have their
important data in the browser,
00:16:17.610 --> 00:16:21.064
because they do online banking. And
that is enough do do a lot of damage.
00:16:21.064 --> 00:16:25.609
Then they had to correct it
after the fact. Chrome now imposes
00:16:25.609 --> 00:16:29.383
even further limits for security reasons
to break ad blockers. It's always
00:16:29.383 --> 00:16:32.601
the same trap that we walk into.
Who of you here use Windows?
00:16:32.601 --> 00:16:37.285
In Windows there is a tool by
Mark Russinovich - by now he has
00:16:37.285 --> 00:16:41.300
sold it to Microsoft, so it is now an
official Microsoft tool.
00:16:41.300 --> 00:16:44.680
And the only functionality of this
tool is to list the different
00:16:44.680 --> 00:16:48.013
plugins that are part of the system.
And I took a relatively
00:16:48.013 --> 00:16:52.285
clean system here. It's not about
this down here or
00:16:52.285 --> 00:16:56.549
the size of the scrollbar, but just
how many tabs there are at the top:
00:16:56.549 --> 00:17:00.745
These are all different options for
plugins to integrate into the system,
00:17:00.745 --> 00:17:04.445
and nobody has an overview of this
anymore, because people always decided
00:17:04.445 --> 00:17:08.798
to go in the wrong direction. I believe
that this is a core problem.
00:17:08.798 --> 00:17:13.857
There is a third approach to this:
My daily life in security consists of
00:17:13.857 --> 00:17:17.926
going to companies. They show me their
source code and I look for bugs. Then
00:17:17.926 --> 00:17:21.920
I tell them, which bugs I found. And
occasionally, there are cases where
00:17:21.920 --> 00:17:25.808
I notice that there are a lot of bugs.
Not just those that I find, but they
00:17:25.808 --> 00:17:30.035
already have their own database,
a bugtracker, and they already
00:17:30.035 --> 00:17:34.955
have a seven-digit number of bugs. Yes,
This happens. And since it is a problem
00:17:34.955 --> 00:17:39.361
that we have so many bugs, there
are now counter-strategies by developers
00:17:39.361 --> 00:17:42.746
that start saying: "Okay, if this bug is
not important then
00:17:42.746 --> 00:17:46.830
I can fix it later." And "later" means
"never" in reality. It just sits there.
00:17:46.830 --> 00:17:52.134
Joke that only makes sense in German
00:17:52.134 --> 00:17:58.087
In the real world, bug
trackers are often just
00:17:58.087 --> 00:18:03.812
massive permanent data disposal sites:
For example, I recently filed a bug report
00:18:03.812 --> 00:18:08.146
for Firefox and got the ID 1590000.
This is already a bad sign.
00:18:08.146 --> 00:18:11.876
But it is also a good sign, that
the bug tracker is open.
00:18:11.876 --> 00:18:16.007
For Microsoft you can't see how
many bugs they have.
00:18:16.007 --> 00:18:19.501
This is only meant for illustration.
Mozilla is not especially bad.
00:18:19.501 --> 00:18:23.170
Mozilla just has an open tracker,
on which I can show it well.
00:18:23.170 --> 00:18:27.217
What I wanted to show you -
I had a look: "What is the first bug
00:18:27.217 --> 00:18:31.017
that I filed there?" It still had
a six-digit ID.
00:18:31.017 --> 00:18:37.953
That was 2003. If you look at the
history of bug IDs then you notice:
00:18:37.953 --> 00:18:43.047
It is growing exponentially.
And it's not like the bugs somehow
00:18:43.047 --> 00:18:48.431
go away at some point.
I have noticed two major events,
00:18:48.431 --> 00:18:52.235
where bugs are closed:
When a new release is done
00:18:52.235 --> 00:18:55.851
and you throw out the old JavaScript
engine and put in a new one.
00:18:55.851 --> 00:18:59.700
Then you just close all bugs of the old
engine. It looks as if you have achieved
00:18:59.700 --> 00:19:03.568
something. And the second is this one:
I don't know, can you read this in
00:19:03.568 --> 00:19:06.848
the back? Mozilla just closed my bug.
It says:
00:19:06.848 --> 00:19:10.034
"This bug has been automatically
resolved after a period
00:19:10.034 --> 00:19:14.008
of inactivity". Mind you, it was not me
who was inactive. I filed the bug and
00:19:14.008 --> 00:19:17.750
nobody at Mozilla took care of it.
So they just automatically closed it,
00:19:17.750 --> 00:19:21.355
because the statistics look so bad.
This is a big issue,
00:19:21.355 --> 00:19:24.378
not just at Mozilla. As I said, this is
just the example
00:19:24.378 --> 00:19:28.262
that I can show, because
in their case it is public. But
00:19:28.262 --> 00:19:32.349
this leads to a cascade of action
and reaction. For example,
00:19:32.349 --> 00:19:36.089
unimportant bugs are just not fixed
anymore. And then people
00:19:36.089 --> 00:19:39.461
add "important" on their bugs,
because they want them to be fixed.
00:19:39.461 --> 00:19:42.780
Then they say "Okay, the important
bugs also don't get fixed,
00:19:42.780 --> 00:19:46.849
because there are too many of them."
And then people
00:19:46.849 --> 00:19:51.472
write "Security" on their bugs, and now
we have a wave of security-bugs.
00:19:51.472 --> 00:19:56.008
There they negotiate: "Is this really
a problem?" And then we get excuses
00:19:56.008 --> 00:20:01.232
like "It's just a crash."
The point is that there is an unholy
00:20:01.232 --> 00:20:07.589
alliance with another trend,
namely that companies see:
00:20:07.589 --> 00:20:11.476
We have so many bugs open that
solving the bugs is not the goal anymore.
00:20:11.476 --> 00:20:15.295
There are just too many, it is
unrealistic. Instead,
00:20:15.295 --> 00:20:19.598
we introduce metrics like "we do
fuzzing". Fuzzing is not
00:20:19.598 --> 00:20:23.897
a bad idea, but it is not "finding all
bugs", but just the first step
00:20:23.897 --> 00:20:28.090
on a long road. But it gives
out a nice metric.
00:20:28.090 --> 00:20:33.011
We have so-and-so many fuzz-
testcases, and now...
00:20:33.011 --> 00:20:37.402
Are we now better or worse than
before? It's hard to say.
00:20:37.402 --> 00:20:41.769
00:20:41.769 --> 00:20:46.975
00:20:46.975 --> 00:20:51.635
00:20:51.635 --> 00:20:55.373
00:20:55.373 --> 00:20:58.367
00:20:58.367 --> 00:21:01.752
00:21:01.752 --> 00:21:05.930
00:21:05.930 --> 00:21:09.778
00:21:09.778 --> 00:21:13.974
00:21:13.974 --> 00:21:18.438
00:21:18.438 --> 00:21:20.345
00:21:20.345 --> 00:21:23.400
00:21:23.400 --> 00:21:28.686
00:21:28.686 --> 00:21:33.658
00:21:33.658 --> 00:21:37.336
00:21:37.336 --> 00:21:41.931
00:21:41.931 --> 00:21:47.848
00:21:47.848 --> 00:21:51.779
00:21:51.779 --> 00:21:55.288
00:21:55.288 --> 00:21:58.910
00:21:58.910 --> 00:22:03.159
00:22:03.159 --> 00:22:07.913
00:22:07.913 --> 00:22:12.536
00:22:12.536 --> 00:22:17.253
00:22:17.253 --> 00:22:21.690
00:22:21.690 --> 00:22:26.237
00:22:26.237 --> 00:22:30.208
00:22:30.208 --> 00:22:33.999
00:22:33.999 --> 00:22:36.695
00:22:36.695 --> 00:22:41.009
00:22:41.009 --> 00:22:45.459
00:22:45.459 --> 00:22:49.214
00:22:49.214 --> 00:22:52.446
00:22:52.446 --> 00:22:56.310
00:22:56.310 --> 00:23:00.526
00:23:00.526 --> 00:23:04.246
00:23:04.246 --> 00:23:07.796
00:23:07.796 --> 00:23:12.318
00:23:12.318 --> 00:23:16.240
00:23:16.240 --> 00:23:20.475
00:23:20.475 --> 00:23:24.298
00:23:24.298 --> 00:23:27.676
00:23:27.676 --> 00:23:31.460
00:23:31.460 --> 00:23:36.410
00:23:36.410 --> 00:23:41.173
00:23:41.173 --> 00:23:45.020
00:23:45.020 --> 00:23:48.800
00:23:48.800 --> 00:23:53.213
00:23:53.213 --> 00:23:57.578
00:23:57.578 --> 00:24:03.977
00:24:03.977 --> 00:24:10.106
00:24:10.106 --> 00:24:13.625
00:24:13.625 --> 00:24:17.247
00:24:17.247 --> 00:24:21.130
00:24:21.130 --> 00:24:24.965
00:24:24.965 --> 00:24:28.860
00:24:28.860 --> 00:24:33.035
00:24:33.035 --> 00:24:37.879
00:24:37.879 --> 00:24:42.818
00:24:42.818 --> 00:24:48.100
00:24:48.100 --> 00:24:52.500
00:24:52.500 --> 00:24:56.206
00:24:56.206 --> 00:24:59.868
00:24:59.868 --> 00:25:03.716
00:25:03.716 --> 00:25:08.630
00:25:08.630 --> 00:25:13.261
00:25:13.261 --> 00:25:16.930
00:25:16.930 --> 00:25:19.907
00:25:19.907 --> 00:25:23.917
00:25:23.917 --> 00:25:27.947
00:25:27.947 --> 00:25:31.971
00:25:31.971 --> 00:25:35.440
00:25:35.440 --> 00:25:39.425
00:25:39.425 --> 00:25:44.229
00:25:44.229 --> 00:25:48.829
00:25:48.829 --> 00:25:52.582
00:25:52.582 --> 00:25:57.265
00:25:57.265 --> 00:26:02.796
00:26:02.796 --> 00:26:08.344
00:26:08.344 --> 00:26:13.337
00:26:13.337 --> 00:26:16.676
00:26:16.676 --> 00:26:19.707
00:26:19.707 --> 00:26:23.220
00:26:23.220 --> 00:26:26.838
00:26:26.838 --> 00:26:30.901
00:26:30.901 --> 00:26:34.758
00:26:34.758 --> 00:26:38.616
00:26:38.616 --> 00:26:42.095
00:26:42.095 --> 00:26:46.015
00:26:46.015 --> 00:26:49.071
00:26:49.071 --> 00:26:53.416
00:26:53.416 --> 00:26:57.395
00:26:57.395 --> 00:27:01.075
00:27:01.075 --> 00:27:03.522
00:27:03.522 --> 00:27:06.250
00:27:06.250 --> 00:27:09.796
00:27:09.796 --> 00:27:13.000
00:27:13.897 --> 00:27:17.249
00:27:17.249 --> 00:27:22.275
00:27:22.275 --> 00:27:26.830
00:27:26.830 --> 00:27:29.650
00:27:29.650 --> 00:27:33.349
00:27:33.349 --> 00:27:37.155
00:27:37.155 --> 00:27:41.157
00:27:41.157 --> 00:27:46.006
00:27:46.006 --> 00:27:49.877
00:27:49.877 --> 00:27:53.429
00:27:53.429 --> 00:27:56.744
00:27:56.744 --> 00:28:00.948
00:28:00.948 --> 00:28:05.771
00:28:05.771 --> 00:28:10.095
00:28:10.095 --> 00:28:14.460
00:28:14.460 --> 00:28:19.729
00:28:19.729 --> 00:28:24.462
00:28:24.462 --> 00:28:28.774
00:28:28.774 --> 00:28:33.027
00:28:33.027 --> 00:28:38.276
00:28:38.276 --> 00:28:42.951
00:28:42.951 --> 00:28:46.781
00:28:46.781 --> 00:28:50.708
00:28:50.708 --> 00:28:54.292
00:28:54.292 --> 00:28:58.676
00:28:58.676 --> 00:29:03.695
00:29:03.695 --> 00:29:07.750
00:29:07.750 --> 00:29:10.944
00:29:10.944 --> 00:29:14.235
00:29:14.235 --> 00:29:18.170
00:29:18.170 --> 00:29:21.906
00:29:21.906 --> 00:29:25.648
00:29:25.648 --> 00:29:29.813
00:29:29.813 --> 00:29:33.962
00:29:33.962 --> 00:29:37.258
00:29:37.258 --> 00:29:40.789
00:29:40.789 --> 00:29:44.961
00:29:44.961 --> 00:29:49.101
00:29:49.101 --> 00:29:53.353
00:29:53.353 --> 00:29:58.053
00:29:58.053 --> 00:30:02.925
00:30:02.925 --> 00:30:07.068
00:30:07.068 --> 00:30:11.207
00:30:11.207 --> 00:30:15.768
00:30:15.768 --> 00:30:20.235
00:30:20.235 --> 00:30:24.736
00:30:24.736 --> 00:30:28.717
00:30:28.717 --> 00:30:33.343
00:30:33.343 --> 00:30:38.565
00:30:38.565 --> 00:30:43.107
00:30:43.107 --> 00:30:49.769
00:30:49.769 --> 00:30:55.467
00:30:55.467 --> 00:30:59.760
00:30:59.760 --> 00:31:04.521
00:31:04.521 --> 00:31:08.482
00:31:08.482 --> 00:31:12.363
00:31:12.363 --> 00:31:18.015
00:31:18.015 --> 00:31:24.111
00:31:24.111 --> 00:31:29.850
00:31:29.850 --> 00:31:36.526
00:31:36.526 --> 00:31:42.156
00:31:42.156 --> 00:31:46.170
00:31:46.170 --> 00:31:49.830
00:31:49.830 --> 00:31:53.224
00:31:53.224 --> 00:31:56.585
00:31:56.585 --> 00:32:00.581
00:32:00.581 --> 00:32:05.106
00:32:05.106 --> 00:32:08.988
00:32:08.988 --> 00:32:12.573
00:32:12.573 --> 00:32:16.852
00:32:16.852 --> 00:32:21.549
00:32:21.549 --> 00:32:24.654
00:32:24.654 --> 00:32:28.779
00:32:28.779 --> 00:32:34.307
00:32:34.307 --> 00:32:39.450
00:32:39.450 --> 00:32:44.572
00:32:44.572 --> 00:32:49.299
00:32:49.299 --> 00:32:52.794
00:32:52.794 --> 00:32:56.071
00:32:56.071 --> 00:32:59.467
00:32:59.467 --> 00:33:03.335
00:33:03.335 --> 00:33:08.721
00:33:08.721 --> 00:33:13.595
00:33:13.595 --> 00:33:17.871
00:33:17.871 --> 00:33:22.411
00:33:22.411 --> 00:33:26.844
00:33:26.844 --> 00:33:31.971
00:33:31.971 --> 00:33:37.346
00:33:37.346 --> 00:33:41.891
00:33:41.891 --> 00:33:47.106
00:33:47.106 --> 00:33:52.356
00:33:52.356 --> 00:33:56.950
00:33:56.950 --> 00:34:01.703
00:34:01.703 --> 00:34:05.389
00:34:05.389 --> 00:34:10.021
00:34:10.021 --> 00:34:15.049
00:34:15.049 --> 00:34:20.110
00:34:20.110 --> 00:34:25.645
00:34:25.645 --> 00:34:29.835
00:34:29.835 --> 00:34:34.685
00:34:34.685 --> 00:34:39.107
00:34:39.107 --> 00:34:43.660
00:34:43.660 --> 00:34:48.532
00:34:48.532 --> 00:34:54.215
00:34:54.215 --> 00:34:59.203
00:34:59.203 --> 00:35:02.580
00:35:02.580 --> 00:35:07.256
00:35:07.256 --> 00:35:11.814
00:35:11.814 --> 00:35:16.713
00:35:16.713 --> 00:35:21.634
00:35:21.634 --> 00:35:25.555
00:35:25.555 --> 00:35:30.453
00:35:30.453 --> 00:35:34.457
00:35:34.457 --> 00:35:39.444
00:35:39.444 --> 00:35:44.786
00:35:44.786 --> 00:35:50.341
00:35:50.341 --> 00:35:55.009
00:35:55.009 --> 00:35:59.622
00:35:59.622 --> 00:36:04.077
00:36:04.077 --> 00:36:07.868
00:36:07.868 --> 00:36:11.956
00:36:11.956 --> 00:36:16.785
00:36:16.785 --> 00:36:21.632
00:36:21.632 --> 00:36:26.415
00:36:26.415 --> 00:36:30.549
00:36:30.549 --> 00:36:34.335
00:36:34.335 --> 00:36:38.224
00:36:38.224 --> 00:36:41.825
00:36:41.825 --> 00:36:46.801
00:36:46.801 --> 00:36:53.313
00:36:53.313 --> 00:36:58.917
00:36:58.917 --> 00:37:04.896
00:37:04.896 --> 00:37:09.333
00:37:09.333 --> 00:37:12.456
00:37:12.456 --> 00:37:15.758
00:37:15.758 --> 00:37:19.001
00:37:19.001 --> 00:37:23.259
00:37:23.259 --> 00:37:26.800
00:37:26.800 --> 00:37:30.960
00:37:30.960 --> 00:37:35.492
00:37:35.492 --> 00:37:39.209
00:37:39.209 --> 00:37:42.595
00:37:42.595 --> 00:37:47.263
00:37:47.263 --> 00:37:52.198
00:37:52.198 --> 00:37:55.821
00:37:55.821 --> 00:37:59.210
00:37:59.210 --> 00:38:02.628
00:38:02.628 --> 00:38:06.296
00:38:06.296 --> 00:38:10.513
00:38:10.513 --> 00:38:14.693
00:38:14.693 --> 00:38:20.115
00:38:20.115 --> 00:38:25.190
00:38:25.190 --> 00:38:28.810
00:38:28.810 --> 00:38:32.660
00:38:32.660 --> 00:38:36.058
00:38:36.058 --> 00:38:40.541
00:38:40.541 --> 00:38:45.744
00:38:45.744 --> 00:38:50.294
00:38:50.294 --> 00:38:55.405
00:38:55.405 --> 00:39:00.857
00:39:00.857 --> 00:39:04.590
00:39:04.590 --> 00:39:07.374
00:39:07.374 --> 00:39:11.371
00:39:11.371 --> 00:39:16.050
00:39:16.050 --> 00:39:20.072
00:39:20.072 --> 00:39:24.693
00:39:24.693 --> 00:39:29.356
00:39:29.356 --> 00:39:33.258
00:39:33.258 --> 00:39:36.727
00:39:36.727 --> 00:39:40.252
00:39:40.252 --> 00:39:42.612
00:39:42.612 --> 00:39:45.735
00:39:45.735 --> 00:39:51.611
00:39:51.611 --> 00:39:56.651
00:39:56.651 --> 00:40:00.428
00:40:00.428 --> 00:40:03.856
00:40:03.856 --> 00:40:07.917
00:40:07.917 --> 00:40:11.881
00:40:11.881 --> 00:40:15.607
00:40:15.607 --> 00:40:19.101
00:40:19.101 --> 00:40:23.064
00:40:23.064 --> 00:40:27.135
00:40:27.135 --> 00:40:31.788
00:40:31.788 --> 00:40:36.450
00:40:36.450 --> 00:40:40.591
00:40:40.591 --> 00:40:45.691
00:40:45.691 --> 00:40:49.752
00:40:49.752 --> 00:40:53.954
00:40:53.954 --> 00:40:58.110
00:40:58.110 --> 00:41:03.000
00:41:03.000 --> 00:41:06.210
00:41:06.210 --> 00:41:11.460
00:41:11.460 --> 00:41:14.430
00:41:14.430 --> 00:41:19.740
00:41:19.740 --> 00:41:23.460
00:41:23.460 --> 00:41:29.310
00:41:29.310 --> 00:41:32.400
00:41:32.400 --> 00:41:40.080
00:41:40.080 --> 00:41:44.070
00:41:44.070 --> 00:41:48.300
00:41:49.020 --> 00:41:54.660
00:41:54.660 --> 00:42:01.480
00:42:01.480 --> 00:42:04.990
00:42:04.990 --> 00:42:08.170
00:42:08.170 --> 00:42:12.070
00:42:12.070 --> 00:42:16.060
00:42:16.060 --> 00:42:19.240
00:42:19.240 --> 00:42:25.810
00:42:25.810 --> 00:42:29.020
00:42:29.020 --> 00:42:33.160
00:42:33.160 --> 00:42:39.310
00:42:39.310 --> 00:42:42.040
00:42:42.040 --> 00:42:44.950
00:42:44.950 --> 00:42:48.070
00:42:48.070 --> 00:42:51.880
00:42:51.880 --> 00:42:55.120
00:42:55.120 --> 00:42:58.420
00:42:58.420 --> 00:43:05.530
00:43:05.530 --> 00:43:08.680
00:43:08.680 --> 00:43:13.150
00:43:13.150 --> 00:43:17.980
00:43:17.980 --> 00:43:23.380
00:43:23.380 --> 00:43:26.740
00:43:26.740 --> 00:43:31.570
00:43:31.570 --> 00:43:35.260
00:43:35.260 --> 00:43:39.820
00:43:39.820 --> 00:43:43.420
00:43:43.420 --> 00:43:47.980
00:43:47.980 --> 00:43:52.210
00:43:52.210 --> 00:43:55.090
00:43:55.090 --> 00:43:58.570
00:43:58.570 --> 00:44:01.930
00:44:01.930 --> 00:44:08.480
00:44:08.480 --> 00:44:13.310
00:44:13.310 --> 00:44:17.990
00:44:17.990 --> 00:44:22.460
00:44:22.460 --> 00:44:28.370
00:44:28.370 --> 00:44:31.370
00:44:31.370 --> 00:44:34.700
00:44:34.700 --> 00:44:38.870
00:44:38.870 --> 00:44:42.320
00:44:42.320 --> 00:44:44.480
00:44:44.480 --> 00:44:48.500
00:44:48.500 --> 00:44:53.810
00:44:53.810 --> 00:44:56.360
00:44:56.360 --> 00:44:59.630
00:44:59.630 --> 00:45:03.440
00:45:03.440 --> 00:45:07.940
00:45:07.940 --> 00:45:11.570
00:45:11.570 --> 00:45:15.770
00:45:15.770 --> 00:45:21.800
00:45:21.800 --> 00:45:26.690
00:45:26.690 --> 00:45:29.960
00:45:29.960 --> 00:45:33.770
00:45:33.770 --> 00:45:37.070
00:45:37.070 --> 00:45:41.210
00:45:41.210 --> 00:45:51.530
00:45:51.530 --> 00:45:54.320
00:45:54.320 --> 00:45:59.720
00:45:59.720 --> 00:46:05.150
00:46:05.150 --> 00:46:08.750
00:46:08.750 --> 00:46:13.080
00:46:13.080 --> 00:46:19.020
00:46:19.020 --> 00:46:22.200
00:46:22.200 --> 00:46:26.130
00:46:26.130 --> 00:46:30.780
00:46:30.780 --> 00:46:35.250
00:46:35.250 --> 00:46:38.490
00:46:38.490 --> 00:46:42.990
00:46:44.340 --> 00:46:48.390
00:46:48.390 --> 00:46:51.420
00:46:51.420 --> 00:46:56.460
00:46:56.460 --> 00:47:00.480
00:47:00.480 --> 00:47:05.850
00:47:05.850 --> 00:47:09.000
00:47:09.000 --> 00:47:14.670
00:47:14.670 --> 00:47:18.810
00:47:18.810 --> 00:47:24.090
00:47:24.090 --> 00:47:28.920
00:47:28.920 --> 00:47:33.540
00:47:33.540 --> 00:47:37.470
00:47:37.470 --> 00:47:42.540
00:47:42.540 --> 00:47:48.070
00:47:48.070 --> 00:47:51.232
00:47:51.232 --> 00:47:53.575
00:47:53.575 --> 00:47:59.176
00:47:59.176 --> 00:48:06.097
00:48:06.097 --> 00:48:10.152
00:48:10.152 --> 00:48:14.255
00:48:14.255 --> 00:48:18.012
00:48:18.012 --> 00:48:22.567
00:48:22.567 --> 00:48:27.432
00:48:27.432 --> 00:48:31.606
00:48:31.606 --> 00:48:36.538
00:48:36.538 --> 00:48:42.133
00:48:42.133 --> 00:48:46.668
00:48:46.668 --> 00:48:51.040
00:48:51.040 --> 00:48:55.159
00:48:55.159 --> 00:48:59.743
00:48:59.743 --> 00:49:04.345
00:49:04.345 --> 00:49:08.893
00:49:08.893 --> 00:49:13.706
00:49:13.706 --> 00:49:17.203
00:49:17.203 --> 00:49:20.885
00:49:20.885 --> 00:49:26.271
00:49:26.271 --> 00:49:31.933
00:49:31.933 --> 00:49:36.007
00:49:36.007 --> 00:49:39.835
00:49:39.835 --> 00:49:44.031
00:49:44.031 --> 00:49:49.421
00:49:49.421 --> 00:49:54.225
00:49:54.225 --> 00:49:57.723
00:49:57.723 --> 00:50:01.824
00:50:01.824 --> 00:50:06.636
00:50:06.636 --> 00:50:10.657
00:50:10.657 --> 00:50:15.330
00:50:15.330 --> 00:50:20.768
00:50:20.768 --> 00:50:25.259
00:50:25.259 --> 00:50:29.070
00:50:29.070 --> 00:50:33.399
00:50:33.399 --> 00:50:38.295
00:50:38.295 --> 00:50:43.166
00:50:43.166 --> 00:50:47.637
00:50:47.637 --> 00:50:51.557
00:50:51.557 --> 00:50:56.079
00:50:56.079 --> 00:50:59.108
00:51:00.042 --> 00:51:05.852
00:51:05.852 --> 00:51:09.893
00:51:09.893 --> 00:51:14.282
00:51:14.282 --> 00:51:18.240
00:51:18.240 --> 00:51:22.553
00:51:22.553 --> 00:51:26.820
00:51:26.820 --> 00:51:30.654
00:51:30.654 --> 00:51:36.671
00:51:36.671 --> 00:51:41.510
00:51:41.510 --> 00:51:46.426
00:51:46.426 --> 00:51:49.610
00:51:49.610 --> 00:51:53.647
00:51:53.647 --> 00:51:58.069
00:51:58.069 --> 00:52:09.897
00:52:10.565 --> 00:52:13.514
00:52:13.514 --> 00:52:16.685
00:52:16.685 --> 00:52:20.779
00:52:20.779 --> 00:52:24.503
00:52:24.503 --> 00:52:30.910
00:52:30.910 --> 00:52:34.660
00:52:34.660 --> 00:52:40.990
00:52:40.990 --> 00:52:44.440
00:52:44.440 --> 00:52:47.110
00:52:47.110 --> 00:52:52.030
00:52:52.030 --> 00:52:56.830
00:52:56.830 --> 00:53:01.990
00:53:01.990 --> 00:53:06.370
00:53:06.370 --> 00:53:11.620
00:53:11.620 --> 00:53:15.280
00:53:15.280 --> 00:53:19.182
00:53:19.182 --> 00:53:24.310
00:53:24.310 --> 00:53:28.360
00:53:28.360 --> 00:53:32.980
00:53:32.980 --> 00:53:38.800
00:53:38.800 --> 00:53:44.970
00:53:46.505 --> 00:53:52.170
00:53:52.170 --> 00:53:56.280
00:53:56.280 --> 00:53:59.574
00:53:59.574 --> 00:54:04.830
00:54:04.830 --> 00:54:08.150
00:54:08.150 --> 00:54:11.454
00:54:11.454 --> 00:54:15.160
00:54:15.160 --> 00:54:20.400
00:54:20.400 --> 00:54:24.602
00:54:24.602 --> 00:54:29.619
00:54:29.619 --> 00:54:34.831
00:54:34.831 --> 00:54:38.848
00:54:38.848 --> 00:54:43.162
00:54:43.162 --> 00:54:46.937
00:54:46.937 --> 00:54:50.271
00:54:50.271 --> 00:54:54.463
00:54:54.463 --> 00:54:58.681
00:54:58.681 --> 00:55:02.985
00:55:02.985 --> 00:55:07.257
00:55:07.257 --> 00:55:10.891
00:55:10.891 --> 00:55:17.549
00:55:17.549 --> 00:55:24.453
00:55:24.453 --> 00:55:31.290
00:55:31.290 --> 00:55:37.701
00:55:37.701 --> 00:55:43.140
00:55:43.140 --> 00:55:46.838
00:55:47.361 --> 00:55:51.361
00:55:51.361 --> 00:55:54.725
00:55:54.725 --> 00:55:58.953
00:55:58.953 --> 00:56:04.293
00:56:04.293 --> 00:56:09.630
00:56:09.630 --> 00:56:13.611
00:56:13.611 --> 00:56:17.483
00:56:17.483 --> 00:56:21.814
00:56:21.814 --> 00:56:26.136
00:56:26.136 --> 00:56:31.395
00:56:31.395 --> 00:56:35.428
00:56:35.428 --> 00:56:39.722
00:56:39.722 --> 00:56:44.161
00:56:45.811 --> 00:56:50.310
00:56:50.310 --> 00:56:54.073
00:56:54.073 --> 00:56:57.720
00:56:57.720 --> 00:57:02.270
00:57:02.270 --> 00:57:06.510
00:57:06.510 --> 00:57:10.085
00:57:10.085 --> 00:57:12.822
00:57:12.822 --> 00:57:16.161
00:57:16.161 --> 00:57:19.253
00:57:19.253 --> 00:57:22.932
00:57:22.932 --> 00:57:26.620
00:57:26.620 --> 00:57:30.078
00:57:30.078 --> 00:57:33.922
00:57:33.922 --> 00:57:38.379
00:57:38.379 --> 00:57:43.382
00:57:43.382 --> 00:57:46.965
00:57:46.965 --> 00:57:51.137
00:57:51.137 --> 00:57:55.221
00:57:55.221 --> 00:57:58.887
00:57:58.887 --> 00:58:02.851
00:58:02.851 --> 00:58:06.423
00:58:06.423 --> 00:58:09.748
00:58:09.748 --> 00:58:12.763
00:58:12.763 --> 00:58:17.678
00:58:17.678 --> 00:58:22.130
00:58:22.130 --> 00:58:26.171
00:58:26.171 --> 00:58:29.487
00:58:29.487 --> 00:58:33.347
00:58:33.347 --> 00:58:37.136
00:58:37.136 --> 00:58:41.254
00:58:41.254 --> 00:58:45.464
00:58:45.464 --> 00:58:49.020
00:58:49.020 --> 00:58:53.007
00:58:53.007 --> 00:58:55.981
00:58:57.221 --> 00:59:01.799
00:59:01.799 --> 00:59:04.998
00:59:04.998 --> 00:59:10.336
00:59:10.336 --> 00:59:13.820
00:59:13.820 --> 00:59:17.272
00:59:17.272 --> 00:59:21.804
00:59:21.804 --> 00:59:26.937
00:59:26.937 --> 00:59:31.625
00:59:31.625 --> 00:59:36.515
00:59:36.515 --> 00:59:42.040
00:59:42.040 --> 00:59:48.444
00:59:48.444 --> 00:59:54.311
00:59:54.311 --> 00:59:59.230
00:59:59.230 --> 01:00:01.970
01:00:01.970 --> 01:00:05.158
01:00:05.158 --> 01:00:10.747
01:00:10.747 --> 01:00:15.679
01:00:15.679 --> 01:00:19.085
01:00:19.085 --> 01:00:22.790
01:00:22.790 --> 01:00:25.730
01:00:25.730 --> 01:00:29.390
01:00:29.390 --> 01:00:33.530
01:00:33.530 --> 01:00:37.310
01:00:37.310 --> 01:00:41.630
01:00:41.630 --> 01:00:45.650
01:00:45.650 --> 01:00:49.430
01:00:49.430 --> 01:00:54.380
01:00:54.380 --> 01:00:57.560
01:00:57.560 --> 01:01:02.150
01:01:02.150 --> 01:01:08.030
01:01:08.030 --> 01:01:11.210
01:01:11.210 --> 01:01:16.280
01:01:16.280 --> 01:01:20.000
01:01:20.000 --> 01:01:24.080
01:01:24.080 --> 01:01:27.472
01:01:28.688 --> 01:01:33.530
01:01:33.530 --> 01:01:36.380
01:01:36.380 --> 01:01:39.530
01:01:39.530 --> 01:01:42.800
01:01:42.800 --> 01:01:45.988
01:01:45.988 --> 01:01:48.830
01:01:48.830 --> 01:01:52.070
01:01:52.070 --> 01:01:57.260
01:01:57.260 --> 01:02:01.220
01:02:01.220 --> 01:02:05.180
01:02:05.180 --> 01:02:08.030
01:02:08.030 --> 01:02:11.960
01:02:11.960 --> 01:02:17.270
01:02:17.270 --> 01:02:20.240
01:02:20.240 --> 01:02:23.270
01:02:23.270 --> 01:02:27.500
01:02:27.500 --> 01:02:32.760
01:02:32.760 --> 01:02:37.500
01:02:37.500 --> 01:02:41.910
01:02:41.910 --> 01:02:46.920
01:02:46.920 --> 01:02:52.050
01:02:52.050 --> 01:02:55.980
01:02:55.980 --> 01:02:59.670
01:02:59.670 --> 01:03:03.210
01:03:03.210 --> 01:03:06.106
01:03:06.818 --> 01:03:11.855
01:03:11.855 --> 01:03:40.000