36C3 - Das nützlich-unbedenklich Spektrum

36c3 Intro Music

Herald: ...now with the talk "The useful-
harmless spectrum". As I said,

he needs no introduction: Fefe.

Applause

Tapping on the microphone

Fefe: Good morning, I am happy that there
are so many people here.

Thankfully this is not Hall 1. That would
be bad, with so many people.

I have to manage your expectations
before I start,

I actually submitted a different talk
last year about TCB-minimization,

which would have been a bit technical,
about what you can do

as a programmer. It was not accepted,
I don't know why - schedule was full.

I submitted it again this year,
but I didn't want it to look

like I want to bother them, so I
submitted another talk.

...of course they accepted that one.
Which meant, I had to

quickly prepare it now.
Audience laughs

Well, the problem is, this is more of a
thought process than a structured

presentation. I hope that it'll be
helpful none the less. But it's

not as structured as my usual talks.
I will just start. So, there are multiple

approaches, that basically result in
the same result, and I will

just let you listen. Relatively early in
my career, I decided the following:

I will never write software on which
people's lives may depend,

like medical devices, nuclear reactors;
that was my idea.

Of course not military either. And then
I met somebody that writes code for

nuclear reactors. And it was the kind of
guy that says "That's super easy"

So when those that know their
limits don't do it, then

the other people will.
Audience laughs

I don't want to generalize though.
I also met another guy that

was not like this, but I mean,
this type of person exist.

I believe that the problem here
is that you learn programming

exploratively: It's not like a set path,
on which you walk, but rather you

are just walking around and finding
your limits. But by definition this also

means, that you don't know your limits
yet, because you are looking for them.

This also means that you are always
working at your limit though. When people

write software, then they go
just as far as they believe they

can just barely go. In turn, this also
means that the technology that

is being rolled out out there is mainly
not tried and tested

or well understood, but rather it is the
technology, that the programmer

just barely still understood. This is a
bit of a problem, which is further

amplified by today's modularization and
dependency wave, where

people just pull in modules from elsewhere

and just assume that the writer of that
module must know what they are doing,

though without any foundation in reality.
And it is often not the case. Instead,

they are people like you and me, that
also worked exploratively.

You can also do a little thought
experiment and get to this

conclusion yourself; you could even
observe it happening. Let's assume

that somebody finds a better way to
deal with complexity. For example

modularization, or object-oriented
programming, when this was new.

So then you would hope that we would
improve the software that we

had written before, because we now
have it more under control.

But this does not happen.
Instead, we now write bigger

software and work at our limit
again. I think this is not

a problem of software development or
programming, but generally

a problem of humans. Evolution
made us this way, and we have to

learn to deal with it. Let me illustrate
this: I have a theory,

which I call the gradient-theory. The
thesis is, that humans treat their

environment like a process of optimization
in mathematics. This means you

have a terrain and you are looking for
the highest or lowest point - that is an

optimization problem. And you can't
directly aim for it, because you don't

know the terrain. Instead, you have to
make assumptions, and you can observe

this on yourself. If it's too cold, then
you go to the radiator and

you don't set it to the perfect heat,
you set it to "hot", then you wait

until it's too hot, then you
turn it down again.

So we interact with our environment in
a process of approximation.

And not just with heaters, but also when
driving a car, when we have a map.

We look, "where is the limit? Where do
we have to turn?", and

we ignore the journey to the turn,
even if it is nice.

Many things that we do, also including
our choice of speed, is such a gradient.

We accelerate until we feel unwell,
then we slow down again.

Or when searching for something in
a telephone book or dictionary,

we make an assumption of where
it will be. And when it is

too far, we go back again. The essence
of it is: We make an assumption

about what the terrain looks like.
We have smooth transitions here,

so this technique works well.
This is called gradient descent

by the way, when you try to follow
gravity to find the lowest point.

But it does not work well
in two scenarios:

Firstly, when there is a cliff where I
can't go back once I have walked

over it. It also doesn't go well when
you don't notice that you have gone

too far. Well it is similar to the cliff,
and the second problem is

when you can't roll back
for other reasons.

This happens frequently in software
development, and it turns out, that

this is exactly the kind of problem that
human have. For example,

when we have a trial subscription for two
weeks, people forget to cancel it again,

or drug addiction is a classic, or
gambling addiction. And in software

development or project management
in general this is common:

We have already invested so much that
we can't go back. Security is not

a gradient. It may look like one, but it
isn't. I think this is

a fundamental issue in IT security.
You don't notice when you

have gone too far. You only notice
when you get hacked. And then

you can no longer go back, all the data
is already gone. Complexity is also

not a gradient, similarly to security,
but it feels like one. I think

this is the reason why we deal with
it so badly. It feels

as if we have everything under
control. And when we notice,

that we don't, we can't go back.
By the way, giving out data to

facebook is also such a "pseudo-gradient".

When you notice that you gave away too
much, it is too late.

So the conclusion is:
Complexity is evil. We notice it too

late and we get into it too easily.
So we have to counteract that somehow.

If this is our job, we are externalizing
the costs to our customers,

to our users, and to our future self.

This is why you rarely find older software
developers that are happy.

Audience laughs
So, this was the first train of thought,

that led me in this direction. The second
train of thought: Let me just show you

the GNU manifesto, as a representative.
This is not GNU-bashing,

but you can show this pretty well with
the example of the GNU manifesto.

This is the original announcement of the
GNU project by Richard Stallman. He wrote:

"GNU will be able to run Unix programs, but
will not be identical to Unix. We will make

all improvements that are convenient".
This is a very bad sentence.

What does "convenient" mean? For who?

But this is the approach that a lot of
programmers have:

"Oh we can just add this quickly."
We are lacking a corrective, that

we think in advance "what legacy am I
hanging to my leg right now?"

I think this "convenience" thought when
extending software is our "original sin"

- to get a bit catholic here -
in software development.

Everyone has done it before, and you
just can't correct it after the fact.

So the only way of getting rid of it
is to throw away

the whole software or module and
start over again. But software doesn't die.

Only when dealing with software, I learned
that it is good that people die,

because it is a corrective that is needed.
If a system is supposed to improve,

the old stuff has to be able to die at
some point. And this does not

happen with software. It is a feature
that things don't last forever.

In general, you can observe that when
somebody is extending their software and

they have a choice between "We do
something to solve our specific problem"

or "We do something to solve a more
general problem", people will

always try to solve the
more general problem.

"The more danger, the more honor."
And you can see this across the board.

There are very few exceptions to this. And
I had my "aha-moment" when I opened

'gdb' on a project one day. I took '/tmp'
here, but that project was

some checkout.
In my webserver, I have a '.gdbinit' file.

It's a configuration file for the GNU-
debugger, where you can for example say

"Open this application that I want to

debug with these arguments!"
And in there, I write "Don't use Port 80,

that doesn't work, instead use port
8005" or something, to debug it on

localhost. And one day, gdb started
saying "no, I don't accept this

.gdbinit file because it is in a directory

that you have not specifically allowed."
This was exactly such an attempt to fix

an issue after shipping, after the fact.
gdb noticed: "Our config-file has become

so powerful, that it is a security issue",

and then retroactively nailed down the
whole config. And this broke more

than it needed to - perhaps, I don't
know for sure - but it was very annoying

for me. You can put an auto path in here,
but that is when I noticed it

for the first time. This was a few years

ago. I don't know, when exactly that was.
There was a similar case like this

again: With Vim, the editor, that I like
to use. You can do things like

in a comment in the file that is being
edited, you can put some configuration

settings in the first or last three lines.

It is supposed to be used for "I use
tabstop=4 here", or something.

But the parser for this had
a security bug, which made it

possible to create a file that
executes code, when it is

opened in vim, which was of course
not intended. But it is the same

issue. I think you can generalize this

a bit - though earlier I argued
against generalizations, but

in analysis it is good, in software
it is usually bad. let me illustrate

with an example:
Let's assume that we have a CSV file

with some trouble tickets. Field 4
is the one, that we are interested in.

Let's assume it looks like this. It's CSV.
So, now I would like to have the sum

of the four fields. So first I use
cut, we are in Unix here.

Then the first line has to go,

so I use tail. Now the first line
is gone, now I just have to

calculate the sum. There is an
application for this too: paste. that is

how you do it in Unix. Then I have to
calculate it. There we go! But what if

it doesn't say 1 here, but instead "fred"?
We notice: cut does not have a problem,

tail does not have a problem, paste is
fine, but bc falls on its face.

Even worse, bc is programmable.
There could be the

Ackermann-function here and
your computer would be gone

for an hour, while it is trying to
solve some recursion. And I think it

is useful to introduce a concept here
to say: cut, tail and paste are harmless,

bc is not. This is one of the thoughts
where I thought "okay, you can make

a talk about this".
But this is not enough.

There are different kinds of harmless.
But I think this simple idea

already helps us a bit.
Let's make it into a sentence:

Software is harmless, when unexpected
input don't produce unexpected

behavior or unexpected kinds of output.
For example, an SHA-checksum is always

harmless. Regardless of
what data I put in, the output

has a known format. Or word
count (wc) is also one of those.

Now you could say: "Okay, just use
awk!" And in awk I don't have a problem

when it says "fred" instead of "4"
and the interpreter also does not

interpret any functions.
It looks better, but

is it really harmless?
It turns out, awk is a different kind of

not harmless, because you can write
in the filesystem with it. So I don't have

to worry about the input, but I have to
worry about the code, that I hand to it

on the command line. So that is
another distinction you can make.

This is a big problem in the game
industry by the way:

The game development industry
has started putting interpreters

into their games, to be able to write
their business logic - not the AI,

but small scripts - in a scripting
language. One of the most

popular script-interpreters for this
purpose is Lua. And Lua is primarily

used because it can't do anything,
if you don't specifically allow it.

So It can't open files or sockets.
You can enable this manually though,

and then you have a problem again
of course. But this is a real issue.

Many open-source people don't think
about this, because they think "Well,

I will ship it and the rest is no longer
my issue." But I think,

that we generally have to think
about this, and preferably

before shipping, optimally already while
programming. So, this is

a different kind of harmlessness.
The first kind was "Can bad input

cause bad output?" And now: "Can the
application itself do bad things?"

This is a very modern thought,
because we work a lot more with

sandboxing today. In sandboxing, the goal
is to prevent a program from

accidentally or deliberately doing bad
things. And there are again different

things that a program can do.
bc can eat processing time. awk can

read and write in your filesystem, and
this goes on and on. Let's get back

to the GNU manifesto: GNU awk is a special
version of awk and it can open sockets,

without any need. This means, if we
just use awk and thing "Well, awk can

write in the filesystem, but I mounted
that read-only, so nothing

can happen". But then if GNU awk
is being used, it is suddenly

no longer harmless. Bash
can open sockets too by the way!

I don't know, how many people knew that?
This goes on of course: after awk

came Perl. It's even worse, and
Perl can do eval(), which in my

opinion is the worse evil that you can
have in a programming language.

A bit closer to the end-user you can also
observe this in browsers. Let's look at

Netscape for example:
Several times, Netscape had the choice

between "useful" and "harmless" and always
chose "useful". It started with

the plugins. I don't know, who
of you still remembers the Flash-plugin,

or before that we all had the RealPlayer,
and there was also an Acrobat-plugin -

And all of it was shit, because the
plugins were native code: they could do

everything, that their operating system
allowed. This means that it was very

useful, but also very dangerous.
And it was a conscious choice of

the browsers, to allow this.
The actual goal of this talk is

to give the programmers among you a
bit of awareness that you don't just

add a plugin interface that
can do everything.

The next iteration was:
We'll do everything in JavaScript.

At first it looked better, but this
JavaScript eventually also ran with

enough privileges do do bad things
in the system, or at least in the browser.

It turns out: People now have their
important data in the browser,

because they do online banking. And
that is enough do do a lot of damage.

Then they had to correct it
after the fact. Chrome now imposes

even further limits for security reasons
to break ad blockers. It's always

the same trap that we walk into.
Who of you here use Windows?

In Windows there is a tool by
Mark Russinovich - by now he has

sold it to Microsoft, so it is now an
official Microsoft tool.

And the only functionality of this
tool is to list the different

plugins that are part of the system.
And I took a relatively

clean system here. It's not about
this down here or

the size of the scrollbar, but just
how many tabs there are at the top:

These are all different options for
plugins to integrate into the system,

and nobody has an overview of this
anymore, because people always decided

to go in the wrong direction. I believe
that this is a core problem.

There is a third approach to this:
My daily life in security consists of

going to companies. They show me their
source code and I look for bugs. Then

I tell them, which bugs I found. And
occasionally, there are cases where

I notice that there are a lot of bugs.
Not just those that I find, but they

already have their own database,
a bugtracker, and they already

have a seven-digit number of bugs. Yes,
This happens. And since it is a problem

that we have so many bugs, there
are now counter-strategies by developers

that start saying: "Okay, if this bug is
not important then

I can fix it later." And "later" means
"never" in reality. It just sits there.

Joke that only makes sense in German

In the real world, bug
trackers are often just

massive permanent data disposal sites:
For example, I recently filed a bug report

for Firefox and got the ID 1590000.
This is already a bad sign.

But it is also a good sign, that
the bug tracker is open.

For Microsoft you can't see how
many bugs they have.

This is only meant for illustration.
Mozilla is not especially bad.

Mozilla just has an open tracker,
on which I can show it well.

What I wanted to show you -
I had a look: "What is the first bug

that I filed there?" It still had
a six-digit ID.

That was 2003. If you look at the
history of bug IDs then you notice:

It is growing exponentially.
And it's not like the bugs somehow

go away at some point.
I have noticed two major events,

where bugs are closed:
When a new release is done

and you throw out the old JavaScript
engine and put in a new one.

Then you just close all bugs of the old
engine. It looks as if you have achieved

something. And the second is this one:
I don't know, can you read this in

the back? Mozilla just closed my bug.
It says:

"This bug has been automatically
resolved after a period

of inactivity". Mind you, it was not me
who was inactive. I filed the bug and

nobody at Mozilla took care of it.
So they just automatically closed it,

because the statistics look so bad.
This is a big issue,

not just at Mozilla. As I said, this is
just the example

that I can show, because
in their case it is public. But

this leads to a cascade of action
and reaction. For example,

unimportant bugs are just not fixed
anymore. And then people

add "important" on their bugs,
because they want them to be fixed.

Then they say "Okay, the important
bugs also don't get fixed,

because there are too many of them."
And then people

write "Security" on their bugs, and now
we have a wave of security-bugs.

There they negotiate: "Is this really
a problem?" And then we get excuses

like "It's just a crash."
The point is that there is an unholy

alliance with another trend,
namely that companies see:

We have so many bugs open that
solving the bugs is not the goal anymore.

There are just too many, it is
unrealistic. Instead,

we introduce metrics like "we do
fuzzing". Fuzzing is not

a bad idea, but it is not "finding all
bugs", but just the first step

on a long road. But it gives
out a nice metric.

We have so-and-so many fuzz-
testcases, and now...

Are we now better or worse than
before? It's hard to say.