WEBVTT
00:00:09.900 --> 00:00:20.581
Éireann: Things are blowing up, in
industrial systems, here in Germany, this
00:00:20.581 --> 00:00:26.080
year! I had hoped that these things
wouldn't happen. This kind of future
00:00:26.080 --> 00:00:34.210
wouldn't be one that we are living in. But
unfortunately it is. And I hope that we
00:00:34.210 --> 00:00:39.550
can make that better, partly through the
course of this talk. But more, I think, in
00:00:39.550 --> 00:00:45.230
the future with your help and your work.
So I'm sorry to begin this presentation
00:00:45.230 --> 00:00:52.489
with such a dark thought but: This year's
theme is a new dawn. And it's always
00:00:52.489 --> 00:00:57.360
darkest just before the dawn. So we're
going to go through some of that darkness
00:00:57.360 --> 00:01:03.640
in industrial systems and SCADA-systems to
get to a better place, right? Now with
00:01:03.640 --> 00:01:09.550
that said no hacker really gets to be
where they are without the help of other,
00:01:09.550 --> 00:01:14.650
right? We stand on the shoulders of giants
and part of the key is not stepping on
00:01:14.650 --> 00:01:20.350
their toes, on the way up. So I would like
to say thank you to a bunch of people who
00:01:20.350 --> 00:01:24.720
are here and also some people who aren't
here. Particularly the Oslo hackerspace
00:01:24.720 --> 00:01:27.970
where I hang out. And these people have
taught me a lot of things not just about
00:01:27.970 --> 00:01:33.560
technology but about life and on
"aprendo", which is how Goya signed some
00:01:33.560 --> 00:01:40.090
of his last paintings and sketches - which
basically means "I'm still learning". OK.
00:01:40.090 --> 00:01:46.390
So with that said I hope that you will
enjoy this talk with its darkness and its
00:01:46.390 --> 00:01:49.890
humor all at the same time. I used to be
in circus, as you may have guessed from
00:01:49.890 --> 00:01:56.200
the mustache. So I encourage you not just
to view this as a technical vulnerability
00:01:56.200 --> 00:02:02.180
presentation but also as kind of live
technical standup comedy. Instead of jokes
00:02:02.180 --> 00:02:07.450
we have vulnerabilities. And I hope that
you will enjoy them. So these
00:02:07.450 --> 00:02:12.700
vulnerabilities are in switches. I chose
to focus on switches and that will become
00:02:12.700 --> 00:02:19.489
clear throughout the presentation, why I
chose to do that for industrial systems.
00:02:19.489 --> 00:02:23.580
And we are looking primarily at three
different families of switches. Because I
00:02:23.580 --> 00:02:28.049
don't want to pick on any one vendor. In
fact, the whole idea of this talk is to
00:02:28.049 --> 00:02:31.790
continue giving it. I have two other
colleagues who couldn't be here with me
00:02:31.790 --> 00:02:35.519
today, who have some vulnerabilities in
some other switches. And they look forward
00:02:35.519 --> 00:02:39.549
to presenting those vulnerabilities as
part of this presentation in the future.
00:02:39.549 --> 00:02:43.409
So every time we give this presentation
we'd like to give some new vulnerabilities
00:02:43.409 --> 00:02:49.680
and show that this is systemic and endemic
risk. So the three switches we'll be
00:02:49.680 --> 00:02:54.239
looking at today are the Siemens Scalance-
family, the GE Multilin-family and the
00:02:54.239 --> 00:02:58.819
Garrettcom Magnum family. These switches
are usually not very big. They might be 8
00:02:58.819 --> 00:03:05.709
ports, they might be 24 ports. And they're
used in a variety of different locations.
00:03:05.709 --> 00:03:13.260
So this talk is for you, if you work in a
utility, if you test industrial Ethernet
00:03:13.260 --> 00:03:17.480
switches, if you manage industrial
Ethernet networking, if you're comfortable
00:03:17.480 --> 00:03:21.510
at a Linux commandline and you play with
web apps but you don't know as much about
00:03:21.510 --> 00:03:25.450
reverse engineering. Don't worry, I'm
exactly the same. I suck at reverse
00:03:25.450 --> 00:03:31.519
engineering. But I care about this stuff.
And so I'm learning. If you are a
00:03:31.519 --> 00:03:35.909
developer of firmware then I think this
talk is for you as well. I hope you learn
00:03:35.909 --> 00:03:39.709
something from it. If you like
vulnerabilities you'll enjoy this quite a
00:03:39.709 --> 00:03:45.129
lot. I'm going to be sharing with you a
little collection I have, you know. Some
00:03:45.129 --> 00:03:51.629
people collect stamps or stories or jokes.
I collect private keys. And I like to
00:03:51.629 --> 00:03:58.011
share them with other enthusiasts such as
yourself. If you happen to work for one of
00:03:58.011 --> 00:04:01.090
the switch manufacturers you know I've
spoken to before. Some of you I get on
00:04:01.090 --> 00:04:06.809
with very well. We speak regularly. Some
of you not yet - but I hope you'll come
00:04:06.809 --> 00:04:13.199
and have a chat with me later. Ok, most
SCADA or ICS presentations go a bit like
00:04:13.199 --> 00:04:20.640
this: Pwn PLC, the RTU, the HMI - these
are terms, you know, that all of us in
00:04:20.640 --> 00:04:23.830
SCADA know. Maybe most of you know them by
now, they're pretty popular. I hope you
00:04:23.830 --> 00:04:27.780
do. But programmable logic controller,
remote terminal unit or human machine
00:04:27.780 --> 00:04:32.920
interface. And the basic idea of the
presentation is if I pwn these things,
00:04:32.920 --> 00:04:38.090
game over. Physical damage. I win. Isn't
the world a scary place? And I encourage
00:04:38.090 --> 00:04:42.240
you to demand better content. I certainly
grew up with better content. I used to go
00:04:42.240 --> 00:04:46.229
and see the presentations and the talks of
a guy called Jason Larson. And he has a
00:04:46.229 --> 00:04:50.271
fantastic example of this. I want all of
you to try it, right now. Just think
00:04:50.271 --> 00:04:56.110
about: If you had complete control over a
paint factory. What would you do to damage
00:04:56.110 --> 00:04:59.470
it? No one is going to get hurt.
Everything's safe. It's a thought
00:04:59.470 --> 00:05:05.630
experiment, right? What would you do to
damage it? Most people can't answer this
00:05:05.630 --> 00:05:09.740
question. And on certain types of
processes I can't answer this question.
00:05:09.740 --> 00:05:12.790
But other types I've worked with before
and I can answer this question. And I
00:05:12.790 --> 00:05:17.530
encourage you to to ask it. But if you
like and you want to learn more go and see
00:05:17.530 --> 00:05:24.069
Marmusha's talk - I think it's tomorrow.
Think of my talk as a frame for her talk.
00:05:24.069 --> 00:05:28.111
She's going to be talking about how to
damage a chemical process. And what you
00:05:28.111 --> 00:05:32.090
need to do as an engineer to do that. And
the reason she's doing that is to build a
00:05:32.090 --> 00:05:36.290
better process in the future. You have to
break a few things to make them work a
00:05:36.290 --> 00:05:41.379
little bit better. Okay. So what's the
point in industrial control systems
00:05:41.379 --> 00:05:47.750
security? It's not credit card data. It's
not privacy. No disrespect to my privacy
00:05:47.750 --> 00:05:52.240
friends in the room. I have the deepest
love and respect for the work that you do.
00:05:52.240 --> 00:05:58.310
But confidentially ... confidentiality is
the lowest priority for us in industrial
00:05:58.310 --> 00:06:04.830
systems. It would go: Availability,
integrity, confidentiality. And you might
00:06:04.830 --> 00:06:10.970
even swap integrity and availability in
many cases. So, you have to protect the
00:06:10.970 --> 00:06:16.770
sensor data or the control signals.
Everything else is maybe a vulnerability
00:06:16.770 --> 00:06:20.210
on the path to getting this. But it's not
the most important thing that we're trying
00:06:20.210 --> 00:06:25.600
to protect. So that's why I'm attacking
switches. That's where the process is,
00:06:25.600 --> 00:06:32.639
right? Now these may not be core switches.
They're often a little bit further down in
00:06:32.639 --> 00:06:37.069
the chain. They're field devices, right.
So you might find them in any of these
00:06:37.069 --> 00:06:44.590
locations. And this last example is not
necessarily important be cause oil and gas
00:06:44.590 --> 00:06:49.270
is important - but it's important because
it gives you the general format of all
00:06:49.270 --> 00:06:53.300
industrial systems. You have sensor
network. And sensor data is traveling back
00:06:53.300 --> 00:06:58.610
and forth. And you have control signal
data. That's it, basically. You might have
00:06:58.610 --> 00:07:01.349
different control signals on different
protocols and you might have different
00:07:01.349 --> 00:07:05.410
sensors on different protocols, giving you
different values like pressure or heat or
00:07:05.410 --> 00:07:15.889
whatever. But most processes follow
basically this format. Okay. I don't do
00:07:15.889 --> 00:07:19.509
SCADA 101. There are other people who do
this. I'm trying to do a little bit, to
00:07:19.509 --> 00:07:25.759
set the reference for this talk, but
usually I avoid it. So basically there's
00:07:25.759 --> 00:07:30.819
not much authentication or integrity in
industrial systems protocols. There's not
00:07:30.819 --> 00:07:36.370
much cryptography. You would expect there
to be, maybe. I'm continually surprised
00:07:36.370 --> 00:07:40.319
that I don't find any. And when I do find
it, it's badly implemented and barely
00:07:40.319 --> 00:07:48.229
works. So once you have compromised a
switch or another part of the network you
00:07:48.229 --> 00:07:51.930
can perform man-in-the-middle attacks on
the process. Or you can create malicious
00:07:51.930 --> 00:07:56.400
firmwares on these different switches. And
that's what I'm trying to prevent. I'm
00:07:56.400 --> 00:08:00.029
trying to find some of the different
methods that people can use to produce
00:08:00.029 --> 00:08:09.800
these firmwares - and then get the vendors
to fix them, right. Okay. These are some
00:08:09.800 --> 00:08:14.419
of the protocols. If you are new to this
space, if you want to do some more work in
00:08:14.419 --> 00:08:17.550
this area, but you don't know what to work
on, take a picture of the slide or go and
00:08:17.550 --> 00:08:21.319
find it later. And choose one of these
protocols and go and work on it. We need
00:08:21.319 --> 00:08:24.250
people to go to these different
organizations. Some of them are
00:08:24.250 --> 00:08:27.499
proprietary, some of them are open and
complain that there is not enough
00:08:27.499 --> 00:08:32.250
cryptography going on in this space. And
yes you can use VPNs. But believe me, I
00:08:32.250 --> 00:08:43.720
often don't find them. Okay. These are the
switches, the specific versions of the
00:08:43.720 --> 00:08:46.600
firmware, in case you're here for
vulnerabilities instead of just me
00:08:46.600 --> 00:08:52.020
waffling on about the basics. If you want
to go and look these up, if you're a
00:08:52.020 --> 00:08:57.910
penetration tester working in this space,
you can go and find them all online. And
00:08:57.910 --> 00:09:02.210
you can get a feeling for the kind of
coding practices that go into these
00:09:02.210 --> 00:09:07.440
different devices. Now I've tried to
choose the vulnerabilities that I'm
00:09:07.440 --> 00:09:14.710
presenting very carefully. To take you
gently from web app vulnerabilities into a
00:09:14.710 --> 00:09:19.950
little bit deeper into the firmware. So
the first one we'll be looking at is
00:09:19.950 --> 00:09:24.710
Siemens. And again, I'm not picking on any
particular vendor. In fact I'm very proud
00:09:24.710 --> 00:09:30.310
of Siemens. They're probably here again.
They're here many years. And they fixed
00:09:30.310 --> 00:09:35.200
these vulnerabilities within three months.
And I think that was awesome - especially
00:09:35.200 --> 00:09:41.500
in the space that I work in. The average
patch-time in SCADA and ICS is 18 months.
00:09:41.500 --> 00:09:45.500
So I think Siemens deserves a round of
applause for getting these fixed.
00:09:45.500 --> 00:09:52.870
Applaus
So without further ado let's have some
00:09:52.870 --> 00:09:58.590
fun, right. So MD5, you go to the web page
for this switch. This is the management
00:09:58.590 --> 00:10:03.350
page of a switch, right. And you interact
with this webpage. And you have a look at
00:10:03.350 --> 00:10:12.580
it. And on the client side they do MD5 of
the password. Okay. That's fascinating. I
00:10:12.580 --> 00:10:17.290
don't think that's particularly secure.
But it's done in roughly the same format
00:10:17.290 --> 00:10:20.641
as that Linux command. So I use the Linux
command instead of the JavaScript just to
00:10:20.641 --> 00:10:26.060
make it easier for everyone. You have the
username at the beginning and the password
00:10:26.060 --> 00:10:30.040
is in the middle. And then you have this
nonce that's at the end, a number you use
00:10:30.040 --> 00:10:34.470
once, right. I was surprised to see the
nonce, and it's even called a nonce,
00:10:34.470 --> 00:10:37.140
right. So somebody had done a little bit
of homework on their cryptography. And
00:10:37.140 --> 00:10:41.150
they understood that they wanted to use,
you know, this number used once to prevent
00:10:41.150 --> 00:10:45.340
replay of the hash every time. Okay,
that's some pretty good work.
00:10:45.340 --> 00:10:49.200
Unfortunately this is MD5 and this is
protecting your electric utilities and
00:10:49.200 --> 00:10:56.070
your water and your sewage systems. And
you can brute force this in a few seconds,
00:10:56.070 --> 00:11:00.200
if the passwords are less than eight
characters. and if they're around 15 it
00:11:00.200 --> 00:11:04.460
might take you 20 minutes or something.
You can do this from PCAPs, from network
00:11:04.460 --> 00:11:08.300
traffic captures. And then you have the
cleartext password that you can use
00:11:08.300 --> 00:11:16.420
forever after, with that switch. So, off
to a bad start, in my opinion. So these
00:11:16.420 --> 00:11:22.770
are the nonces that we're looking at. I'm
glad to hear you laughing. It makes me, it
00:11:22.770 --> 00:11:27.421
warms the heart, right. So you can see
that they are incrementing and that they
00:11:27.421 --> 00:11:37.750
are hex. Yeah. What else can you say about
this? The last half is different than the
00:11:37.750 --> 00:11:45.870
first half. Not only is it incrementing,
it is sequential. If you pull them quickly
00:11:45.870 --> 00:11:53.260
enough. For those of you who also do a bit
of reverse engineering you might recognize
00:11:53.260 --> 00:11:58.130
the first half as well. Anybody in the
room see any patterns in the first half of
00:11:58.130 --> 00:12:09.950
the of the nonces? No? Hmm? Very good, IP
address. Mac address would have been a
00:12:09.950 --> 00:12:13.521
good guess as well. I thought it was at
first. And I got very confused when I went
00:12:13.521 --> 00:12:17.340
to look for the IP address. Because I went
to the switch itself. And the switches IP
00:12:17.340 --> 00:12:25.080
address was not this in hex. It's the
clientside address. Which I just couldn't
00:12:25.080 --> 00:12:29.380
believe, right? Like, it seems like it
makes a sort of sense if you're trying to
00:12:29.380 --> 00:12:33.580
keep session IDs in state. And it's like
oh I want a different session for every IP
00:12:33.580 --> 00:12:39.480
address. And then I'll just use time, I
use uptime in hex as the rest of my
00:12:39.480 --> 00:12:45.160
session ID, right? You know, the entire IP
space and time that can't be brute force.
00:12:45.160 --> 00:12:52.250
It has a kind of crazy logic to it, right.
Unfortunately it can be. And you can get
00:12:52.250 --> 00:12:56.730
the uptime from the device using SNMP. And
of course if you don't want to use SNMP
00:12:56.730 --> 00:13:04.470
you can get old-school and use the TCP-
sequence-ID numbers. So, not a lot of
00:13:04.470 --> 00:13:09.550
entropy there, I guess, I would say. And I
think their lawyers agreed when they put
00:13:09.550 --> 00:13:17.640
out the comments on this. All right. Not
only can you perform session hijacking.
00:13:17.640 --> 00:13:21.050
And if you are attacking switches I'd like
to point out that session hijacking is not
00:13:21.050 --> 00:13:25.230
necessarily a great attack in this
environment. Think about it like you would
00:13:25.230 --> 00:13:29.700
at home, right. How often do you log into
your router? In fact even more importantly
00:13:29.700 --> 00:13:33.250
how often do you upgrade the firmware on
your router? Everyone who has upgraded the
00:13:33.250 --> 00:13:37.940
firmware on their router ever raise your
hand. Just for an experiment. Thank
00:13:37.940 --> 00:13:42.140
goodness, right. But wait, keep them up
just for a minute. Everybody who's updated
00:13:42.140 --> 00:13:45.670
it this year, keep your hand up. Everybody
else put them down. Everybody who has
00:13:45.670 --> 00:13:50.420
updated in the last six months ... okay
... So that gives you a sense of how long
00:13:50.420 --> 00:13:55.061
these vulnerabilities can be in play on an
industrial system's environment. If you
00:13:55.061 --> 00:14:01.800
multiply that by about 10, right. Okay, so
you can simply upload a firmware image to
00:14:01.800 --> 00:14:06.140
a Siemens Scalance device with this
version number without authentication. You
00:14:06.140 --> 00:14:15.700
just need to know the URL. Cross-site
request forgery, right. I just say CSRF
00:14:15.700 --> 00:14:20.220
all the time. I don't even remember what
it stands for. So you can upload or you
00:14:20.220 --> 00:14:23.381
can download a logfile. Not that useful
but you get a sense of what's going on on
00:14:23.381 --> 00:14:27.191
the switch. You know what usernames might
be present, whatever. Incidentally all of
00:14:27.191 --> 00:14:32.050
these switches by default or at least this
one only have two usernames, right. So
00:14:32.050 --> 00:14:37.151
it's "admin" and "operator" I think on
this switch. Or maybe it's not. But
00:14:37.151 --> 00:14:42.830
anyway, there's two usernames "admin" and
"manager"? I know I get them mixed up now.
00:14:42.830 --> 00:14:47.130
But the configuration includes password
hashes. I'm actually not even entirely
00:14:47.130 --> 00:14:50.620
convinced they're hashes because when you
increase the length of your password it
00:14:50.620 --> 00:14:55.610
increases. But I'll leave that for future
researchers to examine. You can download
00:14:55.610 --> 00:14:59.241
the firmware image from the device, which
is nice. So you just make a request. You
00:14:59.241 --> 00:15:03.110
just post an HTTP-request to this device.
And it gives you the firmware that it is
00:15:03.110 --> 00:15:07.820
running back. That's not that big a deal,
right. Because you're just viewing data on
00:15:07.820 --> 00:15:14.930
the switch. But you can upload firmware
and configuration to this device. Which is
00:15:14.930 --> 00:15:18.540
an authentication bypass in and of itself.
But it's also interesting because I can
00:15:18.540 --> 00:15:22.430
take a configuration file from one of the
devices that I have at home with a known
00:15:22.430 --> 00:15:27.490
password. I can upload a new configuration
file with a password that I know. I can
00:15:27.490 --> 00:15:31.500
use the device to do whatever I want to
do. And later I can re upload the old
00:15:31.500 --> 00:15:35.560
configuration file that I got from the
device, so no one ever even realizes what's
00:15:35.560 --> 00:15:45.730
been changed, right. So. I think that's a
disappointing state of affairs. And I
00:15:45.730 --> 00:15:49.340
wrote a script to do this. So that you
wouldn't have to when you are doing
00:15:49.340 --> 00:15:53.920
penetration tests of these device. And I
gave you a little ASCII menu because
00:15:53.920 --> 00:15:58.410
sometimes I get bored. Cambridge is a
small town and there's not much to do in
00:15:58.410 --> 00:16:05.640
the evening. So feel free to go and
examine my github-repository where I put
00:16:05.640 --> 00:16:11.910
up some of this stuff. I'm Blackswanburst
on Github, and on Twitter. So like I say,
00:16:11.910 --> 00:16:15.360
Siemens are some of my favorite people. So
I'm going to finish up with them. This is
00:16:15.360 --> 00:16:19.980
old day, if you like all that you have
just seen. But I want you to keep in mind
00:16:19.980 --> 00:16:24.230
that these vulnerabilities will still be
present in the wild for another two or
00:16:24.230 --> 00:16:28.980
three years. And I encourage you to go and
have a look at your systems, if you have
00:16:28.980 --> 00:16:34.170
any of these devices. And check them out.
And upgrade the firmware. I also hope this
00:16:34.170 --> 00:16:38.540
encourages you that if you haven't done
much in industrial systems and SCADA you
00:16:38.540 --> 00:16:42.270
don't have to be intimidated by all of the
engineering and the terminology, and the
00:16:42.270 --> 00:16:47.001
verb beotch(?).. There is plenty for any
of you in this room to do in the
00:16:47.001 --> 00:16:51.700
industrial systems space. You need to
spend a little time speaking to engineers
00:16:51.700 --> 00:16:56.900
and translating your vulnerabilities into
something meaningful for them. But that's
00:16:56.900 --> 00:17:00.250
just a matter of spending more time with
them and getting to know them. And I think
00:17:00.250 --> 00:17:03.740
that's valuable too because they have a
lot of experience. They care very deeply
00:17:03.740 --> 00:17:08.309
about safety. And I've learned quite a lot
of things from engineers. My general point
00:17:08.309 --> 00:17:13.601
here is I'd like you to stop defending
banks and websites and other stuff. We
00:17:13.601 --> 00:17:18.099
need your help in industrial systems, in
the utilities. We could really do with
00:17:18.099 --> 00:17:22.180
living in a safer world rather than one
where you're just protecting other
00:17:22.180 --> 00:17:32.480
people's money. So we're gonna move on to
the GE Multilin line. I worked on a GE
00:17:32.480 --> 00:17:38.830
ML800 but these vulnerabilities affect
seven of the nine switches in this family.
00:17:38.830 --> 00:17:43.410
Seven because one of the other switches is
an unmanaged switch. If you're a hardware
00:17:43.410 --> 00:17:47.880
person maybe you want to go and play
around with those but not so much my thing
00:17:47.880 --> 00:17:51.130
and the other one uses a different
firmware image but seven of the nine
00:17:51.130 --> 00:17:58.020
switches use a similar firmware image GE
offers a worldwide 10 year warranty. So
00:17:58.020 --> 00:18:01.950
let's see if that includes fixing
vulnerabilities. I think it should. What
00:18:01.950 --> 00:18:10.650
do you think. No? Couple noes couple of
yeses, undecided. All right. CCC is
00:18:10.650 --> 00:18:17.851
undecided on something that's novel. Let's
start with some new vulnerabilities. Cross
00:18:17.851 --> 00:18:22.750
site scripting. Reflected, I grant you but
still cross site scripting and I want you
00:18:22.750 --> 00:18:25.530
to pay attention to the details. I'm not
going to go slow for you and ask you to
00:18:25.530 --> 00:18:29.160
think . I know it's morning, I know it's
tough but I am going to ask you to think.
00:18:29.160 --> 00:18:36.970
See flash up there flash.php and the third
one. Yes, it runs flash in your browser.
00:18:36.970 --> 00:18:42.470
So if you know something about Flash come
and have a look at the switch some time. I
00:18:42.470 --> 00:18:47.751
didn't go for active script attacks. There are
so many attacks surface on this device. I
00:18:47.751 --> 00:18:52.460
just I sometimes don't even know how I'm
going to finish looking at all of them. So
00:18:52.460 --> 00:18:55.780
I just work with the web interface to
begin with. So you have this cross site
00:18:55.780 --> 00:19:00.680
scripting times eight and I want you to
notice in the last section there
00:19:00.680 --> 00:19:05.970
arbitrarily supplied URL parameters. I
don't know about you but I think that's
00:19:05.970 --> 00:19:10.180
funny right. You can just make up
parameters to stick your cross site
00:19:10.180 --> 00:19:20.480
scripting in. laughs It's unbelievable
right. Yeah. Anyways what does that look
00:19:20.480 --> 00:19:28.340
like. It looks like that, they have an
error data page. OK maybe I'm using a
00:19:28.340 --> 00:19:33.370
browser that they don't approve or
something but it deserves looking at. And
00:19:33.370 --> 00:19:39.470
you can do quite a lot of things with
javascript on the client side these days.
00:19:39.470 --> 00:19:44.480
Disturbing. Anyways I'm not a big fan of
XSS so I'm going to move on to things that
00:19:44.480 --> 00:19:52.690
I think are worth my time. So if you fetch
the initial web page of this switch before
00:19:52.690 --> 00:20:01.380
you've even logged in you get this config.
So this is pretty authentication. No
00:20:01.380 --> 00:20:06.850
authentication, right. Now keep in mind that
these switches are designed for process
00:20:06.850 --> 00:20:14.610
data, right. It's not carrying traffic to
images of cats. It's supposed to be for
00:20:14.610 --> 00:20:22.630
engineering. So what happens if I add a
nocache parameter and I make it say 500000
00:20:22.630 --> 00:20:30.030
digits long. I should just be able to
crash the web server. Right. Maybe maybe.
00:20:30.030 --> 00:20:41.270
But you would not expect it to reboot the
switch. And it takes a minute or so for
00:20:41.270 --> 00:20:44.800
the switch to reboot which is actually
really impressive comes up pretty quickly.
00:20:44.800 --> 00:20:50.950
But you know obviously you can repeat
this. So I wanted to examine that a lot
00:20:50.950 --> 00:20:56.390
further. I wanted to know more about that
that crash what was rebooting the switch.
00:20:56.390 --> 00:20:59.290
But like I say I'm not a very good reverse
engineer. So you're going to go on a
00:20:59.290 --> 00:21:02.590
little journey with me where I learned a
couple of things about reverse engineering
00:21:02.590 --> 00:21:06.160
and I had to change my approach from
looking at the webapp style loans to
00:21:06.160 --> 00:21:12.470
moving into this other stuff. So why is
why is it DoS even interesting. You'll
00:21:12.470 --> 00:21:18.320
remember that I mentioned Misha's talk. So
the reason I mention her talk, this is it
00:21:18.320 --> 00:21:23.690
right. Denial of Service on a Website. Who
cares it's tearing posters down as xkcd
00:21:23.690 --> 00:21:28.950
once famously explained to us but in the
industrial system's environment it's very
00:21:28.950 --> 00:21:33.980
different. It can be very serious right. A
simplistic example is you have an
00:21:33.980 --> 00:21:38.750
application that has a heartbeat and if
you stop that heartbeat it might go into
00:21:38.750 --> 00:21:44.060
some sort of safety state it might for
example scram a reactor. There is a famous
00:21:44.060 --> 00:21:50.851
denial of service on PLCs that did scram a
reactor in real life. Does anybody know
00:21:50.851 --> 00:21:58.650
what H2S is? Any oil and gas engineers in
the room? Okay so H2S alerts not reaching
00:21:58.650 --> 00:22:02.861
their destinations is pretty serious
business right. For those of you who are
00:22:02.861 --> 00:22:07.850
not aware of H2S it's a byproduct of
producing oil and gas and inhaled in very
00:22:07.850 --> 00:22:12.850
very small amounts you can go unconscious
and in sort of larger amounts. Respiratory
00:22:12.850 --> 00:22:18.480
failure. So if you take CA safety
seriously if you ever work on these rigs
00:22:18.480 --> 00:22:23.140
in these environments you learn to care
about the wind sock. Right one of these
00:22:23.140 --> 00:22:26.620
alerts goes out. An alarm goes off. There
are many different alarms you have to
00:22:26.620 --> 00:22:31.200
memorize how they all sound on a rig and
then react to them and when you hear the
00:22:31.200 --> 00:22:35.330
H2S alert you look up at the wind sock to
keep an eye on where the wind is and
00:22:35.330 --> 00:22:40.420
trying to avoid being downwind of wherever
the leak is. So a simple denial of service
00:22:40.420 --> 00:22:43.510
that we would not care about in a web
application environment in this
00:22:43.510 --> 00:22:47.940
environment can be very serious. I'm not
saying it always is. It just can be
00:22:47.940 --> 00:22:53.350
right. So denial of service goes up in our
list of problems especially when we're
00:22:53.350 --> 00:22:58.270
looking at networking devices. Okay so
that's that's it for the denial of
00:22:58.270 --> 00:23:01.550
service. But like I say we're going to
look at some other stuff. In fact the
00:23:01.550 --> 00:23:07.320
story with the switch began with a
concerned citizen about three or four
00:23:07.320 --> 00:23:12.280
years ago I found 10000 industrial systems
on the Internet as part of my master's
00:23:12.280 --> 00:23:17.990
thesis and I was pretty uncomfortable with
that. So I sent that data to various
00:23:17.990 --> 00:23:23.889
computer emergency response teams around
the world. I believe it was 52 of them
00:23:23.889 --> 00:23:26.860
right. Not all of them were critical
infrastructure. A lot of them were small
00:23:26.860 --> 00:23:31.370
stuff but maybe 1 in 100. I was told or in
one particular country when they got back
00:23:31.370 --> 00:23:38.400
to me one in 20 were considered critical
infrastructure. And after that you have a
00:23:38.400 --> 00:23:42.540
sort of reputation among the computer
emergency response teams of the world. So
00:23:42.540 --> 00:23:47.580
people send you stuff you get anonymous
e-mails from someone called Concerned
00:23:47.580 --> 00:23:53.330
Citizen. Thank you very much. They sent me
a firmware upgrade pcap of this particular
00:23:53.330 --> 00:23:57.350
device. I suspect that they worked at one
of the utilities and they wanted me to see
00:23:57.350 --> 00:24:05.559
how upgrading the firmware of this GE switch
was performed. So it all began with a pcap.
00:24:05.559 --> 00:24:11.290
So I ran TCP trace to carve out all the
files and see what was going on and you
00:24:11.290 --> 00:24:16.590
could see instantly that there was an FTP
session later looking at the switch I see
00:24:16.590 --> 00:24:21.120
that you can also upgrade them over TFTP
so the management of the switch happens in
00:24:21.120 --> 00:24:26.841
HTTPs and is encrypted but the firmware
upload goes across FTP right so you can
00:24:26.841 --> 00:24:33.700
just carve the file out a little bit of
network forensics I guess. So instantly I
00:24:33.700 --> 00:24:36.950
could see that this one is complete and
the ports on the end of the numbers give
00:24:36.950 --> 00:24:40.660
me a clue of what's going on in the larger
stream. This one seems interesting. Let's
00:24:40.660 --> 00:24:48.240
have a look at it. So. I tried running
file and binwalk I don't know about you
00:24:48.240 --> 00:24:52.860
but I believe that hacking is a journey of
understanding and facts hacking is
00:24:52.860 --> 00:24:57.740
understanding a system better than it
understands itself and nudging it to do
00:24:57.740 --> 00:25:03.950
what you want right. And I also feel that
I should understand my tools. I don't
00:25:03.950 --> 00:25:07.420
really understand my tools until I know
where they're going to fail me or they
00:25:07.420 --> 00:25:11.040
have failed me in the past and in this
particular case I think binwalk is a
00:25:11.040 --> 00:25:15.150
fantastic tool and file is a fantastic
tool. But they didn't tell me anything and
00:25:15.150 --> 00:25:18.750
that was that was a journey of discovery
for me. So that was nice. It was like OK
00:25:18.750 --> 00:25:21.700
binwalk doesn't always give me everything.
I think I was running an older version and
00:25:21.700 --> 00:25:25.179
I think it would handle it now. But the
point is after been walked didn't give me
00:25:25.179 --> 00:25:29.950
anything just resort to the old school
stuff right. Go strings and I found these
00:25:29.950 --> 00:25:34.050
deflate and inflate copywrite strings and
I could tell that a certain portion of the
00:25:34.050 --> 00:25:43.670
file was compressed. This is just from the
pcap. Remember this whole story. So I
00:25:43.670 --> 00:25:49.040
tried to deflate the whole thing. That
didn't work again. I just did something
00:25:49.040 --> 00:25:54.561
simple get a python script that checks
every byte to see which parts of the file
00:25:54.561 --> 00:26:00.831
don't produce ZLIB errors when you try and
decompress them and you figure out what
00:26:00.831 --> 00:26:09.170
sectors of this file are compressed. So
you go to your friend dd and you carve out
00:26:09.170 --> 00:26:15.760
this section of the file right. So we have
this larger firmware image with this
00:26:15.760 --> 00:26:21.310
little compressed section and we have now
cut this little compressed section out. I
00:26:21.310 --> 00:26:24.430
suppose I could have loaded this up into
python and use ZLIB to decompress it. But
00:26:24.430 --> 00:26:27.559
at the time I was still trying to use
command line tools and someone said I'll
00:26:27.559 --> 00:26:35.350
just concatenate the gzip bytes on it.
Gzip inherits from inflate and deflate. So
00:26:35.350 --> 00:26:39.100
if you just concatenate the bytes it
should still handle it. So I did that and
00:26:39.100 --> 00:26:43.920
I got a decompressed binary. When you ran
strings on that it started to make a lot
00:26:43.920 --> 00:26:48.750
more sense and you could find the opcodes
in it where previously it didn't make any
00:26:48.750 --> 00:26:53.910
sense at all. So once you've got an image
like that what do you do. Well if you're
00:26:53.910 --> 00:26:58.250
me you just grep for bugs. I think I
learned that from Ilija. If he's here in
00:26:58.250 --> 00:27:05.590
the room thank you. Thank you very much. I
asked him like a year or two ago. How do
00:27:05.590 --> 00:27:10.761
you how do you find so many bugs. And he
said: "Oh, I just, you know, I grep for
00:27:10.761 --> 00:27:16.510
them, I use find." laughs And so I
started thinking about firmware images.
00:27:16.510 --> 00:27:19.640
Like if I was going to grep for a bug in a
firmware image what would it be. And my
00:27:19.640 --> 00:27:23.840
answer is hardcoded credentials and
default keys because you find them every
00:27:23.840 --> 00:27:29.309
single time so I have this command aliased
on my machine and I just grep for it and I
00:27:29.309 --> 00:27:35.270
find private keys and this is how you too
can end up with a private key collection.
00:27:35.270 --> 00:27:40.465
So, there you go.
00:27:40.465 --> 00:27:50.240
Applause
00:27:50.240 --> 00:27:53.770
Yeah they're hardcoded keys,
but what are they for. It doesn't
00:27:53.770 --> 00:27:57.820
stop there. You know you've got the keys,
but what do they do, right? That was the
00:27:57.820 --> 00:28:02.500
next step of the journey for me. Two of
them you can see one sencrypted with a
00:28:02.500 --> 00:28:05.740
password; we'll come back to that one
later. Let's start with the one on the
00:28:05.740 --> 00:28:15.860
left. If you load this key up into
wireshark. and you use it to decrypt the
00:28:15.860 --> 00:28:22.760
SSL you have a self decrypting pcap.
Remember at the beginning it was using
00:28:22.760 --> 00:28:29.590
HTTPS to manage the device and upload this
firmware image. So if you happen to have
00:28:29.590 --> 00:28:37.210
this firmware image you can decrypt all
the traffic. No forward secrecy, right?
00:28:37.210 --> 00:28:41.550
Now you don't have to be lucky and have
concerned citizens send you an email. You
00:28:41.550 --> 00:28:46.490
can download this image from the GE website
and you can carve the keys out of the
00:28:46.490 --> 00:28:50.100
image in the same way that I did and
decrypt the SSL traffic of any pcap that
00:28:50.100 --> 00:29:01.880
is sent to you. Now the passwords
underneath that are in clear text. You can
00:29:01.880 --> 00:29:08.040
see them highlighted down here. Password
Manager and user manager. You can see them
00:29:08.040 --> 00:29:12.750
up there as well and you can see that
we've decrypted the SSL with that key. So
00:29:12.750 --> 00:29:16.559
default keys, right? Is it a big deal? I
believe the vendors in this case say you
00:29:16.559 --> 00:29:21.190
can upload your own key to the device. For
those of you who aren't used to working in
00:29:21.190 --> 00:29:24.290
embedded it sometimes is difficult to
generate a key on the device because you
00:29:24.290 --> 00:29:27.840
don't have enough memory or you don't have
enough entropy or you don't have enough
00:29:27.840 --> 00:29:32.270
processing power. That's the usual
excuses. And they're true I shouldn't say
00:29:32.270 --> 00:29:36.090
excuses those those things are true. But
you could of course generate it on the
00:29:36.090 --> 00:29:39.850
client side and upload it to the device
and that's what they allow you to do with
00:29:39.850 --> 00:29:44.790
this switch which is great but where is
your encrypted channel in which to upload
00:29:44.790 --> 00:29:52.801
this key? laughs So you can use the serial
device and make sure visually that there's no man
00:29:52.801 --> 00:29:55.340
in the middle. But if you're doing this
remotely – and I'd like you to keep in
00:29:55.340 --> 00:29:59.460
mind that most substations are remote –
if anyone here works in a utility are you
00:29:59.460 --> 00:30:03.530
going to drive to every substation, plug
in a serial cable to change the keys on
00:30:03.530 --> 00:30:07.850
all these devices? It's the sort of thing
you need to know in advance right? So the
00:30:07.850 --> 00:30:12.100
problem with key management, particularly
with SSL and the industrial systems
00:30:12.100 --> 00:30:19.049
environment, is that you have to manage
the keys. And these particular keys, well
00:30:19.049 --> 00:30:23.670
the certificates are self signed so you
can't revoke them. And besides industrial
00:30:23.670 --> 00:30:27.189
systems are never connected to the
Internet. So it wouldn't have made any
00:30:27.189 --> 00:30:32.299
difference. So these are the kind of
problems we're dealing with in this space.
00:30:32.299 --> 00:30:35.271
And that's why I'm trying to encourage
you. Whether you do crypto or privacy or
00:30:35.271 --> 00:30:37.640
whatever spend a little time in the
embedded space, just for bit: there's
00:30:37.640 --> 00:30:46.130
plenty of easy work. OK. So what about the
second key. It requires a password. I
00:30:46.130 --> 00:30:50.990
didn't feel like brute forcing it. Maybe
you do. I don't know. I tried all the
00:30:50.990 --> 00:30:54.340
strings in the image. A classic technique,
just in case someone had a hard coded the
00:30:54.340 --> 00:30:56.580
password. I mean the hard coded
credentials were there but not the hard
00:30:56.580 --> 00:31:00.460
coded password. So I guess I gotta start
reversing, and as I previously said I suck
00:31:00.460 --> 00:31:06.380
at reversing. That's why I come to CCC, so
I can learn, right? But I did find this
00:31:06.380 --> 00:31:11.970
PowerPC ROM image. and I think its running
eCos and redboot and I haven't even gotten
00:31:11.970 --> 00:31:15.330
down to doing hardware stuff: taking it
apart, having look at, it but I probably
00:31:15.330 --> 00:31:19.200
will in the future. So there's the image
I'm slowly starting to learn my way around
00:31:19.200 --> 00:31:27.140
and figure out what's going on. So I had a
look at the image and I figured out that
00:31:27.140 --> 00:31:32.100
this key is used for SSH, right? Well it
would be the other encrypted thing. But I
00:31:32.100 --> 00:31:36.261
couldn't enable SSH on the device. I try
and enable SSH on the device and I'm
00:31:36.261 --> 00:31:39.100
logged in as manager by the way. which is
highest level user on this particular
00:31:39.100 --> 00:31:43.580
device, and I put it in the passwords that
I know and a bunch of other passwords and
00:31:43.580 --> 00:31:47.590
they don't work. Like I said, I tried all
the strings in the image. So apparently to
00:31:47.590 --> 00:31:51.720
enable ssh, I need a password for
something. Now maybe I'm just
00:31:51.720 --> 00:31:55.530
misunderstanding or I'm not so clear on
what's going on but I don't know about
00:31:55.530 --> 00:31:59.070
you. I kind of feel like if I buy a device
that's supposed to be used for a safety
00:31:59.070 --> 00:32:03.440
critical process I should be allowed to
use SSH without having to call up the
00:32:03.440 --> 00:32:11.120
vendor and get some special magic
password. So considering I don't like that
00:32:11.120 --> 00:32:17.420
approach. What if I patched my own key
into the image right. I don't know the
00:32:17.420 --> 00:32:22.201
password of their key but I know the
password of a key I can generate. So I
00:32:22.201 --> 00:32:27.290
just need to make sure it's roughly the
right size and try and patch it in. Then
00:32:27.290 --> 00:32:29.600
I've got some problems with compression
because I've got to reverse the whole
00:32:29.600 --> 00:32:33.570
process that I just described to you patch
it into the larger binary. Will there be
00:32:33.570 --> 00:32:44.200
any CRC or firmware signing? I don't know,
right. So the uploaded image is not a
00:32:44.200 --> 00:32:50.530
valid image for this device. That's
correct: I messed with it. But I got this
00:32:50.530 --> 00:32:54.440
error and it gave me a clue. It gave me a
clue that I did indeed have some of my
00:32:54.440 --> 00:33:02.410
CRCs wrong so when I altered the image
again I got to this state. So you're
00:33:02.410 --> 00:33:05.510
learning all the time by having a real
device. Now some of my friends they do
00:33:05.510 --> 00:33:10.051
static analysis and they don't buy these
devices. I decided to buy this one. I
00:33:10.051 --> 00:33:15.750
found one on eBay. It wasn't very
expensive. I mean it depends on your range
00:33:15.750 --> 00:33:20.179
for expensive. But if you're helping
defend industrial systems I thought it was
00:33:20.179 --> 00:33:26.880
worth the money. So I bought it and this
enables me to try firmware images out and
00:33:26.880 --> 00:33:31.210
I can slowly start to figure out what I
need to patch on these firmware images to
00:33:31.210 --> 00:33:37.321
do whatever I want. Luckily I just tried
to patch mine to have SSH because I
00:33:37.321 --> 00:33:43.799
thought people deserve to have SSH. So
that's an Adler 32 up there on the left
00:33:43.799 --> 00:33:50.270
and the other CRC is on the bottom so that
Adler 32 and some adjustment of file
00:33:50.270 --> 00:33:54.420
length although zeros in that line just
above it eventually got me to the point
00:33:54.420 --> 00:33:59.570
where it believes it's a corrupted binary.
And then we have this CRC on the end that
00:33:59.570 --> 00:34:08.210
we need to have a look at. Now I'm a big
fan of suspense. I love suspense. I'm
00:34:08.210 --> 00:34:14.849
going to leave that one is a cliffhanger
and an exercise for you watching. So I
00:34:14.849 --> 00:34:18.099
said I was going to talk about GE ML800
but I'm also going to talk about
00:34:18.099 --> 00:34:21.219
Garrettcom. Luckily it's not very
difficult. Garrettcom is the original
00:34:21.219 --> 00:34:27.480
equipment manufacturer for the GE ML800
series. I noticed that because the
00:34:27.480 --> 00:34:31.299
certificate I found attached to those
private keys said Garrettcom in it and I
00:34:31.299 --> 00:34:35.789
went and looked at their firmware images
and they have similar CRC similar file
00:34:35.789 --> 00:34:39.710
structures similar everything so I believe
that they are affected by the cross site
00:34:39.710 --> 00:34:45.929
scripting, the denial of service, and
hardcoded keys. I understand from some
00:34:45.929 --> 00:34:50.530
people that they have been in contact with
GE to try and fix some of this stuff but
00:34:50.530 --> 00:34:57.960
their response to GE was mainly "Sorry,
this is the end of life on this device".
00:34:57.960 --> 00:35:02.890
That's fine. I understand you're running a
business but you're selling equipment to
00:35:02.890 --> 00:35:08.339
people who manage utilities that we all
depend on. If Sony goes bankrupt because
00:35:08.339 --> 00:35:13.799
they get hacked that's one thing right.
But you can't just dissolve a utility and
00:35:13.799 --> 00:35:18.670
start again. As my friend Klaus points out
regularly – fantastic insights into the
00:35:18.670 --> 00:35:23.150
industrial system world, Klaus and Vanessa
– you can't just dissolve the utility and
00:35:23.150 --> 00:35:25.970
start again. You still have the same
infrastructure you still have the same
00:35:25.970 --> 00:35:31.249
workers. It doesn't work that way. You
can't bail out utilities that we depend
00:35:31.249 --> 00:35:38.329
on. So sorry. End of Life... I don't even
understand why people buy these devices
00:35:38.329 --> 00:35:43.130
and this code without code escrow. When
you buy the code make sure you have the
00:35:43.130 --> 00:35:48.700
code in perpetuity for these systems so
that you can fix them when something like
00:35:48.700 --> 00:35:53.860
this or something worse happens. If I'm
your worst nightmare, you have real
00:35:53.860 --> 00:35:59.190
problems because there are very dark
people in the world actually damaging
00:35:59.190 --> 00:36:05.460
furnaces in Germany. So me disclosing keys
on stage is scary for you. You need to get
00:36:05.460 --> 00:36:12.689
a grip. So, garrettcom?
Here's your key too.
00:36:12.689 --> 00:36:20.104
Applause
00:36:20.104 --> 00:36:25.629
The strings come from the images.
Developers are funny people really. I like
00:36:25.629 --> 00:36:32.110
this. I just put them up because they're
funny. Some people had some hard times, I
00:36:32.110 --> 00:36:36.490
guess, writing some of this code. And my
respect to them! They do great work but
00:36:36.490 --> 00:36:43.159
you know, there's a couple of things we
can improve on security in these devices.
00:36:43.159 --> 00:36:47.840
So I once had the opportunity to stand in
front of six different vendors at the same
00:36:47.840 --> 00:36:53.440
time their computer emergency response
teams at a conference and I said to them,
00:36:53.440 --> 00:36:59.659
"Will any of you commit to an average
patch time for vulnerabilities of three
00:36:59.659 --> 00:37:05.350
months?" An average patch time, because it
might take 8 months, as it so far has
00:37:05.350 --> 00:37:10.130
taken in the case of GE and Garrettcom, to
work on these issues. It might take a long
00:37:10.130 --> 00:37:15.050
time in some cases but as an average patch
time I think 3 months for things that we
00:37:15.050 --> 00:37:20.440
all depend on is reasonable. So I asked
these six different teams in the same
00:37:20.440 --> 00:37:29.410
room. If any of them would commit to this
and I heard silence for 30 seconds. So my
00:37:29.410 --> 00:37:35.220
friend decided to call this the silence of
the vendors right. And I think that's that
00:37:35.220 --> 00:37:42.029
sums it up. I'd like to see better patch
times. I'd like to see a computer
00:37:42.029 --> 00:37:45.200
emergency response teams in each of these
vendors and I'd like to see someone
00:37:45.200 --> 00:37:53.600
responsible for security in each of these
different utilities. I can dream, right? I
00:37:53.600 --> 00:37:57.369
think that key management... the current
practice industrial systems is to take
00:37:57.369 --> 00:38:02.679
some insecure protocol and wrap it in SSL
or TLS which is why we need the help of
00:38:02.679 --> 00:38:10.180
you privacy people because TLS and SSL
are not the be all and end all. They often
00:38:10.180 --> 00:38:16.430
sort of go the wrong way, right. For
example you can use TLS to do integrity
00:38:16.430 --> 00:38:20.679
without encryption so you can verify that
every message has reached its destination
00:38:20.679 --> 00:38:25.920
intact but it is not encrypted. And this
means that you can still do intrusion
00:38:25.920 --> 00:38:32.530
detection analysis of the packets. That's
really good. But nobody uses that in SSL
00:38:32.530 --> 00:38:36.669
in other ways right. I'm a big fan of
Shodan and use Shodan for a variety of
00:38:36.669 --> 00:38:41.450
different things usually to get a sense of
the Internet as a whole, right? Let me
00:38:41.450 --> 00:38:44.729
back up a little bit. When I was at
Cambridge I went to Darwin college and
00:38:44.729 --> 00:38:47.690
because you're at Darwin college you read
up a bit on Darwin and you think about how
00:38:47.690 --> 00:38:51.870
Darwin thought and I think the Internet is
kind of like that. When it was built by
00:38:51.870 --> 00:38:56.870
the IETF and various people, who did
fantastic work, they imagined it one way
00:38:56.870 --> 00:39:01.450
and then we inherited it and it grew and
it became an ecosystem and stuff happens
00:39:01.450 --> 00:39:05.429
out there that you wouldn't expect. And so
that's why I like Shodan. It's kind of
00:39:05.429 --> 00:39:09.869
like being a natural scientist: what's a
survey of the world, what kind of machines
00:39:09.869 --> 00:39:13.429
are out there, what versions are they
running, when do people update their SSL..
00:39:13.429 --> 00:39:17.559
err, you know, their certificates do they
do it before or after the certificate is
00:39:17.559 --> 00:39:22.600
invalid. Do they always upgrade the
algorithm. Do they increase the key size.
00:39:22.600 --> 00:39:26.380
You know how do things change right you
need to sort of study it as a whole and
00:39:26.380 --> 00:39:30.440
that's my point when it comes to just
taking SSL and slapping it over a
00:39:30.440 --> 00:39:37.759
protocol. It's not quite that simple. So
again we need your help. Where can we go
00:39:37.759 --> 00:39:42.289
with these attacks. And you remember at
the beginning I pointed out the underpants
00:39:42.289 --> 00:39:49.770
gnome. The emperor wears no clothes.
Altering switch configurations is a big
00:39:49.770 --> 00:39:57.410
deal because you can exfiltrate process
data. That gives you a map of the process
00:39:57.410 --> 00:40:02.500
because industrial systems are bespoke.
Each one of them is different. It does run
00:40:02.500 --> 00:40:06.539
different traffic and we are lucky to work
on security in this space because our
00:40:06.539 --> 00:40:10.880
users are numerate and literate and they
care about safety. They don't always
00:40:10.880 --> 00:40:14.239
understand security but they do care about
safety. So if you can make it a safety
00:40:14.239 --> 00:40:18.229
concern they care. There are also
engineers that many of these utilities who
00:40:18.229 --> 00:40:24.219
look at the network 24/7. Not all of them
but some of them. Can you imagine a home
00:40:24.219 --> 00:40:28.899
network or something else with that kind
of user base. We're lucky we should be
00:40:28.899 --> 00:40:35.030
taking advantage of that user base. So
getting back to the point you know denial
00:40:35.030 --> 00:40:38.979
of service attacks to disrupt the process
go and see Marmusha's talk. This will all
00:40:38.979 --> 00:40:43.039
make a lot more sense when you go and see
her talk. Basically any man in the middle
00:40:43.039 --> 00:40:47.990
attack can disrupt alter or drop traffic
at this point. If you can affect the
00:40:47.990 --> 00:40:51.740
switches and the substation. And
exfiltrating in the data gives you a map
00:40:51.740 --> 00:40:58.109
of the process which leads towards further
potential damage for the utilities. Now
00:40:58.109 --> 00:41:01.410
it's not always that simple people will
get up on stage and they will tell you I
00:41:01.410 --> 00:41:07.309
am awesome and this is how it's done and
it's easy to blow shit up. It's not true.
00:41:07.309 --> 00:41:10.249
It takes a little bit of thought it takes
a little bit of work. I am certainly not
00:41:10.249 --> 00:41:15.560
awesome. I am just a quality assurance
person from a former vendor. I just
00:41:15.560 --> 00:41:22.509
decided to get into security and keep
going with it. So you can't always perform
00:41:22.509 --> 00:41:25.389
these man in the middle attacks. People
will say you can. But the reason you can't
00:41:25.389 --> 00:41:30.800
is real-time system constraints. Some
systems will stop receiving traffic five
00:41:30.800 --> 00:41:34.539
milliseconds or microseconds later and
ignore anything. If a value doesn't arrive
00:41:34.539 --> 00:41:39.209
in this time it doesn't care. So the idea
that you can route the traffic out to some
00:41:39.209 --> 00:41:43.590
other country and then back in and disrupt
the process is bollocks. Sometimes you
00:41:43.590 --> 00:41:48.029
have to alter the firmware to achieve
that. That depends on the process but I'm
00:41:48.029 --> 00:41:52.830
just trying to give you a sense of how
performing actual attacks give you a sense
00:41:52.830 --> 00:41:56.120
of what the limits are, what the
logistical burdens are for the attacker
00:41:56.120 --> 00:42:04.940
and that's important stuff for us to know.
All right. Little bit of an overview.
00:42:04.940 --> 00:42:11.810
Drunk session IDs. brute forcing
MD5+NONCE, cross site request forgery for
00:42:11.810 --> 00:42:17.419
firmware upload (of all things),
reflected cross-site scripting (8 cases of
00:42:17.419 --> 00:42:23.050
it) pre authentication denial of service,
hardcoded keys times 2 in a firmware
00:42:23.050 --> 00:42:28.730
image, SSL without forward secrecy, self
signed certificates so there's no revoking
00:42:28.730 --> 00:42:32.280
there's no managing of the keys on these
devices right. Not to mention utility
00:42:32.280 --> 00:42:35.989
workers are busy already. They may not
have time to manage all of these devices
00:42:35.989 --> 00:42:40.250
we might need to rethink that approach
right. Clear text passwords under SSL
00:42:40.250 --> 00:42:44.049
because well no one can break SSL unless
you hard code the key in the firmware
00:42:44.049 --> 00:42:49.539
that's downloadable from the internet.
Enable ssh with a password and three
00:42:49.539 --> 00:42:55.289
quarter of a year waiting for fixes for
some of this stuff. I'm not happy with
00:42:55.289 --> 00:43:00.699
that. I think that we could live in a much
better, much safer world. And to do so we
00:43:00.699 --> 00:43:07.909
need to talk very seriously about some of
these issues. Don't take my opinion for
00:43:07.909 --> 00:43:11.700
it. Listen to some other people. The best
thing about doing industrial systems work
00:43:11.700 --> 00:43:15.480
is the diversity of approach. You know I
love that there are so many other people
00:43:15.480 --> 00:43:20.080
doing SCADA and ICS. And I love that
they're going different directions. So in
00:43:20.080 --> 00:43:26.030
the future I plan to be on another stage
with some friends and show you some more.
00:43:26.030 --> 00:43:30.449
Thank you for listening mustache fans and
as a parting thought. More tax money is
00:43:30.449 --> 00:43:35.349
spent on surveillance than on
defending common utilities.
00:43:35.349 --> 00:43:44.394
Applaus
00:43:44.394 --> 00:43:51.160
Herald: Thank you. It made me a scary
Sunday morning. They got a utility *<<
00:43:51.160 --> 00:43:58.119
guess, mostly incomprehensable* down the
road. OK. We'll have some questions taken
00:43:58.119 --> 00:44:06.459
please. As the session is recorded and
streamed anything you say, say it into a
00:44:06.459 --> 00:44:17.049
mic. Any questions up? Wow, it is Sunday
morning.
00:44:17.049 --> 00:44:18.029
Éireann: Number three, sure
00:44:18.029 --> 00:44:21.280
Herald: everybody understood everything?
You're kidding me.
00:44:21.280 --> 00:44:23.569
Éireann: I've got one right here
Herald: here is a question.
00:44:23.569 --> 00:44:30.089
Question: Hey thanks I enjoyed your talk
and I think it's very important to raise
00:44:30.089 --> 00:44:37.660
awareness. But I think it's not to raise
awareness. Not much in this community, but
00:44:37.660 --> 00:44:43.880
within the engineering community and I see
it a lot of times and many engineers
00:44:43.880 --> 00:44:49.730
having lots of problems doing that for
several reasons. There is maybe the
00:44:49.730 --> 00:44:55.239
engineer who is thinking about this but
has its miniatures in the back has to deal
00:44:55.239 --> 00:45:03.069
with service personnel which know how to
work a hammer and a screwdriver and on the
00:45:03.069 --> 00:45:11.450
other side, engineers have to work with
customers which more those lazy people.
00:45:11.450 --> 00:45:16.309
And so that's how these things happen. And
I think it's more important to raise
00:45:16.309 --> 00:45:22.000
awareness of these kinds of things in the
engineering community.
00:45:22.000 --> 00:45:24.730
Éireann: So just to repeat a little bit
for anybody else that couldn't hear it or
00:45:24.730 --> 00:45:29.170
for the recording it's very important to
work with the engineers some of the
00:45:29.170 --> 00:45:32.469
engineers understand the problem. But
typically management or lower level
00:45:32.469 --> 00:45:37.680
service personnel don't always understand
the problem. And it's not important to
00:45:37.680 --> 00:45:41.690
raise the awareness in the hacker
community. But more with the engineers is
00:45:41.690 --> 00:45:46.299
what you were saying. Right. OK.
Absolutely true. Completely agree with
00:45:46.299 --> 00:45:50.920
you. I don't just come to these
conferences and present to you guys. I go
00:45:50.920 --> 00:45:54.430
and I present to the engineers too. And in
fact a couple of engineers have come to
00:45:54.430 --> 00:45:58.599
this conference because we did work at
other conferences to see what the hacker
00:45:58.599 --> 00:46:01.741
community is about and learn things from
the hacker community because this is a
00:46:01.741 --> 00:46:05.360
place where you can learn if you're just
not afraid of getting pwned a couple of
00:46:05.360 --> 00:46:10.999
times right. And it happens to me too
right. I learned a lot from getting
00:46:10.999 --> 00:46:14.249
compromised on my machine and watching
someone do something. Anyways back to the
00:46:14.249 --> 00:46:18.380
point I don't just work with engineers or
hackers. I also work with C-level
00:46:18.380 --> 00:46:21.920
executives so I'm on a sabbatical from
IOActive at the moment. at the Cambridge
00:46:21.920 --> 00:46:26.469
Center for Risk studies, and I'm working
with the insurance people which has its
00:46:26.469 --> 00:46:31.441
challenges shall we say. But some of them
are very intelligent people and they want
00:46:31.441 --> 00:46:34.670
to understand what's going on with hacking
attacks and they want to approach this
00:46:34.670 --> 00:46:40.839
from a slightly different angle. My stake
in that is to be sure that when the
00:46:40.839 --> 00:46:45.479
insurance people do get involved that they
actually ask for fixes and improve stuff.
00:46:45.479 --> 00:46:49.809
So yes I do my best to raise awareness
wherever I can. And I'm not alone. You can
00:46:49.809 --> 00:46:53.769
help me.
Questioner: Thank you
00:46:53.769 --> 00:46:58.019
applause
00:46:58.019 --> 00:47:05.570
Herald: OK, there's another question here.
Number two. Oh, and up there too, yes we
00:47:05.570 --> 00:47:09.380
saw you. OK number two was first I think.
Go ahead
00:47:09.380 --> 00:47:13.570
Question: incomprehensible. So you
mentioned a couple of things, err a couple
00:47:13.570 --> 00:47:18.440
of vulnerabilities and I was wondering
what you would think an ideal system would
00:47:18.440 --> 00:47:24.150
look like. You mentioned key provisioning
of course putting certificates. I assume
00:47:24.150 --> 00:47:28.470
that they were different certificates for
different devices rather than the same
00:47:28.470 --> 00:47:37.430
certificate for all devices. Okay that's a bad
thing. And and also sort of the way how
00:47:37.430 --> 00:47:44.839
the software update management works. So
how would you if you could give them some
00:47:44.839 --> 00:47:48.950
advice how to design a system
how would you do it?
00:47:48.950 --> 00:47:55.420
Éireann: Okay. So first of all I wouldn't
hard code the keys as you as you discussed
00:47:55.420 --> 00:48:01.859
to be in every device the same. It's one
thing to put in your documentation hey you
00:48:01.859 --> 00:48:07.630
should update the keys but I mean if I can
patch binary file with a key then there's
00:48:07.630 --> 00:48:11.089
no reason you couldn't do that on the
website where you download the firmware
00:48:11.089 --> 00:48:15.160
image right. Just as an example as a
thought experiment sort of makes that
00:48:15.160 --> 00:48:18.420
clear. The upgrade path for these devices
is download the firmware image from the
00:48:18.420 --> 00:48:25.280
website to some machine and then carry it,
because all these systems are airgapped.
00:48:25.280 --> 00:48:29.229
to some other location and then upload it
onto the switch right with hardcoded
00:48:29.229 --> 00:48:33.869
credentials. So first off whenever you
provision a switch initially you provision
00:48:33.869 --> 00:48:36.920
all of the credentials for that device.
That's standard practice of many routers
00:48:36.920 --> 00:48:41.900
and other pieces of equipment today. And I
would think less about defending and
00:48:41.900 --> 00:48:46.230
securing the device than on being
able to regularly check its integrity,
00:48:46.230 --> 00:48:48.539
the integrity of the firmware that is
running and the integrity of the
00:48:48.539 --> 00:48:54.289
configuration. So I'd focus on that and I'd
focus on being able to recover the switch
00:48:54.289 --> 00:48:57.740
after it's been attacked. So you reverse
your thinking. You assume that one day
00:48:57.740 --> 00:49:01.309
someone is going to crack your firmware
signing and crack this and crack that and
00:49:01.309 --> 00:49:05.930
you focus on how can I quickly upload a
new firmware image that is known to be
00:49:05.930 --> 00:49:12.250
good and verify that the one that is
uploaded is good to this device.
00:49:12.250 --> 00:49:16.059
Questioner: Thank you.
Herald: There was a question up there on
00:49:16.059 --> 00:49:18.769
the balcony.
Signal angel: Yes we have two questions
00:49:18.769 --> 00:49:25.549
here on the net. So the first one is how
would you solve the end of life issue.
00:49:25.549 --> 00:49:29.900
Sometimes incomprehensible clients just
gets really outdated.
00:49:29.900 --> 00:49:33.420
Éireann: That's absolutely true and it is
slightly unfair of me to be a hard on the
00:49:33.420 --> 00:49:38.349
vendors. But it's my job to take the
debate a little bit too far the other way.
00:49:38.349 --> 00:49:43.229
So how would I solve the end of life issue
is the question from the internet. I don't
00:49:43.229 --> 00:49:47.759
know. I think that's not a technical
problem it's a societal problem. Like when
00:49:47.759 --> 00:49:55.970
we buy bridges they are bridges until they
fall down. When we buy roads they stay
00:49:55.970 --> 00:49:59.130
there until they go away. I mean there is
probably some end of life issues in there
00:49:59.130 --> 00:50:04.960
but it's almost more of a contractual
legal issue and someone should study that.
00:50:04.960 --> 00:50:08.339
There are people studying that but it's
not my area of expertise but I'll try and
00:50:08.339 --> 00:50:12.969
answer as best I can. I think code escrow
is a good way to go when you buy some of
00:50:12.969 --> 00:50:18.079
these devices you say I want the code for
this device in the future. I want to have
00:50:18.079 --> 00:50:22.369
access to it. If your company goes
bankrupt I need you to give up the source
00:50:22.369 --> 00:50:26.329
code for these devices when you go
bankrupt or when you disappear or when
00:50:26.329 --> 00:50:30.380
it's the end of life. There are a couple
of manufacturers out there doing open
00:50:30.380 --> 00:50:35.200
source switches. There's a company called
Open gear who are awesome. They gave me a
00:50:35.200 --> 00:50:39.790
switch to play with that I haven't had
time to look at yet. I think that's amazing
00:50:39.790 --> 00:50:42.700
right. And their code is open source and
you can go and examine it. So you would
00:50:42.700 --> 00:50:46.309
have the code anyway. Those are two
different approaches. I think there are
00:50:46.309 --> 00:50:49.979
others you can solve this problem
technically or legally or socially but as
00:50:49.979 --> 00:50:55.869
a society we depend on these utilities and
that code should not just vanish when it's
00:50:55.869 --> 00:51:05.369
difficult or costly to keep it upgraded.
applause
00:51:05.369 --> 00:51:08.089
Herald: There was a second
question from the Internet.
00:51:08.089 --> 00:51:14.179
Signal angel: Yes, so the second one is:
what should a non-technical person in
00:51:14.179 --> 00:51:19.890
the respect of incomprehensible set non-
technical person sent to manage small town
00:51:19.890 --> 00:51:25.440
utility do as best practice?
Éireann: I think the first and most
00:51:25.440 --> 00:51:29.930
important thing is to look for attacks.
I'm sorry I should probably repeat that
00:51:29.930 --> 00:51:33.609
question just to be sure. What should
someone in a small town who manages
00:51:33.609 --> 00:51:37.420
utility do to defend themselves and
protect himself. So the first thing is
00:51:37.420 --> 00:51:43.129
look for attacks. Even if you spend a few
hours a week looking for something you
00:51:43.129 --> 00:51:46.249
script something up or you hire some
college kid to come in and script
00:51:46.249 --> 00:51:49.579
something and look for things on your
network and ask questions and yes they're
00:51:49.579 --> 00:51:52.279
going to be a pain in the ass and is going
to be difficult. But you're going to learn
00:51:52.279 --> 00:51:55.599
things about your network and you might
detect some attacks. The first problem in
00:51:55.599 --> 00:52:01.059
utilities is no one is responsible for
security. It's not my job. It's kind of
00:52:01.059 --> 00:52:05.480
the mantra so for a small utility find
someone whose job it is if you're a very
00:52:05.480 --> 00:52:09.130
small utility there's probably some other
small utilities near you and you can hire
00:52:09.130 --> 00:52:13.789
a resource together to come and visit your
different utilities and help you out. The
00:52:13.789 --> 00:52:17.380
second one is watch your relationship with
your vendor when you purchase this
00:52:17.380 --> 00:52:21.220
equipment you spend a lot of money on it.
Spend a little bit of time doing
00:52:21.220 --> 00:52:25.069
penetration tests. Yes I like it when you
hire me but you don't have to hire me.
00:52:25.069 --> 00:52:28.071
There are plenty of other people you can
hire who will have a look at the device
00:52:28.071 --> 00:52:31.770
and find the simple vulnerabilities. So
when you purchase something make sure you
00:52:31.770 --> 00:52:35.469
test it for security purposes and that's
very important because you can even put
00:52:35.469 --> 00:52:40.879
into your contract if you fail the
security tests we will pay you less money.
00:52:40.879 --> 00:52:44.480
And the vendors are not going to react
to security until you do that. So that's
00:52:44.480 --> 00:52:51.429
the second answer. And I wish I had a
third to make it very neat but I don't.
00:52:51.429 --> 00:52:55.729
Herald: OK. There was one more
question at mic 4 I think
00:52:55.729 --> 00:52:58.500
Questioner: Yes hi thank you for
your time.
00:52:58.500 --> 00:53:03.539
Herald: Talk into the mike please. Thank
you for your talk. Q Hi. I'm kind of a
00:53:03.539 --> 00:53:12.739
newbie to the C3 community and I am not
sure about the question I want to ask you.
00:53:12.739 --> 00:53:16.579
Probably many people understand in this
room but I don't know if I would like to
00:53:16.579 --> 00:53:23.780
ask you what exactly do you
mean by arbitrary firmware.
00:53:23.780 --> 00:53:28.799
Éireann: No problem. So the question was
What do you mean by arbitrary firmware. I
00:53:28.799 --> 00:53:34.349
mean the firmware that I have altered that
was not manufactured by the vendor to do
00:53:34.349 --> 00:53:39.230
whatever I want. How do you trust that
this switch sends all the packets that it
00:53:39.230 --> 00:53:45.049
should send. What if it's, you know, my
handle is BSB right. What if it drops
00:53:45.049 --> 00:53:51.230
every packet that has BSB in the packet.
Right. You can rewrite a firmware image to
00:53:51.230 --> 00:53:54.950
do whatever the device can do and in some
cases more things than the device usually
00:53:54.950 --> 00:53:59.959
does to damage itself for example. So an
arbitrary firmware is one in which anyone
00:53:59.959 --> 00:54:03.489
writes the firmware and there is no
checking to be sure that this is the image
00:54:03.489 --> 00:54:08.490
that you want on this device whether it's
provided by the vendor or the community
00:54:08.490 --> 00:54:13.239
right. You still want checking that this
is the correct code or the code that you
00:54:13.239 --> 00:54:18.309
wanted anyway. Right.
Herald: Okay thank you. Is that a question
00:54:18.309 --> 00:54:22.489
here mic 1? OK go ahead.
Questioner: Yes please. In your
00:54:22.489 --> 00:54:29.739
hypothetical question, you asked what
damage could I do in that paint factory.
00:54:29.739 --> 00:54:39.690
But you can also reverse it. What kind of
company secrets can I obtain for example,
00:54:39.690 --> 00:54:45.859
your favorite recipe for your hot
chocolate or the recipes of Coca-Cola.
00:54:45.859 --> 00:54:52.839
They are vulnerable as well aren't they.
Éireann: Yes. So the question just again
00:54:52.839 --> 00:54:56.559
for everyone else. You don't just have to
talk about damage in a paint factory or
00:54:56.559 --> 00:55:01.819
any industrial system. You can also talk
about intellectual property and protecting
00:55:01.819 --> 00:55:07.309
the recipes that we use to bake cookies or
make beer or whatever pharmaceuticals
00:55:07.309 --> 00:55:12.641
whatever. And that's a fantastic question
and I'm glad you brought it up a couple of
00:55:12.641 --> 00:55:15.809
years ago when I was doing... well, more
than a couple of years like eight years
00:55:15.809 --> 00:55:19.249
ago, when I was doing industrial system
security I realized I wasn't getting a lot
00:55:19.249 --> 00:55:23.489
of traction. It was before stuxnet, I was
a quality assurance guy. Everybody thought
00:55:23.489 --> 00:55:34.309
I was fucking crazy right. Stuxnet,
career. It's wrong. It's really wrong. But
00:55:34.309 --> 00:55:39.579
the point is I tried to take that
approach. I tried to say you have a
00:55:39.579 --> 00:55:43.019
process in which you manufacture something
and you make money by the fact that that
00:55:43.019 --> 00:55:47.979
process is relatively secret and if you
don't care about defending your workers
00:55:47.979 --> 00:55:52.589
from being damaged then at least care
about the intellectual property because
00:55:52.589 --> 00:55:56.059
I'll get security in by some sort of back
door right. I'm a little bit of a security
00:55:56.059 --> 00:56:00.200
Machiavellian. I'll find a way to get
security into the system somehow. So I
00:56:00.200 --> 00:56:05.349
tried to say intellectual property you
should be protected. And I found that they
00:56:05.349 --> 00:56:09.320
didn't care so much. I mean maybe you'll
have more luck maybe post-stuxnet that
00:56:09.320 --> 00:56:14.069
that's a better argument. I hope you do.
But it is an important question as well.
00:56:14.069 --> 00:56:18.719
Right. It's not, it's not just potential
for damage. I think there's a lot more
00:56:18.719 --> 00:56:25.459
espionage going on on these networks than
there is damage and sabotage. Herald: Okay
00:56:25.459 --> 00:56:32.069
we'll take one more question on mike four.
Questioner: Thank you okay. My question
00:56:32.069 --> 00:56:38.319
concerns the concepts of software defined
networking and open flow. So when I first
00:56:38.319 --> 00:56:44.880
heard about software defined networking I
thought well this is a huge security issue
00:56:44.880 --> 00:56:50.589
and there may be huge vulnerabilities.
After your joke I think this might
00:56:50.589 --> 00:56:56.420
actually be a good idea to dumb down the
switches and put the intelligence
00:56:56.420 --> 00:57:01.900
somewhere locked up in a safe place.
What's your opinion on that. Can they
00:57:01.900 --> 00:57:05.839
actually improve security.
Éireann: Yes. So the question is what role
00:57:05.839 --> 00:57:09.969
could software defined networking play in
these sorts of environments. And is it a
00:57:09.969 --> 00:57:15.210
good idea from a security perspective.
Anytime someone has a revolution in
00:57:15.210 --> 00:57:19.240
computing we also have to update our
security paradigm. So I think with
00:57:19.240 --> 00:57:23.039
software defined networking it's not
whether it's good or bad it's that you
00:57:23.039 --> 00:57:28.339
defend that network differently than you
defend one of these networks. So it's not
00:57:28.339 --> 00:57:31.400
so much that as good as good or bad it's
neutral if you know how to defend your
00:57:31.400 --> 00:57:34.779
network. I don't care what it is. As long
as someone is looking to defend it and
00:57:34.779 --> 00:57:38.989
cares about how the flows are working. So
I think software defined networking in
00:57:38.989 --> 00:57:42.449
these environments could be a very good
thing but the refresh rate on these
00:57:42.449 --> 00:57:45.799
devices is not that high. So I don't think
we'll see it there for a little while even
00:57:45.799 --> 00:57:50.859
though it might be a good thing
philosophically. It takes 5 10 15 20 years
00:57:50.859 --> 00:57:56.410
to refresh these networks so it'll be a little
while. But it's not good or bad. It's just
00:57:56.410 --> 00:57:59.909
learn to defend what you got is the
problem right.
00:57:59.909 --> 00:58:06.489
Questioner: Okay thanks a lot.
Herald: Okay okay let's give a big hand
00:58:06.489 --> 00:58:09.639
for Éireann and thank you.
Éireann: Thank you
00:58:09.639 --> 00:58:13.320
applause
00:58:13.320 --> 00:58:24.000
subtitles created by c3subtitles.de
Join, and help us!