0:00:09.900,0:00:20.581
Éireann: Things are blowing up, in[br]industrial systems, here in Germany, this

0:00:20.581,0:00:26.080
year! I had hoped that these things[br]wouldn't happen. This kind of future

0:00:26.080,0:00:34.210
wouldn't be one that we are living in. But[br]unfortunately it is. And I hope that we

0:00:34.210,0:00:39.550
can make that better, partly through the[br]course of this talk. But more, I think, in

0:00:39.550,0:00:45.230
the future with your help and your work.[br]So I'm sorry to begin this presentation

0:00:45.230,0:00:52.489
with such a dark thought but: This year's[br]theme is a new dawn. And it's always

0:00:52.489,0:00:57.360
darkest just before the dawn. So we're[br]going to go through some of that darkness

0:00:57.360,0:01:03.640
in industrial systems and SCADA-systems to[br]get to a better place, right? Now with

0:01:03.640,0:01:09.550
that said no hacker really gets to be[br]where they are without the help of other,

0:01:09.550,0:01:14.650
right? We stand on the shoulders of giants[br]and part of the key is not stepping on

0:01:14.650,0:01:20.350
their toes, on the way up. So I would like[br]to say thank you to a bunch of people who

0:01:20.350,0:01:24.720
are here and also some people who aren't[br]here. Particularly the Oslo hackerspace

0:01:24.720,0:01:27.970
where I hang out. And these people have[br]taught me a lot of things not just about

0:01:27.970,0:01:33.560
technology but about life and on[br]"aprendo", which is how Goya signed some

0:01:33.560,0:01:40.090
of his last paintings and sketches - which[br]basically means "I'm still learning". OK.

0:01:40.090,0:01:46.390
So with that said I hope that you will[br]enjoy this talk with its darkness and its

0:01:46.390,0:01:49.890
humor all at the same time. I used to be[br]in circus, as you may have guessed from

0:01:49.890,0:01:56.200
the mustache. So I encourage you not just[br]to view this as a technical vulnerability

0:01:56.200,0:02:02.180
presentation but also as kind of live[br]technical standup comedy. Instead of jokes

0:02:02.180,0:02:07.450
we have vulnerabilities. And I hope that[br]you will enjoy them. So these

0:02:07.450,0:02:12.700
vulnerabilities are in switches. I chose[br]to focus on switches and that will become

0:02:12.700,0:02:19.489
clear throughout the presentation, why I[br]chose to do that for industrial systems.

0:02:19.489,0:02:23.580
And we are looking primarily at three[br]different families of switches. Because I

0:02:23.580,0:02:28.049
don't want to pick on any one vendor. In[br]fact, the whole idea of this talk is to

0:02:28.049,0:02:31.790
continue giving it. I have two other[br]colleagues who couldn't be here with me

0:02:31.790,0:02:35.519
today, who have some vulnerabilities in[br]some other switches. And they look forward

0:02:35.519,0:02:39.549
to presenting those vulnerabilities as[br]part of this presentation in the future.

0:02:39.549,0:02:43.409
So every time we give this presentation[br]we'd like to give some new vulnerabilities

0:02:43.409,0:02:49.680
and show that this is systemic and endemic[br]risk. So the three switches we'll be

0:02:49.680,0:02:54.239
looking at today are the Siemens Scalance-[br]family, the GE Multilin-family and the

0:02:54.239,0:02:58.819
Garrettcom Magnum family. These switches[br]are usually not very big. They might be 8

0:02:58.819,0:03:05.709
ports, they might be 24 ports. And they're[br]used in a variety of different locations.

0:03:05.709,0:03:13.260
So this talk is for you, if you work in a[br]utility, if you test industrial Ethernet

0:03:13.260,0:03:17.480
switches, if you manage industrial[br]Ethernet networking, if you're comfortable

0:03:17.480,0:03:21.510
at a Linux commandline and you play with[br]web apps but you don't know as much about

0:03:21.510,0:03:25.450
reverse engineering. Don't worry, I'm[br]exactly the same. I suck at reverse

0:03:25.450,0:03:31.519
engineering. But I care about this stuff.[br]And so I'm learning. If you are a

0:03:31.519,0:03:35.909
developer of firmware then I think this[br]talk is for you as well. I hope you learn

0:03:35.909,0:03:39.709
something from it. If you like[br]vulnerabilities you'll enjoy this quite a

0:03:39.709,0:03:45.129
lot. I'm going to be sharing with you a[br]little collection I have, you know. Some

0:03:45.129,0:03:51.629
people collect stamps or stories or jokes.[br]I collect private keys. And I like to

0:03:51.629,0:03:58.011
share them with other enthusiasts such as[br]yourself. If you happen to work for one of

0:03:58.011,0:04:01.090
the switch manufacturers you know I've[br]spoken to before. Some of you I get on

0:04:01.090,0:04:06.809
with very well. We speak regularly. Some[br]of you not yet - but I hope you'll come

0:04:06.809,0:04:13.199
and have a chat with me later. Ok, most[br]SCADA or ICS presentations go a bit like

0:04:13.199,0:04:20.640
this: Pwn PLC, the RTU, the HMI - these[br]are terms, you know, that all of us in

0:04:20.640,0:04:23.830
SCADA know. Maybe most of you know them by[br]now, they're pretty popular. I hope you

0:04:23.830,0:04:27.780
do. But programmable logic controller,[br]remote terminal unit or human machine

0:04:27.780,0:04:32.920
interface. And the basic idea of the[br]presentation is if I pwn these things,

0:04:32.920,0:04:38.090
game over. Physical damage. I win. Isn't[br]the world a scary place? And I encourage

0:04:38.090,0:04:42.240
you to demand better content. I certainly[br]grew up with better content. I used to go

0:04:42.240,0:04:46.229
and see the presentations and the talks of[br]a guy called Jason Larson. And he has a

0:04:46.229,0:04:50.271
fantastic example of this. I want all of[br]you to try it, right now. Just think

0:04:50.271,0:04:56.110
about: If you had complete control over a[br]paint factory. What would you do to damage

0:04:56.110,0:04:59.470
it? No one is going to get hurt.[br]Everything's safe. It's a thought

0:04:59.470,0:05:05.630
experiment, right? What would you do to[br]damage it? Most people can't answer this

0:05:05.630,0:05:09.740
question. And on certain types of[br]processes I can't answer this question.

0:05:09.740,0:05:12.790
But other types I've worked with before[br]and I can answer this question. And I

0:05:12.790,0:05:17.530
encourage you to to ask it. But if you[br]like and you want to learn more go and see

0:05:17.530,0:05:24.069
Marmusha's talk - I think it's tomorrow.[br]Think of my talk as a frame for her talk.

0:05:24.069,0:05:28.111
She's going to be talking about how to[br]damage a chemical process. And what you

0:05:28.111,0:05:32.090
need to do as an engineer to do that. And[br]the reason she's doing that is to build a

0:05:32.090,0:05:36.290
better process in the future. You have to[br]break a few things to make them work a

0:05:36.290,0:05:41.379
little bit better. Okay. So what's the[br]point in industrial control systems

0:05:41.379,0:05:47.750
security? It's not credit card data. It's[br]not privacy. No disrespect to my privacy

0:05:47.750,0:05:52.240
friends in the room. I have the deepest[br]love and respect for the work that you do.

0:05:52.240,0:05:58.310
But confidentially ... confidentiality is[br]the lowest priority for us in industrial

0:05:58.310,0:06:04.830
systems. It would go: Availability,[br]integrity, confidentiality. And you might

0:06:04.830,0:06:10.970
even swap integrity and availability in[br]many cases. So, you have to protect the

0:06:10.970,0:06:16.770
sensor data or the control signals.[br]Everything else is maybe a vulnerability

0:06:16.770,0:06:20.210
on the path to getting this. But it's not[br]the most important thing that we're trying

0:06:20.210,0:06:25.600
to protect. So that's why I'm attacking[br]switches. That's where the process is,

0:06:25.600,0:06:32.639
right? Now these may not be core switches.[br]They're often a little bit further down in

0:06:32.639,0:06:37.069
the chain. They're field devices, right.[br]So you might find them in any of these

0:06:37.069,0:06:44.590
locations. And this last example is not[br]necessarily important be cause oil and gas

0:06:44.590,0:06:49.270
is important - but it's important because[br]it gives you the general format of all

0:06:49.270,0:06:53.300
industrial systems. You have sensor[br]network. And sensor data is traveling back

0:06:53.300,0:06:58.610
and forth. And you have control signal[br]data. That's it, basically. You might have

0:06:58.610,0:07:01.349
different control signals on different[br]protocols and you might have different

0:07:01.349,0:07:05.410
sensors on different protocols, giving you[br]different values like pressure or heat or

0:07:05.410,0:07:15.889
whatever. But most processes follow[br]basically this format. Okay. I don't do

0:07:15.889,0:07:19.509
SCADA 101. There are other people who do[br]this. I'm trying to do a little bit, to

0:07:19.509,0:07:25.759
set the reference for this talk, but[br]usually I avoid it. So basically there's

0:07:25.759,0:07:30.819
not much authentication or integrity in[br]industrial systems protocols. There's not

0:07:30.819,0:07:36.370
much cryptography. You would expect there[br]to be, maybe. I'm continually surprised

0:07:36.370,0:07:40.319
that I don't find any. And when I do find[br]it, it's badly implemented and barely

0:07:40.319,0:07:48.229
works. So once you have compromised a[br]switch or another part of the network you

0:07:48.229,0:07:51.930
can perform man-in-the-middle attacks on[br]the process. Or you can create malicious

0:07:51.930,0:07:56.400
firmwares on these different switches. And[br]that's what I'm trying to prevent. I'm

0:07:56.400,0:08:00.029
trying to find some of the different[br]methods that people can use to produce

0:08:00.029,0:08:09.800
these firmwares - and then get the vendors[br]to fix them, right. Okay. These are some

0:08:09.800,0:08:14.419
of the protocols. If you are new to this[br]space, if you want to do some more work in

0:08:14.419,0:08:17.550
this area, but you don't know what to work[br]on, take a picture of the slide or go and

0:08:17.550,0:08:21.319
find it later. And choose one of these[br]protocols and go and work on it. We need

0:08:21.319,0:08:24.250
people to go to these different[br]organizations. Some of them are

0:08:24.250,0:08:27.499
proprietary, some of them are open and[br]complain that there is not enough

0:08:27.499,0:08:32.250
cryptography going on in this space. And[br]yes you can use VPNs. But believe me, I

0:08:32.250,0:08:43.720
often don't find them. Okay. These are the[br]switches, the specific versions of the

0:08:43.720,0:08:46.600
firmware, in case you're here for[br]vulnerabilities instead of just me

0:08:46.600,0:08:52.020
waffling on about the basics. If you want[br]to go and look these up, if you're a

0:08:52.020,0:08:57.910
penetration tester working in this space,[br]you can go and find them all online. And

0:08:57.910,0:09:02.210
you can get a feeling for the kind of[br]coding practices that go into these

0:09:02.210,0:09:07.440
different devices. Now I've tried to[br]choose the vulnerabilities that I'm

0:09:07.440,0:09:14.710
presenting very carefully. To take you[br]gently from web app vulnerabilities into a

0:09:14.710,0:09:19.950
little bit deeper into the firmware. So[br]the first one we'll be looking at is

0:09:19.950,0:09:24.710
Siemens. And again, I'm not picking on any[br]particular vendor. In fact I'm very proud

0:09:24.710,0:09:30.310
of Siemens. They're probably here again.[br]They're here many years. And they fixed

0:09:30.310,0:09:35.200
these vulnerabilities within three months.[br]And I think that was awesome - especially

0:09:35.200,0:09:41.500
in the space that I work in. The average[br]patch-time in SCADA and ICS is 18 months.

0:09:41.500,0:09:45.500
So I think Siemens deserves a round of[br]applause for getting these fixed.

0:09:45.500,0:09:52.870
<i>Applaus</i>[br]So without further ado let's have some

0:09:52.870,0:09:58.590
fun, right. So MD5, you go to the web page[br]for this switch. This is the management

0:09:58.590,0:10:03.350
page of a switch, right. And you interact[br]with this webpage. And you have a look at

0:10:03.350,0:10:12.580
it. And on the client side they do MD5 of[br]the password. Okay. That's fascinating. I

0:10:12.580,0:10:17.290
don't think that's particularly secure.[br]But it's done in roughly the same format

0:10:17.290,0:10:20.641
as that Linux command. So I use the Linux[br]command instead of the JavaScript just to

0:10:20.641,0:10:26.060
make it easier for everyone. You have the[br]username at the beginning and the password

0:10:26.060,0:10:30.040
is in the middle. And then you have this[br]nonce that's at the end, a number you use

0:10:30.040,0:10:34.470
once, right. I was surprised to see the[br]nonce, and it's even called a nonce,

0:10:34.470,0:10:37.140
right. So somebody had done a little bit[br]of homework on their cryptography. And

0:10:37.140,0:10:41.150
they understood that they wanted to use,[br]you know, this number used once to prevent

0:10:41.150,0:10:45.340
replay of the hash every time. Okay,[br]that's some pretty good work.

0:10:45.340,0:10:49.200
Unfortunately this is MD5 and this is[br]protecting your electric utilities and

0:10:49.200,0:10:56.070
your water and your sewage systems. And[br]you can brute force this in a few seconds,

0:10:56.070,0:11:00.200
if the passwords are less than eight[br]characters. and if they're around 15 it

0:11:00.200,0:11:04.460
might take you 20 minutes or something.[br]You can do this from PCAPs, from network

0:11:04.460,0:11:08.300
traffic captures. And then you have the[br]cleartext password that you can use

0:11:08.300,0:11:16.420
forever after, with that switch. So, off[br]to a bad start, in my opinion. So these

0:11:16.420,0:11:22.770
are the nonces that we're looking at. I'm[br]glad to hear you laughing. It makes me, it

0:11:22.770,0:11:27.421
warms the heart, right. So you can see[br]that they are incrementing and that they

0:11:27.421,0:11:37.750
are hex. Yeah. What else can you say about[br]this? The last half is different than the

0:11:37.750,0:11:45.870
first half. Not only is it incrementing,[br]it is sequential. If you pull them quickly

0:11:45.870,0:11:53.260
enough. For those of you who also do a bit[br]of reverse engineering you might recognize

0:11:53.260,0:11:58.130
the first half as well. Anybody in the[br]room see any patterns in the first half of

0:11:58.130,0:12:09.950
the of the nonces? No? Hmm? Very good, IP[br]address. Mac address would have been a

0:12:09.950,0:12:13.521
good guess as well. I thought it was at[br]first. And I got very confused when I went

0:12:13.521,0:12:17.340
to look for the IP address. Because I went[br]to the switch itself. And the switches IP

0:12:17.340,0:12:25.080
address was not this in hex. It's the[br]clientside address. Which I just couldn't

0:12:25.080,0:12:29.380
believe, right? Like, it seems like it[br]makes a sort of sense if you're trying to

0:12:29.380,0:12:33.580
keep session IDs in state. And it's like[br]oh I want a different session for every IP

0:12:33.580,0:12:39.480
address. And then I'll just use time, I[br]use uptime in hex as the rest of my

0:12:39.480,0:12:45.160
session ID, right? You know, the entire IP[br]space and time that can't be brute force.

0:12:45.160,0:12:52.250
It has a kind of crazy logic to it, right.[br]Unfortunately it can be. And you can get

0:12:52.250,0:12:56.730
the uptime from the device using SNMP. And[br]of course if you don't want to use SNMP

0:12:56.730,0:13:04.470
you can get old-school and use the TCP-[br]sequence-ID numbers. So, not a lot of

0:13:04.470,0:13:09.550
entropy there, I guess, I would say. And I[br]think their lawyers agreed when they put

0:13:09.550,0:13:17.640
out the comments on this. All right. Not[br]only can you perform session hijacking.

0:13:17.640,0:13:21.050
And if you are attacking switches I'd like[br]to point out that session hijacking is not

0:13:21.050,0:13:25.230
necessarily a great attack in this[br]environment. Think about it like you would

0:13:25.230,0:13:29.700
at home, right. How often do you log into[br]your router? In fact even more importantly

0:13:29.700,0:13:33.250
how often do you upgrade the firmware on[br]your router? Everyone who has upgraded the

0:13:33.250,0:13:37.940
firmware on their router ever raise your[br]hand. Just for an experiment. Thank

0:13:37.940,0:13:42.140
goodness, right. But wait, keep them up[br]just for a minute. Everybody who's updated

0:13:42.140,0:13:45.670
it this year, keep your hand up. Everybody[br]else put them down. Everybody who has

0:13:45.670,0:13:50.420
updated in the last six months ... okay[br]... So that gives you a sense of how long

0:13:50.420,0:13:55.061
these vulnerabilities can be in play on an[br]industrial system's environment. If you

0:13:55.061,0:14:01.800
multiply that by about 10, right. Okay, so[br]you can simply upload a firmware image to

0:14:01.800,0:14:06.140
a Siemens Scalance device with this[br]version number without authentication. You

0:14:06.140,0:14:15.700
just need to know the URL. Cross-site[br]request forgery, right. I just say CSRF

0:14:15.700,0:14:20.220
all the time. I don't even remember what[br]it stands for. So you can upload or you

0:14:20.220,0:14:23.381
can download a logfile. Not that useful[br]but you get a sense of what's going on on

0:14:23.381,0:14:27.191
the switch. You know what usernames might[br]be present, whatever. Incidentally all of

0:14:27.191,0:14:32.050
these switches by default or at least this[br]one only have two usernames, right. So

0:14:32.050,0:14:37.151
it's "admin" and "operator" I think on[br]this switch. Or maybe it's not. But

0:14:37.151,0:14:42.830
anyway, there's two usernames "admin" and[br]"manager"? I know I get them mixed up now.

0:14:42.830,0:14:47.130
But the configuration includes password[br]hashes. I'm actually not even entirely

0:14:47.130,0:14:50.620
convinced they're hashes because when you[br]increase the length of your password it

0:14:50.620,0:14:55.610
increases. But I'll leave that for future[br]researchers to examine. You can download

0:14:55.610,0:14:59.241
the firmware image from the device, which[br]is nice. So you just make a request. You

0:14:59.241,0:15:03.110
just post an HTTP-request to this device.[br]And it gives you the firmware that it is

0:15:03.110,0:15:07.820
running back. That's not that big a deal,[br]right. Because you're just viewing data on

0:15:07.820,0:15:14.930
the switch. But you can upload firmware[br]and configuration to this device. Which is

0:15:14.930,0:15:18.540
an authentication bypass in and of itself.[br]But it's also interesting because I can

0:15:18.540,0:15:22.430
take a configuration file from one of the[br]devices that I have at home with a known

0:15:22.430,0:15:27.490
password. I can upload a new configuration[br]file with a password that I know. I can

0:15:27.490,0:15:31.500
use the device to do whatever I want to[br]do. And later I can re upload the old

0:15:31.500,0:15:35.560
configuration file that I got from the[br]device, so no one ever even realizes what's

0:15:35.560,0:15:45.730
been changed, right. So. I think that's a[br]disappointing state of affairs. And I

0:15:45.730,0:15:49.340
wrote a script to do this. So that you[br]wouldn't have to when you are doing

0:15:49.340,0:15:53.920
penetration tests of these device. And I[br]gave you a little ASCII menu because

0:15:53.920,0:15:58.410
sometimes I get bored. Cambridge is a[br]small town and there's not much to do in

0:15:58.410,0:16:05.640
the evening. So feel free to go and[br]examine my github-repository where I put

0:16:05.640,0:16:11.910
up some of this stuff. I'm Blackswanburst[br]on Github, and on Twitter. So like I say,

0:16:11.910,0:16:15.360
Siemens are some of my favorite people. So[br]I'm going to finish up with them. This is

0:16:15.360,0:16:19.980
old day, if you like all that you have[br]just seen. But I want you to keep in mind

0:16:19.980,0:16:24.230
that these vulnerabilities will still be[br]present in the wild for another two or

0:16:24.230,0:16:28.980
three years. And I encourage you to go and[br]have a look at your systems, if you have

0:16:28.980,0:16:34.170
any of these devices. And check them out.[br]And upgrade the firmware. I also hope this

0:16:34.170,0:16:38.540
encourages you that if you haven't done[br]much in industrial systems and SCADA you

0:16:38.540,0:16:42.270
don't have to be intimidated by all of the[br]engineering and the terminology, and the

0:16:42.270,0:16:47.001
verb beotch(?).. There is plenty for any[br]of you in this room to do in the

0:16:47.001,0:16:51.700
industrial systems space. You need to[br]spend a little time speaking to engineers

0:16:51.700,0:16:56.900
and translating your vulnerabilities into[br]something meaningful for them. But that's

0:16:56.900,0:17:00.250
just a matter of spending more time with[br]them and getting to know them. And I think

0:17:00.250,0:17:03.740
that's valuable too because they have a[br]lot of experience. They care very deeply

0:17:03.740,0:17:08.309
about safety. And I've learned quite a lot[br]of things from engineers. My general point

0:17:08.309,0:17:13.601
here is I'd like you to stop defending[br]banks and websites and other stuff. We

0:17:13.601,0:17:18.099
need your help in industrial systems, in[br]the utilities. We could really do with

0:17:18.099,0:17:22.180
living in a safer world rather than one[br]where you're just protecting other

0:17:22.180,0:17:32.480
people's money. So we're gonna move on to[br]the GE Multilin line. I worked on a GE

0:17:32.480,0:17:38.830
ML800 but these vulnerabilities affect[br]seven of the nine switches in this family.

0:17:38.830,0:17:43.410
Seven because one of the other switches is[br]an unmanaged switch. If you're a hardware

0:17:43.410,0:17:47.880
person maybe you want to go and play[br]around with those but not so much my thing

0:17:47.880,0:17:51.130
and the other one uses a different[br]firmware image but seven of the nine

0:17:51.130,0:17:58.020
switches use a similar firmware image GE[br]offers a worldwide 10 year warranty. So

0:17:58.020,0:18:01.950
let's see if that includes fixing[br]vulnerabilities. I think it should. What

0:18:01.950,0:18:10.650
do you think. No? Couple noes couple of[br]yeses, undecided. All right. CCC is

0:18:10.650,0:18:17.851
undecided on something that's novel. Let's[br]start with some new vulnerabilities. Cross

0:18:17.851,0:18:22.750
site scripting. Reflected, I grant you but[br]still cross site scripting and I want you

0:18:22.750,0:18:25.530
to pay attention to the details. I'm not[br]going to go slow for you and ask you to

0:18:25.530,0:18:29.160
think . I know it's morning, I know it's[br]tough but I am going to ask you to think.

0:18:29.160,0:18:36.970
See flash up there flash.php and the third[br]one. Yes, it runs flash in your browser.

0:18:36.970,0:18:42.470
So if you know something about Flash come[br]and have a look at the switch some time. I

0:18:42.470,0:18:47.751
didn't go for active script attacks. There are[br]so many attacks surface on this device. I

0:18:47.751,0:18:52.460
just I sometimes don't even know how I'm[br]going to finish looking at all of them. So

0:18:52.460,0:18:55.780
I just work with the web interface to[br]begin with. So you have this cross site

0:18:55.780,0:19:00.680
scripting times eight and I want you to[br]notice in the last section there

0:19:00.680,0:19:05.970
arbitrarily supplied URL parameters. I[br]don't know about you but I think that's

0:19:05.970,0:19:10.180
funny right. You can just make up[br]parameters to stick your cross site

0:19:10.180,0:19:20.480
scripting in. <i>laughs</i> It's unbelievable[br]right. Yeah. Anyways what does that look

0:19:20.480,0:19:28.340
like. It looks like that, they have an[br]error data page. OK maybe I'm using a

0:19:28.340,0:19:33.370
browser that they don't approve or[br]something but it deserves looking at. And

0:19:33.370,0:19:39.470
you can do quite a lot of things with[br]javascript on the client side these days.

0:19:39.470,0:19:44.480
Disturbing. Anyways I'm not a big fan of[br]XSS so I'm going to move on to things that

0:19:44.480,0:19:52.690
I think are worth my time. So if you fetch[br]the initial web page of this switch before

0:19:52.690,0:20:01.380
you've even logged in you get this config.[br]So this is pretty authentication. No

0:20:01.380,0:20:06.850
authentication, right. Now keep in mind that[br]these switches are designed for process

0:20:06.850,0:20:14.610
data, right. It's not carrying traffic to[br]images of cats. It's supposed to be for

0:20:14.610,0:20:22.630
engineering. So what happens if I add a[br]nocache parameter and I make it say 500000

0:20:22.630,0:20:30.030
digits long. I should just be able to[br]crash the web server. Right. Maybe maybe.

0:20:30.030,0:20:41.270
But you would not expect it to reboot the[br]switch. And it takes a minute or so for

0:20:41.270,0:20:44.800
the switch to reboot which is actually[br]really impressive comes up pretty quickly.

0:20:44.800,0:20:50.950
But you know obviously you can repeat[br]this. So I wanted to examine that a lot

0:20:50.950,0:20:56.390
further. I wanted to know more about that[br]that crash what was rebooting the switch.

0:20:56.390,0:20:59.290
But like I say I'm not a very good reverse[br]engineer. So you're going to go on a

0:20:59.290,0:21:02.590
little journey with me where I learned a[br]couple of things about reverse engineering

0:21:02.590,0:21:06.160
and I had to change my approach from[br]looking at the webapp style loans to

0:21:06.160,0:21:12.470
moving into this other stuff. So why is[br]why is it DoS even interesting. You'll

0:21:12.470,0:21:18.320
remember that I mentioned Misha's talk. So[br]the reason I mention her talk, this is it

0:21:18.320,0:21:23.690
right. Denial of Service on a Website. Who[br]cares it's tearing posters down as xkcd

0:21:23.690,0:21:28.950
once famously explained to us but in the[br]industrial system's environment it's very

0:21:28.950,0:21:33.980
different. It can be very serious right. A[br]simplistic example is you have an

0:21:33.980,0:21:38.750
application that has a heartbeat and if[br]you stop that heartbeat it might go into

0:21:38.750,0:21:44.060
some sort of safety state it might for[br]example scram a reactor. There is a famous

0:21:44.060,0:21:50.851
denial of service on PLCs that did scram a[br]reactor in real life. Does anybody know

0:21:50.851,0:21:58.650
what H2S is? Any oil and gas engineers in[br]the room? Okay so H2S alerts not reaching

0:21:58.650,0:22:02.861
their destinations is pretty serious[br]business right. For those of you who are

0:22:02.861,0:22:07.850
not aware of H2S it's a byproduct of[br]producing oil and gas and inhaled in very

0:22:07.850,0:22:12.850
very small amounts you can go unconscious[br]and in sort of larger amounts. Respiratory

0:22:12.850,0:22:18.480
failure. So if you take CA safety[br]seriously if you ever work on these rigs

0:22:18.480,0:22:23.140
in these environments you learn to care[br]about the wind sock. Right one of these

0:22:23.140,0:22:26.620
alerts goes out. An alarm goes off. There[br]are many different alarms you have to

0:22:26.620,0:22:31.200
memorize how they all sound on a rig and[br]then react to them and when you hear the

0:22:31.200,0:22:35.330
H2S alert you look up at the wind sock to[br]keep an eye on where the wind is and

0:22:35.330,0:22:40.420
trying to avoid being downwind of wherever[br]the leak is. So a simple denial of service

0:22:40.420,0:22:43.510
that we would not care about in a web[br]application environment in this

0:22:43.510,0:22:47.940
environment can be very serious. I'm not[br]saying it always is. It just can be

0:22:47.940,0:22:53.350
right. So denial of service goes up in our[br]list of problems especially when we're

0:22:53.350,0:22:58.270
looking at networking devices. Okay so[br]that's that's it for the denial of

0:22:58.270,0:23:01.550
service. But like I say we're going to[br]look at some other stuff. In fact the

0:23:01.550,0:23:07.320
story with the switch began with a[br]concerned citizen about three or four

0:23:07.320,0:23:12.280
years ago I found 10000 industrial systems[br]on the Internet as part of my master's

0:23:12.280,0:23:17.990
thesis and I was pretty uncomfortable with[br]that. So I sent that data to various

0:23:17.990,0:23:23.889
computer emergency response teams around[br]the world. I believe it was 52 of them

0:23:23.889,0:23:26.860
right. Not all of them were critical[br]infrastructure. A lot of them were small

0:23:26.860,0:23:31.370
stuff but maybe 1 in 100. I was told or in[br]one particular country when they got back

0:23:31.370,0:23:38.400
to me one in 20 were considered critical[br]infrastructure. And after that you have a

0:23:38.400,0:23:42.540
sort of reputation among the computer[br]emergency response teams of the world. So

0:23:42.540,0:23:47.580
people send you stuff you get anonymous[br]e-mails from someone called Concerned

0:23:47.580,0:23:53.330
Citizen. Thank you very much. They sent me[br]a firmware upgrade pcap of this particular

0:23:53.330,0:23:57.350
device. I suspect that they worked at one[br]of the utilities and they wanted me to see

0:23:57.350,0:24:05.559
how upgrading the firmware of this GE switch[br]was performed. So it all began with a pcap.

0:24:05.559,0:24:11.290
So I ran TCP trace to carve out all the[br]files and see what was going on and you

0:24:11.290,0:24:16.590
could see instantly that there was an FTP[br]session later looking at the switch I see

0:24:16.590,0:24:21.120
that you can also upgrade them over TFTP[br]so the management of the switch happens in

0:24:21.120,0:24:26.841
HTTPs and is encrypted but the firmware[br]upload goes across FTP right so you can

0:24:26.841,0:24:33.700
just carve the file out a little bit of[br]network forensics I guess. So instantly I

0:24:33.700,0:24:36.950
could see that this one is complete and[br]the ports on the end of the numbers give

0:24:36.950,0:24:40.660
me a clue of what's going on in the larger[br]stream. This one seems interesting. Let's

0:24:40.660,0:24:48.240
have a look at it. So. I tried running[br]file and binwalk I don't know about you

0:24:48.240,0:24:52.860
but I believe that hacking is a journey of[br]understanding and facts hacking is

0:24:52.860,0:24:57.740
understanding a system better than it[br]understands itself and nudging it to do

0:24:57.740,0:25:03.950
what you want right. And I also feel that[br]I should understand my tools. I don't

0:25:03.950,0:25:07.420
really understand my tools until I know[br]where they're going to fail me or they

0:25:07.420,0:25:11.040
have failed me in the past and in this[br]particular case I think binwalk is a

0:25:11.040,0:25:15.150
fantastic tool and file is a fantastic[br]tool. But they didn't tell me anything and

0:25:15.150,0:25:18.750
that was that was a journey of discovery[br]for me. So that was nice. It was like OK

0:25:18.750,0:25:21.700
binwalk doesn't always give me everything.[br]I think I was running an older version and

0:25:21.700,0:25:25.179
I think it would handle it now. But the[br]point is after been walked didn't give me

0:25:25.179,0:25:29.950
anything just resort to the old school[br]stuff right. Go strings and I found these

0:25:29.950,0:25:34.050
deflate and inflate copywrite strings and[br]I could tell that a certain portion of the

0:25:34.050,0:25:43.670
file was compressed. This is just from the[br]pcap. Remember this whole story. So I

0:25:43.670,0:25:49.040
tried to deflate the whole thing. That[br]didn't work again. I just did something

0:25:49.040,0:25:54.561
simple get a python script that checks[br]every byte to see which parts of the file

0:25:54.561,0:26:00.831
don't produce ZLIB errors when you try and[br]decompress them and you figure out what

0:26:00.831,0:26:09.170
sectors of this file are compressed. So[br]you go to your friend dd and you carve out

0:26:09.170,0:26:15.760
this section of the file right. So we have[br]this larger firmware image with this

0:26:15.760,0:26:21.310
little compressed section and we have now[br]cut this little compressed section out. I

0:26:21.310,0:26:24.430
suppose I could have loaded this up into[br]python and use ZLIB to decompress it. But

0:26:24.430,0:26:27.559
at the time I was still trying to use[br]command line tools and someone said I'll

0:26:27.559,0:26:35.350
just concatenate the gzip bytes on it.[br]Gzip inherits from inflate and deflate. So

0:26:35.350,0:26:39.100
if you just concatenate the bytes it[br]should still handle it. So I did that and

0:26:39.100,0:26:43.920
I got a decompressed binary. When you ran[br]strings on that it started to make a lot

0:26:43.920,0:26:48.750
more sense and you could find the opcodes[br]in it where previously it didn't make any

0:26:48.750,0:26:53.910
sense at all. So once you've got an image[br]like that what do you do. Well if you're

0:26:53.910,0:26:58.250
me you just grep for bugs. I think I[br]learned that from Ilija. If he's here in

0:26:58.250,0:27:05.590
the room thank you. Thank you very much. I[br]asked him like a year or two ago. How do

0:27:05.590,0:27:10.761
you how do you find so many bugs. And he[br]said: "Oh, I just, you know, I grep for

0:27:10.761,0:27:16.510
them, I use find." <i>laughs</i> And so I[br]started thinking about firmware images.

0:27:16.510,0:27:19.640
Like if I was going to grep for a bug in a[br]firmware image what would it be. And my

0:27:19.640,0:27:23.840
answer is hardcoded credentials and[br]default keys because you find them every

0:27:23.840,0:27:29.309
single time so I have this command aliased[br]on my machine and I just grep for it and I

0:27:29.309,0:27:35.270
find private keys and this is how you too[br]can end up with a private key collection.

0:27:35.270,0:27:40.465
So, there you go.

0:27:40.465,0:27:50.240
<i>Applause</i>

0:27:50.240,0:27:53.770
Yeah they're hardcoded keys,[br]but what are they for. It doesn't

0:27:53.770,0:27:57.820
stop there. You know you've got the keys,[br]but what do they do, right? That was the

0:27:57.820,0:28:02.500
next step of the journey for me. Two of[br]them you can see one sencrypted with a

0:28:02.500,0:28:05.740
password; we'll come back to that one[br]later. Let's start with the one on the

0:28:05.740,0:28:15.860
left. If you load this key up into[br]wireshark. and you use it to decrypt the

0:28:15.860,0:28:22.760
SSL you have a self decrypting pcap.[br]Remember at the beginning it was using

0:28:22.760,0:28:29.590
HTTPS to manage the device and upload this[br]firmware image. So if you happen to have

0:28:29.590,0:28:37.210
this firmware image you can decrypt all[br]the traffic. No forward secrecy, right?

0:28:37.210,0:28:41.550
Now you don't have to be lucky and have[br]concerned citizens send you an email. You

0:28:41.550,0:28:46.490
can download this image from the GE website[br]and you can carve the keys out of the

0:28:46.490,0:28:50.100
image in the same way that I did and[br]decrypt the SSL traffic of any pcap that

0:28:50.100,0:29:01.880
is sent to you. Now the passwords[br]underneath that are in clear text. You can

0:29:01.880,0:29:08.040
see them highlighted down here. Password[br]Manager and user manager. You can see them

0:29:08.040,0:29:12.750
up there as well and you can see that[br]we've decrypted the SSL with that key. So

0:29:12.750,0:29:16.559
default keys, right? Is it a big deal? I[br]believe the vendors in this case say you

0:29:16.559,0:29:21.190
can upload your own key to the device. For[br]those of you who aren't used to working in

0:29:21.190,0:29:24.290
embedded it sometimes is difficult to[br]generate a key on the device because you

0:29:24.290,0:29:27.840
don't have enough memory or you don't have[br]enough entropy or you don't have enough

0:29:27.840,0:29:32.270
processing power. That's the usual[br]excuses. And they're true I shouldn't say

0:29:32.270,0:29:36.090
excuses those those things are true. But[br]you could of course generate it on the

0:29:36.090,0:29:39.850
client side and upload it to the device[br]and that's what they allow you to do with

0:29:39.850,0:29:44.790
this switch which is great but where is[br]your encrypted channel in which to upload

0:29:44.790,0:29:52.801
this key? <i>laughs</i> So you can use the serial[br]device and make sure visually that there's no man

0:29:52.801,0:29:55.340
in the middle. But if you're doing this[br]remotely – and I'd like you to keep in

0:29:55.340,0:29:59.460
mind that most substations are remote –[br]if anyone here works in a utility are you

0:29:59.460,0:30:03.530
going to drive to every substation, plug[br]in a serial cable to change the keys on

0:30:03.530,0:30:07.850
all these devices? It's the sort of thing[br]you need to know in advance right? So the

0:30:07.850,0:30:12.100
problem with key management, particularly[br]with SSL and the industrial systems

0:30:12.100,0:30:19.049
environment, is that you have to manage[br]the keys. And these particular keys, well

0:30:19.049,0:30:23.670
the certificates are self signed so you[br]can't revoke them. And besides industrial

0:30:23.670,0:30:27.189
systems are never connected to the[br]Internet. So it wouldn't have made any

0:30:27.189,0:30:32.299
difference. So these are the kind of[br]problems we're dealing with in this space.

0:30:32.299,0:30:35.271
And that's why I'm trying to encourage[br]you. Whether you do crypto or privacy or

0:30:35.271,0:30:37.640
whatever spend a little time in the[br]embedded space, just for bit: there's

0:30:37.640,0:30:46.130
plenty of easy work. OK. So what about the[br]second key. It requires a password. I

0:30:46.130,0:30:50.990
didn't feel like brute forcing it. Maybe[br]you do. I don't know. I tried all the

0:30:50.990,0:30:54.340
strings in the image. A classic technique,[br]just in case someone had a hard coded the

0:30:54.340,0:30:56.580
password. I mean the hard coded[br]credentials were there but not the hard

0:30:56.580,0:31:00.460
coded password. So I guess I gotta start[br]reversing, and as I previously said I suck

0:31:00.460,0:31:06.380
at reversing. That's why I come to CCC, so[br]I can learn, right? But I did find this

0:31:06.380,0:31:11.970
PowerPC ROM image. and I think its running[br]eCos and redboot and I haven't even gotten

0:31:11.970,0:31:15.330
down to doing hardware stuff: taking it[br]apart, having look at, it but I probably

0:31:15.330,0:31:19.200
will in the future. So there's the image[br]I'm slowly starting to learn my way around

0:31:19.200,0:31:27.140
and figure out what's going on. So I had a[br]look at the image and I figured out that

0:31:27.140,0:31:32.100
this key is used for SSH, right? Well it[br]would be the other encrypted thing. But I

0:31:32.100,0:31:36.261
couldn't enable SSH on the device. I try[br]and enable SSH on the device and I'm

0:31:36.261,0:31:39.100
logged in as manager by the way. which is[br]highest level user on this particular

0:31:39.100,0:31:43.580
device, and I put it in the passwords that[br]I know and a bunch of other passwords and

0:31:43.580,0:31:47.590
they don't work. Like I said, I tried all[br]the strings in the image. So apparently to

0:31:47.590,0:31:51.720
enable ssh, I need a password for[br]something. Now maybe I'm just

0:31:51.720,0:31:55.530
misunderstanding or I'm not so clear on[br]what's going on but I don't know about

0:31:55.530,0:31:59.070
you. I kind of feel like if I buy a device[br]that's supposed to be used for a safety

0:31:59.070,0:32:03.440
critical process I should be allowed to[br]use SSH without having to call up the

0:32:03.440,0:32:11.120
vendor and get some special magic[br]password. So considering I don't like that

0:32:11.120,0:32:17.420
approach. What if I patched my own key[br]into the image right. I don't know the

0:32:17.420,0:32:22.201
password of their key but I know the[br]password of a key I can generate. So I

0:32:22.201,0:32:27.290
just need to make sure it's roughly the[br]right size and try and patch it in. Then

0:32:27.290,0:32:29.600
I've got some problems with compression[br]because I've got to reverse the whole

0:32:29.600,0:32:33.570
process that I just described to you patch[br]it into the larger binary. Will there be

0:32:33.570,0:32:44.200
any CRC or firmware signing? I don't know,[br]right. So the uploaded image is not a

0:32:44.200,0:32:50.530
valid image for this device. That's[br]correct: I messed with it. But I got this

0:32:50.530,0:32:54.440
error and it gave me a clue. It gave me a[br]clue that I did indeed have some of my

0:32:54.440,0:33:02.410
CRCs wrong so when I altered the image[br]again I got to this state. So you're

0:33:02.410,0:33:05.510
learning all the time by having a real[br]device. Now some of my friends they do

0:33:05.510,0:33:10.051
static analysis and they don't buy these[br]devices. I decided to buy this one. I

0:33:10.051,0:33:15.750
found one on eBay. It wasn't very[br]expensive. I mean it depends on your range

0:33:15.750,0:33:20.179
for expensive. But if you're helping[br]defend industrial systems I thought it was

0:33:20.179,0:33:26.880
worth the money. So I bought it and this[br]enables me to try firmware images out and

0:33:26.880,0:33:31.210
I can slowly start to figure out what I[br]need to patch on these firmware images to

0:33:31.210,0:33:37.321
do whatever I want. Luckily I just tried[br]to patch mine to have SSH because I

0:33:37.321,0:33:43.799
thought people deserve to have SSH. So[br]that's an Adler 32 up there on the left

0:33:43.799,0:33:50.270
and the other CRC is on the bottom so that[br]Adler 32 and some adjustment of file

0:33:50.270,0:33:54.420
length although zeros in that line just[br]above it eventually got me to the point

0:33:54.420,0:33:59.570
where it believes it's a corrupted binary.[br]And then we have this CRC on the end that

0:33:59.570,0:34:08.210
we need to have a look at. Now I'm a big[br]fan of suspense. I love suspense. I'm

0:34:08.210,0:34:14.849
going to leave that one is a cliffhanger[br]and an exercise for you watching. So I

0:34:14.849,0:34:18.099
said I was going to talk about GE ML800[br]but I'm also going to talk about

0:34:18.099,0:34:21.219
Garrettcom. Luckily it's not very[br]difficult. Garrettcom is the original

0:34:21.219,0:34:27.480
equipment manufacturer for the GE ML800[br]series. I noticed that because the

0:34:27.480,0:34:31.299
certificate I found attached to those[br]private keys said Garrettcom in it and I

0:34:31.299,0:34:35.789
went and looked at their firmware images[br]and they have similar CRC similar file

0:34:35.789,0:34:39.710
structures similar everything so I believe[br]that they are affected by the cross site

0:34:39.710,0:34:45.929
scripting, the denial of service, and[br]hardcoded keys. I understand from some

0:34:45.929,0:34:50.530
people that they have been in contact with[br]GE to try and fix some of this stuff but

0:34:50.530,0:34:57.960
their response to GE was mainly "Sorry,[br]this is the end of life on this device".

0:34:57.960,0:35:02.890
That's fine. I understand you're running a[br]business but you're selling equipment to

0:35:02.890,0:35:08.339
people who manage utilities that we all[br]depend on. If Sony goes bankrupt because

0:35:08.339,0:35:13.799
they get hacked that's one thing right.[br]But you can't just dissolve a utility and

0:35:13.799,0:35:18.670
start again. As my friend Klaus points out[br]regularly – fantastic insights into the

0:35:18.670,0:35:23.150
industrial system world, Klaus and Vanessa[br]– you can't just dissolve the utility and

0:35:23.150,0:35:25.970
start again. You still have the same[br]infrastructure you still have the same

0:35:25.970,0:35:31.249
workers. It doesn't work that way. You[br]can't bail out utilities that we depend

0:35:31.249,0:35:38.329
on. So sorry. End of Life... I don't even[br]understand why people buy these devices

0:35:38.329,0:35:43.130
and this code without code escrow. When[br]you buy the code make sure you have the

0:35:43.130,0:35:48.700
code in perpetuity for these systems so[br]that you can fix them when something like

0:35:48.700,0:35:53.860
this or something worse happens. If I'm[br]your worst nightmare, you have real

0:35:53.860,0:35:59.190
problems because there are very dark[br]people in the world actually damaging

0:35:59.190,0:36:05.460
furnaces in Germany. So me disclosing keys[br]on stage is scary for you. You need to get

0:36:05.460,0:36:12.689
a grip. So, garrettcom?[br]Here's your key too.

0:36:12.689,0:36:20.104
<i>Applause</i>

0:36:20.104,0:36:25.629
The strings come from the images.[br]Developers are funny people really. I like

0:36:25.629,0:36:32.110
this. I just put them up because they're[br]funny. Some people had some hard times, I

0:36:32.110,0:36:36.490
guess, writing some of this code. And my[br]respect to them! They do great work but

0:36:36.490,0:36:43.159
you know, there's a couple of things we[br]can improve on security in these devices.

0:36:43.159,0:36:47.840
So I once had the opportunity to stand in[br]front of six different vendors at the same

0:36:47.840,0:36:53.440
time their computer emergency response[br]teams at a conference and I said to them,

0:36:53.440,0:36:59.659
"Will any of you commit to an average[br]patch time for vulnerabilities of three

0:36:59.659,0:37:05.350
months?" An average patch time, because it[br]might take 8 months, as it so far has

0:37:05.350,0:37:10.130
taken in the case of GE and Garrettcom, to[br]work on these issues. It might take a long[br]

0:37:10.130,0:37:15.050
time in some cases but as an average patch[br]time I think 3 months for things that we

0:37:15.050,0:37:20.440
all depend on is reasonable. So I asked[br]these six different teams in the same[br]

0:37:20.440,0:37:29.410
room. If any of them would commit to this[br]and I heard silence for 30 seconds. So my

0:37:29.410,0:37:35.220
friend decided to call this the silence of[br]the vendors right. And I think that's that

0:37:35.220,0:37:42.029
sums it up. I'd like to see better patch[br]times. I'd like to see a computer

0:37:42.029,0:37:45.200
emergency response teams in each of these[br]vendors and I'd like to see someone

0:37:45.200,0:37:53.600
responsible for security in each of these[br]different utilities. I can dream, right? I

0:37:53.600,0:37:57.369
think that key management... the current[br]practice industrial systems is to take

0:37:57.369,0:38:02.679
some insecure protocol and wrap it in SSL[br]or TLS which is why we need the help of

0:38:02.679,0:38:10.180
you privacy people because TLS and SSL[br]are not the be all and end all. They often

0:38:10.180,0:38:16.430
sort of go the wrong way, right. For[br]example you can use TLS to do integrity

0:38:16.430,0:38:20.679
without encryption so you can verify that[br]every message has reached its destination[br]

0:38:20.679,0:38:25.920
intact but it is not encrypted. And this[br]means that you can still do intrusion

0:38:25.920,0:38:32.530
detection analysis of the packets. That's[br]really good. But nobody uses that in SSL

0:38:32.530,0:38:36.669
in other ways right. I'm a big fan of[br]Shodan and use Shodan for a variety of

0:38:36.669,0:38:41.450
different things usually to get a sense of[br]the Internet as a whole, right? Let me

0:38:41.450,0:38:44.729
back up a little bit. When I was at[br]Cambridge I went to Darwin college and

0:38:44.729,0:38:47.690
because you're at Darwin college you read[br]up a bit on Darwin and you think about how

0:38:47.690,0:38:51.870
Darwin thought and I think the Internet is[br]kind of like that. When it was built by

0:38:51.870,0:38:56.870
the IETF and various people, who did[br]fantastic work, they imagined it one way

0:38:56.870,0:39:01.450
and then we inherited it and it grew and[br]it became an ecosystem and stuff happens

0:39:01.450,0:39:05.429
out there that you wouldn't expect. And so[br]that's why I like Shodan. It's kind of

0:39:05.429,0:39:09.869
like being a natural scientist: what's a[br]survey of the world, what kind of machines[br]

0:39:09.869,0:39:13.429
are out there, what versions are they[br]running, when do people update their SSL..[br]

0:39:13.429,0:39:17.559
err, you know, their certificates do they[br]do it before or after the certificate is[br]

0:39:17.559,0:39:22.600
invalid. Do they always upgrade the[br]algorithm. Do they increase the key size.[br]

0:39:22.600,0:39:26.380
You know how do things change right you[br]need to sort of study it as a whole and

0:39:26.380,0:39:30.440
that's my point when it comes to just[br]taking SSL and slapping it over a

0:39:30.440,0:39:37.759
protocol. It's not quite that simple. So[br]again we need your help. Where can we go[br]

0:39:37.759,0:39:42.289
with these attacks. And you remember at[br]the beginning I pointed out the underpants

0:39:42.289,0:39:49.770
gnome. The emperor wears no clothes.[br]Altering switch configurations is a big

0:39:49.770,0:39:57.410
deal because you can exfiltrate process[br]data. That gives you a map of the process

0:39:57.410,0:40:02.500
because industrial systems are bespoke.[br]Each one of them is different. It does run[br]

0:40:02.500,0:40:06.539
different traffic and we are lucky to work[br]on security in this space because our

0:40:06.539,0:40:10.880
users are numerate and literate and they[br]care about safety. They don't always

0:40:10.880,0:40:14.239
understand security but they do care about[br]safety. So if you can make it a safety

0:40:14.239,0:40:18.229
concern they care. There are also[br]engineers that many of these utilities who

0:40:18.229,0:40:24.219
look at the network 24/7. Not all of them[br]but some of them. Can you imagine a home

0:40:24.219,0:40:28.899
network or something else with that kind[br]of user base. We're lucky we should be

0:40:28.899,0:40:35.030
taking advantage of that user base. So[br]getting back to the point you know denial

0:40:35.030,0:40:38.979
of service attacks to disrupt the process[br]go and see Marmusha's talk. This will all

0:40:38.979,0:40:43.039
make a lot more sense when you go and see[br]her talk. Basically any man in the middle

0:40:43.039,0:40:47.990
attack can disrupt alter or drop traffic[br]at this point. If you can affect the

0:40:47.990,0:40:51.740
switches and the substation. And[br]exfiltrating in the data gives you a map

0:40:51.740,0:40:58.109
of the process which leads towards further[br]potential damage for the utilities. Now

0:40:58.109,0:41:01.410
it's not always that simple people will[br]get up on stage and they will tell you I

0:41:01.410,0:41:07.309
am awesome and this is how it's done and[br]it's easy to blow shit up. It's not true.

0:41:07.309,0:41:10.249
It takes a little bit of thought it takes[br]a little bit of work. I am certainly not

0:41:10.249,0:41:15.560
awesome. I am just a quality assurance[br]person from a former vendor. I just

0:41:15.560,0:41:22.509
decided to get into security and keep[br]going with it. So you can't always perform

0:41:22.509,0:41:25.389
these man in the middle attacks. People[br]will say you can. But the reason you can't

0:41:25.389,0:41:30.800
is real-time system constraints. Some[br]systems will stop receiving traffic five

0:41:30.800,0:41:34.539
milliseconds or microseconds later and[br]ignore anything. If a value doesn't arrive[br]

0:41:34.539,0:41:39.209
in this time it doesn't care. So the idea[br]that you can route the traffic out to some

0:41:39.209,0:41:43.590
other country and then back in and disrupt[br]the process is bollocks. Sometimes you

0:41:43.590,0:41:48.029
have to alter the firmware to achieve[br]that. That depends on the process but I'm[br]

0:41:48.029,0:41:52.830
just trying to give you a sense of how[br]performing actual attacks give you a sense

0:41:52.830,0:41:56.120
of what the limits are, what the[br]logistical burdens are for the attacker

0:41:56.120,0:42:04.940
and that's important stuff for us to know.[br]All right. Little bit of an overview.[br]

0:42:04.940,0:42:11.810
Drunk session IDs. brute forcing[br]MD5+NONCE, cross site request forgery for[br]

0:42:11.810,0:42:17.419
firmware upload (of all things),[br]reflected cross-site scripting (8 cases of

0:42:17.419,0:42:23.050
it) pre authentication denial of service,[br]hardcoded keys times 2 in a firmware

0:42:23.050,0:42:28.730
image, SSL without forward secrecy, self[br]signed certificates so there's no revoking

0:42:28.730,0:42:32.280
there's no managing of the keys on these[br]devices right. Not to mention utility

0:42:32.280,0:42:35.989
workers are busy already. They may not[br]have time to manage all of these devices

0:42:35.989,0:42:40.250
we might need to rethink that approach[br]right. Clear text passwords under SSL

0:42:40.250,0:42:44.049
because well no one can break SSL unless[br]you hard code the key in the firmware

0:42:44.049,0:42:49.539
that's downloadable from the internet.[br]Enable ssh with a password and three

0:42:49.539,0:42:55.289
quarter of a year waiting for fixes for[br]some of this stuff. I'm not happy with

0:42:55.289,0:43:00.699
that. I think that we could live in a much[br]better, much safer world. And to do so we

0:43:00.699,0:43:07.909
need to talk very seriously about some of[br]these issues. Don't take my opinion for

0:43:07.909,0:43:11.700
it. Listen to some other people. The best[br]thing about doing industrial systems work[br]

0:43:11.700,0:43:15.480
is the diversity of approach. You know I[br]love that there are so many other people[br]

0:43:15.480,0:43:20.080
doing SCADA and ICS. And I love that[br]they're going different directions. So in

0:43:20.080,0:43:26.030
the future I plan to be on another stage[br]with some friends and show you some more.

0:43:26.030,0:43:30.449
Thank you for listening mustache fans and[br]as a parting thought. More tax money is

0:43:30.449,0:43:35.349
spent on surveillance than on[br]defending common utilities.

0:43:35.349,0:43:44.394
<i>Applaus</i>

0:43:44.394,0:43:51.160
Herald: Thank you. It made me a scary[br]Sunday morning. They got a utility *&lt;&lt;

0:43:51.160,0:43:58.119
guess, mostly incomprehensable* down the[br]road. OK. We'll have some questions taken

0:43:58.119,0:44:06.459
please. As the session is recorded and[br]streamed anything you say, say it into a

0:44:06.459,0:44:17.049
mic. Any questions up? Wow, it is Sunday[br]morning.

0:44:17.049,0:44:18.029
Éireann: Number three, sure

0:44:18.029,0:44:21.280
Herald: everybody understood everything?[br]You're kidding me.

0:44:21.280,0:44:23.569
Éireann: I've got one right here[br]Herald: here is a question.

0:44:23.569,0:44:30.089
Question: Hey thanks I enjoyed your talk[br]and I think it's very important to raise

0:44:30.089,0:44:37.660
awareness. But I think it's not to raise[br]awareness. Not much in this community, but

0:44:37.660,0:44:43.880
within the engineering community and I see[br]it a lot of times and many engineers

0:44:43.880,0:44:49.730
having lots of problems doing that for[br]several reasons. There is maybe the

0:44:49.730,0:44:55.239
engineer who is thinking about this but[br]has its miniatures in the back has to deal

0:44:55.239,0:45:03.069
with service personnel which know how to[br]work a hammer and a screwdriver and on the

0:45:03.069,0:45:11.450
other side, engineers have to work with[br]customers which more those lazy people.

0:45:11.450,0:45:16.309
And so that's how these things happen. And[br]I think it's more important to raise

0:45:16.309,0:45:22.000
awareness of these kinds of things in the[br]engineering community.

0:45:22.000,0:45:24.730
Éireann: So just to repeat a little bit[br]for anybody else that couldn't hear it or

0:45:24.730,0:45:29.170
for the recording it's very important to[br]work with the engineers some of the

0:45:29.170,0:45:32.469
engineers understand the problem. But[br]typically management or lower level

0:45:32.469,0:45:37.680
service personnel don't always understand[br]the problem. And it's not important to

0:45:37.680,0:45:41.690
raise the awareness in the hacker[br]community. But more with the engineers is

0:45:41.690,0:45:46.299
what you were saying. Right. OK.[br]Absolutely true. Completely agree with

0:45:46.299,0:45:50.920
you. I don't just come to these[br]conferences and present to you guys. I go

0:45:50.920,0:45:54.430
and I present to the engineers too. And in[br]fact a couple of engineers have come to

0:45:54.430,0:45:58.599
this conference because we did work at[br]other conferences to see what the hacker

0:45:58.599,0:46:01.741
community is about and learn things from[br]the hacker community because this is a

0:46:01.741,0:46:05.360
place where you can learn if you're just[br]not afraid of getting pwned a couple of

0:46:05.360,0:46:10.999
times right. And it happens to me too[br]right. I learned a lot from getting

0:46:10.999,0:46:14.249
compromised on my machine and watching[br]someone do something. Anyways back to the

0:46:14.249,0:46:18.380
point I don't just work with engineers or[br]hackers. I also work with C-level

0:46:18.380,0:46:21.920
executives so I'm on a sabbatical from[br]IOActive at the moment. at the Cambridge

0:46:21.920,0:46:26.469
Center for Risk studies, and I'm working[br]with the insurance people which has its

0:46:26.469,0:46:31.441
challenges shall we say. But some of them[br]are very intelligent people and they want

0:46:31.441,0:46:34.670
to understand what's going on with hacking[br]attacks and they want to approach this

0:46:34.670,0:46:40.839
from a slightly different angle. My stake[br]in that is to be sure that when the

0:46:40.839,0:46:45.479
insurance people do get involved that they[br]actually ask for fixes and improve stuff.

0:46:45.479,0:46:49.809
So yes I do my best to raise awareness[br]wherever I can. And I'm not alone. You can

0:46:49.809,0:46:53.769
help me.[br]Questioner: Thank you

0:46:53.769,0:46:58.019
<i>applause</i>

0:46:58.019,0:47:05.570
Herald: OK, there's another question here.[br]Number two. Oh, and up there too, yes we

0:47:05.570,0:47:09.380
saw you. OK number two was first I think.[br]Go ahead

0:47:09.380,0:47:13.570
Question: <i>incomprehensible</i>. So you[br]mentioned a couple of things, err a couple

0:47:13.570,0:47:18.440
of vulnerabilities and I was wondering[br]what you would think an ideal system would

0:47:18.440,0:47:24.150
look like. You mentioned key provisioning[br]of course putting certificates. I assume

0:47:24.150,0:47:28.470
that they were different certificates for[br]different devices rather than the same

0:47:28.470,0:47:37.430
certificate for all devices. Okay that's a bad[br]thing. And and also sort of the way how

0:47:37.430,0:47:44.839
the software update management works. So[br]how would you if you could give them some

0:47:44.839,0:47:48.950
advice how to design a system[br]how would you do it?

0:47:48.950,0:47:55.420
Éireann: Okay. So first of all I wouldn't[br]hard code the keys as you as you discussed

0:47:55.420,0:48:01.859
to be in every device the same. It's one[br]thing to put in your documentation hey you

0:48:01.859,0:48:07.630
should update the keys but I mean if I can[br]patch binary file with a key then there's

0:48:07.630,0:48:11.089
no reason you couldn't do that on the[br]website where you download the firmware

0:48:11.089,0:48:15.160
image right. Just as an example as a[br]thought experiment sort of makes that

0:48:15.160,0:48:18.420
clear. The upgrade path for these devices[br]is download the firmware image from the

0:48:18.420,0:48:25.280
website to some machine and then carry it,[br]because all these systems are airgapped.

0:48:25.280,0:48:29.229
to some other location and then upload it[br]onto the switch right with hardcoded

0:48:29.229,0:48:33.869
credentials. So first off whenever you[br]provision a switch initially you provision

0:48:33.869,0:48:36.920
all of the credentials for that device.[br]That's standard practice of many routers

0:48:36.920,0:48:41.900
and other pieces of equipment today. And I[br]would think less about defending and

0:48:41.900,0:48:46.230
securing the device than on being[br]able to regularly check its integrity,

0:48:46.230,0:48:48.539
the integrity of the firmware that is[br]running and the integrity of the

0:48:48.539,0:48:54.289
configuration. So I'd focus on that and I'd[br]focus on being able to recover the switch

0:48:54.289,0:48:57.740
after it's been attacked. So you reverse[br]your thinking. You assume that one day

0:48:57.740,0:49:01.309
someone is going to crack your firmware[br]signing and crack this and crack that and

0:49:01.309,0:49:05.930
you focus on how can I quickly upload a[br]new firmware image that is known to be

0:49:05.930,0:49:12.250
good and verify that the one that is[br]uploaded is good to this device.

0:49:12.250,0:49:16.059
Questioner: Thank you.[br]Herald: There was a question up there on

0:49:16.059,0:49:18.769
the balcony.[br]Signal angel: Yes we have two questions

0:49:18.769,0:49:25.549
here on the net. So the first one is how[br]would you solve the end of life issue.

0:49:25.549,0:49:29.900
Sometimes <i>incomprehensible</i> clients just[br]gets really outdated.

0:49:29.900,0:49:33.420
Éireann: That's absolutely true and it is[br]slightly unfair of me to be a hard on the

0:49:33.420,0:49:38.349
vendors. But it's my job to take the[br]debate a little bit too far the other way.

0:49:38.349,0:49:43.229
So how would I solve the end of life issue[br]is the question from the internet. I don't

0:49:43.229,0:49:47.759
know. I think that's not a technical[br]problem it's a societal problem. Like when

0:49:47.759,0:49:55.970
we buy bridges they are bridges until they[br]fall down. When we buy roads they stay

0:49:55.970,0:49:59.130
there until they go away. I mean there is[br]probably some end of life issues in there

0:49:59.130,0:50:04.960
but it's almost more of a contractual[br]legal issue and someone should study that.

0:50:04.960,0:50:08.339
There are people studying that but it's[br]not my area of expertise but I'll try and

0:50:08.339,0:50:12.969
answer as best I can. I think code escrow[br]is a good way to go when you buy some of

0:50:12.969,0:50:18.079
these devices you say I want the code for[br]this device in the future. I want to have

0:50:18.079,0:50:22.369
access to it. If your company goes[br]bankrupt I need you to give up the source

0:50:22.369,0:50:26.329
code for these devices when you go[br]bankrupt or when you disappear or when

0:50:26.329,0:50:30.380
it's the end of life. There are a couple[br]of manufacturers out there doing open

0:50:30.380,0:50:35.200
source switches. There's a company called[br]Open gear who are awesome. They gave me a

0:50:35.200,0:50:39.790
switch to play with that I haven't had[br]time to look at yet. I think that's amazing

0:50:39.790,0:50:42.700
right. And their code is open source and[br]you can go and examine it. So you would

0:50:42.700,0:50:46.309
have the code anyway. Those are two[br]different approaches. I think there are

0:50:46.309,0:50:49.979
others you can solve this problem[br]technically or legally or socially but as

0:50:49.979,0:50:55.869
a society we depend on these utilities and[br]that code should not just vanish when it's

0:50:55.869,0:51:05.369
difficult or costly to keep it upgraded.[br]<i>applause</i>

0:51:05.369,0:51:08.089
Herald: There was a second[br]question from the Internet.

0:51:08.089,0:51:14.179
Signal angel: Yes, so the second one is:[br]what should a non-technical person in

0:51:14.179,0:51:19.890
the respect of <i>incomprehensible</i> set non-[br]technical person sent to manage small town

0:51:19.890,0:51:25.440
utility do as best practice?[br]Éireann: I think the first and most

0:51:25.440,0:51:29.930
important thing is to look for attacks.[br]I'm sorry I should probably repeat that

0:51:29.930,0:51:33.609
question just to be sure. What should[br]someone in a small town who manages

0:51:33.609,0:51:37.420
utility do to defend themselves and[br]protect himself. So the first thing is

0:51:37.420,0:51:43.129
look for attacks. Even if you spend a few[br]hours a week looking for something you

0:51:43.129,0:51:46.249
script something up or you hire some[br]college kid to come in and script

0:51:46.249,0:51:49.579
something and look for things on your[br]network and ask questions and yes they're

0:51:49.579,0:51:52.279
going to be a pain in the ass and is going[br]to be difficult. But you're going to learn

0:51:52.279,0:51:55.599
things about your network and you might[br]detect some attacks. The first problem in

0:51:55.599,0:52:01.059
utilities is no one is responsible for[br]security. It's not my job. It's kind of

0:52:01.059,0:52:05.480
the mantra so for a small utility find[br]someone whose job it is if you're a very

0:52:05.480,0:52:09.130
small utility there's probably some other[br]small utilities near you and you can hire

0:52:09.130,0:52:13.789
a resource together to come and visit your[br]different utilities and help you out. The

0:52:13.789,0:52:17.380
second one is watch your relationship with[br]your vendor when you purchase this

0:52:17.380,0:52:21.220
equipment you spend a lot of money on it.[br]Spend a little bit of time doing

0:52:21.220,0:52:25.069
penetration tests. Yes I like it when you[br]hire me but you don't have to hire me.

0:52:25.069,0:52:28.071
There are plenty of other people you can[br]hire who will have a look at the device

0:52:28.071,0:52:31.770
and find the simple vulnerabilities. So[br]when you purchase something make sure you

0:52:31.770,0:52:35.469
test it for security purposes and that's[br]very important because you can even put

0:52:35.469,0:52:40.879
into your contract if you fail the[br]security tests we will pay you less money.

0:52:40.879,0:52:44.480
And the vendors are not going to react[br]to security until you do that. So that's

0:52:44.480,0:52:51.429
the second answer. And I wish I had a[br]third to make it very neat but I don't.

0:52:51.429,0:52:55.729
Herald: OK. There was one more[br]question at mic 4 I think

0:52:55.729,0:52:58.500
Questioner: Yes hi thank you for[br]your time.

0:52:58.500,0:53:03.539
Herald: Talk into the mike please. Thank[br]you for your talk. Q Hi. I'm kind of a

0:53:03.539,0:53:12.739
newbie to the C3 community and I am not[br]sure about the question I want to ask you.

0:53:12.739,0:53:16.579
Probably many people understand in this[br]room but I don't know if I would like to

0:53:16.579,0:53:23.780
ask you what exactly do you[br]mean by arbitrary firmware.

0:53:23.780,0:53:28.799
Éireann: No problem. So the question was[br]What do you mean by arbitrary firmware. I

0:53:28.799,0:53:34.349
mean the firmware that I have altered that[br]was not manufactured by the vendor to do

0:53:34.349,0:53:39.230
whatever I want. How do you trust that[br]this switch sends all the packets that it

0:53:39.230,0:53:45.049
should send. What if it's, you know, my[br]handle is BSB right. What if it drops

0:53:45.049,0:53:51.230
every packet that has BSB in the packet.[br]Right. You can rewrite a firmware image to

0:53:51.230,0:53:54.950
do whatever the device can do and in some[br]cases more things than the device usually

0:53:54.950,0:53:59.959
does to damage itself for example. So an[br]arbitrary firmware is one in which anyone

0:53:59.959,0:54:03.489
writes the firmware and there is no[br]checking to be sure that this is the image

0:54:03.489,0:54:08.490
that you want on this device whether it's[br]provided by the vendor or the community

0:54:08.490,0:54:13.239
right. You still want checking that this[br]is the correct code or the code that you

0:54:13.239,0:54:18.309
wanted anyway. Right.[br]Herald: Okay thank you. Is that a question

0:54:18.309,0:54:22.489
here mic 1? OK go ahead.[br]Questioner: Yes please. In your

0:54:22.489,0:54:29.739
hypothetical question, you asked what[br]damage could I do in that paint factory.

0:54:29.739,0:54:39.690
But you can also reverse it. What kind of[br]company secrets can I obtain for example,

0:54:39.690,0:54:45.859
your favorite recipe for your hot[br]chocolate or the recipes of Coca-Cola.

0:54:45.859,0:54:52.839
They are vulnerable as well aren't they.[br]Éireann: Yes. So the question just again

0:54:52.839,0:54:56.559
for everyone else. You don't just have to[br]talk about damage in a paint factory or

0:54:56.559,0:55:01.819
any industrial system. You can also talk[br]about intellectual property and protecting

0:55:01.819,0:55:07.309
the recipes that we use to bake cookies or[br]make beer or whatever pharmaceuticals

0:55:07.309,0:55:12.641
whatever. And that's a fantastic question[br]and I'm glad you brought it up a couple of

0:55:12.641,0:55:15.809
years ago when I was doing... well, more[br]than a couple of years like eight years

0:55:15.809,0:55:19.249
ago, when I was doing industrial system[br]security I realized I wasn't getting a lot

0:55:19.249,0:55:23.489
of traction. It was before stuxnet, I was[br]a quality assurance guy. Everybody thought

0:55:23.489,0:55:34.309
I was fucking crazy right. Stuxnet,[br]career. It's wrong. It's really wrong. But

0:55:34.309,0:55:39.579
the point is I tried to take that[br]approach. I tried to say you have a

0:55:39.579,0:55:43.019
process in which you manufacture something[br]and you make money by the fact that that

0:55:43.019,0:55:47.979
process is relatively secret and if you[br]don't care about defending your workers

0:55:47.979,0:55:52.589
from being damaged then at least care[br]about the intellectual property because

0:55:52.589,0:55:56.059
I'll get security in by some sort of back[br]door right. I'm a little bit of a security

0:55:56.059,0:56:00.200
Machiavellian. I'll find a way to get[br]security into the system somehow. So I

0:56:00.200,0:56:05.349
tried to say intellectual property you[br]should be protected. And I found that they

0:56:05.349,0:56:09.320
didn't care so much. I mean maybe you'll[br]have more luck maybe post-stuxnet that

0:56:09.320,0:56:14.069
that's a better argument. I hope you do.[br]But it is an important question as well.

0:56:14.069,0:56:18.719
Right. It's not, it's not just potential[br]for damage. I think there's a lot more

0:56:18.719,0:56:25.459
espionage going on on these networks than[br]there is damage and sabotage. Herald: Okay

0:56:25.459,0:56:32.069
we'll take one more question on mike four.[br]Questioner: Thank you okay. My question

0:56:32.069,0:56:38.319
concerns the concepts of software defined[br]networking and open flow. So when I first

0:56:38.319,0:56:44.880
heard about software defined networking I[br]thought well this is a huge security issue

0:56:44.880,0:56:50.589
and there may be huge vulnerabilities.[br]After your joke I think this might

0:56:50.589,0:56:56.420
actually be a good idea to dumb down the[br]switches and put the intelligence

0:56:56.420,0:57:01.900
somewhere locked up in a safe place.[br]What's your opinion on that. Can they

0:57:01.900,0:57:05.839
actually improve security.[br]Éireann: Yes. So the question is what role

0:57:05.839,0:57:09.969
could software defined networking play in[br]these sorts of environments. And is it a

0:57:09.969,0:57:15.210
good idea from a security perspective.[br]Anytime someone has a revolution in

0:57:15.210,0:57:19.240
computing we also have to update our[br]security paradigm. So I think with

0:57:19.240,0:57:23.039
software defined networking it's not[br]whether it's good or bad it's that you

0:57:23.039,0:57:28.339
defend that network differently than you[br]defend one of these networks. So it's not

0:57:28.339,0:57:31.400
so much that as good as good or bad it's[br]neutral if you know how to defend your

0:57:31.400,0:57:34.779
network. I don't care what it is. As long[br]as someone is looking to defend it and

0:57:34.779,0:57:38.989
cares about how the flows are working. So[br]I think software defined networking in

0:57:38.989,0:57:42.449
these environments could be a very good[br]thing but the refresh rate on these

0:57:42.449,0:57:45.799
devices is not that high. So I don't think[br]we'll see it there for a little while even

0:57:45.799,0:57:50.859
though it might be a good thing[br]philosophically. It takes 5 10 15 20 years

0:57:50.859,0:57:56.410
to refresh these networks so it'll be a little[br]while. But it's not good or bad. It's just

0:57:56.410,0:57:59.909
learn to defend what you got is the[br]problem right.

0:57:59.909,0:58:06.489
Questioner: Okay thanks a lot.[br]Herald: Okay okay let's give a big hand

0:58:06.489,0:58:09.639
for Éireann and thank you.[br]Éireann: Thank you

0:58:09.639,0:58:13.320
<i>applause</i>

0:58:13.320,0:58:24.000
subtitles created by c3subtitles.de[br]Join, and help us!