0:00:00.000,0:00:19.759
35c3 prerol music
0:00:19.759,0:00:26.630
Herald: So Trammell Hudson, who is[br]standing here, he's taking things apart.
0:00:26.630,0:00:34.370
Don't worry not life on stage, but he will[br]give us a proof of concept and some
0:00:34.370,0:00:39.559
details and functionalities about hardware[br]implants. So the same things that we heard
0:00:39.559,0:00:45.480
from Bloomberg article talking about Apple[br]and super microcomputers with implants
0:00:45.480,0:00:52.879
that, yeah, were implanted into those,[br]into those computers. And I'm really
0:00:52.879,0:00:57.680
excited to see this in action. Please give[br]a warm round of applause to Trammel
0:00:57.680,0:01:02.590
Hudson!
0:01:02.590,0:01:07.510
applause
0:01:07.510,0:01:11.600
Trammell: Before we begin talking about[br]hardware implants just two quick
0:01:11.600,0:01:16.310
disclaimers. The first from my employer[br]Two Sigma investments as it says are
0:01:16.310,0:01:21.910
chocolate bars. This is not investment[br]advice. And secondly I don't actually know
0:01:21.910,0:01:26.920
what the story is behind the super micro[br]story. No one outside of Bloomberg and
0:01:26.920,0:01:32.450
their sources do. But I have spent a lot[br]of time thinking about hardware implants
0:01:32.450,0:01:38.200
starting with the thunderstrike firmware[br]attack against mac books as well as the
0:01:38.200,0:01:45.420
thunderstrike 2 where we were able to get[br]software to write into the firmware on the
0:01:45.420,0:01:50.560
mac books. I've also been thinking a lot[br]about how to defend against hardware
0:01:50.560,0:01:54.420
implants with things like the heads[br]firmware for slightly more secure laptops
0:01:54.420,0:01:59.420
and also as part of my co-lead on the[br]Linux boot project. We're thinking about
0:01:59.420,0:02:10.080
how to protect servers from physical and[br]software attacks. So with all of this
0:02:10.080,0:02:14.910
concentrated thinking about firmware and[br]hardware attacks, I was really excited
0:02:14.910,0:02:20.720
when I saw the Bloomberg story back in[br]October. But what really intrigued me was
0:02:20.720,0:02:26.440
the animated image that they had at the[br]header that highlighted one small part of
0:02:26.440,0:02:32.920
the board as where the implant was, but[br]what I found really interesting is that is
0:02:32.920,0:02:40.250
exactly where I would install a hardware[br]implant as they described on the SPI bus.
0:02:40.250,0:02:44.610
A lot of other people in the hardware and[br]from our security community thought it
0:02:44.610,0:02:50.140
sounded plausible. Other people pointed[br]out that supply chain attacks come up
0:02:50.140,0:02:56.070
periodically and they are definitely a[br]concern. Some people thought the attack as
0:02:56.070,0:03:03.320
described was entirely implausible and in[br]general we sort of had a Whiskey Tango
0:03:03.320,0:03:08.160
Foxtrot moment as everybody scrambled to[br]figure out what's going on inside their
0:03:08.160,0:03:14.540
machines. So, let's step back very quickly[br]and review what the key claims that
0:03:14.540,0:03:22.340
Bloomberg alleged happened. First they[br]said that Amazon's testers found a tiny
0:03:22.340,0:03:27.250
microchip that wasn't part of the board's[br]original design that had been disguised to
0:03:27.250,0:03:33.350
look like a signaling condition signal[br]condition coupler and that these illicit
0:03:33.350,0:03:39.790
chips were connected to the baseboard[br]management controller or the BMC which
0:03:39.790,0:03:44.210
gave them access to machines that were[br]turned off. That might sound kind of
0:03:44.210,0:03:49.870
extreme, but that's actually what the role[br]of the BMC is, that in most servers the
0:03:49.870,0:03:55.280
BMC is running any time the machine is[br]hooked up to power and it's connected to
0:03:55.280,0:04:01.910
the power supplies so that it can turn the[br]machine on and turn it off. Frequently you
0:04:01.910,0:04:06.780
want to be able to do this over a network[br]so it has its own dedicated LAN port but
0:04:06.780,0:04:14.180
it can also share the LAN port with the[br]with the main system. Serial over LAN is a
0:04:14.180,0:04:19.180
really useful way to debug the systems so[br]it provides that functionality. It can
0:04:19.180,0:04:27.350
also provide fake USB volumes to allow to[br]to do unintended OS installation. A lot of
0:04:27.350,0:04:33.430
sites also won't remote KVM so it has VGA[br]but that VGA support means that it's on
0:04:33.430,0:04:40.370
the PCIe BUS and because some PCIe it can[br]do DMA into main memory. It also is
0:04:40.370,0:04:47.000
typically muxed into the SPI flash for[br]the host firmware, which allows it to
0:04:47.000,0:04:51.820
modify it and on some systems it's even[br]connected to the TPM which allows it to
0:04:51.820,0:04:59.930
circumvent the corporate of trust. So with[br]all of this capability inside this chip
0:04:59.930,0:05:06.919
it's really unfortunate that they are[br]really not well put together. The head of
0:05:06.919,0:05:11.150
Azure security says they have no[br]protection against attacks. There's no
0:05:11.150,0:05:15.530
ability to detect if an attack has[br]happened and there's no ability to recover
0:05:15.530,0:05:22.449
from an attack. So having a hardware[br]implant on the BMC is a really big
0:05:22.449,0:05:32.030
concern. The other claim in the article is[br]that it affected 30 different companies
0:05:32.030,0:05:39.930
including Apple and Bloomberg alleges that[br]Apple found malicious chips independently
0:05:39.930,0:05:44.980
on their super micro boards. Went to the[br]FBI about it and that they then severed
0:05:44.980,0:05:52.100
ties with Super Micro. This particular[br]claim was interesting because it
0:05:52.100,0:05:57.570
corroborated a story that had shown up[br]back in early 2017 that Apple had removed
0:05:57.570,0:06:03.050
Super Micro from their data centers. Apple[br]denied that there was a firmware issue.
0:06:03.050,0:06:10.190
But it's interesting that perhaps these[br]two were related. The third set of claims
0:06:10.190,0:06:16.090
is that on some of these implants they[br]were actually put between the layers on
0:06:16.090,0:06:23.210
the PCB and then the most explosive claim[br]is that this was done by operatives from
0:06:23.210,0:06:33.580
China, the Chinese People's Liberation[br]Army. With a story with this you know this
0:06:33.580,0:06:39.389
many claims and this significant of[br]allegations we'd hoped that it would be
0:06:39.389,0:06:45.430
really well sourced and for a normal story[br]17 independent sources that Bloomberg
0:06:45.430,0:06:52.490
editors agreed to grant anonymity to,[br]including six national security, two
0:06:52.490,0:06:57.340
people inside of AWS and three senior[br]insiders at Apple seems like pretty solid
0:06:57.340,0:07:03.110
sourcing, except as soon as this article[br]is published everyone denied it. The
0:07:03.110,0:07:09.080
Director of National Intelligence said[br]they'd seen no evidence of this. Amazon
0:07:09.080,0:07:13.990
said that they've never found any issues[br]of modified hardware nor have they been
0:07:13.990,0:07:21.000
engaged with the government over it. Apple[br]was even more blunt. CEO Tim Cook said
0:07:21.000,0:07:27.590
this did not happen. There is no truth to[br]this. And Super Micro wrote a fairly
0:07:27.590,0:07:32.150
lengthy letter about what they do to[br]protect their supply chain and why they
0:07:32.150,0:07:38.990
think this attack did not happen. And it[br]is worth going through to look at some of
0:07:38.990,0:07:44.880
the things that they say that they do to[br]protect their supply chain. They point out
0:07:44.880,0:07:50.700
that if there's any unauthorized physical[br]alterations during the manufacturing
0:07:50.700,0:07:56.949
process other design elements would not[br]match and those things would be detected.
0:07:56.949,0:08:03.300
To sort of understand how circuit boards[br]are made, I recently visited a PCB factory
0:08:03.300,0:08:11.080
in Guangzhou. This is not a super micro[br]factory. This is just a holiday photos. So
0:08:11.080,0:08:16.760
in order to add new vias they would have[br]to modify the drill files which would then
0:08:16.760,0:08:22.050
get electroplated. If they had to add new[br]traces, they would have to be able to
0:08:22.050,0:08:29.400
subvert the masking and etching process[br]and any changes to either the drills or
0:08:29.400,0:08:34.909
the etching on individual layers would be[br]caught by the optical inspection that's
0:08:34.909,0:08:41.479
done on these bare circuit boards.[br]Additionally the allegation that things
0:08:41.479,0:08:47.110
were inserted between circuit boards would[br]require that the lamination process be
0:08:47.110,0:08:55.329
subverted and that the implant somehow[br]aligned into the system. If that implant
0:08:55.329,0:09:00.410
changes any of the connectivity the flying[br]protesters would pick it up or the bed of
0:09:00.410,0:09:05.980
nails testers which checks all of the[br]connectivity of all the traces to make
0:09:05.980,0:09:09.300
sure that there are no shorts and to make[br]sure that everything that is supposed to
0:09:09.300,0:09:16.679
be connected is electrically conductive.[br]So it would be very difficult to
0:09:16.679,0:09:22.110
circumvent the production process at this[br]stage. And it also would be very difficult
0:09:22.110,0:09:27.709
to contain because the PCB factory doesn't[br]know which customers are going to receive
0:09:27.709,0:09:34.470
those circuit boards. Super Micro also[br]points out that during the assembly
0:09:34.470,0:09:40.480
process when the parts are installed they[br]have their employees on site the whole
0:09:40.480,0:09:47.559
time. On my same holiday trip I also[br]visited some PCB assembly companies and
0:09:47.559,0:09:53.589
spoke with companies that are using doing[br]contract manufacturing and they said that
0:09:53.589,0:09:59.089
they also send their employees to the[br]production line to observe the pick and
0:09:59.089,0:10:05.600
place machines and the reflow and the rest[br]of the surface mount assembly. Their big
0:10:05.600,0:10:10.089
concern is that if they don't have someone[br]there the parts that are fed in the pick
0:10:10.089,0:10:17.660
in place will be replaced with either[br]counterfeits or with salvaged parts. I
0:10:17.660,0:10:23.459
visited the electronics market in ???????[br]bay where there are people desoldering
0:10:23.459,0:10:29.190
e-waste and then sorting the parts into[br]bins and selling these salvaged components
0:10:29.190,0:10:34.589
by the kilo and for a few extra renminbi[br]they'll put them on rails for you so that
0:10:34.589,0:10:41.660
you can save a few pennies on your[br]production process. The other concern that
0:10:41.660,0:10:46.489
these companies have, is not just salvaged[br]parts but straight up counterfeits.
0:10:46.489,0:10:52.439
Especially for things that cost more than[br]a few dollars each. The Arduino community
0:10:52.439,0:10:59.139
was hit a few years ago with a bunch of[br]counterfeit FTDI chips where the internal
0:10:59.139,0:11:07.600
construction was entirely different. In[br]this case it caused reliability issues but
0:11:07.600,0:11:11.550
you can imagine from a security[br]perspective this is really worrisome that
0:11:11.550,0:11:15.709
parts that look identical might have[br]completely different functionality inside
0:11:15.709,0:11:25.379
of them. Super Micro also mentions that[br]they X-ray their main boards to look for
0:11:25.379,0:11:32.000
anomalies and I wasn't able to take any[br]photos inside the factory there was doing
0:11:32.000,0:11:38.230
x-rays. But in this Wikipedia photo we can[br]clearly see active components like this
0:11:38.230,0:11:45.670
SOIC chip are different from things like[br]the SMD resistors and capacitors. So if an
0:11:45.670,0:11:51.220
attacker were trying to subvert the supply[br]chain by putting a disguise component it
0:11:51.220,0:11:56.670
could be detected at this step. Another[br]interesting thing in this photo are these
0:11:56.670,0:12:02.680
inductors that are encased in dip[br]packages. This is really common in a lot
0:12:02.680,0:12:07.439
of Ethernet boards and occasionally people[br]have thought they had some sort of
0:12:07.439,0:12:13.589
hardware implant when they found inductors[br]in their ethernet jacks but it's pretty
0:12:13.589,0:12:19.799
it's fairly common and it shows it pretty[br]clearly on the x-ray. Some other security
0:12:19.799,0:12:26.069
researchers like Sophia D'Antoine did an[br]extensive teardown of Super Micro boards
0:12:26.069,0:12:33.439
including X-ray analysis and her group[br]found a few oddities but nothing.. they
0:12:33.439,0:12:37.529
didn't find anything malicious. There were[br]no smoking guns. They just appeared to be
0:12:37.529,0:12:43.190
sort of supply chain type things. You can[br]read her blog post for more details about
0:12:43.190,0:12:49.319
where they found things that shouldn't[br]have been there. But turned out to be just
0:12:49.319,0:13:00.879
actual signal condition components. So[br]super micro in their ???? letter, they
0:13:00.879,0:13:07.239
keep reenforcing that the manufacturing[br]process that is the assembly process, it's
0:13:07.239,0:13:11.179
during the manufacturing process and I[br]agree with them. It would be very
0:13:11.179,0:13:17.939
difficult to circumvent security in a[br]reasonable way in that part of the
0:13:17.939,0:13:23.189
process. But that's not the only place[br]this could happen. We know that national
0:13:23.189,0:13:30.309
security agencies intercept shipments of[br]computer hardware and then have their
0:13:30.309,0:13:37.249
tailored access operations open the[br]computers, install hardware implants,
0:13:37.249,0:13:43.670
reseal them and then have them continue on[br]their way in shipment. The NSA even has a
0:13:43.670,0:13:51.199
catalog of hardware implants like this[br]JTAG implant Ethernet jacks with embedded
0:13:51.199,0:13:57.009
computers in them as well as firmware[br]specific ones that target servers SNM(?)
0:13:57.009,0:14:05.490
and then some that can do data[br]exfiltration via RF. So that's sort of
0:14:05.490,0:14:09.481
tailored access operations is really ideal[br]for this supply chain attack because it
0:14:09.481,0:14:16.699
allows them to contain the exploit to a[br]single customer. It allows them fairly
0:14:16.699,0:14:21.180
good concealment as well as good cover[br]that if it's discovered it's really hard
0:14:21.180,0:14:25.769
to attribute where things went wrong. Now[br]unlike if you find something inside your
0:14:25.769,0:14:34.230
motherboard between the layers you know[br]that had to have happened at the factory.
0:14:34.230,0:14:47.040
So Super Micro also claim that this was[br]technically implausible, that it was
0:14:47.040,0:14:52.559
highly unlikely that unauthorized hardware[br]would function properly because a third
0:14:52.559,0:15:02.470
party with lack of complete knowledge of[br]the design. I think that's inaccurate,
0:15:02.470,0:15:07.639
both because we know the NSA does it and[br]also because I have done it.
0:15:07.639,0:15:10.319
laughter
0:15:10.319,0:15:16.059
Really, all that you need to know is that[br]these are common components. These flash
0:15:16.059,0:15:20.310
chips show up on all the boards. You can[br]search the internet for the data sheet and
0:15:20.310,0:15:25.989
find exactly how it's wired into the rest[br]of the system. And the only thing that we
0:15:25.989,0:15:33.499
need to know to communicate to the BMC is[br]the serial output pin from this component,
0:15:33.499,0:15:43.429
so the BMC flash is connected over to the[br]BMC CPU via the serial output and it goes
0:15:43.429,0:15:51.589
through a small series resistor and that[br]is where my implant goes in. Mine's a
0:15:51.589,0:15:56.670
little bit larger than that resistor. It[br]clicks onto the board and it has a small
0:15:56.670,0:16:03.009
FPGA that hangs offside but it's[br]completely plausible to fit it into
0:16:03.009,0:16:12.139
something that small in fact a modern ARM[br]M0 fits in the space of two transistors
0:16:12.139,0:16:18.350
from a 65 002 from a few years ago. The[br]Moore's Law means we can pack an amazing
0:16:18.350,0:16:28.329
amount of CPU into a very very small[br]amount of space. So on that 0 6 0 3
0:16:28.329,0:16:36.100
resistor could fit around 100 cortex M0 it[br]would be plenty powerful for this system.
0:16:36.100,0:16:42.379
The problem is we only have those two pins[br]so ordinarily on the spy flashing you need
0:16:42.379,0:16:47.720
at least six pens but we don't have power[br]and ground so we have to passively power
0:16:47.720,0:16:53.059
this through the data signal that's[br]passing through it. We don't have the chip
0:16:53.059,0:16:59.959
select pin so we have to guess when this[br]chip has been talked to. We don't have the
0:16:59.959,0:17:04.980
data input pin so we don't know what[br]addresses are being read or what commands
0:17:04.980,0:17:11.190
are being sent. We have to reconstruct it[br]from the data output pin and we also don't
0:17:11.190,0:17:18.900
have a clock pin so we have to figure out[br]how to synchronize to that clock. Lastly
0:17:18.900,0:17:22.890
we don't have the ability to make[br]arbitrary data changes. All we can do is
0:17:22.890,0:17:29.060
disconnect the pin from the BMC so we can[br]only turn 1 bits into 0 bits. We can't go
0:17:29.060,0:17:35.300
the other way around. So with these[br]limitations we can still do some pretty
0:17:35.300,0:17:40.920
interesting things. Recovering the clock[br]is actually pretty easy. We can look at
0:17:40.920,0:17:49.670
the data stream and find the shortest bit[br]transitions from 0 1 0 or 1 0 1 to
0:17:49.670,0:17:55.060
estimate what the clock is which allows us[br]to then reconstruct that data stream being
0:17:55.060,0:18:00.870
sent to the BMC and if we look at the[br]flash contents we can see that a lot of it
0:18:00.870,0:18:07.570
is being fairly random noise but a lot of[br]it is all white which in this case would
0:18:07.570,0:18:15.110
mean that it's all one bits. So if we look[br]at the way the flash is organized we can
0:18:15.110,0:18:19.380
see there's the u-boot bootloader and[br]that's executable. That's kind of
0:18:19.380,0:18:25.230
difficult to make useful changes in, the[br]kernel and the root file system are both
0:18:25.230,0:18:33.040
compressed so that they look effectively[br]like random noise but the nvram region is
0:18:33.040,0:18:41.660
a jffs2 file system and this file system[br]??? 3 Megs, it's mostly empty and all that
0:18:41.660,0:18:50.040
empty space is F F which is all ones. So[br]this is plenty of ones for us to work on.
0:18:50.040,0:18:55.380
Additionally it has fairly nice headers[br]that we can we can match on. So when we
0:18:55.380,0:19:00.570
see these magic bit masks we know when[br]we've entered different parts of the file
0:19:00.570,0:19:06.990
system. So given that we can now[br]reconstruct the clock we can figure out
0:19:06.990,0:19:13.310
where we are in the file system. This[br]hardware implant can start to inject new
0:19:13.310,0:19:20.320
data into what was the empty space. So[br]this short file that we put in here is a
0:19:20.320,0:19:28.020
small shell script and it is one of the[br]network configuration scripts, so this is
0:19:28.020,0:19:37.350
where I'm going to try a live demo and I[br]hope this works. We're running in qemu
0:19:37.350,0:19:45.660
since I didn't bring a Super Micro board[br]and what we have on the left is the flash
0:19:45.660,0:19:50.530
console excuse me the hardware implant[br]console. And then on the right we have the
0:19:50.530,0:19:57.353
serial console from the BMC so we can see[br]it has loaded the kernel and in a second
0:19:57.353,0:20:03.430
it's going to we should see a bunch of[br]traffic, okay, so the implant is active.
0:20:03.430,0:20:10.450
It has replaced the data when that nvram[br]file system was mounted the BMC is now
0:20:10.450,0:20:18.780
continuing on doing its set up. It's going[br]to load a bunch of device drivers for that
0:20:18.780,0:20:24.250
video. It pauses here for some reason that[br]I haven't diagnosed because that's that's
0:20:24.250,0:20:27.040
not my job.
0:20:27.040,0:20:29.140
laughter
0:20:29.140,0:20:33.020
And eventually it's going to configure the[br]networks and it does that by running that
0:20:33.020,0:20:43.010
shell script off of the nvram partition[br]here it starts KVM stuff brings up some
0:20:43.010,0:20:53.390
things. Allright.[br]applause
0:20:53.390,0:21:01.920
OK. So luckily we got to that point[br]without having to fake the demo. In the
0:21:01.920,0:21:07.820
hardware it's really flaky. My version[br]works about one in eight times. But it
0:21:07.820,0:21:12.041
doesn't typically cause a crash. So that's[br]actually good for concealment because it
0:21:12.041,0:21:17.850
becomes now much harder to determine which[br]machines are affected. In qemu because
0:21:17.850,0:21:21.640
it's emulating, it's a little more[br]reliable but it's still it's only two out
0:21:21.640,0:21:26.760
of three. If we let the BMC boot a little[br]bit further it actually prints out this
0:21:26.760,0:21:32.120
message. And if you hit enter it drops you[br]to a shell with no password and you can
0:21:32.120,0:21:38.170
then just run commands as root on the BMC[br]and that's a lot easier than all this
0:21:38.170,0:21:43.440
stuff with the SPI bus if you wanted to[br]build a hardware implant against it. I
0:21:43.440,0:21:48.540
don't know where the serial port is on the[br]on the Super Micro but on a different tier
0:21:48.540,0:21:54.030
1 server mainboard I was able to probe[br]around the oscilloscope and locate the
0:21:54.030,0:22:00.830
serial console for the BMC. Figure out[br]it's 115 kbaud and it has the same code
0:22:00.830,0:22:06.050
that you hit enter and you can run[br]commands there. So that's a much easier
0:22:06.050,0:22:11.990
way to do it. A big question a lot of[br]people have is how do we actually detect
0:22:11.990,0:22:18.100
this sort of flash implant. A lot of high[br]assurance sites replace all of their roms
0:22:18.100,0:22:22.760
with ones that they flash themselves but[br]that doesn't get rid of the implant
0:22:22.760,0:22:28.960
because it's outside of the ROM chip.[br]Likewise reading the ROM chip doesn't show
0:22:28.960,0:22:35.321
anything because it's not in the ROM[br]itself it's it's outside of it. Even
0:22:35.321,0:22:40.650
hooking up a logic analyzer to the bus and[br]watching as the machine boots and seeing
0:22:40.650,0:22:45.780
the data stream coming out of the flash[br]won't actually reveal the implant because
0:22:45.780,0:22:51.770
you'd have to put the logic probes on the[br]PGA pads on the flat on the BMC itself.
0:22:51.770,0:22:58.140
And that's a much harder task. Some people[br]think "oh well we can see the weird
0:22:58.140,0:23:03.150
network traffic when the BMC tries to[br]exfiltrate the data" but that would be
0:23:03.150,0:23:08.030
that's only one way for the BMC to affect[br]things. There is a great talk a few years
0:23:08.030,0:23:13.410
ago at DefCon from Intel ATR where they[br]showed how something that can control the
0:23:13.410,0:23:19.020
system firmware can backdoor hypervisors.[br]And then they gave a use case where a
0:23:19.020,0:23:26.180
unprivileged guest on a cloud system could[br]read all of the rest of physical memory so
0:23:26.180,0:23:34.760
it could see all of the other guests[br]memory. So what do we do? The big problems
0:23:34.760,0:23:39.560
is the BMC has way too many privileges.[br]It's connected to pretty much everything
0:23:39.560,0:23:46.650
in the system but the BMC is not our only[br]concern. As @whitequark said, our PCs are
0:23:46.650,0:23:52.300
just a bunch of embedded devices in a[br]trench coat and they all have firmware. In
0:23:52.300,0:23:56.680
fact pretty much everything on your system[br]more complex than a resistor probably has
0:23:56.680,0:24:01.270
firmware and if you have one of those[br]Super Micro implants maybe even your
0:24:01.270,0:24:08.500
resistors have firmware as well. I've[br]found that the firmware and things like
0:24:08.500,0:24:15.150
the power supplies can be used to gain[br]code execution on the BMC. It's really
0:24:15.150,0:24:20.750
interesting how tightly connected all of[br]our systems are. And as Joe Fit's pointed
0:24:20.750,0:24:26.700
out in his blackhat ???? talk, these are[br]not multimillion dollar attacks these are
0:24:26.700,0:24:33.500
five euro bits of hardware that we now[br]have to really be worried about. I really
0:24:33.500,0:24:38.480
like the guidelines that NIST has[br]published that suggests that we think
0:24:38.480,0:24:43.650
about our systems more in this holistic[br]manner. Although the interpreting pretty
0:24:43.650,0:24:50.290
much everything into the TPM is the[br]trusted platform module for doing this
0:24:50.290,0:24:55.580
attestation and I think we as a community[br]need to do more to use the TPM. There
0:24:55.580,0:25:01.060
actually a really good tool for securing[br]our systems but they are also potentially
0:25:01.060,0:25:08.030
subject to their own hardware implants.[br]The NCC Group TPM genie is able to subvert
0:25:08.030,0:25:14.600
the core root of trust by interposing on[br]the TPM. So a lot of folks are proposing
0:25:14.600,0:25:19.160
we should move to other trusted execution[br]environments like SGX or Trustzone. And I
0:25:19.160,0:25:24.960
think these have a lot of promise[br]especially for trusted cloud computing.
0:25:24.960,0:25:30.970
There also is a lot of innovation in the[br]hardware roots of trust going on right now
0:25:30.970,0:25:34.860
between the Google Titan, which initially[br]was for their servers and is now showing
0:25:34.860,0:25:39.740
up on all of their chrome books. The[br]Microsoft Cerberus chip which again is the
0:25:39.740,0:25:46.710
Azure system. They're actually publishing[br]their firmware and the ASIC design so that
0:25:46.710,0:25:49.880
people can have a little more faith in it[br]and they hope it will become an open
0:25:49.880,0:25:56.780
standard. And companies like Apple have[br]also gone their own way. With the T2 and
0:25:56.780,0:26:00.620
the T2's are really amazing chip for[br]securing systems. But it does so at the
0:26:00.620,0:26:06.790
expense of user freedom and that gets in[br]the way of what I think the real way that
0:26:06.790,0:26:11.130
we need to.. we need to solve this[br]problem. We need to get rid of a lot of
0:26:11.130,0:26:18.830
these secrets. Counter to what the Super[br]Micro CEO said, having a secret
0:26:18.830,0:26:22.770
motherboard design does not make you more[br]secure. Things like the Open Compute
0:26:22.770,0:26:27.140
hardware I think is a good vision for how[br]we can move forward that when you buy an
0:26:27.140,0:26:33.030
Open Compute server it comes with full[br]schematics and gerber files. So that
0:26:33.030,0:26:37.910
motivated customers can verify that the[br]systems that they're buying are the ones
0:26:37.910,0:26:42.140
that they think they that they're buying[br]that all of the components are what they
0:26:42.140,0:26:49.250
think they should be. I think the firmware[br]also needs more openness. Ronald Minnich,
0:26:49.250,0:26:56.150
Google is my co-lead on Linux boot project[br]and we think that Linux in the firmware is
0:26:56.150,0:27:03.821
a way forward to get a more secure more[br]flexible and more resilient system. We're
0:27:03.821,0:27:09.981
working with a spin off project called[br]micro BMC that is using the Linux boot
0:27:09.981,0:27:16.580
tools to build BMC firmware and this is[br]opensource. It's reproducibly built it can
0:27:16.580,0:27:22.740
work with roots of trust attestation. It's[br]written in a memory safe language since
0:27:22.740,0:27:27.740
it's a Google collaboration and go. And[br]more importantly we've thrown away all of
0:27:27.740,0:27:31.240
the legacy features that have been a[br]source of a lot of security
0:27:31.240,0:27:40.960
vulnerabilities in these systems. So did[br]it happen? I don't know. Is it technically
0:27:40.960,0:27:44.520
possible? I think so. I hope I've[br]convinced all of you that this is
0:27:44.520,0:27:50.770
definitely a technical possibility that we[br]need to be concerned about and I hope that
0:27:50.770,0:27:56.260
the way forward through hardware roots of[br]trust with attestation and more
0:27:56.260,0:28:01.400
importantly with open hardware so that we[br]know that what the machines were running
0:28:01.400,0:28:07.130
are running code that we know.. the code[br]that we've built that we understand and
0:28:07.130,0:28:13.080
that we can actually have a good chance of[br]being able to take control back of them.
0:28:13.080,0:28:18.300
If you're interested in more discussion on[br]this and also on open firmware, there's an
0:28:18.300,0:28:23.850
assembly here in this hall that has a[br]bunch folks working on a core boot and
0:28:23.850,0:28:29.110
Linux boot and a lot of these projects[br]where you can help contribute and you can
0:28:29.110,0:28:37.510
help also pressure vendors to make these[br]this standard and a way forward for a more
0:28:37.510,0:28:42.000
secure computing. So thank you all for[br]coming. And I really enjoyed the chance to
0:28:42.000,0:28:50.380
show off my modship of the state.
0:28:50.380,0:28:56.030
applause
0:28:56.030,0:29:02.600
Herald: Geat talk, thank you very much[br]Trammel. We have 10 minutes for questions
0:29:02.600,0:29:11.080
so please line up at the microphones if[br]you have questions. And we also have a
0:29:11.080,0:29:25.390
signal angel probably with questions from[br]the internet. So any questions? Microphone
0:29:25.390,0:29:29.870
number three?[br]Mic 3: Yes, I was going to ask, what's
0:29:29.870,0:29:35.650
your opinion on the Talos systems? The[br]openPOWER based ones?
0:29:35.650,0:29:41.830
Trammell: So the question is about the[br]Talos power 9 based systems power 9 is a
0:29:41.830,0:29:48.490
really interesting architecture. The.. it[br]is using a open firmware very similar to
0:29:48.490,0:29:55.250
Linux boot called Petty(??) boot that[br]moves Linux into the bootloader. I'm a big
0:29:55.250,0:29:58.860
fan. There's a lot of folks in the[br]opensource community who are very excited
0:29:58.860,0:30:07.540
about it. I'm hoping that there would be[br]more power nine systems coming out. I'm
0:30:07.540,0:30:13.130
also very excited about the brisque five[br]systems. I think having open source CPUs
0:30:13.130,0:30:19.310
use is a real way that we can have more[br]assurance that our systems are what we
0:30:19.310,0:30:22.800
think they are.[br]Herald: Thank you, microphone number two
0:30:22.800,0:30:27.100
please.[br]Mic 2: Yes, thanks for the talk. I was
0:30:27.100,0:30:32.810
wondering if you have just a scope probe[br]over this serial, cause it's just a serial
0:30:32.810,0:30:37.320
resistor which we're replacing. If you put[br]just two scope probes on there and measure
0:30:37.320,0:30:41.270
the voltage over it, in your situation[br]would the voltage change there once in a
0:30:41.270,0:30:42.400
while?[br]Trammell: Yes, yes, yes.
0:30:42.400,0:30:46.540
Mic 2: Well okay, in the normal case would[br]it actually be quite consistent current.
0:30:46.540,0:30:56.890
Or if you lowered the input impedance of[br]the BMC chip who might already have fixed
0:30:56.890,0:31:01.760
a part of the attack because the output[br]sourcing current of your exploit is
0:31:01.760,0:31:04.900
probably limited due to the limited supply[br]you only can..
0:31:04.900,0:31:12.390
Herald: Your question please?[br]Mic 2: Yes.. but.. do you see a way to get
0:31:12.390,0:31:17.710
more power into your setup? Maybe using,[br]well other power sources, other than the
0:31:17.710,0:31:22.650
two pins, or maybe somewhere of..[br]Trammell: Well, so the question is about,
0:31:22.650,0:31:28.420
would there be a way to do more arbitrary[br]changes through redesigning the implant.
0:31:28.420,0:31:34.190
One of the goals was to fit with only[br]those two pins so that a single piece on
0:31:34.190,0:31:38.900
the motherboard could be replaced. With a[br]dual probe soldering iron and you can pop
0:31:38.900,0:31:45.500
it out and stick a new one down in a[br]matter of seconds. So, yes, if you have
0:31:45.500,0:31:51.809
more pins where you can get more power[br]from you can do much more interesting
0:31:51.809,0:31:57.460
things. But that's.. would require a[br]different set of changes to the
0:31:57.460,0:32:02.480
motherboard.[br]Herald: Thank you. Microphone 1 please.
0:32:02.480,0:32:09.350
Mic 1: So, a lot of the -like- arguments[br]that these implants were not feasible by a
0:32:09.350,0:32:13.820
Super Micro where you also show the[br]picture from the fab that you had to
0:32:13.820,0:32:19.390
change the etching and the optical[br]inspection and so on and so on. But how
0:32:19.390,0:32:27.870
probable would you rate the fact that some[br]acto just intercepted the manufacturing
0:32:27.870,0:32:33.570
files and added that component already in[br]the file because then all the optical
0:32:33.570,0:32:38.810
inspection and that would all say well[br]that matches what was sent to us. But that
0:32:38.810,0:32:41.650
was not necessarily what Super Micro sent[br]to the fab.
0:32:41.650,0:32:44.900
Trammell: So the question is, could[br]someone have modified all of the
0:32:44.900,0:32:48.620
manufacturing files that went to the[br]factory, and that's absolutely a
0:32:48.620,0:32:54.520
possibility. But that's also very likely[br]that that would be detected by Super Micro
0:32:54.520,0:33:01.170
itself that in a lot of cases you don't[br]necessarily want to trust the company that
0:33:01.170,0:33:05.930
is making the product to also test it. And[br]you probably want to have a separate
0:33:05.930,0:33:11.059
company that does random spot checks to[br]verify that the boards are actually being
0:33:11.059,0:33:16.460
produced to the specification that you..[br]that you desire. So it's certainly
0:33:16.460,0:33:24.050
possible and I really don't want to[br]speculate as to the accuracy of that part
0:33:24.050,0:33:31.030
of the story but yeah it would require[br]quite a bit more changes. And also would
0:33:31.030,0:33:34.679
be much more likely to be detected in the[br]spot check.
0:33:34.679,0:33:38.230
Herald: Great. Microphone number two[br]please.
0:33:38.230,0:33:44.510
Mic 2: Yes, for a lot of motherboards[br]there are also quite a few components not
0:33:44.510,0:33:53.750
populated some of which are on which you[br]could consider sensitive myths. Wouldn't
0:33:53.750,0:33:59.430
that make it. Yeah exactly. Wouldn't that[br]make it very easy to do just pop something
0:33:59.430,0:34:04.540
on there in parallel with one of the[br]components and not have it be detected
0:34:04.540,0:34:08.329
because it's like the board is modified.[br]There is a component or you have no way of
0:34:08.329,0:34:11.490
telling whether it had to be populated or[br]not?
0:34:11.490,0:34:18.599
Trammell: Super Micro puts a lot of extra[br]pads on the board in this one particular
0:34:18.599,0:34:28.700
one they have both 8 pin and 16 pin flash[br]chip pads that are just in parallel
0:34:28.700,0:34:32.989
together. So depending on which chip is[br]cheaper that day of the week or who knows
0:34:32.989,0:34:38.419
what, they will populate one or the other.[br]So that's why in this particular photo
0:34:38.419,0:34:47.950
having the position of that circle on the[br]data output pin is very very interesting.
0:34:47.950,0:34:56.659
Herald: Question answered? Okay. So one[br]more question on microphone number two
0:34:56.659,0:35:00.400
please?[br]Mic 2: How far can signing of firmware be
0:35:00.400,0:35:06.470
a solution to this problem?[br]Trammell: Signing firmware solves a lot of
0:35:06.470,0:35:13.401
the issues. It does however not all[br]typically not all of the firmware are
0:35:13.401,0:35:21.020
signed specifically is probably to be[br]signed in in a modern BMC. The kernel and
0:35:21.020,0:35:25.789
maybe the root file system might be[br]signed. But the envy of RAM file system in
0:35:25.789,0:35:32.589
this BMC is designed to be user modifiable[br]so it can't be signed by the manufacturer,
0:35:32.589,0:35:41.340
so this sort of attack would work against[br]a signed BMC just as well. Also the "Hit
0:35:41.340,0:35:49.509
enter to get a serial console" attack[br]circumvents any signing. There are things
0:35:49.509,0:35:56.140
on the host firmware on the x86 like boot[br]card that do a really good job of making
0:35:56.140,0:36:01.520
it harder to get code execution during the[br]boot process. But there have been several
0:36:01.520,0:36:07.720
CVEs where it has been implemented poorly.[br]So even though signature's the firmware is
0:36:07.720,0:36:13.800
signed, people have still managed to get[br]code execution during that process.
0:36:13.800,0:36:18.329
Herald: Great. Thank you Trammell Hudson[br]again, a warm round of applause, thank you
0:36:18.329,0:36:21.009
very much!
0:36:21.009,0:36:24.009
applause
0:36:24.009,0:36:25.529
35c3 postrol music
0:36:25.529,0:36:52.000
Subtitles created by c3subtitles.de[br]in the year 2021. Join, and help us!