WEBVTT
00:00:00.000 --> 00:00:19.759
35c3 prerol music
00:00:19.759 --> 00:00:26.630
Herald: So Trammell Hudson, who is
standing here, he's taking things apart.
00:00:26.630 --> 00:00:34.370
Don't worry not life on stage, but he will
give us a proof of concept and some
00:00:34.370 --> 00:00:39.559
details and functionalities about hardware
implants. So the same things that we heard
00:00:39.559 --> 00:00:45.480
from Bloomberg article talking about Apple
and super microcomputers with implants
00:00:45.480 --> 00:00:52.879
that, yeah, were implanted into those,
into those computers. And I'm really
00:00:52.879 --> 00:00:57.680
excited to see this in action. Please give
a warm round of applause to Trammel
00:00:57.680 --> 00:01:02.590
Hudson!
00:01:02.590 --> 00:01:07.510
applause
00:01:07.510 --> 00:01:11.600
Trammell: Before we begin talking about
hardware implants just two quick
00:01:11.600 --> 00:01:16.310
disclaimers. The first from my employer
Two Sigma investments as it says are
00:01:16.310 --> 00:01:21.910
chocolate bars. This is not investment
advice. And secondly I don't actually know
00:01:21.910 --> 00:01:26.920
what the story is behind the super micro
story. No one outside of Bloomberg and
00:01:26.920 --> 00:01:32.450
their sources do. But I have spent a lot
of time thinking about hardware implants
00:01:32.450 --> 00:01:38.200
starting with the thunderstrike firmware
attack against mac books as well as the
00:01:38.200 --> 00:01:45.420
thunderstrike 2 where we were able to get
software to write into the firmware on the
00:01:45.420 --> 00:01:50.560
mac books. I've also been thinking a lot
about how to defend against hardware
00:01:50.560 --> 00:01:54.420
implants with things like the heads
firmware for slightly more secure laptops
00:01:54.420 --> 00:01:59.420
and also as part of my co-leas on the
Linux boot project. We're thinking about
00:01:59.420 --> 00:02:10.080
how to protect servers from physical and
software attacks. So with all of this
00:02:10.080 --> 00:02:14.910
concentrated thinking about firmware and
hardware attacks, I was really excited
00:02:14.910 --> 00:02:20.720
when I saw the Bloomberg story back in
October. But what really intrigued me was
00:02:20.720 --> 00:02:26.440
the animated image that they had at the
header that highlighted one small part of
00:02:26.440 --> 00:02:32.920
the board as where the implant was, but
what I found really interesting is that is
00:02:32.920 --> 00:02:40.250
exactly where I would install a hardware
implant as they described on the SPI bus.
00:02:40.250 --> 00:02:44.610
A lot of other people in the hardware and
from our security community thought it
00:02:44.610 --> 00:02:50.140
sounded plausible. Other people pointed
out that supply chain attacks come up
00:02:50.140 --> 00:02:56.070
periodically and they are definitely a
concern. Some people thought the attack as
00:02:56.070 --> 00:03:03.320
described was entirely implausible and in
general we sort of had a Whiskey Tango
00:03:03.320 --> 00:03:08.160
Foxtrot moment as everybody scrambled to
figure out what's going on inside their
00:03:08.160 --> 00:03:14.540
machines. So, let's step back very quickly
and review what the key claims that
00:03:14.540 --> 00:03:22.340
Bloomberg alleged happened. First they
said that Amazon's testers found a tiny
00:03:22.340 --> 00:03:27.250
microchip that wasn't part of the board's
original design that had been disguised to
00:03:27.250 --> 00:03:33.350
look like a signaling condition signal
condition coupler and that these illicit
00:03:33.350 --> 00:03:39.790
chips were connected to the baseboard
management controller or the BMC which
00:03:39.790 --> 00:03:44.390
gives them access to machines that were
turned off. That might sound kind of
00:03:44.390 --> 00:03:49.870
extreme, but that's actually what the role
of the BMC is, that in most servers the
00:03:49.870 --> 00:03:55.280
BMC is running any time the machine is
hooked up to power and it's connected to
00:03:55.280 --> 00:04:01.910
the power supplies so that it can turn the
machine on and turn it off. Frequently you
00:04:01.910 --> 00:04:06.780
want to be able to do this over a network
so it has its own dedicated LAN port but
00:04:06.780 --> 00:04:14.180
it can also share the LAN port with the
with the main system. Serial over LAN is a
00:04:14.180 --> 00:04:19.180
really useful way to debug the systems so
it provides that functionality. It can
00:04:19.180 --> 00:04:27.350
also provide fake USB volumes to allow to
to do unintended OS installation. A lot of
00:04:27.350 --> 00:04:33.430
sites also won't remote KVM so it has VGA
but that VGA support means that it's on
00:04:33.430 --> 00:04:40.370
the PCIe BUS and because some PCIe it can
do DMA into main memory. It also is
00:04:40.370 --> 00:04:47.000
typically mixed(?) into the SPI flash for
the host firmware, which allows it to
00:04:47.000 --> 00:04:51.820
modify it and on some systems it's even
connected to the TPM which allows it to
00:04:51.820 --> 00:04:59.930
circumvent the corporate of trust. So with
all of this capability inside this chip
00:04:59.930 --> 00:05:06.919
it's really unfortunate that they are
really not well put together. The head of
00:05:06.919 --> 00:05:11.150
Azure security says they have no
protection against attacks. There's no
00:05:11.150 --> 00:05:15.530
ability to detect if an attack has
happened and there's no ability to recover
00:05:15.530 --> 00:05:22.449
from an attack. So having a hardware
implant on the BMC is a really big
00:05:22.449 --> 00:05:32.030
concern. The other claim in the article is
that it affected 30 different companies
00:05:32.030 --> 00:05:39.930
including Apple and Bloomberg alleges that
Apple found malicious chips independently
00:05:39.930 --> 00:05:44.980
on their super micro boards. Went to the
FBI about it and that they then severed
00:05:44.980 --> 00:05:52.100
ties with Super Micro. This particular
claim was interesting because it
00:05:52.100 --> 00:05:57.570
corroborated a story that had shown up
back in early 2017 that Apple had removed
00:05:57.570 --> 00:06:03.050
Super Micro from their data centers. Apple
denied that there was a firmware issue.
00:06:03.050 --> 00:06:10.190
But it's interesting that perhaps these
two were related. The third set of claims
00:06:10.190 --> 00:06:16.090
is that on some of these implants they
were actually put between the layers on
00:06:16.090 --> 00:06:23.210
the PCB and then the most explosive claim
is that this was done by operatives from
00:06:23.210 --> 00:06:33.580
China, the Chinese People's Liberation
Army. With a story with this you know this
00:06:33.580 --> 00:06:39.389
many claims and this significant of
allegations we'd hoped that it would be
00:06:39.389 --> 00:06:45.430
really well sourced and for a normal story
17 independent sources that Bloomberg
00:06:45.430 --> 00:06:52.490
editors agreed to grant anonymity to,
including six national security, two
00:06:52.490 --> 00:06:57.680
people inside of AWS and three senior
insiders at Apple seems like pretty solid
00:06:57.680 --> 00:07:03.110
sourcing, except as soon as this article
is published everyone denied it. The
00:07:03.110 --> 00:07:09.080
Director of National Intelligence said
they'd seen no evidence of this. Amazon
00:07:09.080 --> 00:07:13.990
said that they've never found any issues
of modified hardware nor have they been
00:07:13.990 --> 00:07:21.000
engaged with the government over it. Apple
was even more blunt. CEO Tim Cook said
00:07:21.000 --> 00:07:27.590
this did not happen. There is no truth to
this. And Super Micro wrote a fairly
00:07:27.590 --> 00:07:32.150
lengthy letter about what they do to
protect their supply chain and why they
00:07:32.150 --> 00:07:38.990
think this attack did not happen. And it
is worth going through to look at some of
00:07:38.990 --> 00:07:44.880
the things that they say that they do to
protect their supply chain. They point out
00:07:44.880 --> 00:07:50.700
that if there's any unauthorized physical
alterations during the manufacturing
00:07:50.700 --> 00:07:56.949
process other design elements would not
match and those things would be detected.
00:07:56.949 --> 00:08:03.300
To sort of understand how circuit boards
are made, I recently visited a PCB factory
00:08:03.300 --> 00:08:11.080
in Guangzhou. This is not a super micro
factory. This is just a holiday photos. So
00:08:11.080 --> 00:08:16.760
in order to add new vias they would have
to modify the drill files which would then
00:08:16.760 --> 00:08:22.050
get electroplated. If they had to add new
traces, they would have to be able to
00:08:22.050 --> 00:08:29.400
subvert the masking and etching process
and any changes to either the drills or
00:08:29.400 --> 00:08:34.909
the etching on individual layers would be
caught by the optical inspection that's
00:08:34.909 --> 00:08:41.479
done on these bare circuit boards.
Additionally the allegation that things
00:08:41.479 --> 00:08:47.110
were inserted between circuit boards would
require that the lamination process be
00:08:47.110 --> 00:08:55.329
subverted and that the implant somehow
aligned into the system. If that implant
00:08:55.329 --> 00:09:00.410
changes any of the connectivity the flying
protesters would pick it up or the bed of
00:09:00.410 --> 00:09:05.980
nails testers which checks all of the
connectivity of all the traces to make
00:09:05.980 --> 00:09:09.300
sure that there are no shorts and to make
sure that everything that is supposed to
00:09:09.300 --> 00:09:16.679
be connected is electrically conductive.
So it would be very difficult to
00:09:16.679 --> 00:09:22.110
circumvent the production process at this
stage. And it also would be very difficult
00:09:22.110 --> 00:09:27.709
to contain because the PCB factory doesn't
know which customers are going to receive
00:09:27.709 --> 00:09:34.470
those circuit boards. Super Micro also
points out that during the assembly
00:09:34.470 --> 00:09:40.480
process when the parts are installed they
have their employees on site the whole
00:09:40.480 --> 00:09:47.559
time. On my same holiday trip I also
visited some PCB assembly companies and
00:09:47.559 --> 00:09:53.589
spoke with companies that are using doing
contract manufacturing and they said that
00:09:53.589 --> 00:09:59.089
they also send their employees to the
production line to observe the pick and
00:09:59.089 --> 00:10:05.600
place machines and the reflow and the rest
of the surface mount assembly. Their big
00:10:05.600 --> 00:10:10.089
concern is that if they don't have someone
there the parts that are fed in the pick
00:10:10.089 --> 00:10:17.660
in place will be replaced with either
counterfeits or with salvaged parts. I
00:10:17.660 --> 00:10:23.459
visited the electronics market in ???????
bay where there are people desoldering
00:10:23.459 --> 00:10:29.190
e-waste and then sorting the parts into
bins and selling these salvaged components
00:10:29.190 --> 00:10:34.589
by the kilo and for a few extra renminbi
they'll put them on rails for you so that
00:10:34.589 --> 00:10:41.660
you can save a few pennies on your
production process. The other concern that
00:10:41.660 --> 00:10:46.489
these companies have, is not just salvaged
parts but straight up counterfeits.
00:10:46.489 --> 00:10:52.439
Especially for things that cost more than
a few dollars each. The Arduino community
00:10:52.439 --> 00:10:59.139
was hit a few years ago with a bunch of
counterfeit FTDI chips where the internal
00:10:59.139 --> 00:11:07.600
construction was entirely different. In
this case it caused reliability issues but
00:11:07.600 --> 00:11:11.550
you can imagine from a security
perspective this is really worrisome that
00:11:11.550 --> 00:11:15.709
parts that look identical might have
completely different functionality inside
00:11:15.709 --> 00:11:25.379
of them. Super Micro also mentions that
they X-ray their main boards to look for
00:11:25.379 --> 00:11:32.000
anomalies and I wasn't able to take any
photos inside the factory there was doing
00:11:32.000 --> 00:11:38.230
x-rays. But in this Wikipedia photo we can
clearly see active components like this
00:11:38.230 --> 00:11:45.670
SOIC chip are different from things like
the SMD resistors and capacitors. So if an
00:11:45.670 --> 00:11:51.220
attacker were trying to subvert the supply
chain by putting a disguise component it
00:11:51.220 --> 00:11:56.670
could be detected at this step. Another
interesting thing in this photo are these
00:11:56.670 --> 00:12:02.680
inductors that are encased in dip
packages. This is really common in a lot
00:12:02.680 --> 00:12:07.439
of Ethernet boards and occasionally people
have thought they had some sort of
00:12:07.439 --> 00:12:13.589
hardware implant when they found inductors
in their ethernet jacks but it's pretty
00:12:13.589 --> 00:12:19.799
it's fairly common and it shows it pretty
clearly on the x-ray. Some other security
00:12:19.799 --> 00:12:26.069
researchers like Sophia D'Antoine did an
extensive teardown of Super Micro boards
00:12:26.069 --> 00:12:33.439
including X-ray analysis and her group
found a few oddities but nothing.. they
00:12:33.439 --> 00:12:37.529
didn't find anything malicious. There were
no smoking guns. They just appeared to be
00:12:37.529 --> 00:12:43.190
sort of supply chain type things. You can
read her blog post for more details about
00:12:43.190 --> 00:12:49.319
where they found things that shouldn't
have been there. But turned out to be just
00:12:49.319 --> 00:13:00.879
actual signal condition components. So
super micro in their ???? letter, they
00:13:00.879 --> 00:13:07.239
keep reenforcing that the manufacturing
process that is the assembly process, it's
00:13:07.239 --> 00:13:11.179
during the manufacturing process and I
agree with them. It would be very
00:13:11.179 --> 00:13:17.939
difficult to circumvent security in a
reasonable way in that part of the
00:13:17.939 --> 00:13:23.189
process. But that's not the only place
this could happen. We know that national
00:13:23.189 --> 00:13:30.309
security agencies intercept shipments of
computer hardware and then have their
00:13:30.309 --> 00:13:37.249
tailored access operations open the
computers, install hardware implants,
00:13:37.249 --> 00:13:43.670
reseal them and then have them continue on
their way in shipment. The NSA even has a
00:13:43.670 --> 00:13:51.199
catalog of hardware implants like this
JTAG implant Ethernet jacks with embedded
00:13:51.199 --> 00:13:57.009
computers in them as well as firmware
specific ones that target servers SNM(?)
00:13:57.009 --> 00:14:05.490
and then some that can do data
exfiltration via RF. So that's sort of
00:14:05.490 --> 00:14:09.481
tailored access operations is really ideal
for this supply chain attack because it
00:14:09.481 --> 00:14:16.699
allows them to contain the exploit to a
single customer. It allows them fairly
00:14:16.699 --> 00:14:21.180
good concealment as well as good cover
that if it's discovered it's really hard
00:14:21.180 --> 00:14:25.769
to attribute where things went wrong. Now
unlike if you find something inside your
00:14:25.769 --> 00:14:34.230
motherboard between the layers you know
that had to have happened at the factory.
00:14:34.230 --> 00:14:47.040
So Super Micro also claim that this was
technically implausible, that it was
00:14:47.040 --> 00:14:52.559
highly unlikely that unauthorized hardware
would function properly because a third
00:14:52.559 --> 00:15:02.470
party with lack of complete knowledge of
the design. I think that's inaccurate,
00:15:02.470 --> 00:15:07.639
both because we know the NSA does it and
also because I have done it.
00:15:07.639 --> 00:15:10.319
laughter
00:15:10.319 --> 00:15:16.059
Really, all that you need to know is that
these are common components. These flash
00:15:16.059 --> 00:15:20.310
chips show up on all the boards. You can
search the internet for the data sheet and
00:15:20.310 --> 00:15:25.989
find exactly how it's wired into the rest
of the system. And the only thing that we
00:15:25.989 --> 00:15:33.499
need to know to communicate to the BMC is
the serial output pin from this component,
00:15:33.499 --> 00:15:43.429
so the BMC flash is connected over to the
BMC CPU via the serial output and it goes
00:15:43.429 --> 00:15:51.589
through a small series resistor and that
is where my implant goes in. Mine's a
00:15:51.589 --> 00:15:56.670
little bit larger than that resistor. It
clicks onto the board and it has a small
00:15:56.670 --> 00:16:03.009
FPGA that hangs offside but it's
completely plausible to fit it into
00:16:03.009 --> 00:16:12.139
something that small in fact a modern ARM
M0 fits in the space of two transistors
00:16:12.139 --> 00:16:18.350
from a 65 002 from a few years ago. The
Moore's Law means we can pack an amazing
00:16:18.350 --> 00:16:28.329
amount of CPU into a very very small
amount of space. So on that 0 6 0 3
00:16:28.329 --> 00:16:36.100
resistor could fit around 100 cortex M0 it
would be plenty powerful for this system.
00:16:36.100 --> 00:16:42.379
The problem is we only have those two pins
so ordinarily on the spy flashing you need
00:16:42.379 --> 00:16:47.720
at least six pens but we don't have power
and ground so we have to passively power
00:16:47.720 --> 00:16:53.059
this through the data signal that's
passing through it. We don't have the chip
00:16:53.059 --> 00:16:59.959
select pin so we have to guess when this
chip has been talked to. We don't have the
00:16:59.959 --> 00:17:04.980
data input pin so we don't know what
addresses are being read or what commands
00:17:04.980 --> 00:17:11.190
are being sent. We have to reconstruct it
from the data output pin and we also don't
00:17:11.190 --> 00:17:18.900
have a clock pin so we have to figure out
how to synchronize to that clock. Lastly
00:17:18.900 --> 00:17:22.890
we don't have the ability to make
arbitrary data changes. All we can do is
00:17:22.890 --> 00:17:29.060
disconnect the pin from the BMC so we can
only turn 1 bits into 0 bits. We can't go
00:17:29.060 --> 00:17:35.300
the other way around. So with these
limitations we can still do some pretty
00:17:35.300 --> 00:17:40.920
interesting things. Recovering the clock
is actually pretty easy. We can look at
00:17:40.920 --> 00:17:49.670
the data stream and find the shortest bit
transitions from 0 1 0 or 1 0 1 to
00:17:49.670 --> 00:17:55.060
estimate what the clock is which allows us
to then reconstruct that data stream being
00:17:55.060 --> 00:18:00.870
sent to the BMC and if we look at the
flash contents we can see that a lot of it
00:18:00.870 --> 00:18:07.570
is being fairly random noise but a lot of
it is all white which in this case would
00:18:07.570 --> 00:18:15.110
mean that it's all one bits. So if we look
at the way the flash is organized we can
00:18:15.110 --> 00:18:19.380
see there's the u-boot bootloader and
that's executable. That's kind of
00:18:19.380 --> 00:18:25.230
difficult to make useful changes in, the
kernel and the root file system are both
00:18:25.230 --> 00:18:33.040
compressed so that they look effectively
like random noise but the nvram region is
00:18:33.040 --> 00:18:41.660
a jffs2 file system and this file system
??? 3 Megs, it's mostly empty and all that
00:18:41.660 --> 00:18:50.040
empty space is F F which is all ones. So
this is plenty of ones for us to work on.
00:18:50.040 --> 00:18:55.380
Additionally it has fairly nice headers
that we can we can match on. So when we
00:18:55.380 --> 00:19:00.570
see these magic bit masks we know when
we've entered different parts of the file
00:19:00.570 --> 00:19:06.990
system. So given that we can now
reconstruct the clock we can figure out
00:19:06.990 --> 00:19:13.310
where we are in the file system. This
hardware implant can start to inject new
00:19:13.310 --> 00:19:20.320
data into what was the empty space. So
this short file that we put in here is a
00:19:20.320 --> 00:19:28.020
small shell script and it is one of the
network configuration scripts, so this is
00:19:28.020 --> 00:19:37.350
where I'm going to try a live demo and I
hope this works. We're running in qemu
00:19:37.350 --> 00:19:45.660
since I didn't bring a Super Micro board
and what we have on the left is the flash
00:19:45.660 --> 00:19:50.530
console excuse me the hardware implant
console. And then on the right we have the
00:19:50.530 --> 00:19:57.353
serial console from the BMC so we can see
it has loaded the kernel and in a second
00:19:57.353 --> 00:20:03.430
it's going to we should see a bunch of
traffic, okay, so the implant is active.
00:20:03.430 --> 00:20:10.450
It has replaced the data when that nvram
file system was mounted the BMC is now
00:20:10.450 --> 00:20:18.780
continuing on doing its set up. It's going
to load a bunch of device drivers for that
00:20:18.780 --> 00:20:24.250
video. It pauses here for some reason that
I haven't diagnosed because that's that's
00:20:24.250 --> 00:20:27.040
not my job.
00:20:27.040 --> 00:20:29.140
laughter
00:20:29.140 --> 00:20:33.020
And eventually it's going to configure the
networks and it does that by running that
00:20:33.020 --> 00:20:43.010
shell script off of the nvram partition
here it starts KVM stuff brings up some
00:20:43.010 --> 00:20:53.390
things. Allright.
applause
00:20:53.390 --> 00:21:01.920
OK. So luckily we got to that point
without having to fake the demo. In the
00:21:01.920 --> 00:21:07.820
hardware it's really flaky. My version
works about one in eight times. But it
00:21:07.820 --> 00:21:12.041
doesn't typically cause a crash. So that's
actually good for concealment because it
00:21:12.041 --> 00:21:17.850
becomes now much harder to determine which
machines are affected. In qemu because
00:21:17.850 --> 00:21:21.640
it's emulating, it's a little more
reliable but it's still it's only two out
00:21:21.640 --> 00:21:26.760
of three. If we let the BMC boot a little
bit further it actually prints out this
00:21:26.760 --> 00:21:32.120
message. And if you hit enter it drops you
to a shell with no password and you can
00:21:32.120 --> 00:21:38.170
then just run commands as root on the BMC
and that's a lot easier than all this
00:21:38.170 --> 00:21:43.440
stuff with the SPI bus if you wanted to
build a hardware implant against it. I
00:21:43.440 --> 00:21:48.540
don't know where the serial port is on the
on the Super Micro but on a different tier
00:21:48.540 --> 00:21:54.030
1 server mainboard I was able to probe
around the oscilloscope and locate the
00:21:54.030 --> 00:22:00.830
serial console for the BMC. Figure out
it's 115 kbaud and it has the same code
00:22:00.830 --> 00:22:06.050
that you hit enter and you can run
commands there. So that's a much easier
00:22:06.050 --> 00:22:11.990
way to do it. A big question a lot of
people have is how do we actually detect
00:22:11.990 --> 00:22:18.100
this sort of flash implant. A lot of high
assurance sites replace all of their roms
00:22:18.100 --> 00:22:22.760
with ones that they flash themselves but
that doesn't get rid of the implant
00:22:22.760 --> 00:22:28.960
because it's outside of the ROM chip.
Likewise reading the ROM chip doesn't show
00:22:28.960 --> 00:22:35.321
anything because it's not in the ROM
itself it's it's outside of it. Even
00:22:35.321 --> 00:22:40.650
hooking up a logic analyzer to the bus and
watching as the machine boots and seeing
00:22:40.650 --> 00:22:45.780
the data stream coming out of the flash
won't actually reveal the implant because
00:22:45.780 --> 00:22:51.770
you'd have to put the logic probes on the
PGA pads on the flat on the BMC itself.
00:22:51.770 --> 00:22:58.140
And that's a much harder task. Some people
think "oh well we can see the weird
00:22:58.140 --> 00:23:03.150
network traffic when the BMC tries to
exfiltrate the data" but that would be
00:23:03.150 --> 00:23:08.030
that's only one way for the BMC to affect
things. There is a great talk a few years
00:23:08.030 --> 00:23:13.410
ago at DefCon from Intel ATR where they
showed how something that can control the
00:23:13.410 --> 00:23:19.020
system firmware can backdoor hypervisors.
And then they gave a use case where a
00:23:19.020 --> 00:23:26.180
unprivileged guest on a cloud system could
read all of the rest of physical memory so
00:23:26.180 --> 00:23:34.760
it could see all of the other guests
memory. So what do we do? The big problems
00:23:34.760 --> 00:23:39.560
is the BMC has way too many privileges.
It's connected to pretty much everything
00:23:39.560 --> 00:23:46.650
in the system but the BMC is not our only
concern. As @whitequark said, our PCs are
00:23:46.650 --> 00:23:52.300
just a bunch of embedded devices in a
trench coat and they all have firmware. In
00:23:52.300 --> 00:23:56.680
fact pretty much everything on your system
more complex than a resistor probably has
00:23:56.680 --> 00:24:01.270
firmware and if you have one of those
Super Micro implants maybe even your
00:24:01.270 --> 00:24:08.500
resistors have firmware as well. I've
found that the firmware and things like
00:24:08.500 --> 00:24:15.150
the power supplies can be used to gain
code execution on the BMC. It's really
00:24:15.150 --> 00:24:20.750
interesting how tightly connected all of
our systems are. And as Joe Fit's pointed
00:24:20.750 --> 00:24:26.700
out in his blackhat ???? talk, these are
not multimillion dollar attacks these are
00:24:26.700 --> 00:24:33.500
five euro bits of hardware that we now
have to really be worried about. I really
00:24:33.500 --> 00:24:38.480
like the guidelines that NIST has
published that suggests that we think
00:24:38.480 --> 00:24:43.650
about our systems more in this holistic
manner. Although the interpreting pretty
00:24:43.650 --> 00:24:50.290
much everything into the TPM is the
trusted platform module for doing this
00:24:50.290 --> 00:24:55.580
attestation and I think we as a community
need to do more to use the TPM. There
00:24:55.580 --> 00:25:01.060
actually a really good tool for securing
our systems but they are also potentially
00:25:01.060 --> 00:25:08.030
subject to their own hardware implants.
The NCC Group TPM genie is able to subvert
00:25:08.030 --> 00:25:14.600
the core root of trust by interposing on
the TPM. So a lot of folks are proposing
00:25:14.600 --> 00:25:19.160
we should move to other trusted execution
environments like SGX or Trustzone. And I
00:25:19.160 --> 00:25:24.960
think these have a lot of promise
especially for trusted cloud computing.
00:25:24.960 --> 00:25:30.970
There also is a lot of innovation in the
hardware roots of trust going on right now
00:25:30.970 --> 00:25:34.860
between the Google Titan, which initially
was for their servers and is now showing
00:25:34.860 --> 00:25:39.740
up on all of their chrome books. The
Microsoft Cerberus chip which again is the
00:25:39.740 --> 00:25:46.710
Azure system. They're actually publishing
their firmware and the ASIC design so that
00:25:46.710 --> 00:25:49.880
people can have a little more faith in it
and they hope it will become an open
00:25:49.880 --> 00:25:56.780
standard. And companies like Apple have
also gone their own way. With the T2 and
00:25:56.780 --> 00:26:00.620
the T2's are really amazing chip for
securing systems. But it does so at the
00:26:00.620 --> 00:26:06.790
expense of user freedom and that gets in
the way of what I think the real way that
00:26:06.790 --> 00:26:11.130
we need to.. we need to solve this
problem. We need to get rid of a lot of
00:26:11.130 --> 00:26:18.830
these secrets. Counter to what the Super
Micro CEO said, having a secret
00:26:18.830 --> 00:26:22.770
motherboard design does not make you more
secure. Things like the Open Compute
00:26:22.770 --> 00:26:27.140
hardware I think is a good vision for how
we can move forward that when you buy an
00:26:27.140 --> 00:26:33.030
Open Compute server it comes with full
schematics and gerber files. So that
00:26:33.030 --> 00:26:37.910
motivated customers can verify that the
systems that they're buying are the ones
00:26:37.910 --> 00:26:42.140
that they think they that they're buying
that all of the components are what they
00:26:42.140 --> 00:26:49.250
think they should be. I think the firmware
also needs more openness. Ronald Minnich,
00:26:49.250 --> 00:26:56.150
Google is my co-lead on Linux boot project
and we think that Linux in the firmware is
00:26:56.150 --> 00:27:03.821
a way forward to get a more secure more
flexible and more resilient system. We're
00:27:03.821 --> 00:27:09.981
working with a spin off project called
micro BMC that is using the Linux boot
00:27:09.981 --> 00:27:16.580
tools to build BMC firmware and this is
opensource. It's reproducibly built it can
00:27:16.580 --> 00:27:22.740
work with roots of trust attestation. It's
written in a memory safe language since
00:27:22.740 --> 00:27:27.740
it's a Google collaboration and go. And
more importantly we've thrown away all of
00:27:27.740 --> 00:27:31.240
the legacy features that have been a
source of a lot of security
00:27:31.240 --> 00:27:40.960
vulnerabilities in these systems. So did
it happen? I don't know. Is it technically
00:27:40.960 --> 00:27:44.520
possible? I think so. I hope I've
convinced all of you that this is
00:27:44.520 --> 00:27:50.770
definitely a technical possibility that we
need to be concerned about and I hope that
00:27:50.770 --> 00:27:56.260
the way forward through hardware roots of
trust with attestation and more
00:27:56.260 --> 00:28:01.400
importantly with open hardware so that we
know that what the machines were running
00:28:01.400 --> 00:28:07.130
are running code that we know.. the code
that we've built that we understand and
00:28:07.130 --> 00:28:13.080
that we can actually have a good chance of
being able to take control back of them.
00:28:13.080 --> 00:28:18.300
If you're interested in more discussion on
this and also on open firmware, there's an
00:28:18.300 --> 00:28:23.850
assembly here in this hall that has a
bunch folks working on a core boot and
00:28:23.850 --> 00:28:29.110
Linux boot and a lot of these projects
where you can help contribute and you can
00:28:29.110 --> 00:28:37.510
help also pressure vendors to make these
this standard and a way forward for a more
00:28:37.510 --> 00:28:42.000
secure computing. So thank you all for
coming. And I really enjoyed the chance to
00:28:42.000 --> 00:28:50.380
show off my modship of the state.
00:28:50.380 --> 00:28:56.030
applause
00:28:56.030 --> 00:29:02.600
Herald: Geat talk, thank you very much
Trammel. We have 10 minutes for questions
00:29:02.600 --> 00:29:11.080
so please line up at the microphones if
you have questions. And we also have a
00:29:11.080 --> 00:29:25.390
signal angel probably with questions from
the internet. So any questions? Microphone
00:29:25.390 --> 00:29:29.870
number three?
Mic 3: Yes, I was going to ask, what's
00:29:29.870 --> 00:29:35.650
your opinion on the Talos systems? The
openPOWER based ones?
00:29:35.650 --> 00:29:41.830
Trammell: So the question is about the
Talos power 9 based systems power 9 is a
00:29:41.830 --> 00:29:48.490
really interesting architecture. The.. it
is using a open firmware very similar to
00:29:48.490 --> 00:29:55.250
Linux boot called Petty(??) boot that
moves Linux into the bootloader. I'm a big
00:29:55.250 --> 00:29:58.860
fan. There's a lot of folks in the
opensource community who are very excited
00:29:58.860 --> 00:30:07.540
about it. I'm hoping that there would be
more power nine systems coming out. I'm
00:30:07.540 --> 00:30:13.130
also very excited about the brisque five
systems. I think having open source CPUs
00:30:13.130 --> 00:30:19.310
use is a real way that we can have more
assurance that our systems are what we
00:30:19.310 --> 00:30:22.800
think they are.
Herald: Thank you, microphone number two
00:30:22.800 --> 00:30:27.100
please.
Mic 2: Yes, thanks for the talk. I was
00:30:27.100 --> 00:30:32.810
wondering if you have just a scope probe
over this serial, cause it's just a serial
00:30:32.810 --> 00:30:37.320
resistor which we're replacing. If you put
just two scope probes on there and measure
00:30:37.320 --> 00:30:41.270
the voltage over it, in your situation
would the voltage change there once in a
00:30:41.270 --> 00:30:42.400
while?
Trammell: Yes, yes, yes.
00:30:42.400 --> 00:30:46.540
Mic 2: Well okay, in the normal case would
it actually be quite consistent current.
00:30:46.540 --> 00:30:56.890
Or if you lowered the input impedance of
the BMC chip who might already have fixed
00:30:56.890 --> 00:31:01.760
a part of the attack because the output
sourcing current of your exploit is
00:31:01.760 --> 00:31:04.900
probably limited due to the limited supply
you only can..
00:31:04.900 --> 00:31:12.390
Herald: Your question please?
Mic 2: Yes.. but.. do you see a way to get
00:31:12.390 --> 00:31:17.710
more power into your setup? Maybe using,
well other power sources, other than the
00:31:17.710 --> 00:31:22.650
two pins, or maybe somewhere of..
Trammell: Well, so the question is about,
00:31:22.650 --> 00:31:28.420
would there be a way to do more arbitrary
changes through redesigning the implant.
00:31:28.420 --> 00:31:34.190
One of the goals was to fit with only
those two pins so that a single piece on
00:31:34.190 --> 00:31:38.900
the motherboard could be replaced. With a
dual probe soldering iron and you can pop
00:31:38.900 --> 00:31:45.500
it out and stick a new one down in a
matter of seconds. So, yes, if you have
00:31:45.500 --> 00:31:51.809
more pins where you can get more power
from you can do much more interesting
00:31:51.809 --> 00:31:57.460
things. But that's.. would require a
different set of changes to the
00:31:57.460 --> 00:32:02.480
motherboard.
Herald: Thank you. Microphone 1 please.
00:32:02.480 --> 00:32:09.350
Mic 1: So, a lot of the -like- arguments
that these implants were not feasible by a
00:32:09.350 --> 00:32:13.820
Super Micro where you also show the
picture from the fab that you had to
00:32:13.820 --> 00:32:19.390
change the etching and the optical
inspection and so on and so on. But how
00:32:19.390 --> 00:32:27.870
probable would you rate the fact that some
acto just intercepted the manufacturing
00:32:27.870 --> 00:32:33.570
files and added that component already in
the file because then all the optical
00:32:33.570 --> 00:32:38.810
inspection and that would all say well
that matches what was sent to us. But that
00:32:38.810 --> 00:32:41.650
was not necessarily what Super Micro sent
to the fab.
00:32:41.650 --> 00:32:44.900
Trammell: So the question is, could
someone have modified all of the
00:32:44.900 --> 00:32:48.620
manufacturing files that went to the
factory, and that's absolutely a
00:32:48.620 --> 00:32:54.520
possibility. But that's also very likely
that that would be detected by Super Micro
00:32:54.520 --> 00:33:01.170
itself that in a lot of cases you don't
necessarily want to trust the company that
00:33:01.170 --> 00:33:05.930
is making the product to also test it. And
you probably want to have a separate
00:33:05.930 --> 00:33:11.059
company that does random spot checks to
verify that the boards are actually being
00:33:11.059 --> 00:33:16.460
produced to the specification that you..
that you desire. So it's certainly
00:33:16.460 --> 00:33:24.050
possible and I really don't want to
speculate as to the accuracy of that part
00:33:24.050 --> 00:33:31.030
of the story but yeah it would require
quite a bit more changes. And also would
00:33:31.030 --> 00:33:34.679
be much more likely to be detected in the
spot check.
00:33:34.679 --> 00:33:38.230
Herald: Great. Microphone number two
please.
00:33:38.230 --> 00:33:44.510
Mic 2: Yes, for a lot of motherboards
there are also quite a few components not
00:33:44.510 --> 00:33:53.750
populated some of which are on which you
could consider sensitive myths. Wouldn't
00:33:53.750 --> 00:33:59.430
that make it. Yeah exactly. Wouldn't that
make it very easy to do just pop something
00:33:59.430 --> 00:34:04.540
on there in parallel with one of the
components and not have it be detected
00:34:04.540 --> 00:34:08.329
because it's like the board is modified.
There is a component or you have no way of
00:34:08.329 --> 00:34:11.490
telling whether it had to be populated or
not?
00:34:11.490 --> 00:34:18.599
Trammell: Super Micro puts a lot of extra
pads on the board in this one particular
00:34:18.599 --> 00:34:28.700
one they have both 8 pin and 16 pin flash
chip pads that are just in parallel
00:34:28.700 --> 00:34:32.989
together. So depending on which chip is
cheaper that day of the week or who knows
00:34:32.989 --> 00:34:38.419
what, they will populate one or the other.
So that's why in this particular photo
00:34:38.419 --> 00:34:47.950
having the position of that circle on the
data output pin is very very interesting.
00:34:47.950 --> 00:34:56.659
Herald: Question answered? Okay. So one
more question on microphone number two
00:34:56.659 --> 00:35:00.400
please?
Mic 2: How far can signing of firmware be
00:35:00.400 --> 00:35:06.470
a solution to this problem?
Trammell: Signing firmware solves a lot of
00:35:06.470 --> 00:35:13.401
the issues. It does however not all
typically not all of the firmware are
00:35:13.401 --> 00:35:21.020
signed specifically is probably to be
signed in in a modern BMC. The kernel and
00:35:21.020 --> 00:35:25.789
maybe the root file system might be
signed. But the envy of RAM file system in
00:35:25.789 --> 00:35:32.589
this BMC is designed to be user modifiable
so it can't be signed by the manufacturer,
00:35:32.589 --> 00:35:41.340
so this sort of attack would work against
a signed BMC just as well. Also the "Hit
00:35:41.340 --> 00:35:49.509
enter to get a serial console" attack
circumvents any signing. There are things
00:35:49.509 --> 00:35:56.140
on the host firmware on the x86 like boot
card that do a really good job of making
00:35:56.140 --> 00:36:01.520
it harder to get code execution during the
boot process. But there have been several
00:36:01.520 --> 00:36:07.720
CVEs where it has been implemented poorly.
So even though signature's the firmware is
00:36:07.720 --> 00:36:13.800
signed, people have still managed to get
code execution during that process.
00:36:13.800 --> 00:36:18.329
Herald: Great. Thank you Trammell Hudson
again, a warm round of applause, thank you
00:36:18.329 --> 00:36:21.009
very much!
00:36:21.009 --> 00:36:24.009
applause
00:36:24.009 --> 00:36:25.529
35c3 postrol music
00:36:25.529 --> 00:36:52.000
Subtitles created by c3subtitles.de
in the year 2021. Join, and help us!