WEBVTT
00:00:00.430 --> 00:00:06.600
36C3 preroll music
00:00:06.600 --> 00:00:12.730
ominous bubbling, ebbing away
00:00:19.880 --> 00:00:26.910
Herald: The next talk is called "Uncover,
Understand and Own - Regaining Control
00:00:26.910 --> 00:00:34.260
Over Your AMD CPU", and I must say, the
days where your homebrew PC would have
00:00:34.260 --> 00:00:40.650
been like one CPU plus a lot of discrete
logic, those days are long, long gone. Now
00:00:40.650 --> 00:00:45.250
every single device, probably even this
microphone, is full of microprocessors.
00:00:45.250 --> 00:00:53.490
It's pretty crazy. Robert, Alexander and
Christian discovered an actual ARM
00:00:53.490 --> 00:01:01.390
processor on an AMD CPU, which I find
quite mind boggling; and it actually
00:01:01.390 --> 00:01:06.540
includes its own firmware. To talk about
that, I'd like to welcome them onto the
00:01:06.540 --> 00:01:12.579
stage. I'm really looking forward to
hearing all about this discovery and what
00:01:12.579 --> 00:01:16.360
it has for consequences for us. So thank
you very much. Give them a hand!
00:01:16.360 --> 00:01:22.570
Applause
00:01:22.570 --> 00:01:26.409
Robert Buhren: All right. Thanks. So,
before we dive into the topic, a quick
00:01:26.409 --> 00:01:32.319
introduction. This is Christian and this
is Alex, I'm Robert. And the reason why
00:01:32.319 --> 00:01:36.531
there's three of us today is, I'm a Ph.D.
student at the Technische Universität in
00:01:36.531 --> 00:01:42.719
Berlin, and beginning of 2018, I was
looking into the Secure Encrypted
00:01:42.719 --> 00:01:47.479
Virtualization (SEV) technology from AMD.
And this technology requires a firmware
00:01:47.479 --> 00:01:52.640
running on the Secure Processor of AMD.
And that's where Christian came into play
00:01:52.640 --> 00:01:57.100
because he was looking for a master's
thesis. Now Christian is done with this
00:01:57.100 --> 00:02:03.469
thesis, and Alex here kind of took over his
work. But today we're going to explain to
00:02:03.469 --> 00:02:09.690
you what the AMD Secure Processor is doing
and what we have uncovered. So with that,
00:02:09.690 --> 00:02:17.100 line:1
I'm going to hand over to Christian.
Christian Werling: So let's dive right
00:02:17.100 --> 00:02:22.010 line:1
into our first part of the presentation,
which is about reverse engineering a
00:02:22.010 --> 00:02:27.950
completely unknown subsystem. And when we
started our research, we had to find out
00:02:27.950 --> 00:02:33.270
what the AMD Secure Processor, formerly
called Platform Security Processor, in
00:02:33.270 --> 00:02:38.220
this talk PSP, actually is. And it's a
dedicated security subsystem that is
00:02:38.220 --> 00:02:46.950
integrated into your AMD CPU both on
server and desktop CPUs. It's an ARM
00:02:46.950 --> 00:02:56.971
Cortex A5 inside your x86 CPU and it's
there since around 2013. It runs a so-
00:02:56.971 --> 00:03:05.170
called secure OS and a kernel. And it's
actually undocumented and proprietary. It
00:03:05.170 --> 00:03:12.340
has access to some secure off-chip storage
for the firmware and some some data, and
00:03:12.340 --> 00:03:19.760
it mainly provides crypto functionality to
the main CPU, as well as, yeah, key
00:03:19.760 --> 00:03:22.600
generation and key management
functionality.
00:03:22.600 --> 00:03:29.400
It is required for the early boot. In
fact, it's required for Secure Boot, and
00:03:29.400 --> 00:03:37.000
it acts as a trust anchor in in your
system. So the PSP is a security
00:03:37.000 --> 00:03:44.780
subsystem, so it adds security to our
system. And that's good, right? You might
00:03:44.780 --> 00:03:49.700
notice that this has some similarities
with the Intel Management Engine, which on
00:03:49.700 --> 00:03:56.330
this very stage we heard a lot about three
hours ago. So let's look into the
00:03:56.330 --> 00:04:04.380
applications of this piece of hardware.
For that, we need to talk about trust.
00:04:04.380 --> 00:04:11.250
The one form of trust AMD tackles in what they
call Secure Encrypted Virtualization (SEV).
00:04:11.250 --> 00:04:17.949
So you as a cloud customer can be
sure that your virtual machine can even
00:04:17.949 --> 00:04:24.210 line:1
run in an untrusted physical location, for
example, in a data center. The PSP that is
00:04:24.210 --> 00:04:30.759 line:1
running inside that server CPU acts as a
remote, trusted entity for you as a
00:04:30.759 --> 00:04:38.930 line:1
customer. And it promises you to protect
your memory, your data from the hypervisor
00:04:38.930 --> 00:04:43.620 line:1
and even from physical access. For
example, through a data center
00:04:43.620 --> 00:04:52.930 line:1
administrator. The other form of trust
that the PSP tries to establish is now
00:04:52.930 --> 00:05:01.880
arriving in the Linux kernel, and that's
an API to a trusted execution environment.
00:05:01.880 --> 00:05:08.000
What that actually is, is that the PSP
acts as a black box inside your system
00:05:08.000 --> 00:05:14.210 line:1
that is trusted by an external entity. For
example, a content provider like Netflix.
00:05:14.210 --> 00:05:20.810 line:1
This would enable, for example, digital
rights management on an untrusted system
00:05:20.810 --> 00:05:29.190 line:1
that is your system, like Linux. So to sum
this all up, the PSP runs code that you
00:05:29.190 --> 00:05:35.290
don't know and that you don't control. And
first of all, let's talk about the
00:05:35.290 --> 00:05:45.000
knowing. What you see here is a Supermicro
motherboard, a server motherboard, from
00:05:45.000 --> 00:05:49.790
the top, and I highlighted three
components here which are required or
00:05:49.790 --> 00:05:58.540 line:1
essential for boot up, of course. That is
the CPU, the disk and so-called SPI flash.
00:05:58.540 --> 00:06:04.230
The SPI flash is a simple storage that is
available during early boot. So if you
00:06:04.230 --> 00:06:09.370
look at the boot procedure in a simplified
manner, then the CPU will first load the
00:06:09.370 --> 00:06:15.710
BIOS from this SPI flash. And only at a
later stage of booting, when the necessary
00:06:15.710 --> 00:06:22.410
drivers are at hand, it will be able to
access the hard disk to load the operating
00:06:22.410 --> 00:06:30.419
system. Now, as we saw from AMD's
marketing slides, there is the PSP now.
00:06:30.419 --> 00:06:38.810 line:1
The PSP is actually part of the CPU and
even boots before the CPU boots and will
00:06:38.810 --> 00:06:45.639 line:1
only after successful initialization of
the system release the x86 CPU. So the PSP
00:06:45.639 --> 00:06:52.260
firmware is loaded first, and after that,
the boot is proceeding as we know it with
00:06:52.260 --> 00:07:00.290
the BIOS and the operating system. So
where is this PSP firmware coming from?
00:07:00.290 --> 00:07:07.340
Well, the BIOS is stored on the just-
mentioned SPI flash memory and it contains
00:07:07.340 --> 00:07:12.850
all the data and code that is used, of
course, during boot up. And it is arranged
00:07:12.850 --> 00:07:18.560 line:1
according to the UEFI image specification.
So it's a standardized format. That's
00:07:18.560 --> 00:07:28.470 line:1
that's good. So maybe we should have a
look into a Supermicro UEFI update. You
00:07:28.470 --> 00:07:34.720
see screenshots from the open source tool,
UEFI tool, which is able to parse the UEFI
00:07:34.720 --> 00:07:40.770
image specification. You see information,
for example, like the full size. This is
00:07:40.770 --> 00:07:44.570
16 megabytes. That's the traditional,
that's the size of a traditional SPI
00:07:44.570 --> 00:07:53.210
flash. And you see several volumes which
contain BIOS code and data. What you can
00:07:53.210 --> 00:07:58.510
also spot are two so-called paddings, non
empty paddings. And these are called
00:07:58.510 --> 00:08:04.147
paddings by the tool because
they are not part of the UEFI standard.
00:08:04.147 --> 00:08:09.060
And we're not able to parse
them with the standardized information
00:08:09.060 --> 00:08:16.850
available. So let's use another tool.
Probably many of you know "binwalk", a
00:08:16.850 --> 00:08:24.190
command line tool for extracting firmware
from images and forensics in general. And
00:08:24.190 --> 00:08:30.080
let's look at the machine instructions we
can find in that UEFI update for the
00:08:30.080 --> 00:08:37.220
Supermicro board. So the second block you
see are Intel x86 instructions. This is
00:08:37.220 --> 00:08:44.940
what we expect, right? It's a BIOS update
for an x86 CPU. So that's not surprising.
00:08:44.940 --> 00:08:51.570
What is more surprising are the ARM
instructions. So we might be very close to
00:08:51.570 --> 00:09:03.430
the PSP firmware. And what we found out by
staring at bytes and a hex editor a lot is
00:09:03.430 --> 00:09:08.540
what we call the firmware file system of
the Platform Security Processor. And the
00:09:08.540 --> 00:09:14.230
central data structure in it is the
directory. A directory starts with a
00:09:14.230 --> 00:09:21.420
magic string, in this case, dollar PSP,
and it will have a checksum. It will have
00:09:21.420 --> 00:09:27.560
a number of elements that it will list and
a field we don't know. And then with each
00:09:27.560 --> 00:09:35.420
line in the screenshot, you will have an
entry in this directory. And each entry
00:09:35.420 --> 00:09:42.260
has a type and a size and an address where
it is located inside that UEFI image. So
00:09:42.260 --> 00:09:47.320
the last entry of this directory is a
special entry. It points to a secondary
00:09:47.320 --> 00:09:56.680
directory or that's how we call it. It's a
continuation of this directory, and each
00:09:56.680 --> 00:10:02.520 line:1
entry points to something like a file. A
file definitely has a body and it might
00:10:02.520 --> 00:10:08.260 line:1
have a header and a signature. But I'm
gonna go into detail about this in just a
00:10:08.260 --> 00:10:13.390
second. So now we just need a reliable
entry point to parse this whole firmware
00:10:13.390 --> 00:10:17.750
file system, and this is the Firmware
Entry Table. The Firmware Entry Table
00:10:17.750 --> 00:10:22.500
begins with a specific byte sequence,
that's how you can find it. And, it lists
00:10:22.500 --> 00:10:29.310 line:1
pointers to firmware blobs such as those
directories inside the UEFI image. Earlier
00:10:29.310 --> 00:10:33.170 line:1
versions of the Firmware Entry Table are
documented in source code of the Coreboot
00:10:33.170 --> 00:10:37.360 line:1
project, an open source BIOS
implementation, and that was very helpful
00:10:37.360 --> 00:10:44.680 line:1
in the beginning of our research. So, to
make use of all that knowledge and all
00:10:44.680 --> 00:10:50.620
that staring at bytes here, we developed
"psptool", a command line utility that is
00:10:50.620 --> 00:11:01.630
able to parse any AMD firmware from UEFI
updates such as the Supermicro update. And
00:11:01.630 --> 00:11:06.589
in the output you will see something like
a directory header here, you will find
00:11:06.589 --> 00:11:12.170
entries like something called PSP Firmware
Bootloader. You will see that it has a
00:11:12.170 --> 00:11:17.490
version, and psptool will even try to
find out whether it's compressed, signed,
00:11:17.490 --> 00:11:25.480
will try to verify the signature and so
on. And, just as a recap here, you can see
00:11:25.480 --> 00:11:29.320
that the last entry of this directory
actually points to another directory,
00:11:29.320 --> 00:11:35.570
which psptool parses for you as well. So
in order to enable you to look into the
00:11:35.570 --> 00:11:41.380
code that is running on your AMD CPU right
now, psptool is available on GitHub and
00:11:41.380 --> 00:11:50.580
you can check it out today. So the PSP
runs code we don't know. Well, now it's a
00:11:50.580 --> 00:11:56.180
matter of binary analysis to actually find
out what it does. Let's talk about the
00:11:56.180 --> 00:12:06.080
control. Are we able to alter the firmware
to run our own code? For that we had to
00:12:06.080 --> 00:12:13.970
play around with hardware, and more
specifically we used an SPI programmer to
00:12:13.970 --> 00:12:21.950
flash any arbitrary UEFI image onto the
SPI flash. After, for example, taking the
00:12:21.950 --> 00:12:27.730
original UEFI image and tinkering around
with one byte or one bit we would then try
00:12:27.730 --> 00:12:37.410
to boot the system, and in most cases it
just wouldn't boot. This was insufficient
00:12:37.410 --> 00:12:42.240
because we only had binary output from
these experiments. So we also used the
00:12:42.240 --> 00:12:49.339
logic analyzer that you can see in the top
of this picture. A logic analyzer is just
00:12:49.339 --> 00:12:54.400
an electronic instrument that can capture
the data that runs through the logic
00:12:54.400 --> 00:13:06.029
lines. In this case, between the SPI flash
and the Supermicro board. So, looking into
00:13:06.029 --> 00:13:16.089
a recording of one of our boot procedures
we would now be able to make sense of this
00:13:16.089 --> 00:13:21.589
data. So, for example, we can see that the
chipset here issues a read command that's
00:13:21.589 --> 00:13:28.240
defined by the byte three, but tried to
read the address E 2 0 0 0 0 and then the
00:13:28.240 --> 00:13:34.720
SPI flash would gladly respond with data
at that location. Now you might argue the
00:13:34.720 --> 00:13:38.560
data is not that interesting because
that's what we control, that's what we can
00:13:38.560 --> 00:13:43.870
program, that's what we can look into with
psptool. So what we were more curious
00:13:43.870 --> 00:13:50.959
about is the order and timing of the
actual accesses. And to make that a bit
00:13:50.959 --> 00:14:00.050
more visual we wrote "psptrace". So,
psptrace takes such a SPI capture and
00:14:00.050 --> 00:14:09.140
correlates it to the output from psptool,
and we will get a enumeration of the
00:14:09.140 --> 00:14:15.459
specific components of the PSP during
boot, and I'll get into detail about this
00:14:15.459 --> 00:14:21.959
also in just a second. "psptrace" is
available as part of the psptool
00:14:21.959 --> 00:14:28.630
repository. If you're more interested
about our hardware in our hardware setup,
00:14:28.630 --> 00:14:36.430
you can check out our talk from the CCCamp
earlier this year where we actually had a
00:14:36.430 --> 00:14:42.970
Ryzen Pro CPU at hand, and just used the
Lenovo ThinkPad. So that might be more
00:14:42.970 --> 00:14:52.529
suitable for your homework. So I want to
share two more insights that we gained
00:14:52.529 --> 00:14:57.810
through our experiments in the beginning.
First of all, cryptographic protections on
00:14:57.810 --> 00:15:05.180
files. Files are protected by signature
and a field in the header determines the
00:15:05.180 --> 00:15:10.339
according public key that can be used to
verify that signature. And that's what
00:15:10.339 --> 00:15:17.779
the PSP does. So there are several keys
actually inside the firmware file system,
00:15:17.779 --> 00:15:22.579 line:1
and then all these keys are signed by the
AMD root public key which does not have a
00:15:22.579 --> 00:15:32.240 line:1
trailing signature; but as we found out,
after it is loaded from flash, it will be
00:15:32.240 --> 00:15:37.870 line:1
compared to a hash in Read Only Memory of
the PSP. So we were not able to alter it
00:15:37.870 --> 00:15:47.040
like that. The second insight is how the
early boot procedure of the PSP works.
NOTE Paragraph
00:15:47.040 --> 00:15:52.264
We have an on-chip bootloader that is
burnt into the chip, into the PSP.
00:15:52.264 --> 00:15:55.431
We have an off-chip bootloader that is
loaded from flash,
00:15:55.431 --> 00:15:59.574
and then we have several applications
that are loaded subsequently.
00:15:59.574 --> 00:16:04.130
So now let's look a bit more closely at
the output of psptrace.
00:16:04.130 --> 00:16:08.860
The first few read
accesses are to the firmware entry table,
00:16:08.860 --> 00:16:15.990
the global data structure, and then the
on-chip boot loader will load the PSP
00:16:15.990 --> 00:16:21.300
directory, it will load the AMD public
key, and verify it as I just told you by
00:16:21.300 --> 00:16:28.720
comparing it to a hash in Read Only
Memory, it will load the PSP firmware
00:16:28.720 --> 00:16:32.829
bootloader. That's what we called the off-
chip to bootloader. And this one will be
00:16:32.829 --> 00:16:40.959 line:1
verified with the AMD public key. Then in
the boot trace of psptrace, we see a delay
00:16:40.959 --> 00:16:46.300 line:1
that's due to some initialization work the
PSP does, and then it will load more
00:16:46.300 --> 00:16:54.579 line:1
directories and will load and verify some
applications eventually. And with this
00:16:54.579 --> 00:17:00.075
rough overview of the boot procedure, I'm
gonna hand you over to Alex.
00:17:00.595 --> 00:17:05.895
Applause
00:17:06.420 --> 00:17:10.659
Alexander Eichner: OK. So now that we
uncovered the basic modules of the
00:17:10.659 --> 00:17:14.002
firmware, we obviously wanted to gain
deeper knowledge about what these
00:17:14.002 --> 00:17:18.120
individual modules do, how the firmware
functions of the PSP is constructed, what
00:17:18.120 --> 00:17:22.959
hardware it provides and how we can
interface it. So in order to do that, we
00:17:22.959 --> 00:17:29.370
need to do a quick recap about how AMD
structures the CPU itself. So what you see
00:17:29.370 --> 00:17:34.330
here is a little x86 core being able to
execute two threads using simultaneous
00:17:34.330 --> 00:17:39.080 line:1
multi threading and AMD groups four of
those cores into what they call a Core
00:17:39.080 --> 00:17:45.000 line:1
CompleX (CCX). It contains up to four
cores based on your exact model, and two
00:17:45.000 --> 00:17:50.330 line:1
of those complexes are put onto a CCD or
Core Complex Die. That is what AMD also
00:17:50.330 --> 00:17:55.690 line:1
calls a chiplet. So it's a single silicon
chip on your CPU and you have multiple of
00:17:55.690 --> 00:18:02.541 line:1
those chips on your CPU. Among the two
CCXs, it contains the memory controller
00:18:02.541 --> 00:18:08.230 line:1
for the DDR4 memory, PCI express lanes,
communication links to communicate with
00:18:08.230 --> 00:18:14.120 line:1
other CPUs in the system and much more.
So, in our setup you saw earlier already,
00:18:14.120 --> 00:18:22.290 line:1
we had a two socket system with two CPUs
and each of these CPUs had four CCDs. And
00:18:22.290 --> 00:18:29.470 line:1
now, we have not just one PSP in this
whole system, but up to eight. So each of
00:18:29.470 --> 00:18:35.000 line:1
these CPUs or each of these little PSPs is
actually executing code even before the
00:18:35.000 --> 00:18:43.110 line:1
x86 cores have executed anything. So AMD
calls the one on CCD 0 the Master PSP, and
00:18:43.110 --> 00:18:48.260 line:1
all the others are referred to as slaves.
The master coordinates the initial bring
00:18:48.260 --> 00:18:52.350 line:1
up of the platform. So for the whole
initialization, for the memory controllers
00:18:52.350 --> 00:18:59.450 line:1
and so on and the slaves respond to
requests made by the master PSP. So each
00:18:59.450 --> 00:19:05.081
of these PSP is identical in the system.
Because they are 32 bit ARM cores, they
00:19:05.081 --> 00:19:10.440 line:1
have a 32 bit address space layout. The
first 256KiB of this layout are backed by
00:19:10.440 --> 00:19:16.090
actual on-chip SRAM. The first, the on-
chip bootloader, will load the off-chip
00:19:16.090 --> 00:19:22.200
bootloader, "PSP_FW_BOOTLOADER", and
place it into memory where it will be
00:19:22.200 --> 00:19:27.400
executed. Among [below] the actual
firmware bootloader, you will also have
00:19:27.400 --> 00:19:31.980
the page tables for the MMU. Yes, the PSP
also has a MMU and [is] virtual-memory-
00:19:31.980 --> 00:19:36.760
enabled. And the code is separated into a
supervisor, or kernel, mode and the user
00:19:36.760 --> 00:19:43.390
mode part. So, the last page you see here
is the so-called boot ROM service page. It
00:19:43.390 --> 00:19:47.360
contains information about the PSP [that]
the code is currently executing on, like
00:19:47.360 --> 00:19:52.500
number of sockets in the system, the
current CCD ID where it's executed. It
00:19:52.500 --> 00:19:58.780
contains some other things, like number
of sockets and so on, and it will become
00:19:58.780 --> 00:20:04.610
important later on. Then the off-chip
bootloader will call the applications.
00:20:04.610 --> 00:20:08.419
They are executed in user mode. They
contain the code and data to bring up the
00:20:08.419 --> 00:20:15.559
actual system and they also contain the
stack memory. And this is done during the
00:20:15.559 --> 00:20:22.180
initial boot-up process by using a fixed
order. And later on when the host OS runs,
00:20:22.180 --> 00:20:27.239
the application, for example, for the SEV
functionality will be loaded on demand.
00:20:27.239 --> 00:20:32.809
So, the rest of the space there we have
to fill is taken up by MMIO.
00:20:32.809 --> 00:20:36.890
So, this PSP has its own cryptographic code
processor which is not shared with the
00:20:36.890 --> 00:20:43.221
x86. You have the hardware registers to
access x86 memory, to access the system
00:20:43.221 --> 00:20:47.310
management network (what this is, we will
come to in a bit), and much more [that] we
00:20:47.310 --> 00:20:53.650
don't know about now, right now. So the
boot process in detail. So, Christian
00:20:53.650 --> 00:20:57.550 line:1
already gave you a rough overview how the
boot process is done and now we will take
00:20:57.550 --> 00:21:01.210 line:1
a deeper look into this. So first, of
course, you have the on-chip bootloader.
00:21:01.210 --> 00:21:05.170 line:1
It loads the off-chip bootloader from
flash and executes it. The off-chip
00:21:05.170 --> 00:21:09.500 line:1
bootloader will execute and initialize a
PSP to a bare minimum and then call the
00:21:09.500 --> 00:21:13.009
apps. The first one we have here,
DebugUnlock and Security Gasket. We have
00:21:13.009 --> 00:21:16.969
no idea what they are actually for, but we
named them after some strings we found in
00:21:16.969 --> 00:21:22.600
the binaries itself. So, the big chunk you
see here is the actual bootstrapping
00:21:22.600 --> 00:21:27.410
phase. AMD calls it "AGESA BootLoader"
(ABL) and it's not just a single binary,
00:21:27.410 --> 00:21:32.220
but it hosts a binary which loads binaries
from the flash furthermore, and then
00:21:32.220 --> 00:21:37.090
executes it in a specific order. So, you
see here ABL one, two, three, four and
00:21:37.090 --> 00:21:41.780
six. ABL five is used for something like a
warm resume from suspend to RAM, for
00:21:41.780 --> 00:21:48.440 line:1
example. So, later on, if the SEV app is
for example loaded, if the OS requests a
00:21:48.440 --> 00:21:55.370
specific SEV functionality and not before
that. Because we have the separation
00:21:55.370 --> 00:21:58.950
between supervisor and user mode, we
obviously need a way that the app can
00:21:58.950 --> 00:22:02.940
communicate with the off-chip bootloader
and that is done using the ARM instruction
00:22:02.940 --> 00:22:09.750
"Supervisor Call" or "SVC". So we
identified 76 syscalls in total. We have
00:22:09.750 --> 00:22:14.200
mostly reverse-engineered 30 by now. We
can access the x86 memory. We can
00:22:14.200 --> 00:22:18.740
communicate with other PSPs in a system.
We can load entries from flash and so on.
00:22:18.740 --> 00:22:24.400
28 are partly reverse-engineered. Those
are mostly CCP operations for RSA public
00:22:24.400 --> 00:22:28.169
key verification, AES encryption and so
on. And there are also more elaborate
00:22:28.169 --> 00:22:32.430
functions to communicate with other PSPs
which are required during the AGESA
00:22:32.430 --> 00:22:37.080
BootLoader stage. And then, we have 18
left, and these we don't know about yet
00:22:37.080 --> 00:22:41.939
because they are not called at all or
they have exactly one call site and are
00:22:41.939 --> 00:22:45.844
non-trivial to reverse-engineer.
00:22:45.844 --> 00:22:48.860
So, "System Management Network".
I already saw on the
00:22:48.860 --> 00:22:52.580
slide already that there was access SMN.
If you Google for "System Management
00:22:52.580 --> 00:22:57.940
Network" or SMN, you won't find much
information about it by AMD or otherwise.
00:22:57.940 --> 00:23:02.190
The only reference you may find is code in
the Linux kernel to read out the thermal
00:23:02.190 --> 00:23:06.750
sensors on the CPU. So the System
Management Network actually is a hidden
00:23:06.750 --> 00:23:11.220
control network inside your CPU. Each and
every hardware block which is in there is
00:23:11.220 --> 00:23:16.330
connected to it and is used for the PSP to
control and initialize the hardware blocks
00:23:16.330 --> 00:23:21.300
during the boot up phase. So it is a
dedicated address space, so the PSP can't
00:23:21.300 --> 00:23:26.630
directly access it using MMIO
instructions. And we have the PSP there.
00:23:26.630 --> 00:23:30.820
We have identified the memory controller,
the System Management Unit for which there
00:23:30.820 --> 00:23:35.560 line:1
was a talk about I think two years ago on
this very Congress, the x86 cores are
00:23:35.560 --> 00:23:41.360 line:1
there as well and a lot of other things we
didn't reverse engineer so far. One other
00:23:41.360 --> 00:23:45.910 line:1
thing. OK. So to access the System
Management Network, the PSP has to map a
00:23:45.910 --> 00:23:49.820 line:1
certain region of the System Management
Network address space into its own address
00:23:49.820 --> 00:23:53.929 line:1
space and then can access the register,
write, read and so on. And it has to unmap
00:23:53.929 --> 00:23:58.370 line:1
it again. And one of the functions we
identified is what we call memory
00:23:58.370 --> 00:24:04.620
protection slots. So the PSP has the
possibility [stutters] to configure the
00:24:04.620 --> 00:24:09.610
memory controller, to revoke access to
certain regions of the DDR4 memory from
00:24:09.610 --> 00:24:14.211
the x86 cores. This is done by using three
registers. We have a start register with a
00:24:14.211 --> 00:24:18.450
physical start address, an end register to
denote the physical end address of the
00:24:18.450 --> 00:24:22.549
region you want to protect, and a control
register where we only know yet so far the
00:24:22.549 --> 00:24:26.620
enable bit to flip it on or off. And what
it does is, if the protection is flipped
00:24:26.620 --> 00:24:31.110
on, the x86 will only read "all bits
set"[?] when it tries to access this
00:24:31.110 --> 00:24:36.320
particular region and writes will have no
effect through this region as well. And
00:24:36.320 --> 00:24:40.600
this is, for example, used for the system
management mode UEFI code, and for certain
00:24:40.600 --> 00:24:44.795
functionality for the Secure Encrypted
Virtualization feature of AMD.
00:24:46.965 --> 00:24:50.015
So, the next thing we did was running
00:24:50.015 --> 00:24:53.510
[the] `strings` [command] over all
modules, obviously. And, what we found
00:24:53.510 --> 00:24:57.520
there were a lot of interesting debug
strings and even a lot of format strings.
00:24:57.520 --> 00:25:02.320
And, we wanted to know what the values
were during the runtime. So, when we
00:25:02.320 --> 00:25:06.140
disassembled the firmware and analyzed it,
we saw that most of these strings were
00:25:06.140 --> 00:25:09.949
referenced right before a special call
called SVC 6, so this must be some sort of
00:25:09.949 --> 00:25:16.260
debug print for the PSP. The problem is,
SVC 6 is not implemented in the release
00:25:16.260 --> 00:25:22.210
firmware. So, we had to find another way
to gain access to these debug strings. And
00:25:22.210 --> 00:25:28.650
this is what I will talk about now. So,
the problem here is, first we need to know
00:25:28.650 --> 00:25:34.770
where we want to store these debug strings,
and we don't have any x86 memory available
00:25:34.770 --> 00:25:39.030
at this time in the process. So we need to
find another device or buffer where you
00:25:39.030 --> 00:25:44.929
can store it for later use. But, the only
device we did know about at this time was
00:25:44.929 --> 00:25:50.669
the SPI flash. Luckily for us, right into
this SPI flash area from, the PSP
00:25:50.669 --> 00:25:57.559
generated the necessary bus cycles on the
SPI bus, without altering the flash. Then
00:25:57.559 --> 00:26:02.030
we need a code execution on the PSP to
inject our own SVC handler. And how we
00:26:02.030 --> 00:26:05.850
gained code execution, Robert will talk
about in the third part of this talk. But
00:26:05.850 --> 00:26:09.559
for now, we assume that we have code
execution on the PSP already, can inject
00:26:09.559 --> 00:26:16.539
our own SVC 6 handler and then leave, let
it run. So the app will call SVC 6, it
00:26:16.539 --> 00:26:20.179
will be forwarded on to the SPI bus where
we can collect it with our already
00:26:20.179 --> 00:26:25.760
existing setup. [We] use a tool to filter
the debug strings from the rest of the
00:26:25.760 --> 00:26:30.610
traffic on the SPI bus [that] we don't
want to have [...] in the debug output and
00:26:30.610 --> 00:26:38.860
then hopefully get a raw PSP log. And we
had success with that. So what you see
00:26:38.860 --> 00:26:43.789
here is the initial boot-up or the very
first stage of the boot-up state. The logs
00:26:43.789 --> 00:26:49.170
are several megabytes long and we didn't
have the chance to go through all of them.
00:26:49.170 --> 00:26:57.010
So, there is a lot of interesting stuff
hiding there already.
00:26:57.010 --> 00:27:05.440
applause
So, the next step was to explore what is
00:27:05.440 --> 00:27:09.510
hidden inside the System Management
Network. And we didn't want to always
00:27:09.510 --> 00:27:13.430
reflash the whole system all the time and
write code for it, debug, because that is
00:27:13.430 --> 00:27:20.669
error prone and tedious. So we created our
own setup where we could dynamically use
00:27:20.669 --> 00:27:25.280
the x86 calls on the system to write and
read from the System Management Network.
00:27:25.280 --> 00:27:29.609
For that, we replaced the SEV app with a
stub and the stub provides three
00:27:29.609 --> 00:27:33.427
primitives. We can read-write a System
Management Network address, we can execute
00:27:33.427 --> 00:27:37.850
an arbitrary syscall from the off-chip
bootloader and we can read-write general
00:27:37.850 --> 00:27:45.289
PSP memory. And because the PSP is exposed
as a separate PCIe device to the x86, we
00:27:45.289 --> 00:27:49.650
use the existing Linux kernel driver and
modified it to expose these requests to
00:27:49.650 --> 00:27:53.850
user land, where we created a user space
library wrapper and some Python bindings.
00:27:53.850 --> 00:27:58.970
And with that we were able to use a Python
shell to dynamically read, write
00:27:58.970 --> 00:28:02.861
registers, headers, spurious reboot in
between if you did the wrong thing, but
00:28:02.861 --> 00:28:06.920
could start over very quickly. So what you
see here in the code snippet is, what we
00:28:06.920 --> 00:28:12.480 line:1
did to discover what these memory
protection slots where about. You can see
00:28:12.480 --> 00:28:16.480 line:1
that we call an syscall handler, that we
write some System Management Network
00:28:16.480 --> 00:28:20.890 line:1
address and so on. And we do it for all
the different PSPs in the system, so the
00:28:20.890 --> 00:28:25.550 line:1
master PSP can also forward these requests
to all of the other PSPs in the whole
00:28:25.550 --> 00:28:34.220
system. Next thing, we wanted to also
analyze the SEV app further and see how
00:28:34.220 --> 00:28:39.600
the code is executed and how the data
flows in this SEV app. But because we
00:28:39.600 --> 00:28:43.730
already had a PSP stub running there and
couldn't share it on the PSP, we had to
00:28:43.730 --> 00:28:48.730
find another method. And we created a PSP
emulator for that and using our
00:28:48.730 --> 00:28:55.010
libpspproxy to forward requests onto the
PSP. So the current state can run the SEV
00:28:55.010 --> 00:29:00.890
app to a certain point and we are still
actively developing that. So, that started
00:29:00.890 --> 00:29:08.549
a few weeks ago, and this will continue in
the development. So, what it does is, what
00:29:08.549 --> 00:29:12.590
you see here is the AMD sev-tool to manage
the host and configure all the keys and
00:29:12.590 --> 00:29:16.320
certificates on the system. And we
modified the Linux kernel driver to
00:29:16.320 --> 00:29:21.039
reroute these requests out to our own PSP
emulator running in user space, which is
00:29:21.039 --> 00:29:25.200
based on the unicorn engine. Any hardware
access, because we don't know much about
00:29:25.200 --> 00:29:29.450
the hardware yet, is forwarded to the real
PSP, results are collected, and when the
00:29:29.450 --> 00:29:35.080
SEV app finishes, it will return the
result back to the AMD sev-tool. And with
00:29:35.080 --> 00:29:39.580
that we are able to execute some of the
requests the SEV app implements
00:29:39.580 --> 00:29:46.130
successfully so far. Yeah. What you see
here is a small snippet from one of the
00:29:46.130 --> 00:29:50.600
traces. You can see a syscall being made.
It's a CCP request. We don't know
00:29:50.600 --> 00:29:55.450
exactly how the arguments are used by now.
That's why there's a lot of unknown stuff,
00:29:55.450 --> 00:29:59.990
but this will aid us in development. And
furthermore, in addition to allowing a
00:29:59.990 --> 00:30:04.070
tracing code execution and observe the
data flow, we later on may be able to
00:30:04.070 --> 00:30:08.240 line:1
provide functionality which is currently
only available on the EPYC server platform
00:30:08.240 --> 00:30:12.600 line:1
from AMD, like Secure Encrypted Virtual
machine. The problem here is we don't know
00:30:12.600 --> 00:30:16.200
yet if all the hardware is there which is
supported, and whether it's only a
00:30:16.200 --> 00:30:18.410
firmware limitation by AMD.
00:30:20.770 --> 00:30:24.796
If you're interested, the code is here
on the repository,
00:30:24.796 --> 00:30:27.980
it will be made available in
the next few days. We have a number of
00:30:27.980 --> 00:30:32.630
repositories available. You already saw
PSPTool. We have some repository where we
00:30:32.630 --> 00:30:36.550
collect documentation about hardware
interfaces, syscalls and so on. We have
00:30:36.550 --> 00:30:41.640
our PSP emulator there and also the psp-
apps repository, if you want to dive into
00:30:41.640 --> 00:30:47.110
writing your own apps for the PSP. And
with that I will hand over to Robert, who
00:30:47.110 --> 00:30:51.870
will talk about how we gained code
execution on the PSP itself.
00:30:51.870 --> 00:30:59.108
applause
00:30:59.108 --> 00:31:02.200
Robert: OK, so for everything that Alex
talked about,
00:31:02.200 --> 00:31:15.159
we need code execution on the PSP.
… [inaudible]. Mike? Better? All right.
00:31:15.159 --> 00:31:22.179
So, this part of owning the PSP is again
split into two parts. Now, Christian
00:31:22.179 --> 00:31:27.179
already talked about the firmware and the
SPI flash. So, this is something we can
00:31:27.179 --> 00:31:31.110
control because we have physical access to
the device. We can flash everything we
00:31:31.110 --> 00:31:36.909
want. So, what can we do with that? So, on
the SPI flash, we have these directories
00:31:36.909 --> 00:31:41.770
which have a header and entries and an
entry is actually compromised (composed)
00:31:41.770 --> 00:31:48.070
of an ID, an address and a size. We've
talked about files. So an entry could be a
00:31:48.070 --> 00:31:54.419
reference to a file. And, we also talked
about these secondary directories. So, an
00:31:54.419 --> 00:31:59.440
entry could refer to another directory.
Now, if you look at the files you see that
00:31:59.440 --> 00:32:04.240
they have a signature usually. So, we
cannot manipulate those files directly. If
00:32:04.240 --> 00:32:08.040
we touch them, this will be noticed and
they won't be loaded and the system will
00:32:08.040 --> 00:32:13.620 line:1
immediately reboot. Now, what we can
manipulate is the directories themselves,
00:32:13.620 --> 00:32:19.400 line:1
because they are not protected at all. So,
specifically, what we can do is, we can,
00:32:19.400 --> 00:32:24.730 line:1
for example, add additional entries. These
entries might point to the same files.
00:32:24.730 --> 00:32:29.260 line:1
That doesn't matter. We can add entries.
What we also can do is, we can remove some
00:32:29.260 --> 00:32:35.860 line:1
of those entries or we can change entries.
So, for example, this reference to the
00:32:35.860 --> 00:32:40.679 line:1
secondary directory, this has a size
parameter. Right. And this size refers to
00:32:40.679 --> 00:32:44.480
the size of that directory. And actually,
what we can do is, we can change that
00:32:44.480 --> 00:32:49.230
size. So we can make the directory appear
to be smaller without removing any of
00:32:49.230 --> 00:32:57.140
those entries. Now, during boot, this PSP
directory, that Christian already talked
00:32:57.140 --> 00:33:03.159
about, is parsed. So this PSP directory
contains, among other things, the
00:33:03.159 --> 00:33:07.289
reference to the AMD public key, which is
used to authenticate all the applications
00:33:07.289 --> 00:33:12.840 line:1
which are loaded. Now, this directory also
has a secondary directory. The content is
00:33:12.840 --> 00:33:18.890 line:1
not really relevant here. So the on-chip
bootloader that executes first will set up
00:33:18.890 --> 00:33:23.680
this boot ROM service page that Alex
talked about. And this boot ROM service
00:33:23.680 --> 00:33:31.900
page contains a copy of those directory
entries, just for the first directory. And
00:33:31.900 --> 00:33:36.270 line:1
also the on-chip bootloader will copy the
AMD public key itself to the boot room
00:33:36.270 --> 00:33:42.529 line:1
service page. So it only copies the AMD
public key if it's been verified before.
00:33:42.529 --> 00:33:47.580 line:1
OK. So now this boot room service page
contains this AMD public key and this
00:33:47.580 --> 00:33:54.050 line:1
public key in memory is from then on used
to authenticate applications. So the off-
00:33:54.050 --> 00:33:59.920 line:1
chip bootloader, which executes later,
will use that boot ROM service page and
00:33:59.920 --> 00:34:05.070 line:1
will extend it. Specifically, it will copy
the entries of the secondary directory to
00:34:05.070 --> 00:34:10.909 line:1
that boot ROM service page. So I guess you
can already see where this is going.
00:34:10.909 --> 00:34:16.424
So, what could possibly go wrong here?
Laughter
00:34:16.424 --> 00:34:20.700 line:1
Well, we have space for 64 entries here.
And if
00:34:20.700 --> 00:34:26.530 line:1
we write more entries to that page, we'll
hit the AMD public key. So the off-chip
00:34:26.530 --> 00:34:32.309 line:1
bootloader should better check that we
only copy at most 64 entries. There it is.
00:34:32.309 --> 00:34:37.129 line:1
There is a check. Let's say this is the
function that appends entries and it says:
00:34:37.129 --> 00:34:43.409 line:1
okay, if the number of entries exceeds 64,
we return an error code and do not copy.
00:34:43.409 --> 00:34:48.889 line:1
Sounds good. Thing is, that number refers
to the number of entries in the secondary
00:34:48.889 --> 00:34:56.919 line:1
directory. So this has a maximum size of
64. But there is already space, there are
00:34:56.919 --> 00:35:00.749 line:1
entries there on this boot ROM service
page. So, actually, what we enforce with
00:35:00.749 --> 00:35:07.690 line:1
this check is, whatever we append can have
at most 64 entries, and within that 64
00:35:07.690 --> 00:35:13.580 line:1
entries, well, there's the AMD public key.
Super convenient. So what we do now, we
00:35:13.580 --> 00:35:18.309 line:1
place our own public key inside the
directory structures of the firmware file
00:35:18.309 --> 00:35:24.579 line:1
system. The off-chip bootloader copies the
entries and copies the AMD public key.
00:35:24.579 --> 00:35:35.369 line:1
Applause
So what does it mean for us? Now, all this
00:35:35.369 --> 00:35:41.469
parsing happens before the first
application is loaded. So that means we
00:35:41.469 --> 00:35:46.180
control the very first application and can
replace the content. And from there on, we
00:35:46.180 --> 00:35:50.239
control the userland part of the Secure
Processor.
00:35:51.409 --> 00:35:54.299
So, now coming to the next part.
00:35:54.299 --> 00:35:59.809
So, the natural next target is, of course,
I mean, we have userland code execution,
00:35:59.809 --> 00:36:06.609
we want to have the rest. Kernel mode. So,
how can we take over the kernel mode? Now,
00:36:06.609 --> 00:36:11.399
let's have a look at how this distinction
between kernel and user mode happens. So,
00:36:11.399 --> 00:36:15.990
if we look at the virtual memory layout,
we'll see that there is a user mode part
00:36:15.990 --> 00:36:21.690
and a fixed split with the kernel mode
where our off-chip bootloader resides. So,
00:36:21.690 --> 00:36:26.560
our application, which we already control,
can try to access that memory, of course,
00:36:26.560 --> 00:36:30.329
but that won't work. Right. The MMU
will prevent any access to privileged
00:36:30.329 --> 00:36:39.729
memory. Okay. So let's see how this works
at runtime. So, this bootloader component,
00:36:39.729 --> 00:36:43.819
if we specify the privileged memory a
little bit more, we have code and data
00:36:43.819 --> 00:36:48.959
there. And at runtime another type of
directory is parsed. And this is called
00:36:48.959 --> 00:36:53.409
the BIOS directory. I mean, it's a similar
structure as the directory before. We have
00:36:53.409 --> 00:36:58.019
entries and the reference to a secondary
directory. The entries here, again, of no
00:36:58.019 --> 00:37:04.900
relevance. So during boot, the off-chip
bootloader will copy those entries into
00:37:04.900 --> 00:37:10.479
its data section. OK? So, for the copy
operation, we need some some information.
00:37:10.479 --> 00:37:15.869
So, let's say this is the copy operation,
kind of looks like `memcopy`. What we need
00:37:15.869 --> 00:37:21.410
is destination, where to copy? We need
source. This is the secondary directory,
00:37:21.410 --> 00:37:26.599
this is the thing we want to copy, which
is already under our control. So,
00:37:26.599 --> 00:37:33.160
convenient, we control whatever data is
copied. And, we need a size value. So,
00:37:33.160 --> 00:37:39.619
where do we get that size? Oh yeah, this
entry here has a size value. Super. It's
00:37:39.619 --> 00:37:44.720
ours also, right? We control the directory
structures. We can manipulate the size. So
00:37:44.720 --> 00:37:49.229
to sum up, we have a copy operation into
privileged memory with attacker-controlled
00:37:49.229 --> 00:37:56.690
data and attacker-controlled size. This is
a very old meme, and I think it's
00:37:56.690 --> 00:38:02.170
appropriate because this this bug is so
easy to prevent, actually. But for us it's
00:38:02.170 --> 00:38:09.819
good, because now we control everything in
red here. So, we control that part. The
00:38:09.819 --> 00:38:16.049
thing is, as you can see, code is not part
of what we control. So, what might be here?
00:38:16.049 --> 00:38:26.910
What is of interest for us to overwrite?
Thing is, it's the page tables. The page
00:38:26.910 --> 00:38:31.170
tables are part of the data section within
the privileged part of the virtual memory
00:38:31.170 --> 00:38:36.640
space. So again, what we do, we place our
own page tables here. The data is copied
00:38:36.640 --> 00:38:41.680
and replaces the page tables in memory of
the Secure Processor. So, now, if we look
00:38:41.680 --> 00:38:47.710
at that virtual memory overview again,
well, our page tables define the virtual
00:38:47.710 --> 00:38:53.529
memory a bit different. We make everything
user-writeable. So, we control the
00:38:53.529 --> 00:38:58.760
application, our application now can touch
the privileged memory and just overwrite
00:38:58.760 --> 00:39:04.140
everything there, if we want to. For that,
we need to reimplement everything. But, we
00:39:04.140 --> 00:39:09.869
can patch now the Secure Operating System,
if we want.
00:39:09.869 --> 00:39:16.979
Applause
00:39:16.979 --> 00:39:19.330
So, that means, this parsing of the
00:39:19.330 --> 00:39:23.170
directory also happens before the first
application. So, we control the first
00:39:23.170 --> 00:39:27.589
application, that takes over the
bootloader, if you want. And from there
00:39:27.589 --> 00:39:34.249
on, we have everything. All those issues I
presented were fixed, were even fixed
00:39:34.249 --> 00:39:39.430
before we discovered them. Right? So, we
might not be the first one that discovered
00:39:39.430 --> 00:39:43.160
them. Some of you (may) remember that
there was some web site called
00:39:43.160 --> 00:39:49.200
AMDFlaws[.com]. They did not present too
many technical details. Maybe what they
00:39:49.200 --> 00:39:54.870
discovered was something I present here. I
don't know. Thing is, it does not really
00:39:54.870 --> 00:39:58.340
matter for us because the Secure Processor
does not implement any rollback
00:39:58.340 --> 00:40:03.650
prevention. So we can always go back and
refresh a vulnerable firmware. And from
00:40:03.650 --> 00:40:12.089
that, use whatever code we want to place
there. So, what what we did is, we used
00:40:12.089 --> 00:40:18.789
all this on an Epyc Naples based server
system. And, you cannot just use that
00:40:18.789 --> 00:40:25.019
issue on every AMD system, because the
bootloader we're using was signed with a
00:40:25.019 --> 00:40:31.670
key specific for the Epyc Naples CPU
series. However, we believe, we have not
00:40:31.670 --> 00:40:35.900
tested it thoroughly yet, but we believe
the same kind of issues exist in
00:40:35.900 --> 00:40:43.130
bootloaders which are signed with a Ryzen
first generation key. And, for the rest,
00:40:43.130 --> 00:40:48.329
we don't know yet. So, maybe for
Threadripper or Epyc Rome, there are
00:40:48.329 --> 00:40:54.480
similar issues, maybe not. We don't know.
So the question is, is this really a
00:40:54.480 --> 00:41:00.190
security issue? I mean, of course it's a
security issue, but, for whom? So,
00:41:00.190 --> 00:41:06.519
everything we did requires physical access
to the device. So, if it were my laptop,
00:41:06.519 --> 00:41:12.910
personally, I wouldn't be concerned too
much. However, there are some things where
00:41:12.910 --> 00:41:17.880
this is a real issue. For example, if you
rely on Secure Boot. Because the Secure
00:41:17.880 --> 00:41:22.400
Processor is the first part that boots up,
and if that is broken, everything later on
00:41:22.400 --> 00:41:28.391 line:1
is also broken. So, Christian already told
you that AMD plans to use this Secure
00:41:28.391 --> 00:41:32.089 line:1
Processor [as] a trusted execution
environment. If your application relies on
00:41:32.089 --> 00:41:38.130 line:1
that, you better not have any security
issues in that Secure Processor. And, for
00:41:38.130 --> 00:41:43.759 line:1
the last part, the Secure Encrypted
Virtualization technology from AMD is
00:41:43.759 --> 00:41:48.819 line:1
dependent on the integrity of the Secure
Processor. If that is broken, this
00:41:48.819 --> 00:41:54.299 line:1
technology is also broken. So, Christian
and I published a paper about that. If
00:41:54.299 --> 00:42:00.259
you're interested, you can read it up.
But, for us here, this is actually more of
00:42:00.259 --> 00:42:04.859
an opportunity, right? Because we can gain
more insight into this PSP with code
00:42:04.859 --> 00:42:10.690
execution. We can do a lot of cool things
with that. So, it allows to do further
00:42:10.690 --> 00:42:15.950
research on other subsystems which are
present in the AMD CPUs. For example, the
00:42:15.950 --> 00:42:22.499
PSP is responsible to load the SMU
firmware. The PSP allows access to the SMM
00:42:22.499 --> 00:42:28.940
mode. So, this is a "ring -2 mode" on the
x86 CPUs. So, [it is] higher privileged
00:42:28.940 --> 00:42:34.749
than your kernel, and there is proprietary
code running in that mode. With the PSP,
00:42:34.749 --> 00:42:40.589
you have access to that code and could
replace it, analyze it, whatever. And, the
00:42:40.589 --> 00:42:44.230
PSP is responsible to kick off the x86
calls at all. So everything that comes
00:42:44.230 --> 00:42:50.785
later is, in theory, now under our
control. Thank you. That's it.
00:42:50.785 --> 00:43:00.250
Applause
00:43:00.250 --> 00:43:03.789
Herald: Yes. Thank you very much, Robert,
Alexander and Christian. That was
00:43:03.789 --> 00:43:10.140
fantastic. Wow. I have a lot of questions
I guess [in my?] head going on. But do we
00:43:10.140 --> 00:43:13.724
have any questions from the audience? And
if you have any questions, we have
00:43:13.724 --> 00:43:18.380
microphones lined up here. A question is,
just so that you know what we're talking
00:43:18.380 --> 00:43:22.930
about with questions, is a sentence with a
question mark behind it and not your life
00:43:22.930 --> 00:43:29.190
story. And I think I saw number one first.
So, let's start with number one.
00:43:29.190 --> 00:43:34.999
Mic 1: Hey, is there a reason why the page
table is located at the end of the data
00:43:34.999 --> 00:43:38.279
segment?
Robert: I don't think so. I mean, ...
00:43:38.279 --> 00:43:41.264
Mic 1: "Just because"?
00:43:41.264 --> 00:43:44.559
Robert: You have to place it somewhere.
should be in the [interrupted]
00:43:44.559 --> 00:43:48.789
Mic 1: Why not in the beginning?
Robert: I don't know. No idea.
00:43:48.789 --> 00:43:53.200
Herald: That's what I meant with "a lot of
weird questions" here. From the signal
00:43:53.200 --> 00:43:56.229
angel we had one question.
Signal Angel: And this question goes to
00:43:56.229 --> 00:44:02.209
the first lecturer. Didn't you have access
to an SPI flasher relay(?) to attempt a
00:44:02.209 --> 00:44:06.599
"Time of Use versus Time of Check"
[TOCTOU] attack?
00:44:06.599 --> 00:44:13.809
Christian: So, we had access to different
tools, but the TOCTOU attack that you
00:44:13.809 --> 00:44:21.089
mentioned was not even necessary to mount
the attacks we talked about. And actually
00:44:21.089 --> 00:44:26.589
so far, we don't see any possibility to
mount a TOCTOU attack.
00:44:26.589 --> 00:44:34.029
Herald: OK. So I think I saw microphone 5
next up. Is there somebody at the
00:44:34.029 --> 00:44:37.039
Microphone?
Mic 5: Yes. So, I was wondering if you
00:44:37.039 --> 00:44:40.329
considered looking at the boot ROM for
issues.
00:44:40.329 --> 00:44:49.009
Robert: Yes, of course. The thing is, we
cannot find its code in the memory any
00:44:49.009 --> 00:44:55.809
more after we mounted our attacks. So, I
believe, the boot ROM code is not there
00:44:55.809 --> 00:45:03.009
anymore, which would make it much easier
to analyze. We tried simple things, like
00:45:03.009 --> 00:45:09.239
increasing directory sizes, which are
processed by the boot ROM itself. We
00:45:09.239 --> 00:45:12.680
haven't found any suspicious thing there,
yet.
00:45:12.680 --> 00:45:18.469
Herald: Microphone 2.
Mic 2: Thanks for your research. You have
00:45:18.469 --> 00:45:26.390
really nice big power over the system
right now. Do you have plans to make a PSP
00:45:26.390 --> 00:45:35.910
firmware which is minimal and which makes
your system work, but without some strange
00:45:35.910 --> 00:45:43.079
untrusted code?
Robert: I wouldn't call it plans yet. Of
00:45:43.079 --> 00:45:46.839
course there are ideas to do that. The
thing is, some of the functionality which
00:45:46.839 --> 00:45:52.020
is implemented from AMD is really
required. So, the stages that Alex talked
00:45:52.020 --> 00:45:58.049
about, they configure and train(?) your
DRAM. So without those stages, you don't
00:45:58.049 --> 00:46:04.839
have access to memory. Your x86 cores
wouldn't work. And to reimplement that
00:46:04.839 --> 00:46:10.479
without having access to any manuals is
really, really hard work. So, I'm not too
00:46:10.479 --> 00:46:13.680
confident that this will be possible in
the near future.
00:46:13.680 --> 00:46:19.249
Mic 2: I just refer to a Management Engine
cleaner, and there is such a project,
00:46:19.249 --> 00:46:23.760
which makes your Management Engine
firmware slim.
00:46:23.760 --> 00:46:30.039
Robert: So, the AMD firmware is already
kind of slim. The only thing that is not
00:46:30.039 --> 00:46:35.709
strictly required on the systems we have
been looking at would be the SEV firmware,
00:46:35.709 --> 00:46:40.680
which is loaded on request, and you can,
like, disable that by just flipping a bit
00:46:40.680 --> 00:46:47.190
inside that file. The system would still
boot, but when it tries to initialize the
00:46:47.190 --> 00:46:52.180
SEV technology, the kernel would say, "OK.
This does not work." The system will still
00:46:52.180 --> 00:46:56.109
work after that.
Mic 2: Thanks. And last little question.
00:46:56.109 --> 00:47:03.479
Does PSP work with microcode somehow?
Alexander: We didn't find anything related
00:47:03.479 --> 00:47:06.220
to any microcode there so far.
Mic 2: Thanks.
00:47:06.220 --> 00:47:11.819
Herald: So let's move on to Microphone 3.
Mic 3: Thank you first for the great talk.
00:47:11.819 --> 00:47:17.839
I have one question. Do you have maybe
found something evil or potentially evil
00:47:17.839 --> 00:47:23.670
in the code that it does?
Alexander: No. So far, they didn't find
00:47:23.670 --> 00:47:30.380
anything which could be used for an
attack, for example. So, what the PSP
00:47:30.380 --> 00:47:35.940
might be able to do is, access PCIe
devices. We found some code related to
00:47:35.940 --> 00:47:41.210
that, but we are [stutters] not sure yet
whether it's actually used, because also
00:47:41.210 --> 00:47:46.930
the PSPs executed or is existing on graphics
cards made by AMD. So, that might be also
00:47:46.930 --> 00:47:51.069
ready to[?] that. We couldn't find
anything there yet, but so far, the PSP
00:47:51.069 --> 00:47:53.959
looks rather clean compared to the entire
Management Engine.
00:47:53.959 --> 00:47:58.089
Mic 3: Thank you.
Herald: So, we have a question from the
00:47:58.089 --> 00:48:03.119
Internet.
Signal: Is the AMD public key an RSA one,
00:48:03.119 --> 00:48:08.680
only 576 bits?
Robert: It's an RSA key, yes, but it's
00:48:08.680 --> 00:48:17.030
2048 bits for the first generation Epyc
CPUs and I think 4069 [meaning 4096] for
00:48:17.030 --> 00:48:20.259
later generations.
Herald: Microphone 2.
00:48:20.259 --> 00:48:27.469
Mic 2: For me, it seems like preventing to
flash old vulnerable firmware is really
00:48:27.469 --> 00:48:33.069
important for a scenario like Secure
Encrypted Virtualization. Can you comment
00:48:33.069 --> 00:48:40.430
on how difficult it is for AMD to add this
retrospectively?
00:48:40.430 --> 00:48:47.509
Robert: Okay. So technically, rollback
prevention is there for, I guess, mobile
00:48:47.509 --> 00:48:53.220
devices, for example. You have that. It
should be possible. For adding this
00:48:53.220 --> 00:48:57.439
functionality afterwards, I don't think
that's really possible, because the on-
00:48:57.439 --> 00:49:02.670
chip bootloader is the thing that loads
the off-chip bootloader and verifies it.
00:49:02.670 --> 00:49:09.079
And that software component has to, like,
stop loading if the firmware version does
00:49:09.079 --> 00:49:14.069
not match, for example. And you have to
change that. And that functionality is not
00:49:14.069 --> 00:49:18.539
there and you cannot update the on-chip
boot ROM. So, in that sense, I don't think
00:49:18.539 --> 00:49:24.089
that that's possible to change. And if you
look at our paper, you will see that the
00:49:24.089 --> 00:49:29.289
former issues are kind of devastating for
the SEV technology, because there are some
00:49:29.289 --> 00:49:36.209
keys which are now accessible, which can
be used for attacking SEV-protected
00:49:36.209 --> 00:49:38.920
guests.
Mic 2: Thanks.
00:49:38.920 --> 00:49:45.380
Herald: Microphone 3, please.
Mic 3: One question. Did you analyze the
00:49:45.380 --> 00:49:54.640
API to the x86 core? Did you find anything
that could be exploited without flashing
00:49:54.640 --> 00:49:59.690
anything so that you could directly go
from x86 to PSP exploitation?
00:49:59.690 --> 00:50:06.599
Alexander: Yeah, we tried to find the
necessary code to interface with the x86.
00:50:06.599 --> 00:50:12.179
We think we found one place where the x86
cores are released after the PSP
00:50:12.179 --> 00:50:15.900
initialized the whole system. But
obviously, we can't do much with it except
00:50:15.900 --> 00:50:21.779
preventing the x86 to boot at all. And
otherwise we couldn't find anything there
00:50:21.779 --> 00:50:26.699
yet. So we focused on, on a bit of other,
like the memory controller, and didn't
00:50:26.699 --> 00:50:32.869
have a deeper look at the x86 interface.
So what there is, the BIOS can interface
00:50:32.869 --> 00:50:38.719
with the PSP using a special mailbox
register which is mapped in MMIO space in
00:50:38.719 --> 00:50:44.139
x86 for requests. So, it can, for example,
the UEFI init boots, it will say to the
00:50:44.139 --> 00:50:48.029
PSP "Hey, this is my system management
mode code region, please protect that for
00:50:48.029 --> 00:50:52.910
me" and it will execute this request. But
apart from that, we couldn't find anything
00:50:52.910 --> 00:50:55.039
so far.
Mic 3: Thank you.
00:50:55.039 --> 00:51:00.200
Herald: So, Microphone 4.
Mic 4: Hi. So, is it correct that your
00:51:00.200 --> 00:51:08.651
work enables 100% open source firmware for
this kind of processors? And if so, have
00:51:08.651 --> 00:51:12.979
you already contacted the CoreBoot team to
make that actually happen?
00:51:12.979 --> 00:51:21.849
Robert: So. 100 percent open source. As
for the PSP, there is this on-chip boot
00:51:21.849 --> 00:51:27.240
ROM which we can't replace, right? So,
this will be closed source. Then there is
00:51:27.240 --> 00:51:33.099
code of the off-chip bootloader, until the
first exploit, which runs, which is not
00:51:33.099 --> 00:51:38.719
open source. In theory, you could from now
on take over the PSP, write your own code.
00:51:38.719 --> 00:51:44.150
But, as I said before, you have to
reimplement a lot of functionality without
00:51:44.150 --> 00:51:49.369
having any documentation, right? So,
technically it's possible, I guess, to do
00:51:49.369 --> 00:51:54.089
something like that. Practically, I'm not
too sure.
00:51:54.089 --> 00:51:57.390
Herald: So we're gonna go to the internet
for another question.
00:51:57.390 --> 00:52:03.039
Signal: Is it possible to block PSP from
within Linux or BSD, for the system's
00:52:03.039 --> 00:52:09.130
runtime, by using search and boot flags?
Robert: Sorry, to block what?
00:52:09.130 --> 00:52:14.630
Signal: To block the PSP from the Linux or
BSD.
00:52:14.630 --> 00:52:19.680
Alexander: So, what you can do is, like
Robert mentiond already, you can flip a
00:52:19.680 --> 00:52:25.189
bit in the SPI flash and then the PSP,
once it initialized the whole system, it
00:52:25.189 --> 00:52:29.719
won't run the SEV app, for example,
because the signatures won't match
00:52:29.719 --> 00:52:36.079
anymore. And there is no other sort of
interface where the PSP is actually
00:52:36.079 --> 00:52:41.499
triggered. Or we couldn't find it so far.
Herald: Microphone 3.
00:52:41.499 --> 00:52:45.500
Someone: I think he was first.
Herald: Oh, okay, all right. Right.
00:52:45.500 --> 00:52:49.099
Microphone 2 then. Sorry.
Mic 2: Did you try to enable any
00:52:49.099 --> 00:52:56.420
superpowers from PSP like JTAG or special
tricks with voltage or something else?
00:52:56.420 --> 00:53:01.579
Robert: When the first application that is
loaded has some strings in it like Debug
00:53:01.579 --> 00:53:08.319
Unlock. Sounds interesting. But then
again, JTAG, where would you access the
00:53:08.319 --> 00:53:13.190
JTAG of the PSP? You need to have some
connection to the lines, right?
00:53:13.190 --> 00:53:18.130
Mic 2: Intel supports USB debugging.
Robert: Yeah, I know. With special
00:53:18.130 --> 00:53:21.209
devices, right?
Mic 2: No, even wire cable.
00:53:21.209 --> 00:53:26.589
Robert: Okay. So anyhow, I have the
suspicion that this DebugUnlock app is
00:53:26.589 --> 00:53:32.069
responsible to to allow some debug mode.
Which then, I assume, with special
00:53:32.069 --> 00:53:36.880
hardware, you can have JTAG. But we have
not touched it yet.
00:53:36.880 --> 00:53:39.469
Mic 3: Thanks.
Herald: Now Microphone 3
00:53:39.469 --> 00:53:45.619
Mic 3: So I'm as far from a liar
laughing, um, a lawyer as possible, but
00:53:45.619 --> 00:53:51.650
could AMD in any way file a cease and
desist for anything you do?
00:53:51.650 --> 00:53:57.089
Robert: Probably not, I guess.
Mic 3: Just curious.
00:53:57.089 --> 00:54:01.069
Robert: I have no idea.
Mic 3: Thank you.
00:54:01.069 --> 00:54:06.920
Robert: And, as I said before, we're not
the ones that initially discovered, or
00:54:06.920 --> 00:54:10.450
probably not the ones that initially
discovered these issues. And it's not
00:54:10.450 --> 00:54:15.680
really about these issues. I mean, for me
personally, these issues are a nice way to
00:54:15.680 --> 00:54:21.140
get more insight into the PSP. And it's
not about having the super new security
00:54:21.140 --> 00:54:26.319
issue, whatever. So if AMD wants to file
something, I guess they would have also
00:54:26.319 --> 00:54:32.619
filed other people that did similar
research before. Maybe they did. I don't
00:54:32.619 --> 00:54:34.849
know.
Herald: So we had another question from
00:54:34.849 --> 00:54:37.619
the Internet.
Signal: How long did it take you to
00:54:37.619 --> 00:54:43.390
reverse engineer and develop all this
stuff?
00:54:43.390 --> 00:54:51.730
Robert: So I think beginning of 2018,
Christian was starting with his master's
00:54:51.730 --> 00:54:57.640
thesis. And we spent a lot of time on
figuring out how this firmware file system
00:54:57.640 --> 00:55:03.579
works, and the boot process and writing
these PSPTrace and PSPtool to better
00:55:03.579 --> 00:55:11.950
understand the components of the firmware.
And Alex joined in May, May-ish this year.
00:55:11.950 --> 00:55:19.049
And, well, we're still working on it,
right? So the emulator, once we figured
00:55:19.049 --> 00:55:26.439
out a lot of information about the PSP, I
think the emulator was easy to develop,
00:55:26.439 --> 00:55:30.979
in the sense that it didn't take too
much time. But of course, there was a lot
00:55:30.979 --> 00:55:37.569
of work going into it before that.
Herald: So I do not see, oh yes I do see
00:55:37.569 --> 00:55:40.359
another question from the internet. Let's
go for that.
00:55:40.359 --> 00:55:44.729
Signal: Yeah, last question. Did you try
to glitch the PSP by manipulating the
00:55:44.729 --> 00:55:53.829
voltage of the socket(?)?
Robert: Why? I think our approach is
00:55:53.829 --> 00:55:59.900
easier, but no, seriously, we did not try.
Herald: So with that, I don't see any
00:55:59.900 --> 00:56:06.439
further questions. And I would like you to
help me thank Robert, Alexander and
00:56:06.439 --> 00:56:08.161
Christian for this fantastic talk.
00:56:08.161 --> 00:56:09.884
postroll music
00:56:09.884 --> 00:56:21.460
Subtitles created by c3subtitles.de
in the year 2020. Join, and help us!