36C3 - Uncover, Understand, Own - Regaining Control Over Your AMD CPU

36C3 preroll music

ominous bubbling, ebbing away

Herald: The next talk is called "Uncover,
Understand and Own - Regaining Control

Over Your AMD CPU", and I must say, the
days where your homebrew PC would have

been like one CPU plus a lot of discrete
logic, those days are long, long gone. Now

every single device, probably even this
microphone, is full of microprocessors.

It's pretty crazy. Robert, Alexander and
Christian discovered an actual ARM

processor on an AMD CPU, which I find
quite mind boggling; and it actually

includes its own firmware. To talk about
that, I'd like to welcome them onto the

stage. I'm really looking forward to
hearing all about this discovery and what

it has for consequences for us. So thank
you very much. Give them a hand!

Applause

Robert Buhren: All right. Thanks. So,
before we dive into the topic, a quick

introduction. This is Christian and this
is Alex, I'm Robert. And the reason why

there's three of us today is, I'm a Ph.D.
student at the Technische Universität in

Berlin, and beginning of 2018, I was
looking into the Secure Encrypted

Virtualization (SEV) technology from AMD.
And this technology requires a firmware

running on the Secure Processor of AMD.
And that's where Christian came into play

because he was looking for a master's
thesis. Now Christian is done with this

thesis, and Alex here kind of took over his
work. But today we're going to explain to

you what the AMD Secure Processor is doing
and what we have uncovered. So with that,

I'm going to hand over to Christian.
Christian Werling: So let's dive right

into our first part of the presentation,
which is about reverse engineering a

completely unknown subsystem. And when we
started our research, we had to find out

what the AMD Secure Processor, formerly
called Platform Security Processor, in

this talk PSP, actually is. And it's a
dedicated security subsystem that is

integrated into your AMD CPU both on
server and desktop CPUs. It's an ARM

Cortex A5 inside your x86 CPU and it's
there since around 2013. It runs a so-

called secure OS and a kernel. And it's
actually undocumented and proprietary. It

has access to some secure off-chip storage
for the firmware and some some data, and

it mainly provides crypto functionality to
the main CPU, as well as, yeah, key

generation and key management
functionality.

It is required for the early boot. In
fact, it's required for Secure Boot, and

it acts as a trust anchor in in your
system. So the PSP is a security

subsystem, so it adds security to our
system. And that's good, right? You might

notice that this has some similarities
with the Intel Management Engine, which on

this very stage we heard a lot about three
hours ago. So let's look into the

applications of this piece of hardware.
For that, we need to talk about trust.

The one form of trust AMD tackles in what they
call Secure Encrypted Virtualization (SEV).

So you as a cloud customer can be
sure that your virtual machine can even

run in an untrusted physical location, for
example, in a data center. The PSP that is

running inside that server CPU acts as a
remote, trusted entity for you as a

customer. And it promises you to protect
your memory, your data from the hypervisor

and even from physical access. For
example, through a data center

administrator. The other form of trust
that the PSP tries to establish is now

arriving in the Linux kernel, and that's
an API to a trusted execution environment.

What that actually is, is that the PSP
acts as a black box inside your system

that is trusted by an external entity. For
example, a content provider like Netflix.

This would enable, for example, digital
rights management on an untrusted system

that is your system, like Linux. So to sum
this all up, the PSP runs code that you

don't know and that you don't control. And
first of all, let's talk about the

knowing. What you see here is a Supermicro
motherboard, a server motherboard, from

the top, and I highlighted three
components here which are required or

essential for boot up, of course. That is
the CPU, the disk and so-called SPI flash.

The SPI flash is a simple storage that is
available during early boot. So if you

look at the boot procedure in a simplified
manner, then the CPU will first load the

BIOS from this SPI flash. And only at a
later stage of booting, when the necessary

drivers are at hand, it will be able to
access the hard disk to load the operating

system. Now, as we saw from AMD's
marketing slides, there is the PSP now.

The PSP is actually part of the CPU and
even boots before the CPU boots and will

only after successful initialization of
the system release the x86 CPU. So the PSP

firmware is loaded first, and after that,
the boot is proceeding as we know it with

the BIOS and the operating system. So
where is this PSP firmware coming from?

Well, the BIOS is stored on the just-
mentioned SPI flash memory and it contains

all the data and code that is used, of
course, during boot up. And it is arranged

according to the UEFI image specification.
So it's a standardized format. That's

that's good. So maybe we should have a
look into a Supermicro UEFI update. You

see screenshots from the open source tool,
UEFI tool, which is able to parse the UEFI

image specification. You see information,
for example, like the full size. This is

16 megabytes. That's the traditional,
that's the size of a traditional SPI

flash. And you see several volumes which
contain BIOS code and data. What you can

also spot are two so-called paddings, non
empty paddings. And these are called

paddings by the tool because
they are not part of the UEFI standard.

And we're not able to parse
them with the standardized information

available. So let's use another tool.
Probably many of you know "binwalk", a

command line tool for extracting firmware
from images and forensics in general. And

let's look at the machine instructions we
can find in that UEFI update for the

Supermicro board. So the second block you
see are Intel x86 instructions. This is

what we expect, right? It's a BIOS update
for an x86 CPU. So that's not surprising.

What is more surprising are the ARM
instructions. So we might be very close to

the PSP firmware. And what we found out by
staring at bytes and a hex editor a lot is

what we call the firmware file system of
the Platform Security Processor. And the

central data structure in it is the
directory. A directory starts with a

magic string, in this case, dollar PSP,
and it will have a checksum. It will have

a number of elements that it will list and
a field we don't know. And then with each

line in the screenshot, you will have an
entry in this directory. And each entry

has a type and a size and an address where
it is located inside that UEFI image. So

the last entry of this directory is a
special entry. It points to a secondary

directory or that's how we call it. It's a
continuation of this directory, and each

entry points to something like a file. A
file definitely has a body and it might

have a header and a signature. But I'm
gonna go into detail about this in just a

second. So now we just need a reliable
entry point to parse this whole firmware

file system, and this is the Firmware
Entry Table. The Firmware Entry Table

begins with a specific byte sequence,
that's how you can find it. And, it lists

pointers to firmware blobs such as those
directories inside the UEFI image. Earlier

versions of the Firmware Entry Table are
documented in source code of the Coreboot

project, an open source BIOS
implementation, and that was very helpful

in the beginning of our research. So, to
make use of all that knowledge and all

that staring at bytes here, we developed
"psptool", a command line utility that is

able to parse any AMD firmware from UEFI
updates such as the Supermicro update. And

in the output you will see something like
a directory header here, you will find

entries like something called PSP Firmware
Bootloader. You will see that it has a

version, and psptool will even try to
find out whether it's compressed, signed,

will try to verify the signature and so
on. And, just as a recap here, you can see

that the last entry of this directory
actually points to another directory,

which psptool parses for you as well. So
in order to enable you to look into the

code that is running on your AMD CPU right
now, psptool is available on GitHub and

you can check it out today. So the PSP
runs code we don't know. Well, now it's a

matter of binary analysis to actually find
out what it does. Let's talk about the

control. Are we able to alter the firmware
to run our own code? For that we had to

play around with hardware, and more
specifically we used an SPI programmer to

flash any arbitrary UEFI image onto the
SPI flash. After, for example, taking the

original UEFI image and tinkering around
with one byte or one bit we would then try

to boot the system, and in most cases it
just wouldn't boot. This was insufficient

because we only had binary output from
these experiments. So we also used the

logic analyzer that you can see in the top
of this picture. A logic analyzer is just

an electronic instrument that can capture
the data that runs through the logic

lines. In this case, between the SPI flash
and the Supermicro board. So, looking into

a recording of one of our boot procedures
we would now be able to make sense of this

data. So, for example, we can see that the
chipset here issues a read command that's

defined by the byte three, but tried to
read the address E 2 0 0 0 0 and then the

SPI flash would gladly respond with data
at that location. Now you might argue the

data is not that interesting because
that's what we control, that's what we can

program, that's what we can look into with
psptool. So what we were more curious

about is the order and timing of the
actual accesses. And to make that a bit

more visual we wrote "psptrace". So,
psptrace takes such a SPI capture and

correlates it to the output from psptool,
and we will get a enumeration of the

specific components of the PSP during
boot, and I'll get into detail about this

also in just a second. "psptrace" is
available as part of the psptool

repository. If you're more interested
about our hardware in our hardware setup,

you can check out our talk from the CCCamp
earlier this year where we actually had a

Ryzen Pro CPU at hand, and just used the
Lenovo ThinkPad. So that might be more

suitable for your homework. So I want to
share two more insights that we gained

through our experiments in the beginning.
First of all, cryptographic protections on

files. Files are protected by signature
and a field in the header determines the

according public key that can be used to
verify that signature. And that's what

the PSP does. So there are several keys
actually inside the firmware file system,

and then all these keys are signed by the
AMD root public key which does not have a

trailing signature; but as we found out,
after it is loaded from flash, it will be

compared to a hash in Read Only Memory of
the PSP. So we were not able to alter it

like that. The second insight is how the
early boot procedure of the PSP works.

We have an on-chip bootloader that is
burnt into the chip, into the PSP.

We have an off-chip bootloader that is
loaded from flash,

and then we have several applications
that are loaded subsequently.

So now let's look a bit more closely at
the output of psptrace.

The first few read
accesses are to the firmware entry table,

the global data structure, and then the
on-chip boot loader will load the PSP

directory, it will load the AMD public
key, and verify it as I just told you by

comparing it to a hash in Read Only
Memory, it will load the PSP firmware

bootloader. That's what we called the off-
chip to bootloader. And this one will be

verified with the AMD public key. Then in
the boot trace of psptrace, we see a delay

that's due to some initialization work the
PSP does, and then it will load more

directories and will load and verify some
applications eventually. And with this

rough overview of the boot procedure, I'm
gonna hand you over to Alex.

Applause

Alexander Eichner: OK. So now that we
uncovered the basic modules of the

firmware, we obviously wanted to gain
deeper knowledge about what these

individual modules do, how the firmware
functions of the PSP is constructed, what

hardware it provides and how we can
interface it. So in order to do that, we

need to do a quick recap about how AMD
structures the CPU itself. So what you see

here is a little x86 core being able to
execute two threads using simultaneous

multi threading and AMD groups four of
those cores into what they call a Core

CompleX (CCX). It contains up to four
cores based on your exact model, and two

of those complexes are put onto a CCD or
Core Complex Die. That is what AMD also

calls a chiplet. So it's a single silicon
chip on your CPU and you have multiple of

those chips on your CPU. Among the two
CCXs, it contains the memory controller

for the DDR4 memory, PCI express lanes,
communication links to communicate with

other CPUs in the system and much more.
So, in our setup you saw earlier already,

we had a two socket system with two CPUs
and each of these CPUs had four CCDs. And

now, we have not just one PSP in this
whole system, but up to eight. So each of

these CPUs or each of these little PSPs is
actually executing code even before the

x86 cores have executed anything. So AMD
calls the one on CCD 0 the Master PSP, and

all the others are referred to as slaves.
The master coordinates the initial bring

up of the platform. So for the whole
initialization, for the memory controllers

and so on and the slaves respond to
requests made by the master PSP. So each

of these PSP is identical in the system.
Because they are 32 bit ARM cores, they

have a 32 bit address space layout. The
first 256KiB of this layout are backed by

actual on-chip SRAM. The first, the on-
chip bootloader, will load the off-chip

bootloader, "PSP_FW_BOOTLOADER", and
place it into memory where it will be

executed. Among [below] the actual
firmware bootloader, you will also have

the page tables for the MMU. Yes, the PSP
also has a MMU and [is] virtual-memory-

enabled. And the code is separated into a
supervisor, or kernel, mode and the user

mode part. So, the last page you see here
is the so-called boot ROM service page. It

contains information about the PSP [that]
the code is currently executing on, like

number of sockets in the system, the
current CCD ID where it's executed. It

contains some other things, like number
of sockets and so on, and it will become

important later on. Then the off-chip
bootloader will call the applications.

They are executed in user mode. They
contain the code and data to bring up the

actual system and they also contain the
stack memory. And this is done during the

initial boot-up process by using a fixed
order. And later on when the host OS runs,

the application, for example, for the SEV
functionality will be loaded on demand.

So, the rest of the space there we have
to fill is taken up by MMIO.

So, this PSP has its own cryptographic code
processor which is not shared with the

x86. You have the hardware registers to
access x86 memory, to access the system

management network (what this is, we will
come to in a bit), and much more [that] we

don't know about now, right now. So the
boot process in detail. So, Christian

already gave you a rough overview how the
boot process is done and now we will take

a deeper look into this. So first, of
course, you have the on-chip bootloader.

It loads the off-chip bootloader from
flash and executes it. The off-chip

bootloader will execute and initialize a
PSP to a bare minimum and then call the

apps. The first one we have here,
DebugUnlock and Security Gasket. We have

no idea what they are actually for, but we
named them after some strings we found in

the binaries itself. So, the big chunk you
see here is the actual bootstrapping

phase. AMD calls it "AGESA BootLoader"
(ABL) and it's not just a single binary,

but it hosts a binary which loads binaries
from the flash furthermore, and then

executes it in a specific order. So, you
see here ABL one, two, three, four and

six. ABL five is used for something like a
warm resume from suspend to RAM, for

example. So, later on, if the SEV app is
for example loaded, if the OS requests a

specific SEV functionality and not before
that. Because we have the separation

between supervisor and user mode, we
obviously need a way that the app can

communicate with the off-chip bootloader
and that is done using the ARM instruction

"Supervisor Call" or "SVC". So we
identified 76 syscalls in total. We have

mostly reverse-engineered 30 by now. We
can access the x86 memory. We can

communicate with other PSPs in a system.
We can load entries from flash and so on.

28 are partly reverse-engineered. Those
are mostly CCP operations for RSA public

key verification, AES encryption and so
on. And there are also more elaborate

functions to communicate with other PSPs
which are required during the AGESA

BootLoader stage. And then, we have 18
left, and these we don't know about yet

because they are not called at all or
they have exactly one call site and are

non-trivial to reverse-engineer.

So, "System Management Network".
I already saw on the

slide already that there was access SMN.
If you Google for "System Management

Network" or SMN, you won't find much
information about it by AMD or otherwise.

The only reference you may find is code in
the Linux kernel to read out the thermal

sensors on the CPU. So the System
Management Network actually is a hidden

control network inside your CPU. Each and
every hardware block which is in there is

connected to it and is used for the PSP to
control and initialize the hardware blocks

during the boot up phase. So it is a
dedicated address space, so the PSP can't

directly access it using MMIO
instructions. And we have the PSP there.

We have identified the memory controller,
the System Management Unit for which there

was a talk about I think two years ago on
this very Congress, the x86 cores are

there as well and a lot of other things we
didn't reverse engineer so far. One other

thing. OK. So to access the System
Management Network, the PSP has to map a

certain region of the System Management
Network address space into its own address

space and then can access the register,
write, read and so on. And it has to unmap

it again. And one of the functions we
identified is what we call memory

protection slots. So the PSP has the
possibility [stutters] to configure the

memory controller, to revoke access to
certain regions of the DDR4 memory from

the x86 cores. This is done by using three
registers. We have a start register with a

physical start address, an end register to
denote the physical end address of the

region you want to protect, and a control
register where we only know yet so far the

enable bit to flip it on or off. And what
it does is, if the protection is flipped

on, the x86 will only read "all bits
set"[?] when it tries to access this

particular region and writes will have no
effect through this region as well. And

this is, for example, used for the system
management mode UEFI code, and for certain

functionality for the Secure Encrypted
Virtualization feature of AMD.

So, the next thing we did was running

[the] `strings` [command] over all
modules, obviously. And, what we found

there were a lot of interesting debug
strings and even a lot of format strings.

And, we wanted to know what the values
were during the runtime. So, when we

disassembled the firmware and analyzed it,
we saw that most of these strings were

referenced right before a special call
called SVC 6, so this must be some sort of

debug print for the PSP. The problem is,
SVC 6 is not implemented in the release

firmware. So, we had to find another way
to gain access to these debug strings. And

this is what I will talk about now. So,
the problem here is, first we need to know

where we want to store these debug strings,
and we don't have any x86 memory available

at this time in the process. So we need to
find another device or buffer where you

can store it for later use. But, the only
device we did know about at this time was

the SPI flash. Luckily for us, right into
this SPI flash area from, the PSP

generated the necessary bus cycles on the
SPI bus, without altering the flash. Then

we need a code execution on the PSP to
inject our own SVC handler. And how we

gained code execution, Robert will talk
about in the third part of this talk. But

for now, we assume that we have code
execution on the PSP already, can inject

our own SVC 6 handler and then leave, let
it run. So the app will call SVC 6, it

will be forwarded on to the SPI bus where
we can collect it with our already

existing setup. [We] use a tool to filter
the debug strings from the rest of the

traffic on the SPI bus [that] we don't
want to have [...] in the debug output and

then hopefully get a raw PSP log. And we
had success with that. So what you see

here is the initial boot-up or the very
first stage of the boot-up state. The logs

are several megabytes long and we didn't
have the chance to go through all of them.

So, there is a lot of interesting stuff
hiding there already.

applause
So, the next step was to explore what is

hidden inside the System Management
Network. And we didn't want to always

reflash the whole system all the time and
write code for it, debug, because that is

error prone and tedious. So we created our
own setup where we could dynamically use

the x86 calls on the system to write and
read from the System Management Network.

For that, we replaced the SEV app with a
stub and the stub provides three

primitives. We can read-write a System
Management Network address, we can execute

an arbitrary syscall from the off-chip
bootloader and we can read-write general

PSP memory. And because the PSP is exposed
as a separate PCIe device to the x86, we

use the existing Linux kernel driver and
modified it to expose these requests to

user land, where we created a user space
library wrapper and some Python bindings.

And with that we were able to use a Python
shell to dynamically read, write

registers, headers, spurious reboot in
between if you did the wrong thing, but

could start over very quickly. So what you
see here in the code snippet is, what we

did to discover what these memory
protection slots where about. You can see

that we call an syscall handler, that we
write some System Management Network

address and so on. And we do it for all
the different PSPs in the system, so the

master PSP can also forward these requests
to all of the other PSPs in the whole

system. Next thing, we wanted to also
analyze the SEV app further and see how

the code is executed and how the data
flows in this SEV app. But because we

already had a PSP stub running there and
couldn't share it on the PSP, we had to

find another method. And we created a PSP
emulator for that and using our

libpspproxy to forward requests onto the
PSP. So the current state can run the SEV

app to a certain point and we are still
actively developing that. So, that started

a few weeks ago, and this will continue in
the development. So, what it does is, what

you see here is the AMD sev-tool to manage
the host and configure all the keys and

certificates on the system. And we
modified the Linux kernel driver to

reroute these requests out to our own PSP
emulator running in user space, which is

based on the unicorn engine. Any hardware
access, because we don't know much about

the hardware yet, is forwarded to the real
PSP, results are collected, and when the

SEV app finishes, it will return the
result back to the AMD sev-tool. And with

that we are able to execute some of the
requests the SEV app implements

successfully so far. Yeah. What you see
here is a small snippet from one of the

traces. You can see a syscall being made.
It's a CCP request. We don't know

exactly how the arguments are used by now.
That's why there's a lot of unknown stuff,

but this will aid us in development. And
furthermore, in addition to allowing a

tracing code execution and observe the
data flow, we later on may be able to

provide functionality which is currently
only available on the EPYC server platform

from AMD, like Secure Encrypted Virtual
machine. The problem here is we don't know

yet if all the hardware is there which is
supported, and whether it's only a

firmware limitation by AMD.

If you're interested, the code is here
on the repository,

it will be made available in
the next few days. We have a number of

repositories available. You already saw
PSPTool. We have some repository where we

collect documentation about hardware
interfaces, syscalls and so on. We have

our PSP emulator there and also the psp-
apps repository, if you want to dive into

writing your own apps for the PSP. And
with that I will hand over to Robert, who

will talk about how we gained code
execution on the PSP itself.

applause

Robert: OK, so for everything that Alex
talked about,

we need code execution on the PSP.
… [inaudible]. Mike? Better? All right.

So, this part of owning the PSP is again
split into two parts. Now, Christian

already talked about the firmware and the
SPI flash. So, this is something we can

control because we have physical access to
the device. We can flash everything we

want. So, what can we do with that? So, on
the SPI flash, we have these directories

which have a header and entries and an
entry is actually compromised (composed)

of an ID, an address and a size. We've
talked about files. So an entry could be a

reference to a file. And, we also talked
about these secondary directories. So, an

entry could refer to another directory.
Now, if you look at the files you see that

they have a signature usually. So, we
cannot manipulate those files directly. If

we touch them, this will be noticed and
they won't be loaded and the system will

immediately reboot. Now, what we can
manipulate is the directories themselves,

because they are not protected at all. So,
specifically, what we can do is, we can,

for example, add additional entries. These
entries might point to the same files.

That doesn't matter. We can add entries.
What we also can do is, we can remove some

of those entries or we can change entries.
So, for example, this reference to the

secondary directory, this has a size
parameter. Right. And this size refers to

the size of that directory. And actually,
what we can do is, we can change that

size. So we can make the directory appear
to be smaller without removing any of

those entries. Now, during boot, this PSP
directory, that Christian already talked

about, is parsed. So this PSP directory
contains, among other things, the

reference to the AMD public key, which is
used to authenticate all the applications

which are loaded. Now, this directory also
has a secondary directory. The content is

not really relevant here. So the on-chip
bootloader that executes first will set up

this boot ROM service page that Alex
talked about. And this boot ROM service

page contains a copy of those directory
entries, just for the first directory. And

also the on-chip bootloader will copy the
AMD public key itself to the boot room

service page. So it only copies the AMD
public key if it's been verified before.

OK. So now this boot room service page
contains this AMD public key and this

public key in memory is from then on used
to authenticate applications. So the off-

chip bootloader, which executes later,
will use that boot ROM service page and

will extend it. Specifically, it will copy
the entries of the secondary directory to

that boot ROM service page. So I guess you
can already see where this is going.

So, what could possibly go wrong here?
Laughter

Well, we have space for 64 entries here.
And if

we write more entries to that page, we'll
hit the AMD public key. So the off-chip

bootloader should better check that we
only copy at most 64 entries. There it is.

There is a check. Let's say this is the
function that appends entries and it says:

okay, if the number of entries exceeds 64,
we return an error code and do not copy.

Sounds good. Thing is, that number refers
to the number of entries in the secondary

directory. So this has a maximum size of
64. But there is already space, there are

entries there on this boot ROM service
page. So, actually, what we enforce with

this check is, whatever we append can have
at most 64 entries, and within that 64

entries, well, there's the AMD public key.
Super convenient. So what we do now, we

place our own public key inside the
directory structures of the firmware file

system. The off-chip bootloader copies the
entries and copies the AMD public key.

Applause
So what does it mean for us? Now, all this

parsing happens before the first
application is loaded. So that means we

control the very first application and can
replace the content. And from there on, we

control the userland part of the Secure
Processor.

So, now coming to the next part.

So, the natural next target is, of course,
I mean, we have userland code execution,

we want to have the rest. Kernel mode. So,
how can we take over the kernel mode? Now,

let's have a look at how this distinction
between kernel and user mode happens. So,

if we look at the virtual memory layout,
we'll see that there is a user mode part

and a fixed split with the kernel mode
where our off-chip bootloader resides. So,

our application, which we already control,
can try to access that memory, of course,

but that won't work. Right. The MMU
will prevent any access to privileged

memory. Okay. So let's see how this works
at runtime. So, this bootloader component,

if we specify the privileged memory a
little bit more, we have code and data

there. And at runtime another type of
directory is parsed. And this is called

the BIOS directory. I mean, it's a similar
structure as the directory before. We have

entries and the reference to a secondary
directory. The entries here, again, of no

relevance. So during boot, the off-chip
bootloader will copy those entries into

its data section. OK? So, for the copy
operation, we need some some information.

So, let's say this is the copy operation,
kind of looks like `memcopy`. What we need

is destination, where to copy? We need
source. This is the secondary directory,

this is the thing we want to copy, which
is already under our control. So,

convenient, we control whatever data is
copied. And, we need a size value. So,

where do we get that size? Oh yeah, this
entry here has a size value. Super. It's

ours also, right? We control the directory
structures. We can manipulate the size. So

to sum up, we have a copy operation into
privileged memory with attacker-controlled

data and attacker-controlled size. This is
a very old meme, and I think it's

appropriate because this this bug is so
easy to prevent, actually. But for us it's

good, because now we control everything in
red here. So, we control that part. The

thing is, as you can see, code is not part
of what we control. So, what might be here?

What is of interest for us to overwrite?
Thing is, it's the page tables. The page

tables are part of the data section within
the privileged part of the virtual memory

space. So again, what we do, we place our
own page tables here. The data is copied

and replaces the page tables in memory of
the Secure Processor. So, now, if we look

at that virtual memory overview again,
well, our page tables define the virtual

memory a bit different. We make everything
user-writeable. So, we control the

application, our application now can touch
the privileged memory and just overwrite

everything there, if we want to. For that,
we need to reimplement everything. But, we

can patch now the Secure Operating System,
if we want.

Applause

So, that means, this parsing of the

directory also happens before the first
application. So, we control the first

application, that takes over the
bootloader, if you want. And from there

on, we have everything. All those issues I
presented were fixed, were even fixed

before we discovered them. Right? So, we
might not be the first one that discovered

them. Some of you (may) remember that
there was some web site called

AMDFlaws[.com]. They did not present too
many technical details. Maybe what they

discovered was something I present here. I
don't know. Thing is, it does not really

matter for us because the Secure Processor
does not implement any rollback

prevention. So we can always go back and
refresh a vulnerable firmware. And from

that, use whatever code we want to place
there. So, what what we did is, we used

all this on an Epyc Naples based server
system. And, you cannot just use that

issue on every AMD system, because the
bootloader we're using was signed with a

key specific for the Epyc Naples CPU
series. However, we believe, we have not

tested it thoroughly yet, but we believe
the same kind of issues exist in

bootloaders which are signed with a Ryzen
first generation key. And, for the rest,

we don't know yet. So, maybe for
Threadripper or Epyc Rome, there are

similar issues, maybe not. We don't know.
So the question is, is this really a

security issue? I mean, of course it's a
security issue, but, for whom? So,

everything we did requires physical access
to the device. So, if it were my laptop,

personally, I wouldn't be concerned too
much. However, there are some things where

this is a real issue. For example, if you
rely on Secure Boot. Because the Secure

Processor is the first part that boots up,
and if that is broken, everything later on

is also broken. So, Christian already told
you that AMD plans to use this Secure

Processor [as] a trusted execution
environment. If your application relies on

that, you better not have any security
issues in that Secure Processor. And, for

the last part, the Secure Encrypted
Virtualization technology from AMD is

dependent on the integrity of the Secure
Processor. If that is broken, this

technology is also broken. So, Christian
and I published a paper about that. If

you're interested, you can read it up.
But, for us here, this is actually more of

an opportunity, right? Because we can gain
more insight into this PSP with code

execution. We can do a lot of cool things
with that. So, it allows to do further

research on other subsystems which are
present in the AMD CPUs. For example, the

PSP is responsible to load the SMU
firmware. The PSP allows access to the SMM

mode. So, this is a "ring -2 mode" on the
x86 CPUs. So, [it is] higher privileged

than your kernel, and there is proprietary
code running in that mode. With the PSP,

you have access to that code and could
replace it, analyze it, whatever. And, the

PSP is responsible to kick off the x86
calls at all. So everything that comes

later is, in theory, now under our
control. Thank you. That's it.

Applause

Herald: Yes. Thank you very much, Robert,
Alexander and Christian. That was

fantastic. Wow. I have a lot of questions
I guess [in my?] head going on. But do we

have any questions from the audience? And
if you have any questions, we have

microphones lined up here. A question is,
just so that you know what we're talking

about with questions, is a sentence with a
question mark behind it and not your life

story. And I think I saw number one first.
So, let's start with number one.

Mic 1: Hey, is there a reason why the page
table is located at the end of the data

segment?
Robert: I don't think so. I mean, ...

Mic 1: "Just because"?

Robert: You have to place it somewhere.
should be in the [interrupted]

Mic 1: Why not in the beginning?
Robert: I don't know. No idea.

Herald: That's what I meant with "a lot of
weird questions" here. From the signal

angel we had one question.
Signal Angel: And this question goes to

the first lecturer. Didn't you have access
to an SPI flasher relay(?) to attempt a

"Time of Use versus Time of Check"
[TOCTOU] attack?

Christian: So, we had access to different
tools, but the TOCTOU attack that you

mentioned was not even necessary to mount
the attacks we talked about. And actually

so far, we don't see any possibility to
mount a TOCTOU attack.

Herald: OK. So I think I saw microphone 5
next up. Is there somebody at the

Microphone?
Mic 5: Yes. So, I was wondering if you

considered looking at the boot ROM for
issues.

Robert: Yes, of course. The thing is, we
cannot find its code in the memory any

more after we mounted our attacks. So, I
believe, the boot ROM code is not there

anymore, which would make it much easier
to analyze. We tried simple things, like

increasing directory sizes, which are
processed by the boot ROM itself. We

haven't found any suspicious thing there,
yet.

Herald: Microphone 2.
Mic 2: Thanks for your research. You have

really nice big power over the system
right now. Do you have plans to make a PSP

firmware which is minimal and which makes
your system work, but without some strange

untrusted code?
Robert: I wouldn't call it plans yet. Of

course there are ideas to do that. The
thing is, some of the functionality which

is implemented from AMD is really
required. So, the stages that Alex talked

about, they configure and train(?) your
DRAM. So without those stages, you don't

have access to memory. Your x86 cores
wouldn't work. And to reimplement that

without having access to any manuals is
really, really hard work. So, I'm not too

confident that this will be possible in
the near future.

Mic 2: I just refer to a Management Engine
cleaner, and there is such a project,

which makes your Management Engine
firmware slim.

Robert: So, the AMD firmware is already
kind of slim. The only thing that is not

strictly required on the systems we have
been looking at would be the SEV firmware,

which is loaded on request, and you can,
like, disable that by just flipping a bit

inside that file. The system would still
boot, but when it tries to initialize the

SEV technology, the kernel would say, "OK.
This does not work." The system will still

work after that.
Mic 2: Thanks. And last little question.

Does PSP work with microcode somehow?
Alexander: We didn't find anything related

to any microcode there so far.
Mic 2: Thanks.

Herald: So let's move on to Microphone 3.
Mic 3: Thank you first for the great talk.

I have one question. Do you have maybe
found something evil or potentially evil

in the code that it does?
Alexander: No. So far, they didn't find

anything which could be used for an
attack, for example. So, what the PSP

might be able to do is, access PCIe
devices. We found some code related to

that, but we are [stutters] not sure yet
whether it's actually used, because also

the PSPs executed or is existing on graphics
cards made by AMD. So, that might be also

ready to[?] that. We couldn't find
anything there yet, but so far, the PSP

looks rather clean compared to the entire
Management Engine.

Mic 3: Thank you.
Herald: So, we have a question from the

Internet.
Signal: Is the AMD public key an RSA one,

only 576 bits?
Robert: It's an RSA key, yes, but it's

2048 bits for the first generation Epyc
CPUs and I think 4069 [meaning 4096] for

later generations.
Herald: Microphone 2.

Mic 2: For me, it seems like preventing to
flash old vulnerable firmware is really

important for a scenario like Secure
Encrypted Virtualization. Can you comment

on how difficult it is for AMD to add this
retrospectively?

Robert: Okay. So technically, rollback
prevention is there for, I guess, mobile

devices, for example. You have that. It
should be possible. For adding this

functionality afterwards, I don't think
that's really possible, because the on-

chip bootloader is the thing that loads
the off-chip bootloader and verifies it.

And that software component has to, like,
stop loading if the firmware version does

not match, for example. And you have to
change that. And that functionality is not

there and you cannot update the on-chip
boot ROM. So, in that sense, I don't think

that that's possible to change. And if you
look at our paper, you will see that the

former issues are kind of devastating for
the SEV technology, because there are some

keys which are now accessible, which can
be used for attacking SEV-protected

guests.
Mic 2: Thanks.

Herald: Microphone 3, please.
Mic 3: One question. Did you analyze the

API to the x86 core? Did you find anything
that could be exploited without flashing

anything so that you could directly go
from x86 to PSP exploitation?

Alexander: Yeah, we tried to find the
necessary code to interface with the x86.

We think we found one place where the x86
cores are released after the PSP

initialized the whole system. But
obviously, we can't do much with it except

preventing the x86 to boot at all. And
otherwise we couldn't find anything there

yet. So we focused on, on a bit of other,
like the memory controller, and didn't

have a deeper look at the x86 interface.
So what there is, the BIOS can interface

with the PSP using a special mailbox
register which is mapped in MMIO space in

x86 for requests. So, it can, for example,
the UEFI init boots, it will say to the

PSP "Hey, this is my system management
mode code region, please protect that for

me" and it will execute this request. But
apart from that, we couldn't find anything

so far.
Mic 3: Thank you.

Herald: So, Microphone 4.
Mic 4: Hi. So, is it correct that your

work enables 100% open source firmware for
this kind of processors? And if so, have

you already contacted the CoreBoot team to
make that actually happen?

Robert: So. 100 percent open source. As
for the PSP, there is this on-chip boot

ROM which we can't replace, right? So,
this will be closed source. Then there is

code of the off-chip bootloader, until the
first exploit, which runs, which is not

open source. In theory, you could from now
on take over the PSP, write your own code.

But, as I said before, you have to
reimplement a lot of functionality without

having any documentation, right? So,
technically it's possible, I guess, to do

something like that. Practically, I'm not
too sure.

Herald: So we're gonna go to the internet
for another question.

Signal: Is it possible to block PSP from
within Linux or BSD, for the system's

runtime, by using search and boot flags?
Robert: Sorry, to block what?

Signal: To block the PSP from the Linux or
BSD.

Alexander: So, what you can do is, like
Robert mentiond already, you can flip a

bit in the SPI flash and then the PSP,
once it initialized the whole system, it

won't run the SEV app, for example,
because the signatures won't match

anymore. And there is no other sort of
interface where the PSP is actually

triggered. Or we couldn't find it so far.
Herald: Microphone 3.

Someone: I think he was first.
Herald: Oh, okay, all right. Right.

Microphone 2 then. Sorry.
Mic 2: Did you try to enable any

superpowers from PSP like JTAG or special
tricks with voltage or something else?

Robert: When the first application that is
loaded has some strings in it like Debug

Unlock. Sounds interesting. But then
again, JTAG, where would you access the

JTAG of the PSP? You need to have some
connection to the lines, right?

Mic 2: Intel supports USB debugging.
Robert: Yeah, I know. With special

devices, right?
Mic 2: No, even wire cable.

Robert: Okay. So anyhow, I have the
suspicion that this DebugUnlock app is

responsible to to allow some debug mode.
Which then, I assume, with special

hardware, you can have JTAG. But we have
not touched it yet.

Mic 3: Thanks.
Herald: Now Microphone 3

Mic 3: So I'm as far from a liar
laughing, um, a lawyer as possible, but

could AMD in any way file a cease and
desist for anything you do?

Robert: Probably not, I guess.
Mic 3: Just curious.

Robert: I have no idea.
Mic 3: Thank you.

Robert: And, as I said before, we're not
the ones that initially discovered, or

probably not the ones that initially
discovered these issues. And it's not

really about these issues. I mean, for me
personally, these issues are a nice way to

get more insight into the PSP. And it's
not about having the super new security

issue, whatever. So if AMD wants to file
something, I guess they would have also

filed other people that did similar
research before. Maybe they did. I don't

know.
Herald: So we had another question from

the Internet.
Signal: How long did it take you to

reverse engineer and develop all this
stuff?

Robert: So I think beginning of 2018,
Christian was starting with his master's

thesis. And we spent a lot of time on
figuring out how this firmware file system

works, and the boot process and writing
these PSPTrace and PSPtool to better

understand the components of the firmware.
And Alex joined in May, May-ish this year.

And, well, we're still working on it,
right? So the emulator, once we figured

out a lot of information about the PSP, I
think the emulator was easy to develop,

in the sense that it didn't take too
much time. But of course, there was a lot

of work going into it before that.
Herald: So I do not see, oh yes I do see

another question from the internet. Let's
go for that.

Signal: Yeah, last question. Did you try
to glitch the PSP by manipulating the

voltage of the socket(?)?
Robert: Why? I think our approach is

easier, but no, seriously, we did not try.
Herald: So with that, I don't see any

further questions. And I would like you to
help me thank Robert, Alexander and

Christian for this fantastic talk.

postroll music

Subtitles created by c3subtitles.de
in the year 2020. Join, and help us!