<i>35C3 preroll music</i>

Herald Angel: So… Yaniv Balmas is a
software engineer and he started tinkering

with Commodore's C64 when he was 8
years old.

He was kind of a teenage hacker of games as well.

And now he's in the security field and he
got interested in the fax machine

together with his friend Eyal Itkin, who
is also a security guy and malware researcher.

And together they're going to tell us
about the fax machines and What The Fax?!

Why still using people those machines?

And it's gonna be really interesting I think.

And the title is also
"Hacking your network likes it's 1980 again"

I'm really excited. Please give a warm
round of applause to those two guys.

<i>applause</i>

<i>fax modem sounds</i>

Yaniv: Thank you, thank you guys.
Hi, CCC!

You probably know this sound, right?
And now get to know us:

My name is Yaniv Balmas, I'm a security
researcher. I work at Check Point Research,

and with me here today is Eyal Itkin, also a
security researcher, also works at

Check Point Research, and let's begin
with talking a bit about the history of fax.

So I guess that not many of you know
that fax started,

it was first invented in 1846 by a scientist
called Alexander Bain.

Fun fact, this happened 20 years before
the invention of the light bulb.

And then it had some more advances to it,
this is the actual first thing that looked

like a fax machine, a standard fax machine.

And again, this thing was invented 20
years before the invention of the telephone.

So humanity was sending faxes before we
had light or talked over the phone.

And then there was some more
advancements like radio fax,

and an another important point in time is
1966, where a small unknown company

called Xerox invented – came out with the
first commercial fax machine.

This is the advertisement for it.

And in 1980 a strange organization
called ITU defined the core standards for fax.

Namely it's T.30, T.4, T.6, and those
standars are still the same standards

that we use today – basically, with just
minor changes to them.

So this was all in the past.
But what's happening today?

I mean today we have far better ways
to send electronic documents

from one to the other, right?

You know, let's compare fax to just, I dunno,
off the top of my head

just, you know, one method like, let's say, email.
And just to, you know, remind you.

We are comparing this… to this, okay?
So… let's look at some of the features here.

In terms of quality, in terms of accessibility,
I'm pretty sure that all of you here

have 24/7 access to emails. Not so sure
you're carrying around your fax machines with you.

In terms of reliability, well, when you
send a fax, you don't really know

if it got received or not. Yes, there is
this strange confirmation page,

but it doesn't really mean anything.
I mean, if there's no paper in the

receiving fax, you still get it. If the
dog ate it, you still get it.

There's absolutely no reliability in fax.
Regarding authenticity, well, we can argue

about emails, if it's authenticated or
not, it could be forged, of course.

But we do have public key cryptography
and stuff like that, that will help us

when talking about emails, while we don't have…
we don't have nothing when it comes to fax.

Absolutely no authenticity. So, if we're
looking at this table, one might think to

himself: Okay, so… Who the hell still
uses fax today? It's 2018.

I mean, it deserves a place in the museum
of great technologies and that's it.

So, nobody is using fax today, right?

Wrong.

Everybody are using fax today.

You see, fax is used to send these very
critical maritime maps to ships at open seas

90% of the japanese population uses fax –
according to Wikipedia at least.

And if you google any kind of combos like
"contact us" and "fax" or stuff like that,

you will come up with something like
300 million results. 300 million published

fax numbers in Google. And that's not
counting the unpublished numbers.

That's a huge amount of numbers. But it's
not all about numbers. It's not "how many

fax machines are out there?", but it's
also "Who is using fax?"

You see, if you're a small corporation, a
medium corporation, a huge corporation,

you have fax. Not necessarily anybody is
sending fax to this number, but there is a

fax machine sitting there waiting for a
fax to be received. If you're a bank,

you simply love faxes. This is
Bank of China, the biggest bank in the

world, and that's the fax number of it.
I think most importantly, if you're a

government organization… you…
<i>laughter</i>

… simply wake up in the morning and you
want to have more fax. This is

Donald Trump's fax number if anybody wants
to send him a fax. Go ahead.

That's it. It's not a secret, it's from
Google… We should send him something

by the way. And the thing is that, you know, those
banks and government institutions, they

don't only support fax, allow you to send
fax, the funny thing is that actually most

of the time, it's mandatory to send fax,
there is no other way. You can either

postal mail it, or fax it. They didn't
hear about anything else.

So we looked at this, state of affairs, 
strange state of affairs,

and said to ourselves: "This looks
strange". I mean, it can't be true.

Humanity came so far and we're still using
these old technologies, so…

What The Fax?!

And we decided and try to do something
about it. And we started very long

research to try and find some security
vulnerabilities in fax. And before we do

that, you need to explain how fax looks
like today. You see, today fax doesn't

look like it looked 20 or 30 years ago. 
Then, it was just standalone fax machines.

Right?

Today, fax is mostly old technology
embedded within newer technology.

So, we have fax to email services or email
to fax services, we have as I said before,

radio fax and fax over satellite and stuff
like that. I think most commonly, we have

this. These machines. All-in-one printers.
You buy them, they scan, they print.

And they fax. It actually comes with a
phone cable out of the box, so you can

connec… I guess most people connect it?
I also think that is the most common

faxing solution today. So we decided to
take a look at these machines.

These fax machines.
If you look at these boxes

from a security point of view you can
imagine them to be just black boxes.

And those black boxes have interfaces.
In one side of the box we have interfaces

like WiFi, bluetooth, ethernet, stuff like
that, these interfaces connect the printer

to the internal network, the external
network, basically it connects it to the

world. And on the other side of this box,
there's this little interface here that

connects this black box to somewhere
to the 1970s I would say.

<i>Laughter</i>

So that's pretty funny.

And if you remember, at the end of the day
these printers are basically nothing but

computers. They have CPUs, they have
memories, they have operating systems,

they are computers. Not standard ones,
but they are computers.

And we were thinking to ourselves, imagine
this scenario: There's an attacker sitting

somewhere in the world. All he has is
access to a phone line and his targets fax

number. What will happen if this attacker,
this guy, would be able to send a

malicious fax and with this malicious fax
he would be able to exploit the printer.

Then he has complete control over the
printer, right? If he does that, he could

then maybe pivot through any one of those 
other interfaces, let's say the Ethernet

and jump from this printer to the rest of
the network, the internal network.

Effectively creating a bridge between the
external world and the internal network

through the phone line.
That's 1980s again!

So we thought this is a really cool attack
scenario and we decided to accept this

challenge and go for it. Try and actually
show this thing happening in reality.

We were really excited about this.
But then after we slept a bit and drank

a bit, sat down and talked about it, we
kind of found out that there is like a lot

of challenges, really hard challenges in
front of us and we're not really sure how

to deal with them. Let me name just a few
of them. One of the challenges is how do

we obtain the firmware. The code that this
printer runs. It's not like you have it

everywhere. And after we get it, how do we
analyze this firmware?

After we analyze it,

we need to understand what operating
system are those printers running.

And then we need to understand how to
debug a printer..

I never debugged a printer before..

I need to understand how to debug
a printer. And after we do all that,

we need to understand… How does fax even
work? We only know the beeping sounds like

most of us I think. And after we did all
that, we can start talking about where can

we find vulnerabilities inside this
big, big, big ecosystem.

And today, we'll try to take you through
these challenges, one-by-one and explain

how to do it until we'll be able to
actually do the scenario that we just

showed you. So, let's start with the first
challenge.

How do we obtain the firmware 
for the printer?

So, meet our nice printer.
It's an HP inkjet printer,

an HP Officejet printer, we chose this
model, first of all we chose HP because

it has like – I think – 40% of the market
share so it's not that we dislike HP, we

really like them, but unfortunately for
them, they are just the biggest target out

there. And this specific model, well we
had a lot of reasons why we chose this

printer. But basically it's the cheapest
one.

<i>Laughter</i>

We bought it. We didn't have a lot of
budget. We bought it and we abused it for

a lot of time. And our goal was to break
fax, but before we do that, we had to

break the printer. I mean literally break
the printer. So yeah, that was the fun

part of the project, we broke it. And
inside the printer we find this thing:

The main PCB, the brains behind the
printer, and it looks like this.

Let's map the critical components of it:

So we have here: Flash ROM, 
SPANSION some model,

and then we have some more memory here,
this might look like not a lot, because

the PCB has two sides to it of course,
and on the other side of it we have the

more interesting components, like USB,
WiFi, electricity, SRAM,

battery – probably for the memory but who
knows – and now we have two very

interesting components here. One of them
is the main CPU. It's a Marvell CPU, and

it's proprietarily manufactured for HP.
So we can't tell anything about it,

there's no available specs, nothing.
We can just find bits of information

here and there. And then we have the fax
modem. It's located here and it's a

CSP1040. What we need to understand now is
how do these two components operate and

what is the relationship between them?
If we do that, we're one step further.

So that's what we tried to do. And as I
said, the first challenge is to get the

firmware of this thing. And when we're
looking a bit closer into this PCB, we

find these 2 very interesting interfaces:
One of them is a serial debug, the other

is JTAG. If you're familiar with them, you
know that they give you debugging

capabilities, or at least memory read,
memory write, exactly what we need to get

the firmware. So we're smiling to
ourselves saying "Haha, this is going to

be really easy". But unfortunately it's
not. Because the JTAG is, of couse,

disabled completely. We can't do anything
with it. And the serial port, we managed

to connect to it. And we get this terminal
that for almost every instruction we type

gives us this error: "I don't understand".
Well, we don't understand either.

<i>laughter</i>

But it looks like this terminal is not
going to get us very far. So we dropped

this path and tried and look for other
ways to get the firmware and obviously one

of the most common ways is to try and grab
the firmware upgrade and after looking a

bit in the internet we find this jewel,
this FTP site by HP that contains

every firmware version for
every HP product

ever produced in the history
of HP and the Internet

and a lot of other stuff.

And it actually took us about, I think, 
two weeks to find our firmware within

<i>Laughter</i>

… this mess of firmwares. But once we
did,

we had a firmware upgrade file.
<i>Applause</i>

Yes, thank you! It's still alive so you
can go there and look for some… there's a

lot of interesting stuff in there. And now
we've got ourselves a file. And this file

is the firmware upgrade file. It's not an
executable file, it's just a binary,

and now we kinda need to understand…

How do you even upgrade 
a printer firmware?

I never did it i before. Anybody did it?
Anybody upgraded these firmwares? Lately?

Ah, good. Good for you. Good for you.

Anyway, the answer to this question is
surprisingly… funny, I would say.

You just print it.

<i>Laughter</i>

That's because, you see, a printer
receives a firmware upgrade just the same

way as it receives a normal print job.
That's the thing and it's actually pretty

nice and it's defined in a HP protocol,
it's called PCL XL Feature Reference

Protocol Class 2.1 Supplement. And if
you're still sane after reading this like

300 pages of insanity you understand that
this thing defines something called a

PJL – print job language. If you ever
scanned from a printer to the network you

see this port I think 9100, something like
that, open, that you send print jobs to,

and it's the same port that you send the
firmware upgrade to, and that's nice.

So when we look at the file, it actually
confirms this,

because it actually begins
with the words: PJL – Print job language.

So that's nice. So now we know it's a
print job language.

But unfortunately this document doesn't
document anything about the firmware

upgrade protocol, or anything, 
because it's HP proprietary.

So unfortunately we had 
to do it ourselves

and analyze this thing. Now I'm not going
to take you through the entire process of

unwrapping this firmware because frankly
it's quite boring. But I'll just tell you

that it's composed of several layers of
compression, one of them is called

NULL decoder, the other is called TIFF
decoder, and another one called Delta Raw

decoder. And the thing is that these
things do something like… If the previous

line was all blanks, and if this line is
also all blanks, just write one instead of

the line, so that gives you some kind of
compression, and it makes really a lot of

sense when you're talking about print jobs
because paper has a lot of spaces in it,

but when you're talking about binary files
it makes absolutely no sense to do it this

way. But still, it just works this way, so
after we understand that, we were able to

decode everything, decompress everything,
and we're talking to ourselves and

laughing, when you're 
a hammer everything looks like a nail,

and when you're a printer, 
everything looks like a print job.

<i>Laughter</i>

So that was nice. And now, after we did
that, we have a big file that hopefully

now is our firmware.

So how do we analyze it?

Looking at this thing right at the
beginning of the file, there's something

that really looks like a table. It doesn't
only really look like a table, it is

a table. We define it, it looks like this.
And what this table defines is a loading

address, section name and location in
binary. So what that means is that our big

file is actually split into several
sections. This table just defines those

sections. So now we are able to split this
big file into several smaller chunks and

inspect each chunk. The most important
chunk, the one that looks most promising

looks like it contains our firmware. So we
took a closer look into that and that's

what we saw: It actually looks like our
firmware. That's because you see: That's

one of the strings that we've seen here.

<i>Laughter</i>

Yeah! We all saw that before, right? It's
"Error: I don't understand". But it's not

completely "Error: I don't understand".
There's some missing bytes in here.

And actually those missing bytes are
pretty consistent throughout the entire

chunk. So although we know that we are
looking at the code, we can't actually

see the code until we have those missing
bytes filled. We need to understand: Why

are they there and what were they replaced
with? So let's try to analyze this thing

together, quickly, now. But first, let's
try to understand what is this thing.

We have a lot of things in mind, every one
seemed crazy, but I think the least crazy

option was that this is yet another form
of compression. A really bad one, again.

Because when we tried to compress this
thing with zlib, for example, we get like

80% better compression than it currently
is, and we know that the printer has zlib,

because we see zlib strings in there, so
why not use zlib? I don't know.

But still, we are left with a challenge.
So this is one snippet of the code that

you just saw, 
so let's try to decompress this.

First of all, you need to understand this
thing is composed of two types of

characters, one are ASCII characters,
stuff that you can read, and some other

are stuff that you can't read, non-ASCII
characters. And those non-ASCII characters

are actually those missing bytes that we
have. So we need to understand what they

are, so let's take a closer look at them.
And if you stare at this thing long enough

you'll start seeing some kind of pattern.
I'll save you the trouble and just show you.

It's composed of these one single bytes,
and then those double bytes in there.

And if the distance between the single
bytes looks suspiciously patterned,

8 bytes, 9 bytes, 9 bytes, 8 bytes, over
and over again, so what does this mean,

where is the pattern here? If you look at
this from a different angle, maybe the

pattern will look a bit clearer. You see
that F7 and F7, they look the same.

The FF and FF, they look the same.
Something here looks really pattern-ish.

In order to understand this pattern, you
need to sharpen your binary view a bit,

and if you understand that FF is just
8 one bits, and if you do that

consistently for all of these chunks, you
will start seeing the pattern.

The pattern is that the zero bit always
falls within this two-byte hole.

It's consistent throughout the file. And
what this means is that the first byte is

just a bitmap describing the following
8 bytes after it. That's what it means.

And that's perfect because now we
understand what is this single bytes, but

we still don't understand, what are those
double bytes? And they were replaced with

something, but with what? So if you know
anything about compression, you know that

there's not a lot of options here really.
It could be either a forward or backward

pointer, it could be a dictionary of some
sort, or it could be a sliding window.

Now we can pretty easily confirm that
it's not a forward/backward pointer just

because we tried to follow the references
in the file, we see nothing that should be

there, same goes for dictionary. We can't
find anything that's consistent enough to

be a dictionary. So it leaves us only with
with the option of a sliding window.

Once we're equipped with this information,
we go to our favorite place, to Google.

And try to find some similar
implementations to this. Luckily for us,

in some very dark corner of the internet,
we find this wiki page. It defines

something called a Softdisk Library
Format. I won't ask if someone knows what

Softdisk is, because probably somebody
knows here, it's CCC after all. But inside

this thing it defines some kind of
compression algorithm that looks very

similar to ours. It looks actually really
really like ours. Actually, it's exactly

our compression algorithm. So yeah. That's
nice. And I think the funny thing here is

that this compression algorithm was used
in the past somewhere, and only there.

Can you guess where?

<i>Waiting for response from the audience</i>

Uh, yeah, somebody who didn't see <i>chuckles</i>
this presentation before?

Yeah! It was used in Commander Keen.

Softdisk is the company who produced
Commander Keen.

So the compression algorithm 
from Commander Keen made its way,

somehow, into the entire HP line of
products.

<i>Laughter</i>

<i>Applause</i>

How? I don't know! You can check if there
was anybody who was fired from Softdisk

and hired in HP. Probably that would be my
guess. But we'll never know.

So now we understand exactly what is this
thing, and how does this compression work.

We have the missing data that we need. And
this data means that those two bytes are

actually composed of window location and
data length. And that's all we need, and

let me show you, like really quickly, how
this compression works. So we have an

input text, output text and sliding
window. We want to compress this string

over here, and let's try and do it.
So first byte is the bitmap, so we leave

it empty for now. Then, second byte, we
start with "A". So we place it both in the

output text and in the sliding window.
Then we go to "B", same thing. "C", same

thing. "D", again, and now we get to "A".
But "A" is already present in the sliding

window, so we don't need to write it in
the output text, we can just do

nothing and then go to "B", same thing,
it's just the following character in the

sliding window, and then when we get to
"E", we just write "00 02". That means

"Go to the sliding window at position 0,
and take the first two bytes". That's what

it means. Then we continue to "E", "F",
"G", after we did that, we input our

bitmap here, and now we know the bitmap
value and that's all there is to it.

That's the compression algorithm.

It's pretty easy
looking at it this way, right?

Looking at it in reverse is a bit more
difficult, but yes, now we can do that.

And now we completely open everything, and
yes, we have our firmware, you can read

everything. It's actual code. And now we
need to understand:

What does this code mean? And basically,
first of all, we need to understand what

architecture is this, what is the
operating system and so on and so on.

So it took us quite some time to do that.
But let me give you a brief explanation.

First of all, the operating system is
called ThreadX. It's a real-time operating

system. The CPU, the processor, is ARM9
big-endian, and then it has several

components to it, like stuff that's
related to system, some common libraries,

and tasks. Tasks are the equivalent of
processes in normal operating systems.

In the system stuff we have boot loaders
and some networking functionality and some

other stuff, Common Libraries we have a
lot of common libraries, and tasks, once

we're able to isolate them, we can
understand exactly the tasks, and once

we do that, we now know that all we need
to do is focus on these tasks, because

they're the tasks relevant
to fax protocols,

we can leave everything else aside.

It will make our work much more easy. We
want to start doing that. But,

just a second before we do that. Looking
at this, we see something that looks not

really… I don't know, it doesn't make
sense a lot. This thing is Spidermonkey.

Every HP printer contains a Spidermonkey
library. I don't know if you know what

Spidermonkey is, but basically it's the
JavaScript implementation by Mozilla.

It's used in Firefox for example. And we
were thinking to ourselves:

Why does a printer need to render
JavaScript? It makes no sense.

I mean yeah, it has a web server, but it's
not a web client. We couldn't think of

any situation in which a printer needs to
render JavaScript.

It looked really strange to us. So we
decided to try and see where this printer

is actually using JavaScript, so we went
back a bit and checked and we found that

JavaScript is used in a feature called
PAC – Proxy Auto Configuration.

It's pretty common, it's a good protocol.
It defines proxies when you're doing DHCP

or something like that. The thing is that
the top layer functionality of this entire

PAC functionality was written by HP.
And when we were looking at that, we see

all this functionality, and we see this
strange thing here. The printer once it

does this PAC functionality, it tries to
connect to this domain:

fakeurl1234.com. Just connect to it and
do nothing with it.

Some sort of sanity test I guess? I don't
really know why.

But the interesting thing here is: Do you
know who owns the domain fakeurl1234.com?

<i>Laughter mixed with murmur</i>

No, it's not HP.

<i>Murmur &amp; responses from the audience</i>

Ehh, Check Point is kinda… eh…, yeah.

I own it.

<i>Laughter</i>

<i>Applause</i>

It just wasn't registered.
So, we registered it for 5 Dollars.

And now every HP printer is connecting to
my domain. <i>Chuckling</i>

<i>Laughter</i>

<i>Applause</i>

So, if anybody wants to buy the domain, I
have a very good price for you:

More than 5 dollars.

And now I'll hand it over
to Eyal to continue.

Eyal Itkin: Okay, thank you Yaniv.
After we've finished messing around with

Spidermonkey, it's time to focus back on
fax, so T.30.

T.30 – in its full name it's
ITU-T recommendation T.30 – is a standard

that defines the fax protocol. Actually
it's a very very long PDF, more than

300 pages. It defines all the phases and
messages we need in order to send and

receive a fax document. It was first
defined very long ago, 1985, and was last

updated more than a decade ago. So from
our perspective that's a very good idea,

because we want to find vulnerabilities in
an old and complicated protocol.

We're most probably going to find some.
After we read through the standard we

started to dynamically look at it, opened
it in IDA and look up on the T.30 task.

And you can see that the state machine is
quite huge as you can see here in IDA, and

actually that's a small state machine.
Because most of the code blocks you can

see over here contain additional state
machines inside them. Meaning that this is

going to be a very very huge and
complicated state machine to reverse.

And if that wasn't enough it turns out
that HP really likes to use

function pointers and global variables in 
their code. Meaning that statically

reverse-engineering this huge task is
going to be very complicated. Although I

personally prefer to statically
reverse-engineer, this time we had to

choose a different tactic, we'll need to
dynamically reverse-engineer this thing

and for this we'll need to have a
debugger.

As Yaniv mentioned earlier, nobody knows
how can we debug a printer.

We already tried built-in JTAG and 
serial port and that failed.

We then searched for a builtin GDB stub we
could use,

but I couldn't find any such stub.

At this point it's very important to
remember that even if we could control the

execution flow, no-one can put a debugger
without controlling the execution flow,

and we can't do anything, it's a black
box, I can send papers and that's it.

And even if I could control the execution
flow and load my debugger, the printer

uses a hardware watchdog. And this is an
external hardware mechanism that monitors

the main CPU and whenever the main CPU
enters an endless loop or it halts,

the watchdog reboots the entire printer.
This means that since essentially a

breakpoint halts the program,

whenever we hit a breakpoint, 
the watchdog will kill us.

So we need to find a way around this
thing, the easiest way we could find out

was to split this enormous task into
chunks, if we could find any code

execution vulnerability, we could try to
execute code over the printer and load our

own debugger. And at this stage we had
luck, and we believe that luck is an

important part in every research project.
On the 19th of July, SENRIO published a

vulnerability called "Devil's Ivy".

Devil's Ivy is a remote code execution in
gSOAP and many embedded devices (and our

printer included) tend to implement a web
server for management and configuration,

and in our case this web server uses
gSOAP, and it even uses a vulnerable

version of gSOAP, so we now have our
vulnerability, and we'll need to exploit

it. For those of you not familiar with
Devil's Ivy, here is the code.

And here is the vulnerability itself.

Devil's Ivy is a signed integer underflow
vulnerability,

meaning that we'll need to send

enough data for the variable to go from
negative back to positive. And that means

we need to send roughly 2 Gigabytes of
data to the printer.

So HP really prides itself on the printing
speed of the printer,

but not on the network speed.

After many optimization rounds we managed
to reduce the exploit time to roughly

7 minutes. So you start the exploit, you
wait, and after 7 minutes you have

your exploit. And here our good luck
ended, because we had a side effect in our

exploit, and after two to ten minutes the
printer will crash. And this means we will

need to wait an additional 7 minutes, 
we'll have 2 minutes to debug it,

and then it will crash again. So we 
waited a lot of 7 minutes in our research.

<i>Laughter</i>

If you recall, we wanted a debugger so we
could dynamically reverse-engineer the

firmware. We wanted read memory and write
memory, and now we have a debugging

vulnerability, so we can load a debugger,
we need to execute this debugger, so

we'll need executing permissions
to load it.

The most important thing is that we need

to execute our debugger without crashing
the firmware. Because we want the debugger

to run and the firmware to debug and we
want them to blend inside the

virtual address space of the printer,
living happily together.

We couldn't find any debugger that achieve
this goal, so I did what my mother usually

tells me not to do, we actually wrote our
own debugger.

So this is Scout. Scout is an instruction
based debugger that supports Intel CPUs

and ARM CPUs, because we have an ARM
printer. As a prototype we had a Linux

kernel driver, and this time we're going
to use it its embedded mode.

In embedded mode we compile it to be fully
positioned in the <i>unintelligible</i>,

because we essentially throw it somewhere
inside the firmware and expect it to

execute. We pre-equip it with useful
addresses like:

memcpy, socket, bind, listen, we
find using IDA.

And whenever it tries to 
call these functions it goes to its

own GAT, finds the address and

jumps to it.

After we compile it, we use it in our
exploit, we jump into this blob, and it

starts up a TCP server, we can now connect
to to send instructions to

read memory, to write memory, 
and whatever we want.

You can find Scout in our GitHub, with the
examples for Linux kernel driver and

embedded mode. And we're actually using it
for some CVEs now,

so it's highly recommended.

Now that we reach this point in our talk,

we haven't yet described to you how a fax
actually works, so with Scout we

dynamically reverse-engineered the
firmware, and now we can actually

describe to you how a fax actually works.
In order to send a fax, we need a sending

machine, we need to send it to some modem,
the packets from the modem will be

processed in the CPU, and afterwards, the
data is going to be processed and probably

printed. Let's see how it starts. We start
with network interaction,

probing and ranging, equalizer and echo
cancelling, more training,

and you actually need to be quite familiar
with these steps,

because they sound like this:

<i>repetitive fax modem sounds</i>

With these beeps, we actually created an
HDLC tunnel. Through this tunnel, we're

going to send our T.30 messages, to
the CPU. In T.30 you have phase A,

in which we send the caller ID, which is
a string. In phase B you negotiate the

capabilities, so I send my capabilities
and receive the printer's capabilities.

Phase C is the important step because here
we actually send our fax data,

line after line, and page after page.
And in phase D, we finish. I send an ACK,

I receive an ACK, and that's it.
Let us now see how a normal black/white

fax document is going to be sent through
the protocol. So we have our document,

it's going to be sent over the HDLC tunnel
using T.30 messages, over phase C, and the

receive document is actually the body of a
TIFF file compressed in G.3 or G.4

compressions. From our perspective, that's
partial good news, because there are

many vulnerabilities when parsing TIFF
headers, and we only control the data

of the file. The headers themselves are
going to be constructed by the printer

itself, using messages from phase A
and phase D.

So, we partially control a TIFF file and
after it's done and ready, the file

is going to be printed. Like every good
protocol – and here it becomes very

interesting – T.30 many extensions.
Can you guess what interesting extensions

there are in the protocol?

There's a security extension, but no-one
uses it, the other extension…

is..

Color Extension!

Actually you can send colorful faxes and

they really use it in hospitals 
for some reason

Let's see how colorful fax works.

We send a document through 
the HDLC tunnel,

over phase C, and the received document is
actually a JPEG file. This time we control

the header and the data of the file, and
we can do whatever we want to it,

and send it for printing.

Now that we know how a fax
actually works,

where should we look for 
vulnerabilities in it?

Well, we have complicated state machines, 
withstand strings, there are

several file layers, but the most
convenient layer is the applicative one,

and most importantly, JPEG, because we
control the entire file.

If we look at a JPEG file, it mainly
consists of markers, we have a

start marker, application marker with
length and data, more markers with length

and data, and so and and so on.

If we zoom in on one such marker, we can
see that in this marker we have a

compression table, a 4x4 compression
matrix for the exact document we send, we

have a header, length field, 4x4 matrix,
and the data itself.

If you zoom in a bit deeper, we can see
that here we get a matrix, we sum up all

of the values. This matrix should be
rather sparse, with zeroes, ones,

and twos. The accumulated value is going
to be our length field,

in this case 6 bytes, and 6 bytes are
going to be copied from the data to

a local, small, stack buffer.
Like this.

So if you consider vulnerabilities, at
this point we were like "What The Fax?!"

because that doesn't make sense. We
control the entire header. If you put huge

values in our matrix, like so, we have a
4 kilobyte length field copied into

a stack buffer of 256 bytes, effectively
having a stack-based buffer overflow in

our printer.

It's a trivial stack buffer overflow, we
have no byte constraints, we can use

whatever we want, null bytes, non-ASCII
bytes, whatever we want. And 4 kilobytes

user-controlled data, that's more than enough
to exploit. At this point we had to bypass

several operating system security
mitigations… Nah, not exactly.

<i>Laughter</i>

It's an …, fixed address spaces, no
canaries, it's the eighties, it's really

simple. We've got the CVEs from HP,
9.10 critical, you should really patch

your printers now. And here you can see
the response we have seen from HP after

we've worked with them to patch these
vulnerabilities,

which is a good time for our demo!

Yaniv Balmas: Unfortunately we couldn't
really live-demo, so we just filmed

something for you. So, this is our
attacker machine, all you need to do is

run this script, it's connected to a modem
that we bought for like 10 dollars

from Amazon. We're sending our malicious
fax to this printer, and… yeah.

Incoming call… from who?

Wait just a second.

Eyal Itkin: Faxes are slow.
Yaniv Balmas: Yeah, they are.

Yaniv Balmas: So, from an evil attacker of
course, we forged this easily. And now,

the printer is receiving the fax, and
processing it, and now it's obviously a

colorful fax, and now we have full control
over the printer, so it's ours.

But that's not enough! Because we want to
show that we can propagate to another

computer, so our malicious fax, contained
EternalBlue in it, so once any computer is

connected to the network, the fax now will
recognize it, and will try to exploit it,

and here you go!

<i>Laughter &amp; Applause</i>

So yeah, we made it after all.
It was a long way.

Some conclusions we have to tell you:
First, PSTN seems to still be

a valid attack surface in 2018. Fax can
be used as a gateway to internal networks,

and old and outdated protocols… probably
not so good for you, try not to use them

if you can. What can you do to defend
yourself against this catastrophy?

A lot of things. First of all, you can
patch your printers, as Eyal said,

this link will just tell you if your
printer is vulnerable, by the way, every

HP Inkjet (or HP Officejet) printer is
vulnerable to this thing, it's the biggest

line of printers from HP, over – I think –
200 or …

Eyal Itkin: 300
Yaniv Balmas: … 300 models are vulnerable

to this thing, so really go and update!
Another thing I could tell you is:

If you don't need fax, don't use it.
Also, if you do need to use fax after all,

try and make sure your printer is
segregated from the rest of the network,

so even if somebody takes over the
printer, he will just be confined to the

printers, and won't be able to take over
your entire network. These are really good

suggestions, all of them, but really,

the best suggestion
I have to give you today is:

Please!
Stop using fax!

<i>Laughter</i>

<i>Applause</i>

Thank you, thank you!

And, just one second before we finish,
this was a long way, a long journey.

We had some very good friends that helped
us a lot along the way,

physically, mentally, technically,

so we must mention them.
These are the guys here. Some of them are

in the crowd, so they deserve come claps.

<i>applause</i>

One special guy that helped us is
Yannay Livneh, he also deserves this, and…

… that's it basically, guys!
So if you want to follow more of our work,

you can find us here. Follow us.
Thank you very much!

<i>Applause</i>

Herald Angel: Thank you very much.
We have 5 minutes for Q&amp;A.

So please line up at the microphones.
If you want to leave now,

please do it to your right side, so this
side. From the stage it's the left side,

but for you it's the right side.
So please line up at the microphones.

I think I can see microphone 4 already,
so we'll start with microphone 4.

Question: First, thank you for this talk.
It's scary to see that these can be

exploited today. You talked about
email-to-fax or fax-to-email services,

and I wondered: Is it possible that there
are vulnerabilities in those as well?

I know Fritz!Box routers allow
fax-to-email, could you attack those,

possibly?

Yaniv Balmas: So basically, those services
use T.30 as well. We didn't look at them,

frankly. We had so much work to do with
the printer, that we didn't look at any

other printers, or any other services.
I can't say for sure, but if you're

looking for vulnerabilities, I would
recommend to go look there as well.

Herald Angel: Great, microphone number 5
please.

Question: What can you disclose about the
data that's hitting your URL?

Yaniv Balmas: The…? Uh!

Question: What can you disclose about the
machines that are knocking on your URL,

the fakeurl1234.

Yaniv Balmas: There are a lot of HP printers
out there.

<i>Laughter</i>

That's all I can disclose. Sorry.

Herald Angel: We have one question from
the Signal Angel, please.

Signal Angel: Did you try to activate JTAG
by upgrading to a modified firmware?

Eyal Itkin: We tried to use the JTAG, we
think it's disabled from the factory

lines, it was too much work. So we decided
to use Devil's Ivy, it's a good

vulnerability. Once we have Devil's Ivy
and we can use Scout, Scout is more than

enough for debugging.

Essentially, after we used the JPEG
vulnerability and we loaded up Scout,

Scout survived for weeks on a printer
without any crash.

So that's more than enough.

Herald Angel: Great, we'll go with
microphone number 2 please.

Question: Yes, thank you for the nice
talk, and I think you're completely right

you can have many problems with legacy
protocols, the only thing I do not really

get was the part how you then can
automatically successfully attack your

laptop on the network. My point would be:
My laptop is as secured as I'm going to

the internet cafe or something else, so
you would not be able – with your HP

printer – to start the calculator on my
Linux or even on my Windows.

Yaniv Balmas: Your laptop might be secure,
I'm sure it is, but many others are not.

We tried to show it using the EternalBlue
exploit, as you know, WannaCry, stuff like

that. This thing created a lot of…
– and there were patches out there –

…and still it was… So… we're not here to
attack anyone. We're just saying that

theoretically, if somebody wants to get
into the network and he has a

vulnerability that you have may have not
patched or secured, fax would be a bad

idea to have.

Question: But it was nothing which was
part of the printer…

Herald Angel: Sorry, unfortunately we do
not have more time for Q&amp;A, so thank you

again very much.

Yaniv Balmas: Thank you!

<i>Applause</i>

<i>Music</i>

subtitles created by c3subtitles.de
in the year 2019. Join, and help us!