36C3 - Email authentication for penetration testers

36C3 preroll music

Herald: So, have you ever wondered how to
almost perfectly fake an email? Then you

might be actually in the right talk here.
We have our next speaker. Andrew, who is

currently working for the National CERT of
Latvia as a security researcher. And he's

going to talk about e-mail counterfeiting
and strategies for modern anti-spoofing.

Stage is yours.

Applause

Andrew: So. Greetings. I'm Andrew and I
work for Latvian National CERT. One of our

current goals is improving the state of
email security in our country and which we

mostly do through raising awareness about
this issue and communicating best

practices. And of course we are not the
only organization that is doing that.

There are many more CERTs in other
countries and there are various

nongovernmental organizations that are
doing the same. And commercial entities.

However, so far, frankly speaking, our
collective progress has been quite

underwhelming. So for example, here is the
one stat which is the usage of one

specific technology, DMARC, which as you
will learn in this talk, is quite

important and I hope that everyone will
start using it. So on the left. There are

twenty thousand domains across all the
world which are important domains for

important organizations that truly should
know better. And on the right side we see

the top 50, top 500 EU retailer domains
and across both of these groups two thirds

haven't even configured DMARC yet. And out
of those that have configured majority

hasn't enabled strict policy yet. So if
there is just one key takeaway from this

talk, I hope that it will be that everyone
should start using DMARC. It is important

to use it even for domains that are not
supposed to send email. So, one

explanation for these low adoption rates,
I think, is that, there are seemingly too

many competing technologies. This is the
contents for my talk. And I really tried

to do my best to trim it down. But as you
can see, there are three abbreviations, well and

SMTP and out of this, SPF, DKIM and DMARC
actually two are i don't even remember the

whole name for them. But still, they are
all important. And of course, this problem

that there are too many buzzwords, too
many technologies, and it's not clear

which one which ones we should use, it's
not specific to email. And we have this

across the industry and, ah, security
industry, i think by now we have found at

least one way to solve it. And it is
penetration testing. So when the

penetration test has been run properly and
the results have been published, then we

can start talking. We can shift the
conversation from talking about whether

your organization prefers technology A or
technology B we can instead start talking

about the questions that really matter,
such like: Is it possible for someone for

some third party to spoof your
organization's e-mails and to send such

e-mails to your, for example, customers or
your partners or to media organizations in

such a way that they will think that the
emails really came from within your

organization? So that's why penetration
testers are the key audience for this

talk. However, I hope that any blue
teamers in the audience also will find

this talking interesting. I'm sure that
you already know all the basics about the

email and about these technologies, but
looking at the problem from the different

side from attacker's perspective sometimes
can really put things into perspective. It

can help for you understand what you
should focus on when protecting your

environment. And finally, the SMTP
protocol. The technology that runs

underneath our e-mail conversations is
actually relatively easy to understand.

And so. And also the lessons learned from
all of this journey from SMTP, how it

became and how it's possible to spoof it
and all the technologies that are trying

to prevent spoofing. I think it's a
interesting case study and it should be

interesting to follow even for people who
are new to e-mail. Um, finally. Threat

landscape. So email security is quite a
wide topic. And so today I will only focus

on one small part of it, which is
successful spoofing of e-mails. Tampering

attacks. And I know that many, penetration
testers already, incorporate some part of

phishing or spear phishing, emulation
into their engagements and. But as far as

I know, they mostly do it from the, social
engineering perspective using such tools

as a social engineering toolkit, for
example. And it's, uh, I don't want to

argue, though, that it's important to do
that and to demonstrate to the customer

that what risks are in regards with social
engineering. However, I think you're doing

a disservice to the customer if that's the
only thing that you are testing from the

email perspective, because from the
customers, from managers perspective that

are reading your reports, if they only
mention social engineering attacks, then

the logical conclusion is, that the best
way to mitigate these threats is by

educating your personnel, especially those
that are least technical, as you will see

in this talk. There are quite a lot of
attacks and many organizations are

susceptible to them, which are much better
than that. And no amount of user education

will help here because we can't expect
users to check headers, for example,

manually. So we actually need to improve
our e-mail infrastructure. No way around

it. And finally, before we move on to
actual technical stuff, there's a little

secret, which I think might help people
that are not working in the email industry

understand why we have such problems and
is that, for email admins historically,

um, they value availability of their
system and reliable reliability much more

than security. And that's because that's
not an ideological decision. It's a very

pragmatic one. So, for example, if you are
an e-mail an email admin in an

organization and some of your customers
stop receiving invoices, your management

will find you and will inform you about it
and will ask you a really nicely to fix it

as soon as possible, even if it's not your
fault, if it might happen that the problem

is on the other side of the email. Not on
your server. And the for example, if,

other example, if you, if some of your,
some of your employees can't receive

e-mail soon enough, for example, to
restore the password or to verify the

email or to use multi factor
authentication token and they can't log

into some important systems again, they
will find you on though you will need to

solve that. But if your system is has some
security vulnerabilities, if it's assessed

susceptible to spoofing attacks and so on,
then not users, no management will

normally notice it. You might not not
notice it, but you are. You have this

vulnerability. So that's why obviously
penetration testers are important. Okay.

Now we can finally start talking about the
technical stuff. So and we will start with

the short introduction to SMTP protocol.
SMTP is the protocol that underlies all

email communications and it's actually
pretty easy to follow. So here's a data

flow of what's happening when one person
sends e-mail to another person. For

example Alice is sending to Bob and
they're using different they are working

for different companies. They use
different domains. So what's happening

here is that both of them would say use
email clients such as Outlook or

Thunderbird. And Alice is sending email.
It's going through this protocol SMTP to

Alice's mail server. But important to note
is that this is an outgoing e-mail server.

Usually organizations will have two types
of servers, one for incoming transactions

and one for outgoing and for smaller
organizations it might be one server, but

again, it's important for penetration
tester to think of this as different

systems because they will have even if
it's physically one machine, it will have

different configuration for outgoing mail
and for incoming mail. So as a penetration

tester you need to check both of them.
Okay. Now, when Alice's server tries to

send email to Bob's server, there is sort
of a problem in that the server needs to

somehow automatically find what is the
other server to send the email and it is

done through this blue box MX which is DNS
specific DNS record type MX. So that's

something that is maintained by Bob's
organization. So Bob's organization, if

they want to receive e-mail, they create
this DNS record. And I say that. Okay. If

you want to send e-mail to us, please use
this particular server. So it should point

to Bob's server. And Alice's outgoing
server knowing Bob's incoming server

address can communicate to that. And then
later, Bob, will receive its e-mail. So

the part that we as penetration testers
will be trying to breach is actually

between Alice's server and between Bob
Server. And then we need to think about

the second example, which is the opposite
way. And you might think that it's a

pointless example because we are just
basically changing the direction of

traffic. But the important part here is
for us as penetration testers to

understand that our client only controls
part of this transaction. If our client,

let's say, for the rest of this
presentation is Alice or Alice's

organization, then in the second example
when we are sending mail from Bob to

Alice, then we'll be sending emails only.
Basically, part of this transaction will

go through Alice's servers. In the first
example, if we were sending email from

Alice to Bob, it wouldn't be so. So if
it's a bit confusing, that's okay. We will

return to that a bit later. And finally,
there is a third example which looks

similar, but not quite. And that's if
Alice is communicating. Alice is our

customer. And if she is communicating with
her coworkers, which are using the same

organization, same e-mail server, same
domain. In that example, again, there will

be to at least logically two email
servers, outgoing server and incoming

server. But both of them will belong to
our customer. So right now, if you are not

familiar with e-mail, you can. It's
just interesting to try to think which of

these scenarios, three scenarios, which of
them are easier to protect? And a bit

later we will see how it's actually
happening. Okay. And then we need to look

at what actually is being sent, when email
is being sent. So again, it's using SMTP

protocol and it's really nice protocol you
can. As you can see, it's just text. So

it's plain text protocol and it's very
easy to play around because you can just

open telnet connection to the right server
and you can try writing down the commands

just with your hands. So you can try
mangling something or modifying or trying

different, different, different types and
see in real time how it was going on. So

on the left side we see here two parts
which are defined by SMTP. So first of

all, there comes SMTP envelope, which
basically you connect the server, say

hello, then you say what. Specify the
sender of email and recipient. "mail from"

is sender. Recipient is Bob, for example.
And then the second part starts with data

and ends with quit. And that's the part
which is called Content/Message. So just

if you want to play around with it, a bit
more, this is defined by a different

standard, which is not that important for
penetration testers but if you want to

look into details and it might be
important. And this internal message,

which is called either Content or SMTP
message, it again, it contains two parts.

One is headers and another is body. And I
think some people might not be familiar

with email, but probably everyone is
familiar in this audience with HTTP and

this looks quite, quite the same. So easy
to understand. But the interesting part

here is that you might have noticed that
we have Alice's and Bob's addresses twice.

Right. For example, Alice's is specified
on the second line "mail from". And then

we have the same address. alice @ her
organization in "From" header. The red

ones are the headers. And the same goes
for Bob. So why is that? Well, it comes

down to how we see e-mail. I as a normal
regular person who has used email in

past quite a lot, i usually see them as
described on the left side, which is a

sort of postcard. So on a postcard there
is someone who has sent it. The sender.

There is the recipient. That's usually me.
I'm receiving. And then there's some

message. So at least that's how I
perceived it before I learned a bit more

about it. But email admins and the
standard bodies, they see this situation

as the one which is shown on the right,
which is. There is an envelope and inside

the envelope then there is this message or
a postcard maybe. So you have two

addresses in this scenario. You specified
the address from and to whom you are

sending the envelope, which is the part
that post office, for example, will look.

But post office won't look generally
inside your envelope and inside the

envelope there is another message, and
that is the internal message is actually

meant for a recipient. So actually, you
could do even more and you could even put

the whole envelope with the message of the
postcard inside another envelope. And this

sounds crazy to me as a regular person,
but actually e-mail allows that. And in

the RFC the standard document, there are
some examples why that would be necessary.

Why why such why such things are allowed.
But but they are confusing. And so as a

result, it is the here in this first
example, we see that we generally we are

specifying the same address twice. But as
a penetration tester the question that

we should be asking is: So is that
required, actually? Is that always true or

is it just like a wishful thinking? And
it's actually wishful thinking. So

standards specifically do not say that you
should be specifying the same address for

recipient or for "From" from the sender on
the envelope and inside a message. So you

could actually tweak them and send
different, different stuff. So, actually,

there are much more headers than what I
showed. The ones I showed I think are just

the ones that we all have experience
because even if you are just using e-mail,

that's usually the stuff that you see or
see the date, you see the subject, you see

who has who sent you something and to whom
it was sent. Usually yourself. And there

might be, of course, more recipients. Oh,
yeah. And the question then another

question is: Which one is actually, if we
have specified for some reason by accident

or especially if we have specified
different addresses in this envelope in

the message which one the user will see
the recipient, it's actually the header.

So inside that the message is the one
which is intended for the user. OK. So and

as I was saying, there are actually
standards allow a bit more headers. And

actually 3 headers "From", "Sender",
"Reply to" which are semantically really

close and in the standard it's actually
explains when you should be using which

one. And the funny thing for me is that,
for example "From" header, which is

usually the one with that we see it might
contain . By reading the RFC you will see

that you shouldn't have more than one such
header, but the header itself might

contain multiple addresses. Personally,
I've never received an email which would

come from different people, but that's
allowed. But the important thing to

understand here again is the backwards
compatibility that I mentioned before. So

even though standards explain how you
should use the each header and that you

shouldn't have more than one of each of
these headers in practice actually can

send malformed email. You could send email
with multiple headers, the same header

"From" header multiple times, or you could
send header which does not contain "From"

but contain "Sender" according to RFC
that's incorrect. But in practice it will

work. Most organizations, most e-mail
service will try their best to pass your

completely malformed email because they
really are concerned about lowering the

support costs. So if something does not
work, then you will come to them. So it is

better to make that everything is working
most of the time. Of course, for

penetration testers that means that you
can play around with this because there

are different implementations and it's
exactly which header, for example, if you

have two headers, will be shown or will be
used for some algorithm. It depends on the

particular implementation. So because
there are so many implementations, they

are interconnected in different ways. You
could and you should as a penetration

tester try various things, for example,
add the same header multiple times. OK.

Now that we have covered these basics,
let's actually look into how you would try

to spoof an e-mail, for example. Yeah. And
here we are again, we are coming back to

this diagram that we have seen before. And
for example, in the first example about

Alice is sending email to Bob. Let's say
we are, Chuck. So we are a third party. We

are penetration tester licensed, we have
an arrangement that we are allowed to do

this and we are trying to send spoofed
e-mail to Bob. And in this example, we are

trying to spoof Alice's message. So our
intention is that Bob wants Bob receives

email. It should look to them, to the Bob,
that email was sent by Alice. So risk for

this. Okay. I will not cover the risk. I
think you can imagine that. So, for

example, you could do fake news is one of
the problems that we have seen in Latvia.

It's one this was used against government
bodies. And when someone sent a fake news

e-mail to other people, organizations and
so on, and were trying to impersonate some

some government person. And of course, you
could could imagine yourself how it's not

a good thing if you if it's possible. But
the interesting thing here is that even

though Chuck is doing attack, it depends
on your perspective. It might look like

attack on Alice or on Bob. But in this
case, email won't go through Alice's

systems. As you can see, Chuck is sending
e-mail directly to Bob's incoming

server. Now, there is a second type of
attack that will be looked at. If we are

sending e-mail in other direction from Bob
to Alice. And our customer is Alice. So we

are testing Alice's server. And in this
case, we are trying, again we are Chuck.

We are sending e-mail. In this case,
e-mail will go through Alice's systems. So

interesting question is, which is easier
to protect. It might seem that since in

the second example, e-mail is actually
going through Alice's systems, that means

that Alice has more power to do something,
to do some additional checks and balances

and so on. But actually, as you will see
in the future, it's easier to protect the

first example. So even though our customer
is Alice, we're trying to protect Alice,

but it's easier to protect in practice
this example where someone is selling,

sending e-mail, trying to impersonate
Alice. Okay. Oh, yeah. That there is the

third example, which is if Alice is
communicating with her colleagues inside

the same organization. Again, we are Chuck
in this case. Again, we will only send the

e-mail to Alice's incoming server. Not to
outgoing server. Right. So important thing

to note. And again, in principle, this
third example is the easiest to notice,

because Alice's organization presumably
knows that her e-mails always should come

from this particular outgoing server.
Right. Like if we are sending e-mail from

Alice's colleague, then incoming server in
principle should have all the power, even

without any standards and stuff like that.
But in practice, sometimes actually quite

often there will be a specific whitelist
for Alice's own organization. So some

checks won't happen if incoming server for
Alice is receiving email, which is coming

from, again, Alice. And by the way,
there's this example. We've seen that for

the past few years. I think it's not
specific to Latvia. So here, for example,

is Canada and others,if you can see. This
are these emails which are fake like

ransomware stuff. Basically, they are
telling you that they have hacked your

computer or your email. In this case, and
they have arranged all sorts of financial

activity or have some blackmailing you.
And please send them the money. Your

money. I mean, your money in bitcoins to
their address. So, these e-mails.

Interesting part about these e-mails is,
that they are usually in order to prove to

you that they have access to your e-mail
account. They are sending e-mail from your

address to your address. So and for many
people, that works. So they see that

someone has hacked their account,
obviously, because they've received e-mail

from themselves. So as you will see a bit
later, it's actually easy to spoof such

e-mails if there haven't been any
protections, haven't been put in place. So

the important thing, I hope that now no
one in this audience is falling for such

scam. But if you have some friends or
colleagues that have contacted you and

told you about such e-mails that they have
received. But one of the things besides

checking the passwords is starting using
more effective authentification on is a

just maybe you could tell them that they
should contact their email administrators

or IT team and ask them about anti
spoofing protection, because obviously if

they are able to receive such e-mail and
it's not filtered, something is wrong.

Okay, and now let's see a spoofed SMTP
conversation, so that's example similar to

previous one. But in this now we are
actually Chuck. So this is sent by Chuck

to Bob, but we are pretending to be Alice.
The question is, can you see the

difference how this is different from from
the previous one? And it's hard to see the

difference because there is none
difference. That is the same conversation.

So the point here is that SMTP protocol by
itself it actually it doesn't have any

protection. So, yeah, you could just for
example, if you are that guy that is

sending the fake ransom letters, you can
just write down this text and just dump it

to telnet and it will work for many
organizations. Not for all. And of course,

the email admins know this stuff, know
that SMTP is not very reliable in this

regard. That's easy to spoof and so on.
And there have been many attempts to add

some protection, just like ad hoc way. So
no standards just to ransom, add some

additional filters and stuff into your own
mail. And some of these protections

actually break RFC. If you read it, but
who cares? Like RFC is not a sacred text

or it's. I absolutely approve this, for
example. So yeah, go on. But the problem

is that there is not enough information.
So if you think back here, if we are Bob

and we are trying to protect our systems.
So we are Bob, some system administrator

probably or Bob is a sys admin and we are
trying to add some additional rules and

stuff, then what actually can we do? So
one example that I listed here is doing

this SMTP callback, and that means that we
are just the when we receive e-mail from

Alice, we actually check does that email
exist at all? Because many spammers, what

they will do, they will just send e-mail
from non existing emails and it will work

by if you are just running raw SMTP
server. So SMTP callback is basically you

are when you are receiving email from, for
example. Alice, you are trying. You are

running, spawning a separate process which
will try to connect back to Alice, etc.

And it will try to send email her. If a
server says that. Yeah, that's okay. Such

email exists and so on. You are not like,
you actually stop the conversation. You

don't continue with sending email, but
then your system can automatically find

that actually this e-mail really exists.
So another way to do this is through

checking this "Hello". And this is the
first line and the first line, it's,

normally it should tell you the hostname
of the server that is sending email.

Interesting part. So according to RFC
again, you shouldn't check it that you

shouldn't verify. And if it doesn't, if
it's a random thing, you should accept

email still. But what many servers will do
is they will try to verify that. First of

all, this hostname, which you are telling
that you have this hostname. First of all,

that it really points to the same IP
address and then they do the opposite. So

they will take IP address and try to run a
reverse DNS PTR query and they will try to

find whether that IP address really
responds to this hostname. So again, as a

penetration testers we should be aware of
these protections, ad hoc protections,

because they are if you don't know about
them, you will try running something and

it won't work for you. But they are easy
if you are aware of them and if you have

to identify that this organization uses
them. They are easy to bypass so that they

don't offer good protection. They are
meant to protect from mass abuse from

spam. OK, so SMTP, as we've seen, by
itself does not do does not offer any

protection. So which additions to the
protocol actually can we use to protect

ourselves? One of such protocols is SPF.
And what SPF does is it's trying to be

like mirror MX system. MX system is the
one which basically Alice can use to

Alice's server can use to automatically
find the server that belongs to Bob and

its incoming server. So. SPF is the
opposite of that. So that's an idea is

here to run the system automatically on
the Bob's incoming server. And now when

Bob receives the e-mail, they can run
again DNS query and they can find what IP

addresses actually should belong to
Alice's outgoing server. Right. So it's I

think it's easy to understand it's
actually a meaningful way. It sounds

meaningful addition. And the one field
that is checked in this example is this

envelope sender. OK. And here's an example
of minimal SPF syntax and the as we can

see. I think it's easy to understand, even
if you don't know the syntax is it lists

IP address, which is IP, should be IP
address of outgoing server, legitimate

outgoing server. And then it says this
"-all" which again, is easy to understand.

In this case, it means that that's the
only one. So if you receive a message,

message comes from this IP address. That's
cool. I accept it. If it's something else,

then just drop it. And there are multiple
ways to specify the IP address. You could

just specify the IP address. You could
specify IP subnet, you could specify DNS

hostname. So it's just for admin. So
basically for a penetration test, it

doesn't do much different, for admins it's
just easier to maintain these systems. And

then there are these qualifiers,
qualifiers. This is what's something which

you put before the methods. For example,
here in this example, IPv4 before doesn't

have any qualifier. There's no plus or
minus or something. That's because plus is

assumed by default. So by default,
everything that is listed in SPF record

will should the match some legitimate SMTP
server, outgoing server. However. There

are other options you could use minus
which is fail. And that means if something

matches this record, for example, minus
all is the one which is the most often

used, it means if it matches this one, so
that's usually the last one, then please

drop the mail. It's not real. It's it's
fake mail. And then there's this third

option, which is softfail, and that's
meant for testing period. So when you are

just starting to implement SPF, there
might be some. So the problem is that you

might forget, for example, to add some
SMTP servers. So because you haven't done

it before, maybe you think you have only
one SMTP actually outgoing server. But in

fact, you have multiple of them or
multiple ways to send e-mail. So in that

case, if you were to start set that SPF
record with "fail" strong policy, then

your users won't be able to send the
message anymore. So that's why testing is

good. However. Here are some other
examples, a bit more complicated. One of

them is was include. So instead of
defining the policy yourself because

you're using third party, for example,
Google in this example, and then you will

just include whatever Google has
published. And the interesting thing is

this usage of SPF. If we just if we just
look at the amount of domains that have

defined some sort of policy, that the
number looks pretty okay. I guess that's

for example for most popular domains
that's around 70 percent. But the problem

is that the majority of them are either
poorly configured or they just use the

softfail option. And what softfail
practically does is nothing. You still can

even if there is policy with softfail, you
can in most cases you can spoof your email

and it will still go because the recipient
side will think that it's just in the

testing mode. You shouldn't drop e-mail
automatically. Yeah. So. Actually, the

percentage isn't that great. However, the
most important thing for us as penetration

testers is to understand. So what do we do
when we see this SPF. That means that now

we can't spoof mail and. No, it does not.
That it's game over for us. We can do some

stuff. So first of all, is this softfail
that I mentioned. And that's basically you

have some rules, rules, rules, and then in
the end, you are putting typically just

this softfail at all. So if we as a
penetration testers will try spoofing from

some unknown IP address that hasn't been
listed in the previous rules. Then do

nothing. Do nothing. I mean, don't drop
email. That is good for us, right? That

means that we can actually spoof just in
the same old way and it will mostly go. So

the one great one note here is that some
systems are you are not using just this

binary classification, whether something
is good or bad, but they are trying to run

some scoring. And then it might be that
even if you have this soft fail, they

won't automatically drop your e-mail, but
maybe they will add some like suspicious

level to it. But important thing is that
it's not automatically a game over.

Another thing is this include. So include
is it very convenient when you are using

third parties. But the problem is that
it's not what it sounds to some people, at

least even in the standard, it mentions
that it was a poorly chosen name. And the

reason for that is that it's not a macro.
So to understand what's happening when

this included, you shouldn't just copy
paste everything from inside recursively

to the top level. It's not how it works.
It will try running all the checks inside

this include. But then if it fails, it
won't automatically drop the message. It

will go to the one level top and it will
try running the other rules. So the

problem with that is that two cases that
are the most common is that either if you

just forget to add this minus all to , or
your system administrator who has

forgotten to do that. In that case, even
if they include has minus all, it won't

work because I mean, it would because when
the recipient will be checking it minus

all inside include does not mean the same
as it does on the top level. And the

second would be if they have added all but
did softfail all. And some admins might

think that. But that's okay because I'm
including GMail and GMail has this hard

fail. Doesn't work that way. And then one,
which actually is I think maybe the most

common case, is that something often you
actually see this type of SPF records, but

there is lots of stuff inside there is IP
addresses. There are these A records,

there is a MX. There is a pointer.
Basically, everything that the admins

could think of and the reason is that the
most commonly, they are just not sure how

it works. They're not sure what they
should put inside. So, for example, one

thing that the point that out is if there
is a MX record inside the SPF, most

commonly most organizations, unless they
are very small and just have one server,

they will have different servers,
different IP addresses for outgoing mail

and for incoming mail. That means there is
no practical for this organization,here is

no practical reason to include MX into SPF
because no, no mail should go out through

their incoming mail server. And another
case might be that the admins understand

how it works, but it's really, truly their
architecture is really messy and they are

sending emails from many, many different
points, which is good for penetration

testers. That means that they are not well
organized. OK. And then there's another

flaw, which is that granularity isn't very
well suited. So the only thing you can.

There are multiple this record types. But
all they do basically are resolve the IP

address. But the as you can imagine, in
many cases, IP is not linked just to mail

server. So on one IP, there might be mail
server and web server or database or

something else. And that means that as a
penetration tester, you can exploit this

something else. Not mail server itself,
because mailserver usually is pretty like

low key. There's not many vulnerabilities
there. You just patch them and that's it.

But those other systems, for example, web,
it's easy to exploit. In most cases. So

then you can elevate like in some sort
elevate privileges by gaining access

through some other server on that IP
address or IP range. You can start sending

mails. They will pass all SPF filters. OK.
So one example is shared hosting, which is

the very common case and the problem with
shared hosting is that. In this case.

Okay. You have IP address of SMTP server.
Maybe that's server only used for sending

mails. But the server itself works not
just for you. It works for many domains,

maybe hundreds of thousand domains. That
means as an attacker, again, you can

exploit at least one of them, or for
shared hosting you can just buy. You can

become a customer of that shared hosting.
You don't even need to exploit anything.

And then you can potentially start sending
email, which will look good as far as SPF

is concerned, just like their own. So. And
the another one is this checking wrong

identifier. And this is probably the
worst, worst problem with SPF. It is that,

as I mentioned before, the one there are
at least two identifiers. Typically

envelope sender, the outer one, which
lists the sender, and then there is

internal one, which is usually "from"
header. But out of those two SPF only

checks, if SPF is the only technology that
you are using, SPF only checks the first

one: envelope sender. And as I mentioned,
in most cases, actual users that will

receive the mail, they won't see envelope
senders. They will see this and this other

one "from" for example, or one of the
other headers they mention. So this

behavior is fixed actually by DMARC, which
is the technology that I mentioned. But

the majority of SPF installations, domains
that are using SPF do not have DMARC, so

they are not protected by this. So even if
their SPF is completely great for

attacker, it means that you only need to,
what you need to do to pass SPF is a to

set envelope sender to something else. For
example, your own controlled address,

which will pass all SPF checks. But then
inside the "from" you can show the header

that will match this organization that you
want to pretend to be. Okay. So then there

is another technology which is supposed to
fix this and it's DKIM. As we have seen,

SPF is not enough. So DKIM. Sorry, the
wrong letters, Domainkeys identified mail.

That's the DKIM and you don't need to
remember the long name, just the short

name. And what it does, basically, it uses
cryptography, which is nice, right? It's

math. It's hard to break for attackers.
And what it does is it signs every mail so

every mail that is going out through the
DKIM enabled server will get signature,

which you can, as a recipient, you can
cryptographically verify. So as you can

see, how it looks is actually pretty hard
to see because it's not meant to be

processed by humans. It's cryptography.
It's meant to be processed by computers.

But the important part here is basically
the yellow stuff is this cryptographic

signature. But the green part is what's
called domain identifier. And the red part

is what's called. I don't remember how
it's called laughs. But basically it's

idea is that you can have multiple keys
for your organization, for example, your

organization might be sending mails from
your original SMTP server, then you might

have a backup one or you might have might
be sending some messages from Google or

some marketing campaign and so on. And
then each of them might have different

"red", this parameter. The problem is and
then the recipient will need to run DNS

query, which is the second example using
this combination of green and red one. And

then they will get the public key and they
can use this public key to verify the

signature. So it's sounds really nice. The
problem here is no, another problem yet.

So how to use it? I think it's easy if you
understand the public cryptography. So on

the sender side, you need to first
generate public and private keypairr. Then

you publish the public part in the DNS.
Then you use private key to sign each

message. Now recipient does sort of the
opposite. They once they receive the

email, they figure out from this red and
green part they figured out the correct

DNS record to run, run it, get the public
key and then compare whether this public

key corresponds to the signature. So it
sounds really nice, right? What's the

problem? So customers. Selectors, that's
the name. So the problem with that is that

the selectors there might be multiple
selectors as a DKIM when you are doing

configuration, you can select as many of
this custom selectors as you want, and the

recipient doesn't know whether you
actually should have used a selector and

what selector you should have used. So the
problem is that while, if we are talking

just about the vanilla DKIM, modifying
existing signature is hard for penetration

tester or for an attacker. But it's easy
to just remove it because if you have

removed DKIM at all the header, the
recipient doesn't know that it should have

been there because in order to check, they
need to. So here, for example, in order to

check the signature, I need to know this
green part. This domain identifier and the

selector which are part of this header.
Right. So that's a huge problem. And that

means that. Yeah. That means that we can
actually while we can't spoof DKIM itself,

we can just trim DKIM, send the message
without it. And if the DKIM was the only

thing which protected this system, it will
work. So it might not get the green

checkmark or whatever, but it will get to
the recipient. So. And another thing is

this domain selector. Why do we even need
to set that? Because the best practice, of

course, is that you have envelope sender
equal to "from" header equal to this DKIM

domain selector. Right. So if you are if I
am sending from Alice, then all three

should be Alice.org or whatever. The
problem is that it's not mentioned in RFC

that that should be the case. So what
exactly happens when it is not that way?

For example, on the right side there is
some real domain which was using Gmail,

Google Apps, Google suite, and in that case
the default by default Google suite will

sign all messages. But if you do not do
your own configuration, it will sign them

with domain it controls, which is this
"gappssmtp". And what it means is that

although technically something has been
signed with DKIM, it wasn't signed in the

way that you can trace back to your
organisation. It's something completely

else. What exactly recipient should do in
that case? Should they just ignore it?

Should they reject the message or
something? So the correct way would be not

to reject it, but just consider it not
valid, at least not not a valid DKIM, but

it actually depends. So some validators
will just see any DKIM, will validate it

and will say that's cool that matches RFC.
So but now the interesting part. Modifying

DKIM, which I don't have time for. But the
idea is that in some cases this is not

always but sometimes you actually can
modify. The easiest part to modify in the

messages are headers because DKIM, since
it's placed in headers itself, it does not

automatically sign old headers. There's
like a chicken and egg problem. So by

default it only signs one or two headers
and you can specify more headers that need

to be signed, but it doesn't happen
automatically. So the easy part for

attacker is to add another header. If
that's somehow helps you in your like

plan, then that's easy to do. You just add
another header. An interesting part is,

although the RFC, as I mentioned before,
mentions that some headers such as

"subject" or "from" should only be present
in one copy. Actually you could add more

than one for example "from" header, and
what happens in that case is pretty

interesting. DKIM will match if you have
told to DKIM that "from" header should be,

for example, signed, then it will match
and sign first "from" header from the

bottom. But quite a lot of software in our
software email clients will actually only

display to the user first from the other
side, from the up side. So what it means

is that the attacker can mangle or
overwrite headers by just adding new

headers to the top. And the this actually
problem is mentioned in the DKIM RFC and

the protection that they propose is this
code Over-Signing or you can go and read

the RFC. But not everyone is doing that
actually. And however, that only goes to

the headers. So sometimes that is good.
Sometimes that's not good. Modifying

message body is actually much harder to
do. Basically the naiv way do it through

cryptography, which we don't want to do.
And another way is through this one

parameter, which is body length, and
that's actually like questionable

functionality that DKIM has. Sometimes you
can specify that the hash like. For

signing purposes, we shouldn't consider
the whole body, but only first something

bytes. So that's actually useful in some
cases regarding was a mailing list, but

for the most part that's not useful. And
in practice, most email software does not

do this. If it does, then it is
susceptible to potentially to this

overwriting body as well. You could add
another mime type and then then modify

headers to show that different mime type
and it will pass DKIM. So in this case, it

actually will show, for example, the green
button or whatever, because DKIM, it will

be valid. So now there's the third
technology, which is called DMARC. And

again, there is the full name, which is
long, but in this case actually it means

something. There are two key words:
reporting and conformance. Reporting is

the one which most admins are familiar
with because that's how DMARC I think

often is being sold to them. Reporting
means that when you have some problems in

this case, you actually get get to tell
other side what to do in that case. So

basically you tell them to send you
reports either once per day or every time

and so on. So for penetration testers,
it's not that useful. Potentially we could

use that to understand what sort of
configuration is running on the other

side. But the currently this functionality
actually is not that widely implemented.

However, the other part conformance, it's
actually really, really, really powerful.

What it does, that it corrects these major
flaws that I mentioned in SPF and DKIM. So

first of all, DKIM had this massive
problem that if you just strip down the

header, then the recipient has no way of
knowing whether you whether there was

should have been DKIM in first place. If
you are using DKIM alongside with DMARC

that fixes the problem, because DMARC
specifies just that you have DMARC itself.

It means that you're automatically at
least one of the SPF or DKIM should pass.

So automatically DKIM is like measure
problem solved. The other thing that

changes is, it changes the semantics for
SPF. Now, SPF, if you have both SPF and

DMARC, it means that SPF should be checked
against "from" header. And as I mentioned,

that was the major flaw with SPF, because
if you're using SPF itself, even, it is

the hard to fail mode and so on, it means
that attackers can modify "from" headers

still and the recipient won't know any
better. So a minimal example of DMARC is

really, really small. And I think it's
easy to understand. You have just a DMARC

reject. You need to like find out the
right place to specify. But it's easy and

all you have to do is create this one DNS
record. And the benefit for that is even

if you don't have DKIM and DMARC, if you
have created. Sorry if you don't have SPF

and DKIM, but you have created DMARC,
effectively what it means is that this

domain should not send any mail because
for recipient to consider a mail valid at

least SPF or DKIM should be valid as well.
If they are not, then they can't be valid.

So in fact what it means is that most
domains out there should consider enabling

DMARC. That's just the right thing to do.
OK. So there are more tags. So in the

wild, these DMARC records might be much
longer, but it's not of much use to

penetration testers. So important part
here is again, this is this policy which

can be three values "none", "quarantine"
and "reject". And if it is "quarantine",

that means if the, if there is a failure,
the message should go to the spam folder.

If it's "reject", it should be rejected
outright. And if it's "none", it means

it's in investing mode. So and this is the
picture that I showed in before, which

shows that actually even though DMARC is
really like the best technology out of

these three, it's not really widely used,
unfortunately for defenders. Quite a nice

fact for all penetration testers out
there. That means that you can, in fact

spoof most of the mails out there. Okay.
So how do we work around it? Sorry. So.

What happens if actually someone has
implemented DMARC? Does that mean that now

penetration testers can't do anything? You
don't don't even need to do any research?

No, it doesn't. So in practice, if someone
has implemented both DKIM and DMARC, but

not SPF, so they have only two of them.
That's a really cool combination. DKIM is

pretty powerful and the major flaw that it
had DMARC solves. So this combination is

really cool in theory. In practice, the
problem is that in order to protect your

own mails, the recipient side should
validate both DKIM and DMARC and

unfortunately, quite a lot of software
still does not do that. One such software

is Microsoft Exchange. And even if you are
not running Microsoft Exchange, chances

are good that some of the partners that
you are communicating with are running

Microsoft Exchange, and by default it
doesn't have any functionality to parse

DKIM. So in fact, most systems still need
to enable SPF just for practical purposes,

which is good for penetration testers
because if SPF and DMARC as enabled by

default together, then again that fixes
one of the major problems with SPF, but

does not automatically fix other problems
because there's not enough granularity and

the potential for misconfiguration. So.
And the interesting fact is that DMARC

only requires that one of the other
technologies SPF or DKIM is passed in

order to consider email valid. There is no
way in DMARC, even though there are many

others like selectors. There is no way to
specify that both of them should be valid

or that DKIM should be preferred to SPF.
In practice, what it means is that for

most systems that enable all three of
them, which is a good practical solution

from penetration tester side we can just
ignore DKIM outright and just focus on SPF

because the SPF is the weakest link in
this situation. Okay. So just a minute for

recap. I'm not sure if I have any more
time. Not many time I have. Okay. So

sorry. Yeah. So one really important note
is, when you are testing the systems,

consider both scenarios. So don't focus
just on send. If you are, for example,

testing Alice. Alice is the organisation
that is your customer. Don't just focus on

testing emails sent impersonating Alice,
but also as the other side. Because in

this here you can see that it's easy to
implement for example, SPF and DMARC

because for both of them only you only
need DNS configuration. Just one record

per each. However actually testing them
like well validating them properly is

harder. For the first you need the
software support, you need to configure it

correctly as well. So in practice it might
be that many of organisations that have

enabled DMARC or SPF on the DNS side for
outgoing mails, they are not actually

properly validating it. Yeah. Okay. Sorry,
I don't have time for that. So probably.

That's it. Sorry. Maybe some questions.

applause

Herald: Thanks, Andrew, for this nice
talk. Sure. We have time for a couple of

questions. So there I already see one
person, microphone number two.

M2: Hey, thanks a lot. Do you know some
good tools to monitor DMARC reports that I

get sent by my recipients?
A: Yeah. So this is a really good

question. We as a CERT, we are really
suggesting everyone to enable this tool,

but unfortunately, as far as I know, all
the tools that are popular on the

Internet, they are collecting some data on
you. So they are using it for marketing

purposes, do they are not very good for
privacy, if you are concerned about that.

So you need to implement something
yourself or you need to look at some,

start some open source project maybe.
Herald: OK. Microphone number one, please.

M1: Thank you for the good talk. Me
myself, I would consider myself an mail

administrator. I sometimes get advised to
shorten your SPF record because if it's

too long, it gets dropped anyway. For
that, I sometimes get advised to drop the

PTR record. But in your talk, you say the
PTR record is useful for reverse DNS

checking, which I find very useful as
well. How are you about shortening your

SPF and how are you about the PTR record
in general?

A: Well, it really depends on your
particular use case. So it might be the

case that some organizations really need
this longer SPF and there's not no way

around that you could do. What you could
do is include this, include use includes

because they won't be they are not macros,
so they won't get expanded. They do not

like your record doesn't become longer if
you include and use many includes. But the

problem, which I would suggest to you is
actually reconsider whether it's a really

whether you really need that many records
if it's still long, because they're a very

common problem, is that unless you are
Google or something like that, you don't

really need that long SPF. It's probably
some problem with some. Yeah. So it's

probably an error for most organizations.
Herald: OK. Well, very. Just briefly.

Number 1
M1: On the PTI rocker record. I heard that

it's dropped. Not dropped from the
standards, but it's not in the standards.

A: It is in the standard. No. PTR record
by itself is if it's really your use case.

I don't I'm not aware that it will be
automatically dropped somewhere. Shouldn't

be a problem.
Herald: We have a couple of more

questions here. So number six in the very,
very back.

M6: Thank you for your talk. That's not
directly related, but even it should be

related. If mail server accepts because
DKIM, DKARC and SPF, everything is fine,

but especially Google for a lot of
organizations, the mail is delivered but

classified as spam. It means on the inbox
of the recipient, it is not displayed.

Have you a solution to solve this problem
against Google?

A: Yeah. OK. So I have like different
opinions about that because one thing

which actually enables which we actually
should be doing. Thank you Google. Is

that they are so strict because that's the
only reason that we even have this high

percentage of even improperly configured
SPF. The only reason there are 70 percent

websites are using SPF is because that
they need to communicate with Google. And

Google won't accept your mail if it
doesn't have even SPF on the baseline. So.

I actually I enjoy it as a job that I do.
I've. I would prefer that Google does what

it does. But I understand the real admins
which have this problem. Google has the

tool. You probably know about it. Where
you can check what it considers about your

domain. So you need to consider this
problem on a case by case basis. Quite

often what happens is that even though you
have this DKIM, DMARC and so on, it's not

configured correctly. So that's what the
talk was about. So you have it. You

probably think that you have configured it
correctly, but there are some errors.

Herald: Okay, let's give priority to the
Internet.

Signal Angel: We have one question from
the Internet. Well, attempting to verify

and address how to handle no reply email
addresses.

A: No reply, I'm sorry. Can you read it
again, please?

Signal Angel: When attempting to verify an
address, how to handle noreply Email

addresses.
A: Maybe it was about the noreply header ?

Or not existing IP addresses ?
Signal Angel: How to handle email. No

reply email adresses.
A: I will try to get an answer to how I

understand it. So what often happens is
that what often happens is that the email

will be sent from nonexisting addresses.
So maybe that's what the question was. For

example, there is "no reply", and it's not
the problem itself. No reply. The problem

is that it's not an real address. There is
no such address. Right. And so I don't

have an answer for that because according
to RFC, you should you should still accept

it. Practically, as I said, lots of mail
systems already are dropping this

addresses if you're sending from not
existing unless you are Google or

something large, so you have been put into
whitelist. You just won't be able to do

that. You won't be able to send email from
non-existing address. So if that's your

situation, create the address, make it
like a remove all the email that comes

there, but create the real address so that
your acceptable. If you are on the other

side. So you are receiving this email. It
depends on this particular use case. So

just check what's going on. If you can
contact them, contact them. If you can't

contact them, then you should decide what
is the risk, if you are dropping these

addresses, are they important for you? So
according to RFC you should receive and

process this addresses.
Herald: Okay. Microphone number four,

please.
M4: Hey, thank you for this talk. Do you

know about effort to solve problems with
big email senders like online booksellers,

which are very great because they don't
seem to have their own SPF records, for

example, in in control.
A: Yeah. So in many cases you can just

contact them. So it's just the question
that they haven't thought about it. Or

maybe no one told them what to do or maybe
they don't know how to do better. Right.

So that's one of the parts that we as a
CERT we are doing. If you have some some

this problem with some large company in
particular country, I would suggest to

contact CERT. Even if it's not a
government organization, for example, in

Latvia, if that will be a latvian company.
We would do the triage. We would try to

try to talk to them, explain to them why
they need to change and so on. So that's

maybe one option for you. But the
practices that if something looks to you

as a third party, as a wrong
configuration, that is one I couldn't

mention in this talk. If something isn't
perfectly secure, it doesn't mean that

it's wrong. There might be actually
business case why it should be this way.

Right. Because, for example, if it's a
large I don't know, Amazon and some for

something like that. And if they have
tested and they know that when they enable

very strict configuration, some percentage
of their emails just doesn't come. Not

because of their problem, because of
someone else's problem. Right. But then

there is actually a real business case
that they they are not. It would be stupid

for them to enable this, you know, to
strict configuration, knowing that it will

damage their business. That makes sense,
right?

Herald: Okay. We are unfortunately running
out of time for those who are on the

microphones. please just line up with the
speaker next to the desk. He's gonna talk

to you. Perfectly sure. And.

applause

36C3 postroll

Subtitles created by c3subtitles.de
in the year 2020. Join, and help us!