36C3 ChaosWest: CTF in a box

We are about to start the next talk right

here. So, I am very happy to introduce

Hanemile. Who is going to talk a little

bit about the struggles you are facing

when trying to find the next capture the

flag (CTF) adventure and how he is

proposing to solve the problem.

Please join me in welcoming Emile.

Hi, I am going to talk about CTF in a box.

It is the story of what problems we

found when playing CTFs. How we plan

to solve the problems; we built a

prototype, tested it and the problems

that came after that. So, first who am I

I am Emile, @hanmile at post platforms.

Studying computer science at Düsseldorf.

Playing CTF with @flexerilla or sometimes

as a single player.

Lets start with the current solutions.

Playing CTF we currently have, like, 3

main platforms. The most used framework

used currently is CTFd.

CTFd is the first thing you'll find if you

google "hey I want to host a CTF, what

do I do?"

Second thing is hack the box

that is another case study, well case

study. More so a framework to host CTFs

but you can't use it, because it is

actually close sourced. Meaning that

you can only play with that. The last

solution is custom frameworks. So,

these are frameworks used by teams.

They build them themselves, like

at this years CTF.

So, CTFd looks like this. People may have

played CTF may have seen it since most

CTFs are hosted on CTFd. Overall

it is pretty basic, looks bit bootstrappy.

I´ll come back to what the problems are

later. Hack the box, the people who have

not seen it, it looks like this. This is

the machine view. Because hack the

box differentiate between machines

and challenges. Challenges are simply

files from where you need to find the

flag. Machines are a bit more, where

you an actual machine from where

you need to find the flag in the actual

services running on the machine.

So, it is a bit more.

And custom ones. This is an image of

a current CTF organised by HXV.

It is pretty much CTFd but, but built by

their own.

So, what are the problems with this?

Well, lets start with CTFd, where there

aren't actual problems, in my opinion.

It is mostly a static hoster, for files

you want people to use for the CTF and

some custom infrastructure for score

board, registration and stuff like that.

Hack the box is kind of close sourced,

why I say "kind of" because you can

actually use it, you can see how it is

built up, you could build it your self

and the problem we had when playing

with hack the box was that we had some

reverse shells at the root of the

challenges. As well as other problems like

multiple people writing in to some

challenges and that some files where

there, that should not have been. Which

was really annoying sometimes. Like we

started a challenges and saw that there

is a reverse shell for getting root in

root, you don't have to do anything.

There are shared challenge instances

the problem we saw that was you

have multiple hundre people playing the

same instance, where we could see what

other people where uploading to the

instance. Which kind of helped us and

found out that it could be kind of

optimised. The third problem, well

problem, but it is custom frameworks.

You might find errors in custom frameworks

allowing to get flags that aren't used

without solving the challenge. So, it is

now a ping pong between finding a problem

and finding a solution. The simplest

solution we tried to implement at our CTF

at a local hackrrspace was to generate

a single challenge instance for every

player/ team. This means that every

challenge we built was simply a docker

container somewhere and for everyone

who wanted to play it started a new docker

contianer. We first thought that this

would bring a lot of overhead, but it

didn't. We started multiple hundred

containers and it worked out fine. The

problem with this is that if you put

everything in a doker container docker

escapes and sandbox escapes get really

useful. It would be fatal if someone could

breakout of the container. We got

solutions for the possible problems.

You could place everything in a VM or

nsjail in order to isolate the process.

Stopping people from actually breaking

out. Another possible solution would be

to make it possible for people to break

out, which you don't actually want to

make possible. But you don't want people

to have anything in case; custom flags

for custom teams.

We did by implementing our docker

containers as - or we implemented the

challenges or the flags get put into the

docker via environment variables.

So when you are starting your docker

container you just set an environment

variable with you flag. And in the docker

container you have a little scrip that is

pushing your flag to the place you want

it to be. Then unsetting the environment

variable and deleting everything else.

Meaning no trace of the flag, where there

should not be. That worked out pretty

well. So, that is the CIRCUS prototype

that we used.

A little story for that - we had the

18th anniversary of our hackerspace

this year and we thought that we

need a CTF for that.

In a week before we realised that it is

in a week so we quickly started building

a prototype for it.

And called it CIRCUS.

Because it looks like a circus.

That is a graph showing how the

containers interact with each other.

The goal with this was that we wanted a

place where the teams could register

and get a known companion.

A companion in our system was a place

where people could go and spawn

individual contianers.

Because companion spawns in VPN

containers impacts s all other containers

in to that network.

So, people would go and get the VPN

config and can access the challenges.

It is really similar to how hack the box

works. A problem with this was that

we got one companion container per user

or per team. And we got n challenges

that can be spawn. Meaning that we got

n teams with m challenge computers

we end up with a lot of containers.

What you are seeing here is just a listing

of all the containers that we had spawn

after day 1 of the CTF, with 10

participants or so. But we had like

50 containers at that point.

Which was quite a bit.

At the end of the CTF we had about

120 container up and running.

You might think that a lot of contianers

and people doing stuff in the contaienrs

that must cost a lot of computational

power. But it actually worked out.

We had set up a virtual machine

8 core 16 bit of RAM, and it always

looked like nothing at all was

happening. Until someone set up

a cryptominer and had fun with that.

Since, we went on a machine and saw

"Where is this load coming from?"

We identified that this was a container

that some of the team set up.

Not me.

We had some people try with names,

we screwed up the sanitation a bit

because it was all really quick and that

is a learning for everything - that

it doesn't work.

The XSS you are seeing here didn't also

work for the person trying it

- which was kind of weird.

We did set up a super basic scoreboard.

So, as you can see we tried to build a

CTF framework on our own.

And it kind of worked, it was all

built in a few days and very much

like shitty CTFd.

What we want to do now is to find out

what we want to do and what we don't.

What we want to do is to allow

people to spawn containers

with their challenges. So, we solved

the problem of multiple people

acting on one challenge or instance

of a challenge.

By allowing this we don't allow them to

spawn infinite containers.

Maybe some of you have played Alice CTF

or GM CTF.

That was pretty fun because there

was a challenge exciting

devops challenge and it was exactly like

this. You could spawn containers/

a complete set up for you to play in.

But you had to do a proof of work, meaning

to calculate something so that you could

not just spawn challenge instances as much

as you liked.

Another thing you might keep in mind

when doing this is to not mount the

docker socket into everything.

As fun as it is to spawn docker

containers from docker containers, it is

a giant security risk. If people have

access to the docker socket they can

docker containers and do shit.

Dos and don'ts.

A lot of players do execute stuff in

containers. Just having a container with

just static files are fun, but we wanted

to have more. Allowing people to

execute stuff in containers can be

problem, but you can limit what people can

do. Meaning that allow people to do stuff

but don't allow them to do too much.

And that worked out in our case.

As said before, we tried it with like 10

people in our local CTF.

Seeing where the problems get when we

put really good CTF teams on it, and to

see if they can break out would be really

interesting to see.

As I said, don't allow, or allow people

to do stuff, but don't allow them to do

too much stuff.

Implement techniques so that it works

out. One thing that I had to keep in mind

was to keep things simple.

During the CTF I realised that, we built

a lot of stuff and it was a little bit

overcomplicated and made things a little

bit too hard to fix. I would keep in mind

for future CTF frameworks to keep it as

simpel as possible, in case anything

breaks it will be a 5 minute job to fix it

If you where to lazy to lisen, here is a

recap. Create new platforms, CTF

platforms are really interesting.

I found a lot of topics I could work in to

while building this and I am not at the

end yet. There is still a lot of things

that I need to look into. But to allow

for a place to play the game and limit

the bad stuff. For people thinking why

docker, people at our local hackerspace

ask all the time "why are you using docker

since there are so many known exploits for

that?"

Finding alternativs would have been an

option, but I am used to docker - and

I actually wanted to used docker.

So, that was kind of nice.

So if you know a better solution: find the

solution, implement it and try out the

CTF. Another thing I wanted to say here

that while using docker it might be

insecure, but you could also implement a

lot of stuff inorder to secure it.

Like implementing custom flags for teams,

so if a teams has got a custom flag

it can't just break out of the container

and get the flag from another team.

Since it is really team specific.

That was what we wanted to do with the

environment variable in the challenge

containers. Because then we could start

the containers as we went.

That's actually the end.

What I still want to say is that sometimes

next year we want to play the CIRCLE CTF

with the platform we built, just to try it

out, but at a larger scale. So, if you

are an active CTF player we are going to

be there and organise a complete new CTF

with us with fun challenges.

I got some of the challenges with me so,

if you are interested in how this might

look or what can be done then just come

to my table.

Also, if you are interested in discussing

solutions on how this could be done better

then just drop by.

If you got questions, watching the live

stream then just tweet me at: @hanemile.

That was it.

Does anyone got any direct questions.

Thanks again Emile.

Translated by CS (ITKST56 course assignment at JYU.FI)