1
00:00:15,799 --> 00:00:20,360
Herald: So could you give a warm welcome
of applause to Stephan Gronke who will be
2
00:00:20,360 --> 00:00:23,329
talking to you in one minute.
3
00:00:23,799 --> 00:00:31,277
Applause
4
00:00:31,277 --> 00:00:35,290
Stephan: So hi everybody my name is
5
00:00:35,290 --> 00:00:39,420
Stephan Gronke. I'm a software developer
since about 15 years, working in solo
6
00:00:39,420 --> 00:00:44,499
projects and larger teams and smaller
teams. So mostly my development stack was
7
00:00:44,499 --> 00:00:50,260
JavaScript and you will find some of the
tools that I mentioned coming from this
8
00:00:50,260 --> 00:00:55,829
world, but I'm very sure you can also find
something for your project that applies
9
00:00:55,829 --> 00:01:02,249
here. Here's my email address, my PGP key
and my favorite social network account.
10
00:01:02,249 --> 00:01:09,350
Yeah, so a little spoiler what will happen
today: I will talk about development
11
00:01:09,350 --> 00:01:14,060
process exploitation. So that means if you
are developing your software and somebody
12
00:01:14,060 --> 00:01:18,460
joins your team and since you code for
review it could happen that it executes
13
00:01:18,460 --> 00:01:23,560
code on your machine without your
knowledge. There are a few things that are
14
00:01:23,560 --> 00:01:27,969
really hard to catch or I found hard to
catch and I want to share. Maybe you have
15
00:01:27,969 --> 00:01:32,170
the same problems and you find that the
same mitigations apply for your project as
16
00:01:32,170 --> 00:01:40,929
well. I will then continue and... let's
start with a software development process
17
00:01:40,929 --> 00:01:44,799
that's a small cycle. So first of all it
starts with an operating system: you need
18
00:01:44,799 --> 00:01:48,600
to have a computer to write a software and
that's something you need to trust first
19
00:01:48,600 --> 00:01:59,720
off. If you... yes, so your operating
system contains keys and credentials, it
20
00:01:59,720 --> 00:02:03,159
contains the source code you want to
develop and the tools that you have in
21
00:02:03,159 --> 00:02:09,180
place. And the major risk is that tools
are vulnerable to some exploitation or
22
00:02:09,180 --> 00:02:13,330
that your host is already compromised and
you write a software, commit it to your
23
00:02:13,330 --> 00:02:20,530
coworkers and it isn't what you intended
to write. That's a larger problem here.
24
00:02:20,530 --> 00:02:25,611
After you start writing code, the editor
is kind of the interface that I have to
25
00:02:25,611 --> 00:02:33,250
write the files and edit the code. I find
it kind of complex to use an editor. On
26
00:02:33,250 --> 00:02:36,420
the left you can see that many of the
editors come with a package manager
27
00:02:36,420 --> 00:02:41,940
included, which is a good sign for the
complexity that these tools have. And I
28
00:02:41,940 --> 00:02:45,150
don't know what tools you need but they
support you in development so it's very
29
00:02:45,150 --> 00:02:49,490
good to, for example, have code linters
and auto-completion in place to write
30
00:02:49,490 --> 00:02:56,750
better code. At the same time it can be a
problem because they can execute code
31
00:02:56,750 --> 00:03:02,472
unattendedly. We will see in a moment. The
mitigation I came up with for the editor
32
00:03:02,472 --> 00:03:06,680
part is that you have a virtualized
environment where you run your editor. So
33
00:03:06,680 --> 00:03:11,080
when something happens and and it is
compromised, not your root system is also
34
00:03:11,080 --> 00:03:15,880
compromised as well. You want to monitor
all your config files that you have in the
35
00:03:15,880 --> 00:03:20,590
project and you want to get awareness what
exactly happens on my system when I run
36
00:03:20,590 --> 00:03:22,450
and view this code.
37
00:03:24,510 --> 00:03:28,430
The next part you will
probably use as a shell integration. So as
38
00:03:28,430 --> 00:03:33,340
soon as you open your repository some of
the shells I saw just tell you what
39
00:03:33,340 --> 00:03:36,840
branch you're working in and what files
were changed and so on. So that's
40
00:03:36,840 --> 00:03:41,790
something that comes very neat if you're
developing but it can be a risk as well.
41
00:03:43,430 --> 00:03:47,240
So my opinion on the shell integrations is
mostly that it's made for software
42
00:03:47,240 --> 00:03:50,980
development on your own system. So when
you write the code and you can trust it
43
00:03:50,980 --> 00:03:55,760
it's not a problem to use those tools at
all, but as soon as you get sources from
44
00:03:55,760 --> 00:04:05,760
foreign developers it can be a problem. So
choose your tools wisely and don't execute
45
00:04:05,760 --> 00:04:12,290
code from others if possible. The
versioning system that you commit your
46
00:04:12,290 --> 00:04:20,040
code to is also a very good choice. For
example Git and yeah Git can execute hooks
47
00:04:20,040 --> 00:04:24,070
on different occasions. For example when
you check out new code, if you commit and
48
00:04:24,070 --> 00:04:30,040
so on. That means if you managed to clone
a repository and a ".git" folder is
49
00:04:30,040 --> 00:04:34,410
included or a ".hg" folder is included, it
could mean that your operating system
50
00:04:34,410 --> 00:04:39,800
decides to execute whatever is in the
hooks. It's not possible to store a ".git"
51
00:04:39,800 --> 00:04:43,130
folder within a Git repository but it's
possible to store it in a Mercurial
52
00:04:43,130 --> 00:04:48,480
repository or in a SVN or something, and
then your shell integration won't know
53
00:04:48,480 --> 00:04:53,210
what the original source was and will
execute it anyway. One thing that was
54
00:04:53,210 --> 00:04:57,990
introduced for example from Visual Studio
Code, this October, is that they now
55
00:04:57,990 --> 00:05:00,890
support git hooks - which is a great
feature, right? chuckles
56
00:05:01,974 --> 00:05:03,090
The mitigations
57
00:05:03,090 --> 00:05:07,030
against this are pretty easy: you can
either set a different hooks path, which
58
00:05:07,030 --> 00:05:12,790
is not within this project repository so
that you don't execute git hooks at all,
59
00:05:12,790 --> 00:05:16,850
or you can use that little wrapper here
that you see, to for example check at
60
00:05:16,850 --> 00:05:21,210
least that there is no file that is a git
hook within that folder before you execute
61
00:05:21,210 --> 00:05:25,210
git. It's a very good choice if you want
to protect yourself from that
62
00:05:25,210 --> 00:05:27,995
vulnerability.
63
00:05:29,745 --> 00:05:33,913
So after you committed the
code and shared it to the versioning
64
00:05:33,913 --> 00:05:38,410
server, you probably are going to build it
automatically. So some services like
65
00:05:38,410 --> 00:05:44,230
Travis CI will run it, will run it for
you. So they will run tests, they will
66
00:05:44,230 --> 00:05:49,090
compile the software and also they do the
package versioning and deployment to some
67
00:05:49,090 --> 00:05:53,880
other places. It becomes a problem if you
can't reproduce the results from your
68
00:05:53,880 --> 00:05:59,740
built runner, because it's an system you
don't control sometimes. And as soon as
69
00:05:59,740 --> 00:06:04,400
you get the binary result from it - if you
compile the software that compiles to
70
00:06:04,400 --> 00:06:09,030
binary - you need to check that result
somehow, because somebody could have
71
00:06:09,030 --> 00:06:15,230
altered it without your knowledge and then
you will ship it to your users. Also a
72
00:06:15,230 --> 00:06:19,280
problem on many of this build workers is:
you want to have this process very fast.
73
00:06:19,280 --> 00:06:23,130
So that means you don't want to wait until
all the dependencies are installed and the
74
00:06:23,130 --> 00:06:28,290
great service is that you have caching in
between these projects. This means that,
75
00:06:28,290 --> 00:06:33,030
for example, if somebody managed to inject
the version to the cache of some CI
76
00:06:33,030 --> 00:06:37,919
system, then it will eventually show up in
other projects as well and you can pivot
77
00:06:37,919 --> 00:06:43,940
across the projects. Usually if you have a
build environment it has access to some
78
00:06:43,940 --> 00:06:48,840
kind of development key. Mostly if you get
pull requests from externals, the keys are
79
00:06:48,840 --> 00:06:52,450
stored encrypted and you don't have access
to them, but as soon as somebody has write
80
00:06:52,450 --> 00:06:58,110
access to your repository also the keys
could be leaked. Let's make an example:
81
00:06:58,110 --> 00:07:01,520
you have somebody offering you a software
and you don't give permission to edit the
82
00:07:01,520 --> 00:07:05,930
master branch of the repository but as
soon as you open an open a branch anywhere
83
00:07:05,930 --> 00:07:10,530
and make a pull request Travis CI or other
build runners will use that and decrypt
84
00:07:10,530 --> 00:07:14,110
the passwords for you and give you access
to the credentials which you can then
85
00:07:14,110 --> 00:07:16,942
print or do whatever you intend to.
86
00:07:18,235 --> 00:07:21,500
Yeah,
and for me the best option here would be
87
00:07:21,500 --> 00:07:26,420
to have reproducible builds because then
you can use different of the build workers
88
00:07:26,420 --> 00:07:30,300
and compare results somehow so that you
see if one gets compromised the other two
89
00:07:30,300 --> 00:07:33,519
will tell you: hey, there's a different
result, have a look please.
90
00:07:34,220 --> 00:07:35,800
That would be great.
91
00:07:36,250 --> 00:07:40,040
And also the build steps; I
mentioned building, testing and packaging
92
00:07:40,040 --> 00:07:43,940
the software are totally different steps
so what you can do is you can have one
93
00:07:43,940 --> 00:07:48,930
compartment per step so that you can have
a data at a finer level and see what
94
00:07:48,930 --> 00:07:55,669
happens here. After you compiled the
software, you built the software, you need
95
00:07:55,669 --> 00:07:59,840
to ship it to the user somehow so either
you store it in your own server or most
96
00:07:59,840 --> 00:08:04,740
often you use a CDN. You just put it there
and it's the asset that's lying around.
97
00:08:04,740 --> 00:08:09,460
Your users will come around, download it
from here and execute it, so what is the
98
00:08:09,460 --> 00:08:13,669
problem here? The problem is, that if you
have an URL it's very hard to prove that
99
00:08:13,669 --> 00:08:19,110
it's actually from the real maintainer. If
you call your software like if - if you
100
00:08:19,110 --> 00:08:23,260
call your account like a different
project, then people won't be able to
101
00:08:23,260 --> 00:08:28,320
notice the difference somehow. What you
can do to mitigate this, is to publish the
102
00:08:28,320 --> 00:08:34,319
URLs that you legitly using and also sign
your assets so that users can check is
103
00:08:34,319 --> 00:08:38,370
that something that the developer intended
to give me or is it something that is
104
00:08:38,370 --> 00:08:42,740
really ... that is really not intended.
105
00:08:43,204 --> 00:08:44,986
So, ...
106
00:08:46,176 --> 00:08:49,359
Yes. And the next part is you need to
reach out to your users
107
00:08:49,359 --> 00:08:52,139
so you make people aware,
that there is a project they can
108
00:08:52,139 --> 00:08:57,160
check out, they can clone and usually you
have the package registries. A few slides
109
00:08:57,160 --> 00:09:03,899
back you saw that the package managers are
also included in the editors, so that's
110
00:09:03,899 --> 00:09:08,290
also something where you can ship the
software but the package manager I was
111
00:09:08,290 --> 00:09:12,790
mostly looking at was for example NPM.
There was an interesting occasion where
112
00:09:12,790 --> 00:09:19,139
somebody had a project called Kik.The
company Kik then tried to take it down and
113
00:09:19,139 --> 00:09:23,259
the person just ignored it for the moment
but then Kik reached out to NPM directly
114
00:09:23,259 --> 00:09:27,860
and they deleted the repository. In
consequence the developer removed all his
115
00:09:27,860 --> 00:09:31,920
projects from the versioning server and a
few hours later malware showed up with the
116
00:09:31,920 --> 00:09:35,779
same project names, so that means if you
have a software that uses that
117
00:09:35,779 --> 00:09:40,649
dependencies and somebody freed up names
it would affect your repository as well
118
00:09:40,649 --> 00:09:46,670
and compromise it. That's something that
needs mitigation. I think the best idea
119
00:09:46,670 --> 00:09:51,480
here is to not only identify the
project by a unique identifier but also
120
00:09:51,480 --> 00:09:58,319
have a GUID or an or a unique identifier
per project that does not change, so that
121
00:09:58,319 --> 00:10:01,700
you can make a difference. That's
something that's up to the package
122
00:10:01,700 --> 00:10:05,310
registries to implement. That's not
something we can do as a user but it's a
123
00:10:05,310 --> 00:10:11,370
very common case - it's a very common case
that these packages fluctuate.
124
00:10:11,370 --> 00:10:14,750
So for example, if somebody deletes it,
you don't have a backup of that.
125
00:10:14,750 --> 00:10:18,495
A very good idea is also to store offline
backups of every package that you
126
00:10:18,495 --> 00:10:22,422
check out and that you install to
your software because it's very bad
127
00:10:22,422 --> 00:10:26,597
if you want to maintain your software and
you figure out there's something, there's
128
00:10:26,597 --> 00:10:29,372
something missing and I can't recover
because it's deleted.
129
00:10:33,165 --> 00:10:38,110
Yes, software developers have some
needs during their work.
130
00:10:38,110 --> 00:10:42,730
I want my tooling to perform if my
code editor for example is in the VM and
131
00:10:42,730 --> 00:10:49,139
the VM is slow, that's something that's
annoying all over the process. So then on
132
00:10:49,139 --> 00:10:53,110
the other hand the velocity is something
that your manager will require from you if
133
00:10:53,110 --> 00:10:59,089
you write commercial software or you try
to get something done and you can't spend
134
00:10:59,089 --> 00:11:03,019
all day to work on chores and improve
the repository, the versioning and so on.
135
00:11:03,019 --> 00:11:09,579
So that's something you need to deal with.
Another big factor for me is the
136
00:11:09,579 --> 00:11:13,480
reliability. So as soon as your software
goes down and you are in holiday or
137
00:11:13,480 --> 00:11:18,240
something, anybody else from the company
or from your team should be able to
138
00:11:18,240 --> 00:11:26,329
recover what was there before, also known
as the bus-factor and, yeah, if you have
139
00:11:26,329 --> 00:11:30,779
convenience like for example Ruby on Rails
gives you. It gives you a very good,
140
00:11:30,779 --> 00:11:34,510
very easy start in the projects and that's
something you don't want to break by
141
00:11:34,510 --> 00:11:38,839
making it too complicated with a
development environment. And also
142
00:11:38,839 --> 00:11:45,490
something I've found to be more annoying
than helpful is, if you want to pair-
143
00:11:45,490 --> 00:11:49,579
program and you have a very
compartmentalized environment, it's very
144
00:11:49,579 --> 00:11:53,269
hard to share the resources that you need
to talk about with other developers,
145
00:11:53,269 --> 00:11:56,699
expecting you're not in the same room but
working remotely, what is for me most
146
00:11:56,699 --> 00:11:58,479
often the case.
147
00:12:02,110 --> 00:12:06,601
A large problem that I saw is, if you
underhand somebody code,
148
00:12:06,601 --> 00:12:13,030
if you go ahead and and check out
code from any online resources, it's
149
00:12:13,030 --> 00:12:17,372
sometimes very hard to tell if the code
that you see in your, e.g. Git diff,
150
00:12:17,372 --> 00:12:22,689
is what you really would expect to
see. I have some examples here, which can
151
00:12:22,689 --> 00:12:25,959
show how this could work and how this
could look like if somebody tries
152
00:12:25,959 --> 00:12:30,459
to inject code to your repository, that
you don't see. First of all, let's start
153
00:12:30,459 --> 00:12:37,529
with something easy, that's phishing. What
you see here on the slide, on the left
154
00:12:37,529 --> 00:12:42,999
side maybe you see the cursor. That's not
a full path, that is just a domain name.
155
00:12:42,999 --> 00:12:49,680
The slashes in here are UTF-8 characters
so that thing here resolves to a hostname,
156
00:12:49,680 --> 00:12:54,089
and if you control this host, you can get
a certificate for it and then the example
157
00:12:54,089 --> 00:12:59,959
below you see, how it would look like, if
you install it. First I have a host that's
158
00:12:59,959 --> 00:13:04,480
just running a web server on port 80, so
that you can see the result. Okay, I was
159
00:13:04,480 --> 00:13:09,120
cheating a little bit. I was putting the
domain in the /etc/hosts so that I don't
160
00:13:09,120 --> 00:13:15,519
have to buy it, for just showing it. It's
strange that dot zip is a domain actually,
161
00:13:15,519 --> 00:13:21,599
but then if you install it, you would see
that you can send somebody a very
162
00:13:21,599 --> 00:13:25,140
nice-looking link which looks like a
totally different project, but it's
163
00:13:25,140 --> 00:13:30,649
pointing to your server instead; and I
found many of the package managers having
164
00:13:30,649 --> 00:13:34,899
the nice feature of executing PostScript
hooks, so that means, if you have
165
00:13:34,899 --> 00:13:38,574
installed it, it will run some commands
afterwards for you.
166
00:13:42,105 --> 00:13:46,499
Then there is invisible code. If you go
online somewhere, find in a forum or in a
167
00:13:46,499 --> 00:13:50,760
blog, you find an article and see, "hey,
that code is actually solving my problem",
168
00:13:50,760 --> 00:13:54,509
you go ahead and copy/paste it. So, on the
left you see the source code how this
169
00:13:54,509 --> 00:14:00,290
would look like in HTML for the blog. On
the right there's the result. So you can
170
00:14:00,290 --> 00:14:03,980
go ahead, you can copy/paste from it and
if you paste it to a text area, you will
171
00:14:03,980 --> 00:14:07,759
see, that the result is something that you
didn't expect. For example, if you copy a
172
00:14:07,759 --> 00:14:11,509
large chunk of code, you won't go ahead
and review it on your local system again
173
00:14:11,509 --> 00:14:15,649
and that could be the compromise for your
project.
174
00:14:26,869 --> 00:14:31,141
Another example here is, you can
use ASCII characters,
175
00:14:31,141 --> 00:14:34,957
the control characters to influence
the output in your terminal.
176
00:14:34,957 --> 00:14:38,379
So if your terminal also
supports the legacy of ASCII control
177
00:14:38,379 --> 00:14:42,540
characters, you can use that to just
revert the line and override it with
178
00:14:42,540 --> 00:14:47,850
something you wouldn't expect. What you
see on top here, that harmless script is
179
00:14:47,850 --> 00:14:52,059
the file. It's a little larger than you
would expect for just a echo foo, but not
180
00:14:52,059 --> 00:14:57,009
something you would notice when you just
see it. Looking at it from a hex editor
181
00:14:57,009 --> 00:15:01,060
you can see, that there is something more
going on than just the foo and if you
182
00:15:01,060 --> 00:15:05,509
actually execute it, it will not print
something, it will create the pwned text,
183
00:15:05,509 --> 00:15:11,498
which is a good example for you that your
host was compromised in this moment.
184
00:15:15,049 --> 00:15:20,089
Another example I found online, so credit
to Ariel for this, so there is a byte
185
00:15:20,089 --> 00:15:23,100
sequence you can use so that this even
works in a Git diff. So when you're
186
00:15:23,100 --> 00:15:27,110
working exclusively in your terminal and
you're not doing reviews on GitHub or some
187
00:15:27,110 --> 00:15:32,710
graphical tool, it could be the case that
you don't notice what was going on. What
188
00:15:32,710 --> 00:15:37,529
you can see here on the left is, I created
an empty repository, I added a small
189
00:15:37,529 --> 00:15:43,129
script and in the next step down here, I
added some improvement to the script,
190
00:15:43,129 --> 00:15:50,569
which is actually the malicious commit
that's here in red. Afterwards, I just ran
191
00:15:50,569 --> 00:15:55,690
a Git diff on the code and I see that
there is only no backdoor, oh sorry that
192
00:15:55,690 --> 00:16:00,809
should be okay in the updated slides. So,
you don't see the evil.sh that it's
193
00:16:00,809 --> 00:16:05,559
executed as well, if you run it. That's
something I consider very dangerous.
194
00:16:10,893 --> 00:16:16,179
So, some mitigations: the best thing you
can do, is to make it expensive for your
195
00:16:16,179 --> 00:16:21,089
attackers to compromise or try to. So
as soon as you have the chance to notice
196
00:16:21,089 --> 00:16:27,009
what is going on, also retrospectively,
you can at least burn the capabilities and
197
00:16:27,009 --> 00:16:31,889
tell others how your project was attempted
to compromise and that's something
198
00:16:31,889 --> 00:16:35,499
that is, in my opinion, the best
mitigation against this complexity.
199
00:16:36,919 --> 00:16:41,750
What you can also do, is you can test your
software from external services directly,
200
00:16:41,750 --> 00:16:47,509
which will tell you, if some compromise
happened. For example, Git has it newly
201
00:16:47,509 --> 00:16:52,399
introduced. They will check your packages,
the dependencies and will warn you about
202
00:16:52,399 --> 00:16:59,499
some vulnerabilities that are commonly
known. The best thing you can do on your
203
00:16:59,499 --> 00:17:02,470
local system is to build small
compartments, so that if some compromise
204
00:17:02,470 --> 00:17:08,709
happens, it doesn't affect your full host.
Also not all your projects you have
205
00:17:08,709 --> 00:17:12,360
access to. And it's very important, that
you have backups on a different system
206
00:17:12,360 --> 00:17:16,270
than the hosts you're working on. So if
the compromise happens you still have
207
00:17:16,270 --> 00:17:20,849
access to the original data and can
compare it and do some forensics on this.
208
00:17:23,589 --> 00:17:27,640
So, the intrusion detection forensics;
there are some great tools available.
209
00:17:27,640 --> 00:17:32,600
For example my favorites are DTrace and
Opensnoop. You can monitor changes and
210
00:17:32,600 --> 00:17:37,101
access on the file system or on your
system at all. And you can e.g. set some
211
00:17:37,101 --> 00:17:40,120
rules for your projects that are
specifically matching. So I am
212
00:17:40,120 --> 00:17:44,359
not going to share some rules that match
for all projects, but you will figure out
213
00:17:44,359 --> 00:17:48,779
what is e.g. important. Very
good start is e.g. to Opensnoop for
214
00:17:48,779 --> 00:17:54,029
/etc/passwd if there was some access, then
you can e.g. say that's not something
215
00:17:54,029 --> 00:18:00,770
what my software would do. And again it's
very important to have the backups of this
216
00:18:00,770 --> 00:18:04,530
because in the moment where you execute
it, you can't trust your host at all.
217
00:18:06,894 --> 00:18:11,919
The idea how to achieve this is,
if you have a VM per project for example,
218
00:18:11,919 --> 00:18:15,610
you let it run for half a year,
you don't approve the situation. Instead
219
00:18:15,610 --> 00:18:19,309
of having one system that you need to
update the software to, you need to update
220
00:18:19,309 --> 00:18:23,950
afterwards all the projects that you're
working on frequently and that's something
221
00:18:23,950 --> 00:18:28,360
that's easy to forget. So it's dangerous.
If you assume that every time you run some
222
00:18:28,360 --> 00:18:31,940
command or every time you work in a
project, you spin up a new server entirely
223
00:18:31,940 --> 00:18:36,090
from scratch, install the dependencies and
so on that's something that's not a risk
224
00:18:36,090 --> 00:18:43,240
for you. Also, if you have for example a
virtualized server environment you can
225
00:18:43,240 --> 00:18:48,380
have memory dumps at any time, you can
monitor the network and you can also diff
226
00:18:48,380 --> 00:18:51,770
the filesystem. For example, if you stop
the server and just compare it to a
227
00:18:51,770 --> 00:18:55,540
previous version and see, "hey, here is
something that was changed that I didn't
228
00:18:55,540 --> 00:18:58,970
plan". It's great to know.
229
00:19:01,960 --> 00:19:05,800
Very important is also to separate your
accounts. E.g., if you see large GitHub
230
00:19:05,800 --> 00:19:09,317
accounts, people are making contributions
every day since years.
231
00:19:11,759 --> 00:19:17,410
So it shows that the people
have access to very, to many projects from
232
00:19:17,410 --> 00:19:22,250
the same machine. And the permission model
from GitHub for example, allows you to
233
00:19:22,250 --> 00:19:26,669
store an SSH key for write access. But it
automatically has access to all the
234
00:19:26,669 --> 00:19:30,940
repositories you control. So the best that
you can have here, is to make a
235
00:19:30,940 --> 00:19:35,459
new GitHub account for, ... or to make a
new account on that versioning system
236
00:19:35,459 --> 00:19:39,020
that only has exclusively write access
to that single repository.
237
00:19:39,740 --> 00:19:43,580
So when you work in your
compartmented system and you want to
238
00:19:43,580 --> 00:19:50,260
upload or pull changes, you can't
influence other repositories. That means
239
00:19:50,260 --> 00:19:54,580
compromise doesn't spread across all your
projects and so on, which would be an
240
00:19:54,580 --> 00:20:00,330
invitation for malware somehow or
ransomware. And you get a better
241
00:20:00,330 --> 00:20:06,049
permission model if you create a GitHub
organization. In this case you can also
242
00:20:06,049 --> 00:20:11,649
limit your own access in a better way. So
my recommendation is not to work on your
243
00:20:11,649 --> 00:20:15,119
personal GitHub account but create an
organization for your project.
244
00:20:18,809 --> 00:20:26,100
Something many projects are missing are to
find responsible persons for security
245
00:20:26,100 --> 00:20:33,310
and to clearly communicate what is the
plan for incident response. Small example:
246
00:20:33,310 --> 00:20:36,770
If you have a new project and you
find a vulnerability, you would like to
247
00:20:36,770 --> 00:20:40,571
commit it, but you don't open an issue
publicly, because then everybody, every
248
00:20:40,571 --> 00:20:44,490
user would be affected. You try to reach
out to some developers and if you don't
249
00:20:44,490 --> 00:20:52,570
have any clue how to securely achieve
this, that can get you into trouble. And
250
00:20:52,570 --> 00:20:58,080
there are quite a few projects which don't
communicate this and some of them don't
251
00:20:58,080 --> 00:21:01,955
even respond to their security@ email
address, which is bad.
252
00:21:05,660 --> 00:21:12,210
In this case I told you what I saw from
my experiences of working on the projects.
253
00:21:12,210 --> 00:21:20,699
So that's basically my summary of
what can be harmful
254
00:21:20,699 --> 00:21:23,210
and what can be good for your project.
255
00:21:23,220 --> 00:21:33,089
Applause
256
00:21:33,089 --> 00:21:38,179
Herald: Thank you and we now have time for
Q&A. In the room you can line up behind
257
00:21:38,179 --> 00:21:43,070
the microphones and I can see we have a
question from the internet already.
258
00:21:44,410 --> 00:21:50,029
Signal Angel: What about Git signed
commits? Any thoughts on that?
259
00:21:50,029 --> 00:21:53,620
Answer: So as soon as you have signed
commits and I find that you also email
260
00:21:53,620 --> 00:21:57,073
with the same PGP key, it's very
interesting that you have the PGP key on
261
00:21:57,073 --> 00:22:01,329
the same host probably, then you have your
Git executable. So if somebody executes
262
00:22:01,329 --> 00:22:06,520
Git hooks, they can steal your PGP keys
from this. I didn't find any tutorial
263
00:22:06,520 --> 00:22:11,590
online which explains you, how to make it
manually, so that you don't use the Git
264
00:22:11,590 --> 00:22:17,049
for signing the commits. But I think it
can be very good to sign the commits,
265
00:22:17,049 --> 00:22:23,210
but it can be also dangerous, because your
email communication can be compromised.
266
00:22:23,210 --> 00:22:28,980
Herald: Microphone number four.
Q: In the Git diff you showed us, there
267
00:22:28,980 --> 00:22:36,874
were some control characters. I think
Git diff pipes to less by default, so
268
00:22:36,874 --> 00:22:41,360
shouldn't they appear there somewhere?
A: No, they don't. I just checked with the
269
00:22:41,360 --> 00:22:47,290
latest version today. So, that's something
that, well, we can also click on the blog
270
00:22:47,290 --> 00:22:49,814
and see, if there is the video available.
271
00:23:04,734 --> 00:23:08,449
Yeah, it's very hard to show from my
HTML slide how this works.
272
00:23:08,449 --> 00:23:11,337
So this video animation, ...
maybe we can enlarge it a bit.
273
00:23:12,777 --> 00:23:14,226
That's how it would work.
274
00:23:30,260 --> 00:23:34,523
So most often, yes, if you pipe to less or
you use a hex editor to review,
275
00:23:34,523 --> 00:23:36,410
then you would notice, yes.
276
00:23:38,270 --> 00:23:44,299
Q: I somehow remember that, maybe it
only shows for longer diffs, but I think
277
00:23:44,299 --> 00:23:46,740
when I type "git diff", I can scroll
around.
278
00:23:46,740 --> 00:23:53,000
A: Ah, that's interesting. I need to try.
Herald: We have a question from microphone
279
00:23:53,000 --> 00:23:58,539
number one.
Q: You mentioned Travis having access to
280
00:23:58,539 --> 00:24:07,470
hidden variables and you being able to
leak those variables during pull requests.
281
00:24:07,470 --> 00:24:10,409
What are your suggestions to mitigate
that?
282
00:24:10,409 --> 00:24:15,220
A: Don't give people write access to your
repository, not even to branches that you
283
00:24:15,220 --> 00:24:18,710
don't trust. So, as soon as they have
write access, they would also know the
284
00:24:18,710 --> 00:24:23,409
secrets behind the variables in this case.
I like the security model, because if you
285
00:24:23,409 --> 00:24:26,779
for example get contributions from
outside, nobody can trigger that and steal
286
00:24:26,779 --> 00:24:29,250
your keys.
But as soon as you build it on your own
287
00:24:29,250 --> 00:24:32,440
branch somewhere in the repository that
changes.
288
00:24:32,440 --> 00:24:38,940
Q: Yes, but if you submit a pull request,
you don't necessarily have to have write
289
00:24:38,940 --> 00:24:43,460
access to that repository.
A: Yes, that's what I mean. If you come
290
00:24:43,460 --> 00:24:48,240
from outside and it's not within the same
repository, the secrets are not decrypted.
291
00:24:48,240 --> 00:24:52,350
So you can't run the steps. For example,
you would not like to deploy directly from
292
00:24:52,350 --> 00:24:57,759
a foreign branch, somewhere.
Herald: We have a question from microphone
293
00:24:57,759 --> 00:25:02,700
number four.
Q: You mentioned the problem with
294
00:25:02,700 --> 00:25:07,409
different compartments and how to exchange
those environments without people. I think
295
00:25:07,409 --> 00:25:10,980
that problem has already been solved with
Vagrant and some kind of provisioning
296
00:25:10,980 --> 00:25:14,899
software like Ansible.
Do you have any experience with checking
297
00:25:14,899 --> 00:25:20,480
those results of those Vagrant boxes that
are automatically provisioned, like
298
00:25:20,480 --> 00:25:25,830
having some server spec software to check
those environments afterwards, or having
299
00:25:25,830 --> 00:25:31,269
some kind of hashing, how to find out, if
they have been reproduced the same way and
300
00:25:31,269 --> 00:25:37,039
or if they have been any exploit used
in that process of setting up the Vagrant
301
00:25:37,039 --> 00:25:40,250
environments.
A: Yes, so different levels you can look
302
00:25:40,250 --> 00:25:46,150
at this. There was some, ..., I try to
find it, yes. You can for example memory
303
00:25:46,150 --> 00:25:50,120
dump at any time, if you have the hosts
trying somewhere or was your question
304
00:25:50,120 --> 00:25:55,150
exactly that you want to check, if your
environment that was spun up was not
305
00:25:55,150 --> 00:25:58,870
compromised yet?
Q: Yeah, there has to be some kind of
306
00:25:58,870 --> 00:26:05,830
process, how to verify that the produced
environments are the ones you expect them
307
00:26:05,830 --> 00:26:10,990
to be, or if they have been compromised
and the problem is, I have used those
308
00:26:10,990 --> 00:26:16,610
environments and tried, first I tried the
full disk encryption for the Vagrant boxes
309
00:26:16,610 --> 00:26:26,960
but the problem is, it's always the same
key for the encryption, so that doesn't
310
00:26:26,960 --> 00:26:30,530
work and even as you mentioned, you can
have a memory dump, so you can read out
311
00:26:30,530 --> 00:26:37,110
that key so there's no real possibility to
set up a Vagrant box that can't be
312
00:26:37,110 --> 00:26:46,350
tampered with afterwards. So there has to
be some kind of hash sum to compare those
313
00:26:46,350 --> 00:26:49,151
produced results.
A: Yeah, so as soon as you have a
314
00:26:49,151 --> 00:26:53,350
reproducible build and the result that
you, for example script languages are much
315
00:26:53,350 --> 00:26:57,650
easier to achieve, because then you can
just diff the filesystem directory and
316
00:26:57,650 --> 00:27:02,030
see, if there was some change. What I
would do in this case, is to run multiple
317
00:27:02,030 --> 00:27:05,559
services and compare the results, if
that's possible. For example, you have
318
00:27:05,559 --> 00:27:10,630
these reproducible builds, then run it on
a few servers which are independent and
319
00:27:10,630 --> 00:27:14,500
compare what you have.
Herald: We have two more questions from
320
00:27:14,500 --> 00:27:17,960
microphone number one and only a few
minutes left. Microphone number one.
321
00:27:17,960 --> 00:27:22,770
Q: So, what's your recommendation for
handling credentials in application
322
00:27:22,770 --> 00:27:28,789
configuration files? We need often some
database user and password or something
323
00:27:28,789 --> 00:27:33,840
like this in, say Spring Boot Application
YML, or things like that?
324
00:27:33,840 --> 00:27:40,640
And is there any best practice or any
framework which can handle such things or
325
00:27:40,640 --> 00:27:46,919
we need to explicitly encrypt these
credentials in this application and then
326
00:27:46,919 --> 00:27:51,710
decrypt for itself in the application, but
then you need symmetric keys, or?
327
00:27:51,710 --> 00:27:56,250
A: Yes, so Ansible for example comes with
a mechanism that's called Ansible Vault,
328
00:27:56,250 --> 00:28:00,000
which encrypts that with a passphrase that
you can enter in your command line as soon
329
00:28:00,000 --> 00:28:03,990
as you touch the file. For example, if you
want to run Ansible then, it will ask you
330
00:28:03,990 --> 00:28:07,679
for the password when starting up. So, if
you want to share that password with your
331
00:28:07,679 --> 00:28:11,220
developers everybody has access to the
same keys, I would prefer to give
332
00:28:11,220 --> 00:28:16,000
everybody, so every person in this team or
even every device a different key, if
333
00:28:16,000 --> 00:28:21,100
that's possible somehow.
That's what I was trying to mention with
334
00:28:21,100 --> 00:28:24,830
the GitHub accounts; that you don't use
one GitHub account but you use many of
335
00:28:24,830 --> 00:28:27,049
them. If you, ... yeah.
336
00:28:28,099 --> 00:28:31,860
Herald: We have one more question from
microphone number one and then a question
337
00:28:31,860 --> 00:28:35,090
from the internet.
Q: Yeah, my question was more about, I
338
00:28:35,090 --> 00:28:38,919
mean some of your recommendations are low-
hanging fruits, but some of them it's
339
00:28:38,919 --> 00:28:42,850
like, it's just impossible. I mean it's
not sustainable, like it's very hard to
340
00:28:42,850 --> 00:28:46,960
maintain and so I'm wondering, if you use
all of them every day or just part of them
341
00:28:46,960 --> 00:28:50,190
or do you just leave like an ??? at the
end?
342
00:28:50,190 --> 00:28:54,399
A: It depends on the project. So what I
try to do on my development system is to
343
00:28:54,399 --> 00:28:59,280
have these compartment, so that one
compromised project would not affect
344
00:28:59,280 --> 00:29:04,289
others. Because I'm not the only person
checking and merging the code, so and
345
00:29:04,289 --> 00:29:08,820
that's something that gets quickly too
much for one person to review. So, I
346
00:29:08,820 --> 00:29:13,150
can't review all the code that I'm running
currently on my computer, that's true. But
347
00:29:13,150 --> 00:29:16,540
I can try to mitigate what the impact of
this will be.
348
00:29:16,890 --> 00:29:20,617
Herald: And the question from the internet.
Signal: What tool would you recommend
349
00:29:20,617 --> 00:29:31,850
for diffing a file system?
A: Diff. giggles Well, it worked for me
350
00:29:31,850 --> 00:29:38,620
so far. Or what exactly is the question
about? Um, maybe you want to see, if did
351
00:29:38,620 --> 00:29:42,179
the hash change in the files? So when you
have e.g. the script file one and
352
00:29:42,179 --> 00:29:45,669
the script file B and they have a
different hash sum, that's something I
353
00:29:45,669 --> 00:29:49,760
would consider something I would look up
manually. So as soon as I have an
354
00:29:49,760 --> 00:29:53,440
indication that there was something wrong,
I would look it up manually and use any
355
00:29:53,440 --> 00:29:57,059
tool that I have. Hex editor or whatever
is available.
356
00:29:57,059 --> 00:30:01,200
Herald: Good. We have less than one minute
left. Are there any final remarks?
357
00:30:01,200 --> 00:30:03,620
Stephan: Thank you.
Herald: Thank you very much.
358
00:30:03,620 --> 00:30:05,927
Applause
359
00:30:10,147 --> 00:30:15,860
postroll music
360
00:30:15,860 --> 00:30:28,000
subtitles created by c3subtitles.de
in the year 2019. Join, and help us!