Return to Video

34C3 - Hardening Open Source Development

  • 0:16 - 0:20
    Herald: So could you give a warm welcome
    of applause to Stephan Gronke who will be
  • 0:20 - 0:23
    talking to you in one minute.
  • 0:24 - 0:31
    Applause
  • 0:31 - 0:35
    Stephan: So hi everybody my name is
  • 0:35 - 0:39
    Stephan Gronke. I'm a software developer
    since about 15 years, working in solo
  • 0:39 - 0:44
    projects and larger teams and smaller
    teams. So mostly my development stack was
  • 0:44 - 0:50
    JavaScript and you will find some of the
    tools that I mentioned coming from this
  • 0:50 - 0:56
    world, but I'm very sure you can also find
    something for your project that applies
  • 0:56 - 1:02
    here. Here's my email address, my PGP key
    and my favorite social network account.
  • 1:02 - 1:09
    Yeah, so a little spoiler what will happen
    today: I will talk about development
  • 1:09 - 1:14
    process exploitation. So that means if you
    are developing your software and somebody
  • 1:14 - 1:18
    joins your team and since you code for
    review it could happen that it executes
  • 1:18 - 1:24
    code on your machine without your
    knowledge. There are a few things that are
  • 1:24 - 1:28
    really hard to catch or I found hard to
    catch and I want to share. Maybe you have
  • 1:28 - 1:32
    the same problems and you find that the
    same mitigations apply for your project as
  • 1:32 - 1:41
    well. I will then continue and... let's
    start with a software development process
  • 1:41 - 1:45
    that's a small cycle. So first of all it
    starts with an operating system: you need
  • 1:45 - 1:49
    to have a computer to write a software and
    that's something you need to trust first
  • 1:49 - 2:00
    off. If you... yes, so your operating
    system contains keys and credentials, it
  • 2:00 - 2:03
    contains the source code you want to
    develop and the tools that you have in
  • 2:03 - 2:09
    place. And the major risk is that tools
    are vulnerable to some exploitation or
  • 2:09 - 2:13
    that your host is already compromised and
    you write a software, commit it to your
  • 2:13 - 2:21
    coworkers and it isn't what you intended
    to write. That's a larger problem here.
  • 2:21 - 2:26
    After you start writing code, the editor
    is kind of the interface that I have to
  • 2:26 - 2:33
    write the files and edit the code. I find
    it kind of complex to use an editor. On
  • 2:33 - 2:36
    the left you can see that many of the
    editors come with a package manager
  • 2:36 - 2:42
    included, which is a good sign for the
    complexity that these tools have. And I
  • 2:42 - 2:45
    don't know what tools you need but they
    support you in development so it's very
  • 2:45 - 2:49
    good to, for example, have code linters
    and auto-completion in place to write
  • 2:49 - 2:57
    better code. At the same time it can be a
    problem because they can execute code
  • 2:57 - 3:02
    unattendedly. We will see in a moment. The
    mitigation I came up with for the editor
  • 3:02 - 3:07
    part is that you have a virtualized
    environment where you run your editor. So
  • 3:07 - 3:11
    when something happens and and it is
    compromised, not your root system is also
  • 3:11 - 3:16
    compromised as well. You want to monitor
    all your config files that you have in the
  • 3:16 - 3:21
    project and you want to get awareness what
    exactly happens on my system when I run
  • 3:21 - 3:22
    and view this code.
  • 3:25 - 3:28
    The next part you will
    probably use as a shell integration. So as
  • 3:28 - 3:33
    soon as you open your repository some of
    the shells I saw just tell you what
  • 3:33 - 3:37
    branch you're working in and what files
    were changed and so on. So that's
  • 3:37 - 3:42
    something that comes very neat if you're
    developing but it can be a risk as well.
  • 3:43 - 3:47
    So my opinion on the shell integrations is
    mostly that it's made for software
  • 3:47 - 3:51
    development on your own system. So when
    you write the code and you can trust it
  • 3:51 - 3:56
    it's not a problem to use those tools at
    all, but as soon as you get sources from
  • 3:56 - 4:06
    foreign developers it can be a problem. So
    choose your tools wisely and don't execute
  • 4:06 - 4:12
    code from others if possible. The
    versioning system that you commit your
  • 4:12 - 4:20
    code to is also a very good choice. For
    example Git and yeah Git can execute hooks
  • 4:20 - 4:24
    on different occasions. For example when
    you check out new code, if you commit and
  • 4:24 - 4:30
    so on. That means if you managed to clone
    a repository and a ".git" folder is
  • 4:30 - 4:34
    included or a ".hg" folder is included, it
    could mean that your operating system
  • 4:34 - 4:40
    decides to execute whatever is in the
    hooks. It's not possible to store a ".git"
  • 4:40 - 4:43
    folder within a Git repository but it's
    possible to store it in a Mercurial
  • 4:43 - 4:48
    repository or in a SVN or something, and
    then your shell integration won't know
  • 4:48 - 4:53
    what the original source was and will
    execute it anyway. One thing that was
  • 4:53 - 4:58
    introduced for example from Visual Studio
    Code, this October, is that they now
  • 4:58 - 5:01
    support git hooks - which is a great
    feature, right? chuckles
  • 5:02 - 5:03
    The mitigations
  • 5:03 - 5:07
    against this are pretty easy: you can
    either set a different hooks path, which
  • 5:07 - 5:13
    is not within this project repository so
    that you don't execute git hooks at all,
  • 5:13 - 5:17
    or you can use that little wrapper here
    that you see, to for example check at
  • 5:17 - 5:21
    least that there is no file that is a git
    hook within that folder before you execute
  • 5:21 - 5:25
    git. It's a very good choice if you want
    to protect yourself from that
  • 5:25 - 5:28
    vulnerability.
  • 5:30 - 5:34
    So after you committed the
    code and shared it to the versioning
  • 5:34 - 5:38
    server, you probably are going to build it
    automatically. So some services like
  • 5:38 - 5:44
    Travis CI will run it, will run it for
    you. So they will run tests, they will
  • 5:44 - 5:49
    compile the software and also they do the
    package versioning and deployment to some
  • 5:49 - 5:54
    other places. It becomes a problem if you
    can't reproduce the results from your
  • 5:54 - 6:00
    built runner, because it's an system you
    don't control sometimes. And as soon as
  • 6:00 - 6:04
    you get the binary result from it - if you
    compile the software that compiles to
  • 6:04 - 6:09
    binary - you need to check that result
    somehow, because somebody could have
  • 6:09 - 6:15
    altered it without your knowledge and then
    you will ship it to your users. Also a
  • 6:15 - 6:19
    problem on many of this build workers is:
    you want to have this process very fast.
  • 6:19 - 6:23
    So that means you don't want to wait until
    all the dependencies are installed and the
  • 6:23 - 6:28
    great service is that you have caching in
    between these projects. This means that,
  • 6:28 - 6:33
    for example, if somebody managed to inject
    the version to the cache of some CI
  • 6:33 - 6:38
    system, then it will eventually show up in
    other projects as well and you can pivot
  • 6:38 - 6:44
    across the projects. Usually if you have a
    build environment it has access to some
  • 6:44 - 6:49
    kind of development key. Mostly if you get
    pull requests from externals, the keys are
  • 6:49 - 6:52
    stored encrypted and you don't have access
    to them, but as soon as somebody has write
  • 6:52 - 6:58
    access to your repository also the keys
    could be leaked. Let's make an example:
  • 6:58 - 7:02
    you have somebody offering you a software
    and you don't give permission to edit the
  • 7:02 - 7:06
    master branch of the repository but as
    soon as you open an open a branch anywhere
  • 7:06 - 7:11
    and make a pull request Travis CI or other
    build runners will use that and decrypt
  • 7:11 - 7:14
    the passwords for you and give you access
    to the credentials which you can then
  • 7:14 - 7:17
    print or do whatever you intend to.
  • 7:18 - 7:22
    Yeah,
    and for me the best option here would be
  • 7:22 - 7:26
    to have reproducible builds because then
    you can use different of the build workers
  • 7:26 - 7:30
    and compare results somehow so that you
    see if one gets compromised the other two
  • 7:30 - 7:34
    will tell you: hey, there's a different
    result, have a look please.
  • 7:34 - 7:36
    That would be great.
  • 7:36 - 7:40
    And also the build steps; I
    mentioned building, testing and packaging
  • 7:40 - 7:44
    the software are totally different steps
    so what you can do is you can have one
  • 7:44 - 7:49
    compartment per step so that you can have
    a data at a finer level and see what
  • 7:49 - 7:56
    happens here. After you compiled the
    software, you built the software, you need
  • 7:56 - 8:00
    to ship it to the user somehow so either
    you store it in your own server or most
  • 8:00 - 8:05
    often you use a CDN. You just put it there
    and it's the asset that's lying around.
  • 8:05 - 8:09
    Your users will come around, download it
    from here and execute it, so what is the
  • 8:09 - 8:14
    problem here? The problem is, that if you
    have an URL it's very hard to prove that
  • 8:14 - 8:19
    it's actually from the real maintainer. If
    you call your software like if - if you
  • 8:19 - 8:23
    call your account like a different
    project, then people won't be able to
  • 8:23 - 8:28
    notice the difference somehow. What you
    can do to mitigate this, is to publish the
  • 8:28 - 8:34
    URLs that you legitly using and also sign
    your assets so that users can check is
  • 8:34 - 8:38
    that something that the developer intended
    to give me or is it something that is
  • 8:38 - 8:43
    really ... that is really not intended.
  • 8:43 - 8:45
    So, ...
  • 8:46 - 8:49
    Yes. And the next part is you need to
    reach out to your users
  • 8:49 - 8:52
    so you make people aware,
    that there is a project they can
  • 8:52 - 8:57
    check out, they can clone and usually you
    have the package registries. A few slides
  • 8:57 - 9:04
    back you saw that the package managers are
    also included in the editors, so that's
  • 9:04 - 9:08
    also something where you can ship the
    software but the package manager I was
  • 9:08 - 9:13
    mostly looking at was for example NPM.
    There was an interesting occasion where
  • 9:13 - 9:19
    somebody had a project called Kik.The
    company Kik then tried to take it down and
  • 9:19 - 9:23
    the person just ignored it for the moment
    but then Kik reached out to NPM directly
  • 9:23 - 9:28
    and they deleted the repository. In
    consequence the developer removed all his
  • 9:28 - 9:32
    projects from the versioning server and a
    few hours later malware showed up with the
  • 9:32 - 9:36
    same project names, so that means if you
    have a software that uses that
  • 9:36 - 9:41
    dependencies and somebody freed up names
    it would affect your repository as well
  • 9:41 - 9:47
    and compromise it. That's something that
    needs mitigation. I think the best idea
  • 9:47 - 9:51
    here is to not only identify the
    project by a unique identifier but also
  • 9:51 - 9:58
    have a GUID or an or a unique identifier
    per project that does not change, so that
  • 9:58 - 10:02
    you can make a difference. That's
    something that's up to the package
  • 10:02 - 10:05
    registries to implement. That's not
    something we can do as a user but it's a
  • 10:05 - 10:11
    very common case - it's a very common case
    that these packages fluctuate.
  • 10:11 - 10:15
    So for example, if somebody deletes it,
    you don't have a backup of that.
  • 10:15 - 10:18
    A very good idea is also to store offline
    backups of every package that you
  • 10:18 - 10:22
    check out and that you install to
    your software because it's very bad
  • 10:22 - 10:27
    if you want to maintain your software and
    you figure out there's something, there's
  • 10:27 - 10:29
    something missing and I can't recover
    because it's deleted.
  • 10:33 - 10:38
    Yes, software developers have some
    needs during their work.
  • 10:38 - 10:43
    I want my tooling to perform if my
    code editor for example is in the VM and
  • 10:43 - 10:49
    the VM is slow, that's something that's
    annoying all over the process. So then on
  • 10:49 - 10:53
    the other hand the velocity is something
    that your manager will require from you if
  • 10:53 - 10:59
    you write commercial software or you try
    to get something done and you can't spend
  • 10:59 - 11:03
    all day to work on chores and improve
    the repository, the versioning and so on.
  • 11:03 - 11:10
    So that's something you need to deal with.
    Another big factor for me is the
  • 11:10 - 11:13
    reliability. So as soon as your software
    goes down and you are in holiday or
  • 11:13 - 11:18
    something, anybody else from the company
    or from your team should be able to
  • 11:18 - 11:26
    recover what was there before, also known
    as the bus-factor and, yeah, if you have
  • 11:26 - 11:31
    convenience like for example Ruby on Rails
    gives you. It gives you a very good,
  • 11:31 - 11:35
    very easy start in the projects and that's
    something you don't want to break by
  • 11:35 - 11:39
    making it too complicated with a
    development environment. And also
  • 11:39 - 11:45
    something I've found to be more annoying
    than helpful is, if you want to pair-
  • 11:45 - 11:50
    program and you have a very
    compartmentalized environment, it's very
  • 11:50 - 11:53
    hard to share the resources that you need
    to talk about with other developers,
  • 11:53 - 11:57
    expecting you're not in the same room but
    working remotely, what is for me most
  • 11:57 - 11:58
    often the case.
  • 12:02 - 12:07
    A large problem that I saw is, if you
    underhand somebody code,
  • 12:07 - 12:13
    if you go ahead and and check out
    code from any online resources, it's
  • 12:13 - 12:17
    sometimes very hard to tell if the code
    that you see in your, e.g. Git diff,
  • 12:17 - 12:23
    is what you really would expect to
    see. I have some examples here, which can
  • 12:23 - 12:26
    show how this could work and how this
    could look like if somebody tries
  • 12:26 - 12:30
    to inject code to your repository, that
    you don't see. First of all, let's start
  • 12:30 - 12:38
    with something easy, that's phishing. What
    you see here on the slide, on the left
  • 12:38 - 12:43
    side maybe you see the cursor. That's not
    a full path, that is just a domain name.
  • 12:43 - 12:50
    The slashes in here are UTF-8 characters
    so that thing here resolves to a hostname,
  • 12:50 - 12:54
    and if you control this host, you can get
    a certificate for it and then the example
  • 12:54 - 13:00
    below you see, how it would look like, if
    you install it. First I have a host that's
  • 13:00 - 13:04
    just running a web server on port 80, so
    that you can see the result. Okay, I was
  • 13:04 - 13:09
    cheating a little bit. I was putting the
    domain in the /etc/hosts so that I don't
  • 13:09 - 13:16
    have to buy it, for just showing it. It's
    strange that dot zip is a domain actually,
  • 13:16 - 13:22
    but then if you install it, you would see
    that you can send somebody a very
  • 13:22 - 13:25
    nice-looking link which looks like a
    totally different project, but it's
  • 13:25 - 13:31
    pointing to your server instead; and I
    found many of the package managers having
  • 13:31 - 13:35
    the nice feature of executing PostScript
    hooks, so that means, if you have
  • 13:35 - 13:39
    installed it, it will run some commands
    afterwards for you.
  • 13:42 - 13:46
    Then there is invisible code. If you go
    online somewhere, find in a forum or in a
  • 13:46 - 13:51
    blog, you find an article and see, "hey,
    that code is actually solving my problem",
  • 13:51 - 13:55
    you go ahead and copy/paste it. So, on the
    left you see the source code how this
  • 13:55 - 14:00
    would look like in HTML for the blog. On
    the right there's the result. So you can
  • 14:00 - 14:04
    go ahead, you can copy/paste from it and
    if you paste it to a text area, you will
  • 14:04 - 14:08
    see, that the result is something that you
    didn't expect. For example, if you copy a
  • 14:08 - 14:12
    large chunk of code, you won't go ahead
    and review it on your local system again
  • 14:12 - 14:16
    and that could be the compromise for your
    project.
  • 14:27 - 14:31
    Another example here is, you can
    use ASCII characters,
  • 14:31 - 14:35
    the control characters to influence
    the output in your terminal.
  • 14:35 - 14:38
    So if your terminal also
    supports the legacy of ASCII control
  • 14:38 - 14:43
    characters, you can use that to just
    revert the line and override it with
  • 14:43 - 14:48
    something you wouldn't expect. What you
    see on top here, that harmless script is
  • 14:48 - 14:52
    the file. It's a little larger than you
    would expect for just a echo foo, but not
  • 14:52 - 14:57
    something you would notice when you just
    see it. Looking at it from a hex editor
  • 14:57 - 15:01
    you can see, that there is something more
    going on than just the foo and if you
  • 15:01 - 15:06
    actually execute it, it will not print
    something, it will create the pwned text,
  • 15:06 - 15:11
    which is a good example for you that your
    host was compromised in this moment.
  • 15:15 - 15:20
    Another example I found online, so credit
    to Ariel for this, so there is a byte
  • 15:20 - 15:23
    sequence you can use so that this even
    works in a Git diff. So when you're
  • 15:23 - 15:27
    working exclusively in your terminal and
    you're not doing reviews on GitHub or some
  • 15:27 - 15:33
    graphical tool, it could be the case that
    you don't notice what was going on. What
  • 15:33 - 15:38
    you can see here on the left is, I created
    an empty repository, I added a small
  • 15:38 - 15:43
    script and in the next step down here, I
    added some improvement to the script,
  • 15:43 - 15:51
    which is actually the malicious commit
    that's here in red. Afterwards, I just ran
  • 15:51 - 15:56
    a Git diff on the code and I see that
    there is only no backdoor, oh sorry that
  • 15:56 - 16:01
    should be okay in the updated slides. So,
    you don't see the evil.sh that it's
  • 16:01 - 16:06
    executed as well, if you run it. That's
    something I consider very dangerous.
  • 16:11 - 16:16
    So, some mitigations: the best thing you
    can do, is to make it expensive for your
  • 16:16 - 16:21
    attackers to compromise or try to. So
    as soon as you have the chance to notice
  • 16:21 - 16:27
    what is going on, also retrospectively,
    you can at least burn the capabilities and
  • 16:27 - 16:32
    tell others how your project was attempted
    to compromise and that's something
  • 16:32 - 16:35
    that is, in my opinion, the best
    mitigation against this complexity.
  • 16:37 - 16:42
    What you can also do, is you can test your
    software from external services directly,
  • 16:42 - 16:48
    which will tell you, if some compromise
    happened. For example, Git has it newly
  • 16:48 - 16:52
    introduced. They will check your packages,
    the dependencies and will warn you about
  • 16:52 - 16:59
    some vulnerabilities that are commonly
    known. The best thing you can do on your
  • 16:59 - 17:02
    local system is to build small
    compartments, so that if some compromise
  • 17:02 - 17:09
    happens, it doesn't affect your full host.
    Also not all your projects you have
  • 17:09 - 17:12
    access to. And it's very important, that
    you have backups on a different system
  • 17:12 - 17:16
    than the hosts you're working on. So if
    the compromise happens you still have
  • 17:16 - 17:21
    access to the original data and can
    compare it and do some forensics on this.
  • 17:24 - 17:28
    So, the intrusion detection forensics;
    there are some great tools available.
  • 17:28 - 17:33
    For example my favorites are DTrace and
    Opensnoop. You can monitor changes and
  • 17:33 - 17:37
    access on the file system or on your
    system at all. And you can e.g. set some
  • 17:37 - 17:40
    rules for your projects that are
    specifically matching. So I am
  • 17:40 - 17:44
    not going to share some rules that match
    for all projects, but you will figure out
  • 17:44 - 17:49
    what is e.g. important. Very
    good start is e.g. to Opensnoop for
  • 17:49 - 17:54
    /etc/passwd if there was some access, then
    you can e.g. say that's not something
  • 17:54 - 18:01
    what my software would do. And again it's
    very important to have the backups of this
  • 18:01 - 18:05
    because in the moment where you execute
    it, you can't trust your host at all.
  • 18:07 - 18:12
    The idea how to achieve this is,
    if you have a VM per project for example,
  • 18:12 - 18:16
    you let it run for half a year,
    you don't approve the situation. Instead
  • 18:16 - 18:19
    of having one system that you need to
    update the software to, you need to update
  • 18:19 - 18:24
    afterwards all the projects that you're
    working on frequently and that's something
  • 18:24 - 18:28
    that's easy to forget. So it's dangerous.
    If you assume that every time you run some
  • 18:28 - 18:32
    command or every time you work in a
    project, you spin up a new server entirely
  • 18:32 - 18:36
    from scratch, install the dependencies and
    so on that's something that's not a risk
  • 18:36 - 18:43
    for you. Also, if you have for example a
    virtualized server environment you can
  • 18:43 - 18:48
    have memory dumps at any time, you can
    monitor the network and you can also diff
  • 18:48 - 18:52
    the filesystem. For example, if you stop
    the server and just compare it to a
  • 18:52 - 18:56
    previous version and see, "hey, here is
    something that was changed that I didn't
  • 18:56 - 18:59
    plan". It's great to know.
  • 19:02 - 19:06
    Very important is also to separate your
    accounts. E.g., if you see large GitHub
  • 19:06 - 19:09
    accounts, people are making contributions
    every day since years.
  • 19:12 - 19:17
    So it shows that the people
    have access to very, to many projects from
  • 19:17 - 19:22
    the same machine. And the permission model
    from GitHub for example, allows you to
  • 19:22 - 19:27
    store an SSH key for write access. But it
    automatically has access to all the
  • 19:27 - 19:31
    repositories you control. So the best that
    you can have here, is to make a
  • 19:31 - 19:35
    new GitHub account for, ... or to make a
    new account on that versioning system
  • 19:35 - 19:39
    that only has exclusively write access
    to that single repository.
  • 19:40 - 19:44
    So when you work in your
    compartmented system and you want to
  • 19:44 - 19:50
    upload or pull changes, you can't
    influence other repositories. That means
  • 19:50 - 19:55
    compromise doesn't spread across all your
    projects and so on, which would be an
  • 19:55 - 20:00
    invitation for malware somehow or
    ransomware. And you get a better
  • 20:00 - 20:06
    permission model if you create a GitHub
    organization. In this case you can also
  • 20:06 - 20:12
    limit your own access in a better way. So
    my recommendation is not to work on your
  • 20:12 - 20:15
    personal GitHub account but create an
    organization for your project.
  • 20:19 - 20:26
    Something many projects are missing are to
    find responsible persons for security
  • 20:26 - 20:33
    and to clearly communicate what is the
    plan for incident response. Small example:
  • 20:33 - 20:37
    If you have a new project and you
    find a vulnerability, you would like to
  • 20:37 - 20:41
    commit it, but you don't open an issue
    publicly, because then everybody, every
  • 20:41 - 20:44
    user would be affected. You try to reach
    out to some developers and if you don't
  • 20:44 - 20:53
    have any clue how to securely achieve
    this, that can get you into trouble. And
  • 20:53 - 20:58
    there are quite a few projects which don't
    communicate this and some of them don't
  • 20:58 - 21:02
    even respond to their security@ email
    address, which is bad.
  • 21:06 - 21:12
    In this case I told you what I saw from
    my experiences of working on the projects.
  • 21:12 - 21:21
    So that's basically my summary of
    what can be harmful
  • 21:21 - 21:23
    and what can be good for your project.
  • 21:23 - 21:33
    Applause
  • 21:33 - 21:38
    Herald: Thank you and we now have time for
    Q&A. In the room you can line up behind
  • 21:38 - 21:43
    the microphones and I can see we have a
    question from the internet already.
  • 21:44 - 21:50
    Signal Angel: What about Git signed
    commits? Any thoughts on that?
  • 21:50 - 21:54
    Answer: So as soon as you have signed
    commits and I find that you also email
  • 21:54 - 21:57
    with the same PGP key, it's very
    interesting that you have the PGP key on
  • 21:57 - 22:01
    the same host probably, then you have your
    Git executable. So if somebody executes
  • 22:01 - 22:07
    Git hooks, they can steal your PGP keys
    from this. I didn't find any tutorial
  • 22:07 - 22:12
    online which explains you, how to make it
    manually, so that you don't use the Git
  • 22:12 - 22:17
    for signing the commits. But I think it
    can be very good to sign the commits,
  • 22:17 - 22:23
    but it can be also dangerous, because your
    email communication can be compromised.
  • 22:23 - 22:29
    Herald: Microphone number four.
    Q: In the Git diff you showed us, there
  • 22:29 - 22:37
    were some control characters. I think
    Git diff pipes to less by default, so
  • 22:37 - 22:41
    shouldn't they appear there somewhere?
    A: No, they don't. I just checked with the
  • 22:41 - 22:47
    latest version today. So, that's something
    that, well, we can also click on the blog
  • 22:47 - 22:50
    and see, if there is the video available.
  • 23:05 - 23:08
    Yeah, it's very hard to show from my
    HTML slide how this works.
  • 23:08 - 23:11
    So this video animation, ...
    maybe we can enlarge it a bit.
  • 23:13 - 23:14
    That's how it would work.
  • 23:30 - 23:35
    So most often, yes, if you pipe to less or
    you use a hex editor to review,
  • 23:35 - 23:36
    then you would notice, yes.
  • 23:38 - 23:44
    Q: I somehow remember that, maybe it
    only shows for longer diffs, but I think
  • 23:44 - 23:47
    when I type "git diff", I can scroll
    around.
  • 23:47 - 23:53
    A: Ah, that's interesting. I need to try.
    Herald: We have a question from microphone
  • 23:53 - 23:59
    number one.
    Q: You mentioned Travis having access to
  • 23:59 - 24:07
    hidden variables and you being able to
    leak those variables during pull requests.
  • 24:07 - 24:10
    What are your suggestions to mitigate
    that?
  • 24:10 - 24:15
    A: Don't give people write access to your
    repository, not even to branches that you
  • 24:15 - 24:19
    don't trust. So, as soon as they have
    write access, they would also know the
  • 24:19 - 24:23
    secrets behind the variables in this case.
    I like the security model, because if you
  • 24:23 - 24:27
    for example get contributions from
    outside, nobody can trigger that and steal
  • 24:27 - 24:29
    your keys.
    But as soon as you build it on your own
  • 24:29 - 24:32
    branch somewhere in the repository that
    changes.
  • 24:32 - 24:39
    Q: Yes, but if you submit a pull request,
    you don't necessarily have to have write
  • 24:39 - 24:43
    access to that repository.
    A: Yes, that's what I mean. If you come
  • 24:43 - 24:48
    from outside and it's not within the same
    repository, the secrets are not decrypted.
  • 24:48 - 24:52
    So you can't run the steps. For example,
    you would not like to deploy directly from
  • 24:52 - 24:58
    a foreign branch, somewhere.
    Herald: We have a question from microphone
  • 24:58 - 25:03
    number four.
    Q: You mentioned the problem with
  • 25:03 - 25:07
    different compartments and how to exchange
    those environments without people. I think
  • 25:07 - 25:11
    that problem has already been solved with
    Vagrant and some kind of provisioning
  • 25:11 - 25:15
    software like Ansible.
    Do you have any experience with checking
  • 25:15 - 25:20
    those results of those Vagrant boxes that
    are automatically provisioned, like
  • 25:20 - 25:26
    having some server spec software to check
    those environments afterwards, or having
  • 25:26 - 25:31
    some kind of hashing, how to find out, if
    they have been reproduced the same way and
  • 25:31 - 25:37
    or if they have been any exploit used
    in that process of setting up the Vagrant
  • 25:37 - 25:40
    environments.
    A: Yes, so different levels you can look
  • 25:40 - 25:46
    at this. There was some, ..., I try to
    find it, yes. You can for example memory
  • 25:46 - 25:50
    dump at any time, if you have the hosts
    trying somewhere or was your question
  • 25:50 - 25:55
    exactly that you want to check, if your
    environment that was spun up was not
  • 25:55 - 25:59
    compromised yet?
    Q: Yeah, there has to be some kind of
  • 25:59 - 26:06
    process, how to verify that the produced
    environments are the ones you expect them
  • 26:06 - 26:11
    to be, or if they have been compromised
    and the problem is, I have used those
  • 26:11 - 26:17
    environments and tried, first I tried the
    full disk encryption for the Vagrant boxes
  • 26:17 - 26:27
    but the problem is, it's always the same
    key for the encryption, so that doesn't
  • 26:27 - 26:31
    work and even as you mentioned, you can
    have a memory dump, so you can read out
  • 26:31 - 26:37
    that key so there's no real possibility to
    set up a Vagrant box that can't be
  • 26:37 - 26:46
    tampered with afterwards. So there has to
    be some kind of hash sum to compare those
  • 26:46 - 26:49
    produced results.
    A: Yeah, so as soon as you have a
  • 26:49 - 26:53
    reproducible build and the result that
    you, for example script languages are much
  • 26:53 - 26:58
    easier to achieve, because then you can
    just diff the filesystem directory and
  • 26:58 - 27:02
    see, if there was some change. What I
    would do in this case, is to run multiple
  • 27:02 - 27:06
    services and compare the results, if
    that's possible. For example, you have
  • 27:06 - 27:11
    these reproducible builds, then run it on
    a few servers which are independent and
  • 27:11 - 27:14
    compare what you have.
    Herald: We have two more questions from
  • 27:14 - 27:18
    microphone number one and only a few
    minutes left. Microphone number one.
  • 27:18 - 27:23
    Q: So, what's your recommendation for
    handling credentials in application
  • 27:23 - 27:29
    configuration files? We need often some
    database user and password or something
  • 27:29 - 27:34
    like this in, say Spring Boot Application
    YML, or things like that?
  • 27:34 - 27:41
    And is there any best practice or any
    framework which can handle such things or
  • 27:41 - 27:47
    we need to explicitly encrypt these
    credentials in this application and then
  • 27:47 - 27:52
    decrypt for itself in the application, but
    then you need symmetric keys, or?
  • 27:52 - 27:56
    A: Yes, so Ansible for example comes with
    a mechanism that's called Ansible Vault,
  • 27:56 - 28:00
    which encrypts that with a passphrase that
    you can enter in your command line as soon
  • 28:00 - 28:04
    as you touch the file. For example, if you
    want to run Ansible then, it will ask you
  • 28:04 - 28:08
    for the password when starting up. So, if
    you want to share that password with your
  • 28:08 - 28:11
    developers everybody has access to the
    same keys, I would prefer to give
  • 28:11 - 28:16
    everybody, so every person in this team or
    even every device a different key, if
  • 28:16 - 28:21
    that's possible somehow.
    That's what I was trying to mention with
  • 28:21 - 28:25
    the GitHub accounts; that you don't use
    one GitHub account but you use many of
  • 28:25 - 28:27
    them. If you, ... yeah.
  • 28:28 - 28:32
    Herald: We have one more question from
    microphone number one and then a question
  • 28:32 - 28:35
    from the internet.
    Q: Yeah, my question was more about, I
  • 28:35 - 28:39
    mean some of your recommendations are low-
    hanging fruits, but some of them it's
  • 28:39 - 28:43
    like, it's just impossible. I mean it's
    not sustainable, like it's very hard to
  • 28:43 - 28:47
    maintain and so I'm wondering, if you use
    all of them every day or just part of them
  • 28:47 - 28:50
    or do you just leave like an ??? at the
    end?
  • 28:50 - 28:54
    A: It depends on the project. So what I
    try to do on my development system is to
  • 28:54 - 28:59
    have these compartment, so that one
    compromised project would not affect
  • 28:59 - 29:04
    others. Because I'm not the only person
    checking and merging the code, so and
  • 29:04 - 29:09
    that's something that gets quickly too
    much for one person to review. So, I
  • 29:09 - 29:13
    can't review all the code that I'm running
    currently on my computer, that's true. But
  • 29:13 - 29:17
    I can try to mitigate what the impact of
    this will be.
  • 29:17 - 29:21
    Herald: And the question from the internet.
    Signal: What tool would you recommend
  • 29:21 - 29:32
    for diffing a file system?
    A: Diff. giggles Well, it worked for me
  • 29:32 - 29:39
    so far. Or what exactly is the question
    about? Um, maybe you want to see, if did
  • 29:39 - 29:42
    the hash change in the files? So when you
    have e.g. the script file one and
  • 29:42 - 29:46
    the script file B and they have a
    different hash sum, that's something I
  • 29:46 - 29:50
    would consider something I would look up
    manually. So as soon as I have an
  • 29:50 - 29:53
    indication that there was something wrong,
    I would look it up manually and use any
  • 29:53 - 29:57
    tool that I have. Hex editor or whatever
    is available.
  • 29:57 - 30:01
    Herald: Good. We have less than one minute
    left. Are there any final remarks?
  • 30:01 - 30:04
    Stephan: Thank you.
    Herald: Thank you very much.
  • 30:04 - 30:06
    Applause
  • 30:10 - 30:16
    postroll music
  • 30:16 - 30:28
    subtitles created by c3subtitles.de
    in the year 2019. Join, and help us!
Title:
34C3 - Hardening Open Source Development
Description:

more » « less
Video Language:
English
Duration:
30:28

English subtitles

Revisions Compare revisions