1 00:00:00,000 --> 00:00:01,885 Know that I'm standing in between, uh 2 00:00:02,523 --> 00:00:04,163 whatever, Wednesday night beer is 3 00:00:04,382 --> 00:00:04,632 or dinner, uh 4 99:59:59,999 --> 99:59:59,999 Actually this will be a pretty short talk 5 99:59:59,999 --> 99:59:59,999 but uh moreover I would like to hear from you, so... 6 99:59:59,999 --> 99:59:59,999 My name is Arseny. 7 99:59:59,999 --> 99:59:59,999 I uh I do not work for Dell 8 99:59:59,999 --> 99:59:59,999 haha by the way haha 9 99:59:59,999 --> 99:59:59,999 I do not work for Dell 10 99:59:59,999 --> 99:59:59,999 I'm a solution[s] architect and I've been working on some Monetary Authority of Singapore, uh 11 99:59:59,999 --> 99:59:59,999 Technology risk guidelines, before, like many years ago 12 99:59:59,999 --> 99:59:59,999 They were related to vmware, and I coauthored comparison 13 99:59:59,999 --> 99:59:59,999 to help the industry basically leverage the best practices 14 99:59:59,999 --> 99:59:59,999 and map, and if there's a regulatory requirement, how to answer it 15 99:59:59,999 --> 99:59:59,999 if there's a security team within a big financial organisation, giving you hard time, how to answer their question. 16 99:59:59,999 --> 99:59:59,999 so I basically want to do the same for containers for Docker 17 99:59:59,999 --> 99:59:59,999 I basically share the digest of open materials that I found 18 99:59:59,999 --> 99:59:59,999 around using Google search engine, 19 99:59:59,999 --> 99:59:59,999 but I really want to hear from you because I believe there's, uh, very different crowd today 20 99:59:59,999 --> 99:59:59,999 and I really wanted to hear what you do 21 99:59:59,999 --> 99:59:59,999 what do you do for security in your organisation 22 99:59:59,999 --> 99:59:59,999 To start with I'm not a security expert 23 99:59:59,999 --> 99:59:59,999 So don't worry, it's fine 24 99:59:59,999 --> 99:59:59,999 I'm not a security expert, have never been it 25 99:59:59,999 --> 99:59:59,999 And I don't want to boil the ocean 26 99:59:59,999 --> 99:59:59,999 It's really not something I will be able to do in 20 minutes 27 99:59:59,999 --> 99:59:59,999 So forget about it 28 99:59:59,999 --> 99:59:59,999 So 2 questions: 29 99:59:59,999 --> 99:59:59,999 Who has in their production environment today any test-driven security pipelines? 30 99:59:59,999 --> 99:59:59,999 anyone does test-driven security approaches today? 31 99:59:59,999 --> 99:59:59,999 Cool 32 99:59:59,999 --> 99:59:59,999 Who has, ok before even going to SAT, this acronym from financial markets 33 99:59:59,999 --> 99:59:59,999 Who works for insurance of some regulated enterprise 34 99:59:59,999 --> 99:59:59,999 That runs Docker in production? 35 99:59:59,999 --> 99:59:59,999 1...2...ok 36 99:59:59,999 --> 99:59:59,999 Got it 37 99:59:59,999 --> 99:59:59,999 That's good for me, because I cannot screw up 38 99:59:59,999 --> 99:59:59,999 that significantly because everything I do is just assuming my experience and what not 39 99:59:59,999 --> 99:59:59,999 So, uh, I wanna just start with a problem statment 40 99:59:59,999 --> 99:59:59,999 Remember the good old days we had very clear demarcation line between 41 99:59:59,999 --> 99:59:59,999 where the developers are and where the the operations people are 42 99:59:59,999 --> 99:59:59,999 The dependencies in here, this is where the gray line is starting to appear with containerisation 43 99:59:59,999 --> 99:59:59,999 So, who actually keeps an eye on all the dependencies for the containers? 44 99:59:59,999 --> 99:59:59,999 Within the container you may have a particular version of Python, with in the container, running on the same host you may have, like, uh, [inaudible] of other dependencies 45 99:59:59,999 --> 99:59:59,999 So, who has to work on that? 46 99:59:59,999 --> 99:59:59,999 So, in ideal containerised world, it looks like that, you have an application and it has all the prerequisites, all the dependencies installed and everything 47 99:59:59,999 --> 99:59:59,999 is beautiful 48 99:59:59,999 --> 99:59:59,999 but then, all of a sudden, your guys call you and they say: 49 99:59:59,999 --> 99:59:59,999 hey, AC, we forgot we actually have a better release 50 99:59:59,999 --> 99:59:59,999 we just migrated into Java, we wanna migrate into Java, can you just make sure that the image we are building on 51 99:59:59,999 --> 99:59:59,999 the production security image is right now running on Java, no longer need all these dependencies 52 99:59:59,999 --> 99:59:59,999 We need it like in 5 minutes 53 99:59:59,999 --> 99:59:59,999 Well, it's DevOps world, you do is what you do 54 99:59:59,999 --> 99:59:59,999 you install the needed prerequisites, update the image of the container 55 99:59:59,999 --> 99:59:59,999 send it back to the rebuild pipeline, everything works, fine, nothing breaks 56 99:59:59,999 --> 99:59:59,999 But then they wake up and they say, like, oh by the way, we needed Java 8, we forgot 57 99:59:59,999 --> 99:59:59,999 It actually is optimised for Java 8, can you do the same but for Java 8? 58 99:59:59,999 --> 99:59:59,999 And you do that and in the end, the risk is with all these images 59 99:59:59,999 --> 99:59:59,999 You end up with enormous amount of, uh, configuration drift. 60 99:59:59,999 --> 99:59:59,999 So you may have, like, ports open and something listening on those ports, and 61 99:59:59,999 --> 99:59:59,999 maybe not even patched through the proper levels 62 99:59:59,999 --> 99:59:59,999 And this is how configuration drift had looked like for that particular image 63 99:59:59,999 --> 99:59:59,999 It on [inaudible].io and there are, like, nginx, image that has been open and downloaded like thousand times 64 99:59:59,999 --> 99:59:59,999 they send it... and it has been used in multiple projects as you can imagine 65 99:59:59,999 --> 99:59:59,999 across, uh, the universe, across the internet, and they were like 66 99:59:59,999 --> 99:59:59,999 264 medium level vulnerabilities, 126 high-level vulnerabilities, 67 99:59:59,999 --> 99:59:59,999 it's just because everyone just patched it a little bit, 68 99:59:59,999 --> 99:59:59,999 patched it a little bit there, update the image and the latest became this huge drift 69 99:59:59,999 --> 99:59:59,999 from, like, where it should have been. 70 99:59:59,999 --> 99:59:59,999 Right? So... 71 99:59:59,999 --> 99:59:59,999 And actually this a very interesting fly that I found 72 99:59:59,999 --> 99:59:59,999 Look at this, information security job postings, they kinda tend to go down... 73 99:59:59,999 --> 99:59:59,999 And then all these jobs, they kinda tend to go up! 74 99:59:59,999 --> 99:59:59,999 So, the problem, I guess, will be signified in future 75 99:59:59,999 --> 99:59:59,999 So if we look at the DevOps factory 76 99:59:59,999 --> 99:59:59,999 I'll try to put it in here 77 99:59:59,999 --> 99:59:59,999 This is the usual process, it's a mixed like logical and process stateful diagram interchange that leads from development of a particular business requirement from business partners and business analysists 78 99:59:59,999 --> 99:59:59,999 And all the way into production that's on the operating side and the greens side 79 99:59:59,999 --> 99:59:59,999 Do you see anything on this diagram, related to security at all? 80 99:59:59,999 --> 99:59:59,999 This is DockerCon, this is, uh, 2017, it's actually from the keynote of DockerCon 2017 from Metlife, from their presentation. 81 99:59:59,999 --> 99:59:59,999 So couple of months ago, nothing here says about security. 82 99:59:59,999 --> 99:59:59,999 It does mention, ok, we have a CI/CD tool, and magically we do push/pull tag and image scanning, no details. 83 99:59:59,999 --> 99:59:59,999 So I guess this is the area where need to at least put a bookmark for ourselves, well we need to research it 84 99:59:59,999 --> 99:59:59,999 Because, like, not too much of material or best practices are currently in easily digestable format 85 99:59:59,999 --> 99:59:59,999 How to do that stuff... 86 99:59:59,999 --> 99:59:59,999 I actually included one marketing slide later... 87 99:59:59,999 --> 99:59:59,999 But again, it doesn't actually give you all the best practices how to actually do stuff. 88 99:59:59,999 --> 99:59:59,999 So the new reality is. There's a new term, DevSecOp -- developer security operations. 89 99:59:59,999 --> 99:59:59,999 The new reality is we have a completely decentralised ownership of deployment, so we need to get as early concerned about security as we can 90 99:59:59,999 --> 99:59:59,999 We should narrow it down to an easily digestable tasks all the way through the pipeline, 91 99:59:59,999 --> 99:59:59,999 and make sure there's no configuration drift in the image for example 92 99:59:59,999 --> 99:59:59,999 and make sure that the injection points are designed together with the developer, so they understand the concerns 93 99:59:59,999 --> 99:59:59,999 So they don't come and rush it to you and ask to give them Java 8 94 99:59:59,999 --> 99:59:59,999 And then, provoke this configuration drift and hence give you a lot og vulnerabilities. 95 99:59:59,999 --> 99:59:59,999 At a 1-1 level, how secure is Docker? 96 99:59:59,999 --> 99:59:59,999 And is it the kind of conversation that we may have with the security guys just over coffee in a big firm? 97 99:59:59,999 --> 99:59:59,999 Or in a Starbucks, if you see someone? 98 99:59:59,999 --> 99:59:59,999 I went into CVE details and opened up what the current open Docker vulnerabilities. 99 99:59:59,999 --> 99:59:59,999 So, for, vendor called Docker, there are currently 14 vulenrabilities 100 99:59:59,999 --> 99:59:59,999 Some still pending from 2016, not yet patched, it's not too bad actually. 101 99:59:59,999 --> 99:59:59,999 14 CVE's, eh! 102 99:59:59,999 --> 99:59:59,999 Just to give you an anchor point, what is kubernetes, kubernetes actually has 3. 103 99:59:59,999 --> 99:59:59,999 So, there are 3 vulnerabilities currently known to the global population of our planet that are related to Kubernetes. 104 99:59:59,999 --> 99:59:59,999 And to give you another anchor point, our favourite product, ESXi, from the very old days, currently has 31 vulnerabilities. 105 99:59:59,999 --> 99:59:59,999 So, answer is, it's quite secure, just by nature. Docker and Kubernetes are well-built, they are not really vulnerable as they are. 106 99:59:59,999 --> 99:59:59,999 How about running in the cloud? 107 99:59:59,999 --> 99:59:59,999 There's a report, and again I'll share all the materials and digest of the links with you, when you run deployment template for cloud formation that Docker provides with Docker Cloud, 108 99:59:59,999 --> 99:59:59,999 There are 5 total issues, and 2 of them are low-risk. Basically, they are over-provisioning cloud formation IAM roles. 109 99:59:59,999 --> 99:59:59,999 Just with little bit too much of permissions than needed. 110 99:59:59,999 --> 99:59:59,999 I would say, brilliant, actually, 111 99:59:59,999 --> 99:59:59,999 Same for Azure, there was an independent research that has gone through... recently this research was published, there are only 3 issues that are related to rolling out Docker on top of Azure. 112 99:59:59,999 --> 99:59:59,999 Bear in mind, our issues again, related to more of like, preferences rather than the problem. 113 99:59:59,999 --> 99:59:59,999 Now, what has to happen, we used to run app, and again I apologies, it's just the digest of multiple different slides, right? 114 99:59:59,999 --> 99:59:59,999 We used to run apps through the development into the user acceptance testing and then we would've sent them to the security acceptance tests, which are mandatory part of the DevOps in the old, legacy world. 115 99:59:59,999 --> 99:59:59,999 So, in a bank for example, security acceptance team would take, like, couple of days, then they will call you and they say "Dude, this is not working, we are not passing the scan..." this used to be, like, a waterfall approach. 116 99:59:59,999 --> 99:59:59,999 Now, they looked for everything, so security acceptance team would've scrutinised all the code that you push into production, they would've gone and checked every particular source code as well, crawled it for customer data, crawled it for some stored credentials, what not. 117 99:59:59,999 --> 99:59:59,999 They were acceptable, when we did not have to do releases 5 times and hour. So it's not really what we can do these days 118 99:59:59,999 --> 99:59:59,999 with DevOps, so the new analysis for vulnerabilities, the new analysis for security has to be bottoms-up one. 119 99:59:59,999 --> 99:59:59,999 We need to identify the specific classes per application, what are the most likely vulnerable areas in this app, or in this microservice on in this microservice, 120 99:59:59,999 --> 99:59:59,999 and we need to only care about just those, and we need to push it as heavily into the elimination of these problems early, before they actually hit any particular last-minute security tests 121 99:59:59,999 --> 99:59:59,999 we need to push for that in the development pipeline as much as we can, 122 99:59:59,999 --> 99:59:59,999 and that would actually support the only way to support, this would be the only way to support the velocity of security 123 99:59:59,999 --> 99:59:59,999 Some tidbits, there are really good security best practices published by Docker themselves 124 99:59:59,999 --> 99:59:59,999 They talk about the usual things, usual things, then I'm saying usual things, the things that regulator, the things that your security officer will care about. 125 99:59:59,999 --> 99:59:59,999 Centralised logging -- yes! 126 99:59:59,999 --> 99:59:59,999 [inaudible] reference architecture -- yes, please do something with CloudWatch, 127 99:59:59,999 --> 99:59:59,999 Lot of cloud deployment in the AWS, please use CloudTrail for the API traces as well, in GCP everything is centralised quite differently with StackDriver 128 99:59:59,999 --> 99:59:59,999 Do the signing of the content if you use a repository, there's a service within Docker, it's called Notary that allows you to create a hashed, some kind of like, snapshots of the code 129 99:59:59,999 --> 99:59:59,999 of a particular revision, so that there's no tempering, there's no man-in-the-middle substitution 130 99:59:59,999 --> 99:59:59,999 All these best practices are covered, there's special very nice document out there. 131 99:59:59,999 --> 99:59:59,999 Now, there's also for the host itself, that runs Docker, and that is something that could be incorporated into the security practices within your firm 132 99:59:59,999 --> 99:59:59,999 There's an audit tool, that kindof comes as a part of the intrusion prevention scanner, 133 99:59:59,999 --> 99:59:59,999 So you run a small container on the worker node itself 134 99:59:59,999 --> 99:59:59,999 And as you can see, it maps into the /var/runs and etc 135 99:59:59,999 --> 99:59:59,999 And pull the usual Linux sockets and figures out, are they configured the host underneath, running the containers? 136 99:59:59,999 --> 99:59:59,999 Are they configured as per the best practices against the known, latest vulnerabilities. 137 99:59:59,999 --> 99:59:59,999 This is part of Docker, this is a audit tool, you run it as one command line, and it produces a report like this. 138 99:59:59,999 --> 99:59:59,999 Is your Docker current? 139 99:59:59,999 --> 99:59:59,999 Is the Docker audit files are in place? 140 99:59:59,999 --> 99:59:59,999 Are you logging into right folders? 141 99:59:59,999 --> 99:59:59,999 Do you do proper logging level, is it like, only exceptional or everything that you do? 142 99:59:59,999 --> 99:59:59,999 etc, etc, etc 143 99:59:59,999 --> 99:59:59,999 now, there's also a separate, and again, I seek your input, maybe you can try [inaudible] more tools around it 144 99:59:59,999 --> 99:59:59,999 there's a separate project that could do pretty much the same thing, but both on-prem and in the cloud 145 99:59:59,999 --> 99:59:59,999 And it's called Clair, it's developed by CoreOS team 146 99:59:59,999 --> 99:59:59,999 What it does is, it also pulls the vulnerability updates into a small data store, and then also runs it per layer analysis of your Docker image 147 99:59:59,999 --> 99:59:59,999 if you actually run it on your version N-2 it takes like an hour, 148 99:59:59,999 --> 99:59:59,999 but since you've done it on layer, that was N-2, on the N-1, it would only scan for the incremental changes on that layer, so you don't have to redo the entire docker scan, you actually significantly reduce the time 149 99:59:59,999 --> 99:59:59,999 for the vulnerabilities scan, unless there's something not neat 150 99:59:59,999 --> 99:59:59,999 take a look at this Clair tool, it's incorporated into [inaudible] providers 151 99:59:59,999 --> 99:59:59,999 Security artefacts, this is a pain point, but I mean, you probably heard about ransomware for MongoDB... 152 99:59:59,999 --> 99:59:59,999 That recently hit the entire planet? 153 99:59:59,999 --> 99:59:59,999 People did not at all set passwords, 154 99:59:59,999 --> 99:59:59,999 but the other problem is, when you set passwords, you hardcode them 155 99:59:59,999 --> 99:59:59,999 so the best practice is to avoid hardcoding and to pass the credentials, pass the security artefacts through environmental variables 156 99:59:59,999 --> 99:59:59,999 and you do that for example, by setting -e variable, that is then available for you app to pull 157 99:59:59,999 --> 99:59:59,999 so you literally pass something into the container from the outer world when you run it 158 99:59:59,999 --> 99:59:59,999 and then you take it and use it, that is a very simple best practice 159 99:59:59,999 --> 99:59:59,999 but for some reason, people, especially developing guys, they tend to forget when they push apps and there are so many bitcoins that are paid to ransom, 160 99:59:59,999 --> 99:59:59,999 because someone then commits the code the public github and it's all there 161 99:59:59,999 --> 99:59:59,999 when I've been looking at a lot of resources, there are like, some, attack vector analyses, it takes like 15 minutes to completely block 162 99:59:59,999 --> 99:59:59,999 especially through Jenkins, I have a slide on that 163 99:59:59,999 --> 99:59:59,999 it's insane, Jenkins allows you to crack it through the console, if it's really password-protected well 164 99:59:59,999 --> 99:59:59,999 Anyhow, please pass them through environmental variables, you can also, if you are using AWS, EC2 container service, you can pass them in the cloud formation template 165 99:59:59,999 --> 99:59:59,999 define them, and use the cloud formation script to retrieve it from some secure vault or from other source 166 99:59:59,999 --> 99:59:59,999 into the cloud formation, so even your cloud formation template that then builds a lot of docker containers on AWS, will not have it hard-coded 167 99:59:59,999 --> 99:59:59,999 you just script around getting it from some other location 168 99:59:59,999 --> 99:59:59,999 and you could retrieve it from the metadata of the instance, if you are running on EC2 or Google Cloud, you can just curl into the instance metadata from within the app, at the bootstrap 169 99:59:59,999 --> 99:59:59,999 and then retrieve the tokens that you need to access other services 170 99:59:59,999 --> 99:59:59,999 Another tip: previous speaker had it on one of their slides, there's a very interesting startup called cilium, does an additional layer of tapping into the network at the packet level 171 99:59:59,999 --> 99:59:59,999 they insert themselves before the virtualised adaptors of Docker, add themselves almost like a kernel module, they allow packet filtering to the container blade running on the worker node defined in BPF format 172 99:59:59,999 --> 99:59:59,999 BFP system is already adopted by Docker, for example, they allow and deny particular syscalls, so form within the Docker container, by default there are some syscalls that you cannot run at all 173 99:59:59,999 --> 99:59:59,999 There are 44 banned syscalls, you cannot run mount or process trace, you cannot reboot the worker node from within the container 174 99:59:59,999 --> 99:59:59,999 They already use the BPF framework in Docker itself, but then there is this cilium company, and this is their process architecture 175 99:59:59,999 --> 99:59:59,999 that allows you to do packet filtering even at L7 on the path level do very complex rules to kill the packets, to deny access, to deny traffic to particular application 176 99:59:59,999 --> 99:59:59,999 It's running as an agent on a Docker node, they also have a project, they insert themselves in Kubernetes kublet, and they have monitoring and CLI special hooks 177 99:59:59,999 --> 99:59:59,999 they configure almost like iptables routing, sorry, iptables filtering on the packet level, and again it's very low-latency, it inserts itself at the kernel level 178 99:59:59,999 --> 99:59:59,999 Very interesting, quite sophisticate startup, there's a very good session about it, about an hour long 179 99:59:59,999 --> 99:59:59,999 The marketing slides from Docker 180 99:59:59,999 --> 99:59:59,999 Docker promotes... anyone from Docker here? 181 99:59:59,999 --> 99:59:59,999 There's none from Dell... none from Docker... 182 99:59:59,999 --> 99:59:59,999 The guy from Puppet left... 183 99:59:59,999 --> 99:59:59,999 No vendors? 184 99:59:59,999 --> 99:59:59,999 This is the marketing slide we discussed, it looks like everything is running out of the box, I cannot, I'm not a user of the enterprise edition of Docker... 185 99:59:59,999 --> 99:59:59,999 But they really made significant effort to add aspects like RBAC, trusted registry, your on-site completely secured key manages source for container images 186 99:59:59,999 --> 99:59:59,999 That avoid MITM attacks substituting your app's code 187 99:59:59,999 --> 99:59:59,999 There's a lot of effort put in place, to just run compliance requirements out of the box with Docker Enterprise Edition 188 99:59:59,999 --> 99:59:59,999 You can basically label a particular container or control group and allow some particular users that are LDAP authenticated, not just the local user 189 99:59:59,999 --> 99:59:59,999 that runs user, but particular LDAP user to do some actions against control groups of containers 190 99:59:59,999 --> 99:59:59,999 or hosts themselves, so it's very definable 191 99:59:59,999 --> 99:59:59,999 Nothing like that actually in Kubernetes, there are similar things in the cloud, but again not as granular. 192 99:59:59,999 --> 99:59:59,999 The best practices of hardening Linux containers, reducing your attack surfaces, very simple, right? 193 99:59:59,999 --> 99:59:59,999 Remember the configuration drift story that we started with... 194 99:59:59,999 --> 99:59:59,999 Lots of ports that who knows what they are doing there 195 99:59:59,999 --> 99:59:59,999 I tend to isolate not only as on the development, staging, user-acceptance and production, but also on the risk, so try to containerise 196 99:59:59,999 --> 99:59:59,999 this is best practice, if anyone worked at a bank, there are business criticalities, so you have to go like, BC5 to top-most one, like, do-or-die go fix it, to BC1, like, ok, let's leave it, if it's down for one week, none will care about it 197 99:59:59,999 --> 99:59:59,999 Do this on the risk, do this on the exposure, and then try to scrutinise the security one by one 198 99:59:59,999 --> 99:59:59,999 Create like a structured tree of decisions, what's my most business critical, what are the exposures? what are the attack vectors? 199 99:59:59,999 --> 99:59:59,999 Apply and enable all security-relevant configuration options 200 99:59:59,999 --> 99:59:59,999 if you already pay for Docker enterprise edition, or if you already have something enabled by default when you install it, do not temper with it 201 99:59:59,999 --> 99:59:59,999 Keep everything up to date 202 99:59:59,999 --> 99:59:59,999 Like the gentleman from Puppet 203 99:59:59,999 --> 99:59:59,999 Regularly test security recommendations 1~4 204 99:59:59,999 --> 99:59:59,999 Everything you do, put an Outlook calnedar 205 99:59:59,999 --> 99:59:59,999 It's absolutely normal to do, put an Outlook calendar reminder once in 2 months, just go and sanity-check, did we change anything? Did we miss anything? 206 99:59:59,999 --> 99:59:59,999 Remember, complexity breeds insecurity 207 99:59:59,999 --> 99:59:59,999 The more complex, the more sophisticated set of scripts are, the more the probability that there will be something hardcoded or something that your dev team will inherit from you 208 99:59:59,999 --> 99:59:59,999 and it will breed the attack vector exposure 209 99:59:59,999 --> 99:59:59,999 Two very cool talks that I've seen: 210 99:59:59,999 --> 99:59:59,999 One was on Enigma, and it's been from a guy who works for Mozilla 211 99:59:59,999 --> 99:59:59,999 And they push a lot of releases, it's pretty much weekly stable 212 99:59:59,999 --> 99:59:59,999 Of course they use the pipeline, they define the baseline for their security, what should be our central, focal point for every release 213 99:59:59,999 --> 99:59:59,999 If they do add-ons, it's much more effort to control add-ons to Firefox, because it's a community thing 214 99:59:59,999 --> 99:59:59,999 Developer submits an add-on, you have to still go through the same pipeline, because it's part of the dependency release for the core browser 215 99:59:59,999 --> 99:59:59,999 They write tests and they insert them into the CI/CD pipeline 216 99:59:59,999 --> 99:59:59,999 the security baseline for them, for Mozilla browser, is something like that 217 99:59:59,999 --> 99:59:59,999 They define scores, what would be the build score for Jenkins to use 218 99:59:59,999 --> 99:59:59,999 They need to pass some may-fail, some have-to-pass, they define the baseline, 219 99:59:59,999 --> 99:59:59,999 They write tests using ZAP, I think it's Zed Attack Proxy 220 99:59:59,999 --> 99:59:59,999 One of their focal points is ZAP, and ZAP has integration plugin for Jenkins 221 99:59:59,999 --> 99:59:59,999 It's been released not too far ago 222 99:59:59,999 --> 99:59:59,999 It's already been, it's quite aggressively, there are only 15 installs 223 99:59:59,999 --> 99:59:59,999 They had a separate plugin for Jenkins and then they moved to the completely new project on Jenkins plugins 224 99:59:59,999 --> 99:59:59,999 It's now currently being used about 400 installs in the world, which is not bad for security 225 99:59:59,999 --> 99:59:59,999 The idea behind it is that you are able to define a ZAP scan, 226 99:59:59,999 --> 99:59:59,999 I'm not sure if I have a slide on it 227 99:59:59,999 --> 99:59:59,999 The idea is here -- you can define a site, is it gonna be like all headers scan, let's try to exploit all possible web server vulnerabilities 228 99:59:59,999 --> 99:59:59,999 Let's crunch it with a lot of header-related blocks and see whether it crashes or not 229 99:59:59,999 --> 99:59:59,999 Some client sessions, you can test you particular app from different paths 230 99:59:59,999 --> 99:59:59,999 It orchestrates your HTTP or HTTPS scans, and it can simulate a total attack on conventional web servers 231 99:59:59,999 --> 99:59:59,999 After that they just hook it into the pipeline of Jenkins, and as you see the scan takes about like 30 seconds 232 99:59:59,999 --> 99:59:59,999 So it's not a really long scan, if you run it on a beefed-up container 233 99:59:59,999 --> 99:59:59,999 and it's a container itself, it only takes like half a minute in the entire build process 234 99:59:59,999 --> 99:59:59,999 There's a big set of different libraries that you can tie into the Jenkins testing, they would test different servers, e.g. node.js, rails, retire.js 235 99:59:59,999 --> 99:59:59,999 All these projects are there for you to test your app on your preferred IDE 236 99:59:59,999 --> 99:59:59,999 Test it against the known, current vulnerabilities, there's a full list of it. 237 99:59:59,999 --> 99:59:59,999 If you google for "awesome devsecops" you'll be able to find this list 238 99:59:59,999 --> 99:59:59,999 Once you've created the criticality, the risk to your applications, start with the threat modelling, like, is data going to be in the wrong hands? 239 99:59:59,999 --> 99:59:59,999 Or is going to be an accidental compromise or a brute-force attack? 240 99:59:59,999 --> 99:59:59,999 Then based on this logic, expand it, if it's going to be a human failure, like someone lost their laptop, 241 99:59:59,999 --> 99:59:59,999 Losing a laptop has special attention in here 242 99:59:59,999 --> 99:59:59,999 Remember this magic ~/. ? 243 99:59:59,999 --> 99:59:59,999 Do not lose laptops, or use encryption of the SSD 244 99:59:59,999 --> 99:59:59,999 If you lose the laptop, there could be a lot of things in your ~ 245 99:59:59,999 --> 99:59:59,999 Focus on code, as early as possible, ensure that not MD5, but SHA-256 is used 246 99:59:59,999 --> 99:59:59,999 Ensure that code repositories are private, not public, especially if you hard-code security artefacts into it 247 99:59:59,999 --> 99:59:59,999 Make sure that the orchestration involves all the testing we've discussed 248 99:59:59,999 --> 99:59:59,999 there are many other cool things that I wanted to share 249 99:59:59,999 --> 99:59:59,999 this is something that I've tried today -- 3pm today, on showdone, I've looked at ex-hudson response HTTP 250 99:59:59,999 --> 99:59:59,999 and I came up with 5 Jenkins that are completely unprotected with password 251 99:59:59,999 --> 99:59:59,999 Like, totally! 252 99:59:59,999 --> 99:59:59,999 I've logged into one of them, and it runs on AWS 253 99:59:59,999 --> 99:59:59,999 Guys actually run it in production, there was like Arun Kumar daily 254 99:59:59,999 --> 99:59:59,999 I've looked him up on LinkedIn, he's actually working for TATA consultancy 255 99:59:59,999 --> 99:59:59,999 I never thought it would be that easy! 256 99:59:59,999 --> 99:59:59,999 Please make sure, all these aspects are reviewed at least twice a year 257 99:59:59,999 --> 99:59:59,999 Or every 2 months, return and sanity-check what's the security exposure 258 99:59:59,999 --> 99:59:59,999 Employees, who loves gist? 259 99:59:59,999 --> 99:59:59,999 Make sure at least it's not Google-crawlable, ok? 260 99:59:59,999 --> 99:59:59,999 Slack tokens are there, AWS creds as we discussed... 261 99:59:59,999 --> 99:59:59,999 One good advice -- Jenkins has to be segmented, if you run it on AWS, it has to be in network ACL, it has to be segmented to only allow access from the known set of hosts 262 99:59:59,999 --> 99:59:59,999 The core take-away form the Jay Kumar example 263 99:59:59,999 --> 99:59:59,999 Everything should be all-across authenticated 264 99:59:59,999 --> 99:59:59,999 I'll file it up in terms of links and everything, short link is bit.ly/sg-docket-security 265 99:59:59,999 --> 99:59:59,999 I spent about 15~20 hours watching these videos and going through these PDFs, a lot of cool material, reach out to me on LinkedIn, if you have any ideas, I'll be happy to learn from you! 266 99:59:59,999 --> 99:59:59,999 That's a big area, we are not experts, let's move on together! 267 99:59:59,999 --> 99:59:59,999 There's a devsecops group in Singapore... 268 99:59:59,999 --> 99:59:59,999 Also did a conference in Feb 269 99:59:59,999 --> 99:59:59,999 Am I in the wrong room? 270 99:59:59,999 --> 99:59:59,999 It's a way I structure my thoughts 271 99:59:59,999 --> 99:59:59,999 Any questions? 272 99:59:59,999 --> 99:59:59,999 [inaudible] 273 99:59:59,999 --> 99:59:59,999 vulnez.com 274 99:59:59,999 --> 99:59:59,999 website looks brilliant 275 99:59:59,999 --> 99:59:59,999 [inaudible] 276 99:59:59,999 --> 99:59:59,999 they have an API 277 99:59:59,999 --> 99:59:59,999 Outlook notification doesn't scale 278 99:59:59,999 --> 99:59:59,999 Scan packages... 279 99:59:59,999 --> 99:59:59,999 Let me correct myself, once is 2 months is reviewing the process 280 99:59:59,999 --> 99:59:59,999 not really checking it once in 2 month, that has to be every build 281 99:59:59,999 --> 99:59:59,999 vulnhub 282 99:59:59,999 --> 99:59:59,999 Thanks for that, and the 2nd one you proposed is SourceClear, right? 283 99:59:59,999 --> 99:59:59,999 Thanks for these 284 99:59:59,999 --> 99:59:59,999 What's the name of the project? 285 99:59:59,999 --> 99:59:59,999 black duck 286 99:59:59,999 --> 99:59:59,999 What sort of applications, what sort of language do you use it for? 287 99:59:59,999 --> 99:59:59,999 Java, allright 288 99:59:59,999 --> 99:59:59,999 I love it 289 99:59:59,999 --> 99:59:59,999 There's twistlock 290 99:59:59,999 --> 99:59:59,999 Are these, these companies are more likely to be startups, they are not long-term? 291 99:59:59,999 --> 99:59:59,999 Twistlock and Aqua 292 99:59:59,999 --> 99:59:59,999 All these websites they are brilliant 293 99:59:59,999 --> 99:59:59,999 Any other questions? 294 99:59:59,999 --> 99:59:59,999 Thank you!