WEBVTT 00:00:30.206 --> 00:00:37.260 Hello and good evening on day two of the chaos communication Camp 2023 Translated by {Yang}{Li} (ITKST56 course assignment at JYU.FI)] 00:00:37.260 --> 00:00:42.187 it's late in the evening this is meleeway stage in case you're wondering 00:00:42.230 --> 00:00:48.176 and the next talk is going to be about incident report responses 00:00:48.476 --> 00:00:59.520 so if you're curious about how to even get there to have an incident response how you could prepare for an incident response and how you could support a new organization 00:00:59.520 --> 00:01:07.258 uh, the incident response team in doing the job and trying to fix whatever broke 00:01:07.258 --> 00:01:11.677 let's put it that way um we have the right talk for you 00:01:11.677 --> 00:01:17.352 this is stories from the life of an incident from incident responders Harry and Chris 00:01:17.352 --> 00:01:23.500 please a very warm Round of Applause [Applause] 00:01:28.925 --> 00:01:36.675 so, good evening and thank you for joining us today um we will tell you a little bit of our 00:01:36.675 --> 00:01:43.664 life as incident responders and I'm Chris I did my computer science 00:01:43.664 --> 00:01:48.784 studies at the University of alang and Nuremberg I do this security stuff for 00:01:48.784 --> 00:01:55.394 over 10 years now so my CV is a little bit longer at the moment I'm a detection 00:01:55.415 --> 00:02:01.425 engineer before that I was a long time working in dfir so digital forensic incident 00:02:01.425 --> 00:02:06.620 response in different organizations and 00:02:07.411 --> 00:02:12.388 yeah I'm Harryr I studied electrical and computer engineering at RWTH 00:02:12.395 --> 00:02:18.165 University and I played a lot of CTF and did some hacking stuff at chaos computer club RWTH 00:02:18.165 --> 00:02:24.523 during my masters I worked at x41 dsac doing pen testing patch analysis 00:02:24.589 --> 00:02:32.359 so I also have some kind of offensive security background on for around one year now I'm working at G data Advanced 00:02:32.359 --> 00:02:36.619 analytics doing digital forensics and incident handling 00:02:38.800 --> 00:02:45.390 first Christian will give you a short introduction and then he will tell you how a classical ransomware attack looks 00:02:45.390 --> 00:02:51.970 like and in the second part of the talk I will tell you how the incident 00:02:51.970 --> 00:02:58.167 responders work and what you can do in advance to make it go as smooth as possible and support the incident 00:02:58.167 --> 00:03:05.350 response team so as Harryr told you I will probably 00:03:05.350 --> 00:03:12.290 we'll talk about ransomware because the customers we usually have are small and 00:03:12.290 --> 00:03:17.543 medium-sized businesses universities and hospitals and those are regularly 00:03:17.543 --> 00:03:23.268 unfortunately regularly hit by um um 00:03:24.170 --> 00:03:29.557 ransomware gangs the main reason for this and that's if you heard the last 00:03:29.557 --> 00:03:35.960 talk um why they maybe not that responsive 00:03:35.960 --> 00:03:42.580 and are not so interested in they just lack the resources so the manpower to do 00:03:42.580 --> 00:03:48.424 uh proper security measurements to secure their systems especially in in erm 00:03:48.424 --> 00:03:53.618 situations where you are for example in a hospital have medical devices 00:03:53.618 --> 00:03:59.378 um which where you cannot simply install an AV on or even patch the system 00:03:59.378 --> 00:04:07.321 because you lose the certification as a medical device then but also in in 00:04:07.321 --> 00:04:12.953 companies manufacturing companies on the shop floor we're talking about systems 00:04:12.953 --> 00:04:21.292 that have run times of 25 plus years so if you look back now 2023 00:04:21.292 --> 00:04:26.823 we're talking about XP and older systems fun fact I was in a ransomware case and 00:04:26.823 --> 00:04:34.230 Wannacry in 2017 when I got a call from from a person from the shop floor 00:04:34.230 --> 00:04:38.000 asking me if we have a nt4 expert, um 00:04:40.200 --> 00:04:47.380 that can tell us if WannaCry is affecting nt4 of course you don't need 00:04:47.380 --> 00:04:54.710 to be a expert for NT-4 this one requires of course not affecting nt4 00:04:54.710 --> 00:04:59.602 systems so due to the time uh slot we thought 00:04:59.602 --> 00:05:04.915 memes are the best way to to tell you those stories and we have a lot of them 00:05:06.453 --> 00:05:12.822 so in the first uh um section I tell you a little bit of how an attack Works 00:05:12.822 --> 00:05:21.620 um there are a lot of different possibilities how you can describe and how to structure the how an attack works 00:05:22.257 --> 00:05:28.993 there's the miter attack framework for example there was for example a talk Yesterday by Maker Salko 00:05:28.993 --> 00:05:34.854 um here on the stage there's the original cyber kill chain from from Lockheed Martin you have 00:05:37.190 --> 00:05:42.480 stuff from from companies like Mandy and their targeted the tech life cycle but 00:05:42.480 --> 00:05:47.550 that's all in my opinion two two fine-grained it's that's the reason I 00:05:47.550 --> 00:05:53.275 just take three simple steps yeah get a foothold in the door 00:05:53.275 --> 00:06:00.645 look move play around and cash out those three uh I will just go over 00:06:03.141 --> 00:06:07.835 so start with uh get a foot in the door so normally we 00:06:07.835 --> 00:06:14.756 see three ways how attackers can can get into the environment in the ransomware 00:06:14.756 --> 00:06:20.655 cases you have vulnerabilities in uh remote uh internet facing systems you 00:06:20.655 --> 00:06:25.875 have the remote Services itself and you have malware 00:06:26.712 --> 00:06:35.507 starting with the with the the vulnerabilities and um I just looked uh up the last four 00:06:35.507 --> 00:06:42.600 years and maybe somebody remembers netscaler the the so-called Citrix 00:06:42.600 --> 00:06:49.789 vulnerability in December 2019 um it was released mid of uh 2019 uh 00:06:49.789 --> 00:06:55.889 December 2019 the first POC publicly available POC was in beginning of 00:06:55.889 --> 00:07:03.293 January and the patch was available in middle of January so there was a round one week to one and a half weeks between 00:07:03.293 --> 00:07:10.494 a public proof of concept for the vulnerability and uh patch for the vulnerability and what we saw 00:07:10.494 --> 00:07:17.194 during 2020 a lot of companies patched but the patch didn't remove the the 00:07:17.194 --> 00:07:25.469 compromise so they were already compromised and um yeah with it with the patch they 00:07:25.469 --> 00:07:31.114 didn't remove the compromise so what we found what we could provable 00:07:31.114 --> 00:07:36.184 see or proof evidence for uh was nine 00:07:36.184 --> 00:07:42.286 month uh customer was breached after nine months using this this vulnerability 00:07:43.176 --> 00:07:51.434 and we had other customers where we could see that the netscaler was affected after two years but we couldn't 00:07:51.434 --> 00:08:00.730 prove that this this compromise was the reason for the actual ransomware case 00:08:00.275 --> 00:08:04.914 and of course such vulnerabilities happen not that often 00:08:06.295 --> 00:08:13.350 yeah so 2021 gave us uh hafnium exchange 00:08:13.350 --> 00:08:18.736 vulnerability also a similar situation the patch 00:08:18.736 --> 00:08:25.406 appeared as an out-of-band patch from Microsoft on a Tuesday evening 10 o'clock in German time 00:08:26.479 --> 00:08:32.529 we saw during our uh incidents or the the assessments we did that 00:08:34.476 --> 00:08:41.516 um the first exploit exploitation attempts were seen on Wednesday in the morning at 00:08:41.516 --> 00:08:50.308 5:00 am so around seven eight hours later um I know one guy who could patch 00:08:50.308 --> 00:08:56.691 because he was online when the patch was released otherwise Germany was unable to patch in 00:08:56.691 --> 00:09:04.149 time and of course we can go on with 2021 proxy shell also 00:09:04.149 --> 00:09:10.390 exchange vulnerability proxy nutshell also exchange vulnerability 00:09:10.390 --> 00:09:16.367 we have uh in 2022 VMware Horizon the the virtual desktop infrastructure 00:09:16.367 --> 00:09:23.627 from VMware just to name also open source stuff Zimbra a collaboration platform 00:09:23.627 --> 00:09:28.922 including an email server uh has had a vulnerability actually the vulnerability 00:09:28.922 --> 00:09:34.675 was in cpio from 2015 I think which led 00:09:34.675 --> 00:09:40.164 to a compromise using via email so you send an email 00:09:40.164 --> 00:09:48.387 with a cpio with a specially crafted archive file and you could drop a web 00:09:48.387 --> 00:09:55.947 shell in one of the directories yeah you have of course 40 OS which is a 00:09:55.947 --> 00:10:02.690 40 gate VPN and firewall operating system 00:10:03.220 --> 00:10:08.250 and if you read the news we start at the beginning again 00:10:08.251 --> 00:10:15.121 netscaler had some issues several weeks ago according to foxIT we have 1900 00:10:15.121 --> 00:10:21.545 still unpatched net scalers worldwide how many patched 00:10:22.393 --> 00:10:27.743 was netscale has exists that um have not been checked for compromise we 00:10:27.743 --> 00:10:32.580 don't know of course so that will be a nice year probably 00:10:33.728 --> 00:10:41.564 um so what can you can you do against this kind of of attack vector patch your systems is one thing as you 00:10:41.810 --> 00:10:49.378 see this that doesn't lead to the the um or what you need to do afterwards in 00:10:49.378 --> 00:10:57.354 such cases you need to check your systems for possible compromise 00:10:57.354 --> 00:11:03.973 that is important to reduce this I highly suggest put your 00:11:03.973 --> 00:11:11.583 uh Services behind some VPN so that only people who already have 00:11:11.583 --> 00:11:17.540 connection to the VPN um can access your services or the services 00:11:17.540 --> 00:11:22.649 they need and that would reduce the attack surface 00:11:22.649 --> 00:11:28.289 at least to the VPN server so but I 00:11:28.289 --> 00:11:32.996 of course we can also think about remote services without vulnerabilities 00:11:34.661 --> 00:11:41.591 um there can be configuration mistakes so the admin does something wrong there can 00:11:41.591 --> 00:11:50.339 be insecure default configurations like this um I don't know if you know it but the 00:11:50.339 --> 00:11:55.614 local admins or the administrators on the Windows system are are 00:11:55.614 --> 00:12:02.101 automatically in the remote desktop users group you know and so 00:12:02.101 --> 00:12:08.428 we had several cases especially in the beginning of the pandemic when everybody moved from uh to the home offices and 00:12:08.428 --> 00:12:15.545 they needed to put people fast in the position to to access their the assist 00:12:15.545 --> 00:12:22.125 the internal systems again they just put a RDP server on the internet and hope for the best 00:12:25.136 --> 00:12:29.767 um additionally if you put services on the internet of course brute forcing and 00:12:29.767 --> 00:12:35.947 credential uh stuffing are attacks that are possible so brute forcing just trying the the 00:12:37.115 --> 00:12:42.195 username and password combinations uh credential stuffing using already leaked 00:12:42.195 --> 00:12:47.636 passwords or credentials from leaks you find on the internet 00:12:48.536 --> 00:12:53.923 what you can do about this kind of of attack Vector is uh just as I said use 00:12:53.923 --> 00:13:00.912 multi-factor Authentication and reduce the attack surface as in the 00:13:00.912 --> 00:13:06.695 point with the vulnerabilities before by moving the services behind a VPN and 00:13:06.695 --> 00:13:09.691 then use multi-factor authentication on VPN of course 00:13:12.791 --> 00:13:18.141 the last Vector that we see normally that the attackers can get in the 00:13:18.141 --> 00:13:23.887 network is malware we all know this about 00:13:23.887 --> 00:13:28.658 those funny emails you get with the attachments 00:13:28.658 --> 00:13:35.310 um include that have either Word documents 00:13:35.310 --> 00:13:41.764 attached either zip files with with Visual Basic scripts javascripts and 00:13:41.764 --> 00:13:47.344 what you can get isos you see a lot these days 00:13:48.850 --> 00:13:54.210 um or what you can also have that you can have just a link inside the email and 00:13:54.210 --> 00:14:01.901 you download the respective file from some some shady file sharing website 00:14:03.381 --> 00:14:09.435 um what we saw over the last year was uh USB sticks again funnily 00:14:10.744 --> 00:14:16.484 um I'm not sure if you have heard about raspberry Robin which is a malware that 00:14:16.484 --> 00:14:26.427 warms via USB sticks um but I haven't seen it as a vector for 00:14:27.234 --> 00:14:31.784 ransomware yet on my own but there are people who said that it's 00:14:33.220 --> 00:14:37.770 an initial access broker for some of the ransomware gangs 00:14:38.734 --> 00:14:42.884 so what can you do about this if you think the 00:14:45.169 --> 00:14:53.420 you can of course ban simply some file extensions in your mail server or you 00:14:53.723 --> 00:15:00.953 change the file Association types in your operating system meaning that you 00:15:00.953 --> 00:15:06.274 don't open the JavaScript and Visual Basic script files using for example the 00:15:06.274 --> 00:15:11.610 windows scripting host but open it with notepad and that will 00:15:11.610 --> 00:15:14.757 of course some people will be 00:15:18.146 --> 00:15:23.600 uh some people will think about what this this is then and ask the IT guys 00:15:23.600 --> 00:15:27.408 but it's better than running the the script itself 00:15:28.260 --> 00:15:35.110 one thing I I I don't like to to say it but keep your AV updated 00:15:35.547 --> 00:15:39.791 um uh this is one thing keep it updated and read the logs 00:15:40.722 --> 00:15:46.660 we see a lot of incidents where we see that the already 00:15:46.544 --> 00:15:51.714 days or weeks before we you can could have seen that there's something going 00:15:51.714 --> 00:15:59.612 on in your network yeah and if you see malware in your AV logs 00:16:00.476 --> 00:16:05.846 then react to it just check it you don't know how long this AV this malware has 00:16:05.846 --> 00:16:11.302 been on your system the thing is that 00:16:11.302 --> 00:16:16.792 just because you're AV detected it now it might have been get received an 00:16:16.792 --> 00:16:22.287 update for its signatures and the malware was active for days or weeks 00:16:22.287 --> 00:16:27.597 before so when they are inside 00:16:29.770 --> 00:16:34.300 then they usually look move and play around a little bit 00:16:36.200 --> 00:16:41.420 so when they look around what they do 00:16:41.420 --> 00:16:47.612 is they they enumerate AD they do Ports scan the you they search for 00:16:47.612 --> 00:16:54.388 vulnerabilities they check uh what they how they can escalate 00:16:54.388 --> 00:16:59.826 their privileges they try to find credentials 00:16:59.826 --> 00:17:03.871 um Kerber roasting we heard in the talk before for example is this one thing 00:17:07.890 --> 00:17:11.700 um they try to identify accounts you around 00:17:11.700 --> 00:17:16.981 have running on your systems they can use they can get the credentials from and you reuse 00:17:18.600 --> 00:17:24.150 and for that reason one of the most important things I think is that you 00:17:24.150 --> 00:17:33.254 have a principle of least privileges in your environment so only what a account needs 00:17:33.365 --> 00:17:38.385 you should be able to do you should use dedicated service 00:17:38.385 --> 00:17:43.661 accounts for your services of course and just for your information 00:17:44.397 --> 00:17:50.470 um service account is not an account that has a SVC underscore in front of it 00:17:50.298 --> 00:17:56.880 and is otherwise a normal user account um there I exist educated service 00:17:56.930 --> 00:18:01.453 accounts in Windows environments so use them use strong passwords I still 00:18:01.469 --> 00:18:06.419 I today I can I know companies who still use eight 00:18:06.419 --> 00:18:13.342 character long passwords um I think that's 20 minutes on a decent 00:18:13.342 --> 00:18:18.671 graphics card today so use strong passwords length matters 00:18:19.665 --> 00:18:25.455 12 plus characters is the minimum in my opinion 00:18:26.617 --> 00:18:31.170 and don't reuse passwords especially on your 00:18:31.622 --> 00:18:42.329 systems the local administrator especially in small and medium-sized businesses you see a lot that people use 00:18:42.329 --> 00:18:47.140 the same password for all local administrators so if I have it on one 00:18:47.140 --> 00:18:51.390 system I have the whole company don't reuse it 00:18:56.895 --> 00:19:00.895 um yeah [Audience LoL] 00:19:04.550 --> 00:19:12.500 so when they they move around in your network using either a password hashes they found valid credentials they they 00:19:12.500 --> 00:19:18.530 discovered somewhere vulnerabilities are also used to to move around in your 00:19:18.530 --> 00:19:22.915 environment they try to to establish persistence mechanisms here we heard 00:19:22.915 --> 00:19:28.579 also in the talk before about the C2 channels command and control channels they use 00:19:28.621 --> 00:19:36.132 um they install in some cases directly any desk team viewer and or other remote 00:19:36.132 --> 00:19:43.572 control software or sometimes they also use tunneling softwares like ngrok or recently they 00:19:43.572 --> 00:19:49.744 started using cloudflare G um and this is what you need to do is 00:19:49.744 --> 00:19:55.906 prop have a proper Network segmentation and with the proper Network segmentation I don't talk about subnetting 00:19:57.492 --> 00:20:04.302 subnetting means you just have different subnets you need to have a firewall between them 00:20:04.302 --> 00:20:08.143 and you have need to have rules between them that 00:20:09.893 --> 00:20:16.823 restrict access between your your subnets and one thing is especially important 00:20:17.881 --> 00:20:24.711 please keep or use Network segmentation to 00:20:24.711 --> 00:20:30.225 restrict the access to your backup and your Management Systems as far as 00:20:30.225 --> 00:20:36.197 possible we see a lot especially as I said in the small and medium-sized businesses or in 00:20:36.197 --> 00:20:40.183 the other organizations we we have as customers that 00:20:42.448 --> 00:20:46.873 yeah they have they tell us in in in in in in workshops yeah we have 00:20:46.873 --> 00:20:54.333 a network segmentation every building is one segment and on the question yeah you 00:20:54.333 --> 00:20:59.603 can move between in a segment you can access everything yes you can and 00:21:00.416 --> 00:21:05.911 between the the buildings you can have also a firewall and you cannot access 00:21:05.911 --> 00:21:11.930 anything no you can access everything and also your your uh VMware Management 00:21:11.930 --> 00:21:17.420 console oh yes yes we can so everybody can access it yes of course and that 00:21:17.461 --> 00:21:21.761 doesn't work so 00:21:26.913 --> 00:21:30.263 um when they play around they normally try 00:21:30.263 --> 00:21:38.288 to to gain more privileges so privilege escalation is assisting local privilege escalations normally but also using 00:21:38.288 --> 00:21:43.440 vulnerabilities um or misconfigurations insecure default 00:21:43.440 --> 00:21:48.981 configurations um my personal favorites are Group 00:21:48.981 --> 00:21:53.211 Policy preferences or passwords in group policy preferences 00:21:54.686 --> 00:22:01.890 this is no longer possible since I think 2014 to put passwords in group policy 00:22:01.890 --> 00:22:07.994 preferences however if you had your password stored in those preferences 00:22:07.994 --> 00:22:16.161 before the patch in 2015 2014 then they're still there and yeah there 00:22:16.161 --> 00:22:20.932 are AES encrypted but the encryption key is on the Microsoft website 00:22:21.783 --> 00:22:26.663 so you can just download it and just take it and decrypt the keys 00:22:28.834 --> 00:22:33.484 um then during that phase they also try to disable your security measures 00:22:34.914 --> 00:22:41.516 the thing you can do is of course patch your system so you know try to get your 00:22:41.516 --> 00:22:46.655 availabilities out of of this equation can try to configure your systems in a 00:22:46.655 --> 00:22:52.580 secure way this is not always possible due to some shitty uh third-party software 00:22:52.897 --> 00:22:58.777 and keep your AV um updated and please please as I said 00:22:58.777 --> 00:23:02.894 already check the locks and act accordingly 00:23:05.511 --> 00:23:09.741 so in the last phase they cash out that's when when they 00:23:10.707 --> 00:23:16.881 start using a uh being your backup service so they copy 00:23:16.881 --> 00:23:21.906 data from your your environment using um 00:23:21.906 --> 00:23:27.574 file sharing platforms for example yeah Mega and set was was once the thing we 00:23:27.574 --> 00:23:33.155 transfer we had already uh every every other file sharing platform you you can 00:23:33.155 --> 00:23:38.891 think about is a possible way to exfiltrate data they also use their 00:23:38.891 --> 00:23:47.270 their C2 communication channels so sometimes you they also they just use the the possibilities in any desk or in 00:23:47.270 --> 00:23:53.014 in RDP clients or they use uh file transfer protocols 00:23:53.014 --> 00:23:57.014 like um SS SFTP um 00:23:59.670 --> 00:24:06.260 we saw for example in one case that they try to install filezilla on every machine they had access to 00:24:06.260 --> 00:24:11.705 um because on the first one it didn't work on the second it didn't work on the third it didn't work yet because SFTP 00:24:11.705 --> 00:24:17.940 was blocked uh outgoing and that is one of the things you can do to to prevent 00:24:17.940 --> 00:24:21.765 exfiltration block at least 00:24:24.607 --> 00:24:30.456 protocols you know that you don't need in your environment and proper Network segmentation of 00:24:30.456 --> 00:24:39.024 course is a general thing so in the last step that's when they 00:24:39.024 --> 00:24:44.781 start the encryption um they're running the ransomware or 00:24:44.781 --> 00:24:50.171 normally they are have domain admins at that point so they can run it on all 00:24:50.171 --> 00:24:57.284 domain connected systems they can also disable of course when they are domain admin they can disable 00:24:57.461 --> 00:25:04.553 the AV before they they start to run somewhere ransomware's today disable services like 00:25:04.553 --> 00:25:10.303 databases and such things so that they have the full power of the machine for 00:25:10.303 --> 00:25:14.590 the uh for the encryption 00:25:15.899 --> 00:25:23.001 um if you get lucky not how everything works perfectly because they use 00:25:23.001 --> 00:25:29.327 group names and windows is especially picky when you have a non-english uh 00:25:32.344 --> 00:25:39.434 windows installed for example in Germany the the group everybody is called yida 00:25:40.319 --> 00:25:47.322 and we had cases where the ransomware didn't really work that well because they couldn't 00:25:47.322 --> 00:25:51.322 change the permissions of the files first um 00:25:53.436 --> 00:25:59.278 they use different encryption schemas normally they they come with the 00:25:59.278 --> 00:26:05.740 asymmetric and the symmetric encryption type the asymmetrics or public key cryptography the public key comes with 00:26:05.740 --> 00:26:12.254 the ransomware and is used to encrypt the symmetric keys they generate on in 00:26:12.254 --> 00:26:20.112 your environment depending on the ransomware they they generate one key for each system or even one key for each 00:26:20.112 --> 00:26:25.876 file it depends a little bit on the on the ransomware how it works but that's 00:26:25.876 --> 00:26:32.998 the usual thing they use um I would never count on the the fact 00:26:32.998 --> 00:26:39.521 that there are possibly maybe there could be 00:26:39.521 --> 00:26:47.544 decryptable uh things um in in my opinion in my uh in my world 00:26:49.120 --> 00:26:55.200 The ransomware Gangs have learned and used the standard Microsoft Windows or 00:26:55.200 --> 00:26:59.232 some other publicly available libraries to to do the encryption 00:27:00.770 --> 00:27:09.224 they executed by a remote tools like PSX Powershell or some use 00:27:09.224 --> 00:27:14.983 gpos group policies to execute the ransomware on every 00:27:14.983 --> 00:27:21.153 machines they they connected to the domain and what can you do about this no it's 00:27:21.153 --> 00:27:26.533 it's hard but the the most important thing is have online backups offline 00:27:26.533 --> 00:27:33.817 backup sorry thanks you you see you see off online backups 00:27:33.817 --> 00:27:39.414 are not that are great but not that great offline backups is the most important this is the most important 00:27:39.414 --> 00:27:45.388 thing so um don't have it connected to your environment 00:27:45.969 --> 00:27:52.207 the the the USB disk on the system is not offline backup 00:27:55.594 --> 00:28:00.794 um in my opinion if you see that something is is still encrypting 00:28:01.440 --> 00:28:07.630 I I'm I'm always hesitant to say shut down the system because you can break 00:28:07.630 --> 00:28:15.529 the encryption and maybe the file that is currently in under encryption or the files will never be decryptable if you 00:28:15.529 --> 00:28:23.588 want to buy a decrypter um or gather the cryptos through some 00:28:25.669 --> 00:28:32.149 discussions with the with the ransomware guys um if it's a VM just suspend it and 00:28:32.149 --> 00:28:38.351 that's it and if everything is already encrypted 00:28:39.205 --> 00:28:43.575 keep cool and call your incident responder 00:28:51.306 --> 00:28:58.606 so now let's talk about incident response what happens when it's already too late and what can you do to support 00:28:58.606 --> 00:29:04.110 your incident response team at first the things I'll say in this chapter are 00:29:04.110 --> 00:29:10.549 for our company and how we work so other companies might work a little bit different than that 00:29:12.792 --> 00:29:17.022 first for some reason incidents always come on Friday afternoon 00:29:18.950 --> 00:29:23.950 so some customers think it is a good idea to try to solve a case by 00:29:23.950 --> 00:29:30.785 themselves maybe until the end of the week and if they didn't solve it until the end of the week they call the 00:29:30.785 --> 00:29:35.876 incident response team please don't do that it doesn't help your company and it 00:29:35.876 --> 00:29:43.792 doesn't make your incident Response Team happy to have to work on the weekend and in addition the longer you wait with 00:29:43.792 --> 00:29:50.859 calling the incident Response Team the longer the incident response will take and the more complicated forensics will 00:29:50.859 --> 00:29:56.523 be because you have lock retention times while trying to do stuff by yourself 00:29:56.702 --> 00:30:01.572 maybe you modify some of the systems and it becomes much more harder to do 00:30:02.397 --> 00:30:08.127 precise forensics so what happens on our site when such a 00:30:08.129 --> 00:30:13.589 new incident ticket arrives the first thing we do is team internal coordination 00:30:13.589 --> 00:30:18.935 so we discussed do we have enough people do we have a person for each role in our 00:30:18.935 --> 00:30:25.084 team we have three roads incident handling forensics analyst and Mayweather analyst 00:30:26.254 --> 00:30:32.524 so first let's talk about incident handling incident Handler is responsible for all 00:30:32.524 --> 00:30:39.791 the tasks that our customer facing and the first point is always get the customer out of that headless chicken 00:30:39.791 --> 00:30:45.121 mode like we call it because when an incident comes at our customer site 00:30:45.121 --> 00:30:52.238 everyone is like running around in so-called like headless chicken doing something but not doing anything helpful 00:30:53.402 --> 00:31:00.952 so this is always the first task for the incident Handler Handler structure the customer do meetings and then do all the 00:31:00.952 --> 00:31:06.612 relevant decisions leading to a secure emergency operation mode that means in 00:31:06.612 --> 00:31:13.484 this case that you have working core infrastructure so working domain controller maybe a working email server 00:31:13.484 --> 00:31:19.302 and whatever you need or whatever you define as very business critical systems 00:31:21.321 --> 00:31:28.785 let's go a little bit more in detail probably the first measure will be to cut off the internet connection because 00:31:28.785 --> 00:31:33.163 you just buy you a lot of time with doing that no matter how many back doors 00:31:33.163 --> 00:31:38.830 the attackers placed in the network if you cut off the internet connection the attackers can't access their back doors 00:31:38.949 --> 00:31:46.279 anymore and then you will start to rebuild your network you will Define everything in 00:31:46.279 --> 00:31:51.317 your current infrastructure as red Network and then start building up a 00:31:51.317 --> 00:31:57.590 green network with clean systems maybe you will start with some admin workstations so that the 00:31:57.590 --> 00:32:05.419 administrators can work properly and then we go through a prioritized system list and build up the most 00:32:05.419 --> 00:32:11.420 important systems this can be of course like I said domain controllers or email 00:32:11.420 --> 00:32:17.839 servers but also whatever is important for your current company and for the 00:32:17.839 --> 00:32:23.075 business this is of course in a hospital something very different from a small 00:32:23.075 --> 00:32:30.742 business or maybe a university and please have such a prioritized list before your first incident because when 00:32:30.742 --> 00:32:37.693 you start discussing in an incident which system is the most important then everyone in the company will tell you 00:32:37.693 --> 00:32:44.799 something else and everyone will tell you that their system is the most important and has to be migrated and 00:32:44.799 --> 00:32:48.697 checked and analyzed at first and this isn't really helpful 00:32:50.494 --> 00:32:56.364 so during incident handling there might also be obstacles for example if you 00:32:56.364 --> 00:33:02.524 have backups and you encrypted your backups this is great but you should think about where to 00:33:02.524 --> 00:33:08.909 store that encryption key because when you store it in your password manager and the password 00:33:08.909 --> 00:33:13.747 manager database is in a VM which then gets encrypted by the ransomware well 00:33:13.747 --> 00:33:18.670 you have backups but you don't have your key to decrypt the backup so this 00:33:18.969 --> 00:33:25.809 doesn't help you very much and this is not only two for password 00:33:25.991 --> 00:33:34.231 this is also true for like everything else which you could need during an incident like contact lists or network 00:33:34.268 --> 00:33:39.168 lists and so on so 00:33:40.571 --> 00:33:47.271 working with third parties is also a task for the incident Handler this could be a data Protection Agency 00:33:47.330 --> 00:33:52.760 or the customers customers because they also often yeah have some panic and have 00:33:52.760 --> 00:33:58.021 questions which the customer maybe can't answer so this is also a talk a task for 00:33:58.021 --> 00:34:03.127 the incident Handler and of course also working with law enforcement but there's 00:34:03.177 --> 00:34:09.765 one thing to keep in mind Law Enforcement wants to do criminal investigation incident response wants to 00:34:09.765 --> 00:34:17.545 bring you back to business as soon as possible this is not necessarily the same goal but it finally makes sense to 00:34:17.571 --> 00:34:25.213 allow your incident responders to share all information with law enforcement because this is beneficial for everyone 00:34:26.652 --> 00:34:32.712 um so why do I do forensics at all one could ask well the less precise the 00:34:32.729 --> 00:34:39.189 forensic results are the more conservative the rebuild needs to be so if you don't know anything about the 00:34:39.189 --> 00:34:45.632 attack you have to rebuild everything from scratch so let's say you have maybe two week or 00:34:45.632 --> 00:34:51.086 backups and you know the initial access of the attackers was only one week ago 00:34:51.161 --> 00:34:56.351 then you could use that two weeks old backups for rebuilding which is really 00:34:56.351 --> 00:35:00.991 great but if you don't know how long the attackers were in the 00:35:01.381 --> 00:35:05.521 network then it's really hard to to say 00:35:06.991 --> 00:35:12.077 a second Point why you should do forensics is to estimate the impact 00:35:12.157 --> 00:35:18.677 especially in an early um phase of an attack it is important to know that the attackers only compromise 00:35:18.677 --> 00:35:23.769 some user devices or did they already gain domain administrative privileges 00:35:24.211 --> 00:35:30.751 because then the incident handling is completely different and of course it is also relevant to 00:35:30.751 --> 00:35:36.478 know which attack path the attack is used so you can Harden your network and 00:35:37.102 --> 00:35:42.752 improve in the future how should incident how should forensics 00:35:42.752 --> 00:35:48.789 be it should of course be correct it should be non-disruptive for the customer so they can concentrate on 00:35:48.967 --> 00:35:55.064 rebuilding their infrastructure and it should be fast what means correct you want to have a 00:35:55.211 --> 00:36:00.501 dedicated forensics analyst so you want to have incident response team which is 00:36:00.686 --> 00:36:07.016 doing incident response in their day-to-day business and you want in your forensics teams one 00:36:07.016 --> 00:36:12.910 person who can really concentrate on forensics and is not also doing the incident handling and the malware 00:36:12.910 --> 00:36:17.228 analysis and so on because then you can't really concentrate on doing the technical work 00:36:19.327 --> 00:36:25.685 it also should be non-disruptive as I said that means for our company we do remote triage collection we use the tool 00:36:25.745 --> 00:36:33.335 videos adapter for it this is an official logo of velociraptor it's a really nice tool because we just send 00:36:33.335 --> 00:36:38.639 our customers an exe file the customer just has to execute the exe file and 00:36:38.639 --> 00:36:43.899 then it collects all the logs we need all the um yeah information we need 00:36:44.836 --> 00:36:49.836 and absolute uploads it to our um to our infrastructure so we don't 00:36:49.836 --> 00:36:58.134 need to drive to the customer and like start copying hard disks for a week for weeks because this doesn't help anyone 00:36:58.493 --> 00:37:04.333 I should also said it should be fast so we have a lot of Automation and tooling 00:37:04.333 --> 00:37:10.535 to make the work really fast automatically pushing our the uploaded files we want 00:37:10.535 --> 00:37:16.702 to lose this infrastructure automatically doing reporting and so on I won't go in detail here because this 00:37:16.702 --> 00:37:22.275 is enough for a single talk about it maybe maybe on Congress let's see 00:37:23.275 --> 00:37:28.445 so what happens when we do manual forensic analysis 00:37:28.619 --> 00:37:35.709 there are some questions you normally want to answer one is of course about the vectors you want to find all vectors 00:37:35.709 --> 00:37:40.968 the attack is placed in the network of course you want to have General IUC 00:37:40.968 --> 00:37:47.529 so indicators of compromise so maybe which IP addresses did the attackers use so you can block them in the firewall or 00:37:47.529 --> 00:37:53.345 which tools did they use so you can scan over the whole it infrastructure and 00:37:53.382 --> 00:37:59.502 find further compromised systems and if you have the iocs you can also cross-correlate between cases or show 00:37:59.768 --> 00:38:05.208 them share them with law enforcement another important thing is of course 00:38:05.270 --> 00:38:13.269 lateral movement because you can use lateral movement to build a timeline of the attack so the mythology here is to 00:38:13.391 --> 00:38:20.819 look where the attackers came from and where the attackers went so if you have a system a and you see the attackers 00:38:20.819 --> 00:38:27.572 came from system B then you analyze system B and go back in time and the same you could do forward in time until 00:38:27.572 --> 00:38:34.294 you found yeah all relevant systems which were compromised and have a proper timeline of the case 00:38:36.214 --> 00:38:41.851 and other common thing in the life of an incident responder is to fall in a rabbit hole 00:38:42.226 --> 00:38:47.436 because it can be really really interesting to do forensics but yeah you need to 00:38:48.464 --> 00:38:55.038 check what the customer really wants to know or what your incident Handler really wants to know 00:38:55.038 --> 00:39:01.528 and not just analyze something and fall into the rabbit hole because this yeah 00:39:01.528 --> 00:39:08.174 really doesn't help anyone here also the 80 20 rule applies so I'd say you get like 80 percent of 00:39:08.275 --> 00:39:14.437 the relevant forensics results in 20 of the time and often this is enough for the incident Handler to do the right 00:39:14.437 --> 00:39:20.347 decisions to rebuild the customer Network customer's networks as fast as possible 00:39:21.634 --> 00:39:26.734 so let's go into reporting nobody really likes reporting 00:39:28.074 --> 00:39:34.095 but producing fancy forensics results doing fancy technical stuff is worthless 00:39:34.095 --> 00:39:40.071 if you can can't explain it to a manager nobody pays us to do fancy technical 00:39:40.071 --> 00:39:45.423 stuff nobody understands but we are paid to help our customers and explain the 00:39:45.423 --> 00:39:52.962 situation to them and to do the right decisions so yeah nobody likes reporting but it is a really really important 00:39:52.962 --> 00:39:56.818 thing so after learning the report the 00:39:56.838 --> 00:40:02.428 incident is over and all work is done right no not really because such a 00:40:02.466 --> 00:40:08.051 report normally contains a lot of recommendations what to do 00:40:08.700 --> 00:40:14.760 for the customers and as I already said incident response more or less stops 00:40:15.356 --> 00:40:20.596 when the customer is in an emergency operation mode with a core infrastructure and the most critical 00:40:20.672 --> 00:40:28.134 systems are working but there's still a lot to do and of course security is a process so 00:40:28.298 --> 00:40:32.408 there's always something to improve to stay ahead of the attackers 00:40:33.606 --> 00:40:38.036 so now let's go into a quick recap 00:40:42.251 --> 00:40:49.361 how to protect against those kind of attacks we had this uh the topics just 00:40:49.520 --> 00:40:55.220 as a recap patch your systems it's important keep them updated 00:40:56.028 --> 00:41:02.038 vulnerabilities come and go that's one thing have a sane privilege 00:41:02.212 --> 00:41:09.043 account management so use the correct user accounts for the correct for the 00:41:09.044 --> 00:41:17.864 the right task use the second two-factor authentication 00:41:17.928 --> 00:41:24.278 for remote services have a proper Network segmentation so 00:41:24.568 --> 00:41:31.301 put firewalls and access rules between your networks check your AV logs regularly and act 00:41:31.502 --> 00:41:37.374 accordingly and not really protecting your you against an attack 00:41:37.862 --> 00:41:46.334 but keeping the the symptoms a little bit um not that that devastating have 00:41:46.334 --> 00:41:52.573 offline backups so how to make incident response easier 00:41:52.756 --> 00:41:58.866 for you for yourself and for the incident Response Team first thing is don't try to solve this stuff by 00:41:58.871 --> 00:42:05.271 yourself just call the incident response team of your trust as soon as you know 00:42:05.271 --> 00:42:11.885 you have an incident the second thing is logging policy because if you have a very short lock 00:42:11.885 --> 00:42:16.928 retention for example and you only have locks for the last like two days then it 00:42:16.928 --> 00:42:22.150 might be hard to find out what happened and if you don't lock stuff at all 00:42:22.479 --> 00:42:28.999 especially in the cloud environments this is a thing where you need to look at then it might also be hard to get 00:42:29.173 --> 00:42:34.213 proper forensic results then as I said have offline contact 00:42:34.213 --> 00:42:41.026 lists because your main server is down your accounts are compromised so you can't use like your Microsoft teams or 00:42:41.026 --> 00:42:47.828 whatever you use in your company you need an alternative way to communicate to your colleagues and or 00:42:48.032 --> 00:42:54.102 employees then as I said prioritized asset list and network plans 00:42:55.168 --> 00:43:01.808 and a disaster recovery plan in best case and please all have all that in a 00:43:01.808 --> 00:43:07.565 secure place in best case on paper well nobody likes doing stuff with their 00:43:07.565 --> 00:43:11.677 trees I know but that trees don't get ransomware right 00:43:14.319 --> 00:43:21.379 so that's it from our site more or less normally this is a point where 00:43:21.379 --> 00:43:26.302 you can ask us questions but we want to spin that around and give you some 00:43:26.302 --> 00:43:31.093 questions maybe someone can guess the right answer 00:43:31.876 --> 00:43:36.976 what do you think is the shortest time to domain admin we saw an incident between initial compromise and the 00:43:36.976 --> 00:43:41.062 attackers gaining domain admin 00:43:42.619 --> 00:43:49.499 yeah it's not it somebody said 17 seconds two minutes well it's not that 00:43:49.499 --> 00:43:54.332 bad six minutes is is what we have seen in incident hmm 00:43:54.742 --> 00:44:00.323 what do you think think is the shortest lock retention on a domain controller so how long 00:44:01.207 --> 00:44:08.677 zero 10 hours a year well it is 2.5 minutes 00:44:12.866 --> 00:44:18.596 what what do you think is the highest number of domain administrator accounts 00:44:18.596 --> 00:44:26.037 we saw in an incident 200 nah 50. 00:44:26.037 --> 00:44:30.796 64 with 120 people working in IT 00:44:33.042 --> 00:44:39.672 what do you think is the longest dwell time so the longest the longest time between being initially compromised and 00:44:39.886 --> 00:44:47.568 realizing that you are compromised 60 now it's it's not that bad I hit here 00:44:47.826 --> 00:44:52.116 a lot of years well it is like around two years 00:44:53.294 --> 00:44:58.824 so well that's really it from our side no time for questions thank you 00:45:01.608 --> 00:45:05.908 if you 00:45:09.297 --> 00:45:13.697 if you have questions we will be there outside waiting come and join us 00:45:15.480 --> 00:45:21.230 thank you thank you thank you Harryr and Kris warmer applause thank you 00:45:23.964 --> 00:45:32.770 End of subtitles:[Translated by {Yang}{Li} (ITKST56 course assignment at JYU.FI)]