1 00:00:30,206 --> 00:00:37,260 Hello and good evening on day two of the chaos communication Camp 2023 Translated by {Yang}{Li} (ITKST56 course assignment at JYU.FI)] 2 00:00:37,260 --> 00:00:42,187 it's late in the evening this is meleeway stage in case you're wondering 3 00:00:42,230 --> 00:00:48,176 and the next talk is going to be about incident report responses 4 00:00:48,476 --> 00:00:59,520 so if you're curious about how to even get there to have an incident response how you could prepare for an incident response and how you could support a new organization 5 00:00:59,520 --> 00:01:07,258 uh, the incident response team in doing the job and trying to fix whatever broke 6 00:01:07,258 --> 00:01:11,677 let's put it that way um we have the right talk for you 7 00:01:11,677 --> 00:01:17,352 this is stories from the life of an incident from incident responders Harry and Chris 8 00:01:17,352 --> 00:01:23,500 please a very warm Round of Applause [Applause] 9 00:01:28,925 --> 00:01:36,675 so, good evening and thank you for joining us today um we will tell you a little bit of our 10 00:01:36,675 --> 00:01:43,664 life as incident responders and I'm Chris I did my computer science 11 00:01:43,664 --> 00:01:48,784 studies at the University of alang and Nuremberg I do this security stuff for 12 00:01:48,784 --> 00:01:55,394 over 10 years now so my CV is a little bit longer at the moment I'm a detection 13 00:01:55,415 --> 00:02:01,425 engineer before that I was a long time working in dfir so digital forensic incident 14 00:02:01,425 --> 00:02:06,620 response in different organizations and 15 00:02:07,411 --> 00:02:12,388 yeah I'm Harryr I studied electrical and computer engineering at RWTH 16 00:02:12,395 --> 00:02:18,165 University and I played a lot of CTF and did some hacking stuff at chaos computer club RWTH 17 00:02:18,165 --> 00:02:24,523 during my masters I worked at x41 dsac doing pen testing patch analysis 18 00:02:24,589 --> 00:02:32,359 so I also have some kind of offensive security background on for around one year now I'm working at G data Advanced 19 00:02:32,359 --> 00:02:36,619 analytics doing digital forensics and incident handling 20 00:02:38,800 --> 00:02:45,390 first Christian will give you a short introduction and then he will tell you how a classical ransomware attack looks 21 00:02:45,390 --> 00:02:51,970 like and in the second part of the talk I will tell you how the incident 22 00:02:51,970 --> 00:02:58,167 responders work and what you can do in advance to make it go as smooth as possible and support the incident 23 00:02:58,167 --> 00:03:05,350 response team so as Harryr told you I will probably 24 00:03:05,350 --> 00:03:12,290 we'll talk about ransomware because the customers we usually have are small and 25 00:03:12,290 --> 00:03:17,543 medium-sized businesses universities and hospitals and those are regularly 26 00:03:17,543 --> 00:03:23,268 unfortunately regularly hit by um um 27 00:03:24,170 --> 00:03:29,557 ransomware gangs the main reason for this and that's if you heard the last 28 00:03:29,557 --> 00:03:35,960 talk um why they maybe not that responsive 29 00:03:35,960 --> 00:03:42,580 and are not so interested in they just lack the resources so the manpower to do 30 00:03:42,580 --> 00:03:48,424 uh proper security measurements to secure their systems especially in in erm 31 00:03:48,424 --> 00:03:53,618 situations where you are for example in a hospital have medical devices 32 00:03:53,618 --> 00:03:59,378 um which where you cannot simply install an AV on or even patch the system 33 00:03:59,378 --> 00:04:07,321 because you lose the certification as a medical device then but also in in 34 00:04:07,321 --> 00:04:12,953 companies manufacturing companies on the shop floor we're talking about systems 35 00:04:12,953 --> 00:04:21,292 that have run times of 25 plus years so if you look back now 2023 36 00:04:21,292 --> 00:04:26,823 we're talking about XP and older systems fun fact I was in a ransomware case and 37 00:04:26,823 --> 00:04:34,230 Wannacry in 2017 when I got a call from from a person from the shop floor 38 00:04:34,230 --> 00:04:38,000 asking me if we have a nt4 expert, um 39 00:04:40,200 --> 00:04:47,380 that can tell us if WannaCry is affecting nt4 of course you don't need 40 00:04:47,380 --> 00:04:54,710 to be a expert for NT-4 this one requires of course not affecting nt4 41 00:04:54,710 --> 00:04:59,602 systems so due to the time uh slot we thought 42 00:04:59,602 --> 00:05:04,915 memes are the best way to to tell you those stories and we have a lot of them 43 00:05:06,453 --> 00:05:12,822 so in the first uh um section I tell you a little bit of how an attack Works 44 00:05:12,822 --> 00:05:21,620 um there are a lot of different possibilities how you can describe and how to structure the how an attack works 45 00:05:22,257 --> 00:05:28,993 there's the miter attack framework for example there was for example a talk Yesterday by Maker Salko 46 00:05:28,993 --> 00:05:34,854 um here on the stage there's the original cyber kill chain from from Lockheed Martin you have 47 00:05:37,190 --> 00:05:42,480 stuff from from companies like Mandy and their targeted the tech life cycle but 48 00:05:42,480 --> 00:05:47,550 that's all in my opinion two two fine-grained it's that's the reason I 49 00:05:47,550 --> 00:05:53,275 just take three simple steps yeah get a foothold in the door 50 00:05:53,275 --> 00:06:00,645 look move play around and cash out those three uh I will just go over 51 00:06:03,141 --> 00:06:07,835 so start with uh get a foot in the door so normally we 52 00:06:07,835 --> 00:06:14,756 see three ways how attackers can can get into the environment in the ransomware 53 00:06:14,756 --> 00:06:20,655 cases you have vulnerabilities in uh remote uh internet facing systems you 54 00:06:20,655 --> 00:06:25,875 have the remote Services itself and you have malware 55 00:06:26,712 --> 00:06:35,507 starting with the with the the vulnerabilities and um I just looked uh up the last four 56 00:06:35,507 --> 00:06:42,600 years and maybe somebody remembers netscaler the the so-called Citrix 57 00:06:42,600 --> 00:06:49,789 vulnerability in December 2019 um it was released mid of uh 2019 uh 58 00:06:49,789 --> 00:06:55,889 December 2019 the first POC publicly available POC was in beginning of 59 00:06:55,889 --> 00:07:03,293 January and the patch was available in middle of January so there was a round one week to one and a half weeks between 60 00:07:03,293 --> 00:07:10,494 a public proof of concept for the vulnerability and uh patch for the vulnerability and what we saw 61 00:07:10,494 --> 00:07:17,194 during 2020 a lot of companies patched but the patch didn't remove the the 62 00:07:17,194 --> 00:07:25,469 compromise so they were already compromised and um yeah with it with the patch they 63 00:07:25,469 --> 00:07:31,114 didn't remove the compromise so what we found what we could provable 64 00:07:31,114 --> 00:07:36,184 see or proof evidence for uh was nine 65 00:07:36,184 --> 00:07:42,286 month uh customer was breached after nine months using this this vulnerability 66 00:07:43,176 --> 00:07:51,434 and we had other customers where we could see that the netscaler was affected after two years but we couldn't 67 00:07:51,434 --> 00:08:00,730 prove that this this compromise was the reason for the actual ransomware case 68 00:08:00,275 --> 00:08:04,914 and of course such vulnerabilities happen not that often 69 00:08:06,295 --> 00:08:13,350 yeah so 2021 gave us uh hafnium exchange 70 00:08:13,350 --> 00:08:18,736 vulnerability also a similar situation the patch 71 00:08:18,736 --> 00:08:25,406 appeared as an out-of-band patch from Microsoft on a Tuesday evening 10 o'clock in German time 72 00:08:26,479 --> 00:08:32,529 we saw during our uh incidents or the the assessments we did that 73 00:08:34,476 --> 00:08:41,516 um the first exploit exploitation attempts were seen on Wednesday in the morning at 74 00:08:41,516 --> 00:08:50,308 5:00 am so around seven eight hours later um I know one guy who could patch 75 00:08:50,308 --> 00:08:56,691 because he was online when the patch was released otherwise Germany was unable to patch in 76 00:08:56,691 --> 00:09:04,149 time and of course we can go on with 2021 proxy shell also 77 00:09:04,149 --> 00:09:10,390 exchange vulnerability proxy nutshell also exchange vulnerability 78 00:09:10,390 --> 00:09:16,367 we have uh in 2022 VMware Horizon the the virtual desktop infrastructure 79 00:09:16,367 --> 00:09:23,627 from VMware just to name also open source stuff Zimbra a collaboration platform 80 00:09:23,627 --> 00:09:28,922 including an email server uh has had a vulnerability actually the vulnerability 81 00:09:28,922 --> 00:09:34,675 was in cpio from 2015 I think which led 82 00:09:34,675 --> 00:09:40,164 to a compromise using via email so you send an email 83 00:09:40,164 --> 00:09:48,387 with a cpio with a specially crafted archive file and you could drop a web 84 00:09:48,387 --> 00:09:55,947 shell in one of the directories yeah you have of course 40 OS which is a 85 00:09:55,947 --> 00:10:02,690 40 gate VPN and firewall operating system 86 00:10:03,220 --> 00:10:08,250 and if you read the news we start at the beginning again 87 00:10:08,251 --> 00:10:15,121 netscaler had some issues several weeks ago according to foxIT we have 1900 88 00:10:15,121 --> 00:10:21,545 still unpatched net scalers worldwide how many patched 89 00:10:22,393 --> 00:10:27,743 was netscale has exists that um have not been checked for compromise we 90 00:10:27,743 --> 00:10:32,580 don't know of course so that will be a nice year probably 91 00:10:33,728 --> 00:10:41,564 um so what can you can you do against this kind of of attack vector patch your systems is one thing as you 92 00:10:41,810 --> 00:10:49,378 see this that doesn't lead to the the um or what you need to do afterwards in 93 00:10:49,378 --> 00:10:57,354 such cases you need to check your systems for possible compromise 94 00:10:57,354 --> 00:11:03,973 that is important to reduce this I highly suggest put your 95 00:11:03,973 --> 00:11:11,583 uh Services behind some VPN so that only people who already have 96 00:11:11,583 --> 00:11:17,540 connection to the VPN um can access your services or the services 97 00:11:17,540 --> 00:11:22,649 they need and that would reduce the attack surface 98 00:11:22,649 --> 00:11:28,289 at least to the VPN server so but I 99 00:11:28,289 --> 00:11:32,996 of course we can also think about remote services without vulnerabilities 100 00:11:34,661 --> 00:11:41,591 um there can be configuration mistakes so the admin does something wrong there can 101 00:11:41,591 --> 00:11:50,339 be insecure default configurations like this um I don't know if you know it but the 102 00:11:50,339 --> 00:11:55,614 local admins or the administrators on the Windows system are are 103 00:11:55,614 --> 00:12:02,101 automatically in the remote desktop users group you know and so 104 00:12:02,101 --> 00:12:08,428 we had several cases especially in the beginning of the pandemic when everybody moved from uh to the home offices and 105 00:12:08,428 --> 00:12:15,545 they needed to put people fast in the position to to access their the assist 106 00:12:15,545 --> 00:12:22,125 the internal systems again they just put a RDP server on the internet and hope for the best 107 00:12:25,136 --> 00:12:29,767 um additionally if you put services on the internet of course brute forcing and 108 00:12:29,767 --> 00:12:35,947 credential uh stuffing are attacks that are possible so brute forcing just trying the the 109 00:12:37,115 --> 00:12:42,195 username and password combinations uh credential stuffing using already leaked 110 00:12:42,195 --> 00:12:47,636 passwords or credentials from leaks you find on the internet 111 00:12:48,536 --> 00:12:53,923 what you can do about this kind of of attack Vector is uh just as I said use 112 00:12:53,923 --> 00:13:00,912 multi-factor Authentication and reduce the attack surface as in the 113 00:13:00,912 --> 00:13:06,695 point with the vulnerabilities before by moving the services behind a VPN and 114 00:13:06,695 --> 00:13:09,691 then use multi-factor authentication on VPN of course 115 00:13:12,791 --> 00:13:18,141 the last Vector that we see normally that the attackers can get in the 116 00:13:18,141 --> 00:13:23,887 network is malware we all know this about 117 00:13:23,887 --> 00:13:28,658 those funny emails you get with the attachments 118 00:13:28,658 --> 00:13:35,310 um include that have either Word documents 119 00:13:35,310 --> 00:13:41,764 attached either zip files with with Visual Basic scripts javascripts and 120 00:13:41,764 --> 00:13:47,344 what you can get isos you see a lot these days 121 00:13:48,850 --> 00:13:54,210 um or what you can also have that you can have just a link inside the email and 122 00:13:54,210 --> 00:14:01,901 you download the respective file from some some shady file sharing website 123 00:14:03,381 --> 00:14:09,435 um what we saw over the last year was uh USB sticks again funnily 124 00:14:10,744 --> 00:14:16,484 um I'm not sure if you have heard about raspberry Robin which is a malware that 125 00:14:16,484 --> 00:14:26,427 warms via USB sticks um but I haven't seen it as a vector for 126 00:14:27,234 --> 00:14:31,784 ransomware yet on my own but there are people who said that it's 127 00:14:33,220 --> 00:14:37,770 an initial access broker for some of the ransomware gangs 128 00:14:38,734 --> 00:14:42,884 so what can you do about this if you think the 129 00:14:45,169 --> 00:14:53,420 you can of course ban simply some file extensions in your mail server or you 130 00:14:53,723 --> 00:15:00,953 change the file Association types in your operating system meaning that you 131 00:15:00,953 --> 00:15:06,274 don't open the JavaScript and Visual Basic script files using for example the 132 00:15:06,274 --> 00:15:11,610 windows scripting host but open it with notepad and that will 133 00:15:11,610 --> 00:15:14,757 of course some people will be 134 00:15:18,146 --> 00:15:23,600 uh some people will think about what this this is then and ask the IT guys 135 00:15:23,600 --> 00:15:27,408 but it's better than running the the script itself 136 00:15:28,260 --> 00:15:35,110 one thing I I I don't like to to say it but keep your AV updated 137 00:15:35,547 --> 00:15:39,791 um uh this is one thing keep it updated and read the logs 138 00:15:40,722 --> 00:15:46,660 we see a lot of incidents where we see that the already 139 00:15:46,544 --> 00:15:51,714 days or weeks before we you can could have seen that there's something going 140 00:15:51,714 --> 00:15:59,612 on in your network yeah and if you see malware in your AV logs 141 00:16:00,476 --> 00:16:05,846 then react to it just check it you don't know how long this AV this malware has 142 00:16:05,846 --> 00:16:11,302 been on your system the thing is that 143 00:16:11,302 --> 00:16:16,792 just because you're AV detected it now it might have been get received an 144 00:16:16,792 --> 00:16:22,287 update for its signatures and the malware was active for days or weeks 145 00:16:22,287 --> 00:16:27,597 before so when they are inside 146 00:16:29,770 --> 00:16:34,300 then they usually look move and play around a little bit 147 00:16:36,200 --> 00:16:41,420 so when they look around what they do 148 00:16:41,420 --> 00:16:47,612 is they they enumerate AD they do Ports scan the you they search for 149 00:16:47,612 --> 00:16:54,388 vulnerabilities they check uh what they how they can escalate 150 00:16:54,388 --> 00:16:59,826 their privileges they try to find credentials 151 00:16:59,826 --> 00:17:03,871 um Kerber roasting we heard in the talk before for example is this one thing 152 00:17:07,890 --> 00:17:11,700 um they try to identify accounts you around 153 00:17:11,700 --> 00:17:16,981 have running on your systems they can use they can get the credentials from and you reuse 154 00:17:18,600 --> 00:17:24,150 and for that reason one of the most important things I think is that you 155 00:17:24,150 --> 00:17:33,254 have a principle of least privileges in your environment so only what a account needs 156 00:17:33,365 --> 00:17:38,385 you should be able to do you should use dedicated service 157 00:17:38,385 --> 00:17:43,661 accounts for your services of course and just for your information 158 00:17:44,397 --> 00:17:50,470 um service account is not an account that has a SVC underscore in front of it 159 00:17:50,298 --> 00:17:56,880 and is otherwise a normal user account um there I exist educated service 160 00:17:56,930 --> 00:18:01,453 accounts in Windows environments so use them use strong passwords I still 161 00:18:01,469 --> 00:18:06,419 I today I can I know companies who still use eight 162 00:18:06,419 --> 00:18:13,342 character long passwords um I think that's 20 minutes on a decent 163 00:18:13,342 --> 00:18:18,671 graphics card today so use strong passwords length matters 164 00:18:19,665 --> 00:18:25,455 12 plus characters is the minimum in my opinion 165 00:18:26,617 --> 00:18:31,170 and don't reuse passwords especially on your 166 00:18:31,622 --> 00:18:42,329 systems the local administrator especially in small and medium-sized businesses you see a lot that people use 167 00:18:42,329 --> 00:18:47,140 the same password for all local administrators so if I have it on one 168 00:18:47,140 --> 00:18:51,390 system I have the whole company don't reuse it 169 00:18:56,895 --> 00:19:00,895 um yeah [Audience LoL] 170 00:19:04,550 --> 00:19:12,500 so when they they move around in your network using either a password hashes they found valid credentials they they 171 00:19:12,500 --> 00:19:18,530 discovered somewhere vulnerabilities are also used to to move around in your 172 00:19:18,530 --> 00:19:22,915 environment they try to to establish persistence mechanisms here we heard 173 00:19:22,915 --> 00:19:28,579 also in the talk before about the C2 channels command and control channels they use 174 00:19:28,621 --> 00:19:36,132 um they install in some cases directly any desk team viewer and or other remote 175 00:19:36,132 --> 00:19:43,572 control software or sometimes they also use tunneling softwares like ngrok or recently they 176 00:19:43,572 --> 00:19:49,744 started using cloudflare G um and this is what you need to do is 177 00:19:49,744 --> 00:19:55,906 prop have a proper Network segmentation and with the proper Network segmentation I don't talk about subnetting 178 00:19:57,492 --> 00:20:04,302 subnetting means you just have different subnets you need to have a firewall between them 179 00:20:04,302 --> 00:20:08,143 and you have need to have rules between them that 180 00:20:09,893 --> 00:20:16,823 restrict access between your your subnets and one thing is especially important 181 00:20:17,881 --> 00:20:24,711 please keep or use Network segmentation to 182 00:20:24,711 --> 00:20:30,225 restrict the access to your backup and your Management Systems as far as 183 00:20:30,225 --> 00:20:36,197 possible we see a lot especially as I said in the small and medium-sized businesses or in 184 00:20:36,197 --> 00:20:40,183 the other organizations we we have as customers that 185 00:20:42,448 --> 00:20:46,873 yeah they have they tell us in in in in in in workshops yeah we have 186 00:20:46,873 --> 00:20:54,333 a network segmentation every building is one segment and on the question yeah you 187 00:20:54,333 --> 00:20:59,603 can move between in a segment you can access everything yes you can and 188 00:21:00,416 --> 00:21:05,911 between the the buildings you can have also a firewall and you cannot access 189 00:21:05,911 --> 00:21:11,930 anything no you can access everything and also your your uh VMware Management 190 00:21:11,930 --> 00:21:17,420 console oh yes yes we can so everybody can access it yes of course and that 191 00:21:17,461 --> 00:21:21,761 doesn't work so 192 00:21:26,913 --> 00:21:30,263 um when they play around they normally try 193 00:21:30,263 --> 00:21:38,288 to to gain more privileges so privilege escalation is assisting local privilege escalations normally but also using 194 00:21:38,288 --> 00:21:43,440 vulnerabilities um or misconfigurations insecure default 195 00:21:43,440 --> 00:21:48,981 configurations um my personal favorites are Group 196 00:21:48,981 --> 00:21:53,211 Policy preferences or passwords in group policy preferences 197 00:21:54,686 --> 00:22:01,890 this is no longer possible since I think 2014 to put passwords in group policy 198 00:22:01,890 --> 00:22:07,994 preferences however if you had your password stored in those preferences 199 00:22:07,994 --> 00:22:16,161 before the patch in 2015 2014 then they're still there and yeah there 200 00:22:16,161 --> 00:22:20,932 are AES encrypted but the encryption key is on the Microsoft website 201 00:22:21,783 --> 00:22:26,663 so you can just download it and just take it and decrypt the keys 202 00:22:28,834 --> 00:22:33,484 um then during that phase they also try to disable your security measures 203 00:22:34,914 --> 00:22:41,516 the thing you can do is of course patch your system so you know try to get your 204 00:22:41,516 --> 00:22:46,655 availabilities out of of this equation can try to configure your systems in a 205 00:22:46,655 --> 00:22:52,580 secure way this is not always possible due to some shitty uh third-party software 206 00:22:52,897 --> 00:22:58,777 and keep your AV um updated and please please as I said 207 00:22:58,777 --> 00:23:02,894 already check the locks and act accordingly 208 00:23:05,511 --> 00:23:09,741 so in the last phase they cash out that's when when they 209 00:23:10,707 --> 00:23:16,881 start using a uh being your backup service so they copy 210 00:23:16,881 --> 00:23:21,906 data from your your environment using um 211 00:23:21,906 --> 00:23:27,574 file sharing platforms for example yeah Mega and set was was once the thing we 212 00:23:27,574 --> 00:23:33,155 transfer we had already uh every every other file sharing platform you you can 213 00:23:33,155 --> 00:23:38,891 think about is a possible way to exfiltrate data they also use their 214 00:23:38,891 --> 00:23:47,270 their C2 communication channels so sometimes you they also they just use the the possibilities in any desk or in 215 00:23:47,270 --> 00:23:53,014 in RDP clients or they use uh file transfer protocols 216 00:23:53,014 --> 00:23:57,014 like um SS SFTP um 217 00:23:59,670 --> 00:24:06,260 we saw for example in one case that they try to install filezilla on every machine they had access to 218 00:24:06,260 --> 00:24:11,705 um because on the first one it didn't work on the second it didn't work on the third it didn't work yet because SFTP 219 00:24:11,705 --> 00:24:17,940 was blocked uh outgoing and that is one of the things you can do to to prevent 220 00:24:17,940 --> 00:24:21,765 exfiltration block at least 221 00:24:24,607 --> 00:24:30,456 protocols you know that you don't need in your environment and proper Network segmentation of 222 00:24:30,456 --> 00:24:39,024 course is a general thing so in the last step that's when they 223 00:24:39,024 --> 00:24:44,781 start the encryption um they're running the ransomware or 224 00:24:44,781 --> 00:24:50,171 normally they are have domain admins at that point so they can run it on all 225 00:24:50,171 --> 00:24:57,284 domain connected systems they can also disable of course when they are domain admin they can disable 226 00:24:57,461 --> 00:25:04,553 the AV before they they start to run somewhere ransomware's today disable services like 227 00:25:04,553 --> 00:25:10,303 databases and such things so that they have the full power of the machine for 228 00:25:10,303 --> 00:25:14,590 the uh for the encryption 229 00:25:15,899 --> 00:25:23,001 um if you get lucky not how everything works perfectly because they use 230 00:25:23,001 --> 00:25:29,327 group names and windows is especially picky when you have a non-english uh 231 00:25:32,344 --> 00:25:39,434 windows installed for example in Germany the the group everybody is called yida 232 00:25:40,319 --> 00:25:47,322 and we had cases where the ransomware didn't really work that well because they couldn't 233 00:25:47,322 --> 00:25:51,322 change the permissions of the files first um 234 00:25:53,436 --> 00:25:59,278 they use different encryption schemas normally they they come with the 235 00:25:59,278 --> 00:26:05,740 asymmetric and the symmetric encryption type the asymmetrics or public key cryptography the public key comes with 236 00:26:05,740 --> 00:26:12,254 the ransomware and is used to encrypt the symmetric keys they generate on in 237 00:26:12,254 --> 00:26:20,112 your environment depending on the ransomware they they generate one key for each system or even one key for each 238 00:26:20,112 --> 00:26:25,876 file it depends a little bit on the on the ransomware how it works but that's 239 00:26:25,876 --> 00:26:32,998 the usual thing they use um I would never count on the the fact 240 00:26:32,998 --> 00:26:39,521 that there are possibly maybe there could be 241 00:26:39,521 --> 00:26:47,544 decryptable uh things um in in my opinion in my uh in my world 242 00:26:49,120 --> 00:26:55,200 The ransomware Gangs have learned and used the standard Microsoft Windows or 243 00:26:55,200 --> 00:26:59,232 some other publicly available libraries to to do the encryption 244 00:27:00,770 --> 00:27:09,224 they executed by a remote tools like PSX Powershell or some use 245 00:27:09,224 --> 00:27:14,983 gpos group policies to execute the ransomware on every 246 00:27:14,983 --> 00:27:21,153 machines they they connected to the domain and what can you do about this no it's 247 00:27:21,153 --> 00:27:26,533 it's hard but the the most important thing is have online backups offline 248 00:27:26,533 --> 00:27:33,817 backup sorry thanks you you see you see off online backups 249 00:27:33,817 --> 00:27:39,414 are not that are great but not that great offline backups is the most important this is the most important 250 00:27:39,414 --> 00:27:45,388 thing so um don't have it connected to your environment 251 00:27:45,969 --> 00:27:52,207 the the the USB disk on the system is not offline backup 252 00:27:55,594 --> 00:28:00,794 um in my opinion if you see that something is is still encrypting 253 00:28:01,440 --> 00:28:07,630 I I'm I'm always hesitant to say shut down the system because you can break 254 00:28:07,630 --> 00:28:15,529 the encryption and maybe the file that is currently in under encryption or the files will never be decryptable if you 255 00:28:15,529 --> 00:28:23,588 want to buy a decrypter um or gather the cryptos through some 256 00:28:25,669 --> 00:28:32,149 discussions with the with the ransomware guys um if it's a VM just suspend it and 257 00:28:32,149 --> 00:28:38,351 that's it and if everything is already encrypted 258 00:28:39,205 --> 00:28:43,575 keep cool and call your incident responder 259 00:28:51,306 --> 00:28:58,606 so now let's talk about incident response what happens when it's already too late and what can you do to support 260 00:28:58,606 --> 00:29:04,110 your incident response team at first the things I'll say in this chapter are 261 00:29:04,110 --> 00:29:10,549 for our company and how we work so other companies might work a little bit different than that 262 00:29:12,792 --> 00:29:17,022 first for some reason incidents always come on Friday afternoon 263 00:29:18,950 --> 00:29:23,950 so some customers think it is a good idea to try to solve a case by 264 00:29:23,950 --> 00:29:30,785 themselves maybe until the end of the week and if they didn't solve it until the end of the week they call the 265 00:29:30,785 --> 00:29:35,876 incident response team please don't do that it doesn't help your company and it 266 00:29:35,876 --> 00:29:43,792 doesn't make your incident Response Team happy to have to work on the weekend and in addition the longer you wait with 267 00:29:43,792 --> 00:29:50,859 calling the incident Response Team the longer the incident response will take and the more complicated forensics will 268 00:29:50,859 --> 00:29:56,523 be because you have lock retention times while trying to do stuff by yourself 269 00:29:56,702 --> 00:30:01,572 maybe you modify some of the systems and it becomes much more harder to do 270 00:30:02,397 --> 00:30:08,127 precise forensics so what happens on our site when such a 271 00:30:08,129 --> 00:30:13,589 new incident ticket arrives the first thing we do is team internal coordination 272 00:30:13,589 --> 00:30:18,935 so we discussed do we have enough people do we have a person for each role in our 273 00:30:18,935 --> 00:30:25,084 team we have three roads incident handling forensics analyst and Mayweather analyst 274 00:30:26,254 --> 00:30:32,524 so first let's talk about incident handling incident Handler is responsible for all 275 00:30:32,524 --> 00:30:39,791 the tasks that our customer facing and the first point is always get the customer out of that headless chicken 276 00:30:39,791 --> 00:30:45,121 mode like we call it because when an incident comes at our customer site 277 00:30:45,121 --> 00:30:52,238 everyone is like running around in so-called like headless chicken doing something but not doing anything helpful 278 00:30:53,402 --> 00:31:00,952 so this is always the first task for the incident Handler Handler structure the customer do meetings and then do all the 279 00:31:00,952 --> 00:31:06,612 relevant decisions leading to a secure emergency operation mode that means in 280 00:31:06,612 --> 00:31:13,484 this case that you have working core infrastructure so working domain controller maybe a working email server 281 00:31:13,484 --> 00:31:19,302 and whatever you need or whatever you define as very business critical systems 282 00:31:21,321 --> 00:31:28,785 let's go a little bit more in detail probably the first measure will be to cut off the internet connection because 283 00:31:28,785 --> 00:31:33,163 you just buy you a lot of time with doing that no matter how many back doors 284 00:31:33,163 --> 00:31:38,830 the attackers placed in the network if you cut off the internet connection the attackers can't access their back doors 285 00:31:38,949 --> 00:31:46,279 anymore and then you will start to rebuild your network you will Define everything in 286 00:31:46,279 --> 00:31:51,317 your current infrastructure as red Network and then start building up a 287 00:31:51,317 --> 00:31:57,590 green network with clean systems maybe you will start with some admin workstations so that the 288 00:31:57,590 --> 00:32:05,419 administrators can work properly and then we go through a prioritized system list and build up the most 289 00:32:05,419 --> 00:32:11,420 important systems this can be of course like I said domain controllers or email 290 00:32:11,420 --> 00:32:17,839 servers but also whatever is important for your current company and for the 291 00:32:17,839 --> 00:32:23,075 business this is of course in a hospital something very different from a small 292 00:32:23,075 --> 00:32:30,742 business or maybe a university and please have such a prioritized list before your first incident because when 293 00:32:30,742 --> 00:32:37,693 you start discussing in an incident which system is the most important then everyone in the company will tell you 294 00:32:37,693 --> 00:32:44,799 something else and everyone will tell you that their system is the most important and has to be migrated and 295 00:32:44,799 --> 00:32:48,697 checked and analyzed at first and this isn't really helpful 296 00:32:50,494 --> 00:32:56,364 so during incident handling there might also be obstacles for example if you 297 00:32:56,364 --> 00:33:02,524 have backups and you encrypted your backups this is great but you should think about where to 298 00:33:02,524 --> 00:33:08,909 store that encryption key because when you store it in your password manager and the password 299 00:33:08,909 --> 00:33:13,747 manager database is in a VM which then gets encrypted by the ransomware well 300 00:33:13,747 --> 00:33:18,670 you have backups but you don't have your key to decrypt the backup so this 301 00:33:18,969 --> 00:33:25,809 doesn't help you very much and this is not only two for password 302 00:33:25,991 --> 00:33:34,231 this is also true for like everything else which you could need during an incident like contact lists or network 303 00:33:34,268 --> 00:33:39,168 lists and so on so 304 00:33:40,571 --> 00:33:47,271 working with third parties is also a task for the incident Handler this could be a data Protection Agency 305 00:33:47,330 --> 00:33:52,760 or the customers customers because they also often yeah have some panic and have 306 00:33:52,760 --> 00:33:58,021 questions which the customer maybe can't answer so this is also a talk a task for 307 00:33:58,021 --> 00:34:03,127 the incident Handler and of course also working with law enforcement but there's 308 00:34:03,177 --> 00:34:09,765 one thing to keep in mind Law Enforcement wants to do criminal investigation incident response wants to 309 00:34:09,765 --> 00:34:17,545 bring you back to business as soon as possible this is not necessarily the same goal but it finally makes sense to 310 00:34:17,571 --> 00:34:25,213 allow your incident responders to share all information with law enforcement because this is beneficial for everyone 311 00:34:26,652 --> 00:34:32,712 um so why do I do forensics at all one could ask well the less precise the 312 00:34:32,729 --> 00:34:39,189 forensic results are the more conservative the rebuild needs to be so if you don't know anything about the 313 00:34:39,189 --> 00:34:45,632 attack you have to rebuild everything from scratch so let's say you have maybe two week or 314 00:34:45,632 --> 00:34:51,086 backups and you know the initial access of the attackers was only one week ago 315 00:34:51,161 --> 00:34:56,351 then you could use that two weeks old backups for rebuilding which is really 316 00:34:56,351 --> 00:35:00,991 great but if you don't know how long the attackers were in the 317 00:35:01,381 --> 00:35:05,521 network then it's really hard to to say 318 00:35:06,991 --> 00:35:12,077 a second Point why you should do forensics is to estimate the impact 319 00:35:12,157 --> 00:35:18,677 especially in an early um phase of an attack it is important to know that the attackers only compromise 320 00:35:18,677 --> 00:35:23,769 some user devices or did they already gain domain administrative privileges 321 00:35:24,211 --> 00:35:30,751 because then the incident handling is completely different and of course it is also relevant to 322 00:35:30,751 --> 00:35:36,478 know which attack path the attack is used so you can Harden your network and 323 00:35:37,102 --> 00:35:42,752 improve in the future how should incident how should forensics 324 00:35:42,752 --> 00:35:48,789 be it should of course be correct it should be non-disruptive for the customer so they can concentrate on 325 00:35:48,967 --> 00:35:55,064 rebuilding their infrastructure and it should be fast what means correct you want to have a 326 00:35:55,211 --> 00:36:00,501 dedicated forensics analyst so you want to have incident response team which is 327 00:36:00,686 --> 00:36:07,016 doing incident response in their day-to-day business and you want in your forensics teams one 328 00:36:07,016 --> 00:36:12,910 person who can really concentrate on forensics and is not also doing the incident handling and the malware 329 00:36:12,910 --> 00:36:17,228 analysis and so on because then you can't really concentrate on doing the technical work 330 00:36:19,327 --> 00:36:25,685 it also should be non-disruptive as I said that means for our company we do remote triage collection we use the tool 331 00:36:25,745 --> 00:36:33,335 videos adapter for it this is an official logo of velociraptor it's a really nice tool because we just send 332 00:36:33,335 --> 00:36:38,639 our customers an exe file the customer just has to execute the exe file and 333 00:36:38,639 --> 00:36:43,899 then it collects all the logs we need all the um yeah information we need 334 00:36:44,836 --> 00:36:49,836 and absolute uploads it to our um to our infrastructure so we don't 335 00:36:49,836 --> 00:36:58,134 need to drive to the customer and like start copying hard disks for a week for weeks because this doesn't help anyone 336 00:36:58,493 --> 00:37:04,333 I should also said it should be fast so we have a lot of Automation and tooling 337 00:37:04,333 --> 00:37:10,535 to make the work really fast automatically pushing our the uploaded files we want 338 00:37:10,535 --> 00:37:16,702 to lose this infrastructure automatically doing reporting and so on I won't go in detail here because this 339 00:37:16,702 --> 00:37:22,275 is enough for a single talk about it maybe maybe on Congress let's see 340 00:37:23,275 --> 00:37:28,445 so what happens when we do manual forensic analysis 341 00:37:28,619 --> 00:37:35,709 there are some questions you normally want to answer one is of course about the vectors you want to find all vectors 342 00:37:35,709 --> 00:37:40,968 the attack is placed in the network of course you want to have General IUC 343 00:37:40,968 --> 00:37:47,529 so indicators of compromise so maybe which IP addresses did the attackers use so you can block them in the firewall or 344 00:37:47,529 --> 00:37:53,345 which tools did they use so you can scan over the whole it infrastructure and 345 00:37:53,382 --> 00:37:59,502 find further compromised systems and if you have the iocs you can also cross-correlate between cases or show 346 00:37:59,768 --> 00:38:05,208 them share them with law enforcement another important thing is of course 347 00:38:05,270 --> 00:38:13,269 lateral movement because you can use lateral movement to build a timeline of the attack so the mythology here is to 348 00:38:13,391 --> 00:38:20,819 look where the attackers came from and where the attackers went so if you have a system a and you see the attackers 349 00:38:20,819 --> 00:38:27,572 came from system B then you analyze system B and go back in time and the same you could do forward in time until 350 00:38:27,572 --> 00:38:34,294 you found yeah all relevant systems which were compromised and have a proper timeline of the case 351 00:38:36,214 --> 00:38:41,851 and other common thing in the life of an incident responder is to fall in a rabbit hole 352 00:38:42,226 --> 00:38:47,436 because it can be really really interesting to do forensics but yeah you need to 353 00:38:48,464 --> 00:38:55,038 check what the customer really wants to know or what your incident Handler really wants to know 354 00:38:55,038 --> 00:39:01,528 and not just analyze something and fall into the rabbit hole because this yeah 355 00:39:01,528 --> 00:39:08,174 really doesn't help anyone here also the 80 20 rule applies so I'd say you get like 80 percent of 356 00:39:08,275 --> 00:39:14,437 the relevant forensics results in 20 of the time and often this is enough for the incident Handler to do the right 357 00:39:14,437 --> 00:39:20,347 decisions to rebuild the customer Network customer's networks as fast as possible 358 00:39:21,634 --> 00:39:26,734 so let's go into reporting nobody really likes reporting 359 00:39:28,074 --> 00:39:34,095 but producing fancy forensics results doing fancy technical stuff is worthless 360 00:39:34,095 --> 00:39:40,071 if you can can't explain it to a manager nobody pays us to do fancy technical 361 00:39:40,071 --> 00:39:45,423 stuff nobody understands but we are paid to help our customers and explain the 362 00:39:45,423 --> 00:39:52,962 situation to them and to do the right decisions so yeah nobody likes reporting but it is a really really important 363 00:39:52,962 --> 00:39:56,818 thing so after learning the report the 364 00:39:56,838 --> 00:40:02,428 incident is over and all work is done right no not really because such a 365 00:40:02,466 --> 00:40:08,051 report normally contains a lot of recommendations what to do 366 00:40:08,700 --> 00:40:14,760 for the customers and as I already said incident response more or less stops 367 00:40:15,356 --> 00:40:20,596 when the customer is in an emergency operation mode with a core infrastructure and the most critical 368 00:40:20,672 --> 00:40:28,134 systems are working but there's still a lot to do and of course security is a process so 369 00:40:28,298 --> 00:40:32,408 there's always something to improve to stay ahead of the attackers 370 00:40:33,606 --> 00:40:38,036 so now let's go into a quick recap 371 00:40:42,251 --> 00:40:49,361 how to protect against those kind of attacks we had this uh the topics just 372 00:40:49,520 --> 00:40:55,220 as a recap patch your systems it's important keep them updated 373 00:40:56,028 --> 00:41:02,038 vulnerabilities come and go that's one thing have a sane privilege 374 00:41:02,212 --> 00:41:09,043 account management so use the correct user accounts for the correct for the 375 00:41:09,044 --> 00:41:17,864 the right task use the second two-factor authentication 376 00:41:17,928 --> 00:41:24,278 for remote services have a proper Network segmentation so 377 00:41:24,568 --> 00:41:31,301 put firewalls and access rules between your networks check your AV logs regularly and act 378 00:41:31,502 --> 00:41:37,374 accordingly and not really protecting your you against an attack 379 00:41:37,862 --> 00:41:46,334 but keeping the the symptoms a little bit um not that that devastating have 380 00:41:46,334 --> 00:41:52,573 offline backups so how to make incident response easier 381 00:41:52,756 --> 00:41:58,866 for you for yourself and for the incident Response Team first thing is don't try to solve this stuff by 382 00:41:58,871 --> 00:42:05,271 yourself just call the incident response team of your trust as soon as you know 383 00:42:05,271 --> 00:42:11,885 you have an incident the second thing is logging policy because if you have a very short lock 384 00:42:11,885 --> 00:42:16,928 retention for example and you only have locks for the last like two days then it 385 00:42:16,928 --> 00:42:22,150 might be hard to find out what happened and if you don't lock stuff at all 386 00:42:22,479 --> 00:42:28,999 especially in the cloud environments this is a thing where you need to look at then it might also be hard to get 387 00:42:29,173 --> 00:42:34,213 proper forensic results then as I said have offline contact 388 00:42:34,213 --> 00:42:41,026 lists because your main server is down your accounts are compromised so you can't use like your Microsoft teams or 389 00:42:41,026 --> 00:42:47,828 whatever you use in your company you need an alternative way to communicate to your colleagues and or 390 00:42:48,032 --> 00:42:54,102 employees then as I said prioritized asset list and network plans 391 00:42:55,168 --> 00:43:01,808 and a disaster recovery plan in best case and please all have all that in a 392 00:43:01,808 --> 00:43:07,565 secure place in best case on paper well nobody likes doing stuff with their 393 00:43:07,565 --> 00:43:11,677 trees I know but that trees don't get ransomware right 394 00:43:14,319 --> 00:43:21,379 so that's it from our site more or less normally this is a point where 395 00:43:21,379 --> 00:43:26,302 you can ask us questions but we want to spin that around and give you some 396 00:43:26,302 --> 00:43:31,093 questions maybe someone can guess the right answer 397 00:43:31,876 --> 00:43:36,976 what do you think is the shortest time to domain admin we saw an incident between initial compromise and the 398 00:43:36,976 --> 00:43:41,062 attackers gaining domain admin 399 00:43:42,619 --> 00:43:49,499 yeah it's not it somebody said 17 seconds two minutes well it's not that 400 00:43:49,499 --> 00:43:54,332 bad six minutes is is what we have seen in incident hmm 401 00:43:54,742 --> 00:44:00,323 what do you think think is the shortest lock retention on a domain controller so how long 402 00:44:01,207 --> 00:44:08,677 zero 10 hours a year well it is 2.5 minutes 403 00:44:12,866 --> 00:44:18,596 what what do you think is the highest number of domain administrator accounts 404 00:44:18,596 --> 00:44:26,037 we saw in an incident 200 nah 50. 405 00:44:26,037 --> 00:44:30,796 64 with 120 people working in IT 406 00:44:33,042 --> 00:44:39,672 what do you think is the longest dwell time so the longest the longest time between being initially compromised and 407 00:44:39,886 --> 00:44:47,568 realizing that you are compromised 60 now it's it's not that bad I hit here 408 00:44:47,826 --> 00:44:52,116 a lot of years well it is like around two years 409 00:44:53,294 --> 00:44:58,824 so well that's really it from our side no time for questions thank you 410 00:45:01,608 --> 00:45:05,908 if you 411 00:45:09,297 --> 00:45:13,697 if you have questions we will be there outside waiting come and join us 412 00:45:15,480 --> 00:45:21,230 thank you thank you thank you Harryr and Kris warmer applause thank you 413 00:45:23,964 --> 00:45:32,770 End of subtitles:[Translated by {Yang}{Li} (ITKST56 course assignment at JYU.FI)]