1 00:00:00,000 --> 00:00:18,070 35C3 preroll music 2 00:00:18,070 --> 00:00:24,070 Herald: The following talk is about email encryption. Who worries that their email 3 00:00:24,070 --> 00:00:32,800 encryption might be capable of being hacked or cracked? Worry not as far as I'm 4 00:00:32,800 --> 00:00:40,050 told it still is rather secure. But. And the but will be answered by Sebastian 5 00:00:40,050 --> 00:00:45,879 Schinzel who is one of the eight security researchers who uncovered the drama and 6 00:00:45,879 --> 00:00:54,829 the details behind e-fail. A recently occurred issue with OpenPGP and S/MIME 7 00:00:54,829 --> 00:00:59,600 in a lot of email clients. All the technical details he will dig into so 8 00:00:59,600 --> 00:01:03,339 please give a warm hand of applause for Sebastian Schinzel. 9 00:01:03,339 --> 00:01:09,850 applause 10 00:01:09,850 --> 00:01:15,080 Sebastian: So it's good to be back. Thank you everyone for coming. My name is 11 00:01:15,080 --> 00:01:22,080 Sebastian. It's my fourth talk here at the CCC. And the other talks that I did were 12 00:01:22,080 --> 00:01:27,260 also on encryption so I somehow like encryption and I like analyzing encryption 13 00:01:27,260 --> 00:01:31,320 and I like to break encryption. So I did a talk on TLS where we found an attack 14 00:01:31,320 --> 00:01:39,070 against TLS and against XML encryption for example. And I usually start my talk by 15 00:01:39,070 --> 00:01:45,030 asking Who's using this this specific technology, right? Because when you when 16 00:01:45,030 --> 00:01:49,710 you usually when you're analyzing or doing a talk about a communication protocol you 17 00:01:49,710 --> 00:01:57,144 ask the people who's using that and you know for e-mail it would be silly, right? 18 00:01:57,144 --> 00:02:00,200 Because, who's using e-mail. Every one of you is using e-mail. 19 00:02:00,200 --> 00:02:04,921 OK. Because I don't know. I mean when you started using the Internet like 20 00:02:04,921 --> 00:02:10,369 in the last 20 years or so usually the first thing that you do you dial in 21 00:02:10,369 --> 00:02:13,409 onto the Internet and you make yourself an e-mail address, right? 22 00:02:13,409 --> 00:02:20,980 So everyone has everyone uses e-mail. But let's take a look at how things work when 23 00:02:20,980 --> 00:02:25,359 when you send an email. So when you click in your email client when you when you 24 00:02:25,359 --> 00:02:30,329 compose an email and you click into your email client "Send". So the first thing is 25 00:02:30,329 --> 00:02:34,989 on one of your devices you're using the local network and there's a green arrow 26 00:02:34,989 --> 00:02:40,790 because it's under your control. So you can define by yourself how secure this 27 00:02:40,790 --> 00:02:45,560 link is gonna be. Then you use the next connection and the next connection will be 28 00:02:45,560 --> 00:02:50,437 to your SMTP server. And this is also a green arrow because it's under your control. 29 00:02:50,437 --> 00:02:53,344 Okay. You can control this and usually it's TLS. 30 00:02:53,344 --> 00:02:58,950 The same as valid for your IMAP folder because always when you send an 31 00:02:58,950 --> 00:03:07,180 email via SMTP you will upload a copy to your "sent" folder on IMAP and it uses TLS 32 00:03:07,180 --> 00:03:12,379 in most of the cases so therefore the arrow is green. The arrow is not that 33 00:03:12,379 --> 00:03:17,659 green anymore when it comes to the backup strategy for example an archiving strategy 34 00:03:17,659 --> 00:03:23,599 of your email provider which may be gmail or gmx for example or your company. OK. 35 00:03:23,599 --> 00:03:27,370 But you may have a chance to know the administrators and ask him whether they 36 00:03:27,370 --> 00:03:33,579 are doing a good job and stuff like that. You're also - it's not under your 37 00:03:33,579 --> 00:03:38,840 control where else your email is uploaded to. So for example in order to check it 38 00:03:38,840 --> 00:03:44,620 for antivirus or anti spam and stuff like that, right? So it gets uploaded to some 39 00:03:44,620 --> 00:03:50,389 cloud, right? Because you want to do a spam filtering and then it goes out to the 40 00:03:50,389 --> 00:03:54,200 Internet and you know we learned what's happening on the Internet. People are 41 00:03:54,200 --> 00:04:00,459 listening in on the Internet and we heard from Edward Snowden for example that many 42 00:04:00,459 --> 00:04:05,150 nation state actors and agencies are doing this as well they are listening into 43 00:04:05,150 --> 00:04:11,549 your e-mails or they at least try to. And then the arrows get red as soon as it hits 44 00:04:11,549 --> 00:04:16,431 the SMTP server of the receiver. Right. Because you don't really know whether they 45 00:04:16,431 --> 00:04:21,250 are using TLS, encryption in any case. You don't know whether they upload it to 46 00:04:21,250 --> 00:04:27,040 another antivirus vendor or so and then it gets uploaded to the IMAP folder to the 47 00:04:27,040 --> 00:04:31,990 inbox of the receiver where it gets archived again, right? There is a backup 48 00:04:31,990 --> 00:04:38,310 is being done and stuff like that, it's archived. And then finally the receiver of 49 00:04:38,310 --> 00:04:42,831 the e-mail has the ability to check his own email and retrieve the e-mail 50 00:04:42,831 --> 00:04:49,150 from his inbox. OK. So even if you know that the other person that you've just 51 00:04:49,150 --> 00:04:53,490 sent the e-mail to does a good job in securing it and uses encryption everywhere 52 00:04:53,490 --> 00:04:56,380 and so on and so on. You never know whether there's some 53 00:04:56,380 --> 00:05:01,780 attacker or so who has compromised the infrastructure and so on and so on. OK. 54 00:05:01,780 --> 00:05:06,790 And this problem multiplies. When you send an e-mail to multiple people 55 00:05:06,790 --> 00:05:11,140 because then all of a sudden the e-mail that you just sent, your e-mail, hits 56 00:05:11,140 --> 00:05:17,270 infrastructures of many other people. OK? So many people have access to this 57 00:05:17,270 --> 00:05:22,140 e-mail and this is the first thing that I want to - this is the first point that I 58 00:05:22,140 --> 00:05:27,430 want to make in this talk. People usually talk about their e-mail. It's my e-mail, 59 00:05:27,430 --> 00:05:31,960 you know? And when I run my own mail server it's going to be secure but this is not 60 00:05:31,960 --> 00:05:36,630 the case. I mean the the point of using e-mail is distributing it to your 61 00:05:36,630 --> 00:05:40,030 friends to your colleagues and so on and so on because we want to exchange. 62 00:05:40,030 --> 00:05:43,430 It's a communication method but the communication is archived so people 63 00:05:43,430 --> 00:05:52,120 archive this and so on and so on. So it's it's a mess basically. OK. So for the rest 64 00:05:52,120 --> 00:05:56,090 of this talk we need to have some assumption that someone has access to the 65 00:05:56,090 --> 00:06:02,091 e-mails. OK. Because right. We just learned that it's distributed. You're 66 00:06:02,091 --> 00:06:05,620 archiving your own e-mails in your IMAP folder and so on and so on I have the 67 00:06:05,620 --> 00:06:09,520 e-mails of the last 15 years or so in my in my IMAP folder and if someone gets 68 00:06:09,520 --> 00:06:14,940 access to it - right - boom. I have a problem. Okay. And in order to cope with 69 00:06:14,940 --> 00:06:19,090 this people came up with end-to-end encryption. And there's two competing 70 00:06:19,090 --> 00:06:28,180 standards out there. The first one is OpenPGP. It was first defined in 1991 from 71 00:06:28,180 --> 00:06:32,340 Phil Zimmermann invited it. And this was like this happened in the first crypto 72 00:06:32,340 --> 00:06:39,400 wars. So this is a very interesting story. And it's the most widely used e-mail 73 00:06:39,400 --> 00:06:42,940 clients they they support OpenPGP but only when you when you download the 74 00:06:42,940 --> 00:06:48,590 plug-in, OK? For S/MIME you usually have native support in most of the 75 00:06:48,590 --> 00:06:52,900 applications so you just get an S/MIME certificate and off you go. 76 00:06:52,900 --> 00:06:57,020 You can just use it and it just works out of the box but it's mostly used by 77 00:06:57,020 --> 00:07:01,530 corporate organizations and military for example right. Not so much by privacy 78 00:07:01,530 --> 00:07:06,700 advocates like OpenPGP. Now we can't measure very well how many 79 00:07:06,700 --> 00:07:11,230 people use S/MIME. We tried but there doesn't seem to be a public source where 80 00:07:11,230 --> 00:07:15,790 we can measure this but we can do it on PGP because PGP has key servers and you 81 00:07:15,790 --> 00:07:19,520 can upload your key there so that people can communicate with you in a secure 82 00:07:19,520 --> 00:07:26,150 manner. And this statistic here is the amount of new published public keys per 83 00:07:26,150 --> 00:07:32,920 month. And it started off very low in 1997. Then in 2003 we saw a constant rise 84 00:07:32,920 --> 00:07:39,600 until 2013 where we see a a spike here where it more than doubled. Does anyone 85 00:07:39,600 --> 00:07:43,330 know why we saw this spike in 2013? Someone in the audience: Snowden! 86 00:07:43,330 --> 00:07:48,560 Sebastian: Snowden right. Edward Snowden we had the Snowden revelations and Edward 87 00:07:48,560 --> 00:07:54,060 Snowden uses PGP to communicate with the journalists who did the actual, like, 88 00:07:54,060 --> 00:07:59,639 disclosure of the documents. And so we see this in the uploader keys so people use 89 00:07:59,639 --> 00:08:05,500 PGP much more. We also see this in the amount of daily users of Enigmail 90 00:08:05,500 --> 00:08:10,560 which is the plugin for Thunderbird. And you can extract this 91 00:08:10,560 --> 00:08:16,400 information and there we also see around 2013 we see a jump and it increased by 50 92 00:08:16,400 --> 00:08:21,140 percent or so something like this. So people use PGP, right? It's not that many 93 00:08:21,140 --> 00:08:25,210 when you compare it to how many people use e-mail a few hundred thousand is not that 94 00:08:25,210 --> 00:08:31,580 much but still it's a substantial number. The problem here is we know for a fact 95 00:08:31,580 --> 00:08:38,000 that in the last 20 years or so especially non-technical users are not able to use 96 00:08:38,000 --> 00:08:43,860 PGP in a secure manner. Okay. So we have the first paper from '99: "Why Johnny 97 00:08:43,860 --> 00:08:49,091 Can't encrypt" and then we have a follow up paper from 2006: "Why Johnny still 98 00:08:49,091 --> 00:08:54,700 can't encrypt", OK? So OK. Must've been better I hope and then in 2015 we 99 00:08:54,700 --> 00:09:00,460 have "why Johnny Still, Still can't encrypt". OK? so we have a long story 100 00:09:00,460 --> 00:09:06,170 where people try to use PGP but failed, OK? And this is not something that is 101 00:09:06,170 --> 00:09:13,310 inherent in an OpenPGP. We have the same papers for S/MIME where they said 102 00:09:13,310 --> 00:09:18,240 novice users - and where they confronted novice users with with S/MIME and they 103 00:09:18,240 --> 00:09:23,260 failed, basically. They weren't able to use it in a secure manner. Okay. 104 00:09:23,260 --> 00:09:29,240 So therefore in order to enable people to use PGP, especially PGP and S/MIME in a 105 00:09:29,240 --> 00:09:34,300 secure manner you have many tutorials out there. And this is a very interesting 106 00:09:34,300 --> 00:09:38,900 tutorial because it's the original video that Edward Snowden recorded in order to 107 00:09:38,900 --> 00:09:44,480 teach Glenn Greenwald how to use openPGP. It's still on Vimeo. The original video is 108 00:09:44,480 --> 00:09:49,330 still there. And what is interesting there is he doesn't recommend a plugin for an 109 00:09:49,330 --> 00:09:55,170 email client. He basically says just use web mail and use this PGP standalone 110 00:09:55,170 --> 00:10:00,460 application type your message there click encrypt and copy and paste the ciphertext 111 00:10:00,460 --> 00:10:06,040 into your web mail, OK? So there's no integration and others recommend for 112 00:10:06,040 --> 00:10:13,120 example Enigmail or GPGtools. Enigmail is the plugin for Thunderbird and GPGtools is 113 00:10:13,120 --> 00:10:18,390 the plug in for Apple mail, which means that most of them had HTML 114 00:10:18,390 --> 00:10:22,970 switched on which is an interesting fact. OK. For something that we are going to 115 00:10:22,970 --> 00:10:29,340 talk about later. So I thought about how can I do this talk. And this is how I came 116 00:10:29,340 --> 00:10:33,650 up with the agenda. So we start off with some attacks that are very I find pretty 117 00:10:33,650 --> 00:10:37,260 interesting. At least for an academic they are pretty interesting, and we slowly 118 00:10:37,260 --> 00:10:42,300 degrade to pretty silly attacks and you decide for yourself whether you would fall 119 00:10:42,300 --> 00:10:47,720 for it or not. I came to the conclusion that I would probably fall for them. OK, 120 00:10:47,720 --> 00:10:53,780 I lied. So we start off with something that is even worse than silly. So what is 121 00:10:53,780 --> 00:10:58,210 it what's the worst thing that can happen when you're sending an encrypted mail. 122 00:10:58,210 --> 00:11:00,570 Audience: You send your private key. 123 00:11:00,570 --> 00:11:03,690 Sebastian: if you send your private key there would be bad. What's even worse? 124 00:11:03,690 --> 00:11:06,290 Audience: You forget to encrypt. 125 00:11:06,290 --> 00:11:11,770 Sebastian: you send a plain text. Okay okay. So in 2014 there was a bug in 126 00:11:11,770 --> 00:11:16,210 Enigmail which is documented in their bug tracker. So basically you compose an e-mail 127 00:11:16,210 --> 00:11:19,860 you tell Enigmail to sign and encrypt it, and then you send it and it will be 128 00:11:19,860 --> 00:11:27,860 sent in plain text. Okay. Oh okay. This is not good. In 2017 Outlook did it much better 129 00:11:27,860 --> 00:11:32,710 because they did encrypt it but they packed the plain text along with the 130 00:11:32,710 --> 00:11:42,350 ciphertext. (Audience: laughing) Okay. Bad. And then in 2018 you may remember this is 131 00:11:42,350 --> 00:11:48,270 just a few months back or so Enigmail or pretty easy privacy. It's a plugin to 132 00:11:48,270 --> 00:11:52,660 Enigmail, so a plugin to a plugin that's actually used for novice users. It gave 133 00:11:52,660 --> 00:11:57,040 the feedback that the email that you're composing will be encrypted but it wasn't. 134 00:11:57,040 --> 00:12:04,529 Okay. And the good news here is: the e-mail attacks don't apply here because, 135 00:12:04,529 --> 00:12:10,230 OK, I mean, it's about I don't want to make fun of any developers. The point that I'm 136 00:12:10,230 --> 00:12:16,010 trying to make here is that email is a plaintext protocol and it's always very 137 00:12:16,010 --> 00:12:21,890 difficult to make a protocol that was made for plain text delivery - to make this 138 00:12:21,890 --> 00:12:26,040 encrypted. When you have a plaintext fallback you're into trouble. Okay. And we 139 00:12:26,040 --> 00:12:33,710 we saw the same basically with HTTP and HTTPS. OK. So HTTPS is like now secure but 140 00:12:33,710 --> 00:12:37,640 they hit the same issues right and browsers and so they were falling back to 141 00:12:37,640 --> 00:12:42,529 HTTP and so on and so on. So it's really really difficult. Now that we have this 142 00:12:42,529 --> 00:12:47,920 out of the way. Okay. We tried to look at the interesting interesting box and before 143 00:12:47,920 --> 00:12:53,580 we start I want to make a short introduction into how the cryptography in 144 00:12:53,580 --> 00:12:59,610 S/MIME and PGP work. And I just make one introduction because both are the same 145 00:12:59,610 --> 00:13:03,000 from the viewpoint of the attacks that I'm going to present. Okay. Both are very 146 00:13:03,000 --> 00:13:08,770 similar actually. So we start off by writing a message, m, this is here we 147 00:13:08,770 --> 00:13:13,760 generate a session key, s, that that's basically a random session key and we use 148 00:13:13,760 --> 00:13:19,780 this session key to encrypt the message and make a cipher text, c, and we use AES 149 00:13:19,780 --> 00:13:26,390 or triple DES for example any symmetric cipher will do here. 150 00:13:26,390 --> 00:13:31,910 Then we use this session key that we just generated and encrypt this with a public 151 00:13:31,910 --> 00:13:37,160 key of the recipient and we get a K and we pack this K into the message so everything 152 00:13:37,160 --> 00:13:43,040 will be packed into a message and sent off to the receiver. The receiver then 153 00:13:43,040 --> 00:13:49,050 obtains the encrypted email he starts extracting the ciphertext K and the 154 00:13:49,050 --> 00:13:56,490 ciphertext C. From K he decrypts s which is the session key, right? And with s he can 155 00:13:56,490 --> 00:14:02,580 decrypt the ciphertext C and retrieves m. And so both have the same message. Okay. 156 00:14:02,580 --> 00:14:06,670 So it's basically a hybrid encryption you use symmetric encryption for the actual 157 00:14:06,670 --> 00:14:11,740 encryption off the message and then asymmetric encryption to just encrypt this 158 00:14:11,740 --> 00:14:17,870 session key. Now, if you attended my previous talks I talk a lot about 159 00:14:17,870 --> 00:14:23,940 ciphertext malleability. So you make tiny changes to a cipher text and it will lead 160 00:14:23,940 --> 00:14:28,589 to predictable changes in the plaintext which is strange, right? So we would 161 00:14:28,589 --> 00:14:33,529 assume when we just flip a bit here somewhere, okay? Here's a bit that just 162 00:14:33,529 --> 00:14:37,470 flipped and where we decrypted we get something like this. Usually you would 163 00:14:37,470 --> 00:14:42,860 think OK. It's total garbage. Nothing sensible should come out of 164 00:14:42,860 --> 00:14:47,120 this but we see something like this. Most of the message is still intact. We just 165 00:14:47,120 --> 00:14:54,920 have a few randomly looking junk binary stuff in there and we see one changed 166 00:14:54,920 --> 00:14:59,850 character here. That used to be "email" and now it's "efail". Okay so there was 167 00:14:59,850 --> 00:15:04,240 just some bit flipping happening here. In order to understand this we need to 168 00:15:04,240 --> 00:15:08,990 introduce something that we called a mode of operation. AES or any other block 169 00:15:08,990 --> 00:15:14,910 cipher has the property that it uses blocks. In most cases it's 16 bytes. 170 00:15:14,910 --> 00:15:20,520 And when you want to encrypt more than 16 bytes you need to call AES multiple times 171 00:15:20,520 --> 00:15:25,470 and you split it into blocks and CBC is one way of connecting these ciphertext 172 00:15:25,470 --> 00:15:30,580 blocks so the decryption works like this. You have the first ciphertext block the 173 00:15:30,580 --> 00:15:36,839 second ciphertext block and the third ciphertext block. Only the second one is 174 00:15:36,839 --> 00:15:43,019 decrypted so C1 is decrypted using AES and your key for example. And the result here 175 00:15:43,019 --> 00:15:50,000 is not directly the plaintext but it will be XORed with C0 which will we would also 176 00:15:50,000 --> 00:15:55,470 call an initialization vector. Okay so whatever is written in here will be XORed 177 00:15:55,470 --> 00:16:00,550 on whatever comes out of this description and we get this plaintext here. Now when 178 00:16:00,550 --> 00:16:06,970 we flip a bit or when we change a certain byte in C0 we change whatever comes out 179 00:16:06,970 --> 00:16:11,930 here of the decryption and we will flip the same bit at the same position in 180 00:16:11,930 --> 00:16:18,891 the plaintext which is interesting because now when we say we use the original c0 and 181 00:16:18,891 --> 00:16:27,410 XOR it with p0. So when we XOR it when we XOR two times the same value we get 182 00:16:27,410 --> 00:16:33,870 basically a block of all zeros which is like a blank sheet of paper that we can 183 00:16:33,870 --> 00:16:38,570 write on and we call this thing - these two blocks here in combination - we call this a 184 00:16:38,570 --> 00:16:45,060 CBC gadget because we can reuse it as as much as we want. And when we now take a 185 00:16:45,060 --> 00:16:51,190 chosen ciphertext the nice thing is here we just XOR it - the chosen ciphertext on 186 00:16:51,190 --> 00:16:55,730 this initialization vector and it means that it will here result in a chosen 187 00:16:55,730 --> 00:17:00,640 ciphertext. So this requires that we know the value of 188 00:17:00,640 --> 00:17:06,870 p0. We need to guess this value of p0. And this is always true at least for S/MIME 189 00:17:06,870 --> 00:17:11,770 because all S/MIME emails start with a content type so we always know the first 190 00:17:11,770 --> 00:17:18,549 bytes. So this is valid. We can do this now. This was like the perfect 191 00:17:18,549 --> 00:17:23,949 scenario. It does have some drawbacks because when we tried to do the same for 192 00:17:23,949 --> 00:17:30,080 C1 we switch, we flip, a bit here then we flip a bit also here. But the decryption 193 00:17:30,080 --> 00:17:35,360 of C1 results in this random junk and we can't do anything about it. We don't know 194 00:17:35,360 --> 00:17:39,890 whatever will come out here and we can't control it. Okay we just have to deal with 195 00:17:39,890 --> 00:17:44,300 it. It's gonna be there when we use the CBC gadgets and we have to deal with it. 196 00:17:44,300 --> 00:17:49,480 We have to find a way to deal with it and now we can explain this behavior, you know? 197 00:17:49,480 --> 00:17:53,799 You remember this e-mail where we just flipped a single bit and we saw one block 198 00:17:53,799 --> 00:17:58,950 was destroyed basically. And then the other block we had like we can flip 199 00:17:58,950 --> 00:18:04,410 arbitrary bits. That's the explanation for it. Now how do you how do 200 00:18:04,410 --> 00:18:09,090 you deal with it. How should a proper crypto protocol deal with it. Usually you 201 00:18:09,090 --> 00:18:13,170 do something that is called a message authentication code. It's not a digital 202 00:18:13,170 --> 00:18:19,010 signature and message authentication code is something like a checksum a 203 00:18:19,010 --> 00:18:24,240 cryptographic checksum that is put right next to the 204 00:18:24,240 --> 00:18:28,170 encrypted message and that can be checked after decryption or for 205 00:18:28,170 --> 00:18:31,160 decryption. OK? The message authentication code is solely there to 206 00:18:31,160 --> 00:18:36,480 detect changes of a ciphertext. Digital signatures are different especially in the 207 00:18:36,480 --> 00:18:40,840 mail context because they're pretty much totally separate from 208 00:18:40,840 --> 00:18:47,460 the decryption part. So digital signatures are merely only used to display an icon 209 00:18:47,460 --> 00:18:55,710 for example for valid signature or invalid signature. And the problem here is a smart 210 00:18:55,710 --> 00:19:01,090 attacker can use a signed email and strip off the signature. Okay. We just strip off 211 00:19:01,090 --> 00:19:06,260 this part that we know there's a signature we can remove the signature and then this 212 00:19:06,260 --> 00:19:09,980 doesn't mean there will be an invalid signature but will get something like 213 00:19:09,980 --> 00:19:15,770 this. This is the icon in thunderbird for encrypted and not signed so we can simply 214 00:19:15,770 --> 00:19:21,970 strip off a signature and there's valid reasons to send an encrypted email without 215 00:19:21,970 --> 00:19:27,720 doing a digital signature on it, right? So we want to do both. So digital 216 00:19:27,720 --> 00:19:34,080 signatures won't prevent this attack. So how does S/MIME look like? 217 00:19:34,080 --> 00:19:39,170 Basically, I mean, most of you will have heard of MIME. MIME is like a standard 218 00:19:39,170 --> 00:19:45,310 that we use in emails anyway. Here we have an e-mail header. It looks very 219 00:19:45,310 --> 00:19:50,600 similar to HTTP. We have content type, colon, empty space, and then it's 220 00:19:50,600 --> 00:19:56,640 whatever is coming now for data. And then we have an e-mail body with envelope data 221 00:19:56,640 --> 00:20:01,040 and there we have the list of the of the encrypted session keys that this 222 00:20:01,040 --> 00:20:08,040 ciphertext was encrypted with. And after that we have the encrypted content info 223 00:20:08,040 --> 00:20:12,580 here. This part is the encrypted part where the actual message where the actual 224 00:20:12,580 --> 00:20:17,490 message lies. And now when you look closely and you squint you see there's no 225 00:20:17,490 --> 00:20:22,540 message authentication code and it's not because I left them out but because S/MIME 226 00:20:22,540 --> 00:20:26,669 doesn't define one. So S/MIME doesn't have a message authentication code which 227 00:20:26,669 --> 00:20:33,040 means it cannot detect bye the standard whether a message was changed - whether 228 00:20:33,040 --> 00:20:36,780 it was changed or not. So how can we attack this? 229 00:20:36,780 --> 00:20:41,590 How can we use it? I think you all have a feeling that this shouldn't work, OK? This 230 00:20:41,590 --> 00:20:46,950 is nothing that is supposed to work. This is the plain text that we want to 231 00:20:46,950 --> 00:20:51,210 exfiltrate and we only know the first block. We can pretty much always guess the 232 00:20:51,210 --> 00:20:56,380 first block because it's always content type: text/html or text/plain something 233 00:20:56,380 --> 00:21:03,960 like this. Now we can use this as a gadget here and write our arbitrary plain text. 234 00:21:03,960 --> 00:21:08,331 We have a random block and then chosen plain text. Then we copy it again. We 235 00:21:08,331 --> 00:21:12,780 again have a random block and some chosen plain text. Then we have a random block 236 00:21:12,780 --> 00:21:17,051 and a chosen plain text. Then we have a random block and a chosen plain text. Then 237 00:21:17,051 --> 00:21:23,780 we copy here the original ciphertext and we close this thing. And when you look 238 00:21:23,780 --> 00:21:28,120 closely again you'll see that we're building HTML here right. That's the base 239 00:21:28,120 --> 00:21:33,920 tag. That's an image tag. It uses a URL. That is not closed here. It will be closed 240 00:21:33,920 --> 00:21:40,490 down here. And this is effectively the URL. OK. A URL-path and will thus be 241 00:21:40,490 --> 00:21:45,200 exfiltrated. So when this XML is interpreted the actual plain text is sent 242 00:21:45,200 --> 00:21:51,750 to the server. And here we have an old Thunderbird. So that is a version of 243 00:21:51,750 --> 00:21:58,100 before May and before we did the disclosure. We first compose a message 244 00:21:58,100 --> 00:22:03,419 just to have some PGP ciphertext. This is a very secret message. And here we have 245 00:22:03,419 --> 00:22:09,850 the attack e-mail that is already prepared. And now it asks us 246 00:22:09,850 --> 00:22:13,820 "This remote content in here. What do you want to do?" And we just for the 247 00:22:13,820 --> 00:22:19,370 laughs just - like this - we just click it and we see always now when we select the 248 00:22:19,370 --> 00:22:27,320 message an HTTP request is done to this to this URL. And when we decode this - it 249 00:22:27,320 --> 00:22:32,330 looks pretty - it's encoded. But basically what we have here now you see here is the 250 00:22:32,330 --> 00:22:37,370 secret message and here is random junk. And here again is random junk, right? So 251 00:22:37,370 --> 00:22:43,400 it was successfully exfiltrated. This was the first proof of concept that we had. 252 00:22:43,400 --> 00:22:50,540 Right? And. Okay right? And so this works. This is like - this 253 00:22:50,540 --> 00:22:54,900 dialog here that you get in Thunderbird that ask if should I should a load remote 254 00:22:54,900 --> 00:22:59,580 images. Yes or no. This is basically everything that is keeping you from losing 255 00:22:59,580 --> 00:23:06,400 your your PGP or S/MIME ciphertext - S/MIME we're talking about S/MIME here. Now what we 256 00:23:06,400 --> 00:23:11,650 did next was we were asking ourselves, OK: It works with remote images. We understand 257 00:23:11,650 --> 00:23:16,200 this ,but what other back channels do we have? Back channel is something where 258 00:23:16,200 --> 00:23:20,690 your email client is communicating over the network. That's. So it's not 259 00:23:20,690 --> 00:23:26,460 necessarily exfiltrating but it's just like how can we convince a client to make 260 00:23:26,460 --> 00:23:33,150 a connection to somewhere else. Now I colored in red all the e-mail clients that 261 00:23:33,150 --> 00:23:37,940 require user interaction - that always require user interaction before it 262 00:23:37,940 --> 00:23:44,600 opens up a network connection somewhere. In orange we say that there's no user 263 00:23:44,600 --> 00:23:49,840 interaction. So these are the clients that allow for example loading remote images by 264 00:23:49,840 --> 00:23:54,470 default. OK. There's mail app for example there's gmail for example. They will not 265 00:23:54,470 --> 00:24:00,470 ask you they will just load it, OK? And but what we think is worse and therefore 266 00:24:00,470 --> 00:24:06,000 we colored it red are the clients that say we won't load remote images or any other 267 00:24:06,000 --> 00:24:11,650 way of communicating over the network. But then they will because we want bypasses. 268 00:24:11,650 --> 00:24:16,711 So because preventing loading remote images is basically a firewall and we 269 00:24:16,711 --> 00:24:23,210 bypass this local firewall and found several ways of extracting it. And there's 270 00:24:23,210 --> 00:24:27,549 even four or five of them that even allow JavaScript execution in a mail 271 00:24:27,549 --> 00:24:32,990 client. I mean come on this is really weird. So basically 40 of 47 clients have 272 00:24:32,990 --> 00:24:37,150 back channels that require no user interaction at all. And now we went out 273 00:24:37,150 --> 00:24:42,750 and tried to think about how can we exfiltrate plain text over it. And this 274 00:24:42,750 --> 00:24:48,809 is the final table the final result table for S/MIME and it looks pretty 275 00:24:48,809 --> 00:24:54,260 grim, right? So the red ones means it's vulnerable. That means we could exfiltrate 276 00:24:54,260 --> 00:24:59,539 plain text successfully. The ones with a dash they don't support S/MIME and Claws 277 00:24:59,539 --> 00:25:04,049 and Mutt we just didn't find any remote connection there. 278 00:25:04,049 --> 00:25:08,460 Okay. Which is good. Which is good. Yeah. That's a round of applause. Right. 279 00:25:08,460 --> 00:25:12,660 applause 280 00:25:12,660 --> 00:25:17,500 So I didn't say what S in S/MIME means. It actually means super. So it's super 281 00:25:17,500 --> 00:25:23,270 MIME. I don't know. Is it Super MIME? And basically his kryptonite. You know the the 282 00:25:23,270 --> 00:25:28,710 super villan of Super MIME is HTML. Right. You could say this. So S/MIME is broken 283 00:25:28,710 --> 00:25:34,070 by HTML and it's a very annoying kryptonite because it jumps you 284 00:25:34,070 --> 00:25:41,560 know like it's ugly colors. Now this is my Thunderbird that I have on my Mac at home. 285 00:25:41,560 --> 00:25:46,090 So this is like something that I recorded two days ago or so. And here I want to try 286 00:25:46,090 --> 00:25:52,940 to show you a way of exfiltrading the data without using HTML. So HTML here is 287 00:25:52,940 --> 00:25:59,030 disabled. This is the message that we want to break. And here we see the same message 288 00:25:59,030 --> 00:26:03,770 where we just changed, using gadgeting, we just changed the URL. 289 00:26:03,770 --> 00:26:10,210 So in Thunderbird when you disable HTML it will still highlight the links and when 290 00:26:10,210 --> 00:26:17,360 you click on it you're gonna exfiltrate plain text, right. So we require a little 291 00:26:17,360 --> 00:26:23,720 bit of user interaction but we can basically change a S/MIME ciphertext in a 292 00:26:23,720 --> 00:26:28,230 way that it will contain links and when you click a single link you will lose your 293 00:26:28,230 --> 00:26:34,000 your plain text. It's just a single click on it and the thing here is - I mean this 294 00:26:34,000 --> 00:26:38,401 is not a zero day in e-mail client, right? You just can't do anything about it. 295 00:26:38,401 --> 00:26:45,919 That's the problem. Okay. So people said okay so efail - just disable HTML and then 296 00:26:45,919 --> 00:26:51,600 you'll be fine. But the problem here is basically efail should work with any 297 00:26:51,600 --> 00:26:56,789 format that supports external connections. So as soon as you have a file format for 298 00:26:56,789 --> 00:27:01,740 example that supports external connections like PDF here, right? This is a PDF. When 299 00:27:01,740 --> 00:27:05,870 you click on it it will warn you and will say "Do you really want to make a 300 00:27:05,870 --> 00:27:12,170 connection to this domain?" And if you click "yes" you will do a connection there, 301 00:27:12,170 --> 00:27:18,281 right? And when you look at the how the URL is encoded in the PDF when you 302 00:27:18,281 --> 00:27:23,090 open the PDF in an hex editor you see it's just a string just like HTML, 303 00:27:23,090 --> 00:27:27,720 basically, right? So we could in theory use this as well. So you could change and 304 00:27:27,720 --> 00:27:32,409 S/MIME e-mail that it contains a PDF attachment and when you click the PDF 305 00:27:32,409 --> 00:27:37,030 attachment it will exfiltrate. With no user interaction, it works with with 306 00:27:37,030 --> 00:27:42,700 Microsoft Word for example. So that's a doc file and doc files allow external 307 00:27:42,700 --> 00:27:50,850 images that will load. It has no user interaction at all, so you just open it and 308 00:27:50,850 --> 00:27:55,059 the request will happen and the only problem that we're having here is it's not 309 00:27:55,059 --> 00:27:59,809 an ASCII URL, it's a Unicode URL. So I'm not sure whether this works with 310 00:27:59,809 --> 00:28:05,620 exfiltration but hey no user interaction, right? And. Now you could say "Okay PDF, 311 00:28:05,620 --> 00:28:10,221 Microsoft Word. That's very bad." - The same works in in LibreOffice and it's not a 312 00:28:10,221 --> 00:28:19,510 bug in the viewers, right? It's a basic feature that has been in these standards 313 00:28:19,510 --> 00:28:23,150 forever, right? So it's a thing that will be supported. It's not a zero day 314 00:28:23,150 --> 00:28:28,350 vulnerability in these viewers. It's just the file format support it, so you can 315 00:28:28,350 --> 00:28:34,000 abuse it. If you have some time: here's a challenge for you. If someone is able to 316 00:28:34,000 --> 00:28:42,780 make a successful demo for exfiltrating an S/MIME e-mail using a PDF or a doc file for 317 00:28:42,780 --> 00:28:49,010 example you will get a crate of Club Mate and some efail swag, if you want to? 318 00:28:49,010 --> 00:28:56,150 OK. So what happened after we disclosed it. Obviously we disclosed it to all the 319 00:28:56,150 --> 00:29:00,780 manufacturers and so on. And there's an S/MIME draft of a new RFC. So there's a 320 00:29:00,780 --> 00:29:05,299 working group and they already massaged the countermeasures into the 321 00:29:05,299 --> 00:29:11,110 current RFC draft. So for example to counter the CBC gadget attack they say: use 322 00:29:11,110 --> 00:29:19,895 a GCM which is not right now in the in the current RFC but in a draft it's already in. 323 00:29:19,895 --> 00:29:22,140 And so they're referencing the paper and they're saying: 324 00:29:22,140 --> 00:29:26,700 "OK you need to do this and do this as quickly as possible." And the second Efail 325 00:29:26,700 --> 00:29:32,820 attack which I'll talk about very soon is also mitigated in this in this RFC draft. 326 00:29:32,820 --> 00:29:39,030 OK. So S/MIME was pretty much broken. And you can just try to convince your e-mail 327 00:29:39,030 --> 00:29:45,790 client not to do any external connections. OK. That's all you have. Which at least, 328 00:29:45,790 --> 00:29:51,720 I mean, for cryptographer this means broken, OK? When you have to rely 329 00:29:51,720 --> 00:29:56,429 on the viewer not making any connections, the cryptography is broken. So what are 330 00:29:56,429 --> 00:30:02,310 the changes to OpenPGP because there are substantial differences to to OpenPGP. 331 00:30:02,310 --> 00:30:08,570 So first of all PGP uses a variation of the CFB-mode. It's not CBC-mode but CFB-mode. 332 00:30:08,570 --> 00:30:15,240 And they have pretty similar, they are very similar. So I won't show it here, 333 00:30:15,240 --> 00:30:19,860 if you are looking for the details, please look at the paper, but it's very similar. 334 00:30:19,860 --> 00:30:25,110 We can also flip bits and we also have these chunk bits in the middle and so on. 335 00:30:25,110 --> 00:30:29,590 What really caused a lot of headaches is the plain text compression. So that means 336 00:30:29,590 --> 00:30:37,580 in OpenPGP you don't encrypt directly the plain text but you deflate it 337 00:30:37,580 --> 00:30:42,010 before you encrypt it and that makes it very hard to guess plain text bytes, because 338 00:30:42,010 --> 00:30:47,340 for our attack to work we need to guess plain text bytes and when we know plain text 339 00:30:47,340 --> 00:30:52,990 certain parts of the plain text but it's deflated we have a problem. Okay? And what 340 00:30:52,990 --> 00:30:57,520 is also the most important thing is that OpenPGP defines a modification 341 00:30:57,520 --> 00:31:02,340 detection code, which is basically this is the message here. And the modification 342 00:31:02,340 --> 00:31:07,900 detection code is a hash of the message appended at the plain text and 343 00:31:07,900 --> 00:31:17,919 then encrypted. Okay? Which should detect plain text modifications. Now 344 00:31:17,919 --> 00:31:25,190 how does the OpenPGP standard say how to deal with broken MDCs when an MDC is not 345 00:31:25,190 --> 00:31:29,320 valid. Well, they say an implementation must treat an MDC failure as a security 346 00:31:29,320 --> 00:31:34,470 problem, but they don't say what is a security problem. What do you need to do 347 00:31:34,470 --> 00:31:39,750 then. And the implementation may allow the user access to the erroneous data. So the 348 00:31:39,750 --> 00:31:44,049 modified data may be passed on to the e-mail clients. 349 00:31:44,049 --> 00:31:49,840 And basically we tried to do this, right? But before we go there we looked at how 350 00:31:49,840 --> 00:31:53,730 many keys on the public key servers support MDCs at all. 351 00:31:53,730 --> 00:32:01,010 So MDC is a feature that came to OpenPGP in the early 2000s and before that it 352 00:32:01,010 --> 00:32:05,390 wasn't possible to use it. And so therefore to make this to make the change 353 00:32:05,390 --> 00:32:11,730 from software that doesn't use MDC versus software that that does use MDC they 354 00:32:11,730 --> 00:32:17,580 encoded in the key whether you support MDC or not. And here you see those keys that 355 00:32:17,580 --> 00:32:22,000 were generated in 2000 they don't support MDCs and the green ones - they support 356 00:32:22,000 --> 00:32:30,710 it. But, the problem is, when you accumulate all the valid keys in PGP that don't 357 00:32:30,710 --> 00:32:37,190 support MDCs we still see that at the key servers right now in 2017 we have 358 00:32:37,190 --> 00:32:42,289 something like one point seven million of the keys that don't support MDC. So when I 359 00:32:42,289 --> 00:32:48,560 sent an email to one of these people my implementation will not use an MDC, or 360 00:32:48,560 --> 00:32:55,330 right? They - yeah I'll come to this later. So we thought about okay how 361 00:32:55,330 --> 00:33:00,070 can we make a structured analysis how the MDC is treated with. The OpenPGP standard 362 00:33:00,070 --> 00:33:05,419 is not clear what to do with it. And so we have three test cases. First of all, we 363 00:33:05,419 --> 00:33:10,909 try to strip the MDC. So we just use this encrypted packet here and strip off at the 364 00:33:10,909 --> 00:33:17,039 end. We just say OK it's incorrect, right? So we make our modification in the 365 00:33:17,039 --> 00:33:24,080 plain text and see whether they will detect this error or we could also do this 366 00:33:24,080 --> 00:33:29,230 downgrade from the packet type that supports MDC to the old packet type that 367 00:33:29,230 --> 00:33:34,840 does not support MDC, right? And when we check this, especially with the most widely 368 00:33:34,840 --> 00:33:40,260 used clients we saw that Thunderbird and Enigmail and Apple Mail and GPG tools they 369 00:33:40,260 --> 00:33:46,070 basically ignored the MDC and that was the reason why we didn't see the MDC as very 370 00:33:46,070 --> 00:33:51,670 important because in our test systems it wasn't just treated, OK? 371 00:33:51,670 --> 00:33:57,700 Now what are the Efail related changes to OpenPGP. And we're talking about the 372 00:33:57,700 --> 00:34:02,610 changes that were done after we disclosed this to them. 373 00:34:02,610 --> 00:34:05,640 First of all they made something that is very important. 374 00:34:05,640 --> 00:34:11,090 They say that the packet type and PGP that does not support MDC is now obsolete. So 375 00:34:11,090 --> 00:34:16,220 you must not use this packet type anymore. You must ignore it. Which also means that 376 00:34:16,220 --> 00:34:21,520 when you have used non MDC messages in your mailbox you will not be able to open 377 00:34:21,520 --> 00:34:28,340 them anymore, OK? Yeah. That's a problem. Then they also said: "OK, what 378 00:34:28,340 --> 00:34:33,860 is supposed to happen when the MDC was wrong?" And this is the text that we 379 00:34:33,860 --> 00:34:38,360 have already seen where we said "the implementation may allow the user access 380 00:34:38,360 --> 00:34:43,240 to the erroneous data" and they changed this in the current draft to "the 381 00:34:43,240 --> 00:34:48,280 implementation should not allow the user to process the erroneous data." Right? They 382 00:34:48,280 --> 00:34:53,190 must still warn the user. And so on and so on. But it should not allow the user, which 383 00:34:53,190 --> 00:35:02,410 is good. That's a good change. So this was the - this MDC check is a bit troublesome 384 00:35:02,410 --> 00:35:08,450 because it's right at the end of the plain text, so you can only check the MDC 385 00:35:08,450 --> 00:35:13,250 when you have fully decrypted the message which means when you have a large e-mail 386 00:35:13,250 --> 00:35:18,510 or a large back-up or so you have to fully decrypt it and afterwards only you can tell 387 00:35:18,510 --> 00:35:24,430 whether it was valid or not. And the solution of GnuPG is that it will just 388 00:35:24,430 --> 00:35:30,619 stream - while it decrypts it will stream the data to to the e-mail client and 389 00:35:30,619 --> 00:35:34,700 afterwards tell it whether it's supposed to use it or not, OK? So here's your 390 00:35:34,700 --> 00:35:39,891 email, here's your e-mail, here's your - Oh no! You must not use it, please forget it, right? 391 00:35:39,891 --> 00:35:44,980 Which is not fine. This is not a safe way of doing it. So the current OpenPGP draft, 392 00:35:44,980 --> 00:35:48,690 before Efail, before we did the disclosure already supported something 393 00:35:48,690 --> 00:35:53,520 that they called chunking. So they not only have an MDC some kind of a MAC at the 394 00:35:53,520 --> 00:35:57,800 end of the plain text, but they chunk it into more chunks and each of these 395 00:35:57,800 --> 00:36:04,670 has a MAC - a real MAC - which is much better because, right? You can authenticate chunks 396 00:36:04,670 --> 00:36:13,660 and cache it before you give it to the app. The problem is that the standard says 397 00:36:13,660 --> 00:36:20,920 that 120MB is like a good chunk size which is far too big, right? Far too big. 398 00:36:20,920 --> 00:36:26,370 And when we look at the Efail related changes to GnuPG. So for example MDCs 399 00:36:26,370 --> 00:36:31,541 where they used to be warnings. So when GnuPG was detecting that an MDC is not 400 00:36:31,541 --> 00:36:37,589 there or it's wrong, they would just warn it but it would still print it out. GnuPG 401 00:36:37,589 --> 00:36:45,260 now, like, fails it makes a hard failure now and GnuPG now always uses the MDC 402 00:36:45,260 --> 00:36:50,811 independently if the key denotes that the receiver can use it or not. Which 403 00:36:50,811 --> 00:36:54,650 is also very good but it is a breaking change, rightß So when you have people that 404 00:36:54,650 --> 00:36:58,290 send you PGP e-mails and you can't decrypt them anymore you have to tell them that 405 00:36:58,290 --> 00:37:05,630 they need to update GnuPG. They also say that the default chunk size 406 00:37:05,630 --> 00:37:11,599 is 120 megabyte which means they still stream unauthenticated plaintext so they 407 00:37:11,599 --> 00:37:17,960 missed this opportunity to make this go away, which is a pity. 408 00:37:17,960 --> 00:37:24,380 Okay. What is not answered neither by S/MIME nor by PGP: How to deal with past 409 00:37:24,380 --> 00:37:28,150 e-mails? So, basically all the emails that you're sending in S/MIME encrypted right 410 00:37:28,150 --> 00:37:34,690 now are insecure and the old ones that don't use MDC in PGP they are also not 411 00:37:34,690 --> 00:37:39,480 secure and nobody talks about what's supposed to happen with them, right? 412 00:37:39,480 --> 00:37:43,920 Okay, that was the interesting attack. That 413 00:37:43,920 --> 00:37:46,790 was the one that was really cryptographically involving, 414 00:37:46,790 --> 00:37:51,130 that required changes to the standards and so on. Now let's look at something that's 415 00:37:51,130 --> 00:37:56,790 a bit more silly. Okay, so we have Alice writes an e-mail to Bob, we encrypt it and 416 00:37:56,790 --> 00:38:02,730 send it off to Bob. Now here we have the original e-mail and now Eve composes an 417 00:38:02,730 --> 00:38:08,680 attack e-mail. And it somehow gained access to the PGP ciphertext, it also works for 418 00:38:08,680 --> 00:38:13,140 S/MIME, right, it's independent. It's a vulnerability in the in the mail clients. 419 00:38:13,140 --> 00:38:18,349 And now we don't change the ciphertext we just surround it with other MIME parts 420 00:38:18,349 --> 00:38:24,310 that are of type HTML and that exfiltrated the same as we did before only that we 421 00:38:24,310 --> 00:38:28,910 don't need to change the ciphertext at all, right? 422 00:38:28,910 --> 00:38:34,109 So basically when this is decrypted it will be replaced with a plain text in 423 00:38:34,109 --> 00:38:38,950 place and then it will be merged into one document. And now I have to tell you 424 00:38:38,950 --> 00:38:44,840 something that many people don't seem to know. All mail clients, except mutt, they 425 00:38:44,840 --> 00:38:51,800 use HTML to view emails. So even when you disable HTML it basically means that they 426 00:38:51,800 --> 00:38:57,531 will use incoming HTML e-mails, parse them, transform them into something that 427 00:38:57,531 --> 00:39:04,450 looks like ASCII and display this in HTML. Okay. So, right? There is no such thing as 428 00:39:04,450 --> 00:39:10,440 ASCII e-mails with most of the clients. So but here now you have one HTML document 429 00:39:10,440 --> 00:39:15,990 which basically means the same, it will exfiltrate and very easy. Okay? So let's 430 00:39:15,990 --> 00:39:22,400 look here and on this video here we have we create one ciphertext we encrypt it. We 431 00:39:22,400 --> 00:39:29,230 use Apple Mail because this was one of the two vulnerable clients and we just open this 432 00:39:29,230 --> 00:39:36,630 email in order to view the PGP ciphertext, right? We look at the raw code of the 433 00:39:36,630 --> 00:39:45,360 ciphertext, raw source and we scroll down and there we see there's the PGP message, 434 00:39:45,360 --> 00:39:52,530 OK? Now in the next minute Eve somehow gained access to this ciphertext and it 435 00:39:52,530 --> 00:40:01,310 will compose a attacker email which is hidden in a python program that we just 436 00:40:01,310 --> 00:40:06,140 use to generate an e-mail, so this happened now in the background, you will 437 00:40:06,140 --> 00:40:13,910 see an e-mail is incoming. Here we just open the access log for this and when 438 00:40:13,910 --> 00:40:20,690 we click on it, right at this time, it will be exfiltrated. Okay. So very easy attack, 439 00:40:20,690 --> 00:40:24,950 anyone can do this, you don't need to know anything about cryptography you only need 440 00:40:24,950 --> 00:40:28,970 to understand a little bit of HTML and a little bit of MIME and you can execute 441 00:40:28,970 --> 00:40:37,119 this. You know what's worse than this? How about we just exfiltrate many e-mails with 442 00:40:37,119 --> 00:40:42,050 a single attack e-mail because we could basically use - here we open an image, 443 00:40:42,050 --> 00:40:46,700 here's the ciphertext. Here would close it. Then we open another image. Here's the 444 00:40:46,700 --> 00:40:51,680 second ciphertext. Then we close it. Then we open a third one and so on and so on. 445 00:40:51,680 --> 00:40:56,910 And what you see here, on the left hand side you see there's 100 ciphertext packed 446 00:40:56,910 --> 00:41:02,630 into one e-mail that is sent to this client. I blurred this because I used 100 447 00:41:02,630 --> 00:41:08,800 GPG e-mails that I sent, right? I didn't want to show them to you. Now this takes 448 00:41:08,800 --> 00:41:13,050 some time. Now it's like decrypting, decrypting, decrypting, decrypting. And when 449 00:41:13,050 --> 00:41:17,580 you look at the terminal on the left side you see 100 e-mails getting exfiltated, 450 00:41:17,580 --> 00:41:23,599 right? Automatically simply by opening a single email. Okay this is horrifying. 451 00:41:23,599 --> 00:41:30,700 Applause 452 00:41:30,700 --> 00:41:34,839 And when I say horrifying I'm really literally mean: this is horrifying. So we 453 00:41:34,839 --> 00:41:40,099 were like afraid OK when we disclose this then people will start attacking this 454 00:41:40,099 --> 00:41:48,410 within the hour or so, OK? And so we disclosed this in early 2018. End of 2017, 455 00:41:48,410 --> 00:41:57,010 early 2018. And in May 2018 we had already shifted the disclosure date twice and in 456 00:41:57,010 --> 00:42:01,680 May we said OK we won't shift this anymore. And so therefore we said OK we go 457 00:42:01,680 --> 00:42:06,309 online but we want to make a preannouncement. We want to 458 00:42:06,309 --> 00:42:11,690 warn the people that if you use PGP or S/MIME for very sensitive 459 00:42:11,690 --> 00:42:17,000 communication you should disable it in your e-mail client for now. These are the 460 00:42:17,000 --> 00:42:22,720 original tweets that we sent and we said: OK, In a day, OK, when you have done this 461 00:42:22,720 --> 00:42:27,050 then we will disclose everything, OK? We will go online with the paper and 462 00:42:27,050 --> 00:42:33,020 everything. And now I mean people started talking about it and then the GnuPG people 463 00:42:33,020 --> 00:42:37,950 started tweeting and they basically broke the embargo. So they told OK the attack is 464 00:42:37,950 --> 00:42:41,310 working like this. First you do this and then you do that and then you do that and 465 00:42:41,310 --> 00:42:45,160 they were blabbing and so on and then they sent it, which is annoying, but then they 466 00:42:45,160 --> 00:42:49,780 sent this. So they said know that new GnuPG team was not contacted by them in 467 00:42:49,780 --> 00:42:55,700 advance. And this, this really, OK? Then it got really hot. Okay. 468 00:42:55,700 --> 00:43:00,380 This is not true. I will show you later, but I mean this tipped off almost anyone 469 00:43:00,380 --> 00:43:06,160 in the INFOSEC community. And we got hate e-mails and yeah people were really mad 470 00:43:06,160 --> 00:43:10,200 with them, I posted "We did contact them." and this is like the, these are the 471 00:43:10,200 --> 00:43:13,820 original dates when I talked with Werner Koch which is the main developer of of 472 00:43:13,820 --> 00:43:18,150 GnuPG. But it was too late. I mean they they recognized that they said: "Oh yeah, oh 473 00:43:18,150 --> 00:43:25,380 I forgot!" Right? That they sent this paper here. That writes EFAIL with our names and 474 00:43:25,380 --> 00:43:31,430 a date and an embargo and so on and so on. And they said they forgot it, OK? But I 475 00:43:31,430 --> 00:43:37,180 mean we lost at this point, OK? People were hating us. I was called a murderer. 476 00:43:37,180 --> 00:43:41,290 And and so on and so on and so it was really, really weird. If you're interested 477 00:43:41,290 --> 00:43:45,150 of how things worked, right, this is an independent summary of the disclosure 478 00:43:45,150 --> 00:43:50,790 timeline from Thomas Ptacek which is not involved with us at all we have never met 479 00:43:50,790 --> 00:43:55,190 him, but who was so kind to just compile this just from public information that are 480 00:43:55,190 --> 00:44:01,590 really reliable. So and then something happened that 481 00:44:01,590 --> 00:44:07,320 we what we could foresee. People were finding - like there were some patches - 482 00:44:07,320 --> 00:44:12,130 and I told you already that we saw that these patches are not sufficient. 483 00:44:12,130 --> 00:44:18,630 And two days later people came up with new style of attacks, new EFAIL attacks and 484 00:44:18,630 --> 00:44:23,040 this was Hanno who found something. So he said I found a trivial bypass EFAIL is 485 00:44:23,040 --> 00:44:27,660 still exploitable with the latest Enigmail and Thunderbird - that is three days after 486 00:44:27,660 --> 00:44:32,030 we did the disclosure and he responsibly disclosed this and so on and so on. So 487 00:44:32,030 --> 00:44:37,460 everything is fine. Micah Lee, from The Intercept, also developed a proof of 488 00:44:37,460 --> 00:44:43,080 concept exploit that works against Apple Mail and GPG tools also 2 or 3 days later. 489 00:44:43,080 --> 00:44:48,030 So we were right, right? So people were finding ways to circumvent the fixes that 490 00:44:48,030 --> 00:44:54,780 were in there. But still I mean, OK, media was blowing up, OK? And we didn't actually 491 00:44:54,780 --> 00:45:01,849 understand what was really going on there. But you have to know I mean PGP is like, 492 00:45:01,849 --> 00:45:07,210 one of the main users of PHP, eh of PGP is journalists, right. 493 00:45:07,210 --> 00:45:11,130 So and they basically, I mean you show it to them look that's the attack and they 494 00:45:11,130 --> 00:45:16,690 will like what? That's how easy it is? And that's what you see in the press I guess. 495 00:45:16,690 --> 00:45:23,050 So what are the lessons learned from the disclosure. So first of all: people kept 496 00:45:23,050 --> 00:45:27,119 complaining, I mean people kept complaining to us that we didn't stick to 497 00:45:27,119 --> 00:45:32,310 the 90 day disclosure deadline. So we had more than 200 days disclosure and 498 00:45:32,310 --> 00:45:36,500 we postponed it and so on and so on. And in the aftermath it didn't make sense 499 00:45:36,500 --> 00:45:40,680 because we were postponing and postponing it and there were still no reliable fixes. 500 00:45:40,680 --> 00:45:43,910 So we should have stuck to the 90 day disclosure deadline because after we 501 00:45:43,910 --> 00:45:48,050 disclosed it they were in a rush. They released patches and stuff like, you know? 502 00:45:48,050 --> 00:45:54,190 So it was, yeah. Be careful with disclosure preannouncements. The problem 503 00:45:54,190 --> 00:46:00,861 here is people will speculate about the details. You know, because when you say: "OK 504 00:46:00,861 --> 00:46:04,650 we'll disclose something tomorrow but we're not going to speak today" - then 505 00:46:04,650 --> 00:46:08,839 journalists will speak to some random people, right? That they just get their 506 00:46:08,839 --> 00:46:14,390 hands on and they will just try to guess what it could be. And this means they will 507 00:46:14,390 --> 00:46:19,700 underrate the risk or they will overrate the risk. Underrate the risk is just 508 00:46:19,700 --> 00:46:23,540 disable external images, loading external images, then you're fine. That is 509 00:46:23,540 --> 00:46:30,240 underrating and overrating is like: "OK, PGP is broken, forever", right? Which is not 510 00:46:30,240 --> 00:46:34,510 the case. And they will spread this false information and you still see this out 511 00:46:34,510 --> 00:46:40,630 there, right? It's documented. People have issued like papers and so on and so on 512 00:46:40,630 --> 00:46:45,320 with wrong information, provably wrong information. Which is bad and so these 513 00:46:45,320 --> 00:46:48,480 disclosure preannouncements are bad because you're not in control of the 514 00:46:48,480 --> 00:46:53,040 communication anymore. That's very bad. And controlling information flow really is 515 00:46:53,040 --> 00:46:57,200 key after you do the disclosure because otherwise you get the wrong story out and 516 00:46:57,200 --> 00:47:05,810 this this doesn't help at all. Okay. Let's look at the last attack, and we call this 517 00:47:05,810 --> 00:47:10,300 reply to attacker. And I have to give credit to the people of 518 00:47:10,300 --> 00:47:16,730 Cure53, which also found this bug or a similar very similar version of this bug 519 00:47:16,730 --> 00:47:20,859 as well and disclosed it. But they were first to disclose it so credit goes to 520 00:47:20,859 --> 00:47:27,270 them, OK? Here, the attacker scenario is I get an e-mail from some person which 521 00:47:27,270 --> 00:47:32,280 makes sense. It's a benign e-mail. Okay. It doesn't look bad at all. And a person 522 00:47:32,280 --> 00:47:37,051 wants to trick me to answer this e-mail. So this is the e-mail that I get. There's 523 00:47:37,051 --> 00:47:42,940 some footer here and stuff like that. And now we look in the video, it's an old 524 00:47:42,940 --> 00:47:51,000 version of Mail that we use here. Right. This is the e-mail. We just - and I answer 525 00:47:51,000 --> 00:47:54,640 it, right? Because people were asking me for an appointment they want to 526 00:47:54,640 --> 00:47:58,730 meet me and I was like yeah sure after my talk let's just meet. And this is the 527 00:47:58,730 --> 00:48:05,000 attacker e-mail that the attackers opened and when you look closely down here you'll 528 00:48:05,000 --> 00:48:10,790 see secret stuff here that shouldn't be in there and that I didn't send actually. 529 00:48:10,790 --> 00:48:17,190 Let's look at the plain text of this e-mail and the source code of this e-mail. 530 00:48:17,190 --> 00:48:23,690 And when you scroll down you see it's a mix of just a normal, just a normal 531 00:48:23,690 --> 00:48:28,330 e-mail, benign e-mail. And down here you find a ciphertext it can be S/MIME it can 532 00:48:28,330 --> 00:48:35,270 be PGP. It doesn't really matter. And what's happening now is: 533 00:48:35,270 --> 00:48:39,700 I get this e-mail, this here will be decrypted from my mail client and I don't 534 00:48:39,700 --> 00:48:44,260 see it because I won't scroll down. I'm a lazy person I'm a top poster. Right, now 535 00:48:44,260 --> 00:48:50,180 you can go boo and so on, OK. But still I mean top posting is considered evil here. 536 00:48:50,180 --> 00:48:55,250 Because I basically - the attacker was sending me an e-mail where ciphertext was 537 00:48:55,250 --> 00:48:59,940 hidden in there that my e-mail client was decrypting and I was replying to this 538 00:48:59,940 --> 00:49:04,200 e-mail sending the attacker the actual plain text that I just decrypted, without 539 00:49:04,200 --> 00:49:10,530 knowing it. Okay that's silly. Okay. Come on. That's really silly. Okay. What are 540 00:49:10,530 --> 00:49:16,099 you supposed to do now. What are the things that you should do. And the 541 00:49:16,099 --> 00:49:20,829 important question here is you should ask yourself are you targeted by motivated 542 00:49:20,829 --> 00:49:23,310 attackers? And a motivated attacker is not 543 00:49:23,310 --> 00:49:29,230 necessarily the NSA or so, right? It can be just, I don't know, Cybercrime people or 544 00:49:29,230 --> 00:49:35,359 so. I mean we we basically came up with these attacks with 8 people abd the better 545 00:49:35,359 --> 00:49:40,270 part of a year. Which is not you know, which is a lot of research actually but 546 00:49:40,270 --> 00:49:44,521 it's not like comparable to a nation state. Right. So if you're targeted by a 547 00:49:44,521 --> 00:49:50,420 motivated attacker and if you say yeah you are probably targeted by motivated 548 00:49:50,420 --> 00:49:55,150 attackers then avoid e-mail in total. E-mail is not made for secure 549 00:49:55,150 --> 00:50:01,990 communication. Okay. If you can't avoid it and there's people who can't avoid using 550 00:50:01,990 --> 00:50:06,790 e-mail they can't just use signal or any other chat client, then definitely use 551 00:50:06,790 --> 00:50:15,000 OpenPGP and encrypt and decrypt outside of the mail client. If you are probably not 552 00:50:15,000 --> 00:50:21,960 targeted by motivated attackers, which is most of you and I presume I would count to 553 00:50:21,960 --> 00:50:27,510 the same people here, definitely prefer OpenPGP over S/mime because S/mime remains 554 00:50:27,510 --> 00:50:32,370 broken. Right, there is a standard coming up which fixes most of the stuff. Disable 555 00:50:32,370 --> 00:50:39,210 HTML for encrypted emails and this is like not that easy. Note that most email 556 00:50:39,210 --> 00:50:45,440 clients use HTML by default and it might be okay if you fix it, the people that I 557 00:50:45,440 --> 00:50:51,020 hear in the audience. But think of all the people who are not here in this audience, 558 00:50:51,020 --> 00:50:56,260 who are not technically versed and so on and so on and they will not disable HTML, 559 00:50:56,260 --> 00:51:00,800 right. So be careful with attachments which is also very difficult, how can you 560 00:51:00,800 --> 00:51:05,359 be careful with attachments. Right. This is not very good. And don't top post, 561 00:51:05,359 --> 00:51:12,589 don't cite text in a reply. Okay. That was my talk. If you want to meet us afterwards 562 00:51:12,589 --> 00:51:16,670 we're gonna go to the Chaos West. There's a huge and lighted palm and we'll be there 563 00:51:16,670 --> 00:51:23,020 if you have questions you can ask us there. Thank you very much. 564 00:51:23,020 --> 00:51:33,190 applause 565 00:51:33,190 --> 00:51:37,780 Herald: Thanks a lot Sebastian, for this interesting talk. We still have some time 566 00:51:37,780 --> 00:51:41,960 for questions, so if you would like to ask some questions to Sebastian then please 567 00:51:41,960 --> 00:51:47,170 queue up behind the microphones, and please try to be concise with your 568 00:51:47,170 --> 00:51:51,319 question, and please get close to the microphone, because the mixing Angel in 569 00:51:51,319 --> 00:51:56,180 the back is able to make it quieter, but hardly much louder, if your distance is 570 00:51:56,180 --> 00:51:59,050 not right. So let's start with microphone number 2. 571 00:51:59,050 --> 00:52:05,950 Microphone 2: Test, test. Hello. I'm actually tested the HTML exfiltrate you 572 00:52:05,950 --> 00:52:18,589 described. But I found out, in Thunderbird, the HTML parsing of 573 00:52:18,589 --> 00:52:20,950 Thunderbird was actually helping against the attack, because the word tags was just 574 00:52:20,950 --> 00:52:25,910 disabled any possibility to exfiltrate the message, because there were some div tag, 575 00:52:25,910 --> 00:52:31,760 which is closing your image tag at every point in time 576 00:52:31,760 --> 00:52:37,140 Sebastian: Yeah, so, this is one of the fixes, that they did. I mean it doesn't 577 00:52:37,140 --> 00:52:40,990 help with the cryptography, but it makes the exeltration a bit more difficult. If 578 00:52:40,990 --> 00:52:45,890 you want to do the testing, try the versions from before May this year. 579 00:52:45,890 --> 00:52:48,942 Microphone 2: I did, I did with the old versions. 580 00:52:48,942 --> 00:52:51,680 Sebastian: We can talk afterwards if you want, and I will show it to you. 581 00:52:51,680 --> 00:52:56,690 Herald: We are streaming all the talks on the Internet and we have people on the 582 00:52:56,690 --> 00:53:00,210 internet that would like to ask a question, so please a question from the 583 00:53:00,210 --> 00:53:02,630 internet. Signal Angel: Yeah, so there were actually 584 00:53:02,630 --> 00:53:03,748 many questions in the direction of problems with MIME standards. So, like, 585 00:53:03,748 --> 00:53:05,040 what they were asking was basically "Woudn't the exploitation be prevented, if 586 00:53:05,040 --> 00:53:21,270 the email clients would handle the standart correctly?" 587 00:53:21,270 --> 00:53:26,790 Sebastian: Mhm, okay. So, I am not sure whether they meant the MIME standards or 588 00:53:26,790 --> 00:53:31,710 the S/MIME standards. So basically, the MIME standards are that one to blame for 589 00:53:31,710 --> 00:53:37,150 the direct exoltration attacks, were you have multiple MIME-parts that are mixed 590 00:53:37,150 --> 00:53:42,710 together, and the MIME standards don't state explicitly how to do this. So they 591 00:53:42,710 --> 00:53:48,339 don't have any rules of how to handle it, and basically I think, as a rule of thumb, 592 00:53:48,339 --> 00:53:53,500 when you try to be concise and complete with implementing the MIME standard you 593 00:53:53,500 --> 00:53:58,680 were vulnerable, and if you were just lazy and just for example decrypted the first 594 00:53:58,680 --> 00:54:04,110 MIME-part or so you were not vulnerable. So, leaving out much of the stuff made you 595 00:54:04,110 --> 00:54:08,810 more secure, which is weird. Herald: Thank you for the answer. We have 596 00:54:08,810 --> 00:54:12,680 another question in this hall, on microphone number 6, and please remember 597 00:54:12,680 --> 00:54:14,401 to be concise and get close to the microphone. 598 00:54:14,401 --> 00:54:16,740 Microphone 6: What do you see Audio feedback loop 599 00:54:16,740 --> 00:54:19,750 Microphone 6: Sorry. Herald: Very good! We can always make it 600 00:54:19,750 --> 00:54:20,750 quieter. Applause 601 00:54:20,750 --> 00:54:22,270 Herald: Laughing 602 00:54:22,270 --> 00:54:25,970 Microphone 6: What do you see as a future 603 00:54:25,970 --> 00:54:31,430 replacement to PGP for email encryption? Sebastian: So actually, we brainstormed a 604 00:54:31,430 --> 00:54:37,579 little, because PGP lacks many of the modern properties, like forward secrecy 605 00:54:37,579 --> 00:54:43,100 and stuff like this, but it turns out that it's not very easy to do this. Especially 606 00:54:43,100 --> 00:54:50,079 not with - when you don't want to do changes to SMTP and IMAP. So, it's gonna 607 00:54:50,079 --> 00:54:57,069 be difficult. It's gonna be difficult to make it more secure than PGP, besides the 608 00:54:57,069 --> 00:55:04,930 critisism that I already brought in. So, E-Mail is not a very good protocol, or the 609 00:55:04,930 --> 00:55:09,069 protocols that are built in E-Mail are not a very good to build proper crypto on top 610 00:55:09,069 --> 00:55:12,650 of it. Herald: Thank you for this answer, so 611 00:55:12,650 --> 00:55:17,560 maybe we need build something entirely new from scratch. There is another question on 612 00:55:17,560 --> 00:55:20,130 Microphone number 2, if I see this correctly. 613 00:55:20,130 --> 00:55:26,460 Microphone 2: So, do I understand it correctly, it looks like there won't be a 614 00:55:26,460 --> 00:55:33,050 fix for that multi part attacks, like there is the normal plain text the 615 00:55:33,050 --> 00:55:36,650 attacker writes, and then there is a ciphertext, which my mail client decrypts; 616 00:55:36,650 --> 00:55:40,650 there won't be a fix for that? Sebastian: There is a fix, so the mail 617 00:55:40,650 --> 00:55:44,839 clients are fixed, so that's because that's a non-breaking fix, that's a good 618 00:55:44,839 --> 00:55:49,089 thing here. So you can't fix S/MIME, for example, because it would be a breaking 619 00:55:49,089 --> 00:55:55,020 fix, but you could fix the email clients, which was a lot of work, because MIME 620 00:55:55,020 --> 00:55:59,190 parsers are very complex, but they fixed it, so it's not supposed to work in 621 00:55:59,190 --> 00:56:03,470 current mail clients anymore. The direct exfiltration part. 622 00:56:03,470 --> 00:56:08,380 Herald: So we still have quite some time left and so I would like to have another 623 00:56:08,380 --> 00:56:11,320 question form the internet. Signal Angel: Yes, so one user asked 624 00:56:11,320 --> 00:56:14,440 "would you recommend a better path to integrate PGP into the E-Mail 625 00:56:14,440 --> 00:56:27,150 clients instead of using extensions?" Sebastian: Yea. I'm not a - I mean 626 00:56:27,150 --> 00:56:31,920 I already talked about this that not many people use PGP. And also not many people 627 00:56:31,920 --> 00:56:36,750 use S/MIME. I'm not sure, whether this will be 628 00:56:36,750 --> 00:56:45,140 increased, when OpenPGP spilled into most of the mail clients, because you could use 629 00:56:45,140 --> 00:56:50,750 S/MIME, you know. As soon as S/MIME is fixed it will be fine to use S/MIME, and 630 00:56:50,750 --> 00:56:54,661 it's already in all the clients in there, but not many people use it. 631 00:56:54,661 --> 00:56:59,900 So it would be nice, if we could have it, but I'm not sure whether it would have 632 00:56:59,900 --> 00:57:06,910 much of an effect on the amount of people who use it. Herald Angel: Thank you. And 633 00:57:06,910 --> 00:57:10,849 we have another question from Microphone number 7. 634 00:57:10,849 --> 00:57:16,819 Microphone 7: Yes, hello. So I work with journalists - here! here! here! 635 00:57:16,819 --> 00:57:20,040 Herald Angel: There, in the very back. Microphone 7: There, Hello. Hello. 636 00:57:20,040 --> 00:57:21,599 Sebastian: Hi. Oh, yeah hi. How's it going? 637 00:57:21,599 --> 00:57:27,340 Microphone 7: So I work with journalists, and May was very annoying, and I cannot 638 00:57:27,340 --> 00:57:32,369 stress enough how important it is to - the point you've made about communicating 639 00:57:32,369 --> 00:57:33,440 around disclosure. Sebastion: Yeah 640 00:57:33,440 --> 00:57:37,470 Microphone 7: We were scrambling. I work with about 200 journalists, who use PGP 641 00:57:37,470 --> 00:57:40,710 daily, and we were scrambling for any bit of information. 642 00:57:40,710 --> 00:57:43,119 Sebastian: Yeah. Microphone 7: What, what should we do? It 643 00:57:43,119 --> 00:57:44,390 was, it was madness. Sebastian: Yes. 644 00:57:44,390 --> 00:57:49,250 Microphone 7: But I actually have a question. The question is: Have you played 645 00:57:49,250 --> 00:57:55,500 with Mailpile? I've read the, I've read the paper. In the paper Mailpile, I think 646 00:57:55,500 --> 00:58:01,700 I remember, was mentioned, and I vaguely remember it was mentioned in a positive 647 00:58:01,700 --> 00:58:07,619 light, but I just want to make sure, if that's the right thing that I should get 648 00:58:07,619 --> 00:58:10,230 from this paper. Sebastian: I don't know it on the top of 649 00:58:10,230 --> 00:58:15,910 my head, but if you come afterwards to Chaos West we'll get out the paper and 650 00:58:15,910 --> 00:58:18,690 look at the tests, and then we can answer this question. Cool. 651 00:58:18,690 --> 00:58:22,060 Microphone 7: Thank you. Herald: Okay. We have another question on 652 00:58:22,060 --> 00:58:27,569 the microphone number 6. Microphone 6: So, I'd like to know if 653 00:58:27,569 --> 00:58:37,050 there's any difference in the attack surface, regarding OpenPGP, if plain, 654 00:58:37,050 --> 00:58:47,190 like, plain PGP is used or PGP/MIME. For example if you set your E-Mail client to 655 00:58:47,190 --> 00:58:58,170 disallow decryption of inline PGP. Sebastian: Yeah, that's a very good 656 00:58:58,170 --> 00:59:02,320 question. I mean the reply to attacker attack that I 657 00:59:02,320 --> 00:59:08,230 just showed at the last. The Q53 people came up with a version that worked with 658 00:59:08,230 --> 00:59:15,729 inline PGP, and we showed that it's also possible with PGP/MIME. So, I wouldn't say 659 00:59:15,729 --> 00:59:22,170 - I mean, most of the people use PGP/MIME. Inline PGP is not used anymore. The 660 00:59:22,170 --> 00:59:27,280 problem is, when you don't decrypt PGP, inline PGP, then you have a problem when 661 00:59:27,280 --> 00:59:32,520 people use PGP at an external application. They can't copy and paste the ciphertext 662 00:59:32,520 --> 00:59:37,170 in there anymore, which is also very annoying. And I also told you that it's, I 663 00:59:37,170 --> 00:59:40,910 think it's a very good idea to do this outside of the mail client, if you're in 664 00:59:40,910 --> 00:59:47,590 a, if you face motivated attackers. So, it's not that easy. it would be a breaking 665 00:59:47,590 --> 00:59:50,380 change that wouldn't, maybe not be too good. 666 00:59:50,380 --> 00:59:54,250 Microphone 6: Okay. Thanks. Sebastian: Okay. 667 00:59:54,250 --> 00:59:58,660 Herald: Thanks a lot for this answer. I think we are done with all the questions 668 00:59:58,660 --> 01:00:03,330 from the hall. If you like to continue the discussion with Sebastian Schinzel, you 669 01:00:03,330 --> 01:00:09,151 may move yourself to the assembly of Chaos West in the next hall in this direction. 670 01:00:09,151 --> 01:00:11,980 And now please give a big round of applause for Sebastian Schinzel and the 671 01:00:11,980 --> 01:00:14,880 awesome team behind. Applause 672 01:00:14,880 --> 01:00:18,815 Outro plays 673 01:00:18,815 --> 01:00:38,000 subtitles created by c3subtitles.de in the year 2020. Join, and help us!