... wanted to be able to use Thunderbird and GnuPG together with Tor, and so we thought: oh, it would be really easy, I bet, to configure Thunderbird to work with Tor - hah - so a new Free software project was born. It's a really simple thing, but basically it's just a package that hooks it all together. So a lot of people were using Thunderbird and TorBirdy, and GnuPG, and Tor, and Debian, together for email, combined with Riseup as an email service. So it's a literally a real peer to peer, Free software driven set of things, actually, that made it possible. [question]: So one thing I never understood about this process was exactly how the documents were handled, and maybe that's because nobody wants to say, but, you know, did you leave them on a server somewhere and download them, hand them over to people, and who took what where, and how do you... in case I need to do something really dangerous with a load of documents, what's the best way of doing it? [laughter] [Jacob]: Hmm! [audience member]: It's a good thing this isn't being streamed. I'm sorry, what? There was a voice from god, what did she say? [audience]: I said good we aren't streaming tonight. Oh yeah, so hello to all of our friends in domestic and international surveillance services. Well, so I won't answer your question, but since you asked the question, it's my turn to talk. So what I would say is that... if you want to do clandestine activities that you fear for your life for, you need to really think about the situation that you're in very carefully. And so a big part of this is operational security and a big part of that is compartmentalization. So certain people had access to certain things, but maybe they couldn't decrypt them, and certain things were moved around, and that's on a need to know basis, and those people who knew, which is not me - I don't know anything, I don't know what you're talking about. Those people knew, and then you know, it'll go with them to their grave. So if you're interested in being the next Edward Snowden, you need to do your homework in finding people that will be able to do the other part of it, let's say. But just in general, I mean compartmentalization is key, right. So it's not just for AppArmor profiles. So you need to think about what you want to do. And I mean a big part of this is to consider that the network itself is the enemy, even though it is useful for communicating. So all the metadata that exists on the network could have tipped people off, could have caused this whole thing to fall apart. It really is amazing, I feel like you know two and half, three years ago, when you talk about Free software, and you talk about the idea of Free software, and you talk about issues relating to autonomy and privacy, and security you have a really different reception now than you did then, and that's really what it took to turn the world half a degree, or something, or a quarter of a degree or something. So I'm not going to tell you about detailed plans for conspiracy, but I highly encourage you to read about South African history, in particular the history of Umkhonto we Sizwe. They are the clandestine communications group for MK, or rather the operation who lay inside of MK, which is Umkhonto we Sizwe, and they are sort of with the African National Congress, and those people have published so many books about the revolutionary activities to overthrow the apartheid state. If you read these books, especially the book "Operation Vula" and "Armed and Dangerous" by Ronnie Kasrils they give you some idea about what you need to do which is to compartmentalize, how to find people to do various tasks, specific tasks, how to work on building trust with each other, what that looks like, how to identify political targets, how you might use things like communications technology to change the political topic on, and the discussion in general. And I think the best way to learn about these things is to study previous people who have tried to do that kind of stuff. And the NSA is not the apartheid regime of South Africa, but there are still lessons to be learned there, so if you really want to know the answer to that, also Che Guevara's manual on guerilla warfare is very interesting, and there's a lot of other books like that. I'd be happy to talk about it with you later. And I have nothing to do with anything that we may or may not have done. [laughter] [question]: Do you think there is a chance that things may get better for example I know that publicly, some programs were not extended but I don't know what is happening in the background so maybe it's the same thing but they are pretending that it's not How do you see this? [Jacob]: Well I think a couple of things. In general I think what happened, not just with this movie but with all of these things is that in inspired hope, and the hope is very important, but hope is not a strategy for survival, or for building alternatives, so what it has also done, is that it has allowed us to raise the profile of the things which actually do make it better. For example ridding ourselves of the chains of proprietary software is something that's a serious discussion with people that wouldn't have previously talked about Free software because they don't care about liberty, they care about security. And even though I think those are really simliar things, previously they just thought we were just Free software hippies, in tie-dye shirts and while that may be true on the weekends and evenings or with Bdale every day [laughter] I think that actually does make it better And it also changes the dialogue, in the sense that it's no longer reasonable to pretend that mass surveillance and surveillance issues don't matter, because if you really go down the rabbit-hole of thinking about what the security services are trying to do it becomes obvious that we want to encrypt everything all the time to beat selector-based surveillance and dragnet-based surveillance. It doesn't matter if something is authenticated You could still trigger some action to take place with these kinds of surveillance machines that could for example drone strike someone, and so it raises that. And that gives me a lot of hope too, because people understand the root of the problem, or the root of many problems and the root of some violence in the world, actually. And so it helps us to reduce that violence by getting people to acknowledge that it's real and also that they care about it and that we care about each other. So that really gives me a lot of hope, and part of that is Snowden and part of that is the documents but the other part of it is that.. I don't want to blow it up and make it sound like we did something like a big deal, but in a sense, Laura, Glen, myself and a number of other people were really not sure we would ever be able to travel home to our country that we wouldn't be arrested. I actually haven't been home in over two and half years, well, two years and three months or something I went out on a small business trip that was supposed to last two weeks and then this happened and I've been hear ever since. It's a really long, crazy trip. But the point is that that's what was necessary to make some of these changes and eventually it will turn around and I will be able to go home, and Laura and Glen will be able to travel to the US again. Obviously, Julian is still stuck in the Ecuadorian embassy Sarah lives in exile in Berlin, I live in exile in Berlin, And Ed is in Moscow So we're not finished with some of these things and it's also possible that we are, the set of people I mentioned, the state we're in, will stay that way forever. But what matters is that the rest of the world can actually move on and fix some of these problems, and I have a lot of hope about that. And I see a lot of change, that's the really big part. Like I see the reproducible build stuff that Holger and Lunar are working on. People really understand the root reason for needing to do that and actually seems quite reasonable to people who would previously have expended energy against it, in support of it, so I think that's really good. And there's a lot of other hopeful things. So I would try and be as uplifting as possible. It's not just the rum! [question]: Near the end of the film we saw something about another source. I may have been missing some news or something but I don't remember anything about that being public. Do you know what happened to them? [Jacob]: As far as I know any other source that was mentioned in the film is still anonymous, and they're still free. I'm not exactly sure because I was not involved in that part but I also saw the end of the film and I've seen a bunch of other reporting which wasn't attributed to anyone in particular So the good news... there's an old slogan from the Dutch hacker community, right? "Someone you trust is one of us, and the leak is higher up in the chain of command than you" And I feel like that might be true again, hopefully. I think that guy has a question as well. [question]: Part of the problem initially was that encryption software was not so easy to use, right? And I think part of the challenge for everyone was to improve on that situation to make it better so I'm asking you if you've observed any change and to the rest of the room have we done anything to improve on that? [Jacob]: I definitely think that there is a lot of free software that makes encryption easier to use, though not always on free platforms, which really is heart-breaking. For example Moxie Marlinspike has done a really good job with Signal, Textsecure and Redphone and making end-to-end, encrypted calling, texting, sexting, and whatever apps, sext-secure is what I think it's nicknamed and I'm very impressed by that, and it works really well and it's something which in the last two years if you have a cell-phone, which I don't recommend but if you have a cell-phone, and you put in everyone's phone number, a lot of people that I would classify as non-technical people, that don't care about Free software as a hobby or as a passion or as a profession. You see their names in those systems often more than some of the Free software people, and that's really impressive to me, and I think there's been a huge shift just generally about those sorts of things also about social responsibility, or people understand they have a responsibility to other people to encrypt communications, and not to put people in harm's way by sending unsafe stuff over unsafe communication lines. So I think in my personal view it's better. But the original problem wasn't actually that the encryption was hard to use. I think the main problem is people didn't understand the reason that it needed to be done and they believed the lie that is targetted versus mass surveillance. And there's a big lie, and the lie is that there is such a thing as targeted surveillance. In the modern era, most so-called targetted surveillance actually happens through mass surveillance. They gather everything up, and then they look through the thing they've already seized. And of course there are targetted, focussed attacks. But the main thing is that the abuse of surveillance often happens on an individual basis. It also has a societal cost. I think a lot of people really understand that. It's probably because I also live in Germany now for the last two years but I feel that German society in particular is extremely aware of these abuses in the modern world and they have a historical context that allows them to talk about it with the rest of the world, where the world doesn't downplay it. So this is how other people relate to Germany not just about Germans relate to each other. And that has also been really good for just meeting regular people who really care about it, and who really want to do things. So people's parents email me, and are like "I want to protect my children, what's the best way to use crypto with them?" You know, things like that. And I didn't every receive emails like that in the past and that's to me is uplifting and very positive. [question]: A quick organisational question. Right now we're live-streaming the Q&A. Are you comfortable with that? [Jacob]: I don't think in the last three years I've ever had a moment that wasn't being recorded. [laughter, applause] [question]: If you're fine with it, moving on... [Jacob]: That's fine, just don't do it when I'm trying to sleep. [question]: I was wondering why Laura and you ended up in Germany because what you said about people in Germany might be true but I'm really ashamed about my Government and how they dealt with ???? and they are doing nothing for this. [Jacob]: The reason that we ended up in Germany is that I'd been attending Chaos Computer Club events for many years and there are bunch of people that are part of the Chaos Computer Club who are really supportive, and good people, who have a stable base, and an infrastructure. The German hacker scene has this phenomenon which is that it's a part of society. So there are people in the CCC who will talk with the constitutional court for example, and that creates a much more stable society and those people were willing to help us. They were willing to hold footage, to hold encrypted data. They were willing to help modify hardware. There was a huge base of support where people, even if they had fear, they did stuff anyway. And that support went back a long time. And so we knew that it would be safe to store footage for the film here. In Berlin, not in Heidelberg, but here in Germany. And we knew that, of course, there were people that would be helpful. In the US there's a much bigger culture of fear. People are afraid of having their houses raided by the police, where there's lots of detainments at the borders, where there's lots of speculative arrests, journalists that are jailed, so the situation was not to say that Germany was perfect. I revealed in Der Speigel with three other journalists that Merkel was spied on by the NSA. And it's clear that the Germany government was complicit with some of this surveillance. But in a sort of pyramid of surveillance there's a sort of colonialism that takes place. And that the NSA and GCHQ are at the top. And the Germans are little bit below that. The thing is that there's not a lot you do about that. And so even though we revealed this about Merkel, it's not clear what she should do. It's not clear what anyone should do. But one thing that was clear was that if they wanted to break into our houses they would do it in a way that would cost them a lot politically. It would be very public. The last time someone raided someone working with Der Speigel was in 1962 during the Speigel affair, and some ministers were kicked out. You may have seen recently the Landersverrat thing with Netzpolitik. The charges against them now have been dropped. That would never happen in the United States. We would not be safe. And I still, for my investigative journalism, and my work with Wikileaks, and my work with the Tor project, I wouldn't even go back to the US, because there's no chance that if they wanted to do something to me that I would have any constitutional liberties, I think, and the same is true of Snowden. You just won't get that fair trial. And we thought at least here we would have ground to stand and fight on. And it's exactly what happened, and we won. [question]: This is also about the fear stuff that you talk about which is in the very old days we used to put red words in the end of every message to make sure that it would be hard to find the actual subversive message among all the noise. And you can think about the same thing here. Should we build our systems so that everything gets encrypted all the time? [Jacob]: So I have a lot of radical suggestions for what to do, but I'm going to talk about them tomorrow in the keynote mostly. But to give you an example, if you install Debian, you can give someone the ability to log into the machine over a Tor hidden service for free. You get a free .onion when you add two lines to a Tor configuration file. We should make encryption not only easy to use but out of the box we should have it possible to have end-to-end reachability and connectivity, and we should reduce the total amount of metadata, to make it harder for people who want to break the law, that want to break into computers. We should solve the problem of adversarial versus non-adversarial forensics so we can verify our systems with open hardware and Free software together. And there's a lot to be done, but the main thing to do is to recognise that if you have the ability to upload to Debian, there are literally intelligence agencies that would like those keys. And we have a great responsiblity to humanity as Debian developers to do the right thing: to build open systems, to build them in a way where users don't need to understand this stuff. There are a lot of people in the world that will never see this film. And we can solve the problems that this film describes largely with Free software. And we can do that without them knowing, and they will be safe for us having done that. And if we can do that, the world will be a better place, I think. And I think the world is a better place because of the efforts that were already done in that area, that made this possible. The Tails project made it so that a bunch of people who were good at investigative journalism, but absolutely terrible with computers, were able to pull this off. And that is entirely the product, in my opinion, of Free software. And a little bit of Laura and Glen, but I'd say a lot of Free software. [question]: How many people do you think NSA has working within the Debian community? [laughter, applause] [Jacob]: Well, I looked in the Snowden archive about that actually. [laughter, applause] Yeah. And as far as I can Debian is not a high priority target for them. I mean they write exploits for all sort of stuff but I never found any systematic attempt to compromise or harm the Debian project. But obviously there are people who are paid by the NSA to infiltrate communities, and that's why we have to open transparent processes so that if those people behave badly, we have an audit trail. We won't ever stop that kind of stuff, but what matters is that people do good things. It doesn't matter who they do bad things for as long as we can correct those things and/or catch them and stop them before it happens. But as far as I know there are only a couple of people that have ever been associated with the NSA in the Debian community. But I think we shouldn't get paranoid about it, but we should just be prudent about our processes, because there are lots of intelligence services around the world that do not like the values of a universal operating system, so I don't think it's super-important to look, but I did actually look, very specifically for a whole bunch of people in the Debian community to see if any of them also were being paid by the NSA and I didn't find any serious thing that raised concern, and if I did, I would have... I mean, there were lots of things I found in the archive that I immediately notified security teams about. Where I worked along with many other people to actually fix those things. And one of those things, if we had found them, like infiltrators in Debian, I absolutely would have just told people about. The problem is that a lot of the journalists don't want to do that because there's a ten year felony where you go to prison - a federal American prison - if you reveal the name of an agent. So there's a tension there, but I think that there's something to be said, if they're actually actively harming the community and they're committing a crime, I think there's something to be said about that. So if I found that I think it would be worthwhile, but just so you know, there's this high cost. So if there were people in the agency now, because they say that we used Tails, and Debian, and they wanted to subvert it, there's a really really high bar for punishment. Which suggests that maybe people won't tell you. So we need to sort of bank on the fact that we'll never know, but we don't need to know, as long as we have good processes that would catch bad behaviour. And that's one of the strengths of Debian. There are very few operating systems, I think, and just in general Free software communities, that are as diverse, and committed to the openness and the Free software nature of this kind of a project, and so it's very important to state that. But I do think one of the things that will happen in the future at some point is that you'll start to find people in the Debian community that are pressured by other people to do bad things so we need to set up processes that will stop that, to create an incentive for that not happening. But it's really tough, so I think that openness, transparency and accountability are the ways that we can combat that, because otherwise we won't really be able to solve it. But don't be paranoid, is the other thing. They really are out to get you, so be prepared. [laughter, applause] [question]: I'm just wondering how trust was established because I'm just realizing that this community, for you to verify your public key and even fingerprint is like, you have you produce your passport, so I'm wondering how Laura managed to exchange her keys with Snowden and make sure that they were really talking to the right person. [Julian]: Well, they had a whole sort of dance for doing key exchange. I think it was a little bit luck, and a little bit transitive trust, there's a little bit of the web of trust, and it worked pretty well. I mean, I don't think that the key-signing stuff that Debian does is anything close to what they were doing. They just wanted to make sure that the keys they had were the right keys, and that they weren't compromised, and that then they would change things. There was a point in the movie where they said: "let's disassociate our meta-data one more time" And what that means is they changed all of the identifiers that are visible to the network, new keys, new email addresses, new Tor circuit, etc and this is like a key consistency thing, where they had the right key to begin with and the continued to rotate over new keys. This is also sometimes called TOFU. This is, I think, weaker than the web of trust, but a lot easier for people to do, and very easy to explain, and it worked out pretty well. It doesn't scale really well, but it has a separate good side which is the web of trust explicitly names a web of co-conspirators. And so you don't want that feature. It's useful for something like Debian; it's not useful for clandestine conspiracies to commit investigative journalism. [laughter] Lots of questions, this is great. [question]: Somebody working on Tail told me that the NSA has a file on every DD. Is that true, do you know? [Julian]: Okay, so when you balance your check-book, just to answer your question in a really strange way, when you balance your check-book, or you balance your bank account, and you think this is how much my rent is, this is how much food is, this is how much I have to spend on some new hardware, you think about money in an individual way. But if you think about is as a state, the way a state thinks about money. They don't balance budgets the same way that you do. They think about long-term investments very differently. They have other people's money. It's a whole different way of managing it. And the NSA is not the Stasi. So it's not that you have to worry about whether they have a file on you, or every Debian developer, but rather there exist some laws in the United States that say for cyber-security purposes, you don't have constitutional rights and based on your accent, you weren't an American anyway, and you aren't in America, so you don't have any rights at all, anyway, according to them. They're just allowed to do whatever they want to you, up to and including murdering you, with the CIA. That's what they do with drones; that was at the very end of the movie. So it's not that they have a file on you. It's that they have giant databases full of information on all of us, and then when they're interested in you, pull up all your data, and associative data, and then they use that, and sometimes they use it to target you, to break into your machines, or to find people to exert pressure on, or to do psychological manipulation on. All that stuff, they do all of those things. And so it's not that they have one file on you. Though maybe, it depends, if you work on a critical package like the Linux kernel they might be more interested in you than if you work on something else. I don't want to denigrate anyone's work, but they have very specific focuses, and so they definitely are interested in being able to compromise systems, right? And so you may also have file, but it's really the meta list that's the new way of thinking about it. And in some senses I think that's actually scarier, because they just hoover up everything, all across the whole Internet, and things that are interesting, then they have them. And depending on what interesting things are there, they maybe put those in a database that lasts for ever, or maybe it's just around for 30 days, or maybe its full content for 9 days, or something like that. And then of course if you are a person of interest they do do the same stuff that the Stasi does, they do that Zersetzung stuff, if you're familiar with this German term, disintegration, they do that kind of stuff, along with JTRIG, from GHCQ, so they harass people, blackmail them, do all sorts of really nasty stuff. And they do that also, so both of those things. So again, I don't think you should be paranoid, you should encrypt your stuff, and help people do the same, and know that in a democratic society with a secret political police, the right place to be is in their database, right? You should be proud of being surveilled by them, it means you're doing the right thing. [laughter, applause] Nonetheless, we should stop them. [question]: I'm curious about your views about Snowden actually coming out and saying he was the whistleblower, because I know, when he came out, I had some fierce discussion with friends about it, so I wanted to know what you thought about it. [Jacob]: What do you mean came out? [question]: He said I'm Edward Snowden, I'm the whistle-blower, here I am, instead of just being anonymous the whole way, just sending files to people. [Jacob]: Well, I think the main thing is that it's about control of your own narrative, right? I mean if we could have done everything here anonymous, and gotten away with it, would that have made the same impact in getting other people to come forward even if they maintain their anonymity? So I think that what Snowden did, what' beautiful about it, is that he basically did enough, where he could then survive. Our job now for the most part, a very good friend told me, he's a little bit of a fatalist, he said: your job, Laura's job, Glen's job, Snowden's job, your job now is just to survive. That's all that you need to do now. You don't need to do anything else. You should go do other things, like drink a glass of wine, relax, be happy, have a nice life, but just survive, so other people can see that you do the right thing, you couldn't have done more, you did enough, and you lived through it. And so Snowden coming out and telling us all of these things, I mean, there are really powerful people saying he should be assassinated, right, hung by the neck until dead, was what one of the CIA people said. So he probably could have continued to be anonymous for a while, but imagine if the NSA had got to reveal his identity. How would that have been framed, what would the first impression have been? I think they called him a narcissist, and they called him all these terrible names. And it didn't really stick, because he basically said "come at me bro', I'm ready, and you can do your worst, but you can't get rid of the facts, so let's talk about the facts." And I think the timing of how he did that is good, because he really cared about the issues, but he also recognized that it was a matter of time, the NSA police went to his house, they really bothered his family, they've done that with my family as well, other people's families have had trouble. So I think think it's tough, because I think he probably would have liked to have been able to not have that happen, but there comes a point at which you're the person who has access to all that information and they're going to figure it out. No amount of anonymity, I think, will last forever, but it can buy you time. He got exactly the amount of time he needed. The really sad part about him coming out in public when he did, though, was that he got stuck in Russia, because my government cancelled his passport. I think mostly for propaganda reasons. Because in the United States, we denigrate all things relating to Russia. And there are lots of problems with Russia, and especially with Vladimir Putin, but at the same time that seems to be the only country that was willing to uphold his fundamental liberties. I went to the Council of Europe, and to the European Parliament, to the German Parliament, to the French, sort of to the French Parliament, they didn't really want to meet with me, but also to the Austrian Parliament, and to a number of other places, and everyone said, oh, we would really live to help anybody who needs help, oh it's Edward Snowden, never mind. [laughter] And so though I have a lot of critiques on Russia, the propaganda aspect of it was very damaging for him to be stuck in Russia, but on the other hand, he's still alive, and he's still mostly free. And they recognized his right to receive asylum. So there's a lot of trade-offs to think identifying one's self, and if you were thinking about being the next Snowden, or helping the next Snowden, or helping Snowden, or something like that, you really have to think that, you really have to think this out many steps ahead, and it's easy to stay, oh he should have just stayed anonymous and nobody would have figured it out, but that's very clearly not planning the case that they do figure it out, and then they're going to be in control of the narrative, and in that case, I think you are better off to do what he did, and he did so quite reluctantly. He's not an egoist, or an narcissist, he's actually a really shy guy from what I can tell. I don't know exactly what conversation you and your friend had, but I would suspect that the notion is that people are more powerful when anonymous. And that is true sometimes, but not always, and it's important to remember that the anonymity technology is there so you have a choice, not a requirement. And that choice is sometimes counter-intuitive, but I think he did the right thing in this way, and I wish that my government had done the right thing by him as well, but they did not. [question]: So there are lot of questions, do you want to keep going on, shall we get in a little Mate? [Jacob]: I would love some of that rum. I think I have to GRsec, right? GRsec kernel. And then rum appears. Rum as a service. [applause] I'm really happy to keep taking questions, because to me, what I want is for every person in this room to feel a part of this, because you really are. A lot of the people I've met in this community really inspire me to action, and it's important to understand that really, it would not have been possible without Debian. For example debootstrap - really important tool, right? With weasel's packaging of Tor, it allowed us to have bootstraps of things, it allowed us to build things, and using Free software really was helpful, so if you guys have any questions at all, really each and every person that helps with Debian should just know that you are a part of that, and I'm just happy to talk for as long as you want, basically, to answer all of your questions, except the ones that put me in prison. Thanks. [laughter] [question]: I just wanted to make a quick note about the question "do they have a file on me?" From all I've read so far, it's just that they're doing the thing that is in the commercial world called "big data". [Jacob]: Yep. Absolutely. Oh boy. GRsec again? [orga]: it's not rum, but it's Bavarian whisky. [Jacob]: Oh boy. It's going to be a heavy morning tomorrow. I saw another couple of hands. [question]: I was just wondering if that you noticed throughout this that you think we could improve in Debian to make the next people's lives easier. [Jacob]: Oh my god, I'm so glad you asked that question, that's so fantastic. I'm going to talk about that tomorrow in my keynote, but let me tell you about one that I have. I revealed a specific document about a wifi injection attack system. It's a classified document, it's a top secret document, for a thing called nightstand, and what nightstand is, it's basically like car metasploit, it's a wifi injector... cheers! Danke schön. It's a wifi injector device... Whew, jesus! [laughter, applause] [orga]: Tonight's whisky sponsored by drunc-tank dot org. [Jacob]: So this wifi injector device, what it does is it basically is able to exploit the kernel of a device by sending malformed data over wifi. Now I have a series of photographs, so all of us.. not all of us, but most of us used these speciallly modified X60s where we removed the microphones, soldered?? down things on the PCI bus, we removed, like, firewire, really modified it, flashed coreboot onto it, flipped the read pin so it was only read-only, so you couldn't easily make a BIOS root kit and make it persistent, we booted TAILS, did all this stuff, often we could boot to RAM so that once the machine was powered off basically it would be done, so if someone kicks down your door, you just pull the power out, and you don't have a battery, and when the power fails you have an instant kill switch. So things that are in TAILS that are really useful include this wiping the kernel memory package which I hear is being packaged for Debian soon, which is very exciting. Because everyone should have access to that so we can tie it into something like GNU panicd or these other things. But one thing I kept having problems with is this wifi injection device, I'm pretty sure, was very close to my house. There was a white van outside, it was vibrating a bit like there was a guy walking around in it, and then all of sudden, an X60 here, an X60 here, and an X60 here, just booted into TAILS, not doing anything at all, but on the wifi network, kernel panic, kernel panic, kernel panic. All the same kernel panic, all the same memory offsets, in the Appletalk driver of the stock kernel for TAILS. I think I filed a bug upstream with TAILS at the time, but this is just incredible because it's clear that all the crap in the default Debian kernel that you really want for your 1992 Apple network makes operational security really hard, and one thing that would be really great would be a GRsec enabled kernel... [applause] Yes, have to drink. But as an example, we built different custom machines, and one of the things that we did for some people and in some circumstances was to build GRsec enabled kernels. And I'm not going to drink again. So we built those kernels [audience]: Which ones? [Jacbob]: Yes, exactly, those ones. And that was work which creates a problem for a bunch of reasons. When you build custom kernels, and you only have a few people that can build those kernels, you actually build a chain of evidence of who helped who. And if that was stable, normal package, that people could install in a Debian pure blend, then it would have been easier to do that. We built a lot more sandbox profiles for various different things, we built some transparent TOR-ification stuff, and that required a lot of bespoke knowledge, and it required a lot of effort that a lot of people did not have, because they had a different set of skills, and it's good to have a division of labour, but having that kind of stuff built into Debian by default, making a Debian installer that could do that, and also verification, would be great, right? So I wrote some custom scripts where I could look at a TAILS disk, or a Debian install, and know if it had been tampered with. And it would be nice if there was just a disk you could boot that did verification of an installed system very very easily, so easily that Glen Greenwald could use it. I love Glen, I saw that very politely, but what I means is it needs to be easier than that, because Glen at least knows that he he a reason to need it. And so that was something that we really needed help with. And we spent a lot of time on that. And there are lots of other little things like that, and I'll talk about some of those things tomorrow, but one of the really big problems is hardware, which is that you cannot buy a modern Intel CPU which doesn't come with a backdoor any more. And that is a huge problem, and I'm not sure that the answer is to use ARM. It seems like the answer is to use ARM. But that's only if assume that ARM didn't just add a backdoor that's obvious. So we really need to think about how to, in moving forward, how to have easy to use, easy to buy on the shelf, Debian hardware, available everywhere, all the time, so you can just go and buy this thing and verify it in some way with some other machine, to know that you would have the right thing. And to that extent we didn't have X-rays for a lot of the circuit boards, so that made it very difficult to know if when you buy something, it's been tampered with. I'll talk about some of that stuff tomorrow, but basically, Debian does a lot of stuff right, and that is also worth mentioning. There's so many things that just work out of the box, that just work perfectly. So the main thing is to keep the quality assurance at the level, or to exceed where it is right now. Because it actually works super super well. The exception being for very specific targetted attacks, the kernel attack surface is pretty big, and pretty bad, I think. And also, we rebuilt some binaries in order to.. sorry, I'll get to you in a second. We rebuilt some binaries to make sure that we had address space randomisation and linker hardening, and stack canary stuff, and for some stuff lately we've been using address space sanitizer, so it would be really great if all the hardening stuff was turned in, if there was PAX plus GRsec as a kernel. [audience]: so the specific problem with GR security is that they don't really want to work with distros. So we could have a Linux kernel package with GR security applied, but it wouldn't have any of the other Debian patches. [Jacob]: So I talked with Brad Spender about this, and I'm so glad that you said that, because what he said was that, as far as I can tell, he's totally interested in helping Debian with this but thinks that Debian is not interested. He actually runs a kernel building service where they actually do individual kernel builds, and I think you'd be interested, and when I told him we'd love to have this in TAILS, he said what patches do I need to include in GRsec to make sure that it'll work? And he offered to do the integration into the GRsec patch if there are not too many things. So I think what we should try and do is build a line of communication, and if it costs money we should find a way to raise that money, I'll put in some of my own personal money for this, and I know other people would too. [distant audience]: I will. [Jacob]: Great. So securedrop, for example, part of what they do for their leaking platform, if you go to the intercepts website, you wan to leak them a document, they actually use free software everywhere, but there are a few things they build specially, and one of those things is a GRsec kernel. So the people at first look, that helped make this movie, and that work on securedrop, they would probably also, I'm not committing them, I don't know that they would actually do this, but I think they would really like it if that was in there, and I think it we could find the community will to do that, I know I would volunteer and other people would, I know that dkg in the back would love to help with this, I would that ??? who is just totally behind funding this work, right? I thought that you were there to protect my civil liberties, buddy. But I really think that it's possible that we could do this, and I definitely think Brad, the author of GRsec, I think he would really love it if Debian shipped GRsec. And it doesn't need to come by default, but if it was possible to just have it all, that would be great. Maybe we could have an affinity group where everyone who is interested can meet sometime tomorrow and we could talk about doing this. I would love to have that conversation. Who are you? [audience]: Ben Hutchings. [Jacob]: Oh, nice to meet you! [laughter, applause] That's awkward. [question]: Hi. Sorry to interrupt the awkwardness, and replace it with more awkwardness. Nice to see you, Jake. So, I remember reading the documents in 2013 and seeing the NSA's internal training guide for how to query their Hadoop data store, aka xkeyscore, and so I thought I would just ask you if you think Free software net helps us or helps them. [Jacob]: I'm really glad you asked that question. I think that Free software helps everyone on the planet, and I think that purpose-based limitations.. I understand why people want them. I think we should try to build a world where we are free, and so putting in purpose-based limitations is really problematic, and I think what we should do is try to mitigate the harm that they can do with those systems, as opposed to pretending that they care about Free software licensing. These guys kill people with flying robots, it's illegal to murder people, and they do it. Limiting their use with licenses, first of all, that just means they'll spend your tax money to rewrite it if they care about the license, and you won't get their bug-fixes or their improvements, and then additionally they're still not going to obey your license anyway, because literally some of these people work on assassinating people. So it is better that we keep our integrity and take the high road, and write Free software, and we give it to every single person on the planet without exception, It's just better. It's better for all of us, right? So the fact that they have Hadoop, the fact that they, for example, use OpenSSL, or maybe they use Tor, or whatever, right? Or they use gdb to debug their exploits. I kind of wish that on them. [laughter, applause] I think it's great, right? So one of the things Che Guevara said in his manual about guerilla warfare, in chapter two, is that (oh, it was chapter three) He talks about when you have to arm a guerrilla army, this is not exactly related, but it's an analog. He says that the most important thing is for the guerrilla army to use the weapons of the people that they're fighting - the oppressor. And the reason is that it allows you to resupply, essentially. When you win a battle, you resupply. When we all use the same Free software, and we're working on these things, the fact that they have to contribute to the same projects and they often do means there's a net win for us. They do have some private things that they don't share, obviously, with the exception of nice people like Edward Snowden, and I think that it is a net positive thing, and if we think of it as a struggle, we are better off to take the high road, and so I really think we should not pretend that we can stop them, and instead we should work together to build solutions. And I think that Debian is doing that, right? I think Debian is much harder to compromise than a lot of other operating systems, and it's much much harder to coerce people, and there's a strong ethos that comes with it that it's not just the technical project, there's a social aspect to it. I think I was in the New Maintainer queue for 11 years, maybe that's a little too long, but there's a huge hazing process, so anyone who wants to help, really really wants to help, and if they want to do something wrong there are processes to catch people doing things wrong. So we should really stay true to the Free software ethos, and it really is a net benefit. [question]: Hi Jake. Thanks a lot for saying so much "GRsec". Just wanted to give a shout out. You mentioned possible backdoors in CPUs and so on, that ARM might not be the next best thing because it's not so open either. You might want to have a look at Power 8. It's basically PowerPC 64, so Debian has support for it as far as I know, and most of the stuff is actually open. Not that actually designs that IBM is using, but you can have, actually, an FPGA implementation of it, and if you have the money make your own ASICs for it, without even knowing how to do it, which is pretty good, I think. [Jacob]: I think there are lots of things we can hack right? I mean I had one of those weird RMS laptops, the Limote, or whatever it's called, for a while. And I was definitely able to get some Free software running on it, in theory it was a Free software laptop. But getting other people to use this is the problem, you need to get everybody to use it, right? There's a sort of old anarchist cliché, "None of us are free until all of us are free" And that really applies here. We really need to have Free software that's usable by everyone, otherwise we're sort of bound by the lowest common denominator of Free, or proprietary tools, depending on what people have to use. So it'll be great when we have that, and there's a thing called the Nokimist??? which is a video mixing board that has an FPGA implementing a Free software CPU that you can boot Debian on, or OpenWRT, and it does work, and I have used it, and in fact I used to use it as a shell, and for a long time I used a Debian trick, actually I've never talked about that in public, let me think about that for a second. So I used to use an IRC client that was really buggy, and I couldn't figure out where all the bugs were, but I knew that if I hung out in certain networks that someone else would help me find those bugs by trying to exploit my client. And I wanted to make it as hard as possible. So I ran my IRC client inside of a Debian machine that was running an S390 emulator. Who here uses Hercules? Thank you to whoever packaged it. And so I would use Hercules, it was a very long install process. Very slow. And I would do this, and what I'd always dreamed of doing at some point was using the Nokimist??? and the Hercules together for maximum ridiculously difficult to exploit, plus GRsec kernel. But that's not a usable thing. So what we need to do is take these kinds of prototypes which actually do represent many steps forward, and we need to make sure that they're produced on a scale where you can go into a store and puchase them anonymously, with cash, in a way that you can then verify. And we're actually really close to that with software defined radios and open hardware, but we're not quite there yet. [question]: What I meant is that Power 8 is basically getting big, currently, on the server market, and it might get big for other stuff also. [Jacob]: Hopefully. [question]: I want to come back to the story about the panic in the Appletalk driver. The common approach against this is to compile your own kernel with all this stuff not compiled in, but on two of my systems I have a modprobe wrapper which has a whitelist of module which may be loaded, and I install that wrapper as the thing that the kernel uses for loading modules. Do you know if such a thing exists elsewhere, or if not, I would be interested in developing it into something which is actually useable for people. [Jacob]: That would be great. In this case we were using Tails. And so, Tails is very finicky about what it will accept, and so having that in Debian will make it a lot easier to get it into something like Tails, I think. But the main thing is really that we have to think about the attack surface of the kernel very differently. The problem is not Appletalk; the problem is the Linux kernel is filled with a lot of code, and you can autoload, in certain cases, certain things come in, and certain things get autoloaded, and I know Bdale loves his ham radio stuff, but I never use ham radio on my machine I used for clandestine conspiracies, you know? That's a separate machine. It's over here. So we just need to find a way to think about that. And part of that could be kernel stuff, but also part of it could be thinking about solutions like that, where we don't need to change the kernel. So if you could package that and develop that, it would be really fantastic. [Ben]: Actually, some time ago, after I think it was the econet exploits, no-one uses econet, it was broken anyway, but you could exploit it, because it was autoloaded. So I actually went through and turned off autoloading on a few of the more obscure network protocols. We could probably go further with that, even in the defaults. [Jacob]: I think it would be great to change some of the kernel stuff so that at least, I mean, Tails is a special use case, where, I think, it's very important, and it doesn't work for everyone, but we should just consider that there are certainly things which are really great, and I want to use Debian for it, because Debian is a universal operating system. But for a modern desktop system where you're using GNOME, and you haven't set anything up, Appletalk for example, maybe we would ask those people to load that module themselves. [Ben]: Yeah, for example you could have, a lot of those things are going to have supporting utilities, so you could put something in the supporting utilities that loads it at boot time. And if you don't have those installed, you don't need it. [Jacob]: Yep, totally. And I think there's lots of ways to do it where the network can't trigger it, and that's important. [Ben]: Yeah, that puzzled me, I can't understand, the protocol module when userland tries to open a socket of that type, it shouldn't happen in response to network traffic. There are things like, I think if you run ifconfig that can autoload a bunch of things, for example. [Jacob]: Yeah, I think on either side it should be more explicit, and in this case with Tails, there was a time when you looked at the kernel module list and it was pretty amazing, like I think there was an X25 thing, an Appletalk, thing, wait, this is all about going over Tor, we don't support any of these things at all. So it's just the way that things are interdependent, right? It's not a dig at the kernel itself. I think the Linux kernel as it works in Debian today works really well for a lot of people, but there is definitely a high security use case, and I, for example, if I were a Debian developer, and I had a development machine where I didn't run a web browser, and I took a lot of effort. It would be really nice if there were a kernel that put in the same threshold of security. And I think that the GRsec kernel with some stuff changed about it, like getting rid of Appletalk and a few other things, would be closer to that, and combined with that guy's tool that he's talking about, you could make autoloadable module, that at least even if the system was going to autoload it, you could stop it, in a failing closed sort of way. And I think there's a lot of stuff, practically, to do on that front, and there's another project called Subgraph OS, which is basically working on becoming in some ways a Debian derivative, and they're going to do stuff like GRsec kernel, and they have a whole sandboxing framework which uses apparmor, seccomp and xpra, and a few other things, and I think that they'll make a lot of interesting security decisions, which might make sense to adopt in Debian later. [Ben]: I think Matthew Garrett has an interesting criticism about that and how it wouldn't really work, and Wayland was a better way to go than xpra. [Jacob]: Yeah, I've heard those criticisms, but Matthew Garrett is wrong. Not usually, but in this particular case. For example, the sandboxing stuff, if you have a GNOME appstore, essentially, that's for one set of users, but for a Debian developer writing your own policies, it might be useful, and if you need Wayland, you might not have a full solution, we might want to have both for a while. And think it'd be great. And the main thing is we just need to find people who will think about those issues and try to integrate them, because most people who write exploits, or who understand how to do offensive security stuff, they don't want to help Free software projects, they just want to exploit them. And so some of the Subgraph guys, what I really like about them is that they're trying to improve the Free software products we all use. Even though they may make different design decisions, they're making Free software all the same. 52:17