WEBVTT 99:59:59.999 --> 99:59:59.999 Thank you very much. 99:59:59.999 --> 99:59:59.999 Thanks everybody for coming,… 99:59:59.999 --> 99:59:59.999 If you are packaging software and you want me to work on with you, 99:59:59.999 --> 99:59:59.999 this is how you can do that. 99:59:59.999 --> 99:59:59.999 It is a very self-??? talk: 99:59:59.999 --> 99:59:59.999 I just want to explain some of the things that I like, 99:59:59.999 --> 99:59:59.999 some practice that I prefer about Debian packaging, 99:59:59.999 --> 99:59:59.999 and I don't pretend this is any sort of official, 99:59:59.999 --> 99:59:59.999 permanent or final thing. 99:59:59.999 --> 99:59:59.999 I just wanted to share some ideas that I have about the way that I work with 99:59:59.999 --> 99:59:59.999 packages, in the hope that maybe, hmm, for two hopes: 99:59:59.999 --> 99:59:59.999 One is that I hope that I can show you something that you have not heard of, 99:59:59.999 --> 99:59:59.999 or maybe you were doing differently, 99:59:59.999 --> 99:59:59.999 or maybe you think it is the right think to do and it is just nice to see somebody 99:59:59.999 --> 99:59:59.999 somebody else doing it. 99:59:59.999 --> 99:59:59.999 My second hope is that you can tell me what I am doing wrong, 99:59:59.999 --> 99:59:59.999 and you can help me learn and improve on my own packaging techniques. 99:59:59.999 --> 99:59:59.999 If you see something that I am proposing up here, 99:59:59.999 --> 99:59:59.999 and you think there is a problem with it, I would like to hear about it too. 99:59:59.999 --> 99:59:59.999 I just want to see more of the culture within Debian, 99:59:59.999 --> 99:59:59.999 of people who are doing packaging, explaining what they are doing, 99:59:59.999 --> 99:59:59.999 and so I thought I would just step up and explain: 99:59:59.999 --> 99:59:59.999 "Here is some of the practice that I do", 99:59:59.999 --> 99:59:59.999 In the hope that other people will do the same and explain what they are doing, 99:59:59.999 --> 99:59:59.999 and maybe they can learn from me and I can learn from them. 99:59:59.999 --> 99:59:59.999 Without much further ????, I am just going to dive into it. 99:59:59.999 --> 99:59:59.999 If you have questions, I am perfectly happy to be interrupted, 99:59:59.999 --> 99:59:59.999 we have some folks with walking mics in the crowd: 99:59:59.999 --> 99:59:59.999 you can just raise your hand. 99:59:59.999 --> 99:59:59.999 I you have got a question or an interruption or whatever, 99:59:59.999 --> 99:59:59.999 that is fine. 99:59:59.999 --> 99:59:59.999 I ??? I got the whole 15 minutes, I think there are 20 minutes, 99:59:59.999 --> 99:59:59.999 I ??? the whole time, so there will be also time for questions at the end 99:59:59.999 --> 99:59:59.999 if you prefer. 99:59:59.999 --> 99:59:59.999 But I do not mind being interrupted. 99:59:59.999 --> 99:59:59.999 So, this is all on this web page here, 99:59:59.999 --> 99:59:59.999 you could probably skip this talk and go read the web page, 99:59:59.999 --> 99:59:59.999 but then you would not have the nice ??? actions, 99:59:59.999 --> 99:59:59.999 and it is easier to tell me that I am wrong in person, 99:59:59.999 --> 99:59:59.999 so I would like to have that happen. 99:59:59.999 --> 99:59:59.999 I put this up on the Debian wiki, 99:59:59.999 --> 99:59:59.999 because I want anyone to be able to find it. 99:59:59.999 --> 99:59:59.999 If you thing you have got some good ideas, you should put it on the Debian Wiki too: 99:59:59.999 --> 99:59:59.999 other people can take advantage of the ideas that you have got. 99:59:59.999 --> 99:59:59.999 First baseline is: I really like revision control. 99:59:59.999 --> 99:59:59.999 And I know that it makes me a certain flavor on nerd, 99:59:59.999 --> 99:59:59.999 but when we are working with things that are as complicated as software packages, 99:59:59.999 --> 99:59:59.999 hmmm, I think a lot of people don't get that in Debian we are not just working on 99:59:59.999 --> 99:59:59.999 one software package: 99:59:59.999 --> 99:59:59.999 you are actually probably, if you are doing a responsibly work, 99:59:59.999 --> 99:59:59.999 on at least two software packages, and maybe 5. 99:59:59.999 --> 99:59:59.999 So you have got the version that is unstable and you have got 99:59:59.999 --> 99:59:59.999 the version that you try to maintain for stable as well. 99:59:59.999 --> 99:59:59.999 And we are committing to doing maintenance work. 99:59:59.999 --> 99:59:59.999 A lot of our work in the project is ??? in nature: 99:59:59.999 --> 99:59:59.999 we want to clean up the mess and we want us to stay out of the way and 99:59:59.999 --> 99:59:59.999 to make sure things work, functionally, 99:59:59.999 --> 99:59:59.999 for people who are relying on the operating system to not get in their way. 99:59:59.999 --> 99:59:59.999 So revision control I think is really helpful because it means you can 99:59:59.999 --> 99:59:59.999 keep track of what changes you have done on different branches of the project 99:59:59.999 --> 99:59:59.999 while you are maintaining both of them. 99:59:59.999 --> 99:59:59.999 Basically, ??? require working with the revision system I am comfortable with, 99:59:59.999 --> 99:59:59.999 I prefer Git, I am not going to have a religious word about it. 99:59:59.999 --> 99:59:59.999 If upstream uses Git, I am even happier, and I try to make my packaging depend on 99:59:59.999 --> 99:59:59.999 upstream's revision control. 99:59:59.999 --> 99:59:59.999 I like to use 'git-buildpackage', and I like to use it with debhelper. 99:59:59.999 --> 99:59:59.999 If you have not tried out 'git-buildpackage', 99:59:59.999 --> 99:59:59.999 we are going to have a 'git-buildpackage' skill share session 99:59:59.999 --> 99:59:59.999 later on today actually, and I welcome you to come and share your tricks with it, 99:59:59.999 --> 99:59:59.999 or learn some tricks from other people. 99:59:59.999 --> 99:59:59.999 It is a particular way that you can keep your Debian packaging in a Git repository, 99:59:59.999 --> 99:59:59.999 and it helps you to keep track of all of the changes that ave happened within 99:59:59.999 --> 99:59:59.999 your packaging and within upstream to make sure you are not accidentally 99:59:59.999 --> 99:59:59.999 making other changes. 99:59:59.999 --> 99:59:59.999 So it is very easy to go back and review what you have done. 99:59:59.999 --> 99:59:59.999 I find that really useful. 99:59:59.999 --> 99:59:59.999 I definitely also like to keep upstream's source code in the same revision control 99:59:59.999 --> 99:59:59.999 system. 99:59:59.999 --> 99:59:59.999 I like to keep the tarballs in the revision control system because it means 99:59:59.999 --> 99:59:59.999 that if someone is interested, they can uses a tool called 'debcheckout'. 99:59:59.999 --> 99:59:59.999 You can use 'debcheckout' with a name of a package: 99:59:59.999 --> 99:59:59.999 you say just "I am really interested in package 'foo', 99:59:59.999 --> 99:59:59.999 let me see the source code for that": 99:59:59.999 --> 99:59:59.999 debcheckout foo 99:59:59.999 --> 99:59:59.999 You get the source code, and you get the source code from a revision control 99:59:59.999 --> 99:59:59.999 system that you can now track and you can just propose changes on. 99:59:59.999 --> 99:59:59.999 You can also extract the tarball from that revision control system. 99:59:59.999 --> 99:59:59.999 'debcheckout' actually works even if you do not have upstream stuff in there, 99:59:59.999 --> 99:59:59.999 but I like to keep it all in one revision control system, 99:59:59.999 --> 99:59:59.999 it is just easier to find everything when you want. 99:59:59.999 --> 99:59:59.999 Some of these things that I prefer have to do with what the upstream software 99:59:59.999 --> 99:59:59.999 developer has done, so I am less inclined to try the package an upstream software 99:59:59.999 --> 99:59:59.999 project if they just throw tarballs here over the wall to an FTP side 99:59:59.999 --> 99:59:59.999 every now and then. 99:59:59.999 --> 99:59:59.999 It makes it more difficult for me to know what they are doing, 99:59:59.999 --> 99:59:59.999 and why they are doing it. 99:59:59.999 --> 99:59:59.999 So i like it, I have already said, when upstream uses Git, 99:59:59.999 --> 99:59:59.999 I also like when upstream signs their releases, 99:59:59.999 --> 99:59:59.999 and say "hey, this is specific release", 99:59:59.999 --> 99:59:59.999 Because that is a signal that I can use, or somebody else that understands the 99:59:59.999 --> 99:59:59.999 project: as said "we think that this something that other people can use", 99:59:59.999 --> 99:59:59.999 or "this is a particular version we would like other people to test". 99:59:59.999 --> 99:59:59.999 There are a lot of other situations where maybe it is not so important. 99:59:59.999 --> 99:59:59.999 And having that be cryptographically signed is really useful. 99:59:59.999 --> 99:59:59.999 I care about cryptographic signature on software because I want to know that 99:59:59.999 --> 99:59:59.999 what I am running is related to the code that somebody else out should be run. 99:59:59.999 --> 99:59:59.999 And if you don't verify your software cryptographically, anyone could 99:59:59.999 --> 99:59:59.999 intercept the network connection between you and that software, 99:59:59.999 --> 99:59:59.999 and modify the software before it gets to you. 99:59:59.999 --> 99:59:59.999 And the cryptographic signature just says: 99:59:59.999 --> 99:59:59.999 "look, this is a version that I am OK with. I am putting it out there and 99:59:59.999 --> 99:59:59.999 it comes from me". 99:59:59.999 --> 99:59:59.999 And so I can have a trace back to that point. 99:59:59.999 --> 99:59:59.999 ??? just talk about briefly about how you do cryptographic verification of upstream. 99:59:59.999 --> 99:59:59.999 You might know upstream: you might know them personally, you know their key 99:59:59.999 --> 99:59:59.999 already, that is fine. 99:59:59.999 --> 99:59:59.999 That is not the usual case: we work on the Internet. 99:59:59.999 --> 99:59:59.999 In the situation where your upstream is signing their tarballs 99:59:59.999 --> 99:59:59.999 and you have not met them, you do not have to sign their key, 99:59:59.999 --> 99:59:59.999 you do not have to say "I announce this is their key". 99:59:59.999 --> 99:59:59.999 This is probably the same one that is signing every release, 99:59:59.999 --> 99:59:59.999 so you should keep track of that. 99:59:59.999 --> 99:59:59.999 Debian has a nice way to keep track of that: 99:59:59.999 --> 99:59:59.999 you can tell Debian how to find the new version of the upstream tarball. 99:59:59.999 --> 99:59:59.999 This is in the Debian 'watch' file. 99:59:59.999 --> 99:59:59.999 If you type 'man uscan', you can learn more about Debian 'watch', 99:59:59.999 --> 99:59:59.999 and Debian 'watch' has now a feature that lets you say 99:59:59.999 --> 99:59:59.999 "that is not only this way you find the tarball, 99:59:59.999 --> 99:59:59.999 but upstream publishes signatures and the signatures look like this". 99:59:59.999 --> 99:59:59.999 You know, they got a '.sig' at the end. 99:59:59.999 --> 99:59:59.999 So there is a particular arcane way to specify that, but if you specify that, 99:59:59.999 --> 99:59:59.999 then 'uscan' can find not only the upstream tarball but can find the 99:59:59.999 --> 99:59:59.999 upstream signature and, if you drop upstream's signing key - 99:59:59.999 --> 99:59:59.999 which of course I did not put on the wiki page, someone should add it that and 99:59:59.999 --> 99:59:59.999 fix it - you can put the upstream signing key in 'debian/upstream/signing-key.asc'. 99:59:59.999 --> 99:59:59.999 And then if you do that, when you say 'uscan', you can tell… 99:59:59.999 --> 99:59:59.999 Maybe some people here do notk now how to use 'uscan'. 99:59:59.999 --> 99:59:59.999 'uscan' is a very simple tool, you run it from a software package that 99:59:59.999 --> 99:59:59.999 has a 'debian' directory, or even one level up if you keep all of your software 99:59:59.999 --> 99:59:59.999 packages in one folder. You can go one level up and say 'uscan', and it will look 99:59:59.999 --> 99:59:59.999 in all of the folder that are children of it, and look for new version by 99:59:59.999 --> 99:59:59.999 trying to find for new upstreams versions in 'debian/watch'. 99:59:59.999 --> 99:59:59.999 And if you have configured 'debian/watch' properly, it can find the new upstream 99:59:59.999 --> 99:59:59.999 signatures, and if you have got the 'upstream/signing-key.asc', then 99:59:59.999 --> 99:59:59.999 it will actually verify the signature for you as part of fetching the new 99:59:59.999 --> 99:59:59.999 upstream tarball. 99:59:59.999 --> 99:59:59.999 So you can get all of those things just by setting ???? that way. 99:59:59.999 --> 99:59:59.999 There is a hand up down there, could we get the mic down to the hand ? 99:59:59.999 --> 99:59:59.999 Or to the person who has that hand, it is not just a hand. [public laugh] 99:59:59.999 --> 99:59:59.999 [someone] Publish a tarball and a hash, '.sha1', and sign that hash, '.sha1.asc'. 99:59:59.999 --> 99:59:59.999 Can 'uscan' cope with this and check the signature on the hash and that the hash 99:59:59.999 --> 99:59:59.999 belongs to that tarball ? 99:59:59.999 --> 99:59:59.999 [Daniel] I do not believe that 'uscan' can do that currently. So anybody out there 99:59:59.999 --> 99:59:59.999 who wants to make things better for the world should go hack on 'uscan': 99:59:59.999 --> 99:59:59.999 that is a pretty straightforward thing that we should fix because I agree 99:59:59.999 --> 99:59:59.999 that is common pattern. 99:59:59.999 --> 99:59:59.999 [someone] I have no answer to this question by I have another question: 99:59:59.999 --> 99:59:59.999 how do you convince upstreams who do not release tarballs or who do 99:59:59.999 --> 99:59:59.999 not set tags in Git ? 99:59:59.999 --> 99:59:59.999 [Daniel] Who do not make tags in Git ? 99:59:59.999 --> 99:59:59.999 [someone] Yes, if there is no tags you can not check out a tarball. 99:59:59.999 --> 99:59:59.999 Is there any good way to convince upstream to do this ? 99:59:59.999 --> 99:59:59.999 [Daniel] Git has this nice feature, which is that you can create a tag, 99:59:59.999 --> 99:59:59.999 which is associate with a particular revision, 99:59:59.999 --> 99:59:59.999 and you would like to have a tag everywhere that a tarball has been 99:59:59.999 --> 99:59:59.999 released from. I am tempted to pull up a Git view and show people some tags. 99:59:59.999 --> 99:59:59.999 The question that you ask is a social one, not just a technical one, 99:59:59.999 --> 99:59:59.999 and I actually find that my upstreams are pretty responsive. 99:59:59.999 --> 99:59:59.999 Usually I frame my request as "hey, it looks like you made this tarball from 99:59:59.999 --> 99:59:59.999 this particular commit 'id'. If you could tag you releases, it would be really 99:59:59.999 --> 99:59:59.999 helpful to me, and here is the command that I would use to tag the release". 99:59:59.999 --> 99:59:59.999 And I say "git tag…" and of course I can never remember so first I look it up, 99:59:59.999 --> 99:59:59.999 but it is either 'tag name' 'commit id' or 'commit id' 'tag name'. 99:59:59.999 --> 99:59:59.999 But I would look it up and I would write the e-mail so that all they have to do is 99:59:59.999 --> 99:59:59.999 they read it, understand my argument, and execute one command. 99:59:59.999 --> 99:59:59.999 And then it starts them ?????? 99:59:59.999 --> 99:59:59.999 And if you say 'tag -s' then your tag will be cryptographically signed, which 99:59:59.999 --> 99:59:59.999 I think is a really good thing to do too. 99:59:59.999 --> 99:59:59.999 So, cryptographic verification of upstream. 99:59:59.999 --> 99:59:59.999 As I said, I want to keep upstream's code in the revision control system. 99:59:59.999 --> 99:59:59.999 I also like to keep… 99:59:59.999 --> 99:59:59.999 In my ideal case upstream is using Git: I am using Git for packaging. 99:59:59.999 --> 99:59:59.999 I actually like to keep upsteam's Git history fully in my repository, 99:59:59.999 --> 99:59:59.999 so that I do not just have the tarballs, but I actually have all of their commits. 99:59:59.999 --> 99:59:59.999 And that turns out to be really useful for two specific cases: 99:59:59.999 --> 99:59:59.999 In one case, there is a common scenario where upstream will fix a bug, 99:59:59.999 --> 99:59:59.999 but they have not made a release yet. 99:59:59.999 --> 99:59:59.999 And that bug is really, really obviously problematic for the folks who are 99:59:59.999 --> 99:59:59.999 using Debian, so want to fix it. 99:59:59.999 --> 99:59:59.999 All I can do, because I have their full revision history, I can use Git to "cherry 99:59:59.999 --> 99:59:59.999 pick" the upstream commit. 99:59:59.999 --> 99:59:59.999 And then I "cherry pick" that upstream commit and I can have it applied 99:59:59.999 --> 99:59:59.999 separately and release an Debian version that has the fix, even before upstream 99:59:59.999 --> 99:59:59.999 has made a release with the fix. 99:59:59.999 --> 99:59:59.999 So one nice thing about having upstream revision is that I can pull fixes from 99:59:59.999 --> 99:59:59.999 upstream before they decided to release it. 99:59:59.999 --> 99:59:59.999 The other advantage is the other way around. 99:59:59.999 --> 99:59:59.999 Often when I am doing packaging, I discover a problem, 99:59:59.999 --> 99:59:59.999 and maybe I can fix the problem. 99:59:59.999 --> 99:59:59.999 And if that maybe I am already shipping a Debian package that fixes the problem. 99:59:59.999 --> 99:59:59.999 If my Debian fixes can be directly applied to upstream, then I can use whatever 99:59:59.999 --> 99:59:59.999 their preferred upstream patch submission guidelines are, 99:59:59.999 --> 99:59:59.999 whether it is a Github pull request, or a patch to a mailing list, 99:59:59.999 --> 99:59:59.999 or a "hey can you pull this from my Git repository over here", e-mail… 99:59:59.999 --> 99:59:59.999 The fact that I am using the same Git history that they are using makes it 99:59:59.999 --> 99:59:59.999 much easier for me to push my changes back to them. 99:59:59.999 --> 99:59:59.999 So, it sort of smooth the interaction if you can consolidate and use the same 99:59:59.999 --> 99:59:59.999 revision control system as their. 99:59:59.999 --> 99:59:59.999 Towards that aim, I use a system now called 'patch q', 99:59:59.999 --> 99:59:59.999 which is part of 'git buildpackage'. 99:59:59.999 --> 99:59:59.999 So 'git buildpackage' is 'gbp', 'patch q' is 'pq', 99:59:59.999 --> 99:59:59.999 so to deal with 'patch q' you say 'gbp pq' 99:59:59.999 --> 99:59:59.999 and then you have some commands. 99:59:59.999 --> 99:59:59.999 And what that does, is it takes… 99:59:59.999 --> 99:59:59.999 How many of you are Debian packagers ? 99:59:59.999 --> 99:59:59.999 How many of you package software for Debian ? 99:59:59.999 --> 99:59:59.999 A very large percentage, but not everyone. 99:59:59.999 --> 99:59:59.999 I hope some folks are considering starting packaging if you have not done it yet. 99:59:59.999 --> 99:59:59.999 Of those of you who package software, how many of you package software 99:59:59.999 --> 99:59:59.999 with modifications, how many of you ship a modified version of upstream sources ? 99:59:59.999 --> 99:59:59.999 Beyond the 'debian' directory, just Debian patches ? 99:59:59.999 --> 99:59:59.999 So the common way to do that, for the Debian 3.0 ??? packaging skill, is that in 99:59:59.999 --> 99:59:59.999 your 'debian' directory you have a 'patches' sub-directory that has a set of 99:59:59.999 --> 99:59:59.999 individual patches that apply certain changes, and they are applied in order 99:59:59.999 --> 99:59:59.999 based on the file called 'debian/patches/series'. 99:59:59.999 --> 99:59:59.999 So maintaining that is kind of a drag when upstream makes big changes: 99:59:59.999 --> 99:59:59.999 then all of sudden you have got this set of patches and they do not quite apply… 99:59:59.999 --> 99:59:59.999 I is a drag even you do not have it in the 'debian/patches/' directory. 99:59:59.999 --> 99:59:59.999 But what Debian 'patch q' does is it maps that directory of patches into a little 99:59:59.999 --> 99:59:59.999 branch on your Git revision history. 99:59:59.999 --> 99:59:59.999 So when you get a new upstream version, you can say 'patch q rebase', 99:59:59.999 --> 99:59:59.999 and it treats it just as Git: it takes the 'patch q'… 99:59:59.999 --> 99:59:59.999 You have already imported the new version, and it re-applies your patches, 99:59:59.999 --> 99:59:59.999 and sometimes that means some minor adjustments. 99:59:59.999 --> 99:59:59.999 Git is really good at figuring out what the right minor adjustments are to make, 99:59:59.999 --> 99:59:59.999 and so all of the sudden the 'patch q' is re-based, you refresh it in your revision 99:59:59.999 --> 99:59:59.999 control system, and there you go. 99:59:59.999 --> 99:59:59.999 So I like to use 'git-buildpackage' 'patch q', tagging, as already brought up, 99:59:59.999 --> 99:59:59.999 thank you for that, I like to to tag everything that I release, 99:59:59.999 --> 99:59:59.999 I like to push that as soon as I can, so that other people who are following 99:59:59.999 --> 99:59:59.999 my work can now where my releases come from. 99:59:59.999 --> 99:59:59.999 The reason that I like other people following my work is 99:59:59.999 --> 99:59:59.999 they can fix my bugs easier. 99:59:59.999 --> 99:59:59.999 I make mistakes, everybody makes mistakes, and it is really important to me that 99:59:59.999 --> 99:59:59.999 if someone catches one of my mistakes, I can accept their feedback, 99:59:59.999 --> 99:59:59.999 their criticism, their improvements, as easily as possible. 99:59:59.999 --> 99:59:59.999 I want a low barrier to entry for people to help me fix my problems, 99:59:59.999 --> 99:59:59.999 it is selfishness. So I try to patch it and publish this things for people 99:59:59.999 --> 99:59:59.999 can find it. 99:59:59.999 --> 99:59:59.999 I am ??? on these pretty fast because were are almost at the time. 99:59:59.999 --> 99:59:59.999 I like to put in some place where other people get to the them, 99:59:59.999 --> 99:59:59.999 at the moment I like to put them in 'collab-maint', 99:59:59.999 --> 99:59:59.999 it has some problems but it is better than not publishing your stuff, 99:59:59.999 --> 99:59:59.999 and it is nice because it is sort of a public use. 99:59:59.999 --> 99:59:59.999 I like to standardize how of my branches are named, so if I am working on 99:59:59.999 --> 99:59:59.999 something that has got a stable version, that is for Jessie, I will name the branch 99:59:59.999 --> 99:59:59.999 'jessie', because I ??? 99:59:59.999 --> 99:59:59.999 ???? multiple branches ??? 99:59:59.999 --> 99:59:59.999 I try to push as frequently as I have made something that looks sensible. 99:59:59.999 --> 99:59:59.999 I do not feel obliged to push my commits to a public repository when I am still 99:59:59.999 --> 99:59:59.999 experimenting, I actually really like to experiment, and I also like to keep track 99:59:59.999 --> 99:59:59.999 of my experiments while I am doing them. 99:59:59.999 --> 99:59:59.999 So I try to push when there is a sensible set of changes, and I am trying to get 99:59:59.999 --> 99:59:59.999 myself to a point where I can understand what I have done, even if it wrong. 99:59:59.999 --> 99:59:59.999 If I can get myself to a conceptual point where it is done, I will push my changes 99:59:59.999 --> 99:59:59.999 so other people can see what I am working on, and then work from there. 99:59:59.999 --> 99:59:59.999 That is OK to push something that is wrong, 99:59:59.999 --> 99:59:59.999 as long as you push something that people can understand.