Preferred_Debian_Packaging.webm

Thank you very much.

Thanks everybody for coming,…

If you are packaging software and you want
me to work on with you,

this is how you can do that.

It is a very self-??? talk:

I just want to explain some of the things
that I like,

some practice that I prefer about Debian
packaging,

and I don't pretend this is any sort of…

official, permanent or final thing.

I just wanted to share some ideas that I
have about the way that I work with

packages, in the hope that maybe, hmm,
for two hopes:

One is that I hope that I can show you
something that you have not heard of,

or maybe you were doing differently,

or maybe you think it is
the right think to do

and it is just nice to see
somebody else doing it.

My second hope is that you can tell me
what I am doing wrong,

and you can help me learn and improve
on my own packaging techniques.

If you see something that
I am proposing up here,

and you think there is a problem with it,
I would like to hear about it too.

I just want to see more of the culture
within Debian,

of people who are doing packaging,
explaining what they are doing,

and so I thought I would
just step up and explain:

"Here is some of the practice that I do",

In the hope that other people will do the
same and explain what they are doing,

and maybe they can learn from
me and I can learn from them.

Without much further ????, I am
just going to dive into it.

If you have questions, I am perfectly
happy to be interrupted,

we have some folks with walking mics
in the crowd:

you can just raise your hand.

If you have got a question or an
interruption or whatever,

that is fine.

I doubt I will go the whole 15 minutes,
I think there are 20 minutes,

I doubt I will go the whole time,
so there will be also

time for questions at
the end if you prefer.

But I do not mind being interrupted.

So, this is all on this web page here,

you could probably skip this talk and go
read the web page,

but then you would not have the nice
??? actions,

and it is easier to tell me that I am
wrong in person,

so I would like to have that happen.

I put this up on the Debian wiki,

because I want anyone
to be able to find it.

If you think you have got some good ideas,
you should put it on the Debian Wiki too:

other people can take advantage of the
ideas that you have got.

First baseline is:
I really like revision control.

And I know that it makes me
a certain flavor on nerd,

but when we are working with things that
are as complicated as software packages,

hmmm…

I think a lot of people don't get
that in Debian you are not

just working on one
software package:

you are actually probably, if you
are doing a responsibly work,

on at least two software packages,
and maybe 5.

So you have got the version that is
unstable,

and you have got the version that you
try to maintain for stable as well.

And we are committing to
doing maintenance work.

A lot of our work in the project is ???
in nature:

we want to clean up the mess and
we want us to stay out of the way

and make sure things work,
functionally,

for people who are relying on the
operating system to not get in their way.

So revision control I think is really
helpful because it means you can

keep track of what changes you have done
on different branches of the project

while you are maintaining both of them.

Basically, ??? require working with
the revision system I am comfortable with,

I prefer Git, I am not going to have a
religious word about it.

If upstream uses Git, I am
even happier, and I try to make

my packaging depend on
upstream's revision control.

I like to use 'git-buildpackage', and I
like to use it with debhelper.

If you have not tried out
'git-buildpackage',

we are going to have a
'git-buildpackage' skill share session

later on today actually, and I welcome
you to come and share your tricks with it,

or learn some tricks from other people.

It is a particular way that you can keep
your Debian packaging in a Git repository,

and it helps you to keep track of all of
the changes that have happened within

your packaging and within upstream to
make sure you are not accidentally

making other changes.

So it is very easy to go back and review
what you have done.

I find that really useful.

I definitely also like to keep upstream's
source code in the same revision control

system.

I like to keep the tarballs in the
revision control system because it means

that if someone is interested, they can
uses a tool called 'debcheckout'.

You can use 'debcheckout' with a name of
a package:

you say just "I am really interested in
package 'foo',

let me see the source code for that":

debcheckout foo

You get the source code, and you get the
source code from a revision control

system that you can now track and you
can just propose changes on.

You can also extract the tarball from that
revision control system.

'debcheckout' actually works even if you
do not have upstream stuff in there,

but I like to keep it all in one revision
control system,

it is just easier to find everything when
you want.

Some of these things that I prefer have
to do with what the upstream software

developer has done, so I am less inclined
to try the package an upstream software

project if they just throw tarballs here
over the wall to an FTP side

every now and then.

It makes it more difficult for me to know
what they are doing,

and why they are doing it.

So i like it, I have already said, when
upstream uses Git,

I also like when upstream signs their
releases,

and say "hey, this is specific release",

Because that is a signal that I can use,
or somebody else that understands the

project: as said "we think that this
something that other people can use",

or "this is a particular version we would
like other people to test".

There are a lot of other situations where
maybe it is not so important.

And having that be cryptographically
signed is really useful.

I care about cryptographic signature on
software because I want to know that

what I am running is related to the code
that somebody else out should be run.

And if you don't verify your software
cryptographically, anyone could

intercept the network connection
between you and that software,

and modify the software before it gets
to you.

And the cryptographic signature just says:

"look, this is a version that I am OK
with. I am putting it out there and

it comes from me".

And so I can have a trace back to that
point.

??? just talk about briefly about how you
do cryptographic verification of upstream.

You might know upstream: you might know
them personally, you know their key

already, that is fine.

That is not the usual case: we work on
the Internet.

In the situation where your upstream is
signing their tarballs

and you have not met them, you do not
have to sign their key,

you do not have to say "I announce this
is their key".

This is probably the same one that is
signing every release,

so you should keep track of that.

Debian has a nice way to keep track of
that:

you can tell Debian how to find the new
version of the upstream tarball.

This is in the Debian 'watch' file.

If you type 'man uscan', you can learn
more about Debian 'watch',

and Debian 'watch' has now a feature that
lets you say

"that is not only this way you find the
tarball,

but upstream publishes signatures
and the signatures look like this".

You know, they got a '.sig' at the end.

So there is a particular arcane way to
specify that, but if you specify that,

then 'uscan' can find not only the
upstream tarball but can find the

upstream signature and, if you drop
upstream's signing key -

which of course I did not put on the wiki
page, someone should add it that and

fix it - you can put the upstream signing
key in 'debian/upstream/signing-key.asc'.

And then if you do that, when you say
'uscan', you can tell…

Maybe some people here do notk
now how to use 'uscan'.

'uscan' is a very simple tool,
you run it from a software package that

has a 'debian' directory, or even one
level up if you keep all of your software

packages in one folder. You can go one
level up and say 'uscan', and it will look

in all of the folder that are children
of it, and look for new version by

trying to find for new upstreams versions
in 'debian/watch'.

And if you have configured 'debian/watch'
properly, it can find the new upstream

signatures, and if you have got the
'upstream/signing-key.asc', then

it will actually verify the signature for
you as part of fetching the new

upstream tarball.

So you can get all of those things just
by setting ???? that way.

There is a hand up down there, could we
get the mic down to the hand ?

Or to the person who has that hand, it is
not just a hand. [public laugh]

[someone] Publish a tarball and a hash, '.sha1',
and sign that hash, '.sha1.asc'.

Can 'uscan' cope with this and check the
signature on the hash and that the hash

belongs to that tarball ?

[Daniel] I do not believe that 'uscan' can
do that currently. So anybody out there

who wants to make things better for the
world should go hack on 'uscan':

that is a pretty straightforward thing
that we should fix because I agree

that is common pattern.

[someone] I have no answer to this
question by I have another question:

how do you convince upstreams who do
not release tarballs or who do

not set tags in Git ?

[Daniel] Who do not make tags in Git ?

[someone] Yes, if there is no tags you
can not check out a tarball.

Is there any good way to convince
upstream to do this ?

[Daniel] Git has this nice feature, which
is that you can create a tag,

which is associate with a particular
revision,

and you would like to have a tag
everywhere that a tarball has been

released from. I am tempted to pull up
a Git view and show people some tags.

The question that you ask is a social
one, not just a technical one,

and I actually find that my upstreams
are pretty responsive.

Usually I frame my request as "hey, it
looks like you made this tarball from

this particular commit 'id'. If you could
tag you releases, it would be really

helpful to me, and here is the command
that I would use to tag the release".

And I say "git tag…" and of course I
can never remember so first I look it up,

but it is either 'tag name' 'commit id' or
'commit id' 'tag name'.

But I would look it up and I would write
the e-mail so that all they have to do is

they read it, understand my argument,
and execute one command.

And then it starts them ??????

And if you say 'tag -s' then your tag will
be cryptographically signed, which

I think is a really good thing to do too.

So, cryptographic verification of
upstream.

As I said, I want to keep upstream's code
in the revision control system.

I also like to keep…

In my ideal case upstream is using Git:
I am using Git for packaging.

I actually like to keep upsteam's Git
history fully in my repository,

so that I do not just have the tarballs,
but I actually have all of their commits.

And that turns out to be really useful
for two specific cases:

In one case, there is a common scenario
where upstream will fix a bug,

but they have not made a release yet.

And that bug is really, really obviously
problematic for the folks who are

using Debian, so want to fix it.

All I can do, because I have their full
revision history, I can use Git to "cherry

pick" the upstream commit.

And then I "cherry pick" that upstream
commit and I can have it applied

separately and release an Debian version
that has the fix, even before upstream

has made a release with the fix.

So one nice thing about having upstream
revision is that I can pull fixes from

upstream before they decided
to release it.

The other advantage is the other
way around.

Often when I am doing packaging,
I discover a problem,

and maybe I can fix the problem.

And if that maybe I am already shipping
a Debian package that fixes the problem.

If my Debian fixes can be directly applied
to upstream, then I can use whatever

their preferred upstream patch
submission guidelines are,

whether it is a Github pull request, or
a patch to a mailing list,

or a "hey can you pull this from my Git
repository over here", e-mail…

The fact that I am using the same Git
history that they are using makes it

much easier for me to push my changes
back to them.

So, it sort of smooth the interaction if
you can consolidate and use the same

revision control system as their.

Towards that aim, I use a system now
called 'patch q',

which is part of 'git buildpackage'.

So 'git buildpackage' is 'gbp', 'patch q'
is 'pq',

so to deal with 'patch q' you say
'gbp pq'

and then you have some commands.

And what that does, is it takes…

How many of you are Debian packagers ?

How many of you package
software for Debian ?

A very large percentage, but not everyone.

I hope some folks are considering starting
packaging if you have not done it yet.

Of those of you who package software,
how many of you package software

with modifications, how many of you ship
a modified version of upstream sources ?

Beyond the 'debian' directory, just Debian
patches ?

So the common way to do that, for the
Debian 3.0 ??? packaging skill, is that in

your 'debian' directory you have a
'patches' sub-directory that has a set of

individual patches that apply certain
changes, and they are applied in order

based on the file called
'debian/patches/series'.

So maintaining that is kind of a drag
when upstream makes big changes:

then all of sudden you have got this set
of patches and they do not quite apply…

I is a drag even you do not have it in
the 'debian/patches/' directory.

But what Debian 'patch q' does is it maps
that directory of patches into a little

branch on your Git revision history.

So when you get a new upstream version,
you can say 'patch q rebase',

and it treats it just as Git: it takes the
'patch q'…

You have already imported the new version,
and it re-applies your patches,

and sometimes that means some minor
adjustments.

Git is really good at figuring out what
the right minor adjustments are to make,

and so all of the sudden the 'patch q' is
re-based, you refresh it in your revision

control system, and there you go.

So I like to use 'git-buildpackage' 'patch q',
tagging, as already brought up,

thank you for that, I like to to tag
everything that I release,

I like to push that as soon as I can,
so that other people who are following

my work can now where my releases
come from.

The reason that I like other people
following my work is

they can fix my bugs easier.

I make mistakes, everybody makes mistakes,
and it is really important to me that

if someone catches one of my mistakes,
I can accept their feedback,

their criticism, their improvements,
as easily as possible.

I want a low barrier to entry for people
to help me fix my problems,

it is selfishness. So I try to patch it
and publish this things for people

can find it.

I am ??? on these pretty fast because
were are almost at the time.

I like to put in some place where other
people get to the them,

at the moment I like to put them in
'collab-maint',

it has some problems but it is better
than not publishing your stuff,

and it is nice because it is sort of
a public use.

I like to standardize how of my branches
are named, so if I am working on

something that has got a stable version,
that is for Jessie, I will name the branch

'jessie', because I ???

???? multiple branches ???

I try to push as frequently as I have made
something that looks sensible.

I do not feel obliged to push my commits
to a public repository when I am still

experimenting, I actually really like to
experiment, and I also like to keep track

of my experiments while I am doing them.

So I try to push when there is a sensible
set of changes, and I am trying to get

myself to a point where I can understand
what I have done, even if it wrong.

If I can get myself to a conceptual point
where it is done, I will push my changes

so other people can see what I am
working on, and then work from there.

That is OK to push something that is
wrong,

as long as you push something that
people can understand.

When you make a 'git commit' (if you are
working with Git),

One of the things that helps me to think
about commit messages…

People often think that commit messages
should say "what change you made".

I think that the 'git patch' shows what
change what change you have made,

and I thin your commit messages should
say "why you made the change".

That is what people really want to read.

If you need to explain technically why
the thing that you did

maps to the conceptual thing that you
wanted to do, that is fine:

do that in your commit message too.
But it is really important to say why

you made the change. It is not just like
"initialize variable to 'no'":

OK, we can see that from the patch,
but you what you are really saying

is "there was a crash if someone did 'x',

and we are avoiding that crash by
setting this to 'no'.

So I like to send patches via email,

so I try to configure Git email, which
make it really easy to just

push patches back upstream.

If I am starting taking over a project
that somebody else has past on,

and they did not use Git, I will try to
restore all of the imports.

I would be happy to talk with people
about how to do that,

if you have questions come find me.

I like to keep my files ???? simple:
there is a tool 'wrap-and-sort',

that just canonicalizes your files to
make them look in a simple and

sensible way. And it is nice because it
means that everything is…

It does things like alphabetize your
list of build depends,

and brake them out one per line.
The nice thing about that,

since you are using revision control,
when you make a change

to your build depends, the changes
become very easy to see:

"oh, they added one new package here,
there is a single '+'".

When ???? so you can see that kind of
thing.

I like to use ? deb five ? to format
Debian copyright to be machine readable,

it is nice for people who are doing scans
of the archive and try reason about

what the patterns are, and licensing of
free software.

And if I am doing something really crazy,
that is going to make a big change,

I like to use a feature branch in
revision control.

So we have got one minute left,
I want to open it up for other questions.

????

[attendee] You said you are using
'wrap-and-sort' which is nice,

I had learned that ???? editors
- 'cme' - do the same job,

and somehow does a better job:
it also ??? standard version

if it does not fit, or it makes VCS
fields properly has it should be.

'cme fix dpkg-control' fixes your
control file.

[Daniel] 'cme' ? And it is in
what package ?

[attendee] The package 'cme', in
unstable ????. In Jessie it is ????

[Daniel] You are developing in unstable,
that is OK.

'cme'
OK, thank you.

Other questions or suggestions,
or complains ?

[attendee] If you change the original
source code, and do some commits,

how do you convert that into a series
of ??? patches ?

[Daniel] I use 'patch q' for that as well,
so what I do is I say

"I want to move over to my 'patch q'
view of the tree",

and then I make may changes, I make
my commits,

and then I say
'gbp pq export',

and it takes the 'patch q' that I am on
and dumps it back

into the Debian patches directory.

If you have not use 'gbp pq', I
recommend looking into it.

It takes a little while to get used to,
and I still screwed it up sometimes,

but it makes easy to fix your
mistakes too.

[organizer] Last question ?

[attendee] Do you think it is possible to
make this 'patch q' branch

"pullable" by upstream ?

[Daniel] I do not actually think it is
possible to make it directly

"pullable" by upstream: I think upstream
can cherry pick patches from it,

but I do not see how to
make it "pullable".

If someone else does, I would be
happy to learn.

[organizer] This was "before last",
so last.

[attendee] Do you have a recording of
you using the tools that you mentioned,

a video recording would be great,
just to show your workflow ?

[Daniel] I do not really know how
to do that: if somebody wants to

help me do that I would be
happy to do it.

I am going to give one last plug,

I know we are out of time here,
sorry.

This tool is called 'gitk'.

This is an example - sorry we
should leave - but this the way

that I visualize my revision
control system.

We could do a whole other session
about 'gitk'.

If you do not try to visualize your
revision control system,

you are missing out: try to find a way
to visualize stuff,

find one that works for you.

Thanks for coming.

[organizer] Thank you.