-
Thank you very much.
-
Thanks everybody for coming,…
-
If you are packaging software and you want
me to work on with you,
-
this is how you can do that.
-
It is a very self-??? talk:
-
I just want to explain some of the things
that I like,
-
some practice that I prefer about Debian
packaging,
-
and I don't pretend this is any sort of…
-
official, permanent or final thing.
-
I just wanted to share some ideas
that I have about
-
the way that I work with packages,
in the hope that maybe…
-
For two hopes:
-
One is that I hope that I can show you
something that you have not heard of,
-
or maybe you were doing differently,
-
or maybe you think it is
the right think to do
-
and it is just nice to see
somebody else doing it.
-
My second hope is that you can tell me
what I am doing wrong,
-
and you can help me learn and improve
on my own packaging techniques.
-
If you see something that
I am proposing up here,
-
and you think there is a problem with it,
I would like to hear about it too.
-
I just want to see more of the culture
within Debian,
-
of people who are doing packaging,
explaining what they are doing,
-
and so I thought I would
just step up and explain:
-
"Here is some of the practice that I do",
-
In the hope that other people will do the
same and explain what they are doing,
-
and maybe they can learn from
me and I can learn from them.
-
Without much further ????, I am
just going to dive into it.
-
If you have questions, I am perfectly
happy to be interrupted,
-
we have some folks with walking mics
in the crowd:
-
you can just raise your hand.
-
If you have got a question or an
interruption or whatever,
-
that is fine.
-
I doubt I will go the whole 15 minutes,
I think there are 20 minutes,
-
I doubt I will go the whole time,
so there will be also
-
time for questions at
the end if you prefer.
-
But I do not mind being interrupted.
-
So, this is all on this web page here,
-
you could probably skip this talk and go
read the web page,
-
but then you would not have the nice
??? actions,
-
and it is easier to tell me that I am
wrong in person,
-
so I would like to have that happen.
-
I put this up on the Debian wiki,
-
because I want anyone
to be able to find it.
-
If you think you have got some good ideas,
you should put it on the Debian Wiki too:
-
other people can take advantage of the
ideas that you have got.
-
First baseline is:
I really like revision control.
-
And I know that it makes me
a certain flavor on nerd,
-
but when we are working with things that
are as complicated as software packages,
-
hmmm…
-
I think a lot of people don't get
that in Debian you are not
-
just working on one
software package:
-
you are actually probably, if you
are doing a responsibly work,
-
on at least two software packages,
and maybe 5.
-
So you have got the version that is
unstable,
-
and you have got the version that you
try to maintain for stable as well.
-
And we are committing to
doing maintenance work.
-
A lot of our work in the project is ???
in nature:
-
we want to clean up the mess and
we want us to stay out of the way
-
and make sure things work,
functionally,
-
for people who are relying on the
operating system to not get in their way.
-
So revision control I think is really
helpful because it means you can
-
keep track of what changes you have done
on different branches of the project
-
while you are maintaining both of them.
-
Basically, ??? require working with
the revision system I am comfortable with,
-
I prefer Git, I am not going to have a
religious word about it.
-
If upstream uses Git, I am
even happier, and I try to make
-
my packaging depend on
upstream's revision control.
-
I like to use 'git-buildpackage', and I
like to use it with debhelper.
-
If you have not tried out
'git-buildpackage',
-
we are going to have a
'git-buildpackage' skill share session
-
later on today actually, and I welcome
you to come and share your tricks with it,
-
or learn some tricks from other people.
-
It is a particular way that you can keep
your Debian packaging in a Git repository,
-
and it helps you to keep track of all of
the changes that have happened within
-
your packaging and within upstream to
make sure you are not accidentally
-
making other changes.
-
So it is very easy to go back and
review what you have done.
-
I find that really useful.
-
I definitely also like to keep
upstream's source code
-
in the same revision control system.
-
I like to keep the tarballs in the
revision control system
-
because it means that
if someone is interested,
-
they can uses a tool called
'debcheckout'.
-
You can use 'debcheckout' with
a name of a package:
-
you just say "I am really
interested in package 'foo',
-
let me see the source code for that":
'debcheckout foo'
-
You get the source code, and
you get the source code…
-
from a revision control system
that you can now track
-
and you can just propose changes on.
-
You can also extract the tarball from that
revision control system.
-
'debcheckout' actually works even if you
do not have upstream stuff in there,
-
but I like to keep it all in one
revision control system,
-
it is just easier to find everything
when you want.
-
Some of these things that
I prefer have to do with
-
what the upstream software
developer has done,
-
so I am less inclined to try the package,
an upstream software project,
-
if they just throw tarballs
here over the wall
-
to an FTP side every now and then.
-
It makes it more difficult for me to
know what they are doing,
-
and why they are doing it.
-
So i like it, I have already said,
when upstream uses Git,
-
I also like it when upstream
signs their releases,
-
and says "hey, this is specific release",
-
because that is a signal
that I can use,
-
or somebody else that
understands the project.
-
As said, "we think that this is
something that other people can use",
-
or "this is a particular version that
we would like other people to test".
-
There are a lot of other situations where
maybe it is not so important.
-
And having that be cryptographically
signed is really useful.
-
I care about cryptographic signature on
software because I want to know that
-
what I am running is related to the code
that somebody else out should be run.
-
And if you don't verify your
software cryptographically,
-
anyone who can intercept
the network connection
-
between you and that software, can
modify the software before it gets to you.
-
And the cryptographic signature just says:
-
"look, this is a version
that I am OK with.
-
I am putting it out there
and it comes from me".
-
So I can have a trace back
to that point.
-
??? just talk about briefly about how you
do cryptographic verification of upstream.
-
One is you might know upstream:
-
you might know them personally, you
know their key already, that is fine.
-
That is not the usual case:
we work on the Internet.
-
In the situation where your upstream
is signing their tarballs
-
and you have not met them,
you do not have to sign their key,
-
you do not have to say
"I announce this is their key".
-
But it is probably the same one
that is signing every release,
-
so you should keep track of that.
-
Debian has a nice way to
keep track of that:
-
you can tell Debian how to find the new
version of the upstream tarball.
-
This is in the Debian 'watch' file.
-
If you type 'man uscan', you can learn
more about Debian 'watch',
-
and Debian 'watch' now has
a feature that lets you say
-
"that is not only this way
you find the tarball,
-
but upstream publishes signatures
and the signatures look like this".
-
You know, they got a '.sig' at the end.
-
So there is a particular arcane way to
specify that, but if you specify that,
-
then 'uscan' can find
not only the upstream tarball,
-
it can find the upstream signature.
-
And, if you drop
upstream's signing key
-
- which of course I did not
put on the wiki page,
-
someone should
edit that and fix it -
-
you can put the upstream signing
key in 'debian/upstream/signing-key.asc'.
-
And then if you do that, when you say
'uscan', you can tell…
-
Maybe some people here do not
know how to use 'uscan':
-
'uscan' is a very simple tool,
-
you run it from a software package
that has a 'debian' directory,
-
or even one level up if you keep all of
your software packages in one folder.
-
You can go one level up
and say 'uscan',
-
and it will look in all of the folder
that are children of it,
-
and look for new versions
-
by trying to find a new upstream
version in 'debian/watch'.
-
And if you have configured
'debian/watch' properly,
-
it can find the new upstream signatures,
-
and if you have got the
'upstream/signing-key.asc',
-
then it will actually verify
the signatures for you
-
as part of fetching the
new upstream tarball.
-
So you can get all of those things just
by setting ???? that way.
-
There is a hand up down there, could we
get the mic down to the hand ?
-
Thanks.
-
Or to the person who has that hand,
it is not just a hand. [public laugh]
-
[attendee] Publish a tarball,
and a hash, '.sha1',
-
and sign that hash,
'.sha1.asc'.
-
Can 'uscan' cope with this and
check the signature on the hash
-
and that the hash belongs
to that tarball ?
-
[Daniel] I do not believe that 'uscan'
can do that currently.
-
So anybody out there who wants to make
things better for the world
-
should go hack on 'uscan': that is a
pretty straightforward thing
-
that we should fix because
I agree that is common pattern.
-
[attendee] I have no answer to this
question by I have another question:
-
how do you convince upstreams
who do not release tarballs
-
or who do not set tags in Git ?
-
[Daniel] Who do not make tags in Git ?
-
[someone] Yes, if there is no tags
you can not check out a tarball.
-
Is there any good way to
convince upstream to do this ?
-
[Daniel] Git has this nice feature, which
is that you can create a tag,
-
which is associated with
a particular revision,
-
and you would like to have a tag
-
everywhere that a tarball
has been released from.
-
I am tempted to pull up a Git
view and show people some tags.
-
The question that you ask is a
social one, not just a technical one,
-
and I actually find that my upstreams
are pretty responsive.
-
Usually I frame my request as
-
"hey, it like you made this tarball
from this particular commit 'id'.
-
If you could tag you releases,
it would be really helpful to me,
-
and here is the command
that I would use to tag the release".
-
And I say "git tag…"
-
and of course I can never
remember so first I look it up,
-
but it is either 'tag name' 'commit id'
or 'commit id' 'tag name'.
-
But I would look it up and
I would write the email so that
-
all they have to do is they read it,
-
understand my argument,
-
and execute one command.
-
And then it starts them ??????
-
And if you say 'tag -s',
-
then your tag will be
cryptographically signed,
-
which I think is a really
good thing to do too.
-
So, cryptographic verification
of upstream.
-
As I said, I want to keep upstream's code
in the revision control system.
-
I also like to keep…
-
In my ideal case upstream is using Git:
I am using Git for packaging.
-
I actually like to keep upsteam's Git
history fully in my repository,
-
so that I do not just have the tarballs,
but I actually have all of their commits.
-
And that turns out to be really useful
for two specific cases:
-
In one case, there is a common scenario
where upstream will fix a bug,
-
but they have not made a release yet.
-
And that bug is really,
really obviously problematic
-
for the folks who are using Debian,
-
so I want to fix it.
-
All I can do, because I have
their full revision history,
-
I can use Git to "cherry pick"
the upstream commit.
-
And then I "cherry pick"
that upstream commit
-
and I can have it applied separately,
-
and release a Debian version
that has the fix,
-
even before upstream has made
a release with the fix.
-
So one nice thing about
having upstream revision
-
is that I can pull fixes from upstream
before they decided to release it.
-
The other advantage is the other
way around.
-
Often when I am doing packaging,
I discover a problem,
-
and maybe I can fix the problem.
-
And if that maybe I am already shipping
a Debian package that fixes the problem.
-
If my Debian fixes can be
directly applied to upstream,
-
then I can use whatever their preferred
upstream patch submission guidelines are,
-
whether it is a Github pull request, or
a patch to a mailing list,
-
or a "hey can you pull this from my Git
repository over here", e-mail…
-
The fact that I am using the same
Git history that they are using
-
makes it much easier for me
to push my changes back to them.
-
So, it sort of smooths the interaction
if you can consolidate
-
and use the same revision control
system as their.
-
Towards that aim,
-
I use a system now
called 'patch q',
-
which is part of 'git-buildpackage'.
-
So 'git buildpackage' is 'gbp',
-
'patch q' is 'pq',
-
so to deal with 'patch q'
you say 'gbp pq'
-
and then you have some commands.
-
And what that does, is it takes…
-
How many of you are Debian packagers ?
-
How many of you package
software for Debian ?
-
[many attendees raise their hand]
-
A very large percentage, but not everyone.
-
I hope some folks are considering starting
packaging if you have not done it yet.
-
Of those of you who package software,
-
how many of you package software
with modifications,
-
how many of you ship a
modified version of upstream sources ?
-
[many attendees raise their hand]
-
Beyond the 'debian' directory,
just Debian patches ?
-
So the common way to do that,
-
for the Debian 3.0 ???
packaging skill,
-
is that in your 'debian' directory you
have a 'patches' sub-directory
-
that has a set of individual patches
that apply certain changes,
-
and they are applied in order based on
the file called 'debian/patches/series'.
-
So maintaining that is kind of a drag
when upstream makes big changes:
-
then all of sudden you have got this set
of patches and they do not quite apply…
-
It is a drag even you do not have it
in the 'debian/patches/' directory.
-
But what Debian 'patch q' does is
it maps that directory of patches
-
into a little branch on your
Git revision history.
-
So when you get a new upstream version,
you can say 'patch q rebase',
-
and it treats it just as Git:
it takes the 'patch q'…
-
You have already imported the new version,
and it re-applies your patches,
-
and sometimes that means
some minor adjustments.
-
Git is really good at figuring out what
the right minor adjustments are to make,
-
and so all of the sudden
the 'patch q' is re-based,
-
you refresh it in your revision
control system…
-
and there you go.
-
So I like to use
'git-buildpackage' 'patch q',
-
tagging, as already brought up,
thank you for that,
-
I like to to tag
everything that I release,
-
I like to push that
as soon as I can,
-
so that other people
who are following my work
-
can know where my releases
come from.
-
The reason that I like other people
following my work is
-
they can fix my bugs easier.
-
I make mistakes,
everybody makes mistakes,
-
and it is really important to me that
if someone catches one of my mistakes,
-
I can accept their feedback,
their criticism, their improvements,
-
as easily as possible.
-
I want a low barrier to entry for people
to help me fix my problems:
-
it is selfishness.
-
So I try to patch it and publish these
things for people can find it.
-
I am ??? on these pretty fast because
were are almost at the time.
-
I like to put my repo in some place
where other people get to the them,
-
at the moment I like to put them in
'collab-maint',
-
it has some problems but it is better
than not publishing your stuff,
-
and it is nice because it is sort of
a public use.
-
I like to standardize how of
my branches are named,
-
so if I am working on something
that has got a stable version,
-
that is for Jessie, I will
name the branch 'jessie',
-
because I ???
???? multiple branches ???
-
I try to push as frequently as I have made
something that looks sensible.
-
I do not feel obliged to push
my commits to a public repository
-
when I am still experimenting,
-
I actually really like to
experiment,
-
and I also like to keep track of my
experiments while I am doing them.
-
So I try to push when there is
a sensible set of changes,
-
and I am trying to get myself
-
to a point where I can understand
what I have done, even if it is wrong.
-
If I can get myself to a conceptual
point where it is done,
-
I will push my changes so other people
can see what I am working on,
-
and then work from there.
-
That is OK to push something
that is wrong,
-
as long as you push something that
people can understand.
-
When you make a 'git commit'
(if you are working with Git),
-
one of the things that helps me to think
about commit messages…
-
People often think that commit messages
should say "what change you made".
-
I think that the 'git patch' shows what
change what change you have made,
-
and I thin your commit messages should
say "why you made the change".
-
That is what people really want to read.
-
If you need to explain technically
why the thing that you did
-
maps to the conceptual thing
that you wanted to do,
-
that is fine: do that in
your commit message too.
-
But it is really important to say
why you made the change.
-
It is not just like
"initialize variable to 'no'":
-
OK, we can see that from the patch,
-
but what you are really saying is
"there was a crash if someone did 'x',
-
and we are avoiding that crash by
setting this to 'no'.
-
So I like to send patches via email,
-
so I try to configure Git email,
-
which make it really easy to just
push patches back upstream.
-
If I am starting taking over a project
that somebody else has past on,
-
and they did not use Git,
I will try to restore all of the imports.
-
I would be happy to talk with people
about how to do that,
-
if you have questions come find me.
-
I like to keep my files ???? simple:
-
there is a tool 'wrap-and-sort',
-
that just canonicalizes your files
to make them look
-
in a simple and sensible way.
-
And it is nice because it
means that everything is…
-
It does things like alphabetize
your list of build depends,
-
and brake them out one per line.
-
The nice thing about that,
since you are using revision control,
-
when you make a change
to your build depends,
-
the changes become
very easy to see:
-
"oh, they added one new package here,
there is a single '+'".
-
When ????
so you can see that kind of thing.
-
I like to use ? deb five ? to format
Debian copyright to be machine readable,
-
it is nice for people who are
doing scans of the archive
-
and try reason about
what the patterns are,
-
and licensing of free software.
-
And if I am doing something really crazy,
that is going to make a big change,
-
I like to use a feature branch in
revision control.
-
So we have got one minute left,
I want to open it up for other questions.
-
????
-
[attendee] You said you are using
'wrap-and-sort' which is nice,
-
I had learned that ???? editors
- 'cme' - do the same job,
-
and somehow does a better job:
it also ??? standard version
-
if it does not fit, or it makes VCS
fields properly has it should be.
-
'cme fix dpkg-control' fixes your
control file.
-
[Daniel] 'cme' ? And it is in
what package ?
-
[attendee] The package 'cme', in
unstable ????. In Jessie it is ????
-
[Daniel] You are developing in unstable,
that is OK.
-
'cme'
OK, thank you.
-
Other questions or suggestions,
or complains ?
-
[attendee] If you change the original
source code, and do some commits,
-
how do you convert that into a series
of ??? patches ?
-
[Daniel] I use 'patch q' for that as well,
so what I do is I say
-
"I want to move over to my 'patch q'
view of the tree",
-
and then I make may changes, I make
my commits,
-
and then I say
'gbp pq export',
-
so that 'patch q export',
-
and it takes the 'patch q'
that I am on
-
and dumps it back into
the Debian patches directory.
-
If you have not use 'gbp pq', I
recommend looking into it.
-
It takes a little while to get used to,
and I still screwed it up sometimes,
-
but it makes easy to fix your
mistakes too.
-
[organizer] Last question ?
-
[attendee] Do you think it is possible
-
to make this 'patch q' branch
"pullable" by upstream ?
-
[Daniel] I do not actually think it is
possible to make it directly
-
"pullable" by upstream: I think upstream
can cherry pick patches from it,
-
but I do not see how to
make it "pullable".
-
If someone else does, I would be
happy to learn.
-
[organizer] This was "before last",
so last.
-
[attendee] Do you have a recording of
you using the tools that you mentioned,
-
a video recording would be great,
just to show your workflow ?
-
[Daniel] I do not really know how
to do that:
-
if somebody wants to help me do that
I would be happy to do it.
-
I am going to give one last plug,
-
I know we are out of time here,
sorry.
-
This tool is called 'gitk'.
-
This is an example…
-
- sorry we should leave -
-
but this the way that I visualize
my revision control system.
-
We could do a whole other session
about 'gitk'.
-
If you do not try to visualize your
revision control system,
-
you are missing out:
-
I recommend try to find a way
to visualize stuff,
-
find one that works for you.
-
Thanks for coming.
-
[organizer] Thank you.