1009_Packaging_for_beginners.ogv

The next talk is packaging for beginners

by Gergely Nagy

This talk will be,

about packaging in a

much, different style than

most would expect.

So, first of all

who amoung you have

read the new maintainers guide

or, the packaging guide or

any of that stuff?

How many of you

enjoyed it? [laughter]

[laughter]

Alright, well that's

because they are very dry

and to make

this talk, better and useful

I will ignore everything that is written in

them. [laughter] I would advise you

to do the same because they are very

useful once you know, what

you are doing but when you are

trying

your first steps, they are

an awful, boring

piece of something.

So, we will

do something very different today.

Another question, how

many of you have read the policy

from beginning to end?

ok, congratulations!

1 person, how many

of you tried to read

the policy ?

How many of you got bored

after the first 5 minutes?

Yes,

because the poilcy is not meant to be

read from begining to

end. You look at it

trying to find things when lintian,

yells at you.

So, that's what we will do

today.

I have one small announcement

to make,

I have bad news. I promised

ponies but it turns out

ponysay is not

free software. At, least

the data is not.

So, this talk we will

package something entirely different,

but since I promised ponies!

Here is one I drew,

today.

Doesn't look

like a pony! but.

Okay.

So, let's look at

what we will be packaging today.

It's a little program I

wrote.

Fit for,

Debian's 20th birthday!

The cake

is a lie,

appearently it isn't, but

it's an ASCII art so it doesn't

look like a cake.

But, if you squint a little you might notice,

that it is one.

And, the resolution

is terrible, so we have

the text all over the place. [laughter]

But that is ok, we don't mind.

It looks better when the resolution

is bigger, trust me on that. Anyway,

we will take this piece of software

it's, really easy and package

it.

It's your

standard how to

based thing written in C.

It has one

source file and that is about it.

How do you go about packaging

it ? Well, the easiest

way is to call

dpkd-buildpackage

command.

And see what it tells you,

well it's missing a changelog and a debian

directory, so let's make one.

There is a handy tool called

'dch', which is

short of deb change

it is mean't to,

help you write change logs.

And we will use it to

create one.

We fill it out

you've probably seen a few change logs.

It's not rocket science.

Oh,

ok,

sorry.

Fill it out.

We do not have an ITP,

bug to close, so that's about it.

Let's see what it says now.

Oh,

lots of text! What did it

type. Oh,

we need a control file! What is a control file?

Well, a control file

tells, dpkg

what

sources you have and what binaries

packages you want to build.

So, we will hack up a really

easy one.

First, of all

every control file

has to have 2

sections. The first section

describes the source

it looks a bit

like,

ok,

sorry I always forget my editor is

a bit dumb!

The first section

describes the source,

which

in this case

is this.

We also

have a couple of other fields we need

here like, maintainer

we need a Section

lets see

games, will be fine

and the

priotity which will be extra.

That should

be enough for starters,

and I need to learn to spell.

The next section,

is about the binary

we will build one

binary package

except it's called Package.

It has the same name.

We give it a description,

and,

the long description starts

right after in the next line.

You have to start it with a space,

because,

this whole control files looks like an email.

And starting

with a space tells the,

parser it

is a continuation,

of what was above.

Ok and that is about it.

So, we have a very

simple control file.

Let's see what the tool

tells us.

Oh,

right we need an Architecture field.

Every package in Debian must,

have an

Architecture associated

with it.

It can be

either

and Arch all

package, which means the same

binary will run on all

architectures.

In our case this is,

wrong, because this is a C

program which is compiled and

it depends on the architecture.

So, we write in here

you could list

the architectures here if your

package is something that

only works on a certain subset of

of architectures.

You can make it Linux,

specific by writing linux-any

or FreeBSD specific.

by writing,

kfreebsd-any.

This is a portable thing so we will write,

any.

Going further,

now it needs a rules

file. We are progressing,

a rules file is used

to do the hard

work.

It tells dpkg

rules

about how to build a

package. In the most simple case

like ours,

we will deal with

this only.

This is 'magic',

'black magic',

but the wonderful thing about

black magic is that you don't need to

understand it. If you need

to change the way,

the magic works.

You only have to understand little

pieces of it, and thankfully

deb helper, which

dh is a part of

has wonderful documentation.

Man pages and so on,

and so forth.

I would suggest,

that if you start packaging,

something. Start with this

and

if something goes wrong, go from there.

We will see how things,

will go wrong and

by the way Debian

rules is a make file thats

why we have this first

line here.

We also need to make

it executable, of course.

Now, lets see what happens

oh, yeah.

Since it's a make file,

I have to make it a makefile.

There we go.

Well,

it did things.

But, it also

gave us a lot of

errors and

other things.

So, we need to tell this

tool to ignore,

the git directory

and a couple of other

things. The best way to do that,

is to set the

source format to,

the one it suggests.

Right, here at the bottom.

We can do that by

creating a new directories,

called debian and then source.

And, creating

a new file in it

called format.

Which has the text,

'3.0

(quilt)' in it.

What this format does you

don't need to know, at least

not right now. It's

pretty complicated and

its a similiar

kind of 'magic' as debhelper.

It just has less documentation.

We also want to tell debhelper that,

we want to use

the latest and greatest version of

it. It was

complaining about that to at the beginning,

but I'm not going to scroll that high up.

Which we can do by,

echoing a '9' into a file called

debian/compat .

Now, lets see what happens.

Well, this is

much, much,

improved.

It compilied and now it is doing,

somethings and it

it is also asking for my password.

No, I'm not going to sign it right now.

So, it did stuff.

It also built,

a debian package for us,

which is great.

But I thing

is, a lot of

things missing from this.

For example,

we can run lintian,

on the changes file
we can run lintian,

on the changes file

and it will tell us all kinds of things.

A lot of bad, bad

things about the package.

Ok,

let's see,

we are missing the maintainer name, which I'm not going

to fix right now.

I'm not going to fix the

changelog either, because I

know better.

It's missing a copyright file,

which I'm not going to add

either, because writing a

copyright file could take an

entire session of its own.

I can add the standards version

field.

Which is as easy,

as this, the standards

version field,

signals which version

of policy this package was

made to conform to.

What is worse, is that

we are missing dependencies.

If we look into the package

debian package.

We will see that there is no depends

line, if we try to install this

It may work

and on my computer it will because

I already have all the dependancies installed.

But since it's a compiled

thing, it will at least

need libc,

and probably a few other

things as well.

So, we do

another kind of magic.

And add

the depends line to

the binary section.

Now,

is this line of magic,

which looks like shell

variable except,

they are not.

The way it works,

is when debhelper

runs, it will

find all the

packages that your binary

is dependant on. At least if they are

in 'C' and

fill

this variable out,

with the results.

It something

like 'ldd'

improved.

Here we see,

the output of 'ldd'

as you can see, this

thing depends on quite a lot of

things.

Ranging from freetype

to libX11

to

otherthings.

Yes?

One moment.

[Question] Wouldn't you have to go through 'configure-ac',

to figure all this stuff?

[Gergely] We can,

and we will just not

right now.

We need

to figure out the

dependices of the binary,

if you go through,

'configureare-ac' you can figure

out the build dependices.

Which will be useful to,

and I'll get to that

in about 5 minutes.

To figure out the,

runtime dependicies you can,

use this 'sh.libs'

depends line.

It will be figured out

automatically most of the time.

To figure out,

the build dependicies

you have to do that manually.

And as I said we will get

to that in a moment.

I'm not going to build the package,

again because that takes a long time.

I will show you

another thing, a very

useful tool callled 'sbuild'.

Which works

by setting up a clean

environment to build your package in.

The last time I ran

'dpkg-buildpackage'

it also built a source package

I believe or if not?

Yes, it did.

'sbuild' works after

setting up which is easy and

and well documented in it's

man page and other

kinds of documentation.

It sets up a clean,

environment with only

'build-essential' installed.

You can give this tool,

debian source

control file. It will grab

all the build dependicies, install

it in the

'chroot'

amd try to build the package. It's very good

at figuring out your build dependicies

are and weather

your package will build on

the autobuilders or

if it will not.

This takes quite a long

time because it will have to

install the things.

Except it won't because

we don't have any build dependicies

right now.

Oh, we do

because

never mind.

I'll just stop this.

I forgot that

I built the source

earlier.

Maybe this time?

One momement,

I'll fix this in a moment.

Ok, lets see.

We should have

no build dependicies now.

Whats the problem

now?

Ok, while I

fix my environment,

lets skip

the next step.

If I had compiled this thing without

out build dependicies, it would fail

in a clean environment.

What we will do next is,

go through 'configure-ac' and figure out

what this thing needs.

Well, lets see.

It needs,

'pkg-config',

because that is what provides

this macro.

It will need,

I'm not sure how to pronounce this.

errr, lib-ca-ca somthing.....

It's like aalib

but with colours. It also

needs,

'imlib2'.

Lets add those

to the build dependency.

And, it also needs,

debhelper because we are using that.

Since, we are using compat level 9

we need 'debhelper'

version 9 or later.

We need,

this

and this should

do.

Rebuild the source package again and

I forgot to tell it, to not sign the package.

There we go.

Lets see how it worked.

Right,

now it figured out

that it needs

quite a few pacakges.

It can also download

the build dependicies from the internet.

I have them downloaded already,

which is why it didn't do that.

Then it,

installs all of them and

will try to compile the package

and make a binary out of it.

If it all succeeds, it will

copy the result

into the same directory,

which we ran the tool from.

If it doesn't work,

then it will

leave

the logfile of the whole

build in the same directory

which we ran it from.

Sadly, it takes a little while

on my laptop because

it is kind of old and

slow, but we are all ready at the

setting up stage.

Meanwhile,

do you have any

questions, so far?

[Question] The setup we see running

thats setting it up in a sandbox

or something. [Gergely] Yes.

'sbuild',

is a tool

with which you can setup a

'chroot'.

A clean one,

and it will

do all the magic in there.

It's also,

useful if you are running,

say, 'stable' or

'testing' but want to build for a

stable, because

everything that goes into debian

must be built on 'unstable'.

[Question] Do you have any recommendations

regarding the build environment? For, example using

'pbuilder' over 'cowbuilder',

or is it just a personal preference?

[Gergely] It's just personal

preference, I have used 'sbuild'

ever since it was

packaged. So, thats what

I'm familiar with,

but other

other people are using 'pbuilder' or

'cowbuilder' and

I think there are maybe one or two

more similiar tools.

They all work, they all

have different properties.

You can choose which ever

you like.

It builds and now it's

removing all the packages

it installed before.

We have a successful build but,

'lintian' failed.

Well, thats

kind of expected because there is a lot of

things missing from this package.

oh, I'm stupid

nevermind.

So,

we have a debian package.

Right here, which we built.

Lets try installing it.

Ooooh,

there are things in it.

and if I

run it.

It works! It

looks a bit silly [clapping] but it works

[clapping]

I just rememebered that I had this terminal

window open, because I wanted to run it

here, because it has

smaller fonts and it

looks much nicer this way.

That was what I had,

prepared.

Oh, wait.

So, if we run

'lintian' on this thing, it

tells us a few

errors

and warnings.

Like, this binary

doesn't have a man page,

it's in section games but

contains no games because games

need to go into

'/usr/games' not in

'/usr/bin'.

We can actually fix that easily.

That will show,

how to.

We have

a package which should

be installed

into

'/usr/games' instead of

'/usr/bin'.

Since we built only one

binary,

we can tell configure

that

binaries, should go into '/usr/games'

not '/usr/bin'.

With debhelper,

using this short form we

can do that by overriding,

the configure step.

That works

by adding a new target

to the

'debian/rules' makefile.

We simply call

the original

tool and

pass it

'--prefix=/usr/games' argument.

This will do all the magic

'dh_auto_configure'

does.

Which includes, adding

hardening flags,

adding

all kinds of other

options.

We will also tell it,

that we want to change

the prefix.

Another, thing

about the 'dch'

tool is,

it can not only create

changelog files it can

add new

entries into it.

By using the

'-i'

option.

Now we can build

the package again.

I stopped the build,

Anyway, if it wasn't

scrolling this fast, you could see the

argument we

added

was actually passed.

Anyway, if we

look at the,

Ok, not going to do that

in midnight commander.

There is another command called,

'dpkg-deb'

With which you can inspect,

extract and

reassemble

debian packages.

Right now we will use the

'-c' option

which means contents.

To list what

is inside it.

Well,

this worked a little

except we didn't want to

pass prefix because

everything under prefix

'/usr/games/' including

the documentation! We only want

to place the binaries

there.

So, it's not '--prefix=/usr/games'

but

'--bindir=/usr/games'.

I think,

yup.

Ok, here we go

again.

This time instead of

looking at the final

package, we will have a look at

the debian directory.

As you can see there are a couple of new

files there, like

'debian/files' or

the

'cake-is-a-lie.debhelper.log'

which is a log of

the 'debhelper' commands

which ran,

or the 'cake-is-a-lie.substvars' file.

There is also a directory

called 'cake-is-a-lie'.

In this

directory are

all the

things

installed which our package will

include.

If we go through that,

will we see that under '/usr/games'

is our program,

there is documentation,

and there are the pictures also,

and there is a DEBIAN

directory.

In all captial letters,

that directory is

special

it contains all the

metadata

which also goes into the

final debian

package. This is all

created by 'debhelper'.

If you want to quickly look at

what goes into your package

you can look

under the debian directory,

and find the

directory with the same name as

your binary package and see what is

there. What is in there will go

in your package.

If we run 'lintian'

on this again.

Well, it complains about a lot less

things.

Our program is finally at the correct

location.

So, what I would advise,

if you start

to package something from scratch

start with a very simple rules

file that pretty much

leaves everything to

'debhelper' and

after each interation

run 'lintian' and try to

correct the problems.

If you need

to change something,

which 'debhelper' can't figure out,

or something it

figures out but wrongly.

Just, add an override,

you don't have to know

all the magic that

is behind 'debhelper'

just a few tiny pieces.

Which you need,

for your package.

That's pretty much my recommendation.

I have found that,

it works fairly well.

Most of my packages are

built this way.

Any questions ?

Here in the front.

[Question] Where does

'make' come into play?

[Gergely] I am not quite sure why 'make' was chosen,

originally for Debian rules.

Some 10 years

ago you only needed a file,

it didn't have

to be a makefile although it was

recommended

but then the policy was changed

to require it being a

make file and I'm not quite sure why.

It's a

mostly likely

habit,

or historical reasons.

Does that answer your question?

[Questioner] Maybe my

question was wrong,

What is the difference between 'dpkg-buildpackage'

and make ?

[Gergely] Oh. Well the main

difference is

that 'dpkg-buildpackage'

does a few other things.

Like,

generating a changes file for you,

it can also build

source package

and

it can do a couple

of other things but the main

is generating

the changes file,

the changes file lists,

well we can just

have a look at it.

No.

The changes

file is what you,

actually upload

to the Debian servers,

along with the rest.

It lists all the

files that belong to this

version of the package

including the source,

if there is any.

This is something that 'make'

can't do, this is

all

done by,

'dpkg-buildpackage' or

actually a tool that it calls

for you.

We can

do something like Debian rules

binary,

if I where in the right directory,

that is.

Which we will do

nothing right now because it's already built,

but I can,

clean and I can run binary

and it will do all the things that

it did before.

'dkpg-buildpackage'

does call this command aswell,

the difference is that,

now I have a deb file,

and that is it.

I don't have a source

tarball, I don't have

the debian tarball,

I don't have the debian

source control file, I don't have the

changes file and I don't

have any of the signed either.

'dpkg-buildpackage'

does that

for you.

Any other questions ?

Another here.

[Question] I have never

packaged anything but I have

some code that has,

the Debian rules for ubuntu.

Is there any difference ?

[Gergely] Not much.

[Questioner] Could it be a start to learn to package ?

[Gergely] Yes. [Questioner] And if it's

a python package ?

[Gergely] Well, that's a little bit different.

You will most

likely

need to adjust the

dependicies by hand, I'm not

quite sure

but you can't figure out python

dependicies automatically.

But otherwise it's very similiar.

I believe.

But, if you have a debian

rules files from ubuntu

that is a good start.

[Questioner] Okay, thanks.

[Gergely] If there are no more questions then,

I think this is it.

I can be

[e-mail]

or on IRC,

if you have any packaging

questions feel free to

turn to me and

I will be happy to answer,

if I can.

If not I'll try to find

appropriate documention.

[Applause]