meetings-archive.debian.net/.../Removing_obsolete_packages_for_fun_and_profit.webm

Hi, everybody, I'm Eric Doland.

I'm the automake maintainer,
among other things.

Today I'm going to talk about removing obsolete packages.

This is meant to be a BOF.

I only have sixteen slides,

so I'm going to talk about my experiences

in removing old versions of automake.

Please feel free to interrupt with questions

or criticisms or comments as we go.

Hopefully with a bit of discussion about how to do this better in the future,

or maybe not better, we'll see.

So just to set the context a little bit,

automake in Debian,

I started maintaining it a really long time ago.

It's making me feel old. 2002.

This was around the time that automake 1.6 came out, I think.

One of the problems automake has is

that new releases of it usually have incompatibilities

with backwards-compatible completely.

There's various reasons why this is,

it provides a very weird interface,

because you're basically embedding things inside make files,

and it's hard to control the API of a make file.

Back in the day, there was, for a very long time,

automake 1.4, and that just sort of worked,

and then, automake 1.5 was introduced,

Debian upgraded to automake 1.5, and lots of packages broke.

So what ended up happening is that automake 1.5 got moved into its own package,

and automake 1.4 was automake package.

Nowadays, the older versions get moved into their own packages,

and generally, the currently released version--the most up-to-date version--

is the automake package.

This has some downsides, because it's not necessarily fully backwards-compatible,

but this is usually what people want.

They want the latest version of automake, and they don't want to have to install

a new package any time a new version of automake comes out.

All the automake packages provide an alternative

so that you can have your /usr/bin/automake

be exactly which automake you want it to be.

The priority of that alternative usually means that the latest version of automake

is the one automatically selected, but of course, you can override that.

There's a note here that, depending on automake directly, is a little risky,

because fancy automake files, new versions of automake, might break you, potentially.

Anyway, this is not the automake BOF, this is just a little context

about why automake is what it is in Debian.

So there's problems with this, because these backwards-compatible issues, as I said,

were packaging all the versions of automake.

Each new version was getting a new package, basically,

and we've gone through a lot of automake packages over the years.

I didn't count them up, but it looks like almost close to ten, I guess,

which is a lot of packages you have to deal with.

People will depend on these individual packages, for whatever reason,

and then getting them out of the system, when they become old, is troublesome.

As of the wheezy release, we had four different automakes in the wheezy release.

So we had what the old, old--as you can see, 1.4, 1.9, 1.10, and 1.11--

and they say that at one point we're going to have five different versions of automake,

which is a lot of versions of automake.

You can also see that the 1.4 version was released in 2002,

when I started maintaining this stuff, so it's really old.

No one should be using it, and no one should have been using it for the last ten years,

but we sort of kept it around because people were like

"oh, maybe there's old software that, you know, still depends on automake 1.4."

Um, yeah.

This is crazy.
There are too many versions of automake.

No one really wants them. So, I started out on this mission

to bring us down to, hopefully, one or two versions of automake for the next release.

I've been doing this over the last year, and I was makign these slides

and I was thinking about, it sort of had these weird parallels

with some things I'd read about, so now I present to you

the five stages of removing an obsolete package from Debian.

The first stage is denial.

[laughter]

"If I send mail to debian-devel asking everyone to very nicely

move off of these old versions of automake, they'll just do it, right?

I mean, doesn't everyone agree that that's what will happen?

Yeah, so that's what I started out by doing.

I'm going to talk a little bit about the tools and stuff that I used in this process,

so this is sort of technical, and sort of procedural.

If you've got questions about either side, just shoot up your hand.

The first thing you do, or the first thing I did, is that I used grep-dctrl to figure out

which packages were build dependent on automake.

Automake basically has no actual binary dependencies, it's all build dependencies.

There were one hundred and sixty-nine packages--source packages.

That's a lot, but in Debian scale, it's not that much. It should be fine, right?

Then, I used the dlist tool to turn that list of packages into a list of maintainers,

with their packages. It's a very nice tool for just ??

Then, I sent out an email debian-devel saying,

"Here's my plan: I want to get rid of these old versions of automake.

Here are the packages that are build-dependent on them.

Please, do your part and fix this.

I sent that mail out on May 27, 2013, so before last DebConf, which I was not at.

Stage Two: Anger.

People aren't fixing these things!

I'm going to have to actually do something other than send an email. All right, fine.

The next thing I did, to try to encourage people again to make this move,

Lintian has this really nice facility. One of tests is actually--there's a list of obsolete packages.

If you put your package in that list,

lintian will complain any time anyone depends on that package,

to try to force maintainers that are paying attention,

that they should't actually be depending on these packages.

That's good. It's cheap to do. Just sent the patch to lintian.

I don't see any reason they wouldn't take it if it made sense.

Now, I have to file bugs. People don't read debian-devel.

It's sort of, it's fair I guess. So I basically just went through the list of packages I had,

with a simple substitution script, so now it mailed a mail that was like

"Please stop depending on automake blank." with a very boiler-plate thing saying

"this is why we're getting rid of these old versions of automake"

and I sent that off to BTS. Which is really easy, right? Because BTS is just email.

So you can just do that. You can file a bunch of bugs that way.

I should note that my initial email was a proposal for this mass bug filing, as well,

which you're supposed to do as part of the procedure of sending out mass bug filings.

When I did this, I had to file one hundred and seventeen bugs.

There's fifty two packages that got fixed between me sending my initial email

and the bugs. So that's good, I shouldn't be that angry, maybe.

I did wait four months between these two events, though, so there's plenty of time.

The other thing I used was using BTS user tags, to track all of these bugs very easily easily

using this automake clean-up 2013 tag. I added the date because

I knew this would not be the last automake clean-up that I would have to do,

so I keep these separate.

Always good to date your work if you think you might have to repeat it.

Yeah, no, I did not finish in 2013. I started in 2013.

It should've been automake clean-up 2015 or something, if I wanted the finish date.

All right, so stage three. Bargaining.

Okay, so, I'm past the anger stage, and now I'm like "okay, I've filed all these bugs,

but not much is happening, so let me just fix this for you.

I'll give you the exact fix on how to much to the newer version of automake.

Here's the patch. Just apply it and upload your package.

I've done all the work; I've done all the hard stuff."

So, before I even did this, thirty four packages were fixed

without supplying a patch. I began the patching in late October of 2013

and--this is the hard part, this is the part that's tough to automate--in many cases,

just switching to the new version of automake worked, and then it was easy,

but that was maybe 50% of the cases.

I didn't keep good statistics about how tough this was exactly,

but in a lot of cases, I had to fiddle with the build system,

or a lot of build rules would run automake itself,

instead of using dh-autoreconf.

There was a lot of actual fiddling with packages, trying to get them to build,

and then just waiting to build these things, which can take a very long time,

to test that this worked. So this is a lot of actual, tough work.

What else? I used pbuilder to do the builds and then if I successfully built something,

with the new version of automake,
I would just mail off a patch

to the existing bug, in BTS, and flip the patch tag, saying "here's the patch."

So I thought, okay, at this point, you know, there's patches out there

for almost all of these problems.

There should be a wave of uploads and everything should be great.

Stage Four: Depression.

That's not what happened. I was pretty much on my own.

The responsible maintainers have already fixed this problem.

So we're into this, sort of, long tale of people who don't care,

or people who are too busy to deal with their packages or do anything here.

Now I have to upload NMUs. I started this in late January, I think