-
Not Synced
[Talkmeister]: Welcome, our next talk will
be about the Debian Long Term support
-
Not Synced
team and the speaker is
Raphaël Hertzog.
-
Not Synced
[Raphaël Hertzog]: Hello.
-
Not Synced
Today I will speak a bit about Debian
long term support.
-
Not Synced
I guess most of you already know about
it.
-
Not Synced
Are there some people who have no
idea what this is about?
-
Not Synced
No, good.
-
Not Synced
I will make my talk in 3 parts.
-
Not Synced
First I will present the team, how it
works
-
Not Synced
I will give some facts about how it
evolved over the first years.
-
Not Synced
I took some time to collect statistics
and believe they are rather interesting
-
Not Synced
I will also speak about the future
-
Not Synced
but there will be a separate discussion
about this in a BoF later this week.
-
Not Synced
I will show you how to help because, like
any other team in Debian it is open
-
Not Synced
to anyone. We welcome help.
-
Not Synced
At the end I will answer your questions.
-
Not Synced
What is LTS about?
-
Not Synced
The idea is really simple.
-
Not Synced
We wanted to extend the support period
of all Debian releases.
-
Not Synced
Currently it is basically for 1 year after
the next stable release comes out.
-
Not Synced
We wanted to extend this to 5 years to
match, at least, Ubuntu's offering.
-
Not Synced
which is not our competitor, but for the
companies that are making choices
-
Not Synced
it is one of the important criteria.
So we wanted to do as well.
-
Not Synced
Since we publish new stable releases
every 2 years it is roughly 3 years.
-
Not Synced
A nice side benefit is that the user can
skip a full release.
-
Not Synced
We don't officially support dist-upgrade
over going from Debian 6 to 8
-
Not Synced
but you can do 2 dist-upgrades at
the same time, limiting the downtime
-
Not Synced
to once every 5 years.
-
Not Synced
By the way, in practice, in simple server
configurations, dist-upgrades tend to
-
Not Synced
work rather well even across 2 releases.
-
Not Synced
Keeping a distribution secure for 5 years
is a real challenge.
-
Not Synced
It is hard work that not everybody is
willing to do.
-
Not Synced
In Debian all the work is done by
volunteers who do the work they enjoy.
-
Not Synced
Generally we enjoy working on new
features on top of latest releases
-
Not Synced
and not really on backporting patches to
crud that was written 5 years ago.
-
Not Synced
The security team has limited resources
so we could not just ask the security
-
Not Synced
team to now do twice the work.
-
Not Synced
But they were still really interested in
the project and wanted to support the idea
-
Not Synced
and really helped to get it bootstrapped.
-
Not Synced
The security team has a slightly larger
scope.
-
Not Synced
They support all architectures, which
means you have lots of problems of
-
Not Synced
coordination when security updates do
not compile and stuff like that.
-
Not Synced
What did we do?
-
Not Synced
We restricted the scope by picking
the 2 most popular architectures
-
Not Synced
that most users care about.
-
Not Synced
We have had some demand for ARM
architectures but up to now we only
-
Not Synced
support amd64 and i386.
-
Not Synced
We also excluded some packages from
security support.
-
Not Synced
Either because they are taking too much
time, like a security issue every 2 weeks
-
Not Synced
or that upstream is not cooperative
enough to be able to support it.
-
Not Synced
This list was basically made by the
current security team based on their
-
Not Synced
experience of doing security support.
-
Not Synced
If you look at the list there are some
important restrictions.
-
Not Synced
There's no xen, no kvm, no rails,
no browser. It sucks a bit.
-
Not Synced
But it's a way to get it started.
-
Not Synced
I think we can do better for wheezy.
-
Not Synced
Basically there is no virtualization
support on the host, only on the guest.
-
Not Synced
The security team helped to bootstrap
the LTS team but it is not the same team.
-
Not Synced
Obviously there are members of the
security team who also work on the LTS
-
Not Synced
team. They work in close collaboration.
-
Not Synced
We have regular contact and they watch our
mailing lists etc.
-
Not Synced
But the policies are different and the
infrastructure is separate,
-
Not Synced
which is a problem but I will talk about
that later.
-
Not Synced
We have a dedicated mailing list
-
Not Synced
and a dedicated IRC channel as well.
-
Not Synced
You are welcome to subscribe and to
join.
-
Not Synced
It's a new team which means new people
and new members.
-
Not Synced
Where do they come from?
-
Not Synced
The first idea was to get help from
people in various companies
-
Not Synced
who are already doing such in-house
support.
-
Not Synced
We had contact with EDF, and still have,
but they were one of the first
-
Not Synced
companies who were pushing for this
because they basically said
-
Not Synced
we are doing this already and we can
share with other companies.
-
Not Synced
The idea was to pool security support of
multiple companies.
-
Not Synced
We sent a press release asking
companies to join.
-
Not Synced
We had a few responses.
-
Not Synced
But I'll come back later to how it evolved
It's not really satisfying.
-
Not Synced
The other thing that we did is that we
offered companies the option to
-
Not Synced
fund the project to bring money and use
this to pay the work of
-
Not Synced
actual Debian contributors to do the
security updates that we need.
-
Not Synced
We have wiki pages listing all the ways
that companies can help with money.
-
Not Synced
In practice, most of the (wanting to be)
paid contributors joined together
-
Not Synced
under a single offer managed by
Freexian SARL which is my own company.
-
Not Synced
I'll quickly explain how this works.
-
Not Synced
Most companies don't want to bother
bringing human resources ??? (08:25)
-
Not Synced
They buy long term support contracts
from Freexian.
-
Not Synced
We have a rate. When you give €85 you
fund 1 hour of LTS work.
-
Not Synced
This is the current list of sponsors.
-
Not Synced
Top level gold sponsors sponsoring
more than 1 day of work per month.
-
Not Synced
On the other side we have Debian
contributors that are doing the work
-
Not Synced
and Freexian is paying them. There is a
small difference between the rate
-
Not Synced
to cover administration costs because I
have to handle the invoices
-
Not Synced
and some customers are using Paypal
which is taking a cut.
-
Not Synced
We ask contributors to follow some rules.
-
Not Synced
There is a requirement to publish a
monthly report on work done
-
Not Synced
on paid time. So they won't get paid until
they have published a report.
-
Not Synced
So everybody can know how the money
has been spent.
-
Not Synced
Currently we have 7 Debian contributors
and about 30 sponsors.
-
Not Synced
Some figures.
-
Not Synced
Who uploaded packages?
How has it evolved since June last year?
-
Not Synced
How is the funding evolving?
-
Not Synced
I just updated those figures a few
days ago.
-
Not Synced
I used this talk before at the mini
DebConf in Lyon in March,
-
Not Synced
but I updated it again.
-
Not Synced
The number of uploads is roughly over
one year since we started last year.
-
Not Synced
Over 300 uploads so it is not so much
but it is almost 1 per day.
-
Not Synced
So it is significant work.
-
Not Synced
I have given a state here of who paid
for the work and who did it on the left
-
Not Synced
The sponsors of Freexian are paying for
most of the uploads. ???
-
Not Synced
None is a separate category grouping all
Debian maintainers.
-
Not Synced
There are maintainers who are taking
care of their own packages in LTS.
-
Not Synced
Security team is members of the security
team who also work within the LTS team.
-
Not Synced
EDF is Électricité de France
-
Not Synced
Individuals are Debian developers that
have listed themselves as members of
-
Not Synced
the LTS team and did uploads for packages
of other maintainers not their own.
-
Not Synced
Credativ is a German company that you
probably know.
-
Not Synced
They have a booth here if you want a
job.
-
Not Synced
Toshiba, Univention, Catalyst etc
are other lower ???
-
Not Synced
On the right are people. The top 5 people
are paid by Freexian.
-
Not Synced
Raphaël Geissert is working for EDF.
-
Not Synced
Thijs is a member of the security team.
-
Not Synced
Kurt is openssl maintainer ???
-
Not Synced
Mike Gabriel is also paid by Freexian.
-
Not Synced
Christoph Bieldl is mainly maintaining
the debian-security-support in Squeeze LTS
-
Not Synced
Nguyen Cong is employed by Toshiba.
-
Not Synced
Christoph Berg is employed by creditv
doing postgresql maintainence.
-
Not Synced
How did it evolve over the year?
-
Not Synced
Again it is by affiliation.
-
Not Synced
The big blue part is paid contributors
-
Not Synced
You don't see it very well but the part
about maintainers is this one [points]
-
Not Synced
It tends to do better over the months
because here we started to
-
Not Synced
contact maintainers every time that we
have a new upload coming up
-
Not Synced
and ask them first 'do you want to handle
it yourself' so it slightly increased.
-
Not Synced
but the contribution of other companies
has not really increased over time.
-
Not Synced
Rather it has disappeared. It is
unfortunate but it looks like paid
-
Not Synced
contributors are more productive than
others.
-
Not Synced
In particular with EDF, they do the work,
but with some lag and we are faster
-
Not Synced
so they just reuse what we have done. I
want to talk to Raphaël to see
-
Not Synced
how we can do better towards this.
-
Not Synced
How did the sponsorship level evolve?
-
Not Synced
We have a steady increase, which is
rather nice. It is not a huge amount but
-
Not Synced
it is significant because we fund almost
80 hours per month.
-
Not Synced
It is close to our first goal. We wanted
that amount to be able to sustain
-
Not Synced
ourselves.
-
Not Synced
If you look at the sponsors, we have a
few big ones, possibly one very big
-
Not Synced
We can't give the name officially yet so
I won't.It will be a big jump in the graph
-
Not Synced
A few gold and many small sponsors.
-
Not Synced
I don't want to be dependent too much on
one big sponsor. I really prefer many
-
Not Synced
sponsors who are doing small donations
but donations which are sustainable
-
Not Synced
year after year because we are not here
for 1 year or 2.
-
Not Synced
We want to do it over the long term.
-
Not Synced
We have some figures about how many
hours have been funded since the start
-
Not Synced
Feel free to interrupt me if you have any
questions. I can take them at any time
-
Not Synced
That's it for evolution. Now, the future.
-
Not Synced
What do we expect for the future?
-
Not Synced
First keep doing what we have been up
to
-
Not Synced
Keep supporting the current set of packages.
-
Not Synced
But for wheezy long term support we
would really like to have more
-
Not Synced
supported packages. A browser would
be nice for desktop deployment.
-
Not Synced
Virtualization support is also important
for many companies so we should be
-
Not Synced
able to support something here.
-
Not Synced
Also we want to avoid some pitfalls that
we had with squeeze LTS.
-
Not Synced
As you know LTS users are currently
required to add a separate source
-
Not Synced
list entry with squeeze-lts repository. The
security.debian.org squeeze
-
Not Synced
repository is unused. It should be
possible for the LTS team to
-
Not Synced
continue using the same repository as
the security team once it no longer
-
Not Synced
use it. This will be the topic of a BoF
next week on Tue at 1800.
-
Not Synced
What's the problem with supporting the
current set of packages?
-
Not Synced
For example, we have MySQL right now in
Squeeze LTS in version 5.1
-
Not Synced
which is no longer supported by Oracle.
-
Not Synced
We don't even know if it's affected by
security issues because
-
Not Synced
Oracle doesn't give info on
??? release.
-
Not Synced
This is a problem, we should consider
switching to a newer version,
-
Not Synced
but newer versions involve library
transition, which is not really nice
-
Not Synced
in Long Term Support releases.
-
Not Synced
There is some work to do here if we want
to do something serious.
-
Not Synced
And we have other problems, other
packages which are similar.
-
Not Synced
From time to time, we backport, we take
newer upstream versions.
-
Not Synced
We did that for wireshark for example.
-
Not Synced
This is what I said before, the limited
scope sucks, we want to be able to
-
Not Synced
support more packages and we need more
support for this.
-
Not Synced
[Q] Speaking of wireshark, we switched to
Wheezy's version, so it's solved.
-
Not Synced
[Raphael] Yes, exactly, that's what I just
said.
-
Not Synced
It used to be a problem.
-
Not Synced
That's a part I did no update since March.
-
Not Synced
Additionally, the problem with a separate
repository is that
-
Not Synced
there is a propagation delay to the
mirrors
-
Not Synced
that we don't have with
security.debian.org.
-
Not Synced
I will speak a bit of how the team works.
-
Not Synced
Basically, the first step is triaging new
security issues.
-
Not Synced
We have a list of CVEs that comes in and
there are added to a text file
-
Not Synced
data/CVE/list.
-
Not Synced
Some dispatches those on source packages
and then someone else reviews
-
Not Synced
status in each release.
-
Not Synced
Then the LTS team reviews the status in
Squeeze and members of security team
-
Not Synced
review status in Wheezy and Jessie.
-
Not Synced
And then, we decide what we must do with
the package.
-
Not Synced
Depending on the analysis, either we say
-
Not Synced
"We need to prepare an update", so we add
it to data/dla-needed.txt
-
Not Synced
and someone will have to take care of
preparing the update.
-
Not Synced
Or we say "The issue does not apply, does
not affect the current version"
-
Not Synced
or we ignore it because the package is
unsupported or because
-
Not Synced
the issue is really minor, not severe
enough.
-
Not Synced
Sometimes, it can be that the issue is
already fixed in Debian
-
Not Synced
due to some maintainer choices.
-
Not Synced
When we do this classification, we contact
the maintainer to keep him in the loop
-
Not Synced
and offer him the possibility to help us.
-
Not Synced
Here's what such a text file looks like.
-
Not Synced
The bold line is the line that we are
adding when we have decided
-
Not Synced
what we're doing with the packages.
-
Not Synced
This is all in the Subversion repository
on Alioth.
-
Not Synced
Then, someone has to prepare the security
update.
-
Not Synced
Basically, looking for a patch, it's often
useful to be able to look up at
-
Not Synced
RedHat or Ubuntu or upstream.
-
Not Synced
Best case is upstream because there are
nice upstream who are providing
-
Not Synced
patches also for older versions.
-
Not Synced
Usually there are already patches
available,
-
Not Synced
not always for the good version.
-
Not Synced
Sometimes we have to backport it.
-
Not Synced
Then we prepare an upload with this
+deb6uX suffix that we're using now
-
Not Synced
for security updates, stable updates.
-
Not Synced
This is a rather known territory for
package maintainers.
-
Not Synced
This is the hardest part sometimes,
-
Not Synced
testing the update, making sure the issue
is fixed.
-
Not Synced
When tools have test suites, it's nice,
-
Not Synced
but sometimes we have to set it up
ourselves.
-
Not Synced
Sometimes it is too hard, so we tend to,
out of safety measure,
-
Not Synced
to mail the mailing list and say
-
Not Synced
"Ok, I've done my best, but please double
check"
-
Not Synced
It doesn't happen often that we have
testers
-
Not Synced
but some lts users are willing to test it
before ???
-
Not Synced
It would be really nice if we had
more of those.
-
Not Synced
And if we don't get any negative feedback,
-
Not Synced
then we upload.