-
Not Synced
[Tollef]: Hello, I'm Tollef.
-
Not Synced
I'm part of the DSA team
-
Not Synced
With me today, we have zobel
-
Not Synced
who's also a DSA member
-
Not Synced
and we're here to talk a little bit about what DSA does
-
Not Synced
and obviously, if you have questions or anything, we'd be happy to try to
-
Not Synced
answer those
-
Not Synced
[zobel]: Try to keep that as some sort of round-table
-
Not Synced
because we want a discussion, this is not going to be a talk
-
Not Synced
so whenever you have questions, just ask
-
Not Synced
[Tollef]: The DSA currently consists of 7 people.
-
Not Synced
Most of us are in Europe,
-
Not Synced
but we also have Luca who is in Canada
-
Not Synced
apart from that, paravoid is on "holiday" and ??
-
Not Synced
and various other people.
-
Not Synced
[zobel]: The duties we have as Debian System Administrators
-
Not Synced
is basically to build and maintain the infrastructure you are all using
-
Not Synced
for running our distribution
-
Not Synced
It's the general sysadmin stuff;
-
Not Synced
we are doing installing security updates,
-
Not Synced
keeping machines up to date,
-
Not Synced
keeping the hardware running,
-
Not Synced
creating accounts for you,
-
Not Synced
running DNS and mail.
-
Not Synced
[Tollef]: One thing we actually don't do is...
-
Not Synced
we provide the base, we provide the OS, support for that
-
Not Synced
we don't run the services
-
Not Synced
so lists.debian.org, we're not the people you want to talk to
-
Not Synced
if there is some problem with spam handling
-
Not Synced
Then, you want to talk to this guy
-
Not Synced
who is also a part of the listmaster team
-
Not Synced
Similar, bugs.debian.org, the web pages
-
Not Synced
We make sure that apache is running
-
Not Synced
but if you find typos on the web page,
-
Not Synced
if there is a typo don't blame us.
-
Not Synced
[zobel]: I don't know how many machines we run in the meantime
-
Not Synced
I think it's around 150/160 machines in total, including the VMs.
-
Not Synced
[Tollef]: No, if you count VMs, it's more like 250
-
Not Synced
[zobel]: OK
-
Not Synced
We run those machines currently at about 30 locations worldwide
-
Not Synced
Also part of our duty is to deal with hosters and the local admins
-
Not Synced
If they have firewalls running in front of our machines
-
Not Synced
we try to convince them to disable the firewall parts for our machines
-
Not Synced
so we get to manage that stuff ourselves.
-
Not Synced
[Tollef]: This is often ??
-
Not Synced
we have some locations where the machines are ?? connected for instance
-
Not Synced
and this breaks secure NTP
-
Not Synced
There are various places where we have to make accommodations
-
Not Synced
because it's hard to get the hardware to be another place
-
Not Synced
maybe it's dev boards for an architecture which is being bootstrapped
-
Not Synced
in some cases we kind of have to endure a little bit of pain for that
-
Not Synced
but most hosters and most local admins are really nice people
-
Not Synced
really easy to deal with and very very accommodating
-
Not Synced
I mean, we don't pay for any of this.
-
Not Synced
It's all sponsored and given to us, free of charge.
-
Not Synced
We're quite lucky.
-
Not Synced
[zobel]: It differs from location to location
-
Not Synced
we currently have locations where we have a full rack
-
Not Synced
which we can populate with hardware,
-
Not Synced
there are other locations where we just have 1 or 2 machines
-
Not Synced
just sitting and doing the jobs for us.
-
Not Synced
Keep in mind all of us, 7 persons, are not paid to do sysadmin jobs
-
Not Synced
We are all doing that on our volunteer time
-
Not Synced
So if you speak up on IRC,
-
Not Synced
sometimes you will not get a reaction within 5 minutes
-
Not Synced
but I think that's mostly clear to all of you.
-
Not Synced
[Tollef]: Because we have to so many machines,
-
Not Synced
we like automation
-
Not Synced
We run puppet everywhere
-
Not Synced
It was chosen some time ago
-
Not Synced
and it generally does the right thing
-
Not Synced
and generally works okay.
-
Not Synced
This often makes for some interesting problems when bootstrapping
-
Not Synced
because apparently ruby is really awesome to bootstrap.
-
Not Synced
Right, Steve? [laughs]
-
Not Synced
Especially on arm.
-
Not Synced
We also like git
-
Not Synced
we have the entire puppet repository in git
-
Not Synced
our domains are in git
-
Not Synced
Our wiki is in git
-
Not Synced
[zobel]: Everything.
-
Not Synced
[Tollef]: Yeah, basically everything can be put into git
-
Not Synced
You probably don't to do it to a database
-
Not Synced
but anything else, put it into git
-
Not Synced
[zobel]: We have some sort of account management tool
-
Not Synced
which we are currently rewriting called userd-LDAP or ud-ldap
-
Not Synced
Luca has done quite a lot of work on the rewrite
-
Not Synced
I think it's already handling the generating stuff
-
Not Synced
just rolled out to the debian.org machines
-
Not Synced
all the other parts of ud-ldap are still using the old codebase
-
Not Synced
which is ugly to read,
-
Not Synced
it reads like perl bash, written in python.
-
Not Synced
If you have spare time and knowledge in python,
-
Not Synced
help us to finish the rewrite jump
-
Not Synced
[Tollef]: The new ud is a django project
-
Not Synced
So it's fairly nice and well written
-
Not Synced
What ud-ldap actually does is...
-
Not Synced
it has a local ldap server which runs on the machine called ??
-
Not Synced
which is db.debian.org
-
Not Synced
and on there it generates static files which are synced out to all machines.
-
Not Synced
So even though we're using LDAP for account information,
-
Not Synced
we don't have a single point of failure.
-
Not Synced
So if that machine goes down,
-
Not Synced
it means you can't update your password or your SSH keys
-
Not Synced
but you can still login, at various places.
-
Not Synced
[zobel]: It also works around network issues between machines
-
Not Synced
If SSH between ?? and the porting machine or whatever,
-
Not Synced
you can login to machines.
-
Not Synced
We monitor our machines using Munin and nowadays Icinga.
-
Not Synced
We had some performance issues with Munin,
-
Not Synced
with the wheezy version, I think it was
-
Not Synced
and I think there were other stuff,
-
Not Synced
Munin works quite well for us
-
Not Synced
In general if there are web pages,
-
Not Synced
like Icinga or Munin asking for a password
-
Not Synced
then this is just dsa-guest and either no password or just a random password.
-
Not Synced
This is just to protect our services against script kiddies
-
Not Synced
or whoever to wants to see what his script is doing
-
Not Synced
in effect to the Debian services, not seeing the results directly
-
Not Synced
but everyone who knows how the Debian system works you get access to there.
-
Not Synced
[Tollef]: It's also so that we don't accidentally end up with spiders walking around
-
Not Synced
because Munin web interface is generating the graphs on the fly
-
Not Synced
and it's using rrdtool and that can consume great amounts of CPU power
-
Not Synced
and web spiders are really good at wasting CPU power, for us
-
Not Synced
so we want to keep them off those pages.
-
Not Synced
[zobel]: To track our issues we currently have with hardware failures,
-
Not Synced
with accounts we need to create and so on,
-
Not Synced
we use request tracker on rt.debian.org
-
Not Synced
which some other teams use as well
-
Not Synced
You can even mail it, or use the web interface.
-
Not Synced
Debian developers, I think only, for viewing the web interface?
-
Not Synced
[Tollef]: For most people it's read-only
-
Not Synced
You can interface with request through mail of course.
-
Not Synced
If you need to send something there, send it to rt@rt.debian.org
-
Not Synced
and make sure to including debian rt in the subject
-
Not Synced
else we'll just throw it away because then it's spam
-
Not Synced
It's a really efficient spam filter
-
Not Synced
Slightly annoying for when you submit the first ticket.
-
Not Synced
[zobel]: The last talk we gave about the DSA team
-
Not Synced
was, I think, 2 years ago.
-
Not Synced
We tried to summarise what we've done in the last 2 years
-
Not Synced
When was that meeting in Oslo, 3 years ago?
-
Not Synced
[Tollef]: 3 years ago, yes.
-
Not Synced
[zobel]: 3 years ago we decided that we want,
-
Not Synced
at least the infrastructure hardware
-
Not Synced
not the porting hardware
-
Not Synced
on machines that are under warranty
-
Not Synced
so we can open a ticket at HP, IBM or whatever
-
Not Synced
and ask them to send replacement parts when hardware breaks.
-
Not Synced
We use server-grade hardware,
-
Not Synced
currently most of the machines are HP machines:
-
Not Synced
80 DL360 DL580
-
Not Synced
[Tollef]: They work quite well,
-
Not Synced
I think we're mostly done with that transition.
-
Not Synced
It turns out having actual servers,
-
Not Synced
rather than something someone put under a desk and forgot about
-
Not Synced
actually makes for less pain and more uptime
-
Not Synced
[zobel]: We try to consolidate the amount of data centres
-
Not Synced
we are having core services running in.
-
Not Synced
So currently we have like 3 to 5 data centres,
-
Not Synced
we have quite a lot of services running in.
-
Not Synced
that's: manda; bytemark; grnet, still a little bit;
-
Not Synced
also OSUOSL, UBC.
-
Not Synced
[Tollef]: UBC-ECE
-
Not Synced
Yes. We also have some other places with fewer machines,
-
Not Synced
but since it's often painful to have a single machine
-
Not Synced
in a location, we try to avoid that.
-
Not Synced
It's kind of a tradeoff,
-
Not Synced
you want to have enough locations that you resilience
-
Not Synced
but you don't want to have so many that you basically have 2 machines everywhere,
-
Not Synced
and each time there is a problem
-
Not Synced
you have to deal with somebody you haven't spoken to in 2 years
-
Not Synced
because that was when the last problem occurred.
-
Not Synced
[zobel]: For the core services we are currently using ganeti for virtualization
-
Not Synced
which is some sort of KVM-based virtualization framework.
-
Not Synced
[Tollef]: It's a cluster manager which came out of Google,
-
Not Synced
and which works really well.
-
Not Synced
Its target is clusters from 1 to 50 machines,
-
Not Synced
free software of course.
-
Not Synced
[zobel]: It works very well for us.
-
Not Synced
The target where I try to work on in the last few months is
-
Not Synced
the single sign on framework web applications.
-
Not Synced
Thankfully together with Enrico, who helped quite a lot with that.
-
Not Synced
Rewritten the ugly perl code I wrote to a python django framework
-
Not Synced
We hope to be able to provide single sign on,
-
Not Synced
also for non debian.org web services.
-
Not Synced
Which with the current software we use for debian.org,
-
Not Synced
didn't work out for security reasons.
-
Not Synced
So let's see where we stand in 2 years with SSO for web stuff.
-
Not Synced
[Tollef]: We had a problem earlier this year,
-
Not Synced
that the backup server we had would die and then die and then die,
-
Not Synced
with various problems: It claimed to have hard drive errors,
-
Not Synced
but it looked more like controller errors and so on.
-
Not Synced
Obviously, running without backups isn't a terribly good idea.
-
Not Synced
We bootstrapped another backup server but
-
Not Synced
it was running at the bytemark data centre
-
Not Synced
and because we have many other services hosted there,
-
Not Synced
that's not a very good situation
-
Not Synced
just because if something happens at that data centre
-
Not Synced
and it burns down, suddenly we've lost both the backups
-
Not Synced
and the services being backed up.
-
Not Synced
So a month ago, we got a new machine
-
Not Synced
it's hosted at DGI?
-
Not Synced
[zobel]: DGI, in Dusseldorf.
-
Not Synced
[Tollef]: and it's happily chugging along making backups.
-
Not Synced
We're currently using Bacula for backups,
-
Not Synced
and it's working okay,
-
Not Synced
we're having some interesting problems with scheduling of backups
-
Not Synced
so we're probably going to need to do something to fix things there.
-
Not Synced
[zobel]: Luca is doing the ud-ldap rewrite,
-
Not Synced
as already mentioned earlier, we need helping hands there.
-
Not Synced
I think Paul and Peter are working on the snapshot infrastructure,
-
Not Synced
giving especially the QA integration for snapshot.
-
Not Synced
[Tollef]: We had a donation from Leaseweb, earlier this year.
-
Not Synced
Similar to the backups, it turns out servers when they grow big enough
-
Not Synced
you get lots of disk dying...
-
Not Synced
Linux isn't terribly good at handling this when you get enough of your disks dying.
-
Not Synced
We had one machine that died,
-
Not Synced
with controller failure again!
-
Not Synced
We tried to revive it, it wasn't really successful.
-
Not Synced
So we ended up getting this donation from Leaseweb
-
Not Synced
and we then have a small cluster of machines in their data centre
-
Not Synced
[zobel]: snapshot is currently about 23 terabytes
-
Not Synced
[Tollef]: something like that.
-
Not Synced
[zobel]: 30 terabytes in size
-
Not Synced
Which is currently the biggest 'archive' we maintain.
-
Not Synced
We tried to roll out SSL everywhere in the past
-
Not Synced
[Tollef]: It's been something we wanted to do for a while,
-
Not Synced
to enable HTTPS and so on everywhere,
-
Not Synced
even on public and open resources.
-
Not Synced
It felt, with the - it wasn't really triggered by
-
Not Synced
but it was in the same way as the Snowden things.
-
Not Synced
It was like we should probably actually move forward with this,
-
Not Synced
because it turns out there are entirely too many people
-
Not Synced
who are TCP dumping too much.
-
Not Synced
[single applaud]
-
Not Synced
We're pushing for more SSL everywhere.
-
Not Synced
There was a little bit of controversy around this when we did it to people.debian.org
-
Not Synced
because it turns out that ??? had some problems with verifying the certificate and so on.
-
Not Synced
It's not a completely uncontroversial and smooth move but
-
Not Synced
sometimes you need to make a little bit of sacrifice
-
Not Synced
to actually get the security we want.
-
Not Synced
Related to that also, we pushed some bits towards using CDNs,
-
Not Synced
which also are interesting in the context of SSL,
-
Not Synced
because you have to give your cert to somebody else,
-
Not Synced
there is a tradeoff there.
-
Not Synced
You kind of have to trust your provider there.
-
Not Synced
[zobel]: What we are also doing due to the fact that
-
Not Synced
we got a huge donation from Bytemark,
-
Not Synced
I think one and a half years ago?
-
Not Synced
It was a full blade centre and 6 MSA shelves
-
Not Synced
[Tollef]: 3 chassis plus 3 ??
-
Not Synced
[zobel]: Currently we still have some spare CPU cycles left at Bytemark.
-
Not Synced
Currently setting up Openstack at the Bytemark datacentre for one or two
-
Not Synced
blades
-
Not Synced
In the end, the idea is that Debian Developers can start VMs there, themselves
-
Not Synced
Similar to the VMs we are using for our infrastructure
-
Not Synced
So we can more easily migrate debian.net services to debian.org services
-
Not Synced
giving you some sort of common infrastructure we use on debian
-
Not Synced
So you can help us to migrate services, or we can help you to migrate
-
Not Synced
from your hardware to the Debian infrastructure
-
Not Synced
[Tollef]: Part of the reason that is:
-
Not Synced
it turns out running various half-official services
-
Not Synced
on peoples' home machines and co-lo machines
-
Not Synced
isn't a terribly good idea.
-
Not Synced
Often they'll run for years and then somebody will get bored
-
Not Synced
or they'll quit debian or they'll go broke,
-
Not Synced
the machine will burn down... something will happen,
-
Not Synced
the services disappear and people get upset.
-
Not Synced
So we try to talk any services that are half-official
-
Not Synced
and we would rather move them onto debian.org hardware
-
Not Synced
So if you have a service which is kind of a half-official thing
-
Not Synced
and you want to make it more official,
-
Not Synced
and actually have somebody do the base OS maintenance for you
-
Not Synced
so you don't have to worry about that,
-
Not Synced
then please come and talk to us.
-
Not Synced
We're quite happy to provide you we reasonable VMs.
-
Not Synced
[zobel]: How to contact us.
-
Not Synced
There are several mailing lists,
-
Not Synced
there is a debian-admin@lists.debian.org list where we discussed that this
-
Not Synced
mailing list will more or less be open to every Debian Developer.
-
Not Synced
Debian devel people can subscribe to that mailing list aswell.
-
Not Synced
There's dsa@debian.org email address,
-
Not Synced
which we changed due to the fact there was
-
Not Synced
a debian-admin@debian.org and there was quite a lot of confusion
-
Not Synced
about the debian-admin@lists and the debian-admin@debian.org email address.
-
Not Synced
So we decided to move to a new email alias which is dsa@debian.org
-
Not Synced
You can hang around on IRC as mentioned earlier, in the #debian-admin channel
-
Not Synced
Feel free to join there if you have any issues,
-
Not Synced
just raise them and talk to us.
-
Not Synced
[Tollef]: Like any people in any teams in Debian,
-
Not Synced
we obviously have more things to do than we actually have time for.
-
Not Synced
So help is very much appreciated.
-
Not Synced
Getting help with sysadmin tasks is kind of an interesting challenge
-
Not Synced
because you can't just give out root to all debian.org machines
-
Not Synced
to somebody who shows up and goes:
-
Not Synced
"I would like to rewrite your authentication infrastructure"
-
Not Synced
However, since we keep the puppet repository and so on, in git
-
Not Synced
it's at least possible for people to get in and contribute.
-
Not Synced
Send us patches, show up, discuss things.
-
Not Synced
If you think something can be improved,
-
Not Synced
that's quite likely and we would be happy to discuss how to do that.
-
Not Synced
Documentation is always welcome,
-
Not Synced
there is a bit of documentation for things like debile.debian.org and so on.
-
Not Synced
But more is always welcome.
-
Not Synced
Also just hanging out on IRC, answering people's questions
-
Not Synced
is often surprisingly useful.
-
Not Synced
[zobel]: We also really want to grow the team,
-
Not Synced
from the seven person team we are currently
-
Not Synced
We had, a few months ago, spoken to a Debian Developer who
-
Not Synced
- is he here in the room? Might be! -
-
Not Synced
who said he currently does not want to become a member of the DSA team due to
-
Not Synced
the fact he has too many other things, other duties in Debian.
-
Not Synced
Just talk to us and help us, and at one point we'd probably get annoying with
-
Not Synced
doing too many tasks for you, so we just give out the root access then.
-
Not Synced
[Tollef]: It's how it usually works in Debian,
-
Not Synced
at some point you've contributed enough,
-
Not Synced
that's it's more annoying to merge your patches
-
Not Synced
and review them than to just give you access.
-
Not Synced
So that happens.
-
Not Synced
[zobel]: I think that's all about the slides,
-
Not Synced
so just ask...
-
Not Synced
[Tollef]: questions!
-
Not Synced
[question1]: I guess this is more DSA
-
Not Synced
the listmaster pieces, are they in puppet as well?
-
Not Synced
[zobel]: No, the list stuff is not in puppet.
-
Not Synced
The exim config we are using on debian.org machines is in puppet
-
Not Synced
But lists uses postfix.
-
Not Synced
Alex Wirt is also sitting here in lecture room,
-
Not Synced
he could easily your questions for lists.debian.org
-
Not Synced
More questions!
-
Not Synced
No more questions?
-
Not Synced
[laughter]
-
Not Synced
[question2]: As one of the local admins for a bunch of buildds,
-
Not Synced
I know that every now and again we get asked for stuff opening up, more ports
-
Not Synced
because we're one of those evil places with a firewall, even for the DMZ.
-
Not Synced
Do you actually have a central list of all of things that you want to be able
-
Not Synced
to get access to, you know?
-
Not Synced
That kind of thing would be awesome,
-
Not Synced
that I could just point, say the ARM network sysadmins at
-
Not Synced
instead of every now and again having to say:
-
Not Synced
"Oh and we need this extra thing"
-
Not Synced
and then backwards are forwards, because their immediate response is:
-
Not Synced
"Well, why?"
-
Not Synced
If we can give them a list and
-
Not Synced
just give them a notification that
-
Not Synced
there are few new things we'd like,
-
Not Synced
it might go easier.
-
Not Synced
[Tollef]: I don't think we have a list as such.
-
Not Synced
What we do have is our firewall config is ??
-
Not Synced
So we have a list of things we want to be able to accept on various servers,
-
Not Synced
even though we don't have a list as in "Go to this web page and here you have
-
Not Synced
these ports and their justification", we can generate that.
-
Not Synced
So yeah that's a good idea, we should do something like that.
-
Not Synced
[question3]: Can you explain more about your backup system?
-
Not Synced
I think you covered it very briefly.
-
Not Synced
[Tollef]: We have bacula, it's a centralised backup system using,
-
Not Synced
it's kind of mix of push and pull, in that you have a central director
-
Not Synced
which tells the machines that are to be backed up,
-
Not Synced
that you are now going to be backing your things up
-
Not Synced
to this storage daemon over here.
-
Not Synced
Then it also tells the storage daemon, that please expect a connection from
-
Not Synced
this machine.
-
Not Synced
We run the director, which is this central component,
-
Not Synced
that runs in adm in Bytemark.
-
Not Synced
The actual storage is at DGI
-
Not Synced
and obviously the various machines being backed up are everywhere.
-
Not Synced
One of the painful things about bacula is that it thinks,
-
Not Synced
even though we are backing up to hard drives,
-
Not Synced
it still thinks we are actually backing up to tape drives
-
Not Synced
and that makes for, the nicest thing about hard drives is that generally you
-
Not Synced
don't really have seek time in the same way you have seek time on tapes.
-
Not Synced
So you don't care about rebinding tapes and switching to a different tape,
-
Not Synced
that's called "opening another file" and it doesn't take very long.
-
Not Synced
We also have the problem that bacula doesn't have the concept of-
-
Not Synced
if you look at a backup system like backuppc,
-
Not Synced
it never does full backups, it will only do incremental backups
-
Not Synced
and then has a hardlink farm.
-
Not Synced
bacula will do a full backup, then incrementals
-
Not Synced
then a full backup then incrementals.
-
Not Synced
This makes less sense when you have hard drives
-
Not Synced
than when you have tapes.
-
Not Synced
Also the scheduler isn't very smart,
-
Not Synced
if it can't back up a machine for some reason
-
Not Synced
then instead of rescheduling that back up
-
Not Synced
it will, depending on how you configure it,
-
Not Synced
it will then just skip it.
-
Not Synced
Some of our hosts actually don't have that good connections,
-
Not Synced
so when you're trying to do a full backup,
-
Not Synced
which can take 24 hours,
-
Not Synced
you really don't want that TCP stream to be disconnected
-
Not Synced
because then you've lost that full backup.
-
Not Synced
And also it ends up batching the full backups so they're very clustered
-
Not Synced
rather than being nicely spread out.
-
Not Synced
One of the things we're looking at is writing a different scheduler for bacula
-
Not Synced
just to basically tell it: "please do a full backup of this host, now."
-
Not Synced
rather than relying on the built-in scheduler.
-
Not Synced
[zobel]: (inaudible)
-
Not Synced
[question3]: I'm the maintainer of a package called 'bup'
-
Not Synced
It's not a full-fledged backup system with a scheduler, et cetera
-
Not Synced
but it does use for its backend, git packfiles rather than tapes
-
Not Synced
If you're interested in git, maybe some interesting technology to take a look at
-
Not Synced
[Tollef]: Look time I looked at bup, it didn't actually support expiring backups
-
Not Synced
Which makes for some pain.
-
Not Synced
[question3]: There are some workarounds,
-
Not Synced
but it's one of the limitations currently
-
Not Synced
[Tollef]: For us, that would mean we would run into-
-
Not Synced
I'm sure ?? or ?? would be very happy but I'm not sure that
-
Not Synced
our treasure would be as happy.
-
Not Synced
We need the ability to expire backups,
-
Not Synced
just because we don't have infinite sized hard drives
-
Not Synced
and backups are actually quite big.
-
Not Synced
[zobel]: One of the other issues with bacula is that currently all of the full
-
Not Synced
backups run at the same time
-
Not Synced
so we run into some sort of ?? limitations
-
Not Synced
which it's not an issue but it's annoying that
-
Not Synced
all machines are doing the full backups at the same time.
-
Not Synced
Any else questions?
-
Not Synced
[question4]: You touched on it earlier, Single Sign On
-
Not Synced
What services are next for that?
-
Not Synced
[Tollef]: Don't run away from the mic!
-
Not Synced
[Enrico]: I'll answer that as far as I know
-
Not Synced
many people may have different plans.
-
Not Synced
Single Sign On is currently using DACS,
-
Not Synced
which I would suggest against, in general...
-
Not Synced
[laughter]
-
Not Synced
Having looked deeply into it,
-
Not Synced
it probably seemed like a good idea at the time
-
Not Synced
but the internet moved in a different direction.
-
Not Synced
But DACS is still useful because it's an apache thing
-
Not Synced
so one can just put a directory of static files under DACS
-
Not Synced
and that can be done quite reasonably simply.
-
Not Synced
At DebConf I want to discuss with the currently available DSAs
-
Not Synced
about finishing the DACS setup, putting the basic stuff in puppet
-
Not Synced
and making a guide for deploying new stuff.
-
Not Synced
Any Debian Developer that deploys services can set up DACS
-
Not Synced
reasonably easily, but the way that I see we should go in the future
-
Not Synced
is OAuth 2, which is what we are using for the conference thing.
-
Not Synced
Because that is a bit more like a standard that may work now
-
Not Synced
and which hopefully supports log out!
-
Not Synced
[laughs]
-
Not Synced
which DACS does not do very well.
-
Not Synced
I have not studied OAuth 2, so I'm not interested,
-
Not Synced
it won't be me who does it.
-
Not Synced
If any of you knows OAuth 2 and wants to sit down with me and explain it step
-
Not Synced
by step during DebConf, then please I would like to migrate NM and Debian
-
Not Synced
Contributors to OAuth 2, if at all possible.
-
Not Synced
But I do want to understand the protocol before I touch it.
-
Not Synced
So the direction as far as I'm concerned, will be OAuth 2.
-
Not Synced
We may get stuck with DACS, because it integrates with apache
-
Not Synced
but I'm not comfortable with it and there are too many hacky things
-
Not Synced
to make things work as expected.
-
Not Synced
I wish, my personal dream would be to at some move to OAuth 2 and then
-
Not Synced
replace DACS with just an OAuth 2 provider.
-
Not Synced
[zobel]: Other limitations of our current DACS set up
-
Not Synced
is that it only works for the debian.org domain
-
Not Synced
otherwise we would need to give out credentials,
-
Not Synced
there is some jurisdiction key and federation key,
-
Not Synced
so we would need to give out access to them to the debian.net services.
-
Not Synced
That's one of the other limitations of our current DACS set up.
-
Not Synced
So probably OAuth 2 might be the way to go.
-
Not Synced
But in the end it's up to you and
-
Not Synced
the Debian Developers helping to extend the single sign on.
-
Not Synced
[Enrico]: As new DACS services, ?? set up something that uses DACS
-
Not Synced
[zobel]: It's a new PTS implementation
-
Not Synced
I think he just wants to if a person is logged in
-
Not Synced
then he can modify some news on the new PTS implementation and so on.
-
Not Synced
[Enrico]: One good thing with DACS at the moment, is that login is optional
-
Not Synced
and it totally supports serving a site as it is
-
Not Synced
and if one is logged in, in single sign on then more stuff can happen.
-
Not Synced
[zobel]: I think OAuth 2 is a better thing for the wiki to do.
-
Not Synced
[Enrico]: Does Moin does OAuth 2?
-
Not Synced
[Steve]: (inaudible)
-
Not Synced
[zobel]: I think I looked that up a few months ago and I think it supports OAuth 2.
-
Not Synced
[Enrico]: DACS will give you a remote user variable, so in theory it's easy
-
Not Synced
but if it does OAuth 2, then it's more future proof in my opinion.
-
Not Synced
[Steve]: The fun thing with the wiki as well
-
Not Synced
I was going touch about on this in my Wiki and Web BoF
-
Not Synced
(see advertising too!)
-
Not Synced
We've also currently got, like thousands of existing user accounts.
-
Not Synced
Now obviously for people who've already got Alioth or a Debian LDAP account
-
Not Synced
then we will encourage people to merge and just move over to those
-
Not Synced
but for the many thousands of others who haven't,
-
Not Synced
we're going to have to come up with something.
-
Not Synced
I don't know what that is.
-
Not Synced
[Tollef]: I don't have any response to that on the spot.
-
Not Synced
It's tempting to say, they can just get themselves an Alioth account.
-
Not Synced
Some people might be upset at that answer.
-
Not Synced
I guess there's also the question of how many
-
Not Synced
of these accounts are actually active?
-
Not Synced
Rather than somebody registered back in 2005
-
Not Synced
and haven't used the account since.
-
Not Synced
[Enrico]: I'd be happy to have a conversation about this during DebConf.
-
Not Synced
Because for Debian Contributors,
-
Not Synced
I require to have an Alioth account to get credited in site
-
Not Synced
because I don't want to have a user database in Debian Contributors.
-
Not Synced
It may be too much of a strict requirement,
-
Not Synced
it may be that we just document that if you do
-
Not Synced
anything in Debian you get an Alioth account.
-
Not Synced
Let's talk about it, separately.
-
Not Synced
[Tollef]: In that case, I think we need to have a conversation with my other hat,
-
Not Synced
which is that hat of various other people, which is the Alioth admin hat.
-
Not Synced
[Steve L.]: As the person who inflicted Alioth logins on everybody for DebConf
-
Not Synced
this year, I have been getting feedback that, in particular the sign up
-
Not Synced
process for Alioth is a bit of an obstacle.
-
Not Synced
So there are a few things there, which I think we should talk about
-
Not Synced
streamlining.
-
Not Synced
As the person who decided that we were, for this year, moving away from Penta
-
Not Synced
and moving to Summit, no I did not want to have an authentication database.
-
Not Synced
I didn't want password hashes in Summit
-
Not Synced
and so I said yes, we're going to have figure out how to hook this up to
-
Not Synced
Debian SSO and the consequence of that was, yes:
-
Not Synced
we had the Debian SSO which was only available to Debian Developers
-
Not Synced
Alioth was the other database that was out there
-
Not Synced
and so I guess, my fault,
-
Not Synced
I apologise for anyone that was stressed about the rollout of that
-
Not Synced
because I didn't entirely co-ordinate with all of the parties ahead of time
-
Not Synced
but I think it's hanging together fairly well.
-
Not Synced
But we should talk sometime this week about where we should go forward with
-
Not Synced
that and if alioth is the right authentication provider.
-
Not Synced
But I think it's important we agree there be an authentication provider,
-
Not Synced
for these kinds of services, whether that lives in Alioth or somewhere else.
-
Not Synced
[Enrico]: With a flat namespace.
-
Not Synced
[Steve L.]: With a flat username space, yes.
-
Not Synced
Which we kind of have, today
-
Not Synced
[Enrico]: (inaudible)
-
Not Synced
[Steve L.]: The way OAuth provides them is
-
Not Synced
you get the domain name with it
-
Not Synced
So in fact all Debian Developers have two different-
-
Not Synced
It's a "flat namespace" and DDs all have two they can use.
-
Not Synced
[laughter]
-
Not Synced
[zobel]: More questions?
-
Not Synced
[question5]: You mentioned that all our hosting is sponsored by the hosts
-
Not Synced
and we get some hardware donations at least.
-
Not Synced
I think we, we buy some, as well, don't we?
-
Not Synced
My question isn't really about that,
-
Not Synced
it's about how much support do we get-
-
Not Synced
there's 1.5, well 2, tending to 1 hardware manufacturers on the sponsors there
-
Not Synced
-how much support do we get from them doing interesting stuff.
-
Not Synced
I'm thinking, you mentioned we get fairly regular controller failures on some of our hardware
-
Not Synced
and all of the sponsors we've got have got nice but hard to set up multipath things.
-
Not Synced
It seems to me it would be interesting and for them
-
Not Synced
to set things up like that, on the Debian infrastructure.
-
Not Synced
Is that kind of thing possible, or?
-
Not Synced
[Tollef]: Yeah, so we do have that in some places
-
Not Synced
Like the Bytemark set-up, the UBC-ECE set-up and so on
-
Not Synced
There we have a SAN, we have a bunch of machines
-
Not Synced
and either it's doing SATA or it's doing Fibre Channel
-
Not Synced
[zobel]: iSCSI
-
Not Synced
[Tollef]: iSCSI, as well, yeah.
-
Not Synced
So we do have a bunch of that,
-
Not Synced
the problem is if you want to do data storage
-
Not Synced
where you have available 25 terabytes
-
Not Synced
and you want to do that on a SAN,
-
Not Synced
that's very not cheap, that's really quite expensive.
-
Not Synced
That's a reason why those machines with special storage requirements,
-
Not Synced
like backups and snapshot, basically, they're different.
-
Not Synced
That's also why they need those two machines,
-
Not Synced
they have like 5 controllers each,
-
Not Synced
that's why they are different in that regard.
-
Not Synced
We do get a bunch of sponsorship from the hardware vendors,
-
Not Synced
we usually buy HP gear, mostly because we've had good experience with it
-
Not Synced
and it generally works
-
Not Synced
[zobel]: We had good connections at HP.
-
Not Synced
[Tollef]: We also had historically good connections,
-
Not Synced
they've been good about giving us hardware in the past
-
Not Synced
they are happy to sponsor Debian and DebConf
-
Not Synced
both in actual terms of money given to us,
-
Not Synced
but also in terms of pretty nice prices.
-
Not Synced
I don't think we've actually approached them about saying,
-
Not Synced
"could you please give us this enormously expensive piece of hardware?"
-
Not Synced
It's often hard for them to give that away,
-
Not Synced
because it has to come out of somebodies' budget
-
Not Synced
and somehow they don't have large SANs just hidden under their desks.
-
Not Synced
[zobel]: More questions? Criticism?
-
Not Synced
[question6]: Hi, I was just curious about your mail infrastructure.
-
Not Synced
It doesn't look like you use DKIM or SPF, or DMARC records.
-
Not Synced
Do you have plans for any of that?
-
Not Synced
[Tollef]: There's been some experimentation with domain keys.
-
Not Synced
Luca has been playing with that.
-
Not Synced
There's this interesting ??
-
Not Synced
?? we generally don't provide outgoing SMTP for random people,
-
Not Synced
because that's painful.
-
Not Synced
[zobel]: We are not a mail provider.
-
Not Synced
[Tollef]: Yes, obviously you get a @debian.org account
-
Not Synced
You get incoming email, which we then forward onto somewhere
-
Not Synced
where you'll hopefully remember to update that when that account expires
-
Not Synced
rather than giving us bounces.
-
Not Synced
That's a big change, which we forgot to mention
-
Not Synced
is that we are actually in the process of reworking the entire way we do mail
-
Not Synced
We have drastically reduced the number of incoming mail servers,
-
Not Synced
so now most mail now goes to a set of two MXs.
-
Not Synced
[zobel]: It will increase in the future.
-
Not Synced
At MIT, we will open up one more mailserver.
-
Not Synced
[Tollef]: Well, we can.
-
Not Synced
Currently we have two and then if there is special mail routing needed,
-
Not Synced
it will be routed to the right internal host.
-
Not Synced
But most hosts no longer listens for incoming mail from the internet.
-
Not Synced
Which is a good thing.
-
Not Synced
Not only because it means we don't have to run spamassassin everywhere.
-
Not Synced
[zobel]: Peter did this DAME, SMTP thing
-
Not Synced
weasel, Peter, wanted to do ?? DAME encryption ?? for outgoing mails.
-
Not Synced
So we're experimenting with a bunch of things.
-
Not Synced
What I was going to say about domain keys,
-
Not Synced
is that because we don't provide outgoing mail servers,
-
Not Synced
you need to be able to provide the infrastructure
-
Not Synced
with what your key is going to be.
-
Not Synced
Luca has been working on some patches to ud-ldap to do this
-
Not Synced
so it can show up in DNS and so on.
-
Not Synced
So yes, things are happening.
-
Not Synced
If you're interested in that, do grab us and we can talk more about it.
-
Not Synced
[zobel]: I think we are done because the timer's almost over.
-
Not Synced
I have one small announcement to make.
-
Not Synced
Luca offered some RIPE NCC ATLASS notes to give away
-
Not Synced
and the persons who applied for those notes
-
Not Synced
and got into the list of getting those notes,
-
Not Synced
please come to me, talk to me directly after the talk
-
Not Synced
so I can hand out those notes.
-
Not Synced
Because Luca is not here at DebConf14 this year.
-
Not Synced
[Anibal]: Any plans to use Yubikeys?
-
Not Synced
[Tollef]: I'm port of the maintainer team of yubikey tools in Debian
-
Not Synced
I would very much like to use them for some things.
-
Not Synced
We need to find out how they should best fit into the infrastructure
-
Not Synced
if we're going to do that.
-
Not Synced
One thing that has been mentioned is,
-
Not Synced
for some cases we want to do actual two-factor.
-
Not Synced
Currently there is no two-factor authentication anywhere.
-
Not Synced
[zobel]: Help us setting up those infrastructure
-
Not Synced
[Tollef]: There are no concrete plans
-
Not Synced
but yes, we are very much aware of yubikeys
-
Not Synced
I'm kind of looking for good places to put them in.
-
Not Synced
I like them. I like both the company and the product.
-
Not Synced
They are also quite happy to sponsor free software stuff.
-
Not Synced
[zobel]: I think we are done.
-
Not Synced
[Tollef]: I think we're out of time
-
Not Synced
[zobel]: Thank you for being here.
-
Not Synced
[Tollef]: If you have any more questions, grab us afterwards.
-
Not Synced
[applause]
-
Not Synced