9:59:59.000,9:59:59.000
Debian GNU/Hurd status update
9:59:59.000,9:59:59.000
Let's talk about GNU/Hurd
9:59:59.000,9:59:59.000
For us it's a bit all about freedom 0,[br]that is, the ability to use software,
9:59:59.000,9:59:59.000
basically, for any purpose.
9:59:59.000,9:59:59.000
And for us, the important thing is that[br]you shouldn't have to ask
9:59:59.000,9:59:59.000
the system administrator for things.
9:59:59.000,9:59:59.000
You should be allowed to do[br]whatever you want.
9:59:59.000,9:59:59.000
So for instance, why is fdisk, mke2fs,[br]etc. hidden in /sbin?
9:59:59.000,9:59:59.000
I want to be able to build disk images,[br]play with them, mount them, etc.
9:59:59.000,9:59:59.000
So just be able to work with the kind of[br]disk and network access I have,
9:59:59.000,9:59:59.000
and do whatever I want with this.
9:59:59.000,9:59:59.000
It's about freedom to innovate as well, if[br]I want to use an experimental filesystem,
9:59:59.000,9:59:59.000
just play with, without being afraid of[br]crashing the machine.
9:59:59.000,9:59:59.000
You should be able to just[br]run the file system
9:59:59.000,9:59:59.000
and let the system administrator be happy[br]with this because it's safe to do this.
9:59:59.000,9:59:59.000
And also, it's a way to provide freedom[br]from misbehaving programs
9:59:59.000,9:59:59.000
like a driver which doesn't work so well,[br]some things like this.
9:59:59.000,9:59:59.000
Just to give an idea, in GNU/Hurd,
9:59:59.000,9:59:59.000
you have the kernel which does basically[br]almost nothing,
9:59:59.000,9:59:59.000
just managing tasks, the memory and[br]inter-process communications,
9:59:59.000,9:59:59.000
and then you have a lot of daemons[br]doing the actual stuff,
9:59:59.000,9:59:59.000
so the pfinet is the TCP/IP stack,[br]and ext2fs does the filesystem thing.
9:59:59.000,9:59:59.000
And then, you have the user,[br]just running programs.
9:59:59.000,9:59:59.000
And these tools just, actually just talk[br]to the daemons through the microkernel,
9:59:59.000,9:59:59.000
the microkernel doesn't do much, it just[br]passes requests along.
9:59:59.000,9:59:59.000
For instance, if a server crashes,[br]then that's fine.
9:59:59.000,9:59:59.000
For instance a driver crashes,[br]or just hangs,
9:59:59.000,9:59:59.000
you just can kill and then pfinet[br]will re-open a new instance of the driver
9:59:59.000,9:59:59.000
and it will just work, thanks to TCP just[br]continuing to ping the other computer.
9:59:59.000,9:59:59.000
So it's just an error, it's not something[br]of the death.
9:59:59.000,9:59:59.000
At some point of my desktop, I could[br]switch off the light,
9:59:59.000,9:59:59.000
and then that would crash my laptop.
9:59:59.000,9:59:59.000
Because switching off the light would[br]reboot my hard disk, USB hard disk,
9:59:59.000,9:59:59.000
and then the kernel of the laptop[br]wouldn't like this.
9:59:59.000,9:59:59.000
This is not something which is[br]supposed to happen.
9:59:59.000,9:59:59.000
So, with a server approach, this is[br]completely fixed.
9:59:59.000,9:59:59.000
It's also easier to debug, it's really[br]nice to be able to gdb a TCP/IP stack,
9:59:59.000,9:59:59.000
when there is something happening in there,[br]just run gdb, you can gprof it, etc.
9:59:59.000,9:59:59.000
You can also dare more crazy things.
9:59:59.000,9:59:59.000
For instance, the Linux console doesn't[br]support much, because we don't want to put
9:59:59.000,9:59:59.000
too much complex code in there.
9:59:59.000,9:59:59.000
On GNU/Hurd the console actually supports[br]things like Chinese,
9:59:59.000,9:59:59.000
double-width support, etc.
9:59:59.000,9:59:59.000
which is not supported[br]by the Linux console,
9:59:59.000,9:59:59.000
and that's right because you don't want[br]to put too crazy stuff.
9:59:59.000,9:59:59.000
Here since it's just a userland program,[br]then you're fine,
9:59:59.000,9:59:59.000
and so we do have Chinese support in,[br]actually, textmode in the Debian Installer.
9:59:59.000,9:59:59.000
Just to show an example, so here I have[br]ftpfs which uses the TCP/IP stack
9:59:59.000,9:59:59.000
to actually mount a remote directory,
9:59:59.000,9:59:59.000
and then I can use isofs to mount an ISO[br]image which is inside that FTP server.
9:59:59.000,9:59:59.000
And then I can just let cp copy a file[br]from the ISO image which is on the server.
9:59:59.000,9:59:59.000
So this translates that way, so I've done[br]this command a long time ago,
9:59:59.000,9:59:59.000
just to say that "ftp:" in my home[br]directory is whatever FTP,
9:59:59.000,9:59:59.000
and then I can take a "~/ftp:/etc." URL[br]and give that to isofs
9:59:59.000,9:59:59.000
and then mount that on my "mnt",
9:59:59.000,9:59:59.000
and then I can just browse inside the ISO[br]image, without having to download
9:59:59.000,9:59:59.000
the whole ISO image, without having to ask[br]root for this kind of things, etc.
9:59:59.000,9:59:59.000
And I can also permanently store this[br]in ext2fs.
9:59:59.000,9:59:59.000
So just to give an example, I have a[br]translator on my signature files,
9:59:59.000,9:59:59.000
which just calls fortune, so when I[br]"cat .signature" [demo],
9:59:59.000,9:59:59.000
I get one signature or another, because[br]each time I open the file,
9:59:59.000,9:59:59.000
it's a new instance of fortune which is[br]started.
9:59:59.000,9:59:59.000
You can see that, indeed, this is stored[br]in my signature file.
9:59:59.000,9:59:59.000
So this is fun!
9:59:59.000,9:59:59.000
Another example: as a user, I can start[br]my own TCP/IP stack,
9:59:59.000,9:59:59.000
tell it to use a virtual network[br]interface,
9:59:59.000,9:59:59.000
and then put the TCP/IP service on some[br]node in my home,
9:59:59.000,9:59:59.000
and then I can run openvpn to actually[br]push and pull packets
9:59:59.000,9:59:59.000
from that virtual interface, and build a[br]VPN with somewhere else.
9:59:59.000,9:59:59.000
And then I can remap the system, what is[br]supposed to be the system TCP/IP stack
9:59:59.000,9:59:59.000
into my own socket,
9:59:59.000,9:59:59.000
and then I get a new shell for which the[br]system TCP/IP stack is actually
9:59:59.000,9:59:59.000
my own TCP/IP stack.
9:59:59.000,9:59:59.000
So I can decide which program actually[br]uses this TCP/IP stack,
9:59:59.000,9:59:59.000
and just do my own VPN without having to[br]ask anything to the administrator.
9:59:59.000,9:59:59.000
But also, for instance it happens quite[br]often that you have a binary,
9:59:59.000,9:59:59.000
maybe not sh, but like, python or perl[br]or whatever,
9:59:59.000,9:59:59.000
you have a program which wants /bin/sh[br]to be actually bash or whatever,
9:59:59.000,9:59:59.000
so I want to change this, so I can remap[br]this, so for instance [demo]
9:59:59.000,9:59:59.000
if I look at sh, so as usual,[br]oh it's green,
9:59:59.000,9:59:59.000
but you can see here that it's dash,
9:59:59.000,9:59:59.000
and if I remap /bin/sh into /bin/bash, for[br]instance, I get a new shell where actually,
9:59:59.000,9:59:59.000
sh is not the same, so it's remapped into[br]/bin/bash,
9:59:59.000,9:59:59.000
and so it's actually bash which actually[br]shows up here.
9:59:59.000,9:59:59.000
So I do really choose how I work, what my[br]environment looks like.
9:59:59.000,9:59:59.000
And for instance, I can remap the whole[br]/bin directory into my own directory,
9:59:59.000,9:59:59.000
where I expose /bin, but also[br]other things,
9:59:59.000,9:59:59.000
so that programs which have /bin/something[br]hardcoded into them,
9:59:59.000,9:59:59.000
I can use them without having to ask the[br]administrator to install stuff inside /bin.
9:59:59.000,9:59:59.000
So it's kind of interesting, a bit like[br]stow, Nix, Guix, but done in a nice way.
9:59:59.000,9:59:59.000
How does it work? Well it's actually[br]relatively simple in the principle,
9:59:59.000,9:59:59.000
it's simply that libc doesn't talk with[br]the kernel or whatever,
9:59:59.000,9:59:59.000
it always uses RPCs, so to ask nicely[br]about opening files etc.,
9:59:59.000,9:59:59.000
and so it's really natural in GNU/Hurd[br]that you can redirect things.
9:59:59.000,9:59:59.000
So for instance, the remap translator here[br]is like, maybe,
9:59:59.000,9:59:59.000
200-300 lines[br][Note of transcriptor: 150 actually],
9:59:59.000,9:59:59.000
because it's just a matter of
9:59:59.000,9:59:59.000
"you open a file, OK, I look at the file[br]path, is it something I want to translate?
9:59:59.000,9:59:59.000
Yes, I translate that, and then I open[br]the real file,
9:59:59.000,9:59:59.000
and give the new handle to the program",
9:59:59.000,9:59:59.000
and that's all, so it's extremely simple.
9:59:59.000,9:59:59.000
So everything in GNU/Hurd is an RPC and[br]so it is interposable,
9:59:59.000,9:59:59.000
and then translators get exposed in the[br]filesystem, we have seen the TCP/IP stack,
9:59:59.000,9:59:59.000
it's just a path inside the filesystem.
9:59:59.000,9:59:59.000
And then the user can decide whatever it[br]wants to do to interpose whatever.
9:59:59.000,9:59:59.000
So, for instance fakeroot, in Linux,[br]is quite big,
9:59:59.000,9:59:59.000
because it has to interpose libc symbols,
9:59:59.000,9:59:59.000
and every time libc invents something new,[br]then it breaks in fakeroot
9:59:59.000,9:59:59.000
because fakeroot has to know about this[br]new symbol, etc. and interpose them,
9:59:59.000,9:59:59.000
either through ptrace or ld or whatever.
9:59:59.000,9:59:59.000
In GNU/Hurd, fakeroot is, like,[br]a thousand lines long,
9:59:59.000,9:59:59.000
because it just implements a few basic[br]things,
9:59:59.000,9:59:59.000
and then everything just works, which just[br]interpose basic authentication hooks,
9:59:59.000,9:59:59.000
and libc uses them all the time.
9:59:59.000,9:59:59.000
So it's fully virtualizable, and with[br]a really fine grain interface,
9:59:59.000,9:59:59.000
because you can precisely decide[br]which RPCs are interposed,
9:59:59.000,9:59:59.000
or which files in the filesystem[br]are interposed.
9:59:59.000,9:59:59.000
And then you can just use your home[br]directory, the TCP/IP stack,
9:59:59.000,9:59:59.000
and pile stuff over it, the way you want.
9:59:59.000,9:59:59.000
Just to give a crazy example, we have[br]a lot of stuff,
9:59:59.000,9:59:59.000
I actually have ISO image inside a[br]partitioned disk image on FTP over a VPN.
9:59:59.000,9:59:59.000
And this is not so crazy.
9:59:59.000,9:59:59.000
Maybe the ISO image inside the partitioned[br]disk, the ISO image is a bit too much,
9:59:59.000,9:59:59.000
but one file inside the partitioned disk[br]image on FTP over VPN is not so crazy,
9:59:59.000,9:59:59.000
because maybe you are on a hostile[br]network, so you have to use a VPN,
9:59:59.000,9:59:59.000
and then you want to access a file[br]you know is inside a disk image,
9:59:59.000,9:59:59.000
I don't know, a known disk image which is[br]provided on a public FTP server,
9:59:59.000,9:59:59.000
and you don't want to download the whole[br]image just to get, I don't know,
9:59:59.000,9:59:59.000
the README file or something like this.
9:59:59.000,9:59:59.000
So it's not so crazy, and it just[br]works nicely.
9:59:59.000,9:59:59.000
So a bit more Debian stuff.
9:59:59.000,9:59:59.000
Porting packages to Hurd is quite easy[br]in principle,
9:59:59.000,9:59:59.000
because it's just a POSIX system, there is[br]a lot more than just POSIX,
9:59:59.000,9:59:59.000
but it provides a POSIX interface.
9:59:59.000,9:59:59.000
So portable programs should be[br]really fine.
9:59:59.000,9:59:59.000
Just for fun, some dumb issues, so for[br]instance some programs think that
9:59:59.000,9:59:59.000
if it's Linux or BSD, then they can[br]include windows.h...
9:59:59.000,9:59:59.000
Why not...
9:59:59.000,9:59:59.000
If the system has mach.h, that must be[br]MacOS,
9:59:59.000,9:59:59.000
because MacOS is the only system in the[br]world that uses Mach, I don't know why...
9:59:59.000,9:59:59.000
Some people try to grep cpuinfo, which[br]doesn't exist on GNU/Hurd yet,
9:59:59.000,9:59:59.000
and so they basically just run "make -j"[br]which just explodes the system,
9:59:59.000,9:59:59.000
I mean even on a Linux system it's just[br]the same, unless it's a small program,
9:59:59.000,9:59:59.000
but with a lot of C++ files it's horrible.
9:59:59.000,9:59:59.000
Some people include limits.h from linux/[br]instead of just the standard one, well...
9:59:59.000,9:59:59.000
A problematic thing is people who[br]hardcoded errno values;
9:59:59.000,9:59:59.000
the values of errno are not standardized,[br]so you shouldn't hardcode them, like,
9:59:59.000,9:59:59.000
in testsuite results or things like this.
9:59:59.000,9:59:59.000
And quite often in configure it's[br]hardcoded that
9:59:59.000,9:59:59.000
only Linux knows -lpthread or -ldl, etc.
9:59:59.000,9:59:59.000
so quite often programs are not[br]generic enough,
9:59:59.000,9:59:59.000
and that's just easy to fix, but we have[br]more and more of these.
9:59:59.000,9:59:59.000
So we have a porter page developing[br]a bit more about these.
9:59:59.000,9:59:59.000
I wanted to talk a bit more about[br]PATH_MAX, it is not defined on GNU/Hurd,
9:59:59.000,9:59:59.000
for very good reasons, and it is allowed[br]by POSIX not to define it,
9:59:59.000,9:59:59.000
just to say that there is no limitation on[br]the PATH_MAX value,
9:59:59.000,9:59:59.000
we don't have a limit on the size of[br]the paths.
9:59:59.000,9:59:59.000
And indeed it has a fragile semantic, it[br]has never meant
9:59:59.000,9:59:59.000
"a reasonable size for a array of[br]characters to store a path".
9:59:59.000,9:59:59.000
On Linux it's 4000, that's a whole page,
9:59:59.000,9:59:59.000
that's a whole TLB entry for[br]just one file name.
9:59:59.000,9:59:59.000
It's extremely costly, most people don't[br]have so long paths,
9:59:59.000,9:59:59.000
and so it's really a pity to use so much[br]memory, because it's always a whole page
9:59:59.000,9:59:59.000
because it will always be aligned[br]on 4k etc.
9:59:59.000,9:59:59.000
So, well, that's a waste for one.
9:59:59.000,9:59:59.000
And paths can actually be longer,[br]there is no strict limitation,
9:59:59.000,9:59:59.000
you can mkdir something, cd into that[br]mkdir again, cd, etc.,
9:59:59.000,9:59:59.000
you can do that as much as you want,[br]there is no limitation on this,
9:59:59.000,9:59:59.000
it's just that when you call
9:59:59.000,9:59:59.000
"get current working directory",[br]you won't get it completely.
9:59:59.000,9:59:59.000
And actually, some programs misbehave[br]in that case,
9:59:59.000,9:59:59.000
because they won't see these files,[br]they will be quite actually hidden,
9:59:59.000,9:59:59.000
or protected, or I don't know,
9:59:59.000,9:59:59.000
you can not remove them just giving[br]the path, you have to cd, cd, cd, cd,
9:59:59.000,9:59:59.000
and then you can access the file.
9:59:59.000,9:59:59.000
And for no reason, actually, because Linux[br]inside doesn't have
9:59:59.000,9:59:59.000
such limitation, actually.
9:59:59.000,9:59:59.000
And also, it's stupid, but POSIX didn't[br]really said precisely whether
9:59:59.000,9:59:59.000
the final \0 actually is included in[br]PATH_MAX or not,
9:59:59.000,9:59:59.000
so people would allocate PATH_MAX+1,[br]or maybe not.
9:59:59.000,9:59:59.000
So we have a lot of code which doesn't,[br]maybe, actually work,
9:59:59.000,9:59:59.000
but nobody tests it, actually, because[br]they would never have so long paths.
9:59:59.000,9:59:59.000
So I'm a bit afraid of all these using[br]PATH_MAX.
9:59:59.000,9:59:59.000
You should be afraid as well.
9:59:59.000,9:59:59.000
Just to give an overview of the state.
9:59:59.000,9:59:59.000
We have a i386 support, we have a 64bit[br]support which has started,
9:59:59.000,9:59:59.000
we have the kernel booting,
9:59:59.000,9:59:59.000
and now it's mostly translating between[br]32 and 64 in our RPCs.
9:59:59.000,9:59:59.000
We have drivers for network boards as a[br]userland translator, using the DDE layer.
9:59:59.000,9:59:59.000
We have disk, we have a Xen port.
9:59:59.000,9:59:59.000
We have a preliminary sound which was[br]announced today, using Rump,
9:59:59.000,9:59:59.000
the Rump kernel.
9:59:59.000,9:59:59.000
We don't have USB yet.
9:59:59.000,9:59:59.000
It is quite stable, I haven't reinstalled[br]my boxes for, like, a decade,
9:59:59.000,9:59:59.000
I don't remember when I installed them,[br]actually.
9:59:59.000,9:59:59.000
And then the buildd machines just keep[br]building packages for weeks
9:59:59.000,9:59:59.000
without a problem.
9:59:59.000,9:59:59.000
We have 81% of the archive.
9:59:59.000,9:59:59.000
We have the native Debian Installer which[br]is really working great.
9:59:59.000,9:59:59.000
Recent work is, like, interesting thing is,[br]a distributed mtab translator
9:59:59.000,9:59:59.000
to provide /proc/mounts in a hurdish way.
9:59:59.000,9:59:59.000
We have quite a few optimizations which[br]went in to improve the performance.
9:59:59.000,9:59:59.000
We had releases quite some time ago,
9:59:59.000,9:59:59.000
I really recommend to have a look at this[br]one, it's fun.
9:59:59.000,9:59:59.000
We've some Wheezy and Jessie snapshots,
9:59:59.000,9:59:59.000
they are not official, but for us it's[br]really an official thing.
9:59:59.000,9:59:59.000
An important thing I wanted to discuss[br]this week is the removal from ftp-master.
9:59:59.000,9:59:59.000
This is due since quite a few years now,[br]honestly,
9:59:59.000,9:59:59.000
it's really not useful to mirror the hurd[br]packages over the whole world,
9:59:59.000,9:59:59.000
because there are not even as many users[br]as the number of mirrors.
9:59:59.000,9:59:59.000
So OK, that's fine for just the removal[br]from the main archive in terms of mirroring.
9:59:59.000,9:59:59.000
But then we have a lot of consequences.
9:59:59.000,9:59:59.000
For instance, buildd.debian.org is really[br]an important thing,
9:59:59.000,9:59:59.000
because that is where the release team[br]schedules transitions,
9:59:59.000,9:59:59.000
and loosing this, for us, would be really[br]tedious work,
9:59:59.000,9:59:59.000
because I've been there, doing, actually,[br]the transition work,
9:59:59.000,9:59:59.000
the same work as the release team, and[br]it's really painful to do this again.
9:59:59.000,9:59:59.000
So we would really like to have a solution[br]for this.
9:59:59.000,9:59:59.000
Maybe get that fed from debian-ports and,
9:59:59.000,9:59:59.000
then that's fine, we can be on[br]debian-ports, as long as at least
9:59:59.000,9:59:59.000
there is some synchronization between[br]something.
9:59:59.000,9:59:59.000
And also, getting exposed on the buildd[br]package status page,
9:59:59.000,9:59:59.000
so that people are aware that there is[br]some port which is failing,
9:59:59.000,9:59:59.000
and maybe they are keen on spending some[br]time on it, maybe not,
9:59:59.000,9:59:59.000
but at least get them know about it.
9:59:59.000,9:59:59.000
And also, a corner thing, when we have[br]a version upgrade, like gcc or perl,
9:59:59.000,9:59:59.000
the release team asks[br]"OK, we'll have to upgrade the buildds",
9:59:59.000,9:59:59.000
and at the moment they don't even have[br]an account on them,
9:59:59.000,9:59:59.000
so they can not check whether the version[br]is good or not.
9:59:59.000,9:59:59.000
Maybe we should just provide an account,
9:59:59.000,9:59:59.000
we'd thus need to know who we need to give[br]an account to.
9:59:59.000,9:59:59.000
Basically, my idea would be[br]"OK, that's fine not being on ftp-master".
9:59:59.000,9:59:59.000
The thing is we still want to have most of[br]the support of Debian,
9:59:59.000,9:59:59.000
to make our life less a burden,[br]as much as possible,
9:59:59.000,9:59:59.000
without any extra load on[br]the release team, etc.
9:59:59.000,9:59:59.000
We do understand well that we don't want[br]to put work on people's hand.
9:59:59.000,9:59:59.000
But we would to still get some benefit and[br]probably there are solutions for this.
9:59:59.000,9:59:59.000
And conversely, all of this, I mean, not[br]putting more work on us Hurd porters,
9:59:59.000,9:59:59.000
would actually be the same solutions that[br]existing ports on debian-ports
9:59:59.000,9:59:59.000
would be really happy to have, to improve[br]their life, to have less work to do, [...]
9:59:59.000,9:59:59.000
So maybe we want to think about a real[br]status for Second Class Citizens,
9:59:59.000,9:59:59.000
like Hurd, but also the sparc, hppa, etc.
9:59:59.000,9:59:59.000
Maybe want to have some BoF at some time,[br]so we can gather and discuss about this.
9:59:59.000,9:59:59.000
Future work, the most interesting thing is[br]probably using the Rump drivers,
9:59:59.000,9:59:59.000
because at the moment we use DDE but[br]it's not really going forward.
9:59:59.000,9:59:59.000
We thought it would be a way to get newer[br]drivers, Linux drivers,
9:59:59.000,9:59:59.000
without extra efforts, but it doesn't[br]actually happen at the moment,
9:59:59.000,9:59:59.000
while Rump does go forward, we see work[br]being done with Xen etc.
9:59:59.000,9:59:59.000
So this is probably a long-term solution.
9:59:59.000,9:59:59.000
Maybe we'll have another distribution[br]through Guix.
9:59:59.000,9:59:59.000
This is progressing, we are quite far from[br]doing this,
9:59:59.000,9:59:59.000
so for now Debian is really the only Hurd[br]distribution that we have, so we'll see.
9:59:59.000,9:59:59.000
And of course, just come and have fun with[br]your own pet project, just join, thanks!
9:59:59.000,9:59:59.000
[Michael Banck] Any quick question before[br]we run to lunch?
9:59:59.000,9:59:59.000
[Steve Chamberlain] Hello, I just wondered
9:59:59.000,9:59:59.000
if you're using Hurd on that laptop[br]for the presentation?
9:59:59.000,9:59:59.000
[Samuel Thibault] Yeah, yeah,[br]this is running Hurd, yes.
9:59:59.000,9:59:59.000
[SC] So it's quite, like, usable everyday?
9:59:59.000,9:59:59.000
[ST] Well, not everyday because[br]without USB,
9:59:59.000,9:59:59.000
you can not mount a USB stick for instance,[br]so that's quite inconvenient,
9:59:59.000,9:59:59.000
but yeah, I could probably use it everyday.
9:59:59.000,9:59:59.000
I don't, I mean, for work,[br]I can not afford this, but yeah.
9:59:59.000,9:59:59.000
Also, we don't have wireless drivers[br]at the moment.
9:59:59.000,9:59:59.000
We hope that with the Rump drivers[br]we would get this.
9:59:59.000,9:59:59.000
So, yes.
9:59:59.000,9:59:59.000
Some people do use it everyday.
9:59:59.000,9:59:59.000
Not me.
9:59:59.000,9:59:59.000
[SC] But those would be the major thing[br]missing for more people
9:59:59.000,9:59:59.000
to be able to use it.
9:59:59.000,9:59:59.000
OK, thanks.
9:59:59.000,9:59:59.000
[MB] Any more questions? We're run out of[br]questions, then thanks again. Thanks.