meetings-archive.debian.net/.../GRUB_ancient_and_modern.webm

Hi, I'm Colin Watson, this is a recruitment drive.

I'm glad to see quite so many people here, that's very useful.

[video team]: (inaudible)

[Colin]: There's a little bit of feedback?

Okay, is that any better?

[video team]: (inaudible)

[Colin]: Alright.

Somehow over the years I've found myself

as the de facto GRUB maintainer in debian

and I'm also a GRUB developer over the years

and I'm here to persuade you all that this is a fun thing to do

and that can you help, because I need more people

So yay

grub has moved on a long way from it's beginnings

It's called the GRAND UNIFIED BOOT LOADER

which is a bit of an aspirational name

and certainly was an aspirational name

back in 1995 and back then most us of just

used it over LILO which is the really traditional

x86 loader, because you didn't have to remember to reinstall

your bootloader when you installed a new kernel which was really

annoying, but even so quite a few debian developers have

hacked on grub over the years and nowadays it's a very powerful

bootloader, it's been ported to many architectures

it's actually quite rewarding to hack on

so I'll be giving you a tour of its history

and of its design and suggesting a some particular areas

where we could really do with help

Now the grub project; this is grub 1

it was originally just called grub and now

we call it grub legacy, but it was started in 1995 by Erich Boleyn,

he was initally trying to build something to boot GNU/Hurd

and among the loaders of the day it was quite unusual in that

you know, you could edit its menus on the fly and

that sort of thing. It had an emacs or bash style interface to that.

Other loaders usually just let you append kernel line options and that was

about all you got

and grub even then had a reasonably capable file system

interface as well

so it was by and large good enough, so many people adopted it as it was.

and of course it was originally designed for the Hurd

to start with so it started out with a focus on a new boot method

they designed which is called multiboot and the history file says

they were determined not to add to the large number of mutually

incompatible PC boot methods

[laughter]

Yes, well.

grub did soon become a little bit more generic

it supported the methods for booting linux and such

and I am being a bit unfair

multiboot has been genuinely useful to people doing academic experiments

with kernels written from scratch and custom payloads

because they don't have to do all of that work from scratch

anyway I won't cover multiboot further.

This is a rough layout of grub legacy back in the day

You had stage 1, this was a tiny thing that fitted

in the 512 byte master boot record.

It just knew how to read the first sector of the next

stage along, stage 1.5, from a fixed location, and jump to it.

That was all it could do, it was really stupid.

There was a seperate stage 1.5 for each separate file

system type that grub understood.

And that had enough file system code to

read stage 2 from an ordinary file system.

So it's the usual kind of bootstrap your way gradually up the stack.

The real meat of the loader lived in stage 2 and that included

all the command line commands, configuration file parsing

and the code to actually boot a payload,

like a Linux kernel image,

which is probably what you wanted to do with it.

So far, so good.

But there were quite a few problems with the design

as it was initially put together.

The file system abstraction was pretty ad-hoc.

It was achieved by building a seperate stage 1.5 blob

for each file system that grub knew about

and shipping that in the package.

The terminal abstraction was pretty reasonable aswell,

but other than that there wasn't much in the way of

internal expressive par(sing??) going on.

So it was very difficult to extend it safely.

It's hard to work out how grub legacy could ever

have been taught to handle LVM, for instance,

because you would then have to have a stage 1.5 that knew

about each of the file systems on LVM and so on.

You end up with a combinatorial explosion.

So it wasn't very elegant, from that point of view.

There were lots of PC BIOS assumptions aswell.

It may have been grand and unified,

but it wasn't particularly portable at that point.

Fedora did manage to get it to work with UEFI.

Actually, probably Red Hat then managed to get it

to work with UEFI, but it wasn't at all

a straight-forward exercise and it was very hacky.

So as time went on, upstream maintenance kind of

fell by the way side, it was such a pain to work

on and even though there wasn't really anything newer

that was usable, nobody wasn really interested in the

upstream so the distributions did what they had to do.

This meant we have ended up with grub legacy packages

in Debian and Ubuntu and Fedora and SUSE and all the rest,

that are actually completely different products.

I don't just mean have a different patch set,

in the usual way that you get,

they have completely different installation

and configuration tools that don't exist in

other multiple branches.

So today it's impossible if someone comes to us upstream,

to support grub 0.97, because it just isn't enough of a thing by itself.

You have to figure out what distribution people are using

and tell them what to do for that.

So we end up punting a lot of that to the distributions to support.

So there is a grub 2 rewrite, started in around 2002.

This is a very rough timeline of it,

as you can see it was a pretty long project, pretty slow project.

Yoshinori K. Okuji, I hope I pronounced his name correctly,

he was the lead grub maintainer at the time.

He started work on what he called:

"Preliminary Universal Programming Architecture for GNU Grub"

or 'PUPA'.

It's GNU so there is a bad pun in the acronym,

a pupa is the next stage along from a larva or grub in an insect's lifecycle.

I'd say it was roughly usable by about 2007,

so there was a long period when it was very experimental.

My pretty biased viewpoint is that distributions started

adopting it properly and started to present it to users in about 2009.

A snowball effect kind of kicked in from there

and it became much better quality quickly,

until we finally got 2.0 out in 2012.

So I also noted in here the point where the first

non-x86 architecture port showed up, that was the PowerPC port

for New World Macs, in 2004.

Because that's a very unusual milestone

in something as architecture specific as bootloaders have historically tended to be.

But right now if you look at some core bits of grub 2,

you can kind of see the echos of bits of grub legacy in there.

There are a few common lines of code

but it was a pretty sweeping rewrite and there isn't much left.

This was a very large project by most standards,

the lead maintainship has changed hands 3 or 4 times along the way,

it took a lot of forward thinking for people to get involved early on.

So the design that we have now, is around a small kernel

that offers core features like initialisation,

a virtual file system layer, a module loader

almost everything is in modules, sound familar?

there is an insmod command,

there is considerable inspiration taken from

full-fledged operating systems here.

You mostly don't have to worry about much of this by hand,

grub-install which is the main installation tool,

works out the bare minimum set of modules you need

to read everything else from /boot/grub,

and builds what we call a core image.

That's the rough equivalent of what stage 1.5 was in grub legacy,

but this time it's built dynamically,

rather than all being shipped statically in the package.

And also at runtime, the loader will load some modules into itself automatically.

So for example if you request a command name that isn't currently loaded,

it will go off and load it for you automatically.

This architecture makes it a lot easier to

construct images that fit into a constrained space,

in a much more flexible way.

We make very heavy use of abstraction layers,

it's rather like the way, not quite identical,

but rather like the way Unix-like systems

present block devices and file systems.

And those are general composable,

you can quite easily boot from XFS on LVM on RAID, whatever.

There's even the loopback module, so you can treat a file as

if it were a disk, much like we have in Linux.

This makes it quite easy to do strange things,

like you can embed an entire Debian installation

in a file on a Windows system,

which we actually did with Ubuntu for a while

and it's quite scary but it largely works.

This design also lets us easily build userspace

tools from very near to the same code,

which is very important and I'll come onto that later.

As you can see from the random stats at the bottom,

there's very little assembly in this,

it's almost all portable C.

Almost all of the C is common,

the assembly is of course a lot more architecture specific

but for a PC BIOS we are only talking about

about 4 and a half thousand lines,

which is considerable but not too much of a maintenance burden.

So here is a brief summary of what architectures we have now,

we have a number of x86 platforms,

the newest of those is that you can now use grub

under Xen paravirtualization, with a bit of effort.

The newest architectures are arm and arm64,

which were added last year.

In practice only some arm32 devices actually work with this,

because they require the platform's Uboot to be built with

the Uboot API so that it can act as enough of a firmware for grub.

A lot of them don't have that turned on, but it's at least a start.

I believe that, in theory,

all of jessie's release architectures other than s390

(no it's s390x now, isn't it?)

have at least some level of grub support.

Which is pretty awesome.

In grub legacy the hardest and one of the most

common problems we had to debug were problems

with reading files from disk in one way or another.

So people had problems loading the kernel and initrd

because their filesystem was in some strange shape.

If you wanted to debug this you were usually stuck with

trying to set up something that roughly matched in an emulator

and either using just crude printf debugging

or in extreme cases trying to attach gdb to the emulated machine over qemu.

Some people in the audience may be familar with this routine,

it's not very much fun, especially without a gdb stub.

So in grub 2, as I said earlier,

we have much of the same code built into utilities called grub-probe and grub-fstest

and these use the operating system's block device as a backend.

So you can ask grub to use its own partition table and file system tools

and parsing code from right there in userspace.

This makes it very much easier to attack lots of common problems;

you can use gdb, you can use valgrind,

you can use the debugging tools of your choice.

Tom ?? was asking me earlier about f2fs and putting that together in grub 2,

it can almost entirely be done in userspace so you can put together a new module,

built a version of grub-probe that's linked against it

and iterate until you get something that works,

rather than having to constantly reboot.

Even having to reboot an emulated machine is a lot of work.

So a similar trick also gave us this spin-off benefit of grub-mount,

that's a FUSE file system implementation that uses grub's filesystem code.

So that gives you a guaranteed true read-only mount,

using the exact same view that the bootloader

will have when it tries to look at your system.

This lets you avoid the caveats that apply to

things like journalling filesystems in Linux,

which it turns out you sometimes can't safely read-only mount.

Or sometimes the kernel will simply refuse

and it's also useful for things like os-prober.

If somebody felt the urge to make that work

as a Hurd translator it would be a pretty good fit too.

Now, you can't do everything in userspace,

at boot time grub gives you a pretty nice bash style interactive

shell with runtime controllable debugging levels.

Set the debug variable to various things and

that will give you different amounts of spew on your console.

So you can often try things out on the fly,

to figure out what's going wrong.

You can do quick, miniature image builds using the grub-mkrescue command,

that gives you an iso image containing the version of grub that you're using,

which you can then boot in an emulator to try out.

A useful trick is to boot your real live system with such an image

with 'qemu -snapshot' so that it won't write anything back

and then you see how the loader that you're working with will boot your laptop,

without actually risking the possibility of breaking anything.

It's easy to pull out bits of configuration files,

run them, write new commands.

There is a Hello World command that's about 30 lines,

so it's quite easy to put new things together.

Getting onto to things that are a little bit move involved and/or broken,

one of the things that was part of debian's grub legacy changes

was the Debian specific update-grub script.

Now this worked by trying to guess which parts of

/boot/grub/menu.lst were user modified,

by way of magic comments and updating everything

else to match the current state of the system.

This was sort of OK, but it caused a lot of complaints,

sometimes it was just because people didn't read the comment

text which said which parts of the file were safe to edit.

But generally, mixing user-editable

and automatically updated content in the same file

usually turns out to be a bad idea

and we've seen that in various other places in Debian.

So for grub 2, my predecessors brought this upstream as grub-mkconfig

and made it generate the whole configuration file from a

small amount of user edited configuration in /etc/default/grub

and a bunch of scripts in /etc/grub.d.

You can still of course write your own grub.cfg directly

if you like, it is about the same length as it would be in grub 1,

but for a general purpose system we would prefer the autogenerated approach.

So far, so good.

Just about everything is customisable,

the problem is that you have to try quite hard in several places.

Anything involved requires editing quite complex shell scripts,

if those are changed then later you'll

have quite a difficult merge resolution to do.

Some changes require things like moving conffiles

around to different positions in the order which isn't

going to be handled well by dpkg conffile resolution in the future.

So all of this probably needs somebody, sadly,

to sit down and design a third iteration of the system

that uses a templating language or something.

but that hasn't really been at all started yet.

Ok, now onto some things that are still genuinely quite hard.

The PC BIOS architecture has, well,

secreted over 30 or 40 years of gradual development.

It's basically pretty awful for modern purposes.

The usual partition table format is called MBR,

for Master Boot Record, or sometimes the msdos partition table.

It doesn't offer any formal space for keeping boot code in,

you can cram a trivial loader into 446 bytes,

that basically gives you the debian mbr package,

which jumps to a loader somewhere else.

There are a few approaches to where you can keep

the rest of, the real meat of, the loader.

You can embed in a file system somewhere,

at an offset that you know won't change.

For instance you put it in a file and trust that

the file system is not going to move that around.

Sadly sometimes file systems do move things around,

you might run fsck or there are some file systems

that will do their own, I think tail packing in reiserfs is one of these,

that will sometimes move blocks around for you

on the fly and that doesn't play very well with a

bootloader that has hardcoded addresses in the MBR.

A few file systems support an embedding area at the start,

I think btrfs does and one or two others,

this is good but not really widespread enough

that we can build the whole installation strategy around it.

You can do what grub generally does now,

you can skate the edge of the specification,

that sort of really doesn't exist anywhere coherent,

and you can use what's sometimes called the boot track

or other names, that's the unallocated space between the MBR and the first partition

and this makes some people itchy, it's a bit nasty.

It means we have to do some alarming things to avoid conflicts

with other software that put things in the boot track.

Fortunately almost all of that software is evil.

[laughter]

There are a few bits of Windows software,

well some of them are backup utilities which I'm not sure

why they put things there, but they probably have a reason.

There are a few things on Windows that like to put a note in a

sector in the boot track to say that they have been installed,

so that you cannot simply uninstall them to work around their 30 day license trial terms,

you need to know to wipe that bit of the boot track aswell.

And that's all terribly nasty.

Grub actually now has error correction on some of it's own code in the core image.

Grub can literally cope with a couple of sectors of the

core image being overwritten with completely random data.

[laughter]

It's horrifying that we need this, but yes that surprisingly works.

Fortunately this is becoming less of a problem,

the alignment requirements on modern disks mean that for decent peformance

you want to have your first partition start at 1 megabyte or sometimes even more,

rather than the traditional 63 sectors.

So we usually now have plenty of space on MBR,

but not quite always.

Things are also unpredictable when you have multiple disks,

you can't tell from Linux which disk the system was actually booted from.

There are some things on some BIOS versions that let you make better guesses,

but it's unfortunately not universal.

You also can't tell what disk it's going to be booted from next time,

it's not necessarily the same.

Sometimes it depends on which order the devices are enumerated

by the PCI bus, even on the BIOS level.

The least bad answer is usually to install to all of the fixed disks on your system,

but then you annoy some people who have complicated multiboot setups,

so you can't really win.

The GUID Partition Table format, or GPT, is much better.

It's probably the best thing to have come out of the UEFI spec.

[knocks down debconf sign]

Excuse me, let me not do that. Right, I'll not do that again.

This gives you allocated space for

the bootloader code in the UEFI system partition,

in a partition type that we were able to safely allocate for

ourselves because now partition types are a GUID, they are ginormous.

We were able to allocate one for ourselves,

even when using that on top of BIOS, which you can do.

There are different intepretations of the spec,

unfortunately this causes some problems.

When you install a system with GPT,

you're supposed to put what's called a protective MBR in place,

that means that anything that trys to parse the disk as MBR

knows that it should stay away

and not try to scribble over the partition table.

The intepretation of that part of the spec is particularly bad.

Apple, at least, used to have a completely incompatible boths ways version,

you were required instead of a single giant partition to do the best job of

representing the GPT partitions in MBR.

That's incompatible both ways, I don't know if that's still the case.

If you're trying to use GPT on BIOS,

some PC class systems require you to set

the active flag on the protective MBR,

which the GPT spec explicitly forbids for some reason.

So you're stuck both ways round.

UEFI is where it's at for PC firmware, apparently.

Even if your new machine looks like it's BIOS,

it's almost certainly a legacy layer

implemented internally on top of UEFI.

The economics for firmware manufacturers

are much more favourable this way.

Instead of having to maintain their own,

I don't know however many thousand lines of code base that have accreted over the years,

they can now fork Intel's TianoCore as their starting point

and just do their driver layer on top of that.

And this seems to have attracted essentially all of the BIOS manufacturers.

We should generally expect even that legacy BIOS layer

to go away soon, the direction that we hear from firmware people is all about that

and we need to cope with that.

Now, grub's core support for UEFI is basically fine,

it has more or less the set of drivers you'd expect

including relatively arcane things like serial support.

Of course the big thing that's been recently

is something called "Secure Boot",

that's a mechanism by which firmware makes sure to

only ever execute signed code, in a pre-boot context.

It's obviously very possible for this to go very wrong,

from the point of view of user freedom.

The FSF calls Secure Boot, "Restricted Boot" for that reason.

But a lot of systems now come with Secure Boot enabled by default,

so we have to figure out how to work with it

in the same way that we've had to figure out

how to work with all kinds of devices over the years,

just as a hardware enablement matter.

But the important thing is to work out how to that

in ways that don't end up impinging on our user's freedom.

and that's been quite a difficult slog.

The community has managed to figure out schemes for this

that don't stop you modifying the operating system on your own computer,

which is the important thing.

We have this working on Ubuntu,

we talked about this at DebConf last year.

To get it working in Debian we need dak to sign

bootloader objects that we submit to it, with the Debian key.

I failed with pushing that forward so that

would be a great project for somebody to take up,

if I've missed it and somebody else is already working on that, brilliant!

I should also mention that there are some outstanding

issues with the layout of the EFI System Partition,

which is UEFI's place for putting things like bootloader code.

The spec proscribes how you're supposed to behave

on fixed disk versus removable disks,

we follow the spec but unfortunately some systems don't

and essentially require the removable layout.

Steve McIntyre's here at the back,

we've gone back and forth a bit on that.

We need some way to select the removable layout

and have that actually persist and ideally detect

that this is a problem in the first place,

so that we don't have to ask an incomprehensible question in the installer.

Now, more cheerily, a number of non-x86 architectures are in a pretty good state.

they get fairly limited testing at the moment.

I know that some people use them because occasionally

I get some bugs when things break but generally they work OK.

powerpc and sparc work fine, we sould probably consider

switching the default bootloader there at some point.

If you're a porter for those architectures,

please get in touch with me.

Some mips-type architectures would well too.

Ryan ?? lent me a Yeeloong laptop a little while back,

so that I could port the debian installer to it

and I found that grub was basically fine.

So I was able to build the installer on top of that.

It was a tenth of the work it might otherwise have been.

There's been a pretty pleasing trend towards having

new arches include a grub port very early on.

arm64, also powerpc64, has had grub working from very early in it's life.

This gives them a pretty full-feature loader

with not a lot of effort, so it actually

works out quite well for the ports now, I think.

The most recent arm64 port, I went back and looked at it,

it was about 2000 lines of patches to do the initial enablement.

And now that got to take the advantage of the arm port

that already existed and also of UEFI code,

so that helps a lot but still I think it

indicates a pretty porting friendly design.

OK, so if I haven't persuaded you already,

why do we default to grub on debian x86?

There are certainly other loaders,

there's the venerable LILO, which still largely works fine.

Syslinux and friends are actively maintained by some very smart people,

including folks who maintain the Linux boot protocol

and there's extlinux which is one of that family

which you can use in the same kind of way that you might use LILO.

There's yaboot and powerpc and so on and so forth.

You can even boot a kernel and an initramfs directly

from the UEFI environment, if you want to.

So some of those are some pretty good bootloaders,

they're almost all smaller and simpler than grub

and that appeals to a lot of people.

I do get the simplicity argument.

But I tend to argue that the result of that

is moving a good deal of complexity elsewhere.

If you have a bootloader that can only handle some setups,

the installer needs to know what those setups are.

It needs to forbid you from doing anything, that you won't be

able to boot, in your partitioner.

You end up with static /boot partitions scattered everywhere.

Having a bootloader that you can trust to handle almost anything

means that you don't have to think too hard when you are moving things around.

You can easily do things like having the bootloader notice

when the last boot failed and behave differently.

All that sort of thing is useful for a general purpose distro to be able to assume.

[Ben]: We used to support some bootloaders on mips,

but they couldn't load an initramfs, they could only load a kernel.

So for that reason the kernel configurations we used on mips

had to be restricted, they don't assume there is an initramfs

and you can't boot off filesystems other than ext2, 3 and 4.

[Colin]: That's CoLo, is it? Or cibyl?

[Ben]: I don't know. We have recently switched over,

but that has been a restriction up to and including wheezy.

[Colin]: Thanks. That's useful ammunition. [laughs]

I think that we do not yet have grub working on

all of the mips big endian architectures,

it's working on little endian,

but I think there are still some problems with big endian.

So we maybe can't save the world just yet,

but it might not be too far off. But thanks.

Plus Debian runs on lots of architectures,

generally we try as an architectural thing

to run the same software across different architectures so

that it all works kind of the same way.

We have the same interfaces, the same tools are available and so on.

And it makes a lot of sense to have that be the case with the bootloader too.

The installer would certainly be simplified by being able to

assume grub-mount is available on all architectures, for instance.

OK, so the important bit, this.

We've had a lot of people involved over the years

Too many to name, I've almost certainly missed some

Robert Millan, Felix Zielcke and Jordi Mallach

have done excellent jobs in Debian a few years ago.

And Vladimir Serbinenko does an excellent job as the upstream maintainer.

But as far as Debian maintenance goes,

it's mostly just me at the moment and has been

for the last couple of years and I can't do it all.

We'd probably have had grub 2 in 2.00

rather than 1.99 in wheezy,

if we had a bit more redundancy there.

I was kind of demotivated by the Secure Boot stuff at the time,

so ended up putting it off until too late

and that's exactly the kind of thing

where having more people involved is more helpful

there are lots of bugs, everybody's problem is critical for them,

quite understandably.

So keeping on top of the release critical list is absolute murder.

But there are a few specific problems I'd like to highlight,

that are more substantial and could really do with

somebody sitting down and thinking quite hard.

I won't go into too much detail,

as I've just had a sign from the video team that I'm low on time.

But you can ask me about any of them if you're interested.

I mentioned earlier that the system for generating

the config file has a bad case of second system effect

and really needs a third system

[laughter]

to make it easier to customise things.

All problems can be solved by another rewrite, right?

[laughter]

On MBR systems, we often have robustness problems

with grub not installing the boot code to the place

that your system actually boots from,

particularly when multiple disks are involved.

This tends to manifest as module loading failures

because what happens is, over time

the core image that you're actually using

that you didn't know about, becomes incompatible with new grub modules

because the ABI changes over time

and eventually becomes incompatible and can't boot anything

I think that needs some kind of overhaul somewhere,

though I'm not quite sure where.

There are the remaining bit of UEFI,

which I mentioned earlier.

There's getting signing sorted out in Debian,

getting a thing called MokManager,

which is part of a seperate project called shim,

nicely integrated to give users control over the whole signing process

which will let us require signed kernels under Secure Boot

and thus make Matthew Garrett want to kill me a little bit less

[laughter]

and there's the issues with EFI System Partition layout

There's some work to be done with upstream Xen

to layout effectively a new boot protocol for PV Grub 2

so we can have Xen guests install the bootloader

in a consistent place in the file system.

In fact, Ian Campbell recently sent me something that

might be useful for this, I didn't have time to read it.

and hopefully this will eventually let us kill off all

of the old strange things that live in Xen, to do this.

Finally, we should take much better advantage of grub's ports

and switch over to them on many more architectures than we have done to date.

I've just got onto questions, so excellent.

[Ian]: One of things I've found difficult getting to grips with grub 1

and grub 2 is that the documentation hasn't always been as good.

LILO has always been exceptional in that it came with

an extremely good document that described

its principles of operation and how to get it to do,

well, almost anything you might want it to.

Is that also something you're looking for help with?

[Colin]: Yes.

About a year or two ago,

I went through and tried to systematically documenting

all of the grub commands and overhauling some other things.

I got about half way through that,

before I ran out of time.

but one of the problems has been with grub 2

that the documentation got out of date during the rewrite

so nobody has an incentive to do it properly for their incremental changes.

So I think getting that into shape

would make it much easier to keep it in shape

It's better than it was in 2010, it's not where it should be.

So, yes.

Anybody else, anything from irc aswell?

[John]: This was a related question to your comment earlier

about how BIOSes are widely inconsistent about

how they choose which disk they're going to boot from.

Is there anything that grub could do to help us

with best practices for failover configurations?

So for example, let's say I have a server

and I want to install two disk that are mirrored,

using lvm or something else.

One of the challenges is, how do I configure grub

on both disk to be configured the right

way, despite all of the weirdness with BIOSes?

To have a reasonable easy to maintain setup,

so that where if one disk actually goes completely offline,

the system could automatically reboot.

[Colin]: One of the things I tried to do,

the first time I systematically

attacked this in Debian grub,

the thing I tried to do was to arrange that we

did as by-uuid install to disks, possibly as by-path, I forget.

So we encourage people to install to all of the disks on a system.

Also the grub-pc.postinst notices when a disk has gone away since it last checked,

and it's supposed to offer you the chance to update this.

But you're right it's a little bit fiddly to manage

particularly if you're in a system that's large enough

so that statistically disks are failing quite often

Then it's a problem.

I think what I'd probably like to do is to make sure that

we install to the disks that are associated with a RAID device or something

so you could just bootstrap off the RAID management.

I think some of that works but isn't hooked up very well in Debian.

But there are several things to do there.

I think there's also a bug in d-i where it doesn't install

to all of the fixed disks in your system but only one.

So there are several things to fix there.

There's a question over there.

[??]: I think my question could be summed up as:

"How tall do you have to be to ride this ride?"

[laughter]

It seems like getting started,

introducing regressions would be a really scary thing.

For someone who wanted to get started with the project

and I'm just wondering what kind of facilities do you have

for before we actually upload a version to

test across the broad range of architectures.

[Colin]: Right well, one of the points of this talk

was to try to emphasise that you don't actually have

to be as tall as people think you do.

But you're right about the need for regression testing.

One thing that really helps is that the different parts

of grub are much more isolated than they used to be.

So you can quite easily change one module,

test changes with that, without having to worry about

strange leakage all over the place.

But there are things that that doesn't cover,

there is some boot testing arrangement in the build system,

so there are some make targets that go off

and do boot checks of real systems against the current version of the code.

And that sort of thing helps a lot,

I would like to have grub hooked up better to some sort of autopkgtest arrangement

so that it does emulated boots across various types

of systems on ci.debian.net, I'd love help with that.

[Steve]: Now of course UEFI, there's a whole slew of things. Tom was asking

about MBR and a reliable fallback, of course. I'm not aware of any UEFI

implementation that actually does fallback for the system partition either.

So that is an utter trainwreck.

[Colin]: I suspect what you're,

maybe you're meant to use hardware RAID or something?

[Steve]: Clearly that's what you're expected to use and it's a mess.

We should definitely get together

and talk about the removable media path.

Again there isn't a good answer but let's try

and make it as not crap as we can.

[Colin]: Do you know of any way to detect that

system even if it's something like a blacklist of broken DMI decoders?

[Steve]: I think that's all we can do.

We had a system, as an example

for the people that may not have come across this,

there are firmware implementiatons out there

that straight after install may not actually recognise

that you've put grub into the path where you're meant to put it.

Once you've booted off a rescue system, however,

and done some not particularly well understood sequence of things,

including boot off a removable media path, reboot a couple more times

shuffle things around, spin twice clockwise...

[laughter]

suddenly they then will recognise that you've put grub there.

But god forbid, you ever want to update it again.

It's painful.

[Colin]: There are some exciting problems on Apple Macs,

that are a shadow of the same class of problem.

[Steve]: And of course Secure Boot,

kibi asked Steven about this a few days ago.

Of course, we've been talking about this for like 2 years.

I feel rubbish as well, that we haven't really got any further with it.

If someone really really wants to help get involved with that, oh absolutely,

there's a whole load of people who would love to see you help.

[Colin]: Not only are there are a whole load of people who would like to see you help,

but there's actually roughly a roadmap of what needs to be done.

You don't have to blaze a trail.

[??]: Scrape the names off the roadmap and put your name on it.

[laughter]

[Colin]: It's fine, we don't really care, we just want to see it done!

The things that need to be done are, as I say,

the next step I think is in dak and then

somebody needs to chase up getting shim into debian

with the right kind of signatures on it and so on

but none of it is actually fundamentally hard,

it just requires time to push all of that forward.

[Steve]: Exactly. I'll pass onto anybody else.

[video team]: Actually, we're out of time.

[Colin]: OK, thank you all very much.

[applause]