WEBVTT

99:59:59.999 --> 99:59:59.999
Today, I talk about FAI.me, which is a
build for images.

99:59:59.999 --> 99:59:59.999
First, anybody that never heard anything
about FAI?

99:59:59.999 --> 99:59:59.999
Ok

99:59:59.999 --> 99:59:59.999
I started this project in 1999.

99:59:59.999 --> 99:59:59.999
I'm not sure…

99:59:59.999 --> 99:59:59.999
No, I'm sure that during those times, the
Debian installer did not have

99:59:59.999 --> 99:59:59.999
the preseeding stuff, so we needed
something automatically.

99:59:59.999 --> 99:59:59.999
I installed the first cluster with FAI and
I always do talks on FAI or

99:59:59.999 --> 99:59:59.999
today in the lightning talks, I talk
a little bit about dracut,

99:59:59.999 --> 99:59:59.999
which is used in FAI.

99:59:59.999 --> 99:59:59.999
So, what was the motivation.

99:59:59.999 --> 99:59:59.999
A neighbour of mine, she came to me with

99:59:59.999 --> 99:59:59.999
"My Windows desktop is broken,
can you reinstall it?"

99:59:59.999 --> 99:59:59.999
And in the end, I installed her Linux,
and I was shortly thinking about

99:59:59.999 --> 99:59:59.999
"Should I use FAI for installing her
desktop with Linux?"

99:59:59.999 --> 99:59:59.999
And in the end, I did not use it because
FAI is too complicated,

99:59:59.999 --> 99:59:59.999
like the Debian installer, I guess it's
not really that easy for beginners

99:59:59.999 --> 99:59:59.999
because there are a lot of questions

99:59:59.999 --> 99:59:59.999
but also FAI is not really for beginners.

99:59:59.999 --> 99:59:59.999
So this was the motivation about thinking
about FAI.

99:59:59.999 --> 99:59:59.999
The target group was always advanced
sysadmins

99:59:59.999 --> 99:59:59.999
but I thought maybe it's possible to make
FAI usable also for people

99:59:59.999 --> 99:59:59.999
that are not that advanced sysadmins.

99:59:59.999 --> 99:59:59.999
The idea is that an installer should cover
most installations.

99:59:59.999 --> 99:59:59.999
The Debian installer is really perfect
because I think it covers

99:59:59.999 --> 99:59:59.999
all different kinds and strange environments

99:59:59.999 --> 99:59:59.999
You can do a lot of things, you can configure
very strange combination of language,

99:59:59.999 --> 99:59:59.999
keyboard layout and so on

99:59:59.999 --> 99:59:59.999
but I was thinking about an installer
that covers 90 or 95% of the installations

99:59:59.999 --> 99:59:59.999
A lot of special cases can be ignored and
since the Debian installer has like

99:59:59.999 --> 99:59:59.999
more than 20 questions, I thought it would
be much nicer if there were only

99:59:59.999 --> 99:59:59.999
3 to 5 questions and I looked at Linux Mint
and Mageia installers, CentOS installer,

99:59:59.999 --> 99:59:59.999
and they all ask much less questions.

99:59:59.999 --> 99:59:59.999
In the Debian installer, we sometimes
have also things that are asked

99:59:59.999 --> 99:59:59.999
during the installation, so not everything
is asked at the very beginning.

99:59:59.999 --> 99:59:59.999
For example, the task selection, where you
select your desktop,

99:59:59.999 --> 99:59:59.999
is done after the base installation.

99:59:59.999 --> 99:59:59.999
This was also very important, I would like
to have something that

99:59:59.999 --> 99:59:59.999
asks everything at the very beginning.

99:59:59.999 --> 99:59:59.999
Then, maybe some tool could create
a customized installation image

99:59:59.999 --> 99:59:59.999
and this installation image should run
then completely unattended

99:59:59.999 --> 99:59:59.999
so you can get yourself a coffee and
when you come back, your machine is ready.

99:59:59.999 --> 99:59:59.999
There are 3 things to customize installation
image,

99:59:59.999 --> 99:59:59.999
you just put this image, you do not have
to touch anything, and then it's ready.

99:59:59.999 --> 99:59:59.999
I thought "Oh yes, this is FAI, maybe
FAI can do this."

99:59:59.999 --> 99:59:59.999
As I said, FAI is only, or was until now
only a tool for experienced sysadmins

99:59:59.999 --> 99:59:59.999
and you have to adjust several config
files, these are ASCII files

99:59:59.999 --> 99:59:59.999
but still you have to touch 5 to 10
config files to make a customization.

99:59:59.999 --> 99:59:59.999
So, how can I make FAI usable for
beginners?

99:59:59.999 --> 99:59:59.999
That's the beginning of FAI.me.

99:59:59.999 --> 99:59:59.999
There's a web page, we'll show it
in more detail later,

99:59:59.999 --> 99:59:59.999
where you can just click some things, and
then you get a customized image.

99:59:59.999 --> 99:59:59.999
This image can be put onto a CD, DVD or
USB stick, just with dd

99:59:59.999 --> 99:59:59.999
and the customization is just by using
the web interface

99:59:59.999 --> 99:59:59.999
so there's no need for you to edit
a text file, a config file inside FAI.

99:59:59.999 --> 99:59:59.999
I hope I covered most important thangs
that you want to adjust

99:59:59.999 --> 99:59:59.999
or a little bit customize.

99:59:59.999 --> 99:59:59.999
You can add additional packages, I think
that's the most important thing

99:59:59.999 --> 99:59:59.999
that people say "I want to have the normal
Debian installation

99:59:59.999 --> 99:59:59.999
but with some additional packages."

99:59:59.999 --> 99:59:59.999
And you can select different different
distributions, so it's not only

99:59:59.999 --> 99:59:59.999
the installation image for the stable
release, you can create

99:59:59.999 --> 99:59:59.999
3 variants of the installation.

99:59:59.999 --> 99:59:59.999
This is the web page and thanks to Juri,
he did a great job

99:59:59.999 --> 99:59:59.999
during the first and second day, he added
a new feature that we now have

99:59:59.999 --> 99:59:59.999
a toggle button.

99:59:59.999 --> 99:59:59.999
Is it big enough or should I zoom in?

99:59:59.999 --> 99:59:59.999
Ok.

99:59:59.999 --> 99:59:59.999
So, we have a toggle button, what you see
now is just the bare minimum or questions

99:59:59.999 --> 99:59:59.999
and we can toggle it to more advanced
settings.

99:59:59.999 --> 99:59:59.999
You have to select or just leave this as
it is, username,

99:59:59.999 --> 99:59:59.999
if you do not enter a password, a password
will be generated and shown to you

99:59:59.999 --> 99:59:59.999
and sent by e-mail.

99:59:59.999 --> 99:59:59.999
I will now just type in the password.

99:59:59.999 --> 99:59:59.999
It's here in clear text, for me that's fine
because

99:59:59.999 --> 99:59:59.999
there's also a comment that you should
change the password after the installation

99:59:59.999 --> 99:59:59.999
and I do not like to enter passwords twice
so you can see what you typed in

99:59:59.999 --> 99:59:59.999
and hopefully do not make any wrong
mistakes.

99:59:59.999 --> 99:59:59.999
For example, we could select the Stretch
distribution with backports,

99:59:59.999 --> 99:59:59.999
so we will get a 4.15 kernel with Stretch.

99:59:59.999 --> 99:59:59.999
There are some buttons we can say we want
to have some Debian developer tools.

99:59:59.999 --> 99:59:59.999
This is what I defined in the FAI
configuration, so just a list of packages.

99:59:59.999 --> 99:59:59.999
Here, you can enter you own packages.


99:59:59.999 --> 99:59:59.999
I will select the desktop.

99:59:59.999 --> 99:59:59.999
You can have an installation without any
desktop, so a very small installation.

99:59:59.999 --> 99:59:59.999
I will select the XFCE desktop, but all
the other desktops are here.

99:59:59.999 --> 99:59:59.999
The language, these are just task packages
that are…

99:59:59.999 --> 99:59:59.999
I think Debian has much more task packages,
I just searched which are

99:59:59.999 --> 99:59:59.999
the most common languages, and what I do
if I say I want the spanish language,

99:59:59.999 --> 99:59:59.999
also the keyboard layout is spanish.

99:59:59.999 --> 99:59:59.999
I know there are different combinations
and with local time,

99:59:59.999 --> 99:59:59.999
it's getting more difficult.

99:59:59.999 --> 99:59:59.999
This installation will install the clock
with UTC, so if you want to set

99:59:59.999 --> 99:59:59.999
your time, you have to do this manually.

99:59:59.999 --> 99:59:59.999
I want to cover the most common installations.

99:59:59.999 --> 99:59:59.999
We select english US, the desktop and,
as an example, the midnight commander

99:59:59.999 --> 99:59:59.999
and GIMP.

99:59:59.999 --> 99:59:59.999
I can add an email address so if it would
take longer,

99:59:59.999 --> 99:59:59.999
for example if this service will have
success and a lot of people are using it,

99:59:59.999 --> 99:59:59.999
you may wait for some minutes so your job
will be finished.

99:59:59.999 --> 99:59:59.999
So here are the comments, how to reconfigure
the keyboard or the timezone

99:59:59.999 --> 99:59:59.999
and then you just click "Create
the installation image".

99:59:59.999 --> 99:59:59.999
Now, in the background, there's some job,
a script, looking "Oh, there's a new job"

99:59:59.999 --> 99:59:59.999
and there's a summary of the configuration,
of the web configuration.

99:59:59.999 --> 99:59:59.999
Down here you see these are the
FAI classes,

99:59:59.999 --> 99:59:59.999
I will explain a little bit more about this.

99:59:59.999 --> 99:59:59.999
But with this information, FAI configuration
is generated,

99:59:59.999 --> 99:59:59.999
that's what normally the experienced
sysadmins have to create

99:59:59.999 --> 99:59:59.999
but here you just click on some buttons
and it will be done for you.

99:59:59.999 --> 99:59:59.999
In the meantime, we have some more
advanced features

99:59:59.999 --> 99:59:59.999
which I will also show you later.

99:59:59.999 --> 99:59:59.999
For example, this very simple installation
just creates one partition

99:59:59.999 --> 99:59:59.999
but you can also select that you want
to have a separate /home partition

99:59:59.999 --> 99:59:59.999
or using lvm just by selecting this
on the web interface.

99:59:59.999 --> 99:59:59.999
You can also add your SSH public key
for logging as root without a password

99:59:59.999 --> 99:59:59.999
or what's very nice, I found the new
Ubuntu installer does this,

99:59:59.999 --> 99:59:59.999
you can give your github account and
then there's a comment which

99:59:59.999 --> 99:59:59.999
receives the public key from your
github account and puts it

99:59:59.999 --> 99:59:59.999
into the root account so you can log in
without password.

99:59:59.999 --> 99:59:59.999
I think that's very neat.

99:59:59.999 --> 99:59:59.999
And if you have a repository with your own
packages, you could also add this and say

99:59:59.999 --> 99:59:59.999
"Please install those packages from
my publicly available repository."

99:59:59.999 --> 99:59:59.999
Let's see.

99:59:59.999 --> 99:59:59.999
As we see, this job finished in 74 seconds.

99:59:59.999 --> 99:59:59.999
Now, this customised installation image
is available for download.

99:59:59.999 --> 99:59:59.999
You can also download the log file.

99:59:59.999 --> 99:59:59.999
Since this is an installation image,
I first have to create

99:59:59.999 --> 99:59:59.999
a partial package mirror.

99:59:59.999 --> 99:59:59.999
This is done by the command 'fai-mirror'
and you can also read the log of

99:59:59.999 --> 99:59:59.999
this call of the fai-mirror, where a list
of all you packages with all the dependencies

99:59:59.999 --> 99:59:59.999
are available.

99:59:59.999 --> 99:59:59.999
So you see, these are the list of packages
and later they are downloaded

99:59:59.999 --> 99:59:59.999
and in the end, it says it created a mirror
of 1G of packages

99:59:59.999 --> 99:59:59.999
and since I have a local mirror,
it's very fast.

99:59:59.999 --> 99:59:59.999
This is the one part on the installation
image partial mirror with all the packages

99:59:59.999 --> 99:59:59.999
and the other is that the config space
which you can also download.

99:59:59.999 --> 99:59:59.999
So this is the config space that was
really created for you

99:59:59.999 --> 99:59:59.999
by clicking the web interface.

99:59:59.999 --> 99:59:59.999
If you want to do more things with FAI,
you can set up your own FAI server

99:59:59.999 --> 99:59:59.999
and use this configuration space.

99:59:59.999 --> 99:59:59.999
And, that's also very new, the two commands
that are used for creating this ISO image

99:59:59.999 --> 99:59:59.999
are now also listed there.

99:59:59.999 --> 99:59:59.999
First, create the partial mirror and then
create the installation image.

99:59:59.999 --> 99:59:59.999
Ok, copy link location…

99:59:59.999 --> 99:59:59.999
Let's see how good the network is here.

99:59:59.999 --> 99:59:59.999
[Q] It's a rather large image.

99:59:59.999 --> 99:59:59.999
Yeah, because it includes all the packages
and with Xfce, LibreOffice and so on

99:59:59.999 --> 99:59:59.999
and the installation environment is maybe
about 200MB.

99:59:59.999 --> 99:59:59.999
That's not much bigger than the Debian
installer that you need to download.

99:59:59.999 --> 99:59:59.999
So, 2, 1, done.

99:59:59.999 --> 99:59:59.999
I have a little wrapper which calls
a fresh kvm machine

99:59:59.999 --> 99:59:59.999
with an empty disk and boots this ISO image

99:59:59.999 --> 99:59:59.999
and then we will see how this installation
runs.

99:59:59.999 --> 99:59:59.999
So this is dracut booting the image

99:59:59.999 --> 99:59:59.999
and now you see there are already
some parted commands executed

99:59:59.999 --> 99:59:59.999
and now the packages are installed
and everything runs on

99:59:59.999 --> 99:59:59.999
and in the end some customization
script.

99:59:59.999 --> 99:59:59.999
We use only shell scripts for doing
some customizations

99:59:59.999 --> 99:59:59.999
and you see the files are downloaded
from /media/mirror

99:59:59.999 --> 99:59:59.999
so this is local on the ISO image.

99:59:59.999 --> 99:59:59.999
It would also be possible to create
an image without the packages

99:59:59.999 --> 99:59:59.999
and then give another sources.list file
so the packages would be downloaded

99:59:59.999 --> 99:59:59.999
from the internet but this default
in the FAI service,

99:59:59.999 --> 99:59:59.999
we put everything onto the ISO image.

99:59:59.999 --> 99:59:59.999
I guess it will run for 4 minutes.

99:59:59.999 --> 99:59:59.999
[Q] ???

99:59:59.999 --> 99:59:59.999
Yeah.

99:59:59.999 --> 99:59:59.999
What I will show you now is…

99:59:59.999 --> 99:59:59.999
So, this was the simple one, now I toggle
this web page and

99:59:59.999 --> 99:59:59.999
you will see that there are some more
questions you may answer,

99:59:59.999 --> 99:59:59.999
for example you can give a root password.

99:59:59.999 --> 99:59:59.999
If you leave this empty, sudo will be
configured.

99:59:59.999 --> 99:59:59.999
Here you can upload the SSH key or give
you Github account,

99:59:59.999 --> 99:59:59.999
that would be Mrfai for me.

99:59:59.999 --> 99:59:59.999
With the partitioning schemes, we have
one partition

99:59:59.999 --> 99:59:59.999
or one partition and /home separated
or these two versions with LVM.

99:59:59.999 --> 99:59:59.999
FAI itself can do much more, we could do
soft raid set ups, cryptsetup

99:59:59.999 --> 99:59:59.999
but here I want to cover the most common
installation, so very simple

99:59:59.999 --> 99:59:59.999
we have only 4 things that you can choose.

99:59:59.999 --> 99:59:59.999
[Q] For encryption?

99:59:59.999 --> 99:59:59.999
[A] Yes.

99:59:59.999 --> 99:59:59.999
So, this was the partitioning things.

99:59:59.999 --> 99:59:59.999
This is the new feature where you can add
an URL for your local package repository

99:59:59.999 --> 99:59:59.999
and the rest is the same, you can add
packages you like, your email address

99:59:59.999 --> 99:59:59.999
and then also create an installation image.

99:59:59.999 --> 99:59:59.999
I normally set by default, I include
the nonfree linux firmware.

99:59:59.999 --> 99:59:59.999
This is because my target audience is
an end user and I want to make it

99:59:59.999 --> 99:59:59.999
very comfortable for them, so yeah,
they can just install it

99:59:59.999 --> 99:59:59.999
and do not have these problems.

99:59:59.999 --> 99:59:59.999
And since this is not on an officiel
debian.org web site,

99:59:59.999 --> 99:59:59.999
I can do this with this default.

99:59:59.999 --> 99:59:59.999
Let's see, the installation is still running.

99:59:59.999 --> 99:59:59.999
So, advanced features.

99:59:59.999 --> 99:59:59.999
The next thing after this installation,

99:59:59.999 --> 99:59:59.999
I will show you how to create cloud
images.

99:59:59.999 --> 99:59:59.999
Currently, we create an installation image,

99:59:59.999 --> 99:59:59.999
when you boot it, the installation is run
fully automatically.

99:59:59.999 --> 99:59:59.999
The other type of service FAI.me gives
to you is that it creates a raw image

99:59:59.999 --> 99:59:59.999
or some other formats as you see here,
qcow2 and whatever

99:59:59.999 --> 99:59:59.999
which you can just boot and
the installation is already done.

99:59:59.999 --> 99:59:59.999
But first, see if the installation finish.

99:59:59.999 --> 99:59:59.999
Ah! Ok.

99:59:59.999 --> 99:59:59.999
These are now the shell scripts that are
executed for the customization

99:59:59.999 --> 99:59:59.999
of like /etc/messages of today,
/etc/network/interfaces is written

99:59:59.999 --> 99:59:59.999
and so on.

99:59:59.999 --> 99:59:59.999
You see the installation took 236s, it says
there are some errors

99:59:59.999 --> 99:59:59.999
but that's not really true.

99:59:59.999 --> 99:59:59.999
And it stops here, but we can also
disable this, this is only for showing

99:59:59.999 --> 99:59:59.999
everything went well

99:59:59.999 --> 99:59:59.999
and now we just reboot the machine.

99:59:59.999 --> 99:59:59.999
You see the grub.

99:59:59.999 --> 99:59:59.999
Ok, Xfce desktop.

99:59:59.999 --> 99:59:59.999
debian was a user with password FAI.

99:59:59.999 --> 99:59:59.999
We have "uname -a", this is 4.16,
the backports kernel was installed there.

99:59:59.999 --> 99:59:59.999
We have only one partition, no LVM

99:59:59.999 --> 99:59:59.999
and I told it to install gimp which is
not installed by default.

99:59:59.999 --> 99:59:59.999
Gimp is there, so this is nice.

99:59:59.999 --> 99:59:59.999
And the midnight commander is also there.

99:59:59.999 --> 99:59:59.999
And now we just throw this machine.
Gone.

99:59:59.999 --> 99:59:59.999
What's very nice with this wrapper script,
it creates the local disk

99:59:59.999 --> 99:59:59.999
of the virtual machine in /tmp which is
a RAM disk and I love RAM,

99:59:59.999 --> 99:59:59.999
it's so nice and fast.

99:59:59.999 --> 99:59:59.999
So, this was installation image and now we
look at the cloud image.

99:59:59.999 --> 99:59:59.999
First, you can say how big should your
disk image be.

99:59:59.999 --> 99:59:59.999
Here, I say 8GB, you will see it's not
an 8GB image

99:59:59.999 --> 99:59:59.999
that you have to download later.

99:59:59.999 --> 99:59:59.999
By default, I use zstd compression.

99:59:59.999 --> 99:59:59.999
Anyone who dose not know
this compression?

99:59:59.999 --> 99:59:59.999
This is very fast, very new, created
by Facebook if I'm correct.

99:59:59.999 --> 99:59:59.999
It's for very big files and what you should
never use is gzip with sparse images.

99:59:59.999 --> 99:59:59.999
The disk image is sparse and gzip
cannot handle this

99:59:59.999 --> 99:59:59.999
so if you compress it and uncompress it
it will be very large

99:59:59.999 --> 99:59:59.999
and all the other, xz, zstd, can handle
sparse files very nicely.

99:59:59.999 --> 99:59:59.999
So, the hostname is set, the root password,
username with a password.

99:59:59.999 --> 99:59:59.999
Now we want to install Buster.

99:59:59.999 --> 99:59:59.999
Maybe with no…

99:59:59.999 --> 99:59:59.999
Oh, we also do the Xfce desktop.

99:59:59.999 --> 99:59:59.999
Any packages you'd like to have in this
cloud image.

99:59:59.999 --> 99:59:59.999
"desktop" and "cloud" image does not
make that much sense, maybe.

99:59:59.999 --> 99:59:59.999
Emacs25, ok.

99:59:59.999 --> 99:59:59.999
And now "Create disk image".

99:59:59.999 --> 99:59:59.999
This will take a little bit longer because
we are doing the installation

99:59:59.999 --> 99:59:59.999
inside a file image.

99:59:59.999 --> 99:59:59.999
But no problem, I can tell you what
other ideas I have.

99:59:59.999 --> 99:59:59.999
So, currently we have the installation and
the cloud or virtual machine images for amd64

99:59:59.999 --> 99:59:59.999
FAI itself can also do cross-architecture
images so it would be some work

99:59:59.999 --> 99:59:59.999
to extend the web page to say
"Please create an arm64 image"

99:59:59.999 --> 99:59:59.999
It would be very nice to have predefined
configs for raspberry pi or

99:59:59.999 --> 99:59:59.999
all the very different boards

99:59:59.999 --> 99:59:59.999
but that would also be possible.

99:59:59.999 --> 99:59:59.999
I guess the next thing I will implement
is other distributions

99:59:59.999 --> 99:59:59.999
because I know people are always asking it.

99:59:59.999 --> 99:59:59.999
Not you but the Ubuntu guys.

99:59:59.999 --> 99:59:59.999
Yesterday I did the first test with Ubuntu
bionic, the LTS release

99:59:59.999 --> 99:59:59.999
and FAI just works out of the box with it.

99:59:59.999 --> 99:59:59.999
So what I have to do is to integrate it
in these FAI.me processing scripts.

99:59:59.999 --> 99:59:59.999
Ready-to-go cloud images for the big
cloud providers.

99:59:59.999 --> 99:59:59.999
That's only a different FAI config space
that I have to use.

99:59:59.999 --> 99:59:59.999
Currently, for example, in what I call
cloud images, I do not install

99:59:59.999 --> 99:59:59.999
the package cloud image.

99:59:59.999 --> 99:59:59.999
That's needed for all the ones.

99:59:59.999 --> 99:59:59.999
I'm also working in the Debian cloud team
and this team decided 2 years ago

99:59:59.999 --> 99:59:59.999
that the tool chain in the future for
the official Debian package will be FAI.

99:59:59.999 --> 99:59:59.999
Amazon is already using it, so if you
boot or if you use a Debian cloud image

99:59:59.999 --> 99:59:59.999
in Amazon, Noah Meyerhans did this and
he's using the FAI tool chain for it.

99:59:59.999 --> 99:59:59.999
Google is not yet using because there was
a very small problem

99:59:59.999 --> 99:59:59.999
in a config file we had one space too much

99:59:59.999 --> 99:59:59.999
which caused grub to hang forever

99:59:59.999 --> 99:59:59.999
and that was the reason why they decided
for Stretch to use their auto tool chain.

99:59:59.999 --> 99:59:59.999
But the things are working so we have
the config space also for Google.

99:59:59.999 --> 99:59:59.999
And also for Azure, some people from
Credativ did this.

99:59:59.999 --> 99:59:59.999
The Debian cloud team already has
the FAI configuration for

99:59:59.999 --> 99:59:59.999
the big tool providers,
cloud providers.

99:59:59.999 --> 99:59:59.999
We could also think on a more generic
FAI installation image.

99:59:59.999 --> 99:59:59.999
It's an image that you would boot up
and then enter your job id of the web page

99:59:59.999 --> 99:59:59.999
and then the configuration would be
downloaded

99:59:59.999 --> 99:59:59.999
and the packages would be just
received from the internet.

99:59:59.999 --> 99:59:59.999
That was one…

99:59:59.999 --> 99:59:59.999
So, the image would be much smaller
because the packages do not need to be

99:59:59.999 --> 99:59:59.999
on the installation image.

99:59:59.999 --> 99:59:59.999
It's also possible to create live images
with FAI.

99:59:59.999 --> 99:59:59.999
It is a little bit more…

99:59:59.999 --> 99:59:59.999
Currently, you need some manual work
but that should be also possible

99:59:59.999 --> 99:59:59.999
to use FAI for creating live image
and then also to provide this

99:59:59.999 --> 99:59:59.999
on the FAI.me web service.

99:59:59.999 --> 99:59:59.999
If you want to customize much more
inside the image, you just say

99:59:59.999 --> 99:59:59.999
"Oh, I have some ansible scripts that
I want to execute at the very end"

99:59:59.999 --> 99:59:59.999
then I say "Ok, this is just a starting
point, use the FAI.me service

99:59:59.999 --> 99:59:59.999
and if you're happy with the FAI tools,
then set up your own FAI server,

99:59:59.999 --> 99:59:59.999
create your own configuration space
and then you can do all the crazy things."

99:59:59.999 --> 99:59:59.999
So, how does FAI.me work internally.

99:59:59.999 --> 99:59:59.999
We have a web server where there are
some CGI scripts and

99:59:59.999 --> 99:59:59.999
this is not the build server, so on the
web server, you click "Submit"

99:59:59.999 --> 99:59:59.999
"Create my image", all the input
is validated so you cannot make nasty things

99:59:59.999 --> 99:59:59.999
and then the CGI writes or creates
a subdirectory and puts files in it,

99:59:59.999 --> 99:59:59.999
a config and a meta file

99:59:59.999 --> 99:59:59.999
and writes a status
"waiting for processing".

99:59:59.999 --> 99:59:59.999
Then, the other server, the build server
reads this config and

99:59:59.999 --> 99:59:59.999
this is just an NFS mounted directory,
and sees

99:59:59.999 --> 99:59:59.999
"Oh there's a new job I have to process".

99:59:59.999 --> 99:59:59.999
In this processing script we pass for
some errors.

99:59:59.999 --> 99:59:59.999
What's happening very often that people
type in a package that's not available

99:59:59.999 --> 99:59:59.999
and this will be detected and then a new
version of the web page will pop up and say

99:59:59.999 --> 99:59:59.999
"Oh, when creating the package mirror,
there was an error

99:59:59.999 --> 99:59:59.999
because this package was not known."

99:59:59.999 --> 99:59:59.999
Sometimes I have to…

99:59:59.999 --> 99:59:59.999
Every night, I create new nfsroots
for Buster.

99:59:59.999 --> 99:59:59.999
If there are security updates, I have
to create new nfsroots

99:59:59.999 --> 99:59:59.999
for Stretch and backports.

99:59:59.999 --> 99:59:59.999
I have some cleanup, so if a lot of jobs
are created,

99:59:59.999 --> 99:59:59.999
the images are on the disk after,

99:59:59.999 --> 99:59:59.999
normally I say after one day I just
remove the images

99:59:59.999 --> 99:59:59.999
so you have one day to download
the images.

99:59:59.999 --> 99:59:59.999
There's 3 different configurations

99:59:59.999 --> 99:59:59.999
/etc/fai-stretch, /etc/buster,
/etc/fai-stretch-bpo (backports)

99:59:59.999 --> 99:59:59.999
We need for the installation image

99:59:59.999 --> 99:59:59.999
We need a different nfsroot, but
the config space that is shared

99:59:59.999 --> 99:59:59.999
about all configurations,

99:59:59.999 --> 99:59:59.999
so it doesn't matter if I install

99:59:59.999 --> 99:59:59.999
Stretch or Stretch backports or
Buster,

99:59:59.999 --> 99:59:59.999
I can use the same FAI configuration.

99:59:59.999 --> 99:59:59.999
Also, for building the cloud images,
I use the same FAI configuration.

99:59:59.999 --> 99:59:59.999
A new job is detected, then a copy of
the configuration space will be made

99:59:59.999 --> 99:59:59.999
and it will be customized a little bit.

99:59:59.999 --> 99:59:59.999
So there are a very very few changes,

99:59:59.999 --> 99:59:59.999
for example I have to put the SSH key
into your customized configuration space

99:59:59.999 --> 99:59:59.999
or the list of packages or the user and
root password.

99:59:59.999 --> 99:59:59.999
Then we have two things, if we want to
create the installation image,

99:59:59.999 --> 99:59:59.999
I first have to create the partial package
mirror and then create the installation image

99:59:59.999 --> 99:59:59.999
For the cloud images, we do not need
the nfsroot, we just need

99:59:59.999 --> 99:59:59.999
the configuration space which is
customized a little bit

99:59:59.999 --> 99:59:59.999
and then we can just create the disk image

99:59:59.999 --> 99:59:59.999
so there's one step less compared to
creating the installation ISO.

99:59:59.999 --> 99:59:59.999
The status on the web page will be
updated, log files written

99:59:59.999 --> 99:59:59.999
and if the user said "Please send me
an email if my job is ready",

99:59:59.999 --> 99:59:59.999
this will also be sent to the user.

99:59:59.999 --> 99:59:59.999
The we have the ISO or the disk image
and this will be copied back

99:59:59.999 --> 99:59:59.999
to the web server where the user can then
download it.

99:59:59.999 --> 99:59:59.999
And since I have a lot of RAM
in this machine,

99:59:59.999 --> 99:59:59.999
everything is run in RAM, very very nice.

99:59:59.999 --> 99:59:59.999
As I said, we need an nfsroot,
a configuration space and FAI classes.

99:59:59.999 --> 99:59:59.999
This is a very central component in FAI

99:59:59.999 --> 99:59:59.999
and this is just a list of names.

99:59:59.999 --> 99:59:59.999
So in HOME_LVM, this is the class name,
the FAI class we describe

99:59:59.999 --> 99:59:59.999
and I think this is that example:

99:59:59.999 --> 99:59:59.999
HOME_LVM describes how to partition
the local hard disk.

99:59:59.999 --> 99:59:59.999
This is our very flexible tool where
we can do LVM, cryptsetups,

99:59:59.999 --> 99:59:59.999
software RAIDs and so on.

99:59:59.999 --> 99:59:59.999
But for the FAI.me service, I just created
4 different types of partitioning

99:59:59.999 --> 99:59:59.999
and this is the HOME_LVM example.

99:59:59.999 --> 99:59:59.999
So we have a list of classes and,
as I said,

99:59:59.999 --> 99:59:59.999
just two commands for the installation
image with a list of classes

99:59:59.999 --> 99:59:59.999
and for the cloud image, I have to say
how big should the disk image be,

99:59:59.999 --> 99:59:59.999
the list of classes and what's the target
file that should be created.

99:59:59.999 --> 99:59:59.999
Let's see if this is ready.

99:59:59.999 --> 99:59:59.999
Yes, it's ready.

99:59:59.999 --> 99:59:59.999
So…

99:59:59.999 --> 99:59:59.999
It's 1.1GB.

99:59:59.999 --> 99:59:59.999
Is this really the… oh yeah, raw.

99:59:59.999 --> 99:59:59.999
No problem, let's download it, it should
be fast.

99:59:59.999 --> 99:59:59.999
This is the normal architecture if you use
FAI in a client/server set up.

99:59:59.999 --> 99:59:59.999
You should just look on the left side
where you see

99:59:59.999 --> 99:59:59.999
you need the config space, an nfsroot
and a mirror

99:59:59.999 --> 99:59:59.999
and these parts will put onto the CD.

99:59:59.999 --> 99:59:59.999
If you ??? network installation thing,

99:59:59.999 --> 99:59:59.999
this is of things get from the server
to the client.

99:59:59.999 --> 99:59:59.999
For the software installation, we have
another subdirectory called package_config

99:59:59.999 --> 99:59:59.999
and there you also see several files
where the file name is a FAI class.

99:59:59.999 --> 99:59:59.999
Since in the FAI.me service every client
belongs to the class DEBIAN,

99:59:59.999 --> 99:59:59.999
it will install the packages that are listed
on the ???

99:59:59.999 --> 99:59:59.999
and here we have an other class, NONFREE

99:59:59.999 --> 99:59:59.999
These packages are only installed if
you also said

99:59:59.999 --> 99:59:59.999
"Please install the nonfree packages"

99:59:59.999 --> 99:59:59.999
and this is mapped to a FAI class
called NONFREE.

99:59:59.999 --> 99:59:59.999
And there's an other class for AMD64
and so on.

99:59:59.999 --> 99:59:59.999
Some references.

99:59:59.999 --> 99:59:59.999
In the past, it looked more like this
when I said

99:59:59.999 --> 99:59:59.999
"Oh, who's using FAI?" and during
the last month I collected some logos

99:59:59.999 --> 99:59:59.999
just because it's much nicer.

99:59:59.999 --> 99:59:59.999
Let's see if the download was ready.

99:59:59.999 --> 99:59:59.999
We unzstd the FAI.me image,
faime-013Z image

99:59:59.999 --> 99:59:59.999
On the web site, I said I want to have
a 8GB partition,

99:59:59.999 --> 99:59:59.999
so now let's see how big it is.

99:59:59.999 --> 99:59:59.999
The file is 8, but since it's a sparse file
it's only 3.5GB

99:59:59.999 --> 99:59:59.999
and the compressed was 1.1GB.

99:59:59.999 --> 99:59:59.999
Now I use my wrapper

99:59:59.999 --> 99:59:59.999
and I say "Boot from disk" and this is
the FAI.me raw image, disk image

99:59:59.999 --> 99:59:59.999
that should be booted up.

99:59:59.999 --> 99:59:59.999
That's it.

99:59:59.999 --> 99:59:59.999
debian/fai

99:59:59.999 --> 99:59:59.999
Let's see if emacs is installed, yes.

99:59:59.999 --> 99:59:59.999
Gimp is already there, hopefully, and
the blue midnight commander.

99:59:59.999 --> 99:59:59.999
Let's see.

99:59:59.999 --> 99:59:59.999
Questions.

99:59:59.999 --> 99:59:59.999
[Q] I'm using the preseed file for
the debian-installer,

99:59:59.999 --> 99:59:59.999
do you have a conversion between
your syntax and your configuration files

99:59:59.999 --> 99:59:59.999
and the preseed file or maybe can you add
a download button for the preseed file

99:59:59.999 --> 99:59:59.999
to your web site because I think it's
rather nice to have it displayed

99:59:59.999 --> 99:59:59.999
in web site first.

99:59:59.999 --> 99:59:59.999
[A] I'm not using the debian-installer.

99:59:59.999 --> 99:59:59.999
I use preseeding, yes, the debconf
preseeding for the normal packages

99:59:59.999 --> 99:59:59.999
you can do this also in FAI and it's
the same format

99:59:59.999 --> 99:59:59.999
you get with debconf-get-selections.

99:59:59.999 --> 99:59:59.999
And what you get is you can download
your own FAI config space

99:59:59.999 --> 99:59:59.999
and this includes all information you need
to set up,

99:59:59.999 --> 99:59:59.999
to do this mirror FAI CD or the FAI disk
image command.

99:59:59.999 --> 99:59:59.999
But you cannot convert this config into
a d-i preseeding or vice versa,

99:59:59.999 --> 99:59:59.999
that's not possible.

99:59:59.999 --> 99:59:59.999
Because for example, for the partitioning
part I do not like to create

99:59:59.999 --> 99:59:59.999
from my disk config partman preseeding
file.

99:59:59.999 --> 99:59:59.999
You can pay me a lot of money, I will
never do this.

99:59:59.999 --> 99:59:59.999
You know that the partman preseeding
is very ugly and very heavy.

99:59:59.999 --> 99:59:59.999
For other things, yes, selection of,
for example, the selection of the language

99:59:59.999 --> 99:59:59.999
These are the normal preseeding we use.

99:59:59.999 --> 99:59:59.999
And the list of packages, task selec…

99:59:59.999 --> 99:59:59.999
I think it's much easier to do this in
the FAI configuration than to create

99:59:59.999 --> 99:59:59.999
a debian-installer preseeding.

99:59:59.999 --> 99:59:59.999
And why use d-i if this works for you?

99:59:59.999 --> 99:59:59.999
[Q] d-i works as well for me.

99:59:59.999 --> 99:59:59.999
[A] Yes, then fine, use it.

99:59:59.999 --> 99:59:59.999
[Q] Hi Thomas. Thank you very much for
this new feature in the FAI project,

99:59:59.999 --> 99:59:59.999
it's very nice and I found very great that
you have the output of the commands

99:59:59.999 --> 99:59:59.999
that you used to create the ISO image or
the cloud file.

99:59:59.999 --> 99:59:59.999
A question that I have is, in which servers
are located the files that we create,

99:59:59.999 --> 99:59:59.999
the ISO or the cloud.

99:59:59.999 --> 99:59:59.999
Is it a server that you own host or…

99:59:59.999 --> 99:59:59.999
[A] Trust me.

99:59:59.999 --> 99:59:59.999
Currently,

99:59:59.999 --> 99:59:59.999
both the web server and the FAI.me
processing build server are run

99:59:59.999 --> 99:59:59.999
on two machines at the university where
I work as a system administrator

99:59:59.999 --> 99:59:59.999
so that's also where we have a very fast
connection.

99:59:59.999 --> 99:59:59.999
The CGI script and shell script that is
processing these jobs is currently

99:59:59.999 --> 99:59:59.999
not open source.

99:59:59.999 --> 99:59:59.999
There are plans to do this, I'm not sure
when.

99:59:59.999 --> 99:59:59.999
If you want to reproduce the things, you
have the config file and you can download

99:59:59.999 --> 99:59:59.999
the FAI software and use these one or two
commands to reproduce it.

99:59:59.999 --> 99:59:59.999
Some people said "Oh, very nice service,
I would like to set up in my company".

99:59:59.999 --> 99:59:59.999
Then please yes, contact me and…

99:59:59.999 --> 99:59:59.999
Currently there are no concrete plans
to make these background scripts open source

99:59:59.999 --> 99:59:59.999
but it will be in some future.

99:59:59.999 --> 99:59:59.999
But currently, you have to trust me as
you also have to trust the package maintainers

99:59:59.999 --> 99:59:59.999
that will be installed there.

99:59:59.999 --> 99:59:59.999
But you can verify it or say "I do not
trust Thomas but I will just grab

99:59:59.999 --> 99:59:59.999
the FAI config space and this on my own".

99:59:59.999 --> 99:59:59.999
[Q] Thank you.

99:59:59.999 --> 99:59:59.999
[Q] There's a question from the internet.

99:59:59.999 --> 99:59:59.999
Why not use a proper job queuing system
like grid engine or similar?

99:59:59.999 --> 99:59:59.999
[A] I'm using grid engine at work for
different things.

99:59:59.999 --> 99:59:59.999
It started as a very simple project, so
in the end it's just a loop which

99:59:59.999 --> 99:59:59.999
checks if there's new jobs on that.

99:59:59.999 --> 99:59:59.999
Currently, I do not process jobs
in parallel, currently there's no need for it

99:59:59.999 --> 99:59:59.999
If this project will be very successful,
yeah, I have to use a queuing system.

99:59:59.999 --> 99:59:59.999
It's, yeah, a very simple script.

99:59:59.999 --> 99:59:59.999
But it would be also possible with
a proper queuing system.

99:59:59.999 --> 99:59:59.999
More questions?

99:59:59.999 --> 99:59:59.999
[Q] I have a bunch of questions.

99:59:59.999 --> 99:59:59.999
First, what is it that you use for
partitioning?

99:59:59.999 --> 99:59:59.999
[A] I'm using a poll script that we wrote
several years ago in FAI

99:59:59.999 --> 99:59:59.999
and we defined this config file,
this package config

99:59:59.999 --> 99:59:59.999
and the Perl script parses this script
and then executes the parted and mkfs command

99:59:59.999 --> 99:59:59.999
which you can see in the log files, so if
you want to see what does FAI do

99:59:59.999 --> 99:59:59.999
after parsing this, which commands are
executed, you see everything

99:59:59.999 --> 99:59:59.999
on the log files.

99:59:59.999 --> 99:59:59.999
[Q] Right, but so you turn this text
into partitioning…

99:59:59.999 --> 99:59:59.999
[A] commands, yeah.

99:59:59.999 --> 99:59:59.999
[Q] But the text looks like this,
like with the spaces and everything.

99:59:59.999 --> 99:59:59.999
[A] You can use more or less spaces
or do you like,

99:59:59.999 --> 99:59:59.999
should I convert it to XML?

99:59:59.999 --> 99:59:59.999
[Q] Ok, then my next question is

99:59:59.999 --> 99:59:59.999
what are you using the nfsroot for when
you're generating the images?

99:59:59.999 --> 99:59:59.999
[A] The nfsroot is used only for the
installation image.

99:59:59.999 --> 99:59:59.999
When I do the installation, I need to boot
the machine as a diskless client,

99:59:59.999 --> 99:59:59.999
so it's just what the debian-installer
into RAM,

99:59:59.999 --> 99:59:59.999
you need a running Linux system.

99:59:59.999 --> 99:59:59.999
This is our nfsroot, on the installation
image.

99:59:59.999 --> 99:59:59.999
When you boot installation image, this
nfsroot with all the commands we need

99:59:59.999 --> 99:59:59.999
are started without using the local disk
and then we can do

99:59:59.999 --> 99:59:59.999
everything on the disk, /root and /target
and so on.

99:59:59.999 --> 99:59:59.999
The nfsroot is the system that is running
during the installation.

99:59:59.999 --> 99:59:59.999
[Q] Ok, but there's no need for this to be
nfs, it could be a

99:59:59.999 --> 99:59:59.999
[A] It's called nfsroot.

99:59:59.999 --> 99:59:59.999
This is very common that people call it
nfsroot

99:59:59.999 --> 99:59:59.999
and if you have this network installation
thing, it's really an nfsroot.

99:59:59.999 --> 99:59:59.999
But you're right.

99:59:59.999 --> 99:59:59.999
On the installation ISO, it's not nfs,
it's just a local file system, yes.

99:59:59.999 --> 99:59:59.999
[Q] Alright. So, I think it's my last
comment.

99:59:59.999 --> 99:59:59.999
You have the ISO from which you install
and when you install from the ISO

99:59:59.999 --> 99:59:59.999
you're installing then packages on the
machine, and then you have the image

99:59:59.999 --> 99:59:59.999
which is like a disk image that has
the packages already installed,

99:59:59.999 --> 99:59:59.999
so you skip the installing step.

99:59:59.999 --> 99:59:59.999
Have you thought about having
an intermediate thing

99:59:59.999 --> 99:59:59.999
where you download an image that
already has the packages installed?

99:59:59.999 --> 99:59:59.999
[A] That's also possible.

99:59:59.999 --> 99:59:59.999
When you do an installation, before you
can change root in the new system

99:59:59.999 --> 99:59:59.999
for adding packages, you have to call
debootstrap.

99:59:59.999 --> 99:59:59.999
What we do, we call debootstrap once and
create a tar file out of it.

99:59:59.999 --> 99:59:59.999
This is our minimal… in the former days
it was the floppy disk,

99:59:59.999 --> 99:59:59.999
our base tar.gz file,

99:59:59.999 --> 99:59:59.999
so you could exchange the minimal tar file
with whatever tar file you have.

99:59:59.999 --> 99:59:59.999
That's for example what we do if we install
Ubuntu.

99:59:59.999 --> 99:59:59.999
We boot the installation system which is
a Debian system

99:59:59.999 --> 99:59:59.999
and then create the local filesystem and
extract an Ubuntu base image

99:59:59.999 --> 99:59:59.999
and then we can change root into
the Ubunto or the same for CentOS and so on

99:59:59.999 --> 99:59:59.999
then we can change root into the other
Linux system and add packages there.

99:59:59.999 --> 99:59:59.999
If you have already a bigger image with
some more packages added there,

99:59:59.999 --> 99:59:59.999
it's very easy to say "Do not extract the
Debian Stretch image

99:59:59.999 --> 99:59:59.999
but use my image which also
includes other tools."

99:59:59.999 --> 99:59:59.999
And if you are fine with that, you can
just extract the tar file.

99:59:59.999 --> 99:59:59.999
[Q] Ok. Any more questions?

99:59:59.999 --> 99:59:59.999
[Q] The heading is in german.

99:59:59.999 --> 99:59:59.999
[A] What?

99:59:59.999 --> 99:59:59.999
[Q] The heading is in german.

99:59:59.999 --> 99:59:59.999
[A] Oh, because it's a copy of my german
slides.

99:59:59.999 --> 99:59:59.999
Thank you for this.

99:59:59.999 --> 99:59:59.999
And, what's also missing.

99:59:59.999 --> 99:59:59.999
The web page, where you can select german
or other languages,

99:59:59.999 --> 99:59:59.999
it would be nice if people are interested
to help translate them

99:59:59.999 --> 99:59:59.999
so that it's more easy for people that
do not speak english

99:59:59.999 --> 99:59:59.999
to use the website and create their own
installation image with their language.

99:59:59.999 --> 99:59:59.999
[Q] Someone on the stream said that
the fai.me web site is not yours

99:59:59.999 --> 99:59:59.999
and it's a hack thing, it's a scam, you go
there and get hacked.

99:59:59.999 --> 99:59:59.999
Do you have any plans to try to buy
the domain because it's pretty confusing.

99:59:59.999 --> 99:59:59.999
The first thing I would have done
by seeing that talk would have been

99:59:59.999 --> 99:59:59.999
to go to fai.me.

99:59:59.999 --> 99:59:59.999
[A] Yeah, I was thinking about which name
I should choose.

99:59:59.999 --> 99:59:59.999
I didn't check which web domains are free
and in then end I thought

99:59:59.999 --> 99:59:59.999
"Do I like to have a different domain name
for the service?"

99:59:59.999 --> 99:59:59.999
But since it's only a part of the FAI
project, I thought,

99:59:59.999 --> 99:59:59.999
and technically it was also easier just
to host it under a subdirectory

99:59:59.999 --> 99:59:59.999
and yeah, if people now grab the fai.me
domain and do other things with it, yeah.

99:59:59.999 --> 99:59:59.999
I think one question would be to use
a debian.net or maybe debian.org domain

99:59:59.999 --> 99:59:59.999
because people trust much more.

99:59:59.999 --> 99:59:59.999
I get a lot of comments "Oh this would
be very nice if this would be hosted

99:59:59.999 --> 99:59:59.999
on a Debian machine"

99:59:59.999 --> 99:59:59.999
but this would be much more complicated
because the DSA team

99:59:59.999 --> 99:59:59.999
has much more restriction what to execute
on their machines.

99:59:59.999 --> 99:59:59.999
currently, we need root access because
we mount some things

99:59:59.999 --> 99:59:59.999
and DSA would not give root access
on any Debian machine.

99:59:59.999 --> 99:59:59.999
That's the same problem we have
in the Debian cloud team

99:59:59.999 --> 99:59:59.999
where we want to create the official
images for the cloud providers

99:59:59.999 --> 99:59:59.999
where the Debian cloud team will also
not have root access

99:59:59.999 --> 99:59:59.999
and so there's much more work to get
empty virtual machines

99:59:59.999 --> 99:59:59.999
starting up, putting data into it,
creating the images,

99:59:59.999 --> 99:59:59.999
receiving them from inside the image.

99:59:59.999 --> 99:59:59.999
And since on those machines I have
root access, that's much easier for me.

99:59:59.999 --> 99:59:59.999
We are out of time, so thank you Thomas.

99:59:59.999 --> 99:59:59.999
[Applause]