WEBVTT 00:00:05.885 --> 00:00:12.871 Today, I talk about FAI.me, which is a build server for images. 00:00:14.947 --> 00:00:20.359 First, anybody that never heard anything about FAI? 00:00:21.781 --> 00:00:22.593 Ok 00:00:24.376 --> 00:00:27.468 I started this project in 1999. 00:00:29.297 --> 00:00:30.679 I'm not sure… 00:00:31.656 --> 00:00:35.883 No, I'm sure that during those times, the Debian installer did not have 00:00:35.883 --> 00:00:39.703 the preseeding stuff, so we needed something automatically. 00:00:42.751 --> 00:00:50.347 I installed the first cluster with FAI and I always do talks on FAI or 00:00:50.347 --> 00:00:54.211 today in the lightning talks, I talk a little bit about dracut, 00:00:54.211 --> 00:00:55.794 which is used in FAI. 00:00:56.964 --> 00:00:58.838 So, what was the motivation. 00:00:59.448 --> 00:01:02.780 A neighbour of mine, she came to me with 00:01:02.780 --> 00:01:08.109 "My Windows desktop is broken, can you reinstall it?" 00:01:08.395 --> 00:01:13.673 And in the end, I installed her Linux, and I was shortly thinking about 00:01:13.673 --> 00:01:18.223 "Should I use FAI for installing her desktop with Linux?" 00:01:18.995 --> 00:01:24.809 And in the end, I did not use it because FAI is too complicated, 00:01:24.809 --> 00:01:31.030 like the Debian installer, I guess it's not really that easy for beginners 00:01:31.030 --> 00:01:32.740 because there are a lot of questions 00:01:32.740 --> 00:01:34.973 but also FAI is not really for beginners. 00:01:35.785 --> 00:01:38.787 So this was the motivation about thinking about FAI. 00:01:40.255 --> 00:01:43.776 The target group was always advanced sysadmins 00:01:43.776 --> 00:01:49.313 but I thought maybe it's possible to make FAI usable also for people 00:01:49.313 --> 00:01:52.490 that are not that advanced sysadmins. 00:01:54.359 --> 00:01:58.790 The idea is that an installer should cover most installations. 00:01:59.075 --> 00:02:02.890 The Debian installer is really perfect because I think it covers 00:02:02.890 --> 00:02:07.114 all different kinds and strange environments 00:02:07.114 --> 00:02:14.146 You can do a lot of things, you can configure very strange combination of language, 00:02:14.146 --> 00:02:16.057 keyboard layout and so on 00:02:16.546 --> 00:02:23.068 but I was thinking about an installer that covers 90 or 95% of the installations 00:02:24.677 --> 00:02:32.092 A lot of special cases can be ignored and since the Debian installer has like 00:02:32.862 --> 00:02:38.345 more than 20 questions, I thought it would be much nicer if there were only 00:02:38.345 --> 00:02:46.155 3 to 5 questions and I looked at Linux Mint and Mageia installers, CentOS installer, 00:02:46.155 --> 00:02:50.181 and they all ask much less questions. 00:02:51.767 --> 00:02:57.408 In the Debian installer, we sometimes have also things that are asked 00:02:57.408 --> 00:03:01.512 during the installation, so not everything is asked at the very beginning. 00:03:02.079 --> 00:03:05.898 For example, the task selection, where you select your desktop, 00:03:05.898 --> 00:03:08.538 is done after the base installation. 00:03:08.904 --> 00:03:13.130 This was also very important, I would like to have something that 00:03:13.130 --> 00:03:15.366 asks everything at the very beginning. 00:03:16.584 --> 00:03:22.271 Then, maybe some tool could create a customized installation image 00:03:22.271 --> 00:03:27.149 and this installation image should run then completely unattended 00:03:27.149 --> 00:03:31.572 so you can get yourself a coffee and when you come back, your machine is ready. 00:03:32.792 --> 00:03:37.590 There are 3 things to customize installation image, 00:03:37.590 --> 00:03:43.440 you just put this image, you do not have to touch anything, and then it's ready. 00:03:43.603 --> 00:03:48.239 I thought "Oh yes, this is FAI, maybe FAI can do this." 00:03:50.028 --> 00:03:57.059 As I said, FAI is only, or was until now only a tool for experienced sysadmins 00:03:57.059 --> 00:04:02.297 and you have to adjust several config files, these are ASCII files 00:04:02.297 --> 00:04:08.677 but still you have to touch 5 to 10 config files to make a customization. 00:04:09.610 --> 00:04:12.698 So, how can I make FAI usable for beginners? 00:04:13.023 --> 00:04:15.294 That's the beginning of FAI.me. 00:04:17.647 --> 00:04:21.430 There's a web page, we'll show it in more detail later, 00:04:21.430 --> 00:04:26.831 where you can just click some things, and then you get a customized image. 00:04:28.815 --> 00:04:35.606 This image can be put onto a CD, DVD or USB stick, just with dd 00:04:35.606 --> 00:04:40.724 and the customization is just by using the web interface 00:04:40.724 --> 00:04:46.257 so there's no need for you to edit a text file, a config file inside FAI. 00:04:48.333 --> 00:04:53.454 I hope I covered most important things that you want to adjust 00:04:53.454 --> 00:04:55.852 or a little bit customize. 00:04:56.123 --> 00:04:59.733 You can add additional packages, I think that's the most important thing 00:04:59.733 --> 00:05:03.752 that people say "I want to have the normal Debian installation 00:05:03.752 --> 00:05:05.778 but with some additional packages." 00:05:06.836 --> 00:05:11.796 And you can select different distributions, so it's not only 00:05:11.796 --> 00:05:16.755 the installation image for the stable release, you can create 00:05:16.755 --> 00:05:19.190 3 variants of the installation. 00:05:21.553 --> 00:05:28.747 This is the web page and thanks to Juri, he did a great job 00:05:28.747 --> 00:05:34.316 during the first and second day, he added a new feature that we now have 00:05:34.316 --> 00:05:35.616 a toggle button. 00:05:36.388 --> 00:05:39.522 Is it big enough or should I zoom in? 00:05:40.823 --> 00:05:41.633 Ok. 00:05:42.688 --> 00:05:49.963 So, we have a toggle button, what you see now is just the bare minimum of questions 00:05:49.963 --> 00:05:54.111 and we can toggle it to more advanced settings. 00:05:54.876 --> 00:06:00.225 You have to select or just leave this as it is, username, 00:06:00.225 --> 00:06:06.120 if you do not enter a password, a password will be generated and shown to you 00:06:06.120 --> 00:06:07.706 and sent by e-mail. 00:06:08.435 --> 00:06:11.445 I will now just type in the password. 00:06:11.932 --> 00:06:15.019 It's here in clear text, for me that's fine because 00:06:15.019 --> 00:06:21.430 there's also a comment that you should change the password after the installation 00:06:21.430 --> 00:06:27.337 and I do not like to enter passwords twice so you can see what you typed in 00:06:27.337 --> 00:06:33.311 and hopefully do not make any wrong mistakes. 00:06:33.636 --> 00:06:38.311 For example, we could select the Stretch distribution with backports, 00:06:38.311 --> 00:06:42.208 so we will get a 4.15 kernel with Stretch. 00:06:43.221 --> 00:06:47.687 There are some buttons we can say we want to have some Debian developer tools. 00:06:48.246 --> 00:06:54.145 This is what I defined in the FAI configuration, so just a list of packages. 00:06:55.041 --> 00:06:59.219 Here, you can enter you own packages. 00:07:01.301 --> 00:07:04.548 I will select the desktop. 00:07:04.832 --> 00:07:09.340 You can have an installation without any desktop, so a very small installation. 00:07:10.963 --> 00:07:16.042 I will select the XFCE desktop, but all the other desktops are here. 00:07:16.368 --> 00:07:21.935 The language, these are just task packages that are… 00:07:22.343 --> 00:07:26.532 I think Debian has much more task packages, I just searched which are 00:07:26.532 --> 00:07:35.516 the most common languages, and what I do if I say I want the spanish language, 00:07:35.516 --> 00:07:37.986 also the keyboard layout is spanish. 00:07:39.254 --> 00:07:43.239 I know there are different combinations and with local time, 00:07:43.239 --> 00:07:47.071 it's getting more difficult. 00:07:47.999 --> 00:07:53.121 This installation will install the clock with UTC, so if you want to set 00:07:53.121 --> 00:07:55.315 your time, you have to do this manually. 00:07:55.751 --> 00:07:59.951 I want to cover the most common installations. 00:08:02.389 --> 00:08:11.372 We select english US, the desktop and, as an example, the midnight commander 00:08:11.372 --> 00:08:12.746 and GIMP. 00:08:13.242 --> 00:08:21.050 I can add an email address so if it would take longer, 00:08:21.050 --> 00:08:26.333 for example if this service will have success and a lot of people are using it, 00:08:26.333 --> 00:08:30.992 you may wait for some minutes so your job will be finished. 00:08:31.774 --> 00:08:37.990 So here are the comments, how to reconfigure the keyboard or the timezone 00:08:37.990 --> 00:08:41.568 and then you just click "Create the installation image". 00:08:42.138 --> 00:08:49.333 Now, in the background, there's some job, a script, looking "Oh, there's a new job" 00:08:49.333 --> 00:08:54.117 and there's a summary of the configuration, of the web configuration. 00:08:55.504 --> 00:08:59.164 Down here you see these are the FAI classes, 00:08:59.164 --> 00:09:01.849 I will explain a little bit more about this. 00:09:02.092 --> 00:09:06.552 But with this information, FAI configuration is generated, 00:09:06.884 --> 00:09:10.179 that's what normally the experienced sysadmins have to create 00:09:10.909 --> 00:09:17.541 but here you just click on some buttons and it will be done for you. 00:09:21.889 --> 00:09:26.272 In the meantime, we have some more advanced features 00:09:26.272 --> 00:09:28.744 which I will also show you later. 00:09:29.036 --> 00:09:34.752 For example, this very simple installation just creates one partition 00:09:34.752 --> 00:09:39.923 but you can also select that you want to have a separate /home partition 00:09:39.923 --> 00:09:45.733 or using lvm just by selecting this on the web interface. 00:09:47.278 --> 00:09:55.123 You can also add your SSH public key for logging as root without a password 00:09:55.123 --> 00:10:00.649 or what's very nice, I found the new Ubuntu installer does this, 00:10:00.649 --> 00:10:05.277 you can give your github account and then there's a command which 00:10:05.277 --> 00:10:09.791 receives the public key from your github account and puts it 00:10:09.791 --> 00:10:12.959 into the root account so you can log in without password. 00:10:13.239 --> 00:10:14.778 I think that's very neat. 00:10:15.296 --> 00:10:22.037 And if you have a repository with your own packages, you could also add this and say 00:10:22.037 --> 00:10:26.508 "Please install those packages from my publicly available repository." 00:10:28.663 --> 00:10:29.994 Let's see. 00:10:30.479 --> 00:10:34.747 As we see, this job finished in 74 seconds. 00:10:35.891 --> 00:10:40.004 Now, this customised installation image is available for download. 00:10:40.658 --> 00:10:43.100 You can also download the log file. 00:10:43.548 --> 00:10:47.325 Since this is an installation image, I first have to create 00:10:47.325 --> 00:10:50.213 a partial package mirror. 00:10:51.960 --> 00:10:56.882 This is done by the command 'fai-mirror' and you can also read the log of 00:10:56.882 --> 00:11:04.569 this call of the fai-mirror, where a list of all you packages with all the dependencies 00:11:04.569 --> 00:11:06.155 are available. 00:11:06.400 --> 00:11:12.941 So you see, these are the list of packages and later they are downloaded 00:11:12.941 --> 00:11:21.402 and in the end, it says it created a mirror of 1G of packages 00:11:21.402 --> 00:11:24.328 and since I have a local mirror, it's very fast. 00:11:24.979 --> 00:11:31.113 This is the one part on the installation image partial mirror with all the packages 00:11:31.113 --> 00:11:36.427 and the other is that the config space which you can also download. 00:11:36.973 --> 00:11:41.353 So this is the config space that was really created for you 00:11:41.353 --> 00:11:43.627 by clicking the web interface. 00:11:44.025 --> 00:11:47.977 If you want to do more things with FAI, you can set up your own FAI server 00:11:47.977 --> 00:11:50.047 and use this configuration space. 00:11:51.596 --> 00:11:56.921 And, that's also very new, the two commands that are used for creating this ISO image 00:11:56.921 --> 00:11:59.562 are now also listed there. 00:12:00.010 --> 00:12:04.723 First, create the partial mirror and then create the installation image. 00:12:06.355 --> 00:12:10.988 Ok, copy link location… 00:12:11.804 --> 00:12:14.446 Let's see how good the network is here. 00:12:20.096 --> 00:12:21.638 [Q] It's a rather large image. 00:12:22.167 --> 00:12:28.789 Yeah, because it includes all the packages and with Xfce, LibreOffice and so on 00:12:29.114 --> 00:12:32.931 and the installation environment is maybe about 200MB. 00:12:33.413 --> 00:12:38.053 That's not much bigger than the Debian installer that you need to download. 00:12:39.193 --> 00:12:42.851 So, 2, 1, done. 00:12:47.731 --> 00:12:56.278 I have a little wrapper which calls a fresh kvm machine 00:12:56.278 --> 00:13:01.115 with an empty disk and boots this ISO image 00:13:01.115 --> 00:13:05.914 and then we will see how this installation runs. 00:13:07.990 --> 00:13:10.257 So this is dracut booting the image 00:13:11.200 --> 00:13:17.289 and now you see there are already some parted commands executed 00:13:17.289 --> 00:13:21.955 and now the packages are installed and everything runs on 00:13:21.955 --> 00:13:24.313 and in the end some customization script. 00:13:24.880 --> 00:13:29.392 We use only shell scripts for doing some customizations 00:13:29.392 --> 00:13:33.250 and you see the files are downloaded from /media/mirror 00:13:33.250 --> 00:13:35.893 so this is local on the ISO image. 00:13:36.422 --> 00:13:40.336 It would also be possible to create an image without the packages 00:13:40.336 --> 00:13:45.452 and then give another sources.list file so the packages would be downloaded 00:13:45.452 --> 00:13:51.585 from the internet but this default in the FAI service, 00:13:51.585 --> 00:13:54.632 we put everything onto the ISO image. 00:13:58.213 --> 00:14:01.140 I guess it will run for 4 minutes. 00:14:03.418 --> 00:14:07.310 [Q] ??? 00:14:08.332 --> 00:14:09.305 Yeah. 00:14:14.019 --> 00:14:16.575 What I will show you now is… 00:14:17.019 --> 00:14:21.083 So, this was the simple one, now I toggle this web page and 00:14:21.083 --> 00:14:25.594 you will see that there are some more questions you may answer, 00:14:25.594 --> 00:14:29.415 for example you can give a root password. 00:14:29.740 --> 00:14:32.869 If you leave this empty, sudo will be configured. 00:14:33.633 --> 00:14:39.418 Here you can upload the SSH key or give your Github account, 00:14:39.418 --> 00:14:41.373 that would be Mrfai for me. 00:14:42.554 --> 00:14:46.290 With the partitioning schemes, we have one partition 00:14:46.290 --> 00:14:52.508 or one partition and /home separated or these two versions with LVM. 00:14:53.157 --> 00:14:59.376 FAI itself can do much more, we could do soft raid set ups, cryptsetup 00:14:59.376 --> 00:15:03.802 but here I want to cover the most common installation, so very simple 00:15:03.802 --> 00:15:06.888 we have only 4 things that you can choose. 00:15:08.188 --> 00:15:10.671 [Q] For encryption? 00:15:11.037 --> 00:15:11.802 [A] Yes. 00:15:15.757 --> 00:15:19.009 So, this was the partitioning things. 00:15:19.374 --> 00:15:25.270 This is the new feature where you can add an URL for your local package repository 00:15:25.270 --> 00:15:30.723 and the rest is the same, you can add packages you like, your email address 00:15:30.723 --> 00:15:33.366 and then also create an installation image. 00:15:36.378 --> 00:15:44.434 I normally set… By default, I include the nonfree linux firmware. 00:15:45.408 --> 00:15:51.824 This is because my target audience is an end user and I want to make it 00:15:51.824 --> 00:15:56.087 very comfortable for them, so yeah, they can just install it 00:15:56.087 --> 00:15:58.083 and do not have these problems. 00:15:58.411 --> 00:16:02.431 And since this is not on an official debian.org web site, 00:16:02.431 --> 00:16:07.634 I can do this with this default. 00:16:11.373 --> 00:16:14.020 Let's see, the installation is still running. 00:16:17.277 --> 00:16:19.847 So, advanced features. 00:16:21.340 --> 00:16:23.618 The next thing after this installation, 00:16:23.618 --> 00:16:26.584 I will show you how to create cloud images. 00:16:27.115 --> 00:16:31.582 Currently, we create an installation image, 00:16:31.582 --> 00:16:34.590 when you boot it, the installation is run fully automatically. 00:16:35.160 --> 00:16:42.761 The other type of service FAI.me gives to you is that it creates a raw image 00:16:42.761 --> 00:16:47.798 or some other formats as you see here, qcow2 and whatever 00:16:47.798 --> 00:16:51.491 which you can just boot and the installation is already done. 00:16:53.857 --> 00:16:58.330 But first, see if the installation finish. 00:17:00.066 --> 00:17:01.538 Ah! Ok. 00:17:02.027 --> 00:17:07.878 These are now the shell scripts that are executed for the customization 00:17:07.878 --> 00:17:13.810 of like /etc/messages of today, /etc/network/interfaces is written 00:17:13.810 --> 00:17:15.394 and so on. 00:17:15.759 --> 00:17:21.528 You see the installation took 236s, it says there are some errors 00:17:21.528 --> 00:17:24.219 but that's not really true. 00:17:24.786 --> 00:17:29.182 And it stops here, but we can also disable this, this is only for showing 00:17:29.182 --> 00:17:31.414 everything went well 00:17:31.414 --> 00:17:34.628 and now we just reboot the machine. 00:17:37.351 --> 00:17:39.259 You see the grub. 00:17:46.696 --> 00:17:49.503 Ok, Xfce desktop. 00:17:51.049 --> 00:17:55.597 debian was the user with password FAI. 00:18:00.025 --> 00:18:08.474 We have "uname -a", this is 4.16, the backports kernel was installed there. 00:18:08.474 --> 00:18:13.955 We have only one partition, no LVM 00:18:13.955 --> 00:18:21.094 and I told it to install gimp which is not installed by default. 00:18:21.414 --> 00:18:24.436 Gimp is there, so this is nice. 00:18:24.954 --> 00:18:27.160 And the midnight commander is also there. 00:18:30.529 --> 00:18:34.310 And now we just throw this machine. Gone. 00:18:35.734 --> 00:18:43.061 What's very nice with this wrapper script, it creates the local disk 00:18:43.061 --> 00:18:47.529 of the virtual machine in /tmp which is a RAM disk and I love RAM, 00:18:47.854 --> 00:18:50.051 it's so nice and fast. 00:18:50.983 --> 00:18:55.578 So, this was installation image and now we look at the cloud image. 00:18:56.877 --> 00:19:01.183 First, you can say how big should your disk image be. 00:19:04.120 --> 00:19:10.328 Here, I say 8GB, you will see it's not an 8GB image 00:19:10.328 --> 00:19:12.810 that you have to download later. 00:19:13.218 --> 00:19:17.651 By default, I use zstd compression. 00:19:18.424 --> 00:19:21.514 Anyone who does not know this compression? 00:19:22.495 --> 00:19:27.543 This is very fast, very new, created by Facebook if I'm correct. 00:19:29.159 --> 00:19:37.583 It's for very big files and what you should never use is gzip with sparse images. 00:19:38.394 --> 00:19:41.643 The disk image is sparse and gzip cannot handle this 00:19:41.643 --> 00:19:45.710 so if you compress it and uncompress it it will be very large 00:19:45.710 --> 00:19:52.579 and all the other, xz, zstd, can handle sparse files very nicely. 00:19:54.000 --> 00:20:02.125 So, the hostname is set, the root password, username with a password. 00:20:02.524 --> 00:20:04.526 Now we want to install Buster. 00:20:07.087 --> 00:20:08.957 Maybe with no… 00:20:12.007 --> 00:20:15.385 Oh, we also do the Xfce desktop. 00:20:16.443 --> 00:20:20.097 Any packages you'd like to have in this cloud image. 00:20:21.108 --> 00:20:24.687 "desktop" and "cloud" image does not make that much sense, maybe. 00:20:26.717 --> 00:20:30.780 Emacs25, ok. 00:20:32.888 --> 00:20:36.679 And now "Create disk image". 00:20:37.451 --> 00:20:42.127 This will take a little bit longer because we are doing the installation 00:20:42.127 --> 00:20:45.417 inside a file image. 00:20:46.350 --> 00:20:52.252 But no problem, I can tell you what other ideas I have. 00:20:52.823 --> 00:21:01.591 So, currently we have the installation and the cloud or virtual machine images for amd64 00:21:01.591 --> 00:21:08.462 FAI itself can also do cross-architecture images so it would be some work 00:21:08.462 --> 00:21:14.191 to extend the web page to say "Please create an arm64 image" 00:21:14.191 --> 00:21:21.998 It would be very nice to have predefined configs for raspberry pi or 00:21:21.998 --> 00:21:24.110 all the very different boards 00:21:24.556 --> 00:21:26.785 but that would also be possible. 00:21:28.577 --> 00:21:33.041 I guess the next thing I will implement is other distributions 00:21:33.041 --> 00:21:35.852 because I know people are always asking it. 00:21:36.299 --> 00:21:38.414 Not you but the Ubuntu guys. 00:21:39.798 --> 00:21:44.925 Yesterday I did the first test with Ubuntu bionic, the LTS release 00:21:44.925 --> 00:21:47.325 and FAI just works out of the box with it. 00:21:47.854 --> 00:21:53.867 So what I have to do is to integrate it in these FAI.me processing scripts. 00:21:55.405 --> 00:22:00.323 Ready-to-go cloud images for the big cloud providers. 00:22:02.643 --> 00:22:06.438 That's only a different FAI config space that I have to use. 00:22:06.884 --> 00:22:11.070 Currently, for example, in what I call cloud images, I do not install 00:22:11.070 --> 00:22:12.693 the package cloud image. 00:22:13.912 --> 00:22:16.674 That's needed for all the ones. 00:22:17.842 --> 00:22:24.865 I'm also working in the Debian cloud team and this team decided 2 years ago 00:22:24.865 --> 00:22:29.873 that the tool chain in the future for the official Debian package will be FAI. 00:22:31.052 --> 00:22:36.937 Amazon is already using it, so if you boot or if you use a Debian cloud image 00:22:36.937 --> 00:22:41.284 in Amazon, Noah Meyerhans did this and he's using the FAI tool chain for it. 00:22:41.975 --> 00:22:48.562 Google is not yet using it because there was a very small problem 00:22:48.562 --> 00:22:51.737 in a config file we had one space too much 00:22:51.737 --> 00:22:55.273 which caused grub to hang forever 00:22:55.273 --> 00:23:00.657 and that was the reason why they decided for Stretch to use their own tool chain. 00:23:01.041 --> 00:23:06.125 But the things are working so we have the config space also for Google. 00:23:07.293 --> 00:23:12.264 And also for Azure, some people from Credativ did this. 00:23:14.911 --> 00:23:18.482 The Debian cloud team already has the FAI configuration for 00:23:18.482 --> 00:23:23.521 the big tool providers, cloud providers. 00:23:24.821 --> 00:23:30.955 We could also think on a more generic FAI installation image. 00:23:31.889 --> 00:23:37.337 It's an image that you would boot up and then enter your job id of the web page 00:23:37.337 --> 00:23:39.778 and then the configuration would be downloaded 00:23:39.778 --> 00:23:42.945 and the packages would be just received from the internet. 00:23:43.637 --> 00:23:45.144 That was one… 00:23:45.511 --> 00:23:49.697 So, the image would be much smaller because the packages do not need to be 00:23:49.697 --> 00:23:52.865 on the installation image. 00:23:54.690 --> 00:23:57.859 It's also possible to create live images with FAI. 00:23:58.590 --> 00:24:00.136 It is a little bit more… 00:24:01.272 --> 00:24:06.558 Currently, you need some manual work but that should be also possible 00:24:06.558 --> 00:24:11.157 to use FAI for creating live image and then also to provide this 00:24:11.157 --> 00:24:13.802 on the FAI.me web service. 00:24:14.574 --> 00:24:17.738 If you want to customize much more inside the image, you just say 00:24:17.738 --> 00:24:21.882 "Oh, I have some Ansible scripts that I want to execute at the very end" 00:24:22.070 --> 00:24:26.967 then I say "Ok, this is just a starting point, use the FAI.me service 00:24:26.967 --> 00:24:31.844 and if you're happy with the FAI tools, then set up your own FAI server, 00:24:31.844 --> 00:24:37.856 create your own configuration space and then you can do all the crazy things." 00:24:39.773 --> 00:24:42.335 So, how does FAI.me work internally. 00:24:43.148 --> 00:24:47.861 We have a web server where there are some CGI scripts and 00:24:47.861 --> 00:24:53.548 this is not the build server, so on the web server, you click "Submit" 00:24:53.548 --> 00:25:02.054 "Create my image", all the input is validated so you cannot make nasty things 00:25:02.054 --> 00:25:08.682 and then the CGI writes or creates a subdirectory and puts 2 files in it, 00:25:08.916 --> 00:25:10.301 a config and a meta file 00:25:11.561 --> 00:25:14.937 and writes a status "waiting for processing". 00:25:15.344 --> 00:25:19.451 Then, the other server, the build server reads this config and 00:25:19.451 --> 00:25:25.057 this is just an NFS mounted directory, and sees 00:25:25.057 --> 00:25:27.169 "Oh there's a new job I have to process". 00:25:30.147 --> 00:25:33.615 In this processing script we pass for some errors. 00:25:34.304 --> 00:25:39.624 What's happening very often that people type in a package that's not available 00:25:39.624 --> 00:25:45.239 and this will be detected and then a new version of the web page will pop up and say 00:25:45.239 --> 00:25:48.692 "Oh, when creating the package mirror, there was an error 00:25:48.692 --> 00:25:50.929 because this package was not known." 00:25:52.882 --> 00:25:54.062 Sometimes I have to… 00:25:54.508 --> 00:25:57.645 Every night, I create new nfsroots for Buster. 00:25:59.105 --> 00:26:03.172 If there are security updates, I have to create new nfsroots 00:26:03.172 --> 00:26:05.073 for Stretch and backports. 00:26:05.598 --> 00:26:10.079 I have some cleanup, so if a lot of jobs are created, 00:26:10.079 --> 00:26:12.758 the images are on the disk after, 00:26:12.758 --> 00:26:17.270 normally I say after one day I just remove the images 00:26:17.270 --> 00:26:20.567 so you have one day to download the images. 00:26:22.761 --> 00:26:25.763 There's 3 different configurations 00:26:25.763 --> 00:26:28.894 /etc/fai-stretch, /etc/buster, /etc/fai-stretch-bpo (backports) 00:26:29.129 --> 00:26:30.846 We need for the installation image 00:26:30.846 --> 00:26:36.336 We need a different nfsroot, but the config space that is shared 00:26:36.336 --> 00:26:38.283 about all configurations, 00:26:38.283 --> 00:26:41.209 so it doesn't matter if I install 00:26:41.209 --> 00:26:46.293 Stretch or Stretch backports or Buster, 00:26:46.293 --> 00:26:49.583 I can use the same FAI configuration. 00:26:49.949 --> 00:26:53.520 Also, for building the cloud images, I use the same FAI configuration. 00:26:56.327 --> 00:27:04.656 A new job is detected, then a copy of the configuration space will be made 00:27:04.656 --> 00:27:07.913 and it will be customized a little bit. 00:27:08.352 --> 00:27:10.430 So there are a very very few changes, 00:27:10.430 --> 00:27:16.686 for example I have to put the SSH key into your customized configuration space 00:27:16.686 --> 00:27:20.626 or the list of packages or the user and root password. 00:27:21.475 --> 00:27:25.697 Then we have two things, if we want to create the installation image, 00:27:25.697 --> 00:27:31.148 I first have to create the partial package mirror and then create the installation image 00:27:31.148 --> 00:27:35.700 For the cloud images, we do not need the nfsroot, we just need 00:27:35.700 --> 00:27:38.914 the configuration space which is customized a little bit 00:27:38.914 --> 00:27:41.543 and then we can just create the disk image 00:27:41.543 --> 00:27:46.590 so there's one step less compared to creating the installation ISO. 00:27:47.765 --> 00:27:51.507 The status on the web page will be updated, log files written 00:27:51.507 --> 00:27:55.375 and if the user said "Please send me an email if my job is ready", 00:27:55.375 --> 00:27:57.736 this will also be sent to the user. 00:27:58.593 --> 00:28:03.430 Then we have the ISO or the disk image and this will be copied back 00:28:03.430 --> 00:28:07.659 to the web server where the user can then download it. 00:28:08.706 --> 00:28:12.084 And since I have a lot of RAM in this machine, 00:28:12.084 --> 00:28:15.533 everything is run in RAM, very very nice. 00:28:17.563 --> 00:28:22.804 As I said, we need an nfsroot, a configuration space and FAI classes. 00:28:23.171 --> 00:28:26.260 This is a very central component in FAI 00:28:26.625 --> 00:28:28.819 and this is just a list of names. 00:28:29.387 --> 00:28:36.010 So in HOME_LVM, this is the class name, the FAI class we describe 00:28:36.010 --> 00:28:38.619 and I think this is that example: 00:28:38.619 --> 00:28:43.452 HOME_LVM describes how to partition the local hard disk. 00:28:44.795 --> 00:28:50.657 This is our very flexible tool where we can do LVM, cryptsetups, 00:28:50.657 --> 00:28:52.807 software RAIDs and so on. 00:28:53.417 --> 00:28:57.840 But for the FAI.me service, I just created 4 different types of partitioning 00:28:57.840 --> 00:29:00.443 and this is the HOME_LVM example. 00:29:03.733 --> 00:29:07.469 So we have a list of classes and, as I said, 00:29:07.469 --> 00:29:12.183 just two commands for the installation image with a list of classes 00:29:12.183 --> 00:29:17.762 and for the cloud image, I have to say how big should the disk image be, 00:29:17.762 --> 00:29:21.994 the list of classes and what's the target file that should be created. 00:29:23.856 --> 00:29:25.891 Let's see if this is ready. 00:29:28.083 --> 00:29:29.137 Yes, it's ready. 00:29:29.504 --> 00:29:30.357 So… 00:29:31.820 --> 00:29:33.567 It's 1.1GB. 00:29:34.620 --> 00:29:36.898 Is this really the… oh yeah, raw. 00:29:41.076 --> 00:29:45.555 No problem, let's download it, it should be fast. 00:29:48.911 --> 00:29:53.522 This is the normal architecture if you use FAI in a client/server set up. 00:29:54.092 --> 00:29:57.507 You should just look on the left side where you see 00:29:57.507 --> 00:30:01.535 you need the config space, an nfsroot and a mirror 00:30:01.535 --> 00:30:04.048 and these parts will put onto the CD. 00:30:05.309 --> 00:30:08.683 If you set up a network installation thing, 00:30:08.914 --> 00:30:13.139 this is how things get from the server to the client. 00:30:15.506 --> 00:30:21.403 For the software installation, we have another subdirectory called package_config 00:30:21.403 --> 00:30:27.014 and there you also see several files where the file name is a FAI class. 00:30:27.542 --> 00:30:34.003 Since in the FAI.me service every client belongs to the class DEBIAN, 00:30:34.003 --> 00:30:38.723 it will install the packages that are listed on the top 00:30:39.768 --> 00:30:42.774 and here we have an other class, NONFREE 00:30:42.774 --> 00:30:46.679 These packages are only installed if you also said 00:30:46.679 --> 00:30:49.685 "Please install the nonfree packages" 00:30:49.685 --> 00:30:53.018 and this is mapped to a FAI class called NONFREE. 00:30:53.832 --> 00:30:56.799 And there's an other class for AMD64 and so on. 00:30:58.870 --> 00:31:01.516 Some references. 00:31:02.127 --> 00:31:04.771 In the past, it looked more like this when I said 00:31:04.771 --> 00:31:12.899 "Oh, who's using FAI?" and during the last month I collected some logos 00:31:12.899 --> 00:31:15.537 just because it's much nicer. 00:31:18.865 --> 00:31:21.828 Let's see if the download was ready. 00:31:22.519 --> 00:31:34.922 We unzstd the FAI.me image, faime-013Z image 00:31:41.253 --> 00:31:46.135 On the web site, I said I want to have a 8GB partition, 00:31:46.951 --> 00:31:51.213 so now let's see how big it is. 00:31:51.621 --> 00:31:58.692 The file is 8, but since it's a sparse file it's only 3.5GB 00:31:58.692 --> 00:32:03.324 and the compressed was 1.1GB. 00:32:04.381 --> 00:32:08.879 Now I use my wrapper 00:32:11.169 --> 00:32:17.374 and I say "Boot from disk" and this is the FAI.me raw image, disk image 00:32:18.676 --> 00:32:20.630 that should be booted up. 00:32:33.558 --> 00:32:34.697 That's it. 00:32:35.836 --> 00:32:37.419 debian/fai 00:32:53.367 --> 00:32:57.176 Let's see if emacs is installed, yes. 00:32:58.886 --> 00:33:04.621 Gimp is already there, hopefully, and the blue midnight commander. 00:33:14.131 --> 00:33:15.923 Let's see. 00:33:21.447 --> 00:33:23.113 Questions. 00:33:32.188 --> 00:33:37.431 [Q] I'm using the preseed file for the debian-installer, 00:33:37.431 --> 00:33:40.842 do you have a conversion between your syntax and your configuration files 00:33:40.842 --> 00:33:45.556 and the preseed file or maybe can you add a download button for the preseed file 00:33:45.556 --> 00:33:49.010 to your web site because I think it's rather nice to have it displayed 00:33:49.010 --> 00:33:50.637 in web site first. 00:33:51.045 --> 00:33:53.109 [A] I'm not using the debian-installer. 00:33:54.044 --> 00:33:58.931 I use preseeding, yes, the debconf preseeding for the normal packages 00:33:58.931 --> 00:34:02.571 you can do this also in FAI and it's the same format 00:34:02.571 --> 00:34:06.396 you get with debconf-get-selections. 00:34:07.371 --> 00:34:11.795 And what you get is you can download your own FAI config space 00:34:11.795 --> 00:34:14.881 and this includes all information you need to set up, 00:34:14.881 --> 00:34:20.614 to do this mirror FAI CD or the FAI disk image command. 00:34:21.025 --> 00:34:27.814 But you cannot convert this config into a d-i preseeding or vice versa, 00:34:27.814 --> 00:34:30.008 that's not possible. 00:34:31.268 --> 00:34:37.634 Because for example, for the partitioning part I do not like to create 00:34:37.634 --> 00:34:42.636 from my disk config partman preseeding file. 00:34:43.817 --> 00:34:46.338 You can pay me a lot of money, I will never do this. 00:34:46.705 --> 00:34:51.948 You know that the partman preseeding is very ugly and very heavy. 00:34:53.721 --> 00:35:01.159 For other things, yes, selection of, for example, the selection of the language 00:35:01.159 --> 00:35:04.570 These are the normal preseeding we use. 00:35:05.026 --> 00:35:07.749 And the list of packages, task selec… 00:35:08.074 --> 00:35:13.840 I think it's much easier to do this in the FAI configuration than to create 00:35:13.840 --> 00:35:17.710 a debian-installer preseeding. 00:35:19.370 --> 00:35:22.828 And why use d-i if this works for you? 00:35:23.112 --> 00:35:24.729 [Q] d-i works as well for me. 00:35:25.095 --> 00:35:27.339 [A] Yes, then fine, use it. 00:35:29.905 --> 00:35:36.008 [Q] Hi Thomas. Thank you very much for this new feature in the FAI project, 00:35:36.008 --> 00:35:42.998 it's very nice and I found very great that you have the output of the commands 00:35:42.998 --> 00:35:49.019 that you used to create the ISO image or the cloud file. 00:35:50.392 --> 00:35:59.641 A question that I have is, in which servers are located the files that we create, 00:35:59.641 --> 00:36:01.307 the ISO or the cloud. 00:36:01.632 --> 00:36:04.635 Is it a server that you own host or… 00:36:04.635 --> 00:36:05.860 [A] Trust me. 00:36:09.160 --> 00:36:10.500 Currently, 00:36:10.500 --> 00:36:18.832 both the web server and the FAI.me processing build server are run 00:36:18.832 --> 00:36:24.960 on two machines at the university where I work as a system administrator 00:36:24.960 --> 00:36:27.568 so that's also where we have a very fast connection. 00:36:30.206 --> 00:36:35.859 The CGI script and shell script that is processing these jobs is currently 00:36:35.859 --> 00:36:37.528 not open source. 00:36:38.180 --> 00:36:40.522 There are plans to do this, I'm not sure when. 00:36:43.336 --> 00:36:48.012 If you want to reproduce the things, you have the config file and you can download 00:36:48.012 --> 00:36:52.245 the FAI software and use these one or two commands to reproduce it. 00:36:53.180 --> 00:36:57.203 Some people said "Oh, very nice service, I would like to set up in my company". 00:36:57.775 --> 00:37:00.499 Then please yes, contact me and… 00:37:01.149 --> 00:37:06.677 Currently there are no concrete plans to make these background scripts open source 00:37:06.677 --> 00:37:09.196 but it will be in some future. 00:37:09.764 --> 00:37:16.628 But currently, you have to trust me as you also have to trust the package maintainers 00:37:16.628 --> 00:37:18.659 that will be installed there. 00:37:19.018 --> 00:37:23.375 But you can verify it or say "I do not trust Thomas but I will just grab 00:37:23.375 --> 00:37:26.588 the FAI config space and this on my own". 00:37:27.156 --> 00:37:28.297 [Q] Thank you. 00:37:30.333 --> 00:37:33.095 [Q] There's a question from the internet. 00:37:34.641 --> 00:37:39.597 Why not use a proper job queuing system like grid engine or similar? 00:37:40.816 --> 00:37:45.696 [A] I'm using grid engine at work for different things. 00:37:47.652 --> 00:37:53.824 It started as a very simple project, so in the end it's just a loop which 00:37:53.824 --> 00:37:56.885 checks if there's new jobs on that. 00:37:57.169 --> 00:38:01.719 Currently, I do not process jobs in parallel, currently there's no need for it 00:38:02.898 --> 00:38:07.245 If this project will be very successful, yeah, I have to use a queuing system. 00:38:07.854 --> 00:38:10.492 It's, yeah, a very simple script. 00:38:10.775 --> 00:38:14.677 But it would be also possible with a proper queuing system. 00:38:17.116 --> 00:38:18.254 More questions? 00:38:18.781 --> 00:38:20.491 [Q] I have a bunch of questions. 00:38:21.913 --> 00:38:25.113 First, what is it that you use for partitioning? 00:38:26.825 --> 00:38:31.558 [A] I'm using a Perl script that we wrote several years ago in FAI 00:38:31.558 --> 00:38:37.709 and we defined this config file, this package config 00:38:37.709 --> 00:38:46.083 and the Perl script parses this script and then executes the parted and mkfs command 00:38:46.083 --> 00:38:51.447 which you can see in the log files, so if you want to see what does FAI do 00:38:51.447 --> 00:38:55.673 after parsing this, which commands are executed, you see everything 00:38:55.673 --> 00:38:57.216 on the log files. 00:38:57.641 --> 00:39:03.334 [Q] Right, but so you turn this text into partitioning… 00:39:04.190 --> 00:39:05.328 [A] commands, yeah. 00:39:05.573 --> 00:39:09.351 [Q] But the text looks like this, like with the spaces and everything. 00:39:09.351 --> 00:39:14.345 [A] You can use more or less spaces or do you like, 00:39:14.345 --> 00:39:16.509 should I convert it to XML? 00:39:18.641 --> 00:39:20.537 [Q] Ok, then my next question is 00:39:20.537 --> 00:39:25.090 what are you using the nfsroot for when you're generating the images? 00:39:25.983 --> 00:39:32.079 [A] The nfsroot is used only for the installation image. 00:39:32.079 --> 00:39:37.090 When I do the installation, I need to boot the machine as a diskless client, 00:39:37.090 --> 00:39:41.774 so it's just what the debian-installer loads into RAM, 00:39:41.774 --> 00:39:43.358 you need a running Linux system. 00:39:43.508 --> 00:39:46.196 This is our nfsroot, on the installation image. 00:39:47.536 --> 00:39:52.081 When you boot installation image, this nfsroot with all the commands we need 00:39:52.081 --> 00:39:55.706 are started without using the local disk and then we can do 00:39:55.706 --> 00:39:59.035 everything on the disk, /root and /target and so on. 00:39:59.606 --> 00:40:04.519 The nfsroot is the system that is running during the installation. 00:40:07.361 --> 00:40:11.590 [Q] Ok, but there's no need for this to be nfs, it could be a 00:40:11.590 --> 00:40:14.397 [A] It's called nfsroot. 00:40:14.397 --> 00:40:19.794 This is very common that people call it nfsroot 00:40:19.794 --> 00:40:23.862 and if you have this network installation thing, it's really an nfsroot. 00:40:24.145 --> 00:40:26.256 But you're right. 00:40:26.256 --> 00:40:32.186 On the installation ISO, it's not nfs, it's just a local file system, yes. 00:40:35.229 --> 00:40:38.891 [Q] Alright. So, I think it's my last comment. 00:40:38.891 --> 00:40:45.355 You have the ISO from which you install and when you install from the ISO 00:40:45.355 --> 00:40:50.669 you're installing then packages on the machine, and then you have the image 00:40:50.669 --> 00:40:54.660 which is like a disk image that has the packages already installed, 00:40:54.660 --> 00:40:57.585 so you skip the installing step. 00:40:59.089 --> 00:41:02.753 Have you thought about having an intermediate thing 00:41:02.753 --> 00:41:06.810 where you download an image that already has the packages installed? 00:41:09.987 --> 00:41:11.733 [A] That's also possible. 00:41:11.733 --> 00:41:18.199 When you do an installation, before you can change root in the new system 00:41:18.199 --> 00:41:21.650 for adding packages, you have to call debootstrap. 00:41:22.501 --> 00:41:26.363 What we do, we call debootstrap once and create a tar file out of it. 00:41:27.415 --> 00:41:31.935 This is our minimal… in the former days it was the floppy disk, 00:41:31.935 --> 00:41:34.491 our base tar.gz file, 00:41:34.491 --> 00:41:39.779 so you could exchange the minimal tar file with whatever tar file you have. 00:41:40.756 --> 00:41:44.169 That's for example what we do if we install Ubuntu. 00:41:44.864 --> 00:41:49.739 We boot the installation system which is a Debian system 00:41:49.739 --> 00:41:57.096 and then create the local filesystem and extract an Ubuntu base image 00:41:57.096 --> 00:42:02.012 and then we can change root into the Ubuntu or the same for CentOS and so on 00:42:02.012 --> 00:42:07.014 then we can change root into the other Linux system and add packages there. 00:42:07.540 --> 00:42:13.108 If you have already a bigger image with some more packages added there, 00:42:13.108 --> 00:42:18.188 it's very easy to say "Do not extract the Debian Stretch image 00:42:18.188 --> 00:42:22.494 but use my image which also includes other tools." 00:42:23.273 --> 00:42:27.248 And if you are fine with that, you can just extract the tar file. 00:42:30.583 --> 00:42:33.097 [Q] Ok. Any more questions? 00:42:41.268 --> 00:42:43.136 [Q] The heading is in german. 00:42:43.623 --> 00:42:44.352 [A] What? 00:42:44.679 --> 00:42:45.568 [Q] The heading is in german. 00:42:45.568 --> 00:42:48.943 [A] Oh, because it's a copy of my german slides. 00:42:50.641 --> 00:42:53.050 Thank you for this. 00:42:53.326 --> 00:42:55.287 And, what's also missing. 00:42:55.287 --> 00:43:00.357 The web page, where you can select german or other languages, 00:43:00.357 --> 00:43:04.543 it would be nice if people are interested to help translate them 00:43:04.543 --> 00:43:09.792 so that it's more easy for people that do not speak english 00:43:09.792 --> 00:43:14.871 to use the website and create their own installation image with their language. 00:43:23.500 --> 00:43:28.099 [Q] Someone on the stream said that the fai.me web site is not yours 00:43:28.099 --> 00:43:33.343 and it's a hack thing, it's a scam, you go there and get hacked. 00:43:33.790 --> 00:43:36.755 Do you have any plans to try to buy the domain because it's pretty confusing. 00:43:37.159 --> 00:43:40.732 The first thing I would have done by seeing that talk would have been 00:43:40.732 --> 00:43:41.991 to go to fai.me. 00:43:42.436 --> 00:43:48.812 [A] Yeah, I was thinking about which name I should choose. 00:43:49.339 --> 00:43:54.222 I didn't check which web domains are free and in then end I thought 00:43:54.222 --> 00:43:58.612 "Do I like to have a different domain name for the service?" 00:43:58.612 --> 00:44:03.006 But since it's only a part of the FAI project, I thought, 00:44:03.006 --> 00:44:07.771 and technically it was also easier just to host it under a subdirectory 00:44:07.771 --> 00:44:16.921 and yeah, if people now grab the fai.me domain and do other things with it, yeah. 00:44:17.445 --> 00:44:24.900 I think one question would be to use a debian.net or maybe debian.org domain 00:44:24.900 --> 00:44:27.735 because people trust much more. 00:44:28.024 --> 00:44:32.374 I get a lot of comments "Oh this would be very nice if this would be hosted 00:44:32.374 --> 00:44:34.283 on a Debian machine" 00:44:34.607 --> 00:44:38.510 but this would be much more complicated because the DSA team 00:44:38.510 --> 00:44:43.064 has much more restriction what to execute on their machines. 00:44:43.796 --> 00:44:48.631 currently, we need root access because we mount some things 00:44:48.631 --> 00:44:55.019 and DSA would not give me root access on any Debian machine. 00:44:55.831 --> 00:44:59.535 That's the same problem we have in the Debian cloud team 00:44:59.535 --> 00:45:04.367 where we want to create the official images for the cloud providers 00:45:04.367 --> 00:45:08.560 where the Debian cloud team will also not have root access 00:45:08.560 --> 00:45:13.388 and so there's much more work to get empty virtual machines 00:45:13.388 --> 00:45:18.155 starting up, putting data into it, creating the images, 00:45:18.155 --> 00:45:20.914 receiving them from inside the image. 00:45:22.173 --> 00:45:26.846 And since on those machines I have root access, that's much easier for me. 00:45:29.966 --> 00:45:33.393 We are out of time, so thank you Thomas. 00:45:34.652 --> 00:45:40.099 [Applause]