0:00:05.885,0:00:12.871 Today, I talk about FAI.me, which is a[br]build server for images. 0:00:14.947,0:00:20.359 First, anybody that never heard anything[br]about FAI? 0:00:21.781,0:00:22.593 Ok 0:00:24.376,0:00:27.468 I started this project in 1999. 0:00:29.297,0:00:30.679 I'm not sure… 0:00:31.656,0:00:35.883 No, I'm sure that during those times, the[br]Debian installer did not have 0:00:35.883,0:00:39.703 the preseeding stuff, so we needed[br]something automatically. 0:00:42.751,0:00:50.347 I installed the first cluster with FAI and[br]I always do talks on FAI or 0:00:50.347,0:00:54.211 today in the lightning talks, I talk[br]a little bit about dracut, 0:00:54.211,0:00:55.794 which is used in FAI. 0:00:56.964,0:00:58.838 So, what was the motivation. 0:00:59.448,0:01:02.780 A neighbour of mine, she came to me with 0:01:02.780,0:01:08.109 "My Windows desktop is broken,[br]can you reinstall it?" 0:01:08.395,0:01:13.673 And in the end, I installed her Linux,[br]and I was shortly thinking about 0:01:13.673,0:01:18.223 "Should I use FAI for installing her[br]desktop with Linux?" 0:01:18.995,0:01:24.809 And in the end, I did not use it because[br]FAI is too complicated, 0:01:24.809,0:01:31.030 like the Debian installer, I guess it's[br]not really that easy for beginners 0:01:31.030,0:01:32.740 because there are a lot of questions 0:01:32.740,0:01:34.973 but also FAI is not really for beginners. 0:01:35.785,0:01:38.787 So this was the motivation about thinking[br]about FAI. 0:01:40.255,0:01:43.776 The target group was always advanced[br]sysadmins 0:01:43.776,0:01:49.313 but I thought maybe it's possible to make[br]FAI usable also for people 0:01:49.313,0:01:52.490 that are not that advanced sysadmins. 0:01:54.359,0:01:58.790 The idea is that an installer should cover[br]most installations. 0:01:59.075,0:02:02.890 The Debian installer is really perfect[br]because I think it covers 0:02:02.890,0:02:07.114 all different kinds and strange environments 0:02:07.114,0:02:14.146 You can do a lot of things, you can configure[br]very strange combination of language, 0:02:14.146,0:02:16.057 keyboard layout and so on 0:02:16.546,0:02:23.068 but I was thinking about an installer[br]that covers 90 or 95% of the installations 0:02:24.677,0:02:32.092 A lot of special cases can be ignored and[br]since the Debian installer has like 0:02:32.862,0:02:38.345 more than 20 questions, I thought it would[br]be much nicer if there were only 0:02:38.345,0:02:46.155 3 to 5 questions and I looked at Linux Mint[br]and Mageia installers, CentOS installer, 0:02:46.155,0:02:50.181 and they all ask much less questions. 0:02:51.767,0:02:57.408 In the Debian installer, we sometimes[br]have also things that are asked 0:02:57.408,0:03:01.512 during the installation, so not everything[br]is asked at the very beginning. 0:03:02.079,0:03:05.898 For example, the task selection, where you[br]select your desktop, 0:03:05.898,0:03:08.538 is done after the base installation. 0:03:08.904,0:03:13.130 This was also very important, I would like[br]to have something that 0:03:13.130,0:03:15.366 asks everything at the very beginning. 0:03:16.584,0:03:22.271 Then, maybe some tool could create[br]a customized installation image 0:03:22.271,0:03:27.149 and this installation image should run[br]then completely unattended 0:03:27.149,0:03:31.572 so you can get yourself a coffee and[br]when you come back, your machine is ready. 0:03:32.792,0:03:37.590 There are 3 things to customize installation[br]image, 0:03:37.590,0:03:43.440 you just put this image, you do not have[br]to touch anything, and then it's ready. 0:03:43.603,0:03:48.239 I thought "Oh yes, this is FAI, maybe[br]FAI can do this." 0:03:50.028,0:03:57.059 As I said, FAI is only, or was until now[br]only a tool for experienced sysadmins 0:03:57.059,0:04:02.297 and you have to adjust several config[br]files, these are ASCII files 0:04:02.297,0:04:08.677 but still you have to touch 5 to 10[br]config files to make a customization. 0:04:09.610,0:04:12.698 So, how can I make FAI usable for[br]beginners? 0:04:13.023,0:04:15.294 That's the beginning of FAI.me. 0:04:17.647,0:04:21.430 There's a web page, we'll show it[br]in more detail later, 0:04:21.430,0:04:26.831 where you can just click some things, and[br]then you get a customized image. 0:04:28.815,0:04:35.606 This image can be put onto a CD, DVD or[br]USB stick, just with dd 0:04:35.606,0:04:40.724 and the customization is just by using[br]the web interface 0:04:40.724,0:04:46.257 so there's no need for you to edit[br]a text file, a config file inside FAI. 0:04:48.333,0:04:53.454 I hope I covered most important things[br]that you want to adjust 0:04:53.454,0:04:55.852 or a little bit customize. 0:04:56.123,0:04:59.733 You can add additional packages, I think[br]that's the most important thing 0:04:59.733,0:05:03.752 that people say "I want to have the normal[br]Debian installation 0:05:03.752,0:05:05.778 but with some additional packages." 0:05:06.836,0:05:11.796 And you can select different[br]distributions, so it's not only 0:05:11.796,0:05:16.755 the installation image for the stable[br]release, you can create 0:05:16.755,0:05:19.190 3 variants of the installation. 0:05:21.553,0:05:28.747 This is the web page and thanks to Juri,[br]he did a great job 0:05:28.747,0:05:34.316 during the first and second day, he added[br]a new feature that we now have 0:05:34.316,0:05:35.616 a toggle button. 0:05:36.388,0:05:39.522 Is it big enough or should I zoom in? 0:05:40.823,0:05:41.633 Ok. 0:05:42.688,0:05:49.963 So, we have a toggle button, what you see[br]now is just the bare minimum of questions 0:05:49.963,0:05:54.111 and we can toggle it to more advanced[br]settings. 0:05:54.876,0:06:00.225 You have to select or just leave this as[br]it is, username, 0:06:00.225,0:06:06.120 if you do not enter a password, a password[br]will be generated and shown to you 0:06:06.120,0:06:07.706 and sent by e-mail. 0:06:08.435,0:06:11.445 I will now just type in the password. 0:06:11.932,0:06:15.019 It's here in clear text, for me that's fine[br]because 0:06:15.019,0:06:21.430 there's also a comment that you should[br]change the password after the installation 0:06:21.430,0:06:27.337 and I do not like to enter passwords twice[br]so you can see what you typed in 0:06:27.337,0:06:33.311 and hopefully do not make any wrong[br]mistakes. 0:06:33.636,0:06:38.311 For example, we could select the Stretch[br]distribution with backports, 0:06:38.311,0:06:42.208 so we will get a 4.15 kernel with Stretch. 0:06:43.221,0:06:47.687 There are some buttons we can say we want[br]to have some Debian developer tools. 0:06:48.246,0:06:54.145 This is what I defined in the FAI[br]configuration, so just a list of packages. 0:06:55.041,0:06:59.219 Here, you can enter you own packages.[br] 0:07:01.301,0:07:04.548 I will select the desktop. 0:07:04.832,0:07:09.340 You can have an installation without any[br]desktop, so a very small installation. 0:07:10.963,0:07:16.042 I will select the XFCE desktop, but all[br]the other desktops are here. 0:07:16.368,0:07:21.935 The language, these are just task packages[br]that are… 0:07:22.343,0:07:26.532 I think Debian has much more task packages,[br]I just searched which are 0:07:26.532,0:07:35.516 the most common languages, and what I do[br]if I say I want the spanish language, 0:07:35.516,0:07:37.986 also the keyboard layout is spanish. 0:07:39.254,0:07:43.239 I know there are different combinations[br]and with local time, 0:07:43.239,0:07:47.071 it's getting more difficult. 0:07:47.999,0:07:53.121 This installation will install the clock[br]with UTC, so if you want to set 0:07:53.121,0:07:55.315 your time, you have to do this manually. 0:07:55.751,0:07:59.951 I want to cover the most common installations. 0:08:02.389,0:08:11.372 We select english US, the desktop and,[br]as an example, the midnight commander 0:08:11.372,0:08:12.746 and GIMP. 0:08:13.242,0:08:21.050 I can add an email address so if it would[br]take longer, 0:08:21.050,0:08:26.333 for example if this service will have[br]success and a lot of people are using it, 0:08:26.333,0:08:30.992 you may wait for some minutes so your job[br]will be finished. 0:08:31.774,0:08:37.990 So here are the comments, how to reconfigure[br]the keyboard or the timezone 0:08:37.990,0:08:41.568 and then you just click "Create[br]the installation image". 0:08:42.138,0:08:49.333 Now, in the background, there's some job,[br]a script, looking "Oh, there's a new job" 0:08:49.333,0:08:54.117 and there's a summary of the configuration,[br]of the web configuration. 0:08:55.504,0:08:59.164 Down here you see these are the[br]FAI classes, 0:08:59.164,0:09:01.849 I will explain a little bit more about this. 0:09:02.092,0:09:06.552 But with this information, FAI configuration[br]is generated, 0:09:06.884,0:09:10.179 that's what normally the experienced[br]sysadmins have to create 0:09:10.909,0:09:17.541 but here you just click on some buttons[br]and it will be done for you. 0:09:21.889,0:09:26.272 In the meantime, we have some more[br]advanced features 0:09:26.272,0:09:28.744 which I will also show you later. 0:09:29.036,0:09:34.752 For example, this very simple installation[br]just creates one partition 0:09:34.752,0:09:39.923 but you can also select that you want[br]to have a separate /home partition 0:09:39.923,0:09:45.733 or using lvm just by selecting this[br]on the web interface. 0:09:47.278,0:09:55.123 You can also add your SSH public key[br]for logging as root without a password 0:09:55.123,0:10:00.649 or what's very nice, I found the new[br]Ubuntu installer does this, 0:10:00.649,0:10:05.277 you can give your github account and[br]then there's a command which 0:10:05.277,0:10:09.791 receives the public key from your[br]github account and puts it 0:10:09.791,0:10:12.959 into the root account so you can log in[br]without password. 0:10:13.239,0:10:14.778 I think that's very neat. 0:10:15.296,0:10:22.037 And if you have a repository with your own[br]packages, you could also add this and say 0:10:22.037,0:10:26.508 "Please install those packages from[br]my publicly available repository." 0:10:28.663,0:10:29.994 Let's see. 0:10:30.479,0:10:34.747 As we see, this job finished in 74 seconds. 0:10:35.891,0:10:40.004 Now, this customised installation image[br]is available for download. 0:10:40.658,0:10:43.100 You can also download the log file. 0:10:43.548,0:10:47.325 Since this is an installation image,[br]I first have to create 0:10:47.325,0:10:50.213 a partial package mirror. 0:10:51.960,0:10:56.882 This is done by the command 'fai-mirror'[br]and you can also read the log of 0:10:56.882,0:11:04.569 this call of the fai-mirror, where a list[br]of all you packages with all the dependencies 0:11:04.569,0:11:06.155 are available. 0:11:06.400,0:11:12.941 So you see, these are the list of packages[br]and later they are downloaded 0:11:12.941,0:11:21.402 and in the end, it says it created a mirror[br]of 1G of packages 0:11:21.402,0:11:24.328 and since I have a local mirror,[br]it's very fast. 0:11:24.979,0:11:31.113 This is the one part on the installation[br]image partial mirror with all the packages 0:11:31.113,0:11:36.427 and the other is that the config space[br]which you can also download. 0:11:36.973,0:11:41.353 So this is the config space that was[br]really created for you 0:11:41.353,0:11:43.627 by clicking the web interface. 0:11:44.025,0:11:47.977 If you want to do more things with FAI,[br]you can set up your own FAI server 0:11:47.977,0:11:50.047 and use this configuration space. 0:11:51.596,0:11:56.921 And, that's also very new, the two commands[br]that are used for creating this ISO image 0:11:56.921,0:11:59.562 are now also listed there. 0:12:00.010,0:12:04.723 First, create the partial mirror and then[br]create the installation image. 0:12:06.355,0:12:10.988 Ok, copy link location… 0:12:11.804,0:12:14.446 Let's see how good the network is here. 0:12:20.096,0:12:21.638 [Q] It's a rather large image. 0:12:22.167,0:12:28.789 Yeah, because it includes all the packages[br]and with Xfce, LibreOffice and so on 0:12:29.114,0:12:32.931 and the installation environment is maybe[br]about 200MB. 0:12:33.413,0:12:38.053 That's not much bigger than the Debian[br]installer that you need to download. 0:12:39.193,0:12:42.851 So, 2, 1, done. 0:12:47.731,0:12:56.278 I have a little wrapper which calls[br]a fresh kvm machine 0:12:56.278,0:13:01.115 with an empty disk and boots this ISO image 0:13:01.115,0:13:05.914 and then we will see how this installation[br]runs. 0:13:07.990,0:13:10.257 So this is dracut booting the image 0:13:11.200,0:13:17.289 and now you see there are already[br]some parted commands executed 0:13:17.289,0:13:21.955 and now the packages are installed[br]and everything runs on 0:13:21.955,0:13:24.313 and in the end some customization[br]script. 0:13:24.880,0:13:29.392 We use only shell scripts for doing[br]some customizations 0:13:29.392,0:13:33.250 and you see the files are downloaded[br]from /media/mirror 0:13:33.250,0:13:35.893 so this is local on the ISO image. 0:13:36.422,0:13:40.336 It would also be possible to create[br]an image without the packages 0:13:40.336,0:13:45.452 and then give another sources.list file[br]so the packages would be downloaded 0:13:45.452,0:13:51.585 from the internet but this default[br]in the FAI service, 0:13:51.585,0:13:54.632 we put everything onto the ISO image. 0:13:58.213,0:14:01.140 I guess it will run for 4 minutes. 0:14:03.418,0:14:07.310 [Q] ??? 0:14:08.332,0:14:09.305 Yeah. 0:14:14.019,0:14:16.575 What I will show you now is… 0:14:17.019,0:14:21.083 So, this was the simple one, now I toggle[br]this web page and 0:14:21.083,0:14:25.594 you will see that there are some more[br]questions you may answer, 0:14:25.594,0:14:29.415 for example you can give a root password. 0:14:29.740,0:14:32.869 If you leave this empty, sudo will be[br]configured. 0:14:33.633,0:14:39.418 Here you can upload the SSH key or give[br]your Github account, 0:14:39.418,0:14:41.373 that would be Mrfai for me. 0:14:42.554,0:14:46.290 With the partitioning schemes, we have[br]one partition 0:14:46.290,0:14:52.508 or one partition and /home separated[br]or these two versions with LVM. 0:14:53.157,0:14:59.376 FAI itself can do much more, we could do[br]soft raid set ups, cryptsetup 0:14:59.376,0:15:03.802 but here I want to cover the most common[br]installation, so very simple 0:15:03.802,0:15:06.888 we have only 4 things that you can choose. 0:15:08.188,0:15:10.671 [Q] For encryption? 0:15:11.037,0:15:11.802 [A] Yes. 0:15:15.757,0:15:19.009 So, this was the partitioning things. 0:15:19.374,0:15:25.270 This is the new feature where you can add[br]an URL for your local package repository 0:15:25.270,0:15:30.723 and the rest is the same, you can add[br]packages you like, your email address 0:15:30.723,0:15:33.366 and then also create an installation image. 0:15:36.378,0:15:44.434 I normally set… By default, I include[br]the nonfree linux firmware. 0:15:45.408,0:15:51.824 This is because my target audience is[br]an end user and I want to make it 0:15:51.824,0:15:56.087 very comfortable for them, so yeah,[br]they can just install it 0:15:56.087,0:15:58.083 and do not have these problems. 0:15:58.411,0:16:02.431 And since this is not on an official[br]debian.org web site, 0:16:02.431,0:16:07.634 I can do this with this default. 0:16:11.373,0:16:14.020 Let's see, the installation is still running. 0:16:17.277,0:16:19.847 So, advanced features. 0:16:21.340,0:16:23.618 The next thing after this installation, 0:16:23.618,0:16:26.584 I will show you how to create cloud[br]images. 0:16:27.115,0:16:31.582 Currently, we create an installation image, 0:16:31.582,0:16:34.590 when you boot it, the installation is run[br]fully automatically. 0:16:35.160,0:16:42.761 The other type of service FAI.me gives[br]to you is that it creates a raw image 0:16:42.761,0:16:47.798 or some other formats as you see here,[br]qcow2 and whatever 0:16:47.798,0:16:51.491 which you can just boot and[br]the installation is already done. 0:16:53.857,0:16:58.330 But first, see if the installation finish. 0:17:00.066,0:17:01.538 Ah! Ok. 0:17:02.027,0:17:07.878 These are now the shell scripts that are[br]executed for the customization 0:17:07.878,0:17:13.810 of like /etc/messages of today,[br]/etc/network/interfaces is written 0:17:13.810,0:17:15.394 and so on. 0:17:15.759,0:17:21.528 You see the installation took 236s, it says[br]there are some errors 0:17:21.528,0:17:24.219 but that's not really true. 0:17:24.786,0:17:29.182 And it stops here, but we can also[br]disable this, this is only for showing 0:17:29.182,0:17:31.414 everything went well 0:17:31.414,0:17:34.628 and now we just reboot the machine. 0:17:37.351,0:17:39.259 You see the grub. 0:17:46.696,0:17:49.503 Ok, Xfce desktop. 0:17:51.049,0:17:55.597 debian was the user with password FAI. 0:18:00.025,0:18:08.474 We have "uname -a", this is 4.16,[br]the backports kernel was installed there. 0:18:08.474,0:18:13.955 We have only one partition, no LVM 0:18:13.955,0:18:21.094 and I told it to install gimp which is[br]not installed by default. 0:18:21.414,0:18:24.436 Gimp is there, so this is nice. 0:18:24.954,0:18:27.160 And the midnight commander is also there. 0:18:30.529,0:18:34.310 And now we just throw this machine.[br]Gone. 0:18:35.734,0:18:43.061 What's very nice with this wrapper script,[br]it creates the local disk 0:18:43.061,0:18:47.529 of the virtual machine in /tmp which is[br]a RAM disk and I love RAM, 0:18:47.854,0:18:50.051 it's so nice and fast. 0:18:50.983,0:18:55.578 So, this was installation image and now we[br]look at the cloud image. 0:18:56.877,0:19:01.183 First, you can say how big should your[br]disk image be. 0:19:04.120,0:19:10.328 Here, I say 8GB, you will see it's not[br]an 8GB image 0:19:10.328,0:19:12.810 that you have to download later. 0:19:13.218,0:19:17.651 By default, I use zstd compression. 0:19:18.424,0:19:21.514 Anyone who does not know[br]this compression? 0:19:22.495,0:19:27.543 This is very fast, very new, created[br]by Facebook if I'm correct. 0:19:29.159,0:19:37.583 It's for very big files and what you should[br]never use is gzip with sparse images. 0:19:38.394,0:19:41.643 The disk image is sparse and gzip[br]cannot handle this 0:19:41.643,0:19:45.710 so if you compress it and uncompress it[br]it will be very large 0:19:45.710,0:19:52.579 and all the other, xz, zstd, can handle[br]sparse files very nicely. 0:19:54.000,0:20:02.125 So, the hostname is set, the root password,[br]username with a password. 0:20:02.524,0:20:04.526 Now we want to install Buster. 0:20:07.087,0:20:08.957 Maybe with no… 0:20:12.007,0:20:15.385 Oh, we also do the Xfce desktop. 0:20:16.443,0:20:20.097 Any packages you'd like to have in this[br]cloud image. 0:20:21.108,0:20:24.687 "desktop" and "cloud" image does not[br]make that much sense, maybe. 0:20:26.717,0:20:30.780 Emacs25, ok. 0:20:32.888,0:20:36.679 And now "Create disk image". 0:20:37.451,0:20:42.127 This will take a little bit longer because[br]we are doing the installation 0:20:42.127,0:20:45.417 inside a file image. 0:20:46.350,0:20:52.252 But no problem, I can tell you what[br]other ideas I have. 0:20:52.823,0:21:01.591 So, currently we have the installation and[br]the cloud or virtual machine images for amd64 0:21:01.591,0:21:08.462 FAI itself can also do cross-architecture[br]images so it would be some work 0:21:08.462,0:21:14.191 to extend the web page to say[br]"Please create an arm64 image" 0:21:14.191,0:21:21.998 It would be very nice to have predefined[br]configs for raspberry pi or 0:21:21.998,0:21:24.110 all the very different boards 0:21:24.556,0:21:26.785 but that would also be possible. 0:21:28.577,0:21:33.041 I guess the next thing I will implement[br]is other distributions 0:21:33.041,0:21:35.852 because I know people are always asking it. 0:21:36.299,0:21:38.414 Not you but the Ubuntu guys. 0:21:39.798,0:21:44.925 Yesterday I did the first test with Ubuntu[br]bionic, the LTS release 0:21:44.925,0:21:47.325 and FAI just works out of the box with it. 0:21:47.854,0:21:53.867 So what I have to do is to integrate it[br]in these FAI.me processing scripts. 0:21:55.405,0:22:00.323 Ready-to-go cloud images for the big[br]cloud providers. 0:22:02.643,0:22:06.438 That's only a different FAI config space[br]that I have to use. 0:22:06.884,0:22:11.070 Currently, for example, in what I call[br]cloud images, I do not install 0:22:11.070,0:22:12.693 the package cloud image. 0:22:13.912,0:22:16.674 That's needed for all the ones. 0:22:17.842,0:22:24.865 I'm also working in the Debian cloud team[br]and this team decided 2 years ago 0:22:24.865,0:22:29.873 that the tool chain in the future for[br]the official Debian package will be FAI. 0:22:31.052,0:22:36.937 Amazon is already using it, so if you[br]boot or if you use a Debian cloud image 0:22:36.937,0:22:41.284 in Amazon, Noah Meyerhans did this and[br]he's using the FAI tool chain for it. 0:22:41.975,0:22:48.562 Google is not yet using it because there was[br]a very small problem 0:22:48.562,0:22:51.737 in a config file we had one space too much 0:22:51.737,0:22:55.273 which caused grub to hang forever 0:22:55.273,0:23:00.657 and that was the reason why they decided[br]for Stretch to use their own tool chain. 0:23:01.041,0:23:06.125 But the things are working so we have[br]the config space also for Google. 0:23:07.293,0:23:12.264 And also for Azure, some people from[br]Credativ did this. 0:23:14.911,0:23:18.482 The Debian cloud team already has[br]the FAI configuration for 0:23:18.482,0:23:23.521 the big tool providers,[br]cloud providers. 0:23:24.821,0:23:30.955 We could also think on a more generic[br]FAI installation image. 0:23:31.889,0:23:37.337 It's an image that you would boot up[br]and then enter your job id of the web page 0:23:37.337,0:23:39.778 and then the configuration would be[br]downloaded 0:23:39.778,0:23:42.945 and the packages would be just[br]received from the internet. 0:23:43.637,0:23:45.144 That was one… 0:23:45.511,0:23:49.697 So, the image would be much smaller[br]because the packages do not need to be 0:23:49.697,0:23:52.865 on the installation image. 0:23:54.690,0:23:57.859 It's also possible to create live images[br]with FAI. 0:23:58.590,0:24:00.136 It is a little bit more… 0:24:01.272,0:24:06.558 Currently, you need some manual work[br]but that should be also possible 0:24:06.558,0:24:11.157 to use FAI for creating live image[br]and then also to provide this 0:24:11.157,0:24:13.802 on the FAI.me web service. 0:24:14.574,0:24:17.738 If you want to customize much more[br]inside the image, you just say 0:24:17.738,0:24:21.882 "Oh, I have some Ansible scripts that[br]I want to execute at the very end" 0:24:22.070,0:24:26.967 then I say "Ok, this is just a starting[br]point, use the FAI.me service 0:24:26.967,0:24:31.844 and if you're happy with the FAI tools,[br]then set up your own FAI server, 0:24:31.844,0:24:37.856 create your own configuration space[br]and then you can do all the crazy things." 0:24:39.773,0:24:42.335 So, how does FAI.me work internally. 0:24:43.148,0:24:47.861 We have a web server where there are[br]some CGI scripts and 0:24:47.861,0:24:53.548 this is not the build server, so on the[br]web server, you click "Submit" 0:24:53.548,0:25:02.054 "Create my image", all the input[br]is validated so you cannot make nasty things 0:25:02.054,0:25:08.682 and then the CGI writes or creates[br]a subdirectory and puts 2 files in it, 0:25:08.916,0:25:10.301 a config and a meta file 0:25:11.561,0:25:14.937 and writes a status[br]"waiting for processing". 0:25:15.344,0:25:19.451 Then, the other server, the build server[br]reads this config and 0:25:19.451,0:25:25.057 this is just an NFS mounted directory,[br]and sees 0:25:25.057,0:25:27.169 "Oh there's a new job I have to process". 0:25:30.147,0:25:33.615 In this processing script we pass for[br]some errors. 0:25:34.304,0:25:39.624 What's happening very often that people[br]type in a package that's not available 0:25:39.624,0:25:45.239 and this will be detected and then a new[br]version of the web page will pop up and say 0:25:45.239,0:25:48.692 "Oh, when creating the package mirror,[br]there was an error 0:25:48.692,0:25:50.929 because this package was not known." 0:25:52.882,0:25:54.062 Sometimes I have to… 0:25:54.508,0:25:57.645 Every night, I create new nfsroots[br]for Buster. 0:25:59.105,0:26:03.172 If there are security updates, I have[br]to create new nfsroots 0:26:03.172,0:26:05.073 for Stretch and backports. 0:26:05.598,0:26:10.079 I have some cleanup, so if a lot of jobs[br]are created, 0:26:10.079,0:26:12.758 the images are on the disk after, 0:26:12.758,0:26:17.270 normally I say after one day I just[br]remove the images 0:26:17.270,0:26:20.567 so you have one day to download[br]the images. 0:26:22.761,0:26:25.763 There's 3 different configurations 0:26:25.763,0:26:28.894 /etc/fai-stretch, /etc/buster,[br]/etc/fai-stretch-bpo (backports) 0:26:29.129,0:26:30.846 We need for the installation image 0:26:30.846,0:26:36.336 We need a different nfsroot, but[br]the config space that is shared 0:26:36.336,0:26:38.283 about all configurations, 0:26:38.283,0:26:41.209 so it doesn't matter if I install 0:26:41.209,0:26:46.293 Stretch or Stretch backports or[br]Buster, 0:26:46.293,0:26:49.583 I can use the same FAI configuration. 0:26:49.949,0:26:53.520 Also, for building the cloud images,[br]I use the same FAI configuration. 0:26:56.327,0:27:04.656 A new job is detected, then a copy of[br]the configuration space will be made 0:27:04.656,0:27:07.913 and it will be customized a little bit. 0:27:08.352,0:27:10.430 So there are a very very few changes, 0:27:10.430,0:27:16.686 for example I have to put the SSH key[br]into your customized configuration space 0:27:16.686,0:27:20.626 or the list of packages or the user and[br]root password. 0:27:21.475,0:27:25.697 Then we have two things, if we want to[br]create the installation image, 0:27:25.697,0:27:31.148 I first have to create the partial package[br]mirror and then create the installation image 0:27:31.148,0:27:35.700 For the cloud images, we do not need[br]the nfsroot, we just need 0:27:35.700,0:27:38.914 the configuration space which is[br]customized a little bit 0:27:38.914,0:27:41.543 and then we can just create the disk image 0:27:41.543,0:27:46.590 so there's one step less compared to[br]creating the installation ISO. 0:27:47.765,0:27:51.507 The status on the web page will be[br]updated, log files written 0:27:51.507,0:27:55.375 and if the user said "Please send me[br]an email if my job is ready", 0:27:55.375,0:27:57.736 this will also be sent to the user. 9:59:59.000,9:59:59.000 The we have the ISO or the disk image[br]and this will be copied back 9:59:59.000,9:59:59.000 to the web server where the user can then[br]download it. 9:59:59.000,9:59:59.000 And since I have a lot of RAM[br]in this machine, 9:59:59.000,9:59:59.000 everything is run in RAM, very very nice. 9:59:59.000,9:59:59.000 As I said, we need an nfsroot,[br]a configuration space and FAI classes. 9:59:59.000,9:59:59.000 This is a very central component in FAI 9:59:59.000,9:59:59.000 and this is just a list of names. 9:59:59.000,9:59:59.000 So in HOME_LVM, this is the class name,[br]the FAI class we describe 9:59:59.000,9:59:59.000 and I think this is that example: 9:59:59.000,9:59:59.000 HOME_LVM describes how to partition[br]the local hard disk. 9:59:59.000,9:59:59.000 This is our very flexible tool where[br]we can do LVM, cryptsetups, 9:59:59.000,9:59:59.000 software RAIDs and so on. 9:59:59.000,9:59:59.000 But for the FAI.me service, I just created[br]4 different types of partitioning 9:59:59.000,9:59:59.000 and this is the HOME_LVM example. 9:59:59.000,9:59:59.000 So we have a list of classes and,[br]as I said, 9:59:59.000,9:59:59.000 just two commands for the installation[br]image with a list of classes 9:59:59.000,9:59:59.000 and for the cloud image, I have to say[br]how big should the disk image be, 9:59:59.000,9:59:59.000 the list of classes and what's the target[br]file that should be created. 9:59:59.000,9:59:59.000 Let's see if this is ready. 9:59:59.000,9:59:59.000 Yes, it's ready. 9:59:59.000,9:59:59.000 So… 9:59:59.000,9:59:59.000 It's 1.1GB. 9:59:59.000,9:59:59.000 Is this really the… oh yeah, raw. 9:59:59.000,9:59:59.000 No problem, let's download it, it should[br]be fast. 9:59:59.000,9:59:59.000 This is the normal architecture if you use[br]FAI in a client/server set up. 9:59:59.000,9:59:59.000 You should just look on the left side[br]where you see 9:59:59.000,9:59:59.000 you need the config space, an nfsroot[br]and a mirror 9:59:59.000,9:59:59.000 and these parts will put onto the CD. 9:59:59.000,9:59:59.000 If you ??? network installation thing, 9:59:59.000,9:59:59.000 this is of things get from the server[br]to the client. 9:59:59.000,9:59:59.000 For the software installation, we have[br]another subdirectory called package_config 9:59:59.000,9:59:59.000 and there you also see several files[br]where the file name is a FAI class. 9:59:59.000,9:59:59.000 Since in the FAI.me service every client[br]belongs to the class DEBIAN, 9:59:59.000,9:59:59.000 it will install the packages that are listed[br]on the ??? 9:59:59.000,9:59:59.000 and here we have an other class, NONFREE 9:59:59.000,9:59:59.000 These packages are only installed if[br]you also said 9:59:59.000,9:59:59.000 "Please install the nonfree packages" 9:59:59.000,9:59:59.000 and this is mapped to a FAI class[br]called NONFREE. 9:59:59.000,9:59:59.000 And there's an other class for AMD64[br]and so on. 9:59:59.000,9:59:59.000 Some references. 9:59:59.000,9:59:59.000 In the past, it looked more like this[br]when I said 9:59:59.000,9:59:59.000 "Oh, who's using FAI?" and during[br]the last month I collected some logos 9:59:59.000,9:59:59.000 just because it's much nicer. 9:59:59.000,9:59:59.000 Let's see if the download was ready. 9:59:59.000,9:59:59.000 We unzstd the FAI.me image,[br]faime-013Z image 9:59:59.000,9:59:59.000 On the web site, I said I want to have[br]a 8GB partition, 9:59:59.000,9:59:59.000 so now let's see how big it is. 9:59:59.000,9:59:59.000 The file is 8, but since it's a sparse file[br]it's only 3.5GB 9:59:59.000,9:59:59.000 and the compressed was 1.1GB. 9:59:59.000,9:59:59.000 Now I use my wrapper 9:59:59.000,9:59:59.000 and I say "Boot from disk" and this is[br]the FAI.me raw image, disk image 9:59:59.000,9:59:59.000 that should be booted up. 9:59:59.000,9:59:59.000 That's it. 9:59:59.000,9:59:59.000 debian/fai 9:59:59.000,9:59:59.000 Let's see if emacs is installed, yes. 9:59:59.000,9:59:59.000 Gimp is already there, hopefully, and[br]the blue midnight commander. 9:59:59.000,9:59:59.000 Let's see. 9:59:59.000,9:59:59.000 Questions. 9:59:59.000,9:59:59.000 [Q] I'm using the preseed file for[br]the debian-installer, 9:59:59.000,9:59:59.000 do you have a conversion between[br]your syntax and your configuration files 9:59:59.000,9:59:59.000 and the preseed file or maybe can you add[br]a download button for the preseed file 9:59:59.000,9:59:59.000 to your web site because I think it's[br]rather nice to have it displayed 9:59:59.000,9:59:59.000 in web site first. 9:59:59.000,9:59:59.000 [A] I'm not using the debian-installer. 9:59:59.000,9:59:59.000 I use preseeding, yes, the debconf[br]preseeding for the normal packages 9:59:59.000,9:59:59.000 you can do this also in FAI and it's[br]the same format 9:59:59.000,9:59:59.000 you get with debconf-get-selections. 9:59:59.000,9:59:59.000 And what you get is you can download[br]your own FAI config space 9:59:59.000,9:59:59.000 and this includes all information you need[br]to set up, 9:59:59.000,9:59:59.000 to do this mirror FAI CD or the FAI disk[br]image command. 9:59:59.000,9:59:59.000 But you cannot convert this config into[br]a d-i preseeding or vice versa, 9:59:59.000,9:59:59.000 that's not possible. 9:59:59.000,9:59:59.000 Because for example, for the partitioning[br]part I do not like to create 9:59:59.000,9:59:59.000 from my disk config partman preseeding[br]file. 9:59:59.000,9:59:59.000 You can pay me a lot of money, I will[br]never do this. 9:59:59.000,9:59:59.000 You know that the partman preseeding[br]is very ugly and very heavy. 9:59:59.000,9:59:59.000 For other things, yes, selection of,[br]for example, the selection of the language 9:59:59.000,9:59:59.000 These are the normal preseeding we use. 9:59:59.000,9:59:59.000 And the list of packages, task selec… 9:59:59.000,9:59:59.000 I think it's much easier to do this in[br]the FAI configuration than to create 9:59:59.000,9:59:59.000 a debian-installer preseeding. 9:59:59.000,9:59:59.000 And why use d-i if this works for you? 9:59:59.000,9:59:59.000 [Q] d-i works as well for me. 9:59:59.000,9:59:59.000 [A] Yes, then fine, use it. 9:59:59.000,9:59:59.000 [Q] Hi Thomas. Thank you very much for[br]this new feature in the FAI project, 9:59:59.000,9:59:59.000 it's very nice and I found very great that[br]you have the output of the commands 9:59:59.000,9:59:59.000 that you used to create the ISO image or[br]the cloud file. 9:59:59.000,9:59:59.000 A question that I have is, in which servers[br]are located the files that we create, 9:59:59.000,9:59:59.000 the ISO or the cloud. 9:59:59.000,9:59:59.000 Is it a server that you own host or… 9:59:59.000,9:59:59.000 [A] Trust me. 9:59:59.000,9:59:59.000 Currently, 9:59:59.000,9:59:59.000 both the web server and the FAI.me[br]processing build server are run 9:59:59.000,9:59:59.000 on two machines at the university where[br]I work as a system administrator 9:59:59.000,9:59:59.000 so that's also where we have a very fast[br]connection. 9:59:59.000,9:59:59.000 The CGI script and shell script that is[br]processing these jobs is currently 9:59:59.000,9:59:59.000 not open source. 9:59:59.000,9:59:59.000 There are plans to do this, I'm not sure[br]when. 9:59:59.000,9:59:59.000 If you want to reproduce the things, you[br]have the config file and you can download 9:59:59.000,9:59:59.000 the FAI software and use these one or two[br]commands to reproduce it. 9:59:59.000,9:59:59.000 Some people said "Oh, very nice service,[br]I would like to set up in my company". 9:59:59.000,9:59:59.000 Then please yes, contact me and… 9:59:59.000,9:59:59.000 Currently there are no concrete plans[br]to make these background scripts open source 9:59:59.000,9:59:59.000 but it will be in some future. 9:59:59.000,9:59:59.000 But currently, you have to trust me as[br]you also have to trust the package maintainers 9:59:59.000,9:59:59.000 that will be installed there. 9:59:59.000,9:59:59.000 But you can verify it or say "I do not[br]trust Thomas but I will just grab 9:59:59.000,9:59:59.000 the FAI config space and this on my own". 9:59:59.000,9:59:59.000 [Q] Thank you. 9:59:59.000,9:59:59.000 [Q] There's a question from the internet. 9:59:59.000,9:59:59.000 Why not use a proper job queuing system[br]like grid engine or similar? 9:59:59.000,9:59:59.000 [A] I'm using grid engine at work for[br]different things. 9:59:59.000,9:59:59.000 It started as a very simple project, so[br]in the end it's just a loop which 9:59:59.000,9:59:59.000 checks if there's new jobs on that. 9:59:59.000,9:59:59.000 Currently, I do not process jobs[br]in parallel, currently there's no need for it 9:59:59.000,9:59:59.000 If this project will be very successful,[br]yeah, I have to use a queuing system. 9:59:59.000,9:59:59.000 It's, yeah, a very simple script. 9:59:59.000,9:59:59.000 But it would be also possible with[br]a proper queuing system. 9:59:59.000,9:59:59.000 More questions? 9:59:59.000,9:59:59.000 [Q] I have a bunch of questions. 9:59:59.000,9:59:59.000 First, what is it that you use for[br]partitioning? 9:59:59.000,9:59:59.000 [A] I'm using a poll script that we wrote[br]several years ago in FAI 9:59:59.000,9:59:59.000 and we defined this config file,[br]this package config 9:59:59.000,9:59:59.000 and the Perl script parses this script[br]and then executes the parted and mkfs command 9:59:59.000,9:59:59.000 which you can see in the log files, so if[br]you want to see what does FAI do 9:59:59.000,9:59:59.000 after parsing this, which commands are[br]executed, you see everything 9:59:59.000,9:59:59.000 on the log files. 9:59:59.000,9:59:59.000 [Q] Right, but so you turn this text[br]into partitioning… 9:59:59.000,9:59:59.000 [A] commands, yeah. 9:59:59.000,9:59:59.000 [Q] But the text looks like this,[br]like with the spaces and everything. 9:59:59.000,9:59:59.000 [A] You can use more or less spaces[br]or do you like, 9:59:59.000,9:59:59.000 should I convert it to XML? 9:59:59.000,9:59:59.000 [Q] Ok, then my next question is 9:59:59.000,9:59:59.000 what are you using the nfsroot for when[br]you're generating the images? 9:59:59.000,9:59:59.000 [A] The nfsroot is used only for the[br]installation image. 9:59:59.000,9:59:59.000 When I do the installation, I need to boot[br]the machine as a diskless client, 9:59:59.000,9:59:59.000 so it's just what the debian-installer[br]into RAM, 9:59:59.000,9:59:59.000 you need a running Linux system. 9:59:59.000,9:59:59.000 This is our nfsroot, on the installation[br]image. 9:59:59.000,9:59:59.000 When you boot installation image, this[br]nfsroot with all the commands we need 9:59:59.000,9:59:59.000 are started without using the local disk[br]and then we can do 9:59:59.000,9:59:59.000 everything on the disk, /root and /target[br]and so on. 9:59:59.000,9:59:59.000 The nfsroot is the system that is running[br]during the installation. 9:59:59.000,9:59:59.000 [Q] Ok, but there's no need for this to be[br]nfs, it could be a 9:59:59.000,9:59:59.000 [A] It's called nfsroot. 9:59:59.000,9:59:59.000 This is very common that people call it[br]nfsroot 9:59:59.000,9:59:59.000 and if you have this network installation[br]thing, it's really an nfsroot. 9:59:59.000,9:59:59.000 But you're right. 9:59:59.000,9:59:59.000 On the installation ISO, it's not nfs,[br]it's just a local file system, yes. 9:59:59.000,9:59:59.000 [Q] Alright. So, I think it's my last[br]comment. 9:59:59.000,9:59:59.000 You have the ISO from which you install[br]and when you install from the ISO 9:59:59.000,9:59:59.000 you're installing then packages on the[br]machine, and then you have the image 9:59:59.000,9:59:59.000 which is like a disk image that has[br]the packages already installed, 9:59:59.000,9:59:59.000 so you skip the installing step. 9:59:59.000,9:59:59.000 Have you thought about having[br]an intermediate thing 9:59:59.000,9:59:59.000 where you download an image that[br]already has the packages installed? 9:59:59.000,9:59:59.000 [A] That's also possible. 9:59:59.000,9:59:59.000 When you do an installation, before you[br]can change root in the new system 9:59:59.000,9:59:59.000 for adding packages, you have to call[br]debootstrap. 9:59:59.000,9:59:59.000 What we do, we call debootstrap once and[br]create a tar file out of it. 9:59:59.000,9:59:59.000 This is our minimal… in the former days[br]it was the floppy disk, 9:59:59.000,9:59:59.000 our base tar.gz file, 9:59:59.000,9:59:59.000 so you could exchange the minimal tar file[br]with whatever tar file you have. 9:59:59.000,9:59:59.000 That's for example what we do if we install[br]Ubuntu. 9:59:59.000,9:59:59.000 We boot the installation system which is[br]a Debian system 9:59:59.000,9:59:59.000 and then create the local filesystem and[br]extract an Ubuntu base image 9:59:59.000,9:59:59.000 and then we can change root into[br]the Ubunto or the same for CentOS and so on 9:59:59.000,9:59:59.000 then we can change root into the other[br]Linux system and add packages there. 9:59:59.000,9:59:59.000 If you have already a bigger image with[br]some more packages added there, 9:59:59.000,9:59:59.000 it's very easy to say "Do not extract the[br]Debian Stretch image 9:59:59.000,9:59:59.000 but use my image which also[br]includes other tools." 9:59:59.000,9:59:59.000 And if you are fine with that, you can[br]just extract the tar file. 9:59:59.000,9:59:59.000 [Q] Ok. Any more questions? 9:59:59.000,9:59:59.000 [Q] The heading is in german. 9:59:59.000,9:59:59.000 [A] What? 9:59:59.000,9:59:59.000 [Q] The heading is in german. 9:59:59.000,9:59:59.000 [A] Oh, because it's a copy of my german[br]slides. 9:59:59.000,9:59:59.000 Thank you for this. 9:59:59.000,9:59:59.000 And, what's also missing. 9:59:59.000,9:59:59.000 The web page, where you can select german[br]or other languages, 9:59:59.000,9:59:59.000 it would be nice if people are interested[br]to help translate them 9:59:59.000,9:59:59.000 so that it's more easy for people that[br]do not speak english 9:59:59.000,9:59:59.000 to use the website and create their own[br]installation image with their language. 9:59:59.000,9:59:59.000 [Q] Someone on the stream said that[br]the fai.me web site is not yours 9:59:59.000,9:59:59.000 and it's a hack thing, it's a scam, you go[br]there and get hacked. 9:59:59.000,9:59:59.000 Do you have any plans to try to buy[br]the domain because it's pretty confusing. 9:59:59.000,9:59:59.000 The first thing I would have done[br]by seeing that talk would have been 9:59:59.000,9:59:59.000 to go to fai.me. 9:59:59.000,9:59:59.000 [A] Yeah, I was thinking about which name[br]I should choose. 9:59:59.000,9:59:59.000 I didn't check which web domains are free[br]and in then end I thought 9:59:59.000,9:59:59.000 "Do I like to have a different domain name[br]for the service?" 9:59:59.000,9:59:59.000 But since it's only a part of the FAI[br]project, I thought, 9:59:59.000,9:59:59.000 and technically it was also easier just[br]to host it under a subdirectory 9:59:59.000,9:59:59.000 and yeah, if people now grab the fai.me[br]domain and do other things with it, yeah. 9:59:59.000,9:59:59.000 I think one question would be to use[br]a debian.net or maybe debian.org domain 9:59:59.000,9:59:59.000 because people trust much more. 9:59:59.000,9:59:59.000 I get a lot of comments "Oh this would[br]be very nice if this would be hosted 9:59:59.000,9:59:59.000 on a Debian machine" 9:59:59.000,9:59:59.000 but this would be much more complicated[br]because the DSA team 9:59:59.000,9:59:59.000 has much more restriction what to execute[br]on their machines. 9:59:59.000,9:59:59.000 currently, we need root access because[br]we mount some things 9:59:59.000,9:59:59.000 and DSA would not give root access[br]on any Debian machine. 9:59:59.000,9:59:59.000 That's the same problem we have[br]in the Debian cloud team 9:59:59.000,9:59:59.000 where we want to create the official[br]images for the cloud providers 9:59:59.000,9:59:59.000 where the Debian cloud team will also[br]not have root access 9:59:59.000,9:59:59.000 and so there's much more work to get[br]empty virtual machines 9:59:59.000,9:59:59.000 starting up, putting data into it,[br]creating the images, 9:59:59.000,9:59:59.000 receiving them from inside the image. 9:59:59.000,9:59:59.000 And since on those machines I have[br]root access, that's much easier for me. 9:59:59.000,9:59:59.000 We are out of time, so thank you Thomas. 9:59:59.000,9:59:59.000 [Applause]