9:59:59.000,9:59:59.000 Today, I talk about FAI.me, which is a[br]build for images. 9:59:59.000,9:59:59.000 First, anybody that never heard anything[br]about FAI? 9:59:59.000,9:59:59.000 Ok 9:59:59.000,9:59:59.000 I started this project in 1999. 9:59:59.000,9:59:59.000 I'm not sure… 9:59:59.000,9:59:59.000 No, I'm sure that during those times, the[br]Debian installer did not have 9:59:59.000,9:59:59.000 the preseeding stuff, so we needed[br]something automatically. 9:59:59.000,9:59:59.000 I installed the first cluster with FAI and[br]I always do talks on FAI or 9:59:59.000,9:59:59.000 today in the lightning talks, I talk[br]a little bit about dracut, 9:59:59.000,9:59:59.000 which is used in FAI. 9:59:59.000,9:59:59.000 So, what was the motivation. 9:59:59.000,9:59:59.000 A neighbour of mine, she came to me with 9:59:59.000,9:59:59.000 "My Windows desktop is broken,[br]can you reinstall it?" 9:59:59.000,9:59:59.000 And in the end, I installed her Linux,[br]and I was shortly thinking about 9:59:59.000,9:59:59.000 "Should I use FAI for installing her[br]desktop with Linux?" 9:59:59.000,9:59:59.000 And in the end, I did not use it because[br]FAI is too complicated, 9:59:59.000,9:59:59.000 like the Debian installer, I guess it's[br]not really that easy for beginners 9:59:59.000,9:59:59.000 because there are a lot of questions 9:59:59.000,9:59:59.000 but also FAI is not really for beginners. 9:59:59.000,9:59:59.000 So this was the motivation about thinking[br]about FAI. 9:59:59.000,9:59:59.000 The target group was always advanced[br]sysadmins 9:59:59.000,9:59:59.000 but I thought maybe it's possible to make[br]FAI usable also for people 9:59:59.000,9:59:59.000 that are not that advanced sysadmins. 9:59:59.000,9:59:59.000 The idea is that an installer should cover[br]most installations. 9:59:59.000,9:59:59.000 The Debian installer is really perfect[br]because I think it covers 9:59:59.000,9:59:59.000 all different kinds and strange environments 9:59:59.000,9:59:59.000 You can do a lot of things, you can configure[br]very strange combination of language, 9:59:59.000,9:59:59.000 keyboard layout and so on 9:59:59.000,9:59:59.000 but I was thinking about an installer[br]that covers 90 or 95% of the installations 9:59:59.000,9:59:59.000 A lot of special cases can be ignored and[br]since the Debian installer has like 9:59:59.000,9:59:59.000 more than 20 questions, I thought it would[br]be much nicer if there were only 9:59:59.000,9:59:59.000 3 to 5 questions and I looked at Linux Mint[br]and Mageia installers, CentOS installer, 9:59:59.000,9:59:59.000 and they all ask much less questions. 9:59:59.000,9:59:59.000 In the Debian installer, we sometimes[br]have also things that are asked 9:59:59.000,9:59:59.000 during the installation, so not everything[br]is asked at the very beginning. 9:59:59.000,9:59:59.000 For example, the task selection, where you[br]select your desktop, 9:59:59.000,9:59:59.000 is done after the base installation. 9:59:59.000,9:59:59.000 This was also very important, I would like[br]to have something that 9:59:59.000,9:59:59.000 asks everything at the very beginning. 9:59:59.000,9:59:59.000 Then, maybe some tool could create[br]a customized installation image 9:59:59.000,9:59:59.000 and this installation image should run[br]then completely unattended 9:59:59.000,9:59:59.000 so you can get yourself a coffee and[br]when you come back, your machine is ready. 9:59:59.000,9:59:59.000 There are 3 things to customize installation[br]image, 9:59:59.000,9:59:59.000 you just put this image, you do not have[br]to touch anything, and then it's ready. 9:59:59.000,9:59:59.000 I thought "Oh yes, this is FAI, maybe[br]FAI can do this." 9:59:59.000,9:59:59.000 As I said, FAI is only, or was until now[br]only a tool for experienced sysadmins 9:59:59.000,9:59:59.000 and you have to adjust several config[br]files, these are ASCII files 9:59:59.000,9:59:59.000 but still you have to touch 5 to 10[br]config files to make a customization. 9:59:59.000,9:59:59.000 So, how can I make FAI usable for[br]beginners? 9:59:59.000,9:59:59.000 That's the beginning of FAI.me. 9:59:59.000,9:59:59.000 There's a web page, we'll show it[br]in more detail later, 9:59:59.000,9:59:59.000 where you can just click some things, and[br]then you get a customized image. 9:59:59.000,9:59:59.000 This image can be put onto a CD, DVD or[br]USB stick, just with dd 9:59:59.000,9:59:59.000 and the customization is just by using[br]the web interface 9:59:59.000,9:59:59.000 so there's no need for you to edit[br]a text file, a config file inside FAI. 9:59:59.000,9:59:59.000 I hope I covered most important thangs[br]that you want to adjust 9:59:59.000,9:59:59.000 or a little bit customize. 9:59:59.000,9:59:59.000 You can add additional packages, I think[br]that's the most important thing 9:59:59.000,9:59:59.000 that people say "I want to have the normal[br]Debian installation 9:59:59.000,9:59:59.000 but with some additional packages." 9:59:59.000,9:59:59.000 And you can select different different[br]distributions, so it's not only 9:59:59.000,9:59:59.000 the installation image for the stable[br]release, you can create 9:59:59.000,9:59:59.000 3 variants of the installation. 9:59:59.000,9:59:59.000 This is the web page and thanks to Juri,[br]he did a great job 9:59:59.000,9:59:59.000 during the first and second day, he added[br]a new feature that we now have 9:59:59.000,9:59:59.000 a toggle button. 9:59:59.000,9:59:59.000 Is it big enough or should I zoom in? 9:59:59.000,9:59:59.000 Ok. 9:59:59.000,9:59:59.000 So, we have a toggle button, what you see[br]now is just the bare minimum or questions 9:59:59.000,9:59:59.000 and we can toggle it to more advanced[br]settings. 9:59:59.000,9:59:59.000 You have to select or just leave this as[br]it is, username, 9:59:59.000,9:59:59.000 if you do not enter a password, a password[br]will be generated and shown to you 9:59:59.000,9:59:59.000 and sent by e-mail. 9:59:59.000,9:59:59.000 I will now just type in the password. 9:59:59.000,9:59:59.000 It's here in clear text, for me that's fine[br]because 9:59:59.000,9:59:59.000 there's also a comment that you should[br]change the password after the installation 9:59:59.000,9:59:59.000 and I do not like to enter passwords twice[br]so you can see what you typed in 9:59:59.000,9:59:59.000 and hopefully do not make any wrong[br]mistakes. 9:59:59.000,9:59:59.000 For example, we could select the Stretch[br]distribution with backports, 9:59:59.000,9:59:59.000 so we will get a 4.15 kernel with Stretch. 9:59:59.000,9:59:59.000 There are some buttons we can say we want[br]to have some Debian developer tools. 9:59:59.000,9:59:59.000 This is what I defined in the FAI[br]configuration, so just a list of packages. 9:59:59.000,9:59:59.000 Here, you can enter you own packages.[br] 9:59:59.000,9:59:59.000 I will select the desktop. 9:59:59.000,9:59:59.000 You can have an installation without any[br]desktop, so a very small installation. 9:59:59.000,9:59:59.000 I will select the XFCE desktop, but all[br]the other desktops are here. 9:59:59.000,9:59:59.000 The language, these are just task packages[br]that are… 9:59:59.000,9:59:59.000 I think Debian has much more task packages,[br]I just searched which are 9:59:59.000,9:59:59.000 the most common languages, and what I do[br]if I say I want the spanish language, 9:59:59.000,9:59:59.000 also the keyboard layout is spanish. 9:59:59.000,9:59:59.000 I know there are different combinations[br]and with local time, 9:59:59.000,9:59:59.000 it's getting more difficult. 9:59:59.000,9:59:59.000 This installation will install the clock[br]with UTC, so if you want to set 9:59:59.000,9:59:59.000 your time, you have to do this manually. 9:59:59.000,9:59:59.000 I want to cover the most common installations. 9:59:59.000,9:59:59.000 We select english US, the desktop and,[br]as an example, the midnight commander 9:59:59.000,9:59:59.000 and GIMP. 9:59:59.000,9:59:59.000 I can add an email address so if it would[br]take longer, 9:59:59.000,9:59:59.000 for example if this service will have[br]success and a lot of people are using it, 9:59:59.000,9:59:59.000 you may wait for some minutes so your job[br]will be finished. 9:59:59.000,9:59:59.000 So here are the comments, how to reconfigure[br]the keyboard or the timezone 9:59:59.000,9:59:59.000 and then you just click "Create[br]the installation image". 9:59:59.000,9:59:59.000 Now, in the background, there's some job,[br]a script, looking "Oh, there's a new job" 9:59:59.000,9:59:59.000 and there's a summary of the configuration,[br]of the web configuration. 9:59:59.000,9:59:59.000 Down here you see these are the[br]FAI classes, 9:59:59.000,9:59:59.000 I will explain a little bit more about this. 9:59:59.000,9:59:59.000 But with this information, FAI configuration[br]is generated, 9:59:59.000,9:59:59.000 that's what normally the experienced[br]sysadmins have to create 9:59:59.000,9:59:59.000 but here you just click on some buttons[br]and it will be done for you. 9:59:59.000,9:59:59.000 In the meantime, we have some more[br]advanced features 9:59:59.000,9:59:59.000 which I will also show you later. 9:59:59.000,9:59:59.000 For example, this very simple installation[br]just creates one partition 9:59:59.000,9:59:59.000 but you can also select that you want[br]to have a separate /home partition 9:59:59.000,9:59:59.000 or using lvm just by selecting this[br]on the web interface. 9:59:59.000,9:59:59.000 You can also add your SSH public key[br]for logging as root without a password 9:59:59.000,9:59:59.000 or what's very nice, I found the new[br]Ubuntu installer does this, 9:59:59.000,9:59:59.000 you can give your github account and[br]then there's a comment which 9:59:59.000,9:59:59.000 receives the public key from your[br]github account and puts it 9:59:59.000,9:59:59.000 into the root account so you can log in[br]without password. 9:59:59.000,9:59:59.000 I think that's very neat. 9:59:59.000,9:59:59.000 And if you have a repository with your own[br]packages, you could also add this and say 9:59:59.000,9:59:59.000 "Please install those packages from[br]my publicly available repository." 9:59:59.000,9:59:59.000 Let's see. 9:59:59.000,9:59:59.000 As we see, this job finished in 74 seconds. 9:59:59.000,9:59:59.000 Now, this customised installation image[br]is available for download. 9:59:59.000,9:59:59.000 You can also download the log file. 9:59:59.000,9:59:59.000 Since this is an installation image,[br]I first have to create 9:59:59.000,9:59:59.000 a partial package mirror. 9:59:59.000,9:59:59.000 This is done by the command 'fai-mirror'[br]and you can also read the log of 9:59:59.000,9:59:59.000 this call of the fai-mirror, where a list[br]of all you packages with all the dependencies 9:59:59.000,9:59:59.000 are available. 9:59:59.000,9:59:59.000 So you see, these are the list of packages[br]and later they are downloaded 9:59:59.000,9:59:59.000 and in the end, it says it created a mirror[br]of 1G of packages 9:59:59.000,9:59:59.000 and since I have a local mirror,[br]it's very fast. 9:59:59.000,9:59:59.000 This is the one part on the installation[br]image partial mirror with all the packages 9:59:59.000,9:59:59.000 and the other is that the config space[br]which you can also download. 9:59:59.000,9:59:59.000 So this is the config space that was[br]really created for you 9:59:59.000,9:59:59.000 by clicking the web interface. 9:59:59.000,9:59:59.000 If you want to do more things with FAI,[br]you can set up your own FAI server 9:59:59.000,9:59:59.000 and use this configuration space. 9:59:59.000,9:59:59.000 And, that's also very new, the two commands[br]that are used for creating this ISO image 9:59:59.000,9:59:59.000 are now also listed there. 9:59:59.000,9:59:59.000 First, create the partial mirror and then[br]create the installation image. 9:59:59.000,9:59:59.000 Ok, copy link location… 9:59:59.000,9:59:59.000 Let's see how good the network is here. 9:59:59.000,9:59:59.000 [Q] It's a rather large image. 9:59:59.000,9:59:59.000 Yeah, because it includes all the packages[br]and with Xfce, LibreOffice and so on 9:59:59.000,9:59:59.000 and the installation environment is maybe[br]about 200MB. 9:59:59.000,9:59:59.000 That's not much bigger than the Debian[br]installer that you need to download. 9:59:59.000,9:59:59.000 So, 2, 1, done. 9:59:59.000,9:59:59.000 I have a little wrapper which calls[br]a fresh kvm machine 9:59:59.000,9:59:59.000 with an empty disk and boots this ISO image 9:59:59.000,9:59:59.000 and then we will see how this installation[br]runs. 9:59:59.000,9:59:59.000 So this is dracut booting the image 9:59:59.000,9:59:59.000 and now you see there are already[br]some parted commands executed 9:59:59.000,9:59:59.000 and now the packages are installed[br]and everything runs on 9:59:59.000,9:59:59.000 and in the end some customization[br]script. 9:59:59.000,9:59:59.000 We use only shell scripts for doing[br]some customizations 9:59:59.000,9:59:59.000 and you see the files are downloaded[br]from /media/mirror 9:59:59.000,9:59:59.000 so this is local on the ISO image. 9:59:59.000,9:59:59.000 It would also be possible to create[br]an image without the packages 9:59:59.000,9:59:59.000 and then give another sources.list file[br]so the packages would be downloaded 9:59:59.000,9:59:59.000 from the internet but this default[br]in the FAI service, 9:59:59.000,9:59:59.000 we put everything onto the ISO image. 9:59:59.000,9:59:59.000 I guess it will run for 4 minutes. 9:59:59.000,9:59:59.000 [Q] ??? 9:59:59.000,9:59:59.000 Yeah. 9:59:59.000,9:59:59.000 What I will show you now is… 9:59:59.000,9:59:59.000 So, this was the simple one, now I toggle[br]this web page and 9:59:59.000,9:59:59.000 you will see that there are some more[br]questions you may answer, 9:59:59.000,9:59:59.000 for example you can give a root password. 9:59:59.000,9:59:59.000 If you leave this empty, sudo will be[br]configured. 9:59:59.000,9:59:59.000 Here you can upload the SSH key or give[br]you Github account, 9:59:59.000,9:59:59.000 that would be Mrfai for me. 9:59:59.000,9:59:59.000 With the partitioning schemes, we have[br]one partition 9:59:59.000,9:59:59.000 or one partition and /home separated[br]or these two versions with LVM. 9:59:59.000,9:59:59.000 FAI itself can do much more, we could do[br]soft raid set ups, cryptsetup 9:59:59.000,9:59:59.000 but here I want to cover the most common[br]installation, so very simple 9:59:59.000,9:59:59.000 we have only 4 things that you can choose. 9:59:59.000,9:59:59.000 [Q] For encryption? 9:59:59.000,9:59:59.000 [A] Yes. 9:59:59.000,9:59:59.000 So, this was the partitioning things. 9:59:59.000,9:59:59.000 This is the new feature where you can add[br]an URL for your local package repository 9:59:59.000,9:59:59.000 and the rest is the same, you can add[br]packages you like, your email address 9:59:59.000,9:59:59.000 and then also create an installation image. 9:59:59.000,9:59:59.000 I normally set by default, I include[br]the nonfree linux firmware. 9:59:59.000,9:59:59.000 This is because my target audience is[br]an end user and I want to make it 9:59:59.000,9:59:59.000 very comfortable for them, so yeah,[br]they can just install it 9:59:59.000,9:59:59.000 and do not have these problems. 9:59:59.000,9:59:59.000 And since this is not on an officiel[br]debian.org web site, 9:59:59.000,9:59:59.000 I can do this with this default. 9:59:59.000,9:59:59.000 Let's see, the installation is still running. 9:59:59.000,9:59:59.000 So, advanced features. 9:59:59.000,9:59:59.000 The next thing after this installation, 9:59:59.000,9:59:59.000 I will show you how to create cloud[br]images. 9:59:59.000,9:59:59.000 Currently, we create an installation image, 9:59:59.000,9:59:59.000 when you boot it, the installation is run[br]fully automatically. 9:59:59.000,9:59:59.000 The other type of service FAI.me gives[br]to you is that it creates a raw image 9:59:59.000,9:59:59.000 or some other formats as you see here,[br]qcow2 and whatever 9:59:59.000,9:59:59.000 which you can just boot and[br]the installation is already done. 9:59:59.000,9:59:59.000 But first, see if the installation finish. 9:59:59.000,9:59:59.000 Ah! Ok. 9:59:59.000,9:59:59.000 These are now the shell scripts that are[br]executed for the customization 9:59:59.000,9:59:59.000 of like /etc/messages of today,[br]/etc/network/interfaces is written 9:59:59.000,9:59:59.000 and so on. 9:59:59.000,9:59:59.000 You see the installation took 236s, it says[br]there are some errors 9:59:59.000,9:59:59.000 but that's not really true. 9:59:59.000,9:59:59.000 And it stops here, but we can also[br]disable this, this is only for showing 9:59:59.000,9:59:59.000 everything went well 9:59:59.000,9:59:59.000 and now we just reboot the machine. 9:59:59.000,9:59:59.000 You see the grub. 9:59:59.000,9:59:59.000 Ok, Xfce desktop. 9:59:59.000,9:59:59.000 debian was a user with password FAI. 9:59:59.000,9:59:59.000 We have "uname -a", this is 4.16,[br]the backports kernel was installed there. 9:59:59.000,9:59:59.000 We have only one partition, no LVM 9:59:59.000,9:59:59.000 and I told it to install gimp which is[br]not installed by default. 9:59:59.000,9:59:59.000 Gimp is there, so this is nice. 9:59:59.000,9:59:59.000 And the midnight commander is also there. 9:59:59.000,9:59:59.000 And now we just throw this machine.[br]Gone. 9:59:59.000,9:59:59.000 What's very nice with this wrapper script,[br]it creates the local disk 9:59:59.000,9:59:59.000 of the virtual machine in /tmp which is[br]a RAM disk and I love RAM, 9:59:59.000,9:59:59.000 it's so nice and fast. 9:59:59.000,9:59:59.000 So, this was installation image and now we[br]look at the cloud image. 9:59:59.000,9:59:59.000 First, you can say how big should your[br]disk image be. 9:59:59.000,9:59:59.000 Here, I say 8GB, you will see it's not[br]an 8GB image 9:59:59.000,9:59:59.000 that you have to download later. 9:59:59.000,9:59:59.000 By default, I use zstd compression. 9:59:59.000,9:59:59.000 Anyone who dose not know[br]this compression? 9:59:59.000,9:59:59.000 This is very fast, very new, created[br]by Facebook if I'm correct. 9:59:59.000,9:59:59.000 It's for very big files and what you should[br]never use is gzip with sparse images. 9:59:59.000,9:59:59.000 The disk image is sparse and gzip[br]cannot handle this 9:59:59.000,9:59:59.000 so if you compress it and uncompress it[br]it will be very large 9:59:59.000,9:59:59.000 and all the other, xz, zstd, can handle[br]sparse files very nicely. 9:59:59.000,9:59:59.000 So, the hostname is set, the root password,[br]username with a password. 9:59:59.000,9:59:59.000 Now we want to install Buster. 9:59:59.000,9:59:59.000 Maybe with no… 9:59:59.000,9:59:59.000 Oh, we also do the Xfce desktop. 9:59:59.000,9:59:59.000 Any packages you'd like to have in this[br]cloud image. 9:59:59.000,9:59:59.000 "desktop" and "cloud" image does not[br]make that much sense, maybe. 9:59:59.000,9:59:59.000 Emacs25, ok. 9:59:59.000,9:59:59.000 And now "Create disk image". 9:59:59.000,9:59:59.000 This will take a little bit longer because[br]we are doing the installation 9:59:59.000,9:59:59.000 inside a file image. 9:59:59.000,9:59:59.000 But no problem, I can tell you what[br]other ideas I have. 9:59:59.000,9:59:59.000 So, currently we have the installation and[br]the cloud or virtual machine images for amd64 9:59:59.000,9:59:59.000 FAI itself can also do cross-architecture[br]images so it would be some work 9:59:59.000,9:59:59.000 to extend the web page to say[br]"Please create an arm64 image" 9:59:59.000,9:59:59.000 It would be very nice to have predefined[br]configs for raspberry pi or 9:59:59.000,9:59:59.000 all the very different boards 9:59:59.000,9:59:59.000 but that would also be possible. 9:59:59.000,9:59:59.000 I guess the next thing I will implement[br]is other distributions 9:59:59.000,9:59:59.000 because I know people are always asking it. 9:59:59.000,9:59:59.000 Not you but the Ubuntu guys. 9:59:59.000,9:59:59.000 Yesterday I did the first test with Ubuntu[br]bionic, the LTS release 9:59:59.000,9:59:59.000 and FAI just works out of the box with it. 9:59:59.000,9:59:59.000 So what I have to do is to integrate it[br]in these FAI.me processing scripts. 9:59:59.000,9:59:59.000 Ready-to-go cloud images for the big[br]cloud providers. 9:59:59.000,9:59:59.000 That's only a different FAI config space[br]that I have to use. 9:59:59.000,9:59:59.000 Currently, for example, in what I call[br]cloud images, I do not install 9:59:59.000,9:59:59.000 the package cloud image. 9:59:59.000,9:59:59.000 That's needed for all the ones. 9:59:59.000,9:59:59.000 I'm also working in the Debian cloud team[br]and this team decided 2 years ago 9:59:59.000,9:59:59.000 that the tool chain in the future for[br]the official Debian package will be FAI. 9:59:59.000,9:59:59.000 Amazon is already using it, so if you[br]boot or if you use a Debian cloud image 9:59:59.000,9:59:59.000 in Amazon, Noah Meyerhans did this and[br]he's using the FAI tool chain for it. 9:59:59.000,9:59:59.000 Google is not yet using because there was[br]a very small problem 9:59:59.000,9:59:59.000 in a config file we had one space too much 9:59:59.000,9:59:59.000 which caused grub to hang forever 9:59:59.000,9:59:59.000 and that was the reason why they decided[br]for Stretch to use their auto tool chain. 9:59:59.000,9:59:59.000 But the things are working so we have[br]the config space also for Google. 9:59:59.000,9:59:59.000 And also for Azure, some people from[br]Credativ did this. 9:59:59.000,9:59:59.000 The Debian cloud team already has[br]the FAI configuration for 9:59:59.000,9:59:59.000 the big tool providers,[br]cloud providers. 9:59:59.000,9:59:59.000 We could also think on a more generic[br]FAI installation image. 9:59:59.000,9:59:59.000 It's an image that you would boot up[br]and then enter your job id of the web page 9:59:59.000,9:59:59.000 and then the configuration would be[br]downloaded 9:59:59.000,9:59:59.000 and the packages would be just[br]received from the internet. 9:59:59.000,9:59:59.000 That was one… 9:59:59.000,9:59:59.000 So, the image would be much smaller[br]because the packages do not need to be 9:59:59.000,9:59:59.000 on the installation image. 9:59:59.000,9:59:59.000 It's also possible to create live images[br]with FAI. 9:59:59.000,9:59:59.000 It is a little bit more… 9:59:59.000,9:59:59.000 Currently, you need some manual work[br]but that should be also possible 9:59:59.000,9:59:59.000 to use FAI for creating live image[br]and then also to provide this 9:59:59.000,9:59:59.000 on the FAI.me web service. 9:59:59.000,9:59:59.000 If you want to customize much more[br]inside the image, you just say 9:59:59.000,9:59:59.000 "Oh, I have some ansible scripts that[br]I want to execute at the very end" 9:59:59.000,9:59:59.000 then I say "Ok, this is just a starting[br]point, use the FAI.me service 9:59:59.000,9:59:59.000 and if you're happy with the FAI tools,[br]then set up your own FAI server, 9:59:59.000,9:59:59.000 create your own configuration space[br]and then you can do all the crazy things." 9:59:59.000,9:59:59.000 So, how does FAI.me work internally. 9:59:59.000,9:59:59.000 We have a web server where there are[br]some CGI scripts and 9:59:59.000,9:59:59.000 this is not the build server, so on the[br]web server, you click "Submit" 9:59:59.000,9:59:59.000 "Create my image", all the input[br]is validated so you cannot make nasty things 9:59:59.000,9:59:59.000 and then the CGI writes or creates[br]a subdirectory and puts files in it, 9:59:59.000,9:59:59.000 a config and a meta file 9:59:59.000,9:59:59.000 and writes a status[br]"waiting for processing". 9:59:59.000,9:59:59.000 Then, the other server, the build server[br]reads this config and 9:59:59.000,9:59:59.000 this is just an NFS mounted directory,[br]and sees 9:59:59.000,9:59:59.000 "Oh there's a new job I have to process". 9:59:59.000,9:59:59.000 In this processing script we pass for[br]some errors. 9:59:59.000,9:59:59.000 What's happening very often that people[br]type in a package that's not available 9:59:59.000,9:59:59.000 and this will be detected and then a new[br]version of the web page will pop up and say 9:59:59.000,9:59:59.000 "Oh, when creating the package mirror,[br]there was an error 9:59:59.000,9:59:59.000 because this package was not known." 9:59:59.000,9:59:59.000 Sometimes I have to… 9:59:59.000,9:59:59.000 Every night, I create new nfsroots[br]for Buster. 9:59:59.000,9:59:59.000 If there are security updates, I have[br]to create new nfsroots 9:59:59.000,9:59:59.000 for Stretch and backports. 9:59:59.000,9:59:59.000 I have some cleanup, so if a lot of jobs[br]are created, 9:59:59.000,9:59:59.000 the images are on the disk after, 9:59:59.000,9:59:59.000 normally I say after one day I just[br]remove the images 9:59:59.000,9:59:59.000 so you have one day to download[br]the images. 9:59:59.000,9:59:59.000 There's 3 different configurations 9:59:59.000,9:59:59.000 /etc/fai-stretch, /etc/buster,[br]/etc/fai-stretch-bpo (backports) 9:59:59.000,9:59:59.000 We need for the installation image 9:59:59.000,9:59:59.000 We need a different nfsroot, but[br]the config space that is shared 9:59:59.000,9:59:59.000 about all configurations, 9:59:59.000,9:59:59.000 so it doesn't matter if I install 9:59:59.000,9:59:59.000 Stretch or Stretch backports or[br]Buster, 9:59:59.000,9:59:59.000 I can use the same FAI configuration. 9:59:59.000,9:59:59.000 Also, for building the cloud images,[br]I use the same FAI configuration. 9:59:59.000,9:59:59.000 A new job is detected, then a copy of[br]the configuration space will be made 9:59:59.000,9:59:59.000 and it will be customized a little bit. 9:59:59.000,9:59:59.000 So there are a very very few changes, 9:59:59.000,9:59:59.000 for example I have to put the SSH key[br]into your customized configuration space 9:59:59.000,9:59:59.000 or the list of packages or the user and[br]root password. 9:59:59.000,9:59:59.000 Then we have two things, if we want to[br]create the installation image, 9:59:59.000,9:59:59.000 I first have to create the partial package[br]mirror and then create the installation image 9:59:59.000,9:59:59.000 For the cloud images, we do not need[br]the nfsroot, we just need 9:59:59.000,9:59:59.000 the configuration space which is[br]customized a little bit 9:59:59.000,9:59:59.000 and then we can just create the disk image 9:59:59.000,9:59:59.000 so there's one step less compared to[br]creating the installation ISO. 9:59:59.000,9:59:59.000 The status on the web page will be[br]updated, log files written 9:59:59.000,9:59:59.000 and if the user said "Please send me[br]an email if my job is ready", 9:59:59.000,9:59:59.000 this will also be sent to the user. 9:59:59.000,9:59:59.000 The we have the ISO or the disk image[br]and this will be copied back 9:59:59.000,9:59:59.000 to the web server where the user can then[br]download it. 9:59:59.000,9:59:59.000 And since I have a lot of RAM[br]in this machine, 9:59:59.000,9:59:59.000 everything is run in RAM, very very nice. 9:59:59.000,9:59:59.000 As I said, we need an nfsroot,[br]a configuration space and FAI classes. 9:59:59.000,9:59:59.000 This is a very central component in FAI 9:59:59.000,9:59:59.000 and this is just a list of names. 9:59:59.000,9:59:59.000 So in HOME_LVM, this is the class name,[br]the FAI class we describe 9:59:59.000,9:59:59.000 and I think this is that example: 9:59:59.000,9:59:59.000 HOME_LVM describes how to partition[br]the local hard disk. 9:59:59.000,9:59:59.000 This is our very flexible tool where[br]we can do LVM, cryptsetups, 9:59:59.000,9:59:59.000 software RAIDs and so on. 9:59:59.000,9:59:59.000 But for the FAI.me service, I just created[br]4 different types of partitioning 9:59:59.000,9:59:59.000 and this is the HOME_LVM example. 9:59:59.000,9:59:59.000 So we have a list of classes and,[br]as I said, 9:59:59.000,9:59:59.000 just two commands for the installation[br]image with a list of classes 9:59:59.000,9:59:59.000 and for the cloud image, I have to say[br]how big should the disk image be, 9:59:59.000,9:59:59.000 the list of classes and what's the target[br]file that should be created. 9:59:59.000,9:59:59.000 Let's see if this is ready. 9:59:59.000,9:59:59.000 Yes, it's ready. 9:59:59.000,9:59:59.000 So… 9:59:59.000,9:59:59.000 It's 1.1GB. 9:59:59.000,9:59:59.000 Is this really the… oh yeah, raw. 9:59:59.000,9:59:59.000 No problem, let's download it, it should[br]be fast. 9:59:59.000,9:59:59.000 This is the normal architecture if you use[br]FAI in a client/server set up. 9:59:59.000,9:59:59.000 You should just look on the left side[br]where you see 9:59:59.000,9:59:59.000 you need the config space, an nfsroot[br]and a mirror 9:59:59.000,9:59:59.000 and these parts will put onto the CD. 9:59:59.000,9:59:59.000 If you ??? network installation thing, 9:59:59.000,9:59:59.000 this is of things get from the server[br]to the client. 9:59:59.000,9:59:59.000 For the software installation, we have[br]another subdirectory called package_config 9:59:59.000,9:59:59.000 and there you also see several files[br]where the file name is a FAI class. 9:59:59.000,9:59:59.000 Since in the FAI.me service every client[br]belongs to the class DEBIAN, 9:59:59.000,9:59:59.000 it will install the packages that are listed[br]on the ??? 9:59:59.000,9:59:59.000 and here we have an other class, NONFREE 9:59:59.000,9:59:59.000 These packages are only installed if[br]you also said 9:59:59.000,9:59:59.000 "Please install the nonfree packages" 9:59:59.000,9:59:59.000 and this is mapped to a FAI class[br]called NONFREE. 9:59:59.000,9:59:59.000 And there's an other class for AMD64[br]and so on. 9:59:59.000,9:59:59.000 Some references. 9:59:59.000,9:59:59.000 In the past, it looked more like this[br]when I said 9:59:59.000,9:59:59.000 "Oh, who's using FAI?" and during[br]the last month I collected some logos 9:59:59.000,9:59:59.000 just because it's much nicer. 9:59:59.000,9:59:59.000 Let's see if the download was ready. 9:59:59.000,9:59:59.000 We unzstd the FAI.me image,[br]faime-013Z image 9:59:59.000,9:59:59.000 On the web site, I said I want to have[br]a 8GB partition, 9:59:59.000,9:59:59.000 so now let's see how big it is. 9:59:59.000,9:59:59.000 The file is 8, but since it's a sparse file[br]it's only 3.5GB 9:59:59.000,9:59:59.000 and the compressed was 1.1GB. 9:59:59.000,9:59:59.000 Now I use my wrapper 9:59:59.000,9:59:59.000 and I say "Boot from disk" and this is[br]the FAI.me raw image, disk image 9:59:59.000,9:59:59.000 that should be booted up. 9:59:59.000,9:59:59.000 That's it. 9:59:59.000,9:59:59.000 debian/fai 9:59:59.000,9:59:59.000 Let's see if emacs is installed, yes. 9:59:59.000,9:59:59.000 Gimp is already there, hopefully, and[br]the blue midnight commander. 9:59:59.000,9:59:59.000 Let's see. 9:59:59.000,9:59:59.000 Questions. 9:59:59.000,9:59:59.000 [Q] I'm using the preseed file for[br]the debian-installer, 9:59:59.000,9:59:59.000 do you have a conversion between[br]your syntax and your configuration files 9:59:59.000,9:59:59.000 and the preseed file or maybe can you add[br]a download button for the preseed file 9:59:59.000,9:59:59.000 to your web site because I think it's[br]rather nice to have it displayed 9:59:59.000,9:59:59.000 in web site first. 9:59:59.000,9:59:59.000 [A] I'm not using the debian-installer. 9:59:59.000,9:59:59.000 I use preseeding, yes, the debconf[br]preseeding for the normal packages 9:59:59.000,9:59:59.000 you can do this also in FAI and it's[br]the same format 9:59:59.000,9:59:59.000 you get with debconf-get-selections. 9:59:59.000,9:59:59.000 And what you get is you can download[br]your own FAI config space 9:59:59.000,9:59:59.000 and this includes all information you need[br]to set up, 9:59:59.000,9:59:59.000 to do this mirror FAI CD or the FAI disk[br]image command. 9:59:59.000,9:59:59.000 But you cannot convert this config into[br]a d-i preseeding or vice versa, 9:59:59.000,9:59:59.000 that's not possible. 9:59:59.000,9:59:59.000 Because for example, for the partitioning[br]part I do not like to create 9:59:59.000,9:59:59.000 from my disk config partman preseeding[br]file. 9:59:59.000,9:59:59.000 You can pay me a lot of money, I will[br]never do this. 9:59:59.000,9:59:59.000 You know that the partman preseeding[br]is very ugly and very heavy. 9:59:59.000,9:59:59.000 For other things, yes, selection of,[br]for example, the selection of the language 9:59:59.000,9:59:59.000 These are the normal preseeding we use. 9:59:59.000,9:59:59.000 And the list of packages, task selec… 9:59:59.000,9:59:59.000 I think it's much easier to do this in[br]the FAI configuration than to create 9:59:59.000,9:59:59.000 a debian-installer preseeding. 9:59:59.000,9:59:59.000 And why use d-i if this works for you? 9:59:59.000,9:59:59.000 [Q] d-i works as well for me. 9:59:59.000,9:59:59.000 [A] Yes, then fine, use it. 9:59:59.000,9:59:59.000 [Q] Hi Thomas. Thank you very much for[br]this new feature in the FAI project, 9:59:59.000,9:59:59.000 it's very nice and I found very great that[br]you have the output of the commands 9:59:59.000,9:59:59.000 that you used to create the ISO image or[br]the cloud file. 9:59:59.000,9:59:59.000 A question that I have is, in which servers[br]are located the files that we create, 9:59:59.000,9:59:59.000 the ISO or the cloud. 9:59:59.000,9:59:59.000 Is it a server that you own host or… 9:59:59.000,9:59:59.000 [A] Trust me. 9:59:59.000,9:59:59.000 Currently, 9:59:59.000,9:59:59.000 both the web server and the FAI.me[br]processing build server are run 9:59:59.000,9:59:59.000 on two machines at the university where[br]I work as a system administrator 9:59:59.000,9:59:59.000 so that's also where we have a very fast[br]connection. 9:59:59.000,9:59:59.000 The CGI script and shell script that is[br]processing these jobs is currently 9:59:59.000,9:59:59.000 not open source. 9:59:59.000,9:59:59.000 There are plans to do this, I'm not sure[br]when. 9:59:59.000,9:59:59.000 If you want to reproduce the things, you[br]have the config file and you can download 9:59:59.000,9:59:59.000 the FAI software and use these one or two[br]commands to reproduce it. 9:59:59.000,9:59:59.000 Some people said "Oh, very nice service,[br]I would like to set up in my company". 9:59:59.000,9:59:59.000 Then please yes, contact me and… 9:59:59.000,9:59:59.000 Currently there are no concrete plans[br]to make these background scripts open source 9:59:59.000,9:59:59.000 but it will be in some future. 9:59:59.000,9:59:59.000 But currently, you have to trust me as[br]you also have to trust the package maintainers 9:59:59.000,9:59:59.000 that will be installed there. 9:59:59.000,9:59:59.000 But you can verify it or say "I do not[br]trust Thomas but I will just grab 9:59:59.000,9:59:59.000 the FAI config space and this on my own". 9:59:59.000,9:59:59.000 [Q] Thank you. 9:59:59.000,9:59:59.000 [Q] There's a question from the internet. 9:59:59.000,9:59:59.000 Why not use a proper job queuing system[br]like grid engine or similar? 9:59:59.000,9:59:59.000 [A] I'm using grid engine at work for[br]different things. 9:59:59.000,9:59:59.000 It started as a very simple project, so[br]in the end it's just a loop which 9:59:59.000,9:59:59.000 checks if there's new jobs on that. 9:59:59.000,9:59:59.000 Currently, I do not process jobs[br]in parallel, currently there's no need for it 9:59:59.000,9:59:59.000 If this project will be very successful,[br]yeah, I have to use a queuing system. 9:59:59.000,9:59:59.000 It's, yeah, a very simple script. 9:59:59.000,9:59:59.000 But it would be also possible with[br]a proper queuing system. 9:59:59.000,9:59:59.000 More questions? 9:59:59.000,9:59:59.000 [Q] I have a bunch of questions. 9:59:59.000,9:59:59.000 First, what is it that you use for[br]partitioning? 9:59:59.000,9:59:59.000 [A] I'm using a poll script that we wrote[br]several years ago in FAI 9:59:59.000,9:59:59.000 and we defined this config file,[br]this package config 9:59:59.000,9:59:59.000 and the Perl script parses this script[br]and then executes the parted and mkfs command 9:59:59.000,9:59:59.000 which you can see in the log files, so if[br]you want to see what does FAI do 9:59:59.000,9:59:59.000 after parsing this, which commands are[br]executed, you see everything 9:59:59.000,9:59:59.000 on the log files. 9:59:59.000,9:59:59.000 [Q] Right, but so you turn this text[br]into partitioning… 9:59:59.000,9:59:59.000 [A] commands, yeah. 9:59:59.000,9:59:59.000 [Q] But the text looks like this,[br]like with the spaces and everything. 9:59:59.000,9:59:59.000 [A] You can use more or less spaces[br]or do you like, 9:59:59.000,9:59:59.000 should I convert it to XML? 9:59:59.000,9:59:59.000 [Q] Ok, then my next question is 9:59:59.000,9:59:59.000 what are you using the nfsroot for when[br]you're generating the images? 9:59:59.000,9:59:59.000 [A] The nfsroot is used only for the[br]installation image. 9:59:59.000,9:59:59.000 When I do the installation, I need to boot[br]the machine as a diskless client, 9:59:59.000,9:59:59.000 so it's just what the debian-installer[br]into RAM, 9:59:59.000,9:59:59.000 you need a running Linux system. 9:59:59.000,9:59:59.000 This is our nfsroot, on the installation[br]image. 9:59:59.000,9:59:59.000 When you boot installation image, this[br]nfsroot with all the commands we need 9:59:59.000,9:59:59.000 are started without using the local disk[br]and then we can do 9:59:59.000,9:59:59.000 everything on the disk, /root and /target[br]and so on. 9:59:59.000,9:59:59.000 The nfsroot is the system that is running[br]during the installation. 9:59:59.000,9:59:59.000 [Q] Ok, but there's no need for this to be[br]nfs, it could be a 9:59:59.000,9:59:59.000 [A] It's called nfsroot. 9:59:59.000,9:59:59.000 This is very common that people call it[br]nfsroot 9:59:59.000,9:59:59.000 and if you have this network installation[br]thing, it's really an nfsroot. 9:59:59.000,9:59:59.000 But you're right. 9:59:59.000,9:59:59.000 On the installation ISO, it's not nfs,[br]it's just a local file system, yes. 9:59:59.000,9:59:59.000 [Q] Alright. So, I think it's my last[br]comment. 9:59:59.000,9:59:59.000 You have the ISO from which you install[br]and when you install from the ISO 9:59:59.000,9:59:59.000 you're installing then packages on the[br]machine, and then you have the image 9:59:59.000,9:59:59.000 which is like a disk image that has[br]the packages already installed, 9:59:59.000,9:59:59.000 so you skip the installing step. 9:59:59.000,9:59:59.000 Have you thought about having[br]an intermediate thing 9:59:59.000,9:59:59.000 where you download an image that[br]already has the packages installed? 9:59:59.000,9:59:59.000 [A] That's also possible. 9:59:59.000,9:59:59.000 When you do an installation, before you[br]can change root in the new system 9:59:59.000,9:59:59.000 for adding packages, you have to call[br]debootstrap. 9:59:59.000,9:59:59.000 What we do, we call debootstrap once and[br]create a tar file out of it. 9:59:59.000,9:59:59.000 This is our minimal… in the former days[br]it was the floppy disk, 9:59:59.000,9:59:59.000 our base tar.gz file, 9:59:59.000,9:59:59.000 so you could exchange the minimal tar file[br]with whatever tar file you have. 9:59:59.000,9:59:59.000 That's for example what we do if we install[br]Ubuntu. 9:59:59.000,9:59:59.000 We boot the installation system which is[br]a Debian system 9:59:59.000,9:59:59.000 and then create the local filesystem and[br]extract an Ubuntu base image 9:59:59.000,9:59:59.000 and then we can change root into[br]the Ubunto or the same for CentOS and so on 9:59:59.000,9:59:59.000 then we can change root into the other[br]Linux system and add packages there. 9:59:59.000,9:59:59.000 If you have already a bigger image with[br]some more packages added there, 9:59:59.000,9:59:59.000 it's very easy to say "Do not extract the[br]Debian Stretch image 9:59:59.000,9:59:59.000 but use my image which also[br]includes other tools." 9:59:59.000,9:59:59.000 And if you are fine with that, you can[br]just extract the tar file. 9:59:59.000,9:59:59.000 [Q] Ok. Any more questions? 9:59:59.000,9:59:59.000 [Q] The heading is in german. 9:59:59.000,9:59:59.000 [A] What? 9:59:59.000,9:59:59.000 [Q] The heading is in german. 9:59:59.000,9:59:59.000 [A] Oh, because it's a copy of my german[br]slides. 9:59:59.000,9:59:59.000 Thank you for this. 9:59:59.000,9:59:59.000 And, what's also missing. 9:59:59.000,9:59:59.000 The web page, where you can select german[br]or other languages, 9:59:59.000,9:59:59.000 it would be nice if people are interested[br]to help translate them 9:59:59.000,9:59:59.000 so that it's more easy for people that[br]do not speak english 9:59:59.000,9:59:59.000 to use the website and create their own[br]installation image with their language. 9:59:59.000,9:59:59.000 [Q] Someone on the stream said that[br]the fai.me web site is not yours 9:59:59.000,9:59:59.000 and it's a hack thing, it's a scam, you go[br]there and get hacked. 9:59:59.000,9:59:59.000 Do you have any plans to try to buy[br]the domain because it's pretty confusing. 9:59:59.000,9:59:59.000 The first thing I would have done[br]by seeing that talk would have been 9:59:59.000,9:59:59.000 to go to fai.me. 9:59:59.000,9:59:59.000 [A] Yeah, I was thinking about which name[br]I should choose. 9:59:59.000,9:59:59.000 I didn't check which web domains are free[br]and in then end I thought 9:59:59.000,9:59:59.000 "Do I like to have a different domain name[br]for the service?" 9:59:59.000,9:59:59.000 But since it's only a part of the FAI[br]project, I thought, 9:59:59.000,9:59:59.000 and technically it was also easier just[br]to host it under a subdirectory 9:59:59.000,9:59:59.000 and yeah, if people now grab the fai.me[br]domain and do other things with it, yeah. 9:59:59.000,9:59:59.000 I think one question would be to use[br]a debian.net or maybe debian.org domain 9:59:59.000,9:59:59.000 because people trust much more. 9:59:59.000,9:59:59.000 I get a lot of comments "Oh this would[br]be very nice if this would be hosted 9:59:59.000,9:59:59.000 on a Debian machine" 9:59:59.000,9:59:59.000 but this would be much more complicated[br]because the DSA team 9:59:59.000,9:59:59.000 has much more restriction what to execute[br]on their machines. 9:59:59.000,9:59:59.000 currently, we need root access because[br]we mount some things 9:59:59.000,9:59:59.000 and DSA would not give root access[br]on any Debian machine. 9:59:59.000,9:59:59.000 That's the same problem we have[br]in the Debian cloud team 9:59:59.000,9:59:59.000 where we want to create the official[br]images for the cloud providers 9:59:59.000,9:59:59.000 where the Debian cloud team will also[br]not have root access 9:59:59.000,9:59:59.000 and so there's much more work to get[br]empty virtual machines 9:59:59.000,9:59:59.000 starting up, putting data into it,[br]creating the images, 9:59:59.000,9:59:59.000 receiving them from inside the image. 9:59:59.000,9:59:59.000 And since on those machines I have[br]root access, that's much easier for me. 9:59:59.000,9:59:59.000 We are out of time, so thank you Thomas. 9:59:59.000,9:59:59.000 [Applause]