wikipaka preroll music Hello and welcome to my talk on Remixing Linux. My name is The one with the braid, and today I'm going to introduce your own creation of a Linux distribution. We will talk on remixing Linux, on the architecture you require to run an own Linux distribution, and some of the usual use cases. OK, let's start. Well, what are use cases, what are purposes of your own Linux distribution? Of course, you could customize Linux distribution for your enterprise environment. For example, if you require special network settings, your own services to be installed, it's sometimes easier to create an own distribution than using a ready to use distribution and customizing it. Another option is that you discovered something fully new and decide to create your own Linux distribution. That's what we call yet another Linux distribution. Please, please do not do this. We already have enough different flavors of Linux. OK, another valid reason is missing software. If you require a special software for use case and you want to bundle it into your operating system. That's a usual use case. Moreover, embedding for microcontrollers is a common use case for Linux remixing, as well as OEM hardware support for computer vendors to bundle Linux. Some known vendors do this. Yeah, and of course, another desktop or in general different preinstalled applications are a good reason to create your own distribution. OK, what are we going to have a look on today? We will look on live systems, because that's the most common use case. Afterwards we will go through step by step instructions for and next, we take an example of Ubuntu. We talk on making your system installable and we will talk about a simple tool simplifying some of these steps. We will have a look on Linux from scratch, meaning compiling everything yourself without using any templates or anything. And we'll look at the architecture you require to run your own Linux distribution, a.k.a. package mirrors. OK, let's start with a live system. What are properties of a live system? Of course it needs to fulfill some requirements. If you use a tiny pen drive, I don't know, eight gigabyte or 16 gigabyte, it's obvious you need to compress your whole system. Moreover, it must be writable at runtime. Just try to mount your root filesystem in Read-Only and run a web browser. You run into big trouble. Same for network connections. If you connect to wifi without writable root system, you will run into big trouble. And another requirement, it's not really a requirement, but it's recommended is an accessible live media. So if we have our pen drive containing our live Linux distribution, we sometimes need to access the pen drive itself in our live environment. For example, if we'll have packages of our distribution inside or on our pen drive, we may want to install them later. Then it's quite good to have your live media accessible. Well, of course, we could script everything on our own, but someone already did all these steps and has put them into a tiny software called Casper. Casper does exactly what we want, creates a union file system, meaning an overlay on the existing file system and creates a kind of fake writable system in memory, file system in memory. So it allows modification of files, creation of new files while they are actually being stored in the memory instead of on our physical disk. It mounts the live media and it shows up a pretty display manager you can see over here. That's the Ubuntu mate, it asks whether you want to try our system or whether we want to install our system. OK, let's talk about the compression, because the compression we require is tremendous. If we have a normal desktop distribution. Believe me, it consists of about 30 gigabyte of data. Our applications, as soon as you install some more applications like LibreOffice or a Web browser and some tools, you will reach the 30 gigabyte or 20 gigabytes, but if you ever downloaded a live distribution, you will realize it's not 30 gigabyte. It's about two gigabyte or one point five. I noted in my slides. Yeah, it means we need to compress our file system on about 5 or 10 percent of the actual size. That's not difficult. There are many compression algorithms we may use, but we have another requirement. We need performant readability. As soon as we boot a system, our kernel won't stop accessing any files, reading files, writing files. So we need a performant readability because no one wants to wait five minutes to see the Web browser. And that's what the squash file system is made for. The squash file system is a compressed file system. It's kind of a disk image, but more advanced. It's used in diverse Linuxes. It's used in the operating system of Chromecast and in the Linux userland it's used by app images. These are these click- to-run Linux binaries you can download, and it's used by snap packages from canonical's proprietary snap package manager, which is unfortunately very widely spread. Yeah. Now we know something about compression. Let's talk about how the customization works. Well, first of all, we download a template. We create a base system from which we start our customization process. Well, we start the customization itself. We prepare a CD environment. So when you put a bootloader on a CD, we pack our root file system we just created, put it on the CD, put a kernel there, make it, harden it a bit, and at the end we will pack it. OK, let's go through it, step by step. In this case, we will have a look at Ubuntu, we will go step by step through the customization process of Ubuntu. How do you start the customization process? Well, you need an existing ISO file from a live environment of Ubuntu, because it contains the bootloader, the directory structure and everything, all the assets. Of course, you could create them on your own, but, well, it's more work than you have to do. You can easily use the existing bootloader and the existing files and directory structure from an existing ISO file. Well, let's extract it and we're fine. We're done with it. Afterwards, we need to create a base system. A base system is, well, the base of Linux. We have the base utils, the base files, the directory structure, the kernel, a shell. The package manager of our operating system we have. Without any unnecessary tools, without, just the base of everything. You can either download the prepared base system of the distribution for download, but you could create one on your own using tools like the other Debian tool to create base system or the corresponding tools from other distributions. Yeah, as soon as you have your base system, you need to change your root inside this base system. And then you start the customization process by installing the necessary drivers. We already talked about Casper. Casper is this tool providing the live software and the live scripts, the overlay file system in Ubuntu based systems. And it creates an initial RAM- file-system and installes the required kernel modules to load and to boot a squashed file system. So let's install these three packages, Casper, Ubiquity- Casper and Lupin-Casper. And afterwards we can install whatever we want. We can install our loved vim, we can install our loved Alpaka software, and we can install an alternative shell, the packages we want. In the example code I provided here we look on changing the plymouth screen. That's the startup screen which shows up when you boot the distribution. I don't know, you could put an Alpaka in there or the logo of rC3. Yeah, that's what we do in customization or modification of the system. Afterwards, we continue packing everything we created. First of all, we need a Kernel to boot , because well before our base system can be extracted, we need to load a kernel by the bootloader and the kernel must contain the corresponding kernel modules to load, to boot a squash file system. That's what Caspar already prepared for us, we only need to create the initial run file system and if you read the code I provided, you will see I was missing an minus between update and init RAM file system. Let's create our initial RAM file system and copy them or copy it together with our kernel to our extracted live environment. And can delete the prior kernels and so on. We don't need them. We have our own kernels now. Afterwards we pack our base system. Of course we need to unmount any mount points in the base system we created and create a squash file system from it. And these three files, the VM Linux, the initial Ramdisk and the squashfs file system, they are the major pillars of our live environment. If we have those three, we are actually ready to go now. We can either use our existing bootloader from our extracted ISO file or we, of course, you can install your own bootloader in this directory. But well, it's easier to use the existing. For security purposes and for hardening your live environment I would recommend to put the MD5 sums of all files in your ISO file in this environment, because as soon as you, for example, copy your distribution, for example, on a pen drive on which is fat or ext4 formatted, it could be modified by anyone. And that's why it's much better to verify the MD5 sums at the boot process. Casper will automatically detect whether the MD5 sum, the txt file exists and if it exists, it will automatically verify these MD5 sums. OK, now we are actually ready to go, ready to pack up our ISO file. There's a famous tool called genisoimage. So generate an ISO image, that's a command you could use. The command takes care of legacy booting except of output file, the disk and the disk image and the -r parameter, you could leave out everything if you do not require legacy bootloaders. But if you want to obey legacy bootloaders, you should also hybritize your ISO file to be able to dd the ISO file to a pen drive. Otherwise, without the ISO hybrid command, it won't boot from, it would boot from a CD, but not from a pen drive. Yeah. OK, now we created our own Linux distribution. Well not that much. It's actually just a repacked Linux distribution. Now we talk about making it a real distribution. That's why we talk about making it installable. Why should we make it installable? Of course we could manually install our system. We could manually format our hard drive, extract our file system and manually create users and so on, et cetera, et cetera. But most of users prefer graphical way except of these geeks taking part at the rC3. That's why an installer may be useful. An installer does exactly what we need. It creates partitions, creates users, installs language-packs, offers installation of extra software and does everything the user might want. Creates users, connects to wifi, whatever. And there are three very common installer frameworks: Ubiquity, Calamaris and Anaconda. Now we'll have a look at these frameworks. Ubiquity is known from Ubuntu. It's ready to use if installed, but it's actually not at all customizable. Anyway, some hackers at GitHub found some ways to customize it and provided some good information, some helpful information in their repositories. The next one is Calamaris. Calamiris is highly customizable, is used by many distributions, including SUSE Linux, I think Xubuntu, so the XFCE Ubuntu flavor, DeepinOS, I think I am unsure. I think elementary as well. But if you only install it, it won't work. You need to customize it. The third one is known from the Fedora project. It's called Anaconda. It's a very advanced tool. It's able to update existing systems. If we have an old Fedora installation, we can boot our live CD and update the existing installation and moreover, very useful for admins is the remote functionality. It can be started via SSH and automatically update everything and automatically install and deploy the software we need. Um, yeah. Well, I already mentioned, for everything you can manually do and customize, there is a graphical tool. If you prefer the graphical way you could use the tool remastersys, aka LinuxRespin. That's the the older name. It offers a simple graphical user interface for packing your currently installed system into a live distribution, allowing to install it. It does not allow a high amount of customization in the graphical way, of course, you can manually perform it on the command prompt. But it's an easy way to, for example, backup your just installed system or you readily configured system to a pen drive to make it installable later. And now, Linux from scratch. I guess most of you already heard of Linux from scratch from scratch means, yeah, pretty much Linux from scratch. You start without anything. You start on your host system and create an own system from scratch. What does that mean? It means you should have lots of time. Lots of RAM, and many processor cores. Because you need to compile everything yourself. Beside the kernel, you need to compile the whole userland. If you want, you can include a package manager or whatever. But if you make real Linux from scratch, you do not take any binaries but compile everything yourself, everything on your own. Currently, minimally, you need to compile 226 dependencies. But in some years it will be some more dependencies and the amount of dependencies required increases. Yeah, after you compile everything, of course you need to configure everything, need to provide a bootloader. But as soon as this is done, as soon as you created your own base system, your own system from scratch inside a fake root, it's the same as we already talked on during live system customization. As soon as you have this fake root, you simply pack it to an image, so a squash file system, write it on a disk, provide a bootloader, and that's it. OK, one more word. I will talk about maintaining architecture for distributions. I will talk about package servers. If you have a distribution, you usually provide updates, security fixes and whatsoever. Maybe software the users may install later. Yeah, that's a package mirror, a packet server. From Debian, you can simply mirror existing servers, for example, the Debian or the Ubuntu or the Linux Mint servers by using the tool apt- mirror. It simply mirrors, means it does not allow any customization. For ArchLinux it's easy as well. You can simply use vftp and crontab to download the packages and provide them on your host system, on your server and synchronize from them. That's much easier than in the Debian environment. But usually if you have a distribution, you want to customize it. At Debian, the tool aptly is very widely spread because it allows very advanced management of different snapshots from different repositories, mirrors, local repositories, single packages, allows diffs and so on. But it does not allow any optimization. So good luck at scripting on your own. If you look at ArchLinux, for example, you would usually write your own package build files as you know them from the Arch user repository and include the readily build packages into your directory from which you serve your updates. That's much easier than in the Debian environment. I still found one more graphic explaining aptly. That's what aptly can do. It can do everything. Well, you have mirrors, you have local repositories. You can take snapshots, take differences from snapshots, publish these differences, publish merged snapshots of them, publish single snapshots, diff everything and do whatever you want. It's very performant, the tool and yeah, but it's hard to use. It requires some time to be learned. Yeah, that's it already on remixing Linux. Now there should be an Q&A. Thank you very much for your attention of this talk. I hope I could give you a good introduction into remixing Linux and good luck at coding and enjoy your Remote Chaos Experience. Music Subtitles created by c3subtitles.de in the year 2021. Join, and help us!