Ok, welcome back to the second session of the day. It's going to be Alexander Wirt talking about salsa.debian.org. [Applause] Thank you, good morning. I usually don't give talks in english, so please be nice to me. However, I'm here. I want to talk today about our journey for Alioth which is still running, but not for long anymore, to our new service, salsa. I want to get a little bit into the history of old things and what we have already achieved, what we still need to achieve and what are our plans for the future. Let's start with the basic things, who am I. I am the guy who rejects the mails on lists.debian.org, I am a listmaster. I am the guy that rejects your backports. I am the backports ftp master. And I am the guy that will destroy alioth.debian.org. For the last ten years [Applause] I was an admin by accident of alioth.debian.org. This is another story I will tell you in a few minutes. Beside from that, I work as an OpenSource consultant at credativ, which is a small company in Germany which is specialized in OpenSource, we only do OpenSource consulting in Germany. We do what today is called DevOps, we do every kind of consulting. If you do something with OpenSource, we are probably the ones you can talk with. I am a father of two wonderful girls, they're not here unfortunately, but otherwise I wouldn't be able to work. And in my little bit spare time, I do role playing games and Tabletop games. In theory there should be a picture now. There's a picture missing, I don't know why, which should tell "We need you". A little bit of advertisement, if you want to do OpenSource work in Germany, paid, and you need a job, please talk to me. We are always looking for good people, especially in C development, kernel development, but also of course consulting. So please talk to me. Some steps in history. Some years ago, ??? 2008, 2009, I told the alioth channel "Hey, if you need help, I can help with system administration, not the GForge stuff which is running above, but if you need help, tell me." [Audience] Big mistake Yeah. One or two years went by, and step by step all alioth admins left. We were alone in the channel. And around that time, I detected "Hey, I have sudo permissions and I'm admin" Somebody made me an admin. So, I had to decide that I will be the person that is the future alioth admin and I stepped in. So it was the beginning of our alioth journey. Then, in DebConf15, we had a long 'Birds of a Feather' where we talked about several security problems in collab-maint, some of you are maybe not aware of it, but since we use git at filesystem level on alioth, we are introducing a number of interesting security problems like if someone writes a hook, that hook gets executed every time someone pushes. So you have basically shell access. And of course you execute it as your own uid. So, if some DM (Debian Maintainer) or even not DM, nearly the whole world has write access to collab-maint, drops some hooks in, it can make you execute code on Alioth at your uid, which is a problem. We did some things to solve that problem, but the main problem remained. So, along that time, we decided that we would need a successor for git.debian.org. At that point, we are talking about gitolite which we evaluated at that time. However, as ??? Two years went into the land and nothing real happened, we just played with it. Then, May 2017, a thread comes up, "Moving away from fusionforge". What nobody was really aware of, is that alioth is on a Wheezy machine and Wheezy is ??? out of security support end of the month. So time was running up. The thread was long as usual on debian-devel and we decided to do a few steps, like evaluating things and in June 2017, I did a survey about our new alioth services. It was clear at that point that I wouldn't be able to maintain all the things alioth had in the future so we decided to just bring over the important things. What is important? For everyone, everything else is important so I decided to do a survey which was pretty successful with a few hundreds submissions. Then, in… Then we evaluated… "we" as probably "me", evaluated a few solutions, named pagure, which is the git solution Fedora is using, which is a Python thing based on gitolite, gitlab, which is the biggest Github competitor gogs/gitea, which is some golang-based small git service. pagure turned out to be not stable enough for our needs and we would have to do to much coding inside pagure to use it in our infrastructure because pagure is very strongly ??? with the Fedora infrastructure, specially its user authentication and user management stuff. Gitlab had an other problem called "opencore" and "contributor license agreement" which means I and others were not very happy with contributing code to Gitlab which is something that will always happen if you maintain such a service. And gogs and gitea is nice but it's small It will not be able to manage 10,000s of repositories. Next step happened in August 2017 when we had a sprint here in Hamburg at the hackerlab CCC on the other side of the building, where we talked about it. After long discussions, we decided to go with Gitlab because Gitlab, at that point, was the best solution that was already ready. We didn't have to adapt too much, we don't need to patch it which turned out it isn't true, but it's an other problem It had features like continuous integration ready, it had features like code review ready, wiki pretty good working and ??? very scalable in all directions Every component is scalable which is good for us. This is a TODO point, I wanted to add an image about the restaurant where we decided on the name "salsa". Somebody of you may ask yourself where the name is coming from. There's a small mexican restaurant a few hundred meters from here where you can get great burritos and they have a painting at the back with the term "salsa" written and we were deciding on a name which just not describes the type of service on it so we wanted… Yes, it's also a sauce. So salsa had sauce. I wanted to call it Klaus, but we decided against it so somebody came up in the restaurant with the name "salsa" and so it's called salsa. In the meanwhile, we talked a lot with the Gitlab people which were very kind and helped us with our problems. We also talked with them about the CLA problem and after some discussions, the lawyer of SPI was also involved, we made them to remove the CLA and replace it with something better. Contributing patches to Gitlab is now much easier and better which is something we are very proud of [Applause] And between November and the 25th of December, we implemented salsa two times First time on ???.debian.net where we had root but after more discussions we decided having this maintained at a (debian).org box would be better, which made us ??? ansible stuff and develop a ??? to be able to install gitlab as a non-privileged user but we did that. In Christmas, he was able to release salsa into public beta. Things went well, which allowed, at the end of January, salsa to leave the beta Since then it's official, our official git successor. What will happen in the future? Oh no, this is already past. On May, we disable user and project creation on alioth. Still in May, we disabled the not so much used version control systems, bazaar, mercurial and darcs On Thursday (May 17th 2018), I disabled projects web sites. And this is future, at the end the month, all other remaining version control systems on alioth will get disabled. So if you have anything running on alioth, still running on alioth, cron jobs are also disabled so you don't have cron jobs enabled anymore Be it whatever you think of, remove it. 1st of June, alioth will be off, you won't be able to get any data anymore from alioth. You can get the ??? via DSA to get subsequent backups, that's up to you but I don't recommend it and they won't like it. Yeah In June, alioth will come to an end. It served us well for 10, 15 years, but its time is over. Some numbers. Where are we now? Yesterday (May 18th 2018), we had 23,700 repositories on gitlab, 3200 users, 400 groups, which sums up around 90GB on disk, which is nice. For a service running for more or less 6 months, it's a pretty nice number. What are our future plans. ??? Docker registry, by now you can use external registries which is working You can the gitlab registry for Docker images but it will be nicer to have our own registry. That is pretty high on my todo list, after alioth is gone. We want more runners, so you are able to sponsor runners, if you have machines or some money you want to spend on runners, please tell us. What are runners? Runners are the things that are used by Gitlab CI to build code or test code, or do things. You can use it to build your packages, you can use it to autopkgtest you packages you can use it to build websites or whatever you like. It's pretty useful and I think using CI more will be a big step forward for Debian. We should really get more into it. There are already some projects like the reproducible builds, the debci guys that are working on such stuff and now we have the infrastructure that every DD, every developer or package maintainer can use it. There's also an other feature called "devops" which is based on kubernetes which allows you to even deploy and test things properly. So if you have package which implements a web service, you can even run ??? kubernetes part which runs a web server, you can test it, you can even record it, do QA test and so on all based on this devops feature which would also be a nice thing. By now, we don't have a kubernetes instance we can use for it, so if you have a spare kubernetes instance you want to offer Debian, please talk to us. And integration with sso.debian.org, which is another side project of mine and some of ??? students, sitting there. We want to build a successor for the command sso.debian.org which has a problem that it doesn't have a user backend, the user backend is alioth, you see the problem But it just the case for our guest users. The official Debian Developers come from the ldap which will still work, but we have a problem with guest users, so we currently don't have a way to source for managing those guest users, especially give additional groups like "Hey, the user's a DM" I would love to give all DMs access to the Debian group, write access, but I can't currently because I'm not able to ??? which is something we want to solve with the new sso.debian.org feature. sso.debian.org should also develop a new authentication protocol like OAuth2, which we will use for salsa but new services can also rely on, ??? a way from this certificate stuff which is somewhat nice but it's not that good integrated in most browsers anymore and it doesn't work that well. We hope to have, we already have a prototype, and we hope to have it live until the end of the summer. What we left behind. We don't have shells anymore. So you won't be able to run any cron jobs or other stuff on salsa and please don't ask, we won't give anyone a shell on salsa.debian.org or godard which is the host hosting it. We hape APIs, several of them, I will show them Please use them, we won't run any cron jobs or custom stuff on gitlab, it was a nightmare on alioth to maintain and to administrate and I will never, never want to get into this again. What we also don't have are custom domains which is a feature gitlab has, but DSA decided against it, so you will have to live with projectname.pages.debian.net until someone decides for that feature. We also left behind old version… not so much anymore version control systems like darcs, bazaar, subversion which isn't a problem, but we also don't have cvs anymore, which may be a surprise for someone but Debian is still a heavy user of cvs, especially for our web site and translations. But maybe they will now migrate faster away from cvs. They are working on it, I know, they're working on it for 10 years but things are getting faster and they're making progress in migrating away from cvs. Yeah, ???, that's right, we also left mercurial, or whatever people have in their home directory. Yeah we also had rcs on alioth, there were rcs repos, yes. What we got instead. We got a bunch of new features we didn't have before. So, this is such… maybe a start of new ways of working in Debian, we got a bunch of collaboration features. In the past, collaboration often meant finding the right mailing list, sending a patch and hoping. Now we can use merge requests, which allow people to easily fork and modify packages or repositories, and after they are done, they can just hit a button or whatever and create a nice merge request which is already heavily used by some projects like apt or dak or my own redirector. That allows ???, the admins of those repositories/projects to review code easily, they can add comments, they can discuss with ??? people out of the mailing list. If people update a bunch and they commited, those merge requests get updated which is a workflow we are also using very heavily in our company which is pretty nice in my eyes. This also allows contribution to packages from outside people It lowers the barrier for people to collaborate with Debian, which is in my eyes a good feature, something I always liked on Github and I'm happy we are having it too now. Gitlab has a nice feature of good, well designed web frontend, some things could be better, but it's always the case, but in most cases Gitlab is still blazingly fast except if you've hit some of the bugs in the API but that's an other problem. And you can work with it. If you don't like the web frontend, use the API, nearly everything the web frontend supports is exposed via the API And there are also a bunch of command line clients which can integrate into git to allow things like merge requests, allow you to process merge requests from the command line if you don't like web frontends. You can also open merge requests by e-mail if you still like it you can just hit the right buttons, you'll get a mail address that you can use. And if you send a patch to that mail address you will create a merge request, some of the not so known research. Issues. You can track todo items or bugs. Please, this is not intended for Debian packages, so please don't replace the BTS but using it as an issue tracker or todo lists is great. We are using it all the time. We're also having some upstream projects on salsa, like sane or ??? which is ??? So, they're using issues, that's fine too. Issues are disabled by default for a project, but every project has ??? to just enable it and to use it. You have boards where you can organize your work, you can add sprints, you can add milestones and other things, all the basic stuff you need to have an issue tracker is included. And we also enabled reply by mail so you don't have to use the web frontend, you can just use your mail client to reply ??? into gitlab. You can also close issues by merge requests. So, similar to our BTS, Gitlab has this "closes" feature. It's all the same. So "Close", "Closes"… and so on, it's all the same and we close here your issues. You can even close issues in other projects, so if you have projects related together and you fix something in another project you can even close it with that syntax. You can also create issues by mail, which is basically the same as for merge requests, you have that "email new issue" button where you get a custom mail address you can use and then you can use that mail address for the future to submit bugs if you don't want to use the issue tracker. What we also got are webhooks. Custom hooks are not anymore possible because you don't have access to the repositories directly but what you can use are webhooks. Webhooks are common standard in the web world, you can use them to react to events in your repository, events may be things like someone created an issue, someone created a pull request, someone pushed something, someone took something, things like that. And you can use those events to create IRC notifications, we have two IRC bots available for you to use, which is KGB and my own irker instance. You can automatically close or tag bugs If you look into our documentation, wiki.debian.org, you find a small paragraph about it where you can just, as we did before, if you close a bug and you enable the tag pending, tag pending webhook, your bug will be tagged automatically as pending like before if you used the ??? hooks on alioth. And you can also trigger external CI QA systems, like Jenkins or SonarQube or whatever you like to test you code. In the future, we will also use it for collab, for the collaboration stuff from tincho, where we will just forward every push happened on the whole salsa system so you don't have to configure that manually, it will happen automatically So if you contribute something to Debian, it will come up on collab.debian.net If you want to provide webhooks but you don't want to run your own web server, you can come to us, which means you have to code Ruby. We have our own webhook server implementation for salsa.debian.org, which is currently also running on salsa, but that must be the case in the future. So, if you want to run a webhook, provide us a patch for our webhook implementation which is pluggable, so write a plugin which listens to your webhooks, provide a patch, a merge request and we'll happily add it to our webhook implementation so it can be used for everybody. Documentation is in the wiki. Currently provided hooks are, as already mentioned, tagpending which allows you to tag bug as pending if you mention them in you changelog and some project directly working with commits are using the close webhook which allows you to directly close a bug with a commit which is used by some web servers and other stuff directly used in Debian. One of the most powerful features we got is Gitlab CI. Gitlab CI is a system that allows a continuous integration, continuous development on salsa and that allows you to build, test and eventually deploy software and packages from within Gitlab. You can nearly do whatever you want in this CI stuff, you can compile ???, run linter, run autopkgtest, whatever you can imagine you can do. We have two runners provided. One of it is running as an ??? on Google cloud, the other one is hardware sponsored by a sponsor and for every CI run, we launch a docker container in it, You can even provide an image you want to use as this one and then you can do whatever you want with it. But please don't do bitmining or something like that, be kind to them, we all have to use them and we have only two of them, so please, if you want to do something bigger, talk to us like the KDE people already did. How to use it? Using Gitlab CI is surprisingly easy. There is this gitlab-ci.yml file which is usually in the root of your repository but you can add configuration to your repository, for example to add it to your /debian repository which works better for ??? packages or whatever you have, if you don't want to clutter the upstream directories of your gitlab-ci file. [Question about potential conflicts with gitlab-ci files from upstream] We already have a bug opened on the Gitlab issue tracker that allows us to change the default name of the gitlab-ci file because currently, if you import an external repository, which has a gitlab-ci file which can happen, if we ??? run on our infrastructure and for example, it's ansible or some other project ??? for every upstream commit, salsa will ??? run our runners and build the pipeline. After you edit your file, that's it. From then on, you can watch every commit happening on your pipeline. This is a simple gitlab-ci file, gitlab-ci files are yaml-based, documentation is in the Gitlab repository, the documentation repository and as you can see, it's pretty easy, you have a pre-step, which allows you to do things like installing dependencies which is what's happening here. Since Gitlab CI is running a detached head, if you want to ??? use git buildpackage, we will have to checkout master for it to properly work, then you can do a git pull, git buildpackage and, after that, you have build your package. That's basically all. You can also use artifacts. Artifacts allow you to… keep a build artifact for downloading, so if you want someone to use a package, you can just add an artifact stanza here and that allows you to later download your deb files. Now that doesn't allow you to create a repository ??? but it's an other problem. If it's too much, you can also use the ??? yesterday This is a prepared docker container which is prepared for git buildpackage and all you have to do is to execute this. After that, you have Gitlab CI. [Applause] I don't know who provided it, but it popped up in the wiki yesterday or something like that. We also have Gitlab pages. Gitlab pages are like Github pages and allow you to host web sites, static web sites from within Gitlab. Internally they also use Gitlab CI, so you provide a Gitlab CI job that just deploys your website, so it's nothing to do and here's our build artifact feature. All we do here is just add those public files in the public directory to our pages and we only do this on the master branch and basically that's it. The magic is happening here, it's the "pages" step and if you correctly configure pages in your repository configuration, you have a Gitlab page after that. You can also do more fancy things like a Hugo web site, just depend on a docker image which has hugo installed and then execute a script which builds Hugo, add some artifacts and after that you have a Hugo website. You can also use it for blogs, you can use it in personal repositories, then for example for your own web site or blogs. Of course, it's not intended to serve big web sites but providing blogs for planet.debian.org is perfectly fine, for example or web site of ??? project or whatever is Debian-related. This brings me to an other topic not mentioned in my slides some people asked us what is fine to host on salsa. As long as it's open source, as long as it's intended to be Debian-related or open source related or can be included in Debian, it's perfectly fine to host it on salsa. So we invite every upstream which is looking for a home, like the SANE guys, to host them on salsa. What we got with the latest major version is a web editor which is pretty new, probably buggy, but it works. So, if you don't want to clone a repository or you have just to have simple changes, you can add your file in the web editor, you get a web editor with syntax highlighting, you get even a markdown preview if you just do documentation, so that's great for every one just doing documentation that doesn't want to ??? with git or the code inside, you can even preview it, then you can write a commit message, and that's it. What we also have is two-factor authentication, which is a security feature that allows you to add a second factor to your Gitlab login. I can only recommend to use it, that's well integrated and adds a lot of security. It works with Yubikeys or any U2F-compatible key and also with software solutions that implement TOTP, time-based one time passwords. So every TOTP-compatible generator also works, for example the Google authenticator but there are also others which are open source, that all works. Adding it is easy. ??? It's easy. What you can't see now is me getting my smartphone out, scanning the bar code, generating a PIN code, and in 2 seconds I will enter the PIN code What you see here are recovery codes, I will mention them in a few minutes. You can use them to recover your account if you lost your one-time password generator So, if I log in now, I have to use my smartphone to generate an authentication code, add it, and now I'm in, that's it. So it's pretty easy. Some people say "Oh, what if I lost my token, that is such much work." No, it's easy. If you want to recover your token, I do that all the time, you can just use SSH and do "ssh git@salsa.debian.org" and the command "2fa_recovery_codes" which will generate you a number of new recovery codes, and you can use it to log in. So if I don't have my authentication token with, I just use this every time. It works. And SSH also has an other token. Huh? [Q] It's cheating. [A] No, it works, it's a second factor, so it's fine. So, the API. I have to get faster. Gitlab exposes a powerful JSON REST API, that allows you to query and to manipulate nearly all aspects of Gitlab. If I say "all", I really mean all, nearly everything you can do with the web frontend is covered by the API. The API has an extensive documentation in that link. Ensure that you use the CE, for "Community Edition", in the docs, otherwise you will see features that aren't available in our edition. You can use it for everything. Small hint, you often see the term "namespace ID". In nearly all cases, you can replace the namespace ID with the path of the project, of the thing, mostly projects, but if your projects includes slashes, which is nearly always the case, you have to replace a slash with "%2F" and you can use it as the namespace ID so you don't have to look up every time your IDs, you can just use names. Some examples. Get data about your user. If you want to know ??? in your user, you can just ask the API. For every use of things that need authentication, you need to generate a private token, which is easy, just go to your profile and there's a link to access tokens and generate one for your usecase. You can also add expiration dates. Even if I use curl here, I can only recommend to use a proper library. Don't use curl, please do me a favor and don't use curl. There's a bunch of ??? flying around ??? curl then they hit the first time pagination or something like that, and they start wondering why it doesn't work. There are Python libraries, Ruby libraries, Perl libraries for Gitlab, so we have everything in place that you need. What you can do to create a repo in your namespace. If you do it that way, the repo will be private. Of course, there are parameters to create a public repo, but that means you have to read the documentation. List open merge requests of a project, also easy, just go to Projects, which is a namespace for project-related stuff, add your project, in my case it's "AliothRewriter", it's a merge request, ??? specific state and you get a nice formatted JSON to get nice JSON you can format to list all your merge request. So, I have to get to an end. How to get support. If you have any problems with alioth, you can of course talk to me or to ganneff The channel is still #alioth, and will probably stay for some time. You find us on IRC, you can ??? e-mail, salsa-admin@debian.org. We also have an issue tracker, so if you have a problem, open an issue but please don't open upstream gitlab stuff in our issue tracker. If you have a problem with salsa, fine, create a ticket. If you have some problem with how Gitlab is doing things, or have found a bug in Gitlab, please do us a favor and open those bugs in the Gitlab issue tracker because it's the same thing we would have to do, and we won't ??? your bugs. How we do it. If you're interested in how we did those stuff, we are all doing it unprivileged in git on godard.debian.org, and everything we do is automated via ansible and you find it in our ansible repository in the salsa group on salsa.debian.org. Who are we? Salsa currently has 3 administrators: Bastian Blank (waldi), Jörg Jaspert (ganneff) here, and me. My picture is gone. What you would see here is a "thanks" picture. Thank you for your attention, and we have maybe 1 or 2 minutes left for questions. [Applause] [Q] I have a quick question about SSO going down. Is SSO going down with alioth? On the first of June? [A] No, but… As mentioned in the announcement, I usually do wide annoucements and I hope people read it. I will back up authentication database of alioth to ???.debian.org, which is the host running SSO and I will maintain those users by hand until we have the new SSO backend ready. [Q] So, if there's people going to the registers of debconf, which are not Debian Developpers, in Taiwan, can they still register after the first of June, or not? [A] They can, they can't SSO anymore for registration, but they did implement a second authentication, but… It's been 3 years since debconf doesn't need SSO. Any further questions on salsa? [Q] I have a little question about salsa at all. There is some howtos, how to create an account on sals, how to manipulate that account, create repositories for example. I mean, just for people who never came to it, just use it. ??? documentation. [A] Except for user registration, there's nothing specific on salsa.debian.org, so you can just use the gitlab documentation which has videos, webcasts, tutorials and so on. Just use it, it's good documentation, nearly everything is covered by it and if you still have questions after that, ask us. If you have Debian-specific stuff, look at the wiki. wiki.debian.org/salsa/doc has nearly everything, I hope, most things covered. If not, tell us, and in the end it's a wiki so if you find something or think it's interesting for other people, please, please add it to the wiki. Any other questions? Ok, everything's answered. Thanks a lot again. [Applause]