981_Making_your_package_work_with_systemd.ogv

[stapelberg]: Hi! So first of all, the type of this lecture

is that it's going to be a workshop, just to make the expectations

clear, I have a few slides prepared that

should give you a general introduction

and they will take

probably about half of the time we have allocated

for this slot, and afterwards

we can answer your questions

or have some hands-down hacking

on actual packages.

My name is Michael Stapelberg

Just a quick show of hands so that I know my audience --

who of you maintains a Debian package

in Debian or derivatives?

OK, that's almost everybody

Perfect.

How many of you actually added

systemd support to their packages already?

OK, a couple.

Who of you are looking into doing that?

OK, perfect! You are exactly the right audience.

Great. So the topics I want to cover with my slides are:

First of all, how systemd works for package maintainers
Great. So the topics I want to cover with my slides are:

First of all, how systemd works for package maintainers

That means I'm not going to cover all of it
First of all, how systemd works for package maintainers

That means I'm not going to cover all of it

And also it's not so much of a user standpoint

but just what you need to know to get you started on testing your stuff in systemd.

We will then have a look at an example service file

where we'll make clear what all the different directives are supposed to mean

We will look at the tmpfiles mechanism

and at dh-systemd, a debhelper plugin

We will consider a few more advanced examples, and then finally we will

answer your questions, hopefully.

So, non-topics for this presentation are:

systemd sucks, let's just use something else instead

When will debian ever finally switch the default, and essentially anything that ever came up on debian-devel. I'm not going to talk about that.

This is just about how to make it work if you have already accepted that this is something that you want to do.

So, how does systemd work?

Essentially, where we previously had initscripts, we now have service files.

Service files are just a special kind of unit files

so the more generic term is unit files and that which corresponds to an initscript is a service file.

So previously, you would have /etc/init.d/apache2 as an initscript

and the corresponding file would be /lib/systemd/system/apache2.service

Now what's important here to notice

Is that the base name (apart from the .service suffix)

needs to be the same, because what will happen is that

systemd, when booting on a machine

will look at all the services in /etc/init.d

and use them if they are available

but if there is also a service file, that will take precedence.

So in order to make sure that systemd uses your service file and not

your service file plus the old initscript, make sure that the name is correct.

Before you ask, yes there are also mechanisms to make sure that

when you have a name transition, because you've adopted the upstream name for a service file

or something like that, then you can have a compatibility symlink.

We can cover that later.

And essentially, systemctl does what service did.

So if you had service apache2 start

Then you would use systemctl start apache2.service

In recent versions, you can even skip the .service suffix so it would just be systemctl start apache2

The systemctl tool is mostly for the actual users, or for you when testing.

and just like you would usually use /etc/init.d/apache2 start as an actual user

but "service" inside your maintenance script, we have something for that.

Now, what we had in SystemV init were runlevels

and they were poorly defined and different from distribution to distribution

And what systemd does to replace them is that it has something called a target.

A target is precisely the same as a runlevel

it just has a much nicer name

and the names are standardized between all the different distributions.

The most important targets for you are basic.target, multi-user.target and graphical.target

Each of them is more specific than the other

in that it starts more services for a particular use case

So basic.target is what everybody of you will run

and then multi-user.target extends basic.target with more stuff

and graphical.target brings in all the graphical components like a display manager

and stuff that you would normally not use on servers where multi-user.target is what you would want to use.

You can look that up if you care later on in the manpage systemd.special.

Just as previously, with SystemV init

we had symlinks to enable a service

and these symlinks were created by update-rc.d

we still have symlinks to enable a service

now they're just created by systemctl enable

Again, systemctl is what you would use as a user, or while testing, etc.

And we have a maintainer script equivalent of that.

Are there any questions so far? This is really the basic stuff.

[question]: What is the reason there are different commands for the user and the maintainer scripts?

[stapelberg]: I think will go into this later, but I can just briefly explain it

The maintainer scripts also need to work on machines where systemd is not actually installed.
[stapelberg]: I think will go into this later, but I can just briefly explain it

The maintainer scripts also need to work on machines where systemd is not actually installed.

So that's why we can't use systemctl.

Alright, let's look at an example service file.

I just picked vnstat.service

which is a tiny daemon that will just store (and plot later on) your network traffic.

What you can see here is a very clear, human-readable configuration file

And this is the service file. it's called vnstat.service.

It starts with a [Unit] section, and you can see

just like inifiles, which probably everybody of you is comfortable with

or .desktop files, which have the same format, so this is actually a .desktop file.

There are sections which are enclosed in square brackets, and then there's key/value pairs

separated by an equals sign, so that's really simple.

The first section is entirely just for humans.

So the description will tell me what kind of service this is, if I'm looking at my system and wondering what is this that's starting up here.

The second section is the [Service] section, and this is actually where

all the relevant stuff is.

What we specify here in the second line is ExecStart, which is

the command that systemd will run when trying to start that service.

In this case, it's really simple, you just start the vnstatd binary.

What you need to notice here though is that systemd does not invoke a shell to start it up,

so you need to specify the full path, and you don't have your usual

shell stuff in your command line, so just keep that in mind.

We also have the ExecReload line, and there could also be an ExecStop line.

In the absence of an ExecStop line, systemd will just kill the service

And it will do what many many people implemented in initscripts by hand

which is to gently try to kill it, and if that doesn't work within a certain timespan

then it will try to kill it harder, until it finally succeeds.

The ExecReload line is pretty standard

the thing is that not all of the services provide a way of reloading in the first place

so we need to have such a line to tell systemd what to do on a reload command

Some other service (you could probably think of the bind nameserver)

have a different way of reloading (so you could call "rndc reload" in that case)

but this one is really simple, it just sends SIGHUP to the service, and then it will reload.

the $MAINPID is not actually a shell variable or anything

it's just a special thing that you can use in unit files and service files in particular.

And then we also have the User= line which will specify

under which user this program will be started. That's pretty straightforward.

The last section is also interesting

it's called the [Install] section, and it has a line called

WantedBy=multi-user.target

And that essentially says that when this service file

is enabled on the machine

it will be pulled in by multi-user.target, so this specifies

which runlevel it runs in, so to say.

Are there any questions so far?

yes -- microphone?

[question]: can there be more than one WantedBy= line?

[stapelberg]: Yes, there can be multiple WantedBy= lines.

More questions -- over there?

[question]: So just to make sure that I understand correctly

If systemd tries to restart or reload a service

it first executes this reload phrase, and then it execs

the start phrase again?

[stapelberg]: No, there's two different cases here. One is the reload case

and the other one is the restart case.

In the reload case, it just executes what's specified here

If the service file supports reloading; if it doesn't, then you can't reload it.

If you do a restart, it will

first execute stop if present, or kill it if not present, and then start it again.

[question]: Fine. OK, thank you.
first execute stop if present, or kill it if not present, and then start it again.

[stapelberg]: one more question here?
[question]: Fine. OK, thank you.

[stapelberg]: one more question here?

[question]: Is there an equivalent to debian's force-reload,

where you care so much about wanting it to reload that you're OK with it falling back to stop and start if necessary?

which debian's sysvinit system does support in the usual skeleton file.

[stapelberg]: I think there is a try-reload action, but I would have to confirm that. We can do that later.

[question]: Is there also something to stop a service?

[stapelberg]: Yes, I already mentioned that you can specify ExecStop=, and if you don't, it will just kill it.

[question]: Sorry, I didn't hear that.

[stapelberg]: one more question here?

[question, quoting from a man page]: "tries to reload if possible and falls back to restart"

[stapelberg]: He just said there's a "reload-or-restart" command in systemctl that does precisely what you asked for.

[question]: Are there other things than start, reload, stop, and ... can you define other things?

[stapelberg]: Can you clarify what you mean by that?

[question]: For example, status, or I was thinking more about something specific to a certain daemon?

For instance, fetchmail can have an "awaken" option

it does a sleep of 5 minutes or something, and you can say "check right now"

[stapelberg]: What exactly is the question? I still don't get it.

[question]: Can you have some exec line in there --

[stapelberg]: -- Oh, can we have a custom action, like in initscripts, is that what you mean? No, we can't.

There are different ways of coping with that:

Most often there's an alternative to what you want to do

But I think it's really clean and nice that they standardized on a few verbs, and all of these work with all of the services

(except for the reload one, obviously)

but you can't have really custom weird stuff.

You could ship that in an additional shell script. That's what I would usually use as a first suggestion

if there's no specific, no better way of solving the problem. But we can discuss that later on.

for specific services that you might have in mind.

Any more general questions up until here?

[question]: Maybe you answered and I missed it, but about the possibility of querying the status of the service?

[stapelberg]: Yes, obviously. As I said, I will show that in the second part.

Now, tmpfiles:

The tmpfiles mechanism is really useful not only within systemd

but also oftentimes when people read about it they think "hey, this could be useful standalone."

What it does, is it provides a mechanism to configure creating temporary directories, like runtime directories.

As a really simple example, I have here the lighttpd configuration file

where we will essentially say "create a directory called /run/lighttpd"

with this mode, this user, this group, and no more arguments.

There are some arguments you could specify there, for cleaning it up every ten days or some delay like that

that's all supported. This mechanism is much more powerful than this very simple example.

it can also support not only directories, but also files, pipes, symlinks, etc.

I don't want to go into all of the details, but this is the preferred mechanism to create a /run/something

and it's much much cleaner than having all these varying mkdir commands

that sometimes specify a user and a group and sometimes they don't and then they don't have a mode, etc. etc.

So this is what we want to use. Question?

[question]: There's no implicit conflict with other things, so I have to take care to do a proper namespace organization?

and, for example, call my temporary directory like my service and not

choose an arbitrary name which would perhaps clash with another service?

[stapelberg]: Sure. I mean obviously you need to watch out for filesystem clashes in debian packages.

That's just as it always has been.

[question]: So it's the mechanism, you have to provide the policy.

[stapelberg]: Yes, obviously.

Alright -- service file location:

oftentimes, the upstream provider ships a service file, and that's the way it's meant to be.

because the upstream should know best how to install the service on a particular machine

Now, that doesn't always happen

but if it happens please use the upstream service file

and if the upstream file is really broken, please work with upstream to fix it

If it's broken in some minor detail, you might ask upstream

if it's acceptable to change it in the way that it would be better

I don't have any specific details; one example that would come to mind is that some

upstream service files are actually pretty old -- like they were written in 2009 or something

and by now they're, for example, referring to syslog.target as a dependency

whereas syslog is autostarted nowadays by socket activation.

So that could be removed, and then the service file would be simpler

and more idiomatic, and that would be a typical change that you could push upstream

I don't say that I expect any of you to know what the idiomatic service files

are, and contribute that upstream; I'm just saying that it would be the right thing to do.

Question?

[question]: So you say just write the debian/packagename.service and .tmpfiles

but most debhelpers also accept just debian/service and, just the type of the file

without the package prefix if there's only one binary package -- is that supported?

[stapelberg]: To be honest, I didn't test it, but i'm using the exact same debhelper plugin code

that all the others are using, so I would expect it to be supported.

What we also support, even though I didn't actually program it

is "package.(and then some actual package name)" if it's only for a specific package in a set of all packages

[ed: he means sourcepackgename.binarypackagename.service]
is "package.(and then some actual package name)" if it's only for a specific package in a set of all packages

which is a typical debhelper feature in that sense.
[ed: he means sourcepackgename.binarypackagename.service]

which is a typical debhelper feature in that sense.

[question]: Thanks!

[stapelberg]: Sure.
[question]: Thanks!

[stapelberg]: Sure.

Now, to actually cover that point, if upstream actually doesn't ship a service file and/or a tmpfile

then you can just place them in debian/$yourpackagename.service and debian/$yourpackagename.tmpfile and it will be installed.

It will be installed by dh_installinit, which might confuse you

because dh_installinit is for initscripts and not for servicefiles

and in fact, we by now have dh_systemd, so that's weird!

The only reason why that is is because of historic reasons.
and in fact, we by now have dh_systemd, so that's weird!

The only reason why that is is because of historic reasons.

We first started implementing it in dh_installinit,

but then it turned out that it would make the helper very complex and weird
We first started implementing it in dh_installinit,

but then it turned out that it would make the helper very complex and weird

and handling upstart and sysv and systemd all at the same time is really not a good idea
but then it turned out that it would make dhelper very complex and weird

and handling upstart and sysv and systemd all at the same time is really not a good idea

So we decided to leave it there for the time being, but eventually to migrate it

If you use just the dh command it will all just work, you don't have to care

In all the other cases, currently policy dictates that you still need to ship an initscript

so that's all fine, and as soon as policy gets changed, I promise that we will be in a state where you don't have to care about this either.

I already mentioned to please send service files upstream, but I'll just stress it again.

Not only if you have an upstream service file and modify it and fix it, please send it upstream

but also, if you create a service file, please send it upstream so that upstream can distribute it.

Some upstream software might not agree; some are actually very thankful.

Question over there

[question]: Assuming upstream ships a valid service file, what's the best practice:

Is it to call dh_installinit specifying the path in the upstream directory,

or copying it into the debian directory before dh_installinit is invoked?
Is it to call dh_installinit specifying the path in the upstream directory,

or copying it into the debian directory before dh_installinit is invoked?

[stapelberg]: In case upstream installs it, just let upstream install it. I'm not sure I get the question.

[question]: The question is: in case upstream does not install it; they ship it, but they do not install it?

[stapelberg]: OK, in case they don't install it, and then what is the second part of your question? just put it in debian/ as I explained.

[question]: So you can put it manually in debian/ before dh_installinit is invoked.

Or you can invoke dh_installinit with the path from the upstream directory

[stapelberg]: Oh, right, yes.
Or you can invoke dh_installinit with the path from the upstream directory

Or you can invoke dh_installinit with the path from the upstream directory

So if upstream doesn't install it, but ship it

[question]: Yes.
So if upstream doesn't install it, but ship it

[stapelberg]: Yeah, you could either manually copy it before, or just use what you suggest.
[question]: Yes.

[stapelberg]: Yeah, you could either manually copy it before, or just use what you suggest.

Any more questions?

Great, let's move on.

Getting your service enabled: I already mentioned that just as with update-rc.d you need to enable services

and the easiest way, if you already have a service file shipped by upstream

or if you put your own service file into debian/package.service

is you add a build dependency on dh-systemd

and then you use the dh command, which you hopefully already use

and add the --with=systemd flag

and then all automatically happens and it will just work.

The maintainer scripts that are generated

as part of that package build will contain the appropriate code

they will call a binary called deb-systemd-helper instead of systemctl as I mentioned earlier

which avoids having a dependency on systemd on all the packages which is probably politically not a good move.

If you are not using dh, you can also add the dh_systemd_enable and dh_systemd_start calls directly

in the wiki we have instructions, and the link is provided on these slides.

To test your package -- oh, a question here?

[question]: So you mentioned the deb-systemd-helper maintainer scripts

I know the scope of systemd is broader than sysvinit, but

in the sysvinit world, policy recommends

whether or not you are using sysvinit, including other things like sysvrc or openrc or a few of the other minor ones

(and I think also upstart) they recommend

using invoke-rc.d and update-rc.d in debian maintainer scripts as a way of

both abstracting from the specific system and for

handling the case where you're debootstrapping and the daemons are not running because of policy-rc.d

so the question is: why this instead of those

and if policy-rc.d is configured to not start or stop, will this respect that?

[stapelberg]: We have another of these helpers called deb-systemd-invoke which is for the invoke-rc.d part

and it will try to respect policy. Unfortunately, policy is really hard

It's a really horrible standard -- it's underdocumented and I had a really hard time figuring out how it works

so it will support the use cases I could identify

and if you have use cases that are broken please talk to me and we can try to fix it

Also, to answer the first part of your question, update-rc.d is really hard to have a good solution in there

we tried implementing it in there, but it turned out to not work that well, and upstream is not that responsive to our concerns.

so we actually chose to implement a separate helper that we have under tight control and can release independently of sysvrc

which turned out to be a much better solution because we already refactored it once and iterated on it

quite a few times, and by now it's actually in a pretty good state.

So that's why.

Alright! So, to test your package, which is actually the most interesting part

you will just install systemd, and then you can boot with the init=/bin/systemd kernel parameter

Installing systemd does not involve breaking sysvinit or anything

there's no conflicts in that package, you can still have both of them.

If you just install systemd, nothing will happen.

If you boot with init=/bin/systemd, you will actually use systemd

so that distinction is important. You can always switch back and forth.

So testing it is really simple. you just reboot.

You could reboot in a vm if you don't like to reboot your main machine.

Then what you would typically do is you check that your service starts properly on boot

You would probably check the reload action, check stop, start, restart, and that's about it, I would say.

because there's really not that much more to it, except if your service makes use of really advanced features and all that stuff.

In general, I would say that your users will report bug reports if your service does not work with systemd right now

There are plenty of users of systemd and debian that care enough to submit bug reports.

So it's not expected of you to test it all the time

and convert all your systems and run it all the time; a brief test will be enough.

Now to an advanced example:

There's actually a few more features that were maybe already mentioned if you listened to the previous talk by Lennart

We have a nice service called debian codesearch which I happen to maintain

And the service file is actually much more complicated.

It not only specifies a user and a group

It also has some arguments in here, and you can see that

the service file format supports line wrapping, so if your command line is pretty long

then it might make sense to wrap it and have it really nice.

Also, we have standard output redirected to /dev/null because it's really noisy

and mostly used for debugging. so when I feel like debugging, I can just change that to get standard output

but in the default case I just want standard error

Now also, my service does not actually care to log to syslog, so I say that standard error should go to the journal

which will then end up in the syslog.

Also, I cannot obviously exclude that there are bugs, but I also cannot sit in front of a computer 24/7 and restart my server if it crashes

So what I want is that whenever it fails (that is, it exits with an exit code that is not 0)

I want it to just wait a second and then restart it.

So far I think in production I only restarted my servers once

and then promptly fixed the bug so most of the time it's perfect, but you know: better safe than sorry!

The other parts of the unit file are probably clear by now

and what you will see in practice is that most of the service files really look

kind of the same. It's pretty simple, they're pretty short

they all use the same features.

One more interesting feature that you should be aware of

Who of you -- show of hands -- maintains a package that ships a dbus service?

one or two? OK

For the others, this is more of an academic interest

but systemd can actually care about dbus-activated services

so whereas dbus usually would start on its own in older versions

nowadays it's better to use systemd for it

because then it all ends up in the same hierarchy and it all gets tracked and you get all the benefits and stuff

so what you do is you add Type=dbus and you specify the BusName

and you don't have an [Install] section -- it's not missing on the slide, it's just not there.

and then systemd will activate that service whenever that bus name is actually accessed.

This is a more advanced example of the dh_systemd debhelper plugin

what we do here is we install a service that should not actually be installed by default.

and the way we do this is we override the dh_systemd_enable target and specify the --no-enable flag

This should not be a surprise to any of you who has been using debhelper in the past

I just wanted to mention it and make sure that you understand what the options are here.

-- I will answer that in a second --

The second example here is for the second part of the dh_systemd debhelper plugin

It's dh_systemd_start

and what I specify here is the --restart-after-upgrade flag which will make sure that the package does not get stopped

then replaced then started, but will just get replaced and then restarted afterward

which is kind of cleaner, but the package needs to support it. Question?

[question]: Why do you call the dh_systemd_enable with an option of --no-enable

instead of just leaving the target empty?

[stapelberg]: That's an excellent question, and the comment above tries to somewhat explain it

The thing is that when you purge the package

if the user decided to enable it even though you ship it disabled by default

then you need to clean up the symlinks

So that's what dh_systemd_enable also generates maintainer scripts for

So that still needs to run, so we can't just skip it.

In the first version we tried, but it didn't work out.

More questions?

In the back?

[question]: You mentioned services triggered by dbus actions

[stapelberg]: Yes

[question]: Is there a way to disable them even if the service is installed?

[stapelberg]: Yes, you can mask any service. I can show you that later.

One more question?

[question]: So you say --restart-after-upgrade; how does restart -- I guess this is more general, but it's prompted by this

how does restart work if the service is not running?

[stapelberg]: That's a good question. I would have to really look it up.

I think there is a try-restart action

that would restart if it's running

but not start it if it's not running

[question]: For example, in this case you set up the service to

remain disabled upon install in this example

so if you install the package it is not enabled

Then the user takes no action
so if you install the package it is not enabled

Then the user takes no action

and then they upgrade their system

and a new version of the package is installed, it tries to restart?

[stapelberg]: The thing is, as I was saying, there is a restart action in systemctl and there is a try-restart

The difference is try-restart will only restart it if it's running, which probably answers your question

The caveat is that

currently, if you maintain a package that ships

a system V initscript and a systemd service file, it will still use invoke-rc.d for the actual starting/restarting etc.

And the invoke-rc.d actually has code to divert that to systemd

but the problem is it's not flexible enough to use try-restart and the fancy stuff

so this might need some actual hand-tweaking or just ignoring it for now.

Alright! So before we enter the questions and hands-on part of this workshop

I just want to make sure that you're all aware that we will provide help

There is a wiki page which is linked here called https://wiki.debian.org/Systemd/Packaging

which contains most of the information hopefully, or at least pointers

We have an irc channel on irc.debian.org called #debian-systemd

where you can just stop by at any time

Most of the time there's somebody around who actually knows how to write service files and stuff

There is a mailing list that we are all active and we really do mean it

Please ask! Also, during debconf, if there is at any time any questions

from any of you or from your friends

please ask. we're here for answering these sorts of questions.

Now just one more quick note: finding documentation

There are manpages -- a lot of manpages

They are roughly categorized by the sections I previously showed in service files

So there's a systemd.service, there's a systemd.exec, etc.

There's also an overview on the freedesktop.org web site (http://www.freedesktop.org/wiki/Software/systemd/) where it points to all the documentation

The particularly interesting parts of that are the systemd for administrators blog series

where Lennart talks about a lot of features that are interesting and how to actually make use of them in your service files

And then there's a link for package repositories for the various distributions

where you can just look if there already is a service file for that particular package that you maintain

even though it doesn't ship one upstream

So the best thing in that case would also be to adopt the service file and then also make upstream accept that.

Alright! That's the talk part so far. Now I'm ready to answer any questions or look at any packages

[question]: What are the plans for backports?

I mean if you want to make backports to wheezy, can we use the dh_systemd helper?

[stapelberg]: Yes, the dh_systemd helper is available in wheezy backports

Be aware that the systemd version in wheezy is version 44, and we're currently trying to get version 204, which is much more recent

into debian -- it had a version jump because of udev.

So it's not that much more recent, it's just more recent.

So there might be issues and it's up to you if you decide to commit to maintaining support

for that old systemd version with your service files in wheezy, just so you know.

If there is no immediate question, I would just go on to show you a little bit of stuff and we can answer questions as we go

[question]: I have to prepend that at the moment I'm not maintaining any package that has to start daemons at poweron

So I know nearly nothing about it.

But I am a bit confused. Since I understand there are a lot of these kind of systems

systemd, sysvinit, and so on

As a package maintainer, what do I have to do

if I want to support all of them, I have to provide

the script or description file that they need for each of them

[stapelberg]: -- to answer this, the policy mandates --
the script or description file that they need for each of them

[stapelberg]: -- to answer this, the policy mandates --

[question]: For example, just to complete the question, in particular if we consider just sysvinit, it is one that we have
[stapelberg]: -- to answer this, the policy mandates --

[question]: For example, just to complete the question, in particular if we consider just sysvinit, it is one that we have

by default, and systemd, which we are discussing now

If I want to provide a systemd information file

Do I also have to provide a sysvinit scripts

or is there some compatibility layer that enables me to write one thing

and (at least in common cases) expect some magic to make it work for other systems

[stapelberg]: OK, I would be happy to answer that after the talk because that's not really the focus of this talk.

So now, let me just show you a few handy things that might be useful

I have a terminal here that you hopefully can read in the back

Is that OK? Great! So let's just have a look at

thinkfan.service and what I was using here is the systemctl command and you can see multiple interesting things

First of all, it is active and running, so that's good.

It shows that I started it 6 days ago, when I last rebooted my laptop

It also shows the main pid, which is 2588, and that's the binary that corresponds to it.

For more complex services, there are more binaries in the cgroup hierarchy

What is particularly interesting for you is

first of all, where the service file actually lives; so make sure you ship it to the correct path.

There's also a lintian warning for that, so if you use lintian you should catch that

It should go to /lib/systemd/system, as I mentioned

It also should be enabled unless, of course, you decided to not enable it by default

Now, I can just show you systemctl stop, it will be that

[question]: the video projector does not show all of the terminal

[stapelberg]: That is a good point. The first two columns are not shown, so let's make it like this

Should be much better!

Not perfect. Better?

[question, unintelligible]

[stapelberg]: Yeah, whatever. That's good enough.

thinkfan.service -- now it's still enabled. It's still loaded but it's inactive because I stopped it

I can start it again

check that it's started

You can also see that it used this ExecStart line

I can also actually show you the service file. There's really no magic in here.

There is an ExecReload directive here, so we can test if the reload actually works

Which I need to do as root

And then in the status output we will see that it tried to reload the service

code=exited, status = 0/SUCCESS, so that worked.

So those are a few simple things you can check to see if your service actually works

10 minutes, yes. Are there any questions now? Microphone?

[question]: The example has Type=forking. What are the other values which are valid?

[stapelberg]: So that was obviously a suggestion to open the man page and show you that actually it is documented

The manpage in question here is systemd.service, as I tried to explain earlier

There's multiple Types -- there is simple

There's forking, oneshot, dbus, notify or idle

The most interesting ones are simple, forking, oneshot, and dbus

dbus I already explained -- that's if you have an actual dbus service

There's oneshot, which is for stuff like doing one thing and then nothing

like, it's not a permanently running service, it's just a simple command like a shell script

and then it will stay active, as in "started" afterwards

There is forking and there is simple. The difference between simple and forking

is that simple will -- the daemon you're starting, if it's a Type=simple one

should just continue running in the foreground, whereas a forking one will fork itself into background.

The preferred model is using simple because, you know, it's simple.

forking has the implicit assumption, and I think this is important to know, that

as long as the daemon is still running in the foreground, it's not ready

The unit file will be considered started precisely at the moment where the daemon forks.

That is not necessarily what your upstream software implements, but that's how it's

[question, unintelligible]

[stapelberg]: The question was: when it forks, or when the main process exits?

Obviously, I think when the main process exits.

[question]: It would be really strange in the other case

[stapelberg]: That's how it usually works, right? You double-fork and then you exit the main process.

More questions?

Masking processes -- perfect, let's do that!

So I have thinkfan and I now decide

That it's really a shame that my fan is not spinning up as much as I would like

So what I will just do is I will mask thinkfan.service and it will helpfully

print out what it actually did, which is it just created a symlink in /etc/systemd/system/thinkfan.service

pointing to /dev/null

Essentially, it will try to load that service file but fail, because it can't read /dev/null.

So if I now check status

on thinkfan.service, it will tell me that it's masked

but it's also still running. So if I now do systemctl stop

and then status, it's now dead, and it will not be started on my next boot.

This is different from enable and disable because it also works for dbus services and it's really like the last resort

If you really really don't want this thing to be started, mask it. Question?

[question]: In this case, you cannot start it by hand, you have another method implemented in systemd

[question]: If I want it to not start at boot, but I want to be able to run it by hand

[stapelberg]: Yes, then you obviously disable it because disable just means don't start it at boot, but you can still start and stop and restart and all that stuff

masked is really "don't start this at all"

So now, because I don't like loud fans, let me just unmask that

So it told me that it deleted that symlink, and I can start it again and it will just work.

More questions please? Over there.

[question]: When it's active, it says "active (running)" or it says "inactive (dead)"

Is there any particular meaning that there is a part "active" and a part in parenthesis?

[stapelberg]: It could be "active (exited)" and that would be the case for the oneshot services

that just fork one command, then exited, but the unit is still considered active because the command succeeded.

More questions over here?

[question]: Is there any support for -- what happens when a daemon dies, is it restarted, like a few times, or ?

[stapelberg]: I actually had this on my slides earlier

Yes: more features, here we go. You can specify RestartSec=1 and Restart=on-failure

There's more options in that direction to restart stuff when it dies

More questions please? Here.

[question]: It's about the packaging

Is it possible to use dpkg triggers -- instead we just put a file in /lib/systemd

And then the dpkg trigger happens

It means it will work with other packaging systems like cdbs without modification

[stapelberg]: we actually have sent a patch to cdbs and it supports dh_systemd by now.
It means it will work with other packaging systems like cdbs without modification

[stapelberg]: we actually have sent a patch to cdbs and it supports dh_systemd by now.

[question]: OK, but why are you not using trigger support?

[stapelberg]: Not using what, sorry?

[question]: Instead of inserting a debhelper snippet in postinst

you could have registered a trigger

[stapelberg]: It's more complicated than that because

[question]: It will be executed when it detects a file appearing in some allocation
[stapelberg]: It's more complicated than that because

[question]: It will be executed when it detects a file appearing in some allocation

[stapelberg]: Yeah, but we don't want to enable all the services by default

And then we would need to maintain a whitelist or a blacklist of services

and stuff, and it really needs to be more flexible than that

OK, more questions?

Another one?

[question]: This time it's for services that need several instances

Like stunnel

[stapelberg]: Oh, multiple instances, yes

[question]: Yes, multiple instances; usually they come with several configuration files

And we want one instance per configuration file

And be able to control them separately

[stapelberg]: Yes, so there are actually --

[question]: It's a bummer[?] to do that with sysV
[stapelberg]: Yes, so there are actually --

[question]: It's a bummer[?] to do that with sysV

[stapelberg]: There are good examples here of how that works. There's ifup and there's getty
[question]: It's a bummer[?] to do that with sysV

[stapelberg]: There are good examples here of how that works. There's ifup and there's getty

Let me just show you getty

which has [e-mail]

and what's interesting is you can see that

the service file path actually does not include that

so if you have a look at the service file

You can see that it's much more complicated than I would like it to be

But the interesting part for us is that there's %I

which will be replaced with whatever you specify after the @

So you can make your service file contain the @

in the filename and then use %I and then start a specific instance of that

[question]: And the user is expected to create all the @ files?

[stapelberg]: Yes

OK! More questions?

Here.

[question]: How about this socket-based activation?

[stapelberg]: Socket activation -- that is a good question!

Let me just have a look if I have any socket activation files lying around here

I have approx.socket, which seems kind of appropriate for this conference.

Socket activation essentially works like this, where you have

[Socket] instead of [Service] in your service file

In your socket file, sorry

You specify a TCP port

or a unix socket or whatever it should listen on

And then there is Accept=yes

where the default is Accept=no

The difference is that Accept=yes mimics the inetd-style behavior

of just starting one process per connection, which is kind of wasteful

and the actual real good socket activation is

not having that, but patching the service to

when being started, inherit the file descriptor

of the socket and then just integrate it in its event loop and handle that.

And many services are already patched for that.

Some of them are patched, but not in debian, and some of them you would need to patch.

But this is really a thing of like an hour or two.

I did it for bacula once, for example.

OK. Question over here?

[question]: So the socket file just includes information about where to listen

And if to accept or not, or to start. Is that just in the service file then?

[stapelberg]: Yes, the socket file has to match, by name, the service file

And because I used Accept=yes, it needs to be approx@.service, not approx.service

And then this will in turn just take the standard input from the socket, like inetd does, and start it up.

One more question?

[question]: Why do I have to add the @?

[stapelberg]: Because if you use inetd-style stuff, then for every incoming new connection it will start a separate process.

so all these processes show up in your cgroup hierarchy.
[stapelberg]: Because if you use inetd-style stuff, then for every incoming new connection it will start a separate process.

so all these processes show up in your cgroup hierarchy.

So that's why the instance is used in this case.

Does that also relate to ExecStart=- instead of directly the path?

No, in fact, I already looked at this and wondered why it is there

I think it should not be there, because the - means that failures should be ignored.

I'm not entirely sure why it makes sense to have it here

My suspicion would be that if there's a connection and then it goes away

then it shouldn't -- I would need to look that up. Sorry, no answer.

More questions? Last question, we have one minute left.

Last question over there.

[question]: Thank you. Where are dependencies defined in this thing?

[stapelberg]: Oh yes, that's a good question. Ideally, dependencies are not defined anywhere because they are implicit by socket activation.

In case that's not the case, for example, if my

-- I think I should probably have it here, let me just have a look real quick --

dcs-web.service, it's actually more complicated. No it doesn't have it.

So there is, in the [Unit] section, where we also specified the Description, there can be a Before= and After=

So in my case, I would start codesearch after postgresql came up

and I would just specify After=postgresql.service

You can specify multiple services there, you can specify the directive multiple times

But usually it should not be required to specify dependencies, which is nice.

OK, thank you.

OK, time is over. I would like to say, thank you very much for your attention.

If you have any questions, please let me know and we can fix it.

[talkmeister]: Thank you, Michael.

[clapping]
[talkmeister]: Thank you, Michael.

[clapping]