0:00:05.686,0:00:07.190 Thank you everyone for coming. 0:00:08.005,0:00:12.345 If you were expecting the Postgres talk,[br]that was the one before, so 0:00:12.345,0:00:14.836 you might need to watch the video stream. 0:00:16.614,0:00:18.120 So, Ansible best practices, 0:00:18.619,0:00:22.249 I thought about calling it "Ansible,[br]my best practices", 0:00:22.740,0:00:29.806 so, just warning ahead, this is things[br]I stumbled on using Ansible 0:00:29.806,0:00:32.121 for the last 2-3 years and 0:00:32.121,0:00:37.240 those are very specific things I found[br]that worked very well for me. 0:00:39.075,0:00:45.725 About me, I do also freelance work,[br]do a lot of Ansible in there, 0:00:46.081,0:00:51.901 I'm also the Debian maintainer for[br]Ansible with Harlan Lieberman-Berg 0:00:54.055,0:00:57.792 If there are any bugs in the package,[br]just report them. 0:01:06.481,0:01:10.217 The talk will be roughly divided into[br]4 parts. 0:01:14.523,0:01:19.928 The first part will be about why you[br]actually want to use config management 0:01:19.928,0:01:23.478 and why you specifically want to use[br]Ansible. 0:01:24.005,0:01:30.340 So, if you're still SSHing into machines[br]and editing config files, 0:01:30.340,0:01:33.553 you're probably a good candidate[br]for using Ansible. 0:01:35.627,0:01:41.273 Then, the second part will be about good[br]roles and playbook patterns 0:01:41.799,0:01:43.910 that I have found that work really well[br]for me. 0:01:47.130,0:01:52.533 The third chapter will be about typical[br]antipatterns I've stumbled upon, 0:01:52.533,0:01:57.696 either in my work with other people[br]using Ansible, 0:01:57.696,0:02:00.740 or the IRC support channel, for example. 0:02:02.648,0:02:08.537 The fourth part will be like advanced[br]tips and tricks you can use 0:02:08.537,0:02:11.476 like fun things you can do with Ansible. 0:02:12.945,0:02:16.275 Quick elevator pitch, what makes config[br]management good? 0:02:18.263,0:02:25.492 It actually also serves as a documentation[br]of changes on your servers over time 0:02:25.492,0:02:29.237 so if you just put the whole config[br]management in a git repo 0:02:29.237,0:02:30.986 and just regularly commit, 0:02:30.986,0:02:32.577 you will actually be able to say 0:02:32.577,0:02:35.506 "Why doesn't this work? It used to work[br]a year ago" 0:02:35.506,0:02:38.717 You can actually check why. 0:02:41.399,0:02:49.761 Also, most config management tools have[br]a lot better error reporting than 0:02:49.761,0:02:53.295 your self-written bash scripts that do[br]whatever. 0:02:56.180,0:03:02.891 And usually, you have a very good[br]reproducibility with config management 0:03:02.891,0:03:10.813 and also idempotency, meaning that if you[br]run, for example, a playbook several times 0:03:10.813,0:03:12.761 you will always get the same result. 0:03:14.751,0:03:23.655 Also, it's great if you work in small team[br]or you admin ??? in the company 0:03:23.655,0:03:26.830 and you have some people working[br]on a few things too. 0:03:29.384,0:03:33.374 It makes team work a lot easier and[br]you will save a lot of time actually 0:03:33.374,0:03:35.890 debugging things when things break. 0:03:37.840,0:03:39.426 What makes Ansible good? 0:03:40.273,0:03:45.612 Comparing it to Chef or Puppet for example[br]it's really easy to set up, 0:03:45.612,0:03:50.369 you start with two config files, you have[br]it installed and you're ready to go. 0:03:52.399,0:03:56.419 It's also agentless, so whatever machines[br]you actually want to control, 0:03:56.419,0:04:05.237 the only thing you they really need to have[br]is an SSH daemon and Python 2.6+ 0:04:05.480,0:04:10.678 so that's virtually any Debian machine[br]you have installed and 0:04:10.678,0:04:12.710 that is still supported in any way. 0:04:15.084,0:04:21.668 Ansible also supports configuration[br]of many things like 0:04:21.668,0:04:25.896 networking equipment or even Windows[br]machines, 0:04:25.896,0:04:30.742 they don't need SSH but they use the[br]WinRM 0:04:30.742,0:04:39.068 but Ansible came a bit late to the game[br]so Ansible's still not as good 0:04:39.068,0:04:41.435 in coverage like for example Puppet, 0:04:41.916,0:04:46.555 which literally, you can configure any[br]machine on the planet with that, 0:04:46.555,0:04:48.389 as long as it has a CPU. 0:04:50.380,0:04:53.918 Next step, I will talk about good[br]role patterns. 0:04:57.010,0:04:58.837 If you've never worked with Ansible[br]before, 0:04:58.837,0:05:01.835 this is the point when you watch[br]the video stream, 0:05:01.835,0:05:05.698 that you pause it and start working[br]a few weeks with it 0:05:05.698,0:05:08.380 and then unpause the actual video. 0:05:13.338,0:05:17.529 A good role should ideally have[br]the following layout. 0:05:18.790,0:05:24.970 So, in the "roles" directory, you have[br]the name of the role and task/main.yml 0:05:25.945,0:05:29.202 You have the following rough layout. 0:05:31.561,0:05:38.720 At the beginning of the role, you check[br]for various conditions, 0:05:38.720,0:05:44.085 for example using the "assert" task to[br]for example check that 0:05:44.085,0:05:48.190 certain variables are defined, things[br]are set, 0:05:48.190,0:05:53.235 that it's maybe part of a group, things[br]like that you actually want to check. 0:05:54.660,0:06:03.122 Then, usually, you install packages, you[br]can use apt, or on CentOS machines, yum 0:06:03.504,0:06:05.464 or you can do a git checkout or[br]whatever, 0:06:07.092,0:06:14.106 then usually you do some templating of[br]files where you have certain abstraction 0:06:14.106,0:06:18.579 and the variables are actually put into[br]the template and 0:06:18.579,0:06:21.027 make the actual config file. 0:06:22.480,0:06:26.639 There's also good to point out that[br]the template module actually has 0:06:26.639,0:06:29.934 a "validate" parameter, 0:06:30.343,0:06:35.953 that means you can actually use a command[br]to check your config files for syntax errors 0:06:35.953,0:06:44.193 and if that fails, your playbook will fail[br]before actually deploying that config file 0:06:44.193,0:06:53.179 so you can for example use Apache with[br]the right parameters to actually do 0:06:53.179,0:06:56.670 a check on the syntax of the file. 0:06:57.238,0:07:01.829 That way, you never end up with a state[br]where there's a broken config. 0:07:03.581,0:07:05.410 In the end, you usually… 0:07:06.018,0:07:10.038 When you change things, you trigger[br]handlers to restart any daemons. 0:07:12.448,0:07:23.623 If you use variables, I recommend putting[br]sensible defaults in 0:07:23.623,0:07:26.753 defaults/main.yml 0:07:28.126,0:07:34.798 and then you only have to override[br]those variables on specific cases. 0:07:35.486,0:07:41.260 Ideally, you should have sensible defaults[br]you want to have to get whatever things 0:07:41.260,0:07:42.806 you want to have running. 0:07:45.847,0:07:51.949 When you start working with it and do that[br]a bit more, 0:07:51.949,0:07:58.499 you notice a few things and that is 0:07:58.499,0:08:01.950 your role should ideally run in "check mode". 0:08:02.275,0:08:07.560 "ansible-playbook" has --check that[br]basically is just a dry run of 0:08:07.560,0:08:11.586 your complete playbook 0:08:11.586,0:08:17.642 and with --diff, it will actually show you[br]for example file changes, 0:08:17.642,0:08:20.728 or file mode changes, stuff like that 0:08:20.728,0:08:23.860 and won't actually change anything. 0:08:24.184,0:08:31.579 So if you end up editing a lot of stuff,[br]you can use that as a check. 0:08:32.270,0:08:37.229 I'll later get to some antipatterns that[br]actually break that thing. 0:08:40.075,0:08:47.146 And, ideally, the way you change files[br]and configs and states, 0:08:47.146,0:08:50.724 you should make sure that when the actual[br]changes are deployed, 0:08:50.724,0:08:53.164 and you run it a second time, 0:08:53.164,0:08:57.629 that Ansible doesn't report any changes 0:08:57.629,0:09:02.939 because if you end up writing your roles[br]fairly sloppy, you end up having 0:09:02.939,0:09:05.880 a lot of changes and then, 0:09:05.880,0:09:10.718 in the end of the report, you have like[br]20 changes reported and 0:09:10.718,0:09:14.790 you kind of then know those 18,[br]they're always there 0:09:14.790,0:09:18.414 and you kind of miss the 2 that are[br]important, that actually broke your system 0:09:18.414,0:09:25.168 If you want to do it really well, you make[br]sure that it doesn't report any changes 0:09:25.168,0:09:27.407 when you run it twice in a row. 0:09:30.977,0:09:38.494 Also, a thing to consider is you can define[br]variables in the "defaults" folder 0:09:38.494,0:09:40.485 and also in the "vars" folder, 0:09:41.256,0:09:46.095 but if you look up how variables get[br]inherited, you'll notice that 0:09:46.095,0:09:49.715 the "vars" folder is really hard to[br]actually override, 0:09:50.154,0:09:53.495 so you want to avoid that as much as[br]possible. 0:09:58.989,0:10:05.859 That much larger section will be about[br]typical anti-patterns I've noticed 0:10:05.859,0:10:10.490 and I'll come to the first one now. 0:10:11.632,0:10:15.171 It's the shell or command module. 0:10:17.290,0:10:20.462 When people start using Ansible, that's[br]the first thing they go 0:10:20.462,0:10:26.075 "Oh well, I know how to use wget or I know[br]'apt-get install' " 0:10:26.075,0:10:29.768 and then they end up using the shell module[br]to do just that. 0:10:30.540,0:10:35.388 If you use the shell module or the command[br]module, you usually don't want to use that 0:10:35.388,0:10:38.555 and that's for several reasons. 0:10:40.143,0:10:46.520 There's currently, I think, 1300 different[br]modules in Ansible 0:10:46.520,0:10:50.506 so there's likely a big chance that[br]whatever you want to do, 0:10:50.506,0:10:53.768 there's already a module for that, that[br]just does that thing. 0:10:54.664,0:11:02.929 But those two modules also have several[br]problems and that is 0:11:02.929,0:11:09.640 the shell module, of course, gets[br]interpreted by your actual shell, 0:11:09.640,0:11:12.526 so if you have any special variables[br]in there, 0:11:12.526,0:11:21.912 you'd actually also have to take care of[br]any variables you interpret in the shell string. 0:11:24.552,0:11:31.459 Then, one of the biggest problems is if[br]you run your playbook in check mode, 0:11:31.459,0:11:34.263 the shell and the command modules[br]won't get run. 0:11:34.711,0:11:38.044 So if you're actually doing anything[br]with that, they just get skipped 0:11:38.044,0:11:47.595 and that would cause that your actual[br]check mode and the real mode, 0:11:47.595,0:11:51.567 they will start diverging if you use[br]a lot of shell module. 0:11:55.594,0:12:01.283 The worst, also, a bad part about this[br]is that these two modules, 0:12:01.283,0:12:03.597 they'll always ??? changed 0:12:03.597,0:12:06.117 like, you run a command and it exits 0 0:12:06.117,0:12:07.659 it's like "Oh, it changed" 0:12:10.909,0:12:17.855 To get the reporting right on that module,[br]you'd actually have to define for yourself 0:12:17.855,0:12:21.073 when this is actually a change or not. 0:12:21.607,0:12:29.333 So you'd have to probably get the output[br]and then check, for example, 0:12:29.333,0:12:35.303 if there's something on stderr or something[br]to report an actual error or change. 0:12:38.395,0:12:40.592 Then I'll get to the actual examples. 0:12:41.201,0:12:46.237 The left is a bad example for using[br]the shell module, 0:12:46.237,0:12:48.636 I've seen that a lot, it's basically 0:12:48.636,0:12:56.567 "Yeah, I actually want this file, so just[br]use 'cat /path/file' and I'll use 0:12:56.567,0:12:59.825 the register parameter to get the output". 0:13:06.166,0:13:10.965 The actual output goes into the "shell_cmd"[br]and then 0:13:10.965,0:13:16.201 we want to copy it to some other file[br]somewhere else and 0:13:16.201,0:13:25.657 so we use the Jinja "{{ }}" to define[br]the actual content of the file 0:13:25.657,0:13:30.626 and then put it into that destination file 0:13:31.563,0:13:37.333 That is problematic because, first of all[br]if you run it in check mode, 0:13:37.333,0:13:40.583 this gets skipped and then this variable[br]is undefined and 0:13:40.583,0:13:45.499 Ansible will fail with an error, so you[br]won't be able to actually 0:13:45.499,0:13:47.081 run that in check mode. 0:13:48.219,0:13:51.019 The other problem is this will always[br]??? 0:13:51.995,0:13:54.962 so you'd probably have to… 0:13:56.995,0:14:01.386 the most sensible thing would probably[br]be to say just "changed when false" 0:14:01.712,0:14:06.349 and just acknowledge that that shell[br]command won't change anything on this system 0:14:07.609,0:14:13.824 The good example would be to use the[br]actual "slurp" module that will 0:14:13.824,0:14:17.092 just slurp the whole file and base64encode it 0:14:18.279,0:14:28.146 and you can access the actual content with[br]"path_file.contents" and you then just 0:14:28.146,0:14:30.709 base64decode it and write in there. 0:14:31.931,0:14:39.247 The nice thing is slurp will never return[br]any change, so it won't say it changed 0:14:39.247,0:14:42.783 and it also works great in check mode. 0:14:46.482,0:14:48.432 Here's an other quick example. 0:14:49.893,0:14:52.655 The example on the left, oh yeah wget. 0:14:53.876,0:14:59.602 Here's the problem, every time your playbook[br]runs, this file will get downloaded 0:14:59.602,0:15:07.610 and of course if the file can't be[br]retrieved from that URL 0:15:07.610,0:15:12.766 it will throw an error and that will[br]happen all the time. 0:15:14.600,0:15:19.077 The right example is a more clean example[br]using the uri module. 0:15:20.417,0:15:27.569 You define a URL to retrieve a file from,[br]you define where you want to write it to 0:15:27.569,0:15:31.470 and you use the "creates" parameter to say 0:15:31.470,0:15:34.889 "Just skip the whole thing if the file is[br]already there". 0:15:40.048,0:15:43.458 "set_facts", that's my pet peeve. 0:15:44.718,0:15:49.553 set_facts is a module that allows you[br]to define variables 0:15:49.553,0:15:56.945 during your playbook run, so you can say[br]set_facts and then 0:15:56.945,0:16:02.922 this variable = that variable + a third[br]variable or whatever 0:16:02.922,0:16:04.914 you can do things with that. 0:16:06.375,0:16:13.117 It's very problematic, though, because[br]you end up having your variables 0:16:13.117,0:16:15.394 changed during the playbook run 0:16:15.394,0:16:24.781 and that is a problem when you use[br]the "--start-at" parameter 0:16:24.781,0:16:26.403 from ansible-playbook. 0:16:29.979,0:16:36.436 Because this parameter allows you to[br]skip forward to a certain task in a role 0:16:36.436,0:16:40.133 so it skips everything until that point[br]and then continues running there 0:16:40.133,0:16:41.882 and that's really great for debugging 0:16:41.882,0:16:48.874 but if you define a variable with set_facts[br]and you skip over it, 0:16:48.874,0:16:50.856 that variable would just not be defined. 0:16:53.587,0:17:02.028 If you heavily use set_facts, that makes[br]prototyping really horrible. 0:17:04.914,0:17:07.557 Another point is that you can use 0:17:07.557,0:17:13.411 "ansible -m setup" and then the hostname[br]to check what variables are actually defined 0:17:13.411,0:17:18.521 for a specific host and everything set[br]with set_facts is just not there. 0:17:22.227,0:17:27.025 In summary, avoid the shell module,[br]avoid the command module, 0:17:27.025,0:17:29.872 avoid set_facts as much as you can, 0:17:29.872,0:17:36.622 and don't hide changes with "changed_when" 0:17:36.622,0:17:41.538 so the clean approach is always to use one[br]task to check something 0:17:41.538,0:17:46.099 and then a second task to actually execute[br]something for example. 0:17:48.458,0:17:52.327 Also, a bad idea in my opinion is when[br]people say 0:17:52.327,0:17:55.948 "Oh well, it's not important if this[br]throws an error or not, 0:17:55.948,0:17:58.874 I'll just say 'fails when false'" 0:18:00.177,0:18:06.476 That might work sometimes, but the problem[br]there is, if something really breaks, 0:18:06.476,0:18:08.059 you'll never find out. 0:18:09.196,0:18:10.697 Advanced topics. 0:18:13.748,0:18:17.316 This is about the templating. 0:18:18.870,0:18:21.919 The usual approach, for example for[br]postfix role, 0:18:21.919,0:18:24.686 would be to do the following templating. 0:18:25.461,0:18:36.479 You define certain variables in for example[br]group_vars/postfix_servers 0:18:36.479,0:18:40.868 so any host in that group would inherit[br]these variables, 0:18:41.558,0:18:47.902 so this is sort of a list of parameters[br]for smtp recipient restrictions 0:18:48.923,0:18:54.247 and this is just the smtp helo required. 0:18:55.142,0:18:58.152 So the usual approach would be to[br]define variables 0:18:58.152,0:19:02.730 in the host_vars or group_vars, or even[br]in the defaults 0:19:02.730,0:19:08.074 and then you have a template where[br]you just check every single variable 0:19:08.074,0:19:15.226 If it exists, you actually sort of put[br]the actual value there in place. 0:19:18.033,0:19:23.717 Here, I check if this variable is set true[br]and if yes, put the string there 0:19:23.717,0:19:26.778 else, put this string there 0:19:27.824,0:19:34.130 and for example, smtpd_recipient_restrictions[br]I just iterate over this array 0:19:34.130,0:19:38.435 and just output these values in order[br]in that list. 0:19:41.846,0:19:47.290 The problem here is that every time[br]upstream defines a new variable 0:19:47.290,0:19:56.684 you'll end up having to touch the actual[br]template file and touch the actual variables 0:19:56.952,0:20:04.102 so, I thought, "Well, you actually have[br]keys and values and strings and arrays 0:20:04.102,0:20:09.468 and hashes on one side, and actually,[br]a config file is nothing else than that, 0:20:09.947,0:20:11.664 just in a different format". 0:20:12.428,0:20:16.750 So I came up with… 0:20:18.092,0:20:24.354 With Jinja2, you can also define functions 0:20:24.354,0:20:29.475 I'll have to cut short a little bit on[br]explaining it but 0:20:29.475,0:20:36.229 basically, up here, a function is defined[br]and it's called here in the bottom 0:20:36.229,0:20:43.589 Basically, what it just does, it iterates[br]over the whole dictionary defined here, 0:20:43.833,0:20:47.003 "postfix.main", and it just goes… 0:20:48.672,0:20:51.511 It iterates over all the keys and values[br]and it goes… 0:20:53.302,0:20:57.933 If the value is a string, I'll just put[br]"key = value" and 0:20:57.933,0:21:04.063 if it's an array, I just iterate over it[br]and put it there in the format that 0:21:04.063,0:21:05.702 postfix actually wants. 0:21:07.889,0:21:11.808 Basically, you can do the same, for[br]example, for haproxy and 0:21:11.808,0:21:18.428 you can just deserialize all the variables[br]you actually defined. 0:21:20.258,0:21:22.576 The advantages of this is, 0:21:22.576,0:21:27.966 your template file just stays the same[br]and it doesn't get messy 0:21:27.966,0:21:29.725 if you start adding things. 0:21:30.703,0:21:34.524 You have complete whitespace control,[br]usually if you edit stuff, 0:21:34.524,0:21:39.076 you kind of get an extra space, a new[br]line in there, and that changes 0:21:39.076,0:21:42.492 the template files for all machines. 0:21:43.629,0:21:49.319 You have all the settings in alphabetical[br]order, so if you actually run it and 0:21:49.319,0:21:54.723 you see the diff, you don't end up having[br]things going back and forth. 0:21:56.711,0:22:00.564 If you get the syntax on the template file[br]right, you don't have to touch it after that 0:22:00.564,0:22:05.965 and you also don't get any syntax errors[br]by editing them. 0:22:13.889,0:22:16.003 That follows to the next one. 0:22:17.915,0:22:23.889 You can actually set a "hash_behaviour"[br]merge in the Ansible config and 0:22:23.889,0:22:26.974 that allows you to do the following. 0:22:28.241,0:22:39.333 On the left here, you define for example[br]a dictionary and this is, like, in a group 0:22:39.333,0:22:45.350 and then in a specific machine, you define[br]an other setting in this dictionary. 0:22:46.325,0:22:51.206 If you wouldn't use merge, the second[br]setting would just override the first one 0:22:51.206,0:22:53.684 and you'd end up with that, but if you[br]actually do the merge, 0:22:53.684,0:22:55.591 it does a deep merge of the hash. 0:22:56.608,0:23:03.592 So the previous thing I showed would[br]actually benefit from that 0:23:03.805,0:23:06.410 so the combination of both is really good. 0:23:08.438,0:23:09.864 I'll skip that. 0:23:10.312,0:23:16.001 Further resources. Ansible has just[br]a really good documentation, 0:23:16.001,0:23:22.824 there's the IRC and there's also debops[br]which is a project that is 0:23:22.824,0:23:27.571 specific to Debian and derivatives. 0:23:30.341,0:23:31.475 That's it. 0:23:31.765,0:23:37.165 [Applause] 0:23:39.284,0:23:40.906 Thank you very much.