[Script Info] Title: [Events] Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text Dialogue: 0,0:00:00.11,0:00:14.32,Default,,0000,0000,0000,,{\i1}music{\i0} Dialogue: 0,0:00:14.32,0:00:18.81,Default,,0000,0000,0000,,{\i1}applause{\i0} Dialogue: 0,0:00:18.81,0:00:23.56,Default,,0000,0000,0000,,Raichoo: Yeah sorry about that - beamers\Nor projectors, I don't like them. They Dialogue: 0,0:00:23.56,0:00:27.21,Default,,0000,0000,0000,,don't like me either. So this is a little\Nheads up - this is going to be the only Dialogue: 0,0:00:27.21,0:00:32.05,Default,,0000,0000,0000,,slide I'm going to show you today so,\N"slide", because I think doing stuff like Dialogue: 0,0:00:32.05,0:00:35.94,Default,,0000,0000,0000,,that in a terminal might be a little bit\Nmore interesting for you. But sadly Dialogue: 0,0:00:35.94,0:00:40.02,Default,,0000,0000,0000,,something is getting cut off so I we have\Nto improvise a little bit. But anyway, so Dialogue: 0,0:00:40.02,0:00:43.96,Default,,0000,0000,0000,,today I will be able to talk about two of\Nmy favorite things right now which are Dialogue: 0,0:00:43.96,0:00:47.82,Default,,0000,0000,0000,,FreeBSD and DTrace. But this talk has been\Ncapped down to 30 minutes so we'll be Dialogue: 0,0:00:47.82,0:00:53.19,Default,,0000,0000,0000,,focusing a little more on the DTrace part.\NSo there will be a little bit less BSD Dialogue: 0,0:00:53.19,0:00:57.56,Default,,0000,0000,0000,,than I anticipated. And also adjusted\Neverything a little bit to fit better into Dialogue: 0,0:00:57.56,0:01:03.13,Default,,0000,0000,0000,,the resilience track so hopefully you'll\Nenjoy that. So before we begin, who here Dialogue: 0,0:01:03.13,0:01:08.64,Default,,0000,0000,0000,,is actually using DTrace? Okay more than\NI expected but still not as many as I Dialogue: 0,0:01:08.64,0:01:12.61,Default,,0000,0000,0000,,would like to see. So hopefully after this\Ntalk you will think, "oh, this is a really Dialogue: 0,0:01:12.61,0:01:17.17,Default,,0000,0000,0000,,awesome tool, I gotta learn it." Because I\Ntotally love it - it changed the way I do Dialogue: 0,0:01:17.17,0:01:22.11,Default,,0000,0000,0000,,a lot of stuff. So for those of you who do\Nnot know what DTrace is, first, let me Dialogue: 0,0:01:22.11,0:01:27.26,Default,,0000,0000,0000,,fill you in on this stuff. So it's open\Nsource, it originated on Solaris, and been Dialogue: 0,0:01:27.26,0:01:31.64,Default,,0000,0000,0000,,developed currently on illumos which is a\Nfork from OpenSolaris. It has been ported Dialogue: 0,0:01:31.64,0:01:37.93,Default,,0000,0000,0000,,to FreeBSD, NetBSD, OS X, there's also a\Nport for Linux called next called DTrace Dialogue: 0,0:01:37.93,0:01:43.69,Default,,0000,0000,0000,,for Linux. I think it's done by a person\Ncalled Paul Fox. It's been ported to QNX Dialogue: 0,0:01:43.69,0:01:49.81,Default,,0000,0000,0000,,and the OpenBSD folks are currently doing\Nsome work to get the technology like Dialogue: 0,0:01:49.81,0:01:54.04,Default,,0000,0000,0000,,DTrace on their system. And I think\Nthere's a port for Windows? I don't know Dialogue: 0,0:01:54.04,0:01:57.87,Default,,0000,0000,0000,,if this is actually true, but it is it's\Nkind of cool because then that means it's Dialogue: 0,0:01:57.87,0:02:04.65,Default,,0000,0000,0000,,basically everywhere. So, most of you\Nwould probably know static tools like Dialogue: 0,0:02:04.65,0:02:09.47,Default,,0000,0000,0000,,strace. We have a very similar tool on\NFreeBSD that is called truss, and what Dialogue: 0,0:02:09.47,0:02:14.50,Default,,0000,0000,0000,,truss and strace are doing is - you can\Nattach them to a process and look at the Dialogue: 0,0:02:14.50,0:02:18.65,Default,,0000,0000,0000,,system calls that this process is\Nemitting. So in case something is going Dialogue: 0,0:02:18.65,0:02:23.32,Default,,0000,0000,0000,,wrong you can well look inside of the\Nprogram, which can be kind of useful when Dialogue: 0,0:02:23.32,0:02:28.87,Default,,0000,0000,0000,,you're trying to find a problem. It's\Nkind of handy but it's also pretty Dialogue: 0,0:02:28.87,0:02:32.89,Default,,0000,0000,0000,,limited. Because first of all it really\Nreally slows down the process that you're Dialogue: 0,0:02:32.89,0:02:37.25,Default,,0000,0000,0000,,currently looking at. So if you want to\Ndebug a performance issue, you're pretty Dialogue: 0,0:02:37.25,0:02:42.17,Default,,0000,0000,0000,,much out of luck there. And also it's kind\Nof like, narrow down - you can just look Dialogue: 0,0:02:42.17,0:02:47.94,Default,,0000,0000,0000,,at one process. Which is also like bad\Nthing because the system that we currently Dialogue: 0,0:02:47.94,0:02:52.66,Default,,0000,0000,0000,,have - all these systems are very\Ncomplex: we have a lot of layers. You have Dialogue: 0,0:02:52.66,0:02:56.30,Default,,0000,0000,0000,,virtual file systems, you have virtual\Nmemory, you have network, you have Dialogue: 0,0:02:56.30,0:03:00.50,Default,,0000,0000,0000,,databases, processes communicating with\Neach other. And in case you are using a Dialogue: 0,0:03:00.50,0:03:04.71,Default,,0000,0000,0000,,high-level programming language, you might\Nalso have a runtime system. So it's a Dialogue: 0,0:03:04.71,0:03:09.52,Default,,0000,0000,0000,,little operating system on top of your\Noperating system. So when something goes Dialogue: 0,0:03:09.52,0:03:15.00,Default,,0000,0000,0000,,wrong in a system that has such large\Ncomplexity, something happens that we call Dialogue: 0,0:03:15.00,0:03:19.85,Default,,0000,0000,0000,,the blame game. And the blame game - it's\Nnever your fault, it's always someone Dialogue: 0,0:03:19.85,0:03:25.71,Default,,0000,0000,0000,,else's. So what we want to be able to do\Nis we want to look at the system as a Dialogue: 0,0:03:25.71,0:03:30.35,Default,,0000,0000,0000,,whole, so we can correlate all the data\Nand come up with some meaningful answers Dialogue: 0,0:03:30.35,0:03:34.51,Default,,0000,0000,0000,,when something is really going wrong in\Nthere. And also, we don't want to Dialogue: 0,0:03:34.51,0:03:39.26,Default,,0000,0000,0000,,switch out all the processes for\Ndebug processes to make that happen, Dialogue: 0,0:03:39.26,0:03:44.97,Default,,0000,0000,0000,,because as these things are all -- every\Nproblem happens in production. It never Dialogue: 0,0:03:44.97,0:03:48.47,Default,,0000,0000,0000,,happens on the development box. So like,\Nswitching out all the processes - that's Dialogue: 0,0:03:48.47,0:03:55.03,Default,,0000,0000,0000,,totally out of the picture. So to do that\Nin an arbitrary way, to like, instrument Dialogue: 0,0:03:55.03,0:03:59.91,Default,,0000,0000,0000,,the system in an arbitrary way, we sort of\Nneed like a programming language. So, we Dialogue: 0,0:03:59.91,0:04:03.64,Default,,0000,0000,0000,,need to describe - when that happens,\Nplease submit data so I can see what's Dialogue: 0,0:04:03.64,0:04:09.49,Default,,0000,0000,0000,,going on. So this kind of implies a\Nprogramming language. And DTrace comes Dialogue: 0,0:04:09.49,0:04:13.67,Default,,0000,0000,0000,,with such a programming language - it's a\Nlittle bit reminiscent of awk cross with Dialogue: 0,0:04:13.67,0:04:18.80,Default,,0000,0000,0000,,C? It's pretty simple to learn - you can\Npick it up 20 up to pick it up in 20 Dialogue: 0,0:04:18.80,0:04:25.20,Default,,0000,0000,0000,,minutes and you can start churning out\Nyour first DTrace scripts. So like awk, if Dialogue: 0,0:04:25.20,0:04:30.56,Default,,0000,0000,0000,,you know awk, awk can be used to analyze\Nlarge bodies of text. Dtrace is pretty Dialogue: 0,0:04:30.56,0:04:34.75,Default,,0000,0000,0000,,much the same, but for system behavior -\Nso a little bit mind boggling, but Dialogue: 0,0:04:34.75,0:04:40.07,Default,,0000,0000,0000,,probably I can show you what I mean by\Nthat. And also, as a bonus we don't want Dialogue: 0,0:04:40.07,0:04:43.86,Default,,0000,0000,0000,,to slow down the system, so we want to be\Nable to do things like performance Dialogue: 0,0:04:43.86,0:04:52.30,Default,,0000,0000,0000,,debugging, performance tests like that. So\NI've prepared this little demo here, and. Dialogue: 0,0:04:52.30,0:04:58.78,Default,,0000,0000,0000,,So since we had some issues here probably\Nthis is not -- I have to play around a Dialogue: 0,0:04:58.78,0:05:04.25,Default,,0000,0000,0000,,little bit. So what I'm going to do is\NI'm going to look at a very very naive way Dialogue: 0,0:05:04.25,0:05:18.01,Default,,0000,0000,0000,,to -- excuse me for a second -- very naive\Nway to -- give me a second -- so very Dialogue: 0,0:05:18.01,0:05:21.96,Default,,0000,0000,0000,,naive way to authenticate a user. And\Nthere's a lot of stuff wrong with this Dialogue: 0,0:05:21.96,0:05:26.03,Default,,0000,0000,0000,,code, but like what we're going to do is\Nwe're going to take a user string as Dialogue: 0,0:05:26.03,0:05:32.74,Default,,0000,0000,0000,,input, and then we're going to just\Ncompare it to another, to a secret. So I Dialogue: 0,0:05:32.74,0:05:36.42,Default,,0000,0000,0000,,know, the the secret in here is like in\Nplain text I know this is a problem, but Dialogue: 0,0:05:36.42,0:05:41.64,Default,,0000,0000,0000,,this is a little bit artificial. But I\Njust want to get my point across. So from Dialogue: 0,0:05:41.64,0:05:47.16,Default,,0000,0000,0000,,an algorithmic perspective, this check\Nfunction is correct: so we take a string Dialogue: 0,0:05:47.16,0:05:52.45,Default,,0000,0000,0000,,we take another string and we compare\Nthem. So everything's fine and easy. So if Dialogue: 0,0:05:52.45,0:05:58.60,Default,,0000,0000,0000,,you look at the way string compare works\Nand what it does, it's essentially Dialogue: 0,0:05:58.60,0:06:04.45,Default,,0000,0000,0000,,taking these two strings and it's\Ncomparing every character bit by bit. So Dialogue: 0,0:06:04.45,0:06:10.73,Default,,0000,0000,0000,,when it finds the first pair of characters\Nthat do not match up, it's going to stop. Dialogue: 0,0:06:10.73,0:06:17.88,Default,,0000,0000,0000,,So we can we can conclude something about\Nfrom that - so if it takes very short if Dialogue: 0,0:06:17.88,0:06:23.40,Default,,0000,0000,0000,,if this function this check function takes\Na very short amount of time, then, what Dialogue: 0,0:06:23.40,0:06:29.13,Default,,0000,0000,0000,,will happen is it will terminate earlier.\NAnd if our password guess is better, it Dialogue: 0,0:06:29.13,0:06:34.48,Default,,0000,0000,0000,,will take well, it will take longer. And\Nif we can measure that we can basically Dialogue: 0,0:06:34.48,0:06:40.81,Default,,0000,0000,0000,,extract information from that running\Nalgorithm. So I wrote a little driver Dialogue: 0,0:06:40.81,0:06:47.45,Default,,0000,0000,0000,,program in Haskell that basically just\Niterates over an alphabet and just feeds Dialogue: 0,0:06:47.45,0:06:53.38,Default,,0000,0000,0000,,this one letter into that program,\NAnd I'm going to use DTrace to get some Dialogue: 0,0:06:53.38,0:06:59.02,Default,,0000,0000,0000,,timing information. So let me start the\Ndriver. So this is now just running in the Dialogue: 0,0:06:59.02,0:07:04.92,Default,,0000,0000,0000,,background. And you cannot see what I'm\Ntyping there, but don't worry - these Dialogue: 0,0:07:04.92,0:07:12.24,Default,,0000,0000,0000,,scripts will all be; I will push them on\Nmy github. So DTrace now produces this Dialogue: 0,0:07:12.24,0:07:17.24,Default,,0000,0000,0000,,nice little distribution. So if you if you\Nwere if you were able to see the entire Dialogue: 0,0:07:17.24,0:07:22.95,Default,,0000,0000,0000,,alphabet, you would see that everything\Nexcept "D" behaves differently. So if you Dialogue: 0,0:07:22.95,0:07:29.40,Default,,0000,0000,0000,,squint a little, what you see there is\NDTrace the D letter takes a couple of Dialogue: 0,0:07:29.40,0:07:32.95,Default,,0000,0000,0000,,nanoseconds longer. This is the precision\Nthat I'm measuring here - ten to minus Dialogue: 0,0:07:32.95,0:07:39.22,Default,,0000,0000,0000,,nine seconds. Like really precise. And D\Ntakes longer than everything else, so it's Dialogue: 0,0:07:39.22,0:07:43.93,Default,,0000,0000,0000,,a little bit cut off there, but trust me.\NI know it sound like Donald Trump I'm Dialogue: 0,0:07:43.93,0:07:52.76,Default,,0000,0000,0000,,saying that. So yeah, and from that let's\Njust enter a letter. And now the password Dialogue: 0,0:07:52.76,0:07:56.80,Default,,0000,0000,0000,,and now the script clears everything and\Nit's going to guess the next letter. So Dialogue: 0,0:07:56.80,0:08:02.02,Default,,0000,0000,0000,,sadly this is cut off, because you would\Nsee that this distribution radically Dialogue: 0,0:08:02.02,0:08:08.83,Default,,0000,0000,0000,,changed. It looks completely different,\Nand so we can play that game a little bit. Dialogue: 0,0:08:08.83,0:08:13.42,Default,,0000,0000,0000,,So let's just roll with that.\NAnd like every three seconds the script is Dialogue: 0,0:08:13.42,0:08:19.16,Default,,0000,0000,0000,,going to recompute looking at the new\Ndistribution. And you can probably see Dialogue: 0,0:08:19.16,0:08:26.85,Default,,0000,0000,0000,,where this is going. So here you can see,\Nokay, and now it just - it just takes Dialogue: 0,0:08:26.85,0:08:34.56,Default,,0000,0000,0000,,about like three seconds for me to guess\Nthe next letter. So, and this is not a Dialogue: 0,0:08:34.56,0:08:39.81,Default,,0000,0000,0000,,problem that is only of\Nsomething that happens when you do string Dialogue: 0,0:08:39.81,0:08:44.14,Default,,0000,0000,0000,,compares. This can happen with\Nbasically everything - so it's especially Dialogue: 0,0:08:44.14,0:08:48.03,Default,,0000,0000,0000,,in things like cryptographic stuff where\Nyou don't want to have some information Dialogue: 0,0:08:48.03,0:08:56.62,Default,,0000,0000,0000,,leaked out. So this is what we call a\Ntiming side channel attack. So I could Dialogue: 0,0:08:56.62,0:09:02.96,Default,,0000,0000,0000,,essentially use DTrace to analyze\Nthe real binary. So I didn't change the Dialogue: 0,0:09:02.96,0:09:07.04,Default,,0000,0000,0000,,binary - I didn't have some some debug\Ncode there. This is like the actual binary Dialogue: 0,0:09:07.04,0:09:12.50,Default,,0000,0000,0000,,that I would put into production. So\Nwhat's important about out that, is to Dialogue: 0,0:09:12.50,0:09:16.50,Default,,0000,0000,0000,,take the actual binary, is some of these\Nthese timing side channels might be Dialogue: 0,0:09:16.50,0:09:21.62,Default,,0000,0000,0000,,introduced by a compiler optimization. And\Nwhen you insert debug code into that code, Dialogue: 0,0:09:21.62,0:09:26.92,Default,,0000,0000,0000,,then it might actually go away. So, you\Nwant to look at the real code that you're Dialogue: 0,0:09:26.92,0:09:34.42,Default,,0000,0000,0000,,putting into production. Let me show you\Nthe script that I came up with to write Dialogue: 0,0:09:34.42,0:09:40.78,Default,,0000,0000,0000,,that. So there are three interesting\Nthings in this script. So and and don't Dialogue: 0,0:09:40.78,0:09:44.18,Default,,0000,0000,0000,,worry - this is the more\Ncomplicated example, I just want to like Dialogue: 0,0:09:44.18,0:09:48.84,Default,,0000,0000,0000,,inspire your ideas. Because the things\Nthat you can do with DTrace that's pretty Dialogue: 0,0:09:48.84,0:09:54.60,Default,,0000,0000,0000,,much - the sky's the limit. You can\Ncome up with the weirdest ideas, and so Dialogue: 0,0:09:54.60,0:09:59.42,Default,,0000,0000,0000,,this is more complicated example. I'm\Ngoing to show you simpler ones. So to Dialogue: 0,0:09:59.42,0:10:04.44,Default,,0000,0000,0000,,demonstrate how we got here. So there are\Nthree interesting things in this code. The Dialogue: 0,0:10:04.44,0:10:09.51,Default,,0000,0000,0000,,first one is something that we call a\Nprobe. So a probe is a point of Dialogue: 0,0:10:09.51,0:10:15.02,Default,,0000,0000,0000,,instrumentation in the system. So whenever\Na certain event happens in the system this Dialogue: 0,0:10:15.02,0:10:21.27,Default,,0000,0000,0000,,probe is going to fire. And in this case,\Nthe begin probe like marks the state Dialogue: 0,0:10:21.27,0:10:27.38,Default,,0000,0000,0000,,the moment when the script starts. So the\Nsecond interesting thing is this clause. Dialogue: 0,0:10:27.38,0:10:31.68,Default,,0000,0000,0000,,So this clause is basically what this\Nprobe is going to execute - what's going Dialogue: 0,0:10:31.68,0:10:37.78,Default,,0000,0000,0000,,to be executed once that probe fires. So\Nit's a little block of code. Dialogue: 0,0:10:37.78,0:10:42.37,Default,,0000,0000,0000,,And this probe is a little bit more\Ninteresting, because it tells us Dialogue: 0,0:10:42.37,0:10:48.27,Default,,0000,0000,0000,,something about the structure of how such\Na probe looks like. Because every Dialogue: 0,0:10:48.27,0:10:54.10,Default,,0000,0000,0000,,probe is uniquely identified by a four\Ntuple. So it's like four components that Dialogue: 0,0:10:54.10,0:10:59.08,Default,,0000,0000,0000,,uniquely identify a probe. And the first\None is called the first part of this Dialogue: 0,0:10:59.08,0:11:03.27,Default,,0000,0000,0000,,tuple is called the provider, and I'm\Ngoing to talk about providers in a couple Dialogue: 0,0:11:03.27,0:11:07.16,Default,,0000,0000,0000,,of seconds and what they are. The second\None is called the module. Third one is Dialogue: 0,0:11:07.16,0:11:13.45,Default,,0000,0000,0000,,called the function. And the last one is\Ncalled the name. So these four pieces of Dialogue: 0,0:11:13.45,0:11:21.08,Default,,0000,0000,0000,,data, like, they identify a probe\Nuniquely. So the third thing that is Dialogue: 0,0:11:21.08,0:11:25.44,Default,,0000,0000,0000,,interesting here is, sadly something that\NI don't have time to talk about today, Dialogue: 0,0:11:25.44,0:11:31.14,Default,,0000,0000,0000,,this is called an aggregation. And this\Nsingle line that you see here is Dialogue: 0,0:11:31.14,0:11:35.89,Default,,0000,0000,0000,,essentially responsible for accumulating\Nall this data to print out this Dialogue: 0,0:11:35.89,0:11:39.95,Default,,0000,0000,0000,,distribution stuff - to generate this\Ndistribution. So this is built Dialogue: 0,0:11:39.95,0:11:44.63,Default,,0000,0000,0000,,into DTrace. You don't have to do that\Nyourself. As it, when you look at this Dialogue: 0,0:11:44.63,0:11:50.19,Default,,0000,0000,0000,,script, it's like 42 lines of code.\NAnd I came up with the first prototype Dialogue: 0,0:11:50.19,0:11:55.28,Default,,0000,0000,0000,,after five minutes. So it's not a lot\Nof stuff to do to get something out of Dialogue: 0,0:11:55.28,0:12:00.36,Default,,0000,0000,0000,,that. So it's very useful to have things -\Nif you use DTrace you Dialogue: 0,0:12:00.36,0:12:05.06,Default,,0000,0000,0000,,will use this a lot for performance\Ndebugging so it's kind of neat that we Dialogue: 0,0:12:05.06,0:12:11.41,Default,,0000,0000,0000,,have that. So yeah, let's talk a little\Nbit about providers, and this will Dialogue: 0,0:12:11.41,0:12:18.30,Default,,0000,0000,0000,,probably also will be cut off. So I'm\Ngoing to cheat a little bit here - I'm Dialogue: 0,0:12:18.30,0:12:27.65,Default,,0000,0000,0000,,just going to double that. So let's talk\Nabout providers -- oh that's handy -- Dialogue: 0,0:12:27.65,0:12:32.34,Default,,0000,0000,0000,,so I got 27 providers here and the number\Nof providers vary from operating system to Dialogue: 0,0:12:32.34,0:12:38.34,Default,,0000,0000,0000,,operating system. But these are the\Nones that I can see right now. There are Dialogue: 0,0:12:38.34,0:12:44.50,Default,,0000,0000,0000,,other providers that can be come into\Nexistence when you demand them. So I have Dialogue: 0,0:12:44.50,0:12:49.37,Default,,0000,0000,0000,,these 27 providers, and we're going to\Nlook at the syscall provider and the FBT Dialogue: 0,0:12:49.37,0:12:55.13,Default,,0000,0000,0000,,provider first. So, every provider knows\Nhow to instrument a specific part of the Dialogue: 0,0:12:55.13,0:13:01.41,Default,,0000,0000,0000,,system. So the syscall provider knows how\Nto instrument the syscall table. That's not Dialogue: 0,0:13:01.41,0:13:08.70,Default,,0000,0000,0000,,very surprising. So if you can look at the\Nsyscall provider and here you can see Dialogue: 0,0:13:08.70,0:13:16.72,Default,,0000,0000,0000,,essentially every system call entry and\Nreturn that FreeBSD offers. So Dialogue: 0,0:13:16.72,0:13:20.12,Default,,0000,0000,0000,,here you can see this four tuple, like,\Nthe provider syscall, FreeBSD is the Dialogue: 0,0:13:20.12,0:13:28.19,Default,,0000,0000,0000,,module, and so on. So these are all the\Nsystem calls that I have in my system. And Dialogue: 0,0:13:28.19,0:13:32.91,Default,,0000,0000,0000,,the other provider that I want to look at\Nis the so called FBT provider, and that is Dialogue: 0,0:13:32.91,0:13:38.81,Default,,0000,0000,0000,,pretty astonishing. The FBT provider, FBT\Nstands for "function boundary tracer" and Dialogue: 0,0:13:38.81,0:13:45.16,Default,,0000,0000,0000,,what it allows us to do, it allows us to\Ntrace every single function in the kernel. Dialogue: 0,0:13:45.16,0:13:50.85,Default,,0000,0000,0000,,So I can look at the entire kernel at\Nfunctions, as they are being called. So to Dialogue: 0,0:13:50.85,0:13:57.66,Default,,0000,0000,0000,,illustrate that I wrote a little, very\Nsimple DTrace script and this is probably, Dialogue: 0,0:13:57.66,0:14:01.40,Default,,0000,0000,0000,,look at the upper half please, so this is\Nprobably one of the first DTrace scripts Dialogue: 0,0:14:01.40,0:14:05.53,Default,,0000,0000,0000,,that you will come up with, it's a\Nfairly simple example, so let's break it Dialogue: 0,0:14:05.53,0:14:09.68,Default,,0000,0000,0000,,down. So I'm going to instrument the mmap\Nsystem call. For those of you who do not Dialogue: 0,0:14:09.68,0:14:13.72,Default,,0000,0000,0000,,know what the mmap system call is, what\Nyou can do with it is you can so you can Dialogue: 0,0:14:13.72,0:14:20.97,Default,,0000,0000,0000,,take a file and map that into the address\Nspace of your process, so very dumbed down Dialogue: 0,0:14:20.97,0:14:27.45,Default,,0000,0000,0000,,version. So whenever we enter the mmap\Nsystem call we are going to set the Dialogue: 0,0:14:27.45,0:14:32.81,Default,,0000,0000,0000,,variable "follow" to one, and what this\N"self at" means: this is essentially a Dialogue: 0,0:14:32.81,0:14:37.97,Default,,0000,0000,0000,,thread local variable and we're going to\Nassociate that variable with the thread Dialogue: 0,0:14:37.97,0:14:45.23,Default,,0000,0000,0000,,that we're currently inspecting. Then I'm\Ngoing to do something pretty, that sounds Dialogue: 0,0:14:45.23,0:14:49.15,Default,,0000,0000,0000,,scary but I'm going to instrument the\Nentire kernel. Every function entry and Dialogue: 0,0:14:49.15,0:14:53.01,Default,,0000,0000,0000,,every function return, I'm going to\Ninstrument that and say "please emit data Dialogue: 0,0:14:53.01,0:14:57.19,Default,,0000,0000,0000,,when you do that". And this is what we\Ncall a predicate, so this is where the Dialogue: 0,0:14:57.19,0:15:02.01,Default,,0000,0000,0000,,awkiness of the DTrace programming\Nlanguage comes in. So this is a predicate Dialogue: 0,0:15:02.01,0:15:07.06,Default,,0000,0000,0000,,and whenever that evaluates to true\Nthen the probe is going to fire, so in Dialogue: 0,0:15:07.06,0:15:11.14,Default,,0000,0000,0000,,this case when we are in the thread that\Nwe're currently tracing we're going to Dialogue: 0,0:15:11.14,0:15:16.33,Default,,0000,0000,0000,,emit data. And this is just an empty\Nclause, we just want to know "hey we got Dialogue: 0,0:15:16.33,0:15:23.48,Default,,0000,0000,0000,,here". So when we exit the mmap\Nsystem call and the predicate is set we're Dialogue: 0,0:15:23.48,0:15:27.66,Default,,0000,0000,0000,,going to set the variable "follow" to\Nzero, because every uninitialized variable Dialogue: 0,0:15:27.66,0:15:33.86,Default,,0000,0000,0000,,in DTrace is set to zero, so this pretty\Nmuch amounts to deallocating that variable Dialogue: 0,0:15:33.86,0:15:41.28,Default,,0000,0000,0000,,and then we're going to exit cleanly. So\Nlet me run that. So it takes a couple of Dialogue: 0,0:15:41.28,0:15:48.48,Default,,0000,0000,0000,,seconds and boom. So you saw a little\Npause here, that was when the DTrace guard Dialogue: 0,0:15:48.48,0:15:55.01,Default,,0000,0000,0000,,reverted the driver, the kernel. So now\Nyou can see every function call that Dialogue: 0,0:15:55.01,0:15:59.48,Default,,0000,0000,0000,,happens inside the mmap system call. And\Nthis is a little bit hard on the eyes, so Dialogue: 0,0:15:59.48,0:16:08.38,Default,,0000,0000,0000,,let me pass this flag here and now you can\Nhave nice to read indentation. So Dialogue: 0,0:16:08.38,0:16:12.63,Default,,0000,0000,0000,,now you might say "I don't like that. You\Nare injecting code into the kernel. That Dialogue: 0,0:16:12.63,0:16:17.88,Default,,0000,0000,0000,,is, that sounds dangerous" and yeah, but\Nlet me show you something that I find Dialogue: 0,0:16:17.88,0:16:23.98,Default,,0000,0000,0000,,really interesting. So I'm not\Ngoing too much into depth here, but this Dialogue: 0,0:16:23.98,0:16:28.75,Default,,0000,0000,0000,,is a byte code, so every DTrace script\Ngets compiled to bytecode and this Dialogue: 0,0:16:28.75,0:16:34.50,Default,,0000,0000,0000,,bytecode gets sent to the kernel and in\Nthe kernel you have a virtual machine that Dialogue: 0,0:16:34.50,0:16:39.06,Default,,0000,0000,0000,,interprets that bytecode. So in case you\Nwrite a script that for some reason might Dialogue: 0,0:16:39.06,0:16:44.55,Default,,0000,0000,0000,,go rogue on your kernel, it like allocates\Ntoo much memory, takes too much time, this Dialogue: 0,0:16:44.55,0:16:49.28,Default,,0000,0000,0000,,virtual machine can just say "okay, stop\Nit" and just going to revert all the Dialogue: 0,0:16:49.28,0:16:53.89,Default,,0000,0000,0000,,changes that happened to your kernel, and\Nthat's kinda handy. And it's not a new Dialogue: 0,0:16:53.89,0:17:01.20,Default,,0000,0000,0000,,idea, so if you're using TCP dump it's\Nbasically the same approach. They also Dialogue: 0,0:17:01.20,0:17:04.83,Default,,0000,0000,0000,,have this kind of bytecode, so that's just\Na little excursion here. This is called Dialogue: 0,0:17:04.83,0:17:13.25,Default,,0000,0000,0000,,BPF, Berkeley Packet Filter, so it's not\Nan entirely new idea. So everything I Dialogue: 0,0:17:13.25,0:17:19.47,Default,,0000,0000,0000,,showed you until now was "hey, I can look\Nwhen function calls happen". that's not Dialogue: 0,0:17:19.47,0:17:22.52,Default,,0000,0000,0000,,very much information, so we're going to\Nincrease the amount of information that we Dialogue: 0,0:17:22.52,0:17:35.08,Default,,0000,0000,0000,,get out of the system with every example.\NSo let me look at the actual kernel. So I Dialogue: 0,0:17:35.08,0:17:39.98,Default,,0000,0000,0000,,had to restart my machine, so my setup is\Nbasically gone now. So let's look at this Dialogue: 0,0:17:39.98,0:17:45.31,Default,,0000,0000,0000,,VM fault function. So this is, this is the\Nsource code of the operating system that Dialogue: 0,0:17:45.31,0:17:52.90,Default,,0000,0000,0000,,I'm running right now. This is FreeBSD\Ncurrent 12 and the VM fault function; Dialogue: 0,0:17:52.90,0:17:57.54,Default,,0000,0000,0000,,remember the mmap system call that I told\Nyou? So the mmap system call Dialogue: 0,0:17:57.54,0:18:03.90,Default,,0000,0000,0000,,I told you can bring, like map a file\Ninto your address space. And it doesn't Dialogue: 0,0:18:03.90,0:18:10.32,Default,,0000,0000,0000,,necessarily have to load the entire file,\Nso whenever we are touching a page in the Dialogue: 0,0:18:10.32,0:18:15.78,Default,,0000,0000,0000,,system, like a memory page, this machine\Nis four kilobytes and it's no super pages Dialogue: 0,0:18:15.78,0:18:21.43,Default,,0000,0000,0000,,here, so whenever it touches a piece of\Nmemory that you didn't bring into memory Dialogue: 0,0:18:21.43,0:18:25.31,Default,,0000,0000,0000,,yet, we're generating something that's\Ncalled a page fault, and then this Dialogue: 0,0:18:25.31,0:18:31.18,Default,,0000,0000,0000,,function gets called. So here let's look\Nat the arguments, and I'm going to skip Dialogue: 0,0:18:31.18,0:18:36.99,Default,,0000,0000,0000,,the zeroeth argument, to look at the first\Nargument. So this is the address that Dialogue: 0,0:18:36.99,0:18:44.16,Default,,0000,0000,0000,,provoked that page fault, this is the\Ntype and these are the flags and I'm going Dialogue: 0,0:18:44.16,0:18:48.78,Default,,0000,0000,0000,,to show you something to make that a\Nlittle bit more readable. So what about Dialogue: 0,0:18:48.78,0:18:58.96,Default,,0000,0000,0000,,this one? So you see it's a pointer and\Nthis is a big structure, so we want Dialogue: 0,0:18:58.96,0:19:09.96,Default,,0000,0000,0000,,to be able to look at that structure. And\Njust probably should do this here, so Dialogue: 0,0:19:09.96,0:19:17.09,Default,,0000,0000,0000,,let's look at this VM fault script here.\NSo this is, make this a little bit more, Dialogue: 0,0:19:17.09,0:19:20.95,Default,,0000,0000,0000,,so this is, don't pay too much attention\Nto this code, this this is basically just Dialogue: 0,0:19:20.95,0:19:26.05,Default,,0000,0000,0000,,boilerplate to make make stuff readable\Nand this is where the actual action is Dialogue: 0,0:19:26.05,0:19:31.69,Default,,0000,0000,0000,,happening. So this is, so what I'm doing\Nthere is I'm instrumenting the VM Dialogue: 0,0:19:31.69,0:19:36.35,Default,,0000,0000,0000,,fault function and whenever we enter it\Nthen we're going to use some information Dialogue: 0,0:19:36.35,0:19:40.72,Default,,0000,0000,0000,,that DTrace gives us for free. So this is\Nexecname, this is the name of the Dialogue: 0,0:19:40.72,0:19:45.91,Default,,0000,0000,0000,,currently running executable that provoked\Nthe page fault, this is the process ID and Dialogue: 0,0:19:45.91,0:19:53.25,Default,,0000,0000,0000,,here we have a bunch of argument\Nvariables. So these arg1, arg2, arg3, Dialogue: 0,0:19:53.25,0:19:57.96,Default,,0000,0000,0000,,that are essentially just integers, so\Nnothing too fancy there. But we wanna Dialogue: 0,0:19:57.96,0:20:02.38,Default,,0000,0000,0000,,look, wanna be able to look at that\Nstruct. And here I'm going to use this Dialogue: 0,0:20:02.38,0:20:08.14,Default,,0000,0000,0000,,args array, and this args array\Nis kind of special, because it has typing Dialogue: 0,0:20:08.14,0:20:15.87,Default,,0000,0000,0000,,information about the arguments. So when\Nyou run that, so you're referencing that Dialogue: 0,0:20:15.87,0:20:26.57,Default,,0000,0000,0000,,pointer there with the star, excuse me,\Nand let's just run that and maybe, that's Dialogue: 0,0:20:26.57,0:20:32.90,Default,,0000,0000,0000,,a start yeah. So this is an in-kernel\Ndata structure that we can now look Dialogue: 0,0:20:32.90,0:20:40.01,Default,,0000,0000,0000,,at. So DTrace enabled us to look at in-\Nmemory data structures as the system runs. Dialogue: 0,0:20:40.01,0:20:44.33,Default,,0000,0000,0000,,And this is really really powerful.\NIn in the DTrace script I could use all Dialogue: 0,0:20:44.33,0:20:50.49,Default,,0000,0000,0000,,these fields like I can manipulate this\Nargs array, this value in there, just like Dialogue: 0,0:20:50.49,0:20:57.01,Default,,0000,0000,0000,,just like every other variable; I\Ncan pretty much work like I was in C. So Dialogue: 0,0:20:57.01,0:21:02.66,Default,,0000,0000,0000,,how is it doing that? There is something\Nthat's called CTF, that's not capture the Dialogue: 0,0:21:02.66,0:21:10.12,Default,,0000,0000,0000,,flag, it's, this is the, the Compact C\NTracing Format, so you can see that but Dialogue: 0,0:21:10.12,0:21:14.32,Default,,0000,0000,0000,,there is a man page in FreeBSD, and\Nthere's a little segment in the kernel Dialogue: 0,0:21:14.32,0:21:19.19,Default,,0000,0000,0000,,binary, where all this typing information\Nis stored. I don't know how that compares Dialogue: 0,0:21:19.19,0:21:24.32,Default,,0000,0000,0000,,to modern DWARF but yeah this is what\NDTrace is working with. So now you might Dialogue: 0,0:21:24.32,0:21:28.55,Default,,0000,0000,0000,,ask yourself "Why on earth would I do\Nthat? Why on earth would I look at virtual Dialogue: 0,0:21:28.55,0:21:33.59,Default,,0000,0000,0000,,memory, because, yeah, um, this stuff is\Nsafe isn't it? I mean there's no bugs in Dialogue: 0,0:21:33.59,0:21:42.82,Default,,0000,0000,0000,,there." Except when they are. Anyone\Nremembers remembers "Dirty COW"? So this Dialogue: 0,0:21:42.82,0:21:48.51,Default,,0000,0000,0000,,was a very nasty vulnerability in the\NLinux kernel and that that was a problem Dialogue: 0,0:21:48.51,0:21:52.40,Default,,0000,0000,0000,,in the virtual memory management. So it\Nallowed you to write to a file that you Dialogue: 0,0:21:52.40,0:21:56.68,Default,,0000,0000,0000,,didn't own as a regular user. So you could\Nessentially just write to a binary that Dialogue: 0,0:21:56.68,0:22:01.79,Default,,0000,0000,0000,,had "set UID" set. Very unpleasant, but\NI'm not going to bash the Linux folks Dialogue: 0,0:22:01.79,0:22:08.03,Default,,0000,0000,0000,,here, this is just, I just want to show\Nyou these things are hard. And the first Dialogue: 0,0:22:08.03,0:22:15.44,Default,,0000,0000,0000,,fix for this problem was in 2005 and then\Nit came back in 2016. So now that's fixed Dialogue: 0,0:22:15.44,0:22:21.08,Default,,0000,0000,0000,,and then it came back with "Huge Dirty\NCOW" in 2017, so this is, I mean this Dialogue: 0,0:22:21.08,0:22:27.58,Default,,0000,0000,0000,,was there for way over a decade.\NThese things are hard to debug. And this Dialogue: 0,0:22:27.58,0:22:33.11,Default,,0000,0000,0000,,is what I like about these systems, so not\Nhaving, not having tools like DTrace to Dialogue: 0,0:22:33.11,0:22:37.64,Default,,0000,0000,0000,,figure out what's going on inside of the\Nsystem somehow, to me, amounts to security Dialogue: 0,0:22:37.64,0:22:42.36,Default,,0000,0000,0000,,by obscurity. And I've heard that some\Npeople who are developing exploits for Dialogue: 0,0:22:42.36,0:22:46.10,Default,,0000,0000,0000,,systems that have DTrace they say "Oh, I\Nreally like developing exploits on these Dialogue: 0,0:22:46.10,0:22:53.23,Default,,0000,0000,0000,,systems, because the tooling is so great!"\NYeah, but, to be honest this is cool, Dialogue: 0,0:22:53.23,0:22:58.90,Default,,0000,0000,0000,,because an exploit is a proof of concept\Nand coming up with these exploits quickly Dialogue: 0,0:22:58.90,0:23:03.44,Default,,0000,0000,0000,,is very usable, because you know what's\Ngoing on you can show "Hey, this is going Dialogue: 0,0:23:03.44,0:23:07.28,Default,,0000,0000,0000,,wrong". I had situations, where\Npeople were telling me "Oh, this is this Dialogue: 0,0:23:07.28,0:23:11.02,Default,,0000,0000,0000,,is not a problem with our program, this is\Nthis weird operating system that you're Dialogue: 0,0:23:11.02,0:23:18.10,Default,,0000,0000,0000,,using. Like Solaris, weird operating\Nsystem." And, yeah, and then I churned out Dialogue: 0,0:23:18.10,0:23:22.06,Default,,0000,0000,0000,,some DTrace scripts and "No, it's\Nactually your problem". "Oh, now I can see Dialogue: 0,0:23:22.06,0:23:31.42,Default,,0000,0000,0000,,that on my Linux box!" Magic. So,\Neverything I showed you until now was Dialogue: 0,0:23:31.42,0:23:38.18,Default,,0000,0000,0000,,very, very much related to function calls\Nand we want to have a little bit more Dialogue: 0,0:23:38.18,0:23:44.72,Default,,0000,0000,0000,,semantics here, because you might want to\Nwrite a script that inspects protocols, Dialogue: 0,0:23:44.72,0:23:48.76,Default,,0000,0000,0000,,stuff like TCP, UDP stuff like that. So,\Nyou don't want to know which function Dialogue: 0,0:23:48.76,0:23:54.32,Default,,0000,0000,0000,,inside of the kernel is responsible for\Nhandling your TCP/IP stuff, so DTrace Dialogue: 0,0:23:54.32,0:24:00.55,Default,,0000,0000,0000,,comes with something that's called static\Nproviders and I'm just going to show the Dialogue: 0,0:24:00.55,0:24:04.77,Default,,0000,0000,0000,,apropos here. So these are, so every\Nstatic provider has a main page which is Dialogue: 0,0:24:04.77,0:24:10.95,Default,,0000,0000,0000,,kind of handy - documentation whoo - and\Nyou can see there is an I/O provider if Dialogue: 0,0:24:10.95,0:24:17.54,Default,,0000,0000,0000,,you are interested in looking at this guy:\NOh, IP for looking at IPv4 and IPv6, Dialogue: 0,0:24:17.54,0:24:23.57,Default,,0000,0000,0000,,TCP... This one is pretty cool, it's about\Nscheduling behavior. So, "what does my Dialogue: 0,0:24:23.57,0:24:29.01,Default,,0000,0000,0000,,scheduler do?" And if you look at that, you\Ncan see some interesting stuff like length Dialogue: 0,0:24:29.01,0:24:33.15,Default,,0000,0000,0000,,priority if you ever saw things like\Npriority inversion, stuff like that, now Dialogue: 0,0:24:33.15,0:24:36.97,Default,,0000,0000,0000,,you can see that happen. I'm a nerd, I\Nfind this interesting for some reason, I Dialogue: 0,0:24:36.97,0:24:43.23,Default,,0000,0000,0000,,don't know. And it's also pretty\Ninteresting to figure out what's going on, Dialogue: 0,0:24:43.23,0:24:48.28,Default,,0000,0000,0000,,"why is this getting de-scheduled all the\Ntime?" So, some interesting things going Dialogue: 0,0:24:48.28,0:24:55.81,Default,,0000,0000,0000,,on there. So, I'm running a little bit\Nshort on time here, but I just quickly Dialogue: 0,0:24:55.81,0:24:59.34,Default,,0000,0000,0000,,want to show you something - this is all\Nkernel stuff right now - can we do that Dialogue: 0,0:24:59.34,0:25:05.38,Default,,0000,0000,0000,,with userspace? Of course. So, there was\None provider that didn't show up when I Dialogue: 0,0:25:05.38,0:25:09.59,Default,,0000,0000,0000,,had my provider listing, but was in the\NDTrace script where I did this timing Dialogue: 0,0:25:09.59,0:25:16.23,Default,,0000,0000,0000,,attack stuff. And that's called the PID\Nprovider. And the PID provider generates Dialogue: 0,0:25:16.23,0:25:21.08,Default,,0000,0000,0000,,probes on demand, because a process might\Nhave a lot of probes and you will shortly Dialogue: 0,0:25:21.08,0:25:25.19,Default,,0000,0000,0000,,see why and this is why I'm going to use a\Nvery small program which is called "true", Dialogue: 0,0:25:25.19,0:25:31.56,Default,,0000,0000,0000,,and true just exits with exit code zero.\NSo, nothing too exciting going on here, Dialogue: 0,0:25:31.56,0:25:37.81,Default,,0000,0000,0000,,and this dollar target gets substituted\Nin, we get the process ID there. And this Dialogue: 0,0:25:37.81,0:25:44.64,Default,,0000,0000,0000,,is everything that happens when I'm\Nexecuting this program you see this is a Dialogue: 0,0:25:44.64,0:25:48.68,Default,,0000,0000,0000,,little bit more fine-grained than the FBT\Nprovider, because now we can trace every Dialogue: 0,0:25:48.68,0:25:53.52,Default,,0000,0000,0000,,single instruction inside of that\Nfunction, which is kind of a handy. It's a Dialogue: 0,0:25:53.52,0:25:58.09,Default,,0000,0000,0000,,scriptable debugger. So, these numbers are\Nthe instructional offsets inside of that Dialogue: 0,0:25:58.09,0:26:03.36,Default,,0000,0000,0000,,function. We can also look at - so this is\Neverything in the true segment - we can Dialogue: 0,0:26:03.36,0:26:09.90,Default,,0000,0000,0000,,also look at libraries that got linked in\Nand there's a lot of stuff happening in Dialogue: 0,0:26:09.90,0:26:15.78,Default,,0000,0000,0000,,libc for example when you run true.\NSo, one last thing that I wanted to show Dialogue: 0,0:26:15.78,0:26:22.34,Default,,0000,0000,0000,,you because it consumed a week of my life:\NI'm using a lot of Haskell and the Mac OS Dialogue: 0,0:26:22.34,0:26:29.42,Default,,0000,0000,0000,,people, they also have DTrace and they\Nhave GHC Haskell DTrace support - so the Dialogue: 0,0:26:29.42,0:26:38.38,Default,,0000,0000,0000,,Glasgow Haskell compiler - and glorious...\Nthey have probes to analyze what's going Dialogue: 0,0:26:38.38,0:26:41.62,Default,,0000,0000,0000,,on inside of the runtime system. So, I\Nthought "I want to have that, I have Dialogue: 0,0:26:41.62,0:26:47.02,Default,,0000,0000,0000,,DTrace, why doesn't it work on FreeBSD?"\NSo, after a week of fighting with make Dialogue: 0,0:26:47.02,0:26:55.10,Default,,0000,0000,0000,,files and linkers, that works: If you\Ncheck out the recent GHC repository and Dialogue: 0,0:26:55.10,0:27:00.26,Default,,0000,0000,0000,,build it on FreeBSD, you get all the nice\Nstuff that I'm going to show you now. So, Dialogue: 0,0:27:00.26,0:27:05.91,Default,,0000,0000,0000,,this is a very boring program - it just\Nstarts 32 green threads and schedules them Dialogue: 0,0:27:05.91,0:27:10.47,Default,,0000,0000,0000,,all over the place - and now I can do\Nsomething like this: {\i1}phone rings{\i0} I can Dialogue: 0,0:27:10.47,0:27:13.93,Default,,0000,0000,0000,,ring a telephone.\N{\i1}laughter{\i0} Dialogue: 0,0:27:13.93,0:27:18.75,Default,,0000,0000,0000,,No, that would be\Ninteresting... So, you can also use Dialogue: 0,0:27:18.75,0:27:26.97,Default,,0000,0000,0000,,wildcards - and not as name of the probe -\Nand this is what's going on inside, like Dialogue: 0,0:27:26.97,0:27:31.58,Default,,0000,0000,0000,,GC garbage collection and all this stuff.\NNow you can look at this and write useful Dialogue: 0,0:27:31.58,0:27:37.51,Default,,0000,0000,0000,,DTrace scripts that also take my runtime\Nsystem into account. So, stuff like that Dialogue: 0,0:27:37.51,0:27:41.81,Default,,0000,0000,0000,,exists for I think Python - I'm not\Nentirely sure because I don't use it - Dialogue: 0,0:27:41.81,0:27:49.12,Default,,0000,0000,0000,,nodejs same, Postgres - I used it but not\Nwith DTrace right now - and what a find Dialogue: 0,0:27:49.12,0:27:55.21,Default,,0000,0000,0000,,interesting: Firefox. When you run\NJavaScript in your Firefox, it actually Dialogue: 0,0:27:55.21,0:27:59.36,Default,,0000,0000,0000,,has a provider, so you can trace\NJavaScript running in your browser with Dialogue: 0,0:27:59.36,0:28:05.13,Default,,0000,0000,0000,,DTrace, so after everything I just showed\Nyou, there might be some stuff going on Dialogue: 0,0:28:05.13,0:28:10.70,Default,,0000,0000,0000,,there. So yeah, this is basically\Neverything I wanted to show you and I Dialogue: 0,0:28:10.70,0:28:13.76,Default,,0000,0000,0000,,think I'm going to wrap out, because\Notherwise we're not going to have a lot of Dialogue: 0,0:28:13.76,0:28:19.00,Default,,0000,0000,0000,,time for questions and maybe you have\Nsome. So yeah, thanks. Dialogue: 0,0:28:19.00,0:28:29.61,Default,,0000,0000,0000,,{\i1}applause{\i0}\NHerald: Thank you very much Raichoo. We Dialogue: 0,0:28:29.61,0:28:34.26,Default,,0000,0000,0000,,are actually over time already, but we\Nhave two more minutes because we started Dialogue: 0,0:28:34.26,0:28:38.82,Default,,0000,0000,0000,,three minutes late, so if there are any\Nreally quick questions, possibly from the Dialogue: 0,0:28:38.82,0:28:43.03,Default,,0000,0000,0000,,internet... There is one, the signal angel\Nsays, let's hear it. Dialogue: 0,0:28:43.03,0:28:48.01,Default,,0000,0000,0000,,Question: Yeah, hi, okay. So, the question\Nis, "which changes are actually necessary Dialogue: 0,0:28:48.01,0:28:51.81,Default,,0000,0000,0000,,to do in the kernel of an operating system\Nto support DTrace?" Dialogue: 0,0:28:51.81,0:28:56.37,Default,,0000,0000,0000,,Answer: That's a lot of work. So, it's not\Nsomething like you do in a weekend. This Dialogue: 0,0:28:56.37,0:29:03.06,Default,,0000,0000,0000,,is... So, the person who started the work\Non FreeBSD has sadly passed away now, but Dialogue: 0,0:29:03.06,0:29:09.56,Default,,0000,0000,0000,,I think they took a couple of years to\Nhave everything in place, so you have to Dialogue: 0,0:29:09.56,0:29:13.73,Default,,0000,0000,0000,,have stuff like the CTF thing that I\Nshowed you, which is what OpenBSD is Dialogue: 0,0:29:13.73,0:29:19.89,Default,,0000,0000,0000,,currently working on. And then you need\Nall those those magic gizmos, like kernel Dialogue: 0,0:29:19.89,0:29:25.66,Default,,0000,0000,0000,,modules and stuff like that. So, it takes\Na lot of time, but it's been ported to Dialogue: 0,0:29:25.66,0:29:30.89,Default,,0000,0000,0000,,most operating systems that are available\Nand in use right now. So yeah, hope this Dialogue: 0,0:29:30.89,0:29:34.24,Default,,0000,0000,0000,,answers the question.\NHerald: Excellent and there are no more Dialogue: 0,0:29:34.24,0:29:38.84,Default,,0000,0000,0000,,questions here in the room. I will thank\NRaichoo and you can find him outside of Dialogue: 0,0:29:38.84,0:29:46.59,Default,,0000,0000,0000,,the room and also on Twitter at "raichoo"\Nif you have any more further question. Dialogue: 0,0:29:46.59,0:29:51.40,Default,,0000,0000,0000,,{\i1}postroll music{\i0} Dialogue: 0,0:29:51.40,0:30:08.00,Default,,0000,0000,0000,,subtitles created by c3subtitles.de\Nin the year 2020. Join, and help us!