WEBVTT 00:00:16.640 --> 00:00:20.420 PETE HODGSON: OK. So, thanks for coming. 00:00:20.420 --> 00:00:25.800 Today I'm talking about Rails as an SOA client. 00:00:25.800 --> 00:00:30.020 So, in the beginning, there was a Rails application. 00:00:30.020 --> 00:00:32.710 And in the beginning, most of these Rails applications 00:00:32.710 --> 00:00:33.710 were pretty straight-forward. 00:00:33.710 --> 00:00:36.760 Pretty simple things. They were normally 00:00:36.760 --> 00:00:39.489 just talking to a single database. 00:00:39.489 --> 00:00:42.620 And this is how most of our Rails applications start. 00:00:42.620 --> 00:00:46.320 They're green fields, and we're talking to 00:00:46.320 --> 00:00:47.920 a single database. 00:00:47.920 --> 00:00:51.440 Quite often, we, or pretty soon, particularly now days, 00:00:51.440 --> 00:00:54.989 we start talking to services. These might be external 00:00:54.989 --> 00:00:59.109 services like Twillo or Twitter, or they might be 00:00:59.109 --> 00:01:01.129 internal services that we're using as part of doing 00:01:01.129 --> 00:01:04.769 our job in a, in a more enterprise-y situation. 00:01:04.769 --> 00:01:09.509 And as time has gone on, we've noticed, or 00:01:09.509 --> 00:01:12.039 I've noticed, that our Rails applications are depending more 00:01:12.039 --> 00:01:14.740 and more on services. So, I think there's kind 00:01:14.740 --> 00:01:18.030 of two big forces in play here. Force number 00:01:18.030 --> 00:01:21.840 one is, as time has grown and as Rails 00:01:21.840 --> 00:01:25.560 has matured and our community's matured, our Rails applications 00:01:25.560 --> 00:01:27.380 have been kind of growing into these big monoliths, 00:01:27.380 --> 00:01:31.310 mono Rails, these big, bloated applications, and now there's 00:01:31.310 --> 00:01:34.430 this kind of movement, quite a widespread movement, meant 00:01:34.430 --> 00:01:37.659 to break up our large, monolithic Rails apps into, 00:01:37.659 --> 00:01:40.369 into services and, obviously, something needs to talk to 00:01:40.369 --> 00:01:42.140 those services. And normally that is a Rails app 00:01:42.140 --> 00:01:44.060 fronting those services. 00:01:44.060 --> 00:01:47.909 The second big force is, Rails applications, which I 00:01:47.909 --> 00:01:49.920 think started off mainly being used by start ups, 00:01:49.920 --> 00:01:52.930 have moved more and more into enterprise-y places where 00:01:52.930 --> 00:01:56.969 there are a lot, lot, lot of services. And 00:01:56.969 --> 00:02:00.360 in these enterprise contexts, Rails apps are normally talking 00:02:00.360 --> 00:02:01.530 to a lot of services to get their job 00:02:01.530 --> 00:02:02.340 done. 00:02:02.340 --> 00:02:05.590 And this is, even goes to the extreme where 00:02:05.590 --> 00:02:07.030 we have Rails app that don't actually have any 00:02:07.030 --> 00:02:09.310 local data storage at all, are not doing any 00:02:09.310 --> 00:02:12.980 kind of persistence. All they're doing is, is interacting 00:02:12.980 --> 00:02:15.200 with services to do all of their work. 00:02:15.200 --> 00:02:19.310 So, I, I built an application like this fairly 00:02:19.310 --> 00:02:24.810 recently, and I'm gonna talk to you guys about 00:02:24.810 --> 00:02:27.120 what we built, some of the techniques we used 00:02:27.120 --> 00:02:29.610 to be successful in building these kind of services 00:02:29.610 --> 00:02:34.530 only application. Yup. So that's, that's gonna be the 00:02:34.530 --> 00:02:35.800 talk today. 00:02:35.800 --> 00:02:38.450 My name is Pete. I work for this consulting 00:02:38.450 --> 00:02:41.840 company called ThoughtWorks. So, we work with clients to 00:02:41.840 --> 00:02:44.950 help them build software and get better at building 00:02:44.950 --> 00:02:48.700 software. I've been with ThoughtWorks for about four years. 00:02:48.700 --> 00:02:50.069 As you can tell from my accent, I'm in 00:02:50.069 --> 00:02:53.290 the San Francisco office. 00:02:53.290 --> 00:02:55.250 I really am in the San Francisco office. And 00:02:55.250 --> 00:02:57.959 I, I, in my time in ThoughtWorks, I've done 00:02:57.959 --> 00:02:59.870 a lot of different things. I've done a fair 00:02:59.870 --> 00:03:01.989 amount of Ruby and Rails, but I've also done 00:03:01.989 --> 00:03:05.480 some Scala, some JavaScript, some, some iOS. And I've 00:03:05.480 --> 00:03:08.040 also worked in a lot of different contexts, organizations. 00:03:08.040 --> 00:03:10.569 So I've worked in really small start ups all 00:03:10.569 --> 00:03:13.349 the way through to huge, lumbering banks. 00:03:13.349 --> 00:03:15.629 And, one of the things that I love about 00:03:15.629 --> 00:03:17.349 ThoughtWorks is I get to move in all these 00:03:17.349 --> 00:03:19.150 different places. And one of the things that I 00:03:19.150 --> 00:03:23.319 think ThoughtWorks brings to its clients is ideas from 00:03:23.319 --> 00:03:26.790 one place applied in a different place. So, what 00:03:26.790 --> 00:03:28.489 we find ourselves doing a lot is taking ideas 00:03:28.489 --> 00:03:30.590 from the Rails community and introducing them to the 00:03:30.590 --> 00:03:33.709 Scala community. Or taking ideas from mobile applications and 00:03:33.709 --> 00:03:36.750 introducing them to our client-side JavaScript applications. 00:03:36.750 --> 00:03:39.510 And, indeed, taking ideas from start ups and taking 00:03:39.510 --> 00:03:42.980 them to the enterprise. And so, shockingly enough, vice 00:03:42.980 --> 00:03:44.599 versa. So, that's part of what I'm gonna be 00:03:44.599 --> 00:03:47.010 talking about today is, there's this subtext of kind 00:03:47.010 --> 00:03:48.849 of taking ideas from other communities that are successful 00:03:48.849 --> 00:03:53.860 and bringing them into the Rails community. 00:03:53.860 --> 00:03:55.209 And so this is gonna be a kind of 00:03:55.209 --> 00:03:56.760 a talk in two parts. The first part I'm 00:03:56.760 --> 00:03:59.209 gonna talk about some kind of more hand wave-y, 00:03:59.209 --> 00:04:02.860 how do we live in this ecosystem environment. And 00:04:02.860 --> 00:04:04.510 some tools and techniques that can help with that, 00:04:04.510 --> 00:04:06.959 particularly in the context of Rails. And then I'm 00:04:06.959 --> 00:04:09.599 gonna dive into some more kind of nitty gritty 00:04:09.599 --> 00:04:10.810 kind of, how do we actually build these things 00:04:10.810 --> 00:04:12.450 into our Rails applications? 00:04:12.450 --> 00:04:18.238 So, first off. Co-dependence versus independence. So, as I 00:04:18.238 --> 00:04:20.789 said, I worked for a client. We were building 00:04:20.789 --> 00:04:23.370 a large, we were building, actually, a pretty small 00:04:23.370 --> 00:04:26.039 Rails application in front of a very large set 00:04:26.039 --> 00:04:30.050 of services. We were building an online store for, 00:04:30.050 --> 00:04:32.580 for a large book retailer. They were going to 00:04:32.580 --> 00:04:34.099 a new market, and they didn't want to have 00:04:34.099 --> 00:04:37.700 to keep working in Java. So they were taking 00:04:37.700 --> 00:04:40.599 this as an opportunity to build a green-filled Rails 00:04:40.599 --> 00:04:43.719 app for a new market, which was still gonna 00:04:43.719 --> 00:04:46.130 be using all of their existing services, which were 00:04:46.130 --> 00:04:50.800 implemented in a variety of languages. 00:04:50.800 --> 00:04:52.320 So this is what the home page of this 00:04:52.320 --> 00:04:54.779 application kind of looked like. There was this kind 00:04:54.779 --> 00:04:57.719 of list of all the products and prices and 00:04:57.719 --> 00:04:59.550 descriptions and all that kind of stuff. And then 00:04:59.550 --> 00:05:01.089 across the top we had these kind of deals 00:05:01.089 --> 00:05:03.580 of the day. 00:05:03.580 --> 00:05:06.779 And the way that this worked was, was interesting 00:05:06.779 --> 00:05:09.409 or, I, I suppose is, is interesting if you 00:05:09.409 --> 00:05:11.479 haven't worked in like a large enterprise like this. 00:05:11.479 --> 00:05:14.570 But the information to power this page was coming 00:05:14.570 --> 00:05:17.229 from a lot of different places. And our Rails 00:05:17.229 --> 00:05:19.450 app was really just responsible for stitching that information 00:05:19.450 --> 00:05:21.909 together. So, we had the product service that was 00:05:21.909 --> 00:05:24.330 kind of serving up product catalog data, and then 00:05:24.330 --> 00:05:26.800 we had the deals service that was kind of 00:05:26.800 --> 00:05:28.050 telling us what the deals of the day were. 00:05:28.050 --> 00:05:29.349 And, as I said, the Rails app was just 00:05:29.349 --> 00:05:30.880 kind of there to kind of, go over here 00:05:30.880 --> 00:05:32.529 and get some information. Go over here and get 00:05:32.529 --> 00:05:33.820 some information, and then kind of plug it all 00:05:33.820 --> 00:05:38.169 together. So when you're building an app like this, 00:05:38.169 --> 00:05:42.839 you get this interesting phenomenon, where this app, on 00:05:42.839 --> 00:05:45.490 its own, really can't do much at all. In 00:05:45.490 --> 00:05:47.620 fact, it can't do anything. We, there wasn't, there 00:05:47.620 --> 00:05:51.000 was probably not a single page of this app 00:05:51.000 --> 00:05:53.839 that would render if those services weren't there. 00:05:53.839 --> 00:05:57.409 So, what, this is what I mean by co-dependence. 00:05:57.409 --> 00:06:01.310 This application was incredibly co-dependent on the ecosystem of 00:06:01.310 --> 00:06:05.320 services that it lived inside of. But ideally we 00:06:05.320 --> 00:06:07.870 don't want to be co-dependent, because we want to 00:06:07.870 --> 00:06:09.589 be able to run this thing in isolation, to 00:06:09.589 --> 00:06:13.180 test it in isolation, to debug it in isolation. 00:06:13.180 --> 00:06:15.779 So, we don't want to have to stand up 00:06:15.779 --> 00:06:19.640 the entire enterprise on our laptops just in order 00:06:19.640 --> 00:06:20.839 to, to run this thing when we're, when we're 00:06:20.839 --> 00:06:23.700 on a train or in a plane. So, we 00:06:23.700 --> 00:06:26.659 want independence, but we're in a situation of co-dependence. 00:06:26.659 --> 00:06:30.229 And this gets even more interesting when you start 00:06:30.229 --> 00:06:32.899 thinking about it in terms of teams. So, in 00:06:32.899 --> 00:06:35.839 our case, we had our team, let's call us 00:06:35.839 --> 00:06:39.839 the red team. We're building the Rails application. But 00:06:39.839 --> 00:06:42.580 we weren't building any of these services. The deals 00:06:42.580 --> 00:06:44.950 services might have been maintained by the green team 00:06:44.950 --> 00:06:47.159 and the blue team, of course, were working on 00:06:47.159 --> 00:06:48.830 the product service. 00:06:48.830 --> 00:06:53.080 And these teams were teams we sometimes, we didn't 00:06:53.080 --> 00:06:54.899 even know where they, where they were. We had 00:06:54.899 --> 00:06:56.520 vague ways of getting in contact with them, but 00:06:56.520 --> 00:06:58.959 we, we didn't have a very good relationship with 00:06:58.959 --> 00:07:03.659 them. And this leads to lots of interesting side-effects. 00:07:03.659 --> 00:07:06.070 So one thing that you might note here is 00:07:06.070 --> 00:07:08.640 the shape of the teams. Lines up pretty closely 00:07:08.640 --> 00:07:10.930 with the shape of the services. So this gets 00:07:10.930 --> 00:07:15.260 to this thing called Conway's Law. Don't tell DHH 00:07:15.260 --> 00:07:19.700 that I'm talking about laws. 00:07:19.700 --> 00:07:22.260 Conway's Law, coined in the 60s, says that the 00:07:22.260 --> 00:07:25.459 communication patterns of a software system tend to mirror 00:07:25.459 --> 00:07:28.039 the communication patterns of the people who build that 00:07:28.039 --> 00:07:31.719 system. How many people have heard of Conway's Law, 00:07:31.719 --> 00:07:34.140 by the way? 00:07:34.140 --> 00:07:38.180 So, I'm trying to avoid calling it a law 00:07:38.180 --> 00:07:40.250 but I'm not going to be able to do 00:07:40.250 --> 00:07:42.630 that. I think Conway's Law is the most important 00:07:42.630 --> 00:07:46.709 law for us to understand as software engineers. Oh, 00:07:46.709 --> 00:07:53.709 I said. As people. As software writers. Sorry. 00:07:55.909 --> 00:07:58.560 Conway's Law says that how people work together effects 00:07:58.560 --> 00:08:04.269 system architecture. And, and the flip-side. System architecture effects 00:08:04.269 --> 00:08:09.320 how people work together. And, this isn't kind of 00:08:09.320 --> 00:08:13.870 a, a possible outcome. This is reality. This is 00:08:13.870 --> 00:08:18.140 empirically shown for fifty years, sixty years almost. This 00:08:18.140 --> 00:08:19.769 does happen. This is true. 00:08:19.769 --> 00:08:21.890 So you can't, like, decide you don't want it 00:08:21.890 --> 00:08:23.450 to happen. But what you can do is you 00:08:23.450 --> 00:08:26.640 can either wield Conway's Law to your advantage, go 00:08:26.640 --> 00:08:29.279 with the grain and, and use it to your 00:08:29.279 --> 00:08:30.560 advantage, or you can kind of yield to Conway's 00:08:30.560 --> 00:08:32.120 Law and have it beat you up. 00:08:32.120 --> 00:08:34.190 So I'm gonna talk a little bit today about 00:08:34.190 --> 00:08:37.059 how we wielded Conway's Law to our advantage rather 00:08:37.059 --> 00:08:40.330 than have it beat us up. 00:08:40.330 --> 00:08:45.010 So, here's our three teams. One of the most 00:08:45.010 --> 00:08:48.770 fun aspects of, of working in a system that 00:08:48.770 --> 00:08:50.000 has more than one team is, is when you 00:08:50.000 --> 00:08:53.780 find a bug. So let's say we've got our 00:08:53.780 --> 00:08:55.930 deal, deals of the day section, and we've noticed 00:08:55.930 --> 00:08:59.330 that the prices are missing in some of the 00:08:59.330 --> 00:09:01.870 deals section when we run the application. So now 00:09:01.870 --> 00:09:04.250 we start to play this game, it's called Who's 00:09:04.250 --> 00:09:06.330 Bug is it Anyway? 00:09:06.330 --> 00:09:09.830 Hands up if you've played this game. Awesome. OK. 00:09:09.830 --> 00:09:12.080 I wasn't sure if that was a cultural reference 00:09:12.080 --> 00:09:17.200 that people wouldn't get. So, we know that the 00:09:17.200 --> 00:09:19.560 deals aren't loading correctly, but is that the deal 00:09:19.560 --> 00:09:22.750 service's fault? Maybe the product service isn't returning the 00:09:22.750 --> 00:09:25.690 right data for those deals. Maybe we're asking it 00:09:25.690 --> 00:09:27.560 the wrong questions. Or maybe we just have a 00:09:27.560 --> 00:09:32.140 bug in our rendering code. 00:09:32.140 --> 00:09:34.760 The fundamental issue of all of these things is 00:09:34.760 --> 00:09:40.560 this tension between co-dependence and independence. We are co-dependent. 00:09:40.560 --> 00:09:44.650 We want to be independent. Some people solve a 00:09:44.650 --> 00:09:48.640 lot of these problems by embracing co-dependence. So, if 00:09:48.640 --> 00:09:50.140 our Rails app needs the services to be up 00:09:50.140 --> 00:09:52.010 and running in order to run, we'll stand up 00:09:52.010 --> 00:09:54.830 the services. You can do that. You can stand 00:09:54.830 --> 00:09:57.210 up copies of the services locally on your laptop. 00:09:57.210 --> 00:09:59.940 This is more and more easy to do nowadays 00:09:59.940 --> 00:10:01.860 cause we've got kind of trendy dev-ops things like 00:10:01.860 --> 00:10:04.750 Vagrant and Chef and Puppet. 00:10:04.750 --> 00:10:06.720 That will only get you so far. Try standing 00:10:06.720 --> 00:10:10.390 up a bank on your laptop. It's not happening. 00:10:10.390 --> 00:10:12.950 Very, very common approach here is to, to use 00:10:12.950 --> 00:10:15.950 kind of shared services. So a shared dev environment. 00:10:15.950 --> 00:10:20.190 A shared kind of staging environment. That kind of 00:10:20.190 --> 00:10:22.280 thing. And that, that works quite well. But, for 00:10:22.280 --> 00:10:25.390 our team, we decided to go the alternate route. 00:10:25.390 --> 00:10:28.930 We wanted to, as much as possible, push for 00:10:28.930 --> 00:10:33.900 independence. And, the main way we did that was 00:10:33.900 --> 00:10:38.700 by rem- replacing our dependency on services with fake 00:10:38.700 --> 00:10:39.760 services. 00:10:39.760 --> 00:10:44.700 So, when we wanted to test our application, we 00:10:44.700 --> 00:10:47.030 wanted to test just our application in independence. And 00:10:47.030 --> 00:10:49.200 the way that we did that was, for each 00:10:49.200 --> 00:10:51.620 of our dependent services, we replaced those services with 00:10:51.620 --> 00:10:55.750 kind of a fake version of that service. And 00:10:55.750 --> 00:10:57.700 by doing that, we were allowed to run these 00:10:57.700 --> 00:11:01.810 things that, that we called bounded integration tests. 00:11:01.810 --> 00:11:05.750 So, these were tests that tested the entire stack 00:11:05.750 --> 00:11:08.990 of our application, of our Rails application, from top 00:11:08.990 --> 00:11:11.140 to bottom, from kind of html all the way 00:11:11.140 --> 00:11:16.420 down to the network. But, we weren't, we were 00:11:16.420 --> 00:11:19.930 testing that in isolation. We weren't actually hitting those 00:11:19.930 --> 00:11:23.960 shared services. 00:11:23.960 --> 00:11:26.310 And what's really interesting here is, do you see 00:11:26.310 --> 00:11:29.280 the, the boundary of those tests lines up almost 00:11:29.280 --> 00:11:31.850 perfectly with the boundary of our team. So this 00:11:31.850 --> 00:11:33.960 is a really good example of using Conway's Law 00:11:33.960 --> 00:11:37.880 to your advantage. Embracing Conway's Law and shaping your 00:11:37.880 --> 00:11:41.550 approach, shaping your software, with the shape of your 00:11:41.550 --> 00:11:43.610 team. And this was something that helped us win. 00:11:43.610 --> 00:11:46.570 So, if you want to build these fake services, 00:11:46.570 --> 00:11:49.280 what are your options? Comes down to, to two 00:11:49.280 --> 00:11:52.310 different kind of techniques, really. In-process and out-of-process. So 00:11:52.310 --> 00:11:55.200 in-process means you're kind of messing around with the 00:11:55.200 --> 00:11:57.870 actual Rails, running Rails process, and like replacing the 00:11:57.870 --> 00:12:00.690 network stack with a fake network stack maybe. 00:12:00.690 --> 00:12:02.930 Out of process means you're standing up a real 00:12:02.930 --> 00:12:05.380 http server. It's an actual, you know, you can 00:12:05.380 --> 00:12:08.520 actually talk to it over http. And your application, 00:12:08.520 --> 00:12:11.410 your Rails app, rather than pointing to a real 00:12:11.410 --> 00:12:13.210 deal service, it's going to point to this fake 00:12:13.210 --> 00:12:14.830 deal service. It's gonna look and smell like a 00:12:14.830 --> 00:12:16.910 deal service. You send it a request. It gives 00:12:16.910 --> 00:12:19.120 you a response. It's not a real deals service. 00:12:19.120 --> 00:12:21.830 It's a fake service that we have control over. 00:12:21.830 --> 00:12:24.100 So, you can imagine if we wanted to test 00:12:24.100 --> 00:12:26.810 unicode rendering, we could stand up a fake product 00:12:26.810 --> 00:12:30.090 service that returned a product tied tool that was 00:12:30.090 --> 00:12:32.210 all kind of snow man and that kind of 00:12:32.210 --> 00:12:35.350 stuff. And, and that allows us to test how 00:12:35.350 --> 00:12:38.490 we handle unicode titles without having, then, to need 00:12:38.490 --> 00:12:39.460 a unicode title. 00:12:39.460 --> 00:12:43.130 So, in process, a couple of popular options, vcr 00:12:43.130 --> 00:12:45.040 and webmock. We actually used vcr quite heavily on 00:12:45.040 --> 00:12:47.500 this project that I'm talking about. So basically it 00:12:47.500 --> 00:12:49.980 just kind of sits, it kind of injects itself 00:12:49.980 --> 00:12:53.279 in between your application and the network layer and 00:12:53.279 --> 00:12:55.779 records all the interactions between your application and the 00:12:55.779 --> 00:12:57.650 network, and then you can kind of flip it 00:12:57.650 --> 00:12:59.760 into playback mode, and rather than, next you time 00:12:59.760 --> 00:13:01.690 you go and make a, make a network call, 00:13:01.690 --> 00:13:03.320 it's not actually gonna call the network. It's going 00:13:03.320 --> 00:13:07.900 to just play back the previously recorded interaction. So 00:13:07.900 --> 00:13:12.650 that worked out well for us with some caveats. 00:13:12.650 --> 00:13:14.350 Out of process, you've got a lot more options, 00:13:14.350 --> 00:13:16.440 because you're not running out of process. You don't 00:13:16.440 --> 00:13:19.120 have to use Ruby. You can use whatever tool 00:13:19.120 --> 00:13:21.440 makes the most sense. So mimic is a really 00:13:21.440 --> 00:13:23.529 good option here. This is actually a Ruby gem 00:13:23.529 --> 00:13:27.880 written by Luke Redpath. It's a Sinatra application that 00:13:27.880 --> 00:13:30.950 pretends to be whatever you want it to be. 00:13:30.950 --> 00:13:32.900 Outside of the Ruby community, or outside of the 00:13:32.900 --> 00:13:37.060 Ruby language, I suppose, there's a library called moco. 00:13:37.060 --> 00:13:41.410 It's quite powerful. Stubby is another one. And then 00:13:41.410 --> 00:13:44.950 there's this interesting one called montebank. What makes montebank 00:13:44.950 --> 00:13:47.640 interesting is it doesn't just fake out http, it 00:13:47.640 --> 00:13:49.920 will fake out whatever protocol you want. So it 00:13:49.920 --> 00:13:51.529 can fake out smtp to it, so you can 00:13:51.529 --> 00:13:54.090 check emails. It'll fake out web sockets. That kind 00:13:54.090 --> 00:13:57.220 of stuff. So that's an interesting one to look 00:13:57.220 --> 00:13:59.020 at. 00:13:59.020 --> 00:14:01.570 So now we've got these bounded integration tests. And 00:14:01.570 --> 00:14:04.920 that means we have some confidence that it wasn't 00:14:04.920 --> 00:14:08.339 us that, that's creating this bug. Because our bounded 00:14:08.339 --> 00:14:10.660 integration tests are passing. So all was happy and 00:14:10.660 --> 00:14:13.020 we can go on with our day. 00:14:13.020 --> 00:14:15.740 Well, obviously not, because we actually still don't know 00:14:15.740 --> 00:14:18.250 where the bug is. And the goal here is 00:14:18.250 --> 00:14:20.600 not for us to kind of, prove that we, 00:14:20.600 --> 00:14:22.970 it's not our fault. The goal is to identify 00:14:22.970 --> 00:14:24.839 the problem and fix it, because at the end 00:14:24.839 --> 00:14:26.660 of the day it's a system. We're a part 00:14:26.660 --> 00:14:28.220 of that system and we need to fix the 00:14:28.220 --> 00:14:31.279 system and move on. 00:14:31.279 --> 00:14:33.420 So, so we're left saying, OK. We know it's 00:14:33.420 --> 00:14:35.730 not our fault. We'd like to help our, our 00:14:35.730 --> 00:14:40.720 comrades on other teams figure out where the problem 00:14:40.720 --> 00:14:43.410 is. And for that we used a technique called 00:14:43.410 --> 00:14:49.180 contract tests. So, contract tests, also sometimes referred to 00:14:49.180 --> 00:14:55.230 as consumer-driven contracts. The idea with these is we 00:14:55.230 --> 00:15:00.300 write test code that expresses what we expect a 00:15:00.300 --> 00:15:03.120 dependency, an external service to do, and then we 00:15:03.120 --> 00:15:05.050 run those tests against a real version of that 00:15:05.050 --> 00:15:07.290 service, and we find out if that service actually 00:15:07.290 --> 00:15:10.980 does what we expect. Quite simple. 00:15:10.980 --> 00:15:13.339 What's weird is we're testing someone else's code. We're 00:15:13.339 --> 00:15:16.540 not testing our code. We're, we're writing tests, but 00:15:16.540 --> 00:15:21.580 we're testing another team's services. So here's, again, our 00:15:21.580 --> 00:15:27.100 boundary, our bounded integration tests. And, so we verified 00:15:27.100 --> 00:15:29.380 that the way that the Rails app is, that 00:15:29.380 --> 00:15:30.920 the Rails app does the right thing when it 00:15:30.920 --> 00:15:34.480 talks to what we think these services are doing. 00:15:34.480 --> 00:15:37.110 Contract tests, once we add these, actually verify that 00:15:37.110 --> 00:15:39.020 what the service does is what we think it's 00:15:39.020 --> 00:15:39.750 gonna do. 00:15:39.750 --> 00:15:42.420 Cause it could be that there's a bug in 00:15:42.420 --> 00:15:44.730 the service, and these tests will find those bugs, 00:15:44.730 --> 00:15:46.279 hopefully, because we'll ask it to do something and 00:15:46.279 --> 00:15:48.460 it won't do what we expect. There could be 00:15:48.460 --> 00:15:51.290 a bug in our understanding of the service, and 00:15:51.290 --> 00:15:52.839 that's why these are called contract tests, because they 00:15:52.839 --> 00:15:55.220 define a contract between our team and the other 00:15:55.220 --> 00:15:59.660 team. Again, Conway's Law is coming in here. 00:15:59.660 --> 00:16:02.310 How many of you have worked with an external 00:16:02.310 --> 00:16:05.360 service where the, the Wiki page that documented the 00:16:05.360 --> 00:16:08.730 API was out of date? How many of you 00:16:08.730 --> 00:16:10.790 have ever worked on a service where it wasn't 00:16:10.790 --> 00:16:16.890 wrong? It's always wrong. And that's OK. Documentation tends 00:16:16.890 --> 00:16:19.060 to be stale. API documentation I think is never 00:16:19.060 --> 00:16:22.960 not stale. It just is born that way. 00:16:22.960 --> 00:16:25.310 These contract tests are a way to, to, to 00:16:25.310 --> 00:16:28.950 mitigate that. Because you're expressing your expectations in code 00:16:28.950 --> 00:16:32.520 rather than in words. 00:16:32.520 --> 00:16:35.860 So we, on, on my, on my team, we 00:16:35.860 --> 00:16:38.110 wrote contract tests for every single one of our 00:16:38.110 --> 00:16:41.230 dependencies. This was the most productive thing we did 00:16:41.230 --> 00:16:44.190 on this team is, in terms of improving efficiency 00:16:44.190 --> 00:16:49.460 of, of us creating software. It really, really helped 00:16:49.460 --> 00:16:51.490 us nail down, whenever there was an issue, where 00:16:51.490 --> 00:16:53.610 that issue was. And it really helped us communicate 00:16:53.610 --> 00:16:56.180 with, with our friends on other teams. 00:16:56.180 --> 00:16:58.560 So, if you want to do these, you've got 00:16:58.560 --> 00:17:00.500 a few different options for contract tests. We actually 00:17:00.500 --> 00:17:03.910 just did plain old RSpec. We just used standard 00:17:03.910 --> 00:17:07.099 RSpec test runner to make network calls to our 00:17:07.099 --> 00:17:09.000 dependencies and then look at the results and make 00:17:09.000 --> 00:17:11.299 sure that things looked the way that we expected 00:17:11.299 --> 00:17:12.480 them to look. 00:17:12.480 --> 00:17:15.409 There's two more kind of sophisticated options out there. 00:17:15.409 --> 00:17:17.539 There's a gem called pacto and another gem called 00:17:17.539 --> 00:17:20.539 pact. And these are kind of like real power 00:17:20.539 --> 00:17:24.589 tools. They're quite sophisticated. They're, they're very fully-featured. If 00:17:24.589 --> 00:17:26.339 you want to really kind of get into this, 00:17:26.339 --> 00:17:28.910 into this mind set. 00:17:28.910 --> 00:17:31.400 So once we had these bounded integration tests and 00:17:31.400 --> 00:17:34.410 these contract tests, we were left with this kind 00:17:34.410 --> 00:17:38.730 of CI dashboard that looked something like this. So 00:17:38.730 --> 00:17:40.540 whenever we checked in code, we'd run our unit 00:17:40.540 --> 00:17:42.970 tests, we'd run our functional tests. And assuming those 00:17:42.970 --> 00:17:45.140 passed, we'd run our kind of bounded integration tests 00:17:45.140 --> 00:17:46.330 up there at the top right. 00:17:46.330 --> 00:17:50.900 Assuming those passed, we'd then run our end-to-end tests. 00:17:50.900 --> 00:17:54.040 So these were running, testing as much of the 00:17:54.040 --> 00:17:56.929 stack as we could possibly stand up. So, our 00:17:56.929 --> 00:18:00.390 code, our team's code, maybe our team's dependency's code. 00:18:00.390 --> 00:18:02.520 So as much of the stack as possible. So 00:18:02.520 --> 00:18:04.690 really verifying, from a user's point of view, that 00:18:04.690 --> 00:18:08.510 this, this system worked as we expected. 00:18:08.510 --> 00:18:11.390 In theory, this was the picture. In reality it 00:18:11.390 --> 00:18:15.240 was almost never green. That was OK. So, this 00:18:15.240 --> 00:18:16.850 is what things looked like a lot of the 00:18:16.850 --> 00:18:21.110 time. Our tests were passing, the end-to-end tests were 00:18:21.110 --> 00:18:23.890 failing. If that was the end of the story, 00:18:23.890 --> 00:18:26.650 we'd be left constantly fighting fires of, why is 00:18:26.650 --> 00:18:28.590 it not working in product- or, why is it 00:18:28.590 --> 00:18:31.220 not working in staging? And is it the deals 00:18:31.220 --> 00:18:33.140 service or the product service, blah, blah, blah. 00:18:33.140 --> 00:18:35.240 But we had these contract tests. So now we 00:18:35.240 --> 00:18:37.860 can see here that service D's contract tests seem 00:18:37.860 --> 00:18:40.500 to be failing. So we've got some end-to-end tests 00:18:40.500 --> 00:18:44.210 that are failing. This contract is failing. That's where 00:18:44.210 --> 00:18:46.420 we can start looking for, for the cause of 00:18:46.420 --> 00:18:47.160 this problem. 00:18:47.160 --> 00:18:48.620 So this is what we spent a lot of 00:18:48.620 --> 00:18:49.910 our time, not a lot of our time doing, 00:18:49.910 --> 00:18:52.600 but this was a very frequent occurrence. The build 00:18:52.600 --> 00:18:54.429 would go red. We'd have a look at our 00:18:54.429 --> 00:18:58.420 dashboard. Oh, it's service D, again. Service D, you 00:18:58.420 --> 00:19:00.370 guys. 00:19:00.370 --> 00:19:03.309 So we go to our CI system and we 00:19:03.309 --> 00:19:06.860 would look at the logs. We'd get the log 00:19:06.860 --> 00:19:08.640 data that showed the request we were sending the 00:19:08.640 --> 00:19:11.590 response we, we got back. And we'd have a 00:19:11.590 --> 00:19:13.420 look and say, yup. It looks like they've broken 00:19:13.420 --> 00:19:15.170 it again. They forgot to turn on the database 00:19:15.170 --> 00:19:18.840 after doing a deployment, again. And we would write 00:19:18.840 --> 00:19:20.340 an email to them saying, hey, I think it's 00:19:20.340 --> 00:19:23.220 probably a bug in our code, but we've noticed 00:19:23.220 --> 00:19:24.860 when we send this request we get back this 00:19:24.860 --> 00:19:28.670 500 error from your service, I don't know what's 00:19:28.670 --> 00:19:29.950 going on there. 00:19:29.950 --> 00:19:32.640 You can tell I'm a consultant. 00:19:32.640 --> 00:19:37.000 And, things moved a lot smoother. It, it wasn't 00:19:37.000 --> 00:19:38.960 all kind of rainbows and unicorns, but it was 00:19:38.960 --> 00:19:40.280 a lot better than if these tests hadn't have 00:19:40.280 --> 00:19:42.730 been there. We would have still ended up telling 00:19:42.730 --> 00:19:43.880 them and they would have fixed it, but it 00:19:43.880 --> 00:19:46.000 would have sucked a lot more of our time. 00:19:46.000 --> 00:19:49.179 So that's all the hand wave-y, let's all work 00:19:49.179 --> 00:19:51.900 as teams stuff. Let's talk about how we're actually 00:19:51.900 --> 00:19:56.720 gonna build this, this inside of our team. 00:19:56.720 --> 00:19:57.800 So I want to talk about this idea called 00:19:57.800 --> 00:20:01.480 service gateways. The main thing I'm gonna talk about 00:20:01.480 --> 00:20:04.150 with service gateways is this gem called faraday. And 00:20:04.150 --> 00:20:06.250 in order to talk about this gem called faraday, 00:20:06.250 --> 00:20:08.010 I need to talk about this other gem called 00:20:08.010 --> 00:20:08.600 rack. 00:20:08.600 --> 00:20:10.820 So, how many of you know what rack is? 00:20:10.820 --> 00:20:13.410 Yeah, everyone knows what rack is. How many of 00:20:13.410 --> 00:20:15.420 you hate people that ask you raise your hands 00:20:15.420 --> 00:20:19.350 when you're in the audience? I have all the 00:20:19.350 --> 00:20:22.530 power. You guys have the power but don't realize 00:20:22.530 --> 00:20:23.540 that. 00:20:23.540 --> 00:20:27.660 So, rack is this awesome abstraction over http servers. 00:20:27.660 --> 00:20:31.990 And the main idea, the main thing that makes 00:20:31.990 --> 00:20:36.150 rack awesome is, by abstracting over the concept of 00:20:36.150 --> 00:20:38.380 a request and the concept of a response, we 00:20:38.380 --> 00:20:41.299 can kind of stack these middleware components in between 00:20:41.299 --> 00:20:45.929 our application, and the, the underlying http server. 00:20:45.929 --> 00:20:49.429 So, as a request comes in, from the outside 00:20:49.429 --> 00:20:55.470 world, it travels through this stack of middleware. So, 00:20:55.470 --> 00:20:58.299 each piece of middleware can, as this request is 00:20:58.299 --> 00:21:03.540 traveling through, has the opportunity to kind of modify, 00:21:03.540 --> 00:21:06.040 modify the request and kind of transform it in 00:21:06.040 --> 00:21:09.040 some way, add some information to the request, or 00:21:09.040 --> 00:21:11.059 to kind of have some side-effect. 00:21:11.059 --> 00:21:13.410 And then, a Rails app is gonna deal with 00:21:13.410 --> 00:21:16.090 that request and then, and then send back a 00:21:16.090 --> 00:21:18.120 response. And, again, as that response is traveling back 00:21:18.120 --> 00:21:23.670 up through the pipeline, through the, the middleware stacks, 00:21:23.670 --> 00:21:26.890 again, each of those pieces of middleware can modify 00:21:26.890 --> 00:21:29.150 that request in some way, can add some information 00:21:29.150 --> 00:21:32.320 to it or can have some side effect. 00:21:32.320 --> 00:21:35.890 Rails loves rack. So this is the, the kind 00:21:35.890 --> 00:21:40.960 of the stop stack of rack middlewares that comes 00:21:40.960 --> 00:21:42.970 when you, when you just do rails generate new 00:21:42.970 --> 00:21:46.580 project. So, yeah. Pretty, pretty popular. And the reason 00:21:46.580 --> 00:21:48.179 that this is popular is because the guys that 00:21:48.179 --> 00:21:50.210 built rack are smart and the guys that, guys 00:21:50.210 --> 00:21:52.230 and gals, excuse me, that built rack are smart 00:21:52.230 --> 00:21:55.980 and, likewise, Rails. And they realized that this is 00:21:55.980 --> 00:21:57.880 a very powerful abstraction and so they're really leveraging 00:21:57.880 --> 00:22:00.100 this power. 00:22:00.100 --> 00:22:02.150 So why was I talking about rack when I 00:22:02.150 --> 00:22:04.559 was supposed to be talking about faraday? Faraday is 00:22:04.559 --> 00:22:07.600 the exact same idea of rack, but applied for 00:22:07.600 --> 00:22:11.200 an http client. So, again, we have this abstraction 00:22:11.200 --> 00:22:13.720 over request and response. This time in the context 00:22:13.720 --> 00:22:16.170 of making a request and receiving a response rather 00:22:16.170 --> 00:22:19.179 than receiving a request and sending back a response. 00:22:19.179 --> 00:22:23.120 And, again, we can stack these pieces of middleware 00:22:23.120 --> 00:22:27.690 in between our application and the underlying network library. 00:22:27.690 --> 00:22:30.260 So, in this case, it's us making a request. 00:22:30.260 --> 00:22:32.809 So, as we send the request out to the 00:22:32.809 --> 00:22:36.380 network, again, that request travels through all of these 00:22:36.380 --> 00:22:38.420 middlewares, and these middlewares, again, have the opportunity to 00:22:38.420 --> 00:22:40.850 either modify that request in some way, have a 00:22:40.850 --> 00:22:43.720 side effect, add some information to the request. Eventually 00:22:43.720 --> 00:22:45.010 it gets on the wire. Eventually it gets to 00:22:45.010 --> 00:22:47.290 our dependency. Hopefully not service D because they'll probably 00:22:47.290 --> 00:22:50.260 go down again. 00:22:50.260 --> 00:22:53.640 And then service D hopefully sends a response, and 00:22:53.640 --> 00:22:57.160 again, as that response travels back through the stack, 00:22:57.160 --> 00:23:01.110 the stack is able to modify that, that response 00:23:01.110 --> 00:23:02.620 as it comes through, add some information. Have a 00:23:02.620 --> 00:23:04.690 side effect. 00:23:04.690 --> 00:23:07.990 So, how do we use this thing? So here's 00:23:07.990 --> 00:23:10.230 a, this is a service gateway. This is a 00:23:10.230 --> 00:23:14.360 class that represents a, a connection to the outside 00:23:14.360 --> 00:23:19.480 world. Here's us building a faraday middleware stack. So 00:23:19.480 --> 00:23:24.160 we're building a connection. We're saying hey, new faraday. 00:23:24.160 --> 00:23:26.400 Please use the instrumentation middleware and do some JSON 00:23:26.400 --> 00:23:30.840 stuff and follow redirects and do some logging. 00:23:30.840 --> 00:23:33.929 And that's it. We've now got a faraday connection. 00:23:33.929 --> 00:23:37.190 This is how we use a faraday connection. So, 00:23:37.190 --> 00:23:40.490 we say, hey, faraday connection, hey, http client, we 00:23:40.490 --> 00:23:43.160 want to get this url, or this path. And 00:23:43.160 --> 00:23:46.030 we get a response back. Pretty simple stuff. I 00:23:46.030 --> 00:23:47.950 would say this is readable code. DHH would be 00:23:47.950 --> 00:23:49.990 pleased with me. 00:23:49.990 --> 00:23:51.590 And what's really nice, the real power of, of 00:23:51.590 --> 00:23:54.420 faraday, is there's all this extra stuff that we 00:23:54.420 --> 00:23:58.840 set up, this middleware, this technical junk going on. 00:23:58.840 --> 00:24:01.330 But our, the rest of our code doesn't have 00:24:01.330 --> 00:24:03.140 to care about it. So we've kind of abstracted 00:24:03.140 --> 00:24:07.970 over all of that stuff of JSON, following redirects 00:24:07.970 --> 00:24:11.679 and logging and caching. 00:24:11.679 --> 00:24:13.470 So we used faraday a lot in these things 00:24:13.470 --> 00:24:19.250 that, that I was calling service gateways. So the, 00:24:19.250 --> 00:24:20.540 as I said, the really nice thing about things 00:24:20.540 --> 00:24:22.309 like rack and faraday is they allow us to 00:24:22.309 --> 00:24:28.559 segregate the boring tech-y stuff, http, logging, caching, instrumentation, 00:24:28.559 --> 00:24:30.530 the stuff that computer people talk about at conferences 00:24:30.530 --> 00:24:33.580 like this, from the business domain, which is the 00:24:33.580 --> 00:24:35.919 stuff that people actually care about. 00:24:35.919 --> 00:24:38.890 So, these service gateways acted as the place where 00:24:38.890 --> 00:24:43.419 we pushed all of the boring techy goop out 00:24:43.419 --> 00:24:45.960 to the boundaries of our system, so the core 00:24:45.960 --> 00:24:48.630 of our system, the core of our Rails application, 00:24:48.630 --> 00:24:53.630 wasn't talking about JSON requests, urls, caching, logging, deserialization. 00:24:53.630 --> 00:24:56.940 It was talking about books and products and prices. 00:24:56.940 --> 00:25:01.080 Things that our application really should be caring about. 00:25:01.080 --> 00:25:07.090 So this is this idea of hexagonal architectures. Push 00:25:07.090 --> 00:25:09.370 all of the goopy boring techy stuff out to 00:25:09.370 --> 00:25:11.169 the boundaries of your system, so that the core 00:25:11.169 --> 00:25:15.240 of your system can focus on the domain. The 00:25:15.240 --> 00:25:17.059 stuff that you really care about. The stuff that 00:25:17.059 --> 00:25:22.530 makes your system valuable to other people. 00:25:22.530 --> 00:25:24.350 So what kind of things did we put into 00:25:24.350 --> 00:25:26.290 these? What kind of stuff, boring techy stuff did 00:25:26.290 --> 00:25:30.610 we isolate in these service gateways? One thing that 00:25:30.610 --> 00:25:35.760 we did was we isolated serialization. So, serialization means 00:25:35.760 --> 00:25:39.710 JSON parsing or, in our case, XML parsing, cause 00:25:39.710 --> 00:25:44.190 we were in an enterprise. Yay. 00:25:44.190 --> 00:25:47.120 Comes down to the same thing actually. So, when 00:25:47.120 --> 00:25:49.320 we talk about parsing JSON or XML, we're actually 00:25:49.320 --> 00:25:52.020 talking about two different things. And I think often 00:25:52.020 --> 00:25:55.630 we conflate these two. So, step one, is we 00:25:55.630 --> 00:25:57.830 have this stream of bytes or this raw string, 00:25:57.830 --> 00:25:59.390 and we want to turn it into this kind 00:25:59.390 --> 00:26:02.919 of generic structure, like a hash of arrays of, 00:26:02.919 --> 00:26:04.929 of objects, right. So that's, that's something, you know. 00:26:04.929 --> 00:26:06.910 You do JSON dot parse or whatever and you 00:26:06.910 --> 00:26:09.390 get this kind of generic data structure out. 00:26:09.390 --> 00:26:13.760 But, if we are following the principles of hexagonal 00:26:13.760 --> 00:26:17.179 architecture, we don't want to deal with JSON data 00:26:17.179 --> 00:26:18.900 structures or generic data structures. We want to deal 00:26:18.900 --> 00:26:21.730 with products and books and deals. So the next 00:26:21.730 --> 00:26:23.950 step is to actually take that generic structure and 00:26:23.950 --> 00:26:26.520 map it into a domain object that we actually 00:26:26.520 --> 00:26:27.299 want to work with. 00:26:27.299 --> 00:26:30.240 Cause our goal here is to stop talking about 00:26:30.240 --> 00:26:32.580 things like JSON as quickly as possible, at the 00:26:32.580 --> 00:26:35.410 boundaries of our system, and start talking about products 00:26:35.410 --> 00:26:39.940 and prices and deals. So this is two steps. 00:26:39.940 --> 00:26:42.850 So, the other thing that you'll, I noticed working 00:26:42.850 --> 00:26:47.120 particularly in larger organizations, is these responses that you 00:26:47.120 --> 00:26:49.820 get back are huge. And really, often times, you 00:26:49.820 --> 00:26:52.340 only really care about a small subset of the 00:26:52.340 --> 00:26:54.470 response you get back. So you get back this 00:26:54.470 --> 00:26:56.750 big chunk of bytes. You turn it into this 00:26:56.750 --> 00:26:59.450 big generic structure, and then you actually just want 00:26:59.450 --> 00:27:00.730 to pluck three or four or five or six 00:27:00.730 --> 00:27:03.490 things out of that structure, that represent the product 00:27:03.490 --> 00:27:04.980 in your, in your domain. 00:27:04.980 --> 00:27:07.240 You don't care about the, the short description. You 00:27:07.240 --> 00:27:09.690 certainly don't care about the response time. Really all 00:27:09.690 --> 00:27:11.500 you care about is the, you know, the title 00:27:11.500 --> 00:27:13.760 or the author, you know, that kind of thing. 00:27:13.760 --> 00:27:16.570 So there's some gems that can help you make 00:27:16.570 --> 00:27:19.220 this easier. So, very popular one is this thing 00:27:19.220 --> 00:27:20.980 called hashie. Hashie will just make it a little 00:27:20.980 --> 00:27:22.630 bit easier for you to work with these generic 00:27:22.630 --> 00:27:24.470 data structures. But at the end of the day, 00:27:24.470 --> 00:27:27.090 they're still generic data structures. 00:27:27.090 --> 00:27:31.299 The second step that we took was to take 00:27:31.299 --> 00:27:32.880 all of that boring boiler plate that we were 00:27:32.880 --> 00:27:35.070 doing of saying, you know, get this dot that 00:27:35.070 --> 00:27:37.530 dot the other thing and get me the price. 00:27:37.530 --> 00:27:38.940 And then this dot that dot the other thing, 00:27:38.940 --> 00:27:41.650 get me the, the offer details. And turn that 00:27:41.650 --> 00:27:44.100 into a declarative statement that we could just make 00:27:44.100 --> 00:27:46.059 at the top of a class that would make 00:27:46.059 --> 00:27:46.250 these things. 00:27:46.250 --> 00:27:48.809 So we turn that mapping, that boring boilerplate mapping, 00:27:48.809 --> 00:27:52.100 into something that was declarative rather than imperative. We 00:27:52.100 --> 00:27:55.580 actually ended up extracting that little library that we 00:27:55.580 --> 00:27:58.590 built into this thing called lazy_doc. There's another couple 00:27:58.590 --> 00:28:00.970 of gems that are very, that have very similar 00:28:00.970 --> 00:28:04.450 goals. Embedded_doc is another one. And there's this really 00:28:04.450 --> 00:28:08.570 quite powerful one called representable, which is very good. 00:28:08.570 --> 00:28:10.809 It's, it's very powerful. It almost does too many 00:28:10.809 --> 00:28:13.179 things for my taste, but it's definitely, definitely a 00:28:13.179 --> 00:28:14.960 good option. 00:28:14.960 --> 00:28:17.120 The reason, by the way, embedded_doc is called embedded_doc 00:28:17.120 --> 00:28:19.230 is cause it's, what we're kind of talking about 00:28:19.230 --> 00:28:22.490 here is this embedded document pattern. That's the name 00:28:22.490 --> 00:28:24.350 of the pattern. Sorry, now I'm talking about patterns. 00:28:24.350 --> 00:28:28.169 I'm really off-message. 00:28:28.169 --> 00:28:29.549 So what else did we put in our faraday 00:28:29.549 --> 00:28:35.720 stack? Caching. This is the most, for me, the 00:28:35.720 --> 00:28:38.919 most exciting thing to talk about. So let me 00:28:38.919 --> 00:28:41.450 talk about it. So, back to our example, let's 00:28:41.450 --> 00:28:44.000 say we've got our Rails app. And we've got 00:28:44.000 --> 00:28:47.200 our product service and we've got this pricing service. 00:28:47.200 --> 00:28:48.640 Fun fact that I didn't know before I worked 00:28:48.640 --> 00:28:51.450 on this application. Prices can change, like, multiple times 00:28:51.450 --> 00:28:54.220 a minute in some of these systems. They're really 00:28:54.220 --> 00:28:56.840 trying to optimize for prices. 00:28:56.840 --> 00:29:01.820 So, can't really cache pricing information. Product information tends 00:29:01.820 --> 00:29:03.530 to be pretty static. The author of a book 00:29:03.530 --> 00:29:06.110 doesn't change very often. The title of a book 00:29:06.110 --> 00:29:09.650 doesn't change very often. But we're always looking up 00:29:09.650 --> 00:29:12.160 this product information, almost every single request. So let's 00:29:12.160 --> 00:29:13.850 cache that. Let's make this performant. 00:29:13.850 --> 00:29:16.679 So, a lot of you are automatically starting to 00:29:16.679 --> 00:29:18.910 think about how we would implement this and, well, 00:29:18.910 --> 00:29:20.730 we would have a product cache, and then when 00:29:20.730 --> 00:29:22.429 we need a product we'll go to the cache, 00:29:22.429 --> 00:29:24.750 and if it's there, then we'll check the freshness. 00:29:24.750 --> 00:29:27.160 And if the freshness is up to some configured 00:29:27.160 --> 00:29:28.890 thing, then we'll get it. Otherwise we'll go to 00:29:28.890 --> 00:29:30.280 the network. And then when we get back from 00:29:30.280 --> 00:29:31.950 network, put it in the cache, blah, blah, blah. 00:29:31.950 --> 00:29:34.559 All of this stuff, right. And we're, we're back 00:29:34.559 --> 00:29:37.090 to thinking about boring technical goop and we're not 00:29:37.090 --> 00:29:40.500 talking about our domain anymore. We didn't want to 00:29:40.500 --> 00:29:44.789 do this. I'm, honestly, bored with getting caching wrong 00:29:44.789 --> 00:29:46.690 over and over again in each of my applications. 00:29:46.690 --> 00:29:48.700 And so, we just pretended to be a web 00:29:48.700 --> 00:29:50.690 browser. 00:29:50.690 --> 00:29:53.559 As web developers, we know that images are very 00:29:53.559 --> 00:29:56.049 cacheable, and we should put caching headers on our, 00:29:56.049 --> 00:29:59.059 on our images, so that the browser can do 00:29:59.059 --> 00:30:00.450 the right thing and cache this so that every 00:30:00.450 --> 00:30:02.150 time we want to get this image, if it's 00:30:02.150 --> 00:30:04.200 already in the cache, we don't have to go 00:30:04.200 --> 00:30:09.200 onto the network. Kind of very common practice. 00:30:09.200 --> 00:30:12.820 We don't write JavaScript cache repositories and then go 00:30:12.820 --> 00:30:13.980 and check and see if the image is in 00:30:13.980 --> 00:30:16.059 there and then, if it's not, we'll go to 00:30:16.059 --> 00:30:17.230 the network and then put it in the cache 00:30:17.230 --> 00:30:18.320 repository. 00:30:18.320 --> 00:30:19.910 The web browser just does this for us. All 00:30:19.910 --> 00:30:21.990 we need to do is set the caching headers 00:30:21.990 --> 00:30:26.570 on this, this ping. So, my argument is, and 00:30:26.570 --> 00:30:28.990 what we did was, just apply the same principle 00:30:28.990 --> 00:30:33.000 to our API. So product dot, or product slash 00:30:33.000 --> 00:30:35.809 wherever this is at - my pretend url for, 00:30:35.809 --> 00:30:39.750 for this product. We had caching headers on this 00:30:39.750 --> 00:30:43.980 API call, that said this, this url is cacheable 00:30:43.980 --> 00:30:46.590 for twenty minutes. 00:30:46.590 --> 00:30:49.789 In our faraday stack, we had a caching middleware. 00:30:49.789 --> 00:30:52.880 So, the first time that we requested that url 00:30:52.880 --> 00:30:55.240 for that product, first time we wanted that product, 00:30:55.240 --> 00:30:57.140 our Rails app would say to our service gateway, 00:30:57.140 --> 00:30:58.809 I want this product. The service gateway would say, 00:30:58.809 --> 00:31:01.549 OK, it's at this url. I'm gonna make a 00:31:01.549 --> 00:31:05.210 request through my faraday stack for that url. 00:31:05.210 --> 00:31:08.320 That request would go through our stack, and the 00:31:08.320 --> 00:31:09.830 response would go through our response on the way 00:31:09.830 --> 00:31:11.309 back, and the caching layer would say, oh look 00:31:11.309 --> 00:31:13.820 at that. There's this caching header on this JSON 00:31:13.820 --> 00:31:15.440 that says it's gonna be valid for twenty minutes. 00:31:15.440 --> 00:31:17.450 Let me just put this response in my little 00:31:17.450 --> 00:31:19.400 cache on disk here. 00:31:19.400 --> 00:31:23.590 And, five minutes later, another user comes in and 00:31:23.590 --> 00:31:26.429 wants to, by chance, look at the same product. 00:31:26.429 --> 00:31:30.450 Again, our service gateway, our code doesn't do anything 00:31:30.450 --> 00:31:31.950 different. It just says, hey, service gateway, I want 00:31:31.950 --> 00:31:34.690 this product. Our service gateway doesn't do anything different. 00:31:34.690 --> 00:31:36.630 It just says to faraday, hey, I want to 00:31:36.630 --> 00:31:39.080 get this url. 00:31:39.080 --> 00:31:41.289 The request comes through to the caching layer. The 00:31:41.289 --> 00:31:44.049 caching layer says, oh look. That's the same url 00:31:44.049 --> 00:31:46.150 I just pulled five minutes, and the, the caching 00:31:46.150 --> 00:31:48.270 headers say it's valid for twenty minutes. I'll just 00:31:48.270 --> 00:31:49.940 return what I've got in my local cache. I 00:31:49.940 --> 00:31:52.789 won't go on the network. I won't bother with 00:31:52.789 --> 00:31:56.059 all that stuff. I'll just read it from the 00:31:56.059 --> 00:31:56.730 disc. 00:31:56.730 --> 00:31:59.159 This is what web browsers do all the time. 00:31:59.159 --> 00:32:01.440 We can do this all this time with, literally, 00:32:01.440 --> 00:32:04.530 a single line of code. Sorry. That's not a 00:32:04.530 --> 00:32:09.049 single line. It's a single method call. Would be 00:32:09.049 --> 00:32:10.340 a single line, I guess everything could be a 00:32:10.340 --> 00:32:14.110 single line of code, right. 00:32:14.110 --> 00:32:18.960 So, we've just said to our faraday stack, hey, 00:32:18.960 --> 00:32:21.030 add some caching middleware. Write it to disc at 00:32:21.030 --> 00:32:24.450 this location, and there we go. We, we've now 00:32:24.450 --> 00:32:27.000 got all the semantics of a web browser. And, 00:32:27.000 --> 00:32:28.750 we didn't have to do any work apart from 00:32:28.750 --> 00:32:33.460 this single method call. And what's really powerful is 00:32:33.460 --> 00:32:37.610 the team that knows the most about the data, 00:32:37.610 --> 00:32:40.289 so the product team in this case, can define 00:32:40.289 --> 00:32:43.299 the caching semantics of their data. 00:32:43.299 --> 00:32:45.950 So, if they realized, like, you know what, these, 00:32:45.950 --> 00:32:47.970 this, the name of the author keeps on changing. 00:32:47.970 --> 00:32:51.720 We need to drop down the caching freshness, they 00:32:51.720 --> 00:32:53.690 can just say, you know what, it's gonna be 00:32:53.690 --> 00:32:56.690 ten minutes. And we just get that semantics, we 00:32:56.690 --> 00:32:59.580 just start, we change our caching rules. No, no 00:32:59.580 --> 00:33:02.850 code change. No configuration change. No redeployment. It just 00:33:02.850 --> 00:33:03.539 happens. 00:33:03.539 --> 00:33:06.990 So, again, Conway's Law. The team that knows the 00:33:06.990 --> 00:33:10.309 most about the data has control over the caching 00:33:10.309 --> 00:33:15.580 of that data. Really, really, really powerful stuff. 00:33:15.580 --> 00:33:18.720 This goes back to this principle that, that I'm 00:33:18.720 --> 00:33:21.090 really passionate about. I think a lot of times 00:33:21.090 --> 00:33:22.559 we, we, we, we think of ourselves as building 00:33:22.559 --> 00:33:24.650 these systems on the web. We, we're building our 00:33:24.650 --> 00:33:28.240 stuff on top of http. I don't think we 00:33:28.240 --> 00:33:29.320 should be doing that. I think we should be 00:33:29.320 --> 00:33:32.130 building systems that are of the web. 00:33:32.130 --> 00:33:37.110 So, we should be using principles like, like REST, 00:33:37.110 --> 00:33:42.880 like hypermedia, like caching, http caching, because these, these 00:33:42.880 --> 00:33:46.400 are the principles that have made the most successful, 00:33:46.400 --> 00:33:49.539 biggest distributed system in the world work. It's really 00:33:49.539 --> 00:33:53.200 messy, and sometimes it's a bit of a pain 00:33:53.200 --> 00:33:56.080 to use, but it works really well, and it's 00:33:56.080 --> 00:33:58.850 all done for us already. We don't have to 00:33:58.850 --> 00:34:01.299 figure this stuff out ourselves. It's all out there 00:34:01.299 --> 00:34:03.700 and it's been working for decades. 00:34:03.700 --> 00:34:06.320 We use it, in some cases, hypermedia is something 00:34:06.320 --> 00:34:08.949 we're starting to understand more and more. We can 00:34:08.949 --> 00:34:10.829 use it in more places. Caching is, is an 00:34:10.829 --> 00:34:12.969 example of that. 00:34:12.969 --> 00:34:17.030 So, some principles. I could have talked about all 00:34:17.030 --> 00:34:18.690 this stuff for hours and hours and hours, but 00:34:18.690 --> 00:34:20.329 I don't have time to do that. So I'm 00:34:20.329 --> 00:34:22.359 just gonna shove a bunch of words at the 00:34:22.359 --> 00:34:26.210 end of this talk, and, and hope that, that 00:34:26.210 --> 00:34:27.679 if they're new words to you that you go 00:34:27.679 --> 00:34:29.929 out and, and research them. 00:34:29.929 --> 00:34:32.789 Conway's Law is, is the big one. This is 00:34:32.789 --> 00:34:36.429 something I'm really passionate about. If you, software, building 00:34:36.429 --> 00:34:38.690 software today is very much a team sport. If 00:34:38.690 --> 00:34:40.739 you don't understand how to work with other teams, 00:34:40.739 --> 00:34:42.549 then someone else is gonna do it better than 00:34:42.549 --> 00:34:45.139 you. So this is important stuff. 00:34:45.139 --> 00:34:47.909 Domain-Driven Design, I kind of touched on this talking 00:34:47.909 --> 00:34:51.559 about domain objects and talking about kind of bounded 00:34:51.559 --> 00:34:53.960 contexts. It's a really, really good book. It's really, 00:34:53.960 --> 00:34:57.660 really hard to read. Eric Evans is an incredibly 00:34:57.660 --> 00:35:00.200 smart guy who's not necessarily the best at kind 00:35:00.200 --> 00:35:02.969 of succinctly expressing ideas. 00:35:02.969 --> 00:35:04.769 If you do get this book, read the second 00:35:04.769 --> 00:35:07.529 half before you read the first half. He himself 00:35:07.529 --> 00:35:09.130 has said he got the order around wrong in 00:35:09.130 --> 00:35:11.170 retrospect. The, the second half has loads of really 00:35:11.170 --> 00:35:12.940 good stuff in it. The first half does too. 00:35:12.940 --> 00:35:14.930 But the second half, I think, is where the 00:35:14.930 --> 00:35:17.140 real, the real, real great stuff is. 00:35:17.140 --> 00:35:20.529 Postel's Law. I didn't really talk about this at 00:35:20.529 --> 00:35:24.359 all. But if you're building SOA systems, this is 00:35:24.359 --> 00:35:27.950 the way that you evolve these systems over time, 00:35:27.950 --> 00:35:31.089 without having to redeploy your entire enterprise at once. 00:35:31.089 --> 00:35:34.450 So, Postel's Law is very, very valuable. Another reason 00:35:34.450 --> 00:35:36.710 why the web has been so successful is Postel's 00:35:36.710 --> 00:35:41.349 Law. html is incredibly permissive. It draws, it causes 00:35:41.349 --> 00:35:42.960 a bunch of pain, but it also is the 00:35:42.960 --> 00:35:45.670 reason why html is the system that we still 00:35:45.670 --> 00:35:46.609 use today. 00:35:46.609 --> 00:35:50.150 Hexagonal architectures. Talked about this a fair bit. This 00:35:50.150 --> 00:35:52.160 is a really, really important principle. I'm really excited 00:35:52.160 --> 00:35:53.809 that the Rails community is starting to kind of 00:35:53.809 --> 00:35:55.779 get, to get excited about this. But it's been 00:35:55.779 --> 00:35:58.339 around for awhile. There's loads of really good talks 00:35:58.339 --> 00:36:01.130 out there about hexagonal architectures as they apply to 00:36:01.130 --> 00:36:02.099 Rails. There's been a couple of talks at the 00:36:02.099 --> 00:36:03.849 conference already. 00:36:03.849 --> 00:36:05.359 There's a, there's a Birds of a Feather session 00:36:05.359 --> 00:36:08.660 tonight about it, I believe. So there's loads of 00:36:08.660 --> 00:36:10.579 res- loads of good resources out there. Not just 00:36:10.579 --> 00:36:14.099 for Rails specifically, but just, hexagonal architectures in general. 00:36:14.099 --> 00:36:16.739 Because it is a general architectural principle. 00:36:16.739 --> 00:36:21.489 Finally, we should be embracing the web. It's been 00:36:21.489 --> 00:36:23.049 around for a long time. The patterns are very 00:36:23.049 --> 00:36:26.440 well understood, very well established. The tooling is amazingly 00:36:26.440 --> 00:36:30.160 powerful. We keep on reinventing wheels. Let's not keep 00:36:30.160 --> 00:36:32.880 reinventing this particular wheel. When it makes sense to 00:36:32.880 --> 00:36:34.680 make the web and the principles of the web, 00:36:34.680 --> 00:36:35.739 we should be doing that. 00:36:35.739 --> 00:36:37.099 Thank you.