WEBVTT 00:00:16.840 --> 00:00:22.540 JOSH SZMAJDA: All right. You guys ready to talk about SOA? Woo! 00:00:22.540 --> 00:00:29.540 I'm Josh. Hi. How you doing? I'm the CTO 00:00:30.849 --> 00:00:33.390 at Optoro. We make software for retailers. Help them 00:00:33.390 --> 00:00:36.470 deal with their stuff at scale. I'm working on 00:00:36.470 --> 00:00:41.070 a project called RACK-AMQP. RACK-AMQP is a collection of 00:00:41.070 --> 00:00:46.940 different projects, including an RC, some client libraries, and 00:00:46.940 --> 00:00:51.040 most notably, Jackalope, which is a new Ruby web 00:00:51.040 --> 00:00:55.579 application server, like Unicorn, Passenger, Thin, or, or anything 00:00:55.579 --> 00:00:58.899 like that. Except that it's special-purpose. It's for service-oriented 00:00:58.899 --> 00:01:03.510 architectures. And it doesn't actually speak HTTP at all. 00:01:03.510 --> 00:01:07.899 So. Why am I doing this crazy thing? Well, 00:01:07.899 --> 00:01:11.060 my app drove me to it. My app is 00:01:11.060 --> 00:01:14.110 huge. It's really complicated. This is kind of a 00:01:14.110 --> 00:01:16.070 services diagram of my app. Or where we'd like 00:01:16.070 --> 00:01:18.220 to go with it. 00:01:18.220 --> 00:01:21.100 There's over three-hundred and fifty models in one Rails 00:01:21.100 --> 00:01:25.380 application. The test suite takes over eight hours. It's 00:01:25.380 --> 00:01:29.899 kind of a headache. You could put it. So 00:01:29.899 --> 00:01:31.479 we decided the way to move forward with it 00:01:31.479 --> 00:01:34.110 is to break it up into smaller applications that 00:01:34.110 --> 00:01:37.380 work together. And, of course, that means SOA. 00:01:37.380 --> 00:01:40.810 Now, I mean, SOA is really wrapped up in 00:01:40.810 --> 00:01:45.270 a lot of pretty awful baggage, when you come 00:01:45.270 --> 00:01:48.670 down to it. SOA, you think about enterprise service 00:01:48.670 --> 00:01:53.310 buses. You think about strict interface definitions. Versions. I 00:01:53.310 --> 00:01:55.399 mean it's a lot of complexity that you have 00:01:55.399 --> 00:01:59.569 to worry about. Extra load balancers. More configuration. More 00:01:59.569 --> 00:02:02.410 distributed tracing and monitoring. I mean, it's a, it's 00:02:02.410 --> 00:02:04.280 a real pain. 00:02:04.280 --> 00:02:06.890 So I, I hate SOA, right. It makes me 00:02:06.890 --> 00:02:12.140 feel like this. So, but when you get down 00:02:12.140 --> 00:02:14.690 to it, SOA is actually not all of that. 00:02:14.690 --> 00:02:19.770 SOA is really just independent actors communicating. They're working 00:02:19.770 --> 00:02:25.310 together to get something useful done. So, really, SOA 00:02:25.310 --> 00:02:29.790 can mitigate complexity by enforcing simplification. Each service only 00:02:29.790 --> 00:02:32.540 has to think about its own little bubble. It 00:02:32.540 --> 00:02:34.180 only has to worry about its little component. And 00:02:34.180 --> 00:02:35.450 it doesn't have to think about anything else that 00:02:35.450 --> 00:02:38.019 goes on in its entire system. 00:02:38.019 --> 00:02:39.819 You can let your system architect worry about what's 00:02:39.819 --> 00:02:40.969 going on across the whole thing. All you have 00:02:40.969 --> 00:02:42.079 to do is pay attention to your one little 00:02:42.079 --> 00:02:45.340 service and you can be pretty happy. 00:02:45.340 --> 00:02:48.159 So, when you get down to it again, SOA 00:02:48.159 --> 00:02:51.569 is really just independent actors communicating. Now that communicating 00:02:51.569 --> 00:02:54.219 bit is important. How they communicate actually determines how 00:02:54.219 --> 00:02:57.709 your application behaves. And the kinds of communication you 00:02:57.709 --> 00:03:00.739 can use impacts the kind of application you can 00:03:00.739 --> 00:03:01.859 write. 00:03:01.859 --> 00:03:05.010 So, the typical communication pattern that we're most familiar 00:03:05.010 --> 00:03:08.140 with as programmers is sort of direct messaging. I 00:03:08.140 --> 00:03:10.650 am an object. I can send a message to 00:03:10.650 --> 00:03:13.409 my friend. My friend is gonna give me back 00:03:13.409 --> 00:03:15.680 a response. It's very normal. It's what, in fact, 00:03:15.680 --> 00:03:18.849 we use all the time with HTTP. But when 00:03:18.849 --> 00:03:20.159 you think about it, what we actually want to 00:03:20.159 --> 00:03:21.709 do is be able to send different kinds of 00:03:21.709 --> 00:03:24.790 messaging structures. We want to publish that something happened 00:03:24.790 --> 00:03:27.290 in my system and have people subscribe to that 00:03:27.290 --> 00:03:30.469 event and take some interesting action. 00:03:30.469 --> 00:03:31.980 It's all about decoupling. I don't have to know 00:03:31.980 --> 00:03:33.650 about my neighbor anymore. I just need to know 00:03:33.650 --> 00:03:36.180 that I did something, and then somebody else might 00:03:36.180 --> 00:03:37.879 care that, oh, you did something. That's great. Let 00:03:37.879 --> 00:03:38.799 me do something about that. 00:03:38.799 --> 00:03:41.620 But again, like I said, let me. We're Rails 00:03:41.620 --> 00:03:44.780 developers. We're used to writing web applications. We speak 00:03:44.780 --> 00:03:47.859 HTTP all the time. HTTP allows us to do 00:03:47.859 --> 00:03:50.310 direct messaging, so that's what we tend to do. 00:03:50.310 --> 00:03:52.340 I'd say most SOAs out there, when they get 00:03:52.340 --> 00:03:54.480 off the ground, they start building web applications that 00:03:54.480 --> 00:03:56.889 speak with each other through HTTP. It's what we're 00:03:56.889 --> 00:03:58.430 familiar with. 00:03:58.430 --> 00:04:01.120 But what would we actually like our transport mechanism 00:04:01.120 --> 00:04:05.230 to be? We want it to be, you know, 00:04:05.230 --> 00:04:08.920 data structure independent. HTTP specifies that there be a 00:04:08.920 --> 00:04:11.329 body, but it doesn't specify the format of the 00:04:11.329 --> 00:04:13.170 body. We want it to be fast. We want 00:04:13.170 --> 00:04:17.690 it to be open, transparent, understandable, clear. You know, 00:04:17.690 --> 00:04:20.279 debuggable. Observable. And flexible. 00:04:20.279 --> 00:04:25.830 So, is HTTP all of those things. We also 00:04:25.830 --> 00:04:30.080 need, not only direct request response type of messaging, 00:04:30.080 --> 00:04:33.979 like RPC, but we also need pub, subscribe, and 00:04:33.979 --> 00:04:38.180 broadcast. Additionally, we don't need more load balancers. More 00:04:38.180 --> 00:04:43.120 SSL certificates. Distributed logging. Trying to figure out what's 00:04:43.120 --> 00:04:44.860 going on in this service over here versus that 00:04:44.860 --> 00:04:47.210 one over there. It's a nightmare. 00:04:47.210 --> 00:04:50.150 And, you know, looking up the various services in 00:04:50.150 --> 00:04:51.710 your system, it's a pain. So, again, I mean, 00:04:51.710 --> 00:04:54.740 HTTP, you know, it has, it is blind. It's 00:04:54.740 --> 00:04:57.669 fast. It's open, clear, it's well-understood. It's what we 00:04:57.669 --> 00:05:01.680 know. It's really great for RPC. It's a better 00:05:01.680 --> 00:05:02.500 alternative. 00:05:02.500 --> 00:05:07.660 And that's AMQP. AMQP is blind, also. It doesn't 00:05:07.660 --> 00:05:10.220 care what data you're transmitting across it. It's extremely 00:05:10.220 --> 00:05:12.860 fast. It's an open system. It's great for all 00:05:12.860 --> 00:05:16.500 messaging pattern. Not just direct messaging, but pub/sub, and 00:05:16.500 --> 00:05:21.060 queue-based messaging. It's centralized, which in terms of a 00:05:21.060 --> 00:05:23.650 service-oriented architecture is very beneficial, in that you have 00:05:23.650 --> 00:05:26.120 one point to manage, one point to pay attention 00:05:26.120 --> 00:05:28.710 to everything at. And the only issue is that 00:05:28.710 --> 00:05:30.240 it's not what we're familiar with. It's not what 00:05:30.240 --> 00:05:32.030 we know. You know, again, we're web developers. We're 00:05:32.030 --> 00:05:34.280 not AMQP developers. At least not the most of 00:05:34.280 --> 00:05:34.919 us. 00:05:34.919 --> 00:05:36.500 So it's a challenge. 00:05:36.500 --> 00:05:40.020 So the RACK-AMQP project that I'm working on is 00:05:40.020 --> 00:05:44.280 really designed to make AMQP simple. We are trying 00:05:44.280 --> 00:05:47.180 to introduce as few opinions as absolutely necessary in 00:05:47.180 --> 00:05:49.509 order to make this possible. And to give you 00:05:49.509 --> 00:05:51.490 a stepping stone into the world of AMQP where 00:05:51.490 --> 00:05:54.340 you can start to integrate more advanced concepts into 00:05:54.340 --> 00:06:00.539 the system, without having to relearn your entire world. 00:06:00.539 --> 00:06:03.610 So it also lets us continue to build what 00:06:03.610 --> 00:06:06.590 we know. Because it's built on Rack, you know, 00:06:06.590 --> 00:06:09.580 we're web-developers. We can continue to use the web 00:06:09.580 --> 00:06:11.810 concepts that we're familiar with. We can developer locally 00:06:11.810 --> 00:06:13.569 on our web machines using the systems that we're 00:06:13.569 --> 00:06:15.879 comfortable with. And then when we deploy, we simply 00:06:15.879 --> 00:06:18.440 deploy it to Jackalope, instead of to Unicorn or 00:06:18.440 --> 00:06:22.259 Passenger or whatever, and it's magically in this world 00:06:22.259 --> 00:06:24.020 of AMQP. 00:06:24.020 --> 00:06:27.770 So. How does that work? Well, let's look at 00:06:27.770 --> 00:06:29.220 AMQP a little bit more in depth, first. So 00:06:29.220 --> 00:06:32.240 AMQP is the advanced messaging queue, message queueing protocol. 00:06:32.240 --> 00:06:34.490 It is an open standard. It's developed by Oasis, 00:06:34.490 --> 00:06:37.690 which is an open standards body. It, again, like 00:06:37.690 --> 00:06:41.610 I said, uses a centralized distribution pattern. That centralized 00:06:41.610 --> 00:06:44.400 distribution pattern allows you to do the various kinds 00:06:44.400 --> 00:06:46.659 of messaging that are really handy for us. The 00:06:46.659 --> 00:06:49.750 pub/sub and the other interesting thing. This implies that 00:06:49.750 --> 00:06:51.889 we have a broker in the center that all 00:06:51.889 --> 00:06:54.509 of our services connect to and transmit messages through 00:06:54.509 --> 00:06:56.610 that broker. And that broker can make intelligent decisions 00:06:56.610 --> 00:06:58.930 about how to route those messages. 00:06:58.930 --> 00:07:02.569 AMQP messages themselves, they have headers and bodies, just 00:07:02.569 --> 00:07:06.000 like HTTP. There are well-known fields, like content type 00:07:06.000 --> 00:07:11.939 and priority, in this case. And we can actually 00:07:11.939 --> 00:07:14.830 leverage the conventions that we understand from HTTP to 00:07:14.830 --> 00:07:18.939 pull AMQP into the world. So AMQP, like I 00:07:18.939 --> 00:07:21.009 mentioned, has these headers and bodies. It has well-known 00:07:21.009 --> 00:07:23.819 fields. And HTTP essentially has headers and bodies and 00:07:23.819 --> 00:07:26.780 also well-known fields. So we can essentially emulate HTTP 00:07:26.780 --> 00:07:30.250 over AMQP without too much trouble. And, again, the 00:07:30.250 --> 00:07:32.460 AMQP supports the RPC messaging pattern, which is what 00:07:32.460 --> 00:07:34.240 HTTP is. 00:07:34.240 --> 00:07:36.860 So, one of the parts of the project is 00:07:36.860 --> 00:07:41.610 the AMQP HTTP RC, which, it simply does as 00:07:41.610 --> 00:07:44.569 little as it has to to define the structure. 00:07:44.569 --> 00:07:46.060 It's just a documentation on how we're doing the 00:07:46.060 --> 00:07:48.240 mapping. One of the other goals is that we 00:07:48.240 --> 00:07:50.470 want to make inner-operability with other languages and other 00:07:50.470 --> 00:07:54.150 platforms really easy. So having an RC will allow 00:07:54.150 --> 00:07:55.720 it, allow us to have just a central document 00:07:55.720 --> 00:08:00.229 to refer back to in constructing other servers. 00:08:00.229 --> 00:08:01.569 So let's look at HTTP a little bit more 00:08:01.569 --> 00:08:03.270 in-depth. How does HTTP work? Well, you know, when 00:08:03.270 --> 00:08:06.639 we send a request, we send a specific request 00:08:06.639 --> 00:08:09.310 to a server, to a, you know, specific IP 00:08:09.310 --> 00:08:11.710 address and port. And we get back a response. 00:08:11.710 --> 00:08:15.009 And, of course, the request has things like the 00:08:15.009 --> 00:08:19.409 HTTP verb, the path, headers, the bodies. And the 00:08:19.409 --> 00:08:21.219 response comes back with a response code. 00:08:21.219 --> 00:08:23.539 It's all stuff we're familiar with. So all we 00:08:23.539 --> 00:08:26.500 had to do is map that into AMQP terms, 00:08:26.500 --> 00:08:28.490 which is essentially what we've done here. And this 00:08:28.490 --> 00:08:29.699 is just a reference for just a few of 00:08:29.699 --> 00:08:32.090 the things that we're doing. Like content type. There's 00:08:32.090 --> 00:08:34.130 already a content type header in AMQP, so we 00:08:34.130 --> 00:08:36.780 just reuse that. Some things didn't make sense. Like 00:08:36.780 --> 00:08:41.049 the protocol for example. In HTTP you've got HTP 00:08:41.049 --> 00:08:45.940 or HTPS or HTTP plus dev or whatever. But 00:08:45.940 --> 00:08:48.790 with AMQP, your protocol's already negotiated when you connect 00:08:48.790 --> 00:08:50.330 to the broker. So it doesn't really make sense 00:08:50.330 --> 00:08:51.890 for us to worry about that in the messages 00:08:51.890 --> 00:08:55.200 we're passing around the system. 00:08:55.200 --> 00:08:57.490 The host name becomes, essentially, the routing key. The 00:08:57.490 --> 00:08:59.930 queue target that we're sending to in AMQP. And 00:08:59.930 --> 00:09:03.290 it makes it really simple. So, this looks like 00:09:03.290 --> 00:09:07.050 this, essentially, in code. This is an example of 00:09:07.050 --> 00:09:09.910 how you could write a client that speaks AMQP, 00:09:09.910 --> 00:09:12.830 HTTP. You create a call-back queue. We'll get back 00:09:12.830 --> 00:09:15.220 to that in one sec. You publish a message 00:09:15.220 --> 00:09:17.570 that looks like an HTTP message to the target 00:09:17.570 --> 00:09:19.760 queue. And then you wait for a response. 00:09:19.760 --> 00:09:21.710 Now, the callback queue is actually a really important 00:09:21.710 --> 00:09:26.270 thing here. AMQP is a hundred percent queue based. 00:09:26.270 --> 00:09:29.140 It's all asynchronous under the hood. But, you know, 00:09:29.140 --> 00:09:31.710 we want to write a synchronous system. HTTP is 00:09:31.710 --> 00:09:34.080 synchronous. I send a request. I wait for the 00:09:34.080 --> 00:09:37.280 response, or receive the response and move on. So, 00:09:37.280 --> 00:09:41.090 to do that in AMQP, we create a respond 00:09:41.090 --> 00:09:43.330 queue that we are going to listen to in 00:09:43.330 --> 00:09:46.680 the broker. It's an anonymous queue. There's a great 00:09:46.680 --> 00:09:49.280 convention. We're doing this in AMQP. It's very easy. 00:09:49.280 --> 00:09:50.630 So we create a response queue, we get the 00:09:50.630 --> 00:09:52.000 name of that queue, and then as part of 00:09:52.000 --> 00:09:53.950 the message we send to our target, we say, 00:09:53.950 --> 00:09:56.430 here's the response queue to reply back at. And 00:09:56.430 --> 00:09:58.350 then we simply wait for the response on that 00:09:58.350 --> 00:10:00.250 call-back queue to come back. So that lets us 00:10:00.250 --> 00:10:02.710 get the synchrony that we're familiar with, that we 00:10:02.710 --> 00:10:06.160 need, with HTTP style programming, while still having an 00:10:06.160 --> 00:10:09.130 asynchronous system under the hood. 00:10:09.130 --> 00:10:13.890 RACK, of course, we're all using RACK today. Even 00:10:13.890 --> 00:10:15.810 if you're not familiar with it. RACK, of course, 00:10:15.810 --> 00:10:18.190 is what runs all Ruby web application servers. Turns 00:10:18.190 --> 00:10:21.120 out that RACK actually doesn't care about HTTP. All 00:10:21.120 --> 00:10:23.730 that RACK really defines is that you had defined 00:10:23.730 --> 00:10:26.870 a call method that receives an environment, which is 00:10:26.870 --> 00:10:29.920 basically a hash, and it responds back with a 00:10:29.920 --> 00:10:34.080 three-element array. The HTTP response code, headers, and the 00:10:34.080 --> 00:10:34.770 body. 00:10:34.770 --> 00:10:37.230 That hash doesn't actually have to be anything but 00:10:37.230 --> 00:10:39.650 a hash. It just has to look like an 00:10:39.650 --> 00:10:42.850 HTTP environment. So emulation was actually pretty simple. So 00:10:42.850 --> 00:10:45.730 this is basically how it looks. We subscribe to 00:10:45.730 --> 00:10:48.070 a queue on the server, and then for every 00:10:48.070 --> 00:10:51.290 message we receive back for that queue, we unpack 00:10:51.290 --> 00:10:53.910 some things into variables. We create what looks like 00:10:53.910 --> 00:10:56.910 the HTTP environment and we have the request method 00:10:56.910 --> 00:10:58.380 and query string. If you ever wrote CGI way 00:10:58.380 --> 00:11:00.290 back in the day, this might look familiar to 00:11:00.290 --> 00:11:01.900 you. 00:11:01.900 --> 00:11:04.000 And then we essentially pass that environment onto our 00:11:04.000 --> 00:11:06.700 RACK application, receive the response from the RACK app 00:11:06.700 --> 00:11:09.910 and then publish that back to the response queue. 00:11:09.910 --> 00:11:11.850 So this is what Jackalope is doing for us. 00:11:11.850 --> 00:11:15.770 Jackalope is emulating HTTP for our RACK applications. And 00:11:15.770 --> 00:11:18.540 it just works. I'll show you a demo at 00:11:18.540 --> 00:11:20.370 the end. You don't have to modify your code 00:11:20.370 --> 00:11:22.560 at all. You can just simply deploy to it, 00:11:22.560 --> 00:11:26.110 and it's speaking AMQP instead of HTTP. So that's 00:11:26.110 --> 00:11:28.730 it. We're done. We can go on vacation, right? 00:11:28.730 --> 00:11:31.860 Well. I didn't talk about how to actually put 00:11:31.860 --> 00:11:36.930 this in production, other than just use Jackalope. So 00:11:36.930 --> 00:11:40.510 you have to choose a broker in AMQP. So 00:11:40.510 --> 00:11:43.220 what we're using is RabbitMQ. RabbitMQ is a well-known 00:11:43.220 --> 00:11:46.390 AMQP broker. If you're doing any AMQP you're probably 00:11:46.390 --> 00:11:50.770 using RabbitMQ, most likely. It's used by giants out 00:11:50.770 --> 00:11:53.320 there. I mean, Google, VMWare, NASA. All these people 00:11:53.320 --> 00:11:54.730 are using Rabbit pretty heavily. 00:11:54.730 --> 00:12:00.110 It's extremely scalable. It's fault-tolerant. It's distributable. And it's 00:12:00.110 --> 00:12:02.570 secure. It does everything you'd ever want. And it 00:12:02.570 --> 00:12:05.100 also gives you a really great management console. We 00:12:05.100 --> 00:12:06.300 can go in and see what's going on in 00:12:06.300 --> 00:12:08.370 your system. Like I mentioned, the distributed logging we 00:12:08.370 --> 00:12:09.700 saw in the, in the last talk, if you 00:12:09.700 --> 00:12:11.270 were here for that, trying to get an idea 00:12:11.270 --> 00:12:12.880 of what's going on across your system can be 00:12:12.880 --> 00:12:15.630 challenging. Rabbit doesn't tell you everything you'd want to 00:12:15.630 --> 00:12:17.180 know, but it at least gives you an idea 00:12:17.180 --> 00:12:19.660 of what queues are being used. Their depths at 00:12:19.660 --> 00:12:22.080 the point. The kind of information that can help 00:12:22.080 --> 00:12:23.880 you get, at least get a handle on it. 00:12:23.880 --> 00:12:26.980 Additionally, you're gonna have to think about how you 00:12:26.980 --> 00:12:28.550 talk to the real world. I mean, AMQP is 00:12:28.550 --> 00:12:30.610 all well and good behind the scenes, but how 00:12:30.610 --> 00:12:33.290 do I continue to interact and serve my clients 00:12:33.290 --> 00:12:35.260 that I actually care about, to the point of 00:12:35.260 --> 00:12:36.110 getting paid, right. 00:12:36.110 --> 00:12:39.070 So this is the architecture we typically use at 00:12:39.070 --> 00:12:41.470 Optoro. We have our Rabbit server in the middle, 00:12:41.470 --> 00:12:43.790 and all the various services sitting on Jackalope that 00:12:43.790 --> 00:12:45.850 talk to Rabbit. And then additionally we have one 00:12:45.850 --> 00:12:48.320 or more API services that are continuing to be 00:12:48.320 --> 00:12:50.960 deployed on Unicorn, in our case. They talk to 00:12:50.960 --> 00:12:54.820 the outside world and internally translate the, the needs 00:12:54.820 --> 00:12:57.740 through AMQP and the rest of the system. 00:12:57.740 --> 00:12:58.920 One of the things we actually published for each 00:12:58.920 --> 00:13:01.600 of our services is a client gem that kind 00:13:01.600 --> 00:13:04.600 of isolates the worry, the worry about exactly which 00:13:04.600 --> 00:13:07.710 transport we're using away from the consumer, the API 00:13:07.710 --> 00:13:10.890 here. So it actually is, is really simple. 00:13:10.890 --> 00:13:12.850 And so the way that we do that is 00:13:12.850 --> 00:13:15.860 we typically have been using HTTParty for our HTTP 00:13:15.860 --> 00:13:18.760 communication needs. So we wrote, as one of the 00:13:18.760 --> 00:13:22.940 other projects part of RACK-AMQP, is AMQParty, which is 00:13:22.940 --> 00:13:26.790 just a API-compliant version of HTTParty. So we can 00:13:26.790 --> 00:13:30.620 actually just drop in the AMQParty constant in place 00:13:30.620 --> 00:13:32.800 of the HTTParty constant, and you can see down 00:13:32.800 --> 00:13:35.610 below, that transport variable, we typically configure that at 00:13:35.610 --> 00:13:39.310 boot time to be either HTTParty or AMQParty. And 00:13:39.310 --> 00:13:40.920 everything, again, just works. 00:13:40.920 --> 00:13:43.100 We change the, the url a little bit, too. 00:13:43.100 --> 00:13:44.670 But we do that all part of the, part 00:13:44.670 --> 00:13:47.430 of this setup for each server. For each client. 00:13:47.430 --> 00:13:51.620 Of course, if you don't want to use HTTParty, 00:13:51.620 --> 00:13:54.000 some people like the various other options that are 00:13:54.000 --> 00:13:57.000 out there, we also are publishing RACK-AMQP client, which 00:13:57.000 --> 00:13:59.600 is what H, AMQParty is built on. It, it's 00:13:59.600 --> 00:14:03.040 a very simple API. You get a client, you 00:14:03.040 --> 00:14:06.920 tell it which Rabbit to connect to. You send 00:14:06.920 --> 00:14:09.910 a request to your target queue with your uri 00:14:09.910 --> 00:14:12.410 as part of it, with any, you know, your 00:14:12.410 --> 00:14:13.680 HTTP method and all that. You get back your 00:14:13.680 --> 00:14:16.190 response. It's synchronous. It's simple. It's actually built on 00:14:16.190 --> 00:14:18.670 Bunny, which is a really great AMQP gem out 00:14:18.670 --> 00:14:21.780 there, and that also is very easy to use. 00:14:21.780 --> 00:14:27.540 Additionally, we're publishing a sample SOA using Rails. It's 00:14:27.540 --> 00:14:32.790 a work-in-progress at the moment. BUt the userland and 00:14:32.790 --> 00:14:34.430 userland_client are mostly built as, you can see where 00:14:34.430 --> 00:14:37.120 things are at the mo- at the current state. 00:14:37.120 --> 00:14:38.600 It kind of gives you an idea about how 00:14:38.600 --> 00:14:41.510 we think about SOA at Optoro, and how you 00:14:41.510 --> 00:14:45.350 might be able to use Jackalope and the RACK-AMQP 00:14:45.350 --> 00:14:47.050 project in your own projects. 00:14:47.050 --> 00:14:52.190 So userland is a, is Rails service that essentially 00:14:52.190 --> 00:14:54.560 publishes a user's concept out into the world. And 00:14:54.560 --> 00:14:56.790 then userland_client is a gem that consumes the userland 00:14:56.790 --> 00:15:00.260 service and interface uses the userland gem to talk 00:15:00.260 --> 00:15:02.100 to the userland service. 00:15:02.100 --> 00:15:07.330 And so then, you know, how fast is this? 00:15:07.330 --> 00:15:11.060 Well, there's this bench mark I've got. I'll show 00:15:11.060 --> 00:15:13.480 you it in just a sec. The only weird 00:15:13.480 --> 00:15:15.020 thing we have to do here is tell AMQParty 00:15:15.020 --> 00:15:16.320 which Rabbit to talk to, and you kind of 00:15:16.320 --> 00:15:18.040 saw that in the, in the client gem a 00:15:18.040 --> 00:15:19.710 little bit, so that's a little bit of setup 00:15:19.710 --> 00:15:21.290 we do. Again, like I said, when we make 00:15:21.290 --> 00:15:24.300 the decision about which, whether HTTParty or AMQParty. So 00:15:24.300 --> 00:15:26.270 we tell it which Rabbit. And then, inside, we, 00:15:26.270 --> 00:15:30.620 five hundred times, request this JSON using AMQParty and 00:15:30.620 --> 00:15:33.170 then five hundred times request the same thing using 00:15:33.170 --> 00:15:33.450 HTTParty. 00:15:33.450 --> 00:15:37.450 So here, I'll show you that in, in use. 00:15:37.450 --> 00:15:41.360 Let's see. OK. 00:15:41.360 --> 00:15:48.360 Let's make this big. All right. So here I'm 00:15:49.390 --> 00:15:53.080 gonna boot the app using Unicorn. So Unicorn is 00:15:53.080 --> 00:15:57.080 listing on port 8080. Wow. That's really big. Trust 00:15:57.080 --> 00:15:59.550 me, it's 8080 there. I'm gonna boot the same 00:15:59.550 --> 00:16:02.100 map using Jackalope. The only difference I have to 00:16:02.100 --> 00:16:03.280 do to boot the app is I have to 00:16:03.280 --> 00:16:05.330 say with the right queue name, so it's going 00:16:05.330 --> 00:16:08.120 to listen to the test dot simple queue. 00:16:08.120 --> 00:16:10.420 So there it is running. And then down here 00:16:10.420 --> 00:16:13.170 I'm gonna starty the bench mark. And like I 00:16:13.170 --> 00:16:16.390 said, it's gonna five hundred times hit each service. 00:16:16.390 --> 00:16:18.630 What it actually does is hit each service three 00:16:18.630 --> 00:16:20.380 times, just to make sure that it's warm, and 00:16:20.380 --> 00:16:22.730 then it can go through and hit it all 00:16:22.730 --> 00:16:24.650 the way through. And there we go. 00:16:24.650 --> 00:16:26.940 So you can see, it's two point nine seconds 00:16:26.940 --> 00:16:30.050 total to do five hundred AMQP requests, and three 00:16:30.050 --> 00:16:33.300 point six seconds total to do five hundred HTTP 00:16:33.300 --> 00:16:37.040 requests. I was actually surprised when I got this 00:16:37.040 --> 00:16:40.750 result. Jackalope is beta. I mean, it's, we're using 00:16:40.750 --> 00:16:43.930 it at Optoro, but it's not really been battle-tested. 00:16:43.930 --> 00:16:46.950 But it's faster, at the moment. So why is 00:16:46.950 --> 00:16:47.730 it faster? 00:16:47.730 --> 00:16:51.670 Well, maybe I haven't written all the code yet. 00:16:51.670 --> 00:16:55.530 But more likely it, more likely it's because of 00:16:55.530 --> 00:16:59.800 this concept in TCP called the slow-start phenomonon. And 00:16:59.800 --> 00:17:02.630 also, your whole TCP negotiation. So each one of 00:17:02.630 --> 00:17:06.699 these five hundred HTTP requests, close the connection, open 00:17:06.699 --> 00:17:10.770 the connection, send the message, receive the response. When 00:17:10.770 --> 00:17:12.799 you open the TCP connection you go through a 00:17:12.799 --> 00:17:14.970 whole sinaq phase, which I didn't put slides in 00:17:14.970 --> 00:17:17.309 for and maybe should have. But it's. 00:17:17.309 --> 00:17:20.630 I highly recommend checking out Eliot Gregorich's talks, by 00:17:20.630 --> 00:17:23.679 the way, about how HTTP works, how TCP works. 00:17:23.679 --> 00:17:27.069 But basically, you go through protocol negotiation for any 00:17:27.069 --> 00:17:32.279 TCP connection, and then there's a feature of TCP 00:17:32.279 --> 00:17:34.350 called the slow-start feature, which essentially allows you to 00:17:34.350 --> 00:17:37.639 negotiate your connection speed. And that's really great when 00:17:37.639 --> 00:17:39.600 you're talking over the internet, and you're not sure 00:17:39.600 --> 00:17:42.090 of the latency or the availability of routers to 00:17:42.090 --> 00:17:44.700 be decent along the way. But when you're doing 00:17:44.700 --> 00:17:47.720 internal applications, it just gets in your way. It 00:17:47.720 --> 00:17:48.519 just slows you down. 00:17:48.519 --> 00:17:52.370 So AMQP, we hold a persistent connection to the 00:17:52.370 --> 00:17:55.350 AMQP broker. We do one sinaq cycle, and then 00:17:55.350 --> 00:17:57.990 we get our maximum connection speed and we stay 00:17:57.990 --> 00:18:00.309 there. And then we can multiplex information over that 00:18:00.309 --> 00:18:03.179 single connection and it works really well. Now, you 00:18:03.179 --> 00:18:05.070 could write that with HTTP if you did a 00:18:05.070 --> 00:18:08.690 keep-alive connection, held over that connection and talk to 00:18:08.690 --> 00:18:09.559 the same thing back and forth. 00:18:09.559 --> 00:18:11.279 But, the bottom line is that most of us 00:18:11.279 --> 00:18:13.379 don't do that when we write HTTP. We're using 00:18:13.379 --> 00:18:17.919 something like HTTParty or, I always forget the tiforce 00:18:17.919 --> 00:18:21.000 or tif- whatever it is. Sorry. 00:18:21.000 --> 00:18:21.809 Most of the time, you know, we drop the 00:18:21.809 --> 00:18:23.740 connection, you know, in between requests. We don't hold 00:18:23.740 --> 00:18:25.620 it open. It's just not something we think about. 00:18:25.620 --> 00:18:29.860 So using AMQParty or any of the libraries that 00:18:29.860 --> 00:18:32.360 we're published, it holds that open and it just 00:18:32.360 --> 00:18:34.539 makes it more efficient overall. 00:18:34.539 --> 00:18:36.309 So I hope that's the primary explanation of why 00:18:36.309 --> 00:18:43.309 it's so much faster. So I just wanted to 00:18:44.340 --> 00:18:45.929 mention a few references, a few things that were 00:18:45.929 --> 00:18:47.870 really inspriring when we kind of went down this 00:18:47.870 --> 00:18:49.879 road. You know, this is really a departure from 00:18:49.879 --> 00:18:53.340 what we're used to in Rails. And the Architecture: 00:18:53.340 --> 00:18:55.970 The Lost Years talk and Matt Wynne's original Hexegonal 00:18:55.970 --> 00:18:58.320 Rails talks were all about departing from what we're 00:18:58.320 --> 00:19:00.429 used to, I think. So those are really interesting. 00:19:00.429 --> 00:19:02.909 If you haven't checked those out, definitely do. 00:19:02.909 --> 00:19:05.340 Additionally, Martin Fowler posted a recent article that was 00:19:05.340 --> 00:19:08.450 awesome, called Micro Services. Definitely, definitely read this if 00:19:08.450 --> 00:19:12.190 you are looking into service-oriented architectures. It lets you 00:19:12.190 --> 00:19:14.789 not think about that heavy-weight bloat that we think 00:19:14.789 --> 00:19:16.649 about with SOA. 00:19:16.649 --> 00:19:19.309 It's really a revisitation of the idea of, just, 00:19:19.309 --> 00:19:22.610 you have independent services that are collaborating to get 00:19:22.610 --> 00:19:24.820 something done in a very simple way. And that's 00:19:24.820 --> 00:19:27.529 really, again, also influenced the direction we're taking here. 00:19:27.529 --> 00:19:30.899 We want to introduce as few specific choices as 00:19:30.899 --> 00:19:33.090 possible to just kind of give us a simple 00:19:33.090 --> 00:19:36.679 transport to, to build on top of. Additionally, the 00:19:36.679 --> 00:19:39.529 Ruby AMQP gem has some awesome docs about how 00:19:39.529 --> 00:19:42.529 AMQP works, how to use it with Ruby. And 00:19:42.529 --> 00:19:45.509 RabbitMQ also publishes some really great documentation on what 00:19:45.509 --> 00:19:47.620 AMQP is, how it works. How to use it 00:19:47.620 --> 00:19:49.600 in the organization in your applications. 00:19:49.600 --> 00:19:51.649 And there's also this really great article I found 00:19:51.649 --> 00:19:55.029 about how HTTP works, in general. It's, I think 00:19:55.029 --> 00:19:58.429 a piece of school material from a university in 00:19:58.429 --> 00:19:59.960 Singapore. Definitely check it out. It's got really great 00:19:59.960 --> 00:20:00.730 diagrams. 00:20:00.730 --> 00:20:06.750 Thanks. Again, I'm Josh. I'm at jszmajda on the 00:20:06.750 --> 00:20:09.730 Twitters. I am the CTO at Optoro. We have 00:20:09.730 --> 00:20:12.720 blinq dot com. We have retailers with their returned 00:20:12.720 --> 00:20:14.710 goods and excess goods, help them figure out what 00:20:14.710 --> 00:20:16.809 it is, what the current value is and move 00:20:16.809 --> 00:20:18.070 it quickly. 00:20:18.070 --> 00:20:20.129 Additionally, I am the host of the Ruby hangout. 00:20:20.129 --> 00:20:22.480 Just a quick blurb for the Ruby hangout, it's 00:20:22.480 --> 00:20:25.220 a online meetup. It's really handy when you can 00:20:25.220 --> 00:20:27.850 go face-to-face and talk to people if you have 00:20:27.850 --> 00:20:28.960 a local meet up. If you don't have a 00:20:28.960 --> 00:20:30.629 local meet up or if you're too busy or 00:20:30.629 --> 00:20:32.509 if you can't make it for whatever reason, the 00:20:32.509 --> 00:20:35.570 Ruby hangout is an online meet up to, essentially, 00:20:35.570 --> 00:20:38.249 give you that local meet up feel, as much 00:20:38.249 --> 00:20:40.059 as we can online. So check that out. It's 00:20:40.059 --> 00:20:41.899 a lot of fun. I'm one of the co-organizers 00:20:41.899 --> 00:20:44.110 of the DC Ruby Users group. And I want 00:20:44.110 --> 00:20:46.399 to give a quick shoutout to Jonathan. Thanks Jon. 00:20:46.399 --> 00:20:49.559 He's on my team, helped me get Jackalope over 00:20:49.559 --> 00:20:52.799 the, over the finish line. So super helpful on 00:20:52.799 --> 00:20:53.279 that. 00:20:53.279 --> 00:20:55.649 All the code's on GitHub. Github dot com slash 00:20:55.649 --> 00:20:59.419 rack dash amqp slash rack dash amqp is where 00:20:59.419 --> 00:21:01.600 the RC lives and has bookmarks to the other 00:21:01.600 --> 00:21:06.320 parts of the project. And I would love to 00:21:06.320 --> 00:21:10.299 take your questions, because that talk was way short. 00:21:10.299 --> 00:21:12.840 And I've got, actually, a whole bunch of stuffed 00:21:12.840 --> 00:21:16.100 Jackalopes up here for good questions. So please feel 00:21:16.100 --> 00:21:18.300 free to come grab one.