1 00:00:16,840 --> 00:00:22,540 JOSH SZMAJDA: All right. You guys ready to talk about SOA? Woo! 2 00:00:22,540 --> 00:00:29,540 I'm Josh. Hi. How you doing? I'm the CTO 3 00:00:30,849 --> 00:00:33,390 at Optoro. We make software for retailers. Help them 4 00:00:33,390 --> 00:00:36,470 deal with their stuff at scale. I'm working on 5 00:00:36,470 --> 00:00:41,070 a project called RACK-AMQP. RACK-AMQP is a collection of 6 00:00:41,070 --> 00:00:46,940 different projects, including an RC, some client libraries, and 7 00:00:46,940 --> 00:00:51,040 most notably, Jackalope, which is a new Ruby web 8 00:00:51,040 --> 00:00:55,579 application server, like Unicorn, Passenger, Thin, or, or anything 9 00:00:55,579 --> 00:00:58,899 like that. Except that it's special-purpose. It's for service-oriented 10 00:00:58,899 --> 00:01:03,510 architectures. And it doesn't actually speak HTTP at all. 11 00:01:03,510 --> 00:01:07,899 So. Why am I doing this crazy thing? Well, 12 00:01:07,899 --> 00:01:11,060 my app drove me to it. My app is 13 00:01:11,060 --> 00:01:14,110 huge. It's really complicated. This is kind of a 14 00:01:14,110 --> 00:01:16,070 services diagram of my app. Or where we'd like 15 00:01:16,070 --> 00:01:18,220 to go with it. 16 00:01:18,220 --> 00:01:21,100 There's over three-hundred and fifty models in one Rails 17 00:01:21,100 --> 00:01:25,380 application. The test suite takes over eight hours. It's 18 00:01:25,380 --> 00:01:29,899 kind of a headache. You could put it. So 19 00:01:29,899 --> 00:01:31,479 we decided the way to move forward with it 20 00:01:31,479 --> 00:01:34,110 is to break it up into smaller applications that 21 00:01:34,110 --> 00:01:37,380 work together. And, of course, that means SOA. 22 00:01:37,380 --> 00:01:40,810 Now, I mean, SOA is really wrapped up in 23 00:01:40,810 --> 00:01:45,270 a lot of pretty awful baggage, when you come 24 00:01:45,270 --> 00:01:48,670 down to it. SOA, you think about enterprise service 25 00:01:48,670 --> 00:01:53,310 buses. You think about strict interface definitions. Versions. I 26 00:01:53,310 --> 00:01:55,399 mean it's a lot of complexity that you have 27 00:01:55,399 --> 00:01:59,569 to worry about. Extra load balancers. More configuration. More 28 00:01:59,569 --> 00:02:02,410 distributed tracing and monitoring. I mean, it's a, it's 29 00:02:02,410 --> 00:02:04,280 a real pain. 30 00:02:04,280 --> 00:02:06,890 So I, I hate SOA, right. It makes me 31 00:02:06,890 --> 00:02:12,140 feel like this. So, but when you get down 32 00:02:12,140 --> 00:02:14,690 to it, SOA is actually not all of that. 33 00:02:14,690 --> 00:02:19,770 SOA is really just independent actors communicating. They're working 34 00:02:19,770 --> 00:02:25,310 together to get something useful done. So, really, SOA 35 00:02:25,310 --> 00:02:29,790 can mitigate complexity by enforcing simplification. Each service only 36 00:02:29,790 --> 00:02:32,540 has to think about its own little bubble. It 37 00:02:32,540 --> 00:02:34,180 only has to worry about its little component. And 38 00:02:34,180 --> 00:02:35,450 it doesn't have to think about anything else that 39 00:02:35,450 --> 00:02:38,019 goes on in its entire system. 40 00:02:38,019 --> 00:02:39,819 You can let your system architect worry about what's 41 00:02:39,819 --> 00:02:40,969 going on across the whole thing. All you have 42 00:02:40,969 --> 00:02:42,079 to do is pay attention to your one little 43 00:02:42,079 --> 00:02:45,340 service and you can be pretty happy. 44 00:02:45,340 --> 00:02:48,159 So, when you get down to it again, SOA 45 00:02:48,159 --> 00:02:51,569 is really just independent actors communicating. Now that communicating 46 00:02:51,569 --> 00:02:54,219 bit is important. How they communicate actually determines how 47 00:02:54,219 --> 00:02:57,709 your application behaves. And the kinds of communication you 48 00:02:57,709 --> 00:03:00,739 can use impacts the kind of application you can 49 00:03:00,739 --> 00:03:01,859 write. 50 00:03:01,859 --> 00:03:05,010 So, the typical communication pattern that we're most familiar 51 00:03:05,010 --> 00:03:08,140 with as programmers is sort of direct messaging. I 52 00:03:08,140 --> 00:03:10,650 am an object. I can send a message to 53 00:03:10,650 --> 00:03:13,409 my friend. My friend is gonna give me back 54 00:03:13,409 --> 00:03:15,680 a response. It's very normal. It's what, in fact, 55 00:03:15,680 --> 00:03:18,849 we use all the time with HTTP. But when 56 00:03:18,849 --> 00:03:20,159 you think about it, what we actually want to 57 00:03:20,159 --> 00:03:21,709 do is be able to send different kinds of 58 00:03:21,709 --> 00:03:24,790 messaging structures. We want to publish that something happened 59 00:03:24,790 --> 00:03:27,290 in my system and have people subscribe to that 60 00:03:27,290 --> 00:03:30,469 event and take some interesting action. 61 00:03:30,469 --> 00:03:31,980 It's all about decoupling. I don't have to know 62 00:03:31,980 --> 00:03:33,650 about my neighbor anymore. I just need to know 63 00:03:33,650 --> 00:03:36,180 that I did something, and then somebody else might 64 00:03:36,180 --> 00:03:37,879 care that, oh, you did something. That's great. Let 65 00:03:37,879 --> 00:03:38,799 me do something about that. 66 00:03:38,799 --> 00:03:41,620 But again, like I said, let me. We're Rails 67 00:03:41,620 --> 00:03:44,780 developers. We're used to writing web applications. We speak 68 00:03:44,780 --> 00:03:47,859 HTTP all the time. HTTP allows us to do 69 00:03:47,859 --> 00:03:50,310 direct messaging, so that's what we tend to do. 70 00:03:50,310 --> 00:03:52,340 I'd say most SOAs out there, when they get 71 00:03:52,340 --> 00:03:54,480 off the ground, they start building web applications that 72 00:03:54,480 --> 00:03:56,889 speak with each other through HTTP. It's what we're 73 00:03:56,889 --> 00:03:58,430 familiar with. 74 00:03:58,430 --> 00:04:01,120 But what would we actually like our transport mechanism 75 00:04:01,120 --> 00:04:05,230 to be? We want it to be, you know, 76 00:04:05,230 --> 00:04:08,920 data structure independent. HTTP specifies that there be a 77 00:04:08,920 --> 00:04:11,329 body, but it doesn't specify the format of the 78 00:04:11,329 --> 00:04:13,170 body. We want it to be fast. We want 79 00:04:13,170 --> 00:04:17,690 it to be open, transparent, understandable, clear. You know, 80 00:04:17,690 --> 00:04:20,279 debuggable. Observable. And flexible. 81 00:04:20,279 --> 00:04:25,830 So, is HTTP all of those things. We also 82 00:04:25,830 --> 00:04:30,080 need, not only direct request response type of messaging, 83 00:04:30,080 --> 00:04:33,979 like RPC, but we also need pub, subscribe, and 84 00:04:33,979 --> 00:04:38,180 broadcast. Additionally, we don't need more load balancers. More 85 00:04:38,180 --> 00:04:43,120 SSL certificates. Distributed logging. Trying to figure out what's 86 00:04:43,120 --> 00:04:44,860 going on in this service over here versus that 87 00:04:44,860 --> 00:04:47,210 one over there. It's a nightmare. 88 00:04:47,210 --> 00:04:50,150 And, you know, looking up the various services in 89 00:04:50,150 --> 00:04:51,710 your system, it's a pain. So, again, I mean, 90 00:04:51,710 --> 00:04:54,740 HTTP, you know, it has, it is blind. It's 91 00:04:54,740 --> 00:04:57,669 fast. It's open, clear, it's well-understood. It's what we 92 00:04:57,669 --> 00:05:01,680 know. It's really great for RPC. It's a better 93 00:05:01,680 --> 00:05:02,500 alternative. 94 00:05:02,500 --> 00:05:07,660 And that's AMQP. AMQP is blind, also. It doesn't 95 00:05:07,660 --> 00:05:10,220 care what data you're transmitting across it. It's extremely 96 00:05:10,220 --> 00:05:12,860 fast. It's an open system. It's great for all 97 00:05:12,860 --> 00:05:16,500 messaging pattern. Not just direct messaging, but pub/sub, and 98 00:05:16,500 --> 00:05:21,060 queue-based messaging. It's centralized, which in terms of a 99 00:05:21,060 --> 00:05:23,650 service-oriented architecture is very beneficial, in that you have 100 00:05:23,650 --> 00:05:26,120 one point to manage, one point to pay attention 101 00:05:26,120 --> 00:05:28,710 to everything at. And the only issue is that 102 00:05:28,710 --> 00:05:30,240 it's not what we're familiar with. It's not what 103 00:05:30,240 --> 00:05:32,030 we know. You know, again, we're web developers. We're 104 00:05:32,030 --> 00:05:34,280 not AMQP developers. At least not the most of 105 00:05:34,280 --> 00:05:34,919 us. 106 00:05:34,919 --> 00:05:36,500 So it's a challenge. 107 00:05:36,500 --> 00:05:40,020 So the RACK-AMQP project that I'm working on is 108 00:05:40,020 --> 00:05:44,280 really designed to make AMQP simple. We are trying 109 00:05:44,280 --> 00:05:47,180 to introduce as few opinions as absolutely necessary in 110 00:05:47,180 --> 00:05:49,509 order to make this possible. And to give you 111 00:05:49,509 --> 00:05:51,490 a stepping stone into the world of AMQP where 112 00:05:51,490 --> 00:05:54,340 you can start to integrate more advanced concepts into 113 00:05:54,340 --> 00:06:00,539 the system, without having to relearn your entire world. 114 00:06:00,539 --> 00:06:03,610 So it also lets us continue to build what 115 00:06:03,610 --> 00:06:06,590 we know. Because it's built on Rack, you know, 116 00:06:06,590 --> 00:06:09,580 we're web-developers. We can continue to use the web 117 00:06:09,580 --> 00:06:11,810 concepts that we're familiar with. We can developer locally 118 00:06:11,810 --> 00:06:13,569 on our web machines using the systems that we're 119 00:06:13,569 --> 00:06:15,879 comfortable with. And then when we deploy, we simply 120 00:06:15,879 --> 00:06:18,440 deploy it to Jackalope, instead of to Unicorn or 121 00:06:18,440 --> 00:06:22,259 Passenger or whatever, and it's magically in this world 122 00:06:22,259 --> 00:06:24,020 of AMQP. 123 00:06:24,020 --> 00:06:27,770 So. How does that work? Well, let's look at 124 00:06:27,770 --> 00:06:29,220 AMQP a little bit more in depth, first. So 125 00:06:29,220 --> 00:06:32,240 AMQP is the advanced messaging queue, message queueing protocol. 126 00:06:32,240 --> 00:06:34,490 It is an open standard. It's developed by Oasis, 127 00:06:34,490 --> 00:06:37,690 which is an open standards body. It, again, like 128 00:06:37,690 --> 00:06:41,610 I said, uses a centralized distribution pattern. That centralized 129 00:06:41,610 --> 00:06:44,400 distribution pattern allows you to do the various kinds 130 00:06:44,400 --> 00:06:46,659 of messaging that are really handy for us. The 131 00:06:46,659 --> 00:06:49,750 pub/sub and the other interesting thing. This implies that 132 00:06:49,750 --> 00:06:51,889 we have a broker in the center that all 133 00:06:51,889 --> 00:06:54,509 of our services connect to and transmit messages through 134 00:06:54,509 --> 00:06:56,610 that broker. And that broker can make intelligent decisions 135 00:06:56,610 --> 00:06:58,930 about how to route those messages. 136 00:06:58,930 --> 00:07:02,569 AMQP messages themselves, they have headers and bodies, just 137 00:07:02,569 --> 00:07:06,000 like HTTP. There are well-known fields, like content type 138 00:07:06,000 --> 00:07:11,939 and priority, in this case. And we can actually 139 00:07:11,939 --> 00:07:14,830 leverage the conventions that we understand from HTTP to 140 00:07:14,830 --> 00:07:18,939 pull AMQP into the world. So AMQP, like I 141 00:07:18,939 --> 00:07:21,009 mentioned, has these headers and bodies. It has well-known 142 00:07:21,009 --> 00:07:23,819 fields. And HTTP essentially has headers and bodies and 143 00:07:23,819 --> 00:07:26,780 also well-known fields. So we can essentially emulate HTTP 144 00:07:26,780 --> 00:07:30,250 over AMQP without too much trouble. And, again, the 145 00:07:30,250 --> 00:07:32,460 AMQP supports the RPC messaging pattern, which is what 146 00:07:32,460 --> 00:07:34,240 HTTP is. 147 00:07:34,240 --> 00:07:36,860 So, one of the parts of the project is 148 00:07:36,860 --> 00:07:41,610 the AMQP HTTP RC, which, it simply does as 149 00:07:41,610 --> 00:07:44,569 little as it has to to define the structure. 150 00:07:44,569 --> 00:07:46,060 It's just a documentation on how we're doing the 151 00:07:46,060 --> 00:07:48,240 mapping. One of the other goals is that we 152 00:07:48,240 --> 00:07:50,470 want to make inner-operability with other languages and other 153 00:07:50,470 --> 00:07:54,150 platforms really easy. So having an RC will allow 154 00:07:54,150 --> 00:07:55,720 it, allow us to have just a central document 155 00:07:55,720 --> 00:08:00,229 to refer back to in constructing other servers. 156 00:08:00,229 --> 00:08:01,569 So let's look at HTTP a little bit more 157 00:08:01,569 --> 00:08:03,270 in-depth. How does HTTP work? Well, you know, when 158 00:08:03,270 --> 00:08:06,639 we send a request, we send a specific request 159 00:08:06,639 --> 00:08:09,310 to a server, to a, you know, specific IP 160 00:08:09,310 --> 00:08:11,710 address and port. And we get back a response. 161 00:08:11,710 --> 00:08:15,009 And, of course, the request has things like the 162 00:08:15,009 --> 00:08:19,409 HTTP verb, the path, headers, the bodies. And the 163 00:08:19,409 --> 00:08:21,219 response comes back with a response code. 164 00:08:21,219 --> 00:08:23,539 It's all stuff we're familiar with. So all we 165 00:08:23,539 --> 00:08:26,500 had to do is map that into AMQP terms, 166 00:08:26,500 --> 00:08:28,490 which is essentially what we've done here. And this 167 00:08:28,490 --> 00:08:29,699 is just a reference for just a few of 168 00:08:29,699 --> 00:08:32,090 the things that we're doing. Like content type. There's 169 00:08:32,090 --> 00:08:34,130 already a content type header in AMQP, so we 170 00:08:34,130 --> 00:08:36,780 just reuse that. Some things didn't make sense. Like 171 00:08:36,780 --> 00:08:41,049 the protocol for example. In HTTP you've got HTP 172 00:08:41,049 --> 00:08:45,940 or HTPS or HTTP plus dev or whatever. But 173 00:08:45,940 --> 00:08:48,790 with AMQP, your protocol's already negotiated when you connect 174 00:08:48,790 --> 00:08:50,330 to the broker. So it doesn't really make sense 175 00:08:50,330 --> 00:08:51,890 for us to worry about that in the messages 176 00:08:51,890 --> 00:08:55,200 we're passing around the system. 177 00:08:55,200 --> 00:08:57,490 The host name becomes, essentially, the routing key. The 178 00:08:57,490 --> 00:08:59,930 queue target that we're sending to in AMQP. And 179 00:08:59,930 --> 00:09:03,290 it makes it really simple. So, this looks like 180 00:09:03,290 --> 00:09:07,050 this, essentially, in code. This is an example of 181 00:09:07,050 --> 00:09:09,910 how you could write a client that speaks AMQP, 182 00:09:09,910 --> 00:09:12,830 HTTP. You create a call-back queue. We'll get back 183 00:09:12,830 --> 00:09:15,220 to that in one sec. You publish a message 184 00:09:15,220 --> 00:09:17,570 that looks like an HTTP message to the target 185 00:09:17,570 --> 00:09:19,760 queue. And then you wait for a response. 186 00:09:19,760 --> 00:09:21,710 Now, the callback queue is actually a really important 187 00:09:21,710 --> 00:09:26,270 thing here. AMQP is a hundred percent queue based. 188 00:09:26,270 --> 00:09:29,140 It's all asynchronous under the hood. But, you know, 189 00:09:29,140 --> 00:09:31,710 we want to write a synchronous system. HTTP is 190 00:09:31,710 --> 00:09:34,080 synchronous. I send a request. I wait for the 191 00:09:34,080 --> 00:09:37,280 response, or receive the response and move on. So, 192 00:09:37,280 --> 00:09:41,090 to do that in AMQP, we create a respond 193 00:09:41,090 --> 00:09:43,330 queue that we are going to listen to in 194 00:09:43,330 --> 00:09:46,680 the broker. It's an anonymous queue. There's a great 195 00:09:46,680 --> 00:09:49,280 convention. We're doing this in AMQP. It's very easy. 196 00:09:49,280 --> 00:09:50,630 So we create a response queue, we get the 197 00:09:50,630 --> 00:09:52,000 name of that queue, and then as part of 198 00:09:52,000 --> 00:09:53,950 the message we send to our target, we say, 199 00:09:53,950 --> 00:09:56,430 here's the response queue to reply back at. And 200 00:09:56,430 --> 00:09:58,350 then we simply wait for the response on that 201 00:09:58,350 --> 00:10:00,250 call-back queue to come back. So that lets us 202 00:10:00,250 --> 00:10:02,710 get the synchrony that we're familiar with, that we 203 00:10:02,710 --> 00:10:06,160 need, with HTTP style programming, while still having an 204 00:10:06,160 --> 00:10:09,130 asynchronous system under the hood. 205 00:10:09,130 --> 00:10:13,890 RACK, of course, we're all using RACK today. Even 206 00:10:13,890 --> 00:10:15,810 if you're not familiar with it. RACK, of course, 207 00:10:15,810 --> 00:10:18,190 is what runs all Ruby web application servers. Turns 208 00:10:18,190 --> 00:10:21,120 out that RACK actually doesn't care about HTTP. All 209 00:10:21,120 --> 00:10:23,730 that RACK really defines is that you had defined 210 00:10:23,730 --> 00:10:26,870 a call method that receives an environment, which is 211 00:10:26,870 --> 00:10:29,920 basically a hash, and it responds back with a 212 00:10:29,920 --> 00:10:34,080 three-element array. The HTTP response code, headers, and the 213 00:10:34,080 --> 00:10:34,770 body. 214 00:10:34,770 --> 00:10:37,230 That hash doesn't actually have to be anything but 215 00:10:37,230 --> 00:10:39,650 a hash. It just has to look like an 216 00:10:39,650 --> 00:10:42,850 HTTP environment. So emulation was actually pretty simple. So 217 00:10:42,850 --> 00:10:45,730 this is basically how it looks. We subscribe to 218 00:10:45,730 --> 00:10:48,070 a queue on the server, and then for every 219 00:10:48,070 --> 00:10:51,290 message we receive back for that queue, we unpack 220 00:10:51,290 --> 00:10:53,910 some things into variables. We create what looks like 221 00:10:53,910 --> 00:10:56,910 the HTTP environment and we have the request method 222 00:10:56,910 --> 00:10:58,380 and query string. If you ever wrote CGI way 223 00:10:58,380 --> 00:11:00,290 back in the day, this might look familiar to 224 00:11:00,290 --> 00:11:01,900 you. 225 00:11:01,900 --> 00:11:04,000 And then we essentially pass that environment onto our 226 00:11:04,000 --> 00:11:06,700 RACK application, receive the response from the RACK app 227 00:11:06,700 --> 00:11:09,910 and then publish that back to the response queue. 228 00:11:09,910 --> 00:11:11,850 So this is what Jackalope is doing for us. 229 00:11:11,850 --> 00:11:15,770 Jackalope is emulating HTTP for our RACK applications. And 230 00:11:15,770 --> 00:11:18,540 it just works. I'll show you a demo at 231 00:11:18,540 --> 00:11:20,370 the end. You don't have to modify your code 232 00:11:20,370 --> 00:11:22,560 at all. You can just simply deploy to it, 233 00:11:22,560 --> 00:11:26,110 and it's speaking AMQP instead of HTTP. So that's 234 00:11:26,110 --> 00:11:28,730 it. We're done. We can go on vacation, right? 235 00:11:28,730 --> 00:11:31,860 Well. I didn't talk about how to actually put 236 00:11:31,860 --> 00:11:36,930 this in production, other than just use Jackalope. So 237 00:11:36,930 --> 00:11:40,510 you have to choose a broker in AMQP. So 238 00:11:40,510 --> 00:11:43,220 what we're using is RabbitMQ. RabbitMQ is a well-known 239 00:11:43,220 --> 00:11:46,390 AMQP broker. If you're doing any AMQP you're probably 240 00:11:46,390 --> 00:11:50,770 using RabbitMQ, most likely. It's used by giants out 241 00:11:50,770 --> 00:11:53,320 there. I mean, Google, VMWare, NASA. All these people 242 00:11:53,320 --> 00:11:54,730 are using Rabbit pretty heavily. 243 00:11:54,730 --> 00:12:00,110 It's extremely scalable. It's fault-tolerant. It's distributable. And it's 244 00:12:00,110 --> 00:12:02,570 secure. It does everything you'd ever want. And it 245 00:12:02,570 --> 00:12:05,100 also gives you a really great management console. We 246 00:12:05,100 --> 00:12:06,300 can go in and see what's going on in 247 00:12:06,300 --> 00:12:08,370 your system. Like I mentioned, the distributed logging we 248 00:12:08,370 --> 00:12:09,700 saw in the, in the last talk, if you 249 00:12:09,700 --> 00:12:11,270 were here for that, trying to get an idea 250 00:12:11,270 --> 00:12:12,880 of what's going on across your system can be 251 00:12:12,880 --> 00:12:15,630 challenging. Rabbit doesn't tell you everything you'd want to 252 00:12:15,630 --> 00:12:17,180 know, but it at least gives you an idea 253 00:12:17,180 --> 00:12:19,660 of what queues are being used. Their depths at 254 00:12:19,660 --> 00:12:22,080 the point. The kind of information that can help 255 00:12:22,080 --> 00:12:23,880 you get, at least get a handle on it. 256 00:12:23,880 --> 00:12:26,980 Additionally, you're gonna have to think about how you 257 00:12:26,980 --> 00:12:28,550 talk to the real world. I mean, AMQP is 258 00:12:28,550 --> 00:12:30,610 all well and good behind the scenes, but how 259 00:12:30,610 --> 00:12:33,290 do I continue to interact and serve my clients 260 00:12:33,290 --> 00:12:35,260 that I actually care about, to the point of 261 00:12:35,260 --> 00:12:36,110 getting paid, right. 262 00:12:36,110 --> 00:12:39,070 So this is the architecture we typically use at 263 00:12:39,070 --> 00:12:41,470 Optoro. We have our Rabbit server in the middle, 264 00:12:41,470 --> 00:12:43,790 and all the various services sitting on Jackalope that 265 00:12:43,790 --> 00:12:45,850 talk to Rabbit. And then additionally we have one 266 00:12:45,850 --> 00:12:48,320 or more API services that are continuing to be 267 00:12:48,320 --> 00:12:50,960 deployed on Unicorn, in our case. They talk to 268 00:12:50,960 --> 00:12:54,820 the outside world and internally translate the, the needs 269 00:12:54,820 --> 00:12:57,740 through AMQP and the rest of the system. 270 00:12:57,740 --> 00:12:58,920 One of the things we actually published for each 271 00:12:58,920 --> 00:13:01,600 of our services is a client gem that kind 272 00:13:01,600 --> 00:13:04,600 of isolates the worry, the worry about exactly which 273 00:13:04,600 --> 00:13:07,710 transport we're using away from the consumer, the API 274 00:13:07,710 --> 00:13:10,890 here. So it actually is, is really simple. 275 00:13:10,890 --> 00:13:12,850 And so the way that we do that is 276 00:13:12,850 --> 00:13:15,860 we typically have been using HTTParty for our HTTP 277 00:13:15,860 --> 00:13:18,760 communication needs. So we wrote, as one of the 278 00:13:18,760 --> 00:13:22,940 other projects part of RACK-AMQP, is AMQParty, which is 279 00:13:22,940 --> 00:13:26,790 just a API-compliant version of HTTParty. So we can 280 00:13:26,790 --> 00:13:30,620 actually just drop in the AMQParty constant in place 281 00:13:30,620 --> 00:13:32,800 of the HTTParty constant, and you can see down 282 00:13:32,800 --> 00:13:35,610 below, that transport variable, we typically configure that at 283 00:13:35,610 --> 00:13:39,310 boot time to be either HTTParty or AMQParty. And 284 00:13:39,310 --> 00:13:40,920 everything, again, just works. 285 00:13:40,920 --> 00:13:43,100 We change the, the url a little bit, too. 286 00:13:43,100 --> 00:13:44,670 But we do that all part of the, part 287 00:13:44,670 --> 00:13:47,430 of this setup for each server. For each client. 288 00:13:47,430 --> 00:13:51,620 Of course, if you don't want to use HTTParty, 289 00:13:51,620 --> 00:13:54,000 some people like the various other options that are 290 00:13:54,000 --> 00:13:57,000 out there, we also are publishing RACK-AMQP client, which 291 00:13:57,000 --> 00:13:59,600 is what H, AMQParty is built on. It, it's 292 00:13:59,600 --> 00:14:03,040 a very simple API. You get a client, you 293 00:14:03,040 --> 00:14:06,920 tell it which Rabbit to connect to. You send 294 00:14:06,920 --> 00:14:09,910 a request to your target queue with your uri 295 00:14:09,910 --> 00:14:12,410 as part of it, with any, you know, your 296 00:14:12,410 --> 00:14:13,680 HTTP method and all that. You get back your 297 00:14:13,680 --> 00:14:16,190 response. It's synchronous. It's simple. It's actually built on 298 00:14:16,190 --> 00:14:18,670 Bunny, which is a really great AMQP gem out 299 00:14:18,670 --> 00:14:21,780 there, and that also is very easy to use. 300 00:14:21,780 --> 00:14:27,540 Additionally, we're publishing a sample SOA using Rails. It's 301 00:14:27,540 --> 00:14:32,790 a work-in-progress at the moment. BUt the userland and 302 00:14:32,790 --> 00:14:34,430 userland_client are mostly built as, you can see where 303 00:14:34,430 --> 00:14:37,120 things are at the mo- at the current state. 304 00:14:37,120 --> 00:14:38,600 It kind of gives you an idea about how 305 00:14:38,600 --> 00:14:41,510 we think about SOA at Optoro, and how you 306 00:14:41,510 --> 00:14:45,350 might be able to use Jackalope and the RACK-AMQP 307 00:14:45,350 --> 00:14:47,050 project in your own projects. 308 00:14:47,050 --> 00:14:52,190 So userland is a, is Rails service that essentially 309 00:14:52,190 --> 00:14:54,560 publishes a user's concept out into the world. And 310 00:14:54,560 --> 00:14:56,790 then userland_client is a gem that consumes the userland 311 00:14:56,790 --> 00:15:00,260 service and interface uses the userland gem to talk 312 00:15:00,260 --> 00:15:02,100 to the userland service. 313 00:15:02,100 --> 00:15:07,330 And so then, you know, how fast is this? 314 00:15:07,330 --> 00:15:11,060 Well, there's this bench mark I've got. I'll show 315 00:15:11,060 --> 00:15:13,480 you it in just a sec. The only weird 316 00:15:13,480 --> 00:15:15,020 thing we have to do here is tell AMQParty 317 00:15:15,020 --> 00:15:16,320 which Rabbit to talk to, and you kind of 318 00:15:16,320 --> 00:15:18,040 saw that in the, in the client gem a 319 00:15:18,040 --> 00:15:19,710 little bit, so that's a little bit of setup 320 00:15:19,710 --> 00:15:21,290 we do. Again, like I said, when we make 321 00:15:21,290 --> 00:15:24,300 the decision about which, whether HTTParty or AMQParty. So 322 00:15:24,300 --> 00:15:26,270 we tell it which Rabbit. And then, inside, we, 323 00:15:26,270 --> 00:15:30,620 five hundred times, request this JSON using AMQParty and 324 00:15:30,620 --> 00:15:33,170 then five hundred times request the same thing using 325 00:15:33,170 --> 00:15:33,450 HTTParty. 326 00:15:33,450 --> 00:15:37,450 So here, I'll show you that in, in use. 327 00:15:37,450 --> 00:15:41,360 Let's see. OK. 328 00:15:41,360 --> 00:15:48,360 Let's make this big. All right. So here I'm 329 00:15:49,390 --> 00:15:53,080 gonna boot the app using Unicorn. So Unicorn is 330 00:15:53,080 --> 00:15:57,080 listing on port 8080. Wow. That's really big. Trust 331 00:15:57,080 --> 00:15:59,550 me, it's 8080 there. I'm gonna boot the same 332 00:15:59,550 --> 00:16:02,100 map using Jackalope. The only difference I have to 333 00:16:02,100 --> 00:16:03,280 do to boot the app is I have to 334 00:16:03,280 --> 00:16:05,330 say with the right queue name, so it's going 335 00:16:05,330 --> 00:16:08,120 to listen to the test dot simple queue. 336 00:16:08,120 --> 00:16:10,420 So there it is running. And then down here 337 00:16:10,420 --> 00:16:13,170 I'm gonna starty the bench mark. And like I 338 00:16:13,170 --> 00:16:16,390 said, it's gonna five hundred times hit each service. 339 00:16:16,390 --> 00:16:18,630 What it actually does is hit each service three 340 00:16:18,630 --> 00:16:20,380 times, just to make sure that it's warm, and 341 00:16:20,380 --> 00:16:22,730 then it can go through and hit it all 342 00:16:22,730 --> 00:16:24,650 the way through. And there we go. 343 00:16:24,650 --> 00:16:26,940 So you can see, it's two point nine seconds 344 00:16:26,940 --> 00:16:30,050 total to do five hundred AMQP requests, and three 345 00:16:30,050 --> 00:16:33,300 point six seconds total to do five hundred HTTP 346 00:16:33,300 --> 00:16:37,040 requests. I was actually surprised when I got this 347 00:16:37,040 --> 00:16:40,750 result. Jackalope is beta. I mean, it's, we're using 348 00:16:40,750 --> 00:16:43,930 it at Optoro, but it's not really been battle-tested. 349 00:16:43,930 --> 00:16:46,950 But it's faster, at the moment. So why is 350 00:16:46,950 --> 00:16:47,730 it faster? 351 00:16:47,730 --> 00:16:51,670 Well, maybe I haven't written all the code yet. 352 00:16:51,670 --> 00:16:55,530 But more likely it, more likely it's because of 353 00:16:55,530 --> 00:16:59,800 this concept in TCP called the slow-start phenomonon. And 354 00:16:59,800 --> 00:17:02,630 also, your whole TCP negotiation. So each one of 355 00:17:02,630 --> 00:17:06,699 these five hundred HTTP requests, close the connection, open 356 00:17:06,699 --> 00:17:10,770 the connection, send the message, receive the response. When 357 00:17:10,770 --> 00:17:12,799 you open the TCP connection you go through a 358 00:17:12,799 --> 00:17:14,970 whole sinaq phase, which I didn't put slides in 359 00:17:14,970 --> 00:17:17,309 for and maybe should have. But it's. 360 00:17:17,309 --> 00:17:20,630 I highly recommend checking out Eliot Gregorich's talks, by 361 00:17:20,630 --> 00:17:23,679 the way, about how HTTP works, how TCP works. 362 00:17:23,679 --> 00:17:27,069 But basically, you go through protocol negotiation for any 363 00:17:27,069 --> 00:17:32,279 TCP connection, and then there's a feature of TCP 364 00:17:32,279 --> 00:17:34,350 called the slow-start feature, which essentially allows you to 365 00:17:34,350 --> 00:17:37,639 negotiate your connection speed. And that's really great when 366 00:17:37,639 --> 00:17:39,600 you're talking over the internet, and you're not sure 367 00:17:39,600 --> 00:17:42,090 of the latency or the availability of routers to 368 00:17:42,090 --> 00:17:44,700 be decent along the way. But when you're doing 369 00:17:44,700 --> 00:17:47,720 internal applications, it just gets in your way. It 370 00:17:47,720 --> 00:17:48,519 just slows you down. 371 00:17:48,519 --> 00:17:52,370 So AMQP, we hold a persistent connection to the 372 00:17:52,370 --> 00:17:55,350 AMQP broker. We do one sinaq cycle, and then 373 00:17:55,350 --> 00:17:57,990 we get our maximum connection speed and we stay 374 00:17:57,990 --> 00:18:00,309 there. And then we can multiplex information over that 375 00:18:00,309 --> 00:18:03,179 single connection and it works really well. Now, you 376 00:18:03,179 --> 00:18:05,070 could write that with HTTP if you did a 377 00:18:05,070 --> 00:18:08,690 keep-alive connection, held over that connection and talk to 378 00:18:08,690 --> 00:18:09,559 the same thing back and forth. 379 00:18:09,559 --> 00:18:11,279 But, the bottom line is that most of us 380 00:18:11,279 --> 00:18:13,379 don't do that when we write HTTP. We're using 381 00:18:13,379 --> 00:18:17,919 something like HTTParty or, I always forget the tiforce 382 00:18:17,919 --> 00:18:21,000 or tif- whatever it is. Sorry. 383 00:18:21,000 --> 00:18:21,809 Most of the time, you know, we drop the 384 00:18:21,809 --> 00:18:23,740 connection, you know, in between requests. We don't hold 385 00:18:23,740 --> 00:18:25,620 it open. It's just not something we think about. 386 00:18:25,620 --> 00:18:29,860 So using AMQParty or any of the libraries that 387 00:18:29,860 --> 00:18:32,360 we're published, it holds that open and it just 388 00:18:32,360 --> 00:18:34,539 makes it more efficient overall. 389 00:18:34,539 --> 00:18:36,309 So I hope that's the primary explanation of why 390 00:18:36,309 --> 00:18:43,309 it's so much faster. So I just wanted to 391 00:18:44,340 --> 00:18:45,929 mention a few references, a few things that were 392 00:18:45,929 --> 00:18:47,870 really inspriring when we kind of went down this 393 00:18:47,870 --> 00:18:49,879 road. You know, this is really a departure from 394 00:18:49,879 --> 00:18:53,340 what we're used to in Rails. And the Architecture: 395 00:18:53,340 --> 00:18:55,970 The Lost Years talk and Matt Wynne's original Hexegonal 396 00:18:55,970 --> 00:18:58,320 Rails talks were all about departing from what we're 397 00:18:58,320 --> 00:19:00,429 used to, I think. So those are really interesting. 398 00:19:00,429 --> 00:19:02,909 If you haven't checked those out, definitely do. 399 00:19:02,909 --> 00:19:05,340 Additionally, Martin Fowler posted a recent article that was 400 00:19:05,340 --> 00:19:08,450 awesome, called Micro Services. Definitely, definitely read this if 401 00:19:08,450 --> 00:19:12,190 you are looking into service-oriented architectures. It lets you 402 00:19:12,190 --> 00:19:14,789 not think about that heavy-weight bloat that we think 403 00:19:14,789 --> 00:19:16,649 about with SOA. 404 00:19:16,649 --> 00:19:19,309 It's really a revisitation of the idea of, just, 405 00:19:19,309 --> 00:19:22,610 you have independent services that are collaborating to get 406 00:19:22,610 --> 00:19:24,820 something done in a very simple way. And that's 407 00:19:24,820 --> 00:19:27,529 really, again, also influenced the direction we're taking here. 408 00:19:27,529 --> 00:19:30,899 We want to introduce as few specific choices as 409 00:19:30,899 --> 00:19:33,090 possible to just kind of give us a simple 410 00:19:33,090 --> 00:19:36,679 transport to, to build on top of. Additionally, the 411 00:19:36,679 --> 00:19:39,529 Ruby AMQP gem has some awesome docs about how 412 00:19:39,529 --> 00:19:42,529 AMQP works, how to use it with Ruby. And 413 00:19:42,529 --> 00:19:45,509 RabbitMQ also publishes some really great documentation on what 414 00:19:45,509 --> 00:19:47,620 AMQP is, how it works. How to use it 415 00:19:47,620 --> 00:19:49,600 in the organization in your applications. 416 00:19:49,600 --> 00:19:51,649 And there's also this really great article I found 417 00:19:51,649 --> 00:19:55,029 about how HTTP works, in general. It's, I think 418 00:19:55,029 --> 00:19:58,429 a piece of school material from a university in 419 00:19:58,429 --> 00:19:59,960 Singapore. Definitely check it out. It's got really great 420 00:19:59,960 --> 00:20:00,730 diagrams. 421 00:20:00,730 --> 00:20:06,750 Thanks. Again, I'm Josh. I'm at jszmajda on the 422 00:20:06,750 --> 00:20:09,730 Twitters. I am the CTO at Optoro. We have 423 00:20:09,730 --> 00:20:12,720 blinq dot com. We have retailers with their returned 424 00:20:12,720 --> 00:20:14,710 goods and excess goods, help them figure out what 425 00:20:14,710 --> 00:20:16,809 it is, what the current value is and move 426 00:20:16,809 --> 00:20:18,070 it quickly. 427 00:20:18,070 --> 00:20:20,129 Additionally, I am the host of the Ruby hangout. 428 00:20:20,129 --> 00:20:22,480 Just a quick blurb for the Ruby hangout, it's 429 00:20:22,480 --> 00:20:25,220 a online meetup. It's really handy when you can 430 00:20:25,220 --> 00:20:27,850 go face-to-face and talk to people if you have 431 00:20:27,850 --> 00:20:28,960 a local meet up. If you don't have a 432 00:20:28,960 --> 00:20:30,629 local meet up or if you're too busy or 433 00:20:30,629 --> 00:20:32,509 if you can't make it for whatever reason, the 434 00:20:32,509 --> 00:20:35,570 Ruby hangout is an online meet up to, essentially, 435 00:20:35,570 --> 00:20:38,249 give you that local meet up feel, as much 436 00:20:38,249 --> 00:20:40,059 as we can online. So check that out. It's 437 00:20:40,059 --> 00:20:41,899 a lot of fun. I'm one of the co-organizers 438 00:20:41,899 --> 00:20:44,110 of the DC Ruby Users group. And I want 439 00:20:44,110 --> 00:20:46,399 to give a quick shoutout to Jonathan. Thanks Jon. 440 00:20:46,399 --> 00:20:49,559 He's on my team, helped me get Jackalope over 441 00:20:49,559 --> 00:20:52,799 the, over the finish line. So super helpful on 442 00:20:52,799 --> 00:20:53,279 that. 443 00:20:53,279 --> 00:20:55,649 All the code's on GitHub. Github dot com slash 444 00:20:55,649 --> 00:20:59,419 rack dash amqp slash rack dash amqp is where 445 00:20:59,419 --> 00:21:01,600 the RC lives and has bookmarks to the other 446 00:21:01,600 --> 00:21:06,320 parts of the project. And I would love to 447 00:21:06,320 --> 00:21:10,299 take your questions, because that talk was way short. 448 00:21:10,299 --> 00:21:12,840 And I've got, actually, a whole bunch of stuffed 449 00:21:12,840 --> 00:21:16,100 Jackalopes up here for good questions. So please feel 450 00:21:16,100 --> 00:21:18,300 free to come grab one.