0:00:16.840,0:00:22.540
JOSH SZMAJDA: All right. You guys ready to[br]talk about SOA? Woo!

0:00:22.540,0:00:29.540
I'm Josh. Hi. How you doing? I'm the CTO

0:00:30.849,0:00:33.390
at Optoro. We make software for retailers.[br]Help them

0:00:33.390,0:00:36.470
deal with their stuff at scale. I'm working[br]on

0:00:36.470,0:00:41.070
a project called RACK-AMQP. RACK-AMQP is a[br]collection of

0:00:41.070,0:00:46.940
different projects, including an RC, some[br]client libraries, and

0:00:46.940,0:00:51.040
most notably, Jackalope, which is a new Ruby[br]web

0:00:51.040,0:00:55.579
application server, like Unicorn, Passenger,[br]Thin, or, or anything

0:00:55.579,0:00:58.899
like that. Except that it's special-purpose.[br]It's for service-oriented

0:00:58.899,0:01:03.510
architectures. And it doesn't actually speak[br]HTTP at all.

0:01:03.510,0:01:07.899
So. Why am I doing this crazy thing? Well,

0:01:07.899,0:01:11.060
my app drove me to it. My app is

0:01:11.060,0:01:14.110
huge. It's really complicated. This is kind[br]of a

0:01:14.110,0:01:16.070
services diagram of my app. Or where we'd[br]like

0:01:16.070,0:01:18.220
to go with it.

0:01:18.220,0:01:21.100
There's over three-hundred and fifty models[br]in one Rails

0:01:21.100,0:01:25.380
application. The test suite takes over eight[br]hours. It's

0:01:25.380,0:01:29.899
kind of a headache. You could put it. So

0:01:29.899,0:01:31.479
we decided the way to move forward with it

0:01:31.479,0:01:34.110
is to break it up into smaller applications[br]that

0:01:34.110,0:01:37.380
work together. And, of course, that means[br]SOA.

0:01:37.380,0:01:40.810
Now, I mean, SOA is really wrapped up in

0:01:40.810,0:01:45.270
a lot of pretty awful baggage, when you come

0:01:45.270,0:01:48.670
down to it. SOA, you think about enterprise[br]service

0:01:48.670,0:01:53.310
buses. You think about strict interface definitions.[br]Versions. I

0:01:53.310,0:01:55.399
mean it's a lot of complexity that you have

0:01:55.399,0:01:59.569
to worry about. Extra load balancers. More[br]configuration. More

0:01:59.569,0:02:02.410
distributed tracing and monitoring. I mean,[br]it's a, it's

0:02:02.410,0:02:04.280
a real pain.

0:02:04.280,0:02:06.890
So I, I hate SOA, right. It makes me

0:02:06.890,0:02:12.140
feel like this. So, but when you get down

0:02:12.140,0:02:14.690
to it, SOA is actually not all of that.

0:02:14.690,0:02:19.770
SOA is really just independent actors communicating.[br]They're working

0:02:19.770,0:02:25.310
together to get something useful done. So,[br]really, SOA

0:02:25.310,0:02:29.790
can mitigate complexity by enforcing simplification.[br]Each service only

0:02:29.790,0:02:32.540
has to think about its own little bubble.[br]It

0:02:32.540,0:02:34.180
only has to worry about its little component.[br]And

0:02:34.180,0:02:35.450
it doesn't have to think about anything else[br]that

0:02:35.450,0:02:38.019
goes on in its entire system.

0:02:38.019,0:02:39.819
You can let your system architect worry about[br]what's

0:02:39.819,0:02:40.969
going on across the whole thing. All you have

0:02:40.969,0:02:42.079
to do is pay attention to your one little

0:02:42.079,0:02:45.340
service and you can be pretty happy.

0:02:45.340,0:02:48.159
So, when you get down to it again, SOA

0:02:48.159,0:02:51.569
is really just independent actors communicating.[br]Now that communicating

0:02:51.569,0:02:54.219
bit is important. How they communicate actually[br]determines how

0:02:54.219,0:02:57.709
your application behaves. And the kinds of[br]communication you

0:02:57.709,0:03:00.739
can use impacts the kind of application you[br]can

0:03:00.739,0:03:01.859
write.

0:03:01.859,0:03:05.010
So, the typical communication pattern that[br]we're most familiar

0:03:05.010,0:03:08.140
with as programmers is sort of direct messaging.[br]I

0:03:08.140,0:03:10.650
am an object. I can send a message to

0:03:10.650,0:03:13.409
my friend. My friend is gonna give me back

0:03:13.409,0:03:15.680
a response. It's very normal. It's what, in[br]fact,

0:03:15.680,0:03:18.849
we use all the time with HTTP. But when

0:03:18.849,0:03:20.159
you think about it, what we actually want[br]to

0:03:20.159,0:03:21.709
do is be able to send different kinds of

0:03:21.709,0:03:24.790
messaging structures. We want to publish that[br]something happened

0:03:24.790,0:03:27.290
in my system and have people subscribe to[br]that

0:03:27.290,0:03:30.469
event and take some interesting action.

0:03:30.469,0:03:31.980
It's all about decoupling. I don't have to[br]know

0:03:31.980,0:03:33.650
about my neighbor anymore. I just need to[br]know

0:03:33.650,0:03:36.180
that I did something, and then somebody else[br]might

0:03:36.180,0:03:37.879
care that, oh, you did something. That's great.[br]Let

0:03:37.879,0:03:38.799
me do something about that.

0:03:38.799,0:03:41.620
But again, like I said, let me. We're Rails

0:03:41.620,0:03:44.780
developers. We're used to writing web applications.[br]We speak

0:03:44.780,0:03:47.859
HTTP all the time. HTTP allows us to do

0:03:47.859,0:03:50.310
direct messaging, so that's what we tend to[br]do.

0:03:50.310,0:03:52.340
I'd say most SOAs out there, when they get

0:03:52.340,0:03:54.480
off the ground, they start building web applications[br]that

0:03:54.480,0:03:56.889
speak with each other through HTTP. It's what[br]we're

0:03:56.889,0:03:58.430
familiar with.

0:03:58.430,0:04:01.120
But what would we actually like our transport[br]mechanism

0:04:01.120,0:04:05.230
to be? We want it to be, you know,

0:04:05.230,0:04:08.920
data structure independent. HTTP specifies[br]that there be a

0:04:08.920,0:04:11.329
body, but it doesn't specify the format of[br]the

0:04:11.329,0:04:13.170
body. We want it to be fast. We want

0:04:13.170,0:04:17.690
it to be open, transparent, understandable,[br]clear. You know,

0:04:17.690,0:04:20.279
debuggable. Observable. And flexible.

0:04:20.279,0:04:25.830
So, is HTTP all of those things. We also

0:04:25.830,0:04:30.080
need, not only direct request response type[br]of messaging,

0:04:30.080,0:04:33.979
like RPC, but we also need pub, subscribe,[br]and

0:04:33.979,0:04:38.180
broadcast. Additionally, we don't need more[br]load balancers. More

0:04:38.180,0:04:43.120
SSL certificates. Distributed logging. Trying[br]to figure out what's

0:04:43.120,0:04:44.860
going on in this service over here versus[br]that

0:04:44.860,0:04:47.210
one over there. It's a nightmare.

0:04:47.210,0:04:50.150
And, you know, looking up the various services[br]in

0:04:50.150,0:04:51.710
your system, it's a pain. So, again, I mean,

0:04:51.710,0:04:54.740
HTTP, you know, it has, it is blind. It's

0:04:54.740,0:04:57.669
fast. It's open, clear, it's well-understood.[br]It's what we

0:04:57.669,0:05:01.680
know. It's really great for RPC. It's a better

0:05:01.680,0:05:02.500
alternative.

0:05:02.500,0:05:07.660
And that's AMQP. AMQP is blind, also. It doesn't

0:05:07.660,0:05:10.220
care what data you're transmitting across[br]it. It's extremely

0:05:10.220,0:05:12.860
fast. It's an open system. It's great for[br]all

0:05:12.860,0:05:16.500
messaging pattern. Not just direct messaging,[br]but pub/sub, and

0:05:16.500,0:05:21.060
queue-based messaging. It's centralized, which[br]in terms of a

0:05:21.060,0:05:23.650
service-oriented architecture is very beneficial,[br]in that you have

0:05:23.650,0:05:26.120
one point to manage, one point to pay attention

0:05:26.120,0:05:28.710
to everything at. And the only issue is that

0:05:28.710,0:05:30.240
it's not what we're familiar with. It's not[br]what

0:05:30.240,0:05:32.030
we know. You know, again, we're web developers.[br]We're

0:05:32.030,0:05:34.280
not AMQP developers. At least not the most[br]of

0:05:34.280,0:05:34.919
us.

0:05:34.919,0:05:36.500
So it's a challenge.

0:05:36.500,0:05:40.020
So the RACK-AMQP project that I'm working[br]on is

0:05:40.020,0:05:44.280
really designed to make AMQP simple. We are[br]trying

0:05:44.280,0:05:47.180
to introduce as few opinions as absolutely[br]necessary in

0:05:47.180,0:05:49.509
order to make this possible. And to give you

0:05:49.509,0:05:51.490
a stepping stone into the world of AMQP where

0:05:51.490,0:05:54.340
you can start to integrate more advanced concepts[br]into

0:05:54.340,0:06:00.539
the system, without having to relearn your[br]entire world.

0:06:00.539,0:06:03.610
So it also lets us continue to build what

0:06:03.610,0:06:06.590
we know. Because it's built on Rack, you know,

0:06:06.590,0:06:09.580
we're web-developers. We can continue to use[br]the web

0:06:09.580,0:06:11.810
concepts that we're familiar with. We can[br]developer locally

0:06:11.810,0:06:13.569
on our web machines using the systems that[br]we're

0:06:13.569,0:06:15.879
comfortable with. And then when we deploy,[br]we simply

0:06:15.879,0:06:18.440
deploy it to Jackalope, instead of to Unicorn[br]or

0:06:18.440,0:06:22.259
Passenger or whatever, and it's magically[br]in this world

0:06:22.259,0:06:24.020
of AMQP.

0:06:24.020,0:06:27.770
So. How does that work? Well, let's look at

0:06:27.770,0:06:29.220
AMQP a little bit more in depth, first. So

0:06:29.220,0:06:32.240
AMQP is the advanced messaging queue, message[br]queueing protocol.

0:06:32.240,0:06:34.490
It is an open standard. It's developed by[br]Oasis,

0:06:34.490,0:06:37.690
which is an open standards body. It, again,[br]like

0:06:37.690,0:06:41.610
I said, uses a centralized distribution pattern.[br]That centralized

0:06:41.610,0:06:44.400
distribution pattern allows you to do the[br]various kinds

0:06:44.400,0:06:46.659
of messaging that are really handy for us.[br]The

0:06:46.659,0:06:49.750
pub/sub and the other interesting thing. This[br]implies that

0:06:49.750,0:06:51.889
we have a broker in the center that all

0:06:51.889,0:06:54.509
of our services connect to and transmit messages[br]through

0:06:54.509,0:06:56.610
that broker. And that broker can make intelligent[br]decisions

0:06:56.610,0:06:58.930
about how to route those messages.

0:06:58.930,0:07:02.569
AMQP messages themselves, they have headers[br]and bodies, just

0:07:02.569,0:07:06.000
like HTTP. There are well-known fields, like[br]content type

0:07:06.000,0:07:11.939
and priority, in this case. And we can actually

0:07:11.939,0:07:14.830
leverage the conventions that we understand[br]from HTTP to

0:07:14.830,0:07:18.939
pull AMQP into the world. So AMQP, like I

0:07:18.939,0:07:21.009
mentioned, has these headers and bodies. It[br]has well-known

0:07:21.009,0:07:23.819
fields. And HTTP essentially has headers and[br]bodies and

0:07:23.819,0:07:26.780
also well-known fields. So we can essentially[br]emulate HTTP

0:07:26.780,0:07:30.250
over AMQP without too much trouble. And, again,[br]the

0:07:30.250,0:07:32.460
AMQP supports the RPC messaging pattern, which[br]is what

0:07:32.460,0:07:34.240
HTTP is.

0:07:34.240,0:07:36.860
So, one of the parts of the project is

0:07:36.860,0:07:41.610
the AMQP HTTP RC, which, it simply does as

0:07:41.610,0:07:44.569
little as it has to to define the structure.

0:07:44.569,0:07:46.060
It's just a documentation on how we're doing[br]the

0:07:46.060,0:07:48.240
mapping. One of the other goals is that we

0:07:48.240,0:07:50.470
want to make inner-operability with other[br]languages and other

0:07:50.470,0:07:54.150
platforms really easy. So having an RC will[br]allow

0:07:54.150,0:07:55.720
it, allow us to have just a central document

0:07:55.720,0:08:00.229
to refer back to in constructing other servers.

0:08:00.229,0:08:01.569
So let's look at HTTP a little bit more

0:08:01.569,0:08:03.270
in-depth. How does HTTP work? Well, you know,[br]when

0:08:03.270,0:08:06.639
we send a request, we send a specific request

0:08:06.639,0:08:09.310
to a server, to a, you know, specific IP

0:08:09.310,0:08:11.710
address and port. And we get back a response.

0:08:11.710,0:08:15.009
And, of course, the request has things like[br]the

0:08:15.009,0:08:19.409
HTTP verb, the path, headers, the bodies.[br]And the

0:08:19.409,0:08:21.219
response comes back with a response code.

0:08:21.219,0:08:23.539
It's all stuff we're familiar with. So all[br]we

0:08:23.539,0:08:26.500
had to do is map that into AMQP terms,

0:08:26.500,0:08:28.490
which is essentially what we've done here.[br]And this

0:08:28.490,0:08:29.699
is just a reference for just a few of

0:08:29.699,0:08:32.090
the things that we're doing. Like content[br]type. There's

0:08:32.090,0:08:34.130
already a content type header in AMQP, so[br]we

0:08:34.130,0:08:36.780
just reuse that. Some things didn't make sense.[br]Like

0:08:36.780,0:08:41.049
the protocol for example. In HTTP you've got[br]HTP

0:08:41.049,0:08:45.940
or HTPS or HTTP plus dev or whatever. But

0:08:45.940,0:08:48.790
with AMQP, your protocol's already negotiated[br]when you connect

0:08:48.790,0:08:50.330
to the broker. So it doesn't really make sense

0:08:50.330,0:08:51.890
for us to worry about that in the messages

0:08:51.890,0:08:55.200
we're passing around the system.

0:08:55.200,0:08:57.490
The host name becomes, essentially, the routing[br]key. The

0:08:57.490,0:08:59.930
queue target that we're sending to in AMQP.[br]And

0:08:59.930,0:09:03.290
it makes it really simple. So, this looks[br]like

0:09:03.290,0:09:07.050
this, essentially, in code. This is an example[br]of

0:09:07.050,0:09:09.910
how you could write a client that speaks AMQP,

0:09:09.910,0:09:12.830
HTTP. You create a call-back queue. We'll[br]get back

0:09:12.830,0:09:15.220
to that in one sec. You publish a message

0:09:15.220,0:09:17.570
that looks like an HTTP message to the target

0:09:17.570,0:09:19.760
queue. And then you wait for a response.

0:09:19.760,0:09:21.710
Now, the callback queue is actually a really[br]important

0:09:21.710,0:09:26.270
thing here. AMQP is a hundred percent queue[br]based.

0:09:26.270,0:09:29.140
It's all asynchronous under the hood. But,[br]you know,

0:09:29.140,0:09:31.710
we want to write a synchronous system. HTTP[br]is

0:09:31.710,0:09:34.080
synchronous. I send a request. I wait for[br]the

0:09:34.080,0:09:37.280
response, or receive the response and move[br]on. So,

0:09:37.280,0:09:41.090
to do that in AMQP, we create a respond

0:09:41.090,0:09:43.330
queue that we are going to listen to in

0:09:43.330,0:09:46.680
the broker. It's an anonymous queue. There's[br]a great

0:09:46.680,0:09:49.280
convention. We're doing this in AMQP. It's[br]very easy.

0:09:49.280,0:09:50.630
So we create a response queue, we get the

0:09:50.630,0:09:52.000
name of that queue, and then as part of

0:09:52.000,0:09:53.950
the message we send to our target, we say,

0:09:53.950,0:09:56.430
here's the response queue to reply back at.[br]And

0:09:56.430,0:09:58.350
then we simply wait for the response on that

0:09:58.350,0:10:00.250
call-back queue to come back. So that lets[br]us

0:10:00.250,0:10:02.710
get the synchrony that we're familiar with,[br]that we

0:10:02.710,0:10:06.160
need, with HTTP style programming, while still[br]having an

0:10:06.160,0:10:09.130
asynchronous system under the hood.

0:10:09.130,0:10:13.890
RACK, of course, we're all using RACK today.[br]Even

0:10:13.890,0:10:15.810
if you're not familiar with it. RACK, of course,

0:10:15.810,0:10:18.190
is what runs all Ruby web application servers.[br]Turns

0:10:18.190,0:10:21.120
out that RACK actually doesn't care about[br]HTTP. All

0:10:21.120,0:10:23.730
that RACK really defines is that you had defined

0:10:23.730,0:10:26.870
a call method that receives an environment,[br]which is

0:10:26.870,0:10:29.920
basically a hash, and it responds back with[br]a

0:10:29.920,0:10:34.080
three-element array. The HTTP response code,[br]headers, and the

0:10:34.080,0:10:34.770
body.

0:10:34.770,0:10:37.230
That hash doesn't actually have to be anything[br]but

0:10:37.230,0:10:39.650
a hash. It just has to look like an

0:10:39.650,0:10:42.850
HTTP environment. So emulation was actually[br]pretty simple. So

0:10:42.850,0:10:45.730
this is basically how it looks. We subscribe[br]to

0:10:45.730,0:10:48.070
a queue on the server, and then for every

0:10:48.070,0:10:51.290
message we receive back for that queue, we[br]unpack

0:10:51.290,0:10:53.910
some things into variables. We create what[br]looks like

0:10:53.910,0:10:56.910
the HTTP environment and we have the request[br]method

0:10:56.910,0:10:58.380
and query string. If you ever wrote CGI way

0:10:58.380,0:11:00.290
back in the day, this might look familiar[br]to

0:11:00.290,0:11:01.900
you.

0:11:01.900,0:11:04.000
And then we essentially pass that environment[br]onto our

0:11:04.000,0:11:06.700
RACK application, receive the response from[br]the RACK app

0:11:06.700,0:11:09.910
and then publish that back to the response[br]queue.

0:11:09.910,0:11:11.850
So this is what Jackalope is doing for us.

0:11:11.850,0:11:15.770
Jackalope is emulating HTTP for our RACK applications.[br]And

0:11:15.770,0:11:18.540
it just works. I'll show you a demo at

0:11:18.540,0:11:20.370
the end. You don't have to modify your code

0:11:20.370,0:11:22.560
at all. You can just simply deploy to it,

0:11:22.560,0:11:26.110
and it's speaking AMQP instead of HTTP. So[br]that's

0:11:26.110,0:11:28.730
it. We're done. We can go on vacation, right?

0:11:28.730,0:11:31.860
Well. I didn't talk about how to actually[br]put

0:11:31.860,0:11:36.930
this in production, other than just use Jackalope.[br]So

0:11:36.930,0:11:40.510
you have to choose a broker in AMQP. So

0:11:40.510,0:11:43.220
what we're using is RabbitMQ. RabbitMQ is[br]a well-known

0:11:43.220,0:11:46.390
AMQP broker. If you're doing any AMQP you're[br]probably

0:11:46.390,0:11:50.770
using RabbitMQ, most likely. It's used by[br]giants out

0:11:50.770,0:11:53.320
there. I mean, Google, VMWare, NASA. All these[br]people

0:11:53.320,0:11:54.730
are using Rabbit pretty heavily.

0:11:54.730,0:12:00.110
It's extremely scalable. It's fault-tolerant.[br]It's distributable. And it's

0:12:00.110,0:12:02.570
secure. It does everything you'd ever want.[br]And it

0:12:02.570,0:12:05.100
also gives you a really great management console.[br]We

0:12:05.100,0:12:06.300
can go in and see what's going on in

0:12:06.300,0:12:08.370
your system. Like I mentioned, the distributed[br]logging we

0:12:08.370,0:12:09.700
saw in the, in the last talk, if you

0:12:09.700,0:12:11.270
were here for that, trying to get an idea

0:12:11.270,0:12:12.880
of what's going on across your system can[br]be

0:12:12.880,0:12:15.630
challenging. Rabbit doesn't tell you everything[br]you'd want to

0:12:15.630,0:12:17.180
know, but it at least gives you an idea

0:12:17.180,0:12:19.660
of what queues are being used. Their depths[br]at

0:12:19.660,0:12:22.080
the point. The kind of information that can[br]help

0:12:22.080,0:12:23.880
you get, at least get a handle on it.

0:12:23.880,0:12:26.980
Additionally, you're gonna have to think about[br]how you

0:12:26.980,0:12:28.550
talk to the real world. I mean, AMQP is

0:12:28.550,0:12:30.610
all well and good behind the scenes, but how

0:12:30.610,0:12:33.290
do I continue to interact and serve my clients

0:12:33.290,0:12:35.260
that I actually care about, to the point of

0:12:35.260,0:12:36.110
getting paid, right.

0:12:36.110,0:12:39.070
So this is the architecture we typically use[br]at

0:12:39.070,0:12:41.470
Optoro. We have our Rabbit server in the middle,

0:12:41.470,0:12:43.790
and all the various services sitting on Jackalope[br]that

0:12:43.790,0:12:45.850
talk to Rabbit. And then additionally we have[br]one

0:12:45.850,0:12:48.320
or more API services that are continuing to[br]be

0:12:48.320,0:12:50.960
deployed on Unicorn, in our case. They talk[br]to

0:12:50.960,0:12:54.820
the outside world and internally translate[br]the, the needs

0:12:54.820,0:12:57.740
through AMQP and the rest of the system.

0:12:57.740,0:12:58.920
One of the things we actually published for[br]each

0:12:58.920,0:13:01.600
of our services is a client gem that kind

0:13:01.600,0:13:04.600
of isolates the worry, the worry about exactly[br]which

0:13:04.600,0:13:07.710
transport we're using away from the consumer,[br]the API

0:13:07.710,0:13:10.890
here. So it actually is, is really simple.

0:13:10.890,0:13:12.850
And so the way that we do that is

0:13:12.850,0:13:15.860
we typically have been using HTTParty for[br]our HTTP

0:13:15.860,0:13:18.760
communication needs. So we wrote, as one of[br]the

0:13:18.760,0:13:22.940
other projects part of RACK-AMQP, is AMQParty,[br]which is

0:13:22.940,0:13:26.790
just a API-compliant version of HTTParty.[br]So we can

0:13:26.790,0:13:30.620
actually just drop in the AMQParty constant[br]in place

0:13:30.620,0:13:32.800
of the HTTParty constant, and you can see[br]down

0:13:32.800,0:13:35.610
below, that transport variable, we typically[br]configure that at

0:13:35.610,0:13:39.310
boot time to be either HTTParty or AMQParty.[br]And

0:13:39.310,0:13:40.920
everything, again, just works.

0:13:40.920,0:13:43.100
We change the, the url a little bit, too.

0:13:43.100,0:13:44.670
But we do that all part of the, part

0:13:44.670,0:13:47.430
of this setup for each server. For each client.

0:13:47.430,0:13:51.620
Of course, if you don't want to use HTTParty,

0:13:51.620,0:13:54.000
some people like the various other options[br]that are

0:13:54.000,0:13:57.000
out there, we also are publishing RACK-AMQP[br]client, which

0:13:57.000,0:13:59.600
is what H, AMQParty is built on. It, it's

0:13:59.600,0:14:03.040
a very simple API. You get a client, you

0:14:03.040,0:14:06.920
tell it which Rabbit to connect to. You send

0:14:06.920,0:14:09.910
a request to your target queue with your uri

0:14:09.910,0:14:12.410
as part of it, with any, you know, your

0:14:12.410,0:14:13.680
HTTP method and all that. You get back your

0:14:13.680,0:14:16.190
response. It's synchronous. It's simple. It's[br]actually built on

0:14:16.190,0:14:18.670
Bunny, which is a really great AMQP gem out

0:14:18.670,0:14:21.780
there, and that also is very easy to use.

0:14:21.780,0:14:27.540
Additionally, we're publishing a sample SOA[br]using Rails. It's

0:14:27.540,0:14:32.790
a work-in-progress at the moment. BUt the[br]userland and

0:14:32.790,0:14:34.430
userland_client are mostly built as, you can[br]see where

0:14:34.430,0:14:37.120
things are at the mo- at the current state.

0:14:37.120,0:14:38.600
It kind of gives you an idea about how

0:14:38.600,0:14:41.510
we think about SOA at Optoro, and how you

0:14:41.510,0:14:45.350
might be able to use Jackalope and the RACK-AMQP

0:14:45.350,0:14:47.050
project in your own projects.

0:14:47.050,0:14:52.190
So userland is a, is Rails service that essentially

0:14:52.190,0:14:54.560
publishes a user's concept out into the world.[br]And

0:14:54.560,0:14:56.790
then userland_client is a gem that consumes[br]the userland

0:14:56.790,0:15:00.260
service and interface uses the userland gem[br]to talk

0:15:00.260,0:15:02.100
to the userland service.

0:15:02.100,0:15:07.330
And so then, you know, how fast is this?

0:15:07.330,0:15:11.060
Well, there's this bench mark I've got. I'll[br]show

0:15:11.060,0:15:13.480
you it in just a sec. The only weird

0:15:13.480,0:15:15.020
thing we have to do here is tell AMQParty

0:15:15.020,0:15:16.320
which Rabbit to talk to, and you kind of

0:15:16.320,0:15:18.040
saw that in the, in the client gem a

0:15:18.040,0:15:19.710
little bit, so that's a little bit of setup

0:15:19.710,0:15:21.290
we do. Again, like I said, when we make

0:15:21.290,0:15:24.300
the decision about which, whether HTTParty[br]or AMQParty. So

0:15:24.300,0:15:26.270
we tell it which Rabbit. And then, inside,[br]we,

0:15:26.270,0:15:30.620
five hundred times, request this JSON using[br]AMQParty and

0:15:30.620,0:15:33.170
then five hundred times request the same thing[br]using

0:15:33.170,0:15:33.450
HTTParty.

0:15:33.450,0:15:37.450
So here, I'll show you that in, in use.

0:15:37.450,0:15:41.360
Let's see. OK.

0:15:41.360,0:15:48.360
Let's make this big. All right. So here I'm

0:15:49.390,0:15:53.080
gonna boot the app using Unicorn. So Unicorn[br]is

0:15:53.080,0:15:57.080
listing on port 8080. Wow. That's really big.[br]Trust

0:15:57.080,0:15:59.550
me, it's 8080 there. I'm gonna boot the same

0:15:59.550,0:16:02.100
map using Jackalope. The only difference I[br]have to

0:16:02.100,0:16:03.280
do to boot the app is I have to

0:16:03.280,0:16:05.330
say with the right queue name, so it's going

0:16:05.330,0:16:08.120
to listen to the test dot simple queue.

0:16:08.120,0:16:10.420
So there it is running. And then down here

0:16:10.420,0:16:13.170
I'm gonna starty the bench mark. And like[br]I

0:16:13.170,0:16:16.390
said, it's gonna five hundred times hit each[br]service.

0:16:16.390,0:16:18.630
What it actually does is hit each service[br]three

0:16:18.630,0:16:20.380
times, just to make sure that it's warm, and

0:16:20.380,0:16:22.730
then it can go through and hit it all

0:16:22.730,0:16:24.650
the way through. And there we go.

0:16:24.650,0:16:26.940
So you can see, it's two point nine seconds

0:16:26.940,0:16:30.050
total to do five hundred AMQP requests, and[br]three

0:16:30.050,0:16:33.300
point six seconds total to do five hundred[br]HTTP

0:16:33.300,0:16:37.040
requests. I was actually surprised when I[br]got this

0:16:37.040,0:16:40.750
result. Jackalope is beta. I mean, it's, we're[br]using

0:16:40.750,0:16:43.930
it at Optoro, but it's not really been battle-tested.

0:16:43.930,0:16:46.950
But it's faster, at the moment. So why is

0:16:46.950,0:16:47.730
it faster?

0:16:47.730,0:16:51.670
Well, maybe I haven't written all the code[br]yet.

0:16:51.670,0:16:55.530
But more likely it, more likely it's because[br]of

0:16:55.530,0:16:59.800
this concept in TCP called the slow-start[br]phenomonon. And

0:16:59.800,0:17:02.630
also, your whole TCP negotiation. So each[br]one of

0:17:02.630,0:17:06.699
these five hundred HTTP requests, close the[br]connection, open

0:17:06.699,0:17:10.770
the connection, send the message, receive[br]the response. When

0:17:10.770,0:17:12.799
you open the TCP connection you go through[br]a

0:17:12.799,0:17:14.970
whole sinaq phase, which I didn't put slides[br]in

0:17:14.970,0:17:17.309
for and maybe should have. But it's.

0:17:17.309,0:17:20.630
I highly recommend checking out Eliot Gregorich's[br]talks, by

0:17:20.630,0:17:23.679
the way, about how HTTP works, how TCP works.

0:17:23.679,0:17:27.069
But basically, you go through protocol negotiation[br]for any

0:17:27.069,0:17:32.279
TCP connection, and then there's a feature[br]of TCP

0:17:32.279,0:17:34.350
called the slow-start feature, which essentially[br]allows you to

0:17:34.350,0:17:37.639
negotiate your connection speed. And that's[br]really great when

0:17:37.639,0:17:39.600
you're talking over the internet, and you're[br]not sure

0:17:39.600,0:17:42.090
of the latency or the availability of routers[br]to

0:17:42.090,0:17:44.700
be decent along the way. But when you're doing

0:17:44.700,0:17:47.720
internal applications, it just gets in your[br]way. It

0:17:47.720,0:17:48.519
just slows you down.

0:17:48.519,0:17:52.370
So AMQP, we hold a persistent connection to[br]the

0:17:52.370,0:17:55.350
AMQP broker. We do one sinaq cycle, and then

0:17:55.350,0:17:57.990
we get our maximum connection speed and we[br]stay

0:17:57.990,0:18:00.309
there. And then we can multiplex information[br]over that

0:18:00.309,0:18:03.179
single connection and it works really well.[br]Now, you

0:18:03.179,0:18:05.070
could write that with HTTP if you did a

0:18:05.070,0:18:08.690
keep-alive connection, held over that connection[br]and talk to

0:18:08.690,0:18:09.559
the same thing back and forth.

0:18:09.559,0:18:11.279
But, the bottom line is that most of us

0:18:11.279,0:18:13.379
don't do that when we write HTTP. We're using

0:18:13.379,0:18:17.919
something like HTTParty or, I always forget[br]the tiforce

0:18:17.919,0:18:21.000
or tif- whatever it is. Sorry.

0:18:21.000,0:18:21.809
Most of the time, you know, we drop the

0:18:21.809,0:18:23.740
connection, you know, in between requests.[br]We don't hold

0:18:23.740,0:18:25.620
it open. It's just not something we think[br]about.

0:18:25.620,0:18:29.860
So using AMQParty or any of the libraries[br]that

0:18:29.860,0:18:32.360
we're published, it holds that open and it[br]just

0:18:32.360,0:18:34.539
makes it more efficient overall.

0:18:34.539,0:18:36.309
So I hope that's the primary explanation of[br]why

0:18:36.309,0:18:43.309
it's so much faster. So I just wanted to

0:18:44.340,0:18:45.929
mention a few references, a few things that[br]were

0:18:45.929,0:18:47.870
really inspriring when we kind of went down[br]this

0:18:47.870,0:18:49.879
road. You know, this is really a departure[br]from

0:18:49.879,0:18:53.340
what we're used to in Rails. And the Architecture:

0:18:53.340,0:18:55.970
The Lost Years talk and Matt Wynne's original[br]Hexegonal

0:18:55.970,0:18:58.320
Rails talks were all about departing from[br]what we're

0:18:58.320,0:19:00.429
used to, I think. So those are really interesting.

0:19:00.429,0:19:02.909
If you haven't checked those out, definitely[br]do.

0:19:02.909,0:19:05.340
Additionally, Martin Fowler posted a recent[br]article that was

0:19:05.340,0:19:08.450
awesome, called Micro Services. Definitely,[br]definitely read this if

0:19:08.450,0:19:12.190
you are looking into service-oriented architectures.[br]It lets you

0:19:12.190,0:19:14.789
not think about that heavy-weight bloat that[br]we think

0:19:14.789,0:19:16.649
about with SOA.

0:19:16.649,0:19:19.309
It's really a revisitation of the idea of,[br]just,

0:19:19.309,0:19:22.610
you have independent services that are collaborating[br]to get

0:19:22.610,0:19:24.820
something done in a very simple way. And that's

0:19:24.820,0:19:27.529
really, again, also influenced the direction[br]we're taking here.

0:19:27.529,0:19:30.899
We want to introduce as few specific choices[br]as

0:19:30.899,0:19:33.090
possible to just kind of give us a simple

0:19:33.090,0:19:36.679
transport to, to build on top of. Additionally,[br]the

0:19:36.679,0:19:39.529
Ruby AMQP gem has some awesome docs about[br]how

0:19:39.529,0:19:42.529
AMQP works, how to use it with Ruby. And

0:19:42.529,0:19:45.509
RabbitMQ also publishes some really great[br]documentation on what

0:19:45.509,0:19:47.620
AMQP is, how it works. How to use it

0:19:47.620,0:19:49.600
in the organization in your applications.

0:19:49.600,0:19:51.649
And there's also this really great article[br]I found

0:19:51.649,0:19:55.029
about how HTTP works, in general. It's, I[br]think

0:19:55.029,0:19:58.429
a piece of school material from a university[br]in

0:19:58.429,0:19:59.960
Singapore. Definitely check it out. It's got[br]really great

0:19:59.960,0:20:00.730
diagrams.

0:20:00.730,0:20:06.750
Thanks. Again, I'm Josh. I'm at jszmajda on[br]the

0:20:06.750,0:20:09.730
Twitters. I am the CTO at Optoro. We have

0:20:09.730,0:20:12.720
blinq dot com. We have retailers with their[br]returned

0:20:12.720,0:20:14.710
goods and excess goods, help them figure out[br]what

0:20:14.710,0:20:16.809
it is, what the current value is and move

0:20:16.809,0:20:18.070
it quickly.

0:20:18.070,0:20:20.129
Additionally, I am the host of the Ruby hangout.

0:20:20.129,0:20:22.480
Just a quick blurb for the Ruby hangout, it's

0:20:22.480,0:20:25.220
a online meetup. It's really handy when you[br]can

0:20:25.220,0:20:27.850
go face-to-face and talk to people if you[br]have

0:20:27.850,0:20:28.960
a local meet up. If you don't have a

0:20:28.960,0:20:30.629
local meet up or if you're too busy or

0:20:30.629,0:20:32.509
if you can't make it for whatever reason,[br]the

0:20:32.509,0:20:35.570
Ruby hangout is an online meet up to, essentially,

0:20:35.570,0:20:38.249
give you that local meet up feel, as much

0:20:38.249,0:20:40.059
as we can online. So check that out. It's

0:20:40.059,0:20:41.899
a lot of fun. I'm one of the co-organizers

0:20:41.899,0:20:44.110
of the DC Ruby Users group. And I want

0:20:44.110,0:20:46.399
to give a quick shoutout to Jonathan. Thanks[br]Jon.

0:20:46.399,0:20:49.559
He's on my team, helped me get Jackalope over

0:20:49.559,0:20:52.799
the, over the finish line. So super helpful[br]on

0:20:52.799,0:20:53.279
that.

0:20:53.279,0:20:55.649
All the code's on GitHub. Github dot com slash

0:20:55.649,0:20:59.419
rack dash amqp slash rack dash amqp is where

0:20:59.419,0:21:01.600
the RC lives and has bookmarks to the other

0:21:01.600,0:21:06.320
parts of the project. And I would love to

0:21:06.320,0:21:10.299
take your questions, because that talk was[br]way short.

0:21:10.299,0:21:12.840
And I've got, actually, a whole bunch of stuffed

0:21:12.840,0:21:16.100
Jackalopes up here for good questions. So[br]please feel

0:21:16.100,0:21:18.300
free to come grab one.