NEAL KEMP: Hi RailsConf.
Today we're going to be talking about
testing your services.
But before we get into that,
I want to introduce myself.
My name's Neal. I'm originally from Iowa,
so I feel at home here in the Midwest.
Now I live in California, in LA specifically,
and I'm a software developer and
independent consultant who does Rails,
JavaScript, HTML, CSS.
Basically if you name an acronym
I've probably coded in it at some point.
So, today's talk is going to be the what,
the why, and the how of testing your services.
This is not a talk about building testable services.
I could do an entire talk about that on
its own. It's also not necessarily a talk about
test-driven development. While I'm a practitioner, I don't think
that the principles applied here correspond directly to test-driven
or not test-driven development.
[laughter]
So. Got that one. Good.
So first we're gonna talk about the what. So,
we have to ask ourselves, what is a service.
So I break it down into two main categories.
First of all we have external services, like the
Twitter API, Facebook's API, or Amazon Web Services. And
the other category is internal software-oriented architecture, a buzz
word we all know and love.
So, basically for the purpose of this talk, it
means anytime you're making an HTTP request to an
endpoint in another repository. So, basically any network request
you're making outside of your application.
So now we're gonna talk about. Cause we need
some justification before we go ahead and test all
of our services without question. So, first we have
to ask ourselves, why are services themselves important? I
think a lot of these are pretty self-evident. It
allows us to build things faster. It allows us
to scale more easily. And we use them on
basically every application. I don't think I can personally
think of an app I've built in the past
few years that hasn't glued on multiple services.
And I'm also noticing that we're using more and
more services for every application. So I would argue
that services are critical to modern Rails development. So
we have to ask ourselves then, why is testing
those services important? Well, first of all, you should
be testing everything else, so why would it be
different for services? And services often compose critical features
of your application. For example, a stripe integration. If
your billing's down, you're gonna have a lot of
issues. You know, if you have an API request
to S3, you're not gonna be able to serve
images, if that's down.
And you might also encounter problems with these APIs.
I know I sure have. Basically, any time I've
worked with an API there's been some unexpected results.
So the first example I'm gonna take you through
is, you know, an internal API built by consultants
in another part of the company. So this is
the software-oriented architecture we're talking about. And they were
exposing this API for our Rails app to consume,
but we had issues all along the way, and
it served to increase the project length significantly. Sometimes
we'd have random null responses when we were supposed
to get objects. There is random inconsistencies where we'd
get weird symbols being printed out and different formatting.
And in general it was a catastrophe.
So it definitely lengthened the time to completion. And
this was a lot due to a failure on
our part to, you know, test the API thoroughly.
So we couldn't express to them, you know, the
problems we were having until we put it into
production. So this is one problem that could have
been solved by testing first.
So, now we're gonna talk about a few problems
I've had with external APIs. And I'm sure all
of you have encountered similar issues with APIs in
the past.
So do we have any NHL fans in the,
in the house here? Yeah? Chicago Black Hawks. Doing
pretty well in the play-offs so far. We'll see
how they go. I mean, obviously they're gonna get
crushed by the Kings in a few rounds here,
or the Sharks, possibly. But we'll see. I don't
want to start a sports rivalry today.
So you know, this basically ranged from small annoyances
to, you know, major issues with this API. So
we'd have annoyances like this, where some responses would
come back with an id for the team, and
others would come back with a code. And in
this case, both of them refer to Anaheim. So
this is a minor annoyance. You can code around
that.
Here, we have an undocumented bug, where basically the
goals were all supposed to be as a part
of an array, but if you only had one
goal, it would be an object. And sadly we
discovered this one in production, during a game. So
that wasn't ideal.
But, worst of all, after we had gone through
all of the trouble of fixing these, we realized
that there was no versioning on this API. So,
even if we fixed it, we might be fixing
it again a week later. So this is basically
what it felt like to work with their API.
So, another project I worked on, this is just
kind of a fun side project, was a Snapchat
API Client, so I could, you know, work with
the Snapchat private API. And, well, one of these
examples is extreme, in that there is haphazard documentation,
or no documentation, in this case. I think we've
all worked with APIs that have improper documentation.
But in this case we didn't even know what
the requests were, so we had to figure that
out. There's also bizarre obfuscation implanted inside of the
app itself that basically encrypted, on their iPhone, so
people like me couldn't go in and build things
like this. And there's a GitHub link if you're
curious.
So, now that we've talked a little bit about
why it's important and outlined some of the problems
you might encounter, we're gonna talk about how you're
actually going to test these.
So first we need to ask ourselves, what is
different about services than regular code that we're testing?
Well, first of all, we have external network requests
that are being made, and second of all, you
don't own the code, so you can't really do
unit testing on it. It all has to be
done from integration test perspective.
So, what I propose for your tests, in general,
is that you turn Airplane Mode on. This, I
find, is the best way to think about your
tests, because, first of all, failure is really bad
in testing, and you shouldn't be making any, any
network requests.
So I think of this kind of in two
ways. First of all, so it's Airplane Mode in
the test mode so you can't do these things.
But also it should be a test that you
can run on an airplane. Basically meaning that if
you're, you know, on a long trans-Atlantic flight or
in the RailsConf lobby, you can still make your
tests and, and they won't fail because of network
issues.
So this means you should not interact with your
services from your testing environment. And we have a
few caveats which I'll get into now. So, this
includes dummy APIs. So there's some API makers that
have their real API, and then they have a
fake API which you can hit with requests, but
it doesn't make any changes to your data.
So you can't hit those, because those are somewhere
else on the network. But I do allow you
to make pre-recorded responses to those end points, and
that means you can record them within your test
suite, which we'll get into in a bit more
detail later.
So, for these examples, I'm going to be assuming
that you're using Rails, obviously, and that you're using
rspec for simplicities sake. So, it's time to stub
out these requests.
So, when you're stubbing an object, you're basically -
for those who don't know - it's basically putting
like a fake object in front of that object,
so you're hitting that object instead of hitting the
real one and, you know, saving time with, like
setup processes and stuff like that. And we're doing
a similar thing when you're stubbing a request to
an endpoint, except we're saving a lot more time
when we're doing so, because we don't have to
make that additional network request.
So there's some libraries that include built-in stubbing. So
Typhoeus, if I pronounced that correctly, Faraday and Excon
are three examples of pretty widely-used HTTP libraries built
on top of net HTTP, I think, that have
built-in stubbing functionality.
But we can simplify this a little bit and
use something called webmock, which I'm sure many of
you have worked with in the past, which is
a general purpose stubbing library for your test suite,
so you don't have to learn each individual library's
stubbing methods.
So I'll take you through a quick example. Here
is basic spec_helper. Nothing really interesting about this, except
you have to include disable_net_connect! at the bottom. The
rest is boilerplate. So I've highlighted that. And, obviously,
with all of these examples, you should be putting
the gem in your gem file and bundle installing
before you start.
So, when you put this in your code for
the first time, you'll get a really great error
with this, and I really like getting these errors,
because it tells me exactly, in my code, where
I'm making network requests. So if you're not already
doing airplane mode tests, you should just plug this
disable_net_connect! in, and then you'll get this error, which
will tell you where you're making these network requests.
And it also is really handy, and it gives
you, actually, the request you're making at the bottom.
So you can copy and paste that into your
test in order to stub it automatically. And obviously
you'll have to collect the body and the headers
yourself, if you need to use those as well.
So, for the following examples, we're gonna use probably
the most, sorry, most simple FacebookWrapper ever invented. Basically,
all we're doing here is sending a GET request
to Facebook graph, the public API, for a user.
And what this does is it just returns, like,
very basic Facebook information about you. It has your
Facebook username, your name, and an id and a
few other fields.
And then what we're doing with user_id up at
the top is we are just pulling out the
value for key_id. So this, all it does is
return your Facebook id. Super, super simple. And make
sure, since we're putting it in lib, that you
require it at some point in your loading.
So now we're gonna look at a test for
this. So this is a test where we're not
making a network request, but we're stubbing it out
with webmock. So, at the bottom, you can see
we're doing our testing case, and we're setting up
an expectation that our user_id is equal to Arjun's
user_id. And I'm using Arjun because he was the
maker of the Facebook graph API wrapper.
And, you can see, now, above, we are stubbing
the request, just like you'd stub an object. We're
stubbing the method of the HTTP request and then
we're send, putting the link as the second argument.
Next, we have to set up what it returns,
and this is just an HTTP response that we're
returning. So we want to put a status.
You can set headers, which I generally don't do,
but if you're doing any operations with the headers,
you should definitely set up these in your responses.
And the body, we have a really simple JSON
string. I've cut out a few fields for brevity.
But you can see it has an id, a
first_name, and a username.
So, this test will pass, and we're also making
no net requests, work requests. So, the reasons it's
better is it, first of all, it's faster, and
we also aren't getting this intermittent failure that we
were talking about earlier from the network request.
So, that's a good general way, but there's ways
we can also save time with this. So a
lot of the really popular libraries for API wrappers
also include mock-services within themselves or as an additional
library on the side, and they use that for,
you know, internal testing with their gems. So I
recommend, if, if you can find one, to use
this before you can go and use webmock, because
it'll save you a lot of time.
And I'll take you through a quick example. So
we're gonna use Facebook graph mock here. And all
we are doing is putting it into spec_helper. We're
just including the methods and requiring it. Pretty straightforward.
And now we're gonna look at a spec.
So, basically, all we're doing is we're wrapping the
test case within a wrapper that mocks out the
request. So basically, all this one's doing is saying
we're sending a git request to Facebook graph back
slash Arjun, and then the third argument, in this
case, is users/arjun_public, which is where the JSON file
of this response is located in the gem.
So, you can also specify your own responses, and
I'd recommend you do that, because I found, actually,
some issues with the Facebook graph mock mocking, like,
responses, have some outdatedness in them.
So, but this, you know, example, I'm not gonna
take you through all of the gems that have
this. But this can go to show that there
are some benefits that you get from using this.
It's already stubbed for you. You don't have to
learn the API endpoints in order to use it,
and some of these provide prerecorded responses for your
use, so you don't have to go out and
collect these. So it's just a good way of
saving time, if you're using some popular libraries.
Next, I'm going to take you through sham_rack, which
is one of my more favorite ways of doing
this. I kind of find this to be a
fun way. Basically what sham_rock does, sorry, sham_rack does
is it allows you to mount rack-based apps, which
include Rails and Sinatra and others, and it allows
you to make requests against these fake apps.
So, in this case, we're going to get a
little help from Sinatra in order to stub out
these endpoints. So, spec_helper, the only thing interesting is
that we leave in web-mock. Pretty boring there. But
then we get to our fake. So I usually
just put this spec/support and then fake whatever it
is, in this case fake_facebook.
And this just means it'll be loaded when you
run your specs automatically. But it won't be, obviously,
loaded into your production or staging environments, or development.
So, in this case, at the top we can
see, we're calling sham_rack, and we're setting up the
endpoint which we're hitting against, which in this case
is graph dot facebook dot com. And 433 is
just specifying that we're using the HTTPS SSL link,
and dot Sinatra just means we're going to be
passing it in a Sinatra app.
So, basically, contained within this block is a Sinatra
app, and you can do virtually anything you can
do with a regular Sinatra app, which is really
cool. So you can just, you're just basically mounting
this and testing against it.
So, for those of you who don't use Sinatra
very much, all we're doing here is specifying with
the GET keyword that we're making a GET request
to back slash something, and just like Rails, when
you rake routes you'll see the parametrization of things
with a colon before it. We're doing the exact
same thing here with username.
So, you'll see, in the middle, in the link
we interpolate params username, and that's how you pull
that out. So this is essentially just returning a
string that is this response. You can obviously spice
this up by setting status codes, adding conditionals in
here if you need some more dynamic power, and
also setting up the headers. And you can also,
which I sometimes do this in my testing, is
back it with like a small yml database, so
you can get some more realistic data than just
a simple string.
So, that's the response. And. Now, when we're writing
our spec for sham_rack, all we're doing is keeping
it on this base level. We don't have to
wrap it with anything, because it will automatically, in
your tests, pick up the fact that you have
sham_rack mounted, and it will automatically hit against that
endpoint rather than hitting against the network.
So, you might ask, why is this better? I
think there are a few reasons. First, I find
it more dynamic. I find it more expressive as
well, and you can really add, you know, as
much functionality you need to test your integrations as
you want. And you can also back it with
yml if you need, you know, some pre-population of,
you know, real data. And it's also more readable.
Let's go back to this for a second. And,
you can see, like, reading through this is a
lot easier to parse through, and you know where
the API requests are being made to, versus the
stubbing we showed in the first example, with web-mock,
is a little bit hard to read. So that's
why I prefer to use this.
So next, we're going to talk about vcr, which
is a pretty widely-used gem. And this one has
some other benefits that I think are really important
to use. Basically it prerecords your responses, and we'll
take you through an example.
So, spec_helper. The only thing interesting here. We have
the vcr configuration block, and all we're doing is
setting a cassette library. So that's basically where these
responses will be saved. And then we're hooking into
web-mock, because that's the stubbing library we're using.
So, here's a spec. And as you can see,
it's really, really similar to the Facebook graph mock.
So basically what this does is, you're wrapping it
in a block with vcr. So vcr, what it
does, is it goes out to the network and
makes the request for you, in your testing environment,
and it pulls that response back and saves it,
in this case, at Facebook user_arjun.
And the nice thing about this is you don't
have to go out and collect your own responses,
which I find to be pretty tedious and also
error prone. But, it also means you don't have
to break airplane mode with your tests, because you
can run this before, and you can cache all
of the JSON responses and play them back in
your build. So when you're running it on Travis
EI or Circle or whatever you happen to use,
you're not gonna break your build because of network
failure. You're going to be using these cached responses.
And that also just allows you to verify the
responses. So, like I mentioned, it's a little error
prone. I've tried collecting these responses on my own
and, you know, sometimes I copy and paste them
wrong and come up with an issue. So this
kind of allows you to, like, have a nice
programatic way of pulling those in.
So, there's also an additional build process you can
add. So, for the NHL example I talked about,
the problem was there was no versioning. So what
you can do is, if you want bonus points,
and you are really dependent on an API that
doesn't have versioning, you can do some kind of
build process or, you know, test setup, where you're
basically running it outside of your normal test mode,
and you check the casettes for diffs, and verify
that the responses are not changed from before. So
this can help you avoid versioning issues. So I
recommend that if you're using something like NHL API.
So, the next one we're gonna briefly talk about
is puffing-billy. And, aside from having a really cool
name and a nice logo on their GitHub, this
is an interesting gem to use. We're not gonna
use an example here, but basically what it is
is for in-browser requests. So basically if you're having
integrations that are browser-based, you can record and reuse.
Just like vcr, and use those responses again.
So, I don't want you guys to think that
all of this has to be done in Ruby,
and that you have to use vcr to first
record your responses. There's a lot of tools out
there that will help you to collect these responses,
test API endpoints faster, and I want to share
some of those with you.
So, Chrome Dev Tools. Has anyone heard of this
in here? Yeah. Probably, probably all of you. But
this is the first one I'm mentioning because I
use it, probably every day. Obviously it gives you
a really nice way of viewing responses and requests
and resending them. So super useful. I'm not gonna
get too far in-depth in that one because I'm
assuming most people have worked with it. But it
doesn't hurt to mention.
So next, Postman. If you want to stay within
Chrome. This is an extension you can use. And
it basically gives you a user interface around running
these requests so that you can have kind of
an easier way to play with re, requests and
responses. And it allows you save them. It gives
you, you know, a time in milliseconds of completion.
And this one I think is, I was working
on a Tinder API Client for fun, so. That's
what these requests are for. So, that one's actually
up on my GitHub, too, if you're curious.
So, I use that a lot. But if you
like to stay command line based, I would recommend
HTTPie. It's basically an easier-to-use version of curl, and
it doesn't have, like, quite the archaic syntax curl
has. So, I think it's, you know, worthy. Worthy
of use. And, you know, it'd be easier, obviously,
you know, to run a script around this than
it would be to run it around Postman. So
if you need to do something more programatic, this
is probably your best option.
And, one last tool I really like to use
is called Charles. And Charles does a lot of
the same things as Chrome Dev Tools does, but
it acts as a proxy. So it basically captures,
you know, requests between you and your network. So
you can set this up to capture any request
from your Mac machine, or you can proxy in
your phone as well. So I found this really
valuable when I was testing out the request from
the Snapchat client, because it allowed me to see
what my phone was making for requests and record
those. And especially when we didn't know what the
request was, it was very helpful in that case.
And it's also cool because, you know, when you're
building an API on Ruby and you want to
build on iOS client with it, and you're not
really sure how often to pull and stuff, I
sometimes pull this up, and I'll just see what
other apps are doing. So it's a good way
of debugging other peoples work and, you know, seeing
how they're doing it well. So, I highly recommend
you check it out. It's pretty easy to use,
and you can use it with SSL requests as
well.
So, here's some additional reading. I know you won't
have time to write this all down. I'll post
the slides on my Twitter. But, next up, let's
bring it all together.
So, we went over the what, the why, and
the how of testing services. So, we've shown that
testing services is crucial. They make up really important
parts of your app, so skipping tests is pretty
dangerous. I'd have to say, if you're in doubt,
stub it out. Determine, when you're making choices between,
you know, web-mock, sham_rack, or puffing-billy, even, you want
to determine the amount of flexibility the need and
the amount of extra work you're going to have.
For example, it probably takes more time to make
a sham_rack server and have dynamic responses than copying
and pasting the request you get from the web-mock
error. So, you kind of just need to look
at the project you have and determine what use
case best fits these options.
And, also, record responses to save time. I wish
I would have started doing this sooner. It's, like,
super useful. I would highly recommend you do that.
And next up, after me, I'd recommend you stick
around. I had the pleasure of pairing with Austin
yesterday, and I think his talk plays off my
talk a lot, in that it talks a lot
about inconsistent test failures. And he goes a lot
more in-depth on, you know, other kinds of inconsistent
test failures. And you just should definitely stick around
if you have the time.
So that's it for today. Thank you for taking
the time to listen to my talk. And if
I don't get to your question, feel free to
shoot me an email, or if you just want
to chat. And you can also find me on
Twitter. So thanks a lot guys.