NEAL KEMP: Hi RailsConf.
Today we're going to be talking about
testing your services.
But before we get into that,
I want to introduce myself.
My name's Neal. I'm originally from Iowa,
so I feel at home here in the Midwest.
Now I live in California, in LA specifically,
and I'm a software developer and
independent consultant who does Rails,
JavaScript, HTML, CSS.
Basically if you name an acronym
I've probably coded in it at some point.
So, today's talk is going to be the what,
the why, and the how of testing your services.
This is not a talk about building testable
services.
I could do an entire talk about that on
its own. It's also not necessarily a talk
about
test-driven development. While I'm a practitioner,
I don't think
that the principles applied here correspond
directly to test-driven
or not test-driven development.
[laughter]
So. Got that one. Good.
So first we're gonna talk about the what.
So,
we have to ask ourselves, what is a service.
So I break it down into two main categories.
First of all we have external services, like
the
Twitter API, Facebook's API, or Amazon Web
Services. And
the other category is internal software-oriented
architecture, a buzz
word we all know and love.
So, basically for the purpose of this talk,
it
means anytime you're making an HTTP request
to an
endpoint in another repository. So, basically
any network request
you're making outside of your application.
So now we're gonna talk about. Cause we need
some justification before we go ahead and
test all
of our services without question. So, first
we have
to ask ourselves, why are services themselves
important? I
think a lot of these are pretty self-evident.
It
allows us to build things faster. It allows
us
to scale more easily. And we use them on
basically every application. I don't think
I can personally
think of an app I've built in the past
few years that hasn't glued on multiple services.
And I'm also noticing that we're using more
and
more services for every application. So I
would argue
that services are critical to modern Rails
development. So
we have to ask ourselves then, why is testing
those services important? Well, first of all,
you should
be testing everything else, so why would it
be
different for services? And services often
compose critical features
of your application. For example, a stripe
integration. If
your billing's down, you're gonna have a lot
of
issues. You know, if you have an API request
to S3, you're not gonna be able to serve
images, if that's down.
And you might also encounter problems with
these APIs.
I know I sure have. Basically, any time I've
worked with an API there's been some unexpected
results.
So the first example I'm gonna take you through
is, you know, an internal API built by consultants
in another part of the company. So this is
the software-oriented architecture we're talking
about. And they were
exposing this API for our Rails app to consume,
but we had issues all along the way, and
it served to increase the project length significantly.
Sometimes
we'd have random null responses when we were
supposed
to get objects. There is random inconsistencies
where we'd
get weird symbols being printed out and different
formatting.
And in general it was a catastrophe.
So it definitely lengthened the time to completion.
And
this was a lot due to a failure on
our part to, you know, test the API thoroughly.
So we couldn't express to them, you know,
the
problems we were having until we put it into
production. So this is one problem that could
have
been solved by testing first.
So, now we're gonna talk about a few problems
I've had with external APIs. And I'm sure
all
of you have encountered similar issues with
APIs in
the past.
So do we have any NHL fans in the,
in the house here? Yeah? Chicago Black Hawks.
Doing
pretty well in the play-offs so far. We'll
see
how they go. I mean, obviously they're gonna
get
crushed by the Kings in a few rounds here,
or the Sharks, possibly. But we'll see. I
don't
want to start a sports rivalry today.
So you know, this basically ranged from small
annoyances
to, you know, major issues with this API.
So
we'd have annoyances like this, where some
responses would
come back with an id for the team, and
others would come back with a code. And in
this case, both of them refer to Anaheim.
So
this is a minor annoyance. You can code around
that.
Here, we have an undocumented bug, where basically
the
goals were all supposed to be as a part
of an array, but if you only had one
goal, it would be an object. And sadly we
discovered this one in production, during
a game. So
that wasn't ideal.
But, worst of all, after we had gone through
all of the trouble of fixing these, we realized
that there was no versioning on this API.
So,
even if we fixed it, we might be fixing
it again a week later. So this is basically
what it felt like to work with their API.
So, another project I worked on, this is just
kind of a fun side project, was a Snapchat
API Client, so I could, you know, work with
the Snapchat private API. And, well, one of
these
examples is extreme, in that there is haphazard
documentation,
or no documentation, in this case. I think
we've
all worked with APIs that have improper documentation.
But in this case we didn't even know what
the requests were, so we had to figure that
out. There's also bizarre obfuscation implanted
inside of the
app itself that basically encrypted, on their
iPhone, so
people like me couldn't go in and build things
like this. And there's a GitHub link if you're
curious.
So, now that we've talked a little bit about
why it's important and outlined some of the
problems
you might encounter, we're gonna talk about
how you're
actually going to test these.
So first we need to ask ourselves, what is
different about services than regular code
that we're testing?
Well, first of all, we have external network
requests
that are being made, and second of all, you
don't own the code, so you can't really do
unit testing on it. It all has to be
done from integration test perspective.
So, what I propose for your tests, in general,
is that you turn Airplane Mode on. This, I
find, is the best way to think about your
tests, because, first of all, failure is really
bad
in testing, and you shouldn't be making any,
any
network requests.
So I think of this kind of in two
ways. First of all, so it's Airplane Mode
in
the test mode so you can't do these things.
But also it should be a test that you
can run on an airplane. Basically meaning
that if
you're, you know, on a long trans-Atlantic
flight or
in the RailsConf lobby, you can still make
your
tests and, and they won't fail because of
network
issues.
So this means you should not interact with
your
services from your testing environment. And
we have a
few caveats which I'll get into now. So, this
includes dummy APIs. So there's some API makers
that
have their real API, and then they have a
fake API which you can hit with requests,
but
it doesn't make any changes to your data.
So you can't hit those, because those are
somewhere
else on the network. But I do allow you
to make pre-recorded responses to those end
points, and
that means you can record them within your
test
suite, which we'll get into in a bit more
detail later.
So, for these examples, I'm going to be assuming
that you're using Rails, obviously, and that
you're using
rspec for simplicities sake. So, it's time
to stub
out these requests.
So, when you're stubbing an object, you're
basically -
for those who don't know - it's basically
putting
like a fake object in front of that object,
so you're hitting that object instead of hitting
the
real one and, you know, saving time with,
like
setup processes and stuff like that. And we're
doing
a similar thing when you're stubbing a request
to
an endpoint, except we're saving a lot more
time
when we're doing so, because we don't have
to
make that additional network request.
So there's some libraries that include built-in
stubbing. So
Typhoeus, if I pronounced that correctly,
Faraday and Excon
are three examples of pretty widely-used HTTP
libraries built
on top of net HTTP, I think, that have
built-in stubbing functionality.
But we can simplify this a little bit and
use something called webmock, which I'm sure
many of
you have worked with in the past, which is
a general purpose stubbing library for your
test suite,
so you don't have to learn each individual
library's
stubbing methods.
So I'll take you through a quick example.
Here
is basic spec_helper. Nothing really interesting
about this, except
you have to include disable_net_connect! at
the bottom. The
rest is boilerplate. So I've highlighted that.
And, obviously,
with all of these examples, you should be
putting
the gem in your gem file and bundle installing
before you start.
So, when you put this in your code for
the first time, you'll get a really great
error
with this, and I really like getting these
errors,
because it tells me exactly, in my code, where
I'm making network requests. So if you're
not already
doing airplane mode tests, you should just
plug this
disable_net_connect! in, and then you'll get
this error, which
will tell you where you're making these network
requests.
And it also is really handy, and it gives
you, actually, the request you're making at
the bottom.
So you can copy and paste that into your
test in order to stub it automatically. And
obviously
you'll have to collect the body and the headers
yourself, if you need to use those as well.
So, for the following examples, we're gonna
use probably
the most, sorry, most simple FacebookWrapper
ever invented. Basically,
all we're doing here is sending a GET request
to Facebook graph, the public API, for a user.
And what this does is it just returns, like,
very basic Facebook information about you.
It has your
Facebook username, your name, and an id and
a
few other fields.
And then what we're doing with user_id up
at
the top is we are just pulling out the
value for key_id. So this, all it does is
return your Facebook id. Super, super simple.
And make
sure, since we're putting it in lib, that
you
require it at some point in your loading.
So now we're gonna look at a test for
this. So this is a test where we're not
making a network request, but we're stubbing
it out
with webmock. So, at the bottom, you can see
we're doing our testing case, and we're setting
up
an expectation that our user_id is equal to
Arjun's
user_id. And I'm using Arjun because he was
the
maker of the Facebook graph API wrapper.
And, you can see, now, above, we are stubbing
the request, just like you'd stub an object.
We're
stubbing the method of the HTTP request and
then
we're send, putting the link as the second
argument.
Next, we have to set up what it returns,
and this is just an HTTP response that we're
returning. So we want to put a status.
You can set headers, which I generally don't
do,
but if you're doing any operations with the
headers,
you should definitely set up these in your
responses.
And the body, we have a really simple JSON
string. I've cut out a few fields for brevity.
But you can see it has an id, a
first_name, and a username.
So, this test will pass, and we're also making
no net requests, work requests. So, the reasons
it's
better is it, first of all, it's faster, and
we also aren't getting this intermittent failure
that we
were talking about earlier from the network
request.
So, that's a good general way, but there's
ways
we can also save time with this. So a
lot of the really popular libraries for API
wrappers
also include mock-services within themselves
or as an additional
library on the side, and they use that for,
you know, internal testing with their gems.
So I
recommend, if, if you can find one, to use
this before you can go and use webmock, because
it'll save you a lot of time.
And I'll take you through a quick example.
So
we're gonna use Facebook graph mock here.
And all
we are doing is putting it into spec_helper.
We're
just including the methods and requiring it.
Pretty straightforward.
And now we're gonna look at a spec.
So, basically, all we're doing is we're wrapping
the
test case within a wrapper that mocks out
the
request. So basically, all this one's doing
is saying
we're sending a git request to Facebook graph
back
slash Arjun, and then the third argument,
in this
case, is users/arjun_public, which is where
the JSON file
of this response is located in the gem.
So, you can also specify your own responses,
and
I'd recommend you do that, because I found,
actually,
some issues with the Facebook graph mock mocking,
like,
responses, have some outdatedness in them.
So, but this, you know, example, I'm not gonna
take you through all of the gems that have
this. But this can go to show that there
are some benefits that you get from using
this.
It's already stubbed for you. You don't have
to
learn the API endpoints in order to use it,
and some of these provide prerecorded responses
for your
use, so you don't have to go out and
collect these. So it's just a good way of
saving time, if you're using some popular
libraries.
Next, I'm going to take you through sham_rack,
which
is one of my more favorite ways of doing
this. I kind of find this to be a
fun way. Basically what sham_rock does, sorry,
sham_rack does
is it allows you to mount rack-based apps,
which
include Rails and Sinatra and others, and
it allows
you to make requests against these fake apps.
So, in this case, we're going to get a
little help from Sinatra in order to stub
out
these endpoints. So, spec_helper, the only
thing interesting is
that we leave in web-mock. Pretty boring there.
But
then we get to our fake. So I usually
just put this spec/support and then fake whatever
it
is, in this case fake_facebook.
And this just means it'll be loaded when you
run your specs automatically. But it won't
be, obviously,
loaded into your production or staging environments,
or development.
So, in this case, at the top we can
see, we're calling sham_rack, and we're setting
up the
endpoint which we're hitting against, which
in this case
is graph dot facebook dot com. And 433 is
just specifying that we're using the HTTPS
SSL link,
and dot Sinatra just means we're going to
be
passing it in a Sinatra app.
So, basically, contained within this block
is a Sinatra
app, and you can do virtually anything you
can
do with a regular Sinatra app, which is really
cool. So you can just, you're just basically
mounting
this and testing against it.
So, for those of you who don't use Sinatra
very much, all we're doing here is specifying
with
the GET keyword that we're making a GET request
to back slash something, and just like Rails,
when
you rake routes you'll see the parametrization
of things
with a colon before it. We're doing the exact
same thing here with username.
So, you'll see, in the middle, in the link
we interpolate params username, and that's
how you pull
that out. So this is essentially just returning
a
string that is this response. You can obviously
spice
this up by setting status codes, adding conditionals
in
here if you need some more dynamic power,
and
also setting up the headers. And you can also,
which I sometimes do this in my testing, is
back it with like a small yml database, so
you can get some more realistic data than
just
a simple string.
So, that's the response. And. Now, when we're
writing
our spec for sham_rack, all we're doing is
keeping
it on this base level. We don't have to
wrap it with anything, because it will automatically,
in
your tests, pick up the fact that you have
sham_rack mounted, and it will automatically
hit against that
endpoint rather than hitting against the network.
So, you might ask, why is this better? I
think there are a few reasons. First, I find
it more dynamic. I find it more expressive
as
well, and you can really add, you know, as
much functionality you need to test your integrations
as
you want. And you can also back it with
yml if you need, you know, some pre-population
of,
you know, real data. And it's also more readable.
Let's go back to this for a second. And,
you can see, like, reading through this is
a
lot easier to parse through, and you know
where
the API requests are being made to, versus
the
stubbing we showed in the first example, with
web-mock,
is a little bit hard to read. So that's
why I prefer to use this.
So next, we're going to talk about vcr, which
is a pretty widely-used gem. And this one
has
some other benefits that I think are really
important
to use. Basically it prerecords your responses,
and we'll
take you through an example.
So, spec_helper. The only thing interesting
here. We have
the vcr configuration block, and all we're
doing is
setting a cassette library. So that's basically
where these
responses will be saved. And then we're hooking
into
web-mock, because that's the stubbing library
we're using.
So, here's a spec. And as you can see,
it's really, really similar to the Facebook
graph mock.
So basically what this does is, you're wrapping
it
in a block with vcr. So vcr, what it
does, is it goes out to the network and
makes the request for you, in your testing
environment,
and it pulls that response back and saves
it,
in this case, at Facebook user_arjun.
And the nice thing about this is you don't
have to go out and collect your own responses,
which I find to be pretty tedious and also
error prone. But, it also means you don't
have
to break airplane mode with your tests, because
you
can run this before, and you can cache all
of the JSON responses and play them back in
your build. So when you're running it on Travis
EI or Circle or whatever you happen to use,
you're not gonna break your build because
of network
failure. You're going to be using these cached
responses.
And that also just allows you to verify the
responses. So, like I mentioned, it's a little
error
prone. I've tried collecting these responses
on my own
and, you know, sometimes I copy and paste
them
wrong and come up with an issue. So this
kind of allows you to, like, have a nice
programatic way of pulling those in.
So, there's also an additional build process
you can
add. So, for the NHL example I talked about,
the problem was there was no versioning. So
what
you can do is, if you want bonus points,
and you are really dependent on an API that
doesn't have versioning, you can do some kind
of
build process or, you know, test setup, where
you're
basically running it outside of your normal
test mode,
and you check the casettes for diffs, and
verify
that the responses are not changed from before.
So
this can help you avoid versioning issues.
So I
recommend that if you're using something like
NHL API.
So, the next one we're gonna briefly talk
about
is puffing-billy. And, aside from having a
really cool
name and a nice logo on their GitHub, this
is an interesting gem to use. We're not gonna
use an example here, but basically what it
is
is for in-browser requests. So basically if
you're having
integrations that are browser-based, you can
record and reuse.
Just like vcr, and use those responses again.
So, I don't want you guys to think that
all of this has to be done in Ruby,
and that you have to use vcr to first
record your responses. There's a lot of tools
out
there that will help you to collect these
responses,
test API endpoints faster, and I want to share
some of those with you.
So, Chrome Dev Tools. Has anyone heard of
this
in here? Yeah. Probably, probably all of you.
But
this is the first one I'm mentioning because
I
use it, probably every day. Obviously it gives
you
a really nice way of viewing responses and
requests
and resending them. So super useful. I'm not
gonna
get too far in-depth in that one because I'm
assuming most people have worked with it.
But it
doesn't hurt to mention.
So next, Postman. If you want to stay within
Chrome. This is an extension you can use.
And
it basically gives you a user interface around
running
these requests so that you can have kind of
an easier way to play with re, requests and
responses. And it allows you save them. It
gives
you, you know, a time in milliseconds of completion.
And this one I think is, I was working
on a Tinder API Client for fun, so. That's
what these requests are for. So, that one's
actually
up on my GitHub, too, if you're curious.
So, I use that a lot. But if you
like to stay command line based, I would recommend
HTTPie. It's basically an easier-to-use version
of curl, and
it doesn't have, like, quite the archaic syntax
curl
has. So, I think it's, you know, worthy. Worthy
of use. And, you know, it'd be easier, obviously,
you know, to run a script around this than
it would be to run it around Postman. So
if you need to do something more programatic,
this
is probably your best option.
And, one last tool I really like to use
is called Charles. And Charles does a lot
of
the same things as Chrome Dev Tools does,
but
it acts as a proxy. So it basically captures,
you know, requests between you and your network.
So
you can set this up to capture any request
from your Mac machine, or you can proxy in
your phone as well. So I found this really
valuable when I was testing out the request
from
the Snapchat client, because it allowed me
to see
what my phone was making for requests and
record
those. And especially when we didn't know
what the
request was, it was very helpful in that case.
And it's also cool because, you know, when
you're
building an API on Ruby and you want to
build on iOS client with it, and you're not
really sure how often to pull and stuff, I
sometimes pull this up, and I'll just see
what
other apps are doing. So it's a good way
of debugging other peoples work and, you know,
seeing
how they're doing it well. So, I highly recommend
you check it out. It's pretty easy to use,
and you can use it with SSL requests as
well.
So, here's some additional reading. I know
you won't
have time to write this all down. I'll post
the slides on my Twitter. But, next up, let's
bring it all together.
So, we went over the what, the why, and
the how of testing services. So, we've shown
that
testing services is crucial. They make up
really important
parts of your app, so skipping tests is pretty
dangerous. I'd have to say, if you're in doubt,
stub it out. Determine, when you're making
choices between,
you know, web-mock, sham_rack, or puffing-billy,
even, you want
to determine the amount of flexibility the
need and
the amount of extra work you're going to have.
For example, it probably takes more time to
make
a sham_rack server and have dynamic responses
than copying
and pasting the request you get from the web-mock
error. So, you kind of just need to look
at the project you have and determine what
use
case best fits these options.
And, also, record responses to save time.
I wish
I would have started doing this sooner. It's,
like,
super useful. I would highly recommend you
do that.
And next up, after me, I'd recommend you stick
around. I had the pleasure of pairing with
Austin
yesterday, and I think his talk plays off
my
talk a lot, in that it talks a lot
about inconsistent test failures. And he goes
a lot
more in-depth on, you know, other kinds of
inconsistent
test failures. And you just should definitely
stick around
if you have the time.
So that's it for today. Thank you for taking
the time to listen to my talk. And if
I don't get to your question, feel free to
shoot me an email, or if you just want
to chat. And you can also find me on
Twitter. So thanks a lot guys.