-
PAVAN SUDARSHAN: Hi. My name is Pavan.
-
ANANDHA KRISHNAN: And I'm Anandha Krishnan.
I'm
-
also called Jake. Not Anandha. I know. We
work
-
at MavenHive technologies.
-
P.S.: This is probably the only
-
talk where there are two speakers, and we
haven't
-
rehearsed who says what, so we'll be stepping
on
-
each other's toes, so yeah. Bear with us.
So
-
yeah. Quick disclaimer: Most of what we are
going
-
to talk about is actually platform and language
independent,
-
or at least, in a sense, learned. But the
-
reason we are talking here in a Ruby Conf
-
is because of the Ruby community. We really
think
-
that a lot of things we are going to
-
talk about resonates really well with the
community, and
-
we are hoping to, you know, drive a good
-
discussion, conversation, whatever it is,
from this audience. So
-
that's really why we are talking here on this
-
topic.
-
A.K.: And a lot of the points that
-
we're going to talk about kind of naturally
apply
-
to Rails and Ruby. And most of these we
-
learned, and we sort of experience in our
projects,
-
which are mostly in Ruby and Rails, so.
-
P.S.: Yeah, so let's start with condition.
We have screwed
-
up a lot in our careers. Right, between me
-
and Jake, we have no idea how many mistakes
-
we have made. And on those rare occassions,
we
-
have actually learned from it, or at least
we
-
would like to think so. So yeah, this talk
-
is about one such mistake from which we learned,
-
and yeah.
-
A.K.: And, yes, I think most of
-
it, we just put it up front, just based
-
on projects, and you know, as we were talking
-
about what happened with each of us, and things
-
like that. So yeah, just trying to make it,
-
you know, presentable and that stuff. But
we-
-
P.S.: Yeah, like, though we screwed up, we
would like
-
to believe that no employers or paranoid androids
were
-
hurt in the process of our mistakes, so yeah.
-
OK, about three months back, so why pharamcist
and
-
doctors, right? So about three months about
I was
-
in this pharmacy buying diapers for my daughter,
and
-
in walks this guy - he just goes straight
-
to the pharmacist and he's like, hey, can
you
-
give me something for a toothache? There was
something
-
very interesting and weird about this, and
Jake and
-
I, we carpool. So the next morning I was
-
just telling Jake, and between the two of
us,
-
like, between the two of us, we realized that
-
we have seen people ask for all sorts of
-
medicines in a pharmacy. Right. Headaches,
fever, like, like,
-
true story, I even once saw this guy with
-
a lot of cuts on his face from a
-
knife, and yeah. Insane. Insane, right. So
about, when
-
we were talking about this, we thought there
was
-
something fundamentally wrong with this. Is
there anyone here
-
who thinks that it's totally OK for you to
-
just walk up to a pharmacist and ask for
-
a medicine? Oh yeah, cool. Yeah, so. Nice.
OK.
-
Hold that thought. Yeah, but like. So what
we
-
think, yes, a pharmacy potentially has a cure
for
-
pretty much any, most ailments that you could
think
-
of, and but the important thing is, though,
you
-
need to know what ailment you have. Right,
there's
-
that small implementation detail, right. And
if it was
-
that easy for you to just go to the
-
right medicine and get it, this world would
be
-
filled with only pharmacists and not doctors,
right. Yeah,
-
so that's, so that's where the whole analogy
starts,
-
and then we'll get to how we connect to
-
what we're going to talk about.
-
A.K.: Yeah, and that's, that's where we, in
a sort of thought
-
that we use this metaphor to drive home that
-
point. Of course, a lot of you might have
-
your opinions about self-medication and the
whole thing. So
-
we'll stop it at this, and we will give
-
us, our definition of what we think about
these
-
two sort of mindsets actually are and, you
know.
-
So starting off with like doctors, right.
They don't
-
treat, rarely, they don't try to treat the
symptoms,
-
it's about how you deal with them. So they
-
go about just figuring out what the problem
could
-
be, you know, and then probably, you know,
a
-
lot of tests, make you run through a few
-
tests or try and figure out what's what, if
-
indeed it is the problem, and then try and,
-
based on that, prescribe a treatment and,
of course,
-
make sure that you're OK at the end of
-
it, right. The symptoms are gone.
-
P.S.: And we didn't take a look through a
medical textbook, so
-
we don't know if this is right, but assuming
-
it is, though, this is what worked for us,
-
so.
-
A.K.: And, again, in contrast, a pharmacist's
job,
-
we think very different, should be more about
understanding
-
the medicines, the medicines themselves. Probably
even figure out
-
what the disease is based on the medicine,
right.
-
But definitely they don't really think about,
you know,
-
what was the problem originally or what are
we
-
prescribing the treatment for. And they usually
don't do
-
it. Hopefully they don't do it.
-
P.S.: Yeah. OK.
-
So now with this context of what we mean
-
by a doctor and a pharmacist and medicines
and
-
self-medication, all right. Let's get back
to our mistake,
-
which we want to talk about. Right. So our
-
mistake that we want to talk about is a
-
way we have dealt with, or rather we used
-
to deal with bad symptoms on our code bases
-
and on our projects, right. So you, a lot
-
of times you see these in your code bases.
-
There's the symptom, or there are some issues,
right.
-
So we obviously had a lot of those, in
-
every single project we have worked on, and
this
-
is about how we dealt with that, right.
-
A.K.: Let's start off with one very simple
one, or
-
at least the one which was the most easily
-
expressible.
-
P.S.: Yeah, as in, Tejas Dinkar, who has
-
already been mentioned several times in different
talks, he
-
threatened us to throw off the stage if we
-
took anything more than twenty-nine minutes,
fifty-nine seconds. So
-
we had to like really dumbify our, you know,
-
anecdotes. But you learn, we have like a quick
-
mention of different things which we would
love to
-
talk about offline, but yeah. So thanks Dejas.
-
A.K.: So we'll get started on the first one.
So
-
this was a project where we had a familiar
-
problem of regression bugs. We added new features,
and
-
that kept breaking things. So this is what
we
-
designed. We want this down. We want the number
-
of bugs down from 10 to 5, you know.
-
At that time it was, like, let's set up
-
this goal, let's try and achieve this. What
did
-
we do? Oh, before that. Some facts about the
-
project, right. This was not a project we
started
-
on from scratch, it was a lot of legacy
-
code, a lot of code that we did not
-
understand, and probably that's why we thought
it was
-
bad. And the test coverage was-
-
P.S.: How many of you have taken over a code
base from
-
another team and thought the code base was
awesome?
-
Very small samples, so you realize what we
mean
-
when we thought it was bad. So.
-
A.K.: Sure.
-
So test-coverage was low, which was probably
one of
-
the reasons why people complained about the
code base,
-
of course. So what's your guess?
-
P.S.: OK, so the problem we had was every
time we checked
-
in something, built a new feature, touch any
part
-
of the code base, we ended up breaking a
-
whole bunch of other things. And we would
not
-
even know it right away, we would know it
-
over a period of time. Right, so this was
-
a regression problem. And given the facts,
what would
-
you probably have done?
-
A.K.: I'll try and just
-
go back again, and then hopefully forward
- we're
-
done [00:08:04]??.
-
P.S.: Yeah, based on some facts.
-
A.K.: Sure, so the low coverage was definitely
a problem.
-
We thought, I mean, everybody agreed that
we need
-
to start working on that, and you know, fix
-
the coverage. So we went in there, put the
-
coverage tool in place, you know. And then
we
-
decided we will write tests for every bug
we
-
caught. We got the coverage up, not very surprising,
-
I mean, we didn't manage to- you know, improving
-
the coverage-
-
P.S.: Yeah like, so we spend like the whole
time-
-
A.K.: over a period of time.
-
P.S.: OK, so, like so this was a problem,
-
right. When you look at that, when you try
-
to abstract the, into, like what our thought
process
-
was. It was something like this, right. Check.
Yeah.
-
So, we had a symptom. In this case it
-
was low test-coverage, and - Jake - and we
-
had, we decided on what metric and tool to
-
use. In our case it was a simple line
-
coverage, right and, archive?? [00:09:10],
this was back in
-
the days. And then we started solving that
problem
-
that we had, and then, you know, hopefully,
for
-
example, we were TDDing most of the new code
-
that we wrote, so coverage was good on that,
-
then start writing tests for things, which
were, any
-
part of the code base which we touched where
-
there were no tests, we started adding tests
there.
-
You know, a bunch of different things. So
basically,
-
like, the idea was to improve the coverage
and
-
keep on writing, right. So cool, so. And what
-
was the result? We ended up with, of course,
-
drastically improving our test coverage, so
we were in
-
the late '90s for most the part of the
-
code base, which was awesome.
-
A.K.: A hundred, man, a hundred.
-
P.S.: A hundred, yeah. Or, yeah, sure.
-
Very good coverage. But things got only marginally
better.
-
At this point, this was when we realized that
-
inspite of putting so much effort into actually
improving
-
our test coverage, our actual goal was to
actual
-
reduce the number of regression bugs. So we,
we
-
were still no better than what we started
off,
-
about two months back. So the developers were
generally
-
very happy, now they were doing a lot more
-
TDD, they have a lot, they had manag- successfully
-
convinced the manag- you know the product
manager and
-
the stake holders to spend more time on the
-
tech deck?? [00:10:29] and things like that.
They they
-
also were happy. But the project manager was
extremely
-
frustrated, because in spite of spending so
much effort,
-
there was no real benefit from any of it,
-
right. So it's like one of those very classic
-
moments where you know, in Dilbert where,
you know,
-
the, OK, in Dilbert developers are never happy,
but
-
at least here we were happy and the project
-
manager was sad, right. A.K.: And we weren't
happy
-
with the project managers as well.
-
P.S.: Yeah, and eventually we feel there was
something wrong,
-
it's not like we take pleasure out of it.
So, we
-
think this is a very big mistake, where we
-
spent almost two months of time without really
realizing
-
where we were going wrong, right. So this
mistake
-
and several mistakes across different projects
which Dejas won't
-
let us go into, ended up making us realize
-
something very basic, right. And that's, this
is basically
-
what, this is what we were going to say.
-
A.K.: OK, tell us a little bit.
-
P.S.: If we had like a lightning talk, this
is what
-
we probably the only thing we would have put
-
up and left. So never improve a metric. Solving
-
a problem should automatically imrpve the
metric that you're
-
measuring, right. So the focus is on, is never
-
on making a metric better. It's always about
solving
-
the problem. And, the metric-
-
A.K.: This is like
-
one of those, one of those things that is,
-
it's very easily said and-
-
P.S.: Yeah, and it-
-
A.K.: You have to, at least for us, we
-
always fell in that trap of-
-
P.S.: Yeah, like
-
it almost sounds like 'do the right thing,'
but,
-
yeah, like, it's very, it fits your common
sense
-
very well, but then when you're caught up
in
-
the daily, the day-to-day stuff in what you
do
-
in a project, it becomes very easy for you
-
to miss the point here. So yeah, like this
-
is really what, what is essence of what we
-
are trying to say, right.
-
A.K.: So, so what really happened here.
-
Let's go a little bit more
-
into what we were trying earlier and what
we
-
think we should have probably done. Instead
of something
-
like this, which, which actually ended up
attacking the
-
symptoms, or you know, targeting the symptom,
we want
-
to do something like this: There is a symptom,
-
so just like always-
-
P.S.: This is where the
-
whole, like, the pharmacist and the doctor
approach, yeah,
-
like, it's a very long-shot metaphor, we agree,
but-
-
A.K.: The doctor thinking, which we hopefully
want to
-
do, is first is just try and take a
-
guess at least at the problem, at least in
-
our context, maybe not the doctor's. But in
our
-
context, take a quess at the problem, right.
Think
-
what it might be. Then that hopefully will
tell
-
you what you could do to probably solve the
-
problem, solve, you know, that could be the
solution
-
which hope- will hopefully fix the problem,
right. So
-
this kind of very similar, we are iterating??[00:13:00]
over
-
this and hopefully, you know, improving on
what we
-
thought was the problem. And how do you know,
-
then, that you know we are in fact improving
-
on the problem? How do we know that is
-
the problem?
-
P.S.: Like the whole, how do you
-
define them, right. So, how do we define them?
-
A.K.: And that's where we think, that's where
we
-
think the metrics come in. Again, not metric,
hopefully
-
metrics, because that lets us measure the
problem. It
-
tells us at every point that, you know, you're
-
doing better, you know, it's improving. And
hopefully you
-
also, when it gets done, right. So, yeah.
So
-
this is, this is probably the approach that
we
-
would like to try on, try from now on,
-
also. And, right, there, you know, the problem
may
-
not be the, what is the one that we
-
started out to fix. Like, like, probably in
our
-
previous case, you know, it, you should always
be
-
open to the idea that the problem will, what
-
you thought was the problem was never the
case,
-
and it was not showing up. I mean, you
-
were trying to, you were seeing the metrics
improve,
-
but then the symptom never went away, right.
So
-
be open to the notion that the problem could
-
be different, in which case, the important
thing is
-
the solution is different and the metrics
are different,
-
right. So yeah.
-
P.S.: Any guesses on what could
-
have been the problem on that project? Where
the
-
regression bugs were written very high? It
was duplication,
-
as in there was, like, rampant duplication
all over
-
the place. And we would change something but
forget
-
to change the same thing in some other place.
-
But, and because we didn't know about the
code
-
base, yeah. We were just blindly adding tests.
And
-
incrimentally going through different places,
where each place where
-
we found that there were no tests, we added
-
a test, right. But that didn't really help
us
-
with actually solving the problem of duplication.
Yeah. So-
-
A.K.: The coverage number is something which
is, it
-
easily drives you to just keep adding the
specs,
-
and we will talk about, more about that soon.
-
P.S.: Yeah, so, if you really think about
it,
-
of, basically what, at least, we would love
to
-
believe that we have stopped doing this, right.
So
-
Yogi mentioned this in the panel discussion
yesterday, block-force
-
driven decisions, right. So that was really
what we
-
were doing, essentially. Like, we were a bunch
of
-
kids on this project, who'd see a problem
at
-
the first, the, at the first, trigger we would
-
just start crawling the web, start crawling
block force,
-
see different github projects, find a gem,
install it,
-
start, like, monitoring it, measuring it,
try to improve
-
it, you know. We don't really spending too
much
-
time into figuring out what was really the
problem,
-
right. And, especially, so in, OK - where
does-
-
Especially so in Rails projects, where, you
know, or
-
Ruby projects, where we believe that the number
of
-
gems which actually bundle best practices
is actually very
-
high, right. Here it's very easy for us to
-
fall into the trap of, OK, just choose a
-
gem, start using it, and then two months or
-
three months down the way you have no idea
-
where you used it in the first place. But
-
it's just there in your process, right. Yeah.
Like
-
at least we used to find ourselves in that
-
trap all the time. So yeah. This is basically
-
what we stopped doing. OK, at this-
-
A.K.: [indecipherable]
-
P.S.: So this, this dude is basically Hari,
-
he's sitting way over there. Yesterday we
were doing
-
a write-in??[00:16:14], and the only dude
that, after this
-
point it's fine, but it's getting monotonous,
it's very
-
black and white. And you need more images.
And,
-
like, Jake and I were really, we really think
-
that-
-
A.K.: That's our image, Hari!
-
P.S.: We really think that we don't know how
to add images
-
to a presentation. And we were like, OK fine
-
Hari, we'll just add your picture. And, yeah,
so
-
thanks for- and as you notice, we are very
-
receptive of feedback, so.
-
A.K.: He hasn't spoken yet,
-
but his is full of interesting ones.
-
P.S.: Like it would have been funny if like
his was
-
the presentation before we got the schedule,
but yeah,
-
anyway. He'll be talking next. So yeah, when
you
-
look at test coverage, right, how many of
you
-
think you understand test coverage very well?
Well enough.
-
OK, I mean, sure, yeah, like-
-
A.K.: This was one thing which at least took
us as a
-
very obvious metrics thing, which we always
get into,
-
and-
-
P.S.: When we were young and stupid as
-
against now old and stupid, right, we were,
we
-
used to think oh, test coverage, what's that,
it's
-
just - meh. It's so easy, right. But then
-
we realized even something so seemingly obvious
had, like,
-
so many different shades of details. And once
you
-
start understanding it and interpreting it
in context is
-
when you really understand the, the complexity
of that,
-
of that thing that you're measuring, right.
Now think
-
of all the things that people at Flipkart
measure.
-
I don't even know if they have a rational
-
behind every one of it. I'm hoping they do,
-
but you know like, it becomes very OK, we
-
just need to monitor these ten things. Why?
Why
-
are you doing it? Right. So it should not
-
be like a checklist at every project on start.
-
You just start using it. So yeah, test coverage
-
is definitely one thing that we found where,
on
-
start of every project we just set up a
-
coverage tool. We just wanted to talk about
some
-
details on what we learned when doing that,
so
-
yeah.
-
A.K.: So first we want to start off
-
the obvious one. The measuring. So controller
specs versus
-
model specs coverage. I- I'm guessing it,
does it
-
ring any bells? I'm- so, so I'm, think of
-
it this way, like, he has a question for
-
you, right? You have, let's take a simple
case.
-
There is a single controller, you have a spec
-
around it, there's a corresponding module,
you have a
-
spec around it, right. And then, you, with
these
-
two tests you run your coverage, right.
-
P.S.: And you get some coverage from-
-
A.K.: I'm guessing it's going to be a hundred
percent.
-
P.S.: Do you see a problem with this? Could
there, rather, OK,
-
could there be a problem with this?
-
A.K.: What if you just removed the model spec?
-
P.S.: What will the coverage for model be?
Is there a
-
chance that model's coverage is not zero?
-
A.K.: Your controller spec is still gonna
be loading the model.
-
So your coverage-
-
P.S.: Depends on-
-
A.K.: -is still in question.
-
P.S.: The answer is it depends, right,
-
like, really depends on how you are testing
your
-
controller. But most of all things, what we
have
-
seen is, not every model is mocked out in
-
the controller. Well, it's a totally different
debate, whether
-
should you mock your models or not, but if
-
you are not modelin- or, mocking, your models
are
-
being loaded in your controller. So the controller
spec,
-
when it is tested for, like when the coverage
-
is reported your models are being reported
as well.
-
A.K.: While, while the, or the point of, the
-
point we are trying to make is, here, is
-
not whether you should, how you should test
your
-
model specs and controller specs. What we
do implore
-
you to do is make sure that when you're
-
doing, when you're looking at your coverage,
you do
-
have in mind your testing strategy, which
is, am
-
I actually mocking the model out or is it
-
also getting wrote as a part of my controller
-
spec? Is my controller spec also hitting the
models,
-
right? Think about these things when, when
you're looking
-
at these numbers, right. Or something that
worked for
-
us which we tried to do was we started
-
monitoring the model specs coverage independently,
and then started
-
looking at the controller specs in light of
the,
-
in light of the model spec coverage. We wanted
-
the model spec coverage to be high, because
at
-
least we wanted all, hopefully all our business
logic
-
was in the model specs, and you know, that's
-
what we were keen on. Yes. Yeah, and then
-
the next one, the line coverage itself, I
think
-
most commonly when we talk about coverage
we just
-
talk about line coverage. But then there is
a
-
bunch of other things as well, branch coverage,
and
-
then unique path coverage.
-
P.S.: How many of you
-
pay attention to branch coverages?
-
A.K.: Or even monitor it?
-
P.S.: How many of you don't think it's
-
important? How many of you have no opinions?
Cool.
-
Yeah. I mean, sure. We have it on projects
-
where it's been important, it's not been important,
it's
-
fine. But it's just that, you just need to
-
know that it exists, and you need to train
-
your data, right, so.
-
A.K.: Just, hopefully it should
-
not be a single metric. Something usually
seems wrong
-
if it is just gonna be about that one
-
metric. Next one. So reporting. Yeah, this
one is
-
a bit tricky. What I usually don't like about
-
coverage tools and these tools in general
is they
-
sometimes miss out this aspect of it. And
they,
-
in an attempt to try and be nice to
-
you when you are very simple answered is try
-
and give you a number which inherently makes
it
-
good or bad. There's nothing in between, and
then
-
the focus is lost. Like you either start liking
-
it or you don't like it. You don't really
-
think about what is there in between, right.
Yeah.
-
So the focus on, focus on some of the
-
aspects of where is the coverage, you know,
what
-
does it mean, right. One thing that worked
for
-
us was code climate in the region projects.
I
-
really like it because they put a lot of
-
focus into the presentation aspect of it.
Not just
-
the collection aspect of it. It really, you
know,
-
takes you down to the, to the code, which
-
is missing the specs. Of course, they do also
-
other metrics like code quality, which I really
like,
-
by the way. They have some notification things
like
-
that, like, on, like Lexus a lot, whenever
this
-
climate goes in, poof the coverage goes down
or
-
something like that. [00:22:19] It doesn't
break the builder
-
part, it doesn't break the build, but it lets
-
you know, and then you can deal with it
-
if you think it is important to deal with.
-
P.S.: OK, speaking of breaking build, how
many of
-
you know what racheting, in builds? OK. So
the
-
idea of racheting is basically you will never
leave
-
your code base worse than what it already
was,
-
right. So every commit basically makes sure,
even if
-
it doesn't do any good, it doesn't do any
-
bad to your code base. So for example, if
-
you're, your current code coverage is at 70%,
and
-
if this check-in makes it 69%, it will break
-
the build. Even though there's nothing functionally
wrong with
-
it, you know, it's bad, right. We really think
-
it's a double-edged sword. I will, this is
one
-
of those things which I have a, in theory
-
sounds very, very good, and direct, but in
practice,
-
what it typically ends up doing is people
end
-
up fretting the actual metric, and never about
what
-
the problem is, right. Becau- this exactly
does what
-
we said in the previous slide, which is coverage
-
is never red or green, right. Sometimes you
are
-
OK with taking this hit because you want to
-
do something. I mean there are all, there
are
-
so many reasons why which, in practic- in
reality
-
you may have to do some bad things, and
-
eventually have to pay for it, but it's OK,
-
it's a conscious decision, right. But racheting
invariably stops
-
that. It makes it very, you know, black and
-
white, right.
-
A.K.: Difficult for you to proceed at
-
that very moment.
-
P.S.: Yeah and it has a
-
more behavioral impact on the team, which
is your,
-
your team members start hating either the
racheting, or
-
they start hating the metric, or you know,
we
-
had this one person who did not commit for
-
four days because they thought they did not
have
-
enough test coverage. And, like, when we breached
the
-
whole, OK, frequent check-ins, call check-in,
and this person
-
was actually scared because they were about,
they would
-
break the build, is actually a, very, very
bad
-
signal, of, a bad sign from your team, right.
-
Now well, you can always argue, OK, this person
-
totally missed the point, we can say a bunch
-
of things, right, but this is the reality.
So
-
we think that it's a good idea, so that
-
you might want to do it at certain points
-
in time. But yeah, be very, very careful about
-
the freakanomic implication that it has on
your team,
-
right, always keep that in mind.
-
A.K.: And then
-
the very other popular thing, which is, you
just
-
write that test to bump up the coverage, if
-
you're not just-
-
P.S.: Yeah, so there was, there
-
was just one more philosophy of full-on coverage??
[00:24:50],
-
which is saying, OK, you just checked-in code,
but
-
you don't have a test for this, but that's
-
OK. There's this other class which does not
have
-
a test for it, which is easy, so let's
-
start this there and keep my coverage maintained
right
-
there. There's so many, like, weird things
people end
-
up doing, just because they are now worried
about
-
coverage and not really worried about what
it means
-
to- you know, what it means with what they
-
are doing, right. So it's-
-
is done with the best of intentions, but then
-
you get really, really bad, cause, like-
-
P.S.: Yeah.
-
So, OK, how, how did we go, I mean,
-
how do we now improve, like, so we still
-
take on a lot of projects, customer, we are,
-
we are mainly now in consulting, so we do
-
end up taking worker code bases, right. So
what
-
do we do to improve coverage if we have
-
a bad one? One thing we realized is adding
-
unit tests to classes, existing classes, could
be a
-
very dangerous thing, right. What that essentially
means, is
-
you have, you know, you are cementing the
current
-
design. Now I won't say, like prejudiously,
it might
-
be bad to say not a good design, right.
-
But you are cementing it, right. If ever you
-
want to cement something, cement features.
Cement functionality, right.
-
Which means you might want to write like a
-
much higher level test of, and, you know,
ensure
-
that the functionality is the same, so that
you
-
can go and refactor later. On this one project,
-
which Yogi, me, and Jake worked together on
in
-
ThoughtWorks, it worked beautifully for us,
where we were
-
strangling ?? to Hibernate [00:26:14]. Which
meant that all
-
that entire unit tests and database level
tests were
-
completely invalidated, they were useless,
right. And because of
-
the way, and because we wanted to turn on
-
to caches?? [00:26:24], transactions changed.
Like that meant the
-
whole app was rendered useless from a testing
point
-
of view, right, from a safety net point of
-
view. But what came to our help was controller
-
and beyond level tests. So our biggest, we
had
-
such good coverage there, that we went in
and
-
we just modified a whole bunch of things,
and
-
we like started deleting tests, you know.
You get
-
a lot more flexibility and freedom inside
your code
-
base when you know that you're not breaking
any
-
functionality. So yeah like it's a really,
really, like,
-
good thing, so definitely think about it when
you're
-
inheriting a legacy codebase.
-
A.K.: Unit tests, unit tests
-
are great, but do keep in mind that they
-
might also come in the way of-
-
P.S.: Changes.
-
A.K.: Refactoring, and changes.
-
P.S.: Yeah.
-
A.K.: Big, big refactoring-
-
P.S.: So, OK. What - the whole measurement
-
reporting, racheting, improvement, all of
this is basically saying,
-
always keep the problem you're solving in
mind, right.
-
Rachet don't rachet - how do you decide? Well,
-
is it helping you to achieve your goal, or
-
the problem you have at hand? Sure, you know,
-
so do it. Otherwise don't do it, right. It's-
-
Yeah.
-
A.K.: I think-
-
P.S.: So, the second-
-
A.K.: Let's-
-
P.S.: -anecdote- I'll just be, quickly talk
about-
-
A.K.: We have five minutes, so-
-
P.S.: Yeah, like,
-
the second anecdote was basically, OK, we
had this
-
server which was under a very heavy load.
And,
-
like thousands of requests, a minute, and
only about
-
5% of those requests, in very seemingly arbitrary
periods
-
of times, and arbirtrary controllers and actions,
would pause,
-
and it would take a very long time, right.
-
So the- this problem was way more technical.
It
-
had nothing to do with the behavior or, you
-
know, like, or practices part of, or side
of
-
things. It was pure technical problem. And
goal was
-
for us to find the root cause of this
-
unpredictable behavior and fix it, right.
And, yeah, like
-
we were, like we can definitely talk about
how
-
we went into, like a lot of different solving,
-
a lot of different symptoms, at one point
even
-
suspecting JRuby. You know, like, so it's
very, like,
-
it becomes very hard for you to figure out,
-
OK, what is a problem, what is a symptom,
-
and like, go very methodical about it, right.
So
-
that's what this, this problem was going to
be
-
about. But let's take this offline. At this
point
-
we can-
-
A.K.: I mean, before that, yeah, let's
-
get out some questions.
-
P.S.: Yeah, let's take some
-
questions if you have any.
-
A.K.: Cool.
-
P.S.: Cool, thanks.
-
A.K.: Thanks.
-
P.S.: Thanks guys.