Hey, everyone.
I am David Heinemeier Hansson.
I'm here with the Martin Fowler and Kent Beck
to talk about "is TDD dead?" as the topic.
Coming off keynote I did at Rails conference and subsequent articles I posted on the blog.
I had chance to talk with both Kent and Martin
one on one in the days that followed.
and we got to discussed a bunch of these issues
and I got to clarify some thoughts and definitions that I had in my head
and got a lot wiser about where both Martin and Kent are on the issue
and Kent suggested that, "Hey. Maybe we should share that with world."
So that's what we're doing now.
I know bunch of people have been expecting to this is gonna be a big???
there'll be no boxing.
We'll explore bunch of definitions and problems and patterns we've seen.
and what's work for us individually and go from there
But to get started, I thought
I just run over brief recap of the problems that I've seen with TDD
use that to kick off the discussion
and really three major points that all focus on this discussion.
The first is the definition of TDD and its components
specifically unit testing.
I know the TDD and unit testing are certainly not the same thing
but TDD's often used to drive unit test
and as I was looking through
issues I've been in the Rails world around TDD
a lot of that came down to the definition of unit testing.
that people were defining unit testing as
you can't have collaborators
you can't touch the databases
you can't touch the filesystem
goal was to have a suite of unit tests so fast that they could complete in a blink of an eye.
I've seen arguments of
if it takes more than 300 milliseconds to complete the entire suite then it's way too slow.
I can't use it for anything.
and that just wasn't matching my useful definition of unit testing
and it wasn't matching what I was trying to get out of it.
and frankly I wasn't get anything useful out of unit tests that weren't talking to any collaborators.
Second, which follows from that is driving your architecture through Mocks.
If you want everything in your test suite to be so super fast
well obviously you have to talk to mocks
you can't talk to anything slow
and you can't talk to many layers at once
which leads to a mock sort of heavy and enabling architecture
repositories and hexagonal pattrerns
and all these other things that makes very easy to stuff and mock your way out of everything
and turn every test you could imagine into unit tests
MVC framework as I work with rails
the controller, "Oh, no problem. Just stuff out everything talking to the view"
"talking to the model and then that's gonna be great."
I was not seeing that being very great at all
the key point what I thought
this is test induced damage
that the architect is actually suffering damage from this drive towards unit testing all the things
and that the unit testing all the things is coming through the TDD pipe.
And then finally this sort of ..... TDD
is the read green refactor loop.
And that was more of, sort of personal thing.
We can all have our own personal preferences and how we develop things
in TDD red-green refactor seemed .... more of mandate
I never dealt with that for the bulk of the work that I do
at certain locations at work and at certain locations that flow was good
but I felt like there there's tons of code that I was writing were red green refactor
just not an enjoyable of way to do it.
and I didn't feel like I was going to a better place by going through that
So those sort of the three top-line problems that I think with the TDD
Let's sort of the open conversation up
and have Martin and Kent chime in perhaps one at a time
What is unit test?
What's useful definition of unit test?
What's useful definition of unit test and how most people currently defining it in you guys' view.
Well, I propose
in order to understand something
I often find it useful to know how it came about.
And the style of unit testing we familiar with is basically all Kent's fault.
Right? Because Kent did the framework, XUnit frameworks
whole push through extreme programming
so maybe we start with that
where those source of this style of unit testing come from.
You're the guy. You explain it.
Can you hear me?
Yep.
So when I was a kid
I would read books about programming
I was just fascinated by programming.
I'm talking about nine, ten, eleven years old
and my dad was a programmer
so there're lots of books about programming around.
And he had a book and I would read I didn't understand anything
but I was just sorta compulsively reading these books
cause I found the whole topics fascinating
there was a book that said, "Here's how you program."
"You type in the output tape that you expect your program to produce given some input tape"
"And in programming is, you just write program and tell it produces the output tape that you expected."
So that was kind of fixed way back in my brain as the flow of programming.
And it wasn't until after I had been testing for a while
that already come up with the first version of SUnit, the Smalltalk testing framework
that was the precursor to JUnit.
and I remember this idea of writing the test before writing the code
which makes no sense, right?
the test is gonna fail and you want the test succeed.
so why would you write it when you knew it was gonna fail?
So I just tried it.
One of my heuristics is, "If an idea is obviously bad, find the cheap way to try it
because if it turns out it's not bad then it's really interesting
if you have a good idea and it's obviously good
somebody else's probably tried it before.
So, I just figured I'd try and I think I built stack at first time.
and it worked really really for me.
i think there's aspects of my personality that make this kind of peicemeal style work really well
I have a lot of anxiety and I tend to get overwhelmed by big problems
and for me TDD solves those problems.
Even if I don't know how to implement something I can almost always figure out how to wright tests for it
and if I can't figure out how to write a test for it, I have no business in programming in the first places.
So, that's how got into it.
And it was definitely ground up
that is, I started with small green problems and gradually work my way to larger green problem."
Go ahead.
I was thinking about this on reflecting back on our early days together
and if I recall correctly when we began working
??? using extreme programming ???
we did the testing very strongly from the beginning
but it wasn't test first at beginning.
I've got this memory of you standing up and saying
"The stories are not done until the codes're written and the tests're written and the tests are passing.
So we have the notion that you delivered at the end of it
increment of functionality with code and tests together
but test first didn't actually followed till later.
I can't remember when that came because it seemed gradually appeared.
but I feel that at that very beginning the focus was on the tests.
on the tests with that but that's the first as it were
so I think one of the fundamental question is
as a programmer do you deserve to feel confident?
I certainly grew up in culture that didn't encourage that.
You know, you write some stuff and just hope it works
somebody else would tell you whether they thought it worked.
and you go home on Friday and just think
"Oh god. I just hope I didn't break something."
and so I think the going a step back from TDD or automated testing
"do you deserve to feel the confidence?"
"can you sleep at night?"
knowing that your code works
I think the answer should be Yes for programmer.
So if we agree on that then we can talk about how to achieve that.
TDD is one way to achieve that.
There's lots of other ways.
I think that it's a great point because
one of the things that really got me into programming was
was Ruby.
Right. So I've been programming before Ruby but
this language that has a very specific mission statement.
Programmer Happiness
Right. So I think those two are related
the programmer happiness is certainly related to feeling confident in your code
and confident in making changes to it
and confident that's going to work
But it's only one part of that picture
and early days Martin described were
"Oh. You're not done until you also have tests for piece of functionality.
I'm completely on board with that. Right?
Where I'm not on board is that
that whole feeling of feeling great about your development style
feeling programmer happiness as Matz would always put it.
Just I wasn't getting that from TDD, right?
So maybe that's some of the patterns we talked about
different people with different brain style, right?
Why do some people enjoy Python?
Why do some people enjoy Ruby?
and even harder for me to comprehend how some people enjoy Java.
But but we sort of have different ways of looking at things
and feeling good about them.
and what I was finding just from the red green refactor flow
was that the whole notion of writing to test first and seeing that fail
and doing that for every piece of code before I move forward
was just not natural
because a natural you can overcome just by familiarity
but even through familiarity it was just not a pleasurable flow
and I think that's where a lot of the explosive debate comes from
to some people writing your thesis first and filling in the implementation with that works really well.
That's exactly how their brain operates
For other people, me included, it doesn't work like that.
It's much more tactile experience that I actually have to sit in and write it out
that's how I think through it
I don't think through it by proposing hypothesis up first and then try to fill it in.
I think through it by writing it out
If I can't write test first I have no business writing it.
That's I totally understand how that you can arrived at
It's just not at all how I feel about it.
I have a hard time writing a test forward, if I don't see it first, right?
SO you have sort of these some people like it from
going through the test
and other people like going to the test.
I think .
certainly important to realize we're both getting to same place
getting to fell confident about the code
confident making the changes to the code
that's the part I'm not a fan of tying these two things together
I'm not a fan of tying
oh you just have to be confident to TDD
which, I know you're not doing, right?
TDD is one path of getting to that place
I've seen there're a lot of people are mistaken those two things
that you can't have confidence
you can't deliver incremental functionality with test
unless you go through this mandated well paid road of TDD
so, last night was hackathon at Facebook
and I have a project on a internal tool that I was working on.
and half of it I could use TDD for
and half of it I couldn't
so I had a big log that I wanted to process
and the entries in the log were not clearly specified anywhere
but I wanted to put them into a data structure
kind of funky new data structure
the data structure I could build with TDD is just fine
because I could break it down
ok, here's the main case
here's the corner cases
here's how I go from one to many
just nice sequence of tests
and I can get into real flow
just make one test work and the next one and the next one
so that feels a certain way to me
when I had to fiddle live data into it
it feels very different way
because I don't have that clear specification
I just have to run a few million log entries through this first
and see what blows up
so I put in logging
I put in exception handling
so any log entry that's kind of funky that isn't handle by the code
get saved some place
then I can't go back and look at those entries that I can't handle yet
and I can use them
but I can't write a test for it
because I don't even know what the inputs look like
and that feels different way to me
so with the parts of the code that I can use TDD for
I mean a real flow
write a test, make it work
??? 12:30
I really like that feeling
the ones where I have to run on live ???
see what happens
I fee a lot more anxiety
but you know it's called work for reason
so I'm going to use the principles of double checking of regression testing
of short feedback loops
and try to get the shortest possible feedback loop
out of this unpredictable input that I'm getting
so I had to mix the two styles in one night
and it doesn't bother me
if I can play classical and jazz then I'm in a better position to make music for more people
but when I can reduce a problem to this sequence of tests
drive the solution from this sequence of test
it has special kind of feeling
for me it works really well
if I think back to how learned a ??? map
I always used these examples
you know there'd be some general
you know, prove this statement
and I think well, "what's an example of that"
and then I come up with a few examples
and I convince myself either the statement was true or that wasn't
and then I can write the proof
but that's my personal flow
I work from specific to general
this kind of inductive style
and I understand not everybody likes that
what I think is interesting though
even if I've been in that flow too
I had been in flow where TDD was working our well for me
it was ??? I should describe ???
it was very clear input
and was very clear desired output
and there were not a lot of contacts??
and there was not a lot of environment
to operate it
it was very pure
and for that
that would probably one case where I do actually like that flow
what I find is a lot of my work is not like that
right? sort of fundamentally maybe not like that
and then the problem comes in what am I willing to sacrifice to get that
you can reduce any type of work
certainly in the web space ???if you doing to that flow
if you just, that's where the whole mocking thing comes in, right?
if you just find a way and work hard enough
you can remove all the sort of dependencies
all the environment and reduce it to just one piece of input and one piece of output
is the flow important enough that you want to do that as general case?
how much you willing to sacrifice to be in that flow more ???you work
for me the answer has been
"I'm not willing to sacrifice that much."
when the flow is natural
when the work is sort of almost algorithmic nature
separated from all this context
alright. great. let's switch into that mode
but forcing all sort of other kinds of program work into this mode
just such that we can't feel the TDD loop?
that's where really see that we're going off track
and just coming up with some
in my opinion nasty hats we're starting really bad trade offs
for other aspects of the design of the code ????
so I'm curious for both of you guys
how much you guys willing to sacrifice to get to that TDD flow?
are you willing to use heavy amount of mocking
do you find that generally you can get there
in, say, MVC style programming?
or where the boundaries for you guys.
Martin is waiting for me talk
so I guess I'm talking.
so as I see I do this all the time
it's a question of trade off
and my students see me do that at thousands times
so for me TDD is question of trade offs
and I think, David you pointed out an important part of trade off which is
to make a design intermediate results testable comes with a cost
it comes with a benefit too
and you gotta figure out whether the costs outweigh the benefits
sometimes that's a question of understanding how to design things well
so if I'm building a compiler
and I think well the only kind of test I can have are end to end tests
and then I start to build more and more cases of what the input looks like
and I realize, "oh, I have this intermediate representation of parse tree"
then now I can test parsing
now I have this two orthogonal dimensions
I have parsing and then given a parse tree
does it compute the right results?
and then I can do more testing at a finer grain
but it was a design insight that created that moment
and compilers have settled on that as being a reasonable trade off
"ok, it's worth having this intermediate representation"
"so that we can have testability and bunch of other stuffs comes along with it."
so I'm not willing to say
ok, there's some fixed boundaries and I don't wanna twist design just to make things testable
because I'm just missing some design idea
if I had better idea for the design
then I could have both ways
I could have something that was more static more flexible design and it was more testable at the same time.
but if I don't have that today
your point is exactly right
then what do you do?
do you mock absolutely everything?
my personal practice is I mock almost nothing
if I can't figure out how test efficiently with the real stuff
I find another way of creating feed back look for myself
I have to have feed back loop
and a feed back loop has to be repeatable
but like I just don't go very far down the mock path
look at code where you have mocks returning mocks returning mocks
and my experience is
if i use TDD I can refactor stuff
and then I heard this stories
People say, "I use TDD and now I can't refactor anything"
I couldn't understand that
and then I started to looking at their tests
well. if you have mocks returning mocks returning mocks
your test is completely coupled to the implementation. not the interface
but the exact implementation of some object
you know three streets away
of course you can't change anything without breaking the tests
so that for me is too higher price to pay
that's not a trad off I'm willing to make
just to get piecemeal development
And this is, I think, at the heart of much of this
is confusion of terminology
and what these different things are
when I read David's initial blog post
cause I didn't see his talk until last night
one of things that came through very clearly
was his criticism of
David, your criticism of TDD
and sort of design damage that comes through it
had in itself very much tide in a notion
with the strong desire for isolation of mocking
and there, very important point that
there is nothing sort of
with in the idea of how you do either TDD or unit testing
that says you have to have that kind of isolation
some people are very much in favor of it
others are not
and I remember in the early days of extreme programming
unit testing, people would criticize us
that saying, "hey you're not isolating your units properly"
"that's not unit testing"
and we had this whole conversation with 24 different definitions of unit testing or whatever it was
ans said, "Well, our style of unit testing"
"we don't bubble about the isolation"
"and you know, it's working well for us, thank you very much"
so that's one thing
whether TDD in unit testing should be tide with isolation
and I look at it as different schools of thought
and I'm with Kent
I hardly ever use mocks
but I know good people who do
so I don't want to shoot everybody who uses mocks
maybe give it 10 more years
and then ????will ??? out or something
but we'll see
then there's another thing which is distinction between having what I call self testing code and TDD
to me, it's really important have self testing code
the ability to be able to run a single command have your whole system self test itself
in an acceptable amount of time
that is really powerful
because then if you can do that you can refactor with confidence
you've got a good chance of keeping your code base healthy
and that means you're able to be fast deliver new feature, etc
that is really powerful thing
and if I wanna get a ???high horse and say you must do something
I might be inclined to get that particular high horse.
but to me at least
TDD is one way to approach that
TDD is very particular technique
and if done well, one of benefit is that it gives you self testing code
give you all those benefits as well
but self testing code is for me perhaps the primary benefit
and the two get comflated
now fundamentally if somebody get self testing code by another root
such as when we started
we're delivering the code and the test together at the end of iteration
and you've got that self testing confidence
then I'm actually pretty happy
I'm not gonna kind of feel upset that somebody got that by another root
and I share with David and Kent
there're problems where TDD doesn't work terribly well
sadly most of programming I do these days
is not conducive to TDD
and I miss it
because actually I really like the TDD flow
that really works for me
just as it works for Kent
but fundamentally we have to separate TDD from self testing code
on realize they have different benefit
so often get conflated together
and we have to separate the idea of fully isolated unit testing
from not isolated unit testing
and realize that people have different preferences around those
and that's where I see
that was what much of my reaction was against, right?
that we had his prevailing definition of TDD
as not from you guys, from Kent
but from the people currently speaking in the program environment that I found was
A) very mock heavy
B) was very moralistic about the specific technique to get to the place we all want to get to
we all want to get to feel confident about making changes to the code
having self-test system that can run as a command and all these things
but you couldn't be part of that vote???
unless you also signed up for the TDD road to get there
and that's where I found it really frustrating
that we could have talked about whether we were being professionls or not
and that went through
well, you're writing test first
or you're writing them second
and related to that the whole TDD notion was propped up as
oh, well that self testing code you get
that's just a bi-product
that's a side benefit that some leftovers from driving your design through the tests, right?
that a lot of people really ??? onto this idea that TDD was not about self testing code
that was just a nice to have
but the key benefit was
the notion that you could only improve your design by making more testable
those two things equated
that easier to unit system was better designed system
that's the main heart of the fallacy and I want to take a real big swing at
because I started seeing a whole lot of code with this was not true at all
the code was not better
was not better design
just because it was more testable
it was full of these
mock that you guys talk about????
because we're trying to apply the unit test paradigm to areas of code that really didn't fit it very well
and we were discarding all sorts of really useful techniques, really useful patterns like Active Record
because it didn't fit
this notion of unit tests that had to run in 300 milliseconds
couldn't talk to the database
??? wasn't isolated, right?
so we had to throw out all these productive work
just so that we put ??? about alter of Test Driven Design
and that every aspect or our system could be unit tested in isolation
that's where I think we needed reboot
and I think we still need to reboot
we haven't rebooted enough
like these thing to still intermingled
and while I think we're all very much in the same page
and trying to tease these things apart
most people throwing all into one big pot
they're calling that pot, TDD
and TDD means self tested code
and it means red green refactor
and it means mocks because you have to unit test everything
and it means this big ball of shit in some ways
because that's what comes out of it
the dish in the end it's just not tasty
the code is not great
there're other more important sacred principles than it's just testable
as in if the system clear and understandable
and all the other aspects of it
so that's where my frustration really is
so I get to play time cut now
we said we'd do 30 minutes
30 minutes has come up
and it seems that we'll do another one of this hang out
sometime in the future we haven't decided yet
but I think one way we can immediately begin is to explore this question
in what ways can a desire to TDD damage an architecture
and is it damage, right?
some people say the hexagonal rails stuff is much better that crap DHH is imposing on us
is that the case?
how do we evaluate it?
those sound like good questions to tacklle in the text episode
alright, thanks a lot
we'll set up a new date and shoot out another link
that's great
bye, bye