Is TDD dead?

Hey, everyone.

I am David Heinemeier Hansson.

I'm here with the Martin Fowler and Kent Beck

to talk about "is TDD dead?" as the topic.

Coming off keynote I did at Rails conference and subsequent articles I posted on the blog.

I had chance to talk with both Kent and Martin

one on one in the days that followed.

and we got to discussed a bunch of these issues

and I got to clarify some thoughts and definitions that I had in my head

and got a lot wiser about where both Martin and Kent are on the issue

and Kent suggested that, "Hey. Maybe we should share that with world."

So that's what we're doing now.

I know bunch of people have been expecting to this is gonna be a big???

there'll be no boxing.

We'll explore bunch of definitions and problems and patterns we've seen.

and what's work for us individually and go from there

But to get started, I thought

I just run over brief recap of the problems that I've seen with TDD

use that to kick off the discussion

and really three major points that all focus on this discussion.

The first is the definition of TDD and its components

specifically unit testing.

I know the TDD and unit testing are certainly not the same thing

but TDD's often used to drive unit test

and as I was looking through

issues I've been in the Rails world around TDD

a lot of that came down to the definition of unit testing.

that people were defining unit testing as

you can't have collaborators

you can't touch the databases

you can't touch the filesystem

goal was to have a suite of unit tests so fast that they could complete in a blink of an eye.

I've seen arguments of

if it takes more than 300 milliseconds to complete the entire suite then it's way too slow.

I can't use it for anything.

and that just wasn't matching my useful definition of unit testing

and it wasn't matching what I was trying to get out of it.

and frankly I wasn't get anything useful out of unit tests that weren't talking to any collaborators.

Second, which follows from that is driving your architecture through Mocks.

If you want everything in your test suite to be so super fast

well obviously you have to talk to mocks

you can't talk to anything slow

and you can't talk to many layers at once

which leads to a mock sort of heavy and enabling architecture

repositories and hexagonal pattrerns

and all these other things that makes very easy to stuff and mock your way out of everything

and turn every test you could imagine into unit tests

MVC framework as I work with rails

the controller, "Oh, no problem. Just stuff out everything talking to the view"

"talking to the model and then that's gonna be great."

I was not seeing that being very great at all

the key point what I thought

this is test induced damage

that the architect is actually suffering damage from this drive towards unit testing all the things

and that the unit testing all the things is coming through the TDD pipe.

And then finally this sort of ..... TDD

is the read green refactor loop.

And that was more of, sort of personal thing.

We can all have our own personal preferences and how we develop things

in TDD red-green refactor seemed .... more of mandate

I never dealt with that for the bulk of the work that I do

at certain locations at work and at certain locations that flow was good

but I felt like there there's tons of code that I was writing were red green refactor

just not an enjoyable of way to do it.

and I didn't feel like I was going to a better place by going through that

So those sort of the three top-line problems that I think with the TDD

Let's sort of the open conversation up

and have Martin and Kent chime in perhaps one at a time

What is unit test?

What's useful definition of unit test?

What's useful definition of unit test and how most people currently defining it in you guys' view.

Well, I propose

in order to understand something

I often find it useful to know how it came about.

And the style of unit testing we familiar with is basically all Kent's fault.

Right? Because Kent did the framework, XUnit frameworks

whole push through extreme programming

so maybe we start with that

where those source of this style of unit testing come from.

You're the guy. You explain it.

Can you hear me?

Yep.

So when I was a kid

I would read books about programming

I was just fascinated by programming.

I'm talking about nine, ten, eleven years old

and my dad was a programmer

so there're lots of books about programming around.

And he had a book and I would read I didn't understand anything

but I was just sorta compulsively reading these books

cause I found the whole topics fascinating

there was a book that said, "Here's how you program."

"You type in the output tape that you expect your program to produce given some input tape"

"And in programming is, you just write program and tell it produces the output tape that you expected."

So that was kind of fixed way back in my brain as the flow of programming.

And it wasn't until after I had been testing for a while

that already come up with the first version of SUnit, the Smalltalk testing framework

that was the precursor to JUnit.

and I remember this idea of writing the test before writing the code

which makes no sense, right?

the test is gonna fail and you want the test succeed.

so why would you write it when you knew it was gonna fail?

So I just tried it.

One of my heuristics is, "If an idea is obviously bad, find the cheap way to try it

because if it turns out it's not bad then it's really interesting

if you have a good idea and it's obviously good

somebody else's probably tried it before.

So, I just figured I'd try and I think I built stack at first time.

and it worked really really for me.

i think there's aspects of my personality that make this kind of peicemeal style work really well

I have a lot of anxiety and I tend to get overwhelmed by big problems

and for me TDD solves those problems.

Even if I don't know how to implement something I can almost always figure out how to wright tests for it

and if I can't figure out how to write a test for it, I have no business in programming in the first places.

So, that's how got into it.

And it was definitely ground up

that is, I started with small green problems and gradually work my way to larger green problem."

Go ahead.

I was thinking about this on reflecting back on our early days together

and if I recall correctly when we began working

??? using extreme programming ???

we did the testing very strongly from the beginning

but it wasn't test first at beginning.

I've got this memory of you standing up and saying

"The stories are not done until the codes're written and the tests're written and the tests are passing.

So we have the notion that you delivered at the end of it

increment of functionality with code and tests together

but test first didn't actually followed till later.

I can't remember when that came because it seemed gradually appeared.

but I feel that at that very beginning the focus was on the tests.

on the tests with that but that's the first as it were

so I think one of the fundamental question is

as a programmer do you deserve to feel confident?

I certainly grew up in culture that didn't encourage that.

You know, you write some stuff and just hope it works

somebody else would tell you whether they thought it worked.

and you go home on Friday and just think

"Oh god. I just hope I didn't break something."

and so I think the going a step back from TDD or automated testing

"do you deserve to feel the confidence?"

"can you sleep at night?"

knowing that your code works

I think the answer should be Yes for programmer.

So if we agree on that then we can talk about how to achieve that.

TDD is one way to achieve that.

There's lots of other ways.

I think that it's a great point because

one of the things that really got me into programming was

was Ruby.

Right. So I've been programming before Ruby but

this language that has a very specific mission statement.

Programmer Happiness

Right. So I think those two are related

the programmer happiness is certainly related to feeling confident in your code

and confident in making changes to it

and confident that's going to work

But it's only one part of that picture

and early days Martin described were

"Oh. You're not done until you also have tests for piece of functionality.

I'm completely on board with that. Right?

Where I'm not on board is that

that whole feeling of feeling great about your development style

feeling programmer happiness as Matz would always put it.

Just I wasn't getting that from TDD, right?

So maybe that's some of the patterns we talked about

different people with different brain style, right?

Why do some people enjoy Python?

Why do some people enjoy Ruby?

and even harder for me to comprehend how some people enjoy Java.

But but we sort of have different ways of looking at things

and feeling good about them.

and what I was finding just from the red green refactor flow

was that the whole notion of writing to test first and seeing that fail

and doing that for every piece of code before I move forward

was just not natural

because a natural you can overcome just by familiarity

but even through familiarity it was just not a pleasurable flow

and I think that's where a lot of the explosive debate comes from

to some people writing your thesis first and filling in the implementation with that works really well.

That's exactly how their brain operates

For other people, me included, it doesn't work like that.

It's much more tactile experience that I actually have to sit in and write it out

that's how I think through it

I don't think through it by proposing hypothesis up first and then try to fill it in.

I think through it by writing it out

If I can't write test first I have no business writing it.

That's I totally understand how that you can arrived at

It's just not at all how I feel about it.

I have a hard time writing a test forward, if I don't see it first, right?

SO you have sort of these some people like it from

going through the test

and other people like going to the test.

I think .

certainly important to realize we're both getting to same place

getting to fell confident about the code

confident making the changes to the code

that's the part I'm not a fan of tying these two things together

I'm not a fan of tying

oh you just have to be confident to TDD

which, I know you're not doing, right?

TDD is one path of getting to that place

I've seen there're a lot of people are mistaken those two things

that you can't have confidence

you can't deliver incremental functionality with test

unless you go through this mandated well paid road of TDD

so, last night was hackathon at Facebook

and I have a project on a internal tool that I was working on.

and half of it I could use TDD for

and half of it I couldn't

so I had a big log that I wanted to process

and the entries in the log were not clearly specified anywhere

but I wanted to put them into a data structure

kind of funky new data structure

the data structure I could build with TDD is just fine

because I could break it down

ok, here's the main case

here's the corner cases

here's how I go from one to many

just nice sequence of tests

and I can get into real flow

just make one test work and the next one and the next one

so that feels a certain way to me

when I had to fiddle live data into it

it feels very different way

because I don't have that clear specification

I just have to run a few million log entries through this first

and see what blows up

so I put in logging

I put in exception handling

so any log entry that's kind of funky that isn't handle by the code

get saved some place

then I can't go back and look at those entries that I can't handle yet

and I can use them

but I can't write a test for it

because I don't even know what the inputs look like

and that feels different way to me

so with the parts of the code that I can use TDD for

I mean a real flow

write a test, make it work

write a test, make it work

??? 12:30

I really like that feeling

the ones where I have to run on live ???

see what happens

I fee a lot more anxiety

but you know it's called work for reason

so I'm going to use the principles of double checking of regression testing

of short feedback loops

and try to get the shortest possible feedback loop

out of this unpredictable input that I'm getting

so I had to mix the two styles in one night

and it doesn't bother me

if I can play classical and jazz then I'm in a better position to make music for more people

but when I can reduce a problem to this sequence of tests

drive the solution from this sequence of test

it has special kind of feeling

for me it works really well

if I think back to how learned a ??? map

I always used these examples

you know there'd be some general

you know, prove this statement

and I think well, "what's an example of that"

and then I come up with a few examples

and I convince myself either the statement was true or that wasn't

and then I can write the proof

but that's my personal flow

I work from specific to general

this kind of inductive style

and I understand not everybody likes that

what I think is interesting though

even if I've been in that flow too

I had been in flow where TDD was working our well for me

it was ??? I should describe ???

it was very clear input

and was very clear desired output

and there were not a lot of contacts??

and there was not a lot of environment

to operate it

it was very pure

and for that

that would probably one case where I do actually like that flow

what I find is a lot of my work is not like that

right? sort of fundamentally maybe not like that

and then the problem comes in what am I willing to sacrifice to get that

you can reduce any type of work

certainly in the web space ???if you doing to that flow

if you just, that's where the whole mocking thing comes in, right?

if you just find a way and work hard enough

you can remove all the sort of dependencies

all the environment and reduce it to just one piece of input and one piece of output

is the flow important enough that you want to do that as general case?

how much you willing to sacrifice to be in that flow more ???you work

for me the answer has been

"I'm not willing to sacrifice that much."

when the flow is natural

when the work is sort of almost algorithmic nature

separated from all this context

alright. great. let's switch into that mode

but forcing all sort of other kinds of program work into this mode

just such that we can't feel the TDD loop?

that's where really see that we're going off track

and just coming up with some

in my opinion nasty hats we're starting really bad trade offs

for other aspects of the design of the code ????

so I'm curious for both of you guys

how much you guys willing to sacrifice to get to that TDD flow?

are you willing to use heavy amount of mocking

do you find that generally you can get there

in, say, MVC style programming?

or where the boundaries for you guys.

Martin is waiting for me talk

so I guess I'm talking.

so as I see I do this all the time

it's a question of trade off

and my students see me do that at thousands times

so for me TDD is question of trade offs

and I think, David you pointed out an important part of trade off which is

to make a design intermediate results testable comes with a cost

it comes with a benefit too

and you gotta figure out whether the costs outweigh the benefits

sometimes that's a question of understanding how to design things well

so if I'm building a compiler

and I think well the only kind of test I can have are end to end tests

and then I start to build more and more cases of what the input looks like

and I realize, "oh, I have this intermediate representation of parse tree"

then now I can test parsing

now I have this two orthogonal dimensions

I have parsing and then given a parse tree

does it compute the right results?

and then I can do more testing at a finer grain

but it was a design insight that created that moment

and compilers have settled on that as being a reasonable trade off

"ok, it's worth having this intermediate representation"

"so that we can have testability and bunch of other stuffs comes along with it."

so I'm not willing to say

ok, there's some fixed boundaries and I don't wanna twist design just to make things testable

because I'm just missing some design idea

if I had better idea for the design

then I could have both ways

I could have something that was more static more flexible design and it was more testable at the same time.

but if I don't have that today

your point is exactly right

then what do you do?

do you mock absolutely everything?

my personal practice is I mock almost nothing

if I can't figure out how test efficiently with the real stuff

I find another way of creating feed back look for myself

I have to have feed back loop

and a feed back loop has to be repeatable

but like I just don't go very far down the mock path

look at code where you have mocks returning mocks returning mocks

and my experience is

if i use TDD I can refactor stuff

and then I heard this stories

People say, "I use TDD and now I can't refactor anything"

I couldn't understand that

and then I started to looking at their tests

well. if you have mocks returning mocks returning mocks

your test is completely coupled to the implementation. not the interface

but the exact implementation of some object

you know three streets away

of course you can't change anything without breaking the tests

so that for me is too higher price to pay

that's not a trad off I'm willing to make

just to get piecemeal development

And this is, I think, at the heart of much of this

is confusion of terminology

and what these different things are

when I read David's initial blog post

cause I didn't see his talk until last night

one of things that came through very clearly

was his criticism of

David, your criticism of TDD

and sort of design damage that comes through it

had in itself very much tide in a notion

with the strong desire for isolation of mocking

and there, very important point that

there is nothing sort of

with in the idea of how you do either TDD or unit testing

that says you have to have that kind of isolation

some people are very much in favor of it

others are not

and I remember in the early days of extreme programming

unit testing, people would criticize us

that saying, "hey you're not isolating your units properly"

"that's not unit testing"

and we had this whole conversation with 24 different definitions of unit testing or whatever it was

ans said, "Well, our style of unit testing"

"we don't bubble about the isolation"

"and you know, it's working well for us, thank you very much"

so that's one thing

whether TDD in unit testing should be tide with isolation

and I look at it as different schools of thought

and I'm with Kent

I hardly ever use mocks

but I know good people who do

so I don't want to shoot everybody who uses mocks

maybe give it 10 more years

and then ????will ??? out or something

but we'll see

then there's another thing which is distinction between having what I call self testing code and TDD

to me, it's really important have self testing code

the ability to be able to run a single command have your whole system self test itself

in an acceptable amount of time

that is really powerful

because then if you can do that you can refactor with confidence

you've got a good chance of keeping your code base healthy

and that means you're able to be fast deliver new feature, etc

that is really powerful thing

and if I wanna get a ???high horse and say you must do something

I might be inclined to get that particular high horse.

but to me at least

TDD is one way to approach that

TDD is very particular technique

and if done well, one of benefit is that it gives you self testing code

give you all those benefits as well

but self testing code is for me perhaps the primary benefit

and the two get comflated

now fundamentally if somebody get self testing code by another root

such as when we started

we're delivering the code and the test together at the end of iteration

and you've got that self testing confidence

then I'm actually pretty happy

I'm not gonna kind of feel upset that somebody got that by another root

and I share with David and Kent

there're problems where TDD doesn't work terribly well

sadly most of programming I do these days

is not conducive to TDD

and I miss it

because actually I really like the TDD flow

that really works for me

just as it works for Kent

but fundamentally we have to separate TDD from self testing code

on realize they have different benefit

so often get conflated together

and we have to separate the idea of fully isolated unit testing

from not isolated unit testing

and realize that people have different preferences around those

and that's where I see

that was what much of my reaction was against, right?

that we had his prevailing definition of TDD

as not from you guys, from Kent

but from the people currently speaking in the program environment that I found was

A) very mock heavy

B) was very moralistic about the specific technique to get to the place we all want to get to

we all want to get to feel confident about making changes to the code

having self-test system that can run as a command and all these things

but you couldn't be part of that vote???

unless you also signed up for the TDD road to get there

and that's where I found it really frustrating

that we could have talked about whether we were being professionls or not

and that went through

well, you're writing test first

or you're writing them second

and related to that the whole TDD notion was propped up as

oh, well that self testing code you get

that's just a bi-product

that's a side benefit that some leftovers from driving your design through the tests, right?

that a lot of people really ??? onto this idea that TDD was not about self testing code

that was just a nice to have

but the key benefit was

the notion that you could only improve your design by making more testable

those two things equated

that easier to unit system was better designed system

that's the main heart of the fallacy and I want to take a real big swing at

because I started seeing a whole lot of code with this was not true at all

the code was not better

was not better design

just because it was more testable

it was full of these

mock that you guys talk about????

because we're trying to apply the unit test paradigm to areas of code that really didn't fit it very well

and we were discarding all sorts of really useful techniques, really useful patterns like Active Record

because it didn't fit

this notion of unit tests that had to run in 300 milliseconds

couldn't talk to the database

??? wasn't isolated, right?

so we had to throw out all these productive work

just so that we put ??? about alter of Test Driven Design

and that every aspect or our system could be unit tested in isolation

that's where I think we needed reboot

and I think we still need to reboot

we haven't rebooted enough

like these thing to still intermingled

and while I think we're all very much in the same page

and trying to tease these things apart

most people throwing all into one big pot

they're calling that pot, TDD

and TDD means self tested code

and it means red green refactor

and it means mocks because you have to unit test everything

and it means this big ball of shit in some ways

because that's what comes out of it

the dish in the end it's just not tasty

the code is not great

there're other more important sacred principles than it's just testable

as in if the system clear and understandable

and all the other aspects of it

so that's where my frustration really is

so I get to play time cut now

we said we'd do 30 minutes

30 minutes has come up

and it seems that we'll do another one of this hang out

sometime in the future we haven't decided yet

but I think one way we can immediately begin is to explore this question

in what ways can a desire to TDD damage an architecture

and is it damage, right?

some people say the hexagonal rails stuff is much better that crap DHH is imposing on us

is that the case?

how do we evaluate it?

those sound like good questions to tacklle in the text episode

alright, thanks a lot

we'll set up a new date and shoot out another link

that's great

bye, bye