DAVID HEINEMEIER HANSSON: That any better?
Oh, there we go. Woo.
Software's hard, as you can see. Hardware,
too.
So last year I had the personal pleasure of
celebrating ten years of working with Ruby
and ten
years of working with Rails. But this year,
I
have a much more interesting anniversary,
which is ten
years of sharing Ruby on Rails with all of
you and everyone who's been using it over
the
past decade.
The picture in the background is actually
from almost
exactly this time, where I gave the very first
presentation on Ruby on Rails at a Danish
university
ten years ago.
Ten years ago, I had to talk a lot
about what is MVC, for example. Today, not
so
much. There's a lot of things that over the
past ten years, things we were worried about
in
the beginning, sort of leveling up as a community
and as, as programmers, that just aren't relevant
anymore.
We're just taking all that stuff for granted.
Which is awesome. We get to care about a
lot of other stuff.
But as I look back over the past ten
years, which is pretty much the majority of
my
adult life, I've been working on Ruby on Rails.
It's fun to look back even further. I think
there's a common misconception that anybody
who ends up
creating something like Rails or
doing something else within
software development or computer science,
well, they must have
been programming since they were five years
old, right.
The whole notion of a hacker is somebody who,
who sort of got their first computer twenty
years
ago and was just programming the entire time.
Well.
That wasn't me.
I did not learn to program when I was
five years old. I didn't learn to program
until
I was closer to twenty. I'd been interested
in
computers for a long time, but it wasn't really
until the late '90s, early 2000 that I dove
into computer programming as something that
I was going
to do.
I had a lot of friends who were doing
it. I knew a lot of programmers. But somehow
it, it sort of never, never caught on. Before
I started writing software that I needed for
myself,
and before I found sort of a place in
the software world for that to happen.
And the reason I, I say that is, is
I've seen a number of essays of like, what
is a true hacker, and, and as a true
hacker, one of the things that keeps being
thrown
out is those ten years, right. You should
have
been programming for ten years already,
otherwise you're way behind.
Well, I learned to program about three years
before
I released Ruby on Rails. Turned out fine.
And I don't say that as, like, it was
because I grew up on a farm and didn't
know what a computer was. This is me in,
in the center there with the stick, and the
blue shirt in 1985. These are the kids that
were living in my neighborhood.
In 1985 I got introduced to my, my first
computer. And I was about five years old.
And
I got introduced to computers through gaming.
Sort of,
we were playing ninjas in the streets around
our
houses, and then we'd play ninjas on the box.
And I found it fascinating right from the
get
go, right. Computers were really
fascinating and games really
captured my imagination early on. I remember,
at, at
that time, there were certain, lots of parents
were
like, well make sure you get out and play
a lot. Because you don't want to spend your,
or wasting your time, that was the, that was
the word, wasting your time in front of computers
inside, right.
Well, I really did want to waste my time
in front of computers. And this was the computer
to have in 1985, in our neighborhood. But
we
couldn't afford a computer like that. There
was only
one guy in the whole neighborhood that had
a
computer like that. So we all shared it and
we all played Yeah, Kung Fu, in turn.
But then the next year, so, I, I couldn't
afford this computer, but my dad somehow,
he was
fixing TVs and stereos, he traded a stereo
with
this guy for this other weird computer, which
was
an Abstraught 646.
And I was really excited. It didn't actually
play
Yeah, Kung Fu. I was a little disappointed
by
that, but it had some other crappier games.
Anyway,
it was a computer. And that was sort of
my first introduction to, to computers, six
years old.
And, and I tried to learn programming.
I, I got a magazine and, at the back
of these magazines back then, there were programs
you
could type in. And I was like, wow, this
is, this is amazing. I can control this computer.
So I built my first information technology
system.
It was messages to my mom of where I
went, where, I had this clever system that
would
really optimize the fact that writing a message
of
where I went and, and when I was going
to be home, it was too complicated. It would
be much easier if I just wrote a little
note where I gave her the location on the
tape that she had to fast forward to, and
then she could read where I was.
I thought, man this is so clever. I just
have to write down two twenty-eight, and then
I
could preprogram that note, and she'll never
know I
was over at Peter's house playing Yeah, Kung
Fu.
That really wasn't programming, right. I just
typed some
stuff in. I didn't know what the hell I
was doing. I just somehow figured out print,
OK,
that puts stuff up on the screen. That was,
that was the extent of it, right. But it
was my first stab at programming. And I, and
I, it kind of failed.
That was the extent of my programming, that
I
knew how to fast-forward to a pre-recorded
message.
Not that great.
So a couple years later, late 80s, I saw
this game for the first time. Battle Squadrant.
And
I remember thinking, holy shit, these graphics
are amazing.
How can they make this? This looks so good,
when you were used to a Commodore 64, the
graphics of the Amiga 500, which is mind blowing,
right. And once again, I felt this, like,
wow,
wouldn't it be amazing to be part of that?
To be able to create something like that?
I'd love to make games.
So I sort of started looking around, and,
and
I found this thing called Amus. I don't know,
has anybody here ever programmed in Amus?
Not a
single hand. OK. Must have been a very European
thing.
But it was sort of a real programming environment,
and, and I got the box, and sort of
my English was a little, not that great, so
I was sort of just reading through it and
trying to find the code. And it was all
about sprites and vectors and ifs and variables,
and
it, it didn't make any sense to me at
all.
So I thought, eh this is a little bit
too hard. Thankfully, there was something
called Easy Amos,
right. Oh, wow, that's gonna be great. This
other
one was too hard. Now I just have to
do the easy one.
Unfortunately, in Easy Amos, it was still
programming, and
it still had conditionals and variables and
all these
other things I just did not understand. I,
it's
so funny, because, once you learn something,
it can
sometimes be hard to go back and think, like,
how was it before I knew how to do
this? But I distinctly remember, why would
you have
a variable?
Like, if you just assign something once, why
would
you ever sort of want to change that? Why
does it have to be a space. Why can't
it just be the thing. Like, I did not
get the concept of variables.
And this is at, I don't know, age ten
or whatever. So, still not getting programming.
It's still
not making any sense to me. So I gave
up on that, too.
Then, in 1993, I went to something called
the
Gathering. The Gathering three, which was
a big demo
party in Denmark, where people from all over
Europe,
maybe some from the U.S. as well, would gather
to, to show off their skills of creating these
demos.
And demos were basically just, like, little
music videos
with computer graphics. And I thought that
was really
awesome. And, again, I got this sensation
of, wow,
that's amazing. People are creating these
sequences, they look
really good. This is, this is awesome. I'd
love
to be a part of that.
This is '93, so I'm, I'm fourteen. And this
demo party, and then I met pretty much everybody
I knew for the next ten years in, in
computer software, including Allen of TextMate
fame. I was
fourteen and he was part of one of these
demo groups, and we got talking, and then
ten
years later, I'd help him release Textmate,
and this
is now twenty years ago.
Anyway.
I still didn't get the concept, right. Like,
it
was all, it was Assembler. So it was even
harder and weirder to figure out than Amos
was.
It was vectors, it was math, it was, it
was just really hard. So once again, this
is
the third time I tried to sort of figure
out programming, because I want to build stuff.
And the third time, it failed.
So I ended up building another information
system. At
that time, there's something called BBS's.
So pre-internet, you
dialed up to basically every web site individually
through
a modem, and I ran one of those things.
At that time, it was, it was called a
Where's BBS, which is where we traded all
the
illegal software that we couldn't afford,
and games, and
demos.
And I had a lot of fun doing that.
I was fifteen and I was, I was working
at a grocery store, and I spent all my
money buying modems and phone lines.
But sort of the long and the short of
that is that I failed to identify with programming
under the computer-science paradigm.
Computer science, in itself, just didn't really
appeal to
me. Like, it didn't make sense to me. Learning
programming through sort of the lens, the
prism of,
of hard science just, it didn't really, it
just
didn't click. And I was actually pretty disappointed
for
awhile. I had tried to learn programming three
or
four times over the past ten years of my,
my life, and it just, it wasn't clicking.
No surprise to sort of my teachers. This is
my high school diploma, part of it, and it
says math, final exam, F. And, and English,
I
got an A. But math was just never my
thing. Physics never, was never my thing.
Any of the hard sciences were just never my
thing. And you say, oh, well that explains
a
lot. That's why Rails is so fucking slow.
But it, it's true. It just never appealed
to
me. But, I think it also inoculated me with
something really early on, which was, it disabused
me
of the thinking that I was a computer scientist.
That I was ever going to come up with
an algorithm. That I was ever going to make
any ground-breaking discoveries at the low-level
of, of computer
science.
And that was actually really a relief. Because
when
I finally got into programming, I knew that
was
just not what I was going to do with
it. That was never, it wasn't my idol, it
was not what I was chasing. I wanted to
build information systems. Like all these
attempts I had
over the years, they were all about information
systems.
They were about using the computer to build
something
else that really didn't have much to do with
the underlying things. That there were people,
smart people,
who had come up with algorithms underneath
to, to
make it all work, wonderful. I'm not one of
them.
And that's fine.
I think as an industry, very few people have
gotten to that realization. Even if it is,
that
they, on a daily basis, build information
systems. Even
if it is that they're working on yet another
social network for sock puppets, or horror,
in my
case, yet another fucking to-do list. The
aspiration of
the whole industry, everyone in it, is that
we're
all programmers.
Right?
No we're not. I am nothing like Linus, right.
He's actually a real computer scientist. To
figure out
how to, I don't know, fucking improve the
scheduler
in the kernel. Shew. No clue. No interest.
All
good.
I am ever in debt that there are people
like that out there who can do this stuff.
So I don't have to do it. So I
can focus on something else. But I think most
programmers think that, oh yeah, that, that's
what I
do. Yeah, I work information systems, but,
we're kind
of colleagues, right? Me and Linus here.
I'm pretty sure that he would tell you, fuck
you. We're nothing alike. We are not colleagues.
What
you do is making another fucking to do list.
I'm improving the fucking kernel of Linux.
Far more
important work. He would disabuse you of your
delusions
of grandeur real quick.
And I think that's a real shame. I think
it's a real shame that if you sort of
pick your heroes in such a impossible fashion,
that
they're actually nothing like you and you
will be
nothing like them, you're gonna set yourself
up for
a bad time for the whole ride.
The truth of the matter is that most information
system development has very little to do with
science.
Yes, it's all built on top of computer science.
Yes, computer science is what makes it possible
for
us to do what it is that we do.
But it doesn't define what we do.
And I think in many ways that prism of
computer science is harmful to the development
of information
systems. It's actually not a good view on
the
world to have. Just because you can make,
you're
Steingraeber and Sohne, and you can make the
best
piano in the world, that doesn't make you
a
great pianist. It doesn't mean you can play
wonderful
tunes. Just because you can create the foundations
of
which other people can build upon, just because
you're
a great computer scientist, doesn't mean you're
a great
software writer.
Doesn't mean you're a great programmer of
information systems.
And most of all, if you are committed to
building information systems, and I am wholly
committed to
building information systems, I've given up
the notion, long
ago, that I was going to get into games
programming or vector programming or anything
else that sounds
like hard science and is hard.
I think you're gonna be much better off. But
I think it's also really tough, because I
think
most of the paths, the celebrated paths into
programming
go through courses called computer science.
So you're sort
of taught right from the get go that computer
science, like that is the ultimate ideal,
and what
you're doing here is just sort of piddling
along
until you can get to this top of the
mountain.
Even worse, if you actually have a degree
in
computer science, right, and now you're slumming
it, with
yet another social network, or, yet another
fucking to-do
list. I mean, that's a recipe for self-loathing
if
I ever knew one.
But, as I say, this is mostly about the
prism of how you're looking at programming,
what is
programming, what is writing software. What
is that we
do every day when we create information systems?
And if you look at it from this prism
of the hard sciences, you think, well, Law
of
Thermodynamics. Physics. This is, this is
the real serious
hard stuff, right. You will laugh at French
poetry.
Ha, ha, ha, ha! They're all just, what, analyzing
what some schmuck in the 1700s did, and there's
a thousand different interpretations of, of
what that person
actually wrote and what does that actually
mean? Like,
that's pathetic, right. You can't arrive at
any ultimate,
clear, universal truths.
Math! There's a truth. There's a final result.
Physics.
There's a truth. We're knowing more about
the natural
world in a way where we can be completely
confident. Mostly. In what we know. Certainly
in math,
right.
If you carry that over into programming, you
end
up with shit like this. Law of Demeter. Practices
and principles who sort of project that they're
universal
truths about the natural world, that this
is how
good programs are made, and this is not really
an argument. The only argument is whether
you're professional
and following the laws, or you're an amateur
and
you're breaking them.
When I look at computer programming, and when
I
reach most, read most programs, I'm not reading
hard
sciences. It is much more like studying 17th
century
French poetry. What the fuck did this guy
mean?
Like, I can't follow this at all. Like, is
this some weird reference to some play somewhere?
What's
going on here?
It's actually more like forensics. It's more
like analysis.
It's much more subjective. Like, what is actually
going
on? What were they trying to communicate?
What's just
going on here, right?
So, I find it so funny that, that programmers
who work in programming, and they laugh at
all
these subjective fields of endeavor, when
that is what
they do every day. They just, no, what I'm
doing is computer science. This is empirical
truth, blah,
blah, blah, we have laws, blah, blah, blah.
I think the, the bottom line is that is
when you go in with that notion, when you
go in with the notion that we can actually
discover laws of programming, like, Law of
Demeter, of
how we should be creating our programs, you
lull
yourself into this belief that there are some
practices
that are just true. They're not up for debate.
They're not up for discussion.
They're science. That what we do is science.
Well,
I think there's another word for, sort of,
those
delusions. Pseudoscience. When people think
they're doing science and
they're not actually doing science. That's
pseudoscience. I think
a lot of what's going on in software, methodology,
practices, is pseudoscience.
Which would be fine if people would accept
that
and say, yes, what I'm doing is pseudo science.
Like, I'm not finding any grand truths here,
but
they're not, right. They're ex-pouting that
this is, this
is the truth.
Well, here's another pseudoscience. Diet schemes.
I think diets
are actually incredibly similar to most software
methodology approaches.
They all sort of espouse that I have the
truth, what you need to get slim and healthy
is the ten-day green smoothie cleanse.
That is the truth. That's how you get it,
right. And then you, shit, that, that's, OK,
smoothies.
Sounds good. But what about this super shred
diet?
Like, I lose twenty pounds in four weeks?
That's
certainly better than ten pounds in, I don't
know,
ten weeks, or whatever that hungry diet girl
is
promising. I'll go with that super shred guy,
like
he's got to have the truth, right.
And it's so funny, if you read any diet
books, and diet books are incredibly popular.
If you
look at the most popular book on Amazon, the
top one hundred list, a lot of them are
diet books. People want to be told how they
can cheat the basics. I think software development
is
exactly like that.
I think software developers are exactly like
people trying
to lose ten pounds and thinking, you know
what,
all this exercising, just eating healthier,
that's a little
too hard. Let's, let's listen to this super
shred
guy.
He's got to have the answer. An answer that's
less painful, less simple and basic. There's
got to
be some grand secret I just don't know yet.
If I can just learn the secret then everything's
gonna be great, right. But it's pseudoscience.
Those diets
are based on anecdotes. They're based on one
guy
trying something, or, or looking at a few
people,
a tiny sample size, it's just pure, poor,
pure
poor science.
External variables, uncontrolled experiments
that run for too long.
You can't derive any absolute truths from
it. But
people keep arriving at absolute truths.
And just like feeling a little overweight,
and most
people do at some point in their life. Everybody
wants to lose whatever it is. They want to
feel healthier even if they are at the correct
weight. They want to be in better shape. All
our code bases are exactly like that.
Everyone has like, oh I'd love that this part
of the code base, it's not that clean, right.
So we have that same feeling of being a
little insecure about our quote code quality,
just like
most people are a little insecure, at least
at
certain times in their life, about their body,
right.
So we're ripe for somebody to come in and
tell us what's wrong. To fix it for us
by just saying, oh, no, no, no, you don't
have to do any of the hard stuff. Writing
good code, do you know what that's about?
It's
about this one practice. This one secret that
they
don't want you to know.
If I teach you that, then all your code
is going to be wonderful. But right now, you're
not a professional. You're an amateur. You're
writing dirty
code. You should feel really bad about that.
You
have sinned.
But, I will give you absolution. I have the
pathway to clean code.
And it hits a lot of people right in
the impostor plexus. Like, ugh, you're saying
my code
is dirty? Yeah, I guess it is a little
dirty. There's this one part that's, like,
shit, maybe
I'm not really a computer scientist. Maybe
it doesn't
really, I don't really belong here amongst
the programmers.
Can you please tell me, how do I get
to be a computer scientist?
How can I get to belong amongst the esteemed
professional programmers? Can you tell me
how? And there
are lots of people willing to tell you how.
That the salvation will come through these
patterns and
practices, and as long as you follow these
ten
commandments of good code, all shall be well.
OK.
I think the most popular commandment, I'm
gonna spend
some time on that, the most popular practice,
the
most popular pattern for making people feel
shitty about
their code and shitty about themselves and
shitty about
their path through programming, is TDD.
TDD is the most successful software diet of
all
times. It's so alluring, it has such an appeal
in its basic principles, that everyone gets
wrapped up
in it. I got wrapped up in it for
quite awhile. I got wrapped up in the storytelling
that all software before TDD was shit and
unprofessional.
And that the only way to arrive at clean
code was to follow the principles of TDD.
And the principles of TDD, mind you, are not
about the tests. It's about test first. It's
about
test-driven design, right. That we have tests
afterwards, that's
just an accidental side-effect. A benefit,
if you will,
after the fact. And it's the perfect diet.
I tried multiple times, which is usually how
it
goes with diets, we try one and it doesn't
really work and we fall off the wagon and
then a few months later you try again and
you feel bad about it the whole time and
that's how I felt about TDD for a long
time.
I felt like TDD was what I was supposed
to do. I was supposed to write all my
tests first, and then I would be allowed to
write my code. And it just didn't work. I
kept just feeling like, this is not, I'm not
arriving at something better here. When I'm
driving my
design by writing my tests first, the code
I
look at afterwards, it's not better. It's
not cleaner.
The dirty code I wrote without being test-driven
first,
it actually looks better.
But, so successful has TDD been, that for
the
longest time, until actually fairly recently,
I just thought,
well, I'm the wrong-doer. I'm the one doing
it
wrong. TDD is not at fault, right. Just because
everybody's doing TDD wrong, doesn't mean
that there's anything
wrong with TDD. There's just something wrong
with all
of you. That's the problem.
If you would just be more faithful to the
practices, then everything would be great.
And that's what makes it such a great diet.
That it keeps people in the perpetual state
of
feeling inadequate. So, you keep having to
buy more
books, and attend more conference talks, and
attend more
workshops, and hire more consultants, to teach
you to
be truer to the religion of TDD. Hogwash.
Let's look at some code. So here's a very
simple piece of code. Person has an age method,
that calculates how old somebody is. And we
have
a test for it. This piece of code depends
on the world.
It directly refers to date today. It's a explicit
dependency. You cannot change it in there.
Well, in
a lot of languages, that's a problem. Like,
how
are you actually going to test this if you
can't somehow figure out how to change the
date
of today. Like, every time you run your test
it might be a different day and it might
be broken.
Well, in Ruby it's really easy. We just stop
that constant and make it work. That's what
the
travel-to method is about, right.
Proponents of TDD will look at that code and
say, dirty, dirty code. Explicit dependencies
hidden inside. You're
mocking a global object? What the fuck?
You need to shape up. Here's the shaped up
version. We inject our dependency, so we have
a
default of date.today, but we can put in our
own, in the test, we can put in our
own date, right. This is much cleaner. Right.
No.
Great. We improved our code base.
Did we? Is this a better code base? Is
this method better than what we just had there?
Is it simpler? Is it clearer? No. It's easier
to test. And that's the important point, right.
That's
the important point in all of these debates,
is
just, is it easier to test?
That's the measure of success. I think that's
a
shitty measure of success. I think there are
much
higher ideals than just whether something
is easy to
test. But it gets worse.
Here's another example. If you actually have
a method
that depends on another method, we have to
inject
the dependency all the way down, now you're
really
muddying things up and now the code is really
starting to get nasty. And this is such a
simple example. I've actually posted this
example online before
and had arguments with TDD proponents about
that.
And, yes, this is, I'm like, well, what does
it matter? You're just injecting one dependency,
what does
it matter? It's not that big of a deal,
right? Yes it is.
Because this is exactly the type of thinking
that
leads you down a really nasty path. Let's
look
at another example.
Here's a standard Rails controller. It has
reliance on
the world. It relies on a before action that
ensures permissions. It sets up a new object
and
sends out an email and then it responds to
something, right. The whole purpose of the
controller in
Rails is to sort of direct the world. It's
to interact with the world.
But how do you unit test that, right? That's
really hard. It's relying on the entire world.
If
we're following this scientific approach of
unit testing where
we're isolating all the variables, holding
everything else constant
except for these two things, what goes in,
what
goes out. This is bad, right.
If we instead put in something like a person
creation command and hide away all the actual
doing
of the control and then we inject all the
stuff that it depends on we can test person
creation command really well.
Is that code better? Is that code simpler?
Is
it clearer? Would you rather look at this
and
then understand what the system does or would
you
rather look at this and try to figure out
where does this thing go?
And that's the consequence of this chase of
test-first.
It leads you down a path where the gospel
of test-driven design is that anything that's
easier to
test is better. That's it. That's the measure
of
quality. If you can test it easily it's better.
If you can't test it easily, it's worse.
Boo. Exactly right. Boo.
It's not better. We're losing sight of what
we're
actually trying to do. Tests were supposed
to support
us. They weren't supposed to be the main thing.
And I think this is leading to a lot
of Zombie astronautic architectures. Things
that I thought we
moved past long ago. If you look at, at
this person create command, that reminds me
very much
about Struts 1 point 2, and how they had
every action as their own object. And that
it
was great because it was easy to test, and
it was shit when you were trying to put
a whole architecture together, because you
had all these
create commands and all of the sudden you
had
a million objects. Yes, they were easier to
test,
but the system was much harder to reason about.
You see the same thing around ActiveRecord,
for example.
You see, well ActiveRecord should just be
data access
objects. They shouldn't actually have any
logic. They should
just be about interfacing with the database,
because then
we can split out everything else. Our domain
logic,
it's just that it doesn't have to touch the
database, so that our tests can be simple,
so
that our tests can be fast, right?
Again. Order of priority. Test, test fast,
oh, your
architecture. That'll just fall from that,
right?
I recently read James Coplien, has a great
paper
out called "Why Most Unit Testing is Waste."
And
for me, this is the money quote. Splitting
up
functions to support the testing process,
destroys your system
architecture and code comprehension along
with it. Test at
a coarser level of granularity.
TDD is focused on the unit. The unit is
the sacred piece, because that's the science
piece. That's
what we can control all of the variables.
We're
just looking at these few pieces, right.
What James is saying, maybe that's not the
right
level. Maybe testing, the role it should have,
shouldn't
be about the unit most of the time. And,
I sort of alluded to this awhile back, I
wrote a post called Testing Like the TSA,
and
the main thing about that was about over-testing
and
sort of this testing theater that goes on.
But
I hadn't really made the switch that it, the
problem is that we're trying to test at the
wrong level.
It's not testing itself. Testing is great.
I'm not
advocating that we shouldn't have tests. I'm
advocating that
driving your design from unit tests is actually
not
a good idea. That you actually end up destroying
your system architecture and your code comprehension
along with
it.
So if unit tests aren't the thing, what could
we do instead?
Well, I think there are higher levels of testing.
We've already sort of moved to that in Rails.
It's no longer called test unit, where you
place
your tests, it's called test models. That's
already one
step up. That sort of frees you from feeling
bad about the fact that your, your model tests
actually touch the database. That's not a
bad thing.
Yes. They run slower. But they also test more
things.
You can make anything fast if it doesn't have
to work.
And I think that's the problem with testing
in
a lot of cases. We recently had a really
bad bug on base kim where we actually lost
some data for real customers. And it was incredibly
well-tested at the unit level. And all the
tests
passed. And still we lost data. How the fuck
did that happen?
It happened because we were so focused on
driving
our design from the unit test level, we didn't
have any system tests for that particular
thing. And
it was a really simple thing. It was like,
if you were editing an object and you were
editing the attachments, you could lose an
attachment. And
we lost some attachments and it was a terrible
thing.
And after that, sort of thought, wait a minute.
All these unit tests are just focusing on
these
core objects in the system. These individual
unit pieces.
It doesn't say anything about whether the
whole system
works.
Most TDD proponents, I find, are much more
focused
on the unit level, because that's where they're
driving
their design. And they're not very much focused
on
the system level at all, which is what people
actually give a shit about.
Does the system work? I don't care about whether
your units work. Does the whole thing work?
That's
what matters.
So that kind of freed my mind up a
little bit. That if we give up this need
for hard science experience, where we have
to control
all the variables and boil everything down
to a
single unit that can be tested, we can float
freely with the world.
Awesome.
This realization, I came to realize, was why
people
hate fixtures. So fixtures in, in Rails is
about
spinning up a world, it's about setting up,
sort
of, small size version of the whole world.
Where
most approaches, they focus on just one unit.
I don't want to have an account in here
and a project in here if I'm just testing
my message. I just want to test this one
single thing, right. And if you're doing that,
yeah,
fixtures are probably not a good thing. It
doesn't
really work for that.
It works really well when you're not concerned
about
the hard science focus on unit tests. It works
really well when you're focused on a larger
level.
When you're focused on models, when you're
focused on
controllers, and most importantly when you're
focused on systems.
But all that is sort of usually swept away
by the holy trinity of test metrics. Coverage,
ratio,
and speed.
I've been in a lot of internet arguments lately.
In such esteemed establishments as Hacker
News, and, and
elsewhere, and I find it really interesting,
because each
individual argument will make me rage, but
then the
larger set of all arguments, like a meta study,
actually reveals really interesting things
about what people care
about. What they value.
And what I find is, in most discussions about
design, especially around Rails, what people
care about these
things, and these things only. It's about
the test
coverage. It's about the test ratio. And it's
about
how fast your tests run.
Like, that's the pedestal. That's the holy
grail. What
actually happens underneath, the design of
the rest of
the application is, eh. Doesn't really matter.
And thus, a lot of people come to celebrate,
oh, I have a one to four test ratio.
For every line of production code, I have
four
lines of test. Oh yeah.
And they say that with pride. And I'm like,
what? So you're saying for every line of production
code you write, you have to write four lines
of code? And that somehow makes you a hero?
How, how does that work?
So your system is now five times as large,
reasoning about the whole system is now five
times
as complex, and you're proud of this, why?
Well,
of course, because I have a hundred percent
coverage.
My five thousand tests run incredibly fast
because they
never actually test very much.
They certainly do not test the system. They
test
all these little slices of unit. Wonderful.
It's not wonderful. You have anemic fucking
tests. They
don't prove shit. You're gonna have the same
bug
that we have on base camp, and the system
is not going to work even though you've proudly
proclaimed, oh, well, your units are working.
Well, whoopity-doo. This decoupling is now
free. People think
that, oh, this is like that saying, like,
quality's
free. Right. Testing is free. Not when you're
doing
it like this. It's not free.
And most importantly, it's not free, not so
much
in time, but in conceptual overhead. Understanding
a system
that has been test-driven designed from the
unit perspective
is really hard. Because you have all these
levels
of indirection. You have all these levels
of intermediation.
To separate the tests from slow things, like
HTML
or the database or, any of the other parts
of the system that actually makes up your
system.
And they focus just on that one thing. So
they can be very fast, if they don't have
to work. They don't actually have to test
your
system.
So that's sort of two charges in one. It's
a charge first that your design is not going
to improve. Your design is going to deteriorate
by
doing test first programming, because you're
going to construct
your units of testing, your methods, in a
different
way, like we say with the age example. You're
going to inject all your dependencies in a
way
that does not prove things.
And second of all, you're not gonna get the
benefit of great coverage. You might have
a lot
of tests, but they don't test your system.
It
doesn't improve your confidence in actually
the whole thing
working, and then what good is it?
Well, I thought about this for a long time
and thought, like, this is not really, it
doesn't
seem like that great of a revelation. Why,
why
do I keep having these arguments over and
over
again? Why are people focused so hard and
so
intensely on this trinity of test metrics?
How did
that come to be the main thing that people
are arguing about?
How did that come to be the main decider
of what's good design and what's bad design?
Really,
couldn't really figure it out. Until I started
thinking
back of like, what is this prism we're looking
through? We're looking through computer science.
Engineering. Professionalism.
James Harrington wrote a bunch of books on
quality
of engineering, and he has a great quote here.
If you can't measure something, you can't
understand it.
If you can't understand it, you can't control
it.
If you can't control it, you can't improve
it.
That makes a lot of sense. Cause like, yeah,
yeah, that makes sense. That's gotta be why.
And
then I got another great quote. Just because
you
can, just because something is easy to measure
doesn't
mean it's important.
This, I think, is exactly what's going on
here.
Programming of information systems is a lot
more like
French poetry than it is like physics. But
programmers
grow up thinking that they're computer scientists,
so they
want it really badly to be like physics. They
want it really badly to be a hard, professional
science.
And coverage, ratio, and speed. You can get
that
fucking thing down to six decimals. Like,
that, I
can be so precise about how fast my test
runs, eighty-four point seven percent coverage,
boom. Got it.
Doesn't say anything about whether or not
that's actually
important. Doesn't say anything about whether
that's actually producing
a good system. Doesn't say anything about
whether the
person after you, or you yourself, three months
from
now, can understand what the hell is going
on
in this code base.
You haven't necessarily made anything any
clearer. You've made
it very easy to produce metrics. And if there's
one thing we love with, with science, it's
clear
concise objective truths and coverage and
ratio and speed
fit that bill. So we adopted them with open
arms, even though they were not that important.
Second part of it. Cost is not value. A
lot of people have invested so much in building
up their expertise, their time, their four-to-one
ratio in
tests. The investment is massive. Well, of
course they're
gonna be defensive about it. Like, you've
invested so
much of your ego and your time and your
resources on this project, into testing. So
of course
it must be valuable. Of course it must be
important.
That's not how it works. Just because something
is
really expensive, just because something takes
a lot of
your time doesn't mean it's valuable. Doesn't
mean it's
important.
So I was sort of giving a brief description
of this talk yesterday at dinner with Aaron
Patterson,
and he trolled me right back and said, TL;DR,
just don't test, right. Like, that's what
I'm supposed
to get out of this.
No. No. Testing absolutely has value. Regression
testing absolutely
has value. Driving your design through test
first? In
my mind, rarely has value. Not never. There
are
times where I'll write my tests first, usually
when
it's something like a translator of some kind,
where
I know exactly what's going in and I know
exactly what I want out.
That could be a good case to start with
test first. Most information system design
is not like
that. I'm trying to figure out and sell what
the system is supposed to do. What I'm going
to arrive at is the test, is this set
of tests, it's a set of regression tests,
that
make me feel good about that after the fact,
that I can still change my system and not
break it, right.
So TDD. Kent Beck. Main proponent behind TDD
has
a very sensible quote that goes exactly along
these
lines. I get paid for code that works, not
for tests, so my philosophy is to test as
little as possible to reach a given level
of
confidence.
Immensely sensible. I find that that's actually
often the
case with these things that get taken too
far.
TDD started out as a pretty sensible thing.
Kent
has an even more sensible perception of it
now,
I think, than when he wrote the test-driven
book
originally. But that's not what most people
take away.
That's not what most people run with if they
want to build a career on making people feel
shitty about their code bases and their dirty,
dirty
code.
They take a much more extremist approach,
that unless
you're doing test-first, you're not a professional.
Certainly not
what Kent's saying.
OK. So for me, this really boils down to,
we're, we're trying to wear the wrong hat
the
majority of the time. Thinking of yourself
as a
software engineer will lead you down the path
of
coverage, speed, metrics, hard sciences - all
these things
we can measure and it leave you laughing at
shit like interpretation of French poetry.
Of subjective evaluations
of a design in the system, even though these
are the only tools that we have.
So this has taken me awhile to arrive at
this conclusion. I've hated the word software
engineer for
quite awhile. Cause I never felt it fit me.
I never thought of myself as a software engineer.
I kind of tried to be one, a few
times, and I failed all the time. And by
the time I finally arrived at programming
as something
that I wanted to do, it sure as shit
wasn't software engineering.
Yes. It's a hard hat that I wear occasionally,
when I do performance optimization. That's
hard science. You
make a change. You measure it. Was it an
improvement? If not, revert. If yes, deploy.
Very scientific.
Very good.
Great to wear the hard hat when that fits.
Is what, that what I do the majority of
the time? Is that how I think of myself
when I write most of the things that I
write? When I add a new feature to base
camp or do forensics to figure out how a
bug came about, or what somebody meant when
they
wrote a bug in or something? No. That's not
what I do.
So I don't try to think of myself as
a software engineer.
OK. If we're not software engineers, most
of the
time when we make information systems, what
are we
then? What other hat should we try to wear?
What other identity should we try to aspire
to?
I think we had it all along. I think
we had it in the language all along. We're
software writers. Writing is a much more apt
metaphor
for what we do most of the time, than
engineering is. Writing is about clarity.
It's about presenting
information and motivations in a clear-to-follow
manner so that
anybody can understand it.
There are not bonus points for making something
convoluted,
as there often is with engineering and with
test-first
design. Making things more convoluted gives
you the benefits
of, perhaps, easier to test. They don't give
you
clarity. So those things are often in opposition.
Clarity is all about being succinct without
being terse.
WE can write things using as small words as
we know how, using as little complication
as we
know how, using as little conceptual overhead
as we
know how to get the job done.
That's a much better approach. And I think
it
comes very easy if you think of software development
as writing. If you look at a piece of
writing that somebody has written and it's
kind of
convoluted, can't really follow the argument,
and paragraphs are
not broken up neatly, is the first thing you're
gonna say, you know what the problem with
this
is? You're not using big enough words.
If we just shoved some bigger words into this
text, it's gonna be there. That's good. Oh.
The
problem here is like, like your sentences
are too
clear. If you just insert a sentence in the
middle, that'd be great. Oh, do you know what?
This needs more semicolons. That's what this
needs. If
this has more semicolons, boom. You got clarity.
No. More indirection. More third-person. These
are not the
things that make for great, clear writing.
And that's
obvious when we talk about things like writing.
SO
if we talk about software development as writing,
I
think it'll be obvious too. I think if we
supplant the high ideal of what matters for
design
is how easy it is to test, how easy
it is to make engineering driven, and put
clarity
of the code base above all else, we're gonna
be much better off. And arguments are gonna
be
settled much easier. And it's gonna be much
easier
to read the fucking code you wrote three months
ago, because you had that in mind.
That was your motivation. That was what you
were
going for.
So what is clarity? How do you figure that
out, right? That's real easy to say, oh it's
just clarity. Boom. It's open to interpretation,
right. It's
not as clear as just saying, oh if you
can just get your code coverage above 85%
then
you're gold, right.
Clarity doesn't work like that. There's not
a metric
we can just apply, because, again, it's not
hard
science. Clarity of writing is not hard science.
So the easy answer is, I know when I
see it. Right? There's not just gonna be a
list of principles and practices that somebody
can be
taught and then they will automatically produce
clear writing
every time. Right? If you want to be a
good writer, is it enough just to sit and
memorize the dictionary? No. Just knowing
the words available
to you, knowing the patterns of development
is not
gonna make you a good developer.
You have to develop an eye. You have to
develop an eye for clarity, which means first
of
all, you have to decide that that's important
to
you. So that's the first step. If you're still
stuck in, the most important thing about my
system
is how fast my tests run, and how many
of them I have, well. Forget about it. You're
not gonna get to this point.
You have to decide that the most important
thing
for your system is clarity. When you do decide
that, you can start developing an eye. I started
getting into photography maybe six years ago.
When I
first got into photography, like, I would
look at
a picture and it was like, well, looks like
a good picture.
Now, I've been looking at thousands of pictures.
I've
been taking thousands of pictures. And I've
developed an
eye. I can see when the white balance is
off. I can see, oh, this has a blue
tint. Oh, this actually, if we crop it just
a little bit more, only the subjects we want
to have in focus are in focus. Oh, this
would actually be better in black and white.
That eye just came from doing it. A lot.
It didn't come from just sitting down and
reading
a lot of photography books. And I think it's
the same thing with code. I think a lot
of programmers, coming at it from a software
engineering
angle, thing that just all they have to do
is learn the practices, learn the patterns,
memorize them
all, and then they will be good programmers.
No, they won't. The only way to become a
good programmer, where, by definition, I define
good programmers
as somebody who programs, who writes software
with clarity,
is to read a lot of software, write a
lot of software.
Just like how do you become a good photographer?
You take a lot of pictures. And you look
at them. And you practice. And you practice.
And
you practice.
I think it's quite similar to the problem
with
diets, right. The fundamental truth with diets
is to
be healthy you should probably exercise regularly,
you should
probably eat a reasonable amount and it should
be
good stuff. Like, that's three things. Incredibly
hard to
do. Most people do not do that, right. Figuring
out how to write good software: read a lot
of software, write a lot of software, aim
for
clarity. It sounds too simple.
Why is it simple? Because there's not just
a
secret. There's not just one answer somebody
can give
you. The only way you can get there is
by doing it. So when I first started developing
Rails, I read a ton of software. I read
the entire Ruby standard library. Partly because
documentation of
Ruby at that time was pretty poor, and the
only way to figure out how it worked was
to actually look at the code. But that was
also where I learned so much.
Today, we have it so much easier. Bundle open
name of any gem, and it'll open, boom, right
up in your text editor. You can look at
any code.
How many of you have read through a complete
gem recently. Something you did not write.
Awesome. That's
actually really encouraging. I thought it
would be much
less.
I think that is exactly the path you need
to take. You need to read a shit ton
of code. And it's not so much just because
you read this code and then, oh that's all
great stuff. Just as important as it is to,
to develop your sense of writing by reading
a
lot of shit writing, so is it with code.
And I think you will find that that is
actually very easy. Because a lot of things
you'll
do bundle open on will follow Sturgeon's Revelation.
90%
of everything is crap.
Well, at least you know you have company if
you write crap software and I certainly do
from
time to time. And that's a great way to
learn. I actually find that I learn the most
about software and learn the most about what
matters
to me, I learn the most about what clarity
is when I read poor software.
Because what I do is, I take a piece
of software, I take a class or method, and
then I look at it, how could this be
clearer? Like this is, I think this is poorly
written. I think this smells. How can I rewrite
it?
So by just sitting down and going through
that
exercise and rewriting it, I find I learn
a
whole lot about what I care about. So that's
what I've been doing lately. Engaging in a
lot
of these internet arguments. Somebody will
submit a piece
of code, and usually the submission will come
along
with the proposed solution, too, right.
I have this shitty piece of code. Then I
learned about these three patterns. And now
it's wonderful.
And what I have found every single time, and
I've only done this maybe a handful of times,
maybe a little more, is that every single
time,
you just took the shitty code and you stuck
it into some different boxes.
Like, it didn't actually improve. Applying
the pattern to
it did not improve the underlying clarity
of the
code, because you just wrote it poorly. Like,
the
problem with the code was not that it was
missing patterns. The problem with the code
was that
it was crap. That it just had to be
rewritten.
Now, that leads us to sort of a mission,
in some ways, from Rails is and what Ruby
is, and it leads also to clarify some of
the arguments we've had in the Rails community
for
awhile. Readability is incredibly important
to Ruby. We have
a lot of duplicated methods that do exactly
the
same thing, just so we can improve readability.
I remember the first time I saw unless, when
I was learning about Ruby. That was one of
those light bulb moments. It's like, wait
a minute.
Unless is exactly the same as if not, but
it's a different keyword. Huh.
Right? It was not just about making the most
efficient, compact language. It was about
readability. Poof!
Mind blown.
And decade-long love-affair with Ruby established.
And I think
that this is what we're trying to achieve
constantly
in Rails as well. A lot of people will
gripe about, oh, ActiveRecord is too big or
something
is too big, or have too many methods. The
surface area is too big or there's, whatever
it
is, right.
Like, who gives a shit? Is it more readable?
Is it more clear? That's the only thing that
matters. What do I care whether the surface
area
of a method is a hundred methods or it's
two hundred methods? I don't give a shit about
that. The only thing I give a shit about
is whether, is the code I'm actually reading,
is
the system I'm trying to understand, is that
more
clear. When you put clarity as your number
one
mission, a lot of concerns just fall by the
wayside.
It just doesn't matter anymore. And it's liberating.
So, I think this actually comes from sort
of
the same rope. I didn't have time to write
a short letter, so I wrote a long one
instead. I think that describes about eighty
percent of
all that ninety percent of shitty code.
Most people did not take the time to write
a short piece of code, so they wrote a
long one instead. And then they wrote that
long
one, piece of code, and they like, pulled
out
their suspenders and like, oh yeah, I'm done.
My
tests pass.
Boom. Right? Or they look at it and decide,
oh shit, this is too long. I must be
missing some patterns. I just sprinkle some
patterns over
this. Wonders, right? No. What you wrote was
a
draft.
This was just a first draft, right. Any piece
of code you write down is just a draft.
Mark Twain actually had the hard job, right.
He
wrote in ink. If you actually had to change
that, well that was kind of hard, right. So
his drafts were a lot more complicated. Ours?
We
have it so easy. A text editor? You just
delete stuff. And you don't need any of the
little stuff that you, you fill over the page
and you spill it everywhere and so forth.
An
eraser.
You can just delete stuff. Like that's my
favorite
key. The delete key. It's the number one tool
for improving code. Delete key.
So when I was in high school, I submitted
a bunch of first drafts as essays, and they
were really shitty. And of course they were.
They
were the first draft. And my teacher at the
time said, oh, all right, all right, this
is
actually not bad. You just told me the step
one. If you have something on your mind, you
should write it down.
That's what I did. I, I wrote it down.
And then I handed it in. Oh, if you've
written something down, you should rewrite
it.
Oh. That was the step I missed. And I
think that's the step most people miss when
they
write down code. Cause they're focused on
all these
other things. They're not focused on the clarity,
because
when you are focused on the clarity, you will
realize that all your first drafts are terrible.
All my first drafts are terrible. All my first
attempts at writing a good class are poor.
They're
too long. They're not at the same level of
abstraction. They're not clear. And that's
OK. I wrote
something down. I was trying to figure out
what
the system was supposed to do.
That's hard work. And often times you don't
get
perfect code out of that when you're still
trying
to figure out what it is that you're writing.
But once you've written it down, you should
rewrite
it.
And I think the key part of rewriting is
omitting needless words when it comes to regular
writing.
When it comes to programming, it's omitting
needless concepts.
It's omitting needless patterns. It's omitting
needless practices. It's
omitting needless classes. It's omitting all
these extra things
that aren't getting you closer to clarity,
right.
How do you know? You develop an eye for
it. How do you develop an eye? You read
a lot of code. You write a lot of
code. You rewrite a lot of code. And you
forget about fucking patterns for awhile.
You forget about
fucking TDD for awhile. And you focus on just
what's in front of you. The piece of code.
How can I write it simpler?
Write software well. Thank you very much.