Introduction to Differences-in-Differences

MIT's Josh Angrist introduces differences-in-differences with one of the worst economic events in history: the Great Depression.

Economists still argue about the causes of the Great Depression, but most agree that a key piece of the puzzle was an epidemic of bank failures. Over 9,000 banks failed from 1930 to 1933!

Could the Federal Reserve have prevented this catastrophe?

At the time, regional Federal Reserve branches had considerable policy independence. Some branches helped troubled banks with “easy money”. Others did not, following a “tight money” policy.

Metrics wizards Gary Richardson and William Troost used differences-in-differences to analyze a natural experiment in Mississippi, where one half of the state had tight money while the other half had easy. What did they find?

This introduction to differences-in-differences covers the following:
- Bank failures during the Great Depression
- Easy versus tight monetary policies; moral hazard
- Parallel data trends
- Calculating the treatment effect
- Assumptions for a valid differences-in-differences analysis

**INSTRUCTOR RESOURCES**
Troost/Richardson paper: https://www.journals.uchicago.edu/doi/abs/10.1086/649603
Econometrics test bank: https://mru.io/kt2
High school teacher resources: https://mru.io/o15
Professor resources: https://mru.io/t0f
EconInbox: https://mru.io/sm5

**MORE LEARNING**
Try out our practice questions: https://mru.io/wfd
See the full course: https://mru.io/469
Receive updates when we release new videos: https://mru.io/7g2
More from Marginal Revolution University: https://mru.io/c30

The path from cause to effect
is dark and dangerous,

but the weapons
of econometrics are strong,

wield differences-in-differences
when witnessing parallel trends.

♪ [music] ♪

Masters of metrics

look for convincing
ceteris paribus comparisons.

The ideal comparison contrasts
treatment and control groups

that look similar.

But sometimes this sort
of comparability is elusive.

When treatment and control groups

evolve similarly
in the absence of treatment,

even if from different
starting points,

there's hope for causal inference.

The weapon that exploits
parallel evolution,

masters say parallel trends,

is called differences-in-differences...

(voice whispering)
Differences-in-differences

- ...or DD for short.
- Alright. Nice.

Let's see how DD
can help us understand

one of the most important
economic events

in US history.

Look back with me now
at the Great Depression--

the worst economic catastrophe,
our country has ever known.

Unemployment hit 25% in 1933--

a level not seen before or since.

Millions lost their homes
or their land.

Suicide spiked, and hungry families

relied on soup kitchens
and bread lines

to keep from starving.

Economists argue fiercely over
the causes of the Great Depression.

Most agree, however,
that a key piece of the puzzle

is an epidemic of bank failures.

This was before deposit insurance.

So if your bank went bankrupt,
your savings disappeared with it,

Faced with a banking crisis,
the central bank has a choice:

lend freely to troubled banks

or stand aside and refuse to lend.

Lending freely to banks in trouble
is called easy money.

Refusing to lend is called tight money.

Monetarist masters Milton Friedman
and Anna Schwartz

famously called
the Great Depression

the "Great Contraction,"

accusing the Federal Reserve

of inflicting a misguided policy
of tight money

on the nation's teetering
financial institutions.

They argued that easy money

would have kept
many banks in business,

shortening the Great Depression,

But others disagree!

If banks are insolvent

because of unwise
lending decisions,

then bailouts just encourage
more foolishness.

Economists call this problem
"moral hazard."

The debate over bailouts
in moral hazard continues today.

Should financial behemoth
Lehman Brothers

had been allowed to fail
on the eve of the Great Recession,

in an ideal world,
we'd answer this question

applying different Fed policies
to randomly selected regions.

But we can still learn a lot

by using differences-in-differences

to compare trends across areas
with different monetary policies.

How's that even possible?

Don't the same Fed policies
apply to all banks in the US?

- Yeah.
- Good question.

The Federal Reserve System
is divided into 12 districts,

each headed by a regional bank.

Today, Fed policy is set
at the national level.

But in the 1930s, regional Feds
could do pretty much as they liked.

Ah, interesting.

And here's what's
so awesome about that.

In 1930, the Atlanta Fed,
running the 6th District,

followed an easy money policy,

sending wheelbarrows of cash
to rescue insolvent institutions,

The St. Louis Fed,
running the 8th District,

followed a tight money policy.

"Let fail the foolish!"
they said in St. Louis.

And so a natural experiment
in monetary policy was born.

Even better, this is
a within-state experiment.

The border between the 6th
and the 8th districts

ran smack through
the middle of Mississippi.

So northern Mississippi
had tight money,

while southern Mississippi
had easy money,

but under the same state laws
in banking regulations in both.

The treatment group

is the District 6th part
of Mississippi,

which had access to easy money
during the crisis.

The control group

is the District 8th part
of Mississippi,

which had tight money
during the crisis.

The key year
in our natural experiment

was 1930,

Caldwell & Company,

a massive financial empire
in the South

came crashing down.

Banking is a business
built on confidence and trust.

The Caldwell meltdown
caused a panic

that led to a widespread
bank run all at once.

Depositors wanted their money back,

causing banks to go bankrupt
and shut their doors.

We'll use differences-in-differences

to measure the effect
of contrasting monetary policies

in response to the Caldwell crisis.

This figure plots the number
of banks in Mississippi by year,

for the 8th and 6th districts.

Let's start in 1929 - a year
before the Caldwell crash.

There are 169 banks
open in the 8th,

and 141 banks open in the 6th.

Over the next year,

we see a similar handful
of banks fail, in both districts.

The change in the number
of banks in operation

is remarkably similar.

That's what parallel trends look like.

In November 1930, Caldwell crashes,

and the panic begins.

Banks failed frequently
in the 8th district,

which had tight money.

But the decline is slower
in the 6th District,

which had easy money.

The diverging trends in this period

might be attributable
to easy versus tight money.

In July of 1931, the 8th district
abandons tight money,

so now both districts are easy.

Parallel trends are restored.

In a counterfactual world,

where the 6th district
follows a tight money policy,

what might have happened?

If we extrapolate the trend
of the 8th district to the 6th,

it would look like this.

So the treatment effective
easy money

is how much the 6th district
deviated from the path

implied by the 8th district trend.

How many banks
did the easy money treatment save?

This table reports data
for the treatment group, District 6,

in the first row

and data for the control group,
District 8, in the second row.

The first column shows
the number of banks in business

before the crisis began in 1930.

The second column shows 1931.

This is the key period

when each district
had differing monetary policies

during the crisis.

The rightmost column
reports changes within the district.

District 6 lost 14 banks,
while District 8 lost 33.

The mathematical formula
for the treatment effect is simple.

We subtract the change in banks
in operation, in the 8th District

from the change in banks
in operation in the sixth.

Hence. The name
differences in differences,

negative 14, minus negative, 33 equals 9.

19.

We estimate that 19 Banks
were saved by easy money

in practice tables and figures like
those shown here are the beginning

rather than the end of a DD analysis,

the problem of how to gauge the
statistical, significance of DD, estimates

turns out to be exceedingly, tricky

and a regression is typically
part of the solution,

the key assumption behind a

DD analysis is that of parallel Trends
recall, the principle of ceteris paribus,

our ideal comparison would have
the two districts experienced,

an identical business environment, except
for one factor easy or tight money.

Both districts would have identical
types of customers who would go bankrupt

at exactly the same rate.

The skill of their employees
would be equal and so on

perfect ceteris, paribus comparisons
would allow us to clearly see

the causal effect of different fed
policies in this case, that's not possible.

But the idea of parallel Trends
is based on a similar concept.

If we see that the two regions
experienced similar Trends in the number

of banks over.

Time in the absence of treatment, we
can assume they are good comparisons.

We see that the two districts move in
parallel both before the crisis. And after,

when they have the same Fed policy,

the only time the district's behave differently
is when the Fed policy is different.

In view of this Fed policy is a likely
cause of diverging Trends from 1930 to 1931.

But we should also check for other
changes unique to northern, Mississippi.

What do you mean?

Imagine that bad tornadoes? Hit Northern,
but not Southern. Mississippi in 1930,

these tornadoes devastate Farms
causing Farmers to default on loans,

which drives their Banks out of business.

Then the sixth and eighth districts
would differ in not one, but two ways

Fed policy, and whether and
we'd have trouble identifying

Policy as the causal Factor behind
increased bank failures in the eighth.

DD credibility lives or
dies with the claim that the

only reason northern
Mississippi was special in 1930

is differing. Regional Fed policy.

We're in DD heaven with strong Visual
Evidence of parallel Trend in general.

The first step in evaluating whether

to use DD is usually this type of
visual confirmation of parallel Trends

outside of the period.

When we expect to see a treatment
effect, the treatment in our example,

Easy money in the face of bank failures

metrics Masters, use DD to
explore effects of many policies

like the minimum, legal drinking age,

and environmental changes
like access to clean water.

In our next video.

We'll see an example of how regression
is used to implement a DD approach.

Are you a teacher click to explore ways to
use these videos in class if your learner

Make sure this video sticks by taking
a few quick practice questions,

or if you're ready.
Click for the next video.

You can also check out Mr. Use website for
more courses, teacher resources and more.