9:59:59.000,9:59:59.000
- [Instructor] The path from cause [br]to effect is dark and dangerous.

9:59:59.000,9:59:59.000
But the weapons [br]of Econometrics are strong.

9:59:59.000,9:59:59.000
[Attack] with fierce [br]and flexible instrumental variables

9:59:59.000,9:59:59.000
when nature blesses you [br]with fortuitous random assignment.

9:59:59.000,9:59:59.000
[gong rings]

9:59:59.000,9:59:59.000
Randomized trials are the surest [br]path to ceteris parabus comparisons.

9:59:59.000,9:59:59.000
Alas, this powerful tool [br]is often unavailable.

9:59:59.000,9:59:59.000
But sometimes, randomization [br]happens by accident.

9:59:59.000,9:59:59.000
That's when we turn [br]to instrumental variables,

9:59:59.000,9:59:59.000
IV for short.

9:59:59.000,9:59:59.000
- [Voice whispers] Instrumental [br]variables.

9:59:59.000,9:59:59.000
- [Instructor] Today's lesson [br]is the first of two on IV.

9:59:59.000,9:59:59.000
Our first IV lesson begins [br]with a story of schools.

9:59:59.000,9:59:59.000
- [Josh] Charter schools [br]are public schools

9:59:59.000,9:59:59.000
freed from daily district oversight[br]and teacher union contracts.

9:59:59.000,9:59:59.000
The question of whether charters [br]boost achievement

9:59:59.000,9:59:59.000
is one of the most important

9:59:59.000,9:59:59.000
in the history [br]of American education reform.

9:59:59.000,9:59:59.000
- The most popular charter schools[br]have more applicants

9:59:59.000,9:59:59.000
so the luck of the lottery draw [br]decides who's offered a seat.

9:59:59.000,9:59:59.000
A lot is at stake for the students [br]vying for their chance.

9:59:59.000,9:59:59.000
Waiting for the lottery results [br]brings up lots of emotions

9:59:59.000,9:59:59.000
as was captured [br]in the award-winning documentary

9:59:59.000,9:59:59.000
"Waiting For Superman."

9:59:59.000,9:59:59.000
- [Mother] Don't cry.[br]You're gonna make Mommy cry.

9:59:59.000,9:59:59.000
Okay?

9:59:59.000,9:59:59.000
- Do charters really provide[br]a better education?

9:59:59.000,9:59:59.000
Critics most definitely say no,

9:59:59.000,9:59:59.000
arguing that charters enroll [br]better students to begin with,

9:59:59.000,9:59:59.000
smarter or more motivated[br]so differences in later outcomes

9:59:59.000,9:59:59.000
reflects selection bias.

9:59:59.000,9:59:59.000
- [Kamal] Wait, this one seems easy.

9:59:59.000,9:59:59.000
In a lottery, winners [br]are chosen randomly,

9:59:59.000,9:59:59.000
so just compare winners and losers.

9:59:59.000,9:59:59.000
- [Student] Obviously.

9:59:59.000,9:59:59.000
- On the right track, Kamal,

9:59:59.000,9:59:59.000
but charter lotteries [br]don't force kids into

9:59:59.000,9:59:59.000
or out of a particular school.

9:59:59.000,9:59:59.000
They randomize offers [br]of a charter seat.

9:59:59.000,9:59:59.000
Some kids get lucky.

9:59:59.000,9:59:59.000
Some kids don't.

9:59:59.000,9:59:59.000
If we just wanted to know [br]the effect of charter school offers,

9:59:59.000,9:59:59.000
we could treat this [br]as a randomized trial.

9:59:59.000,9:59:59.000
But we we're interested [br]in the effects

9:59:59.000,9:59:59.000
of charter school attendance,[br]not offers.

9:59:59.000,9:59:59.000
And not everyone [br]who is offered, accepts.

9:59:59.000,9:59:59.000
IV turns the effect of being offered[br]a charter seat into the effect

9:59:59.000,9:59:59.000
of actually attending [br]a charter school.

9:59:59.000,9:59:59.000
- [Student] Cool.

9:59:59.000,9:59:59.000
- Oh nice.

9:59:59.000,9:59:59.000
- Let's look at an example, [br]a charter school from

9:59:59.000,9:59:59.000
the Knowledge Is Power [br]Program, or KIPP for short.

9:59:59.000,9:59:59.000
This KIPP school is in Lynn,

9:59:59.000,9:59:59.000
a faded industrial town [br]on the coast of Massachusetts.

9:59:59.000,9:59:59.000
The school has [br]more applicants than seats

9:59:59.000,9:59:59.000
and therefore picks its students[br]using a lottery.

9:59:59.000,9:59:59.000
From 2005 to 2008,[br]371 fourth and fifth graders

9:59:59.000,9:59:59.000
put their names [br]in the KIPP/Lynn lottery,

9:59:59.000,9:59:59.000
253 students won a seat at KIPP,

9:59:59.000,9:59:59.000
118 students lost.

9:59:59.000,9:59:59.000
A year later, lottery winners had [br]much higher match scores

9:59:59.000,9:59:59.000
than lottery losers.

9:59:59.000,9:59:59.000
But remember, [br]we're not trying to figure out

9:59:59.000,9:59:59.000
whether winning a lottery [br]makes you better at math.

9:59:59.000,9:59:59.000
We want to know if attending KIPP[br]makes you better at math.

9:59:59.000,9:59:59.000
Of the 253 lottery winners,[br]only 199 actually went to KIPP.

9:59:59.000,9:59:59.000
The others chose [br]a traditional public school.

9:59:59.000,9:59:59.000
Similarly of the 118 lottery losers,[br]a few actually ended up at KIPP.

9:59:59.000,9:59:59.000
They got an offer later.

9:59:59.000,9:59:59.000
So what was the effect of test scores[br]of actually attending KIPP?

9:59:59.000,9:59:59.000
- [Student] Why can't we just [br]measure their math scores?

9:59:59.000,9:59:59.000
- [Instructor] Great question.

9:59:59.000,9:59:59.000
Who would you compare them to?

9:59:59.000,9:59:59.000
- [Student] Those who didn't attend.

9:59:59.000,9:59:59.000
- [Instructor] Is attendance random?

9:59:59.000,9:59:59.000
- [Camilla] No.

9:59:59.000,9:59:59.000
- Selection bias.

9:59:59.000,9:59:59.000
- [Instructor] Correct.

9:59:59.000,9:59:59.000
- [Otto] What?

9:59:59.000,9:59:59.000
- [Instructor] The KIPP offers [br]are random so we can be confident

9:59:59.000,9:59:59.000
of ceteris parabus,[br]but attendance is not random.

9:59:59.000,9:59:59.000
The choice to accept the offer [br]might be due to characteristics

9:59:59.000,9:59:59.000
that are related [br]to math performance.

9:59:59.000,9:59:59.000
Say, for example, [br]that dedicated parents

9:59:59.000,9:59:59.000
are more likely [br]to accept the offer.

9:59:59.000,9:59:59.000
Their kids are also more likely [br]to do better in math,

9:59:59.000,9:59:59.000
regardless of school.

9:59:59.000,9:59:59.000
- [Student] Right.

9:59:59.000,9:59:59.000
- [Instructor] IV converts [br]the offer effect

9:59:59.000,9:59:59.000
into the effect of KIPP attendance,

9:59:59.000,9:59:59.000
adjusting for the fact [br]that some winners go elsewhere

9:59:59.000,9:59:59.000
and some losers manage [br]to attend KIPP anyway.

9:59:59.000,9:59:59.000
Essentially, IV takes [br]an incomplete randomization

9:59:59.000,9:59:59.000
and makes the appropriate [br]adjustments.

9:59:59.000,9:59:59.000
How? IV describes a chain reaction.

9:59:59.000,9:59:59.000
Why do offers affect achievement?

9:59:59.000,9:59:59.000
Probably because they affect [br]charter attendance

9:59:59.000,9:59:59.000
and charter attendance [br]improves math scores,

9:59:59.000,9:59:59.000
the first link in the chain[br]called the first stage

9:59:59.000,9:59:59.000
is the effect of the lottery [br]on charter attendance.

9:59:59.000,9:59:59.000
The second stage is the length [br]between attending a charter

9:59:59.000,9:59:59.000
and an outcome variable,

9:59:59.000,9:59:59.000
in this case, math scores.

9:59:59.000,9:59:59.000
The instrumental variable [br]or instrument for short

9:59:59.000,9:59:59.000
is the variable [br]that initiates the chain reaction.

9:59:59.000,9:59:59.000
The effect of the instrument [br]on the outcome is called

9:59:59.000,9:59:59.000
the reduced form.

9:59:59.000,9:59:59.000
This chain reaction can be [br]represented mathematically.

9:59:59.000,9:59:59.000
We multiply the first stage,[br]the effect of winning

9:59:59.000,9:59:59.000
on attendance, by the second stage,

9:59:59.000,9:59:59.000
the affect of attendance on scores.

9:59:59.000,9:59:59.000
And we get the reduced form,

9:59:59.000,9:59:59.000
the effect of winning [br]the lottery on scores.

9:59:59.000,9:59:59.000
Reduced form and first stage [br]are observable and easy to compute.

9:59:59.000,9:59:59.000
However, the effect of attendance[br]on achievement

9:59:59.000,9:59:59.000
is not directly observed.

9:59:59.000,9:59:59.000
This is the causal effect[br]we're trying to determine.

9:59:59.000,9:59:59.000
Given some important assumptions [br]we'll discuss shortly,

9:59:59.000,9:59:59.000
we can find the effect [br]of KIPP attendance

9:59:59.000,9:59:59.000
by dividing the reduced form[br]by the first stage.

9:59:59.000,9:59:59.000
This will become more clear [br]as we work through an example.

9:59:59.000,9:59:59.000
- [Student] Let's do this.

9:59:59.000,9:59:59.000
- A quick note on measurement.

9:59:59.000,9:59:59.000
We measure achievement[br]using standard deviations,

9:59:59.000,9:59:59.000
often denoted [br]by the Greek letter sigma (σ).

9:59:59.000,9:59:59.000
One σ is a huge move[br]from around the bottom 15%

9:59:59.000,9:59:59.000
to the middle of most[br]achievement distributions.

9:59:59.000,9:59:59.000
Even a 1/4 or 1/2 σ [br]difference is big.

9:59:59.000,9:59:59.000
- [Instructor] Now we're ready [br]to plug some numbers

9:59:59.000,9:59:59.000
into the equation [br]we introduced earlier.

9:59:59.000,9:59:59.000
First up, what's the effect

9:59:59.000,9:59:59.000
of winning the lottery [br]on math scores?

9:59:59.000,9:59:59.000
KIPP applicant math scores [br]are a third of a standard deviation

9:59:59.000,9:59:59.000
below the state average in [br]the year before they apply to KIPP.

9:59:59.000,9:59:59.000
But a year later, lottery winners[br]score right at the state average

9:59:59.000,9:59:59.000
while the lottery losers [br]are still well behind.

9:59:59.000,9:59:59.000
With an average score [br]around - 0.36 σ.

9:59:59.000,9:59:59.000
The effect of winning the lottery [br]on scores is the difference

9:59:59.000,9:59:59.000
between the winners' scores[br]and the losers' scores.

9:59:59.000,9:59:59.000
Take the winners' [br]average math scores,

9:59:59.000,9:59:59.000
subtract the losers' [br]average math scores,

9:59:59.000,9:59:59.000
and you will have 0.36 σ .

9:59:59.000,9:59:59.000
Next stop: what's the effect [br]of winning the lottery on attendance?

9:59:59.000,9:59:59.000
In other words, [br]if you win the lottery,

9:59:59.000,9:59:59.000
how much more likely are you [br]to attend KIPP than if you lose?

9:59:59.000,9:59:59.000
First, what percentage [br]of lottery winners attend KIPP?

9:59:59.000,9:59:59.000
Divide the number of winners[br]who attended KIPP

9:59:59.000,9:59:59.000
by the total number [br]of lottery winners -- that's 78%.

9:59:59.000,9:59:59.000
To find the percentage [br]of lottery losers who attended KIPP,

9:59:59.000,9:59:59.000
we divide the number of losers [br]who attended KIPP

9:59:59.000,9:59:59.000
by the total number [br]of lottery losers -- that's 4%.

9:59:59.000,9:59:59.000
Subtract 4 from 78, and we find [br]that winning the lottery

9:59:59.000,9:59:59.000
makes you 74% [br]more likely to attend KIPP.

9:59:59.000,9:59:59.000
Now we can find [br]what we're really after,

9:59:59.000,9:59:59.000
the effect of attendance on scores,[br]by dividing 0.36 by 0.74.

9:59:59.000,9:59:59.000
Attending KIPP raises math scores

9:59:59.000,9:59:59.000
by 0.48 standard deviations [br]on average.

9:59:59.000,9:59:59.000
That's an awesome achievement game,

9:59:59.000,9:59:59.000
equal to moving [br]from about the bottom

9:59:59.000,9:59:59.000
to the middle [br]of the achievement distribution.

9:59:59.000,9:59:59.000
- [Student] Whoa, half a sig.

9:59:59.000,9:59:59.000
- [Instructor] These estimates [br]are for kids opting in

9:59:59.000,9:59:59.000
to the KIPP lottery,[br]whose enrollment status

9:59:59.000,9:59:59.000
is changed by winning.

9:59:59.000,9:59:59.000
That's not necessarily [br]a random sample

9:59:59.000,9:59:59.000
of all children in Lynn.

9:59:59.000,9:59:59.000
So we can't assume [br]we'd see the same effect

9:59:59.000,9:59:59.000
for other types of students.

9:59:59.000,9:59:59.000
- [Students] Huh.

9:59:59.000,9:59:59.000
- But this effect [br]on keen for KIPP kids

9:59:59.000,9:59:59.000
is likely to be a good indicator[br]of the consequences

9:59:59.000,9:59:59.000
of adding additional charter seats.

9:59:59.000,9:59:59.000
- [Student] Cool.[br]- [Student] Got it.

9:59:59.000,9:59:59.000
- IV eliminates selection bias [br]but like all of our tools,

9:59:59.000,9:59:59.000
the solution builds on a set [br]of assumptions

9:59:59.000,9:59:59.000
not to be taken for granted.

9:59:59.000,9:59:59.000
First, there must be [br]a substantial first stage --

9:59:59.000,9:59:59.000
that is the instrumental variable,[br]winning or losing the lottery,

9:59:59.000,9:59:59.000
must really change the variable[br]whose effect we're interested in --

9:59:59.000,9:59:59.000
here, KIPP attendance.

9:59:59.000,9:59:59.000
In this case, the first stage [br]is not really in doubt.

9:59:59.000,9:59:59.000
Winning the lottery makes [br]KIPP attendance much more likely.

9:59:59.000,9:59:59.000
Not all IV stories are like that.

9:59:59.000,9:59:59.000
Second, the instrument [br]must be as good

9:59:59.000,9:59:59.000
as randomly assigned [br]meaning lottery winners and losers

9:59:59.000,9:59:59.000
have similar characteristics.

9:59:59.000,9:59:59.000
This is the independence assumption.

9:59:59.000,9:59:59.000
Of course, KIPP lottery wins[br]really are randomly assigned.

9:59:59.000,9:59:59.000
Still, we should check for balance [br]and confirm that winners and losers

9:59:59.000,9:59:59.000
have similar family backgrounds,

9:59:59.000,9:59:59.000
similar aptitudes and so on.

9:59:59.000,9:59:59.000
In essence, we're checking [br]to ensure KIPP lotteries are fair

9:59:59.000,9:59:59.000
with no group of applicants [br]suspiciously likely to win.

9:59:59.000,9:59:59.000
Finally, we require [br]the instrument change outcomes

9:59:59.000,9:59:59.000
solely through [br]the variable of interest,

9:59:59.000,9:59:59.000
in this case, attending KIPP.

9:59:59.000,9:59:59.000
This assumption is called[br]the exclusion restriction.

9:59:59.000,9:59:59.000
- IV only works if you can satisfy [br]these three assumptions.

9:59:59.000,9:59:59.000
- I don't understand [br]the exclusion restriction.

9:59:59.000,9:59:59.000
How could winning the lottery[br]affect math scores

9:59:59.000,9:59:59.000
other than by attending KIPP?

9:59:59.000,9:59:59.000
- [Student] Yeah.

9:59:59.000,9:59:59.000
- [Instructor] Great question.

9:59:59.000,9:59:59.000
Suppose lottery winners[br]are just thrilled to win,

9:59:59.000,9:59:59.000
and this happiness motivates them [br]to study more and learn more math

9:59:59.000,9:59:59.000
regardless of where [br]they go to school.

9:59:59.000,9:59:59.000
This would violate [br]the exclusion restriction

9:59:59.000,9:59:59.000
because the motivational effect [br]of winning is a second channel

9:59:59.000,9:59:59.000
whereby lotteries [br]might affect test scores.

9:59:59.000,9:59:59.000
While it's hard [br]to rule this out entirely,

9:59:59.000,9:59:59.000
there's no evidence [br]of any alternative channels

9:59:59.000,9:59:59.000
in the KIPP study.

9:59:59.000,9:59:59.000
- IV solves the problem [br]of selection bias

9:59:59.000,9:59:59.000
in scenarios like the KIPP lottery[br]where treatment offers are random

9:59:59.000,9:59:59.000
but some of those offered opt out.

9:59:59.000,9:59:59.000
This sort of intentional [br]yet incomplete random assignment

9:59:59.000,9:59:59.000
is surprisingly common.

9:59:59.000,9:59:59.000
Even randomized clinical trials[br]have this feature.

9:59:59.000,9:59:59.000
IV solves the problem [br]of non-random take up

9:59:59.000,9:59:59.000
in lotteries or clinical research.

9:59:59.000,9:59:59.000
But lotteries are not the only[br]source of compelling instruments.

9:59:59.000,9:59:59.000
Many causal questions can be[br]addressed by naturally occurring

9:59:59.000,9:59:59.000
as good as randomly [br]assigned variation.

9:59:59.000,9:59:59.000
Here's a causal question for you --

9:59:59.000,9:59:59.000
do women who have children early [br]in their careers suffer

9:59:59.000,9:59:59.000
a substantial earnings penalty [br]as a result?

9:59:59.000,9:59:59.000
After all, women earn less than men.

9:59:59.000,9:59:59.000
We could, of course, simply [br]compare the earnings of women

9:59:59.000,9:59:59.000
with more and fewer children.

9:59:59.000,9:59:59.000
But such comparisons are fraught[br]with selection bias.

9:59:59.000,9:59:59.000
If only we could randomly assign [br]babies to different households.

9:59:59.000,9:59:59.000
Yeah, right, [br]sounds pretty fanciful.

9:59:59.000,9:59:59.000
Our next IV story -- fantastic[br]and not fanciful --

9:59:59.000,9:59:59.000
illustrates an amazing, [br]naturally-occurring instrument

9:59:59.000,9:59:59.000
for family size.

9:59:59.000,9:59:59.000
- [Instructor] You're on your way [br]to mastering Econometrics.

9:59:59.000,9:59:59.000
Make sure this video sticks

9:59:59.000,9:59:59.000
by taking a few [br]quick practice questions.

9:59:59.000,9:59:59.000
Or, if you're ready, [br]click for the next video.

9:59:59.000,9:59:59.000
You can also check out [br]MRU's website for more courses,

9:59:59.000,9:59:59.000
teacher resources, and more.

9:59:59.000,9:59:59.000
♪ [music] ♪