♪ [music] ♪
- [Joshua] As I stilled my trembling
iPhone early on October 11th,
my thoughts went to the question
of whether Nobel-level recognition
might change life
for the Angrist family.
Ours is a close-knit family.
We lack for nothing.
So I worried
that stressful Nobel celebrity
might not be a plus.
But with the first cup of coffee,
I began to relax.
It occurred to me
that the matter
of how public recognition
affects a scholars life
is, after all,
a simple causal question.
The Nobel intervention
is substantial, sudden,
and well-measured.
Outcomes like health and wealth
are easy to record.
Having just been recognized
with my co-laureates,
Guido Imbens and David Card,
for answering causal questions
using observational data,
my thoughts moved
from personal upheaval
to the more familiar demands
of identification and estimation
of causal effects.
I was able to soothe
my worried mind
by imagining a study
of the Nobel Prize treatment effect.
How would such a study
be organized?
in a 1999 essay published in the
"Handbook of Labor Economics,"
Alan Krueger and I embraced
the phrase "empirical strategy."
The handbook volume
in question was edited
by two of my Princeton
PhD thesis advisors,
Orley Ashenfelter and David Card,
among the most successful
and prolific graduate advisors
economics has known
An empirical strategy
is a research plan
that encompasses data collection,
identification,
and econometric estimation.
Identification is the applied
econometricians term
for research design --
a randomized, clinical trial.
And RCT is the simplest
and most powerful research design.
In RCTs, causal effects
are identified
by the random assignment
of treatment.
Random assignment ensures
that treatment and control groups
are comparable
in the absence of treatment.
So differences
between them afterwards
reflect only the treatment effect.
Nobel prizes are probably
not randomly assigned.
This challenge notwithstanding,
a compelling empirical strategy
for the Nobel treatment effect
comes to mind,
at least as a flight of empirical fancy.
Imagine a pool of prize-eligible
Nobel applicants,
the group under consideration
for the prize.
Applicants need not apply themselves.
They are, I presume, nominated
by their peer scholars.
My fanciful Nobel impact study
looks only at Nobel applicants
since these are all elite scholars.
But that is only the first step.
Credible applicants, I imagine,
are evaluated by judges
using criteria, like publications,
citations, nominating statements.
I imagine this material is reviewed
and assigned a numerical score,
using some kind of scoring rubric.
Top scorers up to three per field,
in any single year,
win a prize.
Having identified applicants
that collected data
on their scores,
the next step
in my Nobel impact study
is to record the relevant cutoffs.
The Nobel cutoff
is the lowest score
among those awarded a prize,
Many Nobel hopefuls
just missed the cutoff.
Looking only at near-misses,
along with the winners,
differences in scores
between those above
and below the cutoff
begin to look serendipitous,
almost randomly assigned.
After all,
near-Nobels are among
the most eminent of scholars, too.
With one more
high-impact publication,
a little more support
from nominators.
they would have been awarded
Nobel gold --
some of them, someday,
surely will be.
The empirical strategy
sketched here is called
a Regression Discontinuity Design,
RD.
RD exploits the jumps
in human affairs,
induced by rules, regulations,
and the need to classify people
for various assignment purposes.
When treatment
or intervention is determined
by whether a tiebreaking variable
crosses a threshold,
those just below the threshold
become a natural control group
for those who clear it.
RD does not require
that the variable,
whose causes we seek,
switch fully on or fully off
at the cutoff.
We require only
that the average value
of this variable
jump at the cutoff
RD can allow, for example,
for the fact
that this year's near-Nobel
might be next year's winner.
Allowing for this leads
to the use of jumps
and the rate at which
treatment is assigned
to construct instrumental variables,
IV,
estimates of the effect
of treatment received.
This sort of RD
is said to be fuzzy,
But as Steve Pischke and I
wrote in our first book:
"fuzzy RD is IV."
(cheering)
The first RD study I contributed to
was written with my frequent
collaborator, Victor Lavy.
This study is motivated
by the high costs
and uncertain returns
to smaller elementary school classes.
We exploited a rule
used by Israeli elementary schools
to determine class size.
This rule is used to estimate
class size effects,
as if in a class size RCT.
In the 1990s,
Israeli classes were large.
Students enrolled
in a grade cohort of 40.
We're likely to be
seated in a class of 40.
That's the relevant cutoff.
Add another child
to the cohort, making 41,
and the cohort
was likely to be split
into two much smaller classes.
This leads to the Maimonide's
rule research design,
so named because
the 12th-century Rambam
proposed a maximum
class size of 40.
This figure plots is rarely
fourth grade class sizes
as a function
of fourth grade enrollment,
overlaid with the theoretical
class size rule,
Maimonides rule.
The fit isn't perfect.
That's a feature that makes
this application of RD fuzzy.
But the gist of the thing
is a marked class size drop,
at each integer multiple of 40,
the relevant cutoff,
just as predicted by the rule.
As it turns out,
these drops in class size
are reflected in jumps
in 4th and 5th grade math scores.
♪ [music] ♪
Would a comparison
of Nobel laureates to near laureates
really be a good natural experiment?
The logic behind this sort of claim
seems more compelling
for comparisons of schools
with 40 and 41 4th graders
than for comparisons
of laureates and near laureates.
Yet, both scenarios exploit
a feature of the physical world.
Provided the tiebreaking variable
known to RD mavens
as the running variable,
has a continuous distribution,
the probability of crossing
the cutoff approaches one-half
when examined in a narrow window
around the cutoff.
In RD empirical work,
the window around such cutoffs
is known as a bandwidth.
Importantly, this limiting
probability is 0.5 for everybody
regardless of how qualified they look
going into the Nobel competition.
This remarkable fact can be seen
in data on applicants
to one of New York's
highly coveted screen schools.
By way of background,
roughly 40% of New York City's
middle and high schools
select their applicants
on the basis of test scores, grades,
and other exacting criteria.
In other words, the admissions
regime for screen schools
is a lot like the scheme
I've imagined
for the Nobel Prize.
Screen schools
are but one of a number
of highly selective systems
within a system
in large US school districts.
Boston, Chicago, San Francisco,
and Washington, D.C
all feature highly selective
institutions,
often known as exam schools.
Exam schools operate
as part of larger
public school systems
that enroll students
without screening.
Motivated by
the enduring controversy
over the equity
of screened admissions,
my blueprint
labs collaborators and I
have examined the causal effects
of exam school attendance
in Boston, Chicago, and New York.
This figure shows the probability
of being offered a seat
at New York's storied
Townsend Harris High School,
ranked 12th, nationwide.
Bar height in the figure
marks the qualification rate --
that is, the likelihood of earning
a Townsend Harris admission score,
above that of the lowest
scoring applicant
offered a seat.
Importantly, the bars show
qualification rates conditional
on a measure of pre-application
baseline achievement.
In particular, the bars mark
qualification rates conditional
on whether an applicant
has upper quartile or lower quartile
6th grade math scores.
Townsend Harris applicants
with high baseline scores
are much more likely to qualify
than applicants
with low baseline scores.
This isn't surprising.
But in a shrinking
symmetric bandwidth
around the schools cutoff,
qualification rates
in the two groups converge
Qualification rate in the last
and smallest groups
are both remarkably close
to one-half
This is what we'd expect to see
where Townsend Harris
to admit students,
by tossing a coin,
rather than by selecting
only those who scored highly
on the school's entrance exam.
Even when admissions
operates by screening,
the data can be arranged
so as to mimic an RCT.
A few of the questions I've studied
are more controversial
than the question of access
to public exam schools,
like the Boston Latin School,
Chicago's Payton and Northside
selective enrollment high schools,
and New York's legendary
Brooklyn Tech,
Bronx Science,
and Stuyvesant
specialized high schools,
which have graduated
14 Nobel laureates between them.
Townsend Harris, the school
we started with today,
graduated three Nobels,
including economist Ken Arrow.
Exam school proponents
see the opportunities
these schools provide
as democratizing public education.
"Wealthy families," they argue,
can access exam school curricula
in the private sector.
Shouldn't ambitious
low-income students
be afforded the same chance
at elite education?
Critics of selective
enrollment schools
argue that rather than
expanding equity,
exam schools are inherently biased
against the Black
and Hispanic students
that make up the bulk
of America's urban districts,
New York's super selective
Stuyvesant, for example,
enrolled only
seven Black students in 2019,
out of an incoming class of 895.
But are exam school seats
really worth fighting for?
My collaborators and I
have repeatedly used
RD empirical strategies
to study the causal effects
of attendance at exam schools
like Townsend Harris
and Boston Latin.
Our first exam school study,
which looks at schools
in Boston and New York
encapsulates
these findings in its title:
"The Elite Illusion."
"The Elite Illusion"
refers to the fact
that while exam school students
undoubtedly have high test scores
and other good outcomes,
this is sot a causal effect
of exam School attendance.
Our estimates consistently suggest
that the causal effects
of exam school attendance
on their students learning
and college-going are 0.
Maybe even negative.
The good performance
of exam school students
reflect selection bias --
that is, the process by which
these students are chosen,
rather than causal effects.
Data from Chicago's large exam
school sector
illustrate the elite illusion.
This figure plots
peer mean achievement --
that is, the 6th grade test scores
of my 9th grade classmates
against the admissions tiebreaker
for a subset of applicants
to any one of Chicago's
nine exam schools.
Applicants to these schools
rank up to 6,
while the exam schools
prioritize their applicants
using a common composite index,
formed from an admissions test,
GPAs, and grade 7
standardized scores.
This composite tiebreaker
is the running variable
for an RD design
that reveals what happens
when any applicant is offered
an exam school seat.
In Chicago's exam school match,
which is actually an application
of the celebrated Gale and Shapley
matching algorithm
exam school applicants are sure
to be offered a seat somewhere
when they clear the lowest
in their set of cutoffs
among the schools they rank.
We call this lowest cutoff
the "qualifying cutoff."
The figure shows a sharp jump
in peer mean achievement
for Chicago exam school applicants
who clear their qualifying cutoff.
This jump reflects the fact
that most applicants
offered an exam school seat take it,
and applicants who enroll
at one of Chicago's selective
enrollment high schools
are sure to be seated
in a 9th grade classroom
filled with academically
precocious peers,
because only the relatively
precocious make it in.
The increase in peer achievement
across the qualifying cutoff
amounts to almost
half a standard deviation --
a very large effect.
And yet, precocious peers,
notwithstanding,
the offer of an exam school seat
does not appear to increase learning.
Let's plot applicants ACT scores
against their tiebreaker values.
This plot shows that exam
school applicants
who clear their qualifying cutoff
perform sharply worse on the ACT.
What explains this?
It takes a tale of IV and RD
to untangle the forces
behind this intriguing
and unexpected negative effect.
But first, some IV theory,
Guido Imbens and I
developed theoretical tools
that enhance economists'
understanding of empirical strategies
involving IV and RD.
The prize we share
is in recognition of this work.
Guido and I overlapped for
only one year at Harvard,
where we had both taken
our first jobs post PhD.
I welcomed Guido
to Cambridge, Massachusetts
with a pair of interesting
instrumental variables.
I had used
the draft lottery instrument
in my PhD thesis
to estimate the long-run
economic consequences
of serving in the Armed Forces
for soldiers who were drafted.
The draft lottery instrument
relies on the fact
that lottery numbers
randomly assigned to birthdays
determined Vietnam-era
conscription risk.
Yet, even then, most soldiers
were volunteers,
as they are today,
The quarter birth instrument
is used in my 1991 paper
with Alan Krueger
to estimate the economic
returns to schooling.
This instrument uses the fact
that men who are born earlier
in the year
are allowed to drop out
of high school
on their 16th birthday
with less schooling completed
than those born later.
Guido and I soon
began asking each other,
"What really do we learn
from the draft eligibility
and quarter of birth
natural experiments?"
An early result in our quest
for a new understanding of IV
was a solution to the problem
of selection bias
in an RCT with partial compliance.
Even in a randomized clinical trial,
some of the people assigned
to treatment may opt out.
This fact has long vexed trialists
because decisions to opt out
are not made by random assignment.
Our first manuscript together
shows that in a randomized trial
with partial compliance,
you can use IV
to estimate the effect
of treatment on the treated,
even when some offered
treatment decline it.
This works in spite of the fact
that those who comply
with treatment
may be a very select group.
Unfortunately, for us,
we were late to the party.
Not long after releasing
our first working paper,
we learned of a concise
contribution from Howard Bloom
that includes this theoretical result.
Remarkably, Bloom had derived
this from first principles
without making a connection to IV.
So Guido and I went back
to the drawing board.
And a few months later, we had LATE --
a theorem showing how to estimate
the local average treatment effect.
The LATE theorem generalizes
the Bloom theorem
and establishes the connection
between compliance and IV.
Maintaining the clinical trials analogy,
let "Zi" indicate whether subject "i"
is offered treatment.
This is randomly assigned.
Also, let "D1i" indicate
subject "i's" treatment status
when assigned to treatment,
and let the "0i" indicate
subject "i's" treatment status
when assigned to control.
I'll use this formal notation
to give a clear statement
of the late result,
and then follow up with examples.
A key piece of the late framework
pioneered by statistician
Don Rubin,
is the pair of potential outcomes.
As is customary,
I denote potential outcomes
for subject "i"
in the treated and untreated states
by "Y1i" and "Y0i" respectively,
The observed outcome
is "Y1i" for the treated
and "Y0i" for those not treated.
"Y1i" minus "Y0i"
is the causal effect
of treatment on individual "i",
but we can never see.
We try, therefore, to estimate
some kind of average causal effect.
The late framework allows us
to do that in an RCT
where some controls are treated.
The theorem says
that the average causal
effect on people,
whose treatment status
can be changed
by the offer of treatment
is the ratio of ITT
to the treatment control difference
in compliance rates.
A mathematical statement of
this result appears here,
where Greek letter Delta
symbolizes the ITT effect
and Greek symbols pi1 and pi0
are compliance rates
in the group assigned to treatment
and the group assigned
to control, respectively,
The print version of this lecture
delves deeper
into LATE intellectual history,
highlighting key contributions
made with Rubin.
For now, though, I'd like to make
the late theorem concrete for you
by sharing one of
my favorite applications of it.
I'll explain the late framework
through a research question
that has fascinated me
for almost two decades.
What is the causal effect
of charter school attendance
on learning?
Charter schools are public schools
that operate independently
of traditional American
public school districts.
A charter, the right
to operate a public school
is typically awarded
for a limited period
subject to renewal, conditional
on good school performance.
Charter schools are free
to structure their curriculum
and school environment.
The most controversial difference
between charters
and traditional public schools
is the fact that the teachers
and staff who work at charter schools
rarely belonged to labor unions.
By contrast, most
big city public school teachers
work under union contracts.
The 2010 documentary film
"Waiting for Superman"
feature schools belonging to
the Knowledge is Power Program,
KIPP.
KIPP schools are emblematic
of the high expectations,
sometimes also called "no excuses"
approach to public education.
The "no excuses" model features
a long school day
and extended school year,
selective teacher hiring
and focuses on traditional
reading and math skills.
The American debate
over education reform
often focuses on the achievement gap --
that's shorthand
for large test score differences
by race and ethnicity.
Because of its focus
on minority students,
KIPP is often central
in this debate
with supporters pointing to the fact
that non-White KIPP students
have markedly higher test scores
than non-White students
from nearby schools.
KIPP skeptics on the other hand,
argue that KIPP's apparent success,
reflects the fact that KIPP
attracts families
whose children would be
more likely to succeed, anyway.
Who's right?
As you've probably guessed by now,
a randomized trial
might prove decisive
in the debate
over schools like KIPP.
Like Nobel Prize is, though,
seats at KIPP
are not randomly assigned.
Well, at least, not entirely.
In fact,
Massachusetts charter schools
with more applicants than seats
must offer their seats by lottery.
Sounds like a good natural experiment.
A little over a decade ago,
my collaborators and I
collected data
on KIPP admissions lotteries
laying the foundation
for two pioneering charter studies,
the first to use lotteries to study KIPP.
Our KIPP analysis is a classic IV story
because many students
offered a seat in the KIPP lottery
failed to show up in the fall,
while a few not offered a seat,
nevertheless, find their way in.
This graphic shows KIPP
middle school applicants math scores
one year after applying to KIPP.
The entries above the line
show that Kip applicants
who were offered a seat
have standardized
math scores close to zero --
that is near the state average.
As before, we're working
with standardized score data
that has a mean of 0 and
a standard deviation of 1.
because KIPP applicants
start with 4th grade scores
that are roughly .3 standard
deviations below the state mean,
achievement at the level
of the state average is impressive.
By contrast, the average math score
among those not offered a seat
is about minus .36 sigma,
that is, .36 standard deviations
below the state mean --
a result typical for urban
students in Massachusetts.
Since lottery offers
are randomly assigned,
we could say with confidence
that the offer of a seated KIPP
boost math scores by
an average of .36 sigma,
a large effect that's
also statistically precise.
We can be confident
this is.n't a chance finding
What does an offer effect
.36 sigma
tell us about the effects
of actually going to KIPP?
IV methods convert KIPP offer
effects into KIPP attendance effects.
I'll use this brief clip from my Marginal
Revolution University short course
to quickly review the key
assumptions behind this conversion.
- [Narrator] IV describes
a chain reaction.
Why do offers affect achievement?
Probably because they affect
charter attendance
and charter attendance
improves math scores.
The first link in the chain
called the "First Stage"
is the effect of the lottery
on charter attendance.
The "Second Stage" is the link
between attending a charter
and an outcome variable --
in this case, math scores.
The instrumental variable
or "Instrument," for short,
is the variable that initiates
the chain reaction.
The effect of the instrument
on the outcome
is called the "Reduced Form."
This chain reaction can be
represented mathematically.
We multiply the first stage --
the effect of winning on attendance,
by the second stage --
the effect of attendance on scores,
and we get the reduced form --
the effect of winning
the lottery on scores.
The Reduced Form and First Stage
are observable and easy to compute.
However, the effect
of attendance on achievement
is not directly observed.
This is the causal effect.
we're trying to determine
Given some important assumptions
will discuss shortly,
we can find the effect
of KIPP attendance
by dividing the reduced form
by the first stage.
- [Joshua] IV eliminates selection bias,
but like all of our tools,
the solution builds
on a set of assumptions
not to be taken for granted
first. There must be a
substantial first stage.
That is the instrumental variable,
winning or losing the
lottery must really change.
The variable, whose effect we're
interested in here, KIPP attendance.
In this case. The first stage is
not really in doubt, winning the lottery,
make skip attendance, much more likely,
not all IV, stories are like that.
S the instrument must
be as good as randomly.
Signed meaning lottery winners and
losers have similar characteristics.
This is the independence Assumption.
Of course,
KIPP lottery wins really
are randomly assigned
still, we should check for balance
and confirm that winners and losers
have similar family, backgrounds,
similar, aptitudes, and so on,
in essence, we're checking to
ensure KIPP lotteries are fair
with no group of applicants,
suspiciously, likely to win.
Finally, we require the
instrument change outcomes soul.
Through the variable of interest.
In this case, attending camp,
this assumption is called
the exclusion restriction.
The causal effect of KIPP attendance can
therefore be written as the ratio of
the effect of offers on
scores in the numerator,
over the effect of offers on KIPP
and Roman in the denominator.
The numerator in this IV formula,
that is the direct effect of the
instrument on outcomes. Has a special name.
This is called the reduced form.
The denominator is the first step.
Stage
the exclusion restriction is often the
trickiest or most controversial part of
an IV story. Here, the exclusion
restriction amounts to the claim
that the .36 score, differential
between lottery. Winners and losers
is entirely attributable to the .74,
win-loss difference in attendance rates.
Plugging in the numbers.
The effect of KIPP attendance works
out to be point four eight Sigma,
almost half a standard deviation gain in
math scores. That's a remarkably large.
Perfect, who exactly benefits.
So spectacularly from KIPP,
does everyone who applies to
KIPP, see such large gains,
late answers. This question.
The late interpretation of the
KIPP IV, empirical strategy,
is illuminated by the
biblical story of Passover,
which explains that there
are four types of children
each with characteristic behaviors
to keep track of these
children and their behavior.
I'll give them a literate
of names. Applicants like
Alvaro are dying to go to KIPP.
If Alvaro loses, the KIPP lottery.
His mother finds a way to
enroll him in KIPP. Anyway,
perhaps by reapplying applicants like
Camilla are happy to go to Camp if they win
a seat in the lottery, but stoically
accept the verdict, if they lose finally,
applicants, like normando worried about
long days and lots of homework at KIPP
normando doesn't really want to go and
refuses to go to KIPP when told that he won
the lottery.
That was called a never taker because
win or lose. He doesn't go to KIPP
at the other end of KIPP commitment.
Alvaro is called an always taker.
He'll happily take a seat. Went offered
while his mother simply finds a way
to make it happen for him.
Even when he loses for Alvaro and
normando. Both choice of school.
KIPP traditional is
unaffected by the lottery.
Camilla is the type of
applicant who gives IV its power
the instrument determines
her treatment status.
I IV strategies depend on applicants,
like, Camilla who are called compliers.
This term comes from the world of
randomized, Trials, introduced earlier
as we've already discussed,
many randomized, trials, randomize
only the opportunity to be treated
while the decision to comply with the
treatment remains voluntary and non-random.
RCT. Compliers are those who take treatment
when the offer of treatment is made?
But not otherwise with
Lottery instruments.
Late is the effect of KIPP attendance on
Camilla and other compliers like her.
Who enroll at KIPP take treatment when
offered treatment through the lottery.
But not otherwise
IV methods are uninformed of for always
takers like Alvaro and never takers
like normando because the instrument
is unrelated to their treatment status.
Hey, didn't I say there
are four types of children.
A fourth type of child in IV Theory
behaves perversely every family has one.
These defiant children and Roland
KIPP only when they lose the lottery.
Actually, the late theorem requires
us to assume there are few defiers,
that seems like a reasonable assumption
for Charter Lottery instruments.
If not in life.
The late theorem is sometimes
seen as limiting the relevance
of econometric estimates
because it focuses attention
on groups of compliers
yet. The population of compliers is a
group. We'd very much like to learn about
in the KIPP example, compliers
our children, likely to be,
Drawn into KIPP where the school to expand
and offer additional seats in a lottery.
How relevant is this a few years ago,
Massachusetts indeed allowed
thriving charter schools to expand
a recent study by some of my lab mates
shows that late estimates like
the one we just computed for KIPP
predict learning gains at the
schools created by Charter expansion.
Late isn't just a theorem.
It's a
framework.
The late framework can be used to
estimate the entire distribution
of potential outcomes for compliers
as if we really did have a
randomized trial for this group.
Although the theory behind this
fact is necessarily technical.
The value of the framework is easily
appreciated in practice by way of illustration.
Recall that the KIPP study
is motivated in part by
differences in test scores by race.
Let's look at the distribution of
4th grade scores separately by.
Race for applicants to Boston
Charter, Middle Schools,
the two sides of this figure
show distributions for treated
and untreated compliers treated.
Compliers are compliers offered.
A charter seat in a lottery. While
untreated compliers are not offered a seat
because these are 4th grade scores
while middle school begins in 5th
or 6th grade. The two sides
of the figure are similar.
Both sides show score distributions for
black applicants shifted to the left
of the corresponding.
Gorgeous tribulations
for Whites by 8th grade.
Treated compliers have completed
Middle School at a Boston Charter.
Well, I'm treated compliers have
remained in traditional Public School.
Remarkably.
This next graphic shows that the
eighth grade score distributions
of black-and-white treated.
Compliers are indistinguishable.
Boston charter middle schools,
close the achievement Gap,
but for the untreated black and white
score distributions remained distinct
with black students.
Hind white students as
they were in 4th grade,
Boston Charters,
close the achievement Gap because
those who enter Charter Schools,
the farthest behind tend to gain
the most from Charter. Enrollment
I elaborate on this point in
the print version of this talk.
Remember the puzzle of -
Chicago exam School effects.
I'll finish the scientific part of my
talk by using IV and RD to explain this.
This surprising finding
the resolution of this puzzle starts
with the fact that economic reasoning
is about Alternatives.
So what's the alternative
to an exam school education
for most applicants to
Chicago exam schools.
The leading non exam. Alternative
is a traditional public school,
but many of Chicago's rejected exam school
applicants enroll in a charter school
exam school offers. Therefore reduce the
likelihood of Charter School attendance.
Specifically exam schools divert
applicants away from high schools
in the noble network of charter schools.
Noble
with pedagogy much, like KIPP is one of
Chicago's most visible Charter providers.
Also like KIPP convincing evidence on
Noble Effectiveness comes from admissions,
lotteries.
The x-axis in this graphic shows Lottery
offer effects on years enrolled at Noble.
This is the noble first stage.
For an IV setup that uses a dummy.
Indicating Noble Lottery offers as
an instrument for Noble enrollment.
Now, this graphic has a
feature that distinguishes.
It from the simpler KIPP analysis,
the plot shows. First stage effects.
For two groups.
One for Noble applicants who live
in Chicago's lowest income
neighborhoods Tier 1 and 1/4 Noble,
applicants who live in higher, income
areas, tier 3, remember the IV chain.
Each point in this graphic has coordinates
given by first stage reduced form
and therefore implies an IV estimate.
The effect of noble enrollment on ACT scores
is the ratio of reduced form coordinate
to First Stage coordinate.
The graphic shows to such ratios.
The relevant results for Tier 1 are .35.
While for tier 3. We have .33 not bad
for Noble applicants from both tears.
These
stage in reduced form estimates
imply a yearly Noble and Roman effective
about a third of a standard deviation gain,
in act math scores.
Notice. There's also a line connecting
the two Ivy estimates in the figure
because this line passes
through the origin,
it's slope rise. Over run is about equal
to the to IV estimates. In this case.
The slope is about Point 3 for
the fact that the line passes
through 0 0 is significant for
Reason by this fact, we've
substantiated the exclusion restriction,
specifically, the exclusion restriction,
says that given a group for which Noble
offers are unrelated to Noble enrollment.
We should expect to see Zero reduced form
effect of these offers made to applicants
in that group.
How consistent is the evidence for a
noble cause learning gain on the order
of point, three, four Sigma per year
in this next graphic? We've added
L've more points to the original to
the red points here, show, first stage
in reduced form, Noble offer effects
for 12, additional groups to more tears,
and 12 groups, defined by demographic,
characteristics related to race sex,
family, income, and Baseline scores.
Although not a perfect fit
these points cluster around a line
with slope Point Three, Six Sigma
much like the line. We saw earlier
for applicants from tiers 1 and 3.
You're likely now.
Ring. What the noble IV estimates
in this figure have to do
with exam School enrollment.
Here's the answer,
the Blue Line in this new graphic
shows. As we should expect
that exam. School exposure jumps up
for applicants who clear their
qualifying cutoff. At the same time.
The Red Line shows that Noble School
enrollment clearly Falls at the same point.
This is the diversion effect of exam
school offers on Noble enrollment.
Many kids offered an exam School
seat, prefer that exam School seat,
to enrollment at Noble IV, affords us,
the opportunity to go out on a limb
with strong claims about the
mechanism behind the causal effect.
Here's a strong causal, claim Regarding why
Chicago exam schools reduce achievement.
The primary force driving
reduced form exam School.
Qualification effects on ACT scores.
I claim is the effect of exam
school offers on Noble and
Vomit in support of this claim,
consider the points plotted here in blue
all well to the left of 0 on the x-axis.
These points are negative because they Mark
the effect of exam School qualification
on Noble School, enrollment for
particular, groups of applicants.
Now, we've already seen
that Noble applicants offered a noble seen
realize large act math, gains as a result.
Now consider exam school offers as
An instrument for Noble enrollment,
as always, Ivy is a chain reaction.
If exam School qualification reduces
time at Noble, by Point 37 years,
and each year of noble, enrollment
boosts act math scores by
about Point Three, Six Sigma.
We should expect reduced form
effects of exam School. Qualification
to reduce ACT scores by the
product of these two numbers.
That is by about Point 1, 3 Sigma
the reduced form qualification
effects at the left of the
Here are broadly consistent with this.
They cluster closer to -
.16. Then 2 minus Point 1 3,
but that difference is well
within the sampling variance
of the underlying estimates.
The causal story told here,
postulates diversion
away from Charter Schools
as the mechanism by which exam
school offers effect achievement.
In other words, it's Noble enrollment,
that's presumed to satisfy
an exclusion restriction.
When we use exam school offers as an
Variable importantly,
as we saw before the line in this final
graphic, with two sets of 14 points,
runs through the origin.
This fact supports our
new exclusion restriction
for any applicant group
for which exam school offers have little
or no effect on Noble School enrollment.
We should also see ACT scores unchanged
at the same time because the blue and
red dots cluster around the same line,
the IV estimates of noble School,
enrollment effects generated.
I both Noble and exam school
offers are about the same.
I hope this empirical story convinces,
you of the power of IV and RD
to generate new causal,
knowledge for decades.
I've been lucky to work on many
equally engaging empirical problems.
I computed the draft
lottery IV estimates in
my Princeton PhD thesis on a
big hairy Mainframe monster.
Using nine track tapes, and Lease
space on a communal. Hard drive,
Princeton graduate students learn
to mount and manipulate tape.
Reels the size of a cheesecake.
Thankfully empirical work today
is a little less labor-intensive.
What else is improved in
the modern empirical era
in a 2010 article, Steve
Pischke, and I coined the phrase.
He Revolution
by this, we mean economic shift towards
transparent empirical strategies.
Applied to concrete, causal questions,
like the questions. David
Carter studied. So convincingly.
The econometrics of my schooldays
focused more on models than on questions.
The modeling concerns of
that era have mostly faded
but econometricians have since
found much to contribute.
I'll save my personal lists of greatest
hits and exciting new artists for the
The print version of this lecture.
I'll wrap up here by saying that I'm
proud to be part of the Contemporary.
Empirical economics, Enterprise
and I'm gratified beyond words
to have been recognized for contributing
to it back at Princeton in the late 80s.
My graduate school classmates and I
chuckled leading Ed lemurs. Lament,
that no Economist takes another
economists. Empirical work. Seriously.
This is no longer, true, empirical
work today aspires to tell convincing.
Also stories, not that every
effort succeeds far from it.
But as any economics job
market candidate will tell you,
empirical work carefully executed and
clearly explained is taken seriously. Indeed.
That is a measure of
our Enterprises success.
If you'd like to learn more from Josh,
check out his free course mastering
econometrics if you'd like to
Spore, Josh's research,
check out the links in the description
or you can click to watch more
of Josh's videos.