Joshua Angrist Nobel Prize Lecture 2021

♪ [music] ♪

- [Joshua] As I stilled my trembling
iPhone early on October 11th,

my thoughts went to the question
of whether Nobel-level recognition

might change life
for the Angrist family.

Ours is a close-knit family.

We lack for nothing.

So I worried
that stressful Nobel celebrity

might not be a plus.

But with the first cup of coffee,

I began to relax.

It occurred to me

that the matter
of how public recognition

affects a scholars life

is, after all,
a simple causal question.

The Nobel intervention

is substantial, sudden,
and well-measured.

Outcomes like health and wealth
are easy to record.

Having just been recognized
with my co-laureates,

Guido Imbens and David Card,

for answering causal questions
using observational data,

my thoughts moved
from personal upheaval

to the more familiar demands

of identification and estimation
of causal effects.

I was able to soothe
my worried mind

by imagining a study
of the Nobel Prize treatment effect.

How would such a study
be organized?

in a 1999 essay published in the
"Handbook of Labor Economics,"

Alan Krueger and I embraced
the phrase "empirical strategy."

The handbook volume
in question was edited

by two of my Princeton
PhD thesis advisors,

Orley Ashenfelter and David Card,

among the most successful
and prolific graduate advisors

economics has known

An empirical strategy
is a research plan

that encompasses data collection,

identification,
and econometric estimation.

Identification is the applied
econometricians term

for research design --

a randomized, clinical trial.

And RCT is the simplest
and most powerful research design.

In RCTs, causal effects
are identified

by the random assignment
of treatment.

Random assignment ensures

that treatment and control groups

are comparable
in the absence of treatment.

So differences
between them afterwards

reflect only the treatment effect.

Nobel prizes are probably
not randomly assigned.

This challenge notwithstanding,

a compelling empirical strategy
for the Nobel treatment effect

comes to mind,

at least as a flight of empirical fancy.

Imagine a pool of prize-eligible
Nobel applicants,

the group under consideration
for the prize.

Applicants need not apply themselves.

They are, I presume, nominated
by their peer scholars.

My fanciful Nobel impact study
looks only at Nobel applicants

since these are all elite scholars.

But that is only the first step.

Credible applicants, I imagine,
are evaluated by judges

using criteria, like publications,
citations, nominating statements.

I imagine this material is reviewed
and assigned a numerical score,

using some kind of scoring rubric.

Top scorers up to three per field,
in any single year,

win a prize.

Having identified applicants

that collected data
on their scores,

the next step
in my Nobel impact study

is to record the relevant cutoffs.

The Nobel cutoff
is the lowest score

among those awarded a prize,

Many Nobel hopefuls
just missed the cutoff.

Looking only at near-misses,
along with the winners,

differences in scores

between those above
and below the cutoff

begin to look serendipitous,
almost randomly assigned.

After all,

near-Nobels are among
the most eminent of scholars, too.

With one more
high-impact publication,

a little more support
from nominators.

they would have been awarded
Nobel gold --

some of them, someday,
surely will be.

The empirical strategy
sketched here is called

a Regression Discontinuity Design,

RD.

RD exploits the jumps
in human affairs,

induced by rules, regulations,

and the need to classify people
for various assignment purposes.

When treatment
or intervention is determined

by whether a tiebreaking variable
crosses a threshold,

those just below the threshold
become a natural control group

for those who clear it.

RD does not require
that the variable,

whose causes we seek,

switch fully on or fully off
at the cutoff.

We require only

that the average value
of this variable

jump at the cutoff

RD can allow, for example,
for the fact

that this year's near-Nobel
might be next year's winner.

Allowing for this leads
to the use of jumps

and the rate at which
treatment is assigned

to construct instrumental variables,

IV,

estimates of the effect
of treatment received.

This sort of RD
is said to be fuzzy,

But as Steve Pischke and I
wrote in our first book:

"fuzzy RD is IV."

(cheering)

The first RD study I contributed to

was written with my frequent
collaborator, Victor Lavy.

This study is motivated

by the high costs
and uncertain returns

to smaller elementary school classes.

We exploited a rule
used by Israeli elementary schools

to determine class size.

This rule is used to estimate
class size effects,

as if in a class size RCT.

In the 1990s,
Israeli classes were large.

Students enrolled
in a grade cohort of 40.

We're likely to be
seated in a class of 40.

That's the relevant cutoff.

Add another child
to the cohort, making 41,

and the cohort
was likely to be split

into two much smaller classes.

This leads to the Maimonide's
rule research design,

so named because
the 12th-century Rambam

proposed a maximum
class size of 40.

This figure plots is rarely
fourth grade class sizes

as a function
of fourth grade enrollment,

overlaid with the theoretical
class size rule,

Maimonides rule.

The fit isn't perfect.

That's a feature that makes
this application of RD fuzzy.

But the gist of the thing
is a marked class size drop,

at each integer multiple of 40,
the relevant cutoff,

just as predicted by the rule.

As it turns out,
these drops in class size

are reflected in jumps

in 4th and 5th grade math scores.

♪ [music] ♪

Would a comparison
of Nobel laureates to near laureates

really be a good natural experiment?

The logic behind this sort of claim

seems more compelling
for comparisons of schools

with 40 and 41 4th graders

than for comparisons
of laureates and near laureates.

Yet, both scenarios exploit
a feature of the physical world.

Provided the tiebreaking variable

known to RD mavens
as the running variable,

has a continuous distribution,

the probability of crossing
the cutoff approaches one-half

when examined in a narrow window

around the cutoff.

In RD empirical work,

the window around such cutoffs
is known as a bandwidth.

Importantly, this limiting
probability is 0.5 for everybody

regardless of how qualified they look

going into the Nobel competition.

This remarkable fact can be seen
in data on applicants

to one of New York's
highly coveted screen schools.

By way of background,

roughly 40% of New York City's
middle and high schools

select their applicants
on the basis of test scores, grades,

and other exacting criteria.

In other words, the admissions
regime for screen schools

is a lot like the scheme
I've imagined

for the Nobel Prize.

Screen schools
are but one of a number

of highly selective systems
within a system

in large US school districts.

Boston, Chicago, San Francisco,
and Washington, D.C

all feature highly selective
institutions,

often known as exam schools.

Exam schools operate

as part of larger
public school systems

that enroll students
without screening.

Motivated by
the enduring controversy

over the equity
of screened admissions,

my blueprint
labs collaborators and I

have examined the causal effects
of exam school attendance

in Boston, Chicago, and New York.

This figure shows the probability
of being offered a seat

at New York's storied
Townsend Harris High School,

ranked 12th, nationwide.

Bar height in the figure
marks the qualification rate --

that is, the likelihood of earning
a Townsend Harris admission score,

above that of the lowest
scoring applicant

offered a seat.

Importantly, the bars show
qualification rates conditional

on a measure of pre-application
baseline achievement.

In particular, the bars mark
qualification rates conditional

on whether an applicant

has upper quartile or lower quartile
6th grade math scores.

Townsend Harris applicants
with high baseline scores

are much more likely to qualify

than applicants
with low baseline scores.

This isn't surprising.

But in a shrinking
symmetric bandwidth

around the schools cutoff,

qualification rates
in the two groups converge

Qualification rate in the last
and smallest groups

are both remarkably close
to one-half

This is what we'd expect to see

where Townsend Harris
to admit students,

by tossing a coin,

rather than by selecting
only those who scored highly

on the school's entrance exam.

Even when admissions
operates by screening,

the data can be arranged
so as to mimic an RCT.

A few of the questions I've studied
are more controversial

than the question of access
to public exam schools,

like the Boston Latin School,

Chicago's Payton and Northside
selective enrollment high schools,

and New York's legendary
Brooklyn Tech,

Bronx Science,

and Stuyvesant
specialized high schools,

which have graduated
14 Nobel laureates between them.

Townsend Harris, the school
we started with today,

graduated three Nobels,
including economist Ken Arrow.

Exam school proponents

see the opportunities
these schools provide

as democratizing public education.

"Wealthy families," they argue,

can access exam school curricula
in the private sector.

Shouldn't ambitious
low-income students

be afforded the same chance
at elite education?

Critics of selective
enrollment schools

argue that rather than
expanding equity,

exam schools are inherently biased

against the Black
and Hispanic students

that make up the bulk
of America's urban districts,

New York's super selective
Stuyvesant, for example,

enrolled only
seven Black students in 2019,

out of an incoming class of 895.

But are exam school seats
really worth fighting for?

My collaborators and I
have repeatedly used

RD empirical strategies
to study the causal effects

of attendance at exam schools

like Townsend Harris
and Boston Latin.

Our first exam school study,

which looks at schools
in Boston and New York

encapsulates
these findings in its title:

"The Elite Illusion."

"The Elite Illusion"
refers to the fact

that while exam school students
undoubtedly have high test scores

and other good outcomes,

this is sot a causal effect
of exam School attendance.

Our estimates consistently suggest

that the causal effects
of exam school attendance

on their students learning
and college-going are 0.

Maybe even negative.

The good performance
of exam school students

reflect selection bias --

that is, the process by which
these students are chosen,

rather than causal effects.

Data from Chicago's large exam
school sector

illustrate the elite illusion.

This figure plots
peer mean achievement --

that is, the 6th grade test scores
of my 9th grade classmates

against the admissions tiebreaker

for a subset of applicants

to any one of Chicago's
nine exam schools.

Applicants to these schools
rank up to 6,

while the exam schools
prioritize their applicants

using a common composite index,

formed from an admissions test,

GPAs, and grade 7
standardized scores.

This composite tiebreaker
is the running variable

for an RD design
that reveals what happens

when any applicant is offered

an exam school seat.

In Chicago's exam school match,

which is actually an application

of the celebrated Gale and Shapley
matching algorithm

exam school applicants are sure
to be offered a seat somewhere

when they clear the lowest
in their set of cutoffs

among the schools they rank.

We call this lowest cutoff
the "qualifying cutoff."

The figure shows a sharp jump
in peer mean achievement

for Chicago exam school applicants

who clear their qualifying cutoff.

This jump reflects the fact

that most applicants
offered an exam school seat take it,

and applicants who enroll

at one of Chicago's selective
enrollment high schools

are sure to be seated
in a 9th grade classroom

filled with academically
precocious peers,

because only the relatively
precocious make it in.

The increase in peer achievement
across the qualifying cutoff

amounts to almost
half a standard deviation --

a very large effect.

And yet, precocious peers,
notwithstanding,

the offer of an exam school seat
does not appear to increase learning.

Let's plot applicants ACT scores
against their tiebreaker values.

This plot shows that exam
school applicants

who clear their qualifying cutoff

perform sharply worse on the ACT.

What explains this?

It takes a tale of IV and RD
to untangle the forces

behind this intriguing
and unexpected negative effect.

But first, some IV theory,

Guido Imbens and I
developed theoretical tools

that enhance economists'
understanding of empirical strategies

involving IV and RD.

The prize we share
is in recognition of this work.

Guido and I overlapped for
only one year at Harvard,

where we had both taken
our first jobs post PhD.

I welcomed Guido
to Cambridge, Massachusetts

with a pair of interesting
instrumental variables.

I had used
the draft lottery instrument

in my PhD thesis

to estimate the long-run
economic consequences

of serving in the Armed Forces

for soldiers who were drafted.

The draft lottery instrument
relies on the fact

that lottery numbers
randomly assigned to birthdays

determined Vietnam-era
conscription risk.

Yet, even then, most soldiers
were volunteers,

as they are today,

The quarter birth instrument

is used in my 1991 paper
with Alan Krueger

to estimate the economic
returns to schooling.

This instrument uses the fact

that men who are born earlier
in the year

are allowed to drop out
of high school

on their 16th birthday

with less schooling completed
than those born later.

Guido and I soon
began asking each other,

"What really do we learn
from the draft eligibility

and quarter of birth
natural experiments?"

An early result in our quest
for a new understanding of IV

was a solution to the problem
of selection bias

in an RCT with partial compliance.

Even in a randomized clinical trial,

some of the people assigned
to treatment may opt out.

This fact has long vexed trialists

because decisions to opt out
are not made by random assignment.

Our first manuscript together

shows that in a randomized trial
with partial compliance,

you can use IV

to estimate the effect
of treatment on the treated,

even when some offered
treatment decline it.

This works in spite of the fact

that those who comply
with treatment

may be a very select group.

Unfortunately, for us,
we were late to the party.

Not long after releasing
our first working paper,

we learned of a concise
contribution from Howard Bloom

that includes this theoretical result.

Remarkably, Bloom had derived
this from first principles

without making a connection to IV.

So Guido and I went back
to the drawing board.

And a few months later, we had LATE --

a theorem showing how to estimate

the local average treatment effect.

The LATE theorem generalizes
the Bloom theorem

and establishes the connection
between compliance and IV.

Maintaining the clinical trials analogy,

let "Zi" indicate whether subject "i"
is offered treatment.

This is randomly assigned.

Also, let "D1i" indicate
subject "i's" treatment status

when assigned to treatment,

and let the "0i" indicate
subject "i's" treatment status

when assigned to control.

I'll use this formal notation

to give a clear statement
of the late result,

and then follow up with examples.

A key piece of the late framework

pioneered by statistician
Don Rubin,

is the pair of potential outcomes.

As is customary,

I denote potential outcomes
for subject "i"

in the treated and untreated states

by "Y1i" and "Y0i" respectively,

The observed outcome
is "Y1i" for the treated

and "Y0i" for those not treated.

"Y1i" minus "Y0i"

is the causal effect
of treatment on individual "i",

but we can never see.

We try, therefore, to estimate
some kind of average causal effect.

The late framework allows us
to do that in an RCT

where some controls are treated.

The theorem says

that the average causal
effect on people,

whose treatment status
can be changed

by the offer of treatment

is the ratio of ITT
to the treatment control difference

in compliance rates.

A mathematical statement of
this result appears here,

where Greek letter Delta
symbolizes the ITT effect

and Greek symbols pi1 and pi0

are compliance rates
in the group assigned to treatment

and the group assigned
to control, respectively,

The print version of this lecture

delves deeper
into LATE intellectual history,

highlighting key contributions
made with Rubin.

For now, though, I'd like to make
the late theorem concrete for you

by sharing one of
my favorite applications of it.

I'll explain the late framework

through a research question
that has fascinated me

for almost two decades.

What is the causal effect

of charter school attendance
on learning?

Charter schools are public schools

that operate independently

of traditional American
public school districts.

A charter, the right
to operate a public school

is typically awarded
for a limited period

subject to renewal, conditional
on good school performance.

Charter schools are free

to structure their curriculum
and school environment.

The most controversial difference

between charters
and traditional public schools

is the fact that the teachers
and staff who work at charter schools

rarely belonged to labor unions.

By contrast, most
big city public school teachers

work under union contracts.

The 2010 documentary film
"Waiting for Superman"

feature schools belonging to
the Knowledge is Power Program,

KIPP.

KIPP schools are emblematic
of the high expectations,

sometimes also called "no excuses"
approach to public education.

The "no excuses" model features

a long school day
and extended school year,

selective teacher hiring

and focuses on traditional
reading and math skills.

The American debate
over education reform

often focuses on the achievement gap --

that's shorthand
for large test score differences

by race and ethnicity.

Because of its focus
on minority students,

KIPP is often central
in this debate

with supporters pointing to the fact

that non-White KIPP students
have markedly higher test scores

than non-White students
from nearby schools.

KIPP skeptics on the other hand,

argue that KIPP's apparent success,

reflects the fact that KIPP
attracts families

whose children would be
more likely to succeed, anyway.

Who's right?

As you've probably guessed by now,

a randomized trial
might prove decisive

in the debate
over schools like KIPP.

Like Nobel Prize is, though,

seats at KIPP
are not randomly assigned.

Well, at least, not entirely.

In fact,

Massachusetts charter schools
with more applicants than seats

must offer their seats by lottery.

Sounds like a good natural experiment.

A little over a decade ago,

my collaborators and I
collected data

on KIPP admissions lotteries

laying the foundation
for two pioneering charter studies,

the first to use lotteries to study KIPP.

Our KIPP analysis is a classic IV story

because many students
offered a seat in the KIPP lottery

failed to show up in the fall,

while a few not offered a seat,
nevertheless, find their way in.

This graphic shows KIPP
middle school applicants math scores

one year after applying to KIPP.

The entries above the line

show that Kip applicants
who were offered a seat

have standardized
math scores close to zero --

that is near the state average.

As before, we're working
with standardized score data

that has a mean of 0 and
a standard deviation of 1.

because KIPP applicants
start with 4th grade scores

that are roughly .3 standard
deviations below the state mean,

achievement at the level
of the state average is impressive.

By contrast, the average math score
among those not offered a seat

is about minus .36 sigma,

that is, .36 standard deviations
below the state mean --

a result typical for urban
students in Massachusetts.

Since lottery offers
are randomly assigned,

we could say with confidence
that the offer of a seated KIPP

boost math scores by

an average of .36 sigma,

a large effect that's
also statistically precise.

We can be confident
this is.n't a chance finding

What does an offer effect
.36 sigma

tell us about the effects
of actually going to KIPP?

IV methods convert KIPP offer
effects into KIPP attendance effects.

I'll use this brief clip from my Marginal
Revolution University short course

to quickly review the key
assumptions behind this conversion.

- [Narrator] IV describes
a chain reaction.

Why do offers affect achievement?

Probably because they affect
charter attendance

and charter attendance
improves math scores.

The first link in the chain
called the "First Stage"

is the effect of the lottery
on charter attendance.

The "Second Stage" is the link

between attending a charter
and an outcome variable --

in this case, math scores.

The instrumental variable
or "Instrument," for short,

is the variable that initiates
the chain reaction.

The effect of the instrument
on the outcome

is called the "Reduced Form."

This chain reaction can be
represented mathematically.

We multiply the first stage --
the effect of winning on attendance,

by the second stage --
the effect of attendance on scores,

and we get the reduced form --

the effect of winning
the lottery on scores.

The Reduced Form and First Stage
are observable and easy to compute.

However, the effect
of attendance on achievement

is not directly observed.

This is the causal effect.
we're trying to determine

Given some important assumptions
will discuss shortly,

we can find the effect
of KIPP attendance

by dividing the reduced form
by the first stage.

- [Joshua] IV eliminates selection bias,

but like all of our tools,

the solution builds
on a set of assumptions

not to be taken for granted

first. There must be a
substantial first stage.

That is the instrumental variable,

winning or losing the
lottery must really change.

The variable, whose effect we're
interested in here, KIPP attendance.

In this case. The first stage is

not really in doubt, winning the lottery,
make skip attendance, much more likely,

not all IV, stories are like that.

S the instrument must
be as good as randomly.

Signed meaning lottery winners and
losers have similar characteristics.

This is the independence Assumption.

Of course,

KIPP lottery wins really
are randomly assigned

still, we should check for balance
and confirm that winners and losers

have similar family, backgrounds,
similar, aptitudes, and so on,

in essence, we're checking to
ensure KIPP lotteries are fair

with no group of applicants,
suspiciously, likely to win.

Finally, we require the
instrument change outcomes soul.

Through the variable of interest.
In this case, attending camp,

this assumption is called
the exclusion restriction.

The causal effect of KIPP attendance can
therefore be written as the ratio of

the effect of offers on
scores in the numerator,

over the effect of offers on KIPP
and Roman in the denominator.

The numerator in this IV formula,

that is the direct effect of the
instrument on outcomes. Has a special name.

This is called the reduced form.
The denominator is the first step.

Stage

the exclusion restriction is often the
trickiest or most controversial part of

an IV story. Here, the exclusion
restriction amounts to the claim

that the .36 score, differential
between lottery. Winners and losers

is entirely attributable to the .74,
win-loss difference in attendance rates.

Plugging in the numbers.

The effect of KIPP attendance works
out to be point four eight Sigma,

almost half a standard deviation gain in
math scores. That's a remarkably large.

Perfect, who exactly benefits.
So spectacularly from KIPP,

does everyone who applies to
KIPP, see such large gains,

late answers. This question.

The late interpretation of the
KIPP IV, empirical strategy,

is illuminated by the
biblical story of Passover,

which explains that there
are four types of children

each with characteristic behaviors

to keep track of these
children and their behavior.

I'll give them a literate
of names. Applicants like

Alvaro are dying to go to KIPP.
If Alvaro loses, the KIPP lottery.

His mother finds a way to
enroll him in KIPP. Anyway,

perhaps by reapplying applicants like
Camilla are happy to go to Camp if they win

a seat in the lottery, but stoically
accept the verdict, if they lose finally,

applicants, like normando worried about
long days and lots of homework at KIPP

normando doesn't really want to go and
refuses to go to KIPP when told that he won

the lottery.

That was called a never taker because
win or lose. He doesn't go to KIPP

at the other end of KIPP commitment.
Alvaro is called an always taker.

He'll happily take a seat. Went offered
while his mother simply finds a way

to make it happen for him.

Even when he loses for Alvaro and
normando. Both choice of school.

KIPP traditional is
unaffected by the lottery.

Camilla is the type of
applicant who gives IV its power

the instrument determines
her treatment status.

I IV strategies depend on applicants,
like, Camilla who are called compliers.

This term comes from the world of
randomized, Trials, introduced earlier

as we've already discussed,

many randomized, trials, randomize
only the opportunity to be treated

while the decision to comply with the
treatment remains voluntary and non-random.

RCT. Compliers are those who take treatment
when the offer of treatment is made?

But not otherwise with
Lottery instruments.

Late is the effect of KIPP attendance on
Camilla and other compliers like her.

Who enroll at KIPP take treatment when
offered treatment through the lottery.

But not otherwise

IV methods are uninformed of for always
takers like Alvaro and never takers

like normando because the instrument
is unrelated to their treatment status.

Hey, didn't I say there
are four types of children.

A fourth type of child in IV Theory
behaves perversely every family has one.

These defiant children and Roland
KIPP only when they lose the lottery.

Actually, the late theorem requires
us to assume there are few defiers,

that seems like a reasonable assumption
for Charter Lottery instruments.

If not in life.

The late theorem is sometimes
seen as limiting the relevance

of econometric estimates

because it focuses attention
on groups of compliers

yet. The population of compliers is a
group. We'd very much like to learn about

in the KIPP example, compliers
our children, likely to be,

Drawn into KIPP where the school to expand
and offer additional seats in a lottery.

How relevant is this a few years ago,

Massachusetts indeed allowed
thriving charter schools to expand

a recent study by some of my lab mates

shows that late estimates like
the one we just computed for KIPP

predict learning gains at the
schools created by Charter expansion.

Late isn't just a theorem.

It's a

framework.

The late framework can be used to
estimate the entire distribution

of potential outcomes for compliers

as if we really did have a
randomized trial for this group.

Although the theory behind this
fact is necessarily technical.

The value of the framework is easily
appreciated in practice by way of illustration.

Recall that the KIPP study

is motivated in part by
differences in test scores by race.

Let's look at the distribution of
4th grade scores separately by.

Race for applicants to Boston
Charter, Middle Schools,

the two sides of this figure
show distributions for treated

and untreated compliers treated.
Compliers are compliers offered.

A charter seat in a lottery. While
untreated compliers are not offered a seat

because these are 4th grade scores
while middle school begins in 5th

or 6th grade. The two sides
of the figure are similar.

Both sides show score distributions for
black applicants shifted to the left

of the corresponding.

Gorgeous tribulations
for Whites by 8th grade.

Treated compliers have completed
Middle School at a Boston Charter.

Well, I'm treated compliers have
remained in traditional Public School.

Remarkably.

This next graphic shows that the
eighth grade score distributions

of black-and-white treated.
Compliers are indistinguishable.

Boston charter middle schools,
close the achievement Gap,

but for the untreated black and white
score distributions remained distinct

with black students.

Hind white students as
they were in 4th grade,

Boston Charters,

close the achievement Gap because
those who enter Charter Schools,

the farthest behind tend to gain
the most from Charter. Enrollment

I elaborate on this point in
the print version of this talk.

Remember the puzzle of -
Chicago exam School effects.

I'll finish the scientific part of my
talk by using IV and RD to explain this.

This surprising finding

the resolution of this puzzle starts
with the fact that economic reasoning

is about Alternatives.

So what's the alternative
to an exam school education

for most applicants to
Chicago exam schools.

The leading non exam. Alternative
is a traditional public school,

but many of Chicago's rejected exam school
applicants enroll in a charter school

exam school offers. Therefore reduce the
likelihood of Charter School attendance.

Specifically exam schools divert
applicants away from high schools

in the noble network of charter schools.

Noble

with pedagogy much, like KIPP is one of
Chicago's most visible Charter providers.

Also like KIPP convincing evidence on
Noble Effectiveness comes from admissions,

lotteries.

The x-axis in this graphic shows Lottery
offer effects on years enrolled at Noble.

This is the noble first stage.

For an IV setup that uses a dummy.

Indicating Noble Lottery offers as
an instrument for Noble enrollment.

Now, this graphic has a
feature that distinguishes.

It from the simpler KIPP analysis,
the plot shows. First stage effects.

For two groups.

One for Noble applicants who live

in Chicago's lowest income
neighborhoods Tier 1 and 1/4 Noble,

applicants who live in higher, income
areas, tier 3, remember the IV chain.

Each point in this graphic has coordinates
given by first stage reduced form

and therefore implies an IV estimate.

The effect of noble enrollment on ACT scores
is the ratio of reduced form coordinate

to First Stage coordinate.

The graphic shows to such ratios.

The relevant results for Tier 1 are .35.

While for tier 3. We have .33 not bad
for Noble applicants from both tears.

These

stage in reduced form estimates

imply a yearly Noble and Roman effective
about a third of a standard deviation gain,

in act math scores.

Notice. There's also a line connecting
the two Ivy estimates in the figure

because this line passes
through the origin,

it's slope rise. Over run is about equal
to the to IV estimates. In this case.

The slope is about Point 3 for

the fact that the line passes
through 0 0 is significant for

Reason by this fact, we've
substantiated the exclusion restriction,

specifically, the exclusion restriction,

says that given a group for which Noble
offers are unrelated to Noble enrollment.

We should expect to see Zero reduced form
effect of these offers made to applicants

in that group.

How consistent is the evidence for a
noble cause learning gain on the order

of point, three, four Sigma per year
in this next graphic? We've added

L've more points to the original to

the red points here, show, first stage
in reduced form, Noble offer effects

for 12, additional groups to more tears,

and 12 groups, defined by demographic,
characteristics related to race sex,

family, income, and Baseline scores.

Although not a perfect fit

these points cluster around a line
with slope Point Three, Six Sigma

much like the line. We saw earlier
for applicants from tiers 1 and 3.

You're likely now.

Ring. What the noble IV estimates
in this figure have to do

with exam School enrollment.

Here's the answer,

the Blue Line in this new graphic
shows. As we should expect

that exam. School exposure jumps up

for applicants who clear their
qualifying cutoff. At the same time.

The Red Line shows that Noble School
enrollment clearly Falls at the same point.

This is the diversion effect of exam
school offers on Noble enrollment.

Many kids offered an exam School
seat, prefer that exam School seat,

to enrollment at Noble IV, affords us,
the opportunity to go out on a limb

with strong claims about the
mechanism behind the causal effect.

Here's a strong causal, claim Regarding why
Chicago exam schools reduce achievement.

The primary force driving
reduced form exam School.

Qualification effects on ACT scores.

I claim is the effect of exam
school offers on Noble and

Vomit in support of this claim,

consider the points plotted here in blue
all well to the left of 0 on the x-axis.

These points are negative because they Mark
the effect of exam School qualification

on Noble School, enrollment for
particular, groups of applicants.

Now, we've already seen

that Noble applicants offered a noble seen

realize large act math, gains as a result.

Now consider exam school offers as

An instrument for Noble enrollment,

as always, Ivy is a chain reaction.

If exam School qualification reduces
time at Noble, by Point 37 years,

and each year of noble, enrollment

boosts act math scores by
about Point Three, Six Sigma.

We should expect reduced form
effects of exam School. Qualification

to reduce ACT scores by the
product of these two numbers.

That is by about Point 1, 3 Sigma

the reduced form qualification
effects at the left of the

Here are broadly consistent with this.

They cluster closer to -
.16. Then 2 minus Point 1 3,

but that difference is well
within the sampling variance

of the underlying estimates.

The causal story told here,

postulates diversion
away from Charter Schools

as the mechanism by which exam
school offers effect achievement.

In other words, it's Noble enrollment,

that's presumed to satisfy
an exclusion restriction.

When we use exam school offers as an

Variable importantly,

as we saw before the line in this final
graphic, with two sets of 14 points,

runs through the origin.

This fact supports our
new exclusion restriction

for any applicant group

for which exam school offers have little
or no effect on Noble School enrollment.

We should also see ACT scores unchanged

at the same time because the blue and
red dots cluster around the same line,

the IV estimates of noble School,
enrollment effects generated.

I both Noble and exam school
offers are about the same.

I hope this empirical story convinces,
you of the power of IV and RD

to generate new causal,
knowledge for decades.

I've been lucky to work on many
equally engaging empirical problems.

I computed the draft
lottery IV estimates in

my Princeton PhD thesis on a
big hairy Mainframe monster.

Using nine track tapes, and Lease
space on a communal. Hard drive,

Princeton graduate students learn
to mount and manipulate tape.

Reels the size of a cheesecake.

Thankfully empirical work today
is a little less labor-intensive.

What else is improved in
the modern empirical era

in a 2010 article, Steve
Pischke, and I coined the phrase.

He Revolution

by this, we mean economic shift towards
transparent empirical strategies.

Applied to concrete, causal questions,

like the questions. David
Carter studied. So convincingly.

The econometrics of my schooldays
focused more on models than on questions.

The modeling concerns of
that era have mostly faded

but econometricians have since
found much to contribute.

I'll save my personal lists of greatest
hits and exciting new artists for the

The print version of this lecture.

I'll wrap up here by saying that I'm
proud to be part of the Contemporary.

Empirical economics, Enterprise

and I'm gratified beyond words

to have been recognized for contributing
to it back at Princeton in the late 80s.

My graduate school classmates and I
chuckled leading Ed lemurs. Lament,

that no Economist takes another
economists. Empirical work. Seriously.

This is no longer, true, empirical
work today aspires to tell convincing.

Also stories, not that every
effort succeeds far from it.

But as any economics job
market candidate will tell you,

empirical work carefully executed and
clearly explained is taken seriously. Indeed.

That is a measure of
our Enterprises success.

If you'd like to learn more from Josh,

check out his free course mastering
econometrics if you'd like to

Spore, Josh's research,

check out the links in the description
or you can click to watch more

of Josh's videos.