Joshua Angrist Nobel Prize Lecture 2021

♪ [music] ♪

As I stilled my trembling
iPhone early on October 11th,

my thoughts went to the question
of whether Nobel-level recognition

might change life
for the Angrist family.

Ours is a close-knit family.

We lack for nothing.

So I worried
that stressful Nobel celebrity

might not be a plus.

But with the first cup of coffee,

I began to relax.

It occurred to me

that the matter
of how public recognition

affects a scholars life

is, after all,
a simple causal question.

The Nobel intervention

is substantial, sudden,
and well-measured.

Outcomes like health and wealth
are easy to record.

Having just been recognized
with my co-laureates,

Guido Imbens and David Card,

for answering causal questions
using observational data,

my thoughts moved
from personal upheaval

to the more familiar demands

of identification and estimation
of causal effects.

I was able to soothe
my worried mind

by imagining a study
of the Nobel Prize treatment effect.

How would such a study
be organized?

in a 1999 essay published in the
"Handbook of Labor Economics,"

Alan Krueger and I embraced
the phrase "empirical strategy."

The handbook volume
in question was edited

by two of my Princeton
PhD thesis advisors,

Orley Ashenfelter and David Card,

among the most successful
and prolific graduate advisors

economics has known

An empirical strategy
is a research plan

that encompasses data collection,

identification,
and econometric estimation.

Identification is the applied
econometricians term

for research design --

a randomized, clinical trial.

And RCT is the simplest
and most powerful research design.

In RCTs, causal effects
are identified

by the random assignment
of treatment.

Random assignment ensures

that treatment and control groups

are comparable
in the absence of treatment.

So differences
between them afterwards

reflect only the treatment effect.

Nobel prizes are probably
not randomly assigned.

This challenge notwithstanding,

a compelling empirical strategy
for the Nobel treatment effect

comes to mind,

at least as a flight of empirical fancy.

Imagine a pool of prize-eligible
Nobel applicants,

the group under consideration
for the prize.

Applicants need not apply themselves.

They are, I presume, nominated
by their peer scholars.

My fanciful Nobel impact study
looks only at Nobel applicants

since these are all elite scholars.

But that is only the first step.

Credible applicants, I imagine,
are evaluated by judges

using criteria, like publications,
citations, nominating statements.

I imagine this material is reviewed
and assigned a numerical score,

using some kind of scoring rubric.

Top scorers up to three per field,
in any single year,

win a prize.

Having identified applicants

that collected data
on their scores,

the next step
in my Nobel impact study

is to record the relevant cutoffs.

The Nobel cutoff
is the lowest score

among those awarded a prize,

Many Nobel hopefuls
just missed the cutoff.

Looking only at near-misses,
along with the winners,

differences in scores

between those above
and below the cutoff

begin to look serendipitous,
almost randomly assigned.

After all,

near-Nobels are among
the most eminent of scholars, too.

With one more
high-impact publication,

a little more support
from nominators.

they would have been awarded
Nobel gold --

some of them, someday,
surely will be.

The empirical strategy
sketched here is called

a Regression Discontinuity Design,

RD.

RD exploits the jumps
in human affairs,

induced by rules, regulations

and the need to classify
people for various assignment purposes.

When treatment or intervention
is determined by whether

a tiebreaking variable
crosses, a threshold.

Those just below the threshold,

become a natural control
group for those who clear it.

Rd does not require that the
variable, whose causes we seek

switch fully on, or
fully off at the cutoff.

We require only that the average value
of this variable jump at the cutoff

Rd can allow. For example,
for the fact that this

Here's near Nobel,

might be next year's winner

allowing for this leads to
the use of jumps and the rate

at which treatment is assigned to
construct instrumental variables IV,

estimates of the effect of treatment
received. This sort of R&D, is said to be

fuzzy, but as Steve Pischke, and I
wrote in our first book fuzzy rdd as IV.

The first Rd study, I contributed to was
written with my frequent collaborator,

Victor livie. This study is
motivated by the high court.

Costs and uncertain returns to
smaller Elementary School classes.

We exploited a rule used by Israeli
elementary schools to determine class size.

This rule is used to
estimate class size effects.

As if in a class size RCT

in the 1990s Israeli classes, where large
students enrolled in a Grade cohort of 40,

we're likely to be
seated in a class of 40.

That's the relevant cut off, add
another child to the cohort making 41.

And the cohort was likely to be
split into two much smaller classes.

This leads to, the maimonides
rule research design

so named because the 12th century rambam
proposed, a maximum class size of 40.

This figure plots is rarely.

Fourth grade class sizes, as a
function of fourth grade enrollment

overlaid with the
theoretical class size rule,

maimonides rule. The fit isn't perfect.

That's a feature that makes this
application of R&D fuzzy, but the gist of

The thing is a marked class size,
drop at each integer. Multiple of 40,

the relevant cut off just
as predicted, by the rule,

as it turns out, these
drops and class size

are reflected in jumps in 4th
and 5th grade math scores.

What a comparison of Nobel
laureates to near laureates?

Really be a good natural experiment.

The logic behind this sort of claim

seems more compelling for comparisons of
schools with 40 and 41, fourth graders

that for comparisons of
laureates and near laureates.

Yet, both scenarios exploit a
feature of the physical world,

provided, the tiebreaking variable known
2rd mavens as the running variable,

has a continuous distribution. The
probability of crossing the cut off.

Is 1/2 when examined in a
narrow window around. The cutoff

in R&D empirical work,

the window around such cut-offs
is known as a band with

importantly this limiting
probability is point 5 for everybody

regardless of how qualified they look
going into the Nobel competition.

This remarkable fact can be seen in
data on applicants to one of New York's,

highly coveted screen. Schools

by way of background roughly, 40%
of New York City's middle and high.

Schools select their applicants
on the basis of test scores grades

and other exacting criteria.

In other words. The admissions
regime for screen schools

is a lot, like the scheme. I've
imagined for the Nobel Prize

screen,

schools are, but one of a number of
Highly selective systems within a system.

In large US school districts, Boston,
Chicago, San Francisco and Washington, d.c.

All feature. Highly selective
institutions, often known as exam, Schools

exam, schools, operate as

Of larger public school systems that
enroll students without screening

motivated by the enduring controversy
over the equity of screened admissions.

My blueprint, Labs collaborators,

and I have examined the causal
effects of exam School, attendance

in Boston, Chicago and New York.

This figure shows the probability

of being offered a seat at New York's
storied Townsend Harris, high school

ranked 12th, Nationwide bar
height. In the figure marks.

The qualification rate.
That is the likelihood.

Hood of earning a Townsend,
Harris admission score

above that of the lowest scoring.
Applicant, offered a seat importantly,

the bars show qualification rates
conditional on a measure of pre-application.

Baseline achievement

in particular.

The bars Mark qualification
rates conditional on

whether an applicant has upper
quartile or lower quartile

6th. Grade math scores Townsend Harris
applicants with high Baseline. Scores

are much more likely to qualify
than applicants with low base.

And scores, this isn't surprising.

But in a shrinking symmetric
bandwidth around the schools. Cut off

qualification rates in
the two groups, converge

qualification rates, in the
last, and smallest groups

are both remarkably close to 1/2.

This is what we'd expect to see where
Townsend Harris to admit, students,

by tossing a coin rather than by
selecting only those who scored highly

on the school's entrance exam, even
when admissions operates by screen,

During the data can be
arranged. So as to mimic an RCT.

A few of the questions. I've studied
are more controversial than the question

of access to public exam schools,
like the Boston Latin School.

Chicago's Peyton and Northside
selective enrollment high schools

and New York's legendary,
Brooklyn Tech Bronx, Science

and Stuyvesant specialized high schools,

which have graduated 14 Nobel laureates
between them Townsend, Harris, the school.

We started with today. Graduated three,
no Bells, including Economist can Arrow

exam School proponents.

The opportunities. These schools provide
as democratizing public education.

Wealthy families. They argue can access
exam School curricula in the private sector.

Shouldn't ambitious. Low-income
students be afforded.

The same chance at Elite Education.

Critics of selective
enrollment schools argue that

rather than expanding Equity

exam. Schools are inherently biased
against the Black and Hispanic students

that make up the bulk of America's urban
districts, New York, super selective.

And for example, enrolled only
seven black students in 2019,

out of an incoming class of 895.

But our exam School seats
really worth fighting for

my collaborators.

And I have repeatedly used Rd empirical
strategies to study the causal effects

of attendance that exam schools like
Townsend, Harris and Boston Latin.

Our first exam School study, which
looks at schools in Boston and New York

encapsulate, these findings and
its title the elite illusion.

The elite illusion refers to

the fact that while exam school students
undoubtedly have high test scores

and other good outcomes.

This is not a causal effect
of exam School attendance,

our estimates consistently suggest that the
causal effects of exam School attendance

on their students learning
and college-going are 0.

Maybe even -

the good performance of exam school
students, reflect selection bias.

That is the process by which
these students are chosen.

Rather than causal effects

data from Chicago's large exam School
sector illustrate the elite illusion.

This figure plot spear mean achievement.

That is the sixth grade test scores

of my ninth grade classmates
against the admissions tiebreaker

for a subset of applicants. To any
one of Chicago's nine exam schools,

applicants to these schools rank up to 6.

While the exam schools, prioritize their
applicants using a common Composite Index.

Formed from an admissions test,

gpas and grade 7 standardized scores.

This composite tiebreaker
is the running variable

for an RD design. That reveals what
happens when any applicant is offered

an exam School seat in
Chicago's exam School match,

which is actually an application
of the celebrated Gale

and shapley matching algorithm exam.

School applicants are sure to
be offered a seat somewhere

when they clear the lowest in their
set of cut-offs among the schools. They

Rank, we call this lowest cut
off the qualifying. Cut off.

The figure shows a sharp jump in peer
mean achievement for Chicago exam school.

Applicants who clear
their qualifying cut off

this jump, reflects.

The fact that most applicants
offered an exam School seat, take it

and applicants who enroll at one of
Chicago's selective enrollment high. Schools

are sure to be seated in a
9th grade classroom filled

with academically precocious peers,

because only the relatively
precocious make it in the end.

Recent pure achievement across the
qualifying cut off amounts to almost half

a standard deviation, a very large effect.

And yet precocious peers, notwithstanding.

The offer of an exam. School, seat
does not appear to increase learning.

Let's plot applicants ACT scores
against their tiebreaker values.

This plot shows that exam
school applicants who clear?

Their qualifying cut off perform.
Sharply worse on the ACT.

What explains this?

It takes a tale of?

V + Rd to untangle, the
forces behind this intriguing

and unexpected negative effect,

but first some IV Theory,

they do and bands.

And I developed theoretical tools
that enhance economists understanding

of empirical strategies
involving IV and RD.

The prize we share is in
recognition of this work.

He do and I overlapped for
only one year at Harvard

where we had both taken
our first jobs posted.

PhD. I welcomed three do
the Cambridge Massachusetts

with a pair of interesting.
Instrumental variables.

I had used the draft lottery
instrument in my PhD thesis

to estimate The long-run Economic
Consequences of serving in the armed forces

for soldiers, who were drafted.

The draft lottery,

instrument relies on the fact that lottery
numbers randomly assigned to birthdays

determined vietnam-era conscription risk

yet. Even then most soldiers
were volunteers as they are.

Today,

the quarter birth instrument is
used in my 1991, paper with Alan,

Krueger to estimate the
economic returns to schooling.

This instrument uses,

the fact that men who are born earlier
in the year or allowed to drop out

of high school on their 16th birthday

with less schooling completed
than those born later.

They do and I soon
began asking each other.

What really do we learn
from the draft eligibility

and quarter of birth natural experiments?

An early result in our quest
for a new understanding of ivy.

As a solution to the problem of selection
bias in an RCT, with partial compliance,

even in a randomized, clinical trial,

some of the people assigned
to treatment May opt out.

This fact has long Vex
trial list because decisions

to opt out are not made
by random assignment.

Our first manuscript together

shows that in a randomized
trial with partial compliance.

You can use IV to estimate the
effect of treatment on the treated.

Even when some offered
treatment decline, it

This works in spite

of the fact that those who comply with
treatment may be a very select group.

Unfortunately for us. We
were late to the party.

Not long after releasing
our first working paper.

We learned of a concise
contribution from Howard Bloom.

That includes this theoretical result,

remarkably bloom had derived
this from first principles

without making a connection to Ivy.

So he do and I went back to the drawing
board and a few months later. We had late

a theorem showing how
to estimate the local a

Which treatment effect the late
theorem generalizes the bloom, theorem

and establishes the connection
between compliance and IV.

Maintaining the clinical trials, analogy.

Let's zi indicate whether subject I is
offered treatment. This is randomly assigned,

also let d1i indicate subject eyes treatment
status, when assigned to treatment,

and let the 0i indicate subject eyes
treatment status, when assigned to control.

I'll use this formal notation to give
a clear statement of the late result.

And then follow up with examples.

A key piece of the late framework
pioneered by statistician.

Don Ruben, is the pair of
potential outcomes as is customary.

I denote potential outcomes for subject
to. I in the treated and untreated states

by y 1i + y 0i respectively,

The observed outcome is why one eye for
the treated and y0i for those not treated

y 1 I minus y 0, I is the causal
effect of treatment on individual I but

We can never see. We try therefore to
estimate some kind of average, causal effect.

The late framework allows us to do that
in an RCT where some controls are treated.

The theorem says that the average causal
effect on people, whose treatment status

can be changed, by the offer of treatment

is the ratio of ITT to the treatment
control difference in compliance rates,

a mathematical statement of
this result appears here,

where Greek letter, Delta symbolizes
the ITT effect and Greek symbols pie.

One and pi0, our compliance rates
in the group assigned to treatment

and the group assigned
to control respectively,

the print version of this lecture delves
deeper into late intellectual history.

Highlighting key contributions
made with Reuben.

For now though. I'd like to make
the late theorem concrete for you

by sharing, one of my
favorite applications of it.

I'll explain the late framework

through a research question
that has fascinated me for.

Almost two decades.

What is the causal effect of Charter
School attendance on learning?

Charter schools are public schools.

That operate independently of traditional
American public school districts a charter

the right to operate a public school is
typically awarded for a limited period

subject to Renewal conditional
on good school performance.

Charter schools are free to structure
their curriculum and school environment.

The most controversial difference between
Charters and traditional Public Schools

is the

Fact that the teachers and staff who
work at Charter Schools rarely belonged

to labor unions by contrast. Most
Big City, public school, teachers.

Work under Union, contracts,

the 2010 documentary film
Waiting for Superman.

Feature schools belonging to
the knowledge is power program.

Kip Kip schools are emblematic
of the high expectations.

Sometimes also called No Excuses
approach to public education.

The no excuses model, features
a long school day and extended.

School year, selective teacher hiring
and focuses on traditional reading

and math skills. The American
debate over education reform,

often focuses on the achievement Gap.

That's shorthand for large,

test score differences by race and
ethnicity because of its focus on minority.

Students kept is often Central in this
debate with supporters pointing to the fact

that non-white Kip students have markedly
higher test scores, the non-white students

from nearby schools Kip
Skeptics on the other.

Hand argued that keeps apparent success,

reflects, the fact that Kip
attracts families, whose children

would be more likely to
succeed. Anyway, who's right?

As you've probably guessed by, now,

a randomized trial, might prove, decisive
in the debate over schools. Like Kip,

like Nobel Prize is though, seats
at KIPP are not randomly assigned.

Well, at least not entirely.

In fact,

Massachusetts charter schools
with more applicants than seats

must offer their seats by Lottery.

Sounds like a good natural experiment,

a little over a decade ago my
collaborators and I collected data

on Kip admissions lotteries laying the
foundation for to pioneering Charter studies.

The first to use lotteries to study Kip.

Our Kip analysis is a classic IV story

because many students offered
a seat in the kip Lottery

failed to show up in the fall.

Well, a few not offered to seat.
Nevertheless find their way in

this graphic shows Kip
middle school applicants.

Math scores, one year
after applying to Kip

the entries above the line

show, that Kip applicants were offered
a seat have standardized math scores,

close to 0 that is near the state average.

As before we're working
with standardized score data

that has a mean of zero and
a standard deviation of one

because Kip applicants start
with fourth grade scores.

That are roughly point. Three standard
deviations below, the state mean

achievement at the level of the
state average is impressive by

Trashed the average math score
among those not offered a seat

is about minus Point Three, Six Sigma

that is point three. Six standard
deviations below the state mean

a result typical for urban
students. In Massachusetts.

Since Lottery offers a randomly assigned.

We could say with confidence that the
offer of a seated Kip boost math scores by

an average of Point Three, Six Sigma

a large effect. That's
also statistically precise.

We can be confident. This
isn't a chance finding

What does an offer effective
Point Three? Six Sigma

tell us about the effects
of actually going to Kip

IV methods convert Kip offer
effects into Kip attendance effects.

I'll use this brief clip from my marginal
Revolution University short course

to quickly review the key
assumptions behind this conversion

IV describes a chain reaction.

Why do offers affect achievement probably?
Because they affect Charter attendance

and Charter attendance. Improved.

Loves math, scores,

the first link in the chain called.

The first stage is the effect of
the lottery on Charter attendance.

The second stage is the link between
attending a charter and an outcome variable.

In this case math scores
the instrumental variable

or instrument. For short is the variable
that initiates the chain reaction.

The effect of the instrument on the
outcome is called the reduced form.

This Chain Reaction can be
represented mathematically.

We multiply, the first stage, the
effect of winning on attendance,

by the second stage, the
effect of attendance on scores,

and we get the reduced form, the effect
of winning the lottery on scores.

The reduced form. And first stage
are observable and easy to compute.

However, the effect of attendance on
achievement is not directly observed.

This is the causal effect.
We're trying to determine

given some important assumptions
will discuss shortly.

We can find the effect of Kip attendance
by dividing the reduced form by

the first stage IV,
eliminate selection bias,

but like all of our tools,
the solution Builds on a

Set of assumptions. Not
to be taken for granted

first. There must be a
substantial first stage.

That is the instrumental variable,

winning or losing the
lottery must really change.

The variable, whose effect we're
interested in here, Kip attendance.

In this case. The first stage is

not really in doubt, winning the lottery,
make skip attendance, much more likely,

not all IV, stories are like that.

S the instrument must
be as good as randomly.

Signed meaning lottery winners and
losers have similar characteristics.

This is the independence Assumption.

Of course,

Kip Lottery wins, really
are randomly assigned

still, we should check for balance
and confirm that winners and losers

have similar family, backgrounds,
similar, aptitudes, and so on,

in essence, we're checking to
ensure Kip. Lotteries are fair

with no group of applicants,
suspiciously, likely to win.

Finally. We require the
instrument change outcomes soul.

Through the variable of interest.
In this case, attending camp,

this assumption is called
the exclusion restriction.

The causal effect of Kip attendance can
therefore be written as the ratio of

the effect of offers on
scores in the numerator,

over the effect of offers on Kip
and Roman in the denominator.

The numerator in this IV formula,

that is the direct effect of the
instrument on outcomes. Has a special name.

This is called the reduced form.
The denominator is the first step.

Stage

the exclusion restriction is often the
trickiest or most controversial part of

an IV story. Here, the exclusion
restriction amounts to the claim

that the .36 score, differential
between lottery. Winners and losers

is entirely attributable to the .74,
win-loss difference in attendance rates.

Plugging in the numbers.

The effect of Kip attendance works
out to be point four eight Sigma,

almost half a standard deviation gain in
math scores. That's a remarkably large.

Perfect, who exactly benefits.
So spectacularly from KIPP,

does everyone who applies to
KIPP, see such large gains,

late answers. This question.

The late interpretation of the
kip IV, empirical strategy,

is illuminated by the
biblical story of Passover,

which explains that there
are four types of children

each with characteristic behaviors

to keep track of these
children and their behavior.

I'll give them a literate
of names. Applicants like

Alvaro are dying to go to Kip.
If Alvaro loses, the kip Lottery.

His mother finds a way to
enroll him in Kip. Anyway,

perhaps by reapplying applicants like
Camilla are happy to go to Camp if they win

a seat in the lottery, but stoically
accept the verdict, if they lose finally,

applicants, like normando worried about
long days and lots of homework at KIPP

normando doesn't really want to go and
refuses to go to Kip when told that he won

the lottery.

That was called a never taker because
win or lose. He doesn't go to Kip

at the other end of Kip commitment.
Alvaro is called an always taker.

He'll happily take a seat. Went offered
while his mother simply finds a way

to make it happen for him.

Even when he loses for Alvaro and
normando. Both choice of school.

Kip traditional is
unaffected by the lottery.

Camilla is the type of
applicant who gives IV its power

the instrument determines
her treatment status.

I IV strategies depend on applicants,
like, Camilla who are called compliers.

This term comes from the world of
randomized, Trials, introduced earlier

as we've already discussed,

many randomized, trials, randomize
only the opportunity to be treated

while the decision to comply with the
treatment remains voluntary and non-random.

RCT. Compliers are those who take treatment
when the offer of treatment is made?

But not otherwise with
Lottery instruments.

Late is the effect of Kip attendance on
Camilla and other compliers like her.

Who enroll at KIPP take treatment when
offered treatment through the lottery.

But not otherwise

IV methods are uninformed of for always
takers like Alvaro and never takers

like normando because the instrument
is unrelated to their treatment status.

Hey, didn't I say there
are four types of children.

A fourth type of child in IV Theory
behaves perversely every family has one.

These defiant children and Roland
Kip only when they lose the lottery.

Actually, the late theorem requires
us to assume there are few defiers,

that seems like a reasonable assumption
for Charter Lottery instruments.

If not in life.

The late theorem is sometimes
seen as limiting the relevance

of econometric estimates

because it focuses attention
on groups of compliers

yet. The population of compliers is a
group. We'd very much like to learn about

in the kip example, compliers
our children, likely to be,

Drawn into Kip where the school to expand
and offer additional seats in a lottery.

How relevant is this a few years ago,

Massachusetts indeed allowed
thriving charter schools to expand

a recent study by some of my lab mates

shows that late estimates like
the one we just computed for Kip

predict learning gains at the
schools created by Charter expansion.

Late isn't just a theorem.

It's a

framework.

The late framework can be used to
estimate the entire distribution

of potential outcomes for compliers

as if we really did have a
randomized trial for this group.

Although the theory behind this
fact is necessarily technical.

The value of the framework is easily
appreciated in practice by way of illustration.

Recall that the kip study

is motivated in part by
differences in test scores by race.

Let's look at the distribution of
fourth grade scores separately by.

Race for applicants to Boston
Charter, Middle Schools,

the two sides of this figure
show distributions for treated

and untreated compliers treated.
Compliers are compliers offered.

A charter seat in a lottery. While
untreated compliers are not offered a seat

because these are fourth grade scores
while middle school begins in fifth

or sixth grade. The two sides
of the figure are similar.

Both sides show score distributions for
black applicants shifted to the left

of the corresponding.

Gorgeous tribulations
for Whites by 8th grade.

Treated compliers have completed
Middle School at a Boston Charter.

Well, I'm treated compliers have
remained in traditional Public School.

Remarkably.

This next graphic shows that the
eighth grade score distributions

of black-and-white treated.
Compliers are indistinguishable.

Boston charter middle schools,
close the achievement Gap,

but for the untreated black and white
score distributions remained distinct

with black students.

Hind white students as
they were in fourth grade,

Boston Charters,

close the achievement Gap because
those who enter Charter Schools,

the farthest behind tend to gain
the most from Charter. Enrollment

I elaborate on this point in
the print version of this talk.

Remember the puzzle of -
Chicago exam School effects.

I'll finish the scientific part of my
talk by using IV and RD to explain this.

This surprising finding

the resolution of this puzzle starts
with the fact that economic reasoning

is about Alternatives.

So what's the alternative
to an exam school education

for most applicants to
Chicago exam schools.

The leading non exam. Alternative
is a traditional public school,

but many of Chicago's rejected exam school
applicants enroll in a charter school

exam school offers. Therefore reduce the
likelihood of Charter School attendance.

Specifically exam schools divert
applicants away from high schools

in the noble network of charter schools.

Noble

with pedagogy much, like Kip is one of
Chicago's most visible Charter providers.

Also like Kip convincing evidence on
Noble Effectiveness comes from admissions,

lotteries.

The x-axis in this graphic shows Lottery
offer effects on years enrolled at Noble.

This is the noble first stage.

For an IV setup that uses a dummy.

Indicating Noble Lottery offers as
an instrument for Noble enrollment.

Now, this graphic has a
feature that distinguishes.

It from the simpler Kip analysis,
the plot shows. First stage effects.

For two groups.

One for Noble applicants who live

in Chicago's lowest income
neighborhoods Tier 1 and 1/4 Noble,

applicants who live in higher, income
areas, tier 3, remember the IV chain.

Each point in this graphic has coordinates
given by first stage reduced form

and therefore implies an IV estimate.

The effect of noble enrollment on ACT scores
is the ratio of reduced form coordinate

to First Stage coordinate.

The graphic shows to such ratios.

The relevant results for Tier 1 are .35.

While for tier 3. We have .33 not bad
for Noble applicants from both tears.

These

stage in reduced form estimates

imply a yearly Noble and Roman effective
about a third of a standard deviation gain,

in act math scores.

Notice. There's also a line connecting
the two Ivy estimates in the figure

because this line passes
through the origin,

it's slope rise. Over run is about equal
to the to IV estimates. In this case.

The slope is about Point 3 for

the fact that the line passes
through 0 0 is significant for

Reason by this fact, we've
substantiated the exclusion restriction,

specifically, the exclusion restriction,

says that given a group for which Noble
offers are unrelated to Noble enrollment.

We should expect to see Zero reduced form
effect of these offers made to applicants

in that group.

How consistent is the evidence for a
noble cause learning gain on the order

of point, three, four Sigma per year
in this next graphic? We've added

L've more points to the original to

the red points here, show, first stage
in reduced form, Noble offer effects

for 12, additional groups to more tears,

and 12 groups, defined by demographic,
characteristics related to race sex,

family, income, and Baseline scores.

Although not a perfect fit

these points cluster around a line
with slope Point Three, Six Sigma

much like the line. We saw earlier
for applicants from tiers 1 and 3.

You're likely now.

Ring. What the noble IV estimates
in this figure have to do

with exam School enrollment.

Here's the answer,

the Blue Line in this new graphic
shows. As we should expect

that exam. School exposure jumps up

for applicants who clear their
qualifying cut off. At the same time.

The Red Line shows that Noble School
enrollment clearly Falls at the same point.

This is the diversion effect of exam
school offers on Noble enrollment.

Many kids offered an exam School
seat, prefer that exam School seat,

to enrollment at Noble IV, affords us,
the opportunity to go out on a limb

with strong claims about the
mechanism behind the causal effect.

Here's a strong causal, claim Regarding why
Chicago exam schools reduce achievement.

The primary force driving
reduced form exam School.

Qualification effects on ACT scores.

I claim is the effect of exam
school offers on Noble and

Vomit in support of this claim,

consider the points plotted here in blue
all well to the left of 0 on the x-axis.

These points are negative because they Mark
the effect of exam School qualification

on Noble School, enrollment for
particular, groups of applicants.

Now, we've already seen

that Noble applicants offered a noble seen

realize large act math, gains as a result.

Now consider exam school offers as

An instrument for Noble enrollment,

as always, Ivy is a chain reaction.

If exam School qualification reduces
time at Noble, by Point 37 years,

and each year of noble, enrollment

boosts act math scores by
about Point Three, Six Sigma.

We should expect reduced form
effects of exam School. Qualification

to reduce ACT scores by the
product of these two numbers.

That is by about Point 1, 3 Sigma

the reduced form qualification
effects at the left of the

Here are broadly consistent with this.

They cluster closer to -
.16. Then 2 minus Point 1 3,

but that difference is well
within the sampling variance

of the underlying estimates.

The causal story told here,

postulates diversion
away from Charter Schools

as the mechanism by which exam
school offers effect achievement.

In other words, it's Noble enrollment,

that's presumed to satisfy
an exclusion restriction.

When we use exam school offers as an

Variable importantly,

as we saw before the line in this final
graphic, with two sets of 14 points,

runs through the origin.

This fact supports our
new exclusion restriction

for any applicant group

for which exam school offers have little
or no effect on Noble School enrollment.

We should also see ACT scores unchanged

at the same time because the blue and
red dots cluster around the same line,

the IV estimates of noble School,
enrollment effects generated.

I both Noble and exam school
offers are about the same.

I hope this empirical story convinces,
you of the power of IV and RD

to generate new causal,
knowledge for decades.

I've been lucky to work on many
equally engaging empirical problems.

I computed the draft
lottery IV estimates in

my Princeton PhD thesis on a
big hairy Mainframe monster.

Using nine track tapes, and Lease
space on a communal. Hard drive,

Princeton graduate students learn
to mount and manipulate tape.

Reels the size of a cheesecake.

Thankfully empirical work today
is a little less labor-intensive.

What else is improved in
the modern empirical era

in a 2010 article, Steve
Pischke, and I coined the phrase.

He Revolution

by this, we mean economic shift towards
transparent empirical strategies.

Applied to concrete, causal questions,

like the questions. David
Carter studied. So convincingly.

The econometrics of my schooldays
focused more on models than on questions.

The modeling concerns of
that era have mostly faded

but econometricians have since
found much to contribute.

I'll save my personal lists of greatest
hits and exciting new artists for the

The print version of this lecture.

I'll wrap up here by saying that I'm
proud to be part of the Contemporary.

Empirical economics, Enterprise

and I'm gratified beyond words

to have been recognized for contributing
to it back at Princeton in the late 80s.

My graduate school classmates and I
chuckled leading Ed lemurs. Lament,

that no Economist takes another
economists. Empirical work. Seriously.

This is no longer, true, empirical
work today aspires to tell convincing.

Also stories, not that every
effort succeeds far from it.

But as any economics job
market candidate will tell you,

empirical work carefully executed and
clearly explained is taken seriously. Indeed.

That is a measure of
our Enterprises success.

If you'd like to learn more from Josh,

check out his free course mastering
econometrics if you'd like to

Spore, Josh's research,

check out the links in the description
or you can click to watch more

of Josh's videos.