♪ [music] ♪
- [Joshua] As I stilled my trembling iPhone early on October 11th,
my thoughts went to the question of whether Nobel-level recognition
might change life for the Angrist family.
Ours is a close-knit family.
We lack for nothing.
So I worried that stressful Nobel celebrity
might not be a plus.
But with the first cup of coffee,
I began to relax.
It occurred to me
that the matter of how public recognition
affects a scholars life
is, after all, a simple causal question.
The Nobel intervention
is substantial, sudden, and well-measured.
Outcomes like health and wealth are easy to record.
Having just been recognized with my co-laureates,
Guido Imbens and David Card,
for answering causal questions using observational data,
my thoughts moved from personal upheaval
to the more familiar demands
of identification and estimation of causal effects.
I was able to soothe my worried mind
by imagining a study of the Nobel Prize treatment effect.
How would such a study be organized?
in a 1999 essay published in the "Handbook of Labor Economics,"
Alan Krueger and I embraced the phrase "empirical strategy."
The handbook volume in question was edited
by two of my Princeton PhD thesis advisors,
Orley Ashenfelter and David Card,
among the most successful and prolific graduate advisors
economics has known
An empirical strategy is a research plan
that encompasses data collection,
identification, and econometric estimation.
Identification is the applied econometricians term
for research design --
a randomized, clinical trial.
And RCT is the simplest and most powerful research design.
In RCTs, causal effects are identified
by the random assignment of treatment.
Random assignment ensures
that treatment and control groups
are comparable in the absence of treatment.
So differences between them afterwards
reflect only the treatment effect.
Nobel prizes are probably not randomly assigned.
This challenge notwithstanding,
a compelling empirical strategy for the Nobel treatment effect
comes to mind,
at least as a flight of empirical fancy.
Imagine a pool of prize-eligible Nobel applicants,
the group under consideration for the prize.
Applicants need not apply themselves.
They are, I presume, nominated by their peer scholars.
My fanciful Nobel impact study looks only at Nobel applicants
since these are all elite scholars.
But that is only the first step.
Credible applicants, I imagine, are evaluated by judges
using criteria, like publications, citations, nominating statements.
I imagine this material is reviewed and assigned a numerical score,
using some kind of scoring rubric.
Top scorers up to three per field, in any single year,
win a prize.
Having identified applicants
that collected data on their scores,
the next step in my Nobel impact study
is to record the relevant cutoffs.
The Nobel cutoff is the lowest score
among those awarded a prize,
Many Nobel hopefuls just missed the cutoff.
Looking only at near-misses, along with the winners,
differences in scores
between those above and below the cutoff
begin to look serendipitous, almost randomly assigned.
After all,
near-Nobels are among the most eminent of scholars, too.
With one more high-impact publication,
a little more support from nominators.
they would have been awarded Nobel gold --
some of them, someday, surely will be.
The empirical strategy sketched here is called
a Regression Discontinuity Design,
RD.
RD exploits the jumps in human affairs,
induced by rules, regulations,
and the need to classify people for various assignment purposes.
When treatment or intervention is determined
by whether a tiebreaking variable crosses a threshold,
those just below the threshold become a natural control group
for those who clear it.
RD does not require that the variable,
whose causes we seek,
switch fully on or fully off at the cutoff.
We require only
that the average value of this variable
jump at the cutoff
RD can allow, for example, for the fact
that this year's near-Nobel might be next year's winner.
Allowing for this leads to the use of jumps
and the rate at which treatment is assigned
to construct instrumental variables,
IV,
estimates of the effect of treatment received.
This sort of RD is said to be fuzzy,
But as Steve Pischke and I wrote in our first book:
"fuzzy RD is IV."
(cheering)
The first RD study I contributed to
was written with my frequent collaborator, Victor Lavy.
This study is motivated
by the high costs and uncertain returns
to smaller elementary school classes.
We exploited a rule used by Israeli elementary schools
to determine class size.
This rule is used to estimate class size effects,
as if in a class size RCT.
In the 1990s, Israeli classes were large.
Students enrolled in a grade cohort of 40.
We're likely to be seated in a class of 40.
That's the relevant cutoff.
Add another child to the cohort, making 41,
and the cohort was likely to be split
into two much smaller classes.
This leads to the Maimonide's rule research design,
so named because the 12th-century Rambam
proposed a maximum class size of 40.
This figure plots is rarely fourth grade class sizes
as a function of fourth grade enrollment,
overlaid with the theoretical class size rule,
Maimonides rule.
The fit isn't perfect.
That's a feature that makes this application of RD fuzzy.
But the gist of the thing is a marked class size drop,
at each integer multiple of 40, the relevant cutoff,
just as predicted by the rule.
As it turns out, these drops in class size
are reflected in jumps
in 4th and 5th grade math scores.
♪ [music] ♪
Would a comparison of Nobel laureates to near laureates
really be a good natural experiment?
The logic behind this sort of claim
seems more compelling for comparisons of schools
with 40 and 41 4th graders
than for comparisons of laureates and near laureates.
Yet, both scenarios exploit a feature of the physical world.
Provided the tiebreaking variable
known to RD mavens as the running variable,
has a continuous distribution,
the probability of crossing the cutoff approaches one-half
when examined in a narrow window
around the cutoff.
In RD empirical work,
the window around such cutoffs is known as a bandwidth.
Importantly, this limiting probability is 0.5 for everybody
regardless of how qualified they look
going into the Nobel competition.
This remarkable fact can be seen in data on applicants
to one of New York's highly coveted screen schools.
By way of background,
roughly 40% of New York City's middle and high schools
select their applicants on the basis of test scores, grades,
and other exacting criteria.
In other words, the admissions regime for screen schools
is a lot like the scheme I've imagined
for the Nobel Prize.
Screen schools are but one of a number
of highly selective systems within a system
in large US school districts.
Boston, Chicago, San Francisco, and Washington, D.C
all feature highly selective institutions,
often known as exam schools.
Exam schools operate
as part of larger public school systems
that enroll students without screening.
Motivated by the enduring controversy
over the equity of screened admissions,
my blueprint labs collaborators and I
have examined the causal effects of exam school attendance
in Boston, Chicago, and New York.
This figure shows the probability of being offered a seat
at New York's storied Townsend Harris High School,
ranked 12th, nationwide.
Bar height in the figure marks the qualification rate --
that is, the likelihood of earning a Townsend Harris admission score,
above that of the lowest scoring applicant
offered a seat.
Importantly, the bars show qualification rates conditional
on a measure of pre-application baseline achievement.
In particular, the bars mark qualification rates conditional
on whether an applicant
has upper quartile or lower quartile 6th grade math scores.
Townsend Harris applicants with high baseline scores
are much more likely to qualify
than applicants with low baseline scores.
This isn't surprising.
But in a shrinking symmetric bandwidth
around the schools cutoff,
qualification rates in the two groups converge
Qualification rate in the last and smallest groups
are both remarkably close to one-half
This is what we'd expect to see
where Townsend Harris to admit students,
by tossing a coin,
rather than by selecting only those who scored highly
on the school's entrance exam.
Even when admissions operates by screening,
the data can be arranged so as to mimic an RCT.
A few of the questions I've studied are more controversial
than the question of access to public exam schools,
like the Boston Latin School,
Chicago's Payton and Northside selective enrollment high schools,
and New York's legendary Brooklyn Tech,
Bronx Science,
and Stuyvesant specialized high schools,
which have graduated 14 Nobel laureates between them.
Townsend Harris, the school we started with today,
graduated three Nobels, including economist Ken Arrow.
Exam school proponents
see the opportunities these schools provide
as democratizing public education.
"Wealthy families," they argue,
can access exam school curricula in the private sector.
Shouldn't ambitious low-income students
be afforded the same chance at elite education?
Critics of selective enrollment schools
argue that rather than expanding equity,
exam schools are inherently biased
against the Black and Hispanic students
that make up the bulk of America's urban districts,
New York's super selective Stuyvesant, for example,
enrolled only seven Black students in 2019,
out of an incoming class of 895.
But are exam school seats really worth fighting for?
My collaborators and I have repeatedly used
RD empirical strategies to study the causal effects
of attendance at exam schools
like Townsend Harris and Boston Latin.
Our first exam school study,
which looks at schools in Boston and New York
encapsulates these findings in its title:
"The Elite Illusion."
"The Elite Illusion" refers to the fact
that while exam school students undoubtedly have high test scores
and other good outcomes,
this is sot a causal effect of exam School attendance.
Our estimates consistently suggest
that the causal effects of exam school attendance
on their students learning and college-going are 0.
Maybe even negative.
The good performance of exam school students
reflect selection bias --
that is, the process by which these students are chosen,
rather than causal effects.
Data from Chicago's large exam school sector
illustrate the elite illusion.
This figure plots peer mean achievement --
that is, the 6th grade test scores of my 9th grade classmates
against the admissions tiebreaker
for a subset of applicants
to any one of Chicago's nine exam schools.
Applicants to these schools rank up to 6,
while the exam schools prioritize their applicants
using a common composite index,
formed from an admissions test,
GPAs, and grade 7 standardized scores.
This composite tiebreaker is the running variable
for an RD design that reveals what happens
when any applicant is offered
an exam school seat.
In Chicago's exam school match,
which is actually an application
of the celebrated Gale and Shapley matching algorithm
exam school applicants are sure to be offered a seat somewhere
when they clear the lowest in their set of cutoffs
among the schools they rank.
We call this lowest cutoff the "qualifying cutoff."
The figure shows a sharp jump in peer mean achievement
for Chicago exam school applicants
who clear their qualifying cutoff.
This jump reflects the fact
that most applicants offered an exam school seat take it,
and applicants who enroll
at one of Chicago's selective enrollment high schools
are sure to be seated in a 9th grade classroom
filled with academically precocious peers,
because only the relatively precocious make it in.
The increase in peer achievement across the qualifying cutoff
amounts to almost half a standard deviation --
a very large effect.
And yet, precocious peers, notwithstanding,
the offer of an exam school seat does not appear to increase learning.
Let's plot applicants ACT scores against their tiebreaker values.
This plot shows that exam school applicants
who clear their qualifying cutoff
perform sharply worse on the ACT.
What explains this?
It takes a tale of IV and RD to untangle the forces
behind this intriguing and unexpected negative effect.
But first, some IV theory,
Guido Imbens and I developed theoretical tools
that enhance economists' understanding of empirical strategies
involving IV and RD.
The prize we share is in recognition of this work.
Guido and I overlapped for only one year at Harvard,
where we had both taken our first jobs post PhD.
I welcomed Guido to Cambridge, Massachusetts
with a pair of interesting instrumental variables.
I had used the draft lottery instrument
in my PhD thesis
to estimate the long-run economic consequences
of serving in the Armed Forces
for soldiers who were drafted.
The draft lottery instrument relies on the fact
that lottery numbers randomly assigned to birthdays
determined Vietnam-era conscription risk.
Yet, even then, most soldiers were volunteers,
as they are today,
The quarter birth instrument
is used in my 1991 paper with Alan Krueger
to estimate the economic returns to schooling.
This instrument uses the fact
that men who are born earlier in the year
are allowed to drop out of high school
on their 16th birthday
with less schooling completed than those born later.
Guido and I soon began asking each other,
"What really do we learn from the draft eligibility
and quarter of birth natural experiments?"
An early result in our quest for a new understanding of IV
was a solution to the problem of selection bias
in an RCT with partial compliance.
Even in a randomized clinical trial,
some of the people assigned to treatment may opt out.
This fact has long vexed trialists
because decisions to opt out are not made by random assignment.
Our first manuscript together
shows that in a randomized trial with partial compliance,
you can use IV
to estimate the effect of treatment on the treated,
even when some offered treatment decline it.
This works in spite of the fact
that those who comply with treatment
may be a very select group.
Unfortunately, for us, we were late to the party.
Not long after releasing our first working paper,
we learned of a concise contribution from Howard Bloom
that includes this theoretical result.
Remarkably, Bloom had derived this from first principles
without making a connection to IV.
So Guido and I went back to the drawing board.
And a few months later, we had LATE --
a theorem showing how to estimate
the local average treatment effect.
The LATE theorem generalizes the Bloom theorem
and establishes the connection between compliance and IV.
Maintaining the clinical trials analogy,
let "Zi" indicate whether subject "i" is offered treatment.
This is randomly assigned.
Also, let "D1i" indicate subject "i's" treatment status
when assigned to treatment,
and let the "0i" indicate subject "i's" treatment status
when assigned to control.
I'll use this formal notation
to give a clear statement of the late result,
and then follow up with examples.
A key piece of the late framework
pioneered by statistician Don Rubin,
is the pair of potential outcomes.
As is customary,
I denote potential outcomes for subject "i"
in the treated and untreated states
by "Y1i" and "Y0i" respectively,
The observed outcome is "Y1i" for the treated
and "Y0i" for those not treated.
"Y1i" minus "Y0i"
is the causal effect of treatment on individual "i",
but we can never see.
We try, therefore, to estimate some kind of average causal effect.
The late framework allows us to do that in an RCT
where some controls are treated.
The theorem says
that the average causal effect on people,
whose treatment status can be changed
by the offer of treatment
is the ratio of ITT to the treatment control difference
in compliance rates.
A mathematical statement of this result appears here,
where Greek letter Delta symbolizes the ITT effect
and Greek symbols pi1 and pi0
are compliance rates in the group assigned to treatment
and the group assigned to control, respectively,
The print version of this lecture
delves deeper into LATE intellectual history,
highlighting key contributions made with Rubin.
For now, though, I'd like to make the late theorem concrete for you
by sharing one of my favorite applications of it.
I'll explain the late framework
through a research question that has fascinated me
for almost two decades.
What is the causal effect
of charter school attendance on learning?
Charter schools are public schools
that operate independently
of traditional American public school districts.
A charter, the right to operate a public school
is typically awarded for a limited period
subject to renewal, conditional on good school performance.
Charter schools are free
to structure their curriculum and school environment.
The most controversial difference
between charters and traditional public schools
is the fact that the teachers and staff who work at charter schools
rarely belonged to labor unions.
By contrast, most big city public school teachers
work under union contracts.
The 2010 documentary film "Waiting for Superman"
feature schools belonging to the Knowledge is Power Program,
KIPP.
KIPP schools are emblematic of the high expectations,
sometimes also called "no excuses" approach to public education.
The "no excuses" model features
a long school day and extended school year,
selective teacher hiring
and focuses on traditional reading and math skills.
The American debate over education reform
often focuses on the achievement gap --
that's shorthand for large test score differences
by race and ethnicity.
Because of its focus on minority students,
KIPP is often central in this debate
with supporters pointing to the fact
that non-White KIPP students have markedly higher test scores
than non-White students from nearby schools.
KIPP skeptics on the other hand,
argue that KIPP's apparent success,
reflects the fact that KIPP attracts families
whose children would be more likely to succeed, anyway.
Who's right?
As you've probably guessed by now,
a randomized trial might prove decisive
in the debate over schools like KIPP.
Like Nobel Prize is, though,
seats at KIPP are not randomly assigned.
Well, at least, not entirely.
In fact,
Massachusetts charter schools with more applicants than seats
must offer their seats by lottery.
Sounds like a good natural experiment.
A little over a decade ago,
my collaborators and I collected data
on KIPP admissions lotteries
laying the foundation for two pioneering charter studies,
the first to use lotteries to study KIPP.
Our KIPP analysis is a classic IV story
because many students offered a seat in the KIPP lottery
failed to show up in the fall,
while a few not offered a seat, nevertheless, find their way in.
This graphic shows KIPP middle school applicants math scores
one year after applying to KIPP.
The entries above the line
show that Kip applicants who were offered a seat
have standardized math scores close to zero --
that is near the state average.
As before, we're working with standardized score data
that has a mean of 0 and a standard deviation of 1.
because KIPP applicants start with 4th grade scores
that are roughly .3 standard deviations below the state mean,
achievement at the level of the state average is impressive.
By contrast, the average math score among those not offered a seat
is about minus .36 sigma,
that is, .36 standard deviations below the state mean --
a result typical for urban students in Massachusetts.
Since lottery offers are randomly assigned,
we could say with confidence that the offer of a seated KIPP
boost math scores by
an average of .36 sigma,
a large effect that's also statistically precise.
We can be confident this is.n't a chance finding
What does an offer effect .36 sigma
tell us about the effects of actually going to KIPP?
IV methods convert KIPP offer effects into KIPP attendance effects.
I'll use this brief clip from my Marginal Revolution University short course
to quickly review the key assumptions behind this conversion.
- [Narrator] IV describes a chain reaction.
Why do offers affect achievement?
Probably because they affect charter attendance
and charter attendance improves math scores.
The first link in the chain called the "First Stage"
is the effect of the lottery on charter attendance.
The "Second Stage" is the link
between attending a charter and an outcome variable --
in this case, math scores.
The instrumental variable or "Instrument," for short,
is the variable that initiates the chain reaction.
The effect of the instrument on the outcome
is called the "Reduced Form."
This chain reaction can be represented mathematically.
We multiply the first stage -- the effect of winning on attendance,
by the second stage -- the effect of attendance on scores,
and we get the reduced form --
the effect of winning the lottery on scores.
The Reduced Form and First Stage are observable and easy to compute.
However, the effect of attendance on achievement
is not directly observed.
This is the causal effect. we're trying to determine
Given some important assumptions will discuss shortly,
we can find the effect of KIPP attendance
by dividing the reduced form by the first stage.
- [Joshua] IV eliminates selection bias,
but like all of our tools,
the solution builds on a set of assumptions
not to be taken for granted
first. There must be a substantial first stage.
That is the instrumental variable,
winning or losing the lottery must really change.
The variable, whose effect we're interested in here, KIPP attendance.
In this case. The first stage is
not really in doubt, winning the lottery, make skip attendance, much more likely,
not all IV, stories are like that.
S the instrument must be as good as randomly.
Signed meaning lottery winners and losers have similar characteristics.
This is the independence Assumption.
Of course,
KIPP lottery wins really are randomly assigned
still, we should check for balance and confirm that winners and losers
have similar family, backgrounds, similar, aptitudes, and so on,
in essence, we're checking to ensure KIPP lotteries are fair
with no group of applicants, suspiciously, likely to win.
Finally, we require the instrument change outcomes soul.
Through the variable of interest. In this case, attending camp,
this assumption is called the exclusion restriction.
The causal effect of KIPP attendance can therefore be written as the ratio of
the effect of offers on scores in the numerator,
over the effect of offers on KIPP and Roman in the denominator.
The numerator in this IV formula,
that is the direct effect of the instrument on outcomes. Has a special name.
This is called the reduced form. The denominator is the first step.
Stage
the exclusion restriction is often the trickiest or most controversial part of
an IV story. Here, the exclusion restriction amounts to the claim
that the .36 score, differential between lottery. Winners and losers
is entirely attributable to the .74, win-loss difference in attendance rates.
Plugging in the numbers.
The effect of KIPP attendance works out to be point four eight Sigma,
almost half a standard deviation gain in math scores. That's a remarkably large.
Perfect, who exactly benefits. So spectacularly from KIPP,
does everyone who applies to KIPP, see such large gains,
late answers. This question.
The late interpretation of the KIPP IV, empirical strategy,
is illuminated by the biblical story of Passover,
which explains that there are four types of children
each with characteristic behaviors
to keep track of these children and their behavior.
I'll give them a literate of names. Applicants like
Alvaro are dying to go to KIPP. If Alvaro loses, the KIPP lottery.
His mother finds a way to enroll him in KIPP. Anyway,
perhaps by reapplying applicants like Camilla are happy to go to Camp if they win
a seat in the lottery, but stoically accept the verdict, if they lose finally,
applicants, like normando worried about long days and lots of homework at KIPP
normando doesn't really want to go and refuses to go to KIPP when told that he won
the lottery.
That was called a never taker because win or lose. He doesn't go to KIPP
at the other end of KIPP commitment. Alvaro is called an always taker.
He'll happily take a seat. Went offered while his mother simply finds a way
to make it happen for him.
Even when he loses for Alvaro and normando. Both choice of school.
KIPP traditional is unaffected by the lottery.
Camilla is the type of applicant who gives IV its power
the instrument determines her treatment status.
I IV strategies depend on applicants, like, Camilla who are called compliers.
This term comes from the world of randomized, Trials, introduced earlier
as we've already discussed,
many randomized, trials, randomize only the opportunity to be treated
while the decision to comply with the treatment remains voluntary and non-random.
RCT. Compliers are those who take treatment when the offer of treatment is made?
But not otherwise with Lottery instruments.
Late is the effect of KIPP attendance on Camilla and other compliers like her.
Who enroll at KIPP take treatment when offered treatment through the lottery.
But not otherwise
IV methods are uninformed of for always takers like Alvaro and never takers
like normando because the instrument is unrelated to their treatment status.
Hey, didn't I say there are four types of children.
A fourth type of child in IV Theory behaves perversely every family has one.
These defiant children and Roland KIPP only when they lose the lottery.
Actually, the late theorem requires us to assume there are few defiers,
that seems like a reasonable assumption for Charter Lottery instruments.
If not in life.
The late theorem is sometimes seen as limiting the relevance
of econometric estimates
because it focuses attention on groups of compliers
yet. The population of compliers is a group. We'd very much like to learn about
in the KIPP example, compliers our children, likely to be,
Drawn into KIPP where the school to expand and offer additional seats in a lottery.
How relevant is this a few years ago,
Massachusetts indeed allowed thriving charter schools to expand
a recent study by some of my lab mates
shows that late estimates like the one we just computed for KIPP
predict learning gains at the schools created by Charter expansion.
Late isn't just a theorem.
It's a
framework.
The late framework can be used to estimate the entire distribution
of potential outcomes for compliers
as if we really did have a randomized trial for this group.
Although the theory behind this fact is necessarily technical.
The value of the framework is easily appreciated in practice by way of illustration.
Recall that the KIPP study
is motivated in part by differences in test scores by race.
Let's look at the distribution of 4th grade scores separately by.
Race for applicants to Boston Charter, Middle Schools,
the two sides of this figure show distributions for treated
and untreated compliers treated. Compliers are compliers offered.
A charter seat in a lottery. While untreated compliers are not offered a seat
because these are 4th grade scores while middle school begins in 5th
or 6th grade. The two sides of the figure are similar.
Both sides show score distributions for black applicants shifted to the left
of the corresponding.
Gorgeous tribulations for Whites by 8th grade.
Treated compliers have completed Middle School at a Boston Charter.
Well, I'm treated compliers have remained in traditional Public School.
Remarkably.
This next graphic shows that the eighth grade score distributions
of black-and-white treated. Compliers are indistinguishable.
Boston charter middle schools, close the achievement Gap,
but for the untreated black and white score distributions remained distinct
with black students.
Hind white students as they were in 4th grade,
Boston Charters,
close the achievement Gap because those who enter Charter Schools,
the farthest behind tend to gain the most from Charter. Enrollment
I elaborate on this point in the print version of this talk.
Remember the puzzle of - Chicago exam School effects.
I'll finish the scientific part of my talk by using IV and RD to explain this.
This surprising finding
the resolution of this puzzle starts with the fact that economic reasoning
is about Alternatives.
So what's the alternative to an exam school education
for most applicants to Chicago exam schools.
The leading non exam. Alternative is a traditional public school,
but many of Chicago's rejected exam school applicants enroll in a charter school
exam school offers. Therefore reduce the likelihood of Charter School attendance.
Specifically exam schools divert applicants away from high schools
in the noble network of charter schools.
Noble
with pedagogy much, like KIPP is one of Chicago's most visible Charter providers.
Also like KIPP convincing evidence on Noble Effectiveness comes from admissions,
lotteries.
The x-axis in this graphic shows Lottery offer effects on years enrolled at Noble.
This is the noble first stage.
For an IV setup that uses a dummy.
Indicating Noble Lottery offers as an instrument for Noble enrollment.
Now, this graphic has a feature that distinguishes.
It from the simpler KIPP analysis, the plot shows. First stage effects.
For two groups.
One for Noble applicants who live
in Chicago's lowest income neighborhoods Tier 1 and 1/4 Noble,
applicants who live in higher, income areas, tier 3, remember the IV chain.
Each point in this graphic has coordinates given by first stage reduced form
and therefore implies an IV estimate.
The effect of noble enrollment on ACT scores is the ratio of reduced form coordinate
to First Stage coordinate.
The graphic shows to such ratios.
The relevant results for Tier 1 are .35.
While for tier 3. We have .33 not bad for Noble applicants from both tears.
These
stage in reduced form estimates
imply a yearly Noble and Roman effective about a third of a standard deviation gain,
in act math scores.
Notice. There's also a line connecting the two Ivy estimates in the figure
because this line passes through the origin,
it's slope rise. Over run is about equal to the to IV estimates. In this case.
The slope is about Point 3 for
the fact that the line passes through 0 0 is significant for
Reason by this fact, we've substantiated the exclusion restriction,
specifically, the exclusion restriction,
says that given a group for which Noble offers are unrelated to Noble enrollment.
We should expect to see Zero reduced form effect of these offers made to applicants
in that group.
How consistent is the evidence for a noble cause learning gain on the order
of point, three, four Sigma per year in this next graphic? We've added
L've more points to the original to
the red points here, show, first stage in reduced form, Noble offer effects
for 12, additional groups to more tears,
and 12 groups, defined by demographic, characteristics related to race sex,
family, income, and Baseline scores.
Although not a perfect fit
these points cluster around a line with slope Point Three, Six Sigma
much like the line. We saw earlier for applicants from tiers 1 and 3.
You're likely now.
Ring. What the noble IV estimates in this figure have to do
with exam School enrollment.
Here's the answer,
the Blue Line in this new graphic shows. As we should expect
that exam. School exposure jumps up
for applicants who clear their qualifying cutoff. At the same time.
The Red Line shows that Noble School enrollment clearly Falls at the same point.
This is the diversion effect of exam school offers on Noble enrollment.
Many kids offered an exam School seat, prefer that exam School seat,
to enrollment at Noble IV, affords us, the opportunity to go out on a limb
with strong claims about the mechanism behind the causal effect.
Here's a strong causal, claim Regarding why Chicago exam schools reduce achievement.
The primary force driving reduced form exam School.
Qualification effects on ACT scores.
I claim is the effect of exam school offers on Noble and
Vomit in support of this claim,
consider the points plotted here in blue all well to the left of 0 on the x-axis.
These points are negative because they Mark the effect of exam School qualification
on Noble School, enrollment for particular, groups of applicants.
Now, we've already seen
that Noble applicants offered a noble seen
realize large act math, gains as a result.
Now consider exam school offers as
An instrument for Noble enrollment,
as always, Ivy is a chain reaction.
If exam School qualification reduces time at Noble, by Point 37 years,
and each year of noble, enrollment
boosts act math scores by about Point Three, Six Sigma.
We should expect reduced form effects of exam School. Qualification
to reduce ACT scores by the product of these two numbers.
That is by about Point 1, 3 Sigma
the reduced form qualification effects at the left of the
Here are broadly consistent with this.
They cluster closer to - .16. Then 2 minus Point 1 3,
but that difference is well within the sampling variance
of the underlying estimates.
The causal story told here,
postulates diversion away from Charter Schools
as the mechanism by which exam school offers effect achievement.
In other words, it's Noble enrollment,
that's presumed to satisfy an exclusion restriction.
When we use exam school offers as an
Variable importantly,
as we saw before the line in this final graphic, with two sets of 14 points,
runs through the origin.
This fact supports our new exclusion restriction
for any applicant group
for which exam school offers have little or no effect on Noble School enrollment.
We should also see ACT scores unchanged
at the same time because the blue and red dots cluster around the same line,
the IV estimates of noble School, enrollment effects generated.
I both Noble and exam school offers are about the same.
I hope this empirical story convinces, you of the power of IV and RD
to generate new causal, knowledge for decades.
I've been lucky to work on many equally engaging empirical problems.
I computed the draft lottery IV estimates in
my Princeton PhD thesis on a big hairy Mainframe monster.
Using nine track tapes, and Lease space on a communal. Hard drive,
Princeton graduate students learn to mount and manipulate tape.
Reels the size of a cheesecake.
Thankfully empirical work today is a little less labor-intensive.
What else is improved in the modern empirical era
in a 2010 article, Steve Pischke, and I coined the phrase.
He Revolution
by this, we mean economic shift towards transparent empirical strategies.
Applied to concrete, causal questions,
like the questions. David Carter studied. So convincingly.
The econometrics of my schooldays focused more on models than on questions.
The modeling concerns of that era have mostly faded
but econometricians have since found much to contribute.
I'll save my personal lists of greatest hits and exciting new artists for the
The print version of this lecture.
I'll wrap up here by saying that I'm proud to be part of the Contemporary.
Empirical economics, Enterprise
and I'm gratified beyond words
to have been recognized for contributing to it back at Princeton in the late 80s.
My graduate school classmates and I chuckled leading Ed lemurs. Lament,
that no Economist takes another economists. Empirical work. Seriously.
This is no longer, true, empirical work today aspires to tell convincing.
Also stories, not that every effort succeeds far from it.
But as any economics job market candidate will tell you,
empirical work carefully executed and clearly explained is taken seriously. Indeed.
That is a measure of our Enterprises success.
If you'd like to learn more from Josh,
check out his free course mastering econometrics if you'd like to
Spore, Josh's research,
check out the links in the description or you can click to watch more
of Josh's videos.