♪ [music] ♪
- [Narrator] Welcome to Nobel Conversations.
In this episode,
Josh Angrist and Guido Imbens sit down with Isaiah Andrews
to discuss how the field of econometrics is evolving.
- [Isaiah] So, Guido and Josh, you're both pioneers
in developing tools for empirical research in economics.
And so I'd like to explore
where you feel like the field is heading --
economics, econometrics, the whole thing.
To start, I'd be interested to hear
about whether you feel the way in which
the local average treatment effects framework took hold
has any lessons for how new empirical methods in economics
develop and spread or how they should.
- [Josh] That's a good question.
You go first.
[laughter]
- [Guido] Yeah, so I think the important thing
is to come up with good convincing cases
where the questions are clear
and where the methods apply in general.
One thing I...
Looking back at the subsequent literature.
I really like the regression discontinuity literature
where there were clearly a bunch of really convincing examples
and that allowed people to think more clearly,
look harder at the methodological questions.
Do clear applications
that then allow you to kind of think about,
"Wow, do these type of assumptions seem reasonable here?
What kind of things do we not like in the early papers?
How can we improve things?"
So having clear applications motivating these literatures
I think is very helpful.
- I'm glad you mentioned
the regression discontinuity, Guido.
I think there's a lot of complementarity between IV and RD,
instrumental variables and regression discontinuity.
A lot of the econometric applications
of regression discontinuity
are what used to be called "fuzzy" RD,
where it's not discrete or deterministic at the cutoff
but just the change in rates or intensity.
And the LATE framework helps us understand
those applications
and gives us a clear interpretation
for something like in my paper with Victor Lavy,
where we use Maimonides' rule, the class size cutoffs,
and what are you getting there?
Of course, you can answer that question
with a linear constant effects model,
but it turns out we're not limited to that,
and RD is still very powerful and illuminating,
even when the correlation
between the cutoff and the variable of interest,
in this case, class size, is partial,
maybe even not that strong.
So there was definitely a parallel development.
It's also interesting...
Nobody talked about regression discontinuity designs
when we were in graduate school --
it was something
that other social scientists were interested in,
and that grew up alongside the LATE framework,
and we've both done work
on both applications and methods there,
and it's been very exciting to see that develop
and become so important.
It's part of a general evolution,
I think, towards credible identification strategies,
causal effects,
making econometrics
more about causal questions than about models.
In terms of the future,
I think one thing that LATE has helped facilitate
is a move towards more creative, randomized trials
where there's something of interest.
It's not possible or straightforward
to simply turn it off or on,
but you can encourage it or discourage it.
So you subsidize schooling with financial aid, for example.
So now we have a whole framework for interpreting that,
and it opens the doors to randomized trials
of things that maybe would
not have seemed possible before.
We've used that a lot in the work we do on schools in our --
in the Blueprint Lab at MIT.
We're exploiting random assignment in very creative ways, I think.
- [Isaiah] Related to that, do you see particular factors
that make for useful research in econometrics?
You've alluded to it having a clear connection
to problems that are actually coming up,
and empirical practice is often a good idea.
- Isn't it always a good idea?
I often find myself sitting in an econometrics theory seminar,
say the Harvard MIT seminar,
and I'm thinking, "What problem is this guy solving?
Who has this problem?"
And, you know...
sometimes there's an embarrassing silence if I ask
or there might be a fairly contrived scenario.
I want to see where the tool is useful.
There are some purely foundational tools --
I do take the point.
There are people who are working on conceptual foundations of...
it becomes more like mathematical statistics.
I mean, I remember an early example of that
that I struggled to understand
was the idea of stochastic equicontinuity,
which one of my thesis advisors, Whitney Newey,
was using to great effect,
and I was trying to understand that.
It's really foundational.
it's not an application that's driving that --
at least not immediately.
But most things are not like that, and so there should be a problem.
And I think it's on the seller of that sort of thing,
because there's opportunity cost,
the time and attention, and effort to understand things.
It's on the seller to say,
"Hey, I'm solving this problem,
and here's a set of results that show that it's useful,
and here's some insight that I get."
- [Isaiah] As you said, Josh, there's been a move
in the direction of thinking more about causality
in economics and empirical work in economics.
Any consequences of the spread of that view that surprised you
or anything that you view as downsides
of the way that empirical economics has gone?
- Sometimes I see somebody does IV,
and they get a result which seems implausibly large --
that's the usual case.
So it might be an extraordinarily large
causal effect of some relatively minor intervention,
which was randomized or for which you could make a case
that there's a good design.
And then when I see that,
I think it's very hard for me to believe
that this relatively minor intervention
has such a large effect.
The author will sometimes resort
to the local average treatment effects theorem
and say, "Well, these compliers --
they're special in some way,
and they just benefit extraordinarily
from this intervention."
I'm reluctant to take that at face value.
I think often when effects are too big,
it's because the exclusion restriction is failing,
so you don't really have the right endogenous variable
to scale that result.
And so, I'm not too happy to see
a generic heterogeneity argument
being used to excuse something
that I think might be a deeper problem.
- [Guido] I think it played somewhat of an unfortunate role
in the discussions between reduced form
and structural approaches,
where I feel that wasn't quite right.
The instrumental variables assumptions
are at the core,
structural assumptions about behavior --
they were coming from economic...
thinking about the economic behavior of agents,
and somehow it got pushed in a direction
that I think wasn't really very helpful.
I think, initially,
we wrote things up,
it was describing what was happening.
There were a set of methods people were using.
We clarified what those methods were doing
in a way that I think contain a fair amount of insight.
But it somehow got pushed into a corner
that I don't think was necessarily very helpful.
- In just the language of reduced form versus structural,
I find kind of funny in the sense
that the local average treatment effect model,
the potential outcomes model
is a nonparametric structural model,
if you want to think about it, as you suggested, Guido.
So there's something a little funny
about putting these two things in opposition when --
- [Guido] Yes. - [Josh] That language, of course,
comes from the simultaneous equations framework
that we inherited.
It has the advantage
that people seem to know what you mean when you use it,
but that might be
that different people are hearing different things.
- [Guido] Yeah. I think reduced form has become
used in a little bit of the pejorative way...
- [Josh] Sometimes.
...which is not really quite what it was originally intended for.
- [Isaiah] I guess something else that strikes me in thinking about
the effects of the local average treatment effect framework
is that often folks will appeal
to a local average treatment effects intuition
for settings well beyond ones
where any sort of formal result has actually been established.
And I'm curious, given all the work that you guys did
to establish LATE results in different settings,
I'm curious, any thoughts on that?
- I think there's going to be a lot of cases
where the intuition does get you some distance,
but it's going to be somewhat limited,
and establishing formal results there
may be a little tricky
and then maybe only work in special circumstances,
and you end up with a lot of formality
that may not quite capture the intuition.
Sometimes I'm somewhat uneasy with them,
and they are not necessarily the papers I would want to write,
but I do think intuition
often does capture part of the problem.
I think, in some sense, we were very fortunate there
in the way that the LATE paper got handled at the journal,
so that, actually, the editor, made it much shorter
and that allowed us to focus on very clear, crisp results.
There's a somewhat unfortunate tendency
in the econometrics literature
of having the papers get longer and longer.
- Well, you should be able to fix that, man.
- I'm trying to fix that. [laughter]
But I think this is an example where it's very clear
that having it be short is actually --
- You should have imposed that no paper
can be longer than the LATE paper.
- That... wow! That may be great.
- At least no theory, no theory paper.
- Yeah, and I think...
I'm trying very hard to get the papers to be shorter,
and I think there is a lot of value today
because it's often the second part of the paper
that doesn't actually get you much further
in understanding things,
and it does make things much harder to read.
It goes back to how I think econometrics should be done.
You should focus on --
It should be reasonably close to empirical problems.
They should be very clear problems.
But then often the theory doesn't need to be quite so long.
- [Josh] Yeah.
- I think things have gone a little off track.
- [Isaiah] A relatively recent change
has been a seeming big increase in demand
for people with econometrics causal effect estimation skills
in the tech sector.
I'm interested, do either of you have thoughts
of how that's going to interact
with the development of empirical methods
or empirical research in economics going forward?
- [Josh] Well, there's sort of a meta point,
which is there's this new kind of employer,
the Amazons and the Uber, and the TripAdvisor world,
and I think that's great.
I like to tell my students about that.
At MIT, we have a lot of computer science majors --
that's our biggest major.
I try to seduce some of those folks into economics by saying
you can go work for these companies
that people are very keen to work for
because the work seems exciting,
that the skills that you get in econometrics
are as good or better
than any competing discipline has to offer.
So you should at least take some econ --
take some econometrics and some econ.
I did a fun project with Uber
on labor supply of Uber drivers,
and it was very, very exciting to be part of that.
Plus, I got to drive for Uber for a while,
and I thought that was fun too.
I did not make enough
that I was tempted to give up my MIT job,
but I enjoyed the experience.
I see a potential challenge
to our model of graduate education here,
which is, if we're training people to go work at Amazon,
it's not clear why we should be paying
graduate stipends for that.
Why should the taxpayer effectively be subsidizing that.
Our graduate education in the US is generously subsidized,
even in private universities --
there's a lot of public money there.
And I think the traditional rationale for that is,
we were training educators and scholars,
and there's a great externality from the work that we do,
it's either the research externality,
or a teaching externality.
But if many of our students are going to work
in the private sector --
that's fine,
but maybe their employers should pay for them.
- For me, it's just so different
from people working for consulting firms.
It's not clear to me
that the number of jobs in academics has changed.
- I feel like this is a growing sector,
whereas consulting...
You're right to raise that.
It might be the same for consulting.
I'm placing more and more students in these businesses,
so it's on my mind, in a way,
that I've not been attentive to consulting jobs.
Consulting was always important,
and I think also there's some movement
from consulting back into research --
it's a little more fluid.
A lot of the work in both domains,
I have to say, it's not really different,
but people who are working in the tech sector
are doing things that are potentially of scientific interest,
but mostly it's hidden.
Then you really have to say,
why is the government paying for this?
- Although, to Guido's point,
I guess there's a data question here of
has the total [non-EC] for-profit sector employment
of econ Ph.D. program graduates increased
or has it just been a substitution from finance
and consulting towards tech?
- I may be reacting to something
that's not really happening.
- I've actually done some work
with some of these tech companies.
I don't disagree with Josh's point that we need to think
a little bit about the funding model
who is, in the end, paying for the graduate education.
But from a scientific perspective,
not only do these places have great data,
and nowadays they tend to be very careful with that
for privacy reasons,
but they also have great questions.
I find it very inspiring to listen to the people there
and see what kind of questions they have,
and often they're questions
that also come up outside of these companies.
I have a couple of papers with Raj Chetty and Susan Athey
where we look at ways of combining experimental data
and observational data.
Raj Chetty was interested in what is the effect
of early childhood programs on outcomes later in life,
not just on test scores but on earnings and stuff,
and we kind of developed methods
that would help you shed light on that under some...
in some settings,
and the same problems came up
in these tech company settings.
And so, from my perspective,
it's the same kind of --
I was talking to people doing empirical work.
I tried to kind of look at these specific problems
and then try to come up with more general problems,
reformulating the problems at a higher level
so that I can think about solutions that work in a range of settings.
And so from that perspective,
the interactions with the tech companies
are just very valuable and very useful.
We do have students now doing internships there
and then coming back and writing more interesting theses
as a result of their experiences there.
- [Narrator] If you'd like to watch more Nobel Conversations,
click here.
Or if you'd like to learn more about econometrics,
check out Josh's "Mastering Econometrics" series.
If you'd like to learn more about Guido, Josh, and Isaiah,
check out the links in the description.
♪ [music] ♪