How Is Econometrics Changing? (Josh Angrist, Guido Imbens, Isaiah Andrews)

♪ [music] ♪

- [Narrator] Welcome
to Nobel Conversations.

In this episode,

Josh Angrist and Guido Imbens
sit down with Isaiah Andrews

to discuss how the field
of econometrics is evolving.

- [Isaiah] So, Guido and Josh,
you're both pioneers

in developing tools for
empirical research in economics.

And so I'd like to explore

where you feel like
the field is heading --

economics, econometrics,
the whole thing.

To start, I'd be interested to hear

about whether you feel
the way in which

the local average treatment
effects framework took hold

has any lessons for how
new empirical methods in economics

develop and spread
or how they should.

- [Josh] That's a good question.

You go first.

[laughter]

- [Guido] Yeah, so I think
the important thing

is to come up
with good convincing cases

where the questions are clear

and where the methods
apply in general.

One thing I...

Looking back
at the subsequent literature.

I really like the regression
discontinuity literature

where there were clearly a bunch
of really convincing examples

and that allowed people
to think more clearly,

look harder
at the methodological questions.

Do clear applications

that then allow you
to kind of think about,

"Wow, do these type of assumptions
seem reasonable here?

What kind of things do we not like
in the early papers?

How can we improve things?"

So having clear applications
motivating these literatures

I think is very helpful.

- I'm glad you mentioned

the regression
discontinuity, Guido.

I think there's a lot of
complementarity between IV and RD,

instrumental variables
and regression discontinuity.

A lot of the econometric
applications

of regression discontinuity

are what used
to be called "fuzzy" RD,

where it's not discrete
or deterministic at the cutoff

but just the change
in rates or intensity.

And the LATE framework
helps us understand

those applications

and gives us a clear interpretation

for something like
in my paper with Victor Lavy,

where we use Maimonides'
rule, the class size cutoffs,

and what are you getting there?

Of course, you can
answer that question

with a linear
constant effects model,

but it turns out
we're not limited to that,

and RD is still very powerful
and illuminating,

even when the correlation

between the cutoff
and the variable of interest,

in this case, class size,
is partial,

maybe even not that strong.

So there was definitely
a parallel development.

It's also interesting...

Nobody talked about
regression discontinuity designs

when we were in graduate school --

it was something

that other social scientists
were interested in,

and that grew up
alongside the LATE framework,

and we've both done work

on both applications
and methods there,

and it's been very exciting
to see that develop

and become so important.

It's part of a general evolution,

I think, towards credible
identification strategies,

causal effects,

making econometrics

more about causal questions
than about models.

In terms of the future,

I think one thing that LATE
has helped facilitate

is a move towards
more creative, randomized trials

where there's
something of interest.

It's not possible
or straightforward

to simply turn it off or on,

but you can encourage it
or discourage it.

So you subsidize schooling
with financial aid, for example.

So now we have a whole
framework for interpreting that,

and it opens the doors
to randomized trials

of things that maybe would

not have seemed possible before.

We've used that a lot in the work
we do on schools in our --

in the Blueprint Lab at MIT.

We're exploiting random assignment
in very creative ways, I think.

- [Isaiah] Related to that,
do you see particular factors

that make for useful research
in econometrics?

You've alluded to it having
a clear connection

to problems
that are actually coming up,

and empirical practice
is often a good idea.

- Isn't it always a good idea?

I often find myself sitting
in an econometrics theory seminar,

say the Harvard MIT seminar,

and I'm thinking, "What problem
is this guy solving?

Who has this problem?"

And, you know...

sometimes there's an
embarrassing silence if I ask

or there might be
a fairly contrived scenario.

I want to see
where the tool is useful.

There are some
purely foundational tools --

I do take the point.

There are people who are working
on conceptual foundations of...

it becomes more like
mathematical statistics.

I mean, I remember
an early example of that

that I struggled to understand

was the idea
of stochastic equicontinuity,

which one of my thesis advisors,
Whitney Newey,

was using to great effect,

and I was trying
to understand that.

It's really foundational.

it's not an application
that's driving that --

at least not immediately.

But most things are not like that,
and so there should be a problem.

And I think it's on the seller
of that sort of thing,

because there's opportunity cost,

the time and attention,
and effort to understand things.

It's on the seller to say,

"Hey, I'm solving this problem,

and here's a set of results
that show that it's useful,

and here's some insight
that I get."

- [Isaiah] As you said, Josh,
there's been a move

in the direction of thinking
more about causality

in economics and empirical
work in economics.

Any consequences of the spread
of that view that surprised you

or anything that you view
as downsides

of the way that empirical
economics has gone?

- Sometimes I see
somebody does IV,

and they get a result
which seems implausibly large --

that's the usual case.

So it might be
an extraordinarily large

causal effect of some
relatively minor intervention,

which was randomized
or for which you could make a case

that there's a good design.

And then when I see that,

I think it's very hard
for me to believe

that this relatively
minor intervention

has such a large effect.

The author will sometimes resort

to the local average
treatment effects theorem

and say, "Well, these compliers --

they're special in some way,

and they just benefit
extraordinarily

from this intervention."

I'm reluctant to take that
at face value.

I think often when effects
are too big,

it's because the exclusion
restriction is failing,

so you don't really have the right
endogenous variable

to scale that result.

And so, I'm not too happy to see

a generic heterogeneity argument

being used to excuse something

that I think might be
a deeper problem.

- [Guido] I think it played
somewhat of an unfortunate role

in the discussions
between reduced form

and structural approaches,

where I feel
that wasn't quite right.

The instrumental
variables assumptions

are at the core,

structural assumptions
about behavior --

they were coming from economic...

thinking about the economic
behavior of agents,

and somehow it got pushed
in a direction

that I think wasn't
really very helpful.

I think, initially,

we wrote things up,

it was describing
what was happening.

There were a set of methods
people were using.

We clarified what
those methods were doing

in a way that I think contain
a fair amount of insight.

But it somehow
got pushed into a corner

that I don't think
was necessarily very helpful.

- In just the language
of reduced form versus structural,

I find kind of funny in the sense

that the local average
treatment effect model,

the potential outcomes model

is a nonparametric
structural model,

if you want to think about it,
as you suggested, Guido.

So there's something a little funny

about putting these
two things in opposition when --

- [Guido] Yes.
- [Josh] That language, of course,

comes from the simultaneous
equations framework

that we inherited.

It has the advantage

that people seem to know
what you mean when you use it,

but that might be

that different people
are hearing different things.

- [Guido] Yeah. I think
reduced form has become

used in a little bit
of the pejorative way...

- [Josh] Sometimes.

...which is not really quite what
it was originally intended for.

- [Isaiah] I guess something else
that strikes me in thinking about

the effects of the local average
treatment effect framework

is that often folks will appeal

to a local average treatment
effects intuition

for settings well beyond ones

where any sort of formal result
has actually been established.

And I'm curious, given all the work
that you guys did

to establish LATE results
in different settings,

I'm curious, any thoughts on that?

- I think there's going
to be a lot of cases

where the intuition
does get you some distance,

but it's going to be
somewhat limited,

and establishing
formal results there

may be a little tricky

and then maybe only work
in special circumstances,

and you end up
with a lot of formality

that may not quite
capture the intuition.

Sometimes I'm somewhat
uneasy with them,

and they are not necessarily
the papers I would want to write,

but I do think intuition

often does capture
part of the problem.

I think, in some sense,
we were very fortunate there

in the way that the LATE paper
got handled at the journal,

so that, actually, the editor,
made it much shorter

and that allowed us to focus
on very clear, crisp results.

There's a somewhat
unfortunate tendency

in the econometrics literature

of having the papers
get longer and longer.

- Well, you should be able
to fix that, man.

- I'm trying to fix that.
[laughter]

But I think this is an example
where it's very clear

that having it be short
is actually --

- You should have imposed
that no paper

can be longer than the LATE paper.

- That... wow! That may be great.

- At least no theory,
no theory paper.

- Yeah, and I think...

I'm trying very hard to get
the papers to be shorter,

and I think there is a lot
of value today

because it's often
the second part of the paper

that doesn't actually
get you much further

in understanding things,

and it does make things
much harder to read.

It goes back to how I think
econometrics should be done.

You should focus on --

It should be reasonably
close to empirical problems.

They should be very clear problems.

But then often the theory
doesn't need to be quite so long.

- [Josh] Yeah.

- I think things have gone
a little off track.

- [Isaiah] A relatively
recent change

has been a seeming
big increase in demand

for people with econometrics
causal effect estimation skills

in the tech sector.

I'm interested,
do either of you have thoughts

of how that's going to interact

with the development
of empirical methods

or empirical research
in economics going forward?

- [Josh] Well, there's
sort of a meta point,

which is there's this new
kind of employer,

the Amazons and the Uber,
and the TripAdvisor world,

and I think that's great.

I like to tell my students
about that.

At MIT, we have a lot
of computer science majors --

that's our biggest major.

I try to seduce some of those folks
into economics by saying

you can go work for these companies

that people
are very keen to work for

because the work seems exciting,

that the skills
that you get in econometrics

are as good or better

than any competing
discipline has to offer.

So you should at least
take some econ --

take some econometrics
and some econ.

I did a fun project with Uber

on labor supply of Uber drivers,

and it was very, very exciting
to be part of that.

Plus, I got to drive
for Uber for a while,

and I thought that was fun too.

I did not make enough

that I was tempted
to give up my MIT job,

but I enjoyed the experience.

I see a potential challenge

to our model
of graduate education here,

which is, if we're training people
to go work at Amazon,

it's not clear why
we should be paying

graduate stipends for that.

Why should the taxpayer
effectively be subsidizing that.

Our graduate education
in the US is generously subsidized,

even in private universities --

there's a lot
of public money there.

And I think the traditional
rationale for that is,

we were training
educators and scholars,

and there's a great externality
from the work that we do,

it's either
the research externality,

or a teaching externality.

But if many of our students
are going to work

in the private sector --

that's fine,

but maybe their employers
should pay for them.

- For me, it's just so different

from people working
for consulting firms.

It's not clear to me

that the number of jobs
in academics has changed.

- I feel like this is
a growing sector,

whereas consulting...

You're right to raise that.

It might be the same
for consulting.

I'm placing more and more
students in these businesses,

so it's on my mind, in a way,

that I've not been attentive
to consulting jobs.

Consulting was always important,

and I think also
there's some movement

from consulting back
into research --

it's a little more fluid.

A lot of the work in both domains,

I have to say,
it's not really different,

but people who are working
in the tech sector

are doing things that are
potentially of scientific interest,

but mostly it's hidden.

Then you really have to say,

why is the government
paying for this?

- Although, to Guido's point,

I guess there's
a data question here of

has the total [non-EC]
for-profit sector employment

of econ Ph.D. program
graduates increased

or has it just been
a substitution from finance

and consulting towards tech?

- I may be reacting to something

that's not really happening.

- I've actually done some work

with some of these tech companies.

I don't disagree with Josh's point
that we need to think

a little bit about
the funding model

who is, in the end, paying
for the graduate education.

But from a scientific perspective,

not only do these places
have great data,

and nowadays they tend to be
very careful with that

for privacy reasons,

but they also have great questions.

I find it very inspiring to listen
to the people there

and see what kind
of questions they have,

and often they're questions

that also come up
outside of these companies.

I have a couple of papers
with Raj Chetty and Susan Athey

where we look at ways
of combining experimental data

and observational data.

Raj Chetty was interested
in what is the effect

of early childhood programs
on outcomes later in life,

not just on test scores
but on earnings and stuff,

and we kind of developed methods

that would help you shed light
on that under some...

in some settings,

and the same problems came up

in these tech company settings.

And so, from my perspective,

it's the same kind of --

I was talking to people
doing empirical work.

I tried to kind of look
at these specific problems

and then try to come up
with more general problems,

reformulating the problems
at a higher level

so that I can think about solutions
that work in a range of settings.

And so from that perspective,

the interactions
with the tech companies

are just very valuable
and very useful.

We do have students now
doing internships there

and then coming back
and writing more interesting theses

as a result of
their experiences there.

- [Narrator] If you'd like to watch
more Nobel Conversations,

click here.

Or if you'd like to learn
more about econometrics,

check out Josh's
"Mastering Econometrics" series.

If you'd like to learn more
about Guido, Josh, and Isaiah,

check out the links
in the description.

♪ [music] ♪