WEBVTT

00:00:00.000 --> 00:00:02.550
♪ [music] ♪

00:00:03.800 --> 00:00:05.800
- [Narrator] Welcome 
to Nobel Conversations.

00:00:07.100 --> 00:00:08.100
In this episode,

00:00:08.100 --> 00:00:11.570
Josh Angrist and Guido Imbens
sit down with Isaiah Andrews

00:00:11.570 --> 00:00:14.600
to discuss how the field 
of econometrics is evolving.

00:00:16.100 --> 00:00:18.750
- [Isaiah] So Guido and Josh, 
you're both pioneers

00:00:18.750 --> 00:00:21.500
in developing tools for
empirical research in economics.

00:00:21.500 --> 00:00:22.930
And so I'd like to explore

00:00:22.930 --> 00:00:25.300
sort of where you feel like 
the field is heading,

00:00:25.300 --> 00:00:28.079
sort of economics, econometrics,
the whole thing.

00:00:28.510 --> 00:00:31.290
To start, I'd be interested to hear

00:00:32.200 --> 00:00:35.200
about whether you feel like
sort of the way in which

00:00:35.200 --> 00:00:38.510
the local average treatment 
effects framework sort of took hold

00:00:38.800 --> 00:00:42.100
has any lessons for how 
new empirical methods in economics

00:00:42.100 --> 00:00:44.300
develop and spread
or how they should.

00:00:44.560 --> 00:00:45.960
- [Josh] That's a good question.

00:00:46.610 --> 00:00:47.790
You go first.

00:00:47.790 --> 00:00:49.460
(laughter)

00:00:49.700 --> 00:00:52.940
Yeah, so I think 
the important thing

00:00:52.940 --> 00:00:58.550
is to come up 
with good convincing cases

00:00:58.550 --> 00:01:02.207
where the questions are clear

00:01:02.400 --> 00:01:05.720
and where kind of 
the methods apply in general.

00:01:05.720 --> 00:01:07.560
So one thing I --

00:01:08.070 --> 00:01:12.000
Kind of looking back
at the subsequent literature,

00:01:12.200 --> 00:01:16.700
so I really like the regression
discontinuity literature

00:01:16.700 --> 00:01:19.670
[where there were] clearly a bunch
of really convincing examples

00:01:19.670 --> 00:01:21.319
and that allowed people to kind of

00:01:22.300 --> 00:01:27.200
think more clearly, look harder 
at the methodological questions.

00:01:27.400 --> 00:01:28.800
Kind of do clear applications

00:01:28.800 --> 00:01:30.600
that then allow you 
to kind of think about,

00:01:30.600 --> 00:01:33.600
"Wow, do this type of assumption
seem reasonable here?

00:01:33.600 --> 00:01:38.000
What kind of things do we not like
in the early papers?

00:01:38.500 --> 00:01:39.802
How can we improve things?"

00:01:39.802 --> 00:01:44.210
So having clear applications 
motivating these literatures,

00:01:44.210 --> 00:01:46.400
I think it's very helpful.

00:01:46.800 --> 00:01:48.050
- [Josh] I'm glad you mentioned

00:01:48.050 --> 00:01:49.382
the regression 
discontinuity, Guido.

00:01:49.382 --> 00:01:53.300
I think there's a lot of 
complementarity between IV and RD,

00:01:54.700 --> 00:01:57.060
Instrumental Variables and
Regression Discontinuity.

00:02:00.350 --> 00:02:03.260
And a lot of 
the econometric applications

00:02:03.260 --> 00:02:04.520
of regression discontinuity

00:02:04.520 --> 00:02:07.230
are what used to be called 
"fuzzy" RD,

00:02:07.230 --> 00:02:11.620
where, you know, it's not discrete 
or deterministic at the cutoff,

00:02:11.620 --> 00:02:14.900
but just the change
in rates or intensity.

00:02:14.900 --> 00:02:18.740
And the late framework helps us
understand those applications

00:02:18.740 --> 00:02:21.140
and gives us a clear interpretation

00:02:21.140 --> 00:02:25.000
for say, something like, 
in my paper with Victor Lavy,

00:02:25.000 --> 00:02:28.100
where we use Maimonides'
rule, the class size cutoffs.

00:02:28.430 --> 00:02:30.030
What are you getting there?

00:02:30.290 --> 00:02:31.820
Of course, you can
answer that question

00:02:31.820 --> 00:02:33.900
with a linear 
constant effects model,

00:02:34.200 --> 00:02:36.310
but it turns out 
we're not limited to that,

00:02:36.310 --> 00:02:39.889
and RD is still very powerful 
and illuminating,

00:02:40.630 --> 00:02:42.100
even when, you know,

00:02:42.100 --> 00:02:45.670
the correlation between the cutoff
and the variable of interest,

00:02:45.670 --> 00:02:48.980
in this case class size, 
is partial,

00:02:48.980 --> 00:02:51.000
maybe even not that strong.

00:02:52.000 --> 00:02:54.999
So there was definitely a kind of,
a parallel development.

00:02:54.999 --> 00:02:56.400
It's also interesting,

00:02:56.600 --> 00:02:59.780
you know, nobody talked about
regression discontinuity designs

00:02:59.780 --> 00:03:01.220
when we were in graduate school,

00:03:01.220 --> 00:03:05.300
it was something that other social
scientists were interested in,

00:03:05.800 --> 00:03:09.507
and that kind of grew up 
alongside the LATE framework

00:03:09.507 --> 00:03:14.787
and we've both done work on 
both applications and methods there

00:03:14.787 --> 00:03:18.377
and it's been very exciting 
to see that kind of develop

00:03:18.377 --> 00:03:19.800
and become so important.

00:03:20.000 --> 00:03:23.700
It's part of a general evolution,
I think, towards, you know,

00:03:24.000 --> 00:03:27.700
credible identification strategies,
causal effects...

00:03:28.630 --> 00:03:30.200
less, you know, making econometrics

00:03:30.400 --> 00:03:33.300
more about causal questions
than about models.

00:03:33.640 --> 00:03:34.650
In terms of the future,

00:03:34.650 --> 00:03:37.860
I think one thing that LATE 
has helped facilitate

00:03:37.860 --> 00:03:42.700
is a move towards more creative,
randomized trials, where,

00:03:42.700 --> 00:03:44.400
you know, there's
something of interest,

00:03:45.500 --> 00:03:48.460
it's not possible 
or straightforward

00:03:48.460 --> 00:03:50.700
to simply turn it off or on,

00:03:51.000 --> 00:03:54.584
but you can encourage it 
or discourage it.

00:03:54.584 --> 00:03:58.200
So you subsidize schooling 
with financial aid, for example.

00:03:59.000 --> 00:04:02.080
So now we have a whole 
framework for interpreting that,

00:04:03.600 --> 00:04:06.900
and it kind of opens
the doors to randomized trials

00:04:06.900 --> 00:04:10.300
of things that maybe would,
you know,

00:04:10.300 --> 00:04:12.471
not have seem possible before.

00:04:14.500 --> 00:04:17.700
We've used that a lot in the work
we do on schools in our --

00:04:17.700 --> 00:04:21.160
in the Blueprint Lab at MIT,

00:04:22.360 --> 00:04:26.600
we're exploiting random assignment
and in very creative ways, I think.

00:04:28.100 --> 00:04:31.509
- [Isaiah] Related to that, do you 
see sort of particular factors

00:04:31.509 --> 00:04:34.300
that make for useful research
in econometrics?

00:04:34.400 --> 00:04:38.290
You've alluded to
it having a clear connection

00:04:38.290 --> 00:04:40.300
to problems 
that are actually coming up

00:04:40.300 --> 00:04:42.650
and empirical practice 
is often a good idea.

00:04:43.290 --> 00:04:45.000
- [Josh] Isn't it always 
a good idea?

00:04:45.700 --> 00:04:50.100
I often find myself sitting in an
econometrics theory seminar,

00:04:50.700 --> 00:04:52.500
say the Harvard MIT seminar,

00:04:53.400 --> 00:04:56.350
and I'm thinking, "What problem
is this guy solving?

00:04:56.350 --> 00:04:57.960
Who has this problem?"

00:04:57.960 --> 00:04:59.800
And, you know,

00:05:01.600 --> 00:05:04.700
sometimes there's an
embarrassing silence if I ask

00:05:04.900 --> 00:05:08.300
or there might be
a fairly contrived scenario.

00:05:08.800 --> 00:05:11.600
I want to see 
where the tool is useful.

00:05:12.500 --> 00:05:14.900
There are some 
purely foundational tools,

00:05:14.900 --> 00:05:17.600
I do take the point, you know, 
there are people who are

00:05:18.200 --> 00:05:22.500
working on conceptual
foundations of, you know,

00:05:22.600 --> 00:05:25.300
it's more -- becomes more like
mathematical statistics.

00:05:25.800 --> 00:05:28.200
I mean, I remember 
an early example of that that I,

00:05:28.200 --> 00:05:29.920
you know, I struggled to understand

00:05:29.920 --> 00:05:32.500
was the idea
of stochastic equicontinuity,

00:05:32.500 --> 00:05:35.070
which one of my thesis advisors,
Whitney Newey,

00:05:35.070 --> 00:05:36.900
was using to great effect

00:05:37.500 --> 00:05:39.900
and I was trying to understand that
and there isn't really --

00:05:40.600 --> 00:05:45.200
It's really foundational, it's not
an application that's driving that,

00:05:45.890 --> 00:05:47.300
at least not immediately.

00:05:48.600 --> 00:05:53.200
But most things are not like that
and so there should be a problem.

00:05:53.800 --> 00:05:59.100
And I think it's on the seller 
of that sort of thing,

00:06:00.100 --> 00:06:02.250
you know, because there's 
opportunity cost,

00:06:02.250 --> 00:06:05.170
the time and attention,
and effort to understand things

00:06:05.170 --> 00:06:07.200
to, you know, 
it's on the seller to say,

00:06:07.400 --> 00:06:08.900
"Hey, I'm solving this problem

00:06:09.400 --> 00:06:12.900
and here's a set of results
that show that it's useful,

00:06:12.900 --> 00:06:15.200
and here's some insight
that I get."

00:06:16.200 --> 00:06:18.280
- [Isaiah] As you said, Josh, great,
sort of there's been a move

00:06:18.280 --> 00:06:20.700
in the direction of thinking 
more about causality

00:06:20.700 --> 00:06:22.800
in economics and empirical
work in economics,

00:06:22.900 --> 00:06:24.800
any consequences of sort of the --

00:06:24.800 --> 00:06:26.570
the spread of that view 
that surprised you

00:06:26.570 --> 00:06:28.705
or anything that you view
as downsides

00:06:28.705 --> 00:06:31.400
of sort of the way 
that empirical economics has gone?

00:06:31.500 --> 00:06:34.190
- [Josh] Sometimes I see, 
somebody does IV

00:06:34.190 --> 00:06:38.304
and they get a result 
which seems implausibly large.

00:06:38.800 --> 00:06:40.200
That's the usual case.

00:06:42.500 --> 00:06:45.220
So it might be, you know,
an extraordinarily large

00:06:45.220 --> 00:06:48.600
causal effect of some 
relatively minor intervention,

00:06:49.100 --> 00:06:52.260
which was randomized
or for which you could make a case

00:06:52.260 --> 00:06:54.490
that there's a good design.

00:06:54.900 --> 00:06:58.330
And then when I see that,
and, you know, I think

00:06:58.900 --> 00:07:00.465
it's very hard for me to believe

00:07:00.465 --> 00:07:02.030
that this relatively 
minor intervention

00:07:02.030 --> 00:07:03.720
has such a large effect.

00:07:04.100 --> 00:07:06.110
The author will sometimes resort

00:07:06.110 --> 00:07:08.690
to the local average 
treatment effects theorem

00:07:08.690 --> 00:07:10.695
and say, "Well, these compliers,

00:07:10.695 --> 00:07:12.700
you know, they're
special in some way."

00:07:13.300 --> 00:07:15.800
And, you know, 
they just benefit extraordinarily

00:07:15.800 --> 00:07:17.600
from this intervention.

00:07:18.100 --> 00:07:20.900
And I'm reluctant 
to take that at face value.

00:07:20.900 --> 00:07:23.750
I think, you know,
often when effects are too big,

00:07:24.300 --> 00:07:26.780
it's because the exclusion
restriction is failing,

00:07:26.780 --> 00:07:29.240
so you don't really have the right
endogenous variable

00:07:29.240 --> 00:07:31.380
to scale that result.

00:07:32.000 --> 00:07:35.700
And so I'm not too happy to see

00:07:35.700 --> 00:07:38.800
you know, just sort of
a generic heterogeneity

00:07:38.900 --> 00:07:41.610
argument being used 
to excuse something

00:07:41.610 --> 00:07:43.800
that I think might be 
a deeper problem.

00:07:45.190 --> 00:07:47.358
- [Guido] I think it played 
somewhat of an unfortunate role

00:07:47.358 --> 00:07:49.979
when the discussions kind of 
between reduced form

00:07:49.979 --> 00:07:51.700
and structural approaches,

00:07:51.700 --> 00:07:55.510
where I feel 
that wasn't quite right.

00:07:56.090 --> 00:07:58.810
The instrumental 
variables assumptions

00:07:58.810 --> 00:08:03.622
are at the core - structural 
assumptions about behavior -

00:08:03.622 --> 00:08:05.200
they were coming from economic --

00:08:07.100 --> 00:08:09.900
thinking about the economic
behavior of agents,

00:08:10.300 --> 00:08:15.000
and somehow it got
pushed in a direction

00:08:15.100 --> 00:08:17.600
that I think wasn't 
really very helpful.

00:08:18.800 --> 00:08:21.700
The way I think, initially the --

00:08:22.800 --> 00:08:26.480
we wrote things up, it was 
describing what was happening,

00:08:26.480 --> 00:08:29.490
there were a set of methods 
people were using,

00:08:29.490 --> 00:08:32.111
we clarified what 
those methods were doing

00:08:32.811 --> 00:08:38.361
and in a way that I think
contain a fair amount of insight,

00:08:39.100 --> 00:08:42.050
but it somehow 
it got pushed into a corner

00:08:42.050 --> 00:08:45.000
that I don't think 
was necessarily very helpful.

00:08:45.100 --> 00:08:46.850
- [Isaiah] I mean, 
just the language

00:08:46.850 --> 00:08:48.600
of reduced form versus structural

00:08:48.600 --> 00:08:50.820
I find kind of funny 
in the sense that,

00:08:50.820 --> 00:08:53.100
right, the local average treatment
effect model, right,

00:08:53.100 --> 00:08:56.110
the potential outcomes model is 
a nonparametric structural model,

00:08:56.110 --> 00:08:58.600
if you want to think about it, 
as you sort of suggested, Guido.

00:08:58.600 --> 00:09:01.263
So there's something a little funny

00:09:01.263 --> 00:09:03.340
about putting these 
two things in oposition when --

00:09:03.340 --> 00:09:04.690
- [Guido] Yes.
- [Josh] Well, that language,

00:09:04.690 --> 00:09:08.165
of course, comes from 
the [inaudible] equations framework

00:09:08.165 --> 00:09:09.520
that we inherited.

00:09:10.400 --> 00:09:11.530
It has the advantage

00:09:11.530 --> 00:09:14.160
that people seem to know 
what you mean when you use it,

00:09:14.160 --> 00:09:16.240
but might be that people 
are hearing different,

00:09:16.240 --> 00:09:18.200
different people
are hearing different things.

00:09:18.300 --> 00:09:20.530
- [Guido] Yeah. I think [inaudible]
has sort of become --

00:09:20.530 --> 00:09:22.560
used in a little bit 
of the pejorative way, yeah?

00:09:22.560 --> 00:09:24.750
- [Josh] Sometimes.
- [Guido] [The word].

00:09:24.750 --> 00:09:28.250
Which is not really quite what 
it was originally intended for.

00:09:30.100 --> 00:09:33.090
- [Isaiah] I guess something else
that strikes me in thinking about

00:09:33.090 --> 00:09:35.645
the effects of the local average
treatment effect framework

00:09:35.645 --> 00:09:38.200
is that often folks will appeal to

00:09:38.200 --> 00:09:41.800
a local average treatment effects
intuition for settings well beyond

00:09:42.000 --> 00:09:43.700
ones where any sort 
of formal result

00:09:43.700 --> 00:09:45.440
has actually been established.

00:09:45.440 --> 00:09:50.090
And I'm curious, given all the work
that you guys did to, you know,

00:09:50.090 --> 00:09:52.390
establish late results 
in different settings,

00:09:52.390 --> 00:09:54.415
I'm curious, any thoughts on that?

00:09:55.360 --> 00:09:57.420
- [Guido] I think there's
going to be a lot of cases

00:09:57.420 --> 00:10:02.200
where the intuition 
does get you some distance,

00:10:02.800 --> 00:10:05.200
but it's going to be 
somewhat limited

00:10:05.200 --> 00:10:07.600
and establishing 
formal results there

00:10:08.400 --> 00:10:09.490
may be a little tricky

00:10:09.490 --> 00:10:12.700
and then maybe only work 
in special circumstances,

00:10:14.600 --> 00:10:16.540
and you end up
with a lot of formality

00:10:16.540 --> 00:10:19.500
that may not quite 
capture the intuition.

00:10:19.900 --> 00:10:21.550
Sometimes I'm somewhat 
uneasy with them

00:10:21.550 --> 00:10:24.438
and they are not necessarily
the papers I would want to write,

00:10:25.148 --> 00:10:27.218
but I do think something --

00:10:27.218 --> 00:10:31.217
intuition often does capture 
part of the problem.

00:10:33.100 --> 00:10:36.300
I think, in some sense we were
kind of very fortunate there

00:10:36.900 --> 00:10:39.250
in the way that the late paper 
got handled at the journal,

00:10:39.250 --> 00:10:41.766
is that, actually, the editor,
made it much shorter

00:10:42.100 --> 00:10:46.300
and that then allowed us to kind of
focus on very clear, crisp results.

00:10:47.100 --> 00:10:51.770
Where if you -- you know, this 
somewhat unfortunate tendency

00:10:51.770 --> 00:10:52.985
in the econometrics literature

00:10:52.985 --> 00:10:55.100
of having the papers 
get longer and longer.

00:10:55.100 --> 00:10:56.690
- [Josh] Well, you should 
be able to fix that, man.

00:10:56.690 --> 00:10:58.800
- [Guido] I'm trying to fix that.

00:10:59.400 --> 00:11:01.625
But I think this is an example 
where it's sort of very clear

00:11:01.625 --> 00:11:03.498
that having it be short
is actually --

00:11:03.498 --> 00:11:04.842
- [Josh] You should impose
that no paper

00:11:04.842 --> 00:11:06.655
can be longer than the late paper.

00:11:06.655 --> 00:11:08.000
- [Guido] That, wow.

00:11:08.000 --> 00:11:09.617
That may be great.

00:11:09.617 --> 00:11:11.685
- [Josh] At least no theory, 
no theory paper.

00:11:11.892 --> 00:11:14.300
- [Guido] Yeah, 
and I think, I think...

00:11:14.500 --> 00:11:16.800
I'm trying very hard to get
the papers to be shorter.

00:11:16.800 --> 00:11:18.700
And I think there's a lot of value

00:11:19.200 --> 00:11:21.480
today because it's often
the second part of the paper

00:11:21.480 --> 00:11:26.395
that doesn't actually get you much
further in understanding things

00:11:27.000 --> 00:11:29.870
and it does make things 
much harder to read

00:11:30.630 --> 00:11:33.200
and, you know, 
it sort of goes back

00:11:33.200 --> 00:11:36.111
to how I think econometrics 
should be done,

00:11:36.111 --> 00:11:38.070
you should focus on --

00:11:38.700 --> 00:11:41.300
It should be reasonably
close to empirical problems.

00:11:41.500 --> 00:11:43.900
They should be very clear problems.

00:11:44.800 --> 00:11:48.900
But then often the theory
doesn't need to be quite so long.

00:11:48.900 --> 00:11:50.010
- [Josh] Yeah.

00:11:51.100 --> 00:11:54.670
- [Guido] I think things have gone
a little off track.

00:11:56.260 --> 00:11:57.750
- [Isaiah] A new relatively 
recent change

00:11:57.750 --> 00:12:00.230
has been a seeming big 
increase in demand

00:12:00.230 --> 00:12:02.200
for people with sort of 
econometrics,

00:12:02.200 --> 00:12:04.800
causal effect estimation skills
in the tech sector.

00:12:05.000 --> 00:12:07.480
I'm interested, 
do either of you have thoughts

00:12:07.480 --> 00:12:09.840
on sort of how 
that's going to interact

00:12:09.840 --> 00:12:11.600
with the development 
of empirical methods,

00:12:11.600 --> 00:12:13.950
or empirical research
in economics going forward?

00:12:14.600 --> 00:12:16.770
- [Josh] Well, there's
sort of a meta point,

00:12:16.770 --> 00:12:21.000
which is, there's this new 
kind of employer,

00:12:21.800 --> 00:12:27.530
the Amazons and the Uber,
and, you know, TripAdvisor world,

00:12:28.000 --> 00:12:29.300
and I think that's great.

00:12:29.300 --> 00:12:32.600
And I like to tell my students
about that, you know, especially --

00:12:32.600 --> 00:12:35.500
at MIT we have a lot of 
computer science majors.

00:12:35.500 --> 00:12:37.000
That's our biggest major.

00:12:37.400 --> 00:12:42.246
And I try to seduce some of those
folks into economics by saying,

00:12:42.246 --> 00:12:45.700
you know, you can go 
work for these,

00:12:45.800 --> 00:12:49.250
you know, companies that people
are very keen to work for

00:12:49.250 --> 00:12:50.800
because the work seems exciting,

00:12:52.000 --> 00:12:54.250
you know, that the skills 
that you get in econometrics

00:12:54.250 --> 00:12:56.100
are as good or better

00:12:56.100 --> 00:12:59.736
than any competing 
discipline has to offer.

00:12:59.736 --> 00:13:01.100
So you should at least

00:13:01.400 --> 00:13:04.200
take some econ, take some
econometrics, and some econ.

00:13:04.800 --> 00:13:07.000
I did a fun project with Uber

00:13:07.600 --> 00:13:09.770
on labor supply of Uber drivers

00:13:09.920 --> 00:13:12.805
and was very, very exciting
to be part of that.

00:13:13.100 --> 00:13:15.400
Plus I got to drive 
for Uber for a while

00:13:15.900 --> 00:13:17.730
and I thought that was fun too.

00:13:17.730 --> 00:13:20.700
I did not make enough
that I was tempted to

00:13:21.100 --> 00:13:25.100
give up my MIT job, 
but I enjoyed the experience.

00:13:25.230 --> 00:13:30.900
I see a potential challenge to our
model of graduate education here,

00:13:31.700 --> 00:13:37.400
which is, if we're training people
to go work at Amazon, you know,

00:13:37.900 --> 00:13:41.190
it's not clear why, you know, 
we should be paying

00:13:41.190 --> 00:13:42.900
graduate stipends for that.

00:13:43.200 --> 00:13:47.280
Why should the taxpayer effectively
be subsidizing that.

00:13:47.280 --> 00:13:51.400
Our graduate education
in the US Is generously subsidized,

00:13:51.400 --> 00:13:53.160
even in private universities,

00:13:53.160 --> 00:13:56.100
it's ultimately -- there's a lot of
public money there,

00:13:56.100 --> 00:13:59.300
and I think the
traditional rationale for that is,

00:13:59.500 --> 00:14:02.137
you know, we were training 
educators and scholars,

00:14:02.137 --> 00:14:05.657
and there's a great externality
from the work that we do,

00:14:05.657 --> 00:14:07.607
it's either 
the research externality,

00:14:07.607 --> 00:14:09.557
or a teaching externality.

00:14:10.100 --> 00:14:12.350
But, you know, 
if many of our students

00:14:12.350 --> 00:14:14.600
are going to work
in the private sector,

00:14:16.300 --> 00:14:21.700
that's fine, but maybe their 
employers should pay for that.

00:14:22.120 --> 00:14:23.370
- [Guido] But maybe
is not so different

00:14:23.370 --> 00:14:26.780
from people working 
for consulting firms.

00:14:27.200 --> 00:14:28.780
It's not clear to me

00:14:28.780 --> 00:14:32.836
that the number of jobs
in academics has changed.

00:14:33.370 --> 00:14:36.325
- [Josh] I feel like this 
is a growing sector,

00:14:36.325 --> 00:14:39.289
whereas consulting -- 
you're right to raise that,

00:14:39.289 --> 00:14:42.100
it might be the same 
for consulting,

00:14:43.300 --> 00:14:44.846
but this, you know,

00:14:44.846 --> 00:14:47.500
I'm placing more and more 
students in these businesses.

00:14:47.500 --> 00:14:50.400
So, it's on my mind 
in a way that I've sort of,

00:14:50.800 --> 00:14:53.960
you know, not been attentive 
to consulting jobs,

00:14:53.960 --> 00:14:56.920
you know, consulting 
was always important,

00:14:56.920 --> 00:14:58.950
and I think also 
there's some movement

00:14:58.950 --> 00:15:02.600
from consulting back into research,
it's a little more fluid.

00:15:03.900 --> 00:15:07.630
A lot of the work in both domains

00:15:07.630 --> 00:15:09.430
I have to say, 
it's not really different

00:15:09.430 --> 00:15:12.730
but, you know, people who 
are working in the tech sector

00:15:12.730 --> 00:15:15.480
are doing things that are 
potentially of scientific interest,

00:15:15.480 --> 00:15:16.800
but mostly it's hidden.

00:15:17.100 --> 00:15:18.550
Then you really I have to say,

00:15:18.550 --> 00:15:20.900
you know, why is the government
paying for this?

00:15:21.800 --> 00:15:23.732
Yeah, although, yeah, 
I mean to Guidos point,

00:15:23.732 --> 00:15:26.102
I guess there's 
a data question here

00:15:26.102 --> 00:15:30.042
of it has the sort of 
total [no-neck] sort of say

00:15:31.300 --> 00:15:34.870
for-profit sector employment 
of econ Ph.D. program graduates

00:15:34.870 --> 00:15:38.290
increased or has it just been
a substitution from finance

00:15:38.290 --> 00:15:40.200
and consulting towards tech.

00:15:40.300 --> 00:15:42.300
- [Josh] I may be reacting 
to something

00:15:42.300 --> 00:15:44.300
that's not really happening.

00:15:44.400 --> 00:15:45.890
- [Guido] I've actually 
done some work

00:15:45.890 --> 00:15:48.200
with some of these tech companies.

00:15:49.100 --> 00:15:52.200
I don't disagree with Josh's point 
that we need to think

00:15:52.200 --> 00:15:53.830
a little bit about 
the funding model,

00:15:53.830 --> 00:15:56.390
who is it in the end paying 
for the graduate education.

00:15:56.710 --> 00:15:59.400
But from a scientific perspective,

00:15:59.980 --> 00:16:02.540
not only do these places
have great data

00:16:02.540 --> 00:16:04.950
and nowadays they tend to be
very careful with that

00:16:04.950 --> 00:16:07.100
for privacy reasons,

00:16:07.380 --> 00:16:08.900
but they also have great questions.

00:16:10.200 --> 00:16:13.213
I find it very inspiring 
kind of to listen

00:16:13.213 --> 00:16:15.950
to the people there and kind of see
what kind of questions they have,

00:16:15.950 --> 00:16:17.330
and often they're questions

00:16:18.200 --> 00:16:21.510
that also come up outside 
of these companies.

00:16:21.510 --> 00:16:27.430
I have a couple of papers 
with Raj Chetty and Susan Athey,

00:16:27.430 --> 00:16:31.600
where we look at ways 
of combining experimental data

00:16:31.600 --> 00:16:34.000
and observational data,
and kind of their --

00:16:35.500 --> 00:16:38.600
Raj Chetty was interested
in what is the effect

00:16:38.600 --> 00:16:42.893
of early childhood programs 
on outcomes later in life,

00:16:42.893 --> 00:16:46.330
not just kind on test scores,
but on earnings and stuff,

00:16:46.330 --> 00:16:48.300
and we kind of developed methods

00:16:48.600 --> 00:16:51.500
that would help you shed
light on that, onto some --

00:16:52.760 --> 00:16:56.920
in some settings
and the same problems came up

00:16:56.920 --> 00:17:00.533
kind of in this 
tech company settings.

00:17:00.800 --> 00:17:03.240
And so from my perspective,

00:17:03.240 --> 00:17:05.420
it's the same kind of --

00:17:05.420 --> 00:17:07.600
I was talking to people 
doing empirical work,

00:17:07.600 --> 00:17:09.700
I tried to kind of look at these
specific problems

00:17:09.700 --> 00:17:13.370
and then try to come up
with more general problems,

00:17:15.110 --> 00:17:18.230
we formulated the problems
at a higher level,

00:17:18.500 --> 00:17:22.900
so that I can think about solutions
that work in a range of settings.

00:17:23.400 --> 00:17:24.840
And so from that perspective,

00:17:24.840 --> 00:17:27.570
the interactions 
with the tech companies

00:17:27.570 --> 00:17:30.300
are just very valuable
and very useful.

00:17:31.700 --> 00:17:35.030
We do have students now
doing internships there

00:17:35.030 --> 00:17:38.390
and then coming back 
and writing more interesting thesis

00:17:38.390 --> 00:17:43.400
as a result of their 
experiences there.

00:17:44.600 --> 00:17:47.020
- [Narrator] If you'd like to watch
more Nobel Conversations,

00:17:47.020 --> 00:17:48.200
click here,

00:17:48.200 --> 00:17:50.500
or if you'd like to learn
more about econometrics,

00:17:50.500 --> 00:17:53.100
check out Josh's 
"Mastering Econometrics" series.

00:17:53.700 --> 00:17:56.720
If you'd like to learn more 
about Guido, Josh and Isaiah

00:17:56.720 --> 00:17:58.300
check out the links 
in the description.

00:17:59.036 --> 00:18:01.495
♪ [music] ♪