1
00:00:00,000 --> 00:00:02,550
♪ [music] ♪

2
00:00:03,800 --> 00:00:05,800
- [Narrator] Welcome 
to Nobel Conversations.

3
00:00:07,040 --> 00:00:08,100
In this episode,

4
00:00:08,100 --> 00:00:11,570
Josh Angrist and Guido Imbens
sit down with Isaiah Andrews

5
00:00:11,570 --> 00:00:14,600
to discuss how the field 
of econometrics is evolving.

6
00:00:16,100 --> 00:00:18,750
- [Isaiah] So, Guido and Josh, 
you're both pioneers

7
00:00:18,750 --> 00:00:21,500
in developing tools for
empirical research in economics.

8
00:00:21,500 --> 00:00:22,930
And so I'd like to explore

9
00:00:22,930 --> 00:00:25,300
where you feel like 
the field is heading --

10
00:00:25,709 --> 00:00:28,079
economics, econometrics,
the whole thing.

11
00:00:28,510 --> 00:00:31,302
To start, I'd be interested to hear

12
00:00:32,171 --> 00:00:35,200
about whether you feel
the way in which

13
00:00:35,200 --> 00:00:38,510
the local average treatment 
effects framework took hold

14
00:00:38,800 --> 00:00:42,187
has any lessons for how 
new empirical methods in economics

15
00:00:42,187 --> 00:00:44,300
develop and spread
or how they should.

16
00:00:44,560 --> 00:00:45,960
- [Josh] That's a good question.

17
00:00:46,610 --> 00:00:47,790
You go first.

18
00:00:47,790 --> 00:00:49,460
[laughter]

19
00:00:49,700 --> 00:00:52,940
- [Guido] Yeah, so I think 
the important thing

20
00:00:52,940 --> 00:00:58,550
is to come up 
with good convincing cases

21
00:00:58,550 --> 00:01:02,207
where the questions are clear

22
00:01:02,400 --> 00:01:05,720
and where the methods
apply in general.

23
00:01:06,253 --> 00:01:07,560
One thing I --

24
00:01:08,192 --> 00:01:12,000
looking back
at the subsequent literature.

25
00:01:12,200 --> 00:01:16,700
So I really like the regression
discontinuity literature

26
00:01:16,700 --> 00:01:19,670
where there were clearly a bunch
of really convincing examples

27
00:01:19,670 --> 00:01:23,378
and that allowed people
to think more clearly,

28
00:01:23,378 --> 00:01:27,200
look harder 
at the methodological questions.

29
00:01:27,400 --> 00:01:28,800
Do clear applications

30
00:01:28,800 --> 00:01:30,600
that then allow you 
to kind of think about,

31
00:01:30,600 --> 00:01:33,600
"Wow, does this type of assumption
seem reasonable here?

32
00:01:33,600 --> 00:01:38,000
What kind of things do we not like
in the early papers?

33
00:01:38,500 --> 00:01:39,802
How can we improve things?"

34
00:01:39,802 --> 00:01:44,210
So having clear applications 
motivating these literatures

35
00:01:44,210 --> 00:01:46,400
I think is very helpful.

36
00:01:46,800 --> 00:01:48,050
- I'm glad you mentioned

37
00:01:48,050 --> 00:01:49,382
the regression 
discontinuity, Guido.

38
00:01:49,382 --> 00:01:53,300
I think there's a lot of 
complementarity between IV and RD,

39
00:01:54,700 --> 00:01:57,060
Instrumental Variables
and Regression Discontinuity.

40
00:02:00,506 --> 00:02:03,260
A lot of the econometric
applications

41
00:02:03,260 --> 00:02:04,520
of regression discontinuity

42
00:02:04,520 --> 00:02:07,230
are what used to be called 
"fuzzy" RD,

43
00:02:07,230 --> 00:02:11,620
where, you know, it's not discrete 
or deterministic at the cutoff

44
00:02:11,620 --> 00:02:14,900
but just the change
in rates or intensity.

45
00:02:14,900 --> 00:02:17,737
And the late framework
helps us understand

46
00:02:17,737 --> 00:02:18,740
those applications

47
00:02:18,740 --> 00:02:21,140
and gives us a clear interpretation

48
00:02:21,140 --> 00:02:25,000
for something like
in my paper with Victor Lavy,

49
00:02:25,000 --> 00:02:28,100
where we use Maimonides'
rule, the class size cutoffs,

50
00:02:28,430 --> 00:02:30,030
and what are you getting there?

51
00:02:30,290 --> 00:02:31,820
Of course, you can
answer that question

52
00:02:31,820 --> 00:02:33,900
with a linear 
constant effects model,

53
00:02:34,200 --> 00:02:36,310
but it turns out 
we're not limited to that,

54
00:02:36,310 --> 00:02:39,889
and RD is still very powerful 
and illuminating,

55
00:02:40,630 --> 00:02:43,092
even when the correlation

56
00:02:43,092 --> 00:02:45,866
between the cutoff
and the variable of interest,

57
00:02:45,866 --> 00:02:49,133
in this case class size, 
is partial,

58
00:02:49,133 --> 00:02:51,000
maybe even not that strong.

59
00:02:52,000 --> 00:02:54,999
So there was definitely kind
of a parallel development.

60
00:02:54,999 --> 00:02:56,400
It's also interesting --

61
00:02:56,600 --> 00:02:59,780
nobody talked about
regression discontinuity designs

62
00:02:59,780 --> 00:03:01,220
when we were in graduate school.

63
00:03:01,220 --> 00:03:05,300
It was something that other social
scientists were interested in,

64
00:03:05,800 --> 00:03:09,507
and that grew up 
alongside the LATE framework,

65
00:03:09,507 --> 00:03:11,927
and we've both done work

66
00:03:11,927 --> 00:03:14,565
on both applications
and methods there,

67
00:03:14,565 --> 00:03:18,377
and it's been very exciting 
to see that develop

68
00:03:18,377 --> 00:03:19,800
and become so important.

69
00:03:20,000 --> 00:03:21,767
It's part of a general evolution,

70
00:03:21,767 --> 00:03:26,086
I think, towards credible
identification strategies,

71
00:03:26,086 --> 00:03:27,441
causal effects...

72
00:03:29,393 --> 00:03:30,642
making econometrics

73
00:03:30,642 --> 00:03:33,300
more about causal questions
than about models.

74
00:03:33,640 --> 00:03:34,650
In terms of the future,

75
00:03:34,650 --> 00:03:37,660
I think one thing that LATE 
has helped facilitate

76
00:03:37,660 --> 00:03:42,008
is a move towards
more creative, randomized trials

77
00:03:42,008 --> 00:03:44,400
where there's
something of interest.

78
00:03:45,500 --> 00:03:48,460
It's not possible 
or straightforward

79
00:03:48,460 --> 00:03:50,700
to simply turn it off or on,

80
00:03:51,000 --> 00:03:54,584
but you can encourage it 
or discourage it.

81
00:03:54,584 --> 00:03:58,200
So you subsidize schooling 
with financial aid, for example.

82
00:03:59,000 --> 00:04:02,080
So now we have a whole 
framework for interpreting that,

83
00:04:03,600 --> 00:04:07,113
and it opens the doors
to randomized trials

84
00:04:07,113 --> 00:04:10,300
of things that maybe would

85
00:04:10,300 --> 00:04:12,471
not have seemed possible before.

86
00:04:14,500 --> 00:04:17,864
We've used that a lot in the work
we do on schools in our --

87
00:04:17,864 --> 00:04:21,160
in the Blueprint Lab at MIT.

88
00:04:22,360 --> 00:04:26,600
We're exploiting random assignment
in very creative ways, I think.

89
00:04:28,100 --> 00:04:31,395
- [Isaiah] Related to that,
do you see particular factors

90
00:04:31,395 --> 00:04:34,445
that make for useful research
in econometrics?

91
00:04:34,445 --> 00:04:38,290
You've alluded to
it having a clear connection

92
00:04:38,290 --> 00:04:40,300
to problems 
that are actually coming up,

93
00:04:40,300 --> 00:04:42,650
and empirical practice 
is often a good idea.

94
00:04:43,290 --> 00:04:45,000
- Isn't it always a good idea?

95
00:04:45,700 --> 00:04:47,292
I often find myself sitting

96
00:04:47,292 --> 00:04:50,100
in an econometrics
theory seminar,

97
00:04:50,700 --> 00:04:52,500
say the Harvard MIT seminar,

98
00:04:53,400 --> 00:04:56,350
and I'm thinking, "What problem
is this guy solving?

99
00:04:56,350 --> 00:04:57,960
Who has this problem?"

100
00:04:57,960 --> 00:04:59,800
And, you know...

101
00:05:01,600 --> 00:05:04,700
sometimes there's an
embarrassing silence if I ask

102
00:05:04,900 --> 00:05:08,300
or there might be
a fairly contrived scenario.

103
00:05:08,800 --> 00:05:11,600
I want to see 
where the tool is useful.

104
00:05:12,500 --> 00:05:14,765
There are some 
purely foundational tools.

105
00:05:14,765 --> 00:05:16,250
I do take the point.

106
00:05:16,250 --> 00:05:21,735
There are people who are working
on conceptual foundations of ...

107
00:05:22,600 --> 00:05:25,300
it becomes more like
mathematical statistics.

108
00:05:25,800 --> 00:05:27,653
I mean, I remember 
an early example of that

109
00:05:27,653 --> 00:05:29,920
that I struggled to understand

110
00:05:29,920 --> 00:05:32,500
was the idea
of stochastic equicontinuity,

111
00:05:32,500 --> 00:05:35,070
which one of my thesis advisors,
Whitney Newey,

112
00:05:35,070 --> 00:05:36,479
was using to great effect,

113
00:05:36,479 --> 00:05:38,821
and I was trying
to understand that.

114
00:05:40,600 --> 00:05:42,034
It's really foundational.

115
00:05:42,034 --> 00:05:45,200
it's not an application
that's driving that --

116
00:05:45,890 --> 00:05:47,300
at least not immediately.

117
00:05:48,600 --> 00:05:53,200
But most things are not like that,
and so there should be a problem.

118
00:05:53,800 --> 00:05:59,100
And I think it's on the seller 
of that sort of thing,

119
00:06:00,480 --> 00:06:02,250
because there's opportunity cost,

120
00:06:02,250 --> 00:06:05,295
the time and attention,
and effort to understand things.

121
00:06:05,980 --> 00:06:07,200
It's on the seller to say,

122
00:06:07,400 --> 00:06:08,900
"Hey, I'm solving this problem,

123
00:06:09,400 --> 00:06:12,900
and here's a set of results
that show that it's useful,

124
00:06:12,900 --> 00:06:15,200
and here's some insight
that I get."

125
00:06:16,200 --> 00:06:18,280
- [Isaiah] As you said, Josh,
there's been a move

126
00:06:18,280 --> 00:06:20,700
in the direction of thinking 
more about causality

127
00:06:20,700 --> 00:06:22,900
in economics and empirical
work in economics.

128
00:06:22,900 --> 00:06:24,800
Any consequences of the --

129
00:06:24,800 --> 00:06:26,570
the spread of that view 
that surprised you

130
00:06:26,570 --> 00:06:28,347
or anything that you view
as downsides

131
00:06:28,705 --> 00:06:31,400
of the way that empirical
economics has gone?

132
00:06:31,500 --> 00:06:34,322
- Sometimes I see 
somebody does IV,

133
00:06:34,322 --> 00:06:38,304
and they get a result 
which seems implausibly large.

134
00:06:38,800 --> 00:06:40,200
That's the usual case.

135
00:06:42,500 --> 00:06:45,220
So it might be, you know,
an extraordinarily large

136
00:06:45,220 --> 00:06:48,600
causal effect of some 
relatively minor intervention,

137
00:06:49,100 --> 00:06:52,260
which was randomized
or for which you could make a case

138
00:06:52,260 --> 00:06:54,490
that there's a good design.

139
00:06:54,900 --> 00:06:58,330
And then when I see that,
and, you know, I think

140
00:06:58,900 --> 00:07:00,465
it's very hard for me to believe

141
00:07:00,465 --> 00:07:02,030
that this relatively 
minor intervention

142
00:07:02,030 --> 00:07:03,720
has such a large effect.

143
00:07:04,100 --> 00:07:06,110
The author will sometimes resort

144
00:07:06,110 --> 00:07:08,690
to the local average 
treatment effects theorem

145
00:07:08,690 --> 00:07:10,695
and say, "Well, these compliers,

146
00:07:10,695 --> 00:07:12,700
you know, they're
special in some way."

147
00:07:13,300 --> 00:07:15,800
And, you know, 
they just benefit extraordinarily

148
00:07:15,800 --> 00:07:17,600
from this intervention.

149
00:07:18,100 --> 00:07:20,900
And I'm reluctant 
to take that at face value.

150
00:07:20,900 --> 00:07:23,750
I think, you know,
often when effects are too big,

151
00:07:24,300 --> 00:07:26,780
it's because the exclusion
restriction is failing,

152
00:07:26,780 --> 00:07:29,240
so you don't really have the right
endogenous variable

153
00:07:29,240 --> 00:07:31,380
to scale that result.

154
00:07:32,000 --> 00:07:35,700
And so I'm not too happy to see

155
00:07:35,700 --> 00:07:38,800
you know, just sort of
a generic heterogeneity

156
00:07:38,900 --> 00:07:41,610
argument being used 
to excuse something

157
00:07:41,610 --> 00:07:43,800
that I think might be 
a deeper problem.

158
00:07:45,190 --> 00:07:47,358
- [Guido] I think it played 
somewhat of an unfortunate role

159
00:07:47,358 --> 00:07:49,979
when the discussions kind of 
between reduced form

160
00:07:49,979 --> 00:07:51,700
and structural approaches,

161
00:07:51,700 --> 00:07:55,510
where I feel 
that wasn't quite right.

162
00:07:56,090 --> 00:07:58,810
The instrumental 
variables assumptions

163
00:07:58,810 --> 00:08:03,622
are at the core - structural 
assumptions about behavior -

164
00:08:03,622 --> 00:08:05,200
they were coming from economic --

165
00:08:07,100 --> 00:08:09,900
thinking about the economic
behavior of agents,

166
00:08:10,300 --> 00:08:15,000
and somehow it got
pushed in a direction

167
00:08:15,100 --> 00:08:17,600
that I think wasn't 
really very helpful.

168
00:08:18,800 --> 00:08:21,700
The way I think, initially the --

169
00:08:22,800 --> 00:08:26,480
we wrote things up, it was 
describing what was happening,

170
00:08:26,480 --> 00:08:29,490
there were a set of methods 
people were using,

171
00:08:29,490 --> 00:08:32,111
we clarified what 
those methods were doing

172
00:08:32,811 --> 00:08:38,361
and in a way that I think
contain a fair amount of insight,

173
00:08:39,100 --> 00:08:42,050
but it somehow 
it got pushed into a corner

174
00:08:42,050 --> 00:08:45,000
that I don't think 
was necessarily very helpful.

175
00:08:45,100 --> 00:08:46,850
- [Isaiah] I mean, 
just the language

176
00:08:46,850 --> 00:08:48,600
of reduced form versus structural

177
00:08:48,600 --> 00:08:50,820
I find kind of funny 
in the sense that,

178
00:08:50,820 --> 00:08:53,100
right, the local average treatment
effect model, right,

179
00:08:53,100 --> 00:08:56,110
the potential outcomes model is 
a nonparametric structural model,

180
00:08:56,110 --> 00:08:58,600
if you want to think about it, 
as you sort of suggested, Guido.

181
00:08:58,600 --> 00:09:01,263
So there's something a little funny

182
00:09:01,263 --> 00:09:03,340
about putting these 
two things in oposition when --

183
00:09:03,340 --> 00:09:04,690
- [Guido] Yes.
- [Josh] Well, that language,

184
00:09:04,690 --> 00:09:08,165
of course, comes from 
the [inaudible] equations framework

185
00:09:08,165 --> 00:09:09,520
that we inherited.

186
00:09:10,400 --> 00:09:11,530
It has the advantage

187
00:09:11,530 --> 00:09:14,160
that people seem to know 
what you mean when you use it,

188
00:09:14,160 --> 00:09:16,240
but might be that people 
are hearing different,

189
00:09:16,240 --> 00:09:18,200
different people
are hearing different things.

190
00:09:18,300 --> 00:09:20,530
- [Guido] Yeah. I think [inaudible]
has sort of become --

191
00:09:20,530 --> 00:09:22,560
used in a little bit 
of the pejorative way, yeah?

192
00:09:22,560 --> 00:09:24,750
- [Josh] Sometimes.
- [Guido] [The word].

193
00:09:24,750 --> 00:09:28,250
Which is not really quite what 
it was originally intended for.

194
00:09:30,100 --> 00:09:33,090
- [Isaiah] I guess something else
that strikes me in thinking about

195
00:09:33,090 --> 00:09:35,645
the effects of the local average
treatment effect framework

196
00:09:35,645 --> 00:09:38,200
is that often folks will appeal to

197
00:09:38,200 --> 00:09:41,800
a local average treatment effects
intuition for settings well beyond

198
00:09:42,000 --> 00:09:43,700
ones where any sort 
of formal result

199
00:09:43,700 --> 00:09:45,440
has actually been established.

200
00:09:45,440 --> 00:09:50,090
And I'm curious, given all the work
that you guys did to, you know,

201
00:09:50,090 --> 00:09:52,390
establish late results 
in different settings,

202
00:09:52,390 --> 00:09:54,415
I'm curious, any thoughts on that?

203
00:09:55,360 --> 00:09:57,420
- [Guido] I think there's
going to be a lot of cases

204
00:09:57,420 --> 00:10:02,200
where the intuition 
does get you some distance,

205
00:10:02,800 --> 00:10:05,200
but it's going to be 
somewhat limited

206
00:10:05,200 --> 00:10:07,600
and establishing 
formal results there

207
00:10:08,400 --> 00:10:09,490
may be a little tricky

208
00:10:09,490 --> 00:10:12,700
and then maybe only work 
in special circumstances,

209
00:10:14,600 --> 00:10:16,540
and you end up
with a lot of formality

210
00:10:16,540 --> 00:10:19,500
that may not quite 
capture the intuition.

211
00:10:19,900 --> 00:10:21,550
Sometimes I'm somewhat 
uneasy with them

212
00:10:21,550 --> 00:10:24,438
and they are not necessarily
the papers I would want to write,

213
00:10:25,148 --> 00:10:27,218
but I do think something --

214
00:10:27,218 --> 00:10:31,217
intuition often does capture 
part of the problem.

215
00:10:33,100 --> 00:10:36,300
I think, in some sense we were
kind of very fortunate there

216
00:10:36,900 --> 00:10:39,250
in the way that the late paper 
got handled at the journal,

217
00:10:39,250 --> 00:10:41,766
is that, actually, the editor,
made it much shorter

218
00:10:42,100 --> 00:10:46,300
and that then allowed us to kind of
focus on very clear, crisp results.

219
00:10:47,100 --> 00:10:51,770
Where if you -- you know, this 
somewhat unfortunate tendency

220
00:10:51,770 --> 00:10:52,985
in the econometrics literature

221
00:10:52,985 --> 00:10:55,100
of having the papers 
get longer and longer.

222
00:10:55,100 --> 00:10:56,690
- [Josh] Well, you should 
be able to fix that, man.

223
00:10:56,690 --> 00:10:58,800
- [Guido] I'm trying to fix that.

224
00:10:59,400 --> 00:11:01,625
But I think this is an example 
where it's sort of very clear

225
00:11:01,625 --> 00:11:03,498
that having it be short
is actually --

226
00:11:03,498 --> 00:11:04,842
- [Josh] You should impose
that no paper

227
00:11:04,842 --> 00:11:06,655
can be longer than the late paper.

228
00:11:06,655 --> 00:11:08,000
- [Guido] That, wow.

229
00:11:08,000 --> 00:11:09,617
That may be great.

230
00:11:09,617 --> 00:11:11,685
- [Josh] At least no theory, 
no theory paper.

231
00:11:11,892 --> 00:11:14,300
- [Guido] Yeah, 
and I think, I think...

232
00:11:14,500 --> 00:11:16,800
I'm trying very hard to get
the papers to be shorter.

233
00:11:16,800 --> 00:11:18,700
And I think there's a lot of value

234
00:11:19,200 --> 00:11:21,480
today because it's often
the second part of the paper

235
00:11:21,480 --> 00:11:26,395
that doesn't actually get you much
further in understanding things

236
00:11:27,000 --> 00:11:29,870
and it does make things 
much harder to read

237
00:11:30,630 --> 00:11:33,200
and, you know, 
it sort of goes back

238
00:11:33,200 --> 00:11:36,111
to how I think econometrics 
should be done,

239
00:11:36,111 --> 00:11:38,070
you should focus on --

240
00:11:38,700 --> 00:11:41,300
It should be reasonably
close to empirical problems.

241
00:11:41,500 --> 00:11:43,900
They should be very clear problems.

242
00:11:44,800 --> 00:11:48,900
But then often the theory
doesn't need to be quite so long.

243
00:11:48,900 --> 00:11:50,010
- [Josh] Yeah.

244
00:11:51,100 --> 00:11:54,670
- [Guido] I think things have gone
a little off track.

245
00:11:56,260 --> 00:11:57,750
- [Isaiah] A new relatively 
recent change

246
00:11:57,750 --> 00:12:00,230
has been a seeming big 
increase in demand

247
00:12:00,230 --> 00:12:02,200
for people with sort of 
econometrics,

248
00:12:02,200 --> 00:12:04,800
causal effect estimation skills
in the tech sector.

249
00:12:05,000 --> 00:12:07,480
I'm interested, 
do either of you have thoughts

250
00:12:07,480 --> 00:12:09,840
on sort of how 
that's going to interact

251
00:12:09,840 --> 00:12:11,600
with the development 
of empirical methods,

252
00:12:11,600 --> 00:12:13,950
or empirical research
in economics going forward?

253
00:12:14,600 --> 00:12:16,770
- [Josh] Well, there's
sort of a meta point,

254
00:12:16,770 --> 00:12:21,000
which is, there's this new 
kind of employer,

255
00:12:21,800 --> 00:12:27,530
the Amazons and the Uber,
and, you know, TripAdvisor world,

256
00:12:28,000 --> 00:12:29,300
and I think that's great.

257
00:12:29,300 --> 00:12:32,600
And I like to tell my students
about that, you know, especially --

258
00:12:32,600 --> 00:12:35,500
at MIT we have a lot of 
computer science majors.

259
00:12:35,500 --> 00:12:37,000
That's our biggest major.

260
00:12:37,400 --> 00:12:42,246
And I try to seduce some of those
folks into economics by saying,

261
00:12:42,246 --> 00:12:45,700
you know, you can go 
work for these,

262
00:12:45,800 --> 00:12:49,250
you know, companies that people
are very keen to work for

263
00:12:49,250 --> 00:12:50,800
because the work seems exciting,

264
00:12:52,000 --> 00:12:54,250
you know, that the skills 
that you get in econometrics

265
00:12:54,250 --> 00:12:56,100
are as good or better

266
00:12:56,100 --> 00:12:59,736
than any competing 
discipline has to offer.

267
00:12:59,736 --> 00:13:01,100
So you should at least

268
00:13:01,400 --> 00:13:04,200
take some econ, take some
econometrics, and some econ.

269
00:13:04,800 --> 00:13:07,000
I did a fun project with Uber

270
00:13:07,600 --> 00:13:09,770
on labor supply of Uber drivers

271
00:13:09,920 --> 00:13:12,805
and was very, very exciting
to be part of that.

272
00:13:13,100 --> 00:13:15,400
Plus I got to drive 
for Uber for a while

273
00:13:15,900 --> 00:13:17,730
and I thought that was fun too.

274
00:13:17,730 --> 00:13:20,700
I did not make enough
that I was tempted to

275
00:13:21,100 --> 00:13:25,100
give up my MIT job, 
but I enjoyed the experience.

276
00:13:25,230 --> 00:13:30,900
I see a potential challenge to our
model of graduate education here,

277
00:13:31,700 --> 00:13:37,400
which is, if we're training people
to go work at Amazon, you know,

278
00:13:37,900 --> 00:13:41,190
it's not clear why, you know, 
we should be paying

279
00:13:41,190 --> 00:13:42,900
graduate stipends for that.

280
00:13:43,200 --> 00:13:47,280
Why should the taxpayer effectively
be subsidizing that.

281
00:13:47,280 --> 00:13:51,400
Our graduate education
in the US Is generously subsidized,

282
00:13:51,400 --> 00:13:53,160
even in private universities,

283
00:13:53,160 --> 00:13:56,100
it's ultimately -- there's a lot of
public money there,

284
00:13:56,100 --> 00:13:59,300
and I think the
traditional rationale for that is,

285
00:13:59,500 --> 00:14:02,137
you know, we were training 
educators and scholars,

286
00:14:02,137 --> 00:14:05,657
and there's a great externality
from the work that we do,

287
00:14:05,657 --> 00:14:07,607
it's either 
the research externality,

288
00:14:07,607 --> 00:14:09,557
or a teaching externality.

289
00:14:10,100 --> 00:14:12,350
But, you know, 
if many of our students

290
00:14:12,350 --> 00:14:14,600
are going to work
in the private sector,

291
00:14:16,300 --> 00:14:21,700
that's fine, but maybe their 
employers should pay for that.

292
00:14:22,120 --> 00:14:23,370
- [Guido] But maybe
is not so different

293
00:14:23,370 --> 00:14:26,780
from people working 
for consulting firms.

294
00:14:27,200 --> 00:14:28,780
It's not clear to me

295
00:14:28,780 --> 00:14:32,836
that the number of jobs
in academics has changed.

296
00:14:33,370 --> 00:14:36,325
- [Josh] I feel like this 
is a growing sector,

297
00:14:36,325 --> 00:14:39,289
whereas consulting -- 
you're right to raise that,

298
00:14:39,289 --> 00:14:42,100
it might be the same 
for consulting,

299
00:14:43,300 --> 00:14:44,846
but this, you know,

300
00:14:44,846 --> 00:14:47,500
I'm placing more and more 
students in these businesses.

301
00:14:47,500 --> 00:14:50,400
So, it's on my mind 
in a way that I've sort of,

302
00:14:50,800 --> 00:14:53,960
you know, not been attentive 
to consulting jobs,

303
00:14:53,960 --> 00:14:56,920
you know, consulting 
was always important,

304
00:14:56,920 --> 00:14:58,950
and I think also 
there's some movement

305
00:14:58,950 --> 00:15:02,600
from consulting back into research,
it's a little more fluid.

306
00:15:03,900 --> 00:15:07,630
A lot of the work in both domains

307
00:15:07,630 --> 00:15:09,430
I have to say, 
it's not really different

308
00:15:09,430 --> 00:15:12,730
but, you know, people who 
are working in the tech sector

309
00:15:12,730 --> 00:15:15,480
are doing things that are 
potentially of scientific interest,

310
00:15:15,480 --> 00:15:16,800
but mostly it's hidden.

311
00:15:17,100 --> 00:15:18,550
Then you really I have to say,

312
00:15:18,550 --> 00:15:20,900
you know, why is the government
paying for this?

313
00:15:21,800 --> 00:15:23,732
Yeah, although, yeah, 
I mean to Guidos point,

314
00:15:23,732 --> 00:15:26,102
I guess there's 
a data question here

315
00:15:26,102 --> 00:15:30,042
of it has the sort of 
total [no-neck] sort of say

316
00:15:31,300 --> 00:15:34,870
for-profit sector employment 
of econ Ph.D. program graduates

317
00:15:34,870 --> 00:15:38,290
increased or has it just been
a substitution from finance

318
00:15:38,290 --> 00:15:40,200
and consulting towards tech.

319
00:15:40,300 --> 00:15:42,300
- [Josh] I may be reacting 
to something

320
00:15:42,300 --> 00:15:44,300
that's not really happening.

321
00:15:44,400 --> 00:15:45,890
- [Guido] I've actually 
done some work

322
00:15:45,890 --> 00:15:48,200
with some of these tech companies.

323
00:15:49,100 --> 00:15:52,200
I don't disagree with Josh's point 
that we need to think

324
00:15:52,200 --> 00:15:53,830
a little bit about 
the funding model,

325
00:15:53,830 --> 00:15:56,390
who is it in the end paying 
for the graduate education.

326
00:15:56,710 --> 00:15:59,400
But from a scientific perspective,

327
00:15:59,980 --> 00:16:02,540
not only do these places
have great data

328
00:16:02,540 --> 00:16:04,950
and nowadays they tend to be
very careful with that

329
00:16:04,950 --> 00:16:07,100
for privacy reasons,

330
00:16:07,380 --> 00:16:08,900
but they also have great questions.

331
00:16:10,200 --> 00:16:13,213
I find it very inspiring 
kind of to listen

332
00:16:13,213 --> 00:16:15,950
to the people there and kind of see
what kind of questions they have,

333
00:16:15,950 --> 00:16:17,330
and often they're questions

334
00:16:18,200 --> 00:16:21,510
that also come up outside 
of these companies.

335
00:16:21,510 --> 00:16:27,430
I have a couple of papers 
with Raj Chetty and Susan Athey,

336
00:16:27,430 --> 00:16:31,600
where we look at ways 
of combining experimental data

337
00:16:31,600 --> 00:16:34,000
and observational data,
and kind of their --

338
00:16:35,500 --> 00:16:38,600
Raj Chetty was interested
in what is the effect

339
00:16:38,600 --> 00:16:42,893
of early childhood programs 
on outcomes later in life,

340
00:16:42,893 --> 00:16:46,330
not just kind on test scores,
but on earnings and stuff,

341
00:16:46,330 --> 00:16:48,300
and we kind of developed methods

342
00:16:48,600 --> 00:16:51,500
that would help you shed
light on that, onto some --

343
00:16:52,760 --> 00:16:56,920
in some settings
and the same problems came up

344
00:16:56,920 --> 00:17:00,533
kind of in this 
tech company settings.

345
00:17:00,800 --> 00:17:03,240
And so from my perspective,

346
00:17:03,240 --> 00:17:05,420
it's the same kind of --

347
00:17:05,420 --> 00:17:07,600
I was talking to people 
doing empirical work,

348
00:17:07,600 --> 00:17:09,700
I tried to kind of look at these
specific problems

349
00:17:09,700 --> 00:17:13,370
and then try to come up
with more general problems,

350
00:17:15,110 --> 00:17:18,230
we formulated the problems
at a higher level,

351
00:17:18,500 --> 00:17:22,900
so that I can think about solutions
that work in a range of settings.

352
00:17:23,400 --> 00:17:24,840
And so from that perspective,

353
00:17:24,840 --> 00:17:27,570
the interactions 
with the tech companies

354
00:17:27,570 --> 00:17:30,300
are just very valuable
and very useful.

355
00:17:31,700 --> 00:17:35,030
We do have students now
doing internships there

356
00:17:35,030 --> 00:17:38,390
and then coming back 
and writing more interesting thesis

357
00:17:38,390 --> 00:17:43,400
as a result of their 
experiences there.

358
00:17:44,600 --> 00:17:47,020
- [Narrator] If you'd like to watch
more Nobel Conversations,

359
00:17:47,020 --> 00:17:48,200
click here,

360
00:17:48,200 --> 00:17:50,500
or if you'd like to learn
more about econometrics,

361
00:17:50,500 --> 00:17:53,100
check out Josh's 
"Mastering Econometrics" series.

362
00:17:53,700 --> 00:17:56,720
If you'd like to learn more 
about Guido, Josh and Isaiah

363
00:17:56,720 --> 00:17:58,300
check out the links 
in the description.

364
00:17:59,036 --> 00:18:01,495
♪ [music] ♪