1
00:00:00,000 --> 00:00:02,550
♪ [music] ♪

2
00:00:03,800 --> 00:00:05,800
- [Narrator] Welcome 
to Nobel Conversations.

3
00:00:07,040 --> 00:00:08,100
In this episode,

4
00:00:08,100 --> 00:00:11,570
Josh Angrist and Guido Imbens
sit down with Isaiah Andrews

5
00:00:11,570 --> 00:00:14,600
to discuss how the field 
of econometrics is evolving.

6
00:00:16,100 --> 00:00:18,750
- [Isaiah] So, Guido and Josh, 
you're both pioneers

7
00:00:18,750 --> 00:00:21,500
in developing tools for
empirical research in economics.

8
00:00:21,500 --> 00:00:23,174
And so I'd like to explore

9
00:00:23,174 --> 00:00:25,300
where you feel like 
the field is heading --

10
00:00:25,709 --> 00:00:28,079
economics, econometrics,
the whole thing.

11
00:00:28,510 --> 00:00:31,302
To start, I'd be interested to hear

12
00:00:32,171 --> 00:00:35,200
about whether you feel
the way in which

13
00:00:35,200 --> 00:00:38,510
the local average treatment 
effects framework took hold

14
00:00:38,800 --> 00:00:42,187
has any lessons for how 
new empirical methods in economics

15
00:00:42,187 --> 00:00:44,300
develop and spread
or how they should.

16
00:00:44,560 --> 00:00:45,960
- [Josh] That's a good question.

17
00:00:46,610 --> 00:00:47,790
You go first.

18
00:00:47,790 --> 00:00:49,240
[laughter]

19
00:00:49,700 --> 00:00:52,940
- [Guido] Yeah, so I think 
the important thing

20
00:00:52,940 --> 00:00:58,550
is to come up 
with good convincing cases

21
00:00:58,550 --> 00:01:02,207
where the questions are clear

22
00:01:02,400 --> 00:01:05,720
and where the methods
apply in general.

23
00:01:06,253 --> 00:01:07,560
One thing I...

24
00:01:08,192 --> 00:01:12,000
Looking back
at the subsequent literature.

25
00:01:12,200 --> 00:01:16,700
I really like the regression
discontinuity literature

26
00:01:16,700 --> 00:01:19,670
where there were clearly a bunch
of really convincing examples

27
00:01:19,670 --> 00:01:23,378
and that allowed people
to think more clearly,

28
00:01:23,378 --> 00:01:27,200
look harder 
at the methodological questions.

29
00:01:27,400 --> 00:01:28,800
Do clear applications

30
00:01:28,800 --> 00:01:30,600
that then allow you 
to kind of think about,

31
00:01:30,600 --> 00:01:33,600
"Wow, do these type of assumptions
seem reasonable here?

32
00:01:33,600 --> 00:01:38,000
What kind of things do we not like
in the early papers?

33
00:01:38,500 --> 00:01:39,802
How can we improve things?"

34
00:01:39,802 --> 00:01:44,210
So having clear applications 
motivating these literatures

35
00:01:44,210 --> 00:01:46,400
I think is very helpful.

36
00:01:46,800 --> 00:01:48,050
- I'm glad you mentioned

37
00:01:48,050 --> 00:01:49,382
the regression 
discontinuity, Guido.

38
00:01:49,382 --> 00:01:53,300
I think there's a lot of 
complementarity between IV and RD,

39
00:01:54,700 --> 00:01:57,060
instrumental variables
and regression discontinuity.

40
00:02:00,506 --> 00:02:03,260
A lot of the econometric
applications

41
00:02:03,260 --> 00:02:04,520
of regression discontinuity

42
00:02:04,520 --> 00:02:07,230
are what used
to be called "fuzzy" RD,

43
00:02:07,230 --> 00:02:11,620
where it's not discrete 
or deterministic at the cutoff

44
00:02:11,620 --> 00:02:14,900
but just the change
in rates or intensity.

45
00:02:14,900 --> 00:02:17,737
And the LATE framework
helps us understand

46
00:02:17,737 --> 00:02:18,740
those applications

47
00:02:18,740 --> 00:02:21,140
and gives us a clear interpretation

48
00:02:21,140 --> 00:02:25,000
for something like
in my paper with Victor Lavy,

49
00:02:25,000 --> 00:02:28,100
where we use Maimonides'
rule, the class size cutoffs,

50
00:02:28,430 --> 00:02:30,030
and what are you getting there?

51
00:02:30,290 --> 00:02:31,820
Of course, you can
answer that question

52
00:02:31,820 --> 00:02:33,900
with a linear 
constant effects model,

53
00:02:34,200 --> 00:02:36,310
but it turns out 
we're not limited to that,

54
00:02:36,310 --> 00:02:39,889
and RD is still very powerful 
and illuminating,

55
00:02:40,630 --> 00:02:43,092
even when the correlation

56
00:02:43,092 --> 00:02:45,866
between the cutoff
and the variable of interest,

57
00:02:45,866 --> 00:02:49,133
in this case, class size, 
is partial,

58
00:02:49,133 --> 00:02:51,000
maybe even not that strong.

59
00:02:52,000 --> 00:02:54,999
So there was definitely
a parallel development.

60
00:02:54,999 --> 00:02:56,400
It's also interesting...

61
00:02:57,253 --> 00:02:59,780
Nobody talked about
regression discontinuity designs

62
00:02:59,780 --> 00:03:01,220
when we were in graduate school --

63
00:03:01,220 --> 00:03:02,843
it was something

64
00:03:02,843 --> 00:03:05,300
that other social scientists
were interested in,

65
00:03:05,800 --> 00:03:09,507
and that grew up 
alongside the LATE framework,

66
00:03:09,507 --> 00:03:11,927
and we've both done work

67
00:03:11,927 --> 00:03:14,565
on both applications
and methods there,

68
00:03:14,565 --> 00:03:18,377
and it's been very exciting 
to see that develop

69
00:03:18,377 --> 00:03:19,800
and become so important.

70
00:03:20,000 --> 00:03:21,767
It's part of a general evolution,

71
00:03:21,767 --> 00:03:26,086
I think, towards credible
identification strategies,

72
00:03:26,086 --> 00:03:27,441
causal effects,

73
00:03:29,393 --> 00:03:30,642
making econometrics

74
00:03:30,642 --> 00:03:33,300
more about causal questions
than about models.

75
00:03:33,640 --> 00:03:34,650
In terms of the future,

76
00:03:34,650 --> 00:03:37,660
I think one thing that LATE 
has helped facilitate

77
00:03:37,660 --> 00:03:42,008
is a move towards
more creative, randomized trials

78
00:03:42,008 --> 00:03:44,400
where there's
something of interest.

79
00:03:45,500 --> 00:03:48,460
It's not possible 
or straightforward

80
00:03:48,460 --> 00:03:50,700
to simply turn it off or on,

81
00:03:51,000 --> 00:03:54,584
but you can encourage it 
or discourage it.

82
00:03:54,584 --> 00:03:58,200
So you subsidize schooling 
with financial aid, for example.

83
00:03:59,000 --> 00:04:02,080
So now we have a whole 
framework for interpreting that,

84
00:04:03,600 --> 00:04:07,113
and it opens the doors
to randomized trials

85
00:04:07,113 --> 00:04:09,265
of things that maybe would

86
00:04:10,300 --> 00:04:12,471
not have seemed possible before.

87
00:04:14,500 --> 00:04:17,864
We've used that a lot in the work
we do on schools in our --

88
00:04:17,864 --> 00:04:21,160
in the Blueprint Lab at MIT.

89
00:04:22,360 --> 00:04:26,600
We're exploiting random assignment
in very creative ways, I think.

90
00:04:28,100 --> 00:04:31,395
- [Isaiah] Related to that,
do you see particular factors

91
00:04:31,395 --> 00:04:34,445
that make for useful research
in econometrics?

92
00:04:34,445 --> 00:04:38,443
You've alluded to it having
a clear connection

93
00:04:38,443 --> 00:04:40,300
to problems 
that are actually coming up,

94
00:04:40,300 --> 00:04:42,862
and empirical practice 
is often a good idea.

95
00:04:43,290 --> 00:04:45,000
- Isn't it always a good idea?

96
00:04:45,700 --> 00:04:50,112
I often find myself sitting
in an econometrics theory seminar,

97
00:04:50,700 --> 00:04:52,500
say the Harvard MIT seminar,

98
00:04:53,400 --> 00:04:56,350
and I'm thinking, "What problem
is this guy solving?

99
00:04:56,350 --> 00:04:57,960
Who has this problem?"

100
00:04:57,960 --> 00:04:59,800
And, you know...

101
00:05:01,600 --> 00:05:04,700
sometimes there's an
embarrassing silence if I ask

102
00:05:04,900 --> 00:05:08,300
or there might be
a fairly contrived scenario.

103
00:05:08,800 --> 00:05:11,600
I want to see 
where the tool is useful.

104
00:05:12,500 --> 00:05:14,765
There are some 
purely foundational tools --

105
00:05:14,765 --> 00:05:16,250
I do take the point.

106
00:05:16,250 --> 00:05:21,735
There are people who are working
on conceptual foundations of...

107
00:05:22,600 --> 00:05:25,300
it becomes more like
mathematical statistics.

108
00:05:25,800 --> 00:05:27,653
I mean, I remember 
an early example of that

109
00:05:27,653 --> 00:05:29,920
that I struggled to understand

110
00:05:29,920 --> 00:05:32,500
was the idea
of stochastic equicontinuity,

111
00:05:32,500 --> 00:05:35,070
which one of my thesis advisors,
Whitney Newey,

112
00:05:35,070 --> 00:05:36,479
was using to great effect,

113
00:05:36,479 --> 00:05:38,821
and I was trying
to understand that.

114
00:05:40,600 --> 00:05:42,034
It's really foundational.

115
00:05:42,034 --> 00:05:45,200
it's not an application
that's driving that --

116
00:05:45,890 --> 00:05:47,300
at least not immediately.

117
00:05:48,600 --> 00:05:53,200
But most things are not like that,
and so there should be a problem.

118
00:05:53,800 --> 00:05:59,247
And I think it's on the seller 
of that sort of thing,

119
00:06:00,480 --> 00:06:02,250
because there's opportunity cost,

120
00:06:02,250 --> 00:06:05,295
the time and attention,
and effort to understand things.

121
00:06:05,980 --> 00:06:07,200
It's on the seller to say,

122
00:06:07,400 --> 00:06:08,900
"Hey, I'm solving this problem,

123
00:06:09,400 --> 00:06:12,900
and here's a set of results
that show that it's useful,

124
00:06:12,900 --> 00:06:15,200
and here's some insight
that I get."

125
00:06:16,097 --> 00:06:18,280
- [Isaiah] As you said, Josh,
there's been a move

126
00:06:18,280 --> 00:06:20,700
in the direction of thinking 
more about causality

127
00:06:20,700 --> 00:06:22,900
in economics and empirical
work in economics.

128
00:06:22,900 --> 00:06:26,570
Any consequences of the spread
of that view that surprised you

129
00:06:26,570 --> 00:06:28,347
or anything that you view
as downsides

130
00:06:28,705 --> 00:06:31,500
of the way that empirical
economics has gone?

131
00:06:31,500 --> 00:06:34,322
- Sometimes I see 
somebody does IV,

132
00:06:34,322 --> 00:06:38,304
and they get a result 
which seems implausibly large --

133
00:06:38,800 --> 00:06:40,200
that's the usual case.

134
00:06:42,500 --> 00:06:45,220
So it might be
an extraordinarily large

135
00:06:45,220 --> 00:06:48,600
causal effect of some 
relatively minor intervention,

136
00:06:49,100 --> 00:06:52,260
which was randomized
or for which you could make a case

137
00:06:52,260 --> 00:06:54,490
that there's a good design.

138
00:06:54,900 --> 00:06:57,205
And then when I see that,

139
00:06:57,944 --> 00:07:00,101
I think it's very hard 
for me to believe

140
00:07:00,101 --> 00:07:02,030
that this relatively 
minor intervention

141
00:07:02,030 --> 00:07:03,720
has such a large effect.

142
00:07:04,100 --> 00:07:06,277
The author will sometimes resort

143
00:07:06,277 --> 00:07:08,690
to the local average 
treatment effects theorem

144
00:07:08,690 --> 00:07:11,066
and say, "Well, these compliers --

145
00:07:11,066 --> 00:07:12,700
they're special in some way,

146
00:07:13,300 --> 00:07:15,800
and they just benefit
extraordinarily

147
00:07:15,800 --> 00:07:17,600
from this intervention."

148
00:07:18,100 --> 00:07:21,175
I'm reluctant to take that
at face value.

149
00:07:21,175 --> 00:07:23,750
I think often when effects
are too big,

150
00:07:24,300 --> 00:07:26,780
it's because the exclusion
restriction is failing,

151
00:07:26,780 --> 00:07:29,456
so you don't really have the right
endogenous variable

152
00:07:29,456 --> 00:07:31,380
to scale that result.

153
00:07:32,000 --> 00:07:35,700
And so, I'm not too happy to see

154
00:07:36,937 --> 00:07:39,692
a generic heterogeneity argument

155
00:07:40,022 --> 00:07:41,760
being used to excuse something

156
00:07:41,760 --> 00:07:43,800
that I think might be 
a deeper problem.

157
00:07:45,190 --> 00:07:47,358
- [Guido] I think it played 
somewhat of an unfortunate role

158
00:07:47,358 --> 00:07:50,083
in the discussions
between reduced form

159
00:07:50,083 --> 00:07:51,700
and structural approaches,

160
00:07:51,700 --> 00:07:55,510
where I feel 
that wasn't quite right.

161
00:07:56,090 --> 00:07:58,810
The instrumental 
variables assumptions

162
00:07:58,810 --> 00:08:01,510
are at the core,

163
00:08:01,510 --> 00:08:03,483
structural assumptions
about behavior --

164
00:08:03,483 --> 00:08:05,200
they were coming from economic...

165
00:08:07,100 --> 00:08:09,900
thinking about the economic
behavior of agents,

166
00:08:10,300 --> 00:08:15,100
and somehow it got pushed
in a direction

167
00:08:15,100 --> 00:08:17,600
that I think wasn't 
really very helpful.

168
00:08:20,426 --> 00:08:21,663
I think, initially,

169
00:08:22,800 --> 00:08:24,067
we wrote things up,

170
00:08:24,067 --> 00:08:26,480
it was describing
what was happening.

171
00:08:26,480 --> 00:08:29,783
There were a set of methods 
people were using.

172
00:08:29,783 --> 00:08:32,111
We clarified what 
those methods were doing

173
00:08:32,811 --> 00:08:38,361
in a way that I think contain
a fair amount of insight.

174
00:08:39,100 --> 00:08:42,050
But it somehow 
got pushed into a corner

175
00:08:42,050 --> 00:08:45,379
that I don't think 
was necessarily very helpful.

176
00:08:45,379 --> 00:08:48,604
- In just the language
of reduced form versus structural,

177
00:08:48,604 --> 00:08:50,306
I find kind of funny in the sense

178
00:08:50,306 --> 00:08:52,985
that the local average
treatment effect model,

179
00:08:52,985 --> 00:08:54,154
the potential outcomes model

180
00:08:54,154 --> 00:08:56,110
is a nonparametric
structural model,

181
00:08:56,110 --> 00:08:58,600
if you want to think about it, 
as you suggested, Guido.

182
00:08:58,600 --> 00:09:01,129
So there's something a little funny

183
00:09:01,129 --> 00:09:03,505
about putting these 
two things in opposition when --

184
00:09:03,505 --> 00:09:05,116
- [Guido] Yes.
- [Josh] That language, of course,

185
00:09:05,116 --> 00:09:08,371
comes from the simultaneous
equations framework

186
00:09:08,371 --> 00:09:09,641
that we inherited.

187
00:09:10,400 --> 00:09:11,440
It has the advantage

188
00:09:11,440 --> 00:09:14,085
that people seem to know 
what you mean when you use it,

189
00:09:14,085 --> 00:09:15,164
but that might be

190
00:09:16,559 --> 00:09:18,300
that different people
are hearing different things.

191
00:09:18,300 --> 00:09:20,480
- [Guido] Yeah. I think
reduced form has become

192
00:09:20,480 --> 00:09:22,200
used in a little bit 
of the pejorative way...

193
00:09:22,200 --> 00:09:23,540
- [Josh] Sometimes.

194
00:09:25,104 --> 00:09:28,250
...which is not really quite what 
it was originally intended for.

195
00:09:30,100 --> 00:09:33,090
- [Isaiah] I guess something else
that strikes me in thinking about

196
00:09:33,090 --> 00:09:35,645
the effects of the local average
treatment effect framework

197
00:09:35,645 --> 00:09:37,676
is that often folks will appeal

198
00:09:37,676 --> 00:09:40,000
to a local average treatment
effects intuition

199
00:09:40,000 --> 00:09:42,358
for settings well beyond ones

200
00:09:42,358 --> 00:09:44,963
where any sort of formal result
has actually been established.

201
00:09:45,440 --> 00:09:49,180
And I'm curious, given all the work
that you guys did

202
00:09:49,180 --> 00:09:52,390
to establish LATE results 
in different settings,

203
00:09:52,390 --> 00:09:54,415
I'm curious, any thoughts on that?

204
00:09:55,360 --> 00:09:57,306
- I think there's going
to be a lot of cases

205
00:09:57,306 --> 00:10:02,200
where the intuition 
does get you some distance,

206
00:10:02,800 --> 00:10:04,989
but it's going to be 
somewhat limited,

207
00:10:04,989 --> 00:10:07,600
and establishing 
formal results there

208
00:10:08,400 --> 00:10:09,490
may be a little tricky

209
00:10:09,490 --> 00:10:12,700
and then maybe only work 
in special circumstances,

210
00:10:14,600 --> 00:10:16,540
and you end up
with a lot of formality

211
00:10:16,540 --> 00:10:19,500
that may not quite 
capture the intuition.

212
00:10:19,900 --> 00:10:21,550
Sometimes I'm somewhat 
uneasy with them,

213
00:10:21,550 --> 00:10:24,438
and they are not necessarily
the papers I would want to write,

214
00:10:25,148 --> 00:10:27,819
but I do think intuition

215
00:10:27,819 --> 00:10:30,912
often does capture 
part of the problem.

216
00:10:33,100 --> 00:10:36,300
I think, in some sense,
we were very fortunate there

217
00:10:36,900 --> 00:10:39,250
in the way that the LATE paper 
got handled at the journal,

218
00:10:39,250 --> 00:10:41,766
so that, actually, the editor,
made it much shorter

219
00:10:42,100 --> 00:10:46,300
and that allowed us to focus
on very clear, crisp results.

220
00:10:49,924 --> 00:10:51,770
There's a somewhat
unfortunate tendency

221
00:10:51,770 --> 00:10:53,118
in the econometrics literature

222
00:10:53,118 --> 00:10:54,870
of having the papers 
get longer and longer.

223
00:10:54,870 --> 00:10:56,550
- Well, you should be able
to fix that, man.

224
00:10:56,550 --> 00:10:58,915
- I'm trying to fix that.
[laughter]

225
00:10:58,915 --> 00:11:01,510
But I think this is an example 
where it's very clear

226
00:11:01,510 --> 00:11:03,097
that having it be short
is actually --

227
00:11:03,097 --> 00:11:04,750
- You should have imposed
that no paper

228
00:11:04,750 --> 00:11:06,802
can be longer than the LATE paper.

229
00:11:07,269 --> 00:11:09,617
- That... wow! That may be great.

230
00:11:09,617 --> 00:11:11,685
- At least no theory, 
no theory paper.

231
00:11:11,892 --> 00:11:13,339
- Yeah, and I think...

232
00:11:14,500 --> 00:11:16,800
I'm trying very hard to get
the papers to be shorter,

233
00:11:16,800 --> 00:11:19,514
and I think there is a lot
of value today

234
00:11:19,514 --> 00:11:21,573
because it's often
the second part of the paper

235
00:11:21,573 --> 00:11:25,049
that doesn't actually
get you much further

236
00:11:25,049 --> 00:11:26,395
in understanding things,

237
00:11:27,000 --> 00:11:29,870
and it does make things 
much harder to read.

238
00:11:32,426 --> 00:11:36,179
It goes back to how I think
econometrics should be done.

239
00:11:36,179 --> 00:11:38,070
You should focus on --

240
00:11:38,700 --> 00:11:41,300
It should be reasonably
close to empirical problems.

241
00:11:41,500 --> 00:11:43,900
They should be very clear problems.

242
00:11:44,800 --> 00:11:48,900
But then often the theory
doesn't need to be quite so long.

243
00:11:48,900 --> 00:11:50,010
- [Josh] Yeah.

244
00:11:51,100 --> 00:11:54,670
- I think things have gone
a little off track.

245
00:11:56,260 --> 00:11:57,750
- [Isaiah] A relatively 
recent change

246
00:11:57,750 --> 00:12:00,230
has been a seeming
big increase in demand

247
00:12:00,230 --> 00:12:03,765
for people with econometrics
causal effect estimation skills

248
00:12:03,765 --> 00:12:04,994
in the tech sector.

249
00:12:04,994 --> 00:12:07,563
I'm interested, 
do either of you have thoughts

250
00:12:07,563 --> 00:12:09,840
of how that's going to interact

251
00:12:09,840 --> 00:12:11,600
with the development 
of empirical methods

252
00:12:11,600 --> 00:12:13,950
or empirical research
in economics going forward?

253
00:12:14,600 --> 00:12:16,770
- [Josh] Well, there's
sort of a meta point,

254
00:12:16,770 --> 00:12:21,000
which is there's this new 
kind of employer,

255
00:12:21,800 --> 00:12:27,530
the Amazons and the Uber,
and the TripAdvisor world,

256
00:12:28,000 --> 00:12:29,300
and I think that's great.

257
00:12:29,300 --> 00:12:32,030
I like to tell my students
about that.

258
00:12:32,600 --> 00:12:35,500
At MIT, we have a lot
of computer science majors --

259
00:12:35,500 --> 00:12:37,000
that's our biggest major.

260
00:12:37,400 --> 00:12:42,362
I try to seduce some of those folks
into economics by saying

261
00:12:43,228 --> 00:12:46,700
you can go work for these companies

262
00:12:46,700 --> 00:12:49,250
that people
are very keen to work for

263
00:12:49,250 --> 00:12:50,800
because the work seems exciting,

264
00:12:52,000 --> 00:12:54,250
that the skills 
that you get in econometrics

265
00:12:54,250 --> 00:12:56,100
are as good or better

266
00:12:56,100 --> 00:12:59,736
than any competing 
discipline has to offer.

267
00:12:59,736 --> 00:13:02,143
So you should at least
take some econ --

268
00:13:02,143 --> 00:13:04,290
take some econometrics
and some econ.

269
00:13:04,800 --> 00:13:07,000
I did a fun project with Uber

270
00:13:07,600 --> 00:13:09,501
on labor supply of Uber drivers,

271
00:13:09,920 --> 00:13:12,805
and it was very, very exciting
to be part of that.

272
00:13:13,100 --> 00:13:15,400
Plus, I got to drive 
for Uber for a while,

273
00:13:15,900 --> 00:13:17,730
and I thought that was fun too.

274
00:13:17,730 --> 00:13:19,231
I did not make enough

275
00:13:19,231 --> 00:13:22,616
that I was tempted
to give up my MIT job,

276
00:13:22,616 --> 00:13:25,100
but I enjoyed the experience.

277
00:13:25,230 --> 00:13:27,534
I see a potential challenge

278
00:13:27,534 --> 00:13:30,900
to our model
of graduate education here,

279
00:13:31,700 --> 00:13:36,068
which is, if we're training people
to go work at Amazon,

280
00:13:37,900 --> 00:13:41,190
it's not clear why
we should be paying

281
00:13:41,190 --> 00:13:42,900
graduate stipends for that.

282
00:13:43,200 --> 00:13:47,280
Why should the taxpayer
effectively be subsidizing that.

283
00:13:47,280 --> 00:13:51,180
Our graduate education
in the US is generously subsidized,

284
00:13:51,180 --> 00:13:53,272
even in private universities --

285
00:13:54,906 --> 00:13:56,477
there's a lot
of public money there.

286
00:13:56,477 --> 00:13:59,300
And I think the traditional
rationale for that is,

287
00:13:59,643 --> 00:14:01,992
we were training 
educators and scholars,

288
00:14:01,992 --> 00:14:05,657
and there's a great externality
from the work that we do,

289
00:14:05,657 --> 00:14:07,910
it's either 
the research externality,

290
00:14:07,910 --> 00:14:09,557
or a teaching externality.

291
00:14:10,100 --> 00:14:13,389
But if many of our students
are going to work

292
00:14:13,389 --> 00:14:14,688
in the private sector --

293
00:14:16,300 --> 00:14:17,414
that's fine,

294
00:14:19,000 --> 00:14:21,700
but maybe their employers
should pay for them.

295
00:14:22,120 --> 00:14:23,370
- For me, it's just so different

296
00:14:23,370 --> 00:14:26,780
from people working 
for consulting firms.

297
00:14:27,200 --> 00:14:28,780
It's not clear to me

298
00:14:28,780 --> 00:14:32,836
that the number of jobs
in academics has changed.

299
00:14:33,370 --> 00:14:36,325
- I feel like this is
a growing sector,

300
00:14:36,325 --> 00:14:37,807
whereas consulting...

301
00:14:37,807 --> 00:14:39,289
You're right to raise that.

302
00:14:39,289 --> 00:14:42,100
It might be the same 
for consulting.

303
00:14:44,846 --> 00:14:47,500
I'm placing more and more 
students in these businesses,

304
00:14:47,500 --> 00:14:49,467
so it's on my mind, in a way,

305
00:14:49,467 --> 00:14:54,212
that I've not been attentive 
to consulting jobs.

306
00:14:54,212 --> 00:14:56,920
Consulting was always important,

307
00:14:56,920 --> 00:14:58,950
and I think also 
there's some movement

308
00:14:58,950 --> 00:15:01,140
from consulting back
into research --

309
00:15:01,140 --> 00:15:02,723
it's a little more fluid.

310
00:15:03,900 --> 00:15:07,630
A lot of the work in both domains,

311
00:15:07,630 --> 00:15:09,430
I have to say, 
it's not really different,

312
00:15:09,430 --> 00:15:12,730
but people who are working
in the tech sector

313
00:15:12,730 --> 00:15:15,480
are doing things that are 
potentially of scientific interest,

314
00:15:15,480 --> 00:15:16,800
but mostly it's hidden.

315
00:15:17,100 --> 00:15:18,550
Then you really have to say,

316
00:15:18,550 --> 00:15:20,900
why is the government
paying for this?

317
00:15:22,104 --> 00:15:23,732
- Although, to Guido's point,

318
00:15:23,732 --> 00:15:26,783
I guess there's 
a data question here of

319
00:15:26,783 --> 00:15:32,772
has the total [non-EC]
for-profit sector employment

320
00:15:32,772 --> 00:15:35,595
of econ Ph.D. program
graduates increased

321
00:15:35,595 --> 00:15:38,290
or has it just been
a substitution from finance

322
00:15:38,290 --> 00:15:40,300
and consulting towards tech?

323
00:15:40,300 --> 00:15:42,300
- I may be reacting to something

324
00:15:42,300 --> 00:15:44,400
that's not really happening.

325
00:15:44,400 --> 00:15:45,890
- I've actually done some work

326
00:15:45,890 --> 00:15:48,200
with some of these tech companies.

327
00:15:49,100 --> 00:15:52,200
I don't disagree with Josh's point 
that we need to think

328
00:15:52,200 --> 00:15:53,675
a little bit about 
the funding model

329
00:15:53,675 --> 00:15:56,390
who is, in the end, paying 
for the graduate education.

330
00:15:56,913 --> 00:15:59,400
But from a scientific perspective,

331
00:15:59,980 --> 00:16:02,840
not only do these places
have great data,

332
00:16:02,840 --> 00:16:05,112
and nowadays they tend to be
very careful with that

333
00:16:05,112 --> 00:16:07,100
for privacy reasons,

334
00:16:07,380 --> 00:16:08,900
but they also have great questions.

335
00:16:10,200 --> 00:16:13,929
I find it very inspiring to listen
to the people there

336
00:16:13,929 --> 00:16:15,950
and see what kind
of questions they have,

337
00:16:15,950 --> 00:16:17,330
and often they're questions

338
00:16:18,200 --> 00:16:21,241
that also come up
outside of these companies.

339
00:16:21,241 --> 00:16:27,430
I have a couple of papers 
with Raj Chetty and Susan Athey

340
00:16:27,430 --> 00:16:31,600
where we look at ways 
of combining experimental data

341
00:16:31,600 --> 00:16:33,274
and observational data.

342
00:16:35,500 --> 00:16:38,600
Raj Chetty was interested
in what is the effect

343
00:16:38,600 --> 00:16:42,893
of early childhood programs 
on outcomes later in life,

344
00:16:42,893 --> 00:16:46,330
not just on test scores
but on earnings and stuff,

345
00:16:46,330 --> 00:16:48,300
and we kind of developed methods

346
00:16:48,600 --> 00:16:51,500
that would help you shed light
on that under some...

347
00:16:52,718 --> 00:16:53,868
in some settings,

348
00:16:53,868 --> 00:16:56,920
and the same problems came up

349
00:16:56,920 --> 00:17:00,533
in these tech company settings.

350
00:17:00,800 --> 00:17:03,240
And so, from my perspective,

351
00:17:03,240 --> 00:17:05,420
it's the same kind of --

352
00:17:05,420 --> 00:17:07,600
I was talking to people 
doing empirical work.

353
00:17:07,600 --> 00:17:09,700
I tried to kind of look
at these specific problems

354
00:17:09,700 --> 00:17:13,370
and then try to come up
with more general problems,

355
00:17:15,110 --> 00:17:18,230
reformulating the problems
at a higher level

356
00:17:18,500 --> 00:17:22,900
so that I can think about solutions
that work in a range of settings.

357
00:17:23,400 --> 00:17:24,933
And so from that perspective,

358
00:17:24,933 --> 00:17:27,570
the interactions 
with the tech companies

359
00:17:27,570 --> 00:17:30,300
are just very valuable
and very useful.

360
00:17:31,700 --> 00:17:35,204
We do have students now
doing internships there

361
00:17:35,204 --> 00:17:38,516
and then coming back 
and writing more interesting theses

362
00:17:38,516 --> 00:17:43,400
as a result of
their experiences there.

363
00:17:44,600 --> 00:17:47,020
- [Narrator] If you'd like to watch
more Nobel Conversations,

364
00:17:47,020 --> 00:17:48,200
click here.

365
00:17:48,200 --> 00:17:50,500
Or if you'd like to learn
more about econometrics,

366
00:17:50,500 --> 00:17:53,100
check out Josh's 
"Mastering Econometrics" series.

367
00:17:53,700 --> 00:17:56,720
If you'd like to learn more 
about Guido, Josh, and Isaiah,

368
00:17:56,720 --> 00:17:58,300
check out the links 
in the description.

369
00:17:59,036 --> 00:18:01,495
♪ [music] ♪