0:00:00.000,0:00:02.550
♪ [music] ♪

0:00:03.800,0:00:05.800
- [Narrator] Welcome [br]to Nobel Conversations.

0:00:07.040,0:00:08.100
In this episode,

0:00:08.100,0:00:11.570
Josh Angrist and Guido Imbens[br]sit down with Isaiah Andrews

0:00:11.570,0:00:14.600
to discuss how the field [br]of econometrics is evolving.

0:00:16.100,0:00:18.750
- [Isaiah] So, Guido and Josh, [br]you're both pioneers

0:00:18.750,0:00:21.500
in developing tools for[br]empirical research in economics.

0:00:21.500,0:00:23.174
And so I'd like to explore

0:00:23.174,0:00:25.300
where you feel like [br]the field is heading --

0:00:25.709,0:00:28.079
economics, econometrics,[br]the whole thing.

0:00:28.510,0:00:31.302
To start, I'd be interested to hear

0:00:32.171,0:00:35.200
about whether you feel[br]the way in which

0:00:35.200,0:00:38.510
the local average treatment [br]effects framework took hold

0:00:38.800,0:00:42.187
has any lessons for how [br]new empirical methods in economics

0:00:42.187,0:00:44.300
develop and spread[br]or how they should.

0:00:44.560,0:00:45.960
- [Josh] That's a good question.

0:00:46.610,0:00:47.790
You go first.

0:00:47.790,0:00:49.240
[laughter]

0:00:49.700,0:00:52.940
- [Guido] Yeah, so I think [br]the important thing

0:00:52.940,0:00:58.550
is to come up [br]with good convincing cases

0:00:58.550,0:01:02.207
where the questions are clear

0:01:02.400,0:01:05.720
and where the methods[br]apply in general.

0:01:06.253,0:01:07.560
One thing I...

0:01:08.192,0:01:12.000
Looking back[br]at the subsequent literature.

0:01:12.200,0:01:16.700
I really like the regression[br]discontinuity literature

0:01:16.700,0:01:19.670
where there were clearly a bunch[br]of really convincing examples

0:01:19.670,0:01:23.378
and that allowed people[br]to think more clearly,

0:01:23.378,0:01:27.200
look harder [br]at the methodological questions.

0:01:27.400,0:01:28.800
Do clear applications

0:01:28.800,0:01:30.600
that then allow you [br]to kind of think about,

0:01:30.600,0:01:33.600
"Wow, do these type of assumptions[br]seem reasonable here?

0:01:33.600,0:01:38.000
What kind of things do we not like[br]in the early papers?

0:01:38.500,0:01:39.802
How can we improve things?"

0:01:39.802,0:01:44.210
So having clear applications [br]motivating these literatures

0:01:44.210,0:01:46.400
I think is very helpful.

0:01:46.800,0:01:48.050
- I'm glad you mentioned

0:01:48.050,0:01:49.382
the regression [br]discontinuity, Guido.

0:01:49.382,0:01:53.300
I think there's a lot of [br]complementarity between IV and RD,

0:01:54.700,0:01:57.060
instrumental variables[br]and regression discontinuity.

0:02:00.506,0:02:03.260
A lot of the econometric[br]applications

0:02:03.260,0:02:04.520
of regression discontinuity

0:02:04.520,0:02:07.230
are what used[br]to be called "fuzzy" RD,

0:02:07.230,0:02:11.620
where it's not discrete [br]or deterministic at the cutoff

0:02:11.620,0:02:14.900
but just the change[br]in rates or intensity.

0:02:14.900,0:02:17.737
And the LATE framework[br]helps us understand

0:02:17.737,0:02:18.740
those applications

0:02:18.740,0:02:21.140
and gives us a clear interpretation

0:02:21.140,0:02:25.000
for something like[br]in my paper with Victor Lavy,

0:02:25.000,0:02:28.100
where we use Maimonides'[br]rule, the class size cutoffs,

0:02:28.430,0:02:30.030
and what are you getting there?

0:02:30.290,0:02:31.820
Of course, you can[br]answer that question

0:02:31.820,0:02:33.900
with a linear [br]constant effects model,

0:02:34.200,0:02:36.310
but it turns out [br]we're not limited to that,

0:02:36.310,0:02:39.889
and RD is still very powerful [br]and illuminating,

0:02:40.630,0:02:43.092
even when the correlation

0:02:43.092,0:02:45.866
between the cutoff[br]and the variable of interest,

0:02:45.866,0:02:49.133
in this case, class size, [br]is partial,

0:02:49.133,0:02:51.000
maybe even not that strong.

0:02:52.000,0:02:54.999
So there was definitely[br]a parallel development.

0:02:54.999,0:02:56.400
It's also interesting...

0:02:57.253,0:02:59.780
Nobody talked about[br]regression discontinuity designs

0:02:59.780,0:03:01.220
when we were in graduate school --

0:03:01.220,0:03:02.843
it was something

0:03:02.843,0:03:05.300
that other social scientists[br]were interested in,

0:03:05.800,0:03:09.507
and that grew up [br]alongside the LATE framework,

0:03:09.507,0:03:11.927
and we've both done work

0:03:11.927,0:03:14.565
on both applications[br]and methods there,

0:03:14.565,0:03:18.377
and it's been very exciting [br]to see that develop

0:03:18.377,0:03:19.800
and become so important.

0:03:20.000,0:03:21.767
It's part of a general evolution,

0:03:21.767,0:03:26.086
I think, towards credible[br]identification strategies,

0:03:26.086,0:03:27.441
causal effects,

0:03:29.393,0:03:30.642
making econometrics

0:03:30.642,0:03:33.300
more about causal questions[br]than about models.

0:03:33.640,0:03:34.650
In terms of the future,

0:03:34.650,0:03:37.660
I think one thing that LATE [br]has helped facilitate

0:03:37.660,0:03:42.008
is a move towards[br]more creative, randomized trials

0:03:42.008,0:03:44.400
where there's[br]something of interest.

0:03:45.500,0:03:48.460
It's not possible [br]or straightforward

0:03:48.460,0:03:50.700
to simply turn it off or on,

0:03:51.000,0:03:54.584
but you can encourage it [br]or discourage it.

0:03:54.584,0:03:58.200
So you subsidize schooling [br]with financial aid, for example.

0:03:59.000,0:04:02.080
So now we have a whole [br]framework for interpreting that,

0:04:03.600,0:04:07.113
and it opens the doors[br]to randomized trials

0:04:07.113,0:04:09.265
of things that maybe would

0:04:10.300,0:04:12.471
not have seemed possible before.

0:04:14.500,0:04:17.864
We've used that a lot in the work[br]we do on schools in our --

0:04:17.864,0:04:21.160
in the Blueprint Lab at MIT.

0:04:22.360,0:04:26.600
We're exploiting random assignment[br]in very creative ways, I think.

0:04:28.100,0:04:31.395
- [Isaiah] Related to that,[br]do you see particular factors

0:04:31.395,0:04:34.445
that make for useful research[br]in econometrics?

0:04:34.445,0:04:38.443
You've alluded to it having[br]a clear connection

0:04:38.443,0:04:40.300
to problems [br]that are actually coming up,

0:04:40.300,0:04:42.862
and empirical practice [br]is often a good idea.

0:04:43.290,0:04:45.000
- Isn't it always a good idea?

0:04:45.700,0:04:50.112
I often find myself sitting[br]in an econometrics theory seminar,

0:04:50.700,0:04:52.500
say the Harvard MIT seminar,

0:04:53.400,0:04:56.350
and I'm thinking, "What problem[br]is this guy solving?

0:04:56.350,0:04:57.960
Who has this problem?"

0:04:57.960,0:04:59.800
And, you know...

0:05:01.600,0:05:04.700
sometimes there's an[br]embarrassing silence if I ask

0:05:04.900,0:05:08.300
or there might be[br]a fairly contrived scenario.

0:05:08.800,0:05:11.600
I want to see [br]where the tool is useful.

0:05:12.500,0:05:14.765
There are some [br]purely foundational tools --

0:05:14.765,0:05:16.250
I do take the point.

0:05:16.250,0:05:21.735
There are people who are working[br]on conceptual foundations of...

0:05:22.600,0:05:25.300
it becomes more like[br]mathematical statistics.

0:05:25.800,0:05:27.653
I mean, I remember [br]an early example of that

0:05:27.653,0:05:29.920
that I struggled to understand

0:05:29.920,0:05:32.500
was the idea[br]of stochastic equicontinuity,

0:05:32.500,0:05:35.070
which one of my thesis advisors,[br]Whitney Newey,

0:05:35.070,0:05:36.479
was using to great effect,

0:05:36.479,0:05:38.821
and I was trying[br]to understand that.

0:05:40.600,0:05:42.034
It's really foundational.

0:05:42.034,0:05:45.200
it's not an application[br]that's driving that --

0:05:45.890,0:05:47.300
at least not immediately.

0:05:48.600,0:05:53.200
But most things are not like that,[br]and so there should be a problem.

0:05:53.800,0:05:59.247
And I think it's on the seller [br]of that sort of thing,

0:06:00.480,0:06:02.250
because there's opportunity cost,

0:06:02.250,0:06:05.295
the time and attention,[br]and effort to understand things.

0:06:05.980,0:06:07.200
It's on the seller to say,

0:06:07.400,0:06:08.900
"Hey, I'm solving this problem,

0:06:09.400,0:06:12.900
and here's a set of results[br]that show that it's useful,

0:06:12.900,0:06:15.200
and here's some insight[br]that I get."

0:06:16.097,0:06:18.280
- [Isaiah] As you said, Josh,[br]there's been a move

0:06:18.280,0:06:20.700
in the direction of thinking [br]more about causality

0:06:20.700,0:06:22.900
in economics and empirical[br]work in economics.

0:06:22.900,0:06:26.570
Any consequences of the spread[br]of that view that surprised you

0:06:26.570,0:06:28.347
or anything that you view[br]as downsides

0:06:28.705,0:06:31.500
of the way that empirical[br]economics has gone?

0:06:31.500,0:06:34.322
- Sometimes I see [br]somebody does IV,

0:06:34.322,0:06:38.304
and they get a result [br]which seems implausibly large --

0:06:38.800,0:06:40.200
that's the usual case.

0:06:42.500,0:06:45.220
So it might be[br]an extraordinarily large

0:06:45.220,0:06:48.600
causal effect of some [br]relatively minor intervention,

0:06:49.100,0:06:52.260
which was randomized[br]or for which you could make a case

0:06:52.260,0:06:54.490
that there's a good design.

0:06:54.900,0:06:57.205
And then when I see that,

0:06:57.944,0:07:00.101
I think it's very hard [br]for me to believe

0:07:00.101,0:07:02.030
that this relatively [br]minor intervention

0:07:02.030,0:07:03.720
has such a large effect.

0:07:04.100,0:07:06.277
The author will sometimes resort

0:07:06.277,0:07:08.690
to the local average [br]treatment effects theorem

0:07:08.690,0:07:11.066
and say, "Well, these compliers --

0:07:11.066,0:07:12.700
they're special in some way,

0:07:13.300,0:07:15.800
and they just benefit[br]extraordinarily

0:07:15.800,0:07:17.600
from this intervention."

0:07:18.100,0:07:21.175
I'm reluctant to take that[br]at face value.

0:07:21.175,0:07:23.750
I think often when effects[br]are too big,

0:07:24.300,0:07:26.780
it's because the exclusion[br]restriction is failing,

0:07:26.780,0:07:29.456
so you don't really have the right[br]endogenous variable

0:07:29.456,0:07:31.380
to scale that result.

0:07:32.000,0:07:35.700
And so, I'm not too happy to see

0:07:36.937,0:07:39.692
a generic heterogeneity argument

0:07:40.022,0:07:41.760
being used to excuse something

0:07:41.760,0:07:43.800
that I think might be [br]a deeper problem.

0:07:45.190,0:07:47.358
- [Guido] I think it played [br]somewhat of an unfortunate role

0:07:47.358,0:07:50.083
in the discussions[br]between reduced form

0:07:50.083,0:07:51.700
and structural approaches,

0:07:51.700,0:07:55.510
where I feel [br]that wasn't quite right.

0:07:56.090,0:07:58.810
The instrumental [br]variables assumptions

0:07:58.810,0:08:01.510
are at the core,

0:08:01.510,0:08:03.483
structural assumptions[br]about behavior --

0:08:03.483,0:08:05.200
they were coming from economic...

0:08:07.100,0:08:09.900
thinking about the economic[br]behavior of agents,

0:08:10.300,0:08:15.100
and somehow it got pushed[br]in a direction

0:08:15.100,0:08:17.600
that I think wasn't [br]really very helpful.

0:08:20.426,0:08:21.663
I think, initially,

0:08:22.800,0:08:24.067
we wrote things up,

0:08:24.067,0:08:26.480
it was describing[br]what was happening.

0:08:26.480,0:08:29.783
There were a set of methods [br]people were using.

0:08:29.783,0:08:32.111
We clarified what [br]those methods were doing

0:08:32.811,0:08:38.361
in a way that I think contain[br]a fair amount of insight.

0:08:39.100,0:08:42.050
But it somehow [br]got pushed into a corner

0:08:42.050,0:08:45.379
that I don't think [br]was necessarily very helpful.

0:08:45.379,0:08:48.604
- In just the language[br]of reduced form versus structural,

0:08:48.604,0:08:50.306
I find kind of funny in the sense

0:08:50.306,0:08:52.985
that the local average[br]treatment effect model,

0:08:52.985,0:08:54.154
the potential outcomes model

0:08:54.154,0:08:56.110
is a nonparametric[br]structural model,

0:08:56.110,0:08:58.600
if you want to think about it, [br]as you suggested, Guido.

0:08:58.600,0:09:01.129
So there's something a little funny

0:09:01.129,0:09:03.505
about putting these [br]two things in opposition when --

0:09:03.505,0:09:05.116
- [Guido] Yes.[br]- [Josh] That language, of course,

0:09:05.116,0:09:08.371
comes from the simultaneous[br]equations framework

0:09:08.371,0:09:09.641
that we inherited.

0:09:10.400,0:09:11.440
It has the advantage

0:09:11.440,0:09:14.085
that people seem to know [br]what you mean when you use it,

0:09:14.085,0:09:15.164
but that might be

0:09:16.559,0:09:18.300
that different people[br]are hearing different things.

0:09:18.300,0:09:20.480
- [Guido] Yeah. I think[br]reduced form has become

0:09:20.480,0:09:22.200
used in a little bit [br]of the pejorative way...

0:09:22.200,0:09:23.540
- [Josh] Sometimes.

0:09:25.104,0:09:28.250
...which is not really quite what [br]it was originally intended for.

0:09:30.100,0:09:33.090
- [Isaiah] I guess something else[br]that strikes me in thinking about

0:09:33.090,0:09:35.645
the effects of the local average[br]treatment effect framework

0:09:35.645,0:09:37.676
is that often folks will appeal

0:09:37.676,0:09:40.000
to a local average treatment[br]effects intuition

0:09:40.000,0:09:42.358
for settings well beyond ones

0:09:42.358,0:09:44.963
where any sort of formal result[br]has actually been established.

0:09:45.440,0:09:49.180
And I'm curious, given all the work[br]that you guys did

0:09:49.180,0:09:52.390
to establish LATE results [br]in different settings,

0:09:52.390,0:09:54.415
I'm curious, any thoughts on that?

0:09:55.360,0:09:57.306
- I think there's going[br]to be a lot of cases

0:09:57.306,0:10:02.200
where the intuition [br]does get you some distance,

0:10:02.800,0:10:04.989
but it's going to be [br]somewhat limited,

0:10:04.989,0:10:07.600
and establishing [br]formal results there

0:10:08.400,0:10:09.490
may be a little tricky

0:10:09.490,0:10:12.700
and then maybe only work [br]in special circumstances,

0:10:14.600,0:10:16.540
and you end up[br]with a lot of formality

0:10:16.540,0:10:19.500
that may not quite [br]capture the intuition.

0:10:19.900,0:10:21.550
Sometimes I'm somewhat [br]uneasy with them,

0:10:21.550,0:10:24.438
and they are not necessarily[br]the papers I would want to write,

0:10:25.148,0:10:27.819
but I do think intuition

0:10:27.819,0:10:30.912
often does capture [br]part of the problem.

0:10:33.100,0:10:36.300
I think, in some sense,[br]we were very fortunate there

0:10:36.900,0:10:39.250
in the way that the LATE paper [br]got handled at the journal,

0:10:39.250,0:10:41.766
so that, actually, the editor,[br]made it much shorter

0:10:42.100,0:10:46.300
and that allowed us to focus[br]on very clear, crisp results.

0:10:49.924,0:10:51.770
There's a somewhat[br]unfortunate tendency

0:10:51.770,0:10:53.118
in the econometrics literature

0:10:53.118,0:10:54.870
of having the papers [br]get longer and longer.

0:10:54.870,0:10:56.550
- Well, you should be able[br]to fix that, man.

0:10:56.550,0:10:58.915
- I'm trying to fix that.[br][laughter]

0:10:58.915,0:11:01.510
But I think this is an example [br]where it's very clear

0:11:01.510,0:11:03.097
that having it be short[br]is actually --

0:11:03.097,0:11:04.750
- You should have imposed[br]that no paper

0:11:04.750,0:11:06.802
can be longer than the LATE paper.

0:11:07.269,0:11:09.617
- That... wow! That may be great.

0:11:09.617,0:11:11.685
- At least no theory, [br]no theory paper.

0:11:11.892,0:11:13.339
- Yeah, and I think...

0:11:14.500,0:11:16.800
I'm trying very hard to get[br]the papers to be shorter,

0:11:16.800,0:11:19.514
and I think there is a lot[br]of value today

0:11:19.514,0:11:21.573
because it's often[br]the second part of the paper

0:11:21.573,0:11:25.049
that doesn't actually[br]get you much further

0:11:25.049,0:11:26.395
in understanding things,

0:11:27.000,0:11:29.870
and it does make things [br]much harder to read.

0:11:32.426,0:11:36.179
It goes back to how I think[br]econometrics should be done.

0:11:36.179,0:11:38.070
You should focus on --

0:11:38.700,0:11:41.300
It should be reasonably[br]close to empirical problems.

0:11:41.500,0:11:43.900
They should be very clear problems.

0:11:44.800,0:11:48.900
But then often the theory[br]doesn't need to be quite so long.

0:11:48.900,0:11:50.010
- [Josh] Yeah.

0:11:51.100,0:11:54.670
- I think things have gone[br]a little off track.

0:11:56.260,0:11:57.750
- [Isaiah] A relatively [br]recent change

0:11:57.750,0:12:00.230
has been a seeming[br]big increase in demand

0:12:00.230,0:12:03.765
for people with econometrics[br]causal effect estimation skills

0:12:03.765,0:12:04.994
in the tech sector.

0:12:04.994,0:12:07.563
I'm interested, [br]do either of you have thoughts

0:12:07.563,0:12:09.840
of how that's going to interact

0:12:09.840,0:12:11.600
with the development [br]of empirical methods

0:12:11.600,0:12:13.950
or empirical research[br]in economics going forward?

0:12:14.600,0:12:16.770
- [Josh] Well, there's[br]sort of a meta point,

0:12:16.770,0:12:21.000
which is there's this new [br]kind of employer,

0:12:21.800,0:12:27.530
the Amazons and the Uber,[br]and the TripAdvisor world,

0:12:28.000,0:12:29.300
and I think that's great.

0:12:29.300,0:12:32.030
I like to tell my students[br]about that.

0:12:32.600,0:12:35.500
At MIT, we have a lot[br]of computer science majors --

0:12:35.500,0:12:37.000
that's our biggest major.

0:12:37.400,0:12:42.362
I try to seduce some of those folks[br]into economics by saying

0:12:43.228,0:12:46.700
you can go work for these companies

0:12:46.700,0:12:49.250
that people[br]are very keen to work for

0:12:49.250,0:12:50.800
because the work seems exciting,

0:12:52.000,0:12:54.250
that the skills [br]that you get in econometrics

0:12:54.250,0:12:56.100
are as good or better

0:12:56.100,0:12:59.736
than any competing [br]discipline has to offer.

0:12:59.736,0:13:02.143
So you should at least[br]take some econ --

0:13:02.143,0:13:04.290
take some econometrics[br]and some econ.

0:13:04.800,0:13:07.000
I did a fun project with Uber

0:13:07.600,0:13:09.501
on labor supply of Uber drivers,

0:13:09.920,0:13:12.805
and it was very, very exciting[br]to be part of that.

0:13:13.100,0:13:15.400
Plus, I got to drive [br]for Uber for a while,

0:13:15.900,0:13:17.730
and I thought that was fun too.

0:13:17.730,0:13:19.231
I did not make enough

0:13:19.231,0:13:22.616
that I was tempted[br]to give up my MIT job,

0:13:22.616,0:13:25.100
but I enjoyed the experience.

0:13:25.230,0:13:27.534
I see a potential challenge

0:13:27.534,0:13:30.900
to our model[br]of graduate education here,

0:13:31.700,0:13:36.068
which is, if we're training people[br]to go work at Amazon,

0:13:37.900,0:13:41.190
it's not clear why[br]we should be paying

0:13:41.190,0:13:42.900
graduate stipends for that.

0:13:43.200,0:13:47.280
Why should the taxpayer[br]effectively be subsidizing that.

0:13:47.280,0:13:51.180
Our graduate education[br]in the US is generously subsidized,

0:13:51.180,0:13:53.272
even in private universities --

0:13:54.906,0:13:56.477
there's a lot[br]of public money there.

0:13:56.477,0:13:59.300
And I think the traditional[br]rationale for that is,

0:13:59.643,0:14:01.992
we were training [br]educators and scholars,

0:14:01.992,0:14:05.657
and there's a great externality[br]from the work that we do,

0:14:05.657,0:14:07.910
it's either [br]the research externality,

0:14:07.910,0:14:09.557
or a teaching externality.

0:14:10.100,0:14:13.389
But if many of our students[br]are going to work

0:14:13.389,0:14:14.688
in the private sector --

0:14:16.300,0:14:17.414
that's fine,

0:14:19.000,0:14:21.700
but maybe their employers[br]should pay for them.

0:14:22.120,0:14:23.370
- For me, it's just so different

0:14:23.370,0:14:26.780
from people working [br]for consulting firms.

0:14:27.200,0:14:28.780
It's not clear to me

0:14:28.780,0:14:32.836
that the number of jobs[br]in academics has changed.

0:14:33.370,0:14:36.325
- I feel like this is[br]a growing sector,

0:14:36.325,0:14:37.807
whereas consulting...

0:14:37.807,0:14:39.289
You're right to raise that.

0:14:39.289,0:14:42.100
It might be the same [br]for consulting.

0:14:44.846,0:14:47.500
I'm placing more and more [br]students in these businesses,

0:14:47.500,0:14:49.467
so it's on my mind, in a way,

0:14:49.467,0:14:54.212
that I've not been attentive [br]to consulting jobs.

0:14:54.212,0:14:56.920
Consulting was always important,

0:14:56.920,0:14:58.950
and I think also [br]there's some movement

0:14:58.950,0:15:01.140
from consulting back[br]into research --

0:15:01.140,0:15:02.723
it's a little more fluid.

0:15:03.900,0:15:07.630
A lot of the work in both domains,

0:15:07.630,0:15:09.430
I have to say, [br]it's not really different,

0:15:09.430,0:15:12.730
but people who are working[br]in the tech sector

0:15:12.730,0:15:15.480
are doing things that are [br]potentially of scientific interest,

0:15:15.480,0:15:16.800
but mostly it's hidden.

0:15:17.100,0:15:18.550
Then you really have to say,

0:15:18.550,0:15:20.900
why is the government[br]paying for this?

0:15:22.104,0:15:23.732
- Although, to Guido's point,

0:15:23.732,0:15:26.783
I guess there's [br]a data question here of

0:15:26.783,0:15:32.772
has the total [non-EC][br]for-profit sector employment

0:15:32.772,0:15:35.595
of econ Ph.D. program[br]graduates increased

0:15:35.595,0:15:38.290
or has it just been[br]a substitution from finance

0:15:38.290,0:15:40.300
and consulting towards tech?

0:15:40.300,0:15:42.300
- I may be reacting to something

0:15:42.300,0:15:44.400
that's not really happening.

0:15:44.400,0:15:45.890
- I've actually done some work

0:15:45.890,0:15:48.200
with some of these tech companies.

0:15:49.100,0:15:52.200
I don't disagree with Josh's point [br]that we need to think

0:15:52.200,0:15:53.675
a little bit about [br]the funding model

0:15:53.675,0:15:56.390
who is, in the end, paying [br]for the graduate education.

0:15:56.913,0:15:59.400
But from a scientific perspective,

0:15:59.980,0:16:02.840
not only do these places[br]have great data,

0:16:02.840,0:16:05.112
and nowadays they tend to be[br]very careful with that

0:16:05.112,0:16:07.100
for privacy reasons,

0:16:07.380,0:16:08.900
but they also have great questions.

0:16:10.200,0:16:13.929
I find it very inspiring to listen[br]to the people there

0:16:13.929,0:16:15.950
and see what kind[br]of questions they have,

0:16:15.950,0:16:17.330
and often they're questions

0:16:18.200,0:16:21.241
that also come up[br]outside of these companies.

0:16:21.241,0:16:27.430
I have a couple of papers [br]with Raj Chetty and Susan Athey

0:16:27.430,0:16:31.600
where we look at ways [br]of combining experimental data

0:16:31.600,0:16:33.274
and observational data.

0:16:35.500,0:16:38.600
Raj Chetty was interested[br]in what is the effect

0:16:38.600,0:16:42.893
of early childhood programs [br]on outcomes later in life,

0:16:42.893,0:16:46.330
not just on test scores[br]but on earnings and stuff,

0:16:46.330,0:16:48.300
and we kind of developed methods

0:16:48.600,0:16:51.500
that would help you shed light[br]on that under some...

0:16:52.718,0:16:53.868
in some settings,

0:16:53.868,0:16:56.920
and the same problems came up

0:16:56.920,0:17:00.533
in these tech company settings.

0:17:00.800,0:17:03.240
And so, from my perspective,

0:17:03.240,0:17:05.420
it's the same kind of --

0:17:05.420,0:17:07.600
I was talking to people [br]doing empirical work.

0:17:07.600,0:17:09.700
I tried to kind of look[br]at these specific problems

0:17:09.700,0:17:13.370
and then try to come up[br]with more general problems,

0:17:15.110,0:17:18.230
reformulating the problems[br]at a higher level

0:17:18.500,0:17:22.900
so that I can think about solutions[br]that work in a range of settings.

0:17:23.400,0:17:24.933
And so from that perspective,

0:17:24.933,0:17:27.570
the interactions [br]with the tech companies

0:17:27.570,0:17:30.300
are just very valuable[br]and very useful.

0:17:31.700,0:17:35.204
We do have students now[br]doing internships there

0:17:35.204,0:17:38.516
and then coming back [br]and writing more interesting theses

0:17:38.516,0:17:43.400
as a result of[br]their experiences there.

0:17:44.600,0:17:47.020
- [Narrator] If you'd like to watch[br]more Nobel Conversations,

0:17:47.020,0:17:48.200
click here.

0:17:48.200,0:17:50.500
Or if you'd like to learn[br]more about econometrics,

0:17:50.500,0:17:53.100
check out Josh's [br]"Mastering Econometrics" series.

0:17:53.700,0:17:56.720
If you'd like to learn more [br]about Guido, Josh, and Isaiah,

0:17:56.720,0:17:58.300
check out the links [br]in the description.

0:17:59.036,0:18:01.495
♪ [music] ♪