0:00:00.900,0:00:02.760
♪ [music] ♪

0:00:03.800,0:00:05.700
<i>- [Narrator] Welcome to [br]Nobel conversations.</i>

0:00:07.300,0:00:10.240
<i>In this episode, Josh Angrist [br]and Guido Imbens,</i>

0:00:10.240,0:00:12.000
<i>sit down with Isaiah Andrews</i>

0:00:12.000,0:00:14.303
<i>to discuss how the research[br]was initially received</i>

0:00:14.900,0:00:17.090
<i>and how they responded [br]to criticism.</i>

0:00:18.700,0:00:21.540
At the time, did you feel like[br]you are on to something,

0:00:21.540,0:00:25.152
you felt this was the beginning[br]of a whole line of work

0:00:25.152,0:00:27.202
that you felt like was going [br]to be important or...?

0:00:27.600,0:00:30.100
Not so much that it was[br]a whole line of work,

0:00:30.100,0:00:32.417
but certainly I felt like, [br]"Wow, this--"

0:00:32.417,0:00:35.045
- [Josh] We proved something [br]that people didn't know before,

0:00:35.045,0:00:36.430
that it was worth knowing.

0:00:36.430,0:00:41.143
Yeah, going back compared to [br]my job market papers having--

0:00:41.600,0:00:45.560
I felt this was actually [br]a very clear crisp result.

0:00:46.400,0:00:49.530
But there were definitely [br]was mixed reception

0:00:49.530,0:00:52.420
and I don't think anybody [br]said that,

0:00:52.420,0:00:55.461
"Oh, wow, this is already,[br]something."

0:00:55.461,0:00:59.386
No, which is the nightmare scenario[br]for a researcher

0:01:00.300,0:01:02.259
where you think you've [br]discovered something

0:01:02.283,0:01:04.461
and then somebody else says, [br]"Oh, I knew that."

0:01:05.000,0:01:07.412
But there were definitely was [br]a need to convince people

0:01:07.430,0:01:10.346
that this was worth knowing,[br]that instrumental variables

0:01:10.370,0:01:12.712
estimates a causal effect [br]for compliers.

0:01:13.200,0:01:16.178
Yeah, but even though it[br]took a long time

0:01:16.178,0:01:19.348
to convince a bigger audience,

0:01:19.820,0:01:24.346
sometimes even fairly quickly, [br]the reception was pretty good

0:01:24.800,0:01:26.645
among a small group of people.

0:01:27.200,0:01:31.297
Gary, clearly liked it a lot [br]from the beginning

0:01:31.800,0:01:35.645
and I remember, because at that point[br]Josh had left for Israel,

0:01:35.645,0:01:38.886
but I remember explaining it [br]to Don Ruben

0:01:39.696,0:01:43.700
and he was like, [br]"Yeah, this really is something here."

0:01:43.700,0:01:47.100
Not right away though,[br]Don took some convincing.

0:01:47.500,0:01:49.210
By the time you got to Don,

0:01:49.210,0:01:51.226
there have been some back[br]and forth with him

0:01:51.226,0:01:53.304
and in correspondence actually.

0:01:53.700,0:01:57.103
But I remember at some point [br]getting a call or email from him

0:01:57.103,0:02:00.020
saying that he was sitting [br]at the airport in Rome

0:02:00.020,0:02:03.700
and looking at the paper [br]and thinking,

0:02:03.700,0:02:07.000
"Yeah, no actually, [br]you guys are onto something."

0:02:07.490,0:02:08.744
We were happy about

0:02:08.744,0:02:10.670
but that took longer [br]than I think you remember.

0:02:10.800,0:02:12.500
Yeah, it wasn't right away

0:02:12.500,0:02:13.700
[laughter]

0:02:13.700,0:02:16.500
because I know that I was back [br]in Israel by the time that happened.

0:02:16.500,0:02:18.400
I'd left for Israel in the summer--

0:02:19.390,0:02:22.600
I was only at Harvard for two years.[br]We had that one year.

0:02:22.600,0:02:25.700
It is remarkable, I mean, that[br]one year was so fateful for us.

0:02:25.900,0:02:27.200
- [Guido] Yes.

0:02:27.690,0:02:30.200
I think we understood there was[br]something good happening,

0:02:30.200,0:02:34.000
but maybe we didn't think it was[br]life-changing, only in retrospect.

0:02:34.000,0:02:35.720
♪ [music] ♪

0:02:35.720,0:02:37.495
- [Isaiah] As you said, it sounds like [br]a small group of people

0:02:37.495,0:02:39.190
were initially quite receptive,

0:02:39.190,0:02:42.190
perhaps took some time for[br]a broader group of people

0:02:43.090,0:02:46.022
to come around to seeing [br]the LATE framework

0:02:46.022,0:02:47.700
as a valuable way to look [br]at the world.

0:02:47.700,0:02:50.100
I guess, in over the course of that,

0:02:50.100,0:02:51.888
were their periods [br]where you thought,

0:02:51.888,0:02:54.290
maybe the people saying [br]this wasn't a useful way

0:02:54.290,0:02:55.751
to look at the world were right?

0:02:55.751,0:02:58.400
Did you get discouraged? [br]How did you think about?

0:02:58.400,0:03:00.900
I don't think I was discouraged[br]but the people who were saying

0:03:00.900,0:03:05.900
that we're smart people, [br]well informed econometricians,

0:03:05.900,0:03:07.800
sophisticated readers

0:03:08.900,0:03:11.800
and I think the substance[br]of the comment was,

0:03:11.800,0:03:14.297
this is not what econometrics [br]is about.

0:03:14.297,0:03:20.572
Econometrics was being transmitted [br]at that time was about structure.

0:03:21.300,0:03:24.290
There was this idea that[br]there's structure in the economy

0:03:25.100,0:03:27.200
and it's our job to discover it

0:03:27.200,0:03:30.952
and what makes it structure[br]is it's essentially invariant

0:03:31.900,0:03:34.900
and so we're saying, [br]in the LATE theorem,

0:03:34.900,0:03:37.699
that every instrument produces[br]its own causal effect,

0:03:37.699,0:03:41.386
which is in contradiction to that[br]to some extent

0:03:41.386,0:03:43.802
and so that was [br]where the tension was.

0:03:43.802,0:03:45.551
People didn't want [br]to give up that idea.

0:03:46.300,0:03:50.203
Yeah, I remember once [br]people were started

0:03:51.200,0:03:55.784
arguing more more vocally [br]against that,

0:03:56.900,0:03:59.483
that never really [br]bothered me that much.

0:03:59.483,0:04:03.051
It seemed clear that [br]we had a result there

0:04:03.051,0:04:06.048
and it became somewhat [br]controversial,

0:04:06.048,0:04:08.395
but controversial in a good way.

0:04:08.620,0:04:10.190
It was clear that people felt

0:04:10.700,0:04:13.835
they had to come out against it [br]because--

0:04:13.970,0:04:15.959
Well, I think what [br]we think it's good now

0:04:17.426,0:04:19.238
we might not have loved it [br]at the time.

0:04:20.168,0:04:22.984
I remember being somewhat,[br]the more upset--

0:04:22.984,0:04:24.910
there was some dinner [br]where someone said,

0:04:24.910,0:04:27.655
"No, no, that paper with Josh,

0:04:27.655,0:04:30.808
that was doing a disservice[br]to the profession."

0:04:32.050,0:04:33.850
We definitely had [br]reactions like that.

0:04:35.410,0:04:38.200
At some level, that may be [br]indicative of the culture

0:04:38.400,0:04:40.000
in general in economics [br]at the time.

0:04:41.400,0:04:44.097
I thought back later, [br]what if that'd happened now,

0:04:44.600,0:04:47.682
if I was a senior person sitting[br]in that conversation,

0:04:48.300,0:04:51.898
I would call that out because it[br]really was not appropriate--

0:04:53.000,0:04:54.200
- [Josh] It wasn't so bad.

0:04:54.600,0:04:56.600
I think the criticism is--

0:04:57.700,0:04:59.298
It wasn't completely misguided,

0:05:00.210,0:05:01.351
it was maybe wrong.

0:05:01.800,0:05:04.485
No, no, but you can say [br]the paper is wrong

0:05:05.400,0:05:06.600
but it's saying that

0:05:06.600,0:05:08.112
it's a disservice [br]to the profession,

0:05:08.128,0:05:09.300
that's not really--

0:05:09.300,0:05:10.300
Personal.

0:05:10.300,0:05:12.646
Yes, and doing that, not to me,

0:05:12.646,0:05:14.612
but in front of [br]my senior colleagues.

0:05:15.391,0:05:17.489
But nobody was saying [br]the result was wrong

0:05:17.489,0:05:18.700
and I remember also,

0:05:18.700,0:05:21.699
some of the comments [br]were thought-provoking

0:05:21.699,0:05:23.159
so we had some negative reviews,

0:05:23.159,0:05:25.861
I think on the average [br]causal response paper.

0:05:26.500,0:05:30.361
Somebody said, "These compliers[br]you can't figure out who they are."

0:05:31.450,0:05:33.000
Right, it's one thing to say

0:05:33.000,0:05:35.040
you're estimating [br]the effect of treatment

0:05:35.040,0:05:36.765
on the treated [br]or something like that.

0:05:36.788,0:05:38.400
You can tell me who's treated,

0:05:38.700,0:05:42.289
people in the CPS,[br]you can't tell me who's a complier.

0:05:42.929,0:05:44.679
So that was a legitimate challenge.

0:05:44.679,0:05:47.800
That's certainly fair [br]and I can see why

0:05:49.880,0:05:53.700
that part made people[br]a little uneasy and uncomfortable.

0:05:54.300,0:05:56.400
But at the same time

0:05:56.900,0:06:00.244
because it showed that you couldn't[br]really go beyond that,

0:06:00.800,0:06:03.775
it was very useful thing [br]to realize.

0:06:04.630,0:06:06.500
I remember on the day,

0:06:06.500,0:06:09.911
we got to the key result [br]that I was thinking,

0:06:09.911,0:06:14.221
"Wow, this is as good as it gets.

0:06:14.221,0:06:16.978
Here we actually have an insight[br]but clearly--"

0:06:17.500,0:06:19.760
And we had to sell it [br]at some point.

0:06:19.760,0:06:22.000
For quite a few years, [br]we had to sell it

0:06:23.480,0:06:24.892
and it's proven to be quite useful.

0:06:25.500,0:06:28.761
I don't think we understood that[br]it would be so useful at the time.

0:06:28.761,0:06:29.871
No.

0:06:30.170,0:06:34.600
I did feel early on this was[br]a substantial insight.

0:06:34.600,0:06:36.440
- [Josh] Yeah we [did] something.

0:06:36.440,0:06:40.041
But I did not think [br]goals were there.

0:06:40.500,0:06:42.400
I don't think we were aiming [br]for the Nobel.

0:06:42.650,0:06:43.650
[laughter]

0:06:43.730,0:06:46.243
We were very happy to get[br]that note in <i>Econometrica.</i>

0:06:46.859,0:06:48.829
♪ [music] ♪

0:06:49.900,0:06:51.500
- [Isaiah] Are there factors [br]or are ways of approaching problems

0:06:51.500,0:06:54.346
that lead people to be better [br]at recognizing the good stuff

0:06:54.346,0:06:56.600
and taking the time to do it [br]as opposed to dismissing it?

0:06:56.600,0:06:57.700
- [Josh] Sometimes [br]I think it's helpful.

0:06:57.700,0:06:59.672
If you're trying to [br]convince somebody

0:06:59.698,0:07:01.355
that you have something [br]useful to say

0:07:01.900,0:07:04.026
and maybe they don't [br]speak your language,

0:07:04.894,0:07:06.761
you might need [br]to learn their language.

0:07:06.761,0:07:08.020
Yes, yes, exactly.

0:07:08.020,0:07:11.736
That's what we did with Don,[br]we figured out how to--

0:07:11.736,0:07:14.052
I remember we had a very hard time

0:07:14.052,0:07:15.686
explaining the exclusion [br]restriction to Don,

0:07:17.430,0:07:18.993
maybe rightfully so,

0:07:19.804,0:07:22.244
I think Guido and I [br]eventually figured out

0:07:22.510,0:07:24.420
that it wasn't formulated [br]very clearly,

0:07:25.400,0:07:27.450
and we came up [br]with a way to do that

0:07:27.450,0:07:29.456
in the potential outcomes framework

0:07:29.456,0:07:32.218
that I think worked[br]for the three of us.

0:07:32.218,0:07:33.419
Yeah.

0:07:33.419,0:07:35.544
Well, it worked for[br]the bigger literature

0:07:35.544,0:07:37.730
but I think what you're saying [br]there is exactly right,

0:07:37.749,0:07:41.080
you need to figure out [br]how not just say,

0:07:41.080,0:07:43.894
"Okay well, I've got this language[br]and this this works great

0:07:43.894,0:07:45.900
and I've got to convince someone[br]else to use the language.

0:07:45.900,0:07:48.357
You could first figure out [br]what language they're using

0:07:48.680,0:07:50.738
and then only then, [br]can you try to say,

0:07:50.738,0:07:53.270
"Wow, but here you thinking of it [br]this way,"

0:07:53.270,0:07:56.880
but that's actually [br]a pretty hard thing to do,

0:07:56.880,0:07:59.268
you get someone from [br]a different discipline,

0:07:59.268,0:08:02.300
convincing them, two junior faculty[br]in a different department

0:08:02.300,0:08:04.366
actually have something [br]to say to you,

0:08:04.366,0:08:06.600
that's that takes [br]a fair amount of effort.

0:08:07.500,0:08:10.240
Yeah, I wrote Don [br]a number of times,

0:08:10.240,0:08:12.048
in fairly long letters.

0:08:12.048,0:08:13.805
I remember thinking [br]this is worth doing,

0:08:14.600,0:08:16.006
that if I could convince Don

0:08:16.780,0:08:19.444
that would validate the framework [br]to some extent.

0:08:20.300,0:08:23.174
I think both you and Don were

0:08:23.174,0:08:25.000
a little bit more confident [br]that you were right.

0:08:25.000,0:08:26.438
Well, we used to argue a lot

0:08:26.438,0:08:28.500
and you would sometimes [br]referee those.

0:08:28.500,0:08:29.500
[laughter]

0:08:29.800,0:08:30.800
That was fun.

0:08:32.580,0:08:33.967
It wasn't hurtful.

0:08:35.200,0:08:37.935
I remember it getting [br]a little testy once,

0:08:37.935,0:08:40.076
we had lunch in The Faculty Club

0:08:40.600,0:08:44.077
and we're talking about [br]the draft lottery paper.

0:08:44.930,0:08:47.700
We were talking about never takes

0:08:47.700,0:08:51.000
as people wounded serve [br]in the military irrespective of

0:08:51.000,0:08:53.500
whether they were getting drafted

0:08:54.500,0:08:58.831
and you or Don said something[br]about shooting yourself in the foot,

0:08:59.800,0:09:01.530
as a way of getting [br]out of the military

0:09:01.530,0:09:03.400
and that may be [br]the exclusion restriction

0:09:03.400,0:09:05.223
for never takes wasn't working

0:09:06.300,0:09:08.560
and then the other one would say,

0:09:08.560,0:09:09.951
"Well, yes you could do that

0:09:09.951,0:09:12.128
but why would you want [br]to shoot yourself in the foot?"

0:09:12.128,0:09:13.225
[laughter]

0:09:13.225,0:09:15.400
It got a little [out of hand] there.

0:09:15.400,0:09:17.600
I usually go for moving to Canada,[br]for my example,

0:09:18.690,0:09:20.096
when I'm teaching that.

0:09:22.030,0:09:24.100
But things are tricky,

0:09:24.860,0:09:26.595
I get students coming[br]from computer science

0:09:26.595,0:09:29.943
and they want to do things [br]on causal inference

0:09:30.400,0:09:33.293
and it takes a huge amount [br]of effort to figure out

0:09:33.460,0:09:35.230
how they actually thinking [br]about problem

0:09:35.230,0:09:37.000
and whether there's something there

0:09:37.000,0:09:38.310
and so, now over the years,

0:09:38.310,0:09:40.242
I've got a little more appreciation[br]for the fact

0:09:40.250,0:09:41.958
that Don was actually willing to--

0:09:42.200,0:09:46.000
It took him a while, [br]but he did engage first with Josh

0:09:46.400,0:09:47.500
and then with both of us

0:09:48.380,0:09:50.413
and rather than dismissing[br]and say,

0:09:50.413,0:09:53.348
"Okay, well I can't figure out[br]what these guys are doing

0:09:53.368,0:09:56.195
and it's probably just [br]not really that interesting."

0:09:57.200,0:09:59.736
Everybody always wants [br]to figure out quickly,

0:09:59.736,0:10:01.376
you want to save time

0:10:01.376,0:10:04.133
and you want to save your brain cells[br]for other things.

0:10:05.000,0:10:07.000
The fastest route to [br]that is to figure out

0:10:07.000,0:10:08.460
why you should dismiss something.

0:10:08.460,0:10:09.560
Yes.

0:10:09.560,0:10:11.300
I don't need to spend time on this.

0:10:11.300,0:10:12.670
♪ [music] ♪

0:10:12.670,0:10:14.070
<i>- [Narrator] If you'd like [br]to watch more </i>

0:10:14.070,0:10:15.737
<i>Nobel conversations, click here,</i>

0:10:16.300,0:10:18.354
<i>or if you'd like to learn[br]more about econometrics,</i>

0:10:18.700,0:10:21.300
<i>check out Josh's "Mastering[br]Econometrics" series.</i>

0:10:21.800,0:10:24.700
<i>If you'd like to learn more[br]about Guido, Josh, and Isaiah</i>

0:10:24.860,0:10:26.702
<i>check out the links [br]in the description.</i>

0:10:26.702,0:10:28.017
♪ [music] ♪