WEBVTT

00:00:00.130 --> 00:00:02.200
♪ [music] ♪

00:00:03.645 --> 00:00:05.600
- [Narrator] Welcome
to Nobel Conversations.

00:00:07.250 --> 00:00:10.240
In this episode, Josh Angrist 
and Guido Imbens

00:00:10.240 --> 00:00:11.920
sit down with Isaiah Andrews

00:00:11.920 --> 00:00:14.303
to discuss how their research
was initially received

00:00:14.999 --> 00:00:17.090
and how they responded 
to criticism.

00:00:18.700 --> 00:00:20.380
- [Isaiah] At the time, 
did you feel like

00:00:20.380 --> 00:00:21.627
you were on to something,

00:00:21.627 --> 00:00:25.152
you felt this was the beginning
of a whole line of work

00:00:25.152 --> 00:00:27.202
that you felt like was going 
to be important or...?

00:00:27.600 --> 00:00:30.000
- [Guido] Not so much
that it was a whole line of work,

00:00:30.000 --> 00:00:31.894
but certainly, I felt like, 
"Wow, this --"

00:00:32.277 --> 00:00:35.045
- [Josh] We've proved something 
we didn't know before,

00:00:35.045 --> 00:00:36.114
that it was worth knowing.

00:00:36.114 --> 00:00:38.033
- Yeah, going back to the...

00:00:38.741 --> 00:00:41.080
compared to my job market
paper or something --

00:00:41.080 --> 00:00:45.560
No, I felt this was actually 
a very clear, crisp result.

00:00:46.400 --> 00:00:49.657
- But there was definitely 
a mixed reception,

00:00:49.657 --> 00:00:52.420
and I don't think
anybody said that,

00:00:52.420 --> 00:00:55.487
"Oh, well, this is 
already something

00:00:56.043 --> 00:00:59.386
which is the nightmare scenario
for a researcher,

00:01:00.230 --> 00:01:02.003
where you think
you've discovered something

00:01:02.003 --> 00:01:04.461
and then somebody else says, 
'Oh, I knew that.'"

00:01:05.000 --> 00:01:07.220
But there definitely was 
a need to convince people

00:01:07.220 --> 00:01:10.370
that this was worth knowing,
that instrumental variables

00:01:10.370 --> 00:01:12.687
estimates a causal effect 
for compliers.

00:01:13.200 --> 00:01:16.178
- Yeah, but even though
it took a long time

00:01:16.178 --> 00:01:19.348
to convince a bigger audience,

00:01:19.820 --> 00:01:24.346
sometimes even fairly quickly, 
the reception was pretty good

00:01:24.800 --> 00:01:26.645
among a small group of people.

00:01:27.200 --> 00:01:31.297
Gary clearly liked it a lot 
from the beginning,

00:01:31.800 --> 00:01:33.289
and I remember...

00:01:33.289 --> 00:01:35.645
because at that point
Josh had left for Israel,

00:01:35.645 --> 00:01:38.886
but I remember explaining it 
to Don Rubin,

00:01:39.696 --> 00:01:43.700
and he was like, "You know, 
this really is something here."

00:01:43.700 --> 00:01:45.197
- Not right away though.

00:01:45.932 --> 00:01:47.173
Don took some convincing.

00:01:47.500 --> 00:01:49.150
By the time you got to Don,

00:01:49.150 --> 00:01:51.226
there have been 
some back and forth with him

00:01:51.226 --> 00:01:53.304
and in correspondence, actually.

00:01:53.700 --> 00:01:57.103
- But I remember at some point 
getting a call or email from him

00:01:57.103 --> 00:02:00.020
saying that he was sitting 
at the airport in Rome

00:02:00.020 --> 00:02:03.700
and looking at the paper 
and thinking,

00:02:03.700 --> 00:02:07.000
"Yeah, no, actually, 
you guys are onto something."

00:02:07.490 --> 00:02:08.594
- We were happy about that.

00:02:08.594 --> 00:02:10.550
But that took longer 
than I think you remember.

00:02:11.030 --> 00:02:12.500
It wasn't right away.

00:02:12.500 --> 00:02:13.700
[laughter]

00:02:13.700 --> 00:02:15.325
Because I know
that I was back in Israel

00:02:15.325 --> 00:02:16.627
by the time that happened.

00:02:16.627 --> 00:02:18.750
I'd left for Israel 
in the summer of --

00:02:19.390 --> 00:02:21.190
I was only at Harvard 
for two years.

00:02:21.190 --> 00:02:22.540
We had that one year.

00:02:22.540 --> 00:02:25.700
It is remarkable, I mean, that
one year was so fateful for us.

00:02:25.900 --> 00:02:27.200
- [Guido] Yes.

00:02:27.690 --> 00:02:30.200
I think we understood there was
something good happening,

00:02:30.200 --> 00:02:33.700
but maybe we didn't think it was
life-changing, only in retrospect.

00:02:33.700 --> 00:02:35.620
♪ [music] ♪

00:02:35.620 --> 00:02:37.495
- [Isaiah] As you said, it sounds
like a small group of people

00:02:37.495 --> 00:02:39.190
were initially quite receptive.

00:02:39.190 --> 00:02:42.190
It perhaps took some time
for a broader group of people

00:02:43.090 --> 00:02:45.912
to come around to seeing 
the LATE framework

00:02:45.912 --> 00:02:47.620
as a valuable way to look 
at the world.

00:02:47.620 --> 00:02:49.331
I guess, in over
the course of that,

00:02:50.100 --> 00:02:52.128
were there periods 
where you thought

00:02:52.128 --> 00:02:54.450
maybe the people saying 
this wasn't a useful way

00:02:54.450 --> 00:02:55.751
to look at the world were right?

00:02:55.751 --> 00:02:58.360
Did you get discouraged? 
How did you think about?

00:02:58.360 --> 00:02:59.721
- I don't think I was discouraged,

00:02:59.721 --> 00:03:01.119
but the people who were saying that

00:03:01.119 --> 00:03:06.117
were smart people,
well-informed econometricians,

00:03:06.117 --> 00:03:07.800
sophisticated readers,

00:03:08.900 --> 00:03:11.324
and I think the substance
of the comment

00:03:11.324 --> 00:03:14.297
was this is not what 
econometrics is about.

00:03:14.297 --> 00:03:20.572
Econometrics being transmitted 
at that time was about structure.

00:03:21.300 --> 00:03:24.490
There was this idea that
there's structure in the economy,

00:03:25.100 --> 00:03:27.200
and it's our job to discover it,

00:03:27.200 --> 00:03:30.952
and what makes its structure
is it's essentially invariant.

00:03:32.570 --> 00:03:34.900
And so we're saying, 
in the LATE theorem,

00:03:34.900 --> 00:03:37.699
that every instrument produces
its own causal effect,

00:03:37.699 --> 00:03:41.386
which is in contradiction to that
to some extent.

00:03:41.386 --> 00:03:43.640
And so that was 
where the tension was.

00:03:43.640 --> 00:03:45.551
People didn't want 
to give up that idea.

00:03:46.300 --> 00:03:50.369
- Yeah, I remember once 
people started

00:03:51.200 --> 00:03:55.664
arguing more vocally
against that --

00:03:56.900 --> 00:03:59.483
that never really 
bothered me that much.

00:03:59.483 --> 00:04:03.051
It seemed clear that 
we had a result there,

00:04:03.051 --> 00:04:05.878
and it became somewhat 
controversial,

00:04:05.878 --> 00:04:08.395
but controversial in a good way.

00:04:08.620 --> 00:04:10.190
It was clear that people felt

00:04:10.820 --> 00:04:13.835
they had to come out
against it because --

00:04:13.970 --> 00:04:15.649
- Well, I think we think
it's good now.

00:04:17.426 --> 00:04:19.238
We might not have loved it 
at the time.

00:04:20.168 --> 00:04:23.080
- I remember being
somewhat more upset --

00:04:23.080 --> 00:04:24.780
there was some dinner 
where someone said,

00:04:24.780 --> 00:04:27.455
"No, no, no, 
that paper with Josh --

00:04:28.855 --> 00:04:30.749
that was doing a disservice
to the profession."

00:04:32.050 --> 00:04:33.850
- We definitely had 
reactions like that.

00:04:35.410 --> 00:04:38.200
- At some level, that may be 
indicative of the culture

00:04:38.400 --> 00:04:40.000
in general in economics 
at the time.

00:04:41.400 --> 00:04:44.097
I thought back later, 
what if that happened now?

00:04:44.600 --> 00:04:47.682
If I was a senior person
sitting in that conversation,

00:04:48.200 --> 00:04:51.952
I would call that out because
it really was not appropriate.

00:04:53.000 --> 00:04:54.200
- [Josh] It wasn't so bad.

00:04:54.600 --> 00:04:56.262
I think the criticism is...

00:04:57.700 --> 00:04:59.298
It wasn't completely misguided.

00:05:00.070 --> 00:05:01.351
It was maybe wrong.

00:05:01.800 --> 00:05:04.485
- No, no, but you can say 
that paper is wrong,

00:05:05.280 --> 00:05:06.440
but it's saying that

00:05:06.440 --> 00:05:08.128
it's a disservice 
to the profession --

00:05:08.128 --> 00:05:10.300
- that's not really...
- [Isaiah] It's a bit personal.

00:05:10.300 --> 00:05:12.646
- Yes, and doing that not to me

00:05:12.646 --> 00:05:14.442
but in front of 
my senior colleagues.

00:05:15.191 --> 00:05:17.369
- But nobody was saying 
the result was wrong,

00:05:17.369 --> 00:05:18.700
and I remember also,

00:05:18.700 --> 00:05:21.579
some of the comments 
were thought-provoking.

00:05:21.579 --> 00:05:23.059
So we had some negative reviews,

00:05:23.059 --> 00:05:25.861
I think, on the average 
causal response paper.

00:05:26.500 --> 00:05:30.361
Somebody said, "These compliers --
you can't figure out who they are."

00:05:31.967 --> 00:05:33.891
It's one thing to say
you're estimating

00:05:33.891 --> 00:05:35.678
the effect of treatment
on the treated

00:05:35.678 --> 00:05:36.840
or something like that.

00:05:36.840 --> 00:05:38.400
You can tell me who's treated.

00:05:38.837 --> 00:05:42.289
People in the CPS,
you can't tell me who's a complier.

00:05:42.929 --> 00:05:44.679
So that was a legitimate challenge.

00:05:44.679 --> 00:05:47.800
- That's certainly fair,
and I can see why

00:05:49.880 --> 00:05:53.502
that part made people
a little uneasy and uncomfortable.

00:05:54.300 --> 00:05:56.400
But at the same time,

00:05:56.900 --> 00:06:00.244
because it showed that you couldn't
really go beyond that,

00:06:00.800 --> 00:06:03.775
it was a very useful thing 
to realize.

00:06:04.630 --> 00:06:08.831
I remember on the day
we got to the key result

00:06:09.238 --> 00:06:13.113
that I was thinking,
"Wow, this is as good as it gets.

00:06:14.221 --> 00:06:16.978
Here we actually have
an insight, but it clearly..."

00:06:17.500 --> 00:06:19.250
- And we had to sell it 
at some point.

00:06:19.480 --> 00:06:21.261
For quite a few years, 
we had to sell it,

00:06:23.480 --> 00:06:24.892
and it's proven to be quite useful.

00:06:25.500 --> 00:06:28.761
I don't think we understood that
it would be so useful at the time.

00:06:28.761 --> 00:06:29.871
- No.

00:06:30.170 --> 00:06:34.600
I did feel early on
this was a substantial insight.

00:06:34.600 --> 00:06:36.440
- [Josh] Yeah, we'd done something.

00:06:36.440 --> 00:06:40.041
- But I did not think 
goals were there.

00:06:40.700 --> 00:06:42.600
- I don't think we were aiming 
for the Nobel.

00:06:42.600 --> 00:06:43.730
[laughter]

00:06:43.730 --> 00:06:46.243
We were very happy to get
that note in Econometrica.

00:06:46.859 --> 00:06:48.829
♪ [music] ♪

00:06:49.770 --> 00:06:51.560
- [Isaiah] Are there factors 
or are ways of approaching problems

00:06:51.560 --> 00:06:54.186
that lead people to be better 
at recognizing the good stuff

00:06:54.186 --> 00:06:56.600
and taking the time to do it 
as opposed to dismissing it?

00:06:56.600 --> 00:06:57.830
- [Josh] Sometimes 
I think it's helpful.

00:06:57.830 --> 00:06:59.478
If you're trying
to convince somebody

00:06:59.478 --> 00:07:01.247
that you have something 
useful to say

00:07:01.900 --> 00:07:04.176
and maybe they don't 
speak your language,

00:07:04.894 --> 00:07:06.761
you might need 
to learn their language.

00:07:06.761 --> 00:07:07.910
- Yes, yes, exactly.

00:07:07.910 --> 00:07:11.736
- That's what we did with Don,
we figured out how to --

00:07:11.736 --> 00:07:14.052
I remember we had a very hard time

00:07:14.052 --> 00:07:16.058
explaining the exclusion
restriction to Don,

00:07:17.430 --> 00:07:18.993
maybe rightfully so.

00:07:19.804 --> 00:07:21.948
I think Guido and I 
eventually figured out

00:07:21.948 --> 00:07:24.420
that it wasn't formulated 
very clearly,

00:07:25.400 --> 00:07:27.450
and we came up 
with a way to do that

00:07:27.450 --> 00:07:29.316
in the potential outcomes framework

00:07:29.316 --> 00:07:32.218
that I think worked
for the three of us.

00:07:32.218 --> 00:07:33.419
- [Guido] Yeah.

00:07:33.419 --> 00:07:35.454
Well, it worked
for the bigger literature,

00:07:35.454 --> 00:07:37.639
but I think what you're saying 
there is exactly right.

00:07:37.639 --> 00:07:40.860
You need to figure out 
how not just say,

00:07:40.860 --> 00:07:43.894
"Okay, I've got this language,
and this works great,

00:07:43.894 --> 00:07:45.900
and I've got to convince
someone else to use the language."

00:07:45.900 --> 00:07:48.188
You could first figure out 
what language they're using,

00:07:48.680 --> 00:07:51.028
and then, only then, 
can you try to say,

00:07:51.028 --> 00:07:53.140
"Well, but here you're thinking
of it this way."

00:07:53.140 --> 00:07:56.880
But that's actually 
a pretty hard thing to do.

00:07:56.880 --> 00:07:59.098
You get someone
from a different discipline,

00:07:59.098 --> 00:08:02.405
convincing them, two junior faculty
in a different department

00:08:02.405 --> 00:08:04.366
actually have something 
to say to you --

00:08:04.596 --> 00:08:06.516
that takes a fair amount of effort.

00:08:07.500 --> 00:08:09.782
- Yeah, I wrote Don 
a number of times,

00:08:10.420 --> 00:08:11.868
in fairly long letters.

00:08:11.868 --> 00:08:13.805
I remember thinking 
this is worth doing,

00:08:14.600 --> 00:08:16.006
that if I could convince Don,

00:08:16.780 --> 00:08:19.444
that would validate
the framework to some extent.

00:08:20.300 --> 00:08:22.522
- I think both you and Don

00:08:22.522 --> 00:08:25.000
were a little bit more confident 
that you were right.

00:08:25.000 --> 00:08:26.438
- Well, we used to argue a lot,

00:08:26.438 --> 00:08:28.320
and you would sometimes 
referee those.

00:08:28.320 --> 00:08:29.500
[laughter]

00:08:29.800 --> 00:08:30.800
That was fun.

00:08:32.760 --> 00:08:34.125
It wasn't hurtful.

00:08:35.200 --> 00:08:37.492
- I remember it getting 
a little testy once.

00:08:37.935 --> 00:08:39.606
We had lunch in The Faculty Club,

00:08:40.600 --> 00:08:44.077
and we were talking about 
the draft lottery paper.

00:08:44.930 --> 00:08:47.430
We were talking about "never takes"

00:08:47.430 --> 00:08:49.761
as people wouldn't serve
in the military

00:08:49.761 --> 00:08:53.500
irrespective of whether
they were getting drafted,

00:08:54.500 --> 00:08:56.936
and you or Don said something

00:08:56.936 --> 00:08:58.800
about shooting yourself
in the foot...

00:08:58.800 --> 00:08:59.800
[laughter]

00:08:59.800 --> 00:09:01.530
...as a way of getting out
of the military

00:09:01.530 --> 00:09:03.230
and that maybe 
the exclusion restriction

00:09:03.230 --> 00:09:05.743
for "never takes" wasn't working.

00:09:06.300 --> 00:09:08.520
And then the other one was going,

00:09:08.520 --> 00:09:09.791
"Well, yes, you could do that,

00:09:09.791 --> 00:09:12.008
but why would you want 
to shoot yourself in the foot?"

00:09:12.008 --> 00:09:13.225
[laughter]

00:09:13.225 --> 00:09:15.400
It got a little there...

00:09:15.400 --> 00:09:17.860
- I usually go for moving
to Canada for my example,

00:09:18.690 --> 00:09:20.096
when I'm teaching that.

00:09:20.096 --> 00:09:21.365
[laughter]

00:09:22.030 --> 00:09:23.375
- But things are tricky.

00:09:24.860 --> 00:09:26.595
I get students coming
from Computer Science,

00:09:26.595 --> 00:09:29.943
and they want to do things 
on causal inference,

00:09:30.190 --> 00:09:33.460
and it takes a huge amount 
of effort to figure out

00:09:33.460 --> 00:09:35.230
how they're actually thinking 
about a problem

00:09:35.230 --> 00:09:37.000
and whether
there's something there.

00:09:37.000 --> 00:09:38.310
And so, now, over the years,

00:09:38.310 --> 00:09:39.720
I've got a little more appreciation

00:09:39.720 --> 00:09:41.958
for the fact that Don
was actually willing to --

00:09:42.630 --> 00:09:46.000
It took him a while, 
but he did engage first with Josh

00:09:46.400 --> 00:09:47.500
and then with both of us,

00:09:48.671 --> 00:09:50.245
rather than dismissing and saying,

00:09:50.245 --> 00:09:53.348
"Okay, well, I can't figure out
what these guys are doing,

00:09:53.348 --> 00:09:56.435
and it's probably just 
not really that interesting."

00:09:57.200 --> 00:09:59.736
- Everybody always wants 
to figure out quickly.

00:10:00.196 --> 00:10:01.376
You want to save time,

00:10:01.376 --> 00:10:03.410
and you want to save 
your brain cells

00:10:03.410 --> 00:10:04.583
for other things.

00:10:05.000 --> 00:10:07.000
The fastest route to that
is to figure out

00:10:07.000 --> 00:10:08.460
why you should dismiss something.

00:10:08.460 --> 00:10:09.560
- Yes.

00:10:09.560 --> 00:10:11.100
- I don't need
to spend time on this.

00:10:11.100 --> 00:10:12.498
♪ [music] ♪

00:10:12.498 --> 00:10:15.143
- [Narrator] If you'd like 
to watch more Nobel Conversations,

00:10:15.143 --> 00:10:16.207
click here.

00:10:16.207 --> 00:10:18.409
Or if you'd like to learn
more about econometrics,

00:10:18.640 --> 00:10:21.240
check out Josh's
Mastering Econometrics series.

00:10:21.800 --> 00:10:24.540
If you'd like to learn more
about Guido, Josh, and Isaiah,

00:10:24.860 --> 00:10:26.502
check out the links 
in the description.

00:10:26.992 --> 00:10:28.697
♪ [music] ♪