WEBVTT 00:00:00.900 --> 00:00:01.900 ♪ [music] ♪ 00:00:03.800 --> 00:00:05.700 - [Narrator] Welcome to Nobel conversations. 00:00:07.300 --> 00:00:10.240 In this episode, Josh Angrist and Guido Imbens, 00:00:10.240 --> 00:00:12.000 sit down with Isaiah Andrews 00:00:12.000 --> 00:00:14.303 to discuss how the research was initially received 00:00:14.900 --> 00:00:17.736 and how they responded to criticism. 00:00:18.700 --> 00:00:19.300 At the time, did you feel like you are on to something, 00:00:20.400 --> 00:00:24.000 you felt like this was the beginning of a whole line of work 00:00:24.000 --> 00:00:27.400 that you felt like was going to be important or...? 00:00:27.600 --> 00:00:30.100 Not so much that it was a whole line of work, 00:00:30.100 --> 00:00:32.600 but certainly I felt like, "Wow, this--" 00:00:32.600 --> 00:00:34.700 We proved something be proved up that people didn't know before, 00:00:34.700 --> 00:00:39.000 that it was worth knowing. 00:00:39.000 --> 00:00:40.000 Yeah, going back compared to my 00:00:40.000 --> 00:00:41.400 job market papers having-- 00:00:41.600 --> 00:00:45.900 I felt this was actually a very clear crisp result. 00:00:46.400 --> 00:00:48.400 But there were definitely 00:00:48.900 --> 00:00:53.200 was mixed reception and I don't think anybody said that, 00:00:53.300 --> 00:00:56.900 "Oh, wow, this is already, something." 00:00:57.100 --> 00:00:59.600 No, which is the nightmare scenario for a researcher 00:01:00.300 --> 00:01:03.200 where you think you've discovered something and then somebody else, 00:01:03.300 --> 00:01:04.800 says, "Oh, I knew that." 00:01:05.000 --> 00:01:08.600 But there were definitely was a need to convince people that this was worth knowing, 00:01:09.000 --> 00:01:12.800 that instrumental variables estimates a causal effect for compliers. 00:01:13.200 --> 00:01:18.000 Yeah, but even though it took a long time to convince 00:01:18.600 --> 00:01:20.400 a bigger audience, 00:01:20.700 --> 00:01:24.600 sometimes even fairly quickly, the reception was pretty good 00:01:24.800 --> 00:01:27.000 among a small group of people. 00:01:27.200 --> 00:01:31.500 Gary, clearly liked it a lot from the beginning 00:01:31.800 --> 00:01:34.600 and I remember, because at that point Josh had left for Israel, 00:01:34.700 --> 00:01:37.400 but I remember explaining it to Don Ruben 00:01:37.600 --> 00:01:43.700 and he was like, "Yeah, this really is something here." 00:01:43.800 --> 00:01:47.200 Not right away though. Don took some convincing. 00:01:47.500 --> 00:01:48.400 By the time you got to Don, 00:01:48.500 --> 00:01:51.500 there have been some back and forth with him 00:01:51.800 --> 00:01:53.500 and in correspondence actually. 00:01:53.700 --> 00:01:56.700 But I remember at some point getting a call or email from him 00:01:56.700 --> 00:02:02.300 saying that he was sitting at the airport in Rome 00:02:02.500 --> 00:02:03.700 and looking at the paper and thinking, 00:02:03.700 --> 00:02:07.000 "Yeah, no actually, you guys are onto something." 00:02:07.000 --> 00:02:10.600 We were happy about but that took longer than I think you remember. 00:02:10.800 --> 00:02:12.500 Yeah, it wasn't right away 00:02:12.600 --> 00:02:13.600 [laughter] 00:02:13.700 --> 00:02:16.500 because I know that I was back in Israel by the time that happened. 00:02:16.500 --> 00:02:18.300 I'd left for Israel in the summer-- 00:02:18.400 --> 00:02:22.300 I was only at Harvard for two years. We had that one year. 00:02:22.600 --> 00:02:25.700 It is remarkable, I mean, that one year was so fateful for us. 00:02:25.900 --> 00:02:27.200 - [Guido] Yes. 00:02:28.500 --> 00:02:30.200 I think we understood there was something good happening, 00:02:30.200 --> 00:02:34.000 but maybe we didn't think it was life-changing, only in retrospect. 00:02:34.400 --> 00:02:35.400 ♪ [music] ♪ 00:02:35.800 --> 00:02:39.300 - [Isaiah] As you said, it sounds like a small group of people were initially quite receptive, 00:02:39.300 --> 00:02:41.000 perhaps took some time for 00:02:41.100 --> 00:02:44.200 a broader group of people to come around to 00:02:44.400 --> 00:02:47.500 seeing the LATE framework as a valuable way to look at the world. 00:02:47.700 --> 00:02:50.000 I guess, in over the course of that, did you 00:02:50.100 --> 00:02:52.200 were their periods where you thought, 00:02:52.300 --> 00:02:53.200 maybe the people 00:02:53.300 --> 00:02:55.800 saying this wasn't a useful way to look at the world were right? 00:02:55.800 --> 00:02:58.400 Did you get discouraged? How did you think about? 00:02:58.400 --> 00:03:00.900 I don't think I was discouraged but the people who were saying 00:03:00.900 --> 00:03:03.900 that we're smart people, well informed metricians, 00:03:05.000 --> 00:03:08.000 sophisticated readers 00:03:08.900 --> 00:03:11.800 and I think the substance of the comment was, 00:03:11.800 --> 00:03:15.600 this is not what econometrics is about. 00:03:16.300 --> 00:03:20.700 Econometrics was being transmitted at that time was about structure. 00:03:21.300 --> 00:03:24.700 There was this idea that there's structure in the economy 00:03:25.100 --> 00:03:27.100 and it's our job to discover it 00:03:27.200 --> 00:03:31.200 and what makes it structure is it's essentially invariant 00:03:31.900 --> 00:03:34.800 and so we're saying, in the late theorem, 00:03:34.900 --> 00:03:39.100 that every instrument produces its own causal effect, 00:03:39.300 --> 00:03:42.100 which is in contradiction to that to some extent 00:03:42.400 --> 00:03:45.300 and so that was where the tension was. 00:03:45.300 --> 00:03:46.300 People didn't want to give up that idea. 00:03:46.300 --> 00:03:47.700 Yeah, I remember 00:03:48.100 --> 00:03:50.500 once people were started 00:03:51.200 --> 00:03:56.100 arguing more more vocally against that, 00:03:56.900 --> 00:04:00.700 that never really bothered me that much. 00:04:01.000 --> 00:04:03.700 It seems clear that we had a result there 00:04:04.900 --> 00:04:08.100 and it was somewhat controversial, 00:04:08.100 --> 00:04:09.100 but controversial in a good way. 00:04:09.100 --> 00:04:10.400 It was clear that people felt 00:04:10.700 --> 00:04:12.800 they had to come out against it because-- 00:04:14.100 --> 00:04:17.500 Well, I think what we think it's good now 00:04:17.500 --> 00:04:20.800 we might not have loved it at the time. 00:04:20.800 --> 00:04:21.800 I remember being somewhat, the more upset-- 00:04:21.800 --> 00:04:26.400 there was some dinner where someone said, 00:04:26.700 --> 00:04:28.300 "No, no, that paper with Josh, 00:04:28.800 --> 00:04:32.600 that was doing a disservice to the profession." 00:04:32.600 --> 00:04:34.400 We definitely had reactions like that. 00:04:34.800 --> 00:04:38.200 At some level, that may be indicative of the culture 00:04:38.400 --> 00:04:40.000 in general in economics at the time. 00:04:41.400 --> 00:04:44.300 I thought back later, what if that'd happened now, 00:04:44.600 --> 00:04:48.200 if I was a senior person sitting in that conversation, 00:04:48.300 --> 00:04:52.200 I would call that out because it really was not appropriate-- 00:04:53.000 --> 00:04:54.200 - [Josh] But it wasn't so bad. 00:04:54.600 --> 00:04:57.300 I think the criticism is-- 00:04:57.700 --> 00:05:01.500 It wasn't completely misguided, it was maybe wrong. 00:05:01.800 --> 00:05:04.700 No, no, but you can say the paper is wrong 00:05:05.400 --> 00:05:09.300 but it's saying that it's a disservice to the profession, that's not really-- 00:05:09.300 --> 00:05:10.300 Personal. 00:05:10.300 --> 00:05:14.700 Yes, and doing that, not to me, but in front of my senior colleagues. 00:05:14.900 --> 00:05:17.700 But nobody was saying the result was wrong 00:05:17.700 --> 00:05:18.700 and I remember also, 00:05:18.700 --> 00:05:20.900 some of the comments were thought-provoking 00:05:20.900 --> 00:05:24.000 so we had some negative reviews, 00:05:24.400 --> 00:05:26.300 I think on the average causal response paper. 00:05:26.500 --> 00:05:30.600 Somebody said, "These compliers you can't figure out who they are." 00:05:31.500 --> 00:05:31.900 Right. 00:05:32.000 --> 00:05:33.000 It's one thing to say 00:05:33.000 --> 00:05:36.200 you're estimating the effect of treatment on the treated or something like that. 00:05:36.200 --> 00:05:38.400 You can tell me who's treated, 00:05:38.600 --> 00:05:42.600 people in the CPS, you can't tell me who's a complier. 00:05:42.800 --> 00:05:46.800 So that was a legitimate challenge. 00:05:46.800 --> 00:05:47.800 That's certainly fair and I can see why 00:05:47.800 --> 00:05:53.700 that part made people a little uneasy and uncomfortable. 00:05:53.800 --> 00:05:54.200 Yeah. 00:05:54.300 --> 00:05:56.400 But it's a at the same time 00:05:56.900 --> 00:06:00.800 because it showed that you couldn't really go beyond that, 00:06:01.500 --> 00:06:05.500 it was very useful thing to realize. 00:06:05.500 --> 00:06:06.500 I remember on the day, 00:06:06.500 --> 00:06:11.900 we got to the key result that I was thinking, "Wow, this is as good as it gets. 00:06:12.100 --> 00:06:16.500 Here we actually have an insight but clearly--" 00:06:17.500 --> 00:06:21.000 And we had to sell it. 00:06:21.000 --> 00:06:22.000 For quite a few years, we had to sell 00:06:22.000 --> 00:06:24.800 and it's proven to be quite useful. 00:06:25.500 --> 00:06:29.300 I don't think we understood that it would be so useful at the time. 00:06:30.100 --> 00:06:34.600 No, I did feel early on this was a substantial insight. 00:06:34.600 --> 00:06:35.600 - [Josh] Yeah we [learned] something. 00:06:35.600 --> 00:06:40.400 But I did not think goals were there. 00:06:40.500 --> 00:06:42.400 I don't think we were aiming for the Nobel. 00:06:42.650 --> 00:06:43.650 [laughter] 00:06:43.900 --> 00:06:46.300 We were very happy to get that note in Econometrica. 00:06:47.600 --> 00:06:48.600 ♪ [music] ♪ 00:06:49.900 --> 00:06:52.800 - [Isaiah] Are there factors or are ways of approaching problems that lead people to be better at 00:06:53.200 --> 00:06:56.600 recognizing the good stuff and taking the time to do it as opposed to dismissing it? 00:06:56.600 --> 00:06:57.700 - [Josh] Sometimes I think it's helpful. 00:06:57.700 --> 00:07:01.300 If you're trying to convince somebody that you have something useful to say 00:07:01.900 --> 00:07:04.500 and maybe they don't speak your language, 00:07:04.700 --> 00:07:09.900 you might need to learn their language. 00:07:10.300 --> 00:07:10.850 Yes. Yes, exactly. 00:07:10.850 --> 00:07:11.400 That's what we did with Don, we figured out how to-- 00:07:12.200 --> 00:07:17.600 I remember we had a very hard time explaining the exclusion restriction to Don, 00:07:17.800 --> 00:07:19.700 maybe rightfully so, 00:07:20.000 --> 00:07:24.600 I think Guido and I eventually figured out that it wasn't formulated very clearly, 00:07:25.400 --> 00:07:29.700 and we came up with a way to do that in the potential outcomes framework 00:07:29.700 --> 00:07:32.700 that I think worked for the three of us. 00:07:33.400 --> 00:07:35.800 Yeah, well, it worked for the bigger literature 00:07:35.800 --> 00:07:38.100 but I think what you're saying there is exactly right, 00:07:38.100 --> 00:07:39.100 you need to figure out how not just say, okay well, I've got this language 00:07:41.400 --> 00:07:43.900 and this this works great 00:07:43.900 --> 00:07:45.900 and I've got to convince someone else to use the language. 00:07:45.900 --> 00:07:47.600 You could first figure out what language they're using 00:07:49.600 --> 00:07:51.200 and then only then, can you try to say, 00:07:51.200 --> 00:07:55.700 "Wow, but here you thinking of it this way," 00:07:55.700 --> 00:07:56.700 but that's actually a pretty hard thing to do, 00:07:56.700 --> 00:08:00.000 get someone from a different discipline, convincing them, 00:08:00.200 --> 00:08:03.300 two junior faculty in a different department actually have something to say to you, 00:08:03.300 --> 00:08:06.600 that's that takes a fair amount of effort. 00:08:07.500 --> 00:08:10.200 Yeah, I wrote on a number of times, in fairly long letters. 00:08:10.700 --> 00:08:14.500 I remember thinking this is worth doing, 00:08:14.600 --> 00:08:17.600 that if I could convince Don that would validate the framework to some extent. 00:08:20.300 --> 00:08:25.000 I think both you and Don were a little bit more confident that you were right. 00:08:25.000 --> 00:08:28.200 Well, we used to argue a lot and you would sometimes referee those. 00:08:28.500 --> 00:08:29.500 [laughter] 00:08:29.800 --> 00:08:30.700 That was fun. 00:08:30.800 --> 00:08:34.100 It wasn't hurtful. 00:08:35.200 --> 00:08:39.800 I remember getting a little testy once, we had lunch in The Faculty Club 00:08:40.600 --> 00:08:44.400 and we're talking about the draft lottery paper. 00:08:45.200 --> 00:08:47.600 we were talking about never takes 00:08:47.700 --> 00:08:51.000 as people wounded serve in the military irrespective of 00:08:51.200 --> 00:08:53.700 whether they were getting drafted 00:08:54.500 --> 00:08:58.700 and you and Don said something about shooting yourself in the foot, 00:08:59.800 --> 00:09:03.400 as a way of getting out of the military and that may be the exclusion restriction 00:09:03.400 --> 00:09:05.300 for never takes wasn't working 00:09:06.300 --> 00:09:08.900 and then the other one would say, 00:09:08.900 --> 00:09:12.100 "Well, yes you could do that but why would you want to shoot yourself in the foot?" 00:09:12.250 --> 00:09:12.825 [laughter] 00:09:12.825 --> 00:09:13.400 It got a little [inaudible] there. 00:09:13.400 --> 00:09:17.600 I usually go for moving to Canada for my example when I'm teaching that. 00:09:19.700 --> 00:09:24.000 But he thinks it's tricky, 00:09:24.100 --> 00:09:27.900 I get students coming from computer science and they want to do 00:09:28.100 --> 00:09:29.900 things on causal inference 00:09:30.400 --> 00:09:33.700 and it takes a huge amount of effort to figure out how 00:09:33.700 --> 00:09:36.800 they actually thinking about problem and whether there's something there 00:09:37.000 --> 00:09:38.500 and so, now over the years, 00:09:38.500 --> 00:09:40.600 I've got a little more appreciation for the fact that Don 00:09:40.800 --> 00:09:42.100 was actually willing to-- 00:09:42.200 --> 00:09:46.000 It took him a while, but he did engage first with Josh 00:09:46.400 --> 00:09:47.500 and then with both of us 00:09:47.700 --> 00:09:51.000 and rather than dismissing and say, 00:09:51.500 --> 00:09:55.700 "Well, okay I can't figure out what these guys are doing and it's probably just not really that 00:09:55.800 --> 00:09:56.700 that interesting." 00:09:57.200 --> 00:10:00.300 Everybody always wants to figure out quickly, 00:10:00.300 --> 00:10:04.300 you want to save time and you want to save your brain cells for other things. 00:10:05.000 --> 00:10:08.000 The fastest route to that is to figure out why you should dismiss something. 00:10:08.000 --> 00:10:09.000 Yes. 00:10:09.000 --> 00:10:11.300 I don't need to spend time on this. 00:10:11.500 --> 00:10:12.500 ♪ [music] ♪ 00:10:12.700 --> 00:10:15.900 - [Narrator] If you'd like to watch more Nobel conversations click here, 00:10:16.300 --> 00:10:18.600 or if you'd like to learn more about econometrics, 00:10:18.700 --> 00:10:21.300 check out Josh's "Mastering Econometrics" series. 00:10:21.800 --> 00:10:24.700 If you'd like to learn more about Guido, Josh, and Isaiah 00:10:24.900 --> 00:10:26.400 check out the links in the description. 00:10:26.702 --> 00:10:28.017 ♪ [music] ♪