1 00:00:00,000 --> 00:00:02,550 ♪ [music] ♪ 2 00:00:03,800 --> 00:00:05,800 - [Narrator] Welcome to Nobel Conversations. 3 00:00:07,040 --> 00:00:08,100 In this episode, 4 00:00:08,100 --> 00:00:11,570 Josh Angrist and Guido Imbens sit down with Isaiah Andrews 5 00:00:11,570 --> 00:00:14,600 to discuss how the field of econometrics is evolving. 6 00:00:16,100 --> 00:00:18,750 - [Isaiah] So, Guido and Josh, you're both pioneers 7 00:00:18,750 --> 00:00:21,500 in developing tools for empirical research in economics. 8 00:00:21,500 --> 00:00:22,930 And so I'd like to explore 9 00:00:22,930 --> 00:00:25,300 where you feel like the field is heading -- 10 00:00:25,709 --> 00:00:28,079 economics, econometrics, the whole thing. 11 00:00:28,510 --> 00:00:31,302 To start, I'd be interested to hear 12 00:00:32,171 --> 00:00:35,200 about whether you feel the way in which 13 00:00:35,200 --> 00:00:38,510 the local average treatment effects framework took hold 14 00:00:38,800 --> 00:00:42,187 has any lessons for how new empirical methods in economics 15 00:00:42,187 --> 00:00:44,300 develop and spread or how they should. 16 00:00:44,560 --> 00:00:45,960 - [Josh] That's a good question. 17 00:00:46,610 --> 00:00:47,790 You go first. 18 00:00:47,790 --> 00:00:49,460 [laughter] 19 00:00:49,700 --> 00:00:52,940 - [Guido] Yeah, so I think the important thing 20 00:00:52,940 --> 00:00:58,550 is to come up with good convincing cases 21 00:00:58,550 --> 00:01:02,207 where the questions are clear 22 00:01:02,400 --> 00:01:05,720 and where the methods apply in general. 23 00:01:06,253 --> 00:01:07,560 One thing I -- 24 00:01:08,192 --> 00:01:12,000 looking back at the subsequent literature. 25 00:01:12,200 --> 00:01:16,700 So I really like the regression discontinuity literature 26 00:01:16,700 --> 00:01:19,670 where there were clearly a bunch of really convincing examples 27 00:01:19,670 --> 00:01:23,378 and that allowed people to think more clearly, 28 00:01:23,378 --> 00:01:27,200 look harder at the methodological questions. 29 00:01:27,400 --> 00:01:28,800 Do clear applications 30 00:01:28,800 --> 00:01:30,600 that then allow you to kind of think about, 31 00:01:30,600 --> 00:01:33,600 "Wow, does this type of assumption seem reasonable here? 32 00:01:33,600 --> 00:01:38,000 What kind of things do we not like in the early papers? 33 00:01:38,500 --> 00:01:39,802 How can we improve things?" 34 00:01:39,802 --> 00:01:44,210 So having clear applications motivating these literatures 35 00:01:44,210 --> 00:01:46,400 I think is very helpful. 36 00:01:46,800 --> 00:01:48,050 - I'm glad you mentioned 37 00:01:48,050 --> 00:01:49,382 the regression discontinuity, Guido. 38 00:01:49,382 --> 00:01:53,300 I think there's a lot of complementarity between IV and RD, 39 00:01:54,700 --> 00:01:57,060 Instrumental Variables and Regression Discontinuity. 40 00:02:00,506 --> 00:02:03,260 A lot of the econometric applications 41 00:02:03,260 --> 00:02:04,520 of regression discontinuity 42 00:02:04,520 --> 00:02:07,230 are what used to be called "fuzzy" RD, 43 00:02:07,230 --> 00:02:11,620 where, you know, it's not discrete or deterministic at the cutoff 44 00:02:11,620 --> 00:02:14,900 but just the change in rates or intensity. 45 00:02:14,900 --> 00:02:17,737 And the late framework helps us understand 46 00:02:17,737 --> 00:02:18,740 those applications 47 00:02:18,740 --> 00:02:21,140 and gives us a clear interpretation 48 00:02:21,140 --> 00:02:25,000 for something like in my paper with Victor Lavy, 49 00:02:25,000 --> 00:02:28,100 where we use Maimonides' rule, the class size cutoffs, 50 00:02:28,430 --> 00:02:30,030 and what are you getting there? 51 00:02:30,290 --> 00:02:31,820 Of course, you can answer that question 52 00:02:31,820 --> 00:02:33,900 with a linear constant effects model, 53 00:02:34,200 --> 00:02:36,310 but it turns out we're not limited to that, 54 00:02:36,310 --> 00:02:39,889 and RD is still very powerful and illuminating, 55 00:02:40,630 --> 00:02:43,092 even when the correlation 56 00:02:43,092 --> 00:02:45,866 between the cutoff and the variable of interest, 57 00:02:45,866 --> 00:02:49,133 in this case class size, is partial, 58 00:02:49,133 --> 00:02:51,000 maybe even not that strong. 59 00:02:52,000 --> 00:02:54,999 So there was definitely kind of a parallel development. 60 00:02:54,999 --> 00:02:56,400 It's also interesting -- 61 00:02:56,600 --> 00:02:59,780 nobody talked about regression discontinuity designs 62 00:02:59,780 --> 00:03:01,220 when we were in graduate school. 63 00:03:01,220 --> 00:03:05,300 It was something that other social scientists were interested in, 64 00:03:05,800 --> 00:03:09,507 and that grew up alongside the LATE framework, 65 00:03:09,507 --> 00:03:11,927 and we've both done work 66 00:03:11,927 --> 00:03:14,565 on both applications and methods there, 67 00:03:14,565 --> 00:03:18,377 and it's been very exciting to see that develop 68 00:03:18,377 --> 00:03:19,800 and become so important. 69 00:03:20,000 --> 00:03:21,767 It's part of a general evolution, 70 00:03:21,767 --> 00:03:26,086 I think, towards credible identification strategies, 71 00:03:26,086 --> 00:03:27,441 causal effects... 72 00:03:29,393 --> 00:03:30,642 making econometrics 73 00:03:30,642 --> 00:03:33,300 more about causal questions than about models. 74 00:03:33,640 --> 00:03:34,650 In terms of the future, 75 00:03:34,650 --> 00:03:37,660 I think one thing that LATE has helped facilitate 76 00:03:37,660 --> 00:03:42,008 is a move towards more creative, randomized trials 77 00:03:42,008 --> 00:03:44,400 where there's something of interest. 78 00:03:45,500 --> 00:03:48,460 It's not possible or straightforward 79 00:03:48,460 --> 00:03:50,700 to simply turn it off or on, 80 00:03:51,000 --> 00:03:54,584 but you can encourage it or discourage it. 81 00:03:54,584 --> 00:03:58,200 So you subsidize schooling with financial aid, for example. 82 00:03:59,000 --> 00:04:02,080 So now we have a whole framework for interpreting that, 83 00:04:03,600 --> 00:04:07,113 and it opens the doors to randomized trials 84 00:04:07,113 --> 00:04:10,300 of things that maybe would 85 00:04:10,300 --> 00:04:12,471 not have seemed possible before. 86 00:04:14,500 --> 00:04:17,864 We've used that a lot in the work we do on schools in our -- 87 00:04:17,864 --> 00:04:21,160 in the Blueprint Lab at MIT. 88 00:04:22,360 --> 00:04:26,600 We're exploiting random assignment in very creative ways, I think. 89 00:04:28,100 --> 00:04:31,395 - [Isaiah] Related to that, do you see particular factors 90 00:04:31,395 --> 00:04:34,445 that make for useful research in econometrics? 91 00:04:34,445 --> 00:04:38,290 You've alluded to it having a clear connection 92 00:04:38,290 --> 00:04:40,300 to problems that are actually coming up, 93 00:04:40,300 --> 00:04:42,650 and empirical practice is often a good idea. 94 00:04:43,290 --> 00:04:45,000 - Isn't it always a good idea? 95 00:04:45,700 --> 00:04:47,292 I often find myself sitting 96 00:04:47,292 --> 00:04:50,100 in an econometrics theory seminar, 97 00:04:50,700 --> 00:04:52,500 say the Harvard MIT seminar, 98 00:04:53,400 --> 00:04:56,350 and I'm thinking, "What problem is this guy solving? 99 00:04:56,350 --> 00:04:57,960 Who has this problem?" 100 00:04:57,960 --> 00:04:59,800 And, you know... 101 00:05:01,600 --> 00:05:04,700 sometimes there's an embarrassing silence if I ask 102 00:05:04,900 --> 00:05:08,300 or there might be a fairly contrived scenario. 103 00:05:08,800 --> 00:05:11,600 I want to see where the tool is useful. 104 00:05:12,500 --> 00:05:14,765 There are some purely foundational tools. 105 00:05:14,765 --> 00:05:16,250 I do take the point. 106 00:05:16,250 --> 00:05:21,735 There are people who are working on conceptual foundations of ... 107 00:05:22,600 --> 00:05:25,300 it becomes more like mathematical statistics. 108 00:05:25,800 --> 00:05:27,653 I mean, I remember an early example of that 109 00:05:27,653 --> 00:05:29,920 that I struggled to understand 110 00:05:29,920 --> 00:05:32,500 was the idea of stochastic equicontinuity, 111 00:05:32,500 --> 00:05:35,070 which one of my thesis advisors, Whitney Newey, 112 00:05:35,070 --> 00:05:36,479 was using to great effect, 113 00:05:36,479 --> 00:05:38,821 and I was trying to understand that. 114 00:05:40,600 --> 00:05:42,034 It's really foundational. 115 00:05:42,034 --> 00:05:45,200 it's not an application that's driving that -- 116 00:05:45,890 --> 00:05:47,300 at least not immediately. 117 00:05:48,600 --> 00:05:53,200 But most things are not like that, and so there should be a problem. 118 00:05:53,800 --> 00:05:59,100 And I think it's on the seller of that sort of thing, 119 00:06:00,480 --> 00:06:02,250 because there's opportunity cost, 120 00:06:02,250 --> 00:06:05,295 the time and attention, and effort to understand things. 121 00:06:05,980 --> 00:06:07,200 It's on the seller to say, 122 00:06:07,400 --> 00:06:08,900 "Hey, I'm solving this problem, 123 00:06:09,400 --> 00:06:12,900 and here's a set of results that show that it's useful, 124 00:06:12,900 --> 00:06:15,200 and here's some insight that I get." 125 00:06:16,200 --> 00:06:18,280 - [Isaiah] As you said, Josh, there's been a move 126 00:06:18,280 --> 00:06:20,700 in the direction of thinking more about causality 127 00:06:20,700 --> 00:06:22,900 in economics and empirical work in economics. 128 00:06:22,900 --> 00:06:24,800 Any consequences of the -- 129 00:06:24,800 --> 00:06:26,570 the spread of that view that surprised you 130 00:06:26,570 --> 00:06:28,347 or anything that you view as downsides 131 00:06:28,705 --> 00:06:31,400 of the way that empirical economics has gone? 132 00:06:31,500 --> 00:06:34,322 - Sometimes I see somebody does IV, 133 00:06:34,322 --> 00:06:38,304 and they get a result which seems implausibly large. 134 00:06:38,800 --> 00:06:40,200 That's the usual case. 135 00:06:42,500 --> 00:06:45,220 So it might be an extraordinarily large 136 00:06:45,220 --> 00:06:48,600 causal effect of some relatively minor intervention, 137 00:06:49,100 --> 00:06:52,260 which was randomized or for which you could make a case 138 00:06:52,260 --> 00:06:54,490 that there's a good design. 139 00:06:54,900 --> 00:06:57,205 And then when I see that, 140 00:06:57,944 --> 00:07:00,101 I think it's very hard for me to believe 141 00:07:00,101 --> 00:07:02,030 that this relatively minor intervention 142 00:07:02,030 --> 00:07:03,720 has such a large effect. 143 00:07:04,100 --> 00:07:06,277 The author will sometimes resort 144 00:07:06,277 --> 00:07:08,690 to the local average treatment effects theorem 145 00:07:08,690 --> 00:07:11,066 and say, "Well, these compliers, 146 00:07:11,066 --> 00:07:12,700 they're special in some way." 147 00:07:13,300 --> 00:07:15,800 And they just benefit extraordinarily 148 00:07:15,800 --> 00:07:17,600 from this intervention. 149 00:07:18,100 --> 00:07:21,175 I'm reluctant to take that at face value. 150 00:07:21,175 --> 00:07:23,750 I think often when effects are too big, 151 00:07:24,300 --> 00:07:26,780 it's because the exclusion restriction is failing, 152 00:07:26,780 --> 00:07:29,456 so you don't really have the right endogenous variable 153 00:07:29,456 --> 00:07:31,380 to scale that result. 154 00:07:32,000 --> 00:07:35,700 And so I'm not too happy to see 155 00:07:36,937 --> 00:07:40,022 a generic heterogeneity argument 156 00:07:40,022 --> 00:07:41,760 being used to excuse something 157 00:07:41,760 --> 00:07:43,800 that I think might be a deeper problem. 158 00:07:45,190 --> 00:07:47,358 - [Guido] I think it played somewhat of an unfortunate role 159 00:07:47,358 --> 00:07:50,083 in the discussions between reduced form 160 00:07:50,083 --> 00:07:51,700 and structural approaches, 161 00:07:51,700 --> 00:07:55,510 where I feel that wasn't quite right. 162 00:07:56,090 --> 00:07:58,810 The instrumental variables assumptions 163 00:07:58,810 --> 00:08:03,483 are at the core, structural assumptions about behavior -- 164 00:08:03,483 --> 00:08:05,200 they were coming from economic... 165 00:08:07,100 --> 00:08:09,900 thinking about the economic behavior of agents, 166 00:08:10,300 --> 00:08:15,100 and somehow it got pushed in a direction 167 00:08:15,100 --> 00:08:17,600 that I think wasn't really very helpful. 168 00:08:20,426 --> 00:08:21,663 I think, initially, 169 00:08:22,800 --> 00:08:24,067 we wrote things up, 170 00:08:24,067 --> 00:08:26,480 it was describing what was happening. 171 00:08:26,480 --> 00:08:29,783 There were a set of methods people were using. 172 00:08:29,783 --> 00:08:32,111 We clarified what those methods were doing 173 00:08:32,811 --> 00:08:38,361 and in a way that I think contain a fair amount of insight. 174 00:08:39,100 --> 00:08:42,050 But it somehow got pushed into a corner 175 00:08:42,050 --> 00:08:45,379 that I don't think was necessarily very helpful. 176 00:08:45,379 --> 00:08:48,604 - In just the language of reduced form versus structural, 177 00:08:48,604 --> 00:08:50,306 I find kind of funny in the sense 178 00:08:50,306 --> 00:08:52,985 that the local average treatment effect model, 179 00:08:52,985 --> 00:08:54,154 the potential outcomes model 180 00:08:54,154 --> 00:08:56,110 is a nonparametric structural model, 181 00:08:56,110 --> 00:08:58,600 if you want to think about it, as you suggested, Guido. 182 00:08:58,600 --> 00:09:01,129 So there's something a little funny 183 00:09:01,129 --> 00:09:03,505 about putting these two things in opposition when -- 184 00:09:03,505 --> 00:09:05,116 - [Guido] Yes. - [Josh] That language, of course, 185 00:09:05,116 --> 00:09:08,371 comes from the simultaneous equations framework 186 00:09:08,371 --> 00:09:09,641 that we inherited. 187 00:09:10,400 --> 00:09:11,440 It has the advantage 188 00:09:11,440 --> 00:09:14,160 that people seem to know what you mean when you use it, 189 00:09:14,160 --> 00:09:16,240 but that might be that people are hearing different -- 190 00:09:16,240 --> 00:09:18,300 different people are hearing different things. 191 00:09:18,300 --> 00:09:20,480 - [Guido] Yeah. I think reduced form has become 192 00:09:20,480 --> 00:09:22,200 used in a little bit of the pejorative way... 193 00:09:22,200 --> 00:09:23,540 - [Josh] Sometimes. 194 00:09:25,104 --> 00:09:28,250 ...which is not really quite what it was originally intended for. 195 00:09:30,100 --> 00:09:33,090 - [Isaiah] I guess something else that strikes me in thinking about 196 00:09:33,090 --> 00:09:35,645 the effects of the local average treatment effect framework 197 00:09:35,645 --> 00:09:37,676 is that often folks will appeal 198 00:09:37,676 --> 00:09:40,000 to a local average treatment effects intuition 199 00:09:40,000 --> 00:09:42,358 for settings well beyond ones 200 00:09:42,358 --> 00:09:44,963 where any sort of formal result has actually been established. 201 00:09:45,440 --> 00:09:49,180 And I'm curious, given all the work that you guys did 202 00:09:49,180 --> 00:09:52,390 to establish late results in different settings, 203 00:09:52,390 --> 00:09:54,415 I'm curious, any thoughts on that? 204 00:09:55,360 --> 00:09:57,306 - I think there's going to be a lot of cases 205 00:09:57,306 --> 00:10:02,200 where the intuition does get you some distance, 206 00:10:02,800 --> 00:10:04,989 but it's going to be somewhat limited, 207 00:10:04,989 --> 00:10:07,600 and establishing formal results there 208 00:10:08,400 --> 00:10:09,490 may be a little tricky 209 00:10:09,490 --> 00:10:12,700 and then maybe only work in special circumstances, 210 00:10:14,600 --> 00:10:16,540 and you end up with a lot of formality 211 00:10:16,540 --> 00:10:19,500 that may not quite capture the intuition. 212 00:10:19,900 --> 00:10:21,550 Sometimes I'm somewhat uneasy with them, 213 00:10:21,550 --> 00:10:24,438 and they are not necessarily the papers I would want to write, 214 00:10:25,148 --> 00:10:27,819 but I do think intuition 215 00:10:27,819 --> 00:10:31,217 often does capture part of the problem. 216 00:10:33,100 --> 00:10:36,300 I think, in some sense we were kind of very fortunate there 217 00:10:36,900 --> 00:10:39,250 in the way that the late paper got handled at the journal, 218 00:10:39,250 --> 00:10:41,766 is that, actually, the editor, made it much shorter 219 00:10:42,100 --> 00:10:46,300 and that then allowed us to kind of focus on very clear, crisp results. 220 00:10:47,100 --> 00:10:51,770 Where if you -- you know, this somewhat unfortunate tendency 221 00:10:51,770 --> 00:10:52,985 in the econometrics literature 222 00:10:52,985 --> 00:10:55,100 of having the papers get longer and longer. 223 00:10:55,100 --> 00:10:56,690 - [Josh] Well, you should be able to fix that, man. 224 00:10:56,690 --> 00:10:58,800 - [Guido] I'm trying to fix that. 225 00:10:59,400 --> 00:11:01,625 But I think this is an example where it's sort of very clear 226 00:11:01,625 --> 00:11:03,498 that having it be short is actually -- 227 00:11:03,498 --> 00:11:04,842 - [Josh] You should impose that no paper 228 00:11:04,842 --> 00:11:06,655 can be longer than the late paper. 229 00:11:06,655 --> 00:11:08,000 - [Guido] That, wow. 230 00:11:08,000 --> 00:11:09,617 That may be great. 231 00:11:09,617 --> 00:11:11,685 - [Josh] At least no theory, no theory paper. 232 00:11:11,892 --> 00:11:14,300 - [Guido] Yeah, and I think, I think... 233 00:11:14,500 --> 00:11:16,800 I'm trying very hard to get the papers to be shorter. 234 00:11:16,800 --> 00:11:18,700 And I think there's a lot of value 235 00:11:19,200 --> 00:11:21,480 today because it's often the second part of the paper 236 00:11:21,480 --> 00:11:26,395 that doesn't actually get you much further in understanding things 237 00:11:27,000 --> 00:11:29,870 and it does make things much harder to read 238 00:11:30,630 --> 00:11:33,200 and, you know, it sort of goes back 239 00:11:33,200 --> 00:11:36,111 to how I think econometrics should be done, 240 00:11:36,111 --> 00:11:38,070 you should focus on -- 241 00:11:38,700 --> 00:11:41,300 It should be reasonably close to empirical problems. 242 00:11:41,500 --> 00:11:43,900 They should be very clear problems. 243 00:11:44,800 --> 00:11:48,900 But then often the theory doesn't need to be quite so long. 244 00:11:48,900 --> 00:11:50,010 - [Josh] Yeah. 245 00:11:51,100 --> 00:11:54,670 - [Guido] I think things have gone a little off track. 246 00:11:56,260 --> 00:11:57,750 - [Isaiah] A new relatively recent change 247 00:11:57,750 --> 00:12:00,230 has been a seeming big increase in demand 248 00:12:00,230 --> 00:12:02,200 for people with sort of econometrics, 249 00:12:02,200 --> 00:12:04,800 causal effect estimation skills in the tech sector. 250 00:12:05,000 --> 00:12:07,480 I'm interested, do either of you have thoughts 251 00:12:07,480 --> 00:12:09,840 on sort of how that's going to interact 252 00:12:09,840 --> 00:12:11,600 with the development of empirical methods, 253 00:12:11,600 --> 00:12:13,950 or empirical research in economics going forward? 254 00:12:14,600 --> 00:12:16,770 - [Josh] Well, there's sort of a meta point, 255 00:12:16,770 --> 00:12:21,000 which is, there's this new kind of employer, 256 00:12:21,800 --> 00:12:27,530 the Amazons and the Uber, and, you know, TripAdvisor world, 257 00:12:28,000 --> 00:12:29,300 and I think that's great. 258 00:12:29,300 --> 00:12:32,600 And I like to tell my students about that, you know, especially -- 259 00:12:32,600 --> 00:12:35,500 at MIT we have a lot of computer science majors. 260 00:12:35,500 --> 00:12:37,000 That's our biggest major. 261 00:12:37,400 --> 00:12:42,246 And I try to seduce some of those folks into economics by saying, 262 00:12:42,246 --> 00:12:45,700 you know, you can go work for these, 263 00:12:45,800 --> 00:12:49,250 you know, companies that people are very keen to work for 264 00:12:49,250 --> 00:12:50,800 because the work seems exciting, 265 00:12:52,000 --> 00:12:54,250 you know, that the skills that you get in econometrics 266 00:12:54,250 --> 00:12:56,100 are as good or better 267 00:12:56,100 --> 00:12:59,736 than any competing discipline has to offer. 268 00:12:59,736 --> 00:13:01,100 So you should at least 269 00:13:01,400 --> 00:13:04,200 take some econ, take some econometrics, and some econ. 270 00:13:04,800 --> 00:13:07,000 I did a fun project with Uber 271 00:13:07,600 --> 00:13:09,770 on labor supply of Uber drivers 272 00:13:09,920 --> 00:13:12,805 and was very, very exciting to be part of that. 273 00:13:13,100 --> 00:13:15,400 Plus I got to drive for Uber for a while 274 00:13:15,900 --> 00:13:17,730 and I thought that was fun too. 275 00:13:17,730 --> 00:13:20,700 I did not make enough that I was tempted to 276 00:13:21,100 --> 00:13:25,100 give up my MIT job, but I enjoyed the experience. 277 00:13:25,230 --> 00:13:30,900 I see a potential challenge to our model of graduate education here, 278 00:13:31,700 --> 00:13:37,400 which is, if we're training people to go work at Amazon, you know, 279 00:13:37,900 --> 00:13:41,190 it's not clear why, you know, we should be paying 280 00:13:41,190 --> 00:13:42,900 graduate stipends for that. 281 00:13:43,200 --> 00:13:47,280 Why should the taxpayer effectively be subsidizing that. 282 00:13:47,280 --> 00:13:51,400 Our graduate education in the US Is generously subsidized, 283 00:13:51,400 --> 00:13:53,160 even in private universities, 284 00:13:53,160 --> 00:13:56,100 it's ultimately -- there's a lot of public money there, 285 00:13:56,100 --> 00:13:59,300 and I think the traditional rationale for that is, 286 00:13:59,500 --> 00:14:02,137 you know, we were training educators and scholars, 287 00:14:02,137 --> 00:14:05,657 and there's a great externality from the work that we do, 288 00:14:05,657 --> 00:14:07,607 it's either the research externality, 289 00:14:07,607 --> 00:14:09,557 or a teaching externality. 290 00:14:10,100 --> 00:14:12,350 But, you know, if many of our students 291 00:14:12,350 --> 00:14:14,600 are going to work in the private sector, 292 00:14:16,300 --> 00:14:21,700 that's fine, but maybe their employers should pay for that. 293 00:14:22,120 --> 00:14:23,370 - [Guido] But maybe is not so different 294 00:14:23,370 --> 00:14:26,780 from people working for consulting firms. 295 00:14:27,200 --> 00:14:28,780 It's not clear to me 296 00:14:28,780 --> 00:14:32,836 that the number of jobs in academics has changed. 297 00:14:33,370 --> 00:14:36,325 - [Josh] I feel like this is a growing sector, 298 00:14:36,325 --> 00:14:39,289 whereas consulting -- you're right to raise that, 299 00:14:39,289 --> 00:14:42,100 it might be the same for consulting, 300 00:14:43,300 --> 00:14:44,846 but this, you know, 301 00:14:44,846 --> 00:14:47,500 I'm placing more and more students in these businesses. 302 00:14:47,500 --> 00:14:50,400 So, it's on my mind in a way that I've sort of, 303 00:14:50,800 --> 00:14:53,960 you know, not been attentive to consulting jobs, 304 00:14:53,960 --> 00:14:56,920 you know, consulting was always important, 305 00:14:56,920 --> 00:14:58,950 and I think also there's some movement 306 00:14:58,950 --> 00:15:02,600 from consulting back into research, it's a little more fluid. 307 00:15:03,900 --> 00:15:07,630 A lot of the work in both domains 308 00:15:07,630 --> 00:15:09,430 I have to say, it's not really different 309 00:15:09,430 --> 00:15:12,730 but, you know, people who are working in the tech sector 310 00:15:12,730 --> 00:15:15,480 are doing things that are potentially of scientific interest, 311 00:15:15,480 --> 00:15:16,800 but mostly it's hidden. 312 00:15:17,100 --> 00:15:18,550 Then you really I have to say, 313 00:15:18,550 --> 00:15:20,900 you know, why is the government paying for this? 314 00:15:21,800 --> 00:15:23,732 Yeah, although, yeah, I mean to Guidos point, 315 00:15:23,732 --> 00:15:26,102 I guess there's a data question here 316 00:15:26,102 --> 00:15:30,042 of it has the sort of total [no-neck] sort of say 317 00:15:31,300 --> 00:15:34,870 for-profit sector employment of econ Ph.D. program graduates 318 00:15:34,870 --> 00:15:38,290 increased or has it just been a substitution from finance 319 00:15:38,290 --> 00:15:40,200 and consulting towards tech. 320 00:15:40,300 --> 00:15:42,300 - [Josh] I may be reacting to something 321 00:15:42,300 --> 00:15:44,300 that's not really happening. 322 00:15:44,400 --> 00:15:45,890 - [Guido] I've actually done some work 323 00:15:45,890 --> 00:15:48,200 with some of these tech companies. 324 00:15:49,100 --> 00:15:52,200 I don't disagree with Josh's point that we need to think 325 00:15:52,200 --> 00:15:53,830 a little bit about the funding model, 326 00:15:53,830 --> 00:15:56,390 who is it in the end paying for the graduate education. 327 00:15:56,710 --> 00:15:59,400 But from a scientific perspective, 328 00:15:59,980 --> 00:16:02,540 not only do these places have great data 329 00:16:02,540 --> 00:16:04,950 and nowadays they tend to be very careful with that 330 00:16:04,950 --> 00:16:07,100 for privacy reasons, 331 00:16:07,380 --> 00:16:08,900 but they also have great questions. 332 00:16:10,200 --> 00:16:13,213 I find it very inspiring kind of to listen 333 00:16:13,213 --> 00:16:15,950 to the people there and kind of see what kind of questions they have, 334 00:16:15,950 --> 00:16:17,330 and often they're questions 335 00:16:18,200 --> 00:16:21,510 that also come up outside of these companies. 336 00:16:21,510 --> 00:16:27,430 I have a couple of papers with Raj Chetty and Susan Athey, 337 00:16:27,430 --> 00:16:31,600 where we look at ways of combining experimental data 338 00:16:31,600 --> 00:16:34,000 and observational data, and kind of their -- 339 00:16:35,500 --> 00:16:38,600 Raj Chetty was interested in what is the effect 340 00:16:38,600 --> 00:16:42,893 of early childhood programs on outcomes later in life, 341 00:16:42,893 --> 00:16:46,330 not just kind on test scores, but on earnings and stuff, 342 00:16:46,330 --> 00:16:48,300 and we kind of developed methods 343 00:16:48,600 --> 00:16:51,500 that would help you shed light on that, onto some -- 344 00:16:52,760 --> 00:16:56,920 in some settings and the same problems came up 345 00:16:56,920 --> 00:17:00,533 kind of in this tech company settings. 346 00:17:00,800 --> 00:17:03,240 And so from my perspective, 347 00:17:03,240 --> 00:17:05,420 it's the same kind of -- 348 00:17:05,420 --> 00:17:07,600 I was talking to people doing empirical work, 349 00:17:07,600 --> 00:17:09,700 I tried to kind of look at these specific problems 350 00:17:09,700 --> 00:17:13,370 and then try to come up with more general problems, 351 00:17:15,110 --> 00:17:18,230 we formulated the problems at a higher level, 352 00:17:18,500 --> 00:17:22,900 so that I can think about solutions that work in a range of settings. 353 00:17:23,400 --> 00:17:24,840 And so from that perspective, 354 00:17:24,840 --> 00:17:27,570 the interactions with the tech companies 355 00:17:27,570 --> 00:17:30,300 are just very valuable and very useful. 356 00:17:31,700 --> 00:17:35,030 We do have students now doing internships there 357 00:17:35,030 --> 00:17:38,390 and then coming back and writing more interesting thesis 358 00:17:38,390 --> 00:17:43,400 as a result of their experiences there. 359 00:17:44,600 --> 00:17:47,020 - [Narrator] If you'd like to watch more Nobel Conversations, 360 00:17:47,020 --> 00:17:48,200 click here, 361 00:17:48,200 --> 00:17:50,500 or if you'd like to learn more about econometrics, 362 00:17:50,500 --> 00:17:53,100 check out Josh's "Mastering Econometrics" series. 363 00:17:53,700 --> 00:17:56,720 If you'd like to learn more about Guido, Josh and Isaiah 364 00:17:56,720 --> 00:17:58,300 check out the links in the description. 365 00:17:59,036 --> 00:18:01,495 ♪ [music] ♪