0:00:00.000,0:00:02.550 ♪ [music] ♪ 0:00:03.800,0:00:05.800 - [Narrator] Welcome [br]to Nobel Conversations. 0:00:07.040,0:00:08.100 In this episode, 0:00:08.100,0:00:11.570 Josh Angrist and Guido Imbens[br]sit down with Isaiah Andrews 0:00:11.570,0:00:14.600 to discuss how the field [br]of econometrics is evolving. 0:00:16.100,0:00:18.750 - [Isaiah] So, Guido and Josh, [br]you're both pioneers 0:00:18.750,0:00:21.500 in developing tools for[br]empirical research in economics. 0:00:21.500,0:00:22.930 And so I'd like to explore 0:00:22.930,0:00:25.300 where you feel like [br]the field is heading -- 0:00:25.709,0:00:28.079 economics, econometrics,[br]the whole thing. 0:00:28.510,0:00:31.302 To start, I'd be interested to hear 0:00:32.171,0:00:35.200 about whether you feel[br]the way in which 0:00:35.200,0:00:38.510 the local average treatment [br]effects framework took hold 0:00:38.800,0:00:42.187 has any lessons for how [br]new empirical methods in economics 0:00:42.187,0:00:44.300 develop and spread[br]or how they should. 0:00:44.560,0:00:45.960 - [Josh] That's a good question. 0:00:46.610,0:00:47.790 You go first. 0:00:47.790,0:00:49.460 [laughter] 0:00:49.700,0:00:52.940 - [Guido] Yeah, so I think [br]the important thing 0:00:52.940,0:00:58.550 is to come up [br]with good convincing cases 0:00:58.550,0:01:02.207 where the questions are clear 0:01:02.400,0:01:05.720 and where the methods[br]apply in general. 0:01:06.253,0:01:07.560 One thing I -- 0:01:08.192,0:01:12.000 looking back[br]at the subsequent literature. 0:01:12.200,0:01:16.700 So I really like the regression[br]discontinuity literature 0:01:16.700,0:01:19.670 where there were clearly a bunch[br]of really convincing examples 0:01:19.670,0:01:23.378 and that allowed people[br]to think more clearly, 0:01:23.378,0:01:27.200 look harder [br]at the methodological questions. 0:01:27.400,0:01:28.800 Do clear applications 0:01:28.800,0:01:30.600 that then allow you [br]to kind of think about, 0:01:30.600,0:01:33.600 "Wow, does this type of assumption[br]seem reasonable here? 0:01:33.600,0:01:38.000 What kind of things do we not like[br]in the early papers? 0:01:38.500,0:01:39.802 How can we improve things?" 0:01:39.802,0:01:44.210 So having clear applications [br]motivating these literatures 0:01:44.210,0:01:46.400 I think is very helpful. 0:01:46.800,0:01:48.050 - I'm glad you mentioned 0:01:48.050,0:01:49.382 the regression [br]discontinuity, Guido. 0:01:49.382,0:01:53.300 I think there's a lot of [br]complementarity between IV and RD, 0:01:54.700,0:01:57.060 Instrumental Variables[br]and Regression Discontinuity. 0:02:00.506,0:02:03.260 A lot of the econometric[br]applications 0:02:03.260,0:02:04.520 of regression discontinuity 0:02:04.520,0:02:07.230 are what used to be called [br]"fuzzy" RD, 0:02:07.230,0:02:11.620 where, you know, it's not discrete [br]or deterministic at the cutoff 0:02:11.620,0:02:14.900 but just the change[br]in rates or intensity. 0:02:14.900,0:02:17.737 And the late framework[br]helps us understand 0:02:17.737,0:02:18.740 those applications 0:02:18.740,0:02:21.140 and gives us a clear interpretation 0:02:21.140,0:02:25.000 for something like[br]in my paper with Victor Lavy, 0:02:25.000,0:02:28.100 where we use Maimonides'[br]rule, the class size cutoffs, 0:02:28.430,0:02:30.030 and what are you getting there? 0:02:30.290,0:02:31.820 Of course, you can[br]answer that question 0:02:31.820,0:02:33.900 with a linear [br]constant effects model, 0:02:34.200,0:02:36.310 but it turns out [br]we're not limited to that, 0:02:36.310,0:02:39.889 and RD is still very powerful [br]and illuminating, 0:02:40.630,0:02:43.092 even when the correlation 0:02:43.092,0:02:45.866 between the cutoff[br]and the variable of interest, 0:02:45.866,0:02:49.133 in this case class size, [br]is partial, 0:02:49.133,0:02:51.000 maybe even not that strong. 0:02:52.000,0:02:54.999 So there was definitely kind[br]of a parallel development. 0:02:54.999,0:02:56.400 It's also interesting -- 0:02:56.600,0:02:59.780 nobody talked about[br]regression discontinuity designs 0:02:59.780,0:03:01.220 when we were in graduate school. 0:03:01.220,0:03:05.300 It was something that other social[br]scientists were interested in, 0:03:05.800,0:03:09.507 and that grew up [br]alongside the LATE framework, 0:03:09.507,0:03:11.927 and we've both done work 0:03:11.927,0:03:14.565 on both applications[br]and methods there, 0:03:14.565,0:03:18.377 and it's been very exciting [br]to see that develop 0:03:18.377,0:03:19.800 and become so important. 0:03:20.000,0:03:21.767 It's part of a general evolution, 0:03:21.767,0:03:26.086 I think, towards credible[br]identification strategies, 0:03:26.086,0:03:27.441 causal effects... 0:03:29.393,0:03:30.642 making econometrics 0:03:30.642,0:03:33.300 more about causal questions[br]than about models. 0:03:33.640,0:03:34.650 In terms of the future, 0:03:34.650,0:03:37.660 I think one thing that LATE [br]has helped facilitate 0:03:37.660,0:03:42.008 is a move towards[br]more creative, randomized trials 0:03:42.008,0:03:44.400 where there's[br]something of interest. 0:03:45.500,0:03:48.460 It's not possible [br]or straightforward 0:03:48.460,0:03:50.700 to simply turn it off or on, 0:03:51.000,0:03:54.584 but you can encourage it [br]or discourage it. 0:03:54.584,0:03:58.200 So you subsidize schooling [br]with financial aid, for example. 0:03:59.000,0:04:02.080 So now we have a whole [br]framework for interpreting that, 0:04:03.600,0:04:07.113 and it opens the doors[br]to randomized trials 0:04:07.113,0:04:10.300 of things that maybe would 0:04:10.300,0:04:12.471 not have seemed possible before. 0:04:14.500,0:04:17.864 We've used that a lot in the work[br]we do on schools in our -- 0:04:17.864,0:04:21.160 in the Blueprint Lab at MIT. 0:04:22.360,0:04:26.600 We're exploiting random assignment[br]in very creative ways, I think. 0:04:28.100,0:04:31.395 - [Isaiah] Related to that,[br]do you see particular factors 0:04:31.395,0:04:34.445 that make for useful research[br]in econometrics? 0:04:34.445,0:04:38.290 You've alluded to[br]it having a clear connection 0:04:38.290,0:04:40.300 to problems [br]that are actually coming up, 0:04:40.300,0:04:42.650 and empirical practice [br]is often a good idea. 0:04:43.290,0:04:45.000 - Isn't it always a good idea? 0:04:45.700,0:04:47.292 I often find myself sitting 0:04:47.292,0:04:50.100 in an econometrics[br]theory seminar, 0:04:50.700,0:04:52.500 say the Harvard MIT seminar, 0:04:53.400,0:04:56.350 and I'm thinking, "What problem[br]is this guy solving? 0:04:56.350,0:04:57.960 Who has this problem?" 0:04:57.960,0:04:59.800 And, you know... 0:05:01.600,0:05:04.700 sometimes there's an[br]embarrassing silence if I ask 0:05:04.900,0:05:08.300 or there might be[br]a fairly contrived scenario. 0:05:08.800,0:05:11.600 I want to see [br]where the tool is useful. 0:05:12.500,0:05:14.765 There are some [br]purely foundational tools. 0:05:14.765,0:05:16.250 I do take the point. 0:05:16.250,0:05:21.735 There are people who are working[br]on conceptual foundations of ... 0:05:22.600,0:05:25.300 it becomes more like[br]mathematical statistics. 0:05:25.800,0:05:27.653 I mean, I remember [br]an early example of that 0:05:27.653,0:05:29.920 that I struggled to understand 0:05:29.920,0:05:32.500 was the idea[br]of stochastic equicontinuity, 0:05:32.500,0:05:35.070 which one of my thesis advisors,[br]Whitney Newey, 0:05:35.070,0:05:36.479 was using to great effect, 0:05:36.479,0:05:38.821 and I was trying[br]to understand that. 0:05:40.600,0:05:42.034 It's really foundational. 0:05:42.034,0:05:45.200 it's not an application[br]that's driving that -- 0:05:45.890,0:05:47.300 at least not immediately. 0:05:48.600,0:05:53.200 But most things are not like that,[br]and so there should be a problem. 0:05:53.800,0:05:59.100 And I think it's on the seller [br]of that sort of thing, 0:06:00.480,0:06:02.250 because there's opportunity cost, 0:06:02.250,0:06:05.295 the time and attention,[br]and effort to understand things. 0:06:05.980,0:06:07.200 It's on the seller to say, 0:06:07.400,0:06:08.900 "Hey, I'm solving this problem, 0:06:09.400,0:06:12.900 and here's a set of results[br]that show that it's useful, 0:06:12.900,0:06:15.200 and here's some insight[br]that I get." 0:06:16.200,0:06:18.280 - [Isaiah] As you said, Josh,[br]there's been a move 0:06:18.280,0:06:20.700 in the direction of thinking [br]more about causality 0:06:20.700,0:06:22.900 in economics and empirical[br]work in economics. 0:06:22.900,0:06:24.800 Any consequences of the -- 0:06:24.800,0:06:26.570 the spread of that view [br]that surprised you 0:06:26.570,0:06:28.347 or anything that you view[br]as downsides 0:06:28.705,0:06:31.400 of the way that empirical[br]economics has gone? 0:06:31.500,0:06:34.322 - Sometimes I see [br]somebody does IV, 0:06:34.322,0:06:38.304 and they get a result [br]which seems implausibly large. 0:06:38.800,0:06:40.200 That's the usual case. 0:06:42.500,0:06:45.220 So it might be[br]an extraordinarily large 0:06:45.220,0:06:48.600 causal effect of some [br]relatively minor intervention, 0:06:49.100,0:06:52.260 which was randomized[br]or for which you could make a case 0:06:52.260,0:06:54.490 that there's a good design. 0:06:54.900,0:06:57.205 And then when I see that, 0:06:57.944,0:07:00.101 I think it's very hard [br]for me to believe 0:07:00.101,0:07:02.030 that this relatively [br]minor intervention 0:07:02.030,0:07:03.720 has such a large effect. 0:07:04.100,0:07:06.277 The author will sometimes resort 0:07:06.277,0:07:08.690 to the local average [br]treatment effects theorem 0:07:08.690,0:07:11.066 and say, "Well, these compliers, 0:07:11.066,0:07:12.700 they're special in some way." 0:07:13.300,0:07:15.800 And they just benefit[br]extraordinarily 0:07:15.800,0:07:17.600 from this intervention. 0:07:18.100,0:07:21.175 I'm reluctant to take that[br]at face value. 0:07:21.175,0:07:23.750 I think often when effects[br]are too big, 0:07:24.300,0:07:26.780 it's because the exclusion[br]restriction is failing, 0:07:26.780,0:07:29.456 so you don't really have the right[br]endogenous variable 0:07:29.456,0:07:31.380 to scale that result. 0:07:32.000,0:07:35.700 And so I'm not too happy to see 0:07:36.937,0:07:40.022 a generic heterogeneity argument 0:07:40.022,0:07:41.760 being used to excuse something 0:07:41.760,0:07:43.800 that I think might be [br]a deeper problem. 0:07:45.190,0:07:47.358 - [Guido] I think it played [br]somewhat of an unfortunate role 0:07:47.358,0:07:50.083 in the discussions[br]between reduced form 0:07:50.083,0:07:51.700 and structural approaches, 0:07:51.700,0:07:55.510 where I feel [br]that wasn't quite right. 0:07:56.090,0:07:58.810 The instrumental [br]variables assumptions 0:07:58.810,0:08:03.483 are at the core, structural [br]assumptions about behavior -- 0:08:03.483,0:08:05.200 they were coming from economic... 0:08:07.100,0:08:09.900 thinking about the economic[br]behavior of agents, 0:08:10.300,0:08:15.100 and somehow it got pushed[br]in a direction 0:08:15.100,0:08:17.600 that I think wasn't [br]really very helpful. 0:08:20.426,0:08:21.663 I think, initially, 0:08:22.800,0:08:24.067 we wrote things up, 0:08:24.067,0:08:26.480 it was describing[br]what was happening. 0:08:26.480,0:08:29.783 There were a set of methods [br]people were using. 0:08:29.783,0:08:32.111 We clarified what [br]those methods were doing 0:08:32.811,0:08:38.361 and in a way that I think[br]contain a fair amount of insight. 0:08:39.100,0:08:42.050 But it somehow [br]got pushed into a corner 0:08:42.050,0:08:45.379 that I don't think [br]was necessarily very helpful. 0:08:45.379,0:08:48.604 - In just the language[br]of reduced form versus structural, 0:08:48.604,0:08:50.306 I find kind of funny in the sense 0:08:50.306,0:08:52.985 that the local average[br]treatment effect model, 0:08:52.985,0:08:54.154 the potential outcomes model 0:08:54.154,0:08:56.110 is a nonparametric[br]structural model, 0:08:56.110,0:08:58.600 if you want to think about it, [br]as you suggested, Guido. 0:08:58.600,0:09:01.129 So there's something a little funny 0:09:01.129,0:09:03.505 about putting these [br]two things in opposition when -- 0:09:03.505,0:09:05.116 - [Guido] Yes.[br]- [Josh] That language, of course, 0:09:05.116,0:09:08.371 comes from the simultaneous[br]equations framework 0:09:08.371,0:09:09.641 that we inherited. 0:09:10.400,0:09:11.440 It has the advantage 0:09:11.440,0:09:14.160 that people seem to know [br]what you mean when you use it, 0:09:14.160,0:09:16.240 but that might be that people [br]are hearing different -- 0:09:16.240,0:09:18.300 different people[br]are hearing different things. 0:09:18.300,0:09:20.480 - [Guido] Yeah. I think[br]reduced form has become 0:09:20.480,0:09:22.200 used in a little bit [br]of the pejorative way... 0:09:22.200,0:09:23.540 - [Josh] Sometimes. 0:09:25.104,0:09:28.250 ...which is not really quite what [br]it was originally intended for. 0:09:30.100,0:09:33.090 - [Isaiah] I guess something else[br]that strikes me in thinking about 0:09:33.090,0:09:35.645 the effects of the local average[br]treatment effect framework 0:09:35.645,0:09:37.676 is that often folks will appeal 0:09:37.676,0:09:40.000 to a local average treatment[br]effects intuition 0:09:40.000,0:09:42.358 for settings well beyond ones 0:09:42.358,0:09:44.963 where any sort of formal result[br]has actually been established. 0:09:45.440,0:09:49.180 And I'm curious, given all the work[br]that you guys did 0:09:49.180,0:09:52.390 to establish late results [br]in different settings, 0:09:52.390,0:09:54.415 I'm curious, any thoughts on that? 0:09:55.360,0:09:57.306 - I think there's going[br]to be a lot of cases 0:09:57.306,0:10:02.200 where the intuition [br]does get you some distance, 0:10:02.800,0:10:04.989 but it's going to be [br]somewhat limited, 0:10:04.989,0:10:07.600 and establishing [br]formal results there 0:10:08.400,0:10:09.490 may be a little tricky 0:10:09.490,0:10:12.700 and then maybe only work [br]in special circumstances, 0:10:14.600,0:10:16.540 and you end up[br]with a lot of formality 0:10:16.540,0:10:19.500 that may not quite [br]capture the intuition. 0:10:19.900,0:10:21.550 Sometimes I'm somewhat [br]uneasy with them, 0:10:21.550,0:10:24.438 and they are not necessarily[br]the papers I would want to write, 0:10:25.148,0:10:27.819 but I do think intuition 0:10:27.819,0:10:31.217 often does capture [br]part of the problem. 0:10:33.100,0:10:36.300 I think, in some sense we were[br]kind of very fortunate there 0:10:36.900,0:10:39.250 in the way that the late paper [br]got handled at the journal, 0:10:39.250,0:10:41.766 is that, actually, the editor,[br]made it much shorter 0:10:42.100,0:10:46.300 and that then allowed us to kind of[br]focus on very clear, crisp results. 0:10:47.100,0:10:51.770 Where if you -- you know, this [br]somewhat unfortunate tendency 0:10:51.770,0:10:52.985 in the econometrics literature 0:10:52.985,0:10:55.100 of having the papers [br]get longer and longer. 0:10:55.100,0:10:56.690 - [Josh] Well, you should [br]be able to fix that, man. 0:10:56.690,0:10:58.800 - [Guido] I'm trying to fix that. 0:10:59.400,0:11:01.625 But I think this is an example [br]where it's sort of very clear 0:11:01.625,0:11:03.498 that having it be short[br]is actually -- 0:11:03.498,0:11:04.842 - [Josh] You should impose[br]that no paper 0:11:04.842,0:11:06.655 can be longer than the late paper. 0:11:06.655,0:11:08.000 - [Guido] That, wow. 0:11:08.000,0:11:09.617 That may be great. 0:11:09.617,0:11:11.685 - [Josh] At least no theory, [br]no theory paper. 0:11:11.892,0:11:14.300 - [Guido] Yeah, [br]and I think, I think... 0:11:14.500,0:11:16.800 I'm trying very hard to get[br]the papers to be shorter. 0:11:16.800,0:11:18.700 And I think there's a lot of value 0:11:19.200,0:11:21.480 today because it's often[br]the second part of the paper 0:11:21.480,0:11:26.395 that doesn't actually get you much[br]further in understanding things 0:11:27.000,0:11:29.870 and it does make things [br]much harder to read 0:11:30.630,0:11:33.200 and, you know, [br]it sort of goes back 0:11:33.200,0:11:36.111 to how I think econometrics [br]should be done, 0:11:36.111,0:11:38.070 you should focus on -- 0:11:38.700,0:11:41.300 It should be reasonably[br]close to empirical problems. 0:11:41.500,0:11:43.900 They should be very clear problems. 0:11:44.800,0:11:48.900 But then often the theory[br]doesn't need to be quite so long. 0:11:48.900,0:11:50.010 - [Josh] Yeah. 0:11:51.100,0:11:54.670 - [Guido] I think things have gone[br]a little off track. 0:11:56.260,0:11:57.750 - [Isaiah] A new relatively [br]recent change 0:11:57.750,0:12:00.230 has been a seeming big [br]increase in demand 0:12:00.230,0:12:02.200 for people with sort of [br]econometrics, 0:12:02.200,0:12:04.800 causal effect estimation skills[br]in the tech sector. 0:12:05.000,0:12:07.480 I'm interested, [br]do either of you have thoughts 0:12:07.480,0:12:09.840 on sort of how [br]that's going to interact 0:12:09.840,0:12:11.600 with the development [br]of empirical methods, 0:12:11.600,0:12:13.950 or empirical research[br]in economics going forward? 0:12:14.600,0:12:16.770 - [Josh] Well, there's[br]sort of a meta point, 0:12:16.770,0:12:21.000 which is, there's this new [br]kind of employer, 0:12:21.800,0:12:27.530 the Amazons and the Uber,[br]and, you know, TripAdvisor world, 0:12:28.000,0:12:29.300 and I think that's great. 0:12:29.300,0:12:32.600 And I like to tell my students[br]about that, you know, especially -- 0:12:32.600,0:12:35.500 at MIT we have a lot of [br]computer science majors. 0:12:35.500,0:12:37.000 That's our biggest major. 0:12:37.400,0:12:42.246 And I try to seduce some of those[br]folks into economics by saying, 0:12:42.246,0:12:45.700 you know, you can go [br]work for these, 0:12:45.800,0:12:49.250 you know, companies that people[br]are very keen to work for 0:12:49.250,0:12:50.800 because the work seems exciting, 0:12:52.000,0:12:54.250 you know, that the skills [br]that you get in econometrics 0:12:54.250,0:12:56.100 are as good or better 0:12:56.100,0:12:59.736 than any competing [br]discipline has to offer. 0:12:59.736,0:13:01.100 So you should at least 0:13:01.400,0:13:04.200 take some econ, take some[br]econometrics, and some econ. 0:13:04.800,0:13:07.000 I did a fun project with Uber 0:13:07.600,0:13:09.770 on labor supply of Uber drivers 0:13:09.920,0:13:12.805 and was very, very exciting[br]to be part of that. 0:13:13.100,0:13:15.400 Plus I got to drive [br]for Uber for a while 0:13:15.900,0:13:17.730 and I thought that was fun too. 0:13:17.730,0:13:20.700 I did not make enough[br]that I was tempted to 0:13:21.100,0:13:25.100 give up my MIT job, [br]but I enjoyed the experience. 0:13:25.230,0:13:30.900 I see a potential challenge to our[br]model of graduate education here, 0:13:31.700,0:13:37.400 which is, if we're training people[br]to go work at Amazon, you know, 0:13:37.900,0:13:41.190 it's not clear why, you know, [br]we should be paying 0:13:41.190,0:13:42.900 graduate stipends for that. 0:13:43.200,0:13:47.280 Why should the taxpayer effectively[br]be subsidizing that. 0:13:47.280,0:13:51.400 Our graduate education[br]in the US Is generously subsidized, 0:13:51.400,0:13:53.160 even in private universities, 0:13:53.160,0:13:56.100 it's ultimately -- there's a lot of[br]public money there, 0:13:56.100,0:13:59.300 and I think the[br]traditional rationale for that is, 0:13:59.500,0:14:02.137 you know, we were training [br]educators and scholars, 0:14:02.137,0:14:05.657 and there's a great externality[br]from the work that we do, 0:14:05.657,0:14:07.607 it's either [br]the research externality, 0:14:07.607,0:14:09.557 or a teaching externality. 0:14:10.100,0:14:12.350 But, you know, [br]if many of our students 0:14:12.350,0:14:14.600 are going to work[br]in the private sector, 0:14:16.300,0:14:21.700 that's fine, but maybe their [br]employers should pay for that. 0:14:22.120,0:14:23.370 - [Guido] But maybe[br]is not so different 0:14:23.370,0:14:26.780 from people working [br]for consulting firms. 0:14:27.200,0:14:28.780 It's not clear to me 0:14:28.780,0:14:32.836 that the number of jobs[br]in academics has changed. 0:14:33.370,0:14:36.325 - [Josh] I feel like this [br]is a growing sector, 0:14:36.325,0:14:39.289 whereas consulting -- [br]you're right to raise that, 0:14:39.289,0:14:42.100 it might be the same [br]for consulting, 0:14:43.300,0:14:44.846 but this, you know, 0:14:44.846,0:14:47.500 I'm placing more and more [br]students in these businesses. 0:14:47.500,0:14:50.400 So, it's on my mind [br]in a way that I've sort of, 0:14:50.800,0:14:53.960 you know, not been attentive [br]to consulting jobs, 0:14:53.960,0:14:56.920 you know, consulting [br]was always important, 0:14:56.920,0:14:58.950 and I think also [br]there's some movement 0:14:58.950,0:15:02.600 from consulting back into research,[br]it's a little more fluid. 0:15:03.900,0:15:07.630 A lot of the work in both domains 0:15:07.630,0:15:09.430 I have to say, [br]it's not really different 0:15:09.430,0:15:12.730 but, you know, people who [br]are working in the tech sector 0:15:12.730,0:15:15.480 are doing things that are [br]potentially of scientific interest, 0:15:15.480,0:15:16.800 but mostly it's hidden. 0:15:17.100,0:15:18.550 Then you really I have to say, 0:15:18.550,0:15:20.900 you know, why is the government[br]paying for this? 0:15:21.800,0:15:23.732 Yeah, although, yeah, [br]I mean to Guidos point, 0:15:23.732,0:15:26.102 I guess there's [br]a data question here 0:15:26.102,0:15:30.042 of it has the sort of [br]total [no-neck] sort of say 0:15:31.300,0:15:34.870 for-profit sector employment [br]of econ Ph.D. program graduates 0:15:34.870,0:15:38.290 increased or has it just been[br]a substitution from finance 0:15:38.290,0:15:40.200 and consulting towards tech. 0:15:40.300,0:15:42.300 - [Josh] I may be reacting [br]to something 0:15:42.300,0:15:44.300 that's not really happening. 0:15:44.400,0:15:45.890 - [Guido] I've actually [br]done some work 0:15:45.890,0:15:48.200 with some of these tech companies. 0:15:49.100,0:15:52.200 I don't disagree with Josh's point [br]that we need to think 0:15:52.200,0:15:53.830 a little bit about [br]the funding model, 0:15:53.830,0:15:56.390 who is it in the end paying [br]for the graduate education. 0:15:56.710,0:15:59.400 But from a scientific perspective, 0:15:59.980,0:16:02.540 not only do these places[br]have great data 0:16:02.540,0:16:04.950 and nowadays they tend to be[br]very careful with that 0:16:04.950,0:16:07.100 for privacy reasons, 0:16:07.380,0:16:08.900 but they also have great questions. 0:16:10.200,0:16:13.213 I find it very inspiring [br]kind of to listen 0:16:13.213,0:16:15.950 to the people there and kind of see[br]what kind of questions they have, 0:16:15.950,0:16:17.330 and often they're questions 0:16:18.200,0:16:21.510 that also come up outside [br]of these companies. 0:16:21.510,0:16:27.430 I have a couple of papers [br]with Raj Chetty and Susan Athey, 0:16:27.430,0:16:31.600 where we look at ways [br]of combining experimental data 0:16:31.600,0:16:34.000 and observational data,[br]and kind of their -- 0:16:35.500,0:16:38.600 Raj Chetty was interested[br]in what is the effect 0:16:38.600,0:16:42.893 of early childhood programs [br]on outcomes later in life, 0:16:42.893,0:16:46.330 not just kind on test scores,[br]but on earnings and stuff, 0:16:46.330,0:16:48.300 and we kind of developed methods 0:16:48.600,0:16:51.500 that would help you shed[br]light on that, onto some -- 0:16:52.760,0:16:56.920 in some settings[br]and the same problems came up 0:16:56.920,0:17:00.533 kind of in this [br]tech company settings. 0:17:00.800,0:17:03.240 And so from my perspective, 0:17:03.240,0:17:05.420 it's the same kind of -- 0:17:05.420,0:17:07.600 I was talking to people [br]doing empirical work, 0:17:07.600,0:17:09.700 I tried to kind of look at these[br]specific problems 0:17:09.700,0:17:13.370 and then try to come up[br]with more general problems, 0:17:15.110,0:17:18.230 we formulated the problems[br]at a higher level, 0:17:18.500,0:17:22.900 so that I can think about solutions[br]that work in a range of settings. 0:17:23.400,0:17:24.840 And so from that perspective, 0:17:24.840,0:17:27.570 the interactions [br]with the tech companies 0:17:27.570,0:17:30.300 are just very valuable[br]and very useful. 0:17:31.700,0:17:35.030 We do have students now[br]doing internships there 0:17:35.030,0:17:38.390 and then coming back [br]and writing more interesting thesis 0:17:38.390,0:17:43.400 as a result of their [br]experiences there. 0:17:44.600,0:17:47.020 - [Narrator] If you'd like to watch[br]more Nobel Conversations, 0:17:47.020,0:17:48.200 click here, 0:17:48.200,0:17:50.500 or if you'd like to learn[br]more about econometrics, 0:17:50.500,0:17:53.100 check out Josh's [br]"Mastering Econometrics" series. 0:17:53.700,0:17:56.720 If you'd like to learn more [br]about Guido, Josh and Isaiah 0:17:56.720,0:17:58.300 check out the links [br]in the description. 0:17:59.036,0:18:01.495 ♪ [music] ♪