[Script Info] Title: [Events] Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text Dialogue: 0,0:00:00.00,0:00:02.55,Default,,0000,0000,0000,,♪ [music] ♪ Dialogue: 0,0:00:03.80,0:00:05.80,Default,,0000,0000,0000,,- [Narrator] Welcome \Nto Nobel Conversations. Dialogue: 0,0:00:07.04,0:00:08.10,Default,,0000,0000,0000,,In this episode, Dialogue: 0,0:00:08.10,0:00:11.57,Default,,0000,0000,0000,,Josh Angrist and Guido Imbens\Nsit down with Isaiah Andrews Dialogue: 0,0:00:11.57,0:00:14.60,Default,,0000,0000,0000,,to discuss how the field \Nof econometrics is evolving. Dialogue: 0,0:00:16.10,0:00:18.75,Default,,0000,0000,0000,,- [Isaiah] So, Guido and Josh, \Nyou're both pioneers Dialogue: 0,0:00:18.75,0:00:21.50,Default,,0000,0000,0000,,in developing tools for\Nempirical research in economics. Dialogue: 0,0:00:21.50,0:00:23.17,Default,,0000,0000,0000,,And so I'd like to explore Dialogue: 0,0:00:23.17,0:00:25.30,Default,,0000,0000,0000,,where you feel like \Nthe field is heading -- Dialogue: 0,0:00:25.71,0:00:28.08,Default,,0000,0000,0000,,economics, econometrics,\Nthe whole thing. Dialogue: 0,0:00:28.51,0:00:31.30,Default,,0000,0000,0000,,To start, I'd be interested to hear Dialogue: 0,0:00:32.17,0:00:35.20,Default,,0000,0000,0000,,about whether you feel\Nthe way in which Dialogue: 0,0:00:35.20,0:00:38.51,Default,,0000,0000,0000,,the local average treatment \Neffects framework took hold Dialogue: 0,0:00:38.80,0:00:42.19,Default,,0000,0000,0000,,has any lessons for how \Nnew empirical methods in economics Dialogue: 0,0:00:42.19,0:00:44.30,Default,,0000,0000,0000,,develop and spread\Nor how they should. Dialogue: 0,0:00:44.56,0:00:45.96,Default,,0000,0000,0000,,- [Josh] That's a good question. Dialogue: 0,0:00:46.61,0:00:47.79,Default,,0000,0000,0000,,You go first. Dialogue: 0,0:00:47.79,0:00:49.24,Default,,0000,0000,0000,,[laughter] Dialogue: 0,0:00:49.70,0:00:52.94,Default,,0000,0000,0000,,- [Guido] Yeah, so I think \Nthe important thing Dialogue: 0,0:00:52.94,0:00:58.55,Default,,0000,0000,0000,,is to come up \Nwith good convincing cases Dialogue: 0,0:00:58.55,0:01:02.21,Default,,0000,0000,0000,,where the questions are clear Dialogue: 0,0:01:02.40,0:01:05.72,Default,,0000,0000,0000,,and where the methods\Napply in general. Dialogue: 0,0:01:06.25,0:01:07.56,Default,,0000,0000,0000,,One thing I... Dialogue: 0,0:01:08.19,0:01:12.00,Default,,0000,0000,0000,,Looking back\Nat the subsequent literature. Dialogue: 0,0:01:12.20,0:01:16.70,Default,,0000,0000,0000,,I really like the regression\Ndiscontinuity literature Dialogue: 0,0:01:16.70,0:01:19.67,Default,,0000,0000,0000,,where there were clearly a bunch\Nof really convincing examples Dialogue: 0,0:01:19.67,0:01:23.38,Default,,0000,0000,0000,,and that allowed people\Nto think more clearly, Dialogue: 0,0:01:23.38,0:01:27.20,Default,,0000,0000,0000,,look harder \Nat the methodological questions. Dialogue: 0,0:01:27.40,0:01:28.80,Default,,0000,0000,0000,,Do clear applications Dialogue: 0,0:01:28.80,0:01:30.60,Default,,0000,0000,0000,,that then allow you \Nto kind of think about, Dialogue: 0,0:01:30.60,0:01:33.60,Default,,0000,0000,0000,,"Wow, do these type of assumptions\Nseem reasonable here? Dialogue: 0,0:01:33.60,0:01:38.00,Default,,0000,0000,0000,,What kind of things do we not like\Nin the early papers? Dialogue: 0,0:01:38.50,0:01:39.80,Default,,0000,0000,0000,,How can we improve things?" Dialogue: 0,0:01:39.80,0:01:44.21,Default,,0000,0000,0000,,So having clear applications \Nmotivating these literatures Dialogue: 0,0:01:44.21,0:01:46.40,Default,,0000,0000,0000,,I think is very helpful. Dialogue: 0,0:01:46.80,0:01:48.05,Default,,0000,0000,0000,,- I'm glad you mentioned Dialogue: 0,0:01:48.05,0:01:49.38,Default,,0000,0000,0000,,the regression \Ndiscontinuity, Guido. Dialogue: 0,0:01:49.38,0:01:53.30,Default,,0000,0000,0000,,I think there's a lot of \Ncomplementarity between IV and RD, Dialogue: 0,0:01:54.70,0:01:57.06,Default,,0000,0000,0000,,instrumental variables\Nand regression discontinuity. Dialogue: 0,0:02:00.51,0:02:03.26,Default,,0000,0000,0000,,A lot of the econometric\Napplications Dialogue: 0,0:02:03.26,0:02:04.52,Default,,0000,0000,0000,,of regression discontinuity Dialogue: 0,0:02:04.52,0:02:07.23,Default,,0000,0000,0000,,are what used\Nto be called "fuzzy" RD, Dialogue: 0,0:02:07.23,0:02:11.62,Default,,0000,0000,0000,,where it's not discrete \Nor deterministic at the cutoff Dialogue: 0,0:02:11.62,0:02:14.90,Default,,0000,0000,0000,,but just the change\Nin rates or intensity. Dialogue: 0,0:02:14.90,0:02:17.74,Default,,0000,0000,0000,,And the LATE framework\Nhelps us understand Dialogue: 0,0:02:17.74,0:02:18.74,Default,,0000,0000,0000,,those applications Dialogue: 0,0:02:18.74,0:02:21.14,Default,,0000,0000,0000,,and gives us a clear interpretation Dialogue: 0,0:02:21.14,0:02:25.00,Default,,0000,0000,0000,,for something like\Nin my paper with Victor Lavy, Dialogue: 0,0:02:25.00,0:02:28.10,Default,,0000,0000,0000,,where we use Maimonides'\Nrule, the class size cutoffs, Dialogue: 0,0:02:28.43,0:02:30.03,Default,,0000,0000,0000,,and what are you getting there? Dialogue: 0,0:02:30.29,0:02:31.82,Default,,0000,0000,0000,,Of course, you can\Nanswer that question Dialogue: 0,0:02:31.82,0:02:33.90,Default,,0000,0000,0000,,with a linear \Nconstant effects model, Dialogue: 0,0:02:34.20,0:02:36.31,Default,,0000,0000,0000,,but it turns out \Nwe're not limited to that, Dialogue: 0,0:02:36.31,0:02:39.89,Default,,0000,0000,0000,,and RD is still very powerful \Nand illuminating, Dialogue: 0,0:02:40.63,0:02:43.09,Default,,0000,0000,0000,,even when the correlation Dialogue: 0,0:02:43.09,0:02:45.87,Default,,0000,0000,0000,,between the cutoff\Nand the variable of interest, Dialogue: 0,0:02:45.87,0:02:49.13,Default,,0000,0000,0000,,in this case, class size, \Nis partial, Dialogue: 0,0:02:49.13,0:02:51.00,Default,,0000,0000,0000,,maybe even not that strong. Dialogue: 0,0:02:52.00,0:02:54.100,Default,,0000,0000,0000,,So there was definitely\Na parallel development. Dialogue: 0,0:02:54.100,0:02:56.40,Default,,0000,0000,0000,,It's also interesting... Dialogue: 0,0:02:57.25,0:02:59.78,Default,,0000,0000,0000,,Nobody talked about\Nregression discontinuity designs Dialogue: 0,0:02:59.78,0:03:01.22,Default,,0000,0000,0000,,when we were in graduate school -- Dialogue: 0,0:03:01.22,0:03:02.84,Default,,0000,0000,0000,,it was something Dialogue: 0,0:03:02.84,0:03:05.30,Default,,0000,0000,0000,,that other social scientists\Nwere interested in, Dialogue: 0,0:03:05.80,0:03:09.51,Default,,0000,0000,0000,,and that grew up \Nalongside the LATE framework, Dialogue: 0,0:03:09.51,0:03:11.93,Default,,0000,0000,0000,,and we've both done work Dialogue: 0,0:03:11.93,0:03:14.56,Default,,0000,0000,0000,,on both applications\Nand methods there, Dialogue: 0,0:03:14.56,0:03:18.38,Default,,0000,0000,0000,,and it's been very exciting \Nto see that develop Dialogue: 0,0:03:18.38,0:03:19.80,Default,,0000,0000,0000,,and become so important. Dialogue: 0,0:03:20.00,0:03:21.77,Default,,0000,0000,0000,,It's part of a general evolution, Dialogue: 0,0:03:21.77,0:03:26.09,Default,,0000,0000,0000,,I think, towards credible\Nidentification strategies, Dialogue: 0,0:03:26.09,0:03:27.44,Default,,0000,0000,0000,,causal effects, Dialogue: 0,0:03:29.39,0:03:30.64,Default,,0000,0000,0000,,making econometrics Dialogue: 0,0:03:30.64,0:03:33.30,Default,,0000,0000,0000,,more about causal questions\Nthan about models. Dialogue: 0,0:03:33.64,0:03:34.65,Default,,0000,0000,0000,,In terms of the future, Dialogue: 0,0:03:34.65,0:03:37.66,Default,,0000,0000,0000,,I think one thing that LATE \Nhas helped facilitate Dialogue: 0,0:03:37.66,0:03:42.01,Default,,0000,0000,0000,,is a move towards\Nmore creative, randomized trials Dialogue: 0,0:03:42.01,0:03:44.40,Default,,0000,0000,0000,,where there's\Nsomething of interest. Dialogue: 0,0:03:45.50,0:03:48.46,Default,,0000,0000,0000,,It's not possible \Nor straightforward Dialogue: 0,0:03:48.46,0:03:50.70,Default,,0000,0000,0000,,to simply turn it off or on, Dialogue: 0,0:03:51.00,0:03:54.58,Default,,0000,0000,0000,,but you can encourage it \Nor discourage it. Dialogue: 0,0:03:54.58,0:03:58.20,Default,,0000,0000,0000,,So you subsidize schooling \Nwith financial aid, for example. Dialogue: 0,0:03:59.00,0:04:02.08,Default,,0000,0000,0000,,So now we have a whole \Nframework for interpreting that, Dialogue: 0,0:04:03.60,0:04:07.11,Default,,0000,0000,0000,,and it opens the doors\Nto randomized trials Dialogue: 0,0:04:07.11,0:04:09.26,Default,,0000,0000,0000,,of things that maybe would Dialogue: 0,0:04:10.30,0:04:12.47,Default,,0000,0000,0000,,not have seemed possible before. Dialogue: 0,0:04:14.50,0:04:17.86,Default,,0000,0000,0000,,We've used that a lot in the work\Nwe do on schools in our -- Dialogue: 0,0:04:17.86,0:04:21.16,Default,,0000,0000,0000,,in the Blueprint Lab at MIT. Dialogue: 0,0:04:22.36,0:04:26.60,Default,,0000,0000,0000,,We're exploiting random assignment\Nin very creative ways, I think. Dialogue: 0,0:04:28.10,0:04:31.40,Default,,0000,0000,0000,,- [Isaiah] Related to that,\Ndo you see particular factors Dialogue: 0,0:04:31.40,0:04:34.44,Default,,0000,0000,0000,,that make for useful research\Nin econometrics? Dialogue: 0,0:04:34.44,0:04:38.44,Default,,0000,0000,0000,,You've alluded to it having\Na clear connection Dialogue: 0,0:04:38.44,0:04:40.30,Default,,0000,0000,0000,,to problems \Nthat are actually coming up, Dialogue: 0,0:04:40.30,0:04:42.86,Default,,0000,0000,0000,,and empirical practice \Nis often a good idea. Dialogue: 0,0:04:43.29,0:04:45.00,Default,,0000,0000,0000,,- Isn't it always a good idea? Dialogue: 0,0:04:45.70,0:04:50.11,Default,,0000,0000,0000,,I often find myself sitting\Nin an econometrics theory seminar, Dialogue: 0,0:04:50.70,0:04:52.50,Default,,0000,0000,0000,,say the Harvard MIT seminar, Dialogue: 0,0:04:53.40,0:04:56.35,Default,,0000,0000,0000,,and I'm thinking, "What problem\Nis this guy solving? Dialogue: 0,0:04:56.35,0:04:57.96,Default,,0000,0000,0000,,Who has this problem?" Dialogue: 0,0:04:57.96,0:04:59.80,Default,,0000,0000,0000,,And, you know... Dialogue: 0,0:05:01.60,0:05:04.70,Default,,0000,0000,0000,,sometimes there's an\Nembarrassing silence if I ask Dialogue: 0,0:05:04.90,0:05:08.30,Default,,0000,0000,0000,,or there might be\Na fairly contrived scenario. Dialogue: 0,0:05:08.80,0:05:11.60,Default,,0000,0000,0000,,I want to see \Nwhere the tool is useful. Dialogue: 0,0:05:12.50,0:05:14.76,Default,,0000,0000,0000,,There are some \Npurely foundational tools -- Dialogue: 0,0:05:14.76,0:05:16.25,Default,,0000,0000,0000,,I do take the point. Dialogue: 0,0:05:16.25,0:05:21.74,Default,,0000,0000,0000,,There are people who are working\Non conceptual foundations of... Dialogue: 0,0:05:22.60,0:05:25.30,Default,,0000,0000,0000,,it becomes more like\Nmathematical statistics. Dialogue: 0,0:05:25.80,0:05:27.65,Default,,0000,0000,0000,,I mean, I remember \Nan early example of that Dialogue: 0,0:05:27.65,0:05:29.92,Default,,0000,0000,0000,,that I struggled to understand Dialogue: 0,0:05:29.92,0:05:32.50,Default,,0000,0000,0000,,was the idea\Nof stochastic equicontinuity, Dialogue: 0,0:05:32.50,0:05:35.07,Default,,0000,0000,0000,,which one of my thesis advisors,\NWhitney Newey, Dialogue: 0,0:05:35.07,0:05:36.48,Default,,0000,0000,0000,,was using to great effect, Dialogue: 0,0:05:36.48,0:05:38.82,Default,,0000,0000,0000,,and I was trying\Nto understand that. Dialogue: 0,0:05:40.60,0:05:42.03,Default,,0000,0000,0000,,It's really foundational. Dialogue: 0,0:05:42.03,0:05:45.20,Default,,0000,0000,0000,,it's not an application\Nthat's driving that -- Dialogue: 0,0:05:45.89,0:05:47.30,Default,,0000,0000,0000,,at least not immediately. Dialogue: 0,0:05:48.60,0:05:53.20,Default,,0000,0000,0000,,But most things are not like that,\Nand so there should be a problem. Dialogue: 0,0:05:53.80,0:05:59.25,Default,,0000,0000,0000,,And I think it's on the seller \Nof that sort of thing, Dialogue: 0,0:06:00.48,0:06:02.25,Default,,0000,0000,0000,,because there's opportunity cost, Dialogue: 0,0:06:02.25,0:06:05.30,Default,,0000,0000,0000,,the time and attention,\Nand effort to understand things. Dialogue: 0,0:06:05.98,0:06:07.20,Default,,0000,0000,0000,,It's on the seller to say, Dialogue: 0,0:06:07.40,0:06:08.90,Default,,0000,0000,0000,,"Hey, I'm solving this problem, Dialogue: 0,0:06:09.40,0:06:12.90,Default,,0000,0000,0000,,and here's a set of results\Nthat show that it's useful, Dialogue: 0,0:06:12.90,0:06:15.20,Default,,0000,0000,0000,,and here's some insight\Nthat I get." Dialogue: 0,0:06:16.10,0:06:18.28,Default,,0000,0000,0000,,- [Isaiah] As you said, Josh,\Nthere's been a move Dialogue: 0,0:06:18.28,0:06:20.70,Default,,0000,0000,0000,,in the direction of thinking \Nmore about causality Dialogue: 0,0:06:20.70,0:06:22.90,Default,,0000,0000,0000,,in economics and empirical\Nwork in economics. Dialogue: 0,0:06:22.90,0:06:26.57,Default,,0000,0000,0000,,Any consequences of the spread\Nof that view that surprised you Dialogue: 0,0:06:26.57,0:06:28.35,Default,,0000,0000,0000,,or anything that you view\Nas downsides Dialogue: 0,0:06:28.70,0:06:31.50,Default,,0000,0000,0000,,of the way that empirical\Neconomics has gone? Dialogue: 0,0:06:31.50,0:06:34.32,Default,,0000,0000,0000,,- Sometimes I see \Nsomebody does IV, Dialogue: 0,0:06:34.32,0:06:38.30,Default,,0000,0000,0000,,and they get a result \Nwhich seems implausibly large -- Dialogue: 0,0:06:38.80,0:06:40.20,Default,,0000,0000,0000,,that's the usual case. Dialogue: 0,0:06:42.50,0:06:45.22,Default,,0000,0000,0000,,So it might be\Nan extraordinarily large Dialogue: 0,0:06:45.22,0:06:48.60,Default,,0000,0000,0000,,causal effect of some \Nrelatively minor intervention, Dialogue: 0,0:06:49.10,0:06:52.26,Default,,0000,0000,0000,,which was randomized\Nor for which you could make a case Dialogue: 0,0:06:52.26,0:06:54.49,Default,,0000,0000,0000,,that there's a good design. Dialogue: 0,0:06:54.90,0:06:57.20,Default,,0000,0000,0000,,And then when I see that, Dialogue: 0,0:06:57.94,0:07:00.10,Default,,0000,0000,0000,,I think it's very hard \Nfor me to believe Dialogue: 0,0:07:00.10,0:07:02.03,Default,,0000,0000,0000,,that this relatively \Nminor intervention Dialogue: 0,0:07:02.03,0:07:03.72,Default,,0000,0000,0000,,has such a large effect. Dialogue: 0,0:07:04.10,0:07:06.28,Default,,0000,0000,0000,,The author will sometimes resort Dialogue: 0,0:07:06.28,0:07:08.69,Default,,0000,0000,0000,,to the local average \Ntreatment effects theorem Dialogue: 0,0:07:08.69,0:07:11.07,Default,,0000,0000,0000,,and say, "Well, these compliers -- Dialogue: 0,0:07:11.07,0:07:12.70,Default,,0000,0000,0000,,they're special in some way, Dialogue: 0,0:07:13.30,0:07:15.80,Default,,0000,0000,0000,,and they just benefit\Nextraordinarily Dialogue: 0,0:07:15.80,0:07:17.60,Default,,0000,0000,0000,,from this intervention." Dialogue: 0,0:07:18.10,0:07:21.18,Default,,0000,0000,0000,,I'm reluctant to take that\Nat face value. Dialogue: 0,0:07:21.18,0:07:23.75,Default,,0000,0000,0000,,I think often when effects\Nare too big, Dialogue: 0,0:07:24.30,0:07:26.78,Default,,0000,0000,0000,,it's because the exclusion\Nrestriction is failing, Dialogue: 0,0:07:26.78,0:07:29.46,Default,,0000,0000,0000,,so you don't really have the right\Nendogenous variable Dialogue: 0,0:07:29.46,0:07:31.38,Default,,0000,0000,0000,,to scale that result. Dialogue: 0,0:07:32.00,0:07:35.70,Default,,0000,0000,0000,,And so, I'm not too happy to see Dialogue: 0,0:07:36.94,0:07:39.69,Default,,0000,0000,0000,,a generic heterogeneity argument Dialogue: 0,0:07:40.02,0:07:41.76,Default,,0000,0000,0000,,being used to excuse something Dialogue: 0,0:07:41.76,0:07:43.80,Default,,0000,0000,0000,,that I think might be \Na deeper problem. Dialogue: 0,0:07:45.19,0:07:47.36,Default,,0000,0000,0000,,- [Guido] I think it played \Nsomewhat of an unfortunate role Dialogue: 0,0:07:47.36,0:07:50.08,Default,,0000,0000,0000,,in the discussions\Nbetween reduced form Dialogue: 0,0:07:50.08,0:07:51.70,Default,,0000,0000,0000,,and structural approaches, Dialogue: 0,0:07:51.70,0:07:55.51,Default,,0000,0000,0000,,where I feel \Nthat wasn't quite right. Dialogue: 0,0:07:56.09,0:07:58.81,Default,,0000,0000,0000,,The instrumental \Nvariables assumptions Dialogue: 0,0:07:58.81,0:08:01.51,Default,,0000,0000,0000,,are at the core, Dialogue: 0,0:08:01.51,0:08:03.48,Default,,0000,0000,0000,,structural assumptions\Nabout behavior -- Dialogue: 0,0:08:03.48,0:08:05.20,Default,,0000,0000,0000,,they were coming from economic... Dialogue: 0,0:08:07.10,0:08:09.90,Default,,0000,0000,0000,,thinking about the economic\Nbehavior of agents, Dialogue: 0,0:08:10.30,0:08:15.10,Default,,0000,0000,0000,,and somehow it got pushed\Nin a direction Dialogue: 0,0:08:15.10,0:08:17.60,Default,,0000,0000,0000,,that I think wasn't \Nreally very helpful. Dialogue: 0,0:08:20.43,0:08:21.66,Default,,0000,0000,0000,,I think, initially, Dialogue: 0,0:08:22.80,0:08:24.07,Default,,0000,0000,0000,,we wrote things up, Dialogue: 0,0:08:24.07,0:08:26.48,Default,,0000,0000,0000,,it was describing\Nwhat was happening. Dialogue: 0,0:08:26.48,0:08:29.78,Default,,0000,0000,0000,,There were a set of methods \Npeople were using. Dialogue: 0,0:08:29.78,0:08:32.11,Default,,0000,0000,0000,,We clarified what \Nthose methods were doing Dialogue: 0,0:08:32.81,0:08:38.36,Default,,0000,0000,0000,,in a way that I think contain\Na fair amount of insight. Dialogue: 0,0:08:39.10,0:08:42.05,Default,,0000,0000,0000,,But it somehow \Ngot pushed into a corner Dialogue: 0,0:08:42.05,0:08:45.38,Default,,0000,0000,0000,,that I don't think \Nwas necessarily very helpful. Dialogue: 0,0:08:45.38,0:08:48.60,Default,,0000,0000,0000,,- In just the language\Nof reduced form versus structural, Dialogue: 0,0:08:48.60,0:08:50.31,Default,,0000,0000,0000,,I find kind of funny in the sense Dialogue: 0,0:08:50.31,0:08:52.98,Default,,0000,0000,0000,,that the local average\Ntreatment effect model, Dialogue: 0,0:08:52.98,0:08:54.15,Default,,0000,0000,0000,,the potential outcomes model Dialogue: 0,0:08:54.15,0:08:56.11,Default,,0000,0000,0000,,is a nonparametric\Nstructural model, Dialogue: 0,0:08:56.11,0:08:58.60,Default,,0000,0000,0000,,if you want to think about it, \Nas you suggested, Guido. Dialogue: 0,0:08:58.60,0:09:01.13,Default,,0000,0000,0000,,So there's something a little funny Dialogue: 0,0:09:01.13,0:09:03.50,Default,,0000,0000,0000,,about putting these \Ntwo things in opposition when -- Dialogue: 0,0:09:03.50,0:09:05.12,Default,,0000,0000,0000,,- [Guido] Yes.\N- [Josh] That language, of course, Dialogue: 0,0:09:05.12,0:09:08.37,Default,,0000,0000,0000,,comes from the simultaneous\Nequations framework Dialogue: 0,0:09:08.37,0:09:09.64,Default,,0000,0000,0000,,that we inherited. Dialogue: 0,0:09:10.40,0:09:11.44,Default,,0000,0000,0000,,It has the advantage Dialogue: 0,0:09:11.44,0:09:14.08,Default,,0000,0000,0000,,that people seem to know \Nwhat you mean when you use it, Dialogue: 0,0:09:14.08,0:09:15.16,Default,,0000,0000,0000,,but that might be Dialogue: 0,0:09:16.56,0:09:18.30,Default,,0000,0000,0000,,that different people\Nare hearing different things. Dialogue: 0,0:09:18.30,0:09:20.48,Default,,0000,0000,0000,,- [Guido] Yeah. I think\Nreduced form has become Dialogue: 0,0:09:20.48,0:09:22.20,Default,,0000,0000,0000,,used in a little bit \Nof the pejorative way... Dialogue: 0,0:09:22.20,0:09:23.54,Default,,0000,0000,0000,,- [Josh] Sometimes. Dialogue: 0,0:09:25.10,0:09:28.25,Default,,0000,0000,0000,,...which is not really quite what \Nit was originally intended for. Dialogue: 0,0:09:30.10,0:09:33.09,Default,,0000,0000,0000,,- [Isaiah] I guess something else\Nthat strikes me in thinking about Dialogue: 0,0:09:33.09,0:09:35.64,Default,,0000,0000,0000,,the effects of the local average\Ntreatment effect framework Dialogue: 0,0:09:35.64,0:09:37.68,Default,,0000,0000,0000,,is that often folks will appeal Dialogue: 0,0:09:37.68,0:09:40.00,Default,,0000,0000,0000,,to a local average treatment\Neffects intuition Dialogue: 0,0:09:40.00,0:09:42.36,Default,,0000,0000,0000,,for settings well beyond ones Dialogue: 0,0:09:42.36,0:09:44.96,Default,,0000,0000,0000,,where any sort of formal result\Nhas actually been established. Dialogue: 0,0:09:45.44,0:09:49.18,Default,,0000,0000,0000,,And I'm curious, given all the work\Nthat you guys did Dialogue: 0,0:09:49.18,0:09:52.39,Default,,0000,0000,0000,,to establish LATE results \Nin different settings, Dialogue: 0,0:09:52.39,0:09:54.42,Default,,0000,0000,0000,,I'm curious, any thoughts on that? Dialogue: 0,0:09:55.36,0:09:57.31,Default,,0000,0000,0000,,- I think there's going\Nto be a lot of cases Dialogue: 0,0:09:57.31,0:10:02.20,Default,,0000,0000,0000,,where the intuition \Ndoes get you some distance, Dialogue: 0,0:10:02.80,0:10:04.99,Default,,0000,0000,0000,,but it's going to be \Nsomewhat limited, Dialogue: 0,0:10:04.99,0:10:07.60,Default,,0000,0000,0000,,and establishing \Nformal results there Dialogue: 0,0:10:08.40,0:10:09.49,Default,,0000,0000,0000,,may be a little tricky Dialogue: 0,0:10:09.49,0:10:12.70,Default,,0000,0000,0000,,and then maybe only work \Nin special circumstances, Dialogue: 0,0:10:14.60,0:10:16.54,Default,,0000,0000,0000,,and you end up\Nwith a lot of formality Dialogue: 0,0:10:16.54,0:10:19.50,Default,,0000,0000,0000,,that may not quite \Ncapture the intuition. Dialogue: 0,0:10:19.90,0:10:21.55,Default,,0000,0000,0000,,Sometimes I'm somewhat \Nuneasy with them, Dialogue: 0,0:10:21.55,0:10:24.44,Default,,0000,0000,0000,,and they are not necessarily\Nthe papers I would want to write, Dialogue: 0,0:10:25.15,0:10:27.82,Default,,0000,0000,0000,,but I do think intuition Dialogue: 0,0:10:27.82,0:10:30.91,Default,,0000,0000,0000,,often does capture \Npart of the problem. Dialogue: 0,0:10:33.10,0:10:36.30,Default,,0000,0000,0000,,I think, in some sense,\Nwe were very fortunate there Dialogue: 0,0:10:36.90,0:10:39.25,Default,,0000,0000,0000,,in the way that the LATE paper \Ngot handled at the journal, Dialogue: 0,0:10:39.25,0:10:41.77,Default,,0000,0000,0000,,so that, actually, the editor,\Nmade it much shorter Dialogue: 0,0:10:42.10,0:10:46.30,Default,,0000,0000,0000,,and that allowed us to focus\Non very clear, crisp results. Dialogue: 0,0:10:49.92,0:10:51.77,Default,,0000,0000,0000,,There's a somewhat\Nunfortunate tendency Dialogue: 0,0:10:51.77,0:10:53.12,Default,,0000,0000,0000,,in the econometrics literature Dialogue: 0,0:10:53.12,0:10:54.87,Default,,0000,0000,0000,,of having the papers \Nget longer and longer. Dialogue: 0,0:10:54.87,0:10:56.55,Default,,0000,0000,0000,,- Well, you should be able\Nto fix that, man. Dialogue: 0,0:10:56.55,0:10:58.92,Default,,0000,0000,0000,,- I'm trying to fix that.\N[laughter] Dialogue: 0,0:10:58.92,0:11:01.51,Default,,0000,0000,0000,,But I think this is an example \Nwhere it's very clear Dialogue: 0,0:11:01.51,0:11:03.10,Default,,0000,0000,0000,,that having it be short\Nis actually -- Dialogue: 0,0:11:03.10,0:11:04.75,Default,,0000,0000,0000,,- You should have imposed\Nthat no paper Dialogue: 0,0:11:04.75,0:11:06.80,Default,,0000,0000,0000,,can be longer than the LATE paper. Dialogue: 0,0:11:07.27,0:11:09.62,Default,,0000,0000,0000,,- That... wow! That may be great. Dialogue: 0,0:11:09.62,0:11:11.68,Default,,0000,0000,0000,,- At least no theory, \Nno theory paper. Dialogue: 0,0:11:11.89,0:11:13.34,Default,,0000,0000,0000,,- Yeah, and I think... Dialogue: 0,0:11:14.50,0:11:16.80,Default,,0000,0000,0000,,I'm trying very hard to get\Nthe papers to be shorter, Dialogue: 0,0:11:16.80,0:11:19.51,Default,,0000,0000,0000,,and I think there is a lot\Nof value today Dialogue: 0,0:11:19.51,0:11:21.57,Default,,0000,0000,0000,,because it's often\Nthe second part of the paper Dialogue: 0,0:11:21.57,0:11:25.05,Default,,0000,0000,0000,,that doesn't actually\Nget you much further Dialogue: 0,0:11:25.05,0:11:26.40,Default,,0000,0000,0000,,in understanding things, Dialogue: 0,0:11:27.00,0:11:29.87,Default,,0000,0000,0000,,and it does make things \Nmuch harder to read. Dialogue: 0,0:11:32.43,0:11:36.18,Default,,0000,0000,0000,,It goes back to how I think\Neconometrics should be done. Dialogue: 0,0:11:36.18,0:11:38.07,Default,,0000,0000,0000,,You should focus on -- Dialogue: 0,0:11:38.70,0:11:41.30,Default,,0000,0000,0000,,It should be reasonably\Nclose to empirical problems. Dialogue: 0,0:11:41.50,0:11:43.90,Default,,0000,0000,0000,,They should be very clear problems. Dialogue: 0,0:11:44.80,0:11:48.90,Default,,0000,0000,0000,,But then often the theory\Ndoesn't need to be quite so long. Dialogue: 0,0:11:48.90,0:11:50.01,Default,,0000,0000,0000,,- [Josh] Yeah. Dialogue: 0,0:11:51.10,0:11:54.67,Default,,0000,0000,0000,,- I think things have gone\Na little off track. Dialogue: 0,0:11:56.26,0:11:57.75,Default,,0000,0000,0000,,- [Isaiah] A relatively \Nrecent change Dialogue: 0,0:11:57.75,0:12:00.23,Default,,0000,0000,0000,,has been a seeming\Nbig increase in demand Dialogue: 0,0:12:00.23,0:12:03.76,Default,,0000,0000,0000,,for people with econometrics\Ncausal effect estimation skills Dialogue: 0,0:12:03.76,0:12:04.99,Default,,0000,0000,0000,,in the tech sector. Dialogue: 0,0:12:04.99,0:12:07.56,Default,,0000,0000,0000,,I'm interested, \Ndo either of you have thoughts Dialogue: 0,0:12:07.56,0:12:09.84,Default,,0000,0000,0000,,of how that's going to interact Dialogue: 0,0:12:09.84,0:12:11.60,Default,,0000,0000,0000,,with the development \Nof empirical methods Dialogue: 0,0:12:11.60,0:12:13.95,Default,,0000,0000,0000,,or empirical research\Nin economics going forward? Dialogue: 0,0:12:14.60,0:12:16.77,Default,,0000,0000,0000,,- [Josh] Well, there's\Nsort of a meta point, Dialogue: 0,0:12:16.77,0:12:21.00,Default,,0000,0000,0000,,which is there's this new \Nkind of employer, Dialogue: 0,0:12:21.80,0:12:27.53,Default,,0000,0000,0000,,the Amazons and the Uber,\Nand the TripAdvisor world, Dialogue: 0,0:12:28.00,0:12:29.30,Default,,0000,0000,0000,,and I think that's great. Dialogue: 0,0:12:29.30,0:12:32.03,Default,,0000,0000,0000,,I like to tell my students\Nabout that. Dialogue: 0,0:12:32.60,0:12:35.50,Default,,0000,0000,0000,,At MIT, we have a lot\Nof computer science majors -- Dialogue: 0,0:12:35.50,0:12:37.00,Default,,0000,0000,0000,,that's our biggest major. Dialogue: 0,0:12:37.40,0:12:42.36,Default,,0000,0000,0000,,I try to seduce some of those folks\Ninto economics by saying Dialogue: 0,0:12:43.23,0:12:46.70,Default,,0000,0000,0000,,you can go work for these companies Dialogue: 0,0:12:46.70,0:12:49.25,Default,,0000,0000,0000,,that people\Nare very keen to work for Dialogue: 0,0:12:49.25,0:12:50.80,Default,,0000,0000,0000,,because the work seems exciting, Dialogue: 0,0:12:52.00,0:12:54.25,Default,,0000,0000,0000,,that the skills \Nthat you get in econometrics Dialogue: 0,0:12:54.25,0:12:56.10,Default,,0000,0000,0000,,are as good or better Dialogue: 0,0:12:56.10,0:12:59.74,Default,,0000,0000,0000,,than any competing \Ndiscipline has to offer. Dialogue: 0,0:12:59.74,0:13:02.14,Default,,0000,0000,0000,,So you should at least\Ntake some econ -- Dialogue: 0,0:13:02.14,0:13:04.29,Default,,0000,0000,0000,,take some econometrics\Nand some econ. Dialogue: 0,0:13:04.80,0:13:07.00,Default,,0000,0000,0000,,I did a fun project with Uber Dialogue: 0,0:13:07.60,0:13:09.50,Default,,0000,0000,0000,,on labor supply of Uber drivers, Dialogue: 0,0:13:09.92,0:13:12.80,Default,,0000,0000,0000,,and it was very, very exciting\Nto be part of that. Dialogue: 0,0:13:13.10,0:13:15.40,Default,,0000,0000,0000,,Plus, I got to drive \Nfor Uber for a while, Dialogue: 0,0:13:15.90,0:13:17.73,Default,,0000,0000,0000,,and I thought that was fun too. Dialogue: 0,0:13:17.73,0:13:19.23,Default,,0000,0000,0000,,I did not make enough Dialogue: 0,0:13:19.23,0:13:22.62,Default,,0000,0000,0000,,that I was tempted\Nto give up my MIT job, Dialogue: 0,0:13:22.62,0:13:25.10,Default,,0000,0000,0000,,but I enjoyed the experience. Dialogue: 0,0:13:25.23,0:13:27.53,Default,,0000,0000,0000,,I see a potential challenge Dialogue: 0,0:13:27.53,0:13:30.90,Default,,0000,0000,0000,,to our model\Nof graduate education here, Dialogue: 0,0:13:31.70,0:13:36.07,Default,,0000,0000,0000,,which is, if we're training people\Nto go work at Amazon, Dialogue: 0,0:13:37.90,0:13:41.19,Default,,0000,0000,0000,,it's not clear why\Nwe should be paying Dialogue: 0,0:13:41.19,0:13:42.90,Default,,0000,0000,0000,,graduate stipends for that. Dialogue: 0,0:13:43.20,0:13:47.28,Default,,0000,0000,0000,,Why should the taxpayer\Neffectively be subsidizing that. Dialogue: 0,0:13:47.28,0:13:51.18,Default,,0000,0000,0000,,Our graduate education\Nin the US is generously subsidized, Dialogue: 0,0:13:51.18,0:13:53.27,Default,,0000,0000,0000,,even in private universities -- Dialogue: 0,0:13:54.91,0:13:56.48,Default,,0000,0000,0000,,there's a lot\Nof public money there. Dialogue: 0,0:13:56.48,0:13:59.30,Default,,0000,0000,0000,,And I think the traditional\Nrationale for that is, Dialogue: 0,0:13:59.64,0:14:01.99,Default,,0000,0000,0000,,we were training \Neducators and scholars, Dialogue: 0,0:14:01.99,0:14:05.66,Default,,0000,0000,0000,,and there's a great externality\Nfrom the work that we do, Dialogue: 0,0:14:05.66,0:14:07.91,Default,,0000,0000,0000,,it's either \Nthe research externality, Dialogue: 0,0:14:07.91,0:14:09.56,Default,,0000,0000,0000,,or a teaching externality. Dialogue: 0,0:14:10.10,0:14:13.39,Default,,0000,0000,0000,,But if many of our students\Nare going to work Dialogue: 0,0:14:13.39,0:14:14.69,Default,,0000,0000,0000,,in the private sector -- Dialogue: 0,0:14:16.30,0:14:17.41,Default,,0000,0000,0000,,that's fine, Dialogue: 0,0:14:19.00,0:14:21.70,Default,,0000,0000,0000,,but maybe their employers\Nshould pay for them. Dialogue: 0,0:14:22.12,0:14:23.37,Default,,0000,0000,0000,,- For me, it's just so different Dialogue: 0,0:14:23.37,0:14:26.78,Default,,0000,0000,0000,,from people working \Nfor consulting firms. Dialogue: 0,0:14:27.20,0:14:28.78,Default,,0000,0000,0000,,It's not clear to me Dialogue: 0,0:14:28.78,0:14:32.84,Default,,0000,0000,0000,,that the number of jobs\Nin academics has changed. Dialogue: 0,0:14:33.37,0:14:36.32,Default,,0000,0000,0000,,- I feel like this is\Na growing sector, Dialogue: 0,0:14:36.32,0:14:37.81,Default,,0000,0000,0000,,whereas consulting... Dialogue: 0,0:14:37.81,0:14:39.29,Default,,0000,0000,0000,,You're right to raise that. Dialogue: 0,0:14:39.29,0:14:42.10,Default,,0000,0000,0000,,It might be the same \Nfor consulting. Dialogue: 0,0:14:44.85,0:14:47.50,Default,,0000,0000,0000,,I'm placing more and more \Nstudents in these businesses, Dialogue: 0,0:14:47.50,0:14:49.47,Default,,0000,0000,0000,,so it's on my mind, in a way, Dialogue: 0,0:14:49.47,0:14:54.21,Default,,0000,0000,0000,,that I've not been attentive \Nto consulting jobs. Dialogue: 0,0:14:54.21,0:14:56.92,Default,,0000,0000,0000,,Consulting was always important, Dialogue: 0,0:14:56.92,0:14:58.95,Default,,0000,0000,0000,,and I think also \Nthere's some movement Dialogue: 0,0:14:58.95,0:15:01.14,Default,,0000,0000,0000,,from consulting back\Ninto research -- Dialogue: 0,0:15:01.14,0:15:02.72,Default,,0000,0000,0000,,it's a little more fluid. Dialogue: 0,0:15:03.90,0:15:07.63,Default,,0000,0000,0000,,A lot of the work in both domains, Dialogue: 0,0:15:07.63,0:15:09.43,Default,,0000,0000,0000,,I have to say, \Nit's not really different, Dialogue: 0,0:15:09.43,0:15:12.73,Default,,0000,0000,0000,,but people who are working\Nin the tech sector Dialogue: 0,0:15:12.73,0:15:15.48,Default,,0000,0000,0000,,are doing things that are \Npotentially of scientific interest, Dialogue: 0,0:15:15.48,0:15:16.80,Default,,0000,0000,0000,,but mostly it's hidden. Dialogue: 0,0:15:17.10,0:15:18.55,Default,,0000,0000,0000,,Then you really have to say, Dialogue: 0,0:15:18.55,0:15:20.90,Default,,0000,0000,0000,,why is the government\Npaying for this? Dialogue: 0,0:15:22.10,0:15:23.73,Default,,0000,0000,0000,,- Although, to Guido's point, Dialogue: 0,0:15:23.73,0:15:26.78,Default,,0000,0000,0000,,I guess there's \Na data question here of Dialogue: 0,0:15:26.78,0:15:32.77,Default,,0000,0000,0000,,has the total [non-EC]\Nfor-profit sector employment Dialogue: 0,0:15:32.77,0:15:35.60,Default,,0000,0000,0000,,of econ Ph.D. program\Ngraduates increased Dialogue: 0,0:15:35.60,0:15:38.29,Default,,0000,0000,0000,,or has it just been\Na substitution from finance Dialogue: 0,0:15:38.29,0:15:40.30,Default,,0000,0000,0000,,and consulting towards tech? Dialogue: 0,0:15:40.30,0:15:42.30,Default,,0000,0000,0000,,- I may be reacting to something Dialogue: 0,0:15:42.30,0:15:44.40,Default,,0000,0000,0000,,that's not really happening. Dialogue: 0,0:15:44.40,0:15:45.89,Default,,0000,0000,0000,,- I've actually done some work Dialogue: 0,0:15:45.89,0:15:48.20,Default,,0000,0000,0000,,with some of these tech companies. Dialogue: 0,0:15:49.10,0:15:52.20,Default,,0000,0000,0000,,I don't disagree with Josh's point \Nthat we need to think Dialogue: 0,0:15:52.20,0:15:53.68,Default,,0000,0000,0000,,a little bit about \Nthe funding model Dialogue: 0,0:15:53.68,0:15:56.39,Default,,0000,0000,0000,,who is, in the end, paying \Nfor the graduate education. Dialogue: 0,0:15:56.91,0:15:59.40,Default,,0000,0000,0000,,But from a scientific perspective, Dialogue: 0,0:15:59.98,0:16:02.84,Default,,0000,0000,0000,,not only do these places\Nhave great data, Dialogue: 0,0:16:02.84,0:16:05.11,Default,,0000,0000,0000,,and nowadays they tend to be\Nvery careful with that Dialogue: 0,0:16:05.11,0:16:07.10,Default,,0000,0000,0000,,for privacy reasons, Dialogue: 0,0:16:07.38,0:16:08.90,Default,,0000,0000,0000,,but they also have great questions. Dialogue: 0,0:16:10.20,0:16:13.93,Default,,0000,0000,0000,,I find it very inspiring to listen\Nto the people there Dialogue: 0,0:16:13.93,0:16:15.95,Default,,0000,0000,0000,,and see what kind\Nof questions they have, Dialogue: 0,0:16:15.95,0:16:17.33,Default,,0000,0000,0000,,and often they're questions Dialogue: 0,0:16:18.20,0:16:21.24,Default,,0000,0000,0000,,that also come up\Noutside of these companies. Dialogue: 0,0:16:21.24,0:16:27.43,Default,,0000,0000,0000,,I have a couple of papers \Nwith Raj Chetty and Susan Athey Dialogue: 0,0:16:27.43,0:16:31.60,Default,,0000,0000,0000,,where we look at ways \Nof combining experimental data Dialogue: 0,0:16:31.60,0:16:33.27,Default,,0000,0000,0000,,and observational data. Dialogue: 0,0:16:35.50,0:16:38.60,Default,,0000,0000,0000,,Raj Chetty was interested\Nin what is the effect Dialogue: 0,0:16:38.60,0:16:42.89,Default,,0000,0000,0000,,of early childhood programs \Non outcomes later in life, Dialogue: 0,0:16:42.89,0:16:46.33,Default,,0000,0000,0000,,not just on test scores\Nbut on earnings and stuff, Dialogue: 0,0:16:46.33,0:16:48.30,Default,,0000,0000,0000,,and we kind of developed methods Dialogue: 0,0:16:48.60,0:16:51.50,Default,,0000,0000,0000,,that would help you shed light\Non that under some... Dialogue: 0,0:16:52.72,0:16:53.87,Default,,0000,0000,0000,,in some settings, Dialogue: 0,0:16:53.87,0:16:56.92,Default,,0000,0000,0000,,and the same problems came up Dialogue: 0,0:16:56.92,0:17:00.53,Default,,0000,0000,0000,,in these tech company settings. Dialogue: 0,0:17:00.80,0:17:03.24,Default,,0000,0000,0000,,And so, from my perspective, Dialogue: 0,0:17:03.24,0:17:05.42,Default,,0000,0000,0000,,it's the same kind of -- Dialogue: 0,0:17:05.42,0:17:07.60,Default,,0000,0000,0000,,I was talking to people \Ndoing empirical work. Dialogue: 0,0:17:07.60,0:17:09.70,Default,,0000,0000,0000,,I tried to kind of look\Nat these specific problems Dialogue: 0,0:17:09.70,0:17:13.37,Default,,0000,0000,0000,,and then try to come up\Nwith more general problems, Dialogue: 0,0:17:15.11,0:17:18.23,Default,,0000,0000,0000,,reformulating the problems\Nat a higher level Dialogue: 0,0:17:18.50,0:17:22.90,Default,,0000,0000,0000,,so that I can think about solutions\Nthat work in a range of settings. Dialogue: 0,0:17:23.40,0:17:24.93,Default,,0000,0000,0000,,And so from that perspective, Dialogue: 0,0:17:24.93,0:17:27.57,Default,,0000,0000,0000,,the interactions \Nwith the tech companies Dialogue: 0,0:17:27.57,0:17:30.30,Default,,0000,0000,0000,,are just very valuable\Nand very useful. Dialogue: 0,0:17:31.70,0:17:35.20,Default,,0000,0000,0000,,We do have students now\Ndoing internships there Dialogue: 0,0:17:35.20,0:17:38.52,Default,,0000,0000,0000,,and then coming back \Nand writing more interesting theses Dialogue: 0,0:17:38.52,0:17:43.40,Default,,0000,0000,0000,,as a result of\Ntheir experiences there. Dialogue: 0,0:17:44.60,0:17:47.02,Default,,0000,0000,0000,,- [Narrator] If you'd like to watch\Nmore Nobel Conversations, Dialogue: 0,0:17:47.02,0:17:48.20,Default,,0000,0000,0000,,click here. Dialogue: 0,0:17:48.20,0:17:50.50,Default,,0000,0000,0000,,Or if you'd like to learn\Nmore about econometrics, Dialogue: 0,0:17:50.50,0:17:53.10,Default,,0000,0000,0000,,check out Josh's \N"Mastering Econometrics" series. Dialogue: 0,0:17:53.70,0:17:56.72,Default,,0000,0000,0000,,If you'd like to learn more \Nabout Guido, Josh, and Isaiah, Dialogue: 0,0:17:56.72,0:17:58.30,Default,,0000,0000,0000,,check out the links \Nin the description. Dialogue: 0,0:17:59.04,0:18:01.50,Default,,0000,0000,0000,,♪ [music] ♪