[Script Info]
Title: 
[Events]
Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
Dialogue: 0,0:00:00.10,0:00:02.35,Default,,0000,0000,0000,,♪ [music] ♪
Dialogue: 0,0:00:03.70,0:00:05.70,Default,,0000,0000,0000,,- [narrator] Welcome\Nto Nobel Conversations.
Dialogue: 0,0:00:07.00,0:00:10.13,Default,,0000,0000,0000,,In this episode, Josh Angrist\Nand Guido Imbens
Dialogue: 0,0:00:10.13,0:00:13.70,Default,,0000,0000,0000,,sit down with Isaiah Andrews\Nto discuss and disagree
Dialogue: 0,0:00:13.70,0:00:16.58,Default,,0000,0000,0000,,over the role of machine learning\Nin applied econometrics.
Dialogue: 0,0:00:18.30,0:00:19.77,Default,,0000,0000,0000,,- [Isaiah] So, of course,\Nthere are a lot of topics
Dialogue: 0,0:00:19.77,0:00:21.09,Default,,0000,0000,0000,,where you guys largely agree,
Dialogue: 0,0:00:21.09,0:00:22.31,Default,,0000,0000,0000,,but I'd like to turn to one
Dialogue: 0,0:00:22.31,0:00:24.24,Default,,0000,0000,0000,,where maybe you have\Nsome differences of opinion.
Dialogue: 0,0:00:24.24,0:00:25.73,Default,,0000,0000,0000,,So I'd love to hear\Nsome of your thoughts
Dialogue: 0,0:00:25.73,0:00:26.88,Default,,0000,0000,0000,,about machine learning
Dialogue: 0,0:00:26.88,0:00:29.90,Default,,0000,0000,0000,,and the goal that it's playing\Nand is going to play in economics.
Dialogue: 0,0:00:30.20,0:00:33.35,Default,,0000,0000,0000,,- [Guido] I've looked at some data\Nlike the proprietary
Dialogue: 0,0:00:33.35,0:00:35.10,Default,,0000,0000,0000,,so that there's\Nno published paper there.
Dialogue: 0,0:00:36.72,0:00:38.16,Default,,0000,0000,0000,,There was an experiment\Nthat was done
Dialogue: 0,0:00:38.16,0:00:39.50,Default,,0000,0000,0000,,on some search algorithm.
Dialogue: 0,0:00:39.70,0:00:41.50,Default,,0000,0000,0000,,And the question was...
Dialogue: 0,0:00:42.90,0:00:45.60,Default,,0000,0000,0000,,it was about ranking things\Nand changing the ranking.
Dialogue: 0,0:00:45.90,0:00:47.50,Default,,0000,0000,0000,,That was sort of clear...
Dialogue: 0,0:00:48.40,0:00:50.60,Default,,0000,0000,0000,,that was going to be\Na lot of heterogeneity there.
Dialogue: 0,0:00:50.60,0:00:51.70,Default,,0000,0000,0000,,Mmm,
Dialogue: 0,0:00:51.70,0:00:58.12,Default,,0000,0000,0000,,You know, if you look for say,
Dialogue: 0,0:00:58.30,0:01:00.35,Default,,0000,0000,0000,,a picture of Britney Spears
Dialogue: 0,0:01:00.35,0:01:02.40,Default,,0000,0000,0000,,that it doesn't really matter\Nwhere you rank it
Dialogue: 0,0:01:02.40,0:01:05.50,Default,,0000,0000,0000,,because you're going to figure out\Nwhat you're looking for,
Dialogue: 0,0:01:06.20,0:01:07.87,Default,,0000,0000,0000,,whether you put it\Nin the first or second
Dialogue: 0,0:01:07.87,0:01:09.80,Default,,0000,0000,0000,,or third position of the ranking.
Dialogue: 0,0:01:10.10,0:01:12.50,Default,,0000,0000,0000,,But if you're looking\Nfor the best econometrics book,
Dialogue: 0,0:01:13.30,0:01:16.50,Default,,0000,0000,0000,,if you put your book\Nfirst or your book tenth,
Dialogue: 0,0:01:16.50,0:01:18.10,Default,,0000,0000,0000,,that's going to make\Na big difference
Dialogue: 0,0:01:18.60,0:01:21.83,Default,,0000,0000,0000,,how much how often people\Nare going to click on it.
Dialogue: 0,0:01:21.83,0:01:23.42,Default,,0000,0000,0000,,And so there you go --
Dialogue: 0,0:01:23.42,0:01:27.22,Default,,0000,0000,0000,,- [Josh] Why do I need\Nmachine learning to discover that?
Dialogue: 0,0:01:27.22,0:01:29.20,Default,,0000,0000,0000,,It seems like could\NI can discover it simply?
Dialogue: 0,0:01:29.20,0:01:30.44,Default,,0000,0000,0000,,- [Guido] So in general--
Dialogue: 0,0:01:30.44,0:01:32.10,Default,,0000,0000,0000,,- [Josh] There were lots\Nof possible...
Dialogue: 0,0:01:32.10,0:01:35.49,Default,,0000,0000,0000,,- You what you want to think about\Nthere being lots of characteristics
Dialogue: 0,0:01:35.49,0:01:37.61,Default,,0000,0000,0000,,of the items
Dialogue: 0,0:01:37.61,0:01:41.68,Default,,0000,0000,0000,,that you want to understand\Nwhat drives the heterogeneity
Dialogue: 0,0:01:42.30,0:01:43.43,Default,,0000,0000,0000,,in the effect of--
Dialogue: 0,0:01:43.43,0:01:45.60,Default,,0000,0000,0000,,- But you're just predicting
Dialogue: 0,0:01:45.60,0:01:47.70,Default,,0000,0000,0000,,In some sense, you're solving\Na marketing problem.
Dialogue: 0,0:01:48.40,0:01:49.58,Default,,0000,0000,0000,,- [inaudible] it's causal effect,
Dialogue: 0,0:01:49.58,0:01:51.80,Default,,0000,0000,0000,,- It's causal, but it has\Nno scientific content.
Dialogue: 0,0:01:51.80,0:01:53.30,Default,,0000,0000,0000,,Think about...
Dialogue: 0,0:01:54.10,0:01:57.30,Default,,0000,0000,0000,,- No, but it's similar things\Nin medical settings.
Dialogue: 0,0:01:58.00,0:02:01.30,Default,,0000,0000,0000,,If you do an experiment, \Nyou may actually be very interested
Dialogue: 0,0:02:01.30,0:02:03.90,Default,,0000,0000,0000,,in whether the treatment\Nworks for some groups or not.
Dialogue: 0,0:02:03.90,0:02:06.50,Default,,0000,0000,0000,,And you have a lot of individual\Ncharacteristics,
Dialogue: 0,0:02:06.50,0:02:08.00,Default,,0000,0000,0000,,and you want\Nto systematically search.
Dialogue: 0,0:02:08.00,0:02:09.50,Default,,0000,0000,0000,,- Yeah. I'm skeptical about that --
Dialogue: 0,0:02:09.50,0:02:12.60,Default,,0000,0000,0000,,that sort of idea that there's\Nthis personal causal effect
Dialogue: 0,0:02:12.60,0:02:13.90,Default,,0000,0000,0000,,that I should care about,
Dialogue: 0,0:02:14.00,0:02:16.06,Default,,0000,0000,0000,,and that machine learning\Ncan discover it
Dialogue: 0,0:02:16.06,0:02:17.60,Default,,0000,0000,0000,,in some way that's useful.
Dialogue: 0,0:02:17.60,0:02:21.40,Default,,0000,0000,0000,,So think about -- I've done\Na lot of work on schools,
Dialogue: 0,0:02:21.40,0:02:23.95,Default,,0000,0000,0000,,going to, say, a charter school,
Dialogue: 0,0:02:23.95,0:02:25.22,Default,,0000,0000,0000,,a publicly funded private school,
Dialogue: 0,0:02:25.22,0:02:26.50,Default,,0000,0000,0000,,effectively, you know,\Nthat's free to structure
Dialogue: 0,0:02:26.50,0:02:29.30,Default,,0000,0000,0000,,its own curriculum\Nfor context there.
Dialogue: 0,0:02:29.30,0:02:31.00,Default,,0000,0000,0000,,Some types of charter schools
Dialogue: 0,0:02:31.00,0:02:32.70,Default,,0000,0000,0000,,generate spectacular\Nachievement gains,
Dialogue: 0,0:02:32.70,0:02:36.40,Default,,0000,0000,0000,,and in the data set\Nthat produces that result,
Dialogue: 0,0:02:36.40,0:02:37.80,Default,,0000,0000,0000,,I have a lot of covariance.
Dialogue: 0,0:02:37.80,0:02:41.35,Default,,0000,0000,0000,,So I have baseline scores,\Nand I have family background,
Dialogue: 0,0:02:41.35,0:02:43.58,Default,,0000,0000,0000,,the education of the parents,
Dialogue: 0,0:02:43.58,0:02:45.80,Default,,0000,0000,0000,,the sex of the child, \Nthe race of the child.
Dialogue: 0,0:02:45.80,0:02:48.30,Default,,0000,0000,0000,,And, well, soon as I put\Nhalf a dozen of those together,
Dialogue: 0,0:02:48.40,0:02:51.90,Default,,0000,0000,0000,,I have a very high dimensional space.
Dialogue: 0,0:02:52.30,0:02:53.60,Default,,0000,0000,0000,,I'm definitely interested\Nin sort of coarse features
Dialogue: 0,0:02:53.60,0:02:54.90,Default,,0000,0000,0000,,of that treatment effect,
Dialogue: 0,0:02:54.90,0:02:57.15,Default,,0000,0000,0000,,like whether it's better for people
Dialogue: 0,0:02:57.15,0:02:59.40,Default,,0000,0000,0000,,who come from\Nlower income families.
Dialogue: 0,0:03:02.60,0:03:06.00,Default,,0000,0000,0000,,I have a hard time believing\Nthat there's an application,
Dialogue: 0,0:03:06.40,0:03:10.30,Default,,0000,0000,0000,,for the very high dimensional\Nversion of that,
Dialogue: 0,0:03:10.50,0:03:11.85,Default,,0000,0000,0000,,where I discovered\Nthat for non-white children
Dialogue: 0,0:03:11.85,0:03:13.20,Default,,0000,0000,0000,,who have high family incomes
Dialogue: 0,0:03:13.80,0:03:17.80,Default,,0000,0000,0000,,but baseline scores\Nin the third quartile
Dialogue: 0,0:03:18.30,0:03:20.65,Default,,0000,0000,0000,,and only went to public school\Nin the third grade
Dialogue: 0,0:03:20.65,0:03:23.00,Default,,0000,0000,0000,,but not the sixth grade.
Dialogue: 0,0:03:23.00,0:03:25.50,Default,,0000,0000,0000,,So that's what that high\Ndimensional analysis produces.
Dialogue: 0,0:03:25.80,0:03:28.10,Default,,0000,0000,0000,,This very elaborate\Nconditional statement.
Dialogue: 0,0:03:28.30,0:03:31.00,Default,,0000,0000,0000,,There's two things that are wrong\Nwith that in my view.
Dialogue: 0,0:03:31.00,0:03:32.50,Default,,0000,0000,0000,,First, I don't see it as...
Dialogue: 0,0:03:32.50,0:03:34.00,Default,,0000,0000,0000,,I just can't imagine\Nwhy it's actionable.
Dialogue: 0,0:03:34.60,0:03:36.60,Default,,0000,0000,0000,,I don't know why\Nyou'd want to act on it.
Dialogue: 0,0:03:36.60,0:03:38.90,Default,,0000,0000,0000,,And I know also\Nthat there's some alternative model
Dialogue: 0,0:03:38.90,0:03:41.20,Default,,0000,0000,0000,,that fits almost as well,
Dialogue: 0,0:03:41.80,0:03:43.00,Default,,0000,0000,0000,,that flips everything,
Dialogue: 0,0:03:43.20,0:03:45.35,Default,,0000,0000,0000,,Because machine learning\Ndoesn't tell me
Dialogue: 0,0:03:45.35,0:03:47.50,Default,,0000,0000,0000,,that this is really\Nthe predictor that matters.
Dialogue: 0,0:03:48.40,0:03:52.30,Default,,0000,0000,0000,,It just tells me that\Nthis is a good predictor.
Dialogue: 0,0:03:52.80,0:03:54.35,Default,,0000,0000,0000,,And so, I think\Nthere is something different
Dialogue: 0,0:03:54.35,0:03:55.90,Default,,0000,0000,0000,,about the social science contest.
Dialogue: 0,0:03:57.94,0:03:59.54,Default,,0000,0000,0000,,- [Guido] I think\Nthe [socialized sign] applications
Dialogue: 0,0:03:59.54,0:04:01.15,Default,,0000,0000,0000,,you're talking about,
Dialogue: 0,0:04:01.15,0:04:02.60,Default,,0000,0000,0000,,once were...
Dialogue: 0,0:04:03.40,0:04:08.10,Default,,0000,0000,0000,,I think there's not a huge amount\Nof heterogeneity in the effects.
Dialogue: 0,0:04:08.40,0:04:11.20,Default,,0000,0000,0000,,- [Josh] There might be
Dialogue: 0,0:04:11.20,0:04:14.00,Default,,0000,0000,0000,,if you allow me\Nto to fill that space.
Dialogue: 0,0:04:14.60,0:04:16.35,Default,,0000,0000,0000,,- No... not even then.
Dialogue: 0,0:04:16.35,0:04:18.10,Default,,0000,0000,0000,,I think for a lot\Nof those interventions,
Dialogue: 0,0:04:18.30,0:04:22.00,Default,,0000,0000,0000,,you would expect that the effect\Nis the same sign for everybody.
Dialogue: 0,0:04:23.40,0:04:27.60,Default,,0000,0000,0000,,There may be small differences\Nin the magnitude, but it's not...
Dialogue: 0,0:04:28.20,0:04:31.70,Default,,0000,0000,0000,,For a lot of these education\Ndefenses -- they're good for everybody.
Dialogue: 0,0:04:32.90,0:04:35.25,Default,,0000,0000,0000,,It's not that they're bad\Nfor some people
Dialogue: 0,0:04:35.25,0:04:37.60,Default,,0000,0000,0000,,and good for other people,
Dialogue: 0,0:04:37.60,0:04:39.20,Default,,0000,0000,0000,,and that is kind\Nof very small pockets
Dialogue: 0,0:04:39.20,0:04:40.80,Default,,0000,0000,0000,,where they're bad there.
Dialogue: 0,0:04:40.90,0:04:43.90,Default,,0000,0000,0000,,But it may be some variation\Nin the magnitude,
Dialogue: 0,0:04:44.00,0:04:48.20,Default,,0000,0000,0000,,but you would need very, \Nvery big data sets to find those.
Dialogue: 0,0:04:48.40,0:04:49.90,Default,,0000,0000,0000,,I agree that in those cases,
Dialogue: 0,0:04:49.90,0:04:51.40,Default,,0000,0000,0000,,they probably wouldn't be\Nvery actionable anyone.
Dialogue: 0,0:04:51.70,0:04:53.80,Default,,0000,0000,0000,,But I think there's a lot\Nof other settings
Dialogue: 0,0:04:54.10,0:04:56.60,Default,,0000,0000,0000,,where there is\Nmuch more heterogeneity.
Dialogue: 0,0:04:57.40,0:04:59.50,Default,,0000,0000,0000,,- Well, I'm open\Nto that possibility,
Dialogue: 0,0:04:59.50,0:05:05.55,Default,,0000,0000,0000,,and I think the example you gave\Nis essentially a marketing example.
Dialogue: 0,0:05:06.43,0:05:10.70,Default,,0000,0000,0000,,- No, those have implications for it\Nand that's the organization,
Dialogue: 0,0:05:10.70,0:05:13.90,Default,,0000,0000,0000,,whether you need\Nto worry about the...
Dialogue: 0,0:05:14.00,0:05:17.90,Default,,0000,0000,0000,,- Well, I need to see that paper.
Dialogue: 0,0:05:18.40,0:05:21.20,Default,,0000,0000,0000,,- So the sense I'm getting...
Dialogue: 0,0:05:21.50,0:05:23.10,Default,,0000,0000,0000,,- We still disagree on something.\N- Yes.
Dialogue: 0,0:05:23.10,0:05:24.10,Default,,0000,0000,0000,,[laughter]
Dialogue: 0,0:05:24.10,0:05:25.40,Default,,0000,0000,0000,,- We haven't converged\Non everything.
Dialogue: 0,0:05:25.40,0:05:26.05,Default,,0000,0000,0000,,- I'm getting that sense.
Dialogue: 0,0:05:26.05,0:05:26.70,Default,,0000,0000,0000,,[laughter]
Dialogue: 0,0:05:27.20,0:05:29.10,Default,,0000,0000,0000,,- Actually, we've diverged on this
Dialogue: 0,0:05:29.10,0:05:30.05,Default,,0000,0000,0000,,because this wasn't around\Nto argue about.
Dialogue: 0,0:05:30.05,0:05:31.00,Default,,0000,0000,0000,,[laughter]
Dialogue: 0,0:05:33.20,0:05:35.60,Default,,0000,0000,0000,,- Is it getting a little warm here?
Dialogue: 0,0:05:35.60,0:05:38.00,Default,,0000,0000,0000,,- Warmed up. Warmed up is good.
Dialogue: 0,0:05:38.10,0:05:40.80,Default,,0000,0000,0000,,The sense I'm getting is, Josh,\Nyou're not saying
Dialogue: 0,0:05:40.90,0:05:43.40,Default,,0000,0000,0000,,that you're confident\Nthat there is no way
Dialogue: 0,0:05:43.40,0:05:45.40,Default,,0000,0000,0000,,that there is an application\Nwhere the stuff.
Dialogue: 0,0:05:45.40,0:05:46.80,Default,,0000,0000,0000,,It's useful you are saying
Dialogue: 0,0:05:46.80,0:05:48.20,Default,,0000,0000,0000,,you are unconvinced by\Nthe existing application to date.
Dialogue: 0,0:05:48.30,0:05:51.28,Default,,0000,0000,0000,,Fair enough.
Dialogue: 0,0:05:51.28,0:05:53.12,Default,,0000,0000,0000,,- I'm very confident.
Dialogue: 0,0:05:53.12,0:05:54.30,Default,,0000,0000,0000,,[laughter]
Dialogue: 0,0:05:54.30,0:05:55.30,Default,,0000,0000,0000,,- In this case.
Dialogue: 0,0:05:55.30,0:05:57.50,Default,,0000,0000,0000,,- I think Josh does have a point
Dialogue: 0,0:05:58.00,0:06:02.10,Default,,0000,0000,0000,,that even in the prediction cases
Dialogue: 0,0:06:02.30,0:06:05.00,Default,,0000,0000,0000,,where a lot of the machine learning\Nmethods really shine
Dialogue: 0,0:06:05.00,0:06:06.60,Default,,0000,0000,0000,,is where there's just a lot\Nof heterogeneity.
Dialogue: 0,0:06:07.30,0:06:10.60,Default,,0000,0000,0000,,- You don't really care much\Nabout the details there, right?
Dialogue: 0,0:06:10.90,0:06:15.00,Default,,0000,0000,0000,,It doesn't have\Na policy angle or something.
Dialogue: 0,0:06:15.20,0:06:18.10,Default,,0000,0000,0000,,- They kind of recognizing\Nhandwritten digits and stuff.
Dialogue: 0,0:06:18.30,0:06:21.15,Default,,0000,0000,0000,,It does much better there
Dialogue: 0,0:06:21.15,0:06:24.00,Default,,0000,0000,0000,,than building\Nsome complicated model.
Dialogue: 0,0:06:24.40,0:06:28.10,Default,,0000,0000,0000,,But a lot of the social science,\Na lot of the economic applications,
Dialogue: 0,0:06:28.30,0:06:30.20,Default,,0000,0000,0000,,we actually know a huge amount\Nabout the relationship
Dialogue: 0,0:06:30.20,0:06:32.10,Default,,0000,0000,0000,,between its variables.
Dialogue: 0,0:06:32.10,0:06:34.60,Default,,0000,0000,0000,,A lot of the relationships\Nare strictly monotone.
Dialogue: 0,0:06:35.40,0:06:39.40,Default,,0000,0000,0000,,Education is going to increase\Npeople's earnings,
Dialogue: 0,0:06:39.80,0:06:41.95,Default,,0000,0000,0000,,irrespective of the demographic,
Dialogue: 0,0:06:41.95,0:06:44.10,Default,,0000,0000,0000,,irrespective of the level\Nof education you already have.
Dialogue: 0,0:06:44.10,0:06:45.95,Default,,0000,0000,0000,,- Until they get to a Ph.D.
Dialogue: 0,0:06:45.95,0:06:47.80,Default,,0000,0000,0000,,- Yeah, there is a graduate school...
Dialogue: 0,0:06:48.15,0:06:49.15,Default,,0000,0000,0000,,[laughter]
Dialogue: 0,0:06:49.50,0:06:50.70,Default,,0000,0000,0000,,but go over a reasonable range.
Dialogue: 0,0:06:51.60,0:06:55.90,Default,,0000,0000,0000,,It's not going\Nto go down very much.
Dialogue: 0,0:06:56.10,0:06:57.90,Default,,0000,0000,0000,,In a lot of the settings
Dialogue: 0,0:06:57.90,0:06:59.70,Default,,0000,0000,0000,,where these machine learning\Nmethods shine,
Dialogue: 0,0:06:59.70,0:07:01.90,Default,,0000,0000,0000,,there's a lot of [ ]
Dialogue: 0,0:07:02.10,0:07:04.90,Default,,0000,0000,0000,,kind of multimodality\Nin these relationships,
Dialogue: 0,0:07:05.30,0:07:08.40,Default,,0000,0000,0000,,and they're going to be\Nvery powerful.
Dialogue: 0,0:07:08.40,0:07:11.50,Default,,0000,0000,0000,,But I still stand by that.
Dialogue: 0,0:07:11.70,0:07:16.10,Default,,0000,0000,0000,,These methods just have\Na huge amount to offer
Dialogue: 0,0:07:16.40,0:07:18.10,Default,,0000,0000,0000,,for economists,
Dialogue: 0,0:07:18.20,0:07:21.70,Default,,0000,0000,0000,,and they're going to be\Na big part of the future.
Dialogue: 0,0:07:23.40,0:07:24.60,Default,,0000,0000,0000,,- [Isaiah] Feels like\Nthere's something interesting
Dialogue: 0,0:07:24.60,0:07:25.80,Default,,0000,0000,0000,,to be said about\Nmachine learning here.
Dialogue: 0,0:07:25.80,0:07:27.70,Default,,0000,0000,0000,,So, Guido, I was wondering,\Ncould you give some more...
Dialogue: 0,0:07:28.00,0:07:29.00,Default,,0000,0000,0000,,maybe some examples\Nof the sorts of examples
Dialogue: 0,0:07:29.00,0:07:32.50,Default,,0000,0000,0000,,you're thinking about\Nwith applications [ ] at the moment?
Dialogue: 0,0:07:32.50,0:07:34.10,Default,,0000,0000,0000,,- So on areas where
Dialogue: 0,0:07:34.70,0:07:36.40,Default,,0000,0000,0000,,instead of looking\Nfor average cause or effects
Dialogue: 0,0:07:36.50,0:07:39.35,Default,,0000,0000,0000,,we're looking for\Nindividualized estimates,
Dialogue: 0,0:07:39.35,0:07:42.20,Default,,0000,0000,0000,,predictions of cause or effects
Dialogue: 0,0:07:42.40,0:07:44.95,Default,,0000,0000,0000,,and the machine learning algorithms\Nhave been very effective,
Dialogue: 0,0:07:48.30,0:07:51.50,Default,,0000,0000,0000,,Traditionally, we would have done\Nthese things using kernel methods.
Dialogue: 0,0:07:51.60,0:07:54.50,Default,,0000,0000,0000,,And theoretically they work great,
Dialogue: 0,0:07:54.60,0:07:56.00,Default,,0000,0000,0000,,and there's some arguments
Dialogue: 0,0:07:56.00,0:07:57.40,Default,,0000,0000,0000,,that, formally, \Nyou can't do any better.
Dialogue: 0,0:07:57.60,0:08:00.50,Default,,0000,0000,0000,,But in practice, \Nthey don't work very well.
Dialogue: 0,0:08:00.90,0:08:03.15,Default,,0000,0000,0000,,Random causal forest-type things
Dialogue: 0,0:08:03.15,0:08:05.40,Default,,0000,0000,0000,,that Stefan Wager and Susan Athey\Nhave been working on
Dialogue: 0,0:08:05.40,0:08:09.50,Default,,0000,0000,0000,,have used very widely.
Dialogue: 0,0:08:09.60,0:08:12.20,Default,,0000,0000,0000,,They've been very effective\Nin these settings
Dialogue: 0,0:08:12.40,0:08:18.10,Default,,0000,0000,0000,,to actually get causal effects\Nthat vary be [ ].
Dialogue: 0,0:08:20.70,0:08:23.20,Default,,0000,0000,0000,,I think this is still just the beginning\Nof these methods.
Dialogue: 0,0:08:23.20,0:08:25.70,Default,,0000,0000,0000,,But in many cases,
Dialogue: 0,0:08:26.40,0:08:31.60,Default,,0000,0000,0000,,these algorithms are very effective\Nas searching over big spaces
Dialogue: 0,0:08:31.80,0:08:35.60,Default,,0000,0000,0000,,and finding the functions that fit very well
Dialogue: 0,0:08:35.90,0:08:41.10,Default,,0000,0000,0000,,in ways that we couldn't\Nreally do beforehand.
Dialogue: 0,0:08:41.50,0:08:43.40,Default,,0000,0000,0000,,- I don't know of an example
Dialogue: 0,0:08:43.40,0:08:45.30,Default,,0000,0000,0000,,where machine learning\Nhas generated insights
Dialogue: 0,0:08:45.30,0:08:48.10,Default,,0000,0000,0000,,about a causal effect\Nthat I'm interested in.
Dialogue: 0,0:08:48.30,0:08:49.80,Default,,0000,0000,0000,,And I do know of examples
Dialogue: 0,0:08:49.80,0:08:51.30,Default,,0000,0000,0000,,where it's potentially\Nvery misleading.
Dialogue: 0,0:08:51.30,0:08:53.70,Default,,0000,0000,0000,,So I've done some work\Nwith Brigham Frandsen,
Dialogue: 0,0:08:54.10,0:08:55.10,Default,,0000,0000,0000,,using, for example, random forest\Nto model covariate effects
Dialogue: 0,0:08:55.10,0:08:59.90,Default,,0000,0000,0000,,in an instrumental\Nvariables problem
Dialogue: 0,0:09:00.20,0:09:01.20,Default,,0000,0000,0000,,Where you need you need\Nto condition on covariance.
Dialogue: 0,0:09:04.40,0:09:06.30,Default,,0000,0000,0000,,And you don't particularly\Nhave strong feelings
Dialogue: 0,0:09:06.30,0:09:08.20,Default,,0000,0000,0000,,about the functional form for that,
Dialogue: 0,0:09:08.20,0:09:10.00,Default,,0000,0000,0000,,so maybe you should curve...
Dialogue: 0,0:09:10.90,0:09:12.70,Default,,0000,0000,0000,,be open to flexible curve fitting,
Dialogue: 0,0:09:12.70,0:09:14.50,Default,,0000,0000,0000,,and that leads you down a path
Dialogue: 0,0:09:14.50,0:09:18.00,Default,,0000,0000,0000,,where there's a lot\Nof nonlinearities in the model,
Dialogue: 0,0:09:18.20,0:09:20.60,Default,,0000,0000,0000,,and that's very dangerous with IV
Dialogue: 0,0:09:20.60,0:09:23.00,Default,,0000,0000,0000,,because any sort\Nof excluded non-linearity
Dialogue: 0,0:09:23.30,0:09:25.45,Default,,0000,0000,0000,,potentially generates\Na spurious causal effect
Dialogue: 0,0:09:25.45,0:09:27.60,Default,,0000,0000,0000,,and Brigham and I\Nshowed that very powerfully.
Dialogue: 0,0:09:27.90,0:09:32.20,Default,,0000,0000,0000,,I think in the case\Nof two instruments
Dialogue: 0,0:09:32.70,0:09:36.00,Default,,0000,0000,0000,,that come from a paper of mine\Nwith Bill Evans,
Dialogue: 0,0:09:36.50,0:09:37.60,Default,,0000,0000,0000,,where if you replace it
Dialogue: 0,0:09:38.10,0:09:40.35,Default,,0000,0000,0000,,a traditional two stage \N[ ] squares estimator
Dialogue: 0,0:09:40.35,0:09:42.60,Default,,0000,0000,0000,,with some kind of random forest,
Dialogue: 0,0:09:42.90,0:09:48.00,Default,,0000,0000,0000,,you get very precisely\Nestimated [non-sense] estimates.
Dialogue: 0,0:09:49.00,0:09:51.10,Default,,0000,0000,0000,,I think that's a big caution.
Dialogue: 0,0:09:51.10,0:09:53.40,Default,,0000,0000,0000,,In view of those findings\Nin an example I care about
Dialogue: 0,0:09:53.70,0:09:57.10,Default,,0000,0000,0000,,where the instruments\Nare very simple
Dialogue: 0,0:09:57.40,0:09:59.10,Default,,0000,0000,0000,,and I believe that they're valid,
Dialogue: 0,0:09:59.30,0:10:01.60,Default,,0000,0000,0000,,I would be skeptical of that.
Dialogue: 0,0:10:02.90,0:10:06.80,Default,,0000,0000,0000,,So non-linearity and IV\Ndon't mix very comfortably.
Dialogue: 0,0:10:07.20,0:10:10.45,Default,,0000,0000,0000,,No, it sounds like that's already\Na more complicated...
Dialogue: 0,0:10:10.45,0:10:11.40,Default,,0000,0000,0000,,- Well, it's IV....\N- Yeah.
Dialogue: 0,0:10:12.50,0:10:16.70,Default,,0000,0000,0000,,- ...and we work on that.
Dialogue: 0,0:10:17.15,0:10:17.88,Default,,0000,0000,0000,,[laughter]
Dialogue: 0,0:10:17.88,0:10:18.60,Default,,0000,0000,0000,,- Fair enough.
Dialogue: 0,0:10:18.60,0:10:20.45,Default,,0000,0000,0000,,- As Editor of Econometric [guy],
Dialogue: 0,0:10:20.45,0:10:22.30,Default,,0000,0000,0000,,a lot of these papers\Ncross by my desk,
Dialogue: 0,0:10:22.70,0:10:26.10,Default,,0000,0000,0000,,but the motivation is not clear
Dialogue: 0,0:10:26.10,0:10:29.50,Default,,0000,0000,0000,,and, in fact, really lacking.
Dialogue: 0,0:10:29.80,0:10:35.10,Default,,0000,0000,0000,,They're not... [we call] type\Nsemi-parametric foundational papers.
Dialogue: 0,0:10:35.40,0:10:37.10,Default,,0000,0000,0000,,So that that's a big problem.
Dialogue: 0,0:10:38.00,0:10:42.40,Default,,0000,0000,0000,,A related problem is that we have\Nthis tradition in econometrics
Dialogue: 0,0:10:42.60,0:10:47.50,Default,,0000,0000,0000,,of being very focused\Non these formal [ ] results.
Dialogue: 0,0:10:48.80,0:10:52.60,Default,,0000,0000,0000,,We have just have a lot of papers\Nwhere people propose a method
Dialogue: 0,0:10:52.80,0:10:55.70,Default,,0000,0000,0000,,and then establish\Nthe asymptotic properties
Dialogue: 0,0:10:56.30,0:10:59.10,Default,,0000,0000,0000,,in a very kind of standardized way.
Dialogue: 0,0:10:59.10,0:11:01.90,Default,,0000,0000,0000,,- Is that bad?
Dialogue: 0,0:11:02.90,0:11:07.20,Default,,0000,0000,0000,,- Well, I think it's sort\Nof closed the door
Dialogue: 0,0:11:07.20,0:11:09.40,Default,,0000,0000,0000,,for a lot of work\Nthat doesn't fit it into that.
Dialogue: 0,0:11:09.40,0:11:11.60,Default,,0000,0000,0000,,where in the machine\Nlearning literature,
Dialogue: 0,0:11:11.90,0:11:14.30,Default,,0000,0000,0000,,a lot of things\Nare more algorithmic.
Dialogue: 0,0:11:14.43,0:11:18.50,Default,,0000,0000,0000,,People had algorithms\Nfor coming up with predictions
Dialogue: 0,0:11:18.80,0:11:21.20,Default,,0000,0000,0000,,that turn out\Nto actually work much better
Dialogue: 0,0:11:21.20,0:11:23.60,Default,,0000,0000,0000,,than, say, nonparametric\Nkernel regression
Dialogue: 0,0:11:24.00,0:11:26.80,Default,,0000,0000,0000,,For a long time, we were doing all\Nthe nonparametrics in econometrics,
Dialogue: 0,0:11:26.80,0:11:28.95,Default,,0000,0000,0000,,we were using kernel regression,
Dialogue: 0,0:11:28.95,0:11:31.10,Default,,0000,0000,0000,,and it was great for proving theorems.
Dialogue: 0,0:11:31.30,0:11:33.05,Default,,0000,0000,0000,,You could get [ ] intervals
Dialogue: 0,0:11:33.05,0:11:34.80,Default,,0000,0000,0000,,and consistency, \Nand asymptotic normality,
Dialogue: 0,0:11:34.80,0:11:35.90,Default,,0000,0000,0000,,and it was all great,
Dialogue: 0,0:11:35.90,0:11:37.00,Default,,0000,0000,0000,,But it wasn't very useful.
Dialogue: 0,0:11:37.30,0:11:39.10,Default,,0000,0000,0000,,And the things they did\Nin machine learning
Dialogue: 0,0:11:39.10,0:11:40.90,Default,,0000,0000,0000,,are just way, way better.
Dialogue: 0,0:11:41.00,0:11:43.05,Default,,0000,0000,0000,,But they didn't have the problem--
Dialogue: 0,0:11:43.05,0:11:44.30,Default,,0000,0000,0000,,- That's not my beef\Nwith machine learning theory.
Dialogue: 0,0:11:44.30,0:11:45.30,Default,,0000,0000,0000,,[laughter]
Dialogue: 0,0:11:45.30,0:11:51.20,Default,,0000,0000,0000,,No, but I'm saying there,\Nfor the prediction part,
Dialogue: 0,0:11:51.40,0:11:52.95,Default,,0000,0000,0000,,it does much better.
Dialogue: 0,0:11:52.95,0:11:54.50,Default,,0000,0000,0000,,- Yeah, it's a better\Ncurve fitting to it.
Dialogue: 0,0:11:54.90,0:11:56.50,Default,,0000,0000,0000,,- But it did so in a way
Dialogue: 0,0:11:57.10,0:11:58.50,Default,,0000,0000,0000,,that would not have made\Nthose papers
Dialogue: 0,0:11:58.50,0:11:59.90,Default,,0000,0000,0000,,initially easy to get into,\Nthe econometrics journals,
Dialogue: 0,0:12:04.65,0:12:06.30,Default,,0000,0000,0000,,because it wasn't proving\Nthe type of things.
Dialogue: 0,0:12:06.40,0:12:08.80,Default,,0000,0000,0000,,When Brigham was doing\Nhis regression trees
Dialogue: 0,0:12:08.80,0:12:11.20,Default,,0000,0000,0000,,that just didn't fit in.
Dialogue: 0,0:12:11.80,0:12:15.10,Default,,0000,0000,0000,,I think he would have had\Na very hard time
Dialogue: 0,0:12:15.20,0:12:18.40,Default,,0000,0000,0000,,publishing these things\Nin econometric journals.
Dialogue: 0,0:12:18.90,0:12:24.40,Default,,0000,0000,0000,,I think we've limited\Nourselves too much
Dialogue: 0,0:12:24.70,0:12:27.90,Default,,0000,0000,0000,,that left us close things off
Dialogue: 0,0:12:28.00,0:12:29.40,Default,,0000,0000,0000,,for a lot of these\Nmachine learning methods
Dialogue: 0,0:12:29.40,0:12:30.80,Default,,0000,0000,0000,,that are actually very useful.
Dialogue: 0,0:12:30.90,0:12:34.00,Default,,0000,0000,0000,,I mean, I think, in general,
Dialogue: 0,0:12:34.90,0:12:36.20,Default,,0000,0000,0000,,that literature, \Nthe computer scientist,
Dialogue: 0,0:12:36.20,0:12:37.75,Default,,0000,0000,0000,,have proposed a huge number\Nof these algorithms
Dialogue: 0,0:12:37.75,0:12:39.30,Default,,0000,0000,0000,,that actually are very useful.
Dialogue: 0,0:12:45.50,0:12:47.30,Default,,0000,0000,0000,,and that are affecting
Dialogue: 0,0:12:47.30,0:12:49.10,Default,,0000,0000,0000,,the way we're going\Nto be doing empirical work.
Dialogue: 0,0:12:49.80,0:12:52.45,Default,,0000,0000,0000,,But we've not fully internalized that
Dialogue: 0,0:12:52.45,0:12:55.10,Default,,0000,0000,0000,,because we're still very focused
Dialogue: 0,0:12:55.30,0:12:57.50,Default,,0000,0000,0000,,on getting point estimates\Nand getting standard errors
Dialogue: 0,0:12:58.60,0:13:01.20,Default,,0000,0000,0000,,and getting P values
Dialogue: 0,0:13:01.70,0:13:03.10,Default,,0000,0000,0000,,in a way that we need to move beyond
Dialogue: 0,0:13:03.30,0:13:04.30,Default,,0000,0000,0000,,to fully harness the force,
Dialogue: 0,0:13:04.30,0:13:10.70,Default,,0000,0000,0000,,the benefits\Nfrom the machine learning literature.
Dialogue: 0,0:13:10.90,0:13:13.00,Default,,0000,0000,0000,,- On the one hand, I guess I very\Nmuch take your point
Dialogue: 0,0:13:13.00,0:13:15.10,Default,,0000,0000,0000,,that sort of the traditional\Neconometrics framework
Dialogue: 0,0:13:15.20,0:13:18.60,Default,,0000,0000,0000,,of sort of propose a method,\Nprove a limit theorem
Dialogue: 0,0:13:18.60,0:13:22.60,Default,,0000,0000,0000,,under some asymptotic story,\Nstory story, story story...
Dialogue: 0,0:13:22.60,0:13:26.90,Default,,0000,0000,0000,,publisher paper is constraining.
Dialogue: 0,0:13:26.90,0:13:29.70,Default,,0000,0000,0000,,And that, in some sense,
Dialogue: 0,0:13:29.70,0:13:30.58,Default,,0000,0000,0000,,by thinking more broadly
Dialogue: 0,0:13:30.58,0:13:31.45,Default,,0000,0000,0000,,about what a methods paper\Ncould look like,
Dialogue: 0,0:13:31.45,0:13:33.20,Default,,0000,0000,0000,,we may [write] in some sense.
Dialogue: 0,0:13:33.20,0:13:35.90,Default,,0000,0000,0000,,Certainly the machine learning\Nliterature has found a bunch of things,
Dialogue: 0,0:13:35.90,0:13:38.30,Default,,0000,0000,0000,,which seem to work quite well\Nfor a number of problems
Dialogue: 0,0:13:38.30,0:13:40.35,Default,,0000,0000,0000,,and are now having\Nsubstantial influence in economics.
Dialogue: 0,0:13:40.35,0:13:42.40,Default,,0000,0000,0000,,I guess a question I'm interested in
Dialogue: 0,0:13:42.40,0:13:44.80,Default,,0000,0000,0000,,is how do you think\Nabout the role of...
Dialogue: 0,0:13:47.90,0:13:51.20,Default,,0000,0000,0000,,sort of -- do you think there is\Nno value in the theory part of it?
Dialogue: 0,0:13:51.60,0:13:54.80,Default,,0000,0000,0000,,Because I guess a question\Nthat I often have
Dialogue: 0,0:13:54.80,0:13:56.90,Default,,0000,0000,0000,,to sort of seeing that output\Nfrom a machine learning tool,
Dialogue: 0,0:13:56.90,0:13:59.40,Default,,0000,0000,0000,,that actually a number of the\Nmethods that you talked about
Dialogue: 0,0:13:59.40,0:14:01.80,Default,,0000,0000,0000,,actually do have inferential results\Ndeveloped for them,
Dialogue: 0,0:14:02.60,0:14:04.50,Default,,0000,0000,0000,,something that\NI always wonder about
Dialogue: 0,0:14:04.50,0:14:06.40,Default,,0000,0000,0000,,of uncertainty quantification\Nand just...
Dialogue: 0,0:14:06.50,0:14:08.00,Default,,0000,0000,0000,,I have my prior,
Dialogue: 0,0:14:08.00,0:14:11.00,Default,,0000,0000,0000,,I come into the world with my view.\NI see the result of this thing.
Dialogue: 0,0:14:11.00,0:14:12.75,Default,,0000,0000,0000,,How should I update based on it?
Dialogue: 0,0:14:12.75,0:14:14.50,Default,,0000,0000,0000,,And in some sense, \Nif I'm in a world
Dialogue: 0,0:14:14.60,0:14:15.10,Default,,0000,0000,0000,,where things are normally distributed,
Dialogue: 0,0:14:15.20,0:14:16.70,Default,,0000,0000,0000,,I know how to do it here --
Dialogue: 0,0:14:16.70,0:14:18.20,Default,,0000,0000,0000,,here I don't.
Dialogue: 0,0:14:18.20,0:14:21.40,Default,,0000,0000,0000,,And so I'm interested to hear\Nwhat you think about that.
Dialogue: 0,0:14:21.50,0:14:24.30,Default,,0000,0000,0000,,- I don't see this as sort\Nof saying, well,
Dialogue: 0,0:14:24.40,0:14:26.50,Default,,0000,0000,0000,,these results are not interesting,
Dialogue: 0,0:14:26.60,0:14:27.70,Default,,0000,0000,0000,,but it's going to be a lot of cases
Dialogue: 0,0:14:28.00,0:14:29.60,Default,,0000,0000,0000,,where it's going\Nto be incredibly hard
Dialogue: 0,0:14:29.60,0:14:31.20,Default,,0000,0000,0000,,to get those results
Dialogue: 0,0:14:31.20,0:14:33.20,Default,,0000,0000,0000,,and we may not be able to get there
Dialogue: 0,0:14:33.40,0:14:35.55,Default,,0000,0000,0000,,and we may need to do it in stages
Dialogue: 0,0:14:35.55,0:14:37.70,Default,,0000,0000,0000,,where first someone says,
Dialogue: 0,0:14:39.60,0:14:40.90,Default,,0000,0000,0000,,"Hey, I have\Nthis interesting algorithm
Dialogue: 0,0:14:40.90,0:14:42.20,Default,,0000,0000,0000,,for doing something
Dialogue: 0,0:14:42.20,0:14:44.80,Default,,0000,0000,0000,,and it works well by some of the criterion
Dialogue: 0,0:14:45.60,0:14:49.90,Default,,0000,0000,0000,,that on this particular data set,
Dialogue: 0,0:14:51.00,0:14:53.40,Default,,0000,0000,0000,,and I'm visit put it out there,
Dialogue: 0,0:14:53.70,0:14:55.85,Default,,0000,0000,0000,,and maybe someone will figure out a way
Dialogue: 0,0:14:55.85,0:14:58.00,Default,,0000,0000,0000,,that you can later actually\Nstill do inference
Dialogue: 0,0:14:58.00,0:14:59.10,Default,,0000,0000,0000,,on the [sum] condition,
Dialogue: 0,0:14:59.10,0:15:02.10,Default,,0000,0000,0000,,and maybe those are not\Nparticularly realistic conditions,
Dialogue: 0,0:15:02.10,0:15:03.80,Default,,0000,0000,0000,,then we kind of go further.
Dialogue: 0,0:15:03.80,0:15:05.50,Default,,0000,0000,0000,,But I think we've been\Nconstraining things too much
Dialogue: 0,0:15:06.70,0:15:09.05,Default,,0000,0000,0000,,where we said,
Dialogue: 0,0:15:09.05,0:15:11.40,Default,,0000,0000,0000,,"This is the type of things\Nthat we need to do.
Dialogue: 0,0:15:12.10,0:15:14.40,Default,,0000,0000,0000,,And in some sense,
Dialogue: 0,0:15:15.70,0:15:18.20,Default,,0000,0000,0000,,that goes back\Nto the way Josh and I
Dialogue: 0,0:15:19.70,0:15:21.90,Default,,0000,0000,0000,,thought about things for the\N[local average treatment] effect.
Dialogue: 0,0:15:21.90,0:15:23.25,Default,,0000,0000,0000,,That wasn't quite the way
Dialogue: 0,0:15:23.25,0:15:24.60,Default,,0000,0000,0000,,people were thinking\Nabout these problems before.
Dialogue: 0,0:15:24.60,0:15:29.20,Default,,0000,0000,0000,,There was a sense\Nthat some of the people said
Dialogue: 0,0:15:29.50,0:15:31.90,Default,,0000,0000,0000,,the way you need to do\Nthese things is you first say,
Dialogue: 0,0:15:32.20,0:15:34.25,Default,,0000,0000,0000,,what you're interested in\Nin estimating
Dialogue: 0,0:15:34.25,0:15:36.30,Default,,0000,0000,0000,,and then you do the best job\Nyou can in estimating that.
Dialogue: 0,0:15:38.10,0:15:44.20,Default,,0000,0000,0000,,and what you guys are doing\Nis you're doing it backwards.
Dialogue: 0,0:15:44.30,0:15:46.70,Default,,0000,0000,0000,,You kind of say,\N"Here, I have an estimator,
Dialogue: 0,0:15:47.30,0:15:49.60,Default,,0000,0000,0000,,and now I'm going to figure out\Nwhat it's estimating,
Dialogue: 0,0:15:51.40,0:15:53.90,Default,,0000,0000,0000,,and I suppose you're going to say\Nwhy you think that's interesting
Dialogue: 0,0:15:53.90,0:15:56.60,Default,,0000,0000,0000,,or maybe why it's not interesting,\Nand that's not okay.
Dialogue: 0,0:15:56.60,0:15:58.60,Default,,0000,0000,0000,,You're not allowed\Nto do that that way.
Dialogue: 0,0:15:59.00,0:16:04.10,Default,,0000,0000,0000,,And I think we should\Njust be a little bit more flexible
Dialogue: 0,0:16:04.30,0:16:06.30,Default,,0000,0000,0000,,in thinking about\Nhow to look at problems
Dialogue: 0,0:16:06.40,0:16:08.85,Default,,0000,0000,0000,,because I think\Nwe've missed some things
Dialogue: 0,0:16:08.85,0:16:11.30,Default,,0000,0000,0000,,by not doing that.
Dialogue: 0,0:16:13.00,0:16:14.80,Default,,0000,0000,0000,,- [Josh] So you've heard\Nour views, Isaiah.
Dialogue: 0,0:16:14.80,0:16:16.60,Default,,0000,0000,0000,,You've seen that we have\Nsome points of disagreement.
Dialogue: 0,0:16:17.00,0:16:20.40,Default,,0000,0000,0000,,Why don't you referee\Nthis dispute for us?
Dialogue: 0,0:16:20.95,0:16:21.95,Default,,0000,0000,0000,,[laughter]
Dialogue: 0,0:16:22.50,0:16:25.30,Default,,0000,0000,0000,,- Oh, it's so nice of you\Nto ask me a small question.
Dialogue: 0,0:16:25.30,0:16:28.10,Default,,0000,0000,0000,,So I guess for one,
Dialogue: 0,0:16:28.20,0:16:33.20,Default,,0000,0000,0000,,I very much agree with something\Nthat Guido said earlier of...
Dialogue: 0,0:16:34.10,0:16:35.10,Default,,0000,0000,0000,,[laughter]
Dialogue: 0,0:16:36.50,0:16:37.90,Default,,0000,0000,0000,,- So one thing where it seems
Dialogue: 0,0:16:37.90,0:16:39.65,Default,,0000,0000,0000,,where the case for machine learning\Nseems relatively clear
Dialogue: 0,0:16:39.65,0:16:41.40,Default,,0000,0000,0000,,is in settings where\Nwe're interested in some version
Dialogue: 0,0:16:41.50,0:16:45.10,Default,,0000,0000,0000,,of a nonparametric\Nprediction problem.
Dialogue: 0,0:16:45.10,0:16:47.40,Default,,0000,0000,0000,,So I'm interested in estimating
Dialogue: 0,0:16:47.40,0:16:49.70,Default,,0000,0000,0000,,a conditional expectation\Nor conditional probability,
Dialogue: 0,0:16:50.00,0:16:52.10,Default,,0000,0000,0000,,and in the past, maybe\NI would have run a kernel...
Dialogue: 0,0:16:52.10,0:16:53.95,Default,,0000,0000,0000,,I would have run\Na kernel regression
Dialogue: 0,0:16:53.95,0:16:55.80,Default,,0000,0000,0000,,or I would have run\Na series regression,
Dialogue: 0,0:16:56.10,0:16:57.40,Default,,0000,0000,0000,,or something along those lines.
Dialogue: 0,0:16:58.70,0:17:00.35,Default,,0000,0000,0000,,It seems like, at this point, \Nwe've a fairly good sense
Dialogue: 0,0:17:00.35,0:17:02.00,Default,,0000,0000,0000,,that in a fairly wide range\Nof applications,
Dialogue: 0,0:17:02.00,0:17:06.30,Default,,0000,0000,0000,,machine learning methods\Nseem to do better
Dialogue: 0,0:17:06.80,0:17:08.80,Default,,0000,0000,0000,,for estimating conditional\Nmean functions
Dialogue: 0,0:17:08.80,0:17:10.40,Default,,0000,0000,0000,,or conditional probabilities
Dialogue: 0,0:17:10.40,0:17:12.00,Default,,0000,0000,0000,,or various other\Nnonparametric objects
Dialogue: 0,0:17:12.40,0:17:14.50,Default,,0000,0000,0000,,than more traditional\Nnonparametric methods
Dialogue: 0,0:17:14.50,0:17:16.60,Default,,0000,0000,0000,,that were studied\Nin econometrics and statistics,
Dialogue: 0,0:17:16.60,0:17:19.10,Default,,0000,0000,0000,,especially\Nin high dimensional settings.
Dialogue: 0,0:17:19.50,0:17:21.30,Default,,0000,0000,0000,,- So you're thinking of maybe\Nthe propensity score
Dialogue: 0,0:17:21.30,0:17:23.10,Default,,0000,0000,0000,,or something like that?
Dialogue: 0,0:17:23.10,0:17:24.20,Default,,0000,0000,0000,,- Yeah, exactly,
Dialogue: 0,0:17:24.20,0:17:25.30,Default,,0000,0000,0000,,- Nuisance functions.
Dialogue: 0,0:17:25.30,0:17:27.10,Default,,0000,0000,0000,,Yeah, so things\Nlike propensity scores,
Dialogue: 0,0:17:27.53,0:17:29.96,Default,,0000,0000,0000,,even objects of more direct
Dialogue: 0,0:17:29.96,0:17:32.40,Default,,0000,0000,0000,,interest-like conditional\Naverage treatment effects,
Dialogue: 0,0:17:32.40,0:17:35.10,Default,,0000,0000,0000,,which of the difference of two\Nconditional expectation functions,
Dialogue: 0,0:17:35.10,0:17:36.30,Default,,0000,0000,0000,,potentially things like that.
Dialogue: 0,0:17:36.50,0:17:40.40,Default,,0000,0000,0000,,Of course, even there, the theory...
Dialogue: 0,0:17:40.50,0:17:43.70,Default,,0000,0000,0000,,inference of the theory\Nfor how to interpret,
Dialogue: 0,0:17:43.70,0:17:45.90,Default,,0000,0000,0000,,how to make large simple statements\Nabout some of these things
Dialogue: 0,0:17:46.00,0:17:48.05,Default,,0000,0000,0000,,are less well-developed\Ndepending on
Dialogue: 0,0:17:48.05,0:17:50.10,Default,,0000,0000,0000,,the machine learning\Nestimator used.
Dialogue: 0,0:17:50.10,0:17:53.80,Default,,0000,0000,0000,,And so I think there's\Nsomething that is tricky
Dialogue: 0,0:17:53.90,0:17:55.70,Default,,0000,0000,0000,,is that we can have these methods,\Nwhich work a lot,
Dialogue: 0,0:17:55.70,0:17:58.00,Default,,0000,0000,0000,,which seemed to work\Na lot better for some purposes,
Dialogue: 0,0:17:58.00,0:18:01.60,Default,,0000,0000,0000,,but which we need to be a bit\Ncareful in how we plug them in
Dialogue: 0,0:18:01.60,0:18:03.30,Default,,0000,0000,0000,,or how we interpret\Nthe resulting statements.
Dialogue: 0,0:18:03.60,0:18:06.20,Default,,0000,0000,0000,,But of course, that's a very,\Nvery active area right now
Dialogue: 0,0:18:06.40,0:18:08.40,Default,,0000,0000,0000,,where people are doing\Ntons of great work.
Dialogue: 0,0:18:08.40,0:18:10.40,Default,,0000,0000,0000,,And so I fully expect\Nand hope to see
Dialogue: 0,0:18:10.40,0:18:12.80,Default,,0000,0000,0000,,much more going forward there.
Dialogue: 0,0:18:13.00,0:18:17.30,Default,,0000,0000,0000,,So one issue with machine learning\Nthat always seems a danger
Dialogue: 0,0:18:17.40,0:18:20.30,Default,,0000,0000,0000,,or that is sometimes a danger
Dialogue: 0,0:18:20.50,0:18:21.55,Default,,0000,0000,0000,,and had sometimes\Nled to applications
Dialogue: 0,0:18:21.55,0:18:22.60,Default,,0000,0000,0000,,that have made less sense
Dialogue: 0,0:18:22.80,0:18:25.10,Default,,0000,0000,0000,,is when folks start with a method\Nthat they're very excited about
Dialogue: 0,0:18:25.30,0:18:28.50,Default,,0000,0000,0000,,rather than a question.
Dialogue: 0,0:18:28.90,0:18:32.10,Default,,0000,0000,0000,,So sort of starting with a question
Dialogue: 0,0:18:32.50,0:18:34.35,Default,,0000,0000,0000,,where here's the object I'm interested in,
Dialogue: 0,0:18:34.35,0:18:36.20,Default,,0000,0000,0000,,here is the parameter of interest.
Dialogue: 0,0:18:37.30,0:18:39.50,Default,,0000,0000,0000,,let me think about how I would\Nidentify that thing,
Dialogue: 0,0:18:39.50,0:18:41.80,Default,,0000,0000,0000,,how I would recover that thing\Nif I had a ton of data.
Dialogue: 0,0:18:41.90,0:18:44.00,Default,,0000,0000,0000,,Oh, here's a conditional\Nexpectation function.
Dialogue: 0,0:18:44.00,0:18:47.10,Default,,0000,0000,0000,,Let me plug in the machine\Nlearning estimator for that.
Dialogue: 0,0:18:47.20,0:18:48.80,Default,,0000,0000,0000,,That seems very, very sensible.
Dialogue: 0,0:18:49.00,0:18:53.10,Default,,0000,0000,0000,,Whereas, you know, \Nif I regress quantity on price
Dialogue: 0,0:18:53.70,0:18:56.00,Default,,0000,0000,0000,,and say that I used\Na machine learning method,
Dialogue: 0,0:18:56.30,0:18:58.90,Default,,0000,0000,0000,,maybe I'm satisfied that \Nthat solves the [ ] problem
Dialogue: 0,0:18:58.90,0:19:01.20,Default,,0000,0000,0000,,we're usually worried\Nabout there... maybe I'm not.
Dialogue: 0,0:19:01.50,0:19:03.20,Default,,0000,0000,0000,,But again, that's something
Dialogue: 0,0:19:03.40,0:19:06.30,Default,,0000,0000,0000,,where the way to address it\Nseems relatively clear.
Dialogue: 0,0:19:06.50,0:19:09.00,Default,,0000,0000,0000,,It's to find your object of interest
Dialogue: 0,0:19:09.20,0:19:10.40,Default,,0000,0000,0000,,and think about--
Dialogue: 0,0:19:10.40,0:19:11.60,Default,,0000,0000,0000,,- Just bring in the economics.
Dialogue: 0,0:19:11.70,0:19:12.20,Default,,0000,0000,0000,,- Exactly.
Dialogue: 0,0:19:12.20,0:19:15.40,Default,,0000,0000,0000,,- And and can I think about heterogeneity,
Dialogue: 0,0:19:15.40,0:19:18.30,Default,,0000,0000,0000,,but harnessed the power\Nof the machine learning methods
Dialogue: 0,0:19:18.50,0:19:20.65,Default,,0000,0000,0000,,for some of the components.
Dialogue: 0,0:19:20.65,0:19:22.80,Default,,0000,0000,0000,,- Precisely. Exactly.
Dialogue: 0,0:19:22.90,0:19:24.25,Default,,0000,0000,0000,,So the question of interest
Dialogue: 0,0:19:24.25,0:19:25.60,Default,,0000,0000,0000,,is the same as the question\Nof interest has always been,
Dialogue: 0,0:19:25.60,0:19:29.50,Default,,0000,0000,0000,,but we now have better methods\Nfor estimating some pieces of this.
Dialogue: 0,0:19:29.90,0:19:31.60,Default,,0000,0000,0000,,The place that seems\Nharder to forecast
Dialogue: 0,0:19:33.40,0:19:34.85,Default,,0000,0000,0000,,is obviously, there's\Na huge amount going on
Dialogue: 0,0:19:34.85,0:19:36.30,Default,,0000,0000,0000,,in the machine learning literature
Dialogue: 0,0:19:37.50,0:19:38.60,Default,,0000,0000,0000,,and the limited ways\Nof plugging it in
Dialogue: 0,0:19:38.60,0:19:39.70,Default,,0000,0000,0000,,that I've referenced so far
Dialogue: 0,0:19:39.70,0:19:42.90,Default,,0000,0000,0000,,are a limited piece of that.
Dialogue: 0,0:19:43.00,0:19:44.55,Default,,0000,0000,0000,,And so I think there are all sorts\Nof other interesting questions
Dialogue: 0,0:19:44.55,0:19:46.10,Default,,0000,0000,0000,,about where...
Dialogue: 0,0:19:47.10,0:19:49.30,Default,,0000,0000,0000,,where does this interaction go? \NWhat else can we learn?
Dialogue: 0,0:19:49.30,0:19:52.00,Default,,0000,0000,0000,,And that's something where\NI think there's a ton going on
Dialogue: 0,0:19:52.20,0:19:54.30,Default,,0000,0000,0000,,which seems very promising,
Dialogue: 0,0:19:54.30,0:19:56.40,Default,,0000,0000,0000,,and I have no idea\Nwhat the answer is.
Dialogue: 0,0:19:57.00,0:19:59.10,Default,,0000,0000,0000,,- No, I totally agree with that,
Dialogue: 0,0:19:59.10,0:20:01.20,Default,,0000,0000,0000,,but that makes it very exciting.
Dialogue: 0,0:20:03.80,0:20:06.10,Default,,0000,0000,0000,,And I think there's just\Na little work to be done there.
Dialogue: 0,0:20:06.60,0:20:09.00,Default,,0000,0000,0000,,Alright. So I say, he agrees\Nwith me there.
Dialogue: 0,0:20:09.00,0:20:11.40,Default,,0000,0000,0000,,[laughter]
Dialogue: 0,0:20:12.45,0:20:13.45,Default,,0000,0000,0000,,- I didn't say that per se.
Dialogue: 0,0:20:14.50,0:20:16.10,Default,,0000,0000,0000,,- [Narrator] If you'd like to watch\Nmore Nobel Conversations,
Dialogue: 0,0:20:16.10,0:20:17.70,Default,,0000,0000,0000,,click here.
Dialogue: 0,0:20:18.00,0:20:20.40,Default,,0000,0000,0000,,Pr if you'd like to learn\Nmore about econometrics,
Dialogue: 0,0:20:20.50,0:20:23.10,Default,,0000,0000,0000,,check out Josh's\NMastering Econometrics series.
Dialogue: 0,0:20:23.60,0:20:26.50,Default,,0000,0000,0000,,If you'd like to learn more\Nabout Guido, Josh, and Isaiah,
Dialogue: 0,0:20:26.70,0:20:28.20,Default,,0000,0000,0000,,check out the links\Nin the description.
Dialogue: 0,0:20:28.55,0:20:30.54,Default,,0000,0000,0000,,♪ [music] ♪