0:00:00.299,0:00:04.644 - [Instructor] The path from cause [br]to effect is dark and dangerous. 0:00:05.041,0:00:08.015 But the weapons [br]of Econometrics are strong. 0:00:08.480,0:00:11.734 Attack with fierce [br]and flexible instrumental variables 0:00:11.734,0:00:15.803 when nature blesses you [br]with fortuitous random assignment. 0:00:19.393,0:00:21.094 [gong rings] 0:00:23.653,0:00:28.704 Randomized trials are the surest [br]path to ceteris parabus comparisons. 0:00:28.704,0:00:32.640 Alas, this powerful tool [br]is often unavailable. 0:00:33.224,0:00:36.940 But sometimes, randomization [br]happens by accident. 0:00:36.940,0:00:40.592 That's when we turn [br]to instrumental variables -- 0:00:40.592,0:00:41.938 IV for short. 0:00:41.938,0:00:44.508 - [Voice whispers] Instrumental [br]variables. 0:00:44.508,0:00:48.186 - [Instructor] Today's lesson [br]is the first of two on IV. 0:00:48.958,0:00:52.951 Our first IV lesson begins [br]with a story of schools. 0:00:52.951,0:00:54.348 [school bell rings] 0:00:54.348,0:00:56.138 - [Josh] Charter schools [br]are public schools 0:00:56.138,0:01:00.112 freed from daily district oversight[br]and teacher union contracts. 0:01:00.895,0:01:03.511 The question of whether charters [br]boost achievement 0:01:03.511,0:01:05.161 is one of the most important 0:01:05.161,0:01:07.761 in the history [br]of American education reform. 0:01:08.145,0:01:12.562 - The most popular charter schools[br]have more applicants than seats 0:01:12.562,0:01:16.462 so the luck of a lottery draw [br]decides who's offered a seat. 0:01:16.870,0:01:20.503 A lot is at stake for the students [br]vying for their chance, 0:01:20.503,0:01:25.003 and waiting for the lottery results [br]brings up lots of emotions 0:01:25.003,0:01:27.832 as was captured [br]in the award-winning documentary 0:01:27.832,0:01:29.699 "Waiting For Superman." 0:01:30.258,0:01:32.916 - [Mother] Don't cry. You're gonna [br]make Mommy cry. Okay? 0:01:37.498,0:01:40.618 - Do charters really provide[br]a better education? 0:01:40.948,0:01:43.183 Critics most definitely say no, 0:01:43.413,0:01:46.479 arguing that charters enroll [br]better students to begin with, 0:01:46.479,0:01:50.164 smarter or more motivated,[br]so differences in later outcomes 0:01:50.164,0:01:52.061 reflects selection bias. 0:01:52.595,0:01:54.729 - [Kamal] Wait, this one seems easy. 0:01:55.139,0:01:57.444 In a lottery, winners [br]are chosen randomly, 0:01:57.498,0:02:00.083 so just compare winners and losers.[br]- [Student] Obviously. 0:02:00.083,0:02:01.698 - On the right track, Kamal, 0:02:01.698,0:02:04.747 but charter lotteries [br]don't force kids into 0:02:04.747,0:02:07.560 or out of a particular school. 0:02:07.749,0:02:10.667 They randomize offers [br]of a charter seat. 0:02:11.650,0:02:13.449 Some kids get lucky. 0:02:13.449,0:02:14.966 Some kids don't. 0:02:14.966,0:02:19.118 If we just wanted to know [br]the effect of charter school offers, 0:02:19.118,0:02:22.417 we could treat this [br]as a randomized trial. 0:02:22.717,0:02:24.684 But we we're interested [br]in the effects 0:02:24.684,0:02:28.283 of charter school attendance,[br]not offers. 0:02:28.568,0:02:31.917 And not everyone [br]who is offered, accepts. 0:02:31.917,0:02:37.234 IV turns the effect of being offered[br]a charter seat into the effect 0:02:37.234,0:02:40.367 of actually attending [br]a charter school. 0:02:40.367,0:02:42.344 - [Student] Cool.[br]- Oh nice. 0:02:45.925,0:02:48.871 - Let's look at an example, [br]a charter school from 0:02:48.871,0:02:52.353 the Knowledge Is Power [br]Program, or KIPP for short. 0:02:52.736,0:02:54.937 This KIPP school is in Lynn, 0:02:54.937,0:02:58.837 a faded industrial town [br]on the coast of Massachusetts. 0:02:59.104,0:03:01.886 The school has [br]more applicants than seats 0:03:01.886,0:03:05.620 and therefore picks its students[br]using a lottery. 0:03:05.834,0:03:11.854 From 2005 to 2008,[br]371 fourth and fifth graders 0:03:11.854,0:03:15.320 put their names [br]in the KIPP Lynn lottery, 0:03:15.382,0:03:18.754 253 students won a seat at KIPP, 0:03:18.754,0:03:21.651 118 students lost. 0:03:21.967,0:03:26.001 A year later, lottery winners had [br]much higher math scores 0:03:26.001,0:03:27.719 than lottery losers. 0:03:27.802,0:03:30.370 But remember, [br]we're not trying to figure out 0:03:30.370,0:03:33.803 whether winning a lottery [br]makes you better at math. 0:03:34.070,0:03:38.471 We want to know if attending KIPP[br]makes you better at math. 0:03:38.788,0:03:45.671 Of the 253 lottery winners,[br]only 199 actually went to KIPP. 0:03:46.139,0:03:48.804 The others chose [br]a traditional public school. 0:03:49.563,0:03:55.370 Similarly of the 118 lottery losers,[br]a few actually ended up at KIPP. 0:03:55.509,0:03:57.452 They got an offer later. 0:03:57.452,0:04:02.377 So what was the effect on test scores[br]of actually attending KIPP? 0:04:03.109,0:04:05.426 - [Kamal] Why can't we just [br]measure their math scores? 0:04:05.426,0:04:07.096 - [Instructor] Great question. 0:04:07.096,0:04:09.302 Who would you compare them to? 0:04:09.302,0:04:11.111 - [Kamal] Those who didn't attend. 0:04:11.111,0:04:12.944 - [Instructor] Is attendance random? 0:04:13.937,0:04:15.057 - [Camilla] No. 0:04:15.057,0:04:16.177 - Selection bias. 0:04:16.177,0:04:17.909 - [Instructor] Correct.[br]- [Otto] What? 0:04:17.909,0:04:21.826 - [Instructor] The KIPP offers [br]are random so we can be confident 0:04:21.826,0:04:26.409 of ceteris parabus,[br]but attendance is not random. 0:04:26.635,0:04:30.601 The choice to accept the offer [br]might be due to characteristics 0:04:30.601,0:04:32.984 that are related [br]to math performance -- 0:04:33.251,0:04:36.157 say, for example, [br]that dedicated parents 0:04:36.157,0:04:38.941 are more likely [br]to accept the offer. 0:04:38.941,0:04:42.646 Their kids are also more likely [br]to do better in math, 0:04:42.646,0:04:44.090 regardless of school. 0:04:44.090,0:04:45.114 - [Student] Right. 0:04:45.114,0:04:47.613 - [Instructor] IV converts [br]the offer effect 0:04:47.613,0:04:50.567 into the effect of KIPP attendance, 0:04:50.573,0:04:53.371 adjusting for the fact [br]that some winners go elsewhere 0:04:53.371,0:04:56.573 and some losers manage [br]to attend KIPP anyway. 0:04:56.950,0:05:00.517 Essentially, IV takes [br]an incomplete randomization 0:05:00.517,0:05:03.007 and makes the appropriate [br]adjustments. 0:05:03.684,0:05:07.107 How? IV describes a chain reaction. 0:05:07.426,0:05:10.343 Why do offers affect achievement? 0:05:10.343,0:05:13.175 Probably because they affect [br]charter attendance 0:05:13.175,0:05:16.643 and charter attendance [br]improves math scores, 0:05:16.643,0:05:20.442 the first link in the chain[br]called the first stage 0:05:20.442,0:05:24.341 is the effect of the lottery [br]on charter attendance. 0:05:24.446,0:05:28.361 The second stage is the length [br]between attending a charter 0:05:28.361,0:05:30.153 and an outcome variable, 0:05:30.153,0:05:32.261 in this case, math scores. 0:05:32.940,0:05:36.441 The instrumental variable, [br]or instrument for short, 0:05:36.441,0:05:40.246 is the variable [br]that initiates the chain reaction. 0:05:40.899,0:05:44.833 The effect of the instrument [br]on the outcome is called 0:05:44.833,0:05:46.631 the reduced form. 0:05:48.143,0:05:51.615 This chain reaction can be [br]represented mathematically. 0:05:51.615,0:05:55.266 We multiply the first stage,[br]the effect of winning 0:05:55.266,0:05:57.866 on attendance, by the second stage, 0:05:57.866,0:06:00.567 the effect of attendance on scores. 0:06:00.630,0:06:02.713 And we get the reduced form, 0:06:02.713,0:06:05.680 the effect of winning [br]the lottery on scores. 0:06:06.780,0:06:11.566 The reduced form and first stage [br]are observable and easy to compute. 0:06:11.752,0:06:14.876 However, the effect of attendance[br]on achievement 0:06:14.876,0:06:16.993 is not directly observed. 0:06:16.993,0:06:20.360 This is the causal effect[br]we're trying to determine. 0:06:21.043,0:06:23.827 Given some important assumptions [br]we'll discuss shortly, 0:06:23.827,0:06:25.977 we can find the effect [br]of KIPP attendance 0:06:25.977,0:06:29.183 by dividing the reduced form[br]by the first stage. 0:06:29.225,0:06:32.774 This will become more clear [br]as we work through an example. 0:06:32.774,0:06:34.207 - [Student] Let's do this. 0:06:37.161,0:06:38.728 - A quick note on measurement. 0:06:38.728,0:06:41.678 We measure achievement[br]using standard deviations, 0:06:41.678,0:06:44.728 often denoted [br]by the Greek letter sigma (σ). 0:06:44.728,0:06:48.862 One σ is a huge move[br]from around the bottom 15% 0:06:48.862,0:06:51.634 to the middle of most[br]achievement distributions. 0:06:51.634,0:06:55.412 Even a ¼ or ½ σ difference is big. 0:06:56.262,0:06:58.389 - [Instructor] Now we're ready [br]to plug some numbers 0:06:58.389,0:07:01.382 into the equation [br]we introduced earlier. 0:07:01.557,0:07:03.231 First up, what's the effect 0:07:03.231,0:07:06.076 of winning the lottery [br]on math scores? 0:07:06.354,0:07:10.437 KIPP applicants' math scores [br]are a third of a standard deviation 0:07:10.504,0:07:14.386 below the state average in [br]the year before they apply to KIPP. 0:07:14.386,0:07:18.120 But a year later, lottery winners[br]score right at the state average 0:07:18.215,0:07:21.482 while the lottery losers [br]are still well behind 0:07:21.482,0:07:25.499 with an average score [br]around - 0.36 σ. 0:07:25.834,0:07:29.619 The effect of winning the lottery [br]on scores is the difference 0:07:29.619,0:07:32.819 between the winners' scores[br]and the losers' scores. 0:07:33.403,0:07:35.784 Take the winners' [br]average math scores, 0:07:35.784,0:07:38.269 subtract the losers' [br]average math scores, 0:07:38.269,0:07:41.502 and you will have 0.36 σ . 0:07:41.908,0:07:46.659 Next up: what's the effect [br]of winning the lottery on attendance? 0:07:46.809,0:07:49.193 In other words, [br]if you win the lottery, 0:07:49.193,0:07:53.293 how much more likely are you [br]to attend KIPP than if you lose? 0:07:53.643,0:07:57.610 First, what percentage [br]of lottery winners attend KIPP? 0:07:57.610,0:08:00.626 Divide the number of winners[br]who attended KIPP 0:08:00.626,0:08:05.361 by the total number [br]of lottery winners -- that's 78%. 0:08:05.810,0:08:09.143 To find the percentage [br]of lottery losers who attended KIPP, 0:08:09.143,0:08:12.293 we divide the number of losers [br]who attended KIPP 0:08:12.293,0:08:16.760 by the total number [br]of lottery losers -- that's 4%. 0:08:17.377,0:08:21.393 Subtract 4 from 78, and we find [br]that winning the lottery 0:08:21.393,0:08:25.512 makes you 74% [br]more likely to attend KIPP. 0:08:25.946,0:08:28.226 Now we can find [br]what we're really after, 0:08:28.383,0:08:34.551 the effect of attendance on scores,[br]by dividing 0.36 by 0.74. 0:08:34.789,0:08:37.585 Attending KIPP raises math scores 0:08:37.585,0:08:41.518 by 0.48 standard deviations [br]on average. 0:08:42.269,0:08:44.503 That's an awesome achievement gain, 0:08:44.503,0:08:47.236 equal to moving [br]from about the bottom third 0:08:47.236,0:08:49.955 to the middle [br]of the achievement distribution. 0:08:49.955,0:08:51.238 - [Student] Whoa, half a sig. 0:08:51.238,0:08:53.507 - [Instructor] These estimates [br]are for kids opting in 0:08:53.507,0:08:56.047 to the KIPP lottery,[br]whose enrollment status 0:08:56.047,0:08:57.762 is changed by winning. 0:08:57.985,0:09:00.617 That's not necessarily [br]a random sample 0:09:00.617,0:09:02.283 of all children in Lynn. 0:09:02.536,0:09:05.035 So we can't assume [br]we'd see the same effect 0:09:05.035,0:09:07.327 for other types of students.[br]- [Student] Huh. 0:09:07.327,0:09:10.218 - But this effect [br]on keen for KIPP kids 0:09:10.218,0:09:13.367 is likely to be a good indicator[br]of the consequences 0:09:13.367,0:09:15.767 of adding additional charter seats. 0:09:15.767,0:09:17.216 - [Student] Cool.[br]- [Student] Got it. 0:09:19.628,0:09:23.145 - IV eliminates selection bias, [br]but like all of our tools, 0:09:23.145,0:09:25.624 the solution builds on a set [br]of assumptions 0:09:25.624,0:09:27.540 not to be taken for granted. 0:09:28.098,0:09:31.347 First, there must be [br]a substantial first stage -- 0:09:31.347,0:09:35.465 that is the instrumental variable,[br]winning or losing the lottery, 0:09:35.465,0:09:38.915 must really change the variable[br]whose effect we're interested in -- 0:09:38.915,0:09:41.031 here, KIPP attendance. 0:09:41.298,0:09:44.415 In this case, the first stage [br]is not really in doubt. 0:09:44.415,0:09:47.894 Winning the lottery makes [br]KIPP attendance much more likely. 0:09:48.386,0:09:50.631 Not all IV stories are like that. 0:09:51.321,0:09:53.698 Second, the instrument [br]must be as good 0:09:53.698,0:09:56.731 as randomly assigned, [br]meaning lottery winners and losers 0:09:56.731,0:09:58.716 have similar characteristics. 0:09:58.893,0:10:01.559 This is the independence assumption. 0:10:01.977,0:10:05.627 Of course, KIPP lottery wins[br]really are randomly assigned. 0:10:05.627,0:10:09.293 Still, we should check for balance [br]and confirm that winners and losers 0:10:09.293,0:10:11.493 have similar family backgrounds, 0:10:11.493,0:10:13.394 similar aptitudes and so on. 0:10:13.543,0:10:16.969 In essence, we're checking [br]to ensure KIPP lotteries are fair 0:10:16.969,0:10:20.017 with no group of applicants [br]suspiciously likely to win. 0:10:21.373,0:10:24.373 Finally, we require [br]the instrument change outcomes 0:10:24.373,0:10:26.252 solely through [br]the variable of interest, 0:10:26.252,0:10:28.100 in this case, attending KIPP. 0:10:28.299,0:10:31.367 This assumption is called[br]the exclusion restriction. 0:10:32.951,0:10:37.500 - IV only works if you can satisfy [br]these three assumptions. 0:10:37.783,0:10:40.418 - I don't understand [br]the exclusion restriction. 0:10:40.917,0:10:43.599 How could winning the lottery[br]affect math scores 0:10:43.599,0:10:45.244 other than by attending KIPP? 0:10:45.244,0:10:47.230 - [Student] Yeah. [br]- [Instructor] Great question. 0:10:47.230,0:10:50.536 Suppose lottery winners[br]are just thrilled to win, 0:10:50.536,0:10:55.045 and this happiness motivates them [br]to study more and learn more math, 0:10:55.045,0:10:57.144 regardless of where [br]they go to school. 0:10:57.231,0:10:59.901 This would violate [br]the exclusion restriction 0:10:59.901,0:11:03.787 because the motivational effect [br]of winning is a second channel 0:11:03.787,0:11:06.569 whereby lotteries [br]might affect test scores. 0:11:06.865,0:11:09.546 While it's hard [br]to rule this out entirely, 0:11:09.546,0:11:12.650 there's no evidence [br]of any alternative channels 0:11:12.650,0:11:14.499 in the KIPP study. 0:11:17.817,0:11:20.700 - IV solves the problem [br]of selection bias 0:11:20.700,0:11:24.850 in scenarios like the KIPP lottery[br]where treatment offers are random 0:11:24.850,0:11:27.083 but some of those offered opt out. 0:11:28.451,0:11:31.700 This sort of intentional [br]yet incomplete random assignment 0:11:31.700,0:11:33.367 is surprisingly common. 0:11:33.367,0:11:36.318 Even randomized clinical trials[br]have this feature. 0:11:37.134,0:11:40.053 IV solves the problem [br]of non-random take up 0:11:40.053,0:11:42.534 in lotteries or clinical research. 0:11:43.054,0:11:46.725 But lotteries are not the only[br]source of compelling instruments. 0:11:46.915,0:11:50.397 Many causal questions can be[br]addressed by naturally occurring 0:11:50.397,0:11:53.831 as good as randomly [br]assigned variation. 0:11:54.731,0:11:56.915 Here's a causal question for you -- 0:11:56.915,0:11:59.955 do women who have children early [br]in their careers suffer 0:11:59.955,0:12:02.648 a substantial earnings penalty [br]as a result? 0:12:02.648,0:12:04.970 After all, women earn less than men. 0:12:05.573,0:12:08.506 We could, of course, simply [br]compare the earnings of women 0:12:08.506,0:12:10.891 with more and fewer children. 0:12:10.891,0:12:14.190 But such comparisons are fraught[br]with selection bias. 0:12:14.806,0:12:19.089 If only we could randomly assign [br]babies to different households. 0:12:19.089,0:12:22.131 Yeah, right, [br]sounds pretty fanciful. 0:12:22.470,0:12:26.601 Our next IV story -- fantastic[br]and not fanciful -- 0:12:26.601,0:12:30.234 illustrates an amazing, [br]naturally-occurring instrument 0:12:30.234,0:12:31.918 for family size. 0:12:33.317,0:12:34.317 ♪ [music] ♪ 0:12:34.551,0:12:37.985 - [Instructor] You're on your way [br]to mastering Econometrics. 0:12:38.153,0:12:40.170 Make sure this video sticks 0:12:40.170,0:12:42.636 by taking a few [br]quick practice questions. 0:12:42.886,0:12:46.336 Or, if you're ready, [br]click for the next video. 0:12:46.529,0:12:50.278 You can also check out [br]MRU's website for more courses, 0:12:50.278,0:12:52.027 teacher resources, and more. 0:12:52.289,0:12:53.772 ♪ [music] ♪