1 99:59:59,999 --> 99:59:59,999 - [Instructor] The path from cause to effect is dark and dangerous. 2 99:59:59,999 --> 99:59:59,999 But the weapons of Econometrics are strong. 3 99:59:59,999 --> 99:59:59,999 [Attack] with fierce and flexible instrumental variables 4 99:59:59,999 --> 99:59:59,999 when nature blesses you with fortuitous random assignment. 5 99:59:59,999 --> 99:59:59,999 [gong rings] 6 99:59:59,999 --> 99:59:59,999 Randomized trials are the surest path to ceteris parabus comparisons. 7 99:59:59,999 --> 99:59:59,999 Alas, this powerful tool is often unavailable. 8 99:59:59,999 --> 99:59:59,999 But sometimes, randomization happens by accident. 9 99:59:59,999 --> 99:59:59,999 That's when we turn to instrumental variables, 10 99:59:59,999 --> 99:59:59,999 IV for short. 11 99:59:59,999 --> 99:59:59,999 - [Voice whispers] Instrumental variables. 12 99:59:59,999 --> 99:59:59,999 - [Instructor] Today's lesson is the first of two on IV. 13 99:59:59,999 --> 99:59:59,999 Our first IV lesson begins with a story of schools. 14 99:59:59,999 --> 99:59:59,999 - [Josh] Charter schools are public schools 15 99:59:59,999 --> 99:59:59,999 freed from daily district oversight and teacher union contracts. 16 99:59:59,999 --> 99:59:59,999 The question of whether charters boost achievement 17 99:59:59,999 --> 99:59:59,999 is one of the most important 18 99:59:59,999 --> 99:59:59,999 in the history of American education reform. 19 99:59:59,999 --> 99:59:59,999 - The most popular charter schools have more applicants 20 99:59:59,999 --> 99:59:59,999 so the luck of the lottery draw decides who's offered a seat. 21 99:59:59,999 --> 99:59:59,999 A lot is at stake for the students vying for their chance. 22 99:59:59,999 --> 99:59:59,999 Waiting for the lottery results brings up lots of emotions 23 99:59:59,999 --> 99:59:59,999 as was captured in the award-winning documentary 24 99:59:59,999 --> 99:59:59,999 "Waiting For Superman." 25 99:59:59,999 --> 99:59:59,999 - [Mother] Don't cry. You're gonna make Mommy cry. 26 99:59:59,999 --> 99:59:59,999 Okay? 27 99:59:59,999 --> 99:59:59,999 - Do charters really provide a better education? 28 99:59:59,999 --> 99:59:59,999 Critics most definitely say no, 29 99:59:59,999 --> 99:59:59,999 arguing that charters enroll better students to begin with, 30 99:59:59,999 --> 99:59:59,999 smarter or more motivated so differences in later outcomes 31 99:59:59,999 --> 99:59:59,999 reflects selection bias. 32 99:59:59,999 --> 99:59:59,999 - [Kamal] Wait, this one seems easy. 33 99:59:59,999 --> 99:59:59,999 In a lottery, winners are chosen randomly, 34 99:59:59,999 --> 99:59:59,999 so just compare winners and losers. 35 99:59:59,999 --> 99:59:59,999 - [Student] Obviously. 36 99:59:59,999 --> 99:59:59,999 - On the right track, Kamal, 37 99:59:59,999 --> 99:59:59,999 but charter lotteries don't force kids into 38 99:59:59,999 --> 99:59:59,999 or out of a particular school. 39 99:59:59,999 --> 99:59:59,999 They randomize offers of a charter seat. 40 99:59:59,999 --> 99:59:59,999 Some kids get lucky. 41 99:59:59,999 --> 99:59:59,999 Some kids don't. 42 99:59:59,999 --> 99:59:59,999 If we just wanted to know the effect of charter school offers, 43 99:59:59,999 --> 99:59:59,999 we could treat this as a randomized trial. 44 99:59:59,999 --> 99:59:59,999 But we we're interested in the effects 45 99:59:59,999 --> 99:59:59,999 of charter school attendance, not offers. 46 99:59:59,999 --> 99:59:59,999 And not everyone who is offered, accepts. 47 99:59:59,999 --> 99:59:59,999 IV turns the effect of being offered a charter seat into the effect 48 99:59:59,999 --> 99:59:59,999 of actually attending a charter school. 49 99:59:59,999 --> 99:59:59,999 - [Student] Cool. 50 99:59:59,999 --> 99:59:59,999 - Oh nice. 51 99:59:59,999 --> 99:59:59,999 - Let's look at an example, a charter school from 52 99:59:59,999 --> 99:59:59,999 the Knowledge Is Power Program, or KIPP for short. 53 99:59:59,999 --> 99:59:59,999 This KIPP school is in Lynn, 54 99:59:59,999 --> 99:59:59,999 a faded industrial town on the coast of Massachusetts. 55 99:59:59,999 --> 99:59:59,999 The school has more applicants than seats 56 99:59:59,999 --> 99:59:59,999 and therefore picks its students using a lottery. 57 99:59:59,999 --> 99:59:59,999 From 2005 to 2008, 371 fourth and fifth graders 58 99:59:59,999 --> 99:59:59,999 put their names in the KIPP/Lynn lottery, 59 99:59:59,999 --> 99:59:59,999 253 students won a seat at KIPP, 60 99:59:59,999 --> 99:59:59,999 118 students lost. 61 99:59:59,999 --> 99:59:59,999 A year later, lottery winners had much higher match scores 62 99:59:59,999 --> 99:59:59,999 than lottery losers. 63 99:59:59,999 --> 99:59:59,999 But remember, we're not trying to figure out 64 99:59:59,999 --> 99:59:59,999 whether winning a lottery makes you better at math. 65 99:59:59,999 --> 99:59:59,999 We want to know if attending KIPP makes you better at math. 66 99:59:59,999 --> 99:59:59,999 Of the 253 lottery winners, only 199 actually went to KIPP. 67 99:59:59,999 --> 99:59:59,999 The others chose a traditional public school. 68 99:59:59,999 --> 99:59:59,999 Similarly of the 118 lottery losers, a few actually ended up at KIPP. 69 99:59:59,999 --> 99:59:59,999 They got an offer later. 70 99:59:59,999 --> 99:59:59,999 So what was the effect of test scores of actually attending KIPP? 71 99:59:59,999 --> 99:59:59,999 - [Student] Why can't we just measure their math scores? 72 99:59:59,999 --> 99:59:59,999 - [Instructor] Great question. 73 99:59:59,999 --> 99:59:59,999 Who would you compare them to? 74 99:59:59,999 --> 99:59:59,999 - [Student] Those who didn't attend. 75 99:59:59,999 --> 99:59:59,999 - [Instructor] Is attendance random? 76 99:59:59,999 --> 99:59:59,999 - [Camilla] No. 77 99:59:59,999 --> 99:59:59,999 - Selection bias. 78 99:59:59,999 --> 99:59:59,999 - [Instructor] Correct. 79 99:59:59,999 --> 99:59:59,999 - [Otto] What? 80 99:59:59,999 --> 99:59:59,999 - [Instructor] The KIPP offers are random so we can be confident 81 99:59:59,999 --> 99:59:59,999 of ceteris parabus, but attendance is not random. 82 99:59:59,999 --> 99:59:59,999 The choice to accept the offer might be due to characteristics 83 99:59:59,999 --> 99:59:59,999 that are related to math performance. 84 99:59:59,999 --> 99:59:59,999 Say, for example, that dedicated parents 85 99:59:59,999 --> 99:59:59,999 are more likely to accept the offer. 86 99:59:59,999 --> 99:59:59,999 Their kids are also more likely to do better in math, 87 99:59:59,999 --> 99:59:59,999 regardless of school. 88 99:59:59,999 --> 99:59:59,999 - [Student] Right. 89 99:59:59,999 --> 99:59:59,999 - [Instructor] IV converts the offer effect 90 99:59:59,999 --> 99:59:59,999 into the effect of KIPP attendance, 91 99:59:59,999 --> 99:59:59,999 adjusting for the fact that some winners go elsewhere 92 99:59:59,999 --> 99:59:59,999 and some losers manage to attend KIPP anyway. 93 99:59:59,999 --> 99:59:59,999 Essentially, IV takes an incomplete randomization 94 99:59:59,999 --> 99:59:59,999 and makes the appropriate adjustments. 95 99:59:59,999 --> 99:59:59,999 How? IV describes a chain reaction. 96 99:59:59,999 --> 99:59:59,999 Why do offers affect achievement? 97 99:59:59,999 --> 99:59:59,999 Probably because they affect charter attendance 98 99:59:59,999 --> 99:59:59,999 and charter attendance improves math scores, 99 99:59:59,999 --> 99:59:59,999 the first link in the chain called the first stage 100 99:59:59,999 --> 99:59:59,999 is the effect of the lottery on charter attendance. 101 99:59:59,999 --> 99:59:59,999 The second stage is the length between attending a charter 102 99:59:59,999 --> 99:59:59,999 and an outcome variable, 103 99:59:59,999 --> 99:59:59,999 in this case, math scores. 104 99:59:59,999 --> 99:59:59,999 The instrumental variable or instrument for short 105 99:59:59,999 --> 99:59:59,999 is the variable that initiates the chain reaction. 106 99:59:59,999 --> 99:59:59,999 The effect of the instrument on the outcome is called 107 99:59:59,999 --> 99:59:59,999 the reduced form. 108 99:59:59,999 --> 99:59:59,999 This chain reaction can be represented mathematically. 109 99:59:59,999 --> 99:59:59,999 We multiply the first stage, the effect of winning 110 99:59:59,999 --> 99:59:59,999 on attendance, by the second stage, 111 99:59:59,999 --> 99:59:59,999 the affect of attendance on scores. 112 99:59:59,999 --> 99:59:59,999 And we get the reduced form, 113 99:59:59,999 --> 99:59:59,999 the effect of winning the lottery on scores. 114 99:59:59,999 --> 99:59:59,999 Reduced form and first stage are observable and easy to compute. 115 99:59:59,999 --> 99:59:59,999 However, the effect of attendance on achievement 116 99:59:59,999 --> 99:59:59,999 is not directly observed. 117 99:59:59,999 --> 99:59:59,999 This is the causal effect we're trying to determine. 118 99:59:59,999 --> 99:59:59,999 Given some important assumptions we'll discuss shortly, 119 99:59:59,999 --> 99:59:59,999 we can find the effect of KIPP attendance 120 99:59:59,999 --> 99:59:59,999 by dividing the reduced form by the first stage. 121 99:59:59,999 --> 99:59:59,999 This will become more clear as we work through an example. 122 99:59:59,999 --> 99:59:59,999 - [Student] Let's do this. 123 99:59:59,999 --> 99:59:59,999 - A quick note on measurement. 124 99:59:59,999 --> 99:59:59,999 We measure achievement using standard deviations, 125 99:59:59,999 --> 99:59:59,999 often denoted by the Greek letter sigma (σ). 126 99:59:59,999 --> 99:59:59,999 One σ is a huge move from around the bottom 15% 127 99:59:59,999 --> 99:59:59,999 to the middle of most achievement distributions. 128 99:59:59,999 --> 99:59:59,999 Even a 1/4 or 1/2 σ difference is big. 129 99:59:59,999 --> 99:59:59,999 - [Instructor] Now we're ready to plug some numbers 130 99:59:59,999 --> 99:59:59,999 into the equation we introduced earlier. 131 99:59:59,999 --> 99:59:59,999 First up, what's the effect 132 99:59:59,999 --> 99:59:59,999 of winning the lottery on math scores?