[Script Info] Title: [Events] Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- [Instructor] The path from cause \Nto effect is dark and dangerous. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,But the weapons \Nof Econometrics are strong. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,[Attack] with fierce \Nand flexible instrumental variables Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,when nature blesses you \Nwith fortuitous random assignment. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,[gong rings] Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Randomized trials are the surest \Npath to ceteris parabus comparisons. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Alas, this powerful tool \Nis often unavailable. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,But sometimes, randomization \Nhappens by accident. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,That's when we turn \Nto instrumental variables, Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,IV for short. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- [Voice whispers] Instrumental \Nvariables. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- [Instructor] Today's lesson \Nis the first of two on IV. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Our first IV lesson begins \Nwith a story of schools. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- [Josh] Charter schools \Nare public schools Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,freed from daily district oversight\Nand teacher union contracts. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,The question of whether charters \Nboost achievement Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,is one of the most important Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,in the history \Nof American education reform. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- The most popular charter schools\Nhave more applicants Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,so the luck of the lottery draw \Ndecides who's offered a seat. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,A lot is at stake for the students \Nvying for their chance. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Waiting for the lottery results \Nbrings up lots of emotions Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,as was captured \Nin the award-winning documentary Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,"Waiting For Superman." Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- [Mother] Don't cry.\NYou're gonna make Mommy cry. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Okay? Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- Do charters really provide\Na better education? Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Critics most definitely say no, Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,arguing that charters enroll \Nbetter students to begin with, Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,smarter or more motivated\Nso differences in later outcomes Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,reflects selection bias. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- [Kamal] Wait, this one seems easy. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,In a lottery, winners \Nare chosen randomly, Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,so just compare winners and losers. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- [Student] Obviously. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- On the right track, Kamal, Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,but charter lotteries \Ndon't force kids into Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,or out of a particular school. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,They randomize offers \Nof a charter seat. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Some kids get lucky. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Some kids don't. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,If we just wanted to know \Nthe effect of charter school offers, Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,we could treat this \Nas a randomized trial. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,But we we're interested \Nin the effects Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,of charter school attendance,\Nnot offers. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,And not everyone \Nwho is offered, accepts. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,IV turns the effect of being offered\Na charter seat into the effect Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,of actually attending \Na charter school. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- [Student] Cool. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- Oh nice. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- Let's look at an example, \Na charter school from Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,the Knowledge Is Power \NProgram, or KIPP for short. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,This KIPP school is in Lynn, Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,a faded industrial town \Non the coast of Massachusetts. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,The school has \Nmore applicants than seats Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and therefore picks its students\Nusing a lottery. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,From 2005 to 2008,\N371 fourth and fifth graders Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,put their names \Nin the KIPP/Lynn lottery, Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,253 students won a seat at KIPP, Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,118 students lost. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,A year later, lottery winners had \Nmuch higher match scores Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,than lottery losers. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,But remember, \Nwe're not trying to figure out Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,whether winning a lottery \Nmakes you better at math. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,We want to know if attending KIPP\Nmakes you better at math. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Of the 253 lottery winners,\Nonly 199 actually went to KIPP. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,The others chose \Na traditional public school. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Similarly of the 118 lottery losers,\Na few actually ended up at KIPP. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,They got an offer later. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,So what was the effect of test scores\Nof actually attending KIPP? Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- [Student] Why can't we just \Nmeasure their math scores? Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- [Instructor] Great question. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Who would you compare them to? Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- [Student] Those who didn't attend. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- [Instructor] Is attendance random? Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- [Camilla] No. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- Selection bias. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- [Instructor] Correct. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- [Otto] What? Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- [Instructor] The KIPP offers \Nare random so we can be confident Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,of ceteris parabus,\Nbut attendance is not random. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,The choice to accept the offer \Nmight be due to characteristics Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,that are related \Nto math performance. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Say, for example, \Nthat dedicated parents Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,are more likely \Nto accept the offer. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Their kids are also more likely \Nto do better in math, Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,regardless of school. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- [Student] Right. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- [Instructor] IV converts \Nthe offer effect Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,into the effect of KIPP attendance, Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,adjusting for the fact \Nthat some winners go elsewhere Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and some losers manage \Nto attend KIPP anyway. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Essentially, IV takes \Nan incomplete randomization Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and makes the appropriate \Nadjustments. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,How? IV describes a chain reaction. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Why do offers affect achievement? Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Probably because they affect \Ncharter attendance Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and charter attendance \Nimproves math scores, Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,the first link in the chain\Ncalled the first stage Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,is the effect of the lottery \Non charter attendance. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,The second stage is the length \Nbetween attending a charter Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and an outcome variable, Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,in this case, math scores. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,The instrumental variable \Nor instrument for short Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,is the variable \Nthat initiates the chain reaction. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,The effect of the instrument \Non the outcome is called Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,the reduced form. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,This chain reaction can be \Nrepresented mathematically. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,We multiply the first stage,\Nthe effect of winning Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,on attendance, by the second stage, Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,the affect of attendance on scores. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,And we get the reduced form, Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,the effect of winning \Nthe lottery on scores. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Reduced form and first stage \Nare observable and easy to compute. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,However, the effect of attendance\Non achievement Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,is not directly observed. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,This is the causal effect\Nwe're trying to determine. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Given some important assumptions \Nwe'll discuss shortly, Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,we can find the effect \Nof KIPP attendance Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,by dividing the reduced form\Nby the first stage. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,This will become more clear \Nas we work through an example. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- [Student] Let's do this. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- A quick note on measurement. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,We measure achievement\Nusing standard deviations, Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,often denoted \Nby the Greek letter sigma (σ). Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,One σ is a huge move\Nfrom around the bottom 15% Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,to the middle of most\Nachievement distributions. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Even a 1/4 or 1/2 σ \Ndifference is big. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- [Instructor] Now we're ready \Nto plug some numbers Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,into the equation \Nwe introduced earlier. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,First up, what's the effect Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,of winning the lottery \Non math scores? Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,KIPP applicant math scores \Nare a third of a standard deviation Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,below the state average in \Nthe year before they apply to KIPP. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,But a year later, lottery winners\Nscore right at the state average Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,while the lottery losers \Nare still well behind. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,With an average score \Naround - 0.36 σ. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,The effect of winning the lottery \Non scores is the difference Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,between the winners' scores\Nand the losers' scores. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Take the winners' \Naverage math scores, Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,subtract the losers' \Naverage math scores, Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and you will have 0.36 σ . Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Next stop: what's the effect \Nof winning the lottery on attendance? Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,In other words, \Nif you win the lottery, Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,how much more likely are you \Nto attend KIPP than if you lose? Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,First, what percentage \Nof lottery winners attend KIPP? Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Divide the number of winners\Nwho attended KIPP Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,by the total number \Nof lottery winners -- that's 78%. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,To find the percentage \Nof lottery losers who attended KIPP, Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,we divide the number of losers \Nwho attended KIPP Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,by the total number \Nof lottery losers -- that's 4%. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Subtract 4 from 78, and we find \Nthat winning the lottery Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,makes you 74% \Nmore likely to attend KIPP. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Now we can find \Nwhat we're really after, Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,the effect of attendance on scores,\Nby dividing 0.36 by 0.74. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Attending KIPP raises math scores Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,by 0.48 standard deviations \Non average. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,That's an awesome achievement game, Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,equal to moving \Nfrom about the bottom Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,to the middle \Nof the achievement distribution. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- [Student] Whoa, half a sig. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- [Instructor] These estimates \Nare for kids opting in Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,to the KIPP lottery,\Nwhose enrollment status Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,is changed by winning. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,That's not necessarily \Na random sample Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,of all children in Lynn. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,So we can't assume \Nwe'd see the same effect Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,for other types of students. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- [Students] Huh. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- But this effect \Non keen for KIPP kids Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,is likely to be a good indicator\Nof the consequences Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,of adding additional charter seats. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- [Student] Cool.\N- [Student] Got it. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,- IV eliminates selection bias \Nbut like all of our tools, Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,the solution builds on a set \Nof assumptions Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,not to be taken for granted. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,First, there must be \Na substantial first stage -- Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,that is the instrumental variable,\Nwinning or losing the lottery, Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,must really change the variable\Nwhose effect we're interested in -- Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,here, KIPP attendance. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,In this case, the first stage \Nis not really in doubt. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Winning the lottery makes \NKIPP attendance much more likely. Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Not all IV stories are like that.