Return to Video

Introduction to Instrumental Variables (IV)

  • 0:00 - 0:05
    - [Instructor] The path from cause
    to effect is dark and dangerous.
  • 0:05 - 0:08
    But the weapons
    of Econometrics are strong.
  • 0:08 - 0:12
    Attack with fierce
    and flexible instrumental variables
  • 0:12 - 0:16
    when nature blesses you
    with fortuitous random assignment.
  • 0:19 - 0:21
    [gong rings]
  • 0:24 - 0:29
    Randomized trials are the surest
    path to ceteris parabus comparisons.
  • 0:29 - 0:33
    Alas, this powerful tool
    is often unavailable.
  • 0:33 - 0:37
    But sometimes, randomization
    happens by accident.
  • 0:37 - 0:41
    That's when we turn
    to instrumental variables --
  • 0:41 - 0:42
    IV for short.
  • 0:42 - 0:45
    - [Voice whispers] Instrumental
    variables.
  • 0:45 - 0:48
    - [Instructor] Today's lesson
    is the first of two on IV.
  • 0:49 - 0:53
    Our first IV lesson begins
    with a story of schools.
  • 0:53 - 0:54
    [school bell rings]
  • 0:54 - 0:56
    - [Josh] Charter schools
    are public schools
  • 0:56 - 1:00
    freed from daily district oversight
    and teacher union contracts.
  • 1:01 - 1:04
    The question of whether charters
    boost achievement
  • 1:04 - 1:05
    is one of the most important
  • 1:05 - 1:08
    in the history
    of American education reform.
  • 1:08 - 1:13
    - The most popular charter schools
    have more applicants than seats
  • 1:13 - 1:16
    so the luck of a lottery draw
    decides who's offered a seat.
  • 1:17 - 1:21
    A lot is at stake for the students
    vying for their chance,
  • 1:21 - 1:25
    and waiting for the lottery results
    brings up lots of emotions
  • 1:25 - 1:28
    as was captured
    in the award-winning documentary
  • 1:28 - 1:30
    "Waiting For Superman."
  • 1:30 - 1:33
    - [Mother] Don't cry. You're gonna
    make Mommy cry. Okay?
  • 1:37 - 1:41
    - Do charters really provide
    a better education?
  • 1:41 - 1:43
    Critics most definitely say no,
  • 1:43 - 1:46
    arguing that charters enroll
    better students to begin with,
  • 1:46 - 1:50
    smarter or more motivated,
    so differences in later outcomes
  • 1:50 - 1:52
    reflects selection bias.
  • 1:53 - 1:55
    - [Kamal] Wait, this one seems easy.
  • 1:55 - 1:57
    In a lottery, winners
    are chosen randomly,
  • 1:57 - 2:00
    so just compare winners and losers.
    - [Student] Obviously.
  • 2:00 - 2:02
    - On the right track, Kamal,
  • 2:02 - 2:05
    but charter lotteries
    don't force kids into
  • 2:05 - 2:08
    or out of a particular school.
  • 2:08 - 2:11
    They randomize offers
    of a charter seat.
  • 2:12 - 2:13
    Some kids get lucky.
  • 2:13 - 2:15
    Some kids don't.
  • 2:15 - 2:19
    If we just wanted to know
    the effect of charter school offers,
  • 2:19 - 2:22
    we could treat this
    as a randomized trial.
  • 2:23 - 2:25
    But we we're interested
    in the effects
  • 2:25 - 2:28
    of charter school attendance,
    not offers.
  • 2:29 - 2:32
    And not everyone
    who is offered, accepts.
  • 2:32 - 2:37
    IV turns the effect of being offered
    a charter seat into the effect
  • 2:37 - 2:40
    of actually attending
    a charter school.
  • 2:40 - 2:42
    - [Student] Cool.
    - Oh nice.
  • 2:46 - 2:49
    - Let's look at an example,
    a charter school from
  • 2:49 - 2:52
    the Knowledge Is Power
    Program, or KIPP for short.
  • 2:53 - 2:55
    This KIPP school is in Lynn,
  • 2:55 - 2:59
    a faded industrial town
    on the coast of Massachusetts.
  • 2:59 - 3:02
    The school has
    more applicants than seats
  • 3:02 - 3:06
    and therefore picks its students
    using a lottery.
  • 3:06 - 3:12
    From 2005 to 2008,
    371 fourth and fifth graders
  • 3:12 - 3:15
    put their names
    in the KIPP Lynn lottery,
  • 3:15 - 3:19
    253 students won a seat at KIPP,
  • 3:19 - 3:22
    118 students lost.
  • 3:22 - 3:26
    A year later, lottery winners had
    much higher math scores
  • 3:26 - 3:28
    than lottery losers.
  • 3:28 - 3:30
    But remember,
    we're not trying to figure out
  • 3:30 - 3:34
    whether winning a lottery
    makes you better at math.
  • 3:34 - 3:38
    We want to know if attending KIPP
    makes you better at math.
  • 3:39 - 3:46
    Of the 253 lottery winners,
    only 199 actually went to KIPP.
  • 3:46 - 3:49
    The others chose
    a traditional public school.
  • 3:50 - 3:55
    Similarly of the 118 lottery losers,
    a few actually ended up at KIPP.
  • 3:56 - 3:57
    They got an offer later.
  • 3:57 - 4:02
    So what was the effect on test scores
    of actually attending KIPP?
  • 4:03 - 4:05
    - [Kamal] Why can't we just
    measure their math scores?
  • 4:05 - 4:07
    - [Instructor] Great question.
  • 4:07 - 4:09
    Who would you compare them to?
  • 4:09 - 4:11
    - [Kamal] Those who didn't attend.
  • 4:11 - 4:13
    - [Instructor] Is attendance random?
  • 4:14 - 4:15
    - [Camilla] No.
  • 4:15 - 4:16
    - Selection bias.
  • 4:16 - 4:18
    - [Instructor] Correct.
    - [Otto] What?
  • 4:18 - 4:22
    - [Instructor] The KIPP offers
    are random so we can be confident
  • 4:22 - 4:26
    of ceteris parabus,
    but attendance is not random.
  • 4:27 - 4:31
    The choice to accept the offer
    might be due to characteristics
  • 4:31 - 4:33
    that are related
    to math performance --
  • 4:33 - 4:36
    say, for example,
    that dedicated parents
  • 4:36 - 4:39
    are more likely
    to accept the offer.
  • 4:39 - 4:43
    Their kids are also more likely
    to do better in math,
  • 4:43 - 4:44
    regardless of school.
  • 4:44 - 4:45
    - [Student] Right.
  • 4:45 - 4:48
    - [Instructor] IV converts
    the offer effect
  • 4:48 - 4:51
    into the effect of KIPP attendance,
  • 4:51 - 4:53
    adjusting for the fact
    that some winners go elsewhere
  • 4:53 - 4:57
    and some losers manage
    to attend KIPP anyway.
  • 4:57 - 5:01
    Essentially, IV takes
    an incomplete randomization
  • 5:01 - 5:03
    and makes the appropriate
    adjustments.
  • 5:04 - 5:07
    How? IV describes a chain reaction.
  • 5:07 - 5:10
    Why do offers affect achievement?
  • 5:10 - 5:13
    Probably because they affect
    charter attendance
  • 5:13 - 5:17
    and charter attendance
    improves math scores,
  • 5:17 - 5:20
    the first link in the chain
    called the first stage
  • 5:20 - 5:24
    is the effect of the lottery
    on charter attendance.
  • 5:24 - 5:28
    The second stage is the length
    between attending a charter
  • 5:28 - 5:30
    and an outcome variable,
  • 5:30 - 5:32
    in this case, math scores.
  • 5:33 - 5:36
    The instrumental variable,
    or instrument for short,
  • 5:36 - 5:40
    is the variable
    that initiates the chain reaction.
  • 5:41 - 5:45
    The effect of the instrument
    on the outcome is called
  • 5:45 - 5:47
    the reduced form.
  • 5:48 - 5:52
    This chain reaction can be
    represented mathematically.
  • 5:52 - 5:55
    We multiply the first stage,
    the effect of winning
  • 5:55 - 5:58
    on attendance, by the second stage,
  • 5:58 - 6:01
    the effect of attendance on scores.
  • 6:01 - 6:03
    And we get the reduced form,
  • 6:03 - 6:06
    the effect of winning
    the lottery on scores.
  • 6:07 - 6:12
    The reduced form and first stage
    are observable and easy to compute.
  • 6:12 - 6:15
    However, the effect of attendance
    on achievement
  • 6:15 - 6:17
    is not directly observed.
  • 6:17 - 6:20
    This is the causal effect
    we're trying to determine.
  • 6:21 - 6:24
    Given some important assumptions
    we'll discuss shortly,
  • 6:24 - 6:26
    we can find the effect
    of KIPP attendance
  • 6:26 - 6:29
    by dividing the reduced form
    by the first stage.
  • 6:29 - 6:33
    This will become more clear
    as we work through an example.
  • 6:33 - 6:34
    - [Student] Let's do this.
  • 6:37 - 6:39
    - A quick note on measurement.
  • 6:39 - 6:42
    We measure achievement
    using standard deviations,
  • 6:42 - 6:45
    often denoted
    by the Greek letter sigma (σ).
  • 6:45 - 6:49
    One σ is a huge move
    from around the bottom 15%
  • 6:49 - 6:52
    to the middle of most
    achievement distributions.
  • 6:52 - 6:55
    Even a ¼ or ½ σ difference is big.
  • 6:56 - 6:58
    - [Instructor] Now we're ready
    to plug some numbers
  • 6:58 - 7:01
    into the equation
    we introduced earlier.
  • 7:02 - 7:03
    First up, what's the effect
  • 7:03 - 7:06
    of winning the lottery
    on math scores?
  • 7:06 - 7:10
    KIPP applicants' math scores
    are a third of a standard deviation
  • 7:11 - 7:14
    below the state average in
    the year before they apply to KIPP.
  • 7:14 - 7:18
    But a year later, lottery winners
    score right at the state average
  • 7:18 - 7:21
    while the lottery losers
    are still well behind
  • 7:21 - 7:25
    with an average score
    around - 0.36 σ.
  • 7:26 - 7:30
    The effect of winning the lottery
    on scores is the difference
  • 7:30 - 7:33
    between the winners' scores
    and the losers' scores.
  • 7:33 - 7:36
    Take the winners'
    average math scores,
  • 7:36 - 7:38
    subtract the losers'
    average math scores,
  • 7:38 - 7:42
    and you will have 0.36 σ .
  • 7:42 - 7:47
    Next up: what's the effect
    of winning the lottery on attendance?
  • 7:47 - 7:49
    In other words,
    if you win the lottery,
  • 7:49 - 7:53
    how much more likely are you
    to attend KIPP than if you lose?
  • 7:54 - 7:58
    First, what percentage
    of lottery winners attend KIPP?
  • 7:58 - 8:01
    Divide the number of winners
    who attended KIPP
  • 8:01 - 8:05
    by the total number
    of lottery winners -- that's 78%.
  • 8:06 - 8:09
    To find the percentage
    of lottery losers who attended KIPP,
  • 8:09 - 8:12
    we divide the number of losers
    who attended KIPP
  • 8:12 - 8:17
    by the total number
    of lottery losers -- that's 4%.
  • 8:17 - 8:21
    Subtract 4 from 78, and we find
    that winning the lottery
  • 8:21 - 8:26
    makes you 74%
    more likely to attend KIPP.
  • 8:26 - 8:28
    Now we can find
    what we're really after,
  • 8:28 - 8:35
    the effect of attendance on scores,
    by dividing 0.36 by 0.74.
  • 8:35 - 8:38
    Attending KIPP raises math scores
  • 8:38 - 8:42
    by 0.48 standard deviations
    on average.
  • 8:42 - 8:45
    That's an awesome achievement gain,
  • 8:45 - 8:47
    equal to moving
    from about the bottom third
  • 8:47 - 8:50
    to the middle
    of the achievement distribution.
  • 8:50 - 8:51
    - [Student] Whoa, half a sig.
  • 8:51 - 8:54
    - [Instructor] These estimates
    are for kids opting in
  • 8:54 - 8:56
    to the KIPP lottery,
    whose enrollment status
  • 8:56 - 8:58
    is changed by winning.
  • 8:58 - 9:01
    That's not necessarily
    a random sample
  • 9:01 - 9:02
    of all children in Lynn.
  • 9:03 - 9:05
    So we can't assume
    we'd see the same effect
  • 9:05 - 9:07
    for other types of students.
    - [Student] Huh.
  • 9:07 - 9:10
    - But this effect
    on keen for KIPP kids
  • 9:10 - 9:13
    is likely to be a good indicator
    of the consequences
  • 9:13 - 9:16
    of adding additional charter seats.
  • 9:16 - 9:17
    - [Student] Cool.
    - [Student] Got it.
  • 9:20 - 9:23
    - IV eliminates selection bias,
    but like all of our tools,
  • 9:23 - 9:26
    the solution builds on a set
    of assumptions
  • 9:26 - 9:28
    not to be taken for granted.
  • 9:28 - 9:31
    First, there must be
    a substantial first stage --
  • 9:31 - 9:35
    that is the instrumental variable,
    winning or losing the lottery,
  • 9:35 - 9:39
    must really change the variable
    whose effect we're interested in --
  • 9:39 - 9:41
    here, KIPP attendance.
  • 9:41 - 9:44
    In this case, the first stage
    is not really in doubt.
  • 9:44 - 9:48
    Winning the lottery makes
    KIPP attendance much more likely.
  • 9:48 - 9:51
    Not all IV stories are like that.
  • 9:51 - 9:54
    Second, the instrument
    must be as good
  • 9:54 - 9:57
    as randomly assigned,
    meaning lottery winners and losers
  • 9:57 - 9:59
    have similar characteristics.
  • 9:59 - 10:02
    This is the independence assumption.
  • 10:02 - 10:06
    Of course, KIPP lottery wins
    really are randomly assigned.
  • 10:06 - 10:09
    Still, we should check for balance
    and confirm that winners and losers
  • 10:09 - 10:11
    have similar family backgrounds,
  • 10:11 - 10:13
    similar aptitudes and so on.
  • 10:14 - 10:17
    In essence, we're checking
    to ensure KIPP lotteries are fair
  • 10:17 - 10:20
    with no group of applicants
    suspiciously likely to win.
  • 10:21 - 10:24
    Finally, we require
    the instrument change outcomes
  • 10:24 - 10:26
    solely through
    the variable of interest,
  • 10:26 - 10:28
    in this case, attending KIPP.
  • 10:28 - 10:31
    This assumption is called
    the exclusion restriction.
  • 10:33 - 10:38
    - IV only works if you can satisfy
    these three assumptions.
  • 10:38 - 10:40
    - I don't understand
    the exclusion restriction.
  • 10:41 - 10:44
    How could winning the lottery
    affect math scores
  • 10:44 - 10:45
    other than by attending KIPP?
  • 10:45 - 10:47
    - [Student] Yeah.
    - [Instructor] Great question.
  • 10:47 - 10:51
    Suppose lottery winners
    are just thrilled to win,
  • 10:51 - 10:55
    and this happiness motivates them
    to study more and learn more math,
  • 10:55 - 10:57
    regardless of where
    they go to school.
  • 10:57 - 11:00
    This would violate
    the exclusion restriction
  • 11:00 - 11:04
    because the motivational effect
    of winning is a second channel
  • 11:04 - 11:07
    whereby lotteries
    might affect test scores.
  • 11:07 - 11:10
    While it's hard
    to rule this out entirely,
  • 11:10 - 11:13
    there's no evidence
    of any alternative channels
  • 11:13 - 11:14
    in the KIPP study.
  • 11:18 - 11:21
    - IV solves the problem
    of selection bias
  • 11:21 - 11:25
    in scenarios like the KIPP lottery
    where treatment offers are random
  • 11:25 - 11:27
    but some of those offered opt out.
  • 11:28 - 11:32
    This sort of intentional
    yet incomplete random assignment
  • 11:32 - 11:33
    is surprisingly common.
  • 11:33 - 11:36
    Even randomized clinical trials
    have this feature.
  • 11:37 - 11:40
    IV solves the problem
    of non-random take up
  • 11:40 - 11:43
    in lotteries or clinical research.
  • 11:43 - 11:47
    But lotteries are not the only
    source of compelling instruments.
  • 11:47 - 11:50
    Many causal questions can be
    addressed by naturally occurring
  • 11:50 - 11:54
    as good as randomly
    assigned variation.
  • 11:55 - 11:57
    Here's a causal question for you --
  • 11:57 - 12:00
    do women who have children early
    in their careers suffer
  • 12:00 - 12:03
    a substantial earnings penalty
    as a result?
  • 12:03 - 12:05
    After all, women earn less than men.
  • 12:06 - 12:09
    We could, of course, simply
    compare the earnings of women
  • 12:09 - 12:11
    with more and fewer children.
  • 12:11 - 12:14
    But such comparisons are fraught
    with selection bias.
  • 12:15 - 12:19
    If only we could randomly assign
    babies to different households.
  • 12:19 - 12:22
    Yeah, right,
    sounds pretty fanciful.
  • 12:22 - 12:27
    Our next IV story -- fantastic
    and not fanciful --
  • 12:27 - 12:30
    illustrates an amazing,
    naturally-occurring instrument
  • 12:30 - 12:32
    for family size.
  • 12:33 - 12:34
    ♪ [music] ♪
  • 12:35 - 12:38
    - [Instructor] You're on your way
    to mastering Econometrics.
  • 12:38 - 12:40
    Make sure this video sticks
  • 12:40 - 12:43
    by taking a few
    quick practice questions.
  • 12:43 - 12:46
    Or, if you're ready,
    click for the next video.
  • 12:47 - 12:50
    You can also check out
    MRU's website for more courses,
  • 12:50 - 12:52
    teacher resources, and more.
  • 12:52 - 12:54
    ♪ [music] ♪
Title:
Introduction to Instrumental Variables (IV)
Description:

more » « less
Video Language:
English
Team:
Marginal Revolution University
Project:
Mastering Econometrics
Duration:
12:57

English subtitles

Revisions Compare revisions