1 00:00:00,400 --> 00:00:03,050 - [woman] The path from cause to effect 2 00:00:03,050 --> 00:00:05,539 is dark and dangerous, 3 00:00:05,539 --> 00:00:08,900 but the weapons of econometrics are strong, 4 00:00:09,211 --> 00:00:13,700 wield differences-in-differences when witnessing parallel trends. 5 00:00:14,450 --> 00:00:16,850 ♪ [music] ♪ 6 00:00:20,100 --> 00:00:21,423 Masters of metrics 7 00:00:21,423 --> 00:00:24,800 look for convincing ceteris paribus comparisons. 8 00:00:25,100 --> 00:00:29,419 The ideal comparison contrasts treatment and control groups 9 00:00:29,419 --> 00:00:30,600 that look similar. 10 00:00:30,600 --> 00:00:34,700 But sometimes this sort of comparability is elusive. 11 00:00:34,700 --> 00:00:36,805 When treatment and control groups 12 00:00:36,805 --> 00:00:40,100 evolve similarly in the absence of treatment, 13 00:00:40,100 --> 00:00:42,307 even if from different starting points, 14 00:00:42,307 --> 00:00:44,900 there's hope for causal inference. 15 00:00:45,600 --> 00:00:48,400 The weapon that exploits parallel evolution, 16 00:00:48,664 --> 00:00:50,886 masters say parallel trends, 17 00:00:50,886 --> 00:00:53,273 is called differences-in-differences... 18 00:00:53,273 --> 00:00:54,400 - [Man whispering] Differences-in-differences 19 00:00:54,400 --> 00:00:56,900 - ...or DD for short. - [Man] Alright. 20 00:00:56,900 --> 00:00:59,987 Let's see how DD can help us understand 21 00:00:59,987 --> 00:01:02,888 one of the most important economic events 22 00:01:02,888 --> 00:01:04,370 in US history. 23 00:01:05,300 --> 00:01:08,300 - [Joshua] Look back with me now at the Great Depression -- 24 00:01:08,800 --> 00:01:12,200 the worst economic catastrophe our country has ever known. 25 00:01:13,100 --> 00:01:16,200 Unemployment hit 25% in 1933 -- 26 00:01:16,600 --> 00:01:19,114 a level not seen before or since. 27 00:01:19,473 --> 00:01:22,100 Millions lost their homes or their land. 28 00:01:22,600 --> 00:01:24,737 Suicide spiked, and hungry families 29 00:01:24,737 --> 00:01:26,766 relied on soup kitchens and bread lines 30 00:01:26,766 --> 00:01:28,155 to keep from starving. 31 00:01:29,400 --> 00:01:34,000 - Economists argue fiercely over the causes of the Great Depression. 32 00:01:34,000 --> 00:01:36,983 Most agree, however, that a key piece of the puzzle 33 00:01:36,983 --> 00:01:39,458 is an epidemic of bank failures. 34 00:01:39,800 --> 00:01:41,900 This was before deposit insurance. 35 00:01:42,100 --> 00:01:46,553 So if your bank went bankrupt, your savings disappeared with it. 36 00:01:46,553 --> 00:01:47,672 - [Cashier] Closing your account? 37 00:01:47,672 --> 00:01:48,892 - [Customer] Yes, sir. I'm closing my account. 38 00:01:48,892 --> 00:01:50,726 - I wouldn't leave a nickel in this bank. 39 00:01:52,600 --> 00:01:56,100 - Faced with a banking crisis, the Central Bank has a choice: 40 00:01:56,400 --> 00:01:58,524 lend freely to troubled banks 41 00:01:58,524 --> 00:02:01,100 or stand aside and refuse to lend. 42 00:02:01,500 --> 00:02:05,440 Lending freely to banks in trouble is called "easy money." 43 00:02:05,440 --> 00:02:08,100 Refusing to lend is called "tight money." 44 00:02:10,200 --> 00:02:12,872 - [Joshua] Monetarist masters Milton Friedman and Anna Schwartz 45 00:02:12,872 --> 00:02:14,882 famously called the Great Depression 46 00:02:14,882 --> 00:02:16,350 the "Great Contraction," 47 00:02:16,800 --> 00:02:18,262 accusing the Federal Reserve 48 00:02:18,262 --> 00:02:21,200 of inflicting a misguided policy of tight money 49 00:02:21,200 --> 00:02:24,000 on the nation's teetering financial institutions. 50 00:02:24,400 --> 00:02:25,873 They argued that easy money 51 00:02:25,873 --> 00:02:27,985 would have kept many banks in business, 52 00:02:27,985 --> 00:02:29,700 shortening the Great Depression. 53 00:02:30,400 --> 00:02:32,239 But others disagree! 54 00:02:32,239 --> 00:02:33,769 If banks are insolvent 55 00:02:33,769 --> 00:02:35,954 because of unwise lending decisions, 56 00:02:35,954 --> 00:02:38,900 then bailouts just encourage more foolishness. 57 00:02:39,600 --> 00:02:42,965 Economists called this problem "moral hazard." 58 00:02:42,965 --> 00:02:46,100 The debate over bailouts in moral hazard continues today. 59 00:02:46,500 --> 00:02:48,599 Should financial behemoth Lehman Brothers 60 00:02:48,599 --> 00:02:51,500 had been allowed to fail on the eve of the Great Recession, 61 00:02:52,000 --> 00:02:54,703 in an ideal world, we'd answer this question 62 00:02:54,703 --> 00:02:58,400 by applying different Fed policies to randomly selected regions. 63 00:02:59,000 --> 00:03:00,250 But we can still learn a lot 64 00:03:00,250 --> 00:03:02,119 by using differences-in-differences 65 00:03:02,119 --> 00:03:06,300 to compare trends across areas with different monetary policies. 66 00:03:10,900 --> 00:03:12,522 How's that even possible? 67 00:03:12,522 --> 00:03:15,779 Don't the same Fed policies apply to all banks in the US? 68 00:03:15,779 --> 00:03:17,400 - Yeah. - Good question. 69 00:03:17,700 --> 00:03:21,484 The Federal Reserve System is divided into 12 districts, 70 00:03:21,484 --> 00:03:24,301 each headed by a regional bank. 71 00:03:24,301 --> 00:03:27,467 Today, Fed policy is set at the national level. 72 00:03:27,467 --> 00:03:32,200 But in the 1930s, regional Feds could do pretty much as they liked. 73 00:03:32,200 --> 00:03:33,276 Ah, interesting. 74 00:03:33,276 --> 00:03:35,500 And here's what's so awesome about that. 75 00:03:35,500 --> 00:03:39,302 In 1930, the Atlanta Fed, running the 6th District, 76 00:03:39,302 --> 00:03:41,473 followed an easy money policy, 77 00:03:41,473 --> 00:03:45,400 sending wheelbarrows of cash to rescue insolvent institutions, 78 00:03:45,900 --> 00:03:48,816 The St. Louis Fed, running the 8th District, 79 00:03:48,816 --> 00:03:50,668 followed a tight money policy. 80 00:03:51,100 --> 00:03:53,900 "Let fail the foolish!" they said in St. Louis. 81 00:03:54,300 --> 00:03:58,701 And so a natural experiment in monetary policy was born. 82 00:03:58,701 --> 00:04:02,000 Even better, this is a within-state experiment. 83 00:04:02,000 --> 00:04:04,347 The border between the 6th and the 8th districts 84 00:04:04,347 --> 00:04:06,762 ran smack through the middle of Mississippi. 85 00:04:07,300 --> 00:04:09,321 So northern Mississippi had tight money, 86 00:04:09,321 --> 00:04:11,870 while southern Mississippi had easy money, 87 00:04:11,870 --> 00:04:15,200 but under the same state laws in banking regulations in both. 88 00:04:15,770 --> 00:04:16,853 The treatment group 89 00:04:16,853 --> 00:04:20,151 is the District 6th part of Mississippi, 90 00:04:20,151 --> 00:04:23,100 which had access to easy money during the crisis. 91 00:04:23,800 --> 00:04:25,091 The control group 92 00:04:25,091 --> 00:04:27,800 is the District 8th part of Mississippi, 93 00:04:27,800 --> 00:04:30,225 which had tight money during the crisis. 94 00:04:31,300 --> 00:04:34,040 The key year in our natural experiment 95 00:04:34,040 --> 00:04:35,400 was 1930, 96 00:04:35,800 --> 00:04:37,563 Caldwell & Company, 97 00:04:37,563 --> 00:04:40,377 a massive financial empire in the South 98 00:04:40,377 --> 00:04:41,987 came crashing down. 99 00:04:42,700 --> 00:04:46,000 Banking is a business built on confidence and trust. 100 00:04:46,500 --> 00:04:49,151 The Caldwell meltdown caused a panic 101 00:04:49,151 --> 00:04:53,049 that led to a widespread bank run all at once. 102 00:04:53,049 --> 00:04:55,114 Depositors wanted their money back, 103 00:04:55,114 --> 00:04:58,200 causing banks to go bankrupt and shut their doors. 104 00:05:01,000 --> 00:05:03,321 We'll use differences-in-differences 105 00:05:03,321 --> 00:05:06,614 to measure the effect of contrasting monetary policies 106 00:05:06,614 --> 00:05:09,164 in response to the Caldwell crisis. 107 00:05:12,500 --> 00:05:16,439 This figure plots the number of banks in Mississippi by year, 108 00:05:16,439 --> 00:05:19,273 for the 8th and 6th districts. 109 00:05:19,273 --> 00:05:24,200 Let's start in 1929 -- a year before the Caldwell crash. 110 00:05:24,200 --> 00:05:27,646 There are 169 banks open in the 8th, 111 00:05:27,646 --> 00:05:31,366 and 141 banks open in the 6th. 112 00:05:31,366 --> 00:05:32,784 Over the next year, 113 00:05:32,784 --> 00:05:37,000 we see a similar handful of banks fail, in both districts. 114 00:05:37,400 --> 00:05:40,135 The change in the number of banks in operation 115 00:05:40,135 --> 00:05:42,168 is remarkably similar. 116 00:05:42,168 --> 00:05:44,600 That's what parallel trends look like. 117 00:05:45,500 --> 00:05:48,783 In November 1930, Caldwell crashes, 118 00:05:48,783 --> 00:05:50,400 and the panic begins. 119 00:05:51,200 --> 00:05:53,917 Banks failed frequently in the 8th district, 120 00:05:53,917 --> 00:05:55,300 which had tight money. 121 00:05:55,800 --> 00:05:58,712 But the decline is slower in the 6th District, 122 00:05:58,712 --> 00:06:00,247 which had easy money. 123 00:06:00,800 --> 00:06:03,000 The diverging trends in this period 124 00:06:03,000 --> 00:06:06,602 might be attributable to easy versus tight money. 125 00:06:06,602 --> 00:06:12,020 In July of 1931, the 8th district abandons tight money, 126 00:06:12,020 --> 00:06:14,420 so now both districts are easy. 127 00:06:14,700 --> 00:06:16,900 Parallel trends are restored. 128 00:06:17,300 --> 00:06:19,229 In a counterfactual world, 129 00:06:19,229 --> 00:06:22,420 where the 6th district follows a tight money policy, 130 00:06:22,420 --> 00:06:23,800 what might have happened? 131 00:06:24,201 --> 00:06:28,547 If we extrapolate the trend of the 8th district to the 6th, 132 00:06:28,547 --> 00:06:29,900 it would look like this. 133 00:06:30,290 --> 00:06:33,095 So the treatment effective easy money 134 00:06:33,095 --> 00:06:36,646 is how much the 6th district deviated from the path 135 00:06:36,646 --> 00:06:38,900 implied by the 8th district trend. 136 00:06:41,244 --> 00:06:44,475 How many banks did the easy money treatment save? 137 00:06:44,475 --> 00:06:49,021 This table reports data for the treatment group, District 6, 138 00:06:49,021 --> 00:06:50,162 in the first row 139 00:06:50,162 --> 00:06:54,237 and data for the control group, District 8, in the second row. 140 00:06:54,237 --> 00:06:57,494 The first column shows the number of banks in business 141 00:06:57,494 --> 00:07:00,613 before the crisis began in 1930. 142 00:07:00,613 --> 00:07:03,976 The second column shows 1931. 143 00:07:03,976 --> 00:07:05,607 This is the key period 144 00:07:05,607 --> 00:07:08,663 when each district had differing monetary policies 145 00:07:08,663 --> 00:07:10,237 during the crisis. 146 00:07:10,237 --> 00:07:14,347 The rightmost column reports changes within the district. 147 00:07:14,347 --> 00:07:20,456 District 6 lost 14 banks, while District 8 lost 33. 148 00:07:20,456 --> 00:07:23,973 The mathematical formula for the treatment effect is simple. 149 00:07:24,400 --> 00:07:28,700 We subtract the change in banks in operation, in the 8th District, 150 00:07:28,900 --> 00:07:32,200 from the change in banks in operation in the 6th. 151 00:07:32,800 --> 00:07:35,700 Hence, the name differences-in-differences. 152 00:07:37,000 --> 00:07:42,500 -14 minus -33 equals 19. 153 00:07:42,500 --> 00:07:46,800 We estimate that 19 banks were saved by easy money. 154 00:07:47,300 --> 00:07:50,730 In practice, tables and figures like those shown here 155 00:07:50,730 --> 00:07:52,912 are the beginning rather than the end 156 00:07:52,912 --> 00:07:54,453 of a DD analysis, 157 00:07:55,400 --> 00:07:57,100 The problem of how to gauge 158 00:07:57,100 --> 00:08:00,167 the statistical significance of DD estimates 159 00:08:00,167 --> 00:08:02,394 turns out to be exceedingly tricky, 160 00:08:02,400 --> 00:08:05,600 and a regression is typically part of the solution. 161 00:08:09,200 --> 00:08:12,336 The key assumption behind a valid DD analysis 162 00:08:12,336 --> 00:08:15,369 is that of parallel trends. 163 00:08:15,369 --> 00:08:17,842 Recall the principle of ceteris paribus -- 164 00:08:17,842 --> 00:08:21,700 our ideal comparison would have the two districts experience 165 00:08:21,700 --> 00:08:24,023 and identical business environment, 166 00:08:24,023 --> 00:08:25,997 except for one factor: 167 00:08:25,997 --> 00:08:27,916 easy or tight money. 168 00:08:29,200 --> 00:08:32,475 Both districts would have identical types of customers 169 00:08:32,475 --> 00:08:35,300 who would go bankrupt at exactly the same rate. 170 00:08:35,700 --> 00:08:38,600 The skill of their employees would be equal, and so on. 171 00:08:39,200 --> 00:08:43,400 Perfect ceteris paribus comparisons would allow us to clearly see 172 00:08:43,600 --> 00:08:46,594 the causal effect of different Fed policies. 173 00:08:46,594 --> 00:08:49,019 In this case, that's not possible. 174 00:08:49,019 --> 00:08:53,600 But the idea of parallel trends is based on a similar concept. 175 00:08:53,600 --> 00:08:57,364 If we see that the two regions experienced similar trends 176 00:08:57,364 --> 00:08:59,814 in the number of banks over time, 177 00:08:59,814 --> 00:09:01,294 in the absence of treatment, 178 00:09:01,294 --> 00:09:04,407 we can assume they are good comparisons. 179 00:09:04,407 --> 00:09:07,434 We see that the two districts move in parallel, 180 00:09:07,434 --> 00:09:09,800 both before the crisis and after. 181 00:09:10,100 --> 00:09:12,400 when they have the same Fed policy. 182 00:09:13,100 --> 00:09:16,028 The only time the districts behaved differently 183 00:09:16,028 --> 00:09:18,300 is when the Fed policy is different. 184 00:09:19,400 --> 00:09:20,599 In view of this, 185 00:09:20,599 --> 00:09:24,268 Fed policy is a likely cause of diverging trends 186 00:09:24,268 --> 00:09:26,700 from 1930 to 1931. 187 00:09:27,800 --> 00:09:30,106 But we should also check for other changes 188 00:09:30,106 --> 00:09:31,703 unique to northern Mississippi. 189 00:09:31,703 --> 00:09:33,200 - Huh? - What do you mean? 190 00:09:33,500 --> 00:09:35,396 Imagine that bad tornadoes 191 00:09:35,396 --> 00:09:39,100 hit northern but not southern Mississippi in 1930. 192 00:09:39,600 --> 00:09:41,950 These tornadoes devastate farms, 193 00:09:41,950 --> 00:09:44,300 causing farmers to default on loans, 194 00:09:44,600 --> 00:09:46,800 which drives their banks out of business. 195 00:09:47,400 --> 00:09:49,438 Then the 6th and 8th districts 196 00:09:49,438 --> 00:09:52,272 would differ in not one but two ways: 197 00:09:52,700 --> 00:09:55,047 Fed policy and weather. 198 00:09:55,047 --> 00:09:58,219 And we'd have trouble identifying Fed policy 199 00:09:58,219 --> 00:10:01,590 as the causal factor behind increased bank failures 200 00:10:01,590 --> 00:10:02,600 in the 8th. 201 00:10:02,600 --> 00:10:04,248 - Ceteris is not paribus. 202 00:10:07,200 --> 00:10:09,802 DD credibility lives or dies with the claim 203 00:10:09,802 --> 00:10:13,862 that the only reason northern Mississippi was special in 1930 204 00:10:13,862 --> 00:10:16,161 is differing regional Fed policy. 205 00:10:16,600 --> 00:10:21,277 We're in DD heaven with strong, visual evidence of parallel trend. 206 00:10:21,277 --> 00:10:25,549 In general, the first step in evaluating whether to use DD 207 00:10:25,549 --> 00:10:30,200 is usually this type of visual confirmation of parallel trends 208 00:10:30,200 --> 00:10:31,700 outside of the period, 209 00:10:31,700 --> 00:10:35,011 when we expect to see a treatment effect. 210 00:10:35,011 --> 00:10:36,985 The treatment in our example 211 00:10:36,985 --> 00:10:39,835 is easy money in the face of bank failures. 212 00:10:40,500 --> 00:10:45,000 Metrics masters use DD to explore effects of many policies 213 00:10:45,800 --> 00:10:47,900 like the minimum legal drinking age, 214 00:10:48,500 --> 00:10:52,200 and environmental changes, like access to clean water. 215 00:10:52,900 --> 00:10:54,200 In our next video. 216 00:10:54,500 --> 00:10:57,035 We'll see an example of how regression is used 217 00:10:57,035 --> 00:10:59,200 to implement a DD approach. 218 00:11:00,800 --> 00:11:02,183 - [Narrator] Are you a teacher? 219 00:11:02,183 --> 00:11:05,788 Click to explore ways to use these videos in class. 220 00:11:05,788 --> 00:11:08,847 If you're a learner, make sure this video sticks 221 00:11:08,847 --> 00:11:11,200 by taking a few quick practice questions. 222 00:11:11,600 --> 00:11:14,200 Or if you're ready. click for the next video. 223 00:11:14,600 --> 00:11:17,189 You can also check out MRU's website 224 00:11:17,189 --> 00:11:20,173 for more courses, teacher resources, and more.