WEBVTT 00:00:00.100 --> 00:00:02.050 ♪ (música) ♪ 00:00:03.620 --> 00:00:05.700 - [Narradora] Bienvenidos a Nobel Conversations. 00:00:07.000 --> 00:00:08.521 En este episodio, Josh Angrist y Guido Imbens 00:00:08.521 --> 00:00:09.282 están con Isaiah Andrews para discutir y discrepar 00:00:09.282 --> 00:00:09.662 sobre el papel del aprendizaje automático 00:00:09.662 --> 00:00:10.043 en la econometría aplicada. 00:00:10.043 --> 00:00:11.859 - [Isaiah] Entonces, por supuesto que hay muchos temas 00:00:11.859 --> 00:00:12.767 en los que ustedes están muy de acuerdo, pero me gustaría pasar a uno 00:00:12.767 --> 00:00:13.675 en el que tal vez tengan algunas discrepancias. 00:00:13.675 --> 00:00:15.127 Me gustaría escuchar algunas de sus opiniones 00:00:15.127 --> 00:00:15.853 sobre el aprendizaje automático 00:00:15.853 --> 00:00:16.580 y el papel que está desempeñando y va a desempeñar en la economía. 00:00:17.897 --> 00:00:18.833 - [Guido] He consultado algunos datos como el propietario----------------------. 00:00:18.833 --> 00:00:19.301 Vemos que no hay ningún documento publicado allí. 00:00:19.301 --> 00:00:19.535 Se hizo un experimento sobre algún algoritmo de búsqueda 00:00:19.535 --> 00:00:19.652 y la cuestión era... 00:00:19.652 --> 00:00:19.710 se trataba de clasificar cosas y cambiar la clasificación. 00:00:19.710 --> 00:00:19.739 Y estaba más o menos claro que iba a haber mucha heterogeneidad. 00:00:19.739 --> 00:00:19.754 Si buscas, digamos, una foto de Britney Spears -- 00:00:19.754 --> 00:00:19.761 realmente no importa dónde la clasifiques----------------------------------- porque vas a encontrar lo que estás buscando, 00:00:19.761 --> 00:00:19.765 ya sea que la pongas en la primera, segunda o tercera posición del ranking. 00:00:19.765 --> 00:00:19.767 Pero si estás buscando el mejor libro de econometría, 00:00:19.767 --> 00:00:19.768 si pones tu libro en primer lugar o en el décimo, 00:00:19.768 --> 00:00:19.768 eso va a suponer una gran diferencia 00:00:19.768 --> 00:00:19.768 en la frecuencia con la que la gente va a hacer clic en él. 00:00:19.768 --> 00:00:19.769 Así que ahí... 00:00:22.313 --> 00:00:23.276 [Josh] ¿Por qué necesito el aprendizaje automático para descubrir eso? 00:00:23.276 --> 00:00:24.240 Parece que --porque puedo descubrirlo simplemente.----------------------- 00:00:24.240 --> 00:00:24.984 - [Guido] En general -- 00:00:24.984 --> 00:00:25.356 - [Josh] Había un montón de posibles... 00:00:25.356 --> 00:00:25.542 - Quieres pensar en que hay un montón de características de los artículos, 00:00:25.542 --> 00:00:25.635 que quieres entender lo que impulsa la heterogeneidad en el efecto de -- 00:00:25.635 --> 00:00:25.681 - Pero solo estás prediciendo. 00:00:25.681 --> 00:00:25.704 En cierto sentido, estás resolviendo un problema de marketing. 00:00:25.704 --> 00:00:25.716 - No, es un efecto causal, 00:00:25.716 --> 00:00:25.722 - Es causal, pero no tiene contenido científico. 00:00:25.722 --> 00:00:25.725 Piensa en... 00:00:25.725 --> 00:00:25.726 - No, pero hay cosas similares en el ámbito médico. 00:00:25.726 --> 00:00:25.727 Si haces un experimento, 00:00:25.727 --> 00:00:25.727 puedes estar muy interesado en si el tratamiento funciona 00:00:25.727 --> 00:00:25.727 para algunos grupos o no. 00:00:25.727 --> 00:00:25.727 Y tienes un montón de características individuales, 00:00:25.727 --> 00:00:25.727 y quieres buscar sistemáticamente... 00:00:25.727 --> 00:00:25.727 - Sí. Soy escéptico sobre esa... 00:00:25.727 --> 00:00:25.727 esa especie de idea de que hay un efecto causal personal 00:00:25.727 --> 00:00:25.727 que me debería importar 00:00:25.727 --> 00:00:25.727 y que el aprendizaje automático puede descubrirlo de alguna manera que sea útil. 00:00:25.727 --> 00:00:25.727 Así que piensa en -- 00:00:25.727 --> 00:00:25.727 he hecho un montón de trabajo en las escuelas, 00:00:25.727 --> 00:00:25.727 ir a, digamos, una escuela charter,------------------------------------------------- 00:00:25.727 --> 00:00:25.727 una escuela privada financiada con fondos públicos, 00:00:25.727 --> 00:00:25.727 efectivamente, que es libre de estructurar su propio plan de estudios para el contexto allí. 00:00:25.727 --> 00:00:25.727 Algunos tipos de escuelas chárter generan espectaculares ganancias de rendimiento 00:00:25.727 --> 00:00:25.727 y en el conjunto de datos que produce ese resultado, 00:00:25.727 --> 00:00:25.727 tengo un montón de covariables. 00:00:25.727 --> 00:00:25.727 Así que tengo las puntuaciones de referencia 00:00:25.727 --> 00:00:25.727 y tengo los antecedentes familiares, 00:00:25.727 --> 00:00:25.727 la educación de los padres, el sexo del niño, la raza del niño. 00:00:25.727 --> 00:00:25.727 Y, bueno, en cuanto reúno media docena de ellas, 00:00:25.727 --> 00:00:25.727 tengo un espacio de muy alta dimensión. 00:00:25.727 --> 00:00:25.727 Definitivamente me interesan las características del curso 00:00:25.727 --> 00:00:25.727 de ese efecto del tratamiento, 00:00:25.727 --> 00:00:25.727 como por ejemplo si es mejor para las personas que provienen de familias con menores ingresos. 00:00:25.727 --> 00:00:25.727 Me cuesta creer que haya una aplicación para la versión de muy alta dimensión de eso, 00:00:25.727 --> 00:00:25.727 en la que descubrí que para los niños no blancos 00:00:25.727 --> 00:00:25.727 que tienen ingresos familiares altos 00:00:25.727 --> 00:00:25.727 pero puntuaciones de referencia en el tercer cuartil 00:00:25.727 --> 00:00:25.728 y que solo fueron a la escuela pública en el tercer grado pero no en el sexto. 00:00:25.728 --> 00:00:26.305 Así que eso es lo que produce ese análisis de alta dimensión. 00:00:26.305 --> 00:00:26.883 Es una declaración condicional muy elaborada. 00:00:30.200 --> 00:00:33.352 Hay dos cosas que están mal, en mi opinión. 00:00:33.352 --> 00:00:34.251 En primer lugar, no lo veo como... 00:00:34.251 --> 00:00:34.700 no puedo imaginar por qué es procesable. 00:00:34.700 --> 00:00:34.925 No sé por qué querrías actuar sobre ello. 00:00:34.925 --> 00:00:35.037 Y también sé que hay algún modelo alternativo que encaja casi igual, que lo invierte todo. 00:00:35.037 --> 00:00:35.093 Porque el aprendizaje automático 00:00:35.093 --> 00:00:35.121 no me dice que este es realmente el predictor que importa -- 00:00:35.121 --> 00:00:35.150 sólo me dice que este es un buen predictor. 00:00:36.122 --> 00:00:37.140 Así que creo que hay algo diferente en el contexto de las ciencias sociales. 00:00:37.140 --> 00:00:37.649 - [Guido] Creo que las aplicaciones de las ciencias sociales 00:00:37.649 --> 00:00:37.904 de las que hablas son aquellas en las que, creo, 00:00:37.904 --> 00:00:38.031 no hay una gran cantidad de heterogeneidad en los efectos. 00:00:38.031 --> 00:00:38.095 - [Josh] Bueno, podría haberla si me permites llenar ese espacio. 00:00:38.095 --> 00:00:38.127 - No... ni siquiera entonces. 00:00:38.127 --> 00:00:38.143 Creo que para muchas de esas intervenciones, 00:00:38.143 --> 00:00:38.151 se espera que el efecto sea del mismo signo para todos. 00:00:38.151 --> 00:00:38.155 Puede haber pequeñas diferencias en la magnitud, pero no es... 00:00:38.155 --> 00:00:38.157 Para muchas de estas defensas educativas - son buenas para todos. 00:00:38.157 --> 00:00:38.158 No es que sean malas para algunas personas y buenas para otras 00:00:38.158 --> 00:00:38.159 y eso es una especie ------------------------------------------------------------------------------------------------------------------------------------. 00:00:38.159 --> 00:00:38.829 Pero puede haber algo de variación en la magnitud, 00:00:38.829 --> 00:00:39.164 pero se necesitarían conjuntos de datos muy, muy grandes para encontrarlos. 00:00:39.164 --> 00:00:39.332 Estoy de acuerdo en que, en esos casos, probablemente no serían muy procesables de todos modos. 00:00:39.332 --> 00:00:39.416 Pero creo que hay muchos otros escenarios donde hay mucha más heterogeneidad. 00:00:39.416 --> 00:00:39.458 - Bueno, estoy abierto a esa posibilidad 00:00:39.458 --> 00:00:39.479 y creo que el ejemplo que has dado es esencialmente un ejemplo de marketing. 00:00:39.479 --> 00:00:39.489 - No, esos tienen implicaciones para ello y esa es la organización, 00:00:39.489 --> 00:00:39.494 si tienes que preocuparte por la... 00:00:39.494 --> 00:00:39.497 - Bueno, necesito ver ese documento. 00:00:39.497 --> 00:00:39.498 - Así que, la sensación que tengo es que... 00:00:39.498 --> 00:00:39.499 - Todavía no estamos de acuerdo en algo. 00:00:39.499 --> 00:00:39.499 - Sí. 00:00:39.499 --> 00:00:39.499 - No hemos convergido en todo. 00:00:39.499 --> 00:00:39.499 - Tengo esa sensación. [risas] 00:00:39.499 --> 00:00:39.499 - En realidad, hemos divergido en esto porque esto no estaba para discutir. 00:00:39.499 --> 00:00:39.499 [risas] 00:00:39.499 --> 00:00:39.499 - ¿Se está calentando un poco aquí? 00:00:39.499 --> 00:00:39.499 - ------------------------------------------------------es bueno. 00:00:39.499 --> 00:00:39.499 La sensación que tengo es, Josh, 00:00:39.499 --> 00:00:39.499 que no estás diciendo que estás seguro de que no hay manera de que haya una aplicación en la que estas cosas sean útiles. 00:00:39.499 --> 00:00:39.499 Estás diciendo que no estás convencido 00:00:39.499 --> 00:00:39.499 de las aplicaciones existentes hasta la fecha. 00:00:39.499 --> 00:00:39.499 - Es justo. 00:00:39.499 --> 00:00:39.499 - Estoy muy seguro. 00:00:39.499 --> 00:00:39.499 [risas] 00:00:39.499 --> 00:00:39.499 - En este caso. 00:00:39.499 --> 00:00:39.499 - Creo que Josh tiene un punto que incluso en los casos de predicción donde muchos de los métodos de aprendizaje de la máquina realmente brillan es donde hay un montón de heterogeneidad. 00:00:39.499 --> 00:00:39.499 - No te importan mucho los detalles, ¿verdad? 00:00:39.499 --> 00:00:39.499 - [Guido] Sí. 00:00:39.499 --> 00:00:39.499 - No tiene un ángulo de política o algo así. 00:00:39.499 --> 00:00:39.500 00:00:39.700 --> 00:00:41.327 00:00:42.901 --> 00:00:45.600 00:00:45.900 --> 00:00:47.290 00:00:48.400 --> 00:00:50.610 00:00:52.161 --> 00:00:56.282 00:00:57.831 --> 00:01:00.617 00:01:00.617 --> 00:01:02.493 00:01:02.493 --> 00:01:05.500 00:01:06.200 --> 00:01:07.867 00:01:07.867 --> 00:01:09.920 00:01:10.100 --> 00:01:12.500 00:01:13.300 --> 00:01:16.430 00:01:16.430 --> 00:01:18.100 00:01:18.600 --> 00:01:20.979 00:01:21.829 --> 00:01:23.417 00:01:23.417 --> 00:01:27.119 00:01:27.119 --> 00:01:29.195 00:01:29.195 --> 00:01:30.435 00:01:30.435 --> 00:01:32.100 00:01:32.100 --> 00:01:35.045 00:01:35.490 --> 00:01:37.280 00:01:37.610 --> 00:01:41.682 00:01:42.177 --> 00:01:43.427 00:01:43.427 --> 00:01:45.008 00:01:45.008 --> 00:01:47.665 00:01:47.665 --> 00:01:49.381 00:01:49.381 --> 00:01:51.911 00:01:51.911 --> 00:01:53.141 00:01:53.657 --> 00:01:57.300 00:01:58.000 --> 00:02:01.300 00:02:01.300 --> 00:02:03.900 00:02:03.900 --> 00:02:06.143 00:02:06.143 --> 00:02:08.000 00:02:08.000 --> 00:02:09.500 00:02:09.500 --> 00:02:12.603 00:02:12.603 --> 00:02:14.000 00:02:14.000 --> 00:02:15.740 00:02:15.740 --> 00:02:17.259 00:02:17.259 --> 00:02:20.045 00:02:20.045 --> 00:02:22.336 00:02:22.336 --> 00:02:24.428 00:02:25.225 --> 00:02:27.392 00:02:27.392 --> 00:02:29.399 00:02:29.399 --> 00:02:31.369 00:02:31.369 --> 00:02:33.703 00:02:33.703 --> 00:02:36.400 00:02:36.400 --> 00:02:37.800 00:02:37.800 --> 00:02:41.353 00:02:41.353 --> 00:02:43.207 00:02:43.576 --> 00:02:45.800 00:02:45.800 --> 00:02:49.795 00:02:49.795 --> 00:02:51.900 00:02:52.300 --> 00:02:55.199 00:02:55.199 --> 00:02:56.457 00:02:56.457 --> 00:02:58.741 00:02:58.741 --> 00:03:02.046 00:03:02.600 --> 00:03:05.760 00:03:07.273 --> 00:03:09.872 00:03:09.872 --> 00:03:12.406 00:03:12.406 --> 00:03:14.971 00:03:14.971 --> 00:03:17.800 00:03:18.166 --> 00:03:21.785 00:03:21.785 --> 00:03:23.000 00:03:23.000 --> 00:03:25.796 00:03:25.800 --> 00:03:28.100 00:03:28.300 --> 00:03:30.605 00:03:30.605 --> 00:03:31.797 00:03:31.797 --> 00:03:34.000 00:03:34.600 --> 00:03:36.600 00:03:36.600 --> 00:03:39.455 00:03:39.455 --> 00:03:41.200 00:03:41.800 --> 00:03:43.398 00:03:43.398 --> 00:03:45.350 00:03:45.350 --> 00:03:48.582 00:03:48.582 --> 00:03:50.965 00:03:51.486 --> 00:03:54.870 00:03:54.870 --> 00:03:57.586 00:03:57.940 --> 00:04:00.186 00:04:00.186 --> 00:04:01.633 00:04:01.633 --> 00:04:02.735 00:04:03.400 --> 00:04:08.100 00:04:09.777 --> 00:04:11.544 00:04:11.544 --> 00:04:13.466 00:04:13.466 --> 00:04:15.740 00:04:15.740 --> 00:04:18.614 00:04:18.614 --> 00:04:22.765 00:04:24.367 --> 00:04:27.600 00:04:28.200 --> 00:04:30.232 00:04:30.232 --> 00:04:31.700 00:04:34.169 --> 00:04:36.034 00:04:36.034 --> 00:04:37.600 00:04:37.600 --> 00:04:39.100 00:04:39.100 --> 00:04:40.900 00:04:40.900 --> 00:04:44.011 00:04:44.011 --> 00:04:46.955 00:04:47.906 --> 00:04:49.078 00:04:49.078 --> 00:04:51.400 00:04:51.700 --> 00:04:53.800 00:04:54.100 --> 00:04:56.600 00:04:57.250 --> 00:04:59.102 00:04:59.102 --> 00:05:04.918 00:05:06.315 --> 00:05:09.656 00:05:09.656 --> 00:05:11.069 00:05:12.252 --> 00:05:14.330 00:05:15.469 --> 00:05:17.900 00:05:18.400 --> 00:05:21.072 00:05:21.467 --> 00:05:23.996 00:05:23.996 --> 00:05:25.440 00:05:25.440 --> 00:05:27.200 00:05:27.200 --> 00:05:28.679 00:05:28.679 --> 00:05:30.833 00:05:30.833 --> 00:05:32.334 00:05:33.057 --> 00:05:34.771 00:05:35.820 --> 00:05:38.147 00:05:38.147 --> 00:05:41.187 00:05:41.187 --> 00:05:43.119 00:05:43.119 --> 00:05:45.347 00:05:45.347 --> 00:05:47.028 00:05:47.028 --> 00:05:49.487 00:05:49.917 --> 00:05:52.022 00:05:52.022 --> 00:05:53.704 00:05:54.156 --> 00:05:55.189 00:05:55.189 --> 00:05:56.555 00:05:56.555 --> 00:06:00.452 00:06:01.639 --> 00:06:04.519 00:06:04.519 --> 00:06:06.738 00:06:07.300 --> 00:06:10.769 00:06:10.769 --> 00:06:11.836 00:06:11.836 --> 00:06:15.000 00:06:15.200 --> 00:06:18.232 00:06:18.795 --> 00:06:20.090 00:06:20.090 --> 00:06:24.000 00:06:24.400 --> 00:06:28.183 00:06:28.183 --> 00:06:30.383 00:06:30.383 --> 00:06:32.100 00:06:32.100 --> 00:06:34.700 00:06:37.166 --> 00:06:39.416 00:06:39.697 --> 00:06:41.950 00:06:41.950 --> 00:06:44.930 00:06:44.930 --> 00:06:46.180 00:06:46.180 --> 00:06:47.956 00:06:47.956 --> 00:06:49.227 00:06:49.227 --> 00:06:50.700 00:06:51.600 --> 00:06:55.488 00:06:56.100 --> 00:06:58.121 00:06:58.121 --> 00:07:00.100 00:07:00.100 --> 00:07:01.900 00:07:02.100 --> 00:07:04.900 00:07:05.300 --> 00:07:08.921 00:07:08.921 --> 00:07:11.787 00:07:12.410 --> 00:07:14.975 00:07:15.925 --> 00:07:17.561 00:07:17.561 --> 00:07:21.700 00:07:21.930 --> 00:07:23.020 00:07:23.020 --> 00:07:24.600 00:07:24.600 --> 00:07:25.912 00:07:25.912 --> 00:07:28.100 00:07:28.100 --> 00:07:29.807 00:07:29.807 --> 00:07:30.908 00:07:30.908 --> 00:07:32.660 00:07:32.660 --> 00:07:34.182 00:07:34.700 --> 00:07:36.947 00:07:36.947 --> 00:07:39.350 00:07:41.354 --> 00:07:43.288 00:07:43.288 --> 00:07:46.112 00:07:46.112 --> 00:07:47.941 00:07:47.941 --> 00:07:51.415 00:07:51.415 --> 00:07:54.003 00:07:54.003 --> 00:07:55.636 00:07:55.636 --> 00:07:57.612 00:07:57.612 --> 00:07:59.579 00:08:00.900 --> 00:08:03.527 00:08:03.527 --> 00:08:06.916 00:08:06.916 --> 00:08:09.453 00:08:09.453 --> 00:08:12.200 00:08:12.400 --> 00:08:19.208 00:08:20.700 --> 00:08:23.734 00:08:23.734 --> 00:08:25.700 00:08:27.351 --> 00:08:31.600 00:08:31.800 --> 00:08:37.133 00:08:37.133 --> 00:08:40.948 00:08:41.500 --> 00:08:42.697 00:08:42.697 --> 00:08:45.300 00:08:45.300 --> 00:08:47.664 00:08:47.664 --> 00:08:49.610 00:08:49.610 --> 00:08:51.300 00:08:51.300 --> 00:08:53.700 00:08:54.100 --> 00:08:57.782 00:08:57.782 --> 00:09:00.269 00:09:00.269 --> 00:09:03.375 00:09:04.400 --> 00:09:06.531 00:09:06.531 --> 00:09:08.200 00:09:08.200 --> 00:09:10.000 00:09:10.900 --> 00:09:12.804 00:09:12.804 --> 00:09:14.501 00:09:14.501 --> 00:09:16.853 00:09:17.384 --> 00:09:19.933 00:09:19.933 --> 00:09:23.000 00:09:23.300 --> 00:09:25.839 00:09:25.839 --> 00:09:29.292 00:09:29.292 --> 00:09:32.200 00:09:32.944 --> 00:09:35.113 00:09:35.113 --> 00:09:37.600 00:09:38.708 --> 00:09:40.825 00:09:40.825 --> 00:09:42.600 00:09:42.900 --> 00:09:46.807 00:09:49.173 --> 00:09:51.100 00:09:51.944 --> 00:09:55.096 00:09:55.096 --> 00:09:57.100 00:09:57.400 --> 00:09:59.100 00:09:59.300 --> 00:10:01.096 00:10:02.900 --> 00:10:06.435 00:10:06.435 --> 00:10:09.424 00:10:10.206 --> 00:10:11.842 00:10:12.591 --> 00:10:14.033 00:10:14.403 --> 00:10:15.907 00:10:15.907 --> 00:10:17.289 00:10:17.289 --> 00:10:18.410 00:10:18.410 --> 00:10:20.001 00:10:20.001 --> 00:10:22.054 00:10:22.700 --> 00:10:26.823 00:10:27.555 --> 00:10:29.500 00:10:29.800 --> 00:10:31.028 00:10:31.591 --> 00:10:34.926 00:10:35.315 --> 00:10:37.151 00:10:38.761 --> 00:10:42.664 00:10:42.664 --> 00:10:46.560 00:10:48.800 --> 00:10:53.289 00:10:53.289 --> 00:10:55.700 00:10:56.300 --> 00:10:59.420 00:11:00.873 --> 00:11:02.055 00:11:02.900 --> 00:11:06.420 00:11:06.420 --> 00:11:09.040 00:11:09.040 --> 00:11:11.600 00:11:11.900 --> 00:11:13.453 00:11:13.808 --> 00:11:18.500 00:11:18.800 --> 00:11:20.885 00:11:20.885 --> 00:11:23.600 00:11:24.000 --> 00:11:26.800 00:11:26.800 --> 00:11:28.950 00:11:28.950 --> 00:11:31.210 00:11:31.210 --> 00:11:32.580 00:11:32.580 --> 00:11:34.684 00:11:34.684 --> 00:11:35.736 00:11:35.736 --> 00:11:37.000 00:11:37.300 --> 00:11:39.100 00:11:39.100 --> 00:11:41.051 00:11:41.051 --> 00:11:42.557 00:11:42.557 --> 00:11:44.449 00:11:44.449 --> 00:11:45.871 00:11:45.871 --> 00:11:47.141 00:11:47.141 --> 00:11:51.250 00:11:51.250 --> 00:11:52.394 00:11:52.394 --> 00:11:54.500 00:11:54.900 --> 00:11:57.608 00:11:57.608 --> 00:11:59.782 00:11:59.782 --> 00:12:04.234 00:12:04.234 --> 00:12:06.270 00:12:06.786 --> 00:12:09.864 00:12:09.864 --> 00:12:11.200 00:12:12.944 --> 00:12:14.934 00:12:14.934 --> 00:12:18.400 00:12:20.189 --> 00:12:23.656 00:12:24.700 --> 00:12:27.830 00:12:27.830 --> 00:12:29.622 00:12:29.622 --> 00:12:31.163 00:12:31.163 --> 00:12:34.000 00:12:34.900 --> 00:12:36.529 00:12:36.529 --> 00:12:40.013 00:12:40.582 --> 00:12:42.632 00:12:42.632 --> 00:12:43.887 00:12:43.887 --> 00:12:46.073 00:12:46.073 --> 00:12:49.100 00:12:49.800 --> 00:12:52.105 00:12:52.105 --> 00:12:53.573 00:12:53.573 --> 00:12:57.500 00:12:58.600 --> 00:13:00.159 00:13:00.159 --> 00:13:03.209 00:13:03.209 --> 00:13:06.090 00:13:06.549 --> 00:13:08.351 00:13:08.351 --> 00:13:10.979 00:13:11.198 --> 00:13:13.548 00:13:13.548 --> 00:13:16.850 00:13:16.850 --> 00:13:19.821 00:13:19.821 --> 00:13:23.870 00:13:24.424 --> 00:13:27.057 00:13:27.218 --> 00:13:30.132 00:13:30.132 --> 00:13:31.829 00:13:31.829 --> 00:13:33.486 00:13:33.486 --> 00:13:35.229 00:13:35.229 --> 00:13:37.189 00:13:37.189 --> 00:13:38.300 00:13:38.300 --> 00:13:41.267 00:13:41.267 --> 00:13:43.261 00:13:43.261 --> 00:13:46.465 00:13:48.657 --> 00:13:51.200 00:13:51.600 --> 00:13:54.187 00:13:54.187 --> 00:13:56.804 00:13:56.804 --> 00:13:58.207 00:13:58.207 --> 00:13:59.220 00:13:59.220 --> 00:14:00.679 00:14:00.679 --> 00:14:01.944 00:14:02.520 --> 00:14:03.963 00:14:03.963 --> 00:14:06.659 00:14:06.659 --> 00:14:08.000 00:14:08.000 --> 00:14:11.000 00:14:11.000 --> 00:14:12.395 00:14:12.395 --> 00:14:13.867 00:14:13.867 --> 00:14:15.914 00:14:15.914 --> 00:14:17.280 00:14:17.280 --> 00:14:18.305 00:14:18.305 --> 00:14:21.028 00:14:21.500 --> 00:14:24.425 00:14:24.698 --> 00:14:26.556 00:14:26.556 --> 00:14:27.968 00:14:27.968 --> 00:14:30.233 00:14:30.233 --> 00:14:32.489 00:14:32.489 --> 00:14:34.942 00:14:34.942 --> 00:14:36.440 00:14:36.440 --> 00:14:40.900 00:14:40.900 --> 00:14:42.200 00:14:42.200 --> 00:14:47.209 00:14:47.209 --> 00:14:49.900 00:14:51.000 --> 00:14:52.602 00:14:52.602 --> 00:14:55.410 00:14:55.410 --> 00:14:57.828 00:14:57.828 --> 00:14:59.463 00:14:59.463 --> 00:15:02.100 00:15:02.100 --> 00:15:03.800 00:15:03.800 --> 00:15:08.418 00:15:08.418 --> 00:15:09.519 00:15:09.519 --> 00:15:13.185 00:15:13.185 --> 00:15:14.502 00:15:15.700 --> 00:15:18.200 00:15:19.700 --> 00:15:21.984 00:15:21.984 --> 00:15:23.137 00:15:23.137 --> 00:15:25.135 00:15:25.805 --> 00:15:28.860 00:15:29.500 --> 00:15:31.900 00:15:32.200 --> 00:15:34.140 00:15:34.140 --> 00:15:37.507 00:15:38.100 --> 00:15:43.874 00:15:44.300 --> 00:15:46.700 00:15:47.300 --> 00:15:50.642 00:15:50.642 --> 00:15:53.900 00:15:53.900 --> 00:15:56.600 00:15:56.600 --> 00:15:58.600 00:15:59.000 --> 00:16:02.026 00:16:02.026 --> 00:16:06.648 00:16:06.648 --> 00:16:08.328 00:16:08.328 --> 00:16:11.300 00:16:11.300 --> 00:16:12.819 00:16:12.819 --> 00:16:14.753 00:16:14.753 --> 00:16:18.191 00:16:18.191 --> 00:16:20.400 00:16:20.950 --> 00:16:22.394 00:16:22.500 --> 00:16:24.999 00:16:24.999 --> 00:16:26.212 00:16:26.425 --> 00:16:27.993 00:16:27.993 --> 00:16:33.200 00:16:34.100 --> 00:16:35.710 00:16:35.920 --> 00:16:37.148 00:16:37.148 --> 00:16:40.066 00:16:40.066 --> 00:16:43.316 00:16:43.316 --> 00:16:45.100 00:16:45.100 --> 00:16:46.392 00:16:46.392 --> 00:16:49.700 00:16:50.000 --> 00:16:52.020 00:16:52.020 --> 00:16:53.526 00:16:53.526 --> 00:16:55.184 00:16:55.184 --> 00:16:57.400 00:16:57.976 --> 00:17:00.350 00:17:00.350 --> 00:17:03.102 00:17:03.102 --> 00:17:05.671 00:17:05.671 --> 00:17:08.610 00:17:08.610 --> 00:17:09.811 00:17:09.811 --> 00:17:12.000 00:17:12.400 --> 00:17:15.309 00:17:15.309 --> 00:17:17.292 00:17:17.292 --> 00:17:19.100 00:17:19.500 --> 00:17:21.849 00:17:21.849 --> 00:17:23.155 00:17:23.155 --> 00:17:25.063 00:17:25.063 --> 00:17:27.100 00:17:27.872 --> 00:17:29.965 00:17:29.965 --> 00:17:32.400 00:17:32.400 --> 00:17:35.100 00:17:35.100 --> 00:17:36.625 00:17:36.625 --> 00:17:40.573 00:17:40.573 --> 00:17:43.620 00:17:43.620 --> 00:17:45.797 00:17:45.797 --> 00:17:47.733 00:17:47.733 --> 00:17:50.100 00:17:50.100 --> 00:17:52.983 00:17:52.983 --> 00:17:55.700 00:17:55.700 --> 00:17:58.000 00:17:58.000 --> 00:18:01.229 00:18:01.229 --> 00:18:03.300 00:18:03.600 --> 00:18:05.985 00:18:05.985 --> 00:18:07.668 00:18:07.668 --> 00:18:10.694 00:18:10.694 --> 00:18:12.800 00:18:13.000 --> 00:18:16.780 00:18:16.780 --> 00:18:18.517 00:18:18.517 --> 00:18:20.938 00:18:20.938 --> 00:18:22.139 00:18:22.139 --> 00:18:27.309 00:18:27.309 --> 00:18:28.676 00:18:28.900 --> 00:18:30.492 00:18:30.492 --> 00:18:33.782 00:18:33.782 --> 00:18:35.228 00:18:35.529 --> 00:18:39.500 00:18:39.500 --> 00:18:41.824 00:18:41.824 --> 00:18:44.000 00:18:44.000 --> 00:18:47.065 00:18:47.065 --> 00:18:48.800 00:18:49.000 --> 00:18:52.964 00:18:53.504 --> 00:18:56.000 00:18:56.300 --> 00:18:58.791 00:18:58.791 --> 00:19:01.200 00:19:01.500 --> 00:19:02.649 00:19:02.649 --> 00:19:06.300 00:19:06.500 --> 00:19:08.181 00:19:08.181 --> 00:19:09.779 00:19:09.779 --> 00:19:11.489 00:19:11.489 --> 00:19:12.741 00:19:12.741 --> 00:19:14.274 00:19:14.274 --> 00:19:17.067 00:19:17.067 --> 00:19:20.148 00:19:20.349 --> 00:19:21.388 00:19:21.388 --> 00:19:23.673 00:19:23.673 --> 00:19:25.801 00:19:25.801 --> 00:19:28.603 00:19:29.900 --> 00:19:32.704 00:19:32.704 --> 00:19:35.816 00:19:35.816 --> 00:19:37.500 00:19:37.500 --> 00:19:40.223 00:19:40.223 --> 00:19:41.388 00:19:41.388 --> 00:19:43.090 00:19:43.090 --> 00:19:45.394 00:19:45.394 --> 00:19:46.630 00:19:47.100 --> 00:19:49.300 00:19:49.300 --> 00:19:52.932 00:19:52.932 --> 00:19:54.414 00:19:54.414 --> 00:19:56.400 00:19:57.000 --> 00:20:00.297 00:20:00.297 --> 00:20:03.539 00:20:03.539 --> 00:20:06.100 00:20:06.600 --> 00:20:08.720 00:20:08.720 --> 00:20:10.174 00:20:10.174 --> 00:20:11.633 00:20:12.926 --> 00:20:14.419 00:20:14.419 --> 00:20:16.833 00:20:16.833 --> 00:20:18.012 00:20:18.012 --> 00:20:20.500 00:20:20.500 --> 00:20:23.100 00:20:23.600 --> 00:20:26.569 00:20:26.569 --> 00:20:28.550 00:20:28.550 --> 00:20:30.535