Showing Revision 5 created 05/24/2016 by Udacity Robot.

1. Title:
   Slac and Big Data - Intro to Computer Science
2. Description:

3. 0 0:00 - 0:04
   We're here at SLAC National Accelerator Lab,
4. 4000 0:04 - 0:07
   and we're going to see how they use computing to understand the mysteries of the universe.
5. 7000 0:07 - 0:12
   [Spencer Gessner:] We're standing in the klystron gallery, formerly the longest building in the world.
6. 15000 0:15 - 0:18
   [Richard Mount:] You're here at SLAC National Accelerator Laboratory.
7. 18000 0:18 - 0:26
   This is a 50-year-old laboratory, as all the flags on the lampposts around the lab are telling you.
8. 26000 0:26 - 0:31
   It was founded to build a 2-mile-long linear accelerator.
9. 31000 0:31 - 0:34
   SLAC is an accelerator laboratory still.
10. 34000 0:34 - 0:41
    Its main science is based on accelerating particles and creating new states of matter
11. 41000 0:41 - 0:46
    or exploring the nature of matter with the accelerated particles.
12. 46000 0:46 - 0:51
    This always has generated a lot of data, a lot of information.
13. 51000 0:51 - 0:55
    It's very data-intensive experimental science.
14. 55000 0:55 - 1:00
    From the earliest days of SLAC computing
15. 60000 1:00 - 1:05
    to analyze data has been a major part of the activity here.
16. 65000 1:05 - 1:11
    You really can only study the cosmos by studying it in a computer.
17. 71000 1:11 - 1:13
    You get one chance to look at it,
18. 73000 1:13 - 1:17
    but to understand how it evolved into the state it is now,
19. 77000 1:17 - 1:19
    you have to do all this in the computer.
20. 79000 1:19 - 1:22
    There are massive computations going on for that sort of simulation,
21. 82000 1:22 - 1:26
    massive computations in catalysis and material science
22. 86000 1:26 - 1:29
    and massive data analysis going on here as well.
23. 89000 1:29 - 1:32
    The particular particle physics experiment
24. 92000 1:32 - 1:40
    that I am involved in right now has some 300 petabytes of disk space--
25. 100000 1:40 - 1:47
    some 300,000 terabytes, some 300 million gigabytes of disk space
26. 107000 1:47 - 1:50
    around the world to do this analysis.
27. 110000 1:50 - 1:54
    Of course, we are far from understanding everything about the universe,
28. 114000 1:54 - 2:00
    but this is probably one of the most data-intensive activity in science today.
29. 120000 2:00 - 2:07
    The raw data rate coming out of the ATLAS detector that I'm involved in
30. 127000 2:07 - 2:10
    is about a petabyte a second.
31. 130000 2:10 - 2:16
    That's 1 million gigabytes a second.
32. 136000 2:16 - 2:19
    You can't store that with any budget known to man,
33. 139000 2:19 - 2:27
    so most of it is inspected on the fly and reduced to a much smaller, but still large, storable amount of data.
34. 147000 2:27 - 2:32
    Right now we are sifting through these many, many petabytes of data
35. 152000 2:32 - 2:37
    to look for signals of the Higgs boson, as no doubt people have heard in the news.
36. 157000 2:37 - 2:43
    There are tantalizing hints that I'm not holding my breath about at all right now,
37. 163000 2:43 - 2:45
    but this is the way we do it.
38. 165000 2:45 - 2:48
    You need to have those vast amounts of data
39. 168000 2:48 - 2:55
    just to pick out the things that will really revolutionize physics in there,
40. 175000 2:55 - 2:59
    and you need to understand all of it in detail, because what you're looking for
41. 179000 2:59 - 3:02
    is something slightly unusual compared with everything else.
42. 182000 3:02 - 3:07
    If you don't understand everything else perfectly then you don't understand anything.
43. 187000 3:07 - 3:10
    [Max Swiatlowski:] We're looking at one of the racks that contains
44. 190000 3:10 - 3:13
    the ATLAS proof buster at SLAC.
45. 193000 3:13 - 3:16
    ATLAS is an experimental Large Hadron Collider in Geneva, Switzerland,
46. 196000 3:16 - 3:22
    that collides protons, fundamental building blocks of nature,
47. 202000 3:22 - 3:27
    traveling at very, very, very close to the speed of light
48. 207000 3:27 - 3:30
    with trillions of times the energy that they have at room temperature.
49. 210000 3:30 - 3:33
    You get many and many of these collisions happening at once
50. 213000 3:33 - 3:39
    and this enormous machine that reads out trillions of data channels.
51. 219000 3:39 - 3:42
    At the end of the day, you have this enormous amount of data--petabytes of data--
52. 222000 3:42 - 3:48
    that you have to analyse looking for very rare, very particular signatures inside of that.
53. 228000 3:48 - 3:53
    If I want to look for a rare signature--something that had a lot of energy
54. 233000 3:53 - 3:56
    and a lot of really strange particles at once--
55. 236000 3:56 - 4:00
    there are trillions and trillions of these events stored on this machine.
56. 240000 4:00 - 4:03
    To look for them in any reasonable amount of time,
57. 243000 4:03 - 4:06
    I have to do many searches at once.
58. 246000 4:06 - 4:10
    I have to use all the cores on the computers--
59. 250000 4:10 - 4:14
    the hundreds of cores on the machine all running at full-speed at the same time--
60. 254000 4:14 - 4:18
    to have any hope of doing it in any reasonable amount of time.
61. 258000 4:18 - 4:22
    [Richard Mount:] This isn't the sort of thing that search engines currently do.
62. 262000 4:22 - 4:28
    They're looking for text strings and indexing all the text strings that they find
63. 268000 4:28 - 4:31
    in some way like this.
64. 271000 4:31 - 4:34
    What we have is very, very structured.
65. 274000 4:34 - 4:36
    We know the structure of these data.
66. 276000 4:36 - 4:41
    We know exactly how to go to anything that we want to get to in these data,
67. 281000 4:41 - 4:46
    because the way in which everything is linked together is very well understood.
68. 286000 4:46 - 4:48
    Things will go wrong all the time.
69. 288000 4:48 - 4:51
    You cannot assume you won't lose data from the disk.
70. 291000 4:51 - 4:55
    You send it by network from one computer center to another.
71. 295000 4:55 - 4:57
    You cannot assume it arrives undamaged.
72. 297000 4:57 - 5:01
    You cannot assume your computers don't die in the middle of calculations.
73. 301000 5:01 - 5:07
    Everything can go wrong, so the computing we do for the LHC
74. 307000 5:07 - 5:11
    has many layers of error correction and retry.
75. 311000 5:11 - 5:14
    Some of the basic failure rates are quite high,
76. 314000 5:14 - 5:18
    but by the time everything has been fairly automatically retried
77. 318000 5:18 - 5:23
    and errors have been corrected, we get high throughput and a high success rate.