Showing Revision 4 created 05/25/2016 by Udacity Robot.

1. Title:
   Why Use CUB - Intro to Parallel Programming
2. Description:

3. 250 0:00 - 0:04
   Now let me focus on a specific aspect of writing high performance kernel code.
4. 4342 0:04 - 0:07
   At a high level, GPU programming looks like this.
5. 7140 0:07 - 0:10
   There's a bunch of data sitting in global memory,
6. 10013 0:10 - 0:13
   and you have an algorithm that you want to run on that data.
7. 12984 0:13 - 0:16
   That algorithm will get executed by threads running on SMs,
8. 16050 0:16 - 0:20
   and for various reasons, you might want to stage the data through shared memory.
9. 20312 0:20 - 0:24
   And you've seen for yourself that loading or storing global memory into shared memory
10. 24401 0:24 - 0:27
    or into local variables of the threads
11. 26731 0:27 - 0:29
    can be complicated if you are striving for really high performance.
12. 29474 0:29 - 0:31
    Think about problem set 2
13. 31302 0:31 - 0:33
    where you loaded an image tile into shared memory
14. 33400 0:33 - 0:37
    and also had to load a halo of extra cells around the image tile
15. 37068 0:37 - 0:40
    in order to account for the width of the blur,
16. 39608 0:40 - 0:46
    and this can be tricky because the number of threads that want to perform a computation
17. 45502 0:46 - 0:48
    on the pixels in that tile
18. 48050 0:48 - 0:52
    is naturally a different number than the number of threads
19. 51995 0:52 - 0:57
    you would launch if your goal was simply to load the pixels in the tile
20. 56600 0:57 - 1:00
    as well as the pixels outside of the tile.
21. 60222 1:00 - 1:04
    Or think about the tile transpose example in Unit 5
22. 63900 1:04 - 1:10
    where we staged through shared memory in order to pay careful attention to coalescent global memory.
23. 70223 1:10 - 1:13
    Think about our discussion of Little's Law
24. 72976 1:13 - 1:16
    and the trade offs that we went over between latency,
25. 75545 1:16 - 1:18
    bandwidth, occupancy, the number of threads,
26. 78447 1:18 - 1:21
    and the transaction size per thread.
27. 80531 1:21 - 1:24
    Remember that we saw a graph that looked sort of like this
28. 84140 1:24 - 1:28
    where the bandwidth that we achieved as we increased the number of threads
29. 88459 1:28 - 1:33
    was higher if we were able to access 4 floating point values in a single load
30. 93037 1:33 - 1:35
    versus 2 floating point values in a single load
31. 95431 1:35 - 1:40
    versus a single floating point value in a load.
32. 99941 1:40 - 1:44
    Finally, there are ninja level optimizations we haven't even talked about in this class,
33. 103707 1:44 - 1:46
    like using the Kepler LDG intrinsic.
34. 106351 1:46 - 1:50
    In short, CUDA's explicit use of user managed shared memory
35. 109885 1:50 - 1:53
    enables predictable high performance.
36. 112550 1:53 - 1:56
    In contrast, the hardware manage caches that you find on CPUs
37. 115981 1:56 - 1:59
    can result in unpredictable performance,
38. 118880 1:59 - 2:03
    especially when you have many multiple threads sharing the same cache, and they're interfering with each other.
39. 122565 2:03 - 2:05
    So that's an advantage of explicit shared memory,
40. 124835 2:05 - 2:08
    but that advantage comes at the cost of an additional burden on the programmer
41. 128422 2:08 - 2:12
    who has to explicitly manage the movement of data in and out from global memory.
42. 132235 2:12 - 2:14
    And this is a big part of where CUB comes in.
43. 134306 2:14 - 2:18
    CUB puts an abstraction around the algorithm and its memory access pattern
44. 138255 2:18 - 2:23
    and deals opaquely with the movement of data from global memory,
45. 142846 2:23 - 2:28
    possibly through shared memory, into the actual local variables of the threads.
46. 147650 2:28 - 2:31
    I want to mention another programming power tool called Cuda DMA
47. 151389 2:31 - 2:36
    that focuses specifically around the movement of data from global into shared memory.