Let's step back for a bit to remind us why we're focusing on memory the way we are.
Our overarching goal, of course, is just to make code fast.
So we say great, GPUs are fast, let's use those. But why are they fast?
Gpus are fast, first, because they are massively parallel, with hundreds or thousands of
processors on a single chip, working for you to solve your problem, but also because they have
an extremely high bandwidth memory system to feed those massively parallel processors, okay?
So if the memory system can't deliver data to all of these processors
and store results from all those processors, then we're not going to get the full speed out of our GPU.
And that's why, on a memory limited kernel like transpose,
our subgoal is really to utilize all the available memory bandwidth.
Hence our focus on global memory coalescing, DRAM utilization, and so on.
Now I really want to ask a question a little bit more rigorously.
What do we mean by utilizing all the available memory bandwidth?
And this is going to bring us to a very important, very simple principle called Little's Law.
Let's have the talented Kim Dilla illustrate this for us.
Now John Little is a MIT professor who studies Marketing.
He formulated his Eponymous Law, when writing about queuing theory in business processes.
And Little's Law is usually used to reason about things like optimizing the number
of customers in a line at Starbucks, or maybe the size of queues in a factory.
But Little's Law is really very general and can be applied
to many things including memory systems in computers.
In that context, Little's Law states that the number of bytes delivered
equals the average latency of each memory transaction times the bandwidth.
Let's be a little more precise and emphasize
that we care about the useful bytes delivered, and the problem with uncoalesced
global memory accesses is that not all of the bytes in every memory transaction are actually being used.
That's why coalescing global memory accesses helps
ensure that every byte delivered in a memory transaction will be used.
So given this definition, what can we do to improve our bandwidth? Let's check all that apply.
We can increase the number of bytes delivered, we can increase the latency--
meaning the time between memory transactions--we can decrease the
number of bytes delivered, or we can decrease the latency or time between transactions.
让我们后退一点来提醒我们， 为什么我们以我们的方式关注内存。
当然，我们的中心目标只是为了让代码快速运行。
所以我们说很棒，GPU很快，让我们使用那些。但为什么它们速度快？
GPU很快，第一，因为它们是大规模并行，在单个芯片上有成百上千的
处理器为你解决问题，而且还因为它们有
一个极高带宽的内存系统，来供给那些大规模并行处理器，对吗？
因此，如果内存系统不能将数据传递到所有这些处理器，
并且存储所有这些处理器的结果，那我们就不能让我们的GPU 全速工作。
这就是为什么，在一个内存受限的内核上，比如转置，
我们的子目标其实是利用所有可用的内存带宽。
因此我们注重全局内存合并、DRAM 利用率，等等。
现在我很想严格点地问一个问题。
我们说利用所有可用的内存带宽是什么意思？
这会把我们带到一个非常重要、 非常简单的原则，叫利特尔法则。
让我们请有才的金•迪拉来为我们说明这一点。
约翰•利特尔现在是一名学习市场营销的麻省理工学院教授。
他在撰写业务流程中的排队理论时， 构想了以他名字命名的法则。
利特尔法则通常用于推断某些事情，例如优化
星巴克排队的顾客数量，或者也许一个工厂里队列的长度。
但利特尔法则真的很普遍，能应用
于很多事情，包括计算机中的内存系统。
在这种背景下，利特尔法则规定传送的字节数
等于每个内存事务的平均延迟时间乘以带宽。
让我们更准确一点并强调
我们关心传送的有用字节，以及未合并全局内存访问的问题
在于不是每个内存事务的所有字节都实际被使用。
这就是为什么合并全局内存访问帮助
确保在内存事务中传递的每一个字节将被使用。
所以鉴于这一定义，我们可以做什么来改善我们的带宽？让我们勾选所有适用的选项。
我们可以增加传递的字节数，我们可以增加延迟时间——
指每个内存事务间的时间——我们可以减少
传递的字节数，或者我们可以减少延迟或者事务之间的时间。