Chapter 4:

1. Modern CPUs are significantly faster than even the best memories. The latency for accessing a random location in memory is several nanoseconds, enough time for the CPU to execute dozens of operations. Even in streaming access, the overall memory bandwidth is not enough to supply the CPU with the data at the same speed it can carry out the computations.
2. The memory system includes a hierarchy of caches between the CPU and the main memory, so the first factor affecting the speed is the size of the data set: this ultimately determines whether the data fits into a cache or not. For a given size, the memory access pattern is critical: if the hardware can predict the next access, it can hide the latency by starting the transfer of data into the cache before this data is requested.
3. Often inefficient memory access is evident from a performance profile or timer output; this is particularly true for well-modularized code with good encapsulation of data. If the timing profile...