Optimization of conditional execution
After the unnecessary computations and inefficient use of memory, the next easiest way to write inefficient code that fails to utilize a large fraction of available computing resources is probably code that does not pipeline well. We have seen the importance of CPU pipelining in Chapter 3, CPU Architecture, Resources, and Performance Implications. We have also learned there that the worst disruptor of pipelining is usually a conditional operation, especially the one that the hardware branch predictor fails to guess.
Unfortunately, optimizing conditional code for better pipelining is one of the hardest C++ optimizations. It should be undertaken only if the profiler shows poor branch prediction. Note, however, that the number of mispredicted branches does not have to be large to be considered “poor”: a good program will typically have less than 0.1% of mispredicted branches. The misprediction rate of 1% is quite large. It is also...
