Tensor Processing Unit

This chapter introduces the Tensor Processing Unit (TPU), a special chip developed at Google for ultra-fast execution of neural network mathematical operations. Similarly to Graphic Processing Units (GPUs), the idea here is to have a special processor focusing only on very fast matrix operations with no support for all the other operations normally supported by Central Processing Units (CPUs). However, the additional improvement with TPUs is to remove from the chip any hardware support for graphics operation normally present in GPUs (rasterization, texture mapping, frame buffer operations, and so on). Think of a TPU as a special purpose co-processor specialized for deep learning, being focused on matrix or tensor operations. In this chapter we are going to compare CPUs and GPUs with the three generations of TPUs and Edge TPUs. All these accelerators are available as November 2019. The chapter will include code examples of using TPUs. So with that, let&apos...