Further Reading
Transformers could not have increased their potential without hardware innovation. Nvidia, for example, offers interesting insights on transformers and the related hardware:https://blogs.nvidia.com/blog/2022/03/25/what-is-a-transformer-model/https://docs.nvidia.com/deeplearning/transformer-engine/user-guide/index.html