Machine Translation with the Transformer
Humans master sequence transduction, transferring a representation to another object. We can easily imagine a mental representation of a sequence. If somebody says, "The flowers in my garden are beautiful," we can easily visualize a garden with flowers in it. We see images of the garden, although we might never have seen that garden. We might even imagine chirping birds and the scent of flowers.
A machine has to learn transduction from scratch with numerical representations. Recurrent or convolutional approaches have produced interesting results but have not reached significant BLEU translation evaluation scores. Translating requires the representation of language A transposed into language B.
The Transformer model's self-attention innovation increases the analytic ability of machine intelligence. A sequence in language A is adequately represented before attempting to translate it into language B. Self-attention brings...
