Encoder-Decoder Architecture
In this section, we will first introduce the general concept of an encoder-decoder architecture. Afterward, we will focus on how the encoder is used in neural machine translation. In the last two subsections, we will concentrate on how the decoder is applied during training and deployment.
One of the possible structures for neural machine translation is the encoder-decoder network. In Chapter 5, Autoencoder for Fraud Detection, we introduced the concept of a neural network consisting of an encoder and a decoder component. Remember, in the case of an autoencoder, the task of the encoder component is to extract a dense representation of the input, while the task of the decoder component is to recreate the input based on the dense representation given by the encoder.
In the case of encoder-decoder networks for neural machine translation, the task of the encoder is to extract the context of the sentence in the source language (the input sentence) into...
