The architecture of the sequence-to-sequence model that we will be using for building the chatbot will have slight modifications to the basic sequence-to-sequence architecture illustrated previously in Figure 8.2. The modified architecture can be seen in the following diagram (Figure 8.3):
Figure 8.3: Sequence-to-sequence model
Instead of feeding the hidden state and the cell state of the last step of the encoder to the initial hidden and cell states of the Decoder LSTM, we feed the hidden state at each input step of the decoder. To predict the target word wt at any step t, the inputs are the previous target word, wt-1, at any step, t-1, and the hidden state .