Defining and Training the Network Architecture
The process of designing and training the network is similar to the process used in the previous NLP case studies.
Designing the Network
In this case, we want to use a network with five layers:
- A Keras input layer to define the input shape
- A Keras LSTM layer for the sequence analysis
- A Keras dropout layer for regularization
- A Keras dense layers with linear activation
- A Keras softmax layer to transform the output into a probability distribution
The number of unique characters in the training set – that is, the character set size – is 95. Since we allow sequences of variable length, the shape of the input layer is ?, 95. The ? stands for a variable sequence length.
Next, we have the Keras LSTM Layer node. This time, it is important to activate the Return sequences and Return state checkboxes, as we need the intermediate output states during the training process and the cell state in...
