The Input Gate and the Candidate Cell State
At each timestep, a new candidate cell state is also calculated using the following expression:
Figure 7.10: Expression for candidate cell state
The input at timestep t is multiplied by a new set of weights, W_c, with the dimensions (n_h, n_x). The activation from the previous timestep (h[t-1]) is multiplied by another new set of weights, U_c, with the dimensions (n_h, n_h). Note that the multiplications are matrix multiplications. These two terms are then added and passed through a hyperbolic tan function to squish the output, f[t], within a range of [-1,1]. The output, C_candidate, has the dimensions (n_h,1). In the diagram that follows, the candidate cell state is represented by C tilde:
Figure 7.11: Input gate and candidate state
The candidate aims at calculating the cell state that it deduces from the current timestep. In our example sentence, this corresponds to calculating the new subject gender value. This candidate...
