GRU network

LSTM Network is computationally expensive, hence, researchers found an almost equally effective configuration of RNNs, known as Gated Recurrent Unit (GRU) architecture. 

In GRU, instead of a working and a long-term memory, only one kind of memory is used, indicated with h (hidden state). The GRU cell adds information to this state memory or removes information from this state memory through reset and update gates.

Following diagram depicts the GRU cell (explanation follows the diagram):

The GRU Cell

The  internal flow through the gates in the GRU cell is as follows:

1. Update gate u( ): The input
   
    and 
   
    flows to the u( ) gate as per the following equation:
   
2. Reset Gate r( ): The input 
   
    and 
   
    flows to the r( )gate as per the following equation:
   

3. Candidate State Memory: The candidate long-term memory is computed from the output of the r( )gate, 
   
   , and
   
   , as per the following equation:
   
4. Next, the preceding three calculations are combined to get the updated state memory, denoted by
   
   , as per following...