Deep Q Learning

The algorithm for deep Q learning is given as follows:

1. Initialize the main network parameter with random values
2. Initialize the target network parameter by copying the main network parameter
3. Initialize the replay buffer
4. For N number of episodes, perform step 5
5. For each step in the episode, that is, for t = 0, . . ., T – 1:
   1. Observe the state s and select an action using the epsilon-greedy policy, that is, with probability epsilon, select random action a, and with probability 1-epsilon, select the action as
   2. Perform the selected action and move to the next state and obtain the reward r
   3. Store the transition information in the replay buffer
   4. Randomly sample a minibatch of K transitions from the replay buffer
   5. Compute the target value, that is,
   6. Compute the loss,
   7. Compute the gradients of the loss and update the main network parameter using gradient descent...