REINFORCE with Baseline

The algorithm for REINFORCE with baseline is given as follows:

1. Initialize the policy network parameter and value network parameter
2. Generate some N number of trajectories following the policy
3. Compute the return (reward-to-go) R_t
4. Compute the policy gradient,
5. Update the policy network parameter using gradient ascent,
6. Compute the mean squared error of the value network,
7. Update the value network parameter using gradient descent,
8. Repeat steps 2 to 7 for several iterations