DDQN stands for dueling DQN and is different from the double DQN, although people often confuse them. Both variations assume some form of duality, but in the first case, the model is assumed to be split at the base, while in the second case, double DQN, the model is assumed to be split into two entirely different DQN models.
The following diagram shows the difference between DDQN and DQN, which is not to be confused with dueling DQN:
The difference between DQN and DDQN
In the preceding diagram, CNN layers are being used in both models but in the upcoming exercises, we will just use linear fully connected layers instead, just to simplify things.
Notice how the DDQN network separates into two parts that then converge back to an answer. This is the dueling part of the DDQN model we will get to shortly. Before that, though, let's explore the double DQN model...
