Determining the depth of the network

When we start building a deep learning model from scratch, it is hard to determine beforehand how many (different types of) layers we should stack. Generally, it is a good idea to have a look at a well-known deep learning model and use it as a basis to build further. In general, it is good to try to overfit to the training data as much as you can first. This ensures that your model is able to train on the input data. Afterwards, apply regularization techniques such as dropout to prevent overfitting and stimulate generalization.