- Was there a significant difference in performance between a wide or deep network?
Not much in the case, we studied here. However, one thing you must remember is that both networks learned fundamentally different things or aspects of the input. Therefore, in other applications, the performance might vary.
- Is deep learning the same as a deep neural network?
No. Deep learning is the area of machine learning focused on all algorithms that train over-parametrized models using novel gradient descent techniques. Deep neural networks are networks with many hidden layers. Therefore, a deep network is deep learning. But deep learning is not uniquely specific to deep networks.
- Could you give an example of when sparse networks are desired?
Let's think about robotics. In this field, most things run on microchips that have memory constraints and storage constraints and computational power constraints; finding neural architectures whose weights are mostly zero would...
