NN architectures, challenges, and optimization
We’ve primarily covered the basics of NNs so far in this chapter, and in this section, we will expand our discussion to include different types of NN architectures that can be used for different types of real-world use cases, as well as some challenges that are often encountered when training them. Finally, we will discuss how to optimize our NNs to address those challenges.
Common NN architectures
The “architecture” of an NN refers to its structure in terms of the number of layers it contains and the number of neurons in each layer, as well as any special characteristics that influence how information is propagated through the network. The NN architectures we’ve described so far in this chapter are the simplest forms of ANNs, which are referred to as feed-forward NNs (FFNNs). Information in these networks travels in one direction only, from the input layer, through the hidden layers, to the output layer...
