Introduction

Neural networks, as a broad field, borrow a lot from biological systems, particularly the brain. Advances in neural science have directly influenced research in to neural networks.

CNNs are inspired by the work of two neural scientists, D.H. Hubel and T.N. Wiesel. Their research focused on the mammalian visual cortex, which is the part of the brain responsible for vision. Through their research back in the sixties, they found that the visual cortex is composed of layers of neurons. Furthermore, these layers are arranged in a hierarchical structure. This hierarchy ranges from simple-to hypercomplex neurons. They also advanced the notion of a 'receptive field,' which is the space within which certain stimuli activate or fire a neuron, with a degree of spatial invariance. Spatial or shift invariance allows animals to detect objects regardless of whether they are rotated, scaled, transformed, or partially obscured.

Figure 4.1: Examples of spatial variance

Inspired...