The name of CNNs comes from their signature operation: convolution. This operation is a mathematical operation that is very common in the signal processing area. Let's go ahead and discuss the convolution operation.
1-dimension
Let's start with the discrete-time convolution function in one dimension. Suppose that we have input data, , and some weights, , we can define the discrete-time convolution operation between the two as follows:
.
In this equation, the convolution operation is denoted by a * symbol. Without complicating things too much, we can say that is inverted, , and then shifted, . The resulting vector is , which can be interpreted as the filtered version of the input when the filter is applied.
If we define the two vectors as follows, and , then the convolution operation yields .
Figure 12.1 shows every single step involved in obtaining this result by inverting and shifting the filter and multiplying across the input data: