Implementing the Affine and Softmax Layers

Affine Layer

In forward propagation in a neural network, the product of matrices (np.dot(), in NumPy) was used to sum the weighted signals (for details, refer to the Calculating Multidimensional Arrays section in Chapter 3, Neural Networks). For example, do you remember the following implementation in Python?

>>> X = np.random.rand(2) # Input values
>>> W = np.random.rand(2,3) # Weights
>>> B = np.random.rand(3) # Biases
>>>
>>> X.shape # (2,)
>>> W.shape # (2, 3)
>>> B.shape # (3,)
>>>
>>> Y = np.dot(X, W) + B

Here, assume that X, W, and B are multidimensional arrays of the shape (2,), (2, 3), and (3,), respectively. With this, you can calculate the weighted sum of neurons as Y = np.dot(X, W) + B. Y is converted by the activation function and propagated to the next layer, which is the flow of forward propagation in a neural network. Note that the number...