A fully connected layer is the last layer of a CNN. Fully connected layers, given an input volume, return as output a multi-dimensional vector. The dimension of the output vector matches the number of classes for the particular problem to solve.
This chapter and others in this book present some examples of CNN implementation and training for digit classification purposes. In those cases, the dimension of the output vector would be 10 (the possible digits are 0 to 9). Each number in the 10-dimensional output vector represents the probability of a certain class (digit). The following is an output vector for a digit classification inference:
[0 0 0 .1 .75 .1 .05 0 0 0]
How do we interpret those values? The network is telling us that it believes that the input image is a four with a 75% probability (which is the highest in this case), with a 10% probability...
