The term artificial neural networks stands for a graph of nodes connected by links where each of the links has a particular weight. The neural node defines a kind of threshold operator that allows the signal to pass only after a specific activation function has been applied. It remotely resembles the way in which neurons in the brain are organized. Typically, the ANN training process consists of selecting the appropriate weight values for all the links within the network. Thus, ANN can approximate any function and can be considered as a universal approximator, which is established by the Universal Approximation Theorem.
For more information on the proof of the Universal Approximation Theorem, take a look at the following papers:
- Cybenko, G. (1989) Approximations by Superpositions of Sigmoidal Functions, Mathematics of Control...
