Neural network learning algorithm optimization
The procedure used to carry out the learning process in a neural network is called the training algorithm. The learning algorithm is what the machine learning algorithm chooses as model with the best optimization. The aim is to minimize the loss function and provide more accuracy. Here we illustrate some of the optimization techniques, other than gradient descent.
The Particle Swarm Optimization (PSO) method is inspired by observations of social and collective behavior on the movements of bird flocks in search of food or survival. It is similar to a fish school trying to move together. We know the position and velocity of the particles, and PSO aims at searching a solution set in a large space controlled by mathematical equations on position and velocity. It is bio-inspired from biological organism behavior for collective intelligence.
Simulated annealing is a method that works on a probabilistic approach to approximate the global optimum for...
