Understanding Population-Based Training

PBT is a population-based heuristic search method, just like the GA method and PSO. However, PBT is not a nature-inspired algorithm like GA or PSO. Instead, inspired by the GA method itself. PBT is suggested for when you are working with a neural-network-based type of model and just need the final trained model without knowing the specifically chosen hyperparameter configurations.

PBT is specifically designed to work only with a neural network-based type of models, such as a multilayer perceptron, deep reinforcement learning, transformers, GAN, and any other neural network-based models. It can be said that PBT does both hyperparameter tuning and model training since the weights of the neural network model are inherited during the process. So, PBT is not only for choosing the most optimal hyperparameter configurations but also for transferring the weights or parameters of the model to other individuals within the population. That’s...