Evolutionary optimization
Evolutionary optimization makes use of algorithms that mimic the selection process within the natural world. Examples of this are genetic algorithms that optimize via natural selection. Each iteration of a hyperparameter value is like a mutation in genetics that is assessed and altered. The process continues using recombined choices until the most effective configuration is reached. Hence, each generation improves with every iteration as it is optimized. Genetic algorithms are often used to train neural networks.
An evolutionary algorithm typically consists of three steps: initialization, selection, and termination. Fitter generations survive and proliferate, like in natural selection. In general, an initial population of a wide range of solutions is randomly created within the constraints of the problem. The population contains an arbitrary number of possible solutions to the problem, or the solutions are roughly centered around what is believed to be...