Genetic algorithms are a family of search algorithms that are inspired by the principles of evolution in nature. By imitating the process of natural selection and reproduction, genetic algorithms can produce high-quality solutions for various problems involving search, optimization, and learning. At the same time, their analogy to natural evolution allows genetic algorithms to overcome some of the hurdles that are encountered by traditional search and optimization algorithms, especially for problems with a large number of parameters and complex mathematical representations.

In the rest of this section, we will review the basic ideas of genetic algorithms, as well as their analogy to the evolutionary processes transpiring in nature.

Darwinian evolution

Genetic algorithms implement a simplified version of the Darwinian evolution that takes place in nature. The principles of the Darwinian evolution theory can be summarized using the following principles...

The building-block hypothesis underlying genetic algorithms is that the optimal solution to the problem at hand is assembled of small building blocks, and as we bring more of these building blocks together, we get closer to this optimal solution.

Individuals in the population who contain some of the desired building blocks are identified by their superior scores. The repeated operations of selection and crossover result in better individuals conveying these building blocks to the next generations, while possibly combining them with other successful building blocks. This creates genetic pressure, thus guiding the population toward having more and more individuals with the building blocks that form the optimal solution.

As a result, each generation is better than the previous one and contains more individuals that are closer to the optimal solution.

For example, consider a population of four-digit binary strings where our goal is to find...

There are several important differences between genetic algorithms and traditional search and optimization algorithms, such as gradient-based algorithms.

The key distinguishing factors are as follows:

• Maintaining a population of solutions
• Using a genetic representation of the solutions
• Utilizing the outcome of a fitness function
• Exhibiting a probabilistic behavior

We will describe these factors in greater detail in the following sections.

Population-based

The genetic search is conducted over a population of candidate solutions (individuals) rather than a single candidate. At any point in the search, the algorithm retains a set of individuals that form the current generation. Each iteration of the genetic algorithm creates the next generation of individuals.

In contrast, most other search algorithms maintain a single solution and iteratively modify it in search of the best solution. The gradient descent algorithm...

The unique characteristics of genetic algorithms that we discussed in the previous sections provide several advantages over traditional search algorithms.

The main advantages of genetic algorithms are as follows:

• Global optimization capability
• Can handle problems with a complex mathematical representation
• Can handle problems that lack mathematical representation
• Resilience to noise
• Support for parallelism and distributed processing
• Suitable for continuous learning

We will cover each of these in the upcoming sections.

Global optimization

In many cases, optimization problems have local maxima and minima points; these represent solutions that are better than those around them, but not the best overall.

The following figure illustrates the differences between global and local maximum and minimum points:

Figure 1.6: The global and local maxima and minima of a function.

Source:...

To get the most out of genetic algorithms, we need to be aware of their limitations and potential pitfalls.

The limitations of genetic algorithms are as follows:

• The need for special definitions
• The need for hyperparameter tuning
• Computationally intensive operations
• The risk of premature convergence
• No guaranteed solution

We will cover each of these in the upcoming sections.

Special definitions

When applying genetic algorithms to a given problem, we need to create a suitable representation for them – define the fitness function and the chromosome structure, as well as the selection, crossover, and mutation operators that will work for this problem. This can often prove to be challenging and time-consuming.

Luckily, genetic algorithms have already been applied to countless different types of problems, and many of these definitions have been standardized. This book covers numerous types of real-life problems...

Based on the material we covered in the previous sections, genetic algorithms are best suited for the following types of problems:

• Problems with complex mathematical representation: Since genetic algorithms only require the outcome of the fitness function, they can be used for problems with target functions that are hard or impossible to differentiate (such as planning and scheduling), problems with a large number of parameters (such as image reconstruction), and problems with a mix of parameter types (such as hyperparameter optimization).
• Problems with no mathematical representation: Genetic algorithms don’t require a mathematical representation of the problem, so long as a score value can be obtained, or a method is available to compare two solutions. This can be useful, for example, when solving reinforcement learning tasks or optimizing the architecture of a deep learning model.
• Problems involving a noisy environment: Genetic...

In this chapter, we started by introducing genetic algorithms, their analogy to Darwinian evolution, and their basic principles of operation, including the use of population, genotype, the fitness function, and the genetic operators of selection, crossover, and mutation.

Then, we covered the theory underlying genetic algorithms by going over the building-block hypothesis and the schema theorem and illustrating how genetic algorithms work by bringing together superior, small building blocks to create the best solutions.

Next, we went over the differences between genetic algorithms and traditional ones, such as maintaining a population of solutions and using a genetic representation of those solutions.

We continued by covering the strengths of genetic algorithms, including their capacity for global optimization, handling problems with complex or non-existent mathematical representations, and resilience to noise, followed by their weaknesses, including the need for special...

For more information on what we covered in this chapter, please refer to Introduction to Genetic Algorithms, from the book Hands-On Artificial Intelligence for IoT, by Amita Kapoor, January 2019, available at https://subscription.packtpub.com/book/big_data_and_business_intelligence/9781788836067.