Book Image

Hands-On Neuroevolution with Python

By : Iaroslav Omelianenko
Book Image

Hands-On Neuroevolution with Python

By: Iaroslav Omelianenko

Overview of this book

Neuroevolution is a form of artificial intelligence learning that uses evolutionary algorithms to simplify the process of solving complex tasks in domains such as games, robotics, and the simulation of natural processes. This book will give you comprehensive insights into essential neuroevolution concepts and equip you with the skills you need to apply neuroevolution-based algorithms to solve practical, real-world problems. You'll start with learning the key neuroevolution concepts and methods by writing code with Python. You'll also get hands-on experience with popular Python libraries and cover examples of classical reinforcement learning, path planning for autonomous agents, and developing agents to autonomously play Atari games. Next, you'll learn to solve common and not-so-common challenges in natural computing using neuroevolution-based algorithms. Later, you'll understand how to apply neuroevolution strategies to existing neural network designs to improve training and inference performance. Finally, you'll gain clear insights into the topology of neural networks and how neuroevolution allows you to develop complex networks, starting with simple ones. By the end of this book, you will not only have explored existing neuroevolution-based algorithms, but also have the skills you need to apply them in your research and work assignments.

Related Content you might be interested in

Current Title:

Small book image
Hands-On Neuroevolution with Python
Iaroslav Omelianenko
Dec, 2019 | 368 pages
Small book image
Hands-On Genetic Algorithms with Python
Eyal Wirsansky
Jan, 2020 | 346 pages
Small book image
The Reinforcement Learning Workshop
Saikat Basak | Quan Nguyen | Anand N S | Mayur Kulkarni | Anthony So | Emanuele Ghelfi | Aritra Sen | Alessandro Palmas | Alexandra Galina Petre
Aug, 2020 | 822 pages
Table of Contents (18 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Get in touch
Free Chapter
1
Section 1: Fundamentals of Evolutionary Computation Algorithms and Neuroevolution Methods
Section 1: Fundamentals of Evolutionary Computation Algorithms and Neuroevolution Methods
2
Overview of Neuroevolution Methods
Overview of Neuroevolution Methods
Evolutionary algorithms and neuroevolution-based methods
NEAT algorithm overview
Hypercube-based NEAT
Evolvable-Substrate HyperNEAT
Novelty Search optimization method
Summary
Further reading
3
Python Libraries and Environment Setup
Python Libraries and Environment Setup
Suitable Python libraries for neuroevolution experiments
Environment setup
Summary
4
Section 2: Applying Neuroevolution Methods to Solve Classic Computer Science Problems
Section 2: Applying Neuroevolution Methods to Solve Classic Computer Science Problems
5
Using NEAT for XOR Solver Optimization
Using NEAT for XOR Solver Optimization
Technical requirements
XOR problem basics
The objective function for the XOR experiment
Hyperparameter selection
Running the XOR experiment
Exercises
Summary
6
Pole-Balancing Experiments
Pole-Balancing Experiments
Technical requirements
The single-pole balancing problem
Objective function for a single-pole balancing experiment
The single-pole balancing experiment
Exercises
The double-pole balancing problem
Objective function for a double-pole balancing experiment
Double-pole balancing experiment
Exercises
Summary
7
Autonomous Maze Navigation
Autonomous Maze Navigation
Technical requirements
Maze navigation problem
Maze simulation environment
Objective function definition using the fitness score
Running the experiment with a simple maze configuration
Exercises
Running the experiment with a hard-to-solve maze configuration
Exercises
Summary
8
Novelty Search Optimization Method
Novelty Search Optimization Method
Technical requirements
The NS optimization method
NS implementation basics
The fitness function with the novelty score
Experimenting with a simple maze configuration
Experimenting with a hard-to-solve maze configuration
Summary
9
Section 3: Advanced Neuroevolution Methods
Section 3: Advanced Neuroevolution Methods
10
Hypercube-Based NEAT for Visual Discrimination
Hypercube-Based NEAT for Visual Discrimination
Technical requirements
Indirect encoding of ANNs with CPPNs
Visual discrimination experiment basics
Visual discrimination experiment setup
Visual discrimination experiment
Exercises
Summary
11
ES-HyperNEAT and the Retina Problem
ES-HyperNEAT and the Retina Problem
Technical requirements
Manual versus evolution-based configuration of the topography of neural nodes
Quadtree information extraction and ES-HyperNEAT basics
Modular retina problem basics
Modular retina experiment setup
Modular retina experiment
Exercises
Summary
12
Co-Evolution and the SAFE Method
Co-Evolution and the SAFE Method
Technical requirements
Common co-evolution strategies
SAFE method
Modified maze experiment
Modified Novelty Search
Modified maze experiment implementation
Modified maze experiment
Exercises
Summary
13
Deep Neuroevolution
Deep Neuroevolution
Technical requirements
Deep neuroevolution for deep reinforcement learning
Evolving an agent to play the Frostbite Atari game using deep neuroevolution
Training an agent to play the Frostbite game
Running the Frostbite Atari experiment
Visual inspector for neuroevolution
Exercises
Summary
14
Section 4: Discussion and Concluding Remarks
Section 4: Discussion and Concluding Remarks
15
Best Practices, Tips, and Tricks
Best Practices, Tips, and Tricks
Starting with problem analysis
Selecting the optimal search optimization method
Advanced visualization
Tuning hyperparameters
Performance metrics
Python coding tips and tricks
Summary
16
Concluding Remarks
Concluding Remarks
What we learned in this book
Where to go from here
Summary
17
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

Objective function for a double-pole balancing experiment

The objective function for this problem is similar to the objective function defined earlier for the single-pole balancing problem. It is given by the following equations:

In these equations, is the expected number of time steps specified in the configuration of the experiment (100,000), and is the actual number of time steps during which the controller was able to maintain a stable state of the pole balancer within the specified limits.

We use logarithmic scales because most of the trials fail in the first several 100 steps, but we are testing against 100,000 steps. With a logarithmic scale, we have a better distribution of fitness scores, even compared with a small number of steps in failed trials.

The first of the preceding equations defines the loss, which is in the [0,1] range, and the second is a fitness score...

Previous Section
End of Section 8
Next Section