Book Image

Hands-On Neuroevolution with Python.

By : Iaroslav Omelianenko
Book Image

Hands-On Neuroevolution with Python.

By: Iaroslav Omelianenko

Overview of this book

Neuroevolution is a form of artificial intelligence learning that uses evolutionary algorithms to simplify the process of solving complex tasks in domains such as games, robotics, and the simulation of natural processes. This book will give you comprehensive insights into essential neuroevolution concepts and equip you with the skills you need to apply neuroevolution-based algorithms to solve practical, real-world problems. You'll start with learning the key neuroevolution concepts and methods by writing code with Python. You'll also get hands-on experience with popular Python libraries and cover examples of classical reinforcement learning, path planning for autonomous agents, and developing agents to autonomously play Atari games. Next, you'll learn to solve common and not-so-common challenges in natural computing using neuroevolution-based algorithms. Later, you'll understand how to apply neuroevolution strategies to existing neural network designs to improve training and inference performance. Finally, you'll gain clear insights into the topology of neural networks and how neuroevolution allows you to develop complex networks, starting with simple ones. By the end of this book, you will not only have explored existing neuroevolution-based algorithms, but also have the skills you need to apply them in your research and work assignments.

Related Content you might be interested in

Current Title:

Small book image
Hands-On Neuroevolution with Python.
Iaroslav Omelianenko
Dec, 2019 | 368 pages
Small book image
Spring 5.0 Projects
Nilang Patel
Feb, 2019 | 442 pages
Table of Contents (18 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Get in touch
Free Chapter
1
Section 1: Fundamentals of Evolutionary Computation Algorithms and Neuroevolution Methods
Section 1: Fundamentals of Evolutionary Computation Algorithms and Neuroevolution Methods
2
Overview of Neuroevolution Methods
Overview of Neuroevolution Methods
Evolutionary algorithms and neuroevolution-based methods
NEAT algorithm overview
Hypercube-based NEAT
Evolvable-Substrate HyperNEAT
Novelty Search optimization method
Summary
Further reading
3
Python Libraries and Environment Setup
Python Libraries and Environment Setup
Suitable Python libraries for neuroevolution experiments
Environment setup
Summary
4
Section 2: Applying Neuroevolution Methods to Solve Classic Computer Science Problems
Section 2: Applying Neuroevolution Methods to Solve Classic Computer Science Problems
5
Using NEAT for XOR Solver Optimization
Using NEAT for XOR Solver Optimization
Technical requirements
XOR problem basics
The objective function for the XOR experiment
Hyperparameter selection
Running the XOR experiment
Exercises
Summary
6
Pole-Balancing Experiments
Pole-Balancing Experiments
Technical requirements
The single-pole balancing problem
Objective function for a single-pole balancing experiment
The single-pole balancing experiment
Exercises
The double-pole balancing problem
Objective function for a double-pole balancing experiment
Double-pole balancing experiment
Exercises
Summary
7
Autonomous Maze Navigation
Autonomous Maze Navigation
Technical requirements
Maze navigation problem
Maze simulation environment
Objective function definition using the fitness score
Running the experiment with a simple maze configuration
Exercises
Running the experiment with a hard-to-solve maze configuration
Exercises
Summary
8
Novelty Search Optimization Method
Novelty Search Optimization Method
Technical requirements
The NS optimization method
NS implementation basics
The fitness function with the novelty score
Experimenting with a simple maze configuration
Experimenting with a hard-to-solve maze configuration
Summary
9
Section 3: Advanced Neuroevolution Methods
Section 3: Advanced Neuroevolution Methods
10
Hypercube-Based NEAT for Visual Discrimination
Hypercube-Based NEAT for Visual Discrimination
Technical requirements
Indirect encoding of ANNs with CPPNs
Visual discrimination experiment basics
Visual discrimination experiment setup
Visual discrimination experiment
Exercises
Summary
11
ES-HyperNEAT and the Retina Problem
ES-HyperNEAT and the Retina Problem
Technical requirements
Manual versus evolution-based configuration of the topography of neural nodes
Quadtree information extraction and ES-HyperNEAT basics
Modular retina problem basics
Modular retina experiment setup
Modular retina experiment
Exercises
Summary
12
Co-Evolution and the SAFE Method
Co-Evolution and the SAFE Method
Technical requirements
Common co-evolution strategies
SAFE method
Modified maze experiment
Modified Novelty Search
Modified maze experiment implementation
Modified maze experiment
Exercises
Summary
13
Deep Neuroevolution
Deep Neuroevolution
Technical requirements
Deep neuroevolution for deep reinforcement learning
Evolving an agent to play the Frostbite Atari game using deep neuroevolution
Training an agent to play the Frostbite game
Running the Frostbite Atari experiment
Visual inspector for neuroevolution
Exercises
Summary
14
Section 4: Discussion and Concluding Remarks
Section 4: Discussion and Concluding Remarks
15
Best Practices, Tips, and Tricks
Best Practices, Tips, and Tricks
Starting with problem analysis
Selecting the optimal search optimization method
Advanced visualization
Tuning hyperparameters
Performance metrics
Python coding tips and tricks
Summary
16
Concluding Remarks
Concluding Remarks
What we learned in this book
Where to go from here
Summary
17
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

Summary

In this chapter, you learned about the Novelty Search optimization method and how it can be used to guide the neuroevolution process in deceptive problem space environments, such as maze navigation. We conducted the same maze navigation experiments as in the previous chapter. After that, we compared the results we obtained to determine if the NS method has advantages over the goal-oriented optimization method introduced in the previous chapter.

You got the practical experience of writing source code using Python and experimented with tuning the important hyperparameters of the NEAT algorithm. Also, we introduced a new visualization method, allowing you to see the path of the agent through the maze. With this method, you can easily compare how different agents are trying to solve the maze navigation problem and whether the path through the maze that was found is optimal...

Previous Section
End of Chapter 8
Next Chapter