Book Image

Hands-On Neuroevolution with Python.

By : Iaroslav Omelianenko
Book Image

Hands-On Neuroevolution with Python.

By: Iaroslav Omelianenko

Overview of this book

Neuroevolution is a form of artificial intelligence learning that uses evolutionary algorithms to simplify the process of solving complex tasks in domains such as games, robotics, and the simulation of natural processes. This book will give you comprehensive insights into essential neuroevolution concepts and equip you with the skills you need to apply neuroevolution-based algorithms to solve practical, real-world problems. You'll start with learning the key neuroevolution concepts and methods by writing code with Python. You'll also get hands-on experience with popular Python libraries and cover examples of classical reinforcement learning, path planning for autonomous agents, and developing agents to autonomously play Atari games. Next, you'll learn to solve common and not-so-common challenges in natural computing using neuroevolution-based algorithms. Later, you'll understand how to apply neuroevolution strategies to existing neural network designs to improve training and inference performance. Finally, you'll gain clear insights into the topology of neural networks and how neuroevolution allows you to develop complex networks, starting with simple ones. By the end of this book, you will not only have explored existing neuroevolution-based algorithms, but also have the skills you need to apply them in your research and work assignments.

Related Content you might be interested in

Current Title:

Small book image
Hands-On Neuroevolution with Python.
Iaroslav Omelianenko
Dec, 2019 | 368 pages
Small book image
Spring 5.0 Projects
Nilang Patel
Feb, 2019 | 442 pages
Table of Contents (18 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Get in touch
Free Chapter
1
Section 1: Fundamentals of Evolutionary Computation Algorithms and Neuroevolution Methods
Section 1: Fundamentals of Evolutionary Computation Algorithms and Neuroevolution Methods
2
Overview of Neuroevolution Methods
Overview of Neuroevolution Methods
Evolutionary algorithms and neuroevolution-based methods
NEAT algorithm overview
Hypercube-based NEAT
Evolvable-Substrate HyperNEAT
Novelty Search optimization method
Summary
Further reading
3
Python Libraries and Environment Setup
Python Libraries and Environment Setup
Suitable Python libraries for neuroevolution experiments
Environment setup
Summary
4
Section 2: Applying Neuroevolution Methods to Solve Classic Computer Science Problems
Section 2: Applying Neuroevolution Methods to Solve Classic Computer Science Problems
5
Using NEAT for XOR Solver Optimization
Using NEAT for XOR Solver Optimization
Technical requirements
XOR problem basics
The objective function for the XOR experiment
Hyperparameter selection
Running the XOR experiment
Exercises
Summary
6
Pole-Balancing Experiments
Pole-Balancing Experiments
Technical requirements
The single-pole balancing problem
Objective function for a single-pole balancing experiment
The single-pole balancing experiment
Exercises
The double-pole balancing problem
Objective function for a double-pole balancing experiment
Double-pole balancing experiment
Exercises
Summary
7
Autonomous Maze Navigation
Autonomous Maze Navigation
Technical requirements
Maze navigation problem
Maze simulation environment
Objective function definition using the fitness score
Running the experiment with a simple maze configuration
Exercises
Running the experiment with a hard-to-solve maze configuration
Exercises
Summary
8
Novelty Search Optimization Method
Novelty Search Optimization Method
Technical requirements
The NS optimization method
NS implementation basics
The fitness function with the novelty score
Experimenting with a simple maze configuration
Experimenting with a hard-to-solve maze configuration
Summary
9
Section 3: Advanced Neuroevolution Methods
Section 3: Advanced Neuroevolution Methods
10
Hypercube-Based NEAT for Visual Discrimination
Hypercube-Based NEAT for Visual Discrimination
Technical requirements
Indirect encoding of ANNs with CPPNs
Visual discrimination experiment basics
Visual discrimination experiment setup
Visual discrimination experiment
Exercises
Summary
11
ES-HyperNEAT and the Retina Problem
ES-HyperNEAT and the Retina Problem
Technical requirements
Manual versus evolution-based configuration of the topography of neural nodes
Quadtree information extraction and ES-HyperNEAT basics
Modular retina problem basics
Modular retina experiment setup
Modular retina experiment
Exercises
Summary
12
Co-Evolution and the SAFE Method
Co-Evolution and the SAFE Method
Technical requirements
Common co-evolution strategies
SAFE method
Modified maze experiment
Modified Novelty Search
Modified maze experiment implementation
Modified maze experiment
Exercises
Summary
13
Deep Neuroevolution
Deep Neuroevolution
Technical requirements
Deep neuroevolution for deep reinforcement learning
Evolving an agent to play the Frostbite Atari game using deep neuroevolution
Training an agent to play the Frostbite game
Running the Frostbite Atari experiment
Visual inspector for neuroevolution
Exercises
Summary
14
Section 4: Discussion and Concluding Remarks
Section 4: Discussion and Concluding Remarks
15
Best Practices, Tips, and Tricks
Best Practices, Tips, and Tricks
Starting with problem analysis
Selecting the optimal search optimization method
Advanced visualization
Tuning hyperparameters
Performance metrics
Python coding tips and tricks
Summary
16
Concluding Remarks
Concluding Remarks
What we learned in this book
Where to go from here
Summary
17
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

Common co-evolution strategies

The natural evolution of biological systems cannot be considered separately from the concept of co-evolution. Co-evolution is one of the central evolutionary drives that leads to the current state of the biosphere, with the diversity of organisms that we can perceive around us.

We can define co-evolution as a mutually beneficial strategy of the simultaneous evolution of multiple genealogies of different organisms. The evolution of one species cannot be possible without other species. During evolution, the co-evolving species mutually interact, and these inter-species relations shape their evolutionary strategy.

There are three main types of co-evolution:

  • Mutualism is when two or more species coexist and mutually benefit each other.
  • Competitive co-evolution:
    • Predation is when one organism kills another and consumes its resources.
    • Parasitism is when...
Previous Section
End of Section 3
Next Section