Genetic Algorithms in Robotics

Transcription

1 Genetic Algorithms in Robotics Julius Mayer Universität Hamburg Fakultät für Mathematik, Informatik und Naturwissenschaften Technische Aspekte Multimodaler Systeme October 31, 2016 J. Mayer 1

2 Outline 1. Introduction Motivation Classification 2. Algorithm Overview Phases 3. Application Neuroevolution 4. Discussion Evaluation Conclusion J. Mayer 2

3 Introduction - Motivation Motivation Problems that are hard to solve with classical optimization because of too many parameters (intractability), large search space, non-differentiable objective functions, varying numbers of variables within the optimization process, a lacking mathematical function specification. [1] J. Mayer 3

4 Introduction - Classification Classification Biologic: Theory of evolution Adaptation of population to the environment Gradual, hereditary change in the individuals of a species Information storage and transfer through genomes Optimizing population fitness by reproduction [3] J. Mayer 4

5 Introduction - Classification Classification Biologic: Theory of evolution Adaptation of population to the environment Gradual, hereditary change in the individuals of a species Information storage and transfer through genomes Optimizing population fitness by reproduction [5] J. Mayer 5

6 Introduction - Classification Classification Algorithmic: Stochastic optimization Can solve NP-hard problems Solution approximation Computational complexity as prohibiting factor e.g. ant colony & particle swarm optimization [7] J. Mayer 6

7 Algorithm - Overview Algorithm 1. Initialization 2. Evaluation 3. Selection 4. Mutation 5. Terminate J. Mayer 7

8 Algorithm - Phases Phase 1. Initialization Initialize random population Choose appropriate representation Individuals as encoded problem solutions e.g. real values or binary strings 1,67 2,34 1,04 5,83 2, [12] J. Mayer 8

9 Algorithm - Phases Phase 2. Evaluation Choose objective function Decode & evaluate chromosome Find minima in objective function Survival of the fittest [9] J. Mayer 9

10 Algorithm - Phases Phase 3. Selection Probabilistic selection Roulette wheel selection Tournament Selection Balance exploration & exploitation dilemma J. Mayer 10

11 Algorithm - Phases Phase 4. Mutation Assign mutation rate Mutate Bitflips Normal distribution mutation Recombination Crossover Permutation Mutation drives change [12] J. Mayer 11

12 Algorithm - Phases Phase 4. Mutation Assign mutation rate Mutate Bitflips Normal distribution mutation Recombination Crossover Permutation Mutation drives change [12] J. Mayer 12

13 Algorithm - Phases Phase 5. Termination If solution is sufficiently close Performance is satisfying Change is stagnant J. Mayer 13

14 Application - Evolutionary robotics Path planing Multivariate parameter optimization Evolving artificial network architectures & optimizing connection weights (learning) [6] J. Mayer 14

15 Application - Neuroevolution Neuroevolution of augmenting topologies (NEAT) Uses crossover Uses speciation to protect structures Increment growth from minimal structures [10] J. Mayer 15

16 Application - Neuroevolution NEAT: Mutation Connection weights Network structure Add new connection (random weight) Add new node (split connection) [10] J. Mayer 16

17 Application - Neuroevolution NEAT: Crossover Genes are lined up according to innovation number Matching Genes are randomly chosen Disjoint / excess are taken from fitter parent [10] J. Mayer 17

18 Application - Neuroevolution NEAT for RoboCup Soccer Keepaway (robot soccer subtask) Large state space (unable to explore exhaustively) Only partial state information for each agent Continuous action space Multiple teammates need to learn simultaneously [4] J. Mayer 18

19 Application - Neuroevolution NEAT vs. Temporal Difference Methods [11] J. Mayer 19

20 Application - Neuroevolution NEAT vs. Temporal Difference Methods [11] J. Mayer 20

21 Discussion - Evaluation Evaluation Challenges Get GA s parameters right Solution encoding Exploration vs. exploitation Premature convergence (local minima) slow convergence big search space Find representative fitness function Drawbacks Approximation instead of exact solution Computational complexity as prohibiting factor No guaranteed convergence to global optimum Non-exhaustive coverage of the complete solution space J. Mayer 21

22 Discussion - Conclusion Conclusion Inspired by biological evolution Stochastic optimization Population containing encoded problem solutions (parameters) Different applications in robotics e.g. evolving ANN (NEAT) for RoboCup Soccer No guarantee for convergence to global optimum [2] J. Mayer 22

23 Discussion - Conclusion References [1] walkers/, [2] [3] [4] [5] hip hop.jpg [6] [7] [8] John H Holland. Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. U Michigan Press, [9] Melanie Mitchell. Complexity: A Guided Tour. In Oxford, volume 1, chapter Cellular A, pages Oxford University Press, Inc, New York, J. Mayer 23

24 Discussion - Conclusion References (cont.) [10] Kenneth O Stanley and Risto Miikkulainen. Evolving neural networks through augmenting topologies. Evolutionary computation, 10(2):99 127, [11] Matthew E Taylor. Comparing Evolutionary and Temporal Difference Methods in a Reinforcement Learning Domain. pages [12] Mattias Wahde. Biologically Inspired Optimization Methods: An introduction. WIT Press, Boston, MA, J. Mayer 24