Genetic Algorithms in Robotics Julius Mayer Universität Hamburg Fakultät für Mathematik, Informatik und Naturwissenschaften Technische Aspekte Multimodaler Systeme October 31, 2016 J. Mayer 1
Outline 1. Introduction Motivation Classification 2. Algorithm Overview Phases 3. Application Neuroevolution 4. Discussion Evaluation Conclusion J. Mayer 2
Introduction - Motivation Motivation Problems that are hard to solve with classical optimization because of too many parameters (intractability), large search space, non-differentiable objective functions, varying numbers of variables within the optimization process, a lacking mathematical function specification. [1] J. Mayer 3
Introduction - Classification Classification Biologic: Theory of evolution Adaptation of population to the environment Gradual, hereditary change in the individuals of a species Information storage and transfer through genomes Optimizing population fitness by reproduction [3] J. Mayer 4
Introduction - Classification Classification Biologic: Theory of evolution Adaptation of population to the environment Gradual, hereditary change in the individuals of a species Information storage and transfer through genomes Optimizing population fitness by reproduction [5] J. Mayer 5
Introduction - Classification Classification Algorithmic: Stochastic optimization Can solve NP-hard problems Solution approximation Computational complexity as prohibiting factor e.g. ant colony & particle swarm optimization [7] J. Mayer 6
Algorithm - Overview Algorithm 1. Initialization 2. Evaluation 3. Selection 4. Mutation 5. Terminate J. Mayer 7
Algorithm - Phases Phase 1. Initialization Initialize random population Choose appropriate representation Individuals as encoded problem solutions e.g. real values or binary strings 1,67 2,34 1,04 5,83 2,10 101010011011010010101 [12] J. Mayer 8
Algorithm - Phases Phase 2. Evaluation Choose objective function Decode & evaluate chromosome Find minima in objective function Survival of the fittest [9] J. Mayer 9
Algorithm - Phases Phase 3. Selection Probabilistic selection Roulette wheel selection Tournament Selection Balance exploration & exploitation dilemma J. Mayer 10
Algorithm - Phases Phase 4. Mutation Assign mutation rate Mutate Bitflips Normal distribution mutation Recombination Crossover Permutation Mutation drives change [12] J. Mayer 11
Algorithm - Phases Phase 4. Mutation Assign mutation rate Mutate Bitflips Normal distribution mutation Recombination Crossover Permutation Mutation drives change [12] J. Mayer 12
Algorithm - Phases Phase 5. Termination If solution is sufficiently close Performance is satisfying Change is stagnant J. Mayer 13
Application - Evolutionary robotics Path planing Multivariate parameter optimization Evolving artificial network architectures & optimizing connection weights (learning) [6] J. Mayer 14
Application - Neuroevolution Neuroevolution of augmenting topologies (NEAT) Uses crossover Uses speciation to protect structures Increment growth from minimal structures [10] J. Mayer 15
Application - Neuroevolution NEAT: Mutation Connection weights Network structure Add new connection (random weight) Add new node (split connection) [10] J. Mayer 16
Application - Neuroevolution NEAT: Crossover Genes are lined up according to innovation number Matching Genes are randomly chosen Disjoint / excess are taken from fitter parent [10] J. Mayer 17
Application - Neuroevolution NEAT for RoboCup Soccer Keepaway (robot soccer subtask) Large state space (unable to explore exhaustively) Only partial state information for each agent Continuous action space Multiple teammates need to learn simultaneously [4] J. Mayer 18
Application - Neuroevolution NEAT vs. Temporal Difference Methods [11] J. Mayer 19
Application - Neuroevolution NEAT vs. Temporal Difference Methods [11] J. Mayer 20
Discussion - Evaluation Evaluation Challenges Get GA s parameters right Solution encoding Exploration vs. exploitation Premature convergence (local minima) slow convergence big search space Find representative fitness function Drawbacks Approximation instead of exact solution Computational complexity as prohibiting factor No guaranteed convergence to global optimum Non-exhaustive coverage of the complete solution space J. Mayer 21
Discussion - Conclusion Conclusion Inspired by biological evolution Stochastic optimization Population containing encoded problem solutions (parameters) Different applications in robotics e.g. evolving ANN (NEAT) for RoboCup Soccer No guarantee for convergence to global optimum [2] J. Mayer 22
Discussion - Conclusion References [1] http://rednuht.org/genetic walkers/, 2016. [2] https://cdn.meme.am/instances/41036988.jpg. 2016. [3] https://i.ytimg.com/vi/3zjnyscv8to/maxresdefault.jpg, 2016. [4] https://i.ytimg.com/vi/hhln0tdglle/maxresdefault.jpg. 2016. [5] http://www.hanskottke.de/wordpress/wp-content/uploads/2012/09/evolutions hip hop.jpg. 2016. [6] http://www.orocos.org. 2016. [7] http://www.turingfinance.com/wp-content/uploads/2014/10/self-organizing-feature-map-3.png. 2016. [8] John H Holland. Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. U Michigan Press, 1975. [9] Melanie Mitchell. Complexity: A Guided Tour. In Oxford, volume 1, chapter Cellular A, pages 145 159. Oxford University Press, Inc, New York, 2009. J. Mayer 23
Discussion - Conclusion References (cont.) [10] Kenneth O Stanley and Risto Miikkulainen. Evolving neural networks through augmenting topologies. Evolutionary computation, 10(2):99 127, 2002. [11] Matthew E Taylor. Comparing Evolutionary and Temporal Difference Methods in a Reinforcement Learning Domain. pages 1321 1328. [12] Mattias Wahde. Biologically Inspired Optimization Methods: An introduction. WIT Press, Boston, MA, 2008. J. Mayer 24