Laboratorio di Intelligenza Artificiale e Robotica A.A. 2008-2009
Outline 2 Machine Learning Unsupervised Learning Supervised Learning Reinforcement Learning Genetic Algorithms Genetics-Based Machine Learning Applications Computational Intelligence and Games Machine Learning for Embedded Systems Design
What is Machine Learning?
What is Machine Learning? 4 The field of machine learning is concerned with the question of how to construct computer programs that automatically improve with experience. Tom Mitchell (1997) A program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E. A well-defined learning task is defined by P, T, and E.
Example: checkers 5 Task T: playing checkers Artificial Intelligence Design and implement a computer-based system that exhibit intelligent action Machine Learning Write a program that can learn how to play It can learn from examples of previous games, by playing against another opponent, by playing against itself
Unsupervised Learning
7
Supervised Learning
What is an apple? 9
What is an apple? 10
Reinforcement Learning
RL framework 12 reward...delay state Agent action Environment
Optimal value function 13 The problem becomes learning how to maximize the amount of reward collected (payoff) Usually RL algorithms try to learn the action-value function Q(s,a) and derive from it the optimal policy
Genetic Algorithms
The population 15
Ranking by fitness 16
Mate Selection 17 Selection probability is proportional to fitness
Crossover 18 Exploit goodness of parents
Mutation 19 Explore unknown
Best solution 20
The GA recipe 21
Genetic Algorithms + Machine Learning = Genetic Based Machine Learning
Examples of GBML 23 Neuroevolution Classifiers structure and parameters optimization with GAs Feature selection with GAs Rule-based Evolutionary Systems GA-based approach for training classifiers
GBML projects 24 Goal Implementation of a GBML approach, or extension of an available implementation, or comparison of different approaches Experimental analysis Technical report Skills Basic Knowledge of EC and ML Good analysis capabilities Good programming skills (C++) Students: max 2
Computational Intelligence and Games
Why games? 26 Modern computer games are ideal benchmarks for computational intelligence techniques Inexpensive Challenging Computational intelligence could be useful for modern games Speedup AI design Improvement of AI Adaptive AI Customization of game content Neverwinter Nights Civilization II Unreal Tournament Urban Combat
Issues in CIG 27 CIG is an attractive field but involves several technical issues: Interface Experiments design Simulation speed Customization Documentation Typical benchmarks are specifically designed for doing research Modern computer games are very complex and with different goals
Projects available on TORCS 28 TORCS is an open source car racing simulator Well suited for CIG research Software API available Examples available Competitions Available projects Automated generation of tracks Visual input for TORCS C++ Learning API Learn your bot!
SmarTrack 29 Goal Generation, evaluation and classification of tracks for TORCS Skills Good programming skills (C++) Good analysis capabilities Basic knowledge of EC and/or ML Students: max 3 Reference http://cig.dei.polimi.it http://torcs.sourceforge.net/
EyeBot 30 Goal Develop a TORCS controller based on visual inputs Skills Some knowledge/experience with computer vision is preferred Good programming skills (C++) Students: max 2 Reference http://cig.dei.polimi.it http://torcs.sourceforge.net/
C++ Learning API 31 Goal Developing a learning framework for TORCS (partially as porting of an existing JAVA API) Skills Basic knowledge of EC and/or ML Good programming skills (C++) Students: max 2 Reference http://cig.dei.polimi.it http://torcs.sourceforge.net/
Learn your bot! 32 Goal Apply any evolutionary computation or machine learning techniques to develop a controller for TORCS Provide the software, the controller and a presentation Skills Knowledge of EC and/or ML Good analysis capabilities Basic programming skills (C++) Students: max 1 Reference http://cig.dei.polimi.it http://torcs.sourceforge.net/
Racing Games for CIG research 33 Goal Analysis of racing games for research purposes Comparison of different games Prototype and tutorial for a specific game Skills Good programming skill Good analysis capabilities Reference http://www.rfactor.net/ http://vdrift.net/