Laboratorio di Intelligenza Artificiale e Robotica A.A. 2008-2009
Outline 2 Machine Learning Unsupervised Learning Supervised Learning Reinforcement Learning Genetic Algorithms Genetics-Based Machine Learning Applications Computational Intelligence and Games Machine Learning for Embedded Systems Design
What is Machine Learning?
What is Machine Learning? 4 The field of machine learning is concerned with the question of how to construct computer programs that automatically improve with experience. Tom Mitchell (1997) A program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E. A well-defined learning task is defined by P, T, and E.
Example: checkers 5 Task T: playing checkers Artificial Intelligence Design and implement a computer-based system that exhibit intelligent action Machine Learning Write a program that can learn how to play It can learn from examples of previous games, by playing against another opponent, by playing against itself
Unsupervised Learning
7
Supervised Learning
What is an apple? 9
What is an apple? 10
Reinforcement Learning
RL framework 12 reward...delay state Agent action Environment
Optimal value function 13 The problem becomes learning how to maximize the amount of reward collected (payoff) Usually RL algorithms try to learn the action-value function Q(s,a) and derive from it the optimal policy
Genetic Algorithms
The population 15
Ranking by fitness 16
Mate Selection 17 Selection probability is proportional to fitness
Crossover 18 Exploit goodness of parents
Mutation 19 Explore unknown
Best solution 20
The GA recipe 21
Genetic Algorithms + Machine Learning = Genetic Based Machine Learning
Examples of GBML 23 Neuroevolution Classifiers structure and parameters optimization with GAs Feature selection with GAs Rule-based Evolutionary Systems GA-based approach for training classifiers
GBML projects 24 Goal Implementation of a GBML approach, or extension of an available implementation, or comparison of different approaches Experimental analysis Technical report Skills Basic Knowledge of EC and ML Good analysis capabilities Good programming skills (C++) Students: max 2
Computational Intelligence and Games
Why games? 26 Modern computer games are ideal benchmarks for computational intelligence techniques Inexpensive Challenging Computational intelligence could be useful for modern games Speedup AI design Improvement of AI Adaptive AI Customization of game content Neverwinter Nights Civilization II Unreal Tournament Urban Combat
Issues in CIG 27 CIG is an attractive field but involves several technical issues: Interface Experiments design Simulation speed Customization Documentation Typical benchmarks are specifically designed for doing research Modern computer games are very complex and with different goals
Projects available on TORCS 28 TORCS is an open source car racing simulator Well suited for CIG research Software API available Examples available Competitions Available projects Automated generation of tracks Visual input for TORCS C++ Learning API Learn your bot!
SmarTrack 29 Goal Generation, evaluation and classification of tracks for TORCS Skills Good programming skills (C++) Good analysis capabilities Basic knowledge of EC and/or ML Students: max 3 Reference http://cig.dei.polimi.it http://torcs.sourceforge.net/
EyeBot 30 Goal Develop a TORCS controller based on visual inputs Skills Some knowledge/experience with computer vision is preferred Good programming skills (C++) Students: max 2 Reference http://cig.dei.polimi.it http://torcs.sourceforge.net/
C++ Learning API 31 Goal Developing a learning framework for TORCS (partially as porting of an existing JAVA API) Skills Basic knowledge of EC and/or ML Good programming skills (C++) Students: max 2 Reference http://cig.dei.polimi.it http://torcs.sourceforge.net/
Learn your bot! 32 Goal Apply any evolutionary computation or machine learning techniques to develop a controller for TORCS Provide the software, the controller and a presentation Skills Knowledge of EC and/or ML Good analysis capabilities Basic programming skills (C++) Students: max 1 Reference http://cig.dei.polimi.it http://torcs.sourceforge.net/
Racing Games for CIG research 33 Goal Analysis of racing games for research purposes Comparison of different games Prototype and tutorial for a specific game Skills Good programming skill Good analysis capabilities Reference http://www.rfactor.net/ http://vdrift.net/
Projects on Unreal Tournament 34 UT is a commercial FPS Easy to extend through UnrealScript GameBots API available for UT2004 Projects BotPrize Analysis and extension of GameBots
The Botprize 35 A Turing test for FPS bots! Goal Analysis of GameBots and of the state of the art Submit an entry to the competition Skills Good programming skills (Java) Strong commitment Students: 4-6 Deadlines Registration: Sunday 5 October Intermediate deadline: Friday 31 October Final deadline: 5 December Reference http://botprize.org/
GameBots for CIG research 36 Goal Analysis of GameBots for CIG research Extending GameBots Example of machine learning application Skills Good programming (Java) Knowledge of EC and ML Reference https://artemis.ms.mff.cuni.cz/pogamut/tiki-index.php
Machine Learning For Embedded Systems Design
Overview 38 Embedded systems are today more and more complex There is need of automated tools to support their design We focus on Performance models Design space exploration for high level synthesis
Performance model 39 Goal Generation/analysis/pre-processing of the dataset Application of a supervised learning technique Experimental analysis Skills Good knowledge of ML Good analysis capabilities Reference http://trac.elet.polimi.it/panda/ Additional contact: Ing. Marco Lattuada, lattuada@elet.polimi.it
High Level Synthesis High-Level Synthesis means going from an algorithmic level specification of a behaviour of a digital system to a register level structure that implements that behavior. McFarland, et al., Proc. IEEE, February 1990. Resource Library Behavioral specification Design constraints Objectives Scheduling High-Level Synthesis tool Allocation Binding Datapath & Controller Controller Synthesis
EC approaches for HLS 41 Goal Integration of an EC-based optimization process into the HLS framework Experimental analysis Technical report Skills Good programming (C++) Good analysis capabilities Reference http://trac.elet.polimi.it/panda Additional contact: Ing. Christian Pilato, pilato@elet.polimi.it
Contacts 42 Email: loiacono@elet.polimi.it Web: http://home.dei.polimi.it/loiacono Stanza 146, Primo Piano, DEI, Tel. 3540 When and how to start? Have a look to the references in the slide Drop me an email for additional info on a specific project Select one or two projects you are interested and send me an email to arrange a meeting on them Ideal start of the projects would be first week of November