Laboratorio di Intelligenza Artificiale e Robotica

Transcription

1 Laboratorio di Intelligenza Artificiale e Robotica A.A

2 Outline 2 Machine Learning Unsupervised Learning Supervised Learning Reinforcement Learning Genetic Algorithms Genetics-Based Machine Learning Applications Computational Intelligence and Games Machine Learning for Embedded Systems Design

3 What is Machine Learning?

4 What is Machine Learning? 4 The field of machine learning is concerned with the question of how to construct computer programs that automatically improve with experience. Tom Mitchell (1997) A program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E. A well-defined learning task is defined by P, T, and E.

5 Example: checkers 5 Task T: playing checkers Artificial Intelligence Design and implement a computer-based system that exhibit intelligent action Machine Learning Write a program that can learn how to play It can learn from examples of previous games, by playing against another opponent, by playing against itself

6 Unsupervised Learning

7 7

8 Supervised Learning

9 What is an apple? 9

10 What is an apple? 10

11 Reinforcement Learning

12 RL framework 12 reward...delay state Agent action Environment

13 Optimal value function 13 The problem becomes learning how to maximize the amount of reward collected (payoff) Usually RL algorithms try to learn the action-value function Q(s,a) and derive from it the optimal policy

14 Genetic Algorithms

15 The population 15

16 Ranking by fitness 16

17 Mate Selection 17 Selection probability is proportional to fitness

18 Crossover 18 Exploit goodness of parents

19 Mutation 19 Explore unknown

20 Best solution 20

21 The GA recipe 21

22 Genetic Algorithms + Machine Learning = Genetic Based Machine Learning

23 Examples of GBML 23 Neuroevolution Classifiers structure and parameters optimization with GAs Feature selection with GAs Rule-based Evolutionary Systems GA-based approach for training classifiers

24 GBML projects 24 Goal Implementation of a GBML approach, or extension of an available implementation, or comparison of different approaches Experimental analysis Technical report Skills Basic Knowledge of EC and ML Good analysis capabilities Good programming skills (C++) Students: max 2

25 Computational Intelligence and Games

26 Why games? 26 Modern computer games are ideal benchmarks for computational intelligence techniques Inexpensive Challenging Computational intelligence could be useful for modern games Speedup AI design Improvement of AI Adaptive AI Customization of game content Neverwinter Nights Civilization II Unreal Tournament Urban Combat

27 Issues in CIG 27 CIG is an attractive field but involves several technical issues: Interface Experiments design Simulation speed Customization Documentation Typical benchmarks are specifically designed for doing research Modern computer games are very complex and with different goals

28 Projects available on TORCS 28 TORCS is an open source car racing simulator Well suited for CIG research Software API available Examples available Competitions Available projects Automated generation of tracks Visual input for TORCS C++ Learning API Learn your bot!

29 SmarTrack 29 Goal Generation, evaluation and classification of tracks for TORCS Skills Good programming skills (C++) Good analysis capabilities Basic knowledge of EC and/or ML Students: max 3 Reference

30 EyeBot 30 Goal Develop a TORCS controller based on visual inputs Skills Some knowledge/experience with computer vision is preferred Good programming skills (C++) Students: max 2 Reference

31 C++ Learning API 31 Goal Developing a learning framework for TORCS (partially as porting of an existing JAVA API) Skills Basic knowledge of EC and/or ML Good programming skills (C++) Students: max 2 Reference

32 Learn your bot! 32 Goal Apply any evolutionary computation or machine learning techniques to develop a controller for TORCS Provide the software, the controller and a presentation Skills Knowledge of EC and/or ML Good analysis capabilities Basic programming skills (C++) Students: max 1 Reference

33 Racing Games for CIG research 33 Goal Analysis of racing games for research purposes Comparison of different games Prototype and tutorial for a specific game Skills Good programming skill Good analysis capabilities Reference