August 29: Lecture 1

Transcription

1 August 29: Lecture 1

2 Machine Learning Fall 2011 Machine learning is a wide collection of concepts and techniques that allows machines to improve their performance at a given task, using 'experience' or 'guidance'. It is a dynamic and evolving field with numerous successes in everyday computing. Machine Learning in Action The Intuition of ML The Definition of ML ML and other disciplines Design Challenges

3 Spot the ML. Labeling as SPAM helps gmail learn to classify correctly future SPAM . This combines the power of many people. Gmail learns which s you consider most important. Helps busy people. Connection with Human Computer Interaction.

4 Spot the ML. User starts tagging people, software learns quickly and makes clusters of faces that appear similar. It even makes suggestions to further teach itself when it has doubts. Connection with Computer Vision. Gmail learns which s you consider most important. Helps busy people

5 Spot the ML..

6 Spot the ML. Playing backgammon against a program, that learned how to play. It can analyze the game and find error rates, good and bad moves, etc. It plays better than most humans. Now humans can learn from the computer!

7 The intuition of ML What do you need to write a chess program? Graphics are secondary. The computer uses an internal representation with symbols. Encode the valid moves. Ready to play! When possible avoid moves that give away your pieces and make moves that kill opponent s pieces..etc

8 The intuition of ML Most people, even people who play good chess apply a set of rules if then else when they play. Very often when a Grandmaster is asked to justify a move they can t verbalize it. They say it s intuition. In a way, many many neurons are fine tuned to respond to different chessboard patterns.

9 The intuition of ML at a high level A fixed algorithm is like a pipe. The same motions, or inputs will always produce the same output. Machine Learning techniques are like algorithms with freedom of tune. Many many parameters are being constantly readjusted based on some sort of feedback. The goal is to make the algorithm sound good (perform better).

10 The definition of ML well-posed learning problems Words like learning are usually imprecise! We need a rigorous definition for Machine Learning. Definition: A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.

11 The definition of ML well-posed learning problems Definition: A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E. A checkers learning problem: Task T: playing checkers Performance Measure P: percent of games won against opponents. Training Experience E: playing practice games against itself.

12 The definition of ML well-posed learning problems Definition: A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E. A handwriting recognition problem: Task T: recognizing and classifying handwritten words within images Performance Measure P: percent of words correctly classified. Training Experience E: a database of handwritten words with given classifications

13 The definition of ML well-posed learning problems Definition: A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E. A robot driving learning problem: Task T: driving on public four-lane highways using vision sensors. Performance Measure P: average distance traveled before an error (as judged by humans) Training Experience E: a sequence of images and steering commands recorded while observing a human driver.

14 Connections of ML with other disciplines Philosophy: Ockham s razor suggesting that the simplest hypothesis is the best. Computational Complexity: How much computational effort, or training examples, or mistakes are required to learn a given task? Biology and Evolution: Modern theories attempt a new explanation and quantification of evolution, by viewing as a learning process. Leslie Valiant Psychology and neurobiology, Statistics, Control Theory, Information Theory.. etc

15 Challenge: Design a Learning System for Checkers What training experience should the system have? A design choice with great impact on the outcome Choice #1: Direct training examples. Just a bunch of board states together with a correct move. Choice #2: Indirect training. A bunch of recorded games, where the correctness of the moves is inferred by the result of the game. credit assignment for each move (how good or bad)

16 Challenge: Design a Learning System for Checkers What amount of interaction should there be between the system and the supervisor? Choice #1: No freedom. Supervisor provides all training examples. Choice #2: Semi-free. Supervisor provides training examples, system constructs its own examples too, and asks questions to the supervisor in cases of doubt. Choice #3: Total-freedeom. System learns to play completely unsupervised How daring the system should be in exploring new boards?

17 Challenge: Design a Learning System for Checkers Which training examples? There is an huge huge number of possible games. No time to try all possible games. System should learn with examples that it will encounter in the future. For example, if the goal is to beat humans, it should be able to do well in situations that humans encounter when they play (this is hard to achieve in practice).

18 Challenge: Design a Learning System for Checkers Next Class A walk through the design of a full system