Machine Learning and Pattern Recognition Introduction

Transcription

1 Machine Learning and Pattern Recognition Introduction Giovanni Maria Farinella

2 What is ML & PR? Interdisciplinary field focusing on both the mathematical foundations and practical applications of systems that learn, reason and act. ML & PR

3 Definition of Machine Learning Arthur Samuel (1959). MachineLearning: Field of study that gives computers the ability to learn without being explicitly programmed. Tom Mitchell (1998) Well-posed Learning Problem: A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E.

4 A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E. SPAM

5 A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E. Suppose your program watches which s you do or do not mark as spam, and based on that learns how to better filter spam. What is the task T in this setting? The task T is classify s as spam or not spam What is the experience E in this problem? Watching the s you have labeled as spam or not spam What is the measure P in this settings? The number of s correctly classified as spam/not spam

6 Why ML and PR are useful?

7 Learning Approaches Imagine an organism/machine which experiences a series of sensory inputs: x (1), x (2), x (n) Supervised learning: the organism/machine is also given desired outputs y (1), y (2),...,y (n), and its goal is to learn to produce the correct output given a new input. Unsupervised learning: the goal of the organism/machine is to build a model of x that can be used for reasoning, decision making,predicting things,etc. Reinforcement learning: the organism/machine can also produce actions a (1), a (2),..a (n), which affect the state of the world, and receives rewards (or punishments) r (1), r (2),... r (n). Its goal is to learn to act in a way that maximizes rewards (or minimize punishments) in the long term.

8 Data Representation

9 Features Classes A feature is the specification of an attribute. It is a measurements which represent the data. For example, color is an attribute. ``Color is blue'' is a feature of an example. The statistical model to be employed is crucially dependent on the choice of features. Hence it is useful to consider alternative representations of the same measurements (i.e. different features).

10 Our Goal

11 Features Types Categorical: A finite number of discrete values. The type nominal denotes that there is no ordering between the values, such as last names and colors. The type ordinal denotes that there is an ordering, such as in an attribute taking on the values low,medium, or high. Continuous (quantitative): Commonly, subset of real numbers, where there is a measurable difference between the possible values. Integers are usually treated as continuous inpractical problems. Hybrid: set of feature composed by both categorical and continuous features

12 Examples (Height,Weight) à BMI (Height,Weight) à {normal, overweight} (Prod1, Prod2,,ProdK) à Find Groups (Rating1, Rating2,, RatingN) à Find Groups Q1,Q2,..,QN à {PD, PDL, M5S, } Q1,Q2,..,QM à % ImageàLike

13 Learning Approach: Supervised Learning x " ($) Notazione x ($) =(x & ($), x " ($) )=(long (4), lat (4) ) n=8 à i=1,,n d=2 y ($) ϵ{0,1}= {Italian,Japanise} x & ($)

14 Typical Supervised Learning Problems Regression: the desidered output consist of one o more continuous variables Classification/Recognition: the aim is to assign each input vector to one of a finite number of category Recommendation systems: seek to predict the rating or preference that a user would give to an item given past ratings/preferences

15 Supervised Learning Problem: Regression y ($) Price ($) in 1000 s Supervised Learning right answers given 0 Hipothesis={h 1,h 2 } h 1 h Size in mq x & ($) Training Set Regression: Predict continuous valued output (price)

16 Supervised Learning Problem: Regression y ($) Price ($) in 1000 s Size in mq x ($) =(x & ($), x " ($). x A ($) )=(? & ($),? " ($).? A ($) ), i=1,,n n=? d=? y ($) =? x & ($) E=Experience=? T=Task =? P=Performances=?

17 Supervised Learning Problem: Regression y ($) Price ($) in 1000 s Size in mq x ($) =(x & ($) )=(Size in mq ($) ), i=1,,n n=11 d=1 y ($) =Price ($) x & ($)

18 Stages

19 Training set Age Supervised Learning Problem: Classification/Recognition Tumor Size More significant features: - Clump Thickness - Uniformity of Cell Size - Uniformity of Cell Shape Homework x ($) =(x & ($), x " ($). x A ($) )=(? & ($),? " ($).? A ($) ), i=1,,n n=? d=? y ($) =? E=Experience=? T=Task =? P=Performances=?

20 Training and Test Sets

21 Training and Test Sets For Learning For Evaluation

22 Training, Validation and Test Sets For Learning For Tuning For Evaluation

23 Evaluation - Common Data Split Validation 5-fold Cross-validation: iteratesover the choice of which fold isthe validation fold, separately from 1-5. The bestmodel ishence evaluated on test data.

24 Supervised Learning Problem: Recommandators Predicting movie ratings User rates movies using zero to five stars Training set Romance Action Movie Alice (1) Bob (2) Carol (3) Dave (4) Love at last Romance forever 5 4.5?? 0 0 Cute puppies of love? ? Nonstop car chases Swords vs. karate ? Nice Boring

25 Supervised Learning: Take Home Message x " ($) input output Training set: x & ($)

26 Learning Approach: Unsupervised Learning x " ($) Build a model of x that can be used for reasoning, decision making, predicting things Clustering Training set: x & ($)

27 Typical Unsupervised Learning Problems Market segmentation: divide a broad target market into subsets of consumers who have common needs and priorities, and then designing and implementing strategies to target them. Social network analysis: views social relationships in terms of network theory, consisting of nodes, representing individual actors within the network, and ties which represent relationships between the individuals, such as friendship, kinship, organizations and sexual relationships

28 Classification by Retrieval K-Nearest Neighbor (KNN) Training must be stored. We need a distance.

29 Classification by Retrieval K-Nearest Neighbor (KNN)

30 Classification by Retrieval

31 The importance of Representation KNN with Euclidean distance on pixel space is not suitable! We need a better representation of the data.