# Decision Tree For Playing Tennis

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Decision Tree For Playing Tennis ROOT NODE BRANCH INTERNAL NODE LEAF NODE Disjunction of conjunctions

2 Another Perspective of a Decision Tree Model Age NoDefault NoDefault + + NoDefault Default Age, income Case A. 30,, \$110K, Default Case B. 50,, \$110K, NoDefault Case C. 45,, \$90K, NoDefault Case A. 32,, \$105K, Default Case B. 49,, \$82K, NoDefault Case C. 29,, \$50K, NoDefault Income

3 Top-Down Tree Induction

4 Which Column and Split Point? Multitude of techniques: Entropy/Information gain Chi square test (CHAID) Test of independence GINI index

5 Information Gain

6 Entropy

7 Data Set

8 Choosing the Next Attribute - 1

9 Choosing the Next Attribute - 2

10 Representational and Search Bias

11 Occam s Razor 14 th Century Franciscan friar; William of Occam. The principle states that "Entities should not be multiplied unnecessarily." People often reinvented Occam's Razor Newton - "We are to admit no more causes of natural things than such as are both true and sufficient to explain their appearances." To most scientist the razor is: "when you have two competing theories which make exactly the same predictions, the one that is simpler is the better."

12 Review of Choosing a Split Entropy= Σ-p.log 2 (p) Entropy Population = 1 Entropy Split on Length =0.42 Entropy Split on Thread =0.85

13 Stopping Criteria What type of tree will perfectly classify the training data (ie. 100% training set accuracy)? Is this a bad thing?, Why? What does this tell you about the relationship between the dependent and independent attributes? Stop growing the tree when: A certain tree depth is reached Number of records at a node goes below some threshold. All potential splits are insignificant

14 How Do We Know When We ve Overfitted The Training Data? Is there any other way?

15 Training Set Error Should Approximately Equal Test Set Error

16 Trimming/Pruning Trees Stopping criterion can be some what arbitrary. Automatic pruning of trees Ask the data, How far should we split the data. Two general approaches: Use part of the training set as a validation set Use entire training set (usually an MDL approach).

17 Using Pruning To Prevent Overfitting

18 Reduced Error Pruning

19 Reduced Error Pruning

20 Results of Reduced Error Pruning Consider the use of learning a tree is to make prediction What is the fundamental assumption that this learning algorithm is making

21 Rule Post-Pruning

22 X-Fold Cross Validation Used to estimate the accuracy of the learner. Feature selection for other supervised learning algorithms. Fold 1 Fold 2 Fold 3 Fold 4 Fold 5

23 MDL Base Pruning Minimize Overall Message Length MessLen(Model, Data) = MessLen(Model) + MessLen(Data Model) Encode model using node encoding. Encode model in terms of classification error. Remove a node if it reduces the cost.

24 Ensemble of Decision Trees Why stop at one decision tree. Adopt the committee of experts approach Build multiple decision trees, each votes on the classification, highest vote wins. What problem will we run up against?

25 Why Does it Work? Brieman Works because decision tree learners are unstable. Friedman Reduces the variance of the learner without reducing bias. Domingos Underlying learners bias towards simplicity is too great Bagging corrects bias.

26 C4.5 - Quinlan Goto Download C4.5 Release 8 Need to untar it (use tar xvf) In R8/Src type make all, builds c4.5 executable May need to remove contents of getopt.c file. Use nroff doc/c4.5.1 more to read documentation. See me during office hours if you have any problems.

27 Building a Model Using C4.5 Options c4.5 - form [ -f filestem] [ -u ] [ -s ] [ -p ] [ -v verb ] [ -t trials ] [ -w wsize ] [ -i incr ] [ -g ] [ -m minobjs ] [ -c cf ] C4.5 f golf m 2 outlook = overcast: Play (4.0) outlook = sunny: humidity <= 75 : Play (2.0) humidity > 75 : Don't Play (3.0) outlook = rain: windy = true: Don't Play (2.0) windy = false: Play (3.0) Size Errors 8 0( 0.0%)

28 Building and Applying a Model Using C4.5 Many data sets in the Data directory can are split into.data (training set) and.test (test set). Use c4.5 f <name> -u To build a model and then test it on the training set. (use labor-neg or vote datasets).

29 Model Uncertainty What s wrong with making predictions from one model? May have two or more equally accurate models that give different predictions. May have two models that are quite fundamentally different

30 Ensemble of Models Techniques Bayesian Modeling Averaging Pr(c,x D, H) = Σ h H Pr(c,x h). Pr(h D) Weight each model s prediction by how good the model is. Can this approach be applied to C4.5 Dtrees? Boosting (Bootstrap Aggregation), Improves accuracy Seminal paper says on 19 of 26 data sets improves accuracy by 4%.

31 Bagging Take a number of bootstrap samples of the training set. Build a decision tree from each When predicting the category for a test set instance: Each tree gets to vote on the decision Ties are resolved by choosing the most populous class Empirical evidence shows that you get consistently better results on most data set.

32 The Bagging Algorithm Building the Models For i = 1 to k // k is the number of bags T i =BootStrap(D) // D is the training set Build Model M i from T i (ie. Induce the tree) End Applying the Models To Make a Prediction For a test set example, x For i = 1 to k // k is the number of bags C i =M i (x) End Prediction is the class with the most vote.

33 Take A Bootstrap Sample Sample with replacement Bootstrapping and model building can be easily parallelized

34 Bagging - Results

35 Example of Bagging Problem Single DT Solution 100 DT s Bagging Solution

36 Boosting The Idea Take weak learners (marginally better than random guessing) make them stronger. Freund and Schapire, 95 AdaBoost AdaBoost premise Each training instances has equal weight Build first Model from training instances Training instances that are classified incorrectly given more weight Build another model with re-weighted instances and so on and so on.

37 Boosting Psuedo Code

38 Some Implementation Comments Difficult to parallelize Factoring instance weights into decision tree induction. Tree vote is weighted inversely to error. Adaptive Boosting (AdaBoosting) according to the tree error Free scaled down version of C5.0 incorporates boosting available at

39 Toy Example (Freund COLT 99) Round 1

40 Round 2 + 3

41 Final Hypothesis Demo at

42 Some Insights into Boosting Final aggregate model will have no training error (given some conditions). Seems to over-fit but reduces test set error Larger margins on training set correspond to better generalization error Margin(x) = y Σ α j h j (x) / Σ α j

43 The Performance of Models and Learners Error of the hypothesis vs error of the learning algorithm? Know the training and test set error, good estimate of the learner s performance? Learners Error = noise + bias 2 + variance How we calculate bias and variance for a learner* T 1 n : Training sets drawn randomly from population Bias is the difference in error over all training sets true error. Variance is the variability of the error. Why would a decision tree be biased? Have a high variance?

44 Errors

45 Bias and Variance

46 Retrospective on Decision Trees Representation and search Does Bagging and Boosting change model representation space? Do they change search preference? Order of data presented does not count.

### Decision Tree for Playing Tennis

Decision Tree Decision Tree for Playing Tennis (outlook=sunny, wind=strong, humidity=normal,? ) DT for prediction C-section risks Characteristics of Decision Trees Decision trees have many appealing properties

### Machine Learning B, Fall 2016

Machine Learning 10-601 B, Fall 2016 Decision Trees (Summary) Lecture 2, 08/31/ 2016 Maria-Florina (Nina) Balcan Learning Decision Trees. Supervised Classification. Useful Readings: Mitchell, Chapter 3

### Supervised learning can be done by choosing the hypothesis that is most probable given the data: = arg max ) = arg max

The learning problem is called realizable if the hypothesis space contains the true function; otherwise it is unrealizable On the other hand, in the name of better generalization ability it may be sensible

### Decision Tree Instability and Active Learning

Decision Tree Instability and Active Learning Kenneth Dwyer and Robert Holte University of Alberta November 14, 2007 Kenneth Dwyer, University of Alberta Decision Tree Instability and Active Learning 1

### 18 LEARNING FROM EXAMPLES

18 LEARNING FROM EXAMPLES An intelligent agent may have to learn, for instance, the following components: A direct mapping from conditions on the current state to actions A means to infer relevant properties

### Machine Learning Tom M. Mitchell Machine Learning Department Carnegie Mellon University. January 11, 2011

Machine Learning 10-701 Tom M. Mitchell Machine Learning Department Carnegie Mellon University January 11, 2011 Today: What is machine learning? Decision tree learning Course logistics Readings: The Discipline

### Introduction to Ensemble Learning Featuring Successes in the Netflix Prize Competition

Introduction to Ensemble Learning Featuring Successes in the Netflix Prize Competition Todd Holloway Two Lecture Series for B551 November 20 & 27, 2007 Indiana University Outline Introduction Bias and

### A Few Useful Things to Know about Machine Learning. Pedro Domingos Department of Computer Science and Engineering University of Washington" 2012"

A Few Useful Things to Know about Machine Learning Pedro Domingos Department of Computer Science and Engineering University of Washington 2012 A Few Useful Things to Know about Machine Learning Machine

### Machine Learning for Language Technology

October 2013 Machine Learning for Language Technology Lecture 6: Ensemble Methods Marina Santini, Uppsala University Department of Linguistics and Philology Where we are Previous lectures, various different

### PRESENTATION TITLE. A Two-Step Data Mining Approach for Graduation Outcomes CAIR Conference

PRESENTATION TITLE A Two-Step Data Mining Approach for Graduation Outcomes 2013 CAIR Conference Afshin Karimi (akarimi@fullerton.edu) Ed Sullivan (esullivan@fullerton.edu) James Hershey (jrhershey@fullerton.edu)

### CSC 4510/9010: Applied Machine Learning Rule Inference

CSC 4510/9010: Applied Machine Learning Rule Inference Dr. Paula Matuszek Paula.Matuszek@villanova.edu Paula.Matuszek@gmail.com (610) 647-9789 CSC 4510.9010 Spring 2015. Paula Matuszek 1 Red Tape Going

### P(A, B) = P(A B) = P(A) + P(B) - P(A B)

AND Probability P(A, B) = P(A B) = P(A) + P(B) - P(A B) P(A B) = P(A) + P(B) - P(A B) Area = Probability of Event AND Probability P(A, B) = P(A B) = P(A) + P(B) - P(A B) If, and only if, A and B are independent,

### Multiple classifiers

Multiple classifiers JERZY STEFANOWSKI Institute of Computing Sciences Poznań University of Technology Zajęcia dla TPD - ZED 2009 Oparte na wykładzie dla Doctoral School, Catania-Troina, April, 2008 Outline

### Multiple classifiers. JERZY STEFANOWSKI Institute of Computing Sciences Poznań University of Technology. Doctoral School, Catania-Troina, April, 2008

Multiple classifiers JERZY STEFANOWSKI Institute of Computing Sciences Poznań University of Technology Doctoral School, Catania-Troina, April, 2008 Outline of the presentation 1. Introduction 2. Why do

### Inductive Learning and Decision Trees

Inductive Learning and Decision Trees Doug Downey EECS 349 Spring 2017 with slides from Pedro Domingos, Bryan Pardo Outline Announcements Homework #1 was assigned on Monday (due in five days!) Inductive

### Tanagra Tutorials. Figure 1 Tree size and generalization error rate (Source:

1 Topic Describing the post pruning process during the induction of decision trees (CART algorithm, Breiman and al., 1984 C RT component into TANAGRA). Determining the appropriate size of the tree is a

### Assignment 6 (Sol.) Introduction to Machine Learning Prof. B. Ravindran

Assignment 6 (Sol.) Introduction to Machine Learning Prof. B. Ravindran 1. Assume that you are given a data set and a neural network model trained on the data set. You are asked to build a decision tree

### Course 395: Machine Learning - Lectures

Course 395: Machine Learning - Lectures Lecture 1-2: Concept Learning (M. Pantic) Lecture 3-4: Decision Trees & CBC Intro (M. Pantic & S. Petridis) Lecture 5-6: Evaluating Hypotheses (S. Petridis) Lecture

### ANALYZING BIG DATA WITH DECISION TREES

San Jose State University SJSU ScholarWorks Master's Projects Master's Theses and Graduate Research Spring 2014 ANALYZING BIG DATA WITH DECISION TREES Lok Kei Leong Follow this and additional works at:

### Machine Learning (Decision Trees and Intro to Neural Nets) CSCI 3202, Fall 2010

Machine Learning (Decision Trees and Intro to Neural Nets) CSCI 3202, Fall 2010 Assignments To read this week: Chapter 18, sections 1-4 and 7 Problem Set 3 due next week! Learning a Decision Tree We look

### Analysis of Different Classifiers for Medical Dataset using Various Measures

Analysis of Different for Medical Dataset using Various Measures Payal Dhakate ME Student, Pune, India. K. Rajeswari Associate Professor Pune,India Deepa Abin Assistant Professor, Pune, India ABSTRACT

### CS 4510/9010 Applied Machine Learning. Evaluation. Paula Matuszek Fall, copyright Paula Matuszek 2016

CS 4510/9010 Applied Machine Learning 1 Evaluation Paula Matuszek Fall, 2016 Evaluating Classifiers 2 With a decision tree, or with any classifier, we need to know how well our trained model performs on

### Cost-Sensitive Learning and the Class Imbalance Problem

To appear in Encyclopedia of Machine Learning. C. Sammut (Ed.). Springer. 2008 Cost-Sensitive Learning and the Class Imbalance Problem Charles X. Ling, Victor S. Sheng The University of Western Ontario,

### Machine Learning Tom M. Mitchell Machine Learning Department Carnegie Mellon University. January 12, 2015

Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University January 12, 2015 Today: What is machine learning? Decision tree learning Course logistics Readings: The Discipline

### Inductive Learning and Decision Trees

Inductive Learning and Decision Trees Doug Downey EECS 349 Winter 2014 with slides from Pedro Domingos, Bryan Pardo Outline Announcements Homework #1 assigned Have you completed it? Inductive learning

### An Empirical Study of Combining Boosting-BAN and Boosting-MultiTAN

Research Journal of Applied Sciences, Engineering and Technology 5(24): 5550-5555, 2013 ISSN: 2040-7459; e-issn: 2040-7467 Maxwell Scientific Organization, 2013 Submitted: September 24, 2012 Accepted:

### Decision Boundary. Hemant Ishwaran and J. Sunil Rao

32 Decision Trees, Advanced Techniques in Constructing define impurity using the log-rank test. As in CART, growing a tree by reducing impurity ensures that terminal nodes are populated by individuals

### Dudon Wai Georgia Institute of Technology CS 7641: Machine Learning Atlanta, GA

Adult Income and Letter Recognition - Supervised Learning Report An objective look at classifier performance for predicting adult income and Letter Recognition Dudon Wai Georgia Institute of Technology

### 10701/15781 Machine Learning, Spring 2005: Homework 1

10701/15781 Machine Learning, Spring 2005: Homework 1 Due: Monday, February 6, beginning of the class 1 [15 Points] Probability and Regression [Stano] 1 1.1 [10 Points] The Matrix Strikes Back The Matrix

### Introduction to Classification

Introduction to Classification Classification: Definition Given a collection of examples (training set ) Each example is represented by a set of features, sometimes called attributes Each example is to

### Foundations of Intelligent Systems CSCI (Fall 2015)

Foundations of Intelligent Systems CSCI-630-01 (Fall 2015) Final Examination, Fri. Dec 18, 2015 Instructor: Richard Zanibbi, Duration: 120 Minutes Name: Instructions The exam questions are worth a total

### IAI : Machine Learning

IAI : Machine Learning John A. Bullinaria, 2005 1. What is Machine Learning? 2. The Need for Learning 3. Learning in Neural and Evolutionary Systems 4. Problems Facing Expert Systems 5. Learning in Rule

### Applied Machine Learning Lecture 1: Introduction

Applied Machine Learning Lecture 1: Introduction Richard Johansson January 16, 2018 welcome to the course! machine learning is getting increasingly popular among students our courses are full! many thesis

### CS 354R: Computer Game Technology

CS 354R: Computer Game Technology AI Decision Trees and Rule Systems Fall 2017 Decision Trees Nodes represent attribute tests One child for each outcome Leaves represent classifications Can have same classification

### Online Ensemble Learning: An Empirical Study

Online Ensemble Learning: An Empirical Study Alan Fern AFERN@ECN.PURDUE.EDU Robert Givan GIVAN@ECN.PURDUE.EDU Department of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 4797

### Online Ensemble Learning: An Empirical Study

Online Ensemble Learning: An Empirical Study Alan Fern AFERN@ECN.PURDUE.EDU Robert Givan GIVAN@ECN.PURDUE.EDU Department of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 4797

### Big Data Analytics Clustering and Classification

E6893 Big Data Analytics Lecture 4: Big Data Analytics Clustering and Classification Ching-Yung Lin, Ph.D. Adjunct Professor, Dept. of Electrical Engineering and Computer Science September 28th, 2017 1

### Practical Methods for the Analysis of Big Data

Practical Methods for the Analysis of Big Data Module 4: Clustering, Decision Trees, and Ensemble Methods Philip A. Schrodt The Pennsylvania State University schrodt@psu.edu Workshop at the Odum Institute

### A Quantitative Study of Small Disjuncts in Classifier Learning

Submitted 1/7/02 A Quantitative Study of Small Disjuncts in Classifier Learning Gary M. Weiss AT&T Labs 30 Knightsbridge Road, Room 31-E53 Piscataway, NJ 08854 USA Keywords: classifier learning, small

### A study of the NIPS feature selection challenge

A study of the NIPS feature selection challenge Nicholas Johnson November 29, 2009 Abstract The 2003 Nips Feature extraction challenge was dominated by Bayesian approaches developed by the team of Radford

### 6 COMBINED MACHINE LEARNING AND FEATURE DESIGN

In the previous chapter, we presented an evaluation of the state-of-the-art machine learning algorithms for the task of classification using a real world problem and dataset. We calculated our results

### Compacting Instances: Creating models

Decision Trees Compacting Instances: Creating models Food Chat Speedy Price Bar BigTip (3) (2) (2) (2) (2) 1 great yes yes adequate no yes 2 great no yes adequate no yes 3 mediocre yes no high no no 4

### Online Ensemble Learning: An Empirical Study

Online Ensemble Learning: An Empirical Study Alan Fern AFERN@ECN.PURDUE.EDU Robert Givan GIVAN@ECN.PURDUE.EDU Department of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 4797

### Outline. Learning from Observations. Learning agents. Learning. Inductive learning (a.k.a. Science) Environment. Agent.

Outline Learning agents Learning from Observations Inductive learning Decision tree learning Measuring learning performance Chapter 18, Sections 1 3 Chapter 18, Sections 1 3 1 Chapter 18, Sections 1 3

### Conditional Independence Trees

Conditional Independence Trees Harry Zhang and Jiang Su Faculty of Computer Science, University of New Brunswick P.O. Box 4400, Fredericton, NB, Canada E3B 5A3 hzhang@unb.ca, WWW home page: http://www.cs.unb.ca/profs/hzhang/

### Ensemble Approaches for Regression: a Survey

Ensemble Approaches for Regression: a Survey João M. Moreira a,, Carlos Soares b,c, Alípio M. Jorge b,c and Jorge Freire de Sousa a a Faculty of Engineering, University of Porto, Rua Dr. Roberto Frias,

### TOWARDS DATA-DRIVEN AUTONOMICS IN DATA CENTERS

TOWARDS DATA-DRIVEN AUTONOMICS IN DATA CENTERS ALINA SIRBU, OZALP BABAOGLU SUMMARIZED BY ARDA GUMUSALAN MOTIVATION 2 MOTIVATION Human-interaction-dependent data centers are not sustainable for future data

### Introduction to Classification, aka Machine Learning

Introduction to Classification, aka Machine Learning Classification: Definition Given a collection of examples (training set ) Each example is represented by a set of features, sometimes called attributes

### A Comparison of Face Detection Algorithms

A Comparison of Face Detection Algorithms Ian R. Fasel 1 and Javier R. Movellan 2 1 Department of Cognitive Science, University of California, San Diego La Jolla, CA, 92093-0515 2 Institute for Neural

### Introduction to Machine Learning

Introduction to Machine Learning D. De Cao R. Basili Corso di Web Mining e Retrieval a.a. 2008-9 April 6, 2009 Outline Outline Introduction to Machine Learning Outline Outline Introduction to Machine Learning

### X-TREPAN: AN EXTENDED TREPAN FOR COMPREHENSIBILITY AND CLASSIFICATION ACCURACY IN ARTIFICIAL NEURAL NETWORKS

X-TREPAN: AN EXTENDED TREPAN FOR COMPREHENSIBILITY AND CLASSIFICATION ACCURACY IN ARTIFICIAL NEURAL NETWORKS Awudu Karim 1, Shangbo Zhou 2 College of Computer Science, Chongqing University, Chongqing,

### MASTER THESIS AUTOMATIC ESSAY SCORING: MACHINE LEARNING MEETS APPLIED LINGUISTICS. Victor Dias de Oliveira Santos July, 2011

1 MASTER THESIS AUTOMATIC ESSAY SCORING: MACHINE LEARNING MEETS APPLIED LINGUISTICS Victor Dias de Oliveira Santos July, 2011 European Masters in Language and Communication Technologies Supervisors: Prof.

### COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection.

COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection. Instructor: (jpineau@cs.mcgill.ca) Class web page: www.cs.mcgill.ca/~jpineau/comp551 Unless otherwise

### Evaluating Probability Estimates from Decision Trees

Evaluating Probability Estimates from Decision Trees Nitesh V. Chawla and David A. Cieslak {nchawla,dcieslak}@cse.nd.edu Department of Computer Science and Engineering University of Notre Dame, IN 46556

### A Practical Tour of Ensemble (Machine) Learning

A Practical Tour of Ensemble (Machine) Learning Nima Hejazi Evan Muzzall Division of Biostatistics, University of California, Berkeley D-Lab, University of California, Berkeley slides: https://googl/wwaqc

### Machine Learning and Data Mining. Ensembles of Learners. Prof. Alexander Ihler

Machine Learning and Data Mining Ensembles of Learners Prof. Alexander Ihler Ensemble methods Why learn one classifier when you can learn many? Ensemble: combine many predictors (Weighted) combina

### K Nearest Neighbor Edition to Guide Classification Tree Learning

K Nearest Neighbor Edition to Guide Classification Tree Learning J. M. Martínez-Otzeta, B. Sierra, E. Lazkano and A. Astigarraga Department of Computer Science and Artificial Intelligence University of

### CS Machine Learning

CS 478 - Machine Learning Projects Data Representation Basic testing and evaluation schemes CS 478 Data and Testing 1 Programming Issues l Program in any platform you want l Realize that you will be doing

### A Combination of Decision Trees and Instance-Based Learning Master s Scholarly Paper Peter Fontana,

A Combination of Decision s and Instance-Based Learning Master s Scholarly Paper Peter Fontana, pfontana@cs.umd.edu March 21, 2008 Abstract People are interested in developing a machine learning algorithm

### Cascade evaluation of clustering algorithms

Cascade evaluation of clustering algorithms Laurent Candillier 1,2, Isabelle Tellier 1, Fabien Torre 1, Olivier Bousquet 2 1 GRAppA - Charles de Gaulle University - Lille 3 candillier@grappa.univ-lille3.fr

### An Adaptive Sampling Ensemble Classifier for Learning from Imbalanced Data Sets

An Adaptive Sampling Ensemble Classifier for Learning from Imbalanced Data Sets Ordonez Jon Geiler, Li Hong, Guo Yue-jian Abstract In Imbalanced datasets, minority classes can be erroneously classified

### A Procedure for Classifying New Respondents into Existing Segments Using Maximum Difference Scaling

A Procedure for Classifying New Respondents into Existing Segments Using Maximum Difference Scaling Background Bryan Orme and Rich Johnson, Sawtooth Software March, 2009 (with minor clarifications September

### Predicting Student Performance by Using Data Mining Methods for Classification

BULGARIAN ACADEMY OF SCIENCES CYBERNETICS AND INFORMATION TECHNOLOGIES Volume 13, No 1 Sofia 2013 Print ISSN: 1311-9702; Online ISSN: 1314-4081 DOI: 10.2478/cait-2013-0006 Predicting Student Performance

### Unsupervised Learning

17s1: COMP9417 Machine Learning and Data Mining Unsupervised Learning May 2, 2017 Acknowledgement: Material derived from slides for the book Machine Learning, Tom M. Mitchell, McGraw-Hill, 1997 http://www-2.cs.cmu.edu/~tom/mlbook.html

### Welcome to CMPS 142 and 242: Machine Learning

Welcome to CMPS 142 and 242: Machine Learning Instructor: David Helmbold, dph@soe.ucsc.edu Office hours: Monday 1:30-2:30, Thursday 4:15-5:00 TA: Aaron Michelony, amichelo@soe.ucsc.edu Web page: www.soe.ucsc.edu/classes/cmps242/fall13/01

### Linear Regression: Predicting House Prices

Linear Regression: Predicting House Prices I am big fan of Kalid Azad writings. He has a knack of explaining hard mathematical concepts like Calculus in simple words and helps the readers to get the intuition

### Practical Feature Subset Selection for Machine Learning

Practical Feature Subset Selection for Machine Learning Mark A. Hall, Lloyd A. Smith {mhall, las}@cs.waikato.ac.nz Department of Computer Science, University of Waikato, Hamilton, New Zealand. Abstract

### Lecture 1: Machine Learning Basics

1/69 Lecture 1: Machine Learning Basics Ali Harakeh University of Waterloo WAVE Lab ali.harakeh@uwaterloo.ca May 1, 2017 2/69 Overview 1 Learning Algorithms 2 Capacity, Overfitting, and Underfitting 3

### Classifying Breast Cancer By Using Decision Tree Algorithms

Classifying Breast Cancer By Using Decision Tree Algorithms Nusaibah AL-SALIHY, Turgay IBRIKCI (Presenter) Cukurova University, TURKEY What Is A Decision Tree? Why A Decision Tree? Why Decision TreeClassification?

### GLMs the Good, the Bad, and the Ugly Midwest Actuarial Forum 23 March Christopher Cooksey, FCAS, MAAA EagleEye Analytics

Midwest Actuarial Forum 23 March 2009 Christopher Cooksey, FCAS, MAAA EagleEye Analytics Agenda 1.A Brief History of GLMs 2.The Good what GLMs do well 3.The Bad what GLMs don t do well 4.The Ugly what

### Neighbourhood Sampling in Bagging for Imbalanced Data

Neighbourhood Sampling in Bagging for Imbalanced Data Jerzy Błaszczyński, Jerzy Stefanowski Institute of Computing Sciences, Poznań University of Technology, 60 965 Poznań, Poland Abstract Various approaches

### Improving Classifier Utility by Altering the Misclassification Cost Ratio

Improving Classifier Utility by Altering the Misclassification Cost Ratio Michelle Ciraco, Michael Rogalewski and Gary Weiss Department of Computer Science Fordham University Rose Hill Campus Bronx, New

### Conformal Prediction Using Decision Trees

2013 IEEE 13th International Conference on Data Mining Conformal Prediction Using Decision Trees Ulf Johansson, Henrik Boström, Tuve Löfström School of Business and IT University of Borås, Sweden Email:

### Lecture 9: Classification and algorithmic methods

1/28 Lecture 9: Classification and algorithmic methods Måns Thulin Department of Mathematics, Uppsala University thulin@math.uu.se Multivariate Methods 17/5 2011 2/28 Outline What are algorithmic methods?

### Learning When Training Data are Costly: The Effect of Class Distribution on Tree Induction

Journal of Artificial Intelligence Research 19 (2003) 315-354 Submitted 12//02; published 10/03 Learning When Training Data are Costly: The Effect of Class Distribution on Tree Induction Gary M. Weiss

### Towards Moment of Learning Accuracy

Towards Moment of Learning Accuracy Zachary A. Pardos and Michael V. Yudelson Massachusetts Institute of Technology 77 Massachusetts Ave., Cambridge, MA 02139 Carnegie Learning, Inc. 437 Grant St., Pittsburgh,

### Admission Prediction System Using Machine Learning

Admission Prediction System Using Machine Learning Jay Bibodi, Aasihwary Vadodaria, Anand Rawat, Jaidipkumar Patel bibodi@csus.edu, aaishwaryvadoda@csus.edu, anandrawat@csus.edu, jaidipkumarpate@csus.edu

### Machine Learning and Auto-Evaluation

Machine Learning and Auto-Evaluation In very simple terms, Machine Learning is about training or teaching computers to take decisions or actions without explicitly programming them. For example, whenever

### Active Learning with Direct Query Construction

Active Learning with Direct Query Construction Charles X. Ling Department of Computer Science The University of Western Ontario London, Ontario N6A 5B7, Canada cling@csd.uwo.ca Jun Du Department of Computer

### COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection.

COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection. Instructor: Herke van Hoof (herke.vanhoof@mail.mcgill.ca) Slides mostly by: Class web page: www.cs.mcgill.ca/~hvanho2/comp551

### Data Mining: A Prediction for Academic Performance Improvement of Science Students using Classification

Data Mining: A Prediction for Academic Performance Improvement of Science Students using Classification I.A Ganiyu Department of Computer Science, Ramon Adedoyin College of Science and Technology, Oduduwa

### SELECTIVE VOTING GETTING MORE FOR LESS IN SENSOR FUSION

International Journal of Pattern Recognition and Artificial Intelligence Vol. 20, No. 3 (2006) 329 350 c World Scientific Publishing Company SELECTIVE VOTING GETTING MORE FOR LESS IN SENSOR FUSION LIOR

### The CN2 Induction Algorithm

Machine Learning 3: 261-283, 1989 1989 Kluwer Academic Publishers - Manufactured in The Netherlands The CN2 Induction Algorithm PETER CLARK (PETE@TURING.AC.UK) TIM NIBLETT (TIM@TURING.AC.UK) The Turing

### Performance Analysis of Various Data Mining Techniques on Banknote Authentication

International Journal of Engineering Science Invention ISSN (Online): 2319 6734, ISSN (Print): 2319 6726 Volume 5 Issue 2 February 2016 PP.62-71 Performance Analysis of Various Data Mining Techniques on

### CSE 258 Lecture 3. Web Mining and Recommender Systems. Supervised learning Classification

CSE 258 Lecture 3 Web Mining and Recommender Systems Supervised learning Classification Last week Last week we started looking at supervised learning problems Last week We studied linear regression, in

### Combating the Class Imbalance Problem in Small Sample Data Sets

Combating the Class Imbalance Problem in Small Sample Data Sets Michael Wasikowski Submitted to the Department of Electrical Engineering & Computer Science and the Graduate Faculty of the University of

### CS 540: Introduction to Artificial Intelligence

CS 540: Introduction to Artificial Intelligence Midterm Exam: 4:00-5:15 pm, October 25, 2016 B130 Van Vleck CLOSED BOOK (one sheet of notes and a calculator allowed) Write your answers on these pages and

### Two hierarchical text categorization approaches for BioASQ semantic indexing challenge. BioASQ challenge 2013 Valencia, September 2013

Two hierarchical text categorization approaches for BioASQ semantic indexing challenge Francisco J. Ribadas Víctor M. Darriba Compilers and Languages Group Universidade de Vigo (Spain) http://www.grupocole.org/

### Number of classifiers in error

Ensemble Methods in Machine Learning Thomas G. Dietterich Oregon State University, Corvallis, Oregon, USA, tgd@cs.orst.edu, WWW home page: http://www.cs.orst.edu/~tgd Abstract. Ensemble methods are learning

### Learning Concept Classification Rules Using Genetic Algorithms

Learning Concept Classification Rules Using Genetic Algorithms Kenneth A. De Jong George Mason University Fairfax, VA 22030 USA kdejong@aic.gmu.edu William M. Spears Naval Research Laboratory Washington,

### KNOWLEDGE INTEGRATION AND FORGETTING

KNOWLEDGE INTEGRATION AND FORGETTING Luís Torgo LIACC - Laboratory of AI and Computer Science University of Porto Rua Campo Alegre, 823-2º 4100 Porto, Portugal Miroslav Kubat Computer Center Technical

### Modelling Student Knowledge as a Latent Variable in Intelligent Tutoring Systems: A Comparison of Multiple Approaches

Modelling Student Knowledge as a Latent Variable in Intelligent Tutoring Systems: A Comparison of Multiple Approaches Qandeel Tariq, Alex Kolchinski, Richard Davis December 6, 206 Introduction This paper

### Python Machine Learning

Python Machine Learning Unlock deeper insights into machine learning with this vital guide to cuttingedge predictive analytics Sebastian Raschka [ PUBLISHING 1 open source I community experience distilled

### Learning Imbalanced Data with Random Forests

Learning Imbalanced Data with Random Forests Chao Chen (Stat., UC Berkeley) chenchao@stat.berkeley.edu Andy Liaw (Merck Research Labs) andy_liaw@merck.com Leo Breiman (Stat., UC Berkeley) leo@stat.berkeley.edu

### A Classification Method using Decision Tree for Uncertain Data

A Classification Method using Decision Tree for Uncertain Data Annie Mary Bhavitha S 1, Sudha Madhuri 2 1 Pursuing M.Tech(CSE), Nalanda Institute of Engineering & Technology, Siddharth Nagar, Sattenapalli,

### Introduction To Ensemble Learning

Educational Series Introduction To Ensemble Learning Dr. Oliver Steinki, CFA, FRM Ziad Mohammad Volume I: Series 1 July 2015 What Is Ensemble Learning? In broad terms, ensemble learning is a procedure

### Noise-Tolerant Windowing

Noise-Tolerant Windowing Johannes Fiirnkranz Austrian Research Institute for Artificial Intelligence Schottengasse 3, A-1010 Wien, Austria E-mail: juf f i@ai. univie. ac. at Abstract Windowing has been