# Machine Learning B, Fall 2016

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Machine Learning B, Fall 2016 Decision Trees (Summary) Lecture 2, 08/31/ 2016 Maria-Florina (Nina) Balcan

2 Learning Decision Trees. Supervised Classification. Useful Readings: Mitchell, Chapter 3 Bishop, Chapter 14.4 DT learning: Method for learning discrete-valued target functions in which the function to be learned is represented by a decision tree.

3 Supervised Classification: Decision Tree Learning Example: learn concept PlayTennis (i.e., decide whether our friend will play tennis or not in a given day) Simple Training Data Set Day Outlook Temperature Humidity Wind Play Tennis example label

4 Supervised Classification: Decision Tree Learning Each internal node: test one (discrete-valued) attribute X i Each branch from a node: corresponds to one possible values for X i Each leaf node: predict Y Example: A Decision tree for f: <Outlook, Temperature, Humidity, Wind> PlayTennis? Day Outlook Temperature Humidity Wind Play Tennis E.g., x=(outlook=sunny, Temperature-Hot, Humidity=Normal,Wind=High), f(x)=yes.

5 Supervised Classification: Problem Setting Input: Training labeled examples {(x (i),y (i) )} of unknown target function f Examples described by their values on some set of features or attributes Day Outlook Temperature Humidity Wind Play Tennis E.g. 4 attributes: Humidity, Wind, Outlook, Temp e.g., <Humidity=High, Wind=weak, Outlook=rain, Temp=Mild> Set of possible instances X (a.k.a instance space) Unknown target function f : X Y e.g., Y={0,1} label space e.g., 1 if we play tennis on this day, else 0 Output: Hypothesis h H that (best) approximates target function f Set of function hypotheses H={ h h : X Y } each hypothesis h is a decision tree

6 Core Aspects in Decision Tree & Supervised Learning How to automatically find a good hypothesis for training data? This is an algorithmic question, the main topic of computer science When do we generalize and do well on unseen data? Learning theory quantifies ability to generalize as a function of the amount of training data and the hypothesis space Occam s razor: use the simplest hypothesis consistent with data! Fewer short hypotheses than long ones a short hypothesis that fits the data is less likely to be a statistical coincidence highly probable that a sufficiently complex hypothesis will fit the data

7 Core Aspects in Decision Tree & Supervised Learning How to automatically find a good hypothesis for training data? This is an algorithmic question, the main topic of computer science When do we generalize and do well on unseen data? Occam s razor: use the simplest hypothesis consistent with data! Decision trees: if we were able to find a small decision tree that explains data well, then good generalization guarantees. NP-hard [Hyafil-Rivest 76]: unlikely to have a poly time algorithm Very nice practical heuristics; top down algorithms, e.g, ID3

8 Top-Down Induction of Decision Trees [ID3, C4.5, Quinlan] ID3: Natural greedy approach to growing a decision tree top-down (from the root to the leaves by repeatedly replacing an existing leaf with an internal node.). Algorithm: Pick best attribute to split at the root based on training data. Recurse on children that are impure (e.g, have both Yes and No). Humidity Outlook Temp Wind Day Outlook Temperature Humidity Wind Play Tennis Day Outlook Temperature Humidity Wind Play Tennis D1 Sunny Hot High Weak No D2 Sunny Hot High Strong No D8 Sunny Mild High Weak No D9 Sunny Cool Normal Weak Yes D11 Sunny Mild Normal Strong Yes Weak High Sunny Cool Overcast Mild Normal Strong Rain Hot Day Outlook Temperature Humidity Wind Play Tennis D4 Rain Mild High Weak Yes D5 Rain Cool Normal Weak Yes D6 Rain Cool Normal Strong No D10 Rain Mild Normal Weak Yes D14 Rain Mild High Strong No Humidity Yes Wind High Normal Strong Weak No Yes No Yes

9 Top-Down Induction of Decision Trees [ID3, C4.5, Quinlan] ID3: Natural greedy approach to growing a decision tree top-down. Algorithm: Day Outlook Temperature Humidity Wind Play Tennis Pick best attribute to split at the root based on training data. Recurse on children that are impure (e.., have both Yes and No). Key question: Which attribute is best? ID3 uses a statistical measure called information gain (how well a given attribute separates the training examples according to the target classification) Information Gain of A is the expected reduction in entropy of target variable Y for data sample S, due to sorting on variable A

10 Properties of ID3 ID3 performs heuristic search through space of decision trees It tends to have the right bias (output short decision trees), but it can still overfit. It might be beneficial to prune the tree by using a validation dataset.

11 Consider a hypothesis h and its Properties of ID3 Overfitting could occur because of noisy data and because ID3 is not guaranteed to output a small hypothesis even if one exists. Error rate over training data: error train (h) True error rate over all data: error true (h) We say h overfits the training data if error true h > error train (h) Amount of overfitting = error true h error train (h)

12 Task: learning which medical patients have a form of diabetes.

13 Key Issues in Machine Learning How can we gauge the accuracy of a hypothesis on unseen data? Occam s razor: use the simplest hypothesis consistent with data! This will help us avoid overfitting. Learning theory will help us quantify our ability to generalize as a function of the amount of training data and the hypothesis space How do we find the best hypothesis? This is an algorithmic question, the main topic of computer science How do we choose a hypothesis space? Often we use prior knowledge to guide this choice How to model applications as machine learning problems? (engineering challenge)

### Machine Learning Tom M. Mitchell Machine Learning Department Carnegie Mellon University. January 11, 2011

Machine Learning 10-701 Tom M. Mitchell Machine Learning Department Carnegie Mellon University January 11, 2011 Today: What is machine learning? Decision tree learning Course logistics Readings: The Discipline

### Machine Learning Tom M. Mitchell Machine Learning Department Carnegie Mellon University. January 12, 2015

Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University January 12, 2015 Today: What is machine learning? Decision tree learning Course logistics Readings: The Discipline

### Machine Learning. June 22, 2006 CS 486/686 University of Waterloo

Machine Learning June 22, 2006 CS 486/686 University of Waterloo Outline Inductive learning Decision trees Reading: R&N Ch 18.1-18.3 CS486/686 Lecture Slides (c) 2006 K.Larson and P. Poupart 2 What is

### Decision Tree for Playing Tennis

Decision Tree Decision Tree for Playing Tennis (outlook=sunny, wind=strong, humidity=normal,? ) DT for prediction C-section risks Characteristics of Decision Trees Decision trees have many appealing properties

### Inductive Learning and Decision Trees

Inductive Learning and Decision Trees Doug Downey EECS 349 Winter 2014 with slides from Pedro Domingos, Bryan Pardo Outline Announcements Homework #1 assigned Have you completed it? Inductive learning

### Course 395: Machine Learning Lectures

Course 395: Machine Learning Lectures Lecture 1-2: Concept Learning (M. Pantic) Lecture 3-4: Decision Trees & CBC Intro (M. Pantic) Lecture 5-6: Artificial Neural Networks (S. Zafeiriou) Lecture 7-8: Instance

### Machine Learning. Announcements (7/15) Announcements (7/16) Comments on the Midterm. Agents that Learn. Agents that Don t Learn

Machine Learning Burr H. Settles CS540, UWMadison www.cs.wisc.edu/~cs5401 Summer 2003 Announcements (7/15) If you haven t already, read Sections 18.118.3 in AI: A Modern Approach Homework #3 due tomorrow

### Inductive Learning and Decision Trees

Inductive Learning and Decision Trees Doug Downey EECS 349 Spring 2017 with slides from Pedro Domingos, Bryan Pardo Outline Announcements Homework #1 was assigned on Monday (due in five days!) Inductive

### Course 395: Machine Learning Lectures

Course 395: Machine Learning Lectures Lecture 1-2: Concept Learning (M. Pantic) Lecture 3-4: Decision Trees & CBC Intro (M. Pantic) Lecture 5-6: Artificial Neural Networks (THs) Lecture 7-8: Instance Based

### Rule Learning (1): Classification Rules

14s1: COMP9417 Machine Learning and Data Mining Rule Learning (1): Classification Rules March 19, 2014 Acknowledgement: Material derived from slides for the book Machine Learning, Tom M. Mitchell, McGraw-Hill,

### CLASSIFICATION: DECISION TREES

CLASSIFICATION: DECISION TREES Gökhan Akçapınar (gokhana@hacettepe.edu.tr) Seminar in Methodology and Statistics John Nerbonne, Çağrı Çöltekin University of Groningen May, 2012 Outline Research question

### Deriving Decision Trees from Case Data

Topic 4 Automatic Kwledge Acquisition PART II Contents 5.1 The Bottleneck of Kwledge Aquisition 5.2 Inductive Learning: Decision Trees 5.3 Converting Decision Trees into Rules 5.4 Generating Decision Trees:

### CSC 4510/9010: Applied Machine Learning. Rule Inference. Dr. Paula Matuszek

CSC 4510/9010: Applied Machine Learning 1 Rule Inference Dr. Paula Matuszek Paula.Matuszek@villanova.edu Paula.Matuszek@gmail.com (610) 647-9789 Classification rules Popular alternative to decision trees

### Decision Tree For Playing Tennis

Decision Tree For Playing Tennis ROOT NODE BRANCH INTERNAL NODE LEAF NODE Disjunction of conjunctions Another Perspective of a Decision Tree Model Age 60 40 20 NoDefault NoDefault + + NoDefault Default

### 18 LEARNING FROM EXAMPLES

18 LEARNING FROM EXAMPLES An intelligent agent may have to learn, for instance, the following components: A direct mapping from conditions on the current state to actions A means to infer relevant properties

### CSC 4510/9010: Applied Machine Learning Rule Inference

CSC 4510/9010: Applied Machine Learning Rule Inference Dr. Paula Matuszek Paula.Matuszek@villanova.edu Paula.Matuszek@gmail.com (610) 647-9789 CSC 4510.9010 Spring 2015. Paula Matuszek 1 Red Tape Going

### Supervised learning can be done by choosing the hypothesis that is most probable given the data: = arg max ) = arg max

The learning problem is called realizable if the hypothesis space contains the true function; otherwise it is unrealizable On the other hand, in the name of better generalization ability it may be sensible

### Introduction to Machine Learning

Introduction to Machine Learning D. De Cao R. Basili Corso di Web Mining e Retrieval a.a. 2008-9 April 6, 2009 Outline Outline Introduction to Machine Learning Outline Outline Introduction to Machine Learning

### COMPARATIVE STUDY ID3, CART AND C4.5 DECISION TREE ALGORITHM: A SURVEY

COMPARATIVE STUDY ID3, CART AND C4.5 DECISION TREE ALGORITHM: A SURVEY Sonia Singh Assistant Professor Department of computer science University of Delhi New Delhi, India 14sonia.singh@gmail.com Priyanka

### ECT7110 Classification Decision Trees. Prof. Wai Lam

ECT7110 Classification Decision Trees Prof. Wai Lam Classification and Decision Tree What is classification? What is prediction? Issues regarding classification and prediction Classification by decision

### Applied Machine Learning Lecture 1: Introduction

Applied Machine Learning Lecture 1: Introduction Richard Johansson January 16, 2018 welcome to the course! machine learning is getting increasingly popular among students our courses are full! many thesis

### PRESENTATION TITLE. A Two-Step Data Mining Approach for Graduation Outcomes CAIR Conference

PRESENTATION TITLE A Two-Step Data Mining Approach for Graduation Outcomes 2013 CAIR Conference Afshin Karimi (akarimi@fullerton.edu) Ed Sullivan (esullivan@fullerton.edu) James Hershey (jrhershey@fullerton.edu)

### IAI : Machine Learning

IAI : Machine Learning John A. Bullinaria, 2005 1. What is Machine Learning? 2. The Need for Learning 3. Learning in Neural and Evolutionary Systems 4. Problems Facing Expert Systems 5. Learning in Rule

### CS 354R: Computer Game Technology

CS 354R: Computer Game Technology AI Decision Trees and Rule Systems Fall 2017 Decision Trees Nodes represent attribute tests One child for each outcome Leaves represent classifications Can have same classification

### Machine Learning and Auto-Evaluation

Machine Learning and Auto-Evaluation In very simple terms, Machine Learning is about training or teaching computers to take decisions or actions without explicitly programming them. For example, whenever

### Towards semantics-enabled infrastructure for knowledge acquisition from distributed data

Towards semantics-enabled infrastructure for knowledge acquisition from distributed data Vasant Honavar and Doina Caragea Artificial Intelligence Research Laboratory Bioinformatics and Computational Biology

### P(A, B) = P(A B) = P(A) + P(B) - P(A B)

AND Probability P(A, B) = P(A B) = P(A) + P(B) - P(A B) P(A B) = P(A) + P(B) - P(A B) Area = Probability of Event AND Probability P(A, B) = P(A B) = P(A) + P(B) - P(A B) If, and only if, A and B are independent,

### X-TREPAN: AN EXTENDED TREPAN FOR COMPREHENSIBILITY AND CLASSIFICATION ACCURACY IN ARTIFICIAL NEURAL NETWORKS

X-TREPAN: AN EXTENDED TREPAN FOR COMPREHENSIBILITY AND CLASSIFICATION ACCURACY IN ARTIFICIAL NEURAL NETWORKS Awudu Karim 1, Shangbo Zhou 2 College of Computer Science, Chongqing University, Chongqing,

### Outline. Learning from Observations. Learning agents. Learning. Inductive learning (a.k.a. Science) Environment. Agent.

Outline Learning agents Learning from Observations Inductive learning Decision tree learning Measuring learning performance Chapter 18, Sections 1 3 Chapter 18, Sections 1 3 1 Chapter 18, Sections 1 3

### Section 18.3 Learning Decision Trees

Section 18.3 Learning Decision Trees CS4811 - Artificial Intelligence Nilufer Onder Department of Computer Science Michigan Technological University Outline Attribute-based representations Decision tree

### Machine Learning :: Introduction. Konstantin Tretyakov

Machine Learning :: Introduction Konstantin Tretyakov (kt@ut.ee) MTAT.03.183 Data Mining November 5, 2009 So far Data mining as knowledge discovery Frequent itemsets Descriptive analysis Clustering Seriation

### Combining multiple models

Combining multiple models Basic idea of meta learning schemes: build different experts and let them vote Advantage: often improves predictive performance Disadvantage: produces output that is very hard

### Assignment 6 (Sol.) Introduction to Machine Learning Prof. B. Ravindran

Assignment 6 (Sol.) Introduction to Machine Learning Prof. B. Ravindran 1. Assume that you are given a data set and a neural network model trained on the data set. You are asked to build a decision tree

### Introduction to Ensemble Learning Featuring Successes in the Netflix Prize Competition

Introduction to Ensemble Learning Featuring Successes in the Netflix Prize Competition Todd Holloway Two Lecture Series for B551 November 20 & 27, 2007 Indiana University Outline Introduction Bias and

### CS534 Machine Learning

CS534 Machine Learning Spring 2013 Lecture 1: Introduction to ML Course logistics Reading: The discipline of Machine learning by Tom Mitchell Course Information Instructor: Dr. Xiaoli Fern Kec 3073, xfern@eecs.oregonstate.edu

### COMP 551 Applied Machine Learning Lecture 11: Ensemble learning

COMP 551 Applied Machine Learning Lecture 11: Ensemble learning Instructor: Herke van Hoof (herke.vanhoof@mcgill.ca) Slides mostly by: (jpineau@cs.mcgill.ca) Class web page: www.cs.mcgill.ca/~hvanho2/comp551

### Data Mining Practical Machine Learning Tools and Techniques. Slides for Chapter 1 of Data Mining by I. H. Witten and E. Frank

Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 1 of Data Mining by I. H. Witten and E. Frank What s it all about Data vs information Data mining and machine learning Structural

### Compacting Instances: Creating models

Decision Trees Compacting Instances: Creating models Food Chat Speedy Price Bar BigTip (3) (2) (2) (2) (2) 1 great yes yes adequate no yes 2 great no yes adequate no yes 3 mediocre yes no high no no 4

### COMP 551 Applied Machine Learning Lecture 12: Ensemble learning

COMP 551 Applied Machine Learning Lecture 12: Ensemble learning Associate Instructor: Herke van Hoof (herke.vanhoof@mcgill.ca) Slides mostly by: (jpineau@cs.mcgill.ca) Class web page: www.cs.mcgill.ca/~jpineau/comp551

### Inducing a Decision Tree

Inducing a Decision Tree In order to learn a decision tree, our agent will need to have some information to learn from: a training set of examples each example is described by its values for the problem

### CS 540: Introduction to Artificial Intelligence

CS 540: Introduction to Artificial Intelligence Midterm Exam: 4:00-5:15 pm, October 25, 2016 B130 Van Vleck CLOSED BOOK (one sheet of notes and a calculator allowed) Write your answers on these pages and

### A Survey on Hoeffding Tree Stream Data Classification Algorithms

CPUH-Research Journal: 2015, 1(2), 28-32 ISSN (Online): 2455-6076 http://www.cpuh.in/academics/academic_journals.php A Survey on Hoeffding Tree Stream Data Classification Algorithms Arvind Kumar 1*, Parminder

### Practical Feature Subset Selection for Machine Learning

Practical Feature Subset Selection for Machine Learning Mark A. Hall, Lloyd A. Smith {mhall, las}@cs.waikato.ac.nz Department of Computer Science, University of Waikato, Hamilton, New Zealand. Abstract

### Tanagra Tutorials. Figure 1 Tree size and generalization error rate (Source:

1 Topic Describing the post pruning process during the induction of decision trees (CART algorithm, Breiman and al., 1984 C RT component into TANAGRA). Determining the appropriate size of the tree is a

### Machine Learning 2nd Edition

INTRODUCTION TO Lecture Slides for Machine Learning 2nd Edition ETHEM ALPAYDIN, modified by Leonardo Bobadilla and some parts from http://www.cs.tau.ac.il/~apartzin/machinelearning/ The MIT Press, 2010

### A Combination of Decision Trees and Instance-Based Learning Master s Scholarly Paper Peter Fontana,

A Combination of Decision s and Instance-Based Learning Master s Scholarly Paper Peter Fontana, pfontana@cs.umd.edu March 21, 2008 Abstract People are interested in developing a machine learning algorithm

### A Few Useful Things to Know about Machine Learning. Pedro Domingos Department of Computer Science and Engineering University of Washington" 2012"

A Few Useful Things to Know about Machine Learning Pedro Domingos Department of Computer Science and Engineering University of Washington 2012 A Few Useful Things to Know about Machine Learning Machine

### Conditional Independence Trees

Conditional Independence Trees Harry Zhang and Jiang Su Faculty of Computer Science, University of New Brunswick P.O. Box 4400, Fredericton, NB, Canada E3B 5A3 hzhang@unb.ca, WWW home page: http://www.cs.unb.ca/profs/hzhang/

### Machine Learning Lecture 1: Introduction

Welcome to CSCE 478/878! Please check off your name on the roster, or write your name if you're not listed Indicate if you wish to register or sit in Policy on sit-ins: You may sit in on the course without

### Selective Bayesian Classifier: Feature Selection for the Naïve Bayesian Classifier Using Decision Trees

Selective Bayesian Classifier: Feature Selection for the Naïve Bayesian Classifier Using Decision Trees Chotirat Ann Ratanamahatana, Dimitrios Gunopulos Department of Computer Science, University of California,

### CS 3030 Artificial Intelligence Review for Exam 1

Part of this document is from the lecture notes of Artificial Intelligence Illuminated. Use this review together with your lecture notes, textbook and quizzes to prepare for the exam. 1. Introduction What

### Linear Regression. Chapter Introduction

Chapter 9 Linear Regression 9.1 Introduction In this class, we have looked at a variety of di erent models and learning methods, such as finite state machines, sequence models, and classification methods.

### Constrained Dynamic Rule Induction Learning

Constrained Dynamic Rule Induction Learning Fadi Thabtah a, Issa Qabajeh b, Francisco Chiclana c a. Applied Business and Computing, NMIT, Auckland, New Zealand b. School of Computer Sciences and Informatics,

### CSE 546 Machine Learning

CSE 546 Machine Learning Instructor: Luke Zettlemoyer TA: Lydia Chilton Slides adapted from Pedro Domingos and Carlos Guestrin Logistics Instructor: Luke Zettlemoyer Email: lsz@cs Office: CSE 658 Office

### What is Machine Learning?

What is Machine Learning? INFO-4604, Applied Machine Learning University of Colorado Boulder August 29-31, 2017 Prof. Michael Paul Definition Murphy: a set of methods that can automatically detect patterns

### Data Mining CAP

Data Mining CAP 5771-001 Administrative Details The text is a high-level overview of data mining. You can supplement this by papers from the bibliography available on the Web. They will provide some details.

### Analysis of Different Classifiers for Medical Dataset using Various Measures

Analysis of Different for Medical Dataset using Various Measures Payal Dhakate ME Student, Pune, India. K. Rajeswari Associate Professor Pune,India Deepa Abin Assistant Professor, Pune, India ABSTRACT

### Principles of Machine Learning

Principles of Machine Learning Lab 5 - Optimization-Based Machine Learning Models Overview In this lab you will explore the use of optimization-based machine learning models. Optimization-based models

### Decision Tree Instability and Active Learning

Decision Tree Instability and Active Learning Kenneth Dwyer and Robert Holte University of Alberta November 14, 2007 Kenneth Dwyer, University of Alberta Decision Tree Instability and Active Learning 1

### 10701/15781 Machine Learning, Spring 2005: Homework 1

10701/15781 Machine Learning, Spring 2005: Homework 1 Due: Monday, February 6, beginning of the class 1 [15 Points] Probability and Regression [Stano] 1 1.1 [10 Points] The Matrix Strikes Back The Matrix

### CSC-272 Exam #2 March 20, 2015

CSC-272 Exam #2 March 20, 2015 Name Questions are weighted as indicated. Show your work and state your assumptions for partial credit consideration. Unless explicitly stated, there are NO intended errors

### An Inductive Learning Algorithm for Production Rule Discovery

An Inductive Learning Algorithm for Production Rule Discovery Mehmet R. Tolun Saleh M. Abu-Soud Department of Computer Engineering Department of Computer Science Middle East Technical University Princess

### Principle Component Analysis for Feature Reduction and Data Preprocessing in Data Science

Principle Component Analysis for Feature Reduction and Data Preprocessing in Data Science Hayden Wimmer Department of Information Technology Georgia Southern University hwimmer@georgiasouthern.edu Loreen

### A Classification Method using Decision Tree for Uncertain Data

A Classification Method using Decision Tree for Uncertain Data Annie Mary Bhavitha S 1, Sudha Madhuri 2 1 Pursuing M.Tech(CSE), Nalanda Institute of Engineering & Technology, Siddharth Nagar, Sattenapalli,

### Machine Learning (Decision Trees and Intro to Neural Nets) CSCI 3202, Fall 2010

Machine Learning (Decision Trees and Intro to Neural Nets) CSCI 3202, Fall 2010 Assignments To read this week: Chapter 18, sections 1-4 and 7 Problem Set 3 due next week! Learning a Decision Tree We look

### MASTER THESIS AUTOMATIC ESSAY SCORING: MACHINE LEARNING MEETS APPLIED LINGUISTICS. Victor Dias de Oliveira Santos July, 2011

1 MASTER THESIS AUTOMATIC ESSAY SCORING: MACHINE LEARNING MEETS APPLIED LINGUISTICS Victor Dias de Oliveira Santos July, 2011 European Masters in Language and Communication Technologies Supervisors: Prof.

### ANALYZING BIG DATA WITH DECISION TREES

San Jose State University SJSU ScholarWorks Master's Projects Master's Theses and Graduate Research Spring 2014 ANALYZING BIG DATA WITH DECISION TREES Lok Kei Leong Follow this and additional works at:

### Introduction to Machine Learning

1, 582631 5 credits Introduction to Machine Learning Lecturer: Teemu Roos Assistant: Ville Hyvönen Department of Computer Science University of Helsinki (based in part on material by Patrik Hoyer and Jyrki

### A Quantitative Study of Small Disjuncts in Classifier Learning

Submitted 1/7/02 A Quantitative Study of Small Disjuncts in Classifier Learning Gary M. Weiss AT&T Labs 30 Knightsbridge Road, Room 31-E53 Piscataway, NJ 08854 USA Keywords: classifier learning, small

### Unified View of Decision Tree Learning Machines for the Purpose of Meta-learning

Unified View of Decision Tree Learning Machines for the Purpose of Meta-learning Krzysztof Grąbczewski Abstract. The experience gained from thorough analysis of many decision tree (DT) induction algorithms,

### Decision Tree C4.5 algorithm and its enhanced approach for Educational Data Mining

Decision Tree C4.5 algorithm and its enhanced approach for Educational Data Mining Preeti Patidar 1, Jitendra Dangra 2, M.K. Rawar 3 Computer Science dept. LNCT Indore, University RGPV Bhopal, India 1

### A Review on Classification Techniques in Machine Learning

A Review on Classification Techniques in Machine Learning R. Vijaya Kumar Reddy 1, Dr. U. Ravi Babu 2 1 Research Scholar, Dept. of. CSE, Acharya Nagarjuna University, Guntur, (India) 2 Principal, DRK College

### Handbook of Perception and Cognition, Vol.14 Chapter 4: Machine Learning

Handbook of Perception and Cognition, Vol.14 Chapter 4: Machine Learning Stuart Russell Computer Science Division University of California Berkeley, CA 94720, USA (510) 642 4964, fax: (510) 642 5775 Contents

### Lecture 1: Basic Concepts of Machine Learning

Lecture 1: Basic Concepts of Machine Learning Cognitive Systems - Machine Learning Ute Schmid (lecture) Johannes Rabold (practice) Based on slides prepared March 2005 by Maximilian Röglinger, updated 2010

### Data Mining. CS57300 Purdue University. Bruno Ribeiro. February 15th, 2018

Data Mining CS573 Purdue University Bruno Ribeiro February 15th, 218 1 Today s Goal Ensemble Methods Supervised Methods Meta-learners Unsupervised Methods 215 Bruno Ribeiro Understanding Ensembles The

### 1 Subject. 2 Dataset. 3 Descriptive statistics. 3.1 Data importation. SIPINA proposes some descriptive statistics functionalities.

1 Subject proposes some descriptive statistics functionalities. In itself, the information is not really exceptional; there is a large number of freeware which do that. It becomes more interesting when

### Ensembles. CS Ensembles 1

Ensembles CS 478 - Ensembles 1 A Holy Grail of Machine Learning Outputs Just a Data Set or just an explanation of the problem Automated Learner Hypothesis Input Features CS 478 - Ensembles 2 Ensembles

### Lecture 1: Machine Learning Basics

1/69 Lecture 1: Machine Learning Basics Ali Harakeh University of Waterloo WAVE Lab ali.harakeh@uwaterloo.ca May 1, 2017 2/69 Overview 1 Learning Algorithms 2 Capacity, Overfitting, and Underfitting 3

### Admission Prediction System Using Machine Learning

Admission Prediction System Using Machine Learning Jay Bibodi, Aasihwary Vadodaria, Anand Rawat, Jaidipkumar Patel bibodi@csus.edu, aaishwaryvadoda@csus.edu, anandrawat@csus.edu, jaidipkumarpate@csus.edu

### A Comparison of Data Mining Tools using the implementation of C4.5 Algorithm

A Comparison of Data Mining Tools using the implementation of C4.5 Algorithm Divya Jain School of Computer Science and Engineering, ITM University, Gurgaon, India Abstract: This paper presents the implementation

### Attribute Discretization for Classification

Association for Information Systems AIS Electronic Library (AISeL) AMCIS 2001 Proceedings Americas Conference on Information Systems (AMCIS) December 2001 Attribute Discretization for Classification Noel

### CS Machine Learning

CS 478 - Machine Learning Projects Data Representation Basic testing and evaluation schemes CS 478 Data and Testing 1 Programming Issues l Program in any platform you want l Realize that you will be doing

### UTILIZING BAYESIAN NETWORKS TO DEVELOP REAL TIME PROGNOSTIC MODELS FOR GROUND VEHICLES

: Distribution Statement A. Approved for public release; distribution is unlimited 2017 NDIA GROUND VEHICLE SYSTEMS ENGINEERING AND TECHNOLOGY SYMPOSIUM SYSTEMS ENGINEERING (SE) TECHNICAL SESSION AUGUST

### Phrase-Based MT: Decoding. February 19, 2015

Phrase-Based MT: Decoding February 19, 2015 Administrative Final proposal draft due Tuesday It needs to be revised Bring 3 printed copies again HW 2 is due two weeks from today Phrase Based MT e = arg

### Predicting Academic Success from Student Enrolment Data using Decision Tree Technique

Predicting Academic Success from Student Enrolment Data using Decision Tree Technique M Narayana Swamy Department of Computer Applications, Presidency College Bangalore,India M. Hanumanthappa Department

### Ensemble Learning CS534

Ensemble Learning CS534 Ensemble Learning How to generate ensembles? There have been a wide range of methods developed We will study to popular approaches Bagging Boosting Both methods take a single (base)

### Classifying Breast Cancer By Using Decision Tree Algorithms

Classifying Breast Cancer By Using Decision Tree Algorithms Nusaibah AL-SALIHY, Turgay IBRIKCI (Presenter) Cukurova University, TURKEY What Is A Decision Tree? Why A Decision Tree? Why Decision TreeClassification?

### Learning Concept Classification Rules Using Genetic Algorithms

Learning Concept Classification Rules Using Genetic Algorithms Kenneth A. De Jong George Mason University Fairfax, VA 22030 USA kdejong@aic.gmu.edu William M. Spears Naval Research Laboratory Washington,

### Evaluation and Comparison of Performance of different Classifiers

Evaluation and Comparison of Performance of different Classifiers Bhavana Kumari 1, Vishal Shrivastava 2 ACE&IT, Jaipur Abstract:- Many companies like insurance, credit card, bank, retail industry require

### Prediction of Bike Sharing Systems for Casual and Registered Users Mahmood Alhusseini CS229: Machine Learning.

Prediction of Bike Sharing Systems for Casual and Registered Users Mahmood Alhusseini mih@stanford.edu CS229: Machine Learning Abstract - In this project, two different approaches to predict Bike Sharing

### Uninformed Search (Ch )

1 Uninformed Search (Ch. 3-3.4) 2 Announcements Will make homework this weekend (~4 days) due next weekend (~13 days) 3 What did we do last time? Take away messages: Lecture 1: Class schedule (ended early)

### Cost-Sensitive Learning and the Class Imbalance Problem

To appear in Encyclopedia of Machine Learning. C. Sammut (Ed.). Springer. 2008 Cost-Sensitive Learning and the Class Imbalance Problem Charles X. Ling, Victor S. Sheng The University of Western Ontario,

### Machine Learning. Basic Concepts. Joakim Nivre. Machine Learning 1(24)

Machine Learning Basic Concepts Joakim Nivre Uppsala University and Växjö University, Sweden E-mail: nivre@msi.vxu.se Machine Learning 1(24) Machine Learning Idea: Synthesize computer programs by learning

### The Health Economics and Outcomes Research Applications and Valuation of Digital Health Technologies and Machine Learning

The Health Economics and Outcomes Research Applications and Valuation of Digital Health Technologies and Machine Learning Workshop W29 - Session V 3:00 4:00pm May 25, 2016 ISPOR 21 st Annual International

### An Empherical Study on Decision Tree Classification Algorithms

An Empherical Study on Decision Tree Classification Algorithms Lakshmi.B.N 1 Dr. Indumathi.T.S 2 Dr. Nandini Ravi 3 Abstract The increasing data with technological advancement has put-forth a challenging

### Data Mining in Oral Medicine Using Decision Trees

Data Mining in Oral Medicine Using Decision Trees Fahad Shahbaz Khan, Rao Muhammad Anwer, Olof Torgersson, and Göran Falkman Abstract Data mining has been used very frequently to extract hidden information

### Lecture 9: Classification and algorithmic methods

1/28 Lecture 9: Classification and algorithmic methods Måns Thulin Department of Mathematics, Uppsala University thulin@math.uu.se Multivariate Methods 17/5 2011 2/28 Outline What are algorithmic methods?

### Active Learning. Yingyu Liang Computer Sciences 760 Fall

Active Learning Yingyu Liang Computer Sciences 760 Fall 2017 http://pages.cs.wisc.edu/~yliang/cs760/ Some of the slides in these lectures have been adapted/borrowed from materials developed by Mark Craven,