# Ensemble Learning CS534

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Ensemble Learning CS534

2 Ensemble Learning

3 How to generate ensembles? There have been a wide range of methods developed We will study to popular approaches Bagging Boosting Both methods take a single (base) learning algorithm and generate ensembles

4 Base Learning Algorithm We are given a black box learning algorithm Learn referred to as the base learner.

5 Bootstrap Aggregating (Bagging) Leo Breiman, Bagging Predictors, Machine Learning, 24, (1996) Consider creating many training data sets by drawing instances from some distribution and then using Learn to output a hypothesis for each dataset. The resulting hypotheses will likely vary in performance due to variation in the training sets What happens if we combine these hypothesesusing a majority vote?

6 Bagging Algorithm Given training set S, bagging works as follows: 1. Create T bootstrap samples { of S as follows: For each : Randomly drawing S examples from S with replacement 2. For each, 3. Output With large S, each will contain % unique examples

7 Stability of Learn A learning algorithm is unstable if small changes in the training data can produce large changes in the output hypothesis (otherwise stable). Clearly bagging will have little benefit when used with stable base learning algorithms (i.e., most ensemble members will be very similar). Bagging generally works best when used with unstable yet relatively accurate base learners

8 The Bias Variance Decomposition Bagging reduces variance of a classifier. Most appropriate for classifiers of low bias and high variance (e.g., decision tree).

9 Target concept Single decision tree 100 bagged decision tree

10

11 Boosting Key difference compared to bagging? Its iterative. Bagging : Individual classifiers were independent. Boosting: Look at errors from previous classifiers to decide what to focus on for the next iteration over data Successive classifiers depends upon its predecessors. Result: more weights on hard examples. (the ones on which we committed mistakes in the previous iterations)

12 Some Boosting History The idea of boosting began with a learning theory question first asked in the late 80 s. The question was answered in 1989 by Robert Shapire resulting in the first theoretical boosting algorithm Shapire and Freund later developed a practical boosting algorithm called Adaboost Many empirical studies show that Adaboost is highly effective (very often they outperform ensembles produced by bagging)

13 History: Strong vs weak learning Strong = weak?

14 Strong = Weak PAC learning The key idea is that we can learn a little on every distribution Produce 3 hypothesis as follows is the result of applying Learn to all training data. is the result of applying Learn to filtered data distribution such that has only 50% accuracy on the data. (e.g., to generate an example flip a coin, if head then draw examples until makes an error, and give it to Learn; if tail then wait until is correct, and give it to Learn) is the result of applying Learn to training data on which and disagree. We can then let them vote, the resulting error rate will be improved. We can repeat this until reaching the target error rate

15 Consider E = <, majorityvote>. If,, have error rates less than, it can be shown that the error rate of E is upper bounded by :3 2 This fact leads to a recursive algorithm that creates a hypothesis of arbitrary accuracy from weak hypotheses. Assume we desire an error rate less than e. These need only achieve an error rate less than As we move down the tree, the error we needs to achieve increases according to Eventually the error rate needed will be attainable by the weak learner

16 AdaBoost The boosting algorithm derived from the original proof is impractical requires to many calls to Learn, though only polynomially many Practically efficient boosting algorithm Adaboost Makes more effective use of each call of Learn

17 Specifying Input Distributions AdaBoost works by invoking Learn many times on different distributions over the training data set. Need to modify base learner protocol to accept a training set distribution as an input. D(i) can be viewed as indicating to base learner Learn the importance of correctly classifying the i th training instance

18 AdaBoost (High level steps) AdaBoost performs L boosting rounds, the operations in each boosting round are: 1. Call Learn on data set S with distribution to produce l th ensemble member, where is the distribution of round. 2. Compute the 1 round distribution by putting more weight on instances that makes mistakes on 3. Compute a voting weight for The ensemble hypothesis returned is: H=<,,,,, >

19

20 Learning with Weights It is often straightforward to convert a base learner to take into account an input distribution D. Decision trees? Neural nets? Logistic regression? When it s not straightforward, we can resample the training data according to D

21

22

23

24

25

26

27 Schapire Letter recognition

28

29 Margin Based Error bound (schapire, Freund, Bartlett and Lee 1989) Boosting increases the margin very aggressively since it concentrates on the hardest examples. If margin is large, more weak learners agree and hence more rounds does not necessarily imply that final classifier is getting more complex. Bound is independent of number of rounds T! Boosting can still overfit if margin is too small, weak learners are too complex or perform arbitrarily close to random guessing

30

31

32 AdaBoost as a Additive Model We will now derive AdaBoost in a way that can be adapted in various ways This recipe will let you derive boosting style algorithms for particular learning settings of interest E.g., general misprediction cost, semi supervised learning these boosting style algorithms will not generally be boosting algorithms in the theoretical sense but they often work quite well

33 AdaBoost: Iterative Learning of Additive Models Consider the final hypothesis: it takes the sign of an additive expansion of a set of base classifiers AdaBoost iteratively finds at each iteration an add to to The goal is to minimize a loss function on the training example:

34 Instead, Adaboost can be viewed as minimizing an exponential loss function, which is a smooth upper bound on 0/1 error:

35 Fix and optimize

36

37 Pitfall of Boosting: sensitive to noise and outliers

38 Summary: Bagging and Boosting Bagging Resample data points Weight of each classifier is the same Only variance reduction Robust to noise and outliers Boosting Reweight data points (modify data distribution) Weight of classifier vary depending on accuracy Reduces both bias and variance Can hurt performance with noise and outliers

### Jeff Howbert Introduction to Machine Learning Winter

Classification Ensemble e Methods 1 Jeff Howbert Introduction to Machine Learning Winter 2012 1 Ensemble methods Basic idea of ensemble methods: Combining predictions from competing models often gives

### Machine Learning for Language Technology

October 2013 Machine Learning for Language Technology Lecture 6: Ensemble Methods Marina Santini, Uppsala University Department of Linguistics and Philology Where we are Previous lectures, various different

### Introduction to Ensemble Learning Featuring Successes in the Netflix Prize Competition

Introduction to Ensemble Learning Featuring Successes in the Netflix Prize Competition Todd Holloway Two Lecture Series for B551 November 20 & 27, 2007 Indiana University Outline Introduction Bias and

### Combining multiple models

Combining multiple models Basic idea of meta learning schemes: build different experts and let them vote Advantage: often improves predictive performance Disadvantage: produces output that is very hard

### Data Mining. CS57300 Purdue University. Bruno Ribeiro. February 15th, 2018

Data Mining CS573 Purdue University Bruno Ribeiro February 15th, 218 1 Today s Goal Ensemble Methods Supervised Methods Meta-learners Unsupervised Methods 215 Bruno Ribeiro Understanding Ensembles The

### Decision Tree For Playing Tennis

Decision Tree For Playing Tennis ROOT NODE BRANCH INTERNAL NODE LEAF NODE Disjunction of conjunctions Another Perspective of a Decision Tree Model Age 60 40 20 NoDefault NoDefault + + NoDefault Default

### Decision Tree Instability and Active Learning

Decision Tree Instability and Active Learning Kenneth Dwyer and Robert Holte University of Alberta November 14, 2007 Kenneth Dwyer, University of Alberta Decision Tree Instability and Active Learning 1

### A Practical Tour of Ensemble (Machine) Learning

A Practical Tour of Ensemble (Machine) Learning Nima Hejazi Evan Muzzall Division of Biostatistics, University of California, Berkeley D-Lab, University of California, Berkeley slides: https://googl/wwaqc

### Decision Boundary. Hemant Ishwaran and J. Sunil Rao

32 Decision Trees, Advanced Techniques in Constructing define impurity using the log-rank test. As in CART, growing a tree by reducing impurity ensures that terminal nodes are populated by individuals

### Multiple classifiers

Multiple classifiers JERZY STEFANOWSKI Institute of Computing Sciences Poznań University of Technology Zajęcia dla TPD - ZED 2009 Oparte na wykładzie dla Doctoral School, Catania-Troina, April, 2008 Outline

### Feature Selection for Ensembles

From: AAAI-99 Proceedings. Copyright 1999, AAAI (www.aaai.org). All rights reserved. Feature Selection for Ensembles David W. Opitz Computer Science Department University of Montana Missoula, MT 59812

### Machine Learning and Data Mining. Ensembles of Learners. Prof. Alexander Ihler

Machine Learning and Data Mining Ensembles of Learners Prof. Alexander Ihler Ensemble methods Why learn one classifier when you can learn many? Ensemble: combine many predictors (Weighted) combina

### Multiple classifiers. JERZY STEFANOWSKI Institute of Computing Sciences Poznań University of Technology. Doctoral School, Catania-Troina, April, 2008

Multiple classifiers JERZY STEFANOWSKI Institute of Computing Sciences Poznań University of Technology Doctoral School, Catania-Troina, April, 2008 Outline of the presentation 1. Introduction 2. Why do

### An Empirical Study of Combining Boosting-BAN and Boosting-MultiTAN

Research Journal of Applied Sciences, Engineering and Technology 5(24): 5550-5555, 2013 ISSN: 2040-7459; e-issn: 2040-7467 Maxwell Scientific Organization, 2013 Submitted: September 24, 2012 Accepted:

### A Few Useful Things to Know about Machine Learning. Pedro Domingos Department of Computer Science and Engineering University of Washington" 2012"

A Few Useful Things to Know about Machine Learning Pedro Domingos Department of Computer Science and Engineering University of Washington 2012 A Few Useful Things to Know about Machine Learning Machine

### 6 COMBINED MACHINE LEARNING AND FEATURE DESIGN

In the previous chapter, we presented an evaluation of the state-of-the-art machine learning algorithms for the task of classification using a real world problem and dataset. We calculated our results

### Dudon Wai Georgia Institute of Technology CS 7641: Machine Learning Atlanta, GA

Adult Income and Letter Recognition - Supervised Learning Report An objective look at classifier performance for predicting adult income and Letter Recognition Dudon Wai Georgia Institute of Technology

### Ensemble Neural Networks Using Interval Neutrosophic Sets and Bagging

Ensemble Neural Networks Using Interval Neutrosophic Sets and Bagging Pawalai Kraipeerapun, Chun Che Fung and Kok Wai Wong School of Information Technology, Murdoch University, Australia Email: {p.kraipeerapun,

### Refine Decision Boundaries of a Statistical Ensemble by Active Learning

Refine Decision Boundaries of a Statistical Ensemble by Active Learning a b * Dingsheng Luo and Ke Chen a National Laboratory on Machine Perception and Center for Information Science, Peking University,

### Analysis of Different Classifiers for Medical Dataset using Various Measures

Analysis of Different for Medical Dataset using Various Measures Payal Dhakate ME Student, Pune, India. K. Rajeswari Associate Professor Pune,India Deepa Abin Assistant Professor, Pune, India ABSTRACT

### 18 LEARNING FROM EXAMPLES

18 LEARNING FROM EXAMPLES An intelligent agent may have to learn, for instance, the following components: A direct mapping from conditions on the current state to actions A means to infer relevant properties

### A study of the NIPS feature selection challenge

A study of the NIPS feature selection challenge Nicholas Johnson November 29, 2009 Abstract The 2003 Nips Feature extraction challenge was dominated by Bayesian approaches developed by the team of Radford

### Supervised learning can be done by choosing the hypothesis that is most probable given the data: = arg max ) = arg max

The learning problem is called realizable if the hypothesis space contains the true function; otherwise it is unrealizable On the other hand, in the name of better generalization ability it may be sensible

### Introduction to Classification

Introduction to Classification Classification: Definition Given a collection of examples (training set ) Each example is represented by a set of features, sometimes called attributes Each example is to

### Lecture 1: Machine Learning Basics

1/69 Lecture 1: Machine Learning Basics Ali Harakeh University of Waterloo WAVE Lab ali.harakeh@uwaterloo.ca May 1, 2017 2/69 Overview 1 Learning Algorithms 2 Capacity, Overfitting, and Underfitting 3

### COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection.

COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection. Instructor: Herke van Hoof (herke.vanhoof@mail.mcgill.ca) Slides mostly by: Class web page: www.cs.mcgill.ca/~hvanho2/comp551

### COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection.

COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection. Instructor: (jpineau@cs.mcgill.ca) Class web page: www.cs.mcgill.ca/~jpineau/comp551 Unless otherwise

### A Comparison of Face Detection Algorithms

A Comparison of Face Detection Algorithms Ian R. Fasel 1 and Javier R. Movellan 2 1 Department of Cognitive Science, University of California, San Diego La Jolla, CA, 92093-0515 2 Institute for Neural

### Big Data Analytics Clustering and Classification

E6893 Big Data Analytics Lecture 4: Big Data Analytics Clustering and Classification Ching-Yung Lin, Ph.D. Adjunct Professor, Dept. of Electrical Engineering and Computer Science September 28th, 2017 1

### Machine Learning 2nd Edition

INTRODUCTION TO Lecture Slides for Machine Learning 2nd Edition ETHEM ALPAYDIN, modified by Leonardo Bobadilla and some parts from http://www.cs.tau.ac.il/~apartzin/machinelearning/ The MIT Press, 2010

### Online Ensemble Learning: An Empirical Study

Online Ensemble Learning: An Empirical Study Alan Fern AFERN@ECN.PURDUE.EDU Robert Givan GIVAN@ECN.PURDUE.EDU Department of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 4797

### ANALYZING BIG DATA WITH DECISION TREES

San Jose State University SJSU ScholarWorks Master's Projects Master's Theses and Graduate Research Spring 2014 ANALYZING BIG DATA WITH DECISION TREES Lok Kei Leong Follow this and additional works at:

### Boosted Mixture of Experts: An Ensemble Learning Scheme

LETTER Communicated by Robert Jacobs Boosted Mixture of Experts: An Ensemble Learning Scheme Ran Avnimelech Nathan Intrator Department of Computer Science, Sackler Faculty of Exact Sciences, Tel-Aviv University,

### Number of classifiers in error

Ensemble Methods in Machine Learning Thomas G. Dietterich Oregon State University, Corvallis, Oregon, USA, tgd@cs.orst.edu, WWW home page: http://www.cs.orst.edu/~tgd Abstract. Ensemble methods are learning

### Foundations of Intelligent Systems CSCI (Fall 2015)

Foundations of Intelligent Systems CSCI-630-01 (Fall 2015) Final Examination, Fri. Dec 18, 2015 Instructor: Richard Zanibbi, Duration: 120 Minutes Name: Instructions The exam questions are worth a total

### Python Machine Learning

Python Machine Learning Unlock deeper insights into machine learning with this vital guide to cuttingedge predictive analytics Sebastian Raschka [ PUBLISHING 1 open source I community experience distilled

### Scheduling Tasks under Constraints CS229 Final Project

Scheduling Tasks under Constraints CS229 Final Project Mike Yu myu3@stanford.edu Dennis Xu dennisx@stanford.edu Kevin Moody kmoody@stanford.edu Abstract The project is based on the principle of unconventional

### An Adaptive Sampling Ensemble Classifier for Learning from Imbalanced Data Sets

An Adaptive Sampling Ensemble Classifier for Learning from Imbalanced Data Sets Ordonez Jon Geiler, Li Hong, Guo Yue-jian Abstract In Imbalanced datasets, minority classes can be erroneously classified

### Machine Learning Tom M. Mitchell Machine Learning Department Carnegie Mellon University. January 11, 2011

Machine Learning 10-701 Tom M. Mitchell Machine Learning Department Carnegie Mellon University January 11, 2011 Today: What is machine learning? Decision tree learning Course logistics Readings: The Discipline

### Online Ensemble Learning: An Empirical Study

Online Ensemble Learning: An Empirical Study Alan Fern AFERN@ECN.PURDUE.EDU Robert Givan GIVAN@ECN.PURDUE.EDU Department of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 4797

### Introduction to Classification, aka Machine Learning

Introduction to Classification, aka Machine Learning Classification: Definition Given a collection of examples (training set ) Each example is represented by a set of features, sometimes called attributes

### Investigation of Property Valuation Models Based on Decision Tree Ensembles Built over Noised Data

Investigation of Property Valuation Models Based on Decision Tree Ensembles Built over Noised Data Tadeusz Lasota 1, Tomasz Łuczak 2, Michał Niemczyk 2, Michał Olszewski 2, Bogdan Trawiński 2 1 Wrocław

### Machine Learning : Hinge Loss

Machine Learning Hinge Loss 16/01/2014 Machine Learning : Hinge Loss Recap tasks considered before Let a training dataset be given with (i) data and (ii) classes The goal is to find a hyper plane that

### CS534 Machine Learning

CS534 Machine Learning Spring 2013 Lecture 1: Introduction to ML Course logistics Reading: The discipline of Machine learning by Tom Mitchell Course Information Instructor: Dr. Xiaoli Fern Kec 3073, xfern@eecs.oregonstate.edu

### Online Ensemble Learning: An Empirical Study

Online Ensemble Learning: An Empirical Study Alan Fern AFERN@ECN.PURDUE.EDU Robert Givan GIVAN@ECN.PURDUE.EDU Department of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 4797

### Stay Alert!: Creating a Classifier to Predict Driver Alertness in Real-time

Stay Alert!: Creating a Classifier to Predict Driver Alertness in Real-time Aditya Sarkar, Julien Kawawa-Beaudan, Quentin Perrot Friday, December 11, 2014 1 Problem Definition Driving while drowsy inevitably

### Clustering Students to Generate an Ensemble to Improve Standard Test Score Predictions

Clustering Students to Generate an Ensemble to Improve Standard Test Score Predictions Shubhendu Trivedi, Zachary A. Pardos, Neil T. Heffernan Department of Computer Science, Worcester Polytechnic Institute,

### TOWARDS DATA-DRIVEN AUTONOMICS IN DATA CENTERS

TOWARDS DATA-DRIVEN AUTONOMICS IN DATA CENTERS ALINA SIRBU, OZALP BABAOGLU SUMMARIZED BY ARDA GUMUSALAN MOTIVATION 2 MOTIVATION Human-interaction-dependent data centers are not sustainable for future data

### Machine Learning L, T, P, J, C 2,0,2,4,4

Subject Code: Objective Expected Outcomes Machine Learning L, T, P, J, C 2,0,2,4,4 It introduces theoretical foundations, algorithms, methodologies, and applications of Machine Learning and also provide

(Sub)Gradient Descent CMSC 422 MARINE CARPUAT marine@cs.umd.edu Figures credit: Piyush Rai Logistics Midterm is on Thursday 3/24 during class time closed book/internet/etc, one page of notes. will include

### CS 4510/9010 Applied Machine Learning. Evaluation. Paula Matuszek Fall, copyright Paula Matuszek 2016

CS 4510/9010 Applied Machine Learning 1 Evaluation Paula Matuszek Fall, 2016 Evaluating Classifiers 2 With a decision tree, or with any classifier, we need to know how well our trained model performs on

### Combating the Class Imbalance Problem in Small Sample Data Sets

Combating the Class Imbalance Problem in Small Sample Data Sets Michael Wasikowski Submitted to the Department of Electrical Engineering & Computer Science and the Graduate Faculty of the University of

### IMBALANCED data sets (IDS) correspond to domains

Diversity Analysis on Imbalanced Data Sets by Using Ensemble Models Shuo Wang and Xin Yao Abstract Many real-world applications have problems when learning from imbalanced data sets, such as medical diagnosis,

### Ensemble Classifier for Solving Credit Scoring Problems

Ensemble Classifier for Solving Credit Scoring Problems Maciej Zięba and Jerzy Świątek Wroclaw University of Technology, Faculty of Computer Science and Management, Wybrzeże Wyspiańskiego 27, 50-370 Wrocław,

### Linear Regression. Chapter Introduction

Chapter 9 Linear Regression 9.1 Introduction In this class, we have looked at a variety of di erent models and learning methods, such as finite state machines, sequence models, and classification methods.

### P(A, B) = P(A B) = P(A) + P(B) - P(A B)

AND Probability P(A, B) = P(A B) = P(A) + P(B) - P(A B) P(A B) = P(A) + P(B) - P(A B) Area = Probability of Event AND Probability P(A, B) = P(A B) = P(A) + P(B) - P(A B) If, and only if, A and B are independent,

### Practical Methods for the Analysis of Big Data

Practical Methods for the Analysis of Big Data Module 4: Clustering, Decision Trees, and Ensemble Methods Philip A. Schrodt The Pennsylvania State University schrodt@psu.edu Workshop at the Odum Institute

### Ensemble Approaches for Regression: a Survey

Ensemble Approaches for Regression: a Survey João M. Moreira a,, Carlos Soares b,c, Alípio M. Jorge b,c and Jorge Freire de Sousa a a Faculty of Engineering, University of Porto, Rua Dr. Roberto Frias,

### Session 1: Gesture Recognition & Machine Learning Fundamentals

IAP Gesture Recognition Workshop Session 1: Gesture Recognition & Machine Learning Fundamentals Nicholas Gillian Responsive Environments, MIT Media Lab Tuesday 8th January, 2013 My Research My Research

### Overview of TreeNet Technology Stochastic Gradient Boosting

Overview of TreeNet Technology Stochastic Gradient Boosting Dan Steinberg January 2009 Introduction to TreeNet: Stochastic Gradient Boosting Powerful new approach to machine learning and function approximation

### Machine Learning with Weka

Machine Learning with Weka SLIDES BY (TOTAL 5 Session of 1.5 Hours Each) ANJALI GOYAL & ASHISH SUREKA (www.ashish-sureka.in) CS 309 INFORMATION RETRIEVAL COURSE ASHOKA UNIVERSITY NOTE: Slides created and

### ROBERTO BATTITI, MAURO BRUNATO. The LION Way: Machine Learning plus Intelligent Optimization. LIONlab, University of Trento, Italy, Apr 2015

ROBERTO BATTITI, MAURO BRUNATO. The LION Way: Machine Learning plus Intelligent Optimization. LIONlab, University of Trento, Italy, Apr 2015 http://intelligentoptimization.org/lionbook Roberto Battiti

### CLASS distribution, i.e., the proportion of instances belonging

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS PART C: APPLICATIONS AND REVIEWS, VOL. 42, NO. 4, JULY 2012 463 A Review on Ensembles for the Class Imbalance Problem: Bagging-, Boosting-, and Hybrid-Based

### Mocking the Draft Predicting NFL Draft Picks and Career Success

Mocking the Draft Predicting NFL Draft Picks and Career Success Wesley Olmsted [wolmsted], Jeff Garnier [jeff1731], Tarek Abdelghany [tabdel] 1 Introduction We started off wanting to make some kind of

### Targeted Feature Dropout for Robust Slot Filling in Natural Language Understanding

Targeted Feature Dropout for Robust Slot Filling in Natural Language Understanding Puyang Xu, Ruhi Sarikaya Microsoft Corporation, Redmond WA 98052, USA {puyangxu, ruhi.sarikaya}@microsoft.com Abstract

### A Procedure for Classifying New Respondents into Existing Segments Using Maximum Difference Scaling

A Procedure for Classifying New Respondents into Existing Segments Using Maximum Difference Scaling Background Bryan Orme and Rich Johnson, Sawtooth Software March, 2009 (with minor clarifications September

### Scaling Quality On Quora Using Machine Learning

Scaling Quality On Quora Using Machine Learning Nikhil Garg @nikhilgarg28 @Quora @QconSF 11/7/16 Goals Of The Talk Introducing specific product problems we need to solve to stay high-quality Describing

### Neighbourhood Sampling in Bagging for Imbalanced Data

Neighbourhood Sampling in Bagging for Imbalanced Data Jerzy Błaszczyński, Jerzy Stefanowski Institute of Computing Sciences, Poznań University of Technology, 60 965 Poznań, Poland Abstract Various approaches

### Introduction To Ensemble Learning

Educational Series Introduction To Ensemble Learning Dr. Oliver Steinki, CFA, FRM Ziad Mohammad Volume I: Series 1 July 2015 What Is Ensemble Learning? In broad terms, ensemble learning is a procedure

### Welcome to CMPS 142 and 242: Machine Learning

Welcome to CMPS 142 and 242: Machine Learning Instructor: David Helmbold, dph@soe.ucsc.edu Office hours: Monday 1:30-2:30, Thursday 4:15-5:00 TA: Aaron Michelony, amichelo@soe.ucsc.edu Web page: www.soe.ucsc.edu/classes/cmps242/fall13/01

### Bird Species Identification from an Image

Bird Species Identification from an Image Aditya Bhandari, 1 Ameya Joshi, 2 Rohit Patki 3 1 Department of Computer Science, Stanford University 2 Department of Electrical Engineering, Stanford University

### Principles of Machine Learning

Principles of Machine Learning Lab 5 - Optimization-Based Machine Learning Models Overview In this lab you will explore the use of optimization-based machine learning models. Optimization-based models

### Compacting Instances: Creating models

Decision Trees Compacting Instances: Creating models Food Chat Speedy Price Bar BigTip (3) (2) (2) (2) (2) 1 great yes yes adequate no yes 2 great no yes adequate no yes 3 mediocre yes no high no no 4

### Decision Tree for Playing Tennis

Decision Tree Decision Tree for Playing Tennis (outlook=sunny, wind=strong, humidity=normal,? ) DT for prediction C-section risks Characteristics of Decision Trees Decision trees have many appealing properties

### Admission Prediction System Using Machine Learning

Admission Prediction System Using Machine Learning Jay Bibodi, Aasihwary Vadodaria, Anand Rawat, Jaidipkumar Patel bibodi@csus.edu, aaishwaryvadoda@csus.edu, anandrawat@csus.edu, jaidipkumarpate@csus.edu

### Machine Learning Lecture 1: Introduction

Welcome to CSCE 478/878! Please check off your name on the roster, or write your name if you're not listed Indicate if you wish to register or sit in Policy on sit-ins: You may sit in on the course without

### Cascade evaluation of clustering algorithms

Cascade evaluation of clustering algorithms Laurent Candillier 1,2, Isabelle Tellier 1, Fabien Torre 1, Olivier Bousquet 2 1 GRAppA - Charles de Gaulle University - Lille 3 candillier@grappa.univ-lille3.fr

### Machine Learning Tom M. Mitchell Machine Learning Department Carnegie Mellon University. January 12, 2015

Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University January 12, 2015 Today: What is machine learning? Decision tree learning Course logistics Readings: The Discipline

### Linear Regression: Predicting House Prices

Linear Regression: Predicting House Prices I am big fan of Kalid Azad writings. He has a knack of explaining hard mathematical concepts like Calculus in simple words and helps the readers to get the intuition

### 10701/15781 Machine Learning, Spring 2005: Homework 1

10701/15781 Machine Learning, Spring 2005: Homework 1 Due: Monday, February 6, beginning of the class 1 [15 Points] Probability and Regression [Stano] 1 1.1 [10 Points] The Matrix Strikes Back The Matrix

### Pattern-Aided Regression Modelling and Prediction Model Analysis

San Jose State University SJSU ScholarWorks Master's Projects Master's Theses and Graduate Research Fall 2015 Pattern-Aided Regression Modelling and Prediction Model Analysis Naresh Avva Follow this and

### Cross-Domain Video Concept Detection Using Adaptive SVMs

Cross-Domain Video Concept Detection Using Adaptive SVMs AUTHORS: JUN YANG, RONG YAN, ALEXANDER G. HAUPTMANN PRESENTATION: JESSE DAVIS CS 3710 VISUAL RECOGNITION Problem-Idea-Challenges Address accuracy

### INTRODUCTION TO DATA SCIENCE

DATA11001 INTRODUCTION TO DATA SCIENCE EPISODE 6: MACHINE LEARNING TODAY S MENU 1. WHAT IS ML? 2. CLASSIFICATION AND REGRESSSION 3. EVALUATING PERFORMANCE & OVERFITTING WHAT IS MACHINE LEARNING? Definition:

### An Automatic Construction and Organization Strategy for Ensemble Learning on Data Streams

An Automatic Construction and Organization Strategy for Ensemble Learning on Data Streams Yi Zhang School of Software Tsinghua University, Beijing, 100084 China zhang-yi@mails.tsinghua.edu.cn Xiaoming

### Modelling Student Knowledge as a Latent Variable in Intelligent Tutoring Systems: A Comparison of Multiple Approaches

Modelling Student Knowledge as a Latent Variable in Intelligent Tutoring Systems: A Comparison of Multiple Approaches Qandeel Tariq, Alex Kolchinski, Richard Davis December 6, 206 Introduction This paper

### Machine Learning (Decision Trees and Intro to Neural Nets) CSCI 3202, Fall 2010

Machine Learning (Decision Trees and Intro to Neural Nets) CSCI 3202, Fall 2010 Assignments To read this week: Chapter 18, sections 1-4 and 7 Problem Set 3 due next week! Learning a Decision Tree We look

### Inductive Learning and Decision Trees

Inductive Learning and Decision Trees Doug Downey EECS 349 Spring 2017 with slides from Pedro Domingos, Bryan Pardo Outline Announcements Homework #1 was assigned on Monday (due in five days!) Inductive

### Programming Social Robots for Human Interaction. Lecture 4: Machine Learning and Pattern Recognition

Programming Social Robots for Human Interaction Lecture 4: Machine Learning and Pattern Recognition Zheng-Hua Tan Dept. of Electronic Systems, Aalborg Univ., Denmark zt@es.aau.dk, http://kom.aau.dk/~zt

### The Effect of Large Training Set Sizes on Online Japanese Kanji and English Cursive Recognizers

The Effect of Large Training Set Sizes on Online Japanese Kanji and English Cursive Recognizers Henry A. Rowley Manish Goyal John Bennett Microsoft Corporation, One Microsoft Way, Redmond, WA 98052, USA

### Model evaluation, multi model ensembles and structural error

ETH Zurich Reto Knutti Model evaluation, multi model ensembles and structural error Reto Knutti, IAC ETH Toy model Model: obs = linear trend + noise(variance, spectrum) Short term predictability, separation

### Learning Imbalanced Data with Random Forests

Learning Imbalanced Data with Random Forests Chao Chen (Stat., UC Berkeley) chenchao@stat.berkeley.edu Andy Liaw (Merck Research Labs) andy_liaw@merck.com Leo Breiman (Stat., UC Berkeley) leo@stat.berkeley.edu

### Classification with Deep Belief Networks. HussamHebbo Jae Won Kim

Classification with Deep Belief Networks HussamHebbo Jae Won Kim Table of Contents Introduction... 3 Neural Networks... 3 Perceptron... 3 Backpropagation... 4 Deep Belief Networks (RBM, Sigmoid Belief

Adaptive Cluster Ensemble Selection Javad Azimi, Xiaoli Fern Department of Electrical Engineering and Computer Science Oregon State University {Azimi, xfern}@eecs.oregonstate.edu Abstract Cluster ensembles

### Linear Models Continued: Perceptron & Logistic Regression

Linear Models Continued: Perceptron & Logistic Regression CMSC 723 / LING 723 / INST 725 Marine Carpuat Slides credit: Graham Neubig, Jacob Eisenstein Linear Models for Classification Feature function

### CSE258 Assignment 2 brb Predicting on Airbnb

CSE258 Assignment 2 brb Predicting on Airbnb Arvind Rao A10735113 a3rao@ucsd.edu Behnam Hedayatnia A09920117 bhedayat@ucsd.edu Daniel Riley A10730856 dgriley@ucsd.edu Ninad Kulkarni A09807450 nkulkarn@ucsd.edu

### Speeding up ResNet training

Speeding up ResNet training Konstantin Solomatov (06246217), Denis Stepanov (06246218) Project mentor: Daniel Kang December 2017 Abstract Time required for model training is an important limiting factor

### Seeing the Forest through the Trees

Seeing the Forest through the Trees Learning a Comprehensible Model from a First Order Ensemble Anneleen Van Assche and Hendrik Blockeel Computer Science Department, Katholieke Universiteit Leuven, Belgium