Welcome to CMPS 142 and 242: Machine Learning

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Welcome to CMPS 142 and 242: Machine Learning"

Transcription

1 Welcome to CMPS 142 and 242: Machine Learning Instructor: David Helmbold, Office hours: Monday 1:30-2:30, Thursday 4:15-5:00 TA: Aaron Michelony, Web page: Text: Pattern Recognition and Machine Learning, by Bishop 1

2 Administrivia Sign up sheet (enrollment) Evaluation: Group Homework 20% Late midterm exam 40 % Projects (group) 40 % Must pass both exam and project Expectations/Style Reading assignments Attendance/participation My hearing/writing Academic honesty Topics: Introduction Bayesian learning and parameter estimation Instance based methods Linear Regression Linear Classification Decision Trees and Neural networks Graphical Models Support Vector Machines Clustering, EM Algorithm Boosting (AdaBoost) On-line prediction Reinforcement learning 2

3 Lecture Slides for INTRODUCTION TO Machine Learning ETHEM ALPAYDIN The MIT Press, 2004 (modified by DPH )

4 CHAPTER 1: Introduction

5 Why Learn? Machine learning is programming computers to optimize a performance criterion using example data or past experience (inference in statistics) There is no need to learn to calculate payroll Learning is used when: Human expertise does not exist (navigating on Mars), Humans are unable to explain their expertise (speech recognition, object detection) Solution changes in time (routing on a computer network) Solution needs to be adapted or customized to particular cases (or users) 5

6 What We Talk About When We Talk About Learning Learning general models from a set of particular examples Data is cheap and abundant (data warehouses, data marts); knowledge is expensive and scarce. Example in retail: Customer transactions to consumer behavior: People who bought Da Vinci Code also bought The Five People You Meet in Heaven ( Build a model that is a good and useful approximation to the data. 6

7 What is Machine Learning? Optimize a performance criterion using example data or past experience. Role of Statistics: Inference from a sample Role of Computer science: Efficient algorithms to Solve the optimization problem Representing and evaluating the model for inference 7

8 Stat. Machine learning is not: Cognitive science (how people think/learn) Teaching computers to think But is related to: Statistics Data Mining - KDD Control theory part of AI, but not traditional AI 8

9 Supervised Batch Learning Assume (unknown) distribution over things Things have measurable attributes or features Get instances x by drawing things from distribution and recording observations. Teacher labels instances making examples (x, y) or (x, t) (Bishop) Set of labeled examples is the training set or sample Create hypothesis (rule) from sample hypothesis predicts on new random instances, evaluated using a loss function 9

10 Supervised Learning Framework learning prediction 10

11 Supervised Learning (cont.) Classification: labels are nominal (unordered set, e.g. {ham, spam} {democrat, republican, indep.}) Binary Classification Regression: labels are numeric (e.g. price of used car) Sometimes predictions are probabilities 11

12 Examples Thing Observations Prediction Written Digit Pixel array Which digit? message Words, Subject, sender Ham or Spam? Customer Recent purchases interest level in a new product Used car Year, make, mpg, options Price or value 12

13 Regression Example: Price of a used car x : car attributes t : price assume t = g (x θ ) g ( ) model (e.g. linear) θ parameters (w, w 0 ) t y(x) = wx+w 0 x 13

14 Batch Assumption: iid Examples Distribution of things and measurements defines some unknown (but fixed) P(x,t) over domain-label pairs Find a hypothesis h that is close to the truth A loss function L(t, t ) measures error of predictions, often L(t, t )=0 if t=t and L(t,t )=1 otherwise (classification) Want to minimize P(x,t) L(t, h(x)) -- probability of error for 0-1 loss Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning The MIT Press (V1.1) 14

15 Supervised Learning: Uses Prediction of future cases: Use the rule to predict the output for future inputs Knowledge extraction: The rule is easy to understand Compression: The rule is simpler than the data it explains Outlier detection: Exceptions that are not covered by the rule, e.g., fraud and data entry errors 15

16 Can we Generalize? Learning is an ill-posed problem: If we assume nothing else, any label t could be likely for an unseen x Need an inductive bias limiting possible P(x,t) Often assume some kind of simplicity (e.g. linearity) based on domain knowledge Bayesian approach: put prior on rules, and balance prior with evidence (data) 16

17 Noise Data not always perfect Unmeasured Features Attribute noise (random or systemic) Label noise (random or systemic) Noise associated with inductive bias errors Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning The MIT Press (V1.1) 17

18 Overfitting and Underfitting Overfitting happens when the hypothesis that is too complex for the truth Underfitting happens when the hypothesis is too simple. 18

19 Bishop fig

20 Sup. Learning as parameter estimation Model (hypothesis set or class): h θ (x) Empirical error Error/Loss function: N E θ = L( t n,h θ (x n )) n =1 + regularization(θ) Optimization procedure: ˆ θ argmin(e θ ) θ Regularization penalizes complex θ Model choice + regularization = inductive bias! Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning The MIT Press (V1.1) 20

21 Don t rely on training error! To estimate generalization error, we need data unseen during training. Often data split into Training set (50%) Validation set (25%) (did training work? Use for Parameter selection/model complexity ) Final Test (publication) set (25%) Resampling when there are few examples cross validation (describe) Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning The MIT Press (V1.1) 21

22 Other kinds of supervised learning Reinforcement learning - learning a policy for influencing or reacting to environment No supervised output, but delayed rewards Credit assignment problem Game playing/robot in a maze, etc. On-line learning: predict on each instance in turn Semi-supervised learning uses both labeled and unlabeled data Active learning request labels for particular instances 22

23 Unsupervised Learning Learning what normally happens No labels Clustering: Grouping similar instances Example applications Segmentation in customer relationship mgmt Image compression: Color quantization Bioinformatics: Learning motifs Identifying unusual Airplane landings Deep learning learn the features 23

24 Resources: Datasets UCI Repository: UCI KDD Archive: Statlib: Delve: MLcomp: 24

25 Resources: Journals Journal of Machine Learning Research Machine Learning Neural Computation Neural Networks IEEE Transactions on Neural Networks IEEE Transactions on Pattern Analysis and Machine Intelligence Annals of Statistics Journal of the American Statistical Association... 25

26 Resources: Conferences International Conference on Machine Learning (ICML) Neural Information Processing Systems (NIPS) Uncertainty in Artificial Intelligence (UAI) Computational Learning Theory (COLT) European Conference on Machine Learning (ECML) Knowledge Discovery and Data Mining (KDD) International Joint Conference on Artificial Intelligence (IJCAI) International Conference on Neural Networks (ICANN)... 26

M. R. Ahmadzadeh Isfahan University of Technology. M. R. Ahmadzadeh Isfahan University of Technology

M. R. Ahmadzadeh Isfahan University of Technology. M. R. Ahmadzadeh Isfahan University of Technology 1 2 M. R. Ahmadzadeh Isfahan University of Technology Ahmadzadeh@cc.iut.ac.ir M. R. Ahmadzadeh Isfahan University of Technology Textbooks 3 Introduction to Machine Learning - Ethem Alpaydin Pattern Recognition

More information

Machine Learning 2nd Edition

Machine Learning 2nd Edition INTRODUCTION TO Lecture Slides for Machine Learning 2nd Edition ETHEM ALPAYDIN, modified by Leonardo Bobadilla and some parts from http://www.cs.tau.ac.il/~apartzin/machinelearning/ The MIT Press, 2010

More information

CS540 Machine learning Lecture 1 Introduction

CS540 Machine learning Lecture 1 Introduction CS540 Machine learning Lecture 1 Introduction Administrivia Overview Supervised learning Unsupervised learning Other kinds of learning Outline Administrivia Class web page www.cs.ubc.ca/~murphyk/teaching/cs540-fall08

More information

T Machine Learning: Advanced Probablistic Methods

T Machine Learning: Advanced Probablistic Methods T-61.5140 Machine Learning: Advanced Probablistic Methods Jaakko Hollmén Department of Information and Computer Science Helsinki University of Technology, Finland e-mail: Jaakko.Hollmen@tkk.fi Web: http://www.cis.hut.fi/opinnot/t-61.5140/

More information

INTRODUCTION TO DATA SCIENCE

INTRODUCTION TO DATA SCIENCE DATA11001 INTRODUCTION TO DATA SCIENCE EPISODE 6: MACHINE LEARNING TODAY S MENU 1. WHAT IS ML? 2. CLASSIFICATION AND REGRESSSION 3. EVALUATING PERFORMANCE & OVERFITTING WHAT IS MACHINE LEARNING? Definition:

More information

CS545 Machine Learning

CS545 Machine Learning Machine learning and related fields CS545 Machine Learning Course Introduction Machine learning: the construction and study of systems that learn from data. Pattern recognition: the same field, different

More information

Lecture 1: Machine Learning Basics

Lecture 1: Machine Learning Basics 1/69 Lecture 1: Machine Learning Basics Ali Harakeh University of Waterloo WAVE Lab ali.harakeh@uwaterloo.ca May 1, 2017 2/69 Overview 1 Learning Algorithms 2 Capacity, Overfitting, and Underfitting 3

More information

Lecture 1: Introduc4on

Lecture 1: Introduc4on CSC2515 Spring 2014 Introduc4on to Machine Learning Lecture 1: Introduc4on All lecture slides will be available as.pdf on the course website: http://www.cs.toronto.edu/~urtasun/courses/csc2515/csc2515_winter15.html

More information

Session 1: Gesture Recognition & Machine Learning Fundamentals

Session 1: Gesture Recognition & Machine Learning Fundamentals IAP Gesture Recognition Workshop Session 1: Gesture Recognition & Machine Learning Fundamentals Nicholas Gillian Responsive Environments, MIT Media Lab Tuesday 8th January, 2013 My Research My Research

More information

Course Outline. Course Grading. Where to go for help. Academic Integrity. EE-589 Introduction to Neural Networks NN 1 EE

Course Outline. Course Grading. Where to go for help. Academic Integrity. EE-589 Introduction to Neural Networks NN 1 EE EE-589 Introduction to Neural Assistant Prof. Dr. Turgay IBRIKCI Room # 305 (322) 338 6868 / 139 Wensdays 9:00-12:00 Course Outline The course is divided in two parts: theory and practice. 1. Theory covers

More information

Module 12. Machine Learning. Version 2 CSE IIT, Kharagpur

Module 12. Machine Learning. Version 2 CSE IIT, Kharagpur Module 12 Machine Learning 12.1 Instructional Objective The students should understand the concept of learning systems Students should learn about different aspects of a learning system Students should

More information

CSE 546 Machine Learning

CSE 546 Machine Learning CSE 546 Machine Learning Instructor: Luke Zettlemoyer TA: Lydia Chilton Slides adapted from Pedro Domingos and Carlos Guestrin Logistics Instructor: Luke Zettlemoyer Email: lsz@cs Office: CSE 658 Office

More information

CSL465/603 - Machine Learning

CSL465/603 - Machine Learning CSL465/603 - Machine Learning Fall 2016 Narayanan C Krishnan ckn@iitrpr.ac.in Introduction CSL465/603 - Machine Learning 1 Administrative Trivia Course Structure 3-0-2 Lecture Timings Monday 9.55-10.45am

More information

P(A, B) = P(A B) = P(A) + P(B) - P(A B)

P(A, B) = P(A B) = P(A) + P(B) - P(A B) AND Probability P(A, B) = P(A B) = P(A) + P(B) - P(A B) P(A B) = P(A) + P(B) - P(A B) Area = Probability of Event AND Probability P(A, B) = P(A B) = P(A) + P(B) - P(A B) If, and only if, A and B are independent,

More information

18 LEARNING FROM EXAMPLES

18 LEARNING FROM EXAMPLES 18 LEARNING FROM EXAMPLES An intelligent agent may have to learn, for instance, the following components: A direct mapping from conditions on the current state to actions A means to infer relevant properties

More information

Machine Learning. Basic Concepts. Joakim Nivre. Machine Learning 1(24)

Machine Learning. Basic Concepts. Joakim Nivre. Machine Learning 1(24) Machine Learning Basic Concepts Joakim Nivre Uppsala University and Växjö University, Sweden E-mail: nivre@msi.vxu.se Machine Learning 1(24) Machine Learning Idea: Synthesize computer programs by learning

More information

CSC 411 MACHINE LEARNING and DATA MINING

CSC 411 MACHINE LEARNING and DATA MINING CSC 411 MACHINE LEARNING and DATA MINING Lectures: Monday, Wednesday 12-1 (section 1), 3-4 (section 2) Lecture Room: MP 134 (section 1); Bahen 1200 (section 2) Instructor (section 1): Richard Zemel Instructor

More information

Machine Learning for NLP

Machine Learning for NLP Natural Language Processing SoSe 2014 Machine Learning for NLP Dr. Mariana Neves April 30th, 2014 (based on the slides of Dr. Saeedeh Momtazi) Introduction Field of study that gives computers the ability

More information

A Few Useful Things to Know about Machine Learning. Pedro Domingos Department of Computer Science and Engineering University of Washington" 2012"

A Few Useful Things to Know about Machine Learning. Pedro Domingos Department of Computer Science and Engineering University of Washington 2012 A Few Useful Things to Know about Machine Learning Pedro Domingos Department of Computer Science and Engineering University of Washington 2012 A Few Useful Things to Know about Machine Learning Machine

More information

CS 445/545 Machine Learning Winter, 2017

CS 445/545 Machine Learning Winter, 2017 CS 445/545 Machine Learning Winter, 2017 See syllabus at http://web.cecs.pdx.edu/~mm/machinelearningwinter2017/ Lecture slides will be posted on this website before each class. What is machine learning?

More information

Introduction to Ensemble Learning Featuring Successes in the Netflix Prize Competition

Introduction to Ensemble Learning Featuring Successes in the Netflix Prize Competition Introduction to Ensemble Learning Featuring Successes in the Netflix Prize Competition Todd Holloway Two Lecture Series for B551 November 20 & 27, 2007 Indiana University Outline Introduction Bias and

More information

Classification with Deep Belief Networks. HussamHebbo Jae Won Kim

Classification with Deep Belief Networks. HussamHebbo Jae Won Kim Classification with Deep Belief Networks HussamHebbo Jae Won Kim Table of Contents Introduction... 3 Neural Networks... 3 Perceptron... 3 Backpropagation... 4 Deep Belief Networks (RBM, Sigmoid Belief

More information

Inductive Learning and Decision Trees

Inductive Learning and Decision Trees Inductive Learning and Decision Trees Doug Downey EECS 349 Spring 2017 with slides from Pedro Domingos, Bryan Pardo Outline Announcements Homework #1 was assigned on Monday (due in five days!) Inductive

More information

Machine Learning Tom M. Mitchell Machine Learning Department Carnegie Mellon University. January 12, 2015

Machine Learning Tom M. Mitchell Machine Learning Department Carnegie Mellon University. January 12, 2015 Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University January 12, 2015 Today: What is machine learning? Decision tree learning Course logistics Readings: The Discipline

More information

Lecture 1. Introduction Bastian Leibe Visual Computing Institute RWTH Aachen University

Lecture 1. Introduction Bastian Leibe Visual Computing Institute RWTH Aachen University Advanced Machine Learning Lecture 1 Introduction 20.10.2015 Bastian Leibe Visual Computing Institute RWTH Aachen University http://www.vision.rwth-aachen.de/ leibe@vision.rwth-aachen.de Organization Lecturer

More information

Introduction to Machine Learning

Introduction to Machine Learning 1, DATA11002 Introduction to Machine Learning Lecturer: Teemu Roos TAs: Ville Hyvönen and Janne Leppä-aho Department of Computer Science University of Helsinki (based in part on material by Patrik Hoyer

More information

(Sub)Gradient Descent

(Sub)Gradient Descent (Sub)Gradient Descent CMSC 422 MARINE CARPUAT marine@cs.umd.edu Figures credit: Piyush Rai Logistics Midterm is on Thursday 3/24 during class time closed book/internet/etc, one page of notes. will include

More information

COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection.

COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection. COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection. Instructor: (jpineau@cs.mcgill.ca) Class web page: www.cs.mcgill.ca/~jpineau/comp551 Unless otherwise

More information

COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection.

COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection. COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection. Instructor: Herke van Hoof (herke.vanhoof@mail.mcgill.ca) Slides mostly by: Class web page: www.cs.mcgill.ca/~hvanho2/comp551

More information

10-702: Statistical Machine Learning

10-702: Statistical Machine Learning 10-702: Statistical Machine Learning Syllabus, Spring 2010 http://www.cs.cmu.edu/~10702 Statistical Machine Learning is a second graduate level course in machine learning, assuming students have taken

More information

Big Data Analytics Clustering and Classification

Big Data Analytics Clustering and Classification E6893 Big Data Analytics Lecture 4: Big Data Analytics Clustering and Classification Ching-Yung Lin, Ph.D. Adjunct Professor, Dept. of Electrical Engineering and Computer Science September 28th, 2017 1

More information

Machine Learning Tom M. Mitchell Machine Learning Department Carnegie Mellon University. January 11, 2011

Machine Learning Tom M. Mitchell Machine Learning Department Carnegie Mellon University. January 11, 2011 Machine Learning 10-701 Tom M. Mitchell Machine Learning Department Carnegie Mellon University January 11, 2011 Today: What is machine learning? Decision tree learning Course logistics Readings: The Discipline

More information

CPSC 340: Machine Learning and Data Mining. Course Review/Preview Fall 2015

CPSC 340: Machine Learning and Data Mining. Course Review/Preview Fall 2015 CPSC 340: Machine Learning and Data Mining Course Review/Preview Fall 2015 Admin Assignment 6 due now. We will have office hours as usual next week. Final exam details: December 15: 8:30-11 (WESB 100).

More information

Predictive Analysis of Text: Concepts, Features, and Instances

Predictive Analysis of Text: Concepts, Features, and Instances of Text: Concepts, Features, and Instances Jaime Arguello jarguell@email.unc.edu August 26, 2015 of Text Objective: developing and evaluating computer programs that automatically detect a particular concept

More information

Computer Vision for Card Games

Computer Vision for Card Games Computer Vision for Card Games Matias Castillo matiasct@stanford.edu Benjamin Goeing bgoeing@stanford.edu Jesper Westell jesperw@stanford.edu Abstract For this project, we designed a computer vision program

More information

Classification of Arrhythmia Using Machine Learning Techniques

Classification of Arrhythmia Using Machine Learning Techniques Classification of Arrhythmia Using Machine Learning Techniques THARA SOMAN PATRICK O. BOBBIE School of Computing and Software Engineering Southern Polytechnic State University (SPSU) 1 S. Marietta Parkway,

More information

Overview COEN 296 Topics in Computer Engineering Introduction to Pattern Recognition and Data Mining Course Goals Syllabus

Overview COEN 296 Topics in Computer Engineering Introduction to Pattern Recognition and Data Mining Course Goals Syllabus Overview COEN 296 Topics in Computer Engineering to Pattern Recognition and Data Mining Instructor: Dr. Giovanni Seni G.Seni@ieee.org Department of Computer Engineering Santa Clara University Course Goals

More information

Statistics and Machine Learning, Master s Programme

Statistics and Machine Learning, Master s Programme DNR LIU-2017-02005 1(9) Statistics and Machine Learning, Master s Programme 120 credits Statistics and Machine Learning, Master s Programme F7MSL Valid from: 2018 Autumn semester Determined by Board of

More information

Syllabus Data Mining for Business Analytics - Managerial INFO-GB.3336, Spring 2018

Syllabus Data Mining for Business Analytics - Managerial INFO-GB.3336, Spring 2018 Syllabus Data Mining for Business Analytics - Managerial INFO-GB.3336, Spring 2018 Course information When: Mondays and Wednesdays 3-4:20pm Where: KMEC 3-65 Professor Manuel Arriaga Email: marriaga@stern.nyu.edu

More information

Machine Learning for SAS Programmers

Machine Learning for SAS Programmers Machine Learning for SAS Programmers The Agenda Introduction of Machine Learning Supervised and Unsupervised Machine Learning Deep Neural Network Machine Learning implementation Questions and Discussion

More information

Python Machine Learning

Python Machine Learning Python Machine Learning Unlock deeper insights into machine learning with this vital guide to cuttingedge predictive analytics Sebastian Raschka [ PUBLISHING 1 open source I community experience distilled

More information

Keywords: data mining, heart disease, Naive Bayes. I. INTRODUCTION. 1.1 Data mining

Keywords: data mining, heart disease, Naive Bayes. I. INTRODUCTION. 1.1 Data mining Heart Disease Prediction System using Naive Bayes Dhanashree S. Medhekar 1, Mayur P. Bote 2, Shruti D. Deshmukh 3 1 dhanashreemedhekar@gmail.com, 2 mayur468@gmail.com, 3 deshshruti88@gmail.com ` Abstract:

More information

Introduction to Machine Learning for NLP I

Introduction to Machine Learning for NLP I Introduction to Machine Learning for NLP I Benjamin Roth CIS LMU München Benjamin Roth (CIS LMU München) Introduction to Machine Learning for NLP I 1 / 49 Outline 1 This Course 2 Overview 3 Machine Learning

More information

Learning Agents: Introduction

Learning Agents: Introduction Learning Agents: Introduction S Luz luzs@cs.tcd.ie October 28, 2014 Learning in agent architectures Agent Learning in agent architectures Agent Learning in agent architectures Agent perception Learning

More information

Linear Models Continued: Perceptron & Logistic Regression

Linear Models Continued: Perceptron & Logistic Regression Linear Models Continued: Perceptron & Logistic Regression CMSC 723 / LING 723 / INST 725 Marine Carpuat Slides credit: Graham Neubig, Jacob Eisenstein Linear Models for Classification Feature function

More information

CSC 411: Lecture 01: Introduction

CSC 411: Lecture 01: Introduction CSC 411: Lecture 01: Introduction Richard Zemel, Raquel Urtasun and Sanja Fidler University of Toronto Zemel, Urtasun, Fidler (UofT) CSC 411: 01-Introduction 1 / 44 Today Administration details Why is

More information

Similarity-Weighted Association Rules for a Name Recommender System

Similarity-Weighted Association Rules for a Name Recommender System Similarity-Weighted Association Rules for a Name Recommender System Benjamin Letham Operations Research Center Massachusetts Institute of Technology Cambridge, MA, USA bletham@mit.edu Abstract. Association

More information

36-350: Data Mining. Fall Lectures: Monday, Wednesday and Friday, 10:30 11:20, Porter Hall 226B

36-350: Data Mining. Fall Lectures: Monday, Wednesday and Friday, 10:30 11:20, Porter Hall 226B 36-350: Data Mining Fall 2009 Instructor: Cosma Shalizi, Statistics Dept., Baker Hall 229C, cshalizi@stat.cmu.edu Teaching Assistant: Joseph Richards, jwrichar@stat.cmu.edu Lectures: Monday, Wednesday

More information

HAMLET JERRY ZHU UNIVERSITY OF WISCONSIN

HAMLET JERRY ZHU UNIVERSITY OF WISCONSIN HAMLET JERRY ZHU UNIVERSITY OF WISCONSIN Collaborators: Rui Castro, Michael Coen, Ricki Colman, Charles Kalish, Joseph Kemnitz, Robert Nowak, Ruichen Qian, Shelley Prudom, Timothy Rogers Somewhere, something

More information

Refine Decision Boundaries of a Statistical Ensemble by Active Learning

Refine Decision Boundaries of a Statistical Ensemble by Active Learning Refine Decision Boundaries of a Statistical Ensemble by Active Learning a b * Dingsheng Luo and Ke Chen a National Laboratory on Machine Perception and Center for Information Science, Peking University,

More information

A Practical Tour of Ensemble (Machine) Learning

A Practical Tour of Ensemble (Machine) Learning A Practical Tour of Ensemble (Machine) Learning Nima Hejazi Evan Muzzall Division of Biostatistics, University of California, Berkeley D-Lab, University of California, Berkeley slides: https://googl/wwaqc

More information

Dudon Wai Georgia Institute of Technology CS 7641: Machine Learning Atlanta, GA

Dudon Wai Georgia Institute of Technology CS 7641: Machine Learning Atlanta, GA Adult Income and Letter Recognition - Supervised Learning Report An objective look at classifier performance for predicting adult income and Letter Recognition Dudon Wai Georgia Institute of Technology

More information

Exploration vs. Exploitation. CS 473: Artificial Intelligence Reinforcement Learning II. How to Explore? Exploration Functions

Exploration vs. Exploitation. CS 473: Artificial Intelligence Reinforcement Learning II. How to Explore? Exploration Functions CS 473: Artificial Intelligence Reinforcement Learning II Exploration vs. Exploitation Dieter Fox / University of Washington [Most slides were taken from Dan Klein and Pieter Abbeel / CS188 Intro to AI

More information

Research Statement. Ricardo Silva Gatsby Computational Neuroscience Unit November 11, 2006

Research Statement. Ricardo Silva Gatsby Computational Neuroscience Unit November 11, 2006 Research Statement Ricardo Silva Gatsby Computational Neuroscience Unit rbas@gatsby.ucl.ac.uk November 11, 2006 1 Philosophy My work lies on the intersection of computer science and statistics. The questions

More information

Modelling Student Knowledge as a Latent Variable in Intelligent Tutoring Systems: A Comparison of Multiple Approaches

Modelling Student Knowledge as a Latent Variable in Intelligent Tutoring Systems: A Comparison of Multiple Approaches Modelling Student Knowledge as a Latent Variable in Intelligent Tutoring Systems: A Comparison of Multiple Approaches Qandeel Tariq, Alex Kolchinski, Richard Davis December 6, 206 Introduction This paper

More information

Analytical Study of Some Selected Classification Algorithms in WEKA Using Real Crime Data

Analytical Study of Some Selected Classification Algorithms in WEKA Using Real Crime Data Analytical Study of Some Selected Classification Algorithms in WEKA Using Real Crime Data Obuandike Georgina N. Department of Mathematical Sciences and IT Federal University Dutsinma Katsina state, Nigeria

More information

Capacity, Learning, Teaching

Capacity, Learning, Teaching Capacity, Learning, Teaching Xiaojin Zhu Department of Computer Sciences University of Wisconsin-Madison jerryzhu@cs.wisc.edu 23 Machine learning human learning Learning capacity and generalization bounds

More information

Supervised learning can be done by choosing the hypothesis that is most probable given the data: = arg max ) = arg max

Supervised learning can be done by choosing the hypothesis that is most probable given the data: = arg max ) = arg max The learning problem is called realizable if the hypothesis space contains the true function; otherwise it is unrealizable On the other hand, in the name of better generalization ability it may be sensible

More information

Decision Tree for Playing Tennis

Decision Tree for Playing Tennis Decision Tree Decision Tree for Playing Tennis (outlook=sunny, wind=strong, humidity=normal,? ) DT for prediction C-section risks Characteristics of Decision Trees Decision trees have many appealing properties

More information

Evaluation and Comparison of Performance of different Classifiers

Evaluation and Comparison of Performance of different Classifiers Evaluation and Comparison of Performance of different Classifiers Bhavana Kumari 1, Vishal Shrivastava 2 ACE&IT, Jaipur Abstract:- Many companies like insurance, credit card, bank, retail industry require

More information

Course 395: Machine Learning - Lectures

Course 395: Machine Learning - Lectures Course 395: Machine Learning - Lectures Lecture 1-2: Concept Learning (M. Pantic) Lecture 3-4: Decision Trees & CBC Intro (M. Pantic & S. Petridis) Lecture 5-6: Evaluating Hypotheses (S. Petridis) Lecture

More information

In-depth: Deep learning (one lecture) Applied to both SL and RL above Code examples

In-depth: Deep learning (one lecture) Applied to both SL and RL above Code examples Introduction to machine learning (two lectures) Supervised learning Reinforcement learning (lab) In-depth: Deep learning (one lecture) Applied to both SL and RL above Code examples 2017-09-30 2 1 To enable

More information

Secondary Masters in Machine Learning

Secondary Masters in Machine Learning Secondary Masters in Machine Learning Student Handbook Revised 8/20/14 Page 1 Table of Contents Introduction... 3 Program Requirements... 4 Core Courses:... 5 Electives:... 6 Double Counting Courses:...

More information

Semi-Supervised Self-Training with Decision Trees: An Empirical Study

Semi-Supervised Self-Training with Decision Trees: An Empirical Study 1 Semi-Supervised Self-Training with Decision Trees: An Empirical Study Jafar Tanha, Maarten van Someren, and Hamideh Afsarmanesh Computer science Department,University of Amsterdam, The Netherlands J.Tanha,M.W.vanSomeren,h.afsarmanesh@uva.nl

More information

A study of the NIPS feature selection challenge

A study of the NIPS feature selection challenge A study of the NIPS feature selection challenge Nicholas Johnson November 29, 2009 Abstract The 2003 Nips Feature extraction challenge was dominated by Bayesian approaches developed by the team of Radford

More information

Cost-Sensitive Learning and the Class Imbalance Problem

Cost-Sensitive Learning and the Class Imbalance Problem To appear in Encyclopedia of Machine Learning. C. Sammut (Ed.). Springer. 2008 Cost-Sensitive Learning and the Class Imbalance Problem Charles X. Ling, Victor S. Sheng The University of Western Ontario,

More information

Government of Russian Federation. Federal State Autonomous Educational Institution of High Professional Education

Government of Russian Federation. Federal State Autonomous Educational Institution of High Professional Education Government of Russian Federation Federal State Autonomous Educational Institution of High Professional Education National Research University Higher School of Economics Syllabus for the course Advanced

More information

Inductive Learning and Decision Trees

Inductive Learning and Decision Trees Inductive Learning and Decision Trees Doug Downey EECS 349 Winter 2014 with slides from Pedro Domingos, Bryan Pardo Outline Announcements Homework #1 assigned Have you completed it? Inductive learning

More information

About This Specialization

About This Specialization About This Specialization The 5 courses in this University of Michigan specialization introduce learners to data science through the python programming language. This skills-based specialization is intended

More information

Artificial Neural Networks in Data Mining

Artificial Neural Networks in Data Mining IOSR Journal of Computer Engineering (IOSR-JCE) e-issn: 2278-0661,p-ISSN: 2278-8727, Volume 18, Issue 6, Ver. III (Nov.-Dec. 2016), PP 55-59 www.iosrjournals.org Artificial Neural Networks in Data Mining

More information

ECE-271A Statistical Learning I

ECE-271A Statistical Learning I ECE-271A Statistical Learning I Nuno Vasconcelos ECE Department, UCSD The course the course is an introductory level course in statistical learning by introductory I mean that you will not need any previous

More information

10701: Intro to Machine Learning. Instructors: Pradeep Ravikumar, Manuela Veloso, Teaching Assistants:

10701: Intro to Machine Learning. Instructors: Pradeep Ravikumar, Manuela Veloso, Teaching Assistants: 10701: Intro to Machine Instructors: Pradeep Ravikumar, pradeepr@cs.cmu.edu Manuela Veloso, mmv@cs.cmu.edu Teaching Assistants: Shaojie Bai shaojieb@andrew.cmu.edu Adarsh Prasad adarshp@andrew.cmu.edu

More information

Machine Learning. Nate Derbinsky Assistant Professor Computer Science and Networking

Machine Learning. Nate Derbinsky Assistant Professor Computer Science and Networking Nate Derbinsky Assistant Professor Computer Science and Networking 1 Founded a computer consulting business in high school About Me PhD from University of Michigan (Go Blue!) Imagineer with Disney Research,

More information

On Multiclass Universum Learning

On Multiclass Universum Learning On Multiclass Universum Learning Sauptik Dhar Naveen Ramakrishnan Vladimir Cherkassky Mohak Shah Robert Bosch Research and Technology Center, CA University of Minnesota, MN University of Illinois at Chicago,

More information

Data Mining: A prediction for Student's Performance Using Classification Method

Data Mining: A prediction for Student's Performance Using Classification Method World Journal of Computer Application and Technoy (: 43-47, 014 DOI: 10.13189/wcat.014.0003 http://www.hrpub.org Data Mining: A prediction for tudent's Performance Using Classification Method Abeer Badr

More information

Unsupervised Learning

Unsupervised Learning 17s1: COMP9417 Machine Learning and Data Mining Unsupervised Learning May 2, 2017 Acknowledgement: Material derived from slides for the book Machine Learning, Tom M. Mitchell, McGraw-Hill, 1997 http://www-2.cs.cmu.edu/~tom/mlbook.html

More information

Analysis of Different Classifiers for Medical Dataset using Various Measures

Analysis of Different Classifiers for Medical Dataset using Various Measures Analysis of Different for Medical Dataset using Various Measures Payal Dhakate ME Student, Pune, India. K. Rajeswari Associate Professor Pune,India Deepa Abin Assistant Professor, Pune, India ABSTRACT

More information

Introduction to Pattern Recognition

Introduction to Pattern Recognition Introduction to Pattern Recognition Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Fall 2017 CS 551, Fall 2017 c 2017, Selim Aksoy (Bilkent University)

More information

Machine Learning in Practice/ Applied Machine Learning ,11-663,05-834,05-434

Machine Learning in Practice/ Applied Machine Learning ,11-663,05-834,05-434 Machine Learning in Practice/ Applied Machine Learning 11-344,11-663,05-834,05-434 Instructor: Dr. Carolyn P. Rosé, cprose@cs.cmu.edu Office Hours: Gates-Hillman Center 5415, Time TBA Teaching Assistants:

More information

Introduction to Classification, aka Machine Learning

Introduction to Classification, aka Machine Learning Introduction to Classification, aka Machine Learning Classification: Definition Given a collection of examples (training set ) Each example is represented by a set of features, sometimes called attributes

More information

CIS 419/519 Introduction to Machine Learning Course Project Guidelines

CIS 419/519 Introduction to Machine Learning Course Project Guidelines CIS 419/519 Introduction to Machine Learning Course Project Guidelines 1 Project Overview One the main goals of this course is to prepare you to apply machine learning algorithms to realworld problems.

More information

GRADUAL INFORMATION MAXIMIZATION IN INFORMATION ENHANCEMENT TO EXTRACT IMPORTANT INPUT NEURONS

GRADUAL INFORMATION MAXIMIZATION IN INFORMATION ENHANCEMENT TO EXTRACT IMPORTANT INPUT NEURONS Proceedings of the IASTED International Conference Artificial Intelligence and Applications (AIA 214) February 17-19, 214 Innsbruck, Austria GRADUAL INFORMATION MAXIMIZATION IN INFORMATION ENHANCEMENT

More information

Azure Machine Learning. Designing Iris Multi-Class Classifier

Azure Machine Learning. Designing Iris Multi-Class Classifier Media Partners Azure Machine Learning Designing Iris Multi-Class Classifier Marcin Szeliga 20 years of experience with SQL Server Trainer & data platform architect Books & articles writer Speaker at numerous

More information

An Educational Data Mining System for Advising Higher Education Students

An Educational Data Mining System for Advising Higher Education Students An Educational Data Mining System for Advising Higher Education Students Heba Mohammed Nagy, Walid Mohamed Aly, Osama Fathy Hegazy Abstract Educational data mining is a specific data mining field applied

More information

Unsupervised Learning Jointly With Image Clustering

Unsupervised Learning Jointly With Image Clustering Unsupervised Learning Jointly With Image Clustering Jianwei Yang Devi Parikh Dhruv Batra Virginia Tech https://filebox.ece.vt.edu/~jw2yang/ 1 2 Huge amount of images!!! 3 Huge amount of images!!! Learning

More information

A Transfer-Learning Approach to Exploit Noisy Information for Classification and Its Application on Sentiment Detection

A Transfer-Learning Approach to Exploit Noisy Information for Classification and Its Application on Sentiment Detection A Transfer-Learning Approach to Exploit Noisy Information for Classification and Its Application on Sentiment Detection Wei-Shih Lin *, Tsung-Ting Kuo *, Yu-Yang Huang *, Wan-Chen Lu +, Shou-De Lin * *

More information

Day 2 Lecture 5. Transfer learning and domain adaptation

Day 2 Lecture 5. Transfer learning and domain adaptation Day 2 Lecture 5 Transfer learning and domain adaptation Semi-supervised and transfer learning Myth: you can t do deep learning unless you have a million labelled examples for your problem. Reality You

More information

Learning. Part 6 in Russell / Norvig Book

Learning. Part 6 in Russell / Norvig Book Wisdom is not the product of schooling but the lifelong attempt to acquire it. - Albert Einstein Learning Part 6 in Russell / Norvig Book Gerhard Fischer AI Course, Fall 1996, Lecture October 14 1 Overview

More information

Reinforcement Learning

Reinforcement Learning Reinforcement Learning LU 1 - Introduction Dr. Joschka Bödecker AG Maschinelles Lernen und Natürlichsprachliche Systeme Albert-Ludwigs-Universität Freiburg jboedeck@informatik.uni-freiburg.de Acknowledgement

More information

Learning and Planning with Tabular Methods

Learning and Planning with Tabular Methods Carnegie Mellon School of Computer Science Deep Reinforcement Learning and Control Learning and Planning with Tabular Methods Lecture 6, CMU 10703 Katerina Fragkiadaki What can I learn by interacting with

More information

CAP 6412 Advanced Computer Vision

CAP 6412 Advanced Computer Vision CAP 6412 Advanced Computer Vision http://www.cs.ucf.edu/~bgong/cap6412.html Boqing Gong Feb 25, 2016 Next week: Vision and language Tuesday (03/01) Javier Lores Thursday (03/03) Aisha Urooji [Relation

More information

Machine Learning and Applications in Finance

Machine Learning and Applications in Finance Machine Learning and Applications in Finance Christian Hesse 1,2,* 1 Autobahn Equity Europe, Global Markets Equity, Deutsche Bank AG, London, UK christian-a.hesse@db.com 2 Department of Computer Science,

More information

Lecture 1.1: Introduction CSC Machine Learning

Lecture 1.1: Introduction CSC Machine Learning Lecture 1.1: Introduction CSC 84020 - Machine Learning Andrew Rosenberg January 29, 2010 Today Introductions and Class Mechanics. Background about me Me: Graduated from Columbia in 2009 Research Speech

More information

Lesson Plan. Preparation. Data Mining Basics BIM 1 Business Management & Administration

Lesson Plan. Preparation. Data Mining Basics BIM 1 Business Management & Administration Data Mining Basics BIM 1 Business Management & Administration Lesson Plan Performance Objective The student understands and is able to recall information on data mining basics. Specific Objectives The

More information

DS 502/MA 543 STATISTICAL METHODS FOR DATA SCIENCE

DS 502/MA 543 STATISTICAL METHODS FOR DATA SCIENCE DS 502/MA 543 STATISTICAL METHODS FOR DATA SCIENCE This course surveys the statistical methods most useful in data science applications. Topics covered include predictive modeling methods, including multiple

More information

When Dictionary Learning Meets Classification

When Dictionary Learning Meets Classification When Dictionary Learning Meets Classification Bufford, Teresa Chen, Yuxin Horning, Mitchell Shee, Liberty Supervised by: Prof. Yohann Tero August 9, 213 Abstract This report details and exts the implementation

More information

Deep Learning for AI Yoshua Bengio. August 28th, DS3 Data Science Summer School

Deep Learning for AI Yoshua Bengio. August 28th, DS3 Data Science Summer School Deep Learning for AI Yoshua Bengio August 28th, 2017 @ DS3 Data Science Summer School A new revolution seems to be in the work after the industrial revolution. And Machine Learning, especially Deep Learning,

More information

COMP 527: Data Mining and Visualization. Danushka Bollegala

COMP 527: Data Mining and Visualization. Danushka Bollegala COMP 527: Data Mining and Visualization Danushka Bollegala Introductions Lecturer: Danushka Bollegala Office: 2.24 Ashton Building (Second Floor) Email: danushka@liverpool.ac.uk Personal web: http://danushka.net/

More information

Applied Machine Learning Lecture 1: Introduction

Applied Machine Learning Lecture 1: Introduction Applied Machine Learning Lecture 1: Introduction Richard Johansson January 16, 2018 welcome to the course! machine learning is getting increasingly popular among students our courses are full! many thesis

More information

arxiv: v3 [cs.lg] 9 Mar 2014

arxiv: v3 [cs.lg] 9 Mar 2014 Learning Factored Representations in a Deep Mixture of Experts arxiv:1312.4314v3 [cs.lg] 9 Mar 2014 David Eigen 1,2 Marc Aurelio Ranzato 1 Ilya Sutskever 1 1 Google, Inc. 2 Dept. of Computer Science, Courant

More information