ECE-271A Statistical Learning I

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "ECE-271A Statistical Learning I"

Transcription

1 ECE-271A Statistical Learning I Nuno Vasconcelos ECE Department, UCSD

2 The course the course is an introductory level course in statistical learning by introductory I mean that you will not need any previous exposure to the field, not that it is basic we will cover the foundations of Bayesian or generative learning 271B is a follow-up course on discriminant learning, in alternating years more on generative vs discriminant later 2

3 Logistics Exams: 1 mid-term - 35% 1 final 45% (covers everything) Homework (20%): one problem set every week. will include a small computational problem. By small, I mean in terms of concepts, thinking, etc. some computational problems will require a fair amount of computer power, e.g. a few hours on a low-end PC. be sure to start early will count 20%, but almost impossible without it. will give you the hands-on experience needed to be able to claim that you really know learning! 3

4 Homework policies homework is individual OK to work on problems with someone else but you have to: write your own solution write down the names of who you collaborated with homework is due one week after it is issued. 4

5 Cheetah statistical learning only makes sense when you try it on data we will test what we learn on a image processing problem given the cheetah image, can we teach a computer to segment it into object and foreground? the question will be answered with different techniques, typically one problem per week a total of 5 computer problems try to keep an eye on the big picture, e.g. did this improve over what we had done before? 5

6 Resources Course web page: all handouts, problem sets, code will be available there TA: TBA, Me: Nuno Vasconcelos, EBU Office hours: TA: TBA mine: Fridays, 9:30-10:30AM for homework talk to TA first, everything else see me My assistant: Travis Spackman outside my office, may sometimes be involved in administrative issues 6

7 Texts required: Pattern Classification, Duda, Hart, and Stork, John Willey and Sons, 2001 will follow closely, hand-outs where needed various other good, but optional, texts: Pattern Recognition and Machine Learning, Bishop, 2006 Elements of Statistical Learning, Hastie, Tibshirani, Fredman, 2001 Bayesian Data Analysis, Gelman, Rubin, Stern, A Probabilistic Theory of Pattern Recognition, Devroye, Gyorfi, Lugosi, 1998 (more than what we need) stuff you must know really well: Linear Algebra, Gilbert Strang, 1988 Fundamentals of Applied Probability, Drake, McGraw-Hill,

8 The course why statistical learning? there are many processes in the world that are ruled by deterministic equations e.g. f = ma; linear systems and convolution, Fourier, etc, various chemical laws there may be some noise, error, variability, but we can leave with those we don t need statistical learning learning is needed when there is a need for predictions about variables in the world, Y that depend on factors (other variables) X in a way that is impossible or too difficult to derive an equation for. 8

9 Examples data-mining view: large amounts of data that does not follow deterministic rules e.g. given an history of thousands of customer records and some questions that I can ask you, how do I predict that you will pay on time? impossible to derive a theorem for this, must be learned while many associate learning with data-mining, it is by no means the only or more important application signal processing view: signals combine in ways that depend on hidden structure (e.g. speech waveforms depend on language, grammar, etc.) signals are usually subject to significant amounts of noise (which sometimes means things we do not know how to model ) 9

10 Examples (cont d) signal processing view: e.g. the cocktail party problem, although there are all these people talking, I can figure everything out. how do I build a chip to separate the speakers? model the hidden dependence as a linear combination of independent sources noise many other examples in the areas of wireless, communications, signal restoration, etc. 10

11 Examples (cont d) perception/ai view: it is a complex world, I cannot model everything in detail rely on probabilistic models that explicitly account for the variability use the laws of probability to make inferences, e.g. what is P( burglar alarm, no earthquake) is high P( burglar alarm, earthquake) is low a whole field that studies perception as Bayesian inference perception really just confirms what you already know priors + observations = robust inference 11

12 Examples (cont d) communications view: detection problems: X channel Y I see Y, and know something about the statistics of the channel. What was X? this is the canonic detection problem that appears all over learning. for example, face detection in computer vision: I see pixel array Y. Is it a face? 12

13 Statistical learning goal: given a function x f (.) y = f (x ) and a collection of example data-points, learn what the function f(.) is. this is called training. two major types of learning: unsupervised: only X is known, usually referred to as clustering; supervised: both are known during training, only X known at test time, usually referred to as classification or regression. 13

14 Supervised learning X can be anything, but the type of Y dictates the type of supervised learning problem Y in {0,1} referred to as detection Y in {0,..., M-1} referred to as classification Y real referred to as regression theory is quite similar, algorithms similar most of the time we will emphasize classification, but will talk about regression when particularly insightful 14

15 Example classifying fish: fish roll down a conveyer belt camera takes a picture goal: is this a salmon or a seabass? Q: what is X? What features do I use to distinguish between the two fish? this is somewhat of an artform. Frequently, the best is to ask experts. e.g. obvious! use length and scale width! 15

16 Classification/detection two major types of classifiers: discriminant: directly recover the decision boundary that best separates the classes; generative: fit a probability model to each class and then analyze the models to find the border. a lot more on this later! focus will be on generative learning. discriminant will be covered by 271B. 16

17 Caution how do we know learning worked? we care about generalization, i.e. accuracy outside training set models that are too powerful can lead to over-fitting: e.g. in regression I can always fit exactly n pts with polynomial of order n-1. is this good? how likely is the error to be small outside the training set? similar problem for classification fundamental LAW: only test set results matter!!! 17

18 Generalization good generalization requires controlling the trade-off between training and test error training error large, test error large training error smaller, test error smaller training error smallest, test error largest this trade-off is known by many names in the generative classification world it is usually due to the bias-variance trade-off of the class models will look at this in detail 18

19 Class-modeling each class is characterized by a probability density function (class conditional density) a model is adopted, e.g. a Gaussian training data used to estimate model parameters overall the process is referred to as density estimation the simplest example would be to use histograms 19

20 Density estimation there are, however, much better models usually, problem has two components: selecting a model estimating model parameters models: we will cover the whole gamut from the exponential family (e.g. Gaussian) to kernel-based density estimates including mixture models and non-parametric approaches (nearest neighbors, histograms, etc.) 20

21 Parameter estimation two main camps: maximum likelihood Bayesian estimates ML will devote most attention to the quality of estimates the bias variance/trade-off a lot more emphasis on Bayes: subjective probability what is really a prior? mechanics: predictive distribution, MAP estimates, etc. priors: conjugate, non-informative, improper why is the exponential family special? 21

22 Decision rules given class models, Bayesian decision theory provides us with optimal rules for classification optimal here means minimum probability of error, for example we will study BDT in detail, establish connections to other decision principles (e.g. linear discriminants) show that Bayesian decisions are usually intuitive derive optimal rules for a range of classifiers 22

23 Reasons to take the course statistical learning tremendous amount of theory but things invariably go wrong too little data, noise, too many dimensions, training sets that do not reflect all possible variability, etc. good learning solutions require: knowledge of the domain (e.g. these are the features to use ) knowledge of the available techniques, their limitations, etc. (e.g. here a Gaussian is enough for AB&C, but there I need a mixture) in the absence of either of these you will fail! we will cover the basics, but will talk about quite advanced concepts. easier scenario in which to understand them 24

24 Reasons to take the course theory together with hands-on experience will cover all theory, every week 5-6 problems hands-on component: one computational problem per week this will center around cheetah segmentation allows evaluation of the benefits of more advance techniques as they are introduced forces you to deal with real, noisy, data exposes you to working on a new domain 25

25 26

26 Cheetah day last class, we will have Cheetah Day what: 5 teams each team will write a report on the 5 cheetah problems each team will give a presentation on one of the problems why: to make sure that we get the big picture out of all this work presenting is always good practice 27

27 Cheetah Day how much: 10% of the final grade (5% report, 5% presentation) what to talk about: report: comparative analysis of all solutions of the problem as if you were writing a conference paper presentation: will be on one single problem review what solution was what did this problem taught us about learning? what tricks did we learn solving it? how well did this solution do compared to others? will talk about this in due time 28

Pattern Classification and Clustering Spring 2006

Pattern Classification and Clustering Spring 2006 Pattern Classification and Clustering Time: Spring 2006 Room: Instructor: Yingen Xiong Office: 621 McBryde Office Hours: Phone: 231-4212 Email: yxiong@cs.vt.edu URL: http://www.cs.vt.edu/~yxiong/pcc/ Detailed

More information

Programming Social Robots for Human Interaction. Lecture 4: Machine Learning and Pattern Recognition

Programming Social Robots for Human Interaction. Lecture 4: Machine Learning and Pattern Recognition Programming Social Robots for Human Interaction Lecture 4: Machine Learning and Pattern Recognition Zheng-Hua Tan Dept. of Electronic Systems, Aalborg Univ., Denmark zt@es.aau.dk, http://kom.aau.dk/~zt

More information

CSCI 5521: Pattern Recognition. Prof. Paul Schrater

CSCI 5521: Pattern Recognition. Prof. Paul Schrater CSCI 5521: Pattern Recognition Prof. Paul Schrater Business Check to make sure you received the test email. If not, you are not officially registered. Course web page: http://gandalf.psych.umn.edu/~schrater/schrater_lab/courses/pattre

More information

36-350: Data Mining. Fall Lectures: Monday, Wednesday and Friday, 10:30 11:20, Porter Hall 226B

36-350: Data Mining. Fall Lectures: Monday, Wednesday and Friday, 10:30 11:20, Porter Hall 226B 36-350: Data Mining Fall 2009 Instructor: Cosma Shalizi, Statistics Dept., Baker Hall 229C, cshalizi@stat.cmu.edu Teaching Assistant: Joseph Richards, jwrichar@stat.cmu.edu Lectures: Monday, Wednesday

More information

Introduction to Machine Learning Reykjavík University Spring Instructor: Dan Lizotte

Introduction to Machine Learning Reykjavík University Spring Instructor: Dan Lizotte Introduction to Machine Learning Reykjavík University Spring 2007 Instructor: Dan Lizotte Logistics To contact Dan: dlizotte@cs.ualberta.ca http://www.cs.ualberta.ca/~dlizotte/teaching/ Books: Introduction

More information

CSC 411 MACHINE LEARNING and DATA MINING

CSC 411 MACHINE LEARNING and DATA MINING CSC 411 MACHINE LEARNING and DATA MINING Lectures: Monday, Wednesday 12-1 (section 1), 3-4 (section 2) Lecture Room: MP 134 (section 1); Bahen 1200 (section 2) Instructor (section 1): Richard Zemel Instructor

More information

CS534 Machine Learning

CS534 Machine Learning CS534 Machine Learning Spring 2013 Lecture 1: Introduction to ML Course logistics Reading: The discipline of Machine learning by Tom Mitchell Course Information Instructor: Dr. Xiaoli Fern Kec 3073, xfern@eecs.oregonstate.edu

More information

Introduction. Industrial AI Lab.

Introduction. Industrial AI Lab. Introduction Industrial AI Lab. 2018 - present: POSTECH Industrial AI Lab. Introduction 2013-2017: UNIST isystems Design Lab. 2010, Ph.D. from the University of Michigan, Ann Arbor S. M. Wu Manufacturing

More information

Introduction to Pattern Recognition

Introduction to Pattern Recognition Introduction to Pattern Recognition Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Fall 2017 CS 551, Fall 2017 c 2017, Selim Aksoy (Bilkent University)

More information

Machine Learning for Computer Vision

Machine Learning for Computer Vision Prof. Daniel Cremers Machine Learning for Computer PD Dr. Rudolph Triebel Lecturers PD Dr. Rudolph Triebel rudolph.triebel@in.tum.de Room number 02.09.059 (Fridays) Main lecture MSc. Ioannis John Chiotellis

More information

Statistical Learning- Classification STAT 441/ 841, CM 764

Statistical Learning- Classification STAT 441/ 841, CM 764 Statistical Learning- Classification STAT 441/ 841, CM 764 Ali Ghodsi Department of Statistics and Actuarial Science University of Waterloo aghodsib@uwaterloo.ca Two Paradigms Classical Statistics Infer

More information

Machine Learning for Computer Vision

Machine Learning for Computer Vision Prof. Daniel Cremers Machine Learning for Computer PD Dr. Rudolph Triebel Lecturers PD Dr. Rudolph Triebel rudolph.triebel@in.tum.de Room number 02.09.058 (Fridays) Main lecture MSc. Ioannis John Chiotellis

More information

10-702: Statistical Machine Learning

10-702: Statistical Machine Learning 10-702: Statistical Machine Learning Syllabus, Spring 2010 http://www.cs.cmu.edu/~10702 Statistical Machine Learning is a second graduate level course in machine learning, assuming students have taken

More information

10701: Intro to Machine Learning. Instructors: Pradeep Ravikumar, Manuela Veloso, Teaching Assistants:

10701: Intro to Machine Learning. Instructors: Pradeep Ravikumar, Manuela Veloso, Teaching Assistants: 10701: Intro to Machine Instructors: Pradeep Ravikumar, pradeepr@cs.cmu.edu Manuela Veloso, mmv@cs.cmu.edu Teaching Assistants: Shaojie Bai shaojieb@andrew.cmu.edu Adarsh Prasad adarshp@andrew.cmu.edu

More information

Overview COEN 296 Topics in Computer Engineering Introduction to Pattern Recognition and Data Mining Course Goals Syllabus

Overview COEN 296 Topics in Computer Engineering Introduction to Pattern Recognition and Data Mining Course Goals Syllabus Overview COEN 296 Topics in Computer Engineering to Pattern Recognition and Data Mining Instructor: Dr. Giovanni Seni G.Seni@ieee.org Department of Computer Engineering Santa Clara University Course Goals

More information

Machine Learning for Computer Vision

Machine Learning for Computer Vision Computer Group Prof. Daniel Cremers Machine Learning for Computer PD Dr. Rudolph Triebel Lecturers PD Dr. Rudolph Triebel rudolph.triebel@in.tum.de Room number 02.09.059 Main lecture MSc. Ioannis John

More information

Machine Learning and Applications in Finance

Machine Learning and Applications in Finance Machine Learning and Applications in Finance Christian Hesse 1,2,* 1 Autobahn Equity Europe, Global Markets Equity, Deutsche Bank AG, London, UK christian-a.hesse@db.com 2 Department of Computer Science,

More information

Statistical Parameter Estimation

Statistical Parameter Estimation Statistical Parameter Estimation ECE 275AB Syllabus AY 2017-2018 Ken Kreutz-Delgado ECE Department, UC San Diego Ken Kreutz-Delgado (UC San Diego) ECE 275AB Syllabus Version 1.1c Fall 2016 1 / 9 Contact

More information

M. R. Ahmadzadeh Isfahan University of Technology. M. R. Ahmadzadeh Isfahan University of Technology

M. R. Ahmadzadeh Isfahan University of Technology. M. R. Ahmadzadeh Isfahan University of Technology 1 2 M. R. Ahmadzadeh Isfahan University of Technology Ahmadzadeh@cc.iut.ac.ir M. R. Ahmadzadeh Isfahan University of Technology Textbooks 3 Introduction to Machine Learning - Ethem Alpaydin Pattern Recognition

More information

Lecture 1: Machine Learning Basics

Lecture 1: Machine Learning Basics 1/69 Lecture 1: Machine Learning Basics Ali Harakeh University of Waterloo WAVE Lab ali.harakeh@uwaterloo.ca May 1, 2017 2/69 Overview 1 Learning Algorithms 2 Capacity, Overfitting, and Underfitting 3

More information

Session 1: Gesture Recognition & Machine Learning Fundamentals

Session 1: Gesture Recognition & Machine Learning Fundamentals IAP Gesture Recognition Workshop Session 1: Gesture Recognition & Machine Learning Fundamentals Nicholas Gillian Responsive Environments, MIT Media Lab Tuesday 8th January, 2013 My Research My Research

More information

L1: Course introduction

L1: Course introduction Introduction Course organization Grading policy Outline What is pattern recognition? Definitions from the literature Related fields and applications L1: Course introduction Components of a pattern recognition

More information

An Introduction to Machine Learning

An Introduction to Machine Learning MindLAB Research Group - Universidad Nacional de Colombia Introducción a los Sistemas Inteligentes Outline 1 2 What s machine learning History Supervised learning Non-supervised learning 3 Observation

More information

W4240 Data Mining. Frank Wood. September 6, 2010

W4240 Data Mining. Frank Wood. September 6, 2010 W4240 Data Mining Frank Wood September 6, 2010 Introduction Data mining is the search for patterns in large collections of data Learning models Applying models to large quantities of data Pattern recognition

More information

Unsupervised Learning: Clustering

Unsupervised Learning: Clustering Unsupervised Learning: Clustering Vibhav Gogate The University of Texas at Dallas Slides adapted from Carlos Guestrin, Dan Klein & Luke Zettlemoyer Machine Learning Supervised Learning Unsupervised Learning

More information

INTRODUCTION TO MACHINE LEARNING SOME CONTENT COURTESY OF PROFESSOR ANDREW NG OF STANFORD UNIVERSITY

INTRODUCTION TO MACHINE LEARNING SOME CONTENT COURTESY OF PROFESSOR ANDREW NG OF STANFORD UNIVERSITY INTRODUCTION TO MACHINE LEARNING SOME CONTENT COURTESY OF PROFESSOR ANDREW NG OF STANFORD UNIVERSITY IQS2: Spring 2013 Machine Learning Definition 2 Arthur Samuel (1959). Machine Learning: Field of study

More information

A Review on Classification Techniques in Machine Learning

A Review on Classification Techniques in Machine Learning A Review on Classification Techniques in Machine Learning R. Vijaya Kumar Reddy 1, Dr. U. Ravi Babu 2 1 Research Scholar, Dept. of. CSE, Acharya Nagarjuna University, Guntur, (India) 2 Principal, DRK College

More information

Government of Russian Federation. Federal State Autonomous Educational Institution of High Professional Education

Government of Russian Federation. Federal State Autonomous Educational Institution of High Professional Education Government of Russian Federation Federal State Autonomous Educational Institution of High Professional Education National Research University Higher School of Economics Syllabus for the course Advanced

More information

CS545 Machine Learning

CS545 Machine Learning Machine learning and related fields CS545 Machine Learning Course Introduction Machine learning: the construction and study of systems that learn from data. Pattern recognition: the same field, different

More information

STA 414/2104 Statistical Methods for Machine Learning and Data Mining

STA 414/2104 Statistical Methods for Machine Learning and Data Mining STA 414/2104 Statistical Methods for Machine Learning and Data Mining Radford M. Neal, University of Toronto, 2014 Week 1 What are Machine Learning and Data Mining? Typical Machine Learning and Data Mining

More information

Course Guide Year GENERAL INFORMATION Course information Name. Machine Learning Code

Course Guide Year GENERAL INFORMATION Course information Name. Machine Learning Code Course Guide Year 2017-2018 ESCUELA TÉCNICA SUPERIOR DE INGENIERÍA GENERAL INFORMATION Course information Name Machine Learning Code DOI-MIC-515 Degree MIC, MII, MIT Year Semester Spring ECTS credits 6

More information

A Brief Introduction to Generative Models

A Brief Introduction to Generative Models Theoretical Neuroscience and Computer Vision A Brief Introduction to Generative Models FIAS, Goethe-Universität Frankfurt, Germany FIAS Summer School Frankfurt, August 2008 Contents Introduction Please

More information

Machine Learning L, T, P, J, C 2,0,2,4,4

Machine Learning L, T, P, J, C 2,0,2,4,4 Subject Code: Objective Expected Outcomes Machine Learning L, T, P, J, C 2,0,2,4,4 It introduces theoretical foundations, algorithms, methodologies, and applications of Machine Learning and also provide

More information

Machine Learning Lecture 1: Introduction

Machine Learning Lecture 1: Introduction Welcome to CSCE 478/878! Please check off your name on the roster, or write your name if you're not listed Indicate if you wish to register or sit in Policy on sit-ins: You may sit in on the course without

More information

CPSC 340: Machine Learning and Data Mining. Course Review/Preview Fall 2015

CPSC 340: Machine Learning and Data Mining. Course Review/Preview Fall 2015 CPSC 340: Machine Learning and Data Mining Course Review/Preview Fall 2015 Admin Assignment 6 due now. We will have office hours as usual next week. Final exam details: December 15: 8:30-11 (WESB 100).

More information

COMS 4771 Introduction to Machine Learning. Nakul Verma

COMS 4771 Introduction to Machine Learning. Nakul Verma COMS 4771 Introduction to Machine Learning Nakul Verma Machine learning: what? Study of making machines learn a concept without having to explicitly program it. Constructing algorithms that can: learn

More information

Lecture 1: Introduc4on

Lecture 1: Introduc4on CSC2515 Spring 2014 Introduc4on to Machine Learning Lecture 1: Introduc4on All lecture slides will be available as.pdf on the course website: http://www.cs.toronto.edu/~urtasun/courses/csc2515/csc2515_winter15.html

More information

COLLEGE OF SCIENCE. School of Mathematical Sciences. NEW (or REVISED) COURSE: COS-STAT-747 Principles of Statistical Data Mining.

COLLEGE OF SCIENCE. School of Mathematical Sciences. NEW (or REVISED) COURSE: COS-STAT-747 Principles of Statistical Data Mining. ROCHESTER INSTITUTE OF TECHNOLOGY COURSE OUTLINE FORM COLLEGE OF SCIENCE School of Mathematical Sciences NEW (or REVISED) COURSE: COS-STAT-747 Principles of Statistical Data Mining 1.0 Course Designations

More information

Theodoridis, S. and K. Koutroumbas, Pattern recognition. 4th ed. 2009, San Diego, CA: Academic Press.

Theodoridis, S. and K. Koutroumbas, Pattern recognition. 4th ed. 2009, San Diego, CA: Academic Press. Pattern Recognition Winter 2013 Andrew Cohen acohen@coe.drexel.edu What is this course about? This course will study state-of-the-art techniques for analyzing data. The goal is to extract meaningful information

More information

ECE Pattern Recognition Syllabus Fall 2014

ECE Pattern Recognition Syllabus Fall 2014 ECE 5258 - Pattern Recognition Syllabus Fall 2014 Dr. Georgios C. Anagnostopoulos August 11, 2014 (ver. 1.0) 1. Contents 2 Course Description 2 2.1 Objectives & Outcomes...................................

More information

CSE 546 Machine Learning

CSE 546 Machine Learning CSE 546 Machine Learning Instructor: Luke Zettlemoyer TA: Lydia Chilton Slides adapted from Pedro Domingos and Carlos Guestrin Logistics Instructor: Luke Zettlemoyer Email: lsz@cs Office: CSE 658 Office

More information

Machine Learning: Summary

Machine Learning: Summary Machine Learning: Summary Greg Grudic CSCI-4830 Machine Learning 1 What is Machine Learning? The goal of machine learning is to build computer systems that can adapt and learn from their experience. Tom

More information

Service courses for graduate students in degree programs other than the MS or PhD programs in Biostatistics.

Service courses for graduate students in degree programs other than the MS or PhD programs in Biostatistics. Course Catalog In order to be assured that all prerequisites are met, students must acquire a permission number from the education coordinator prior to enrolling in any Biostatistics course. Courses are

More information

Master of Science in ECE - Machine Learning & Data Science Focus

Master of Science in ECE - Machine Learning & Data Science Focus Master of Science in ECE - Machine Learning & Data Science Focus Core Coursework (16 units) ECE269: Linear Algebra ECE271A: Statistical Learning I ECE 225A: Probability and Statistics for Data Science

More information

Machine Learning with MATLAB Antti Löytynoja Application Engineer

Machine Learning with MATLAB Antti Löytynoja Application Engineer Machine Learning with MATLAB Antti Löytynoja Application Engineer 2014 The MathWorks, Inc. 1 Goals Overview of machine learning Machine learning models & techniques available in MATLAB MATLAB as an interactive

More information

CS340 Machine learning Lecture 1 Introduction

CS340 Machine learning Lecture 1 Introduction CS340 Machine learning Lecture 1 Introduction Administrivia Class web page (check regularly!): www.cs.ubc.ca/~murphyk/teaching/cs340-fall07 TAs: Hoyt Koepke Erik Zawadzki hoytak@cs.ubc.ca epz@cs.ubc.ca

More information

COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection.

COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection. COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection. Instructor: Herke van Hoof (herke.vanhoof@mail.mcgill.ca) Slides mostly by: Class web page: www.cs.mcgill.ca/~hvanho2/comp551

More information

INTRODUCTION TO PATTERN RECOGNITION SYSTEM 1.1 Overview

INTRODUCTION TO PATTERN RECOGNITION SYSTEM 1.1 Overview CHAPTER 1 INTRODUCTION TO PATTERN RECOGNITION SYSTEM 1.1 Overview One of the most important capabilities of mankind is learning by experience, by our endeavors, by our faults. By the time we attain an

More information

Machine Learning Overview. Lars Schmidt-Thieme

Machine Learning Overview. Lars Schmidt-Thieme Machine Learning 2 0. Overview Lars Schmidt-Thieme Information Systems and Machine Learning Lab (ISMLL) Institute for Computer Science University of Hildesheim, Germany 1 / 6 Outline 1. Lecture Overview

More information

Statistics and Machine Learning, Master s Programme

Statistics and Machine Learning, Master s Programme DNR LIU-2017-02005 1(9) Statistics and Machine Learning, Master s Programme 120 credits Statistics and Machine Learning, Master s Programme F7MSL Valid from: 2018 Autumn semester Determined by Board of

More information

Linear Models Continued: Perceptron & Logistic Regression

Linear Models Continued: Perceptron & Logistic Regression Linear Models Continued: Perceptron & Logistic Regression CMSC 723 / LING 723 / INST 725 Marine Carpuat Slides credit: Graham Neubig, Jacob Eisenstein Linear Models for Classification Feature function

More information

Unsupervised Learning

Unsupervised Learning 09s1: COMP9417 Machine Learning and Data Mining Unsupervised Learning June 3, 2009 Acknowledgement: Material derived from slides for the book Machine Learning, Tom M. Mitchell, McGraw-Hill, 1997 http://www-2.cs.cmu.edu/~tom/mlbook.html

More information

Department of Biostatistics

Department of Biostatistics The University of Kansas 1 Department of Biostatistics The mission of the Department of Biostatistics is to provide an infrastructure of biostatistical and informatics expertise to support and enhance

More information

ECE 5424: Introduction to Machine Learning

ECE 5424: Introduction to Machine Learning ECE 5424: Introduction to Machine Learning Topics: Classification: Naïve Bayes Readings: Barber 10.1-10.3 Stefan Lee Virginia Tech Administrativia HW2 Due: Friday 09/28, 10/3, 11:55pm Implement linear

More information

Introduction to Foundations of Graphical Models

Introduction to Foundations of Graphical Models Introduction to Foundations of Graphical Models David M. Blei Columbia University September 2, 2015 Probabilistic modeling is a mainstay of modern machine learning and statistics research, providing essential

More information

Disclaimer. Copyright. Machine Learning Mastery With Weka

Disclaimer. Copyright. Machine Learning Mastery With Weka i Disclaimer The information contained within this ebook is strictly for educational purposes. If you wish to apply ideas contained in this ebook, you are taking full responsibility for your actions. The

More information

Evaluation and Comparison of Performance of different Classifiers

Evaluation and Comparison of Performance of different Classifiers Evaluation and Comparison of Performance of different Classifiers Bhavana Kumari 1, Vishal Shrivastava 2 ACE&IT, Jaipur Abstract:- Many companies like insurance, credit card, bank, retail industry require

More information

Machine Learning. Introduction. Hamid Beigy. Sharif University of Technology. Fall 1395

Machine Learning. Introduction. Hamid Beigy. Sharif University of Technology. Fall 1395 Machine Learning Introduction Hamid Beigy Sharif University of Technology Fall 1395 Hamid Beigy (Sharif University of Technology) Machine Learning Fall 1395 1 / 15 Table of contents 1 What is machine learning?

More information

learn from the accelerometer data? A close look into privacy Member: Devu Manikantan Shila

learn from the accelerometer data? A close look into privacy Member: Devu Manikantan Shila What can we learn from the accelerometer data? A close look into privacy Team Member: Devu Manikantan Shila Abstract: A handful of research efforts nowadays focus on gathering and analyzing the data from

More information

COMP 551 Applied Machine Learning Lecture 11: Ensemble learning

COMP 551 Applied Machine Learning Lecture 11: Ensemble learning COMP 551 Applied Machine Learning Lecture 11: Ensemble learning Instructor: Herke van Hoof (herke.vanhoof@mcgill.ca) Slides mostly by: (jpineau@cs.mcgill.ca) Class web page: www.cs.mcgill.ca/~hvanho2/comp551

More information

COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection.

COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection. COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection. Instructor: (jpineau@cs.mcgill.ca) Class web page: www.cs.mcgill.ca/~jpineau/comp551 Unless otherwise

More information

Appliance-specific power usage classification and disaggregation

Appliance-specific power usage classification and disaggregation Appliance-specific power usage classification and disaggregation Srinikaeth Thirugnana Sambandam, Jason Hu, EJ Baik Department of Energy Resources Engineering Department, Stanford Univesrity 367 Panama

More information

CSC 411: Lecture 01: Introduction

CSC 411: Lecture 01: Introduction CSC 411: Lecture 01: Introduction Richard Zemel, Raquel Urtasun and Sanja Fidler University of Toronto Zemel, Urtasun, Fidler (UofT) CSC 411: 01-Introduction 1 / 44 Today Administration details Why is

More information

Deep (Structured) Learning

Deep (Structured) Learning Deep (Structured) Learning Yasmine Badr 06/23/2015 NanoCAD Lab UCLA What is Deep Learning? [1] A wide class of machine learning techniques and architectures Using many layers of non-linear information

More information

E9 205 Machine Learning for Signal Processing

E9 205 Machine Learning for Signal Processing E9 205 Machine Learning for Signal Processing Introduction to Machine Learning of Sensory Signals 14-08-2017 Instructor - Sriram Ganapathy (sriram@ee.iisc.ernet.in) Teaching Assistant - Aravind Illa (aravindece77@gmail.com).

More information

Hierarchical Bayesian Methods for Reinforcement Learning

Hierarchical Bayesian Methods for Reinforcement Learning Hierarchical Bayesian Methods for Reinforcement Learning David Wingate wingated@mit.edu Joint work with Noah Goodman, Dan Roy, Leslie Kaelbling and Joshua Tenenbaum My Research: Agents Rich sensory data

More information

Introduction to Machine Learning

Introduction to Machine Learning Introduction to Machine Learning D. De Cao R. Basili Corso di Web Mining e Retrieval a.a. 2008-9 April 7, 2009 Outline Outline Introduction to Machine Learning Decision Tree Naive Bayes K-nearest neighbor

More information

Introduction to Machine Learning

Introduction to Machine Learning Introduction to Machine Learning D. De Cao R. Basili Corso di Web Mining e Retrieval a.a. 2008-9 April 6, 2009 Outline Outline Introduction to Machine Learning Outline Outline Introduction to Machine Learning

More information

CS 510: Lecture 8. Deep Learning, Fairness, and Bias

CS 510: Lecture 8. Deep Learning, Fairness, and Bias CS 510: Lecture 8 Deep Learning, Fairness, and Bias Next Week All Presentations, all the time Upload your presentation before class if using slides Sign up for a timeslot google doc, if you haven t already

More information

ECE521 Lecture1. Introduction

ECE521 Lecture1. Introduction ECE521 Lecture1 Introduction Outline History of machine learning Types of machine learning problems What is machine learning? A scientific field is best defined by the central question it studies. The

More information

CS540 Machine learning Lecture 1 Introduction

CS540 Machine learning Lecture 1 Introduction CS540 Machine learning Lecture 1 Introduction Administrivia Overview Supervised learning Unsupervised learning Other kinds of learning Outline Administrivia Class web page www.cs.ubc.ca/~murphyk/teaching/cs540-fall08

More information

(Sub)Gradient Descent

(Sub)Gradient Descent (Sub)Gradient Descent CMSC 422 MARINE CARPUAT marine@cs.umd.edu Figures credit: Piyush Rai Logistics Midterm is on Thursday 3/24 during class time closed book/internet/etc, one page of notes. will include

More information

Big Data Analytics Clustering and Classification

Big Data Analytics Clustering and Classification E6893 Big Data Analytics Lecture 4: Big Data Analytics Clustering and Classification Ching-Yung Lin, Ph.D. Adjunct Professor, Dept. of Electrical Engineering and Computer Science September 28th, 2017 1

More information

- Introduzione al Corso - (a.a )

- Introduzione al Corso - (a.a ) Short Course on Machine Learning for Web Mining - Introduzione al Corso - (a.a. 2009-2010) Roberto Basili (University of Roma, Tor Vergata) 1 Overview MLxWM: Motivations and perspectives A temptative syllabus

More information

Lecture 1. Introduction Bastian Leibe Visual Computing Institute RWTH Aachen University

Lecture 1. Introduction Bastian Leibe Visual Computing Institute RWTH Aachen University Advanced Machine Learning Lecture 1 Introduction 20.10.2015 Bastian Leibe Visual Computing Institute RWTH Aachen University http://www.vision.rwth-aachen.de/ leibe@vision.rwth-aachen.de Organization Lecturer

More information

Lecture 16 Speaker Recognition

Lecture 16 Speaker Recognition Lecture 16 Speaker Recognition Information College, Shandong University @ Weihai Definition Method of recognizing a Person form his/her voice. Depends on Speaker Specific Characteristics To determine whether

More information

Introduction to Machine Learning NPFL 054

Introduction to Machine Learning NPFL 054 Introduction to Machine Learning NPFL 054 http://ufal.mff.cuni.cz/course/npfl054 Barbora Hladká hladka@ufal.mff.cuni.cz Martin Holub holub@ufal.mff.cuni.cz Charles University, Faculty of Mathematics and

More information

A PROBABILISTIC MODEL FOR SPELLING CORRECTION. Lucian SASU 1

A PROBABILISTIC MODEL FOR SPELLING CORRECTION. Lucian SASU 1 Bulletin of the Transilvania University of Braşov Vol 4(53), No. 2-2011 Series III: Mathematics, Informatics, Physics, 141-146 A PROBABILISTIC MODEL FOR SPELLING CORRECTION Lucian SASU 1 Abstract Spelling

More information

Lecture 7: Distributed Representations

Lecture 7: Distributed Representations Lecture 7: Distributed Representations Roger Grosse 1 Introduction We ll take a break from derivatives and optimization, and look at a particular example of a neural net that we can train using backprop:

More information

Bioinformatics II Theoretical Bioinformatics and Machine Learning Part 1. Sepp Hochreiter

Bioinformatics II Theoretical Bioinformatics and Machine Learning Part 1. Sepp Hochreiter Bioinformatics II Theoretical Bioinformatics and Machine Learning Part 1 Institute of Bioinformatics Johannes Kepler University, Linz, Austria Course 6 ECTS 4 SWS VO (class) 3 ECTS 2 SWS UE (exercise)

More information

10701/15781 Machine Learning, Spring 2005: Homework 1

10701/15781 Machine Learning, Spring 2005: Homework 1 10701/15781 Machine Learning, Spring 2005: Homework 1 Due: Monday, February 6, beginning of the class 1 [15 Points] Probability and Regression [Stano] 1 1.1 [10 Points] The Matrix Strikes Back The Matrix

More information

Introduction to Machine Learning

Introduction to Machine Learning Introduction to Machine Learning Hamed Pirsiavash CMSC 678 http://www.csee.umbc.edu/~hpirsiav/courses/ml_fall17 The slides are closely adapted from Subhransu Maji s slides Course background What is the

More information

MACHINE LEARNING AND PATTERN RECOGNITION Spring 2004, Lecture 1: Introduction and Basic Concepts Yann LeCun

MACHINE LEARNING AND PATTERN RECOGNITION Spring 2004, Lecture 1: Introduction and Basic Concepts Yann LeCun Y. LeCun: Machine Learning and Pattern Recognition p. 1/3 MACHINE LEARNING AND PATTERN RECOGNITION Spring 2004, Lecture 1: Introduction and Basic Concepts Yann LeCun The Courant Institute, New York University

More information

CSE 258 Lecture 3. Web Mining and Recommender Systems. Supervised learning Classification

CSE 258 Lecture 3. Web Mining and Recommender Systems. Supervised learning Classification CSE 258 Lecture 3 Web Mining and Recommender Systems Supervised learning Classification Last week Last week we started looking at supervised learning problems Last week We studied linear regression, in

More information

Welcome to CMPS 142 and 242: Machine Learning

Welcome to CMPS 142 and 242: Machine Learning Welcome to CMPS 142 and 242: Machine Learning Instructor: David Helmbold, dph@soe.ucsc.edu Office hours: Monday 1:30-2:30, Thursday 4:15-5:00 TA: Aaron Michelony, amichelo@soe.ucsc.edu Web page: www.soe.ucsc.edu/classes/cmps242/fall13/01

More information

36-217: Probability Theory and Random Processes Fall 1997 MWF 3:30 4:20 DH 2210 Course Policies and Syllabus

36-217: Probability Theory and Random Processes Fall 1997 MWF 3:30 4:20 DH 2210 Course Policies and Syllabus Vital Information 36-217: Probability Theory and Random Processes Fall 1997 MWF 3:30 4:20 DH 2210 Course Policies and Syllabus Instructor: Pantelis Vlachos, Statistics 232K Baker Hall 268-1883 vlachos@stat.cmu.edu

More information

COMP150 DR Final Project Proposal

COMP150 DR Final Project Proposal COMP150 DR Final Project Proposal Ari Brown and Julie Jiang October 26, 2017 Abstract The problem of sound classification has been studied in depth and has multiple applications related to identity discrimination,

More information

Data Analysis for Business and Industry

Data Analysis for Business and Industry Coordinating unit: Teaching unit: Academic year: Degree: ECTS credits: 2017 240 - ETSEIB - Barcelona School of Industrial Engineering 715 - EIO - Department of Statistics and Operations Research BACHELOR'S

More information

CS4780/ Machine Learning

CS4780/ Machine Learning CS4780/5780 - Machine Learning Fall 2012 Thorsten Joachims Cornell University Department of Computer Science Outline of Today Who we are? Prof: Thorsten Joachims TAs: Joshua Moore, Igor Labutov, Moontae

More information

L16: Speaker recognition

L16: Speaker recognition L16: Speaker recognition Introduction Measurement of speaker characteristics Construction of speaker models Decision and performance Applications [This lecture is based on Rosenberg et al., 2008, in Benesty

More information

ELEC9723 Speech Processing

ELEC9723 Speech Processing ELEC9723 Speech Processing COURSE INTRODUCTION Session 1, 2013 s Course Staff Course conveners: Dr. Vidhyasaharan Sethu, v.sethu@unsw.edu.au (EE304) Laboratory demonstrator: Nicholas Cummins, n.p.cummins@unsw.edu.au

More information

HAMLET JERRY ZHU UNIVERSITY OF WISCONSIN

HAMLET JERRY ZHU UNIVERSITY OF WISCONSIN HAMLET JERRY ZHU UNIVERSITY OF WISCONSIN Collaborators: Rui Castro, Michael Coen, Ricki Colman, Charles Kalish, Joseph Kemnitz, Robert Nowak, Ruichen Qian, Shelley Prudom, Timothy Rogers Somewhere, something

More information

BGS Training Requirement in Statistics

BGS Training Requirement in Statistics BGS Training Requirement in Statistics All BGS students are required to have an understanding of statistical methods and their application to biomedical research. Most students take BIOM611, Statistical

More information

EasyDSP: Problem-Based Learning in Digital Signal Processing

EasyDSP: Problem-Based Learning in Digital Signal Processing EasyDSP: Problem-Based Learning in Digital Signal Processing Kaveh Malakuti and Alexandra Branzan Albu Department of Electrical and Computer Engineering University of Victoria (BC) Canada malakuti@ece.uvic.ca,

More information

Sanjoy Dasgupta Professor, Computer Science and Engineering Faculty-Affiliate, Calit2

Sanjoy Dasgupta Professor, Computer Science and Engineering Faculty-Affiliate, Calit2 Sanjoy Dasgupta Professor, Computer Science and Engineering Faculty-Affiliate, Calit2 Prior to joining the UCSD Jacobs School in 2002, Sanjoy Dasgupta was a senior member of the technical staff at AT&T

More information

CS 6140: Machine Learning Spring 2017

CS 6140: Machine Learning Spring 2017 CS 6140: Machine Learning Spring 2017 Instructor: Lu Wang College of Computer and Informa@on Science Northeastern University Webpage: www.ccs.neu.edu/home/luwang Email: luwang@ccs.neu.edu Time and Loca@on

More information

Lecture 0: Machine Learning

Lecture 0: Machine Learning Lecture 0: Machine Learning Tuo Zhao Schools of ISYE and CSE, Georgia Tech 2017 Fall Questions Course Logistics Why Machine Learning? What is a well-defined learning problem? What questions should we ask

More information

Machine Learning. Introduction. Marc Toussaint. Duy Nguyen-Tuong. Summer University of Stuttgart. Bosch Center for Artificial Intelligence

Machine Learning. Introduction. Marc Toussaint. Duy Nguyen-Tuong. Summer University of Stuttgart. Bosch Center for Artificial Intelligence Machine Learning Introduction Marc Toussaint University of Stuttgart Duy Nguyen-Tuong Bosch Center for Artificial Intelligence Summer 2017 What is Machine Learning? 1) A long list of methods/algorithms

More information

Machine Learning. Professor Sridhar Mahadevan

Machine Learning. Professor Sridhar Mahadevan Machine Learning Professor Sridhar Mahadevan mahadeva@cs.umass.edu Lecture 1 Home page:www-edlab.cs.umass.edu/cs689 Quizzes, mini-projects: moodle.umass.edu Discussion forum:piazza.com CMPSCI 689 p. 1/35

More information