Programming Social Robots for Human Interaction. Lecture 4: Machine Learning and Pattern Recognition

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Programming Social Robots for Human Interaction. Lecture 4: Machine Learning and Pattern Recognition"

Transcription

1 Programming Social Robots for Human Interaction Lecture 4: Machine Learning and Pattern Recognition Zheng-Hua Tan Dept. of Electronic Systems, Aalborg Univ., Denmark Many of the figures are provided by Chris Bishop. Programming Social Robots 4, Zheng-Hua Tan 1 Course outline 1. Introduction to Robot Operating System (ROS) 2. Introduction to isociobot and NAO robot, and demos 3. Social Robots and Applications 4. Machine Learning and Pattern Recognition 5. Speech Processing I: Acquisition of Speech, Feature Extraction and Speaker Localization 6. Speech Processing II: Speaker Identification and Speech Recognition 7. Image Processing I: Image Acquisition, Pre-processing and Feature Extraction 8. Image Processing II: Face Detection and Face Recognition 9. User Modelling 10. Multimodal Human-Robot Interaction Programming Social Robots 4, Zheng-Hua Tan 2

2 Classification examples Handwritten Digit Recognition It s not easy to recognize speech. It s not easy to wreck a nice beach. Programming Social Robots 4, Zheng-Hua Tan 3 Regression example Polynomial Curve Fitting from Bishop Programming Social Robots 4, Zheng-Hua Tan 4

3 Density estimation examples from Bishop Programming Social Robots 4, Zheng-Hua Tan 5 General information References Pattern Recognition and Machine Learning by Bishop Introduction to Machine Learning by Alpaydin For more resources, refer to Machine_learning_resources.htm Programming Social Robots 4, Zheng-Hua Tan 6

4 Lecture outline Introduction Machine learning Concepts, supervised learning, unsupervised learning Memory-based learning Model-based learning Programming Social Robots 4, Zheng-Hua Tan 7 What is machine learning? Machine: computing device. Learning is acquiring and improving performance through experience. is the acquisition and development of memories and behaviors, including skills, knowledge, understanding, values, and wisdom. Machine learning is programming computers to optimize a performance criterion using example data or past experience. is concerned with the design and development of algorithms and techniques that allow computers to learn from examples or experiences. Programming Social Robots 4, Zheng-Hua Tan 8

5 Machine learning s WHs When: We want computers to learn when it is too difficult or too expensive to program them directly to perform a task (e.g., spam filtering). human expertise does not exist (e.g., navigating on Mars), humans are unable to explain their expertise (e.g., speech recognition) Solution changes in time (routing on a computer network) What: Get the computer to learn density, discriminant or regression functions by showing examples of inputs (and outputs). How: We write a parameterized program, and let the learning algorithm find the set of parameters that best approximates the desired function or behavior. Programming Social Robots 4, Zheng-Hua Tan 9 Why study machine learning? Build intelligent computer systems that acquire or improve knowledge from examples adapt to users, customize and be context-aware discover patterns in large databases (data mining) Timing is good ubiquitous computing: computers are cheap, powerful and everywhere progresses in algorithm and theory development abundant data Study is needed develop new algorithms, and understand which algorithms should be applied in which circumstances, primarily aiming at good generalization performance on unseen test data Programming Social Robots 4, Zheng-Hua Tan 10

6 Related subjects and applications Statistics: statistical estimation targets the same problem as machine learning and most learning algorithms are statistical in nature. Pattern Recognition is when the output of the learning machine is a set of discrete categories. Data Mining is when machine learning is applied to a large database. Applications: speech recognition, handwriting recognition, bio-informatics, adaptive control, natural language processing, web search and text classification, fraud detection, time-series prediction, etc. Programming Social Robots 4, Zheng-Hua Tan 11 Types of machine learning Supervised learning: given inputs along with corresponding outputs, find the correct outputs for test inputs Classification: 1-of-N discrete output (pattern recognition) Regression: real-valued output (prediction) Unsupervised learning: given only inputs without outputs as training, find structure in the space density estimation clustering dimensionality reduction Reinforcement learning: given inputs from the environment, take actions that affect the environment and produce action sequences that maximize the expected scalar reward or punishment. This is similar to animal learning. Programming Social Robots 4, Zheng-Hua Tan 12

7 Supervised learning {input, output} Classification: assign each input to one of a finite number of discrete categories. Learn a decision boundary that separates one class from the other. Two separate stages: Inference stage: use a training data to learn a model for p(c k x), being either probabilistic generative or discriminative model. Decision stage: use these posterior probabilities to make optimal class assignments. Alternatively, we can solve both problems together and simply learn a discriminant function that maps inputs x directly into decisions. Regression: the desired output consists of one or more continuous variables. Learn a continuous input-output mapping from a limited number of examples. Regression is also known as curve fitting or function approximation. Programming Social Robots 4, Zheng-Hua Tan 13 Supervised learning {input, output} How to represent the inputs and outputs How to select a both powerful and searchable hypothesis space to represent the relationship between inputs and outputs Programming Social Robots 4, Zheng-Hua Tan 14

8 Unsupervised learning {input} Discover the unknown structure of the inputs. density estimation: determine the probability density distribution of data within the input space, e.g. k-nn, histogram, kernel. clustering: discover groups of similar examples (clumps) within the data, e.g., k-means, EM. Dimensionality reduction: project the data from a highdimensional space down to low dimensions. Compression/quantization: discover a function that for each input computes a compact code from which the input can be reconstructed (clustering). Association E.g., in retail, from customer transactions to consumer behavior: people who bought A also bought B. Programming Social Robots 4, Zheng-Hua Tan 15 Introduction Machine learning Memory-based learning Model-based learning Lecture outline Programming Social Robots 4, Zheng-Hua Tan 16

9 Learning is more than memorization Constructing a lookup table is easy Simply store all the inputs and their corresponding outputs in the training data. For a new input, compare it to all the samples and produce the output associated with the matching prototype. Problem In general, new inputs are different from training prototypes. The key of learning is generalization: the ability to produce correct outputs or behavior on previously unseen inputs. Programming Social Robots 4, Zheng-Hua Tan 17 Memory based learning: a simple trick Compute the distances between the input and all the stored prototypes, instead of identity requirement. 1-nearest neighbor search: choose the class of the nearest prototype. K-nearest neighbor search: choose the class that has the majority among the K nearest prototypes. so called lazy learning, memory based learning, or instancebased learning ; similar to case based reasoning. Challenges What is the right similarity measure? High computational intensity for large number of prototypes The curse of dimensionality and data sparsity. Programming Social Robots 4, Zheng-Hua Tan 18

10 Lecture outline Introduction Machine learning Memory-based learning Model-based learning Over-fitting, bias-variance trade-off Programming Social Robots 4, Zheng-Hua Tan 19 Model based learning Build a model that is a good, useful approximation to the data, or construct a general, explicit description of the target function linear vs nonlinear parametric vs nonparametric Discard learning examples when they are processed efficient computationally efficient in memory use. It is limited by the used learning bias - a coarse approximation of the target function. Programming Social Robots 4, Zheng-Hua Tan 20

11 Linear classifier - two classes g T ( x) = w x + w0 ( x) C1 if g > 0 choose C2 otherwise from Alpaydin Programming Social Robots 4, Zheng-Hua Tan 21 Regression - polynomial curve fitting, again! from Bishop Programming Social Robots 4, Zheng-Hua Tan 22

12 Sum-of-squares error function from Bishop Programming Social Robots 4, Zheng-Hua Tan 23 Polynomials model selection excessively tuned to the random noise! Programming Social Robots 4, Zheng-Hua Tan 24

13 Over-fitting from Bishop The need for a separate validation (or hold-out) set for model selection. Root-Mean-Square (RMS) Error: Programming Social Robots 4, Zheng-Hua Tan 25 Polynomial coefficients various order Programming Social Robots 4, Zheng-Hua Tan 26

14 Regularization Penalize large coefficient values from Bishop Programming Social Robots 4, Zheng-Hua Tan 27 Regularization: vs. from Bishop Programming Social Robots 4, Zheng-Hua Tan 28

15 Polynomial coefficients various λ 9 th Order Polynomial Programming Social Robots 4, Zheng-Hua Tan 29 Data set size 9 th Order Polynomial N = 10 For a given model complexity, the over-fitting problem become less severe as the size of the data set increases. Programming Social Robots 4, Zheng-Hua Tan 30

16 Number of data sets 100 data sets; training multiple polynomials and then averaging them, the contribution from the variance term tended to cancel, leading to improved predictions. The dependence of bias and variance on model complexity variance bias Programming Social Robots 4, Zheng-Hua Tan 31 Bias-variance trade-off There is a trade-off between bias and variance, with very flexible models having low bias and high variance, and relatively rigid models having high bias and low variance. Mean square error of the estimator d for unknown parameter θ : r (d,θ) = E [(d θ) 2 ] = (E [d] θ) 2 + E [(d E [d]) 2 ] = Bias 2 + Variance Programming Social Robots 4, Zheng-Hua Tan 32

17 Beating the bias-variance trade-off We can reduce the variance by averaging lots of models trained on different datasets. This seems silly. If we had lots of different datasets it would be better to combine them into one big training set. (With more training data there will be much less variance.) Weird idea: We can create different datasets by bootstrap sampling of our single training dataset. This is called bagging and it works surprisingly well. But if we have enough computation its better to do the right Bayesian thing: Combine the predictions of many models using the posterior probability of each parameter vector as the combination weight. Programming Social Robots 4, Zheng-Hua Tan 33 Over-fitting a still unsolved problem! from Bishop The least squares approach to finding the model parameters resorts to intuition and represents a specific case of maximum likelihood, and that the over-fitting problem can be understood as a general property of ML. More principled approach - probability theory, foundation for machine learning. Programming Social Robots 4, Zheng-Hua Tan 34

18 Lecture outline Introduction Machine learning Concepts, supervised learning, unsupervised learning Memory-based learning Model-based learning Over-fitting, bias-variance trade-off Programming Social Robots 4, Zheng-Hua Tan 35 Course outline 1. Introduction to Robot Operating System (ROS) 2. Introduction to isociobot and NAO robot, and demos 3. Social Robots and Applications 4. Machine Learning and Pattern Recognition 5. Speech Processing I: Acquisition of Speech, Feature Extraction and Speaker Localization 6. Speech Processing II: Speaker Identification and Speech Recognition 7. Image Processing I: Image Acquisition, Pre-processing and Feature Extraction 8. Image Processing II: Face Detection and Face Recognition 9. User Modelling 10. Multimodal Human-Robot Interaction Programming Social Robots 4, Zheng-Hua Tan 36

M. R. Ahmadzadeh Isfahan University of Technology. M. R. Ahmadzadeh Isfahan University of Technology

M. R. Ahmadzadeh Isfahan University of Technology. M. R. Ahmadzadeh Isfahan University of Technology 1 2 M. R. Ahmadzadeh Isfahan University of Technology Ahmadzadeh@cc.iut.ac.ir M. R. Ahmadzadeh Isfahan University of Technology Textbooks 3 Introduction to Machine Learning - Ethem Alpaydin Pattern Recognition

More information

CHAPTER 1: INTRODUCTION

CHAPTER 1: INTRODUCTION CHAPTER 1: INTRODUCTION Big Data 3 Widespread use of personal computers and wireless communication leads to big data We are both producers and consumers of data Producer: when buy a product, rent a movie,

More information

n Learning is useful as a system construction method n Examples of systems that employ ML? q Supervised learning: correct answers for each example

n Learning is useful as a system construction method n Examples of systems that employ ML? q Supervised learning: correct answers for each example Learning Learning from Data Russell and Norvig Chapter 18 Essential for agents working in unknown environments Learning is useful as a system construction method q Expose the agent to reality rather than

More information

Machine Learning ICS 273A. Instructor: Max Welling

Machine Learning ICS 273A. Instructor: Max Welling Machine Learning ICS 273A Instructor: Max Welling Class Homework What is Expected? Required, (answers will be provided) A Project See webpage Quizzes A quiz every Friday Bring scantron form (buy in UCI

More information

Lecture I Outline. Course information and details Why do machine learning? What is machine learning? Why now? Type of Learning

Lecture I Outline. Course information and details Why do machine learning? What is machine learning? Why now? Type of Learning Lecture I Outline Course information and details Why do machine learning? What is machine learning? Why now? Type of Learning Association Classification Three types: Linear, Decision Tree, and Nearest

More information

Welcome to CMPS 142 Machine Learning

Welcome to CMPS 142 Machine Learning Welcome to CMPS 142 Machine Learning Instructor: David Helmbold, dph@soe.ucsc.edu Office hours: Tentatively after class Tu-Th 12-1:30. TA: Keshav Mathur, kemathur@ucsc.edu Web page: https://courses.soe.ucsc.edu/courses/cmps142/spring15/01

More information

Welcome to CMPS 142 and 242: Machine Learning

Welcome to CMPS 142 and 242: Machine Learning Welcome to CMPS 142 and 242: Machine Learning Instructor: David Helmbold, dph@soe.ucsc.edu Office hours: Monday 1:30-2:30, Thursday 4:15-5:00 TA: Aaron Michelony, amichelo@soe.ucsc.edu Web page: www.soe.ucsc.edu/classes/cmps242/fall13/01

More information

Lecture 1: What is Machine Learning? STAT161/261 Introduction to Pattern Recognition and Machine Learning Spring 2018 Prof.

Lecture 1: What is Machine Learning? STAT161/261 Introduction to Pattern Recognition and Machine Learning Spring 2018 Prof. Lecture 1: What is Machine Learning? STAT161/261 Introduction to Pattern Recognition and Machine Learning Spring 2018 Prof. Allie Fletcher Lecture 1 Outline Course information and details What and why

More information

CS340 Machine learning Lecture 2

CS340 Machine learning Lecture 2 CS340 Machine learning Lecture 2 What is machine learning? ``Learning denotes changes in the system that are adaptive in the sense that they enable the system to do the task or tasks drawn from the same

More information

Welcome to CMPS 142: Machine Learning. Administrivia. Lecture Slides for. Instructor: David Helmbold,

Welcome to CMPS 142: Machine Learning. Administrivia. Lecture Slides for. Instructor: David Helmbold, Welcome to CMPS 142: Machine Learning Instructor: David Helmbold, dph@soe.ucsc.edu Web page: www.soe.ucsc.edu/classes/cmps142/winter07/ Text: Introduction to Machine Learning, Alpaydin Administrivia Sign

More information

Applied Machine Learning

Applied Machine Learning Applied Spring 2018, CS 519 Prof. Liang Huang School of EECS Oregon State University liang.huang@oregonstate.edu is Everywhere A breakthrough in machine learning would be worth ten Microsofts (Bill Gates)

More information

CHAPTER 1: INTRODUCTION

CHAPTER 1: INTRODUCTION CHAPTER 1: INTRODUCTION Big Data 3 Widespread use of personal computers and wireless communication leads to big data We are both producers and consumers of data Data is not random, it has structure, e.g.,

More information

CSC 2515: Lecture 01: Introduction

CSC 2515: Lecture 01: Introduction CSC 2515: Lecture 01: Introduction Richard Zemel & Raquel Urtasun University of Toronto Sep 17, 2015 Zemel & Urtasun (UofT) CSC 2515: 01-Introduction Sep 17, 2015 1 / 50 Today Administration details Why

More information

CS545 Machine Learning

CS545 Machine Learning Machine learning and related fields CS545 Machine Learning Course Introduction Machine learning: the construction and study of systems that learn from data. Pattern recognition: the same field, different

More information

Machine Learning: Summary

Machine Learning: Summary Machine Learning: Summary Greg Grudic CSCI-4830 Machine Learning 1 What is Machine Learning? The goal of machine learning is to build computer systems that can adapt and learn from their experience. Tom

More information

Foundations of Machine Learning and Data Mining Rainer Marrone, Ralf Möller. Today s slides taken partly from E. ALPAYDIN

Foundations of Machine Learning and Data Mining Rainer Marrone, Ralf Möller. Today s slides taken partly from E. ALPAYDIN Foundations of Machine Learning and Data Mining Rainer Marrone, Ralf Möller Today s slides taken partly from E. ALPAYDIN 1 Lab Class and literature Thursday, 13:15 14:45, ES42 2589 Lab Class Fr 9:45-10:30,

More information

CS534 Machine Learning

CS534 Machine Learning CS534 Machine Learning Spring 2013 Lecture 1: Introduction to ML Course logistics Reading: The discipline of Machine learning by Tom Mitchell Course Information Instructor: Dr. Xiaoli Fern Kec 3073, xfern@eecs.oregonstate.edu

More information

MACHINE LEARNING AND PATTERN RECOGNITION Spring 2004, Lecture 1: Introduction and Basic Concepts Yann LeCun

MACHINE LEARNING AND PATTERN RECOGNITION Spring 2004, Lecture 1: Introduction and Basic Concepts Yann LeCun Y. LeCun: Machine Learning and Pattern Recognition p. 1/3 MACHINE LEARNING AND PATTERN RECOGNITION Spring 2004, Lecture 1: Introduction and Basic Concepts Yann LeCun The Courant Institute, New York University

More information

INTRODUCTION TO MACHINE LEARNING SOME CONTENT COURTESY OF PROFESSOR ANDREW NG OF STANFORD UNIVERSITY

INTRODUCTION TO MACHINE LEARNING SOME CONTENT COURTESY OF PROFESSOR ANDREW NG OF STANFORD UNIVERSITY INTRODUCTION TO MACHINE LEARNING SOME CONTENT COURTESY OF PROFESSOR ANDREW NG OF STANFORD UNIVERSITY IQS2: Spring 2013 Machine Learning Definition 2 Arthur Samuel (1959). Machine Learning: Field of study

More information

Machine Learning 101a. Jan Peters Gerhard Neumann

Machine Learning 101a. Jan Peters Gerhard Neumann Machine Learning 101a Jan Peters Gerhard Neumann 1 Purpose of this Lecture Statistics and Math Refresher Foundations of machine learning tools for robotics We focus on regression methods and general principles

More information

Introduction to Machine Learning Stephen Scott, Dept of CSE

Introduction to Machine Learning Stephen Scott, Dept of CSE Introduction to Machine Learning Stephen Scott, Dept of CSE What is Machine Learning? Building machines that automatically learn from experience Sub-area of artificial intelligence (Very) small sampling

More information

Introduction to Computational Linguistics

Introduction to Computational Linguistics Introduction to Computational Linguistics Olga Zamaraeva (2018) Based on Guestrin (2013) University of Washington April 10, 2018 1 / 30 This and last lecture: bird s eye view Next lecture: understand precision

More information

Artificial Intelligence Introduction to Machine Learning

Artificial Intelligence Introduction to Machine Learning Artificial Intelligence Introduction to Machine Learning Artificial Intelligence Chung-Ang University Narration: Prof. Jaesung Lee Introduction Applications which Machine Learning techniques play an important

More information

CS798: Selected topics in Machine Learning

CS798: Selected topics in Machine Learning CS798: Selected topics in Machine Learning Introduction Jakramate Bootkrajang Department of Computer Science Chiang Mai University Jakramate Bootkrajang CS798: Selected topics in Machine Learning 1 / 22

More information

Pattern Classification and Clustering Spring 2006

Pattern Classification and Clustering Spring 2006 Pattern Classification and Clustering Time: Spring 2006 Room: Instructor: Yingen Xiong Office: 621 McBryde Office Hours: Phone: 231-4212 Email: yxiong@cs.vt.edu URL: http://www.cs.vt.edu/~yxiong/pcc/ Detailed

More information

Introduction to Machine Learning 1. Nov., 2018 D. Ratner SLAC National Accelerator Laboratory

Introduction to Machine Learning 1. Nov., 2018 D. Ratner SLAC National Accelerator Laboratory Introduction to Machine Learning 1 Nov., 2018 D. Ratner SLAC National Accelerator Laboratory Introduction What is machine learning? Arthur Samuel (1959): Ability to learn without being explicitly programmed

More information

Course Overview and Introduction CE-717 : Machine Learning Sharif University of Technology. M. Soleymani Fall 2012

Course Overview and Introduction CE-717 : Machine Learning Sharif University of Technology. M. Soleymani Fall 2012 Course Overview and Introduction CE-717 : Machine Learning Sharif University of Technology M. Soleymani Fall 2012 Course Info Instructor: Mahdieh Soleymani Email: soleyman@ce.sharif.edu Lectures: Sun-Tue

More information

Class Overview and General Introduction to Machine Learning

Class Overview and General Introduction to Machine Learning Class Overview and General Introduction to Machine Learning Piyush Rai www.cs.utah.edu/~piyush CS5350/6350: Machine Learning August 23, 2011 (CS5350/6350) Intro to ML August 23, 2011 1 / 25 What is Machine

More information

ECE-271A Statistical Learning I

ECE-271A Statistical Learning I ECE-271A Statistical Learning I Nuno Vasconcelos ECE Department, UCSD The course the course is an introductory level course in statistical learning by introductory I mean that you will not need any previous

More information

Course Overview and Introduction CE-717 : Machine Learning Sharif University of Technology. M. Soleymani Fall 2014

Course Overview and Introduction CE-717 : Machine Learning Sharif University of Technology. M. Soleymani Fall 2014 Course Overview and Introduction CE-717 : Machine Learning Sharif University of Technology M. Soleymani Fall 2014 Course Info Instructor: Mahdieh Soleymani Email: soleymani@sharif.edu Lectures: Sun-Tue

More information

Lecture 1: Introduction to Machine Learning

Lecture 1: Introduction to Machine Learning Statistical Methods for Intelligent Information Processing (SMIIP) Lecture 1: Introduction to Machine Learning Shuigeng Zhou School of Computer Science September 13, 2017 What is machine learning? Machine

More information

Machine Learning: Preliminaries & Overview

Machine Learning: Preliminaries & Overview Machine Learning: Preliminaries & Overview Winter 2018 LOL What is machine learning? Textbook definitions of machine learning : Detecting patterns and regularities with a good and generalizable approximation

More information

Course Overview and Introduction CE-717 : Machine Learning Sharif University of Technology. M. Soleymani Fall 2016

Course Overview and Introduction CE-717 : Machine Learning Sharif University of Technology. M. Soleymani Fall 2016 Course Overview and Introduction CE-717 : Machine Learning Sharif University of Technology M. Soleymani Fall 2016 Course Info Instructor: Mahdieh Soleymani Email: soleymani@sharif.edu Lectures: Sun-Tue

More information

Machine Learning Lecture 1: Introduction

Machine Learning Lecture 1: Introduction What is? Building machines that automatically learn from experience Sub-area of artificial intelligence (Very) small sampling of applications: Lecture 1: Introduction Detection of fraudulent credit card

More information

Lecture 1. Introduction Bastian Leibe Visual Computing Institute RWTH Aachen University

Lecture 1. Introduction Bastian Leibe Visual Computing Institute RWTH Aachen University Advanced Machine Learning Lecture 1 Introduction 20.10.2015 Bastian Leibe Visual Computing Institute RWTH Aachen University http://www.vision.rwth-aachen.de/ leibe@vision.rwth-aachen.de Organization Lecturer

More information

Structured Output Prediction

Structured Output Prediction Structured Output Prediction CS4780/5780 Machine Learning Fall 2011 Thorsten Joachims Cornell University Reading: T. Joachims, T. Hofmann, Yisong Yue, Chun-Nam Yu, Predicting Structured Objects with Support

More information

Outline. Little green men INTRODUCTION TO STATISTICAL MACHINE LEARNING. Representing things in Machine Learning 10/22/2010

Outline. Little green men INTRODUCTION TO STATISTICAL MACHINE LEARNING. Representing things in Machine Learning 10/22/2010 Outline INTRODUCTION TO STATISTICAL MACHINE LEARNING Representing things Feature vector Training sample Unsupervised learning Clustering Supervised learning Classification Regression Xiaojin Zhu jerryzhu@cs.wisc.edu

More information

Machine Learning for Computer Vision

Machine Learning for Computer Vision Computer Group Prof. Daniel Cremers Machine Learning for Computer PD Dr. Rudolph Triebel Lecturers PD Dr. Rudolph Triebel rudolph.triebel@in.tum.de Room number 02.09.059 Main lecture MSc. Ioannis John

More information

Lecture 1: Machine Learning Basics

Lecture 1: Machine Learning Basics 1/69 Lecture 1: Machine Learning Basics Ali Harakeh University of Waterloo WAVE Lab ali.harakeh@uwaterloo.ca May 1, 2017 2/69 Overview 1 Learning Algorithms 2 Capacity, Overfitting, and Underfitting 3

More information

Introduction to statistical learning

Introduction to statistical learning Introduction to statistical learning 1. Introduction V. Lefieux June 2018 1/42 Table of contents 2/42 Table of contents 3/42 Data everywhere 4/42 Data everywhere Before: structured data, generated by companies

More information

Python Machine Learning

Python Machine Learning Python Machine Learning Unlock deeper insights into machine learning with this vital guide to cuttingedge predictive analytics Sebastian Raschka [ PUBLISHING 1 open source I community experience distilled

More information

Machine Learning Basics

Machine Learning Basics Deep Learning Theory and Applications Machine Learning Basics Kevin Moon (kevin.moon@yale.edu) Guy Wolf (guy.wolf@yale.edu) CPSC/AMTH 663 Outline 1. What is machine learning? 2. Supervised Learning Regression

More information

Machine Learning - Introduction

Machine Learning - Introduction Machine Learning - Introduction CSE 4309 Machine Learning Vassilis Athitsos Computer Science and Engineering Department University of Texas at Arlington 1 What is Machine Learning Quote by Tom M. Mitchell:

More information

Lecture 10 Summary and reflections

Lecture 10 Summary and reflections Lecture 10 Summary and reflections Niklas Wahlström Division of Systems and Control Department of Information Technology Uppsala University. Email: niklas.wahlstrom@it.uu.se SML - Lecture 10 Contents Lecture

More information

36-350: Data Mining. Fall Lectures: Monday, Wednesday and Friday, 10:30 11:20, Porter Hall 226B

36-350: Data Mining. Fall Lectures: Monday, Wednesday and Friday, 10:30 11:20, Porter Hall 226B 36-350: Data Mining Fall 2009 Instructor: Cosma Shalizi, Statistics Dept., Baker Hall 229C, cshalizi@stat.cmu.edu Teaching Assistant: Joseph Richards, jwrichar@stat.cmu.edu Lectures: Monday, Wednesday

More information

Pattern Recognition Systems

Pattern Recognition Systems Pattern Recognition Systems Dr. Shuang LIANG School of Software Engineering TongJi University Fall, 2012 Today s Topics An example Pattern recognition systems The design cycle Introduction Pattern Recognition,

More information

Welcome to CSCE 496/896: Deep Learning! Welcome to CSCE 496/896: Deep Learning! Override Policy. Override Policy. Override Policy.

Welcome to CSCE 496/896: Deep Learning! Welcome to CSCE 496/896: Deep Learning! Override Policy. Override Policy. Override Policy. Welcome to CSCE 496/896: Deep! Welcome to CSCE 496/896: Deep! Please check off your name on the roster, or write your name if you're not listed Indicate if you wish to register or sit in Policy on sit-ins:

More information

Lecture 1: Introduc4on

Lecture 1: Introduc4on CSC2515 Spring 2014 Introduc4on to Machine Learning Lecture 1: Introduc4on All lecture slides will be available as.pdf on the course website: http://www.cs.toronto.edu/~urtasun/courses/csc2515/csc2515_winter15.html

More information

CSE 446 Machine Learning

CSE 446 Machine Learning CSE 446 Machine What is Machine? Daniel Weld Xiao Ling Congle Zhang 1 2 Machine Study of algorithms that improve their performance at some task with experience Why? Data Machine Understanding Is this topic

More information

STA 414/2104 Statistical Methods for Machine Learning and Data Mining

STA 414/2104 Statistical Methods for Machine Learning and Data Mining STA 414/2104 Statistical Methods for Machine Learning and Data Mining Radford M. Neal, University of Toronto, 2014 Week 1 What are Machine Learning and Data Mining? Typical Machine Learning and Data Mining

More information

MLD Statistical Machine Learning

MLD Statistical Machine Learning Spring 2008 Syllabus MLD 10-702 Statistical Machine Learning http://www.stat.cmu.edu/ larry/=sml2008 Statistical Machine Learning is a second graduate level course in machine learning, assuming students

More information

Introduction to Pattern Recognition

Introduction to Pattern Recognition Introduction to Pattern Recognition Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Fall 2017 CS 551, Fall 2017 c 2017, Selim Aksoy (Bilkent University)

More information

Machine Learning Lecture 1: Introduction

Machine Learning Lecture 1: Introduction Welcome to CSCE 478/878! Please check off your name on the roster, or write your name if you're not listed Indicate if you wish to register or sit in Policy on sit-ins: You may sit in on the course without

More information

Machine Learning for Computer Vision

Machine Learning for Computer Vision Prof. Daniel Cremers Machine Learning for Computer PD Dr. Rudolph Triebel Lecturers PD Dr. Rudolph Triebel rudolph.triebel@in.tum.de Room number 02.09.059 (Fridays) Main lecture MSc. Ioannis John Chiotellis

More information

Data Classification: Advanced Concepts. Lijun Zhang

Data Classification: Advanced Concepts. Lijun Zhang Data Classification: Advanced Concepts Lijun Zhang zlj@nju.edu.cn http://cs.nju.edu.cn/zlj Outline Introduction Multiclass Learning Rare Class Learning Scalable Classification Semisupervised Learning Active

More information

Lecture 1. Introduction - Part 1. Luigi Freda. ALCOR Lab DIAG University of Rome La Sapienza. October 6, 2016

Lecture 1. Introduction - Part 1. Luigi Freda. ALCOR Lab DIAG University of Rome La Sapienza. October 6, 2016 Lecture 1 Introduction - Part 1 Luigi Freda ALCOR Lab DIAG University of Rome La Sapienza October 6, 2016 Luigi Freda (University of Rome La Sapienza ) Lecture 1 October 6, 2016 1 / 39 Outline 1 General

More information

CS 6375 Advanced Machine Learning (Qualifying Exam Section) Nicholas Ruozzi University of Texas at Dallas

CS 6375 Advanced Machine Learning (Qualifying Exam Section) Nicholas Ruozzi University of Texas at Dallas CS 6375 Advanced Machine Learning (Qualifying Exam Section) Nicholas Ruozzi University of Texas at Dallas Slides adapted from David Sontag and Vibhav Gogate Course Info. Instructor: Nicholas Ruozzi Office:

More information

INTRODUCTION TO DATA SCIENCE

INTRODUCTION TO DATA SCIENCE DATA11001 INTRODUCTION TO DATA SCIENCE EPISODE 6: MACHINE LEARNING TODAY S MENU 1. WHAT IS ML? 2. CLASSIFICATION AND REGRESSSION 3. EVALUATING PERFORMANCE & OVERFITTING WHAT IS MACHINE LEARNING? Definition:

More information

Machine Learning. Introduction. Hamid Beigy. Sharif University of Technology. Fall 1395

Machine Learning. Introduction. Hamid Beigy. Sharif University of Technology. Fall 1395 Machine Learning Introduction Hamid Beigy Sharif University of Technology Fall 1395 Hamid Beigy (Sharif University of Technology) Machine Learning Fall 1395 1 / 15 Table of contents 1 What is machine learning?

More information

6. Learning and Adaptation

6. Learning and Adaptation Autonomous Systems Tutorial: Part II 6. Learning and Adaptation David J. Atkinson, Ph.D Senior Research Scientist Outline Review: Types of Knowledge Why Learn and Adapt? Desired Capabilities Bootstrapping

More information

10-702: Statistical Machine Learning

10-702: Statistical Machine Learning 10-702: Statistical Machine Learning Syllabus, Spring 2010 http://www.cs.cmu.edu/~10702 Statistical Machine Learning is a second graduate level course in machine learning, assuming students have taken

More information

The Fundamentals of Machine Learning

The Fundamentals of Machine Learning The Fundamentals of Machine Learning Willie Brink 1, Nyalleng Moorosi 2 1 Stellenbosch University, South Africa 2 Council for Scientific and Industrial Research, South Africa Deep Learning Indaba 2017

More information

Introduction. Notices. A Learning Agent 22/11/2012. COMP219: Artificial Intelligence. COMP219: Artificial Intelligence

Introduction. Notices. A Learning Agent 22/11/2012. COMP219: Artificial Intelligence. COMP219: Artificial Intelligence COMP219: Artificial Intelligence COMP219: Artificial Intelligence Dr. Annabel Latham Room 2.05 Ashton Building Department of Computer Science University of Liverpool Lecture 27: Introduction to Learning,

More information

Machine Learning for Computer Vision

Machine Learning for Computer Vision Prof. Daniel Cremers Machine Learning for Computer PD Dr. Rudolph Triebel Lecturers PD Dr. Rudolph Triebel rudolph.triebel@in.tum.de Room number 02.09.058 (Fridays) Main lecture MSc. Ioannis John Chiotellis

More information

DS Machine Learning and Data Mining I. Alina Oprea Associate Professor, CCIS Northeastern University

DS Machine Learning and Data Mining I. Alina Oprea Associate Professor, CCIS Northeastern University DS 4400 Machine Learning and Data Mining I Alina Oprea Associate Professor, CCIS Northeastern University January 10 2019 Class Outline Introduction 1 week Probability and linear algebra review Supervised

More information

Optical character recognition (ICDAR - International Conference on Document Analysis and Recognition)

Optical character recognition (ICDAR - International Conference on Document Analysis and Recognition) What is Machine Learning A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves

More information

CSC 411 MACHINE LEARNING and DATA MINING

CSC 411 MACHINE LEARNING and DATA MINING CSC 411 MACHINE LEARNING and DATA MINING Lectures: Monday, Wednesday 12-1 (section 1), 3-4 (section 2) Lecture Room: MP 134 (section 1); Bahen 1200 (section 2) Instructor (section 1): Richard Zemel Instructor

More information

A Review on Machine Learning Algorithms, Tasks and Applications

A Review on Machine Learning Algorithms, Tasks and Applications A Review on Machine Learning Algorithms, Tasks and Applications Diksha Sharma 1, Neeraj Kumar 2 ABSTRACT: Machine learning is a field of computer science which gives computers an ability to learn without

More information

Machine Learning Lecture 1

Machine Learning Lecture 1 Machine Learning Lecture 1 Introduction 12.10.2017 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de/ leibe@vision.rwth-aachen.de Organization Lecturer Prof. Bastian Leibe (leibe@vision.rwth-aachen.de)

More information

Machine Learning. Lecture 1: Introduction to Machine Learning. Nevin L. Zhang

Machine Learning. Lecture 1: Introduction to Machine Learning. Nevin L. Zhang Machine Learning Lecture 1: Introduction to Machine Learning Nevin L. Zhang lzhang@cse.ust.hk Department of Computer Science and Engineering The Hong Kong University of Science and Technology This set

More information

Introduction to Machine Learning & Its Application in Healthcare Lecture 4 Oct 3, 2018 Presentation by: Leila Karimi

Introduction to Machine Learning & Its Application in Healthcare Lecture 4 Oct 3, 2018 Presentation by: Leila Karimi Introduction to Machine Learning & Its Application in Healthcare Lecture 4 Oct 3, 2018 Presentation by: Leila Karimi 1 What Is Machine Learning? A branch of artificial intelligence, concerned with the

More information

Machine Learning Lecture 1

Machine Learning Lecture 1 Machine Learning Lecture 1 Introduction 11.10.2018 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de/ leibe@vision.rwth-aachen.de Organization Lecturer Prof. Bastian Leibe (leibe@vision.rwth-aachen.de)

More information

CS540 Machine learning Lecture 1 Introduction

CS540 Machine learning Lecture 1 Introduction CS540 Machine learning Lecture 1 Introduction Administrivia Overview Supervised learning Unsupervised learning Other kinds of learning Outline Administrivia Class web page www.cs.ubc.ca/~murphyk/teaching/cs540-fall08

More information

CHAPTER 4 IMPROVING THE PERFORMANCE OF A CLASSIFIER USING UNIQUE FEATURES

CHAPTER 4 IMPROVING THE PERFORMANCE OF A CLASSIFIER USING UNIQUE FEATURES 38 CHAPTER 4 IMPROVING THE PERFORMANCE OF A CLASSIFIER USING UNIQUE FEATURES 4.1 INTRODUCTION In classification tasks, the error rate is proportional to the commonality among classes. Conventional GMM

More information

CS148 - Building Intelligent Robots Lecture 6: Learning for Robotics. Instructor: Chad Jenkins (cjenkins)

CS148 - Building Intelligent Robots Lecture 6: Learning for Robotics. Instructor: Chad Jenkins (cjenkins) Lecture 6 Robot Learning Slide 1 CS148 - Building Intelligent Robots Lecture 6: Learning for Robotics Instructor: Chad Jenkins (cjenkins) Lecture 6 Robot Learning Slide 2 Administrivia: good news No class

More information

Harivinod N Dept of CSE Vivekananda College of Engineering Technology, Puttur

Harivinod N Dept of CSE Vivekananda College of Engineering Technology, Puttur 15CS73, VTU CBCS Scheme By Dept of CSE Vivekananda College of Engineering Technology, Puttur What is Learning? Learning - improve automatically with experience Using past experiences to improve future

More information

Machine learning theory

Machine learning theory Machine learning theory Machine learning theory Introduction Hamid Beigy Sharif university of technology February 27, 2017 Hamid Beigy Sharif university of technology February 27, 2017 1 / 28 Machine learning

More information

Practical Advice for Building Machine Learning Applications

Practical Advice for Building Machine Learning Applications Practical Advice for Building Machine Learning Applications Machine Learning Fall 2017 Based on lectures and papers by Andrew Ng, Pedro Domingos, Tom Mitchell and others 1 This lecture: ML and the world

More information

PAC Learning Introduction to Machine Learning. Matt Gormley Lecture 14 March 5, 2018

PAC Learning Introduction to Machine Learning. Matt Gormley Lecture 14 March 5, 2018 10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University PAC Learning Matt Gormley Lecture 14 March 5, 2018 1 ML Big Picture Learning Paradigms:

More information

Introduction. Industrial AI Lab.

Introduction. Industrial AI Lab. Introduction Industrial AI Lab. 2018 - present: POSTECH Industrial AI Lab. Introduction 2013-2017: UNIST isystems Design Lab. 2010, Ph.D. from the University of Michigan, Ann Arbor S. M. Wu Manufacturing

More information

CPSC 340: Machine Learning and Data Mining. Course Review/Preview Fall 2015

CPSC 340: Machine Learning and Data Mining. Course Review/Preview Fall 2015 CPSC 340: Machine Learning and Data Mining Course Review/Preview Fall 2015 Admin Assignment 6 due now. We will have office hours as usual next week. Final exam details: December 15: 8:30-11 (WESB 100).

More information

Lecture 2 Fundamentals of machine learning

Lecture 2 Fundamentals of machine learning Lecture 2 Fundamentals of machine learning Topics of this lecture Formulation of machine learning Taxonomy of learning algorithms Supervised, semi-supervised, and unsupervised learning Parametric and non-parametric

More information

Machine Learning. Introduction. Hamid Beigy. Sharif University of Technology. Fall 1393

Machine Learning. Introduction. Hamid Beigy. Sharif University of Technology. Fall 1393 Machine Learning Introduction Hamid Beigy Sharif University of Technology Fall 1393 Hamid Beigy (Sharif University of Technology) Machine Learning Fall 1393 1 / 15 Table of contents 1 What is machine learning?

More information

Introduction to Machine Learning

Introduction to Machine Learning Andrea Passerini passerini@disi.unitn.it Machine Learning What is Machine Learning A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P,

More information

ID2223 Lecture 2: Distributed ML and Linear Regression

ID2223 Lecture 2: Distributed ML and Linear Regression ID2223 Lecture 2: Distributed ML and Linear Regression Terminology Observations. Entities used for learning/evaluation Features. Attributes (typically numeric) used to represent an observation Labels.

More information

Theodoridis, S. and K. Koutroumbas, Pattern recognition. 4th ed. 2009, San Diego, CA: Academic Press.

Theodoridis, S. and K. Koutroumbas, Pattern recognition. 4th ed. 2009, San Diego, CA: Academic Press. Pattern Recognition Winter 2013 Andrew Cohen acohen@coe.drexel.edu What is this course about? This course will study state-of-the-art techniques for analyzing data. The goal is to extract meaningful information

More information

Overview. Overview of the course. Classification, Clustering, and Dimension reduction. The curse of dimensionality

Overview. Overview of the course. Classification, Clustering, and Dimension reduction. The curse of dimensionality Overview Overview of the course Classification, Clustering, and Dimension reduction The curse of dimensionality Tianwei Yu RSPH Room 334 Tianwei.yu@emory.edu 1 Instructor: Course Outline Tianwei Yu Office:

More information

Introduction. Jun Zhu. Tsinghua University. [ Advanced Machine Learning, Fall, 2012]

Introduction. Jun Zhu. Tsinghua University. [ Advanced Machine Learning, Fall, 2012] [80240603 Advanced Machine Learning, Fall, 2012] Introduction Jun Zhu dcszj@mail.tsinghua.edu.cn Sate Key Lab of Intelligent Tech. & Systems, Tsinghua University Goals of this Lecture Show that machine

More information

Machine Learning for Chemoinformatics An introduction

Machine Learning for Chemoinformatics An introduction Machine Learning for Chemoinformatics An introduction Francesca Grisoni University of Milano-Bicocca, Dept. of Earth and Environmental Sciences, Milan, Italy ETH Zurich, Dept. of Chemistry and Applied

More information

Welcome to CSCE 478/878! Please check off your name on the roster, or write your name if you re not listed

Welcome to CSCE 478/878! Please check off your name on the roster, or write your name if you re not listed Welcome to CSCE 478/878! Please check off your name on the roster, or write your name if you re not listed CSCE 478/878 Lecture 0: Administrivia Policy on sit-ins: You may sit in on the course without

More information

Statistical Pattern Recognition

Statistical Pattern Recognition Statistical Pattern Recognition A Brief Overview of the course Hamid R. Rabiee Jafar Muhammadi, Nima Pourdamghani Spring 2012 http://ce.sharif.edu/courses/90-91/2/ce725-1/ Agenda What is a Pattern? What

More information

Practical Data Science with R

Practical Data Science with R Practical Data Science with R NINAZUMEL JOHN MOUNT Ill MANNING SHELTER ISLAND Practical Data Science with R NINAZUMEL JOHN MOUNT MANNING SHELTER ISLAND brief contents 1 Ill The data science process 3 2

More information

Data Mining: Practical Machine Learning Techniques

Data Mining: Practical Machine Learning Techniques Artificial Intelligence Data Mining: Practical Machine Learning Techniques Dae-Won Kim School of Computer Science & Engineering Chung-Ang University AI Scope 1. Search-based optimization techniques for

More information

State of Machine Learning and Future of Machine Learning

State of Machine Learning and Future of Machine Learning State of Machine Learning and Future of Machine Learning (based on the vision of T.M. Mitchell) Rémi Gilleron Mostrare project Lille university and INRIA Futurs www.grappa.univ-lille3.fr/mostrare Collège

More information

Unsupervised Learning: Clustering

Unsupervised Learning: Clustering Unsupervised Learning: Clustering Vibhav Gogate The University of Texas at Dallas Slides adapted from Carlos Guestrin, Dan Klein & Luke Zettlemoyer Machine Learning Supervised Learning Unsupervised Learning

More information

MACHINE LEARNING. Slide adapted from learning from data book and course, and Berkeley cs188 by Dan Klein, and Pieter Abbeel

MACHINE LEARNING. Slide adapted from learning from data book and course, and Berkeley cs188 by Dan Klein, and Pieter Abbeel MACHINE LEARNING Slide adapted from learning from data book and course, and Berkeley cs188 by Dan Klein, and Pieter Abbeel Machine Learning?? Learning from data Tasks: Prediction Classification Recognition

More information

Optimization Methods for Machine Learning (OMML)

Optimization Methods for Machine Learning (OMML) Optimization Methods for Machine Learning (OMML) 1st lecture (1 slot) Prof. L. Palagi 30/09/2015 1 (6 cfu) TO BE UPDATED Course at a glance http://www.dis.uniroma1.it/~or/gestionale/svm/ Assistant Professor:

More information

Data Mining. Practical Machine Learning Tools and Techniques, Second Edition V

Data Mining. Practical Machine Learning Tools and Techniques, Second Edition V Data Mining Practical Machine Learning Tools and Techniques, Second Edition V Ian H. Witten Department of Computer Science University of Waikato Eibe Frank Department of Computer Science University of

More information

10. Machine Learning

10. Machine Learning Artificial Intelligence 10. Machine Learning Prof. Bojana Dalbelo Bašić Assoc. Prof. Jan Šnajder University of Zagreb Faculty of Electrical Engineering and Computing Academic Year 2016/2017 Creative Commons

More information

Ensemble Methods. Zhi-Hua Zhou. Foundations and Algorithms. Chapman & Hall/CRC. CRC Press. Machine Learning & Pattern Recognition Series

Ensemble Methods. Zhi-Hua Zhou. Foundations and Algorithms. Chapman & Hall/CRC. CRC Press. Machine Learning & Pattern Recognition Series Chapman & Hall/CRC Machine Learning & Pattern Recognition Series Ensemble Methods Foundations and Algorithms Zhi-Hua Zhou CRC Press Taylor & Francis Croup Boca Raton London New York CRC Press is an imprint

More information