INTRODUCTION TO MACHINE LEARNING SOME CONTENT COURTESY OF PROFESSOR ANDREW NG OF STANFORD UNIVERSITY

Save this PDF as:
Size: px
Start display at page:

Download "INTRODUCTION TO MACHINE LEARNING SOME CONTENT COURTESY OF PROFESSOR ANDREW NG OF STANFORD UNIVERSITY"

Transcription

1 INTRODUCTION TO MACHINE LEARNING SOME CONTENT COURTESY OF PROFESSOR ANDREW NG OF STANFORD UNIVERSITY IQS2: Spring 2013

2 Machine Learning Definition 2 Arthur Samuel (1959). Machine Learning: Field of study that gives computers the ability to learn without being explicitly programmed. Samuel s claim to fame: Checkers. Had his program play against itself tens of thousands of times, noting board positions that tended to lead to wins, and those that tended to lead to losses. In time the program became much better at checkers than Samuel ever was!

3 Machine Learning Definition 3 Problem with Samuel s definition: too informal. How do we know when the definition has been satisfied? Tom Mitchell (1988). Well-posed learning problem: A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E.

4 A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E. 4 Suppose your program watches which s you do or do not mark as spam, and based on that learns how to better filter spam. What are T, P, and E?

5 A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E. 5 Suppose your program watches which s you do or do not mark as spam, and based on that learns how to better filter spam. What are T, P, and E? T: Classifying s as spam or not spam. E: Watching you label s as spam or not spam P: The number (or fraction) of s correctly classified as spam/not spam.

6 Machine Learning Algorithms 6 Supervised learning (what we ll do: learning is supervised in the sense that we provide training data) Currently the most common type of machine learning Unsupervised learning Example: Clustering algorithms Example: Some data mining Others: Recommender systems (think Netflix)

7 Terminology: Feature Vector 7 or simply features: The characteristics of the studied phenomenon that provide input to the machine learning algorithm Ex. In Housing prices example, only one feature: the size in square feet of the house. We could use more features, such as whether the house has a garage, whether it has central air conditioning, the number of bathrooms, etc. Then we would have an entire vector of features. NOTE: These are not parameters. We do not change their values to optimize our model. We do, however, try to select sufficient features to allow us to meet our goals.

8 Supervised Learning 8 Supervised learning: Correct answers given Ex. Linear regression: correct housing prices for some square foot values n Note: nothing to prevent two houses of the same size having two different prices Ex. Digit classification: Sample images are given along with the digit they represent n Note that in classification problems, though the answer is a class, it is often coded as a discrete numerical value n E.g., if code classified as malicious, code it 1, else code it 0 n In our digit classifier, the coded value is actually NOT the digit: it s a 10 x 1 column vector, each of whose entries are between 0 and 1 (typical of multi-class classification)

9 Supervised Learning Terminology 9 Training set: the data that is used to teach your program Test set: the data that will be used to test your program (this should definitely NOT be the same as your training data). Cross-validation VERY IMPORTANT: Just because your classifier works well (or even perfectly) on you training data, this does NOT mean that it will perform well on other data! n In fact, classifiers that work perfectly on the training data are often over fitted (more on this later).

10 Cost Function 10 Recall that machine learning, according to our formal definition, requires measured performance. In virtually all cases, this is provided by a cost function Recall from linear regression: In this example (as in most) lower cost means a better solution. So improving performance means finding values of the parameters θ0 and θ1 that minimize the cost function (or at least create a lower cost than our earlier choices).

11 Cost Function 11 Recall that machine learning, according to our formal definition, requires measured performance. In virtually all cases, this is provided by a cost function Recall from linear regression: Note here that θ0 and θ1 are the only variables in this example, in the sense that the values of the xi and yi are provided to us by the training data.

12 12 Your Cost Function

13 13 Where: m = number of training data K = output dimension (10 for us) h is the neural network output, which we ll discuss later Θ (l) is a family of matrices (actually just 2 matrices) n Θ (1) is a 257 x 256 matrix of weights n Θ (2) is a 257 x 10 matrix of weights The matrix entries are the parameters. So I lied. There are not 400 parameters. There are 68,362.

14 Don t Panic! 14 I showed you that to try to condition you to it. Sort of like shock therapy. I won t show it again for at least a few days.

15 Don t Panic! 15 I showed you that to try to condition you to it. Sort of like shock therapy. I won t show it again for at least a few days. And when I do, you ll have a better understanding of what it all means.

16 Don t Panic! 16 I showed you that to try to condition you to it. Sort of like shock therapy. I won t show it again for at least a few days. And when I do, you ll have a better understanding of what it all means. BUT, your head will probably still hurt a bit when you work with it.

17 Regularization 17 Deals with under fitting and over fitting A.K.A. bias and variance Under fitting (a.k.a. high bias): Too few parameters to accurately capture real phenomenon Ex:

18 Solution? 18 Add more parameters to the model (in this case, allow for higher dimensional polynomial fits)

19 Problem: Over fitting 19 Over fitting (a.k.a. high variance): Fitting the model too closely to the data can sometimes create poor model outcomes

20 Problem: Over fitting 20 Over fitting: Fitting the model too closely to the data can sometimes create poor model outcomes What if this is the true value of the data point you want to test?

21 Example: Nearest-Neighbor 21 The k-nearest neighbor algorithm classifies a a given point by looking at the k training data values nearest to the point. The majority class wins. 1-nearest neighbor will always classify the training set perfectly! But that will rarely be what you want to use to classify new values (it s classic over fitting). k-nearest neighbor, for k bigger than 1, will not always classify the training set correctly. But will do a much better job at classifying unknown data

22 22 Example: 1-nearest neighbor

23 23 Example: 15-nearest neighbor

24 For Emphasis 24 The issue with over fitting is that if your model has too many parameters (or too many features), you may fit the training set so well that your model fails to generalize to new examples. Since it s new examples you want to classify, this is a problem!

25 In General 25 Machine learning experts use various statistical tools to determine the likelihood that their model is experiencing over fitting or under fitting (and which of the two is the case). When identified, there are methods for remedying the situation. In general, beyond the scope of our foray into machine learning. For over fitting, I ll mention one obvious one: get rid of some features. n But which ones?

26 Regularization 26 A method for dealing with over fitting Basic idea: keep all the features or parameters, but reduce their magnitude As an intuitive example, think in general about what reducing coefficients does to a polynomial Works well when we have a lot of features, each of which contributes a little bit to predicting the class of the data.

27 So If we have the following cost function, and want to force θ3 and θ4 to be small, how can we do this?

28 So If we have the following cost function, and want to force θ3 and θ4 to be small, how can we do this? How about like this?:

29 Intuitively 29 it s a bit easier to see why shrinking θ 3 and θ4 simplify the model and reduce the effect of over fitting than it is to see why shrinking the values of ALL the parameters has a similar effect But shrinking ALL the parameters does do what we want Creates a simpler model Helps avoid the bad effects of over fitting Easiest way to see this is to play around and see the effects for yourself. But it s not necessary...unless you want to someday work in the machine learning arena.

30 Our Cost Function (Once Again) 30 The regularization term

31 Summary 31 We ve seen the basic concepts involved in machine learning We ve discussed the problems of under fitting and over fitting We ve discussed regularization We ve looked at the cost function we ll be using And have possibly been traumatized by it

Introduction. Welcome. Machine Learning

Introduction. Welcome. Machine Learning Introduction Welcome Machine Learning Machine Learning - Grew out of work in AI - New capability for computers Examples: - Database mining Large datasets from growth of automation/web. E.g., Web click

More information

Course Overview and Introduction CE-717 : Machine Learning Sharif University of Technology. M. Soleymani Fall 2012

Course Overview and Introduction CE-717 : Machine Learning Sharif University of Technology. M. Soleymani Fall 2012 Course Overview and Introduction CE-717 : Machine Learning Sharif University of Technology M. Soleymani Fall 2012 Course Info Instructor: Mahdieh Soleymani Email: soleyman@ce.sharif.edu Lectures: Sun-Tue

More information

Practical Advice for Building Machine Learning Applications

Practical Advice for Building Machine Learning Applications Practical Advice for Building Machine Learning Applications Machine Learning Fall 2017 Based on lectures and papers by Andrew Ng, Pedro Domingos, Tom Mitchell and others 1 This lecture: ML and the world

More information

Course Overview and Introduction CE-717 : Machine Learning Sharif University of Technology. M. Soleymani Fall 2016

Course Overview and Introduction CE-717 : Machine Learning Sharif University of Technology. M. Soleymani Fall 2016 Course Overview and Introduction CE-717 : Machine Learning Sharif University of Technology M. Soleymani Fall 2016 Course Info Instructor: Mahdieh Soleymani Email: soleymani@sharif.edu Lectures: Sun-Tue

More information

Course Overview and Introduction CE-717 : Machine Learning Sharif University of Technology. M. Soleymani Fall 2014

Course Overview and Introduction CE-717 : Machine Learning Sharif University of Technology. M. Soleymani Fall 2014 Course Overview and Introduction CE-717 : Machine Learning Sharif University of Technology M. Soleymani Fall 2014 Course Info Instructor: Mahdieh Soleymani Email: soleymani@sharif.edu Lectures: Sun-Tue

More information

CS534 Machine Learning

CS534 Machine Learning CS534 Machine Learning Spring 2013 Lecture 1: Introduction to ML Course logistics Reading: The discipline of Machine learning by Tom Mitchell Course Information Instructor: Dr. Xiaoli Fern Kec 3073, xfern@eecs.oregonstate.edu

More information

Programming Social Robots for Human Interaction. Lecture 4: Machine Learning and Pattern Recognition

Programming Social Robots for Human Interaction. Lecture 4: Machine Learning and Pattern Recognition Programming Social Robots for Human Interaction Lecture 4: Machine Learning and Pattern Recognition Zheng-Hua Tan Dept. of Electronic Systems, Aalborg Univ., Denmark zt@es.aau.dk, http://kom.aau.dk/~zt

More information

CS 6375 Advanced Machine Learning (Qualifying Exam Section) Nicholas Ruozzi University of Texas at Dallas

CS 6375 Advanced Machine Learning (Qualifying Exam Section) Nicholas Ruozzi University of Texas at Dallas CS 6375 Advanced Machine Learning (Qualifying Exam Section) Nicholas Ruozzi University of Texas at Dallas Slides adapted from David Sontag and Vibhav Gogate Course Info. Instructor: Nicholas Ruozzi Office:

More information

Applied Machine Learning

Applied Machine Learning Applied Spring 2018, CS 519 Prof. Liang Huang School of EECS Oregon State University liang.huang@oregonstate.edu is Everywhere A breakthrough in machine learning would be worth ten Microsofts (Bill Gates)

More information

Session 1: Gesture Recognition & Machine Learning Fundamentals

Session 1: Gesture Recognition & Machine Learning Fundamentals IAP Gesture Recognition Workshop Session 1: Gesture Recognition & Machine Learning Fundamentals Nicholas Gillian Responsive Environments, MIT Media Lab Tuesday 8th January, 2013 My Research My Research

More information

CS545 Machine Learning

CS545 Machine Learning Machine learning and related fields CS545 Machine Learning Course Introduction Machine learning: the construction and study of systems that learn from data. Pattern recognition: the same field, different

More information

(Refer Slide Time: 0:33)

(Refer Slide Time: 0:33) Machine Learning for Engineering and Science Applications. Professor Dr. Balaji Srinivasan. Department of Mechanical Engineering. Indian Institute of Technology, Madras. Overview of Machine Learning. We

More information

IC 669: Introduction to Machine Learning. Lecture 1 Taesup Moon

IC 669: Introduction to Machine Learning. Lecture 1 Taesup Moon IC 669: Introduction to Machine Learning Lecture 1 Taesup Moon Outline Examples of machine learning Course info Course outline - Supervised learning: classification, regression -Unsupervised learning:

More information

STA 414/2104 Statistical Methods for Machine Learning and Data Mining

STA 414/2104 Statistical Methods for Machine Learning and Data Mining STA 414/2104 Statistical Methods for Machine Learning and Data Mining Radford M. Neal, University of Toronto, 2014 Week 1 What are Machine Learning and Data Mining? Typical Machine Learning and Data Mining

More information

INTRODUCTION TO DATA SCIENCE

INTRODUCTION TO DATA SCIENCE DATA11001 INTRODUCTION TO DATA SCIENCE EPISODE 6: MACHINE LEARNING TODAY S MENU 1. WHAT IS ML? 2. CLASSIFICATION AND REGRESSSION 3. EVALUATING PERFORMANCE & OVERFITTING WHAT IS MACHINE LEARNING? Definition:

More information

Lecture 2 Fundamentals of machine learning

Lecture 2 Fundamentals of machine learning Lecture 2 Fundamentals of machine learning Topics of this lecture Formulation of machine learning Taxonomy of learning algorithms Supervised, semi-supervised, and unsupervised learning Parametric and non-parametric

More information

CS 760 Machine Learning Spring 2017

CS 760 Machine Learning Spring 2017 Page 1 University of Wisconsin Madison Department of Computer Sciences CS 760 Machine Learning Spring 2017 Final Examination Duration: 1 hour 15 minutes One set of handwritten notes and calculator allowed.

More information

Introduction to Machine Learning 1. Nov., 2018 D. Ratner SLAC National Accelerator Laboratory

Introduction to Machine Learning 1. Nov., 2018 D. Ratner SLAC National Accelerator Laboratory Introduction to Machine Learning 1 Nov., 2018 D. Ratner SLAC National Accelerator Laboratory Introduction What is machine learning? Arthur Samuel (1959): Ability to learn without being explicitly programmed

More information

Regularization. INFO-4604, Applied Machine Learning University of Colorado Boulder. September 19, 2017 Prof. Michael Paul

Regularization. INFO-4604, Applied Machine Learning University of Colorado Boulder. September 19, 2017 Prof. Michael Paul Regularization INFO-4604, Applied Machine Learning University of Colorado Boulder September 19, 2017 Prof. Michael Paul Generalization Prediction functions that work on the training data might not work

More information

CSE 446 Machine Learning

CSE 446 Machine Learning CSE 446 Machine What is Machine? Daniel Weld Xiao Ling Congle Zhang 1 2 Machine Study of algorithms that improve their performance at some task with experience Why? Data Machine Understanding Is this topic

More information

Machine Learning: Preliminaries & Overview

Machine Learning: Preliminaries & Overview Machine Learning: Preliminaries & Overview Winter 2018 LOL What is machine learning? Textbook definitions of machine learning : Detecting patterns and regularities with a good and generalizable approximation

More information

CPSC 340: Machine Learning and Data Mining. Fundamentals of Learning Fall 2017

CPSC 340: Machine Learning and Data Mining. Fundamentals of Learning Fall 2017 CPSC 340: Machine Learning and Data Mining Fundamentals of Learning Fall 2017 Admin Assignment 0 is due Friday: you should be almost done. Waiting list people: you should be registered. You may be e-mailed

More information

Machine Learning 101a. Jan Peters Gerhard Neumann

Machine Learning 101a. Jan Peters Gerhard Neumann Machine Learning 101a Jan Peters Gerhard Neumann 1 Purpose of this Lecture Statistics and Math Refresher Foundations of machine learning tools for robotics We focus on regression methods and general principles

More information

Introduction to Computational Linguistics

Introduction to Computational Linguistics Introduction to Computational Linguistics Olga Zamaraeva (2018) Based on Guestrin (2013) University of Washington April 10, 2018 1 / 30 This and last lecture: bird s eye view Next lecture: understand precision

More information

Optimization Methods for Machine Learning (OMML)

Optimization Methods for Machine Learning (OMML) Optimization Methods for Machine Learning (OMML) 1st lecture (1 slot) Prof. L. Palagi 30/09/2015 1 (6 cfu) TO BE UPDATED Course at a glance http://www.dis.uniroma1.it/~or/gestionale/svm/ Assistant Professor:

More information

Introduction to Machine Learning Prof. Sudeshna Sarkar Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur

Introduction to Machine Learning Prof. Sudeshna Sarkar Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Introduction to Machine Learning Prof. Sudeshna Sarkar Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Module - 1 Lecture - 03 Hypothesis Space and Inductive Bias

More information

Lecture 1: Introduction to Machine Learning

Lecture 1: Introduction to Machine Learning Statistical Methods for Intelligent Information Processing (SMIIP) Lecture 1: Introduction to Machine Learning Shuigeng Zhou School of Computer Science September 13, 2017 What is machine learning? Machine

More information

COMS 4771 Introduction to Machine Learning. Nakul Verma

COMS 4771 Introduction to Machine Learning. Nakul Verma COMS 4771 Introduction to Machine Learning Nakul Verma Machine learning: what? Study of making machines learn a concept without having to explicitly program it. Constructing algorithms that can: learn

More information

Announcements. Only 104 people have signed up for a project team

Announcements. Only 104 people have signed up for a project team Announcements Only 104 people have signed up for a project team if you have not signed up, or are on a team of 1, please try contacting other folks in the same situation if this fails, please email me

More information

Lecture 7: Distributed Representations

Lecture 7: Distributed Representations Lecture 7: Distributed Representations Roger Grosse 1 Introduction We ll take a break from derivatives and optimization, and look at a particular example of a neural net that we can train using backprop:

More information

CS260: Machine Learning Theory Lecture 1: A Gentle Introduction to Learning Theory September 26, 2011

CS260: Machine Learning Theory Lecture 1: A Gentle Introduction to Learning Theory September 26, 2011 CS260: Machine Learning Theory Lecture 1: A Gentle Introduction to Learning Theory September 26, 2011 Lecturer: Jennifer Wortman Vaughan 1 What is Machine Learning? Machine learning studies automatic techniques

More information

Lecture 1: Machine Learning Basics

Lecture 1: Machine Learning Basics 1/69 Lecture 1: Machine Learning Basics Ali Harakeh University of Waterloo WAVE Lab ali.harakeh@uwaterloo.ca May 1, 2017 2/69 Overview 1 Learning Algorithms 2 Capacity, Overfitting, and Underfitting 3

More information

Outline. Little green men INTRODUCTION TO STATISTICAL MACHINE LEARNING. Representing things in Machine Learning 10/22/2010

Outline. Little green men INTRODUCTION TO STATISTICAL MACHINE LEARNING. Representing things in Machine Learning 10/22/2010 Outline INTRODUCTION TO STATISTICAL MACHINE LEARNING Representing things Feature vector Training sample Unsupervised learning Clustering Supervised learning Classification Regression Xiaojin Zhu jerryzhu@cs.wisc.edu

More information

DS Machine Learning and Data Mining I. Alina Oprea Associate Professor, CCIS Northeastern University

DS Machine Learning and Data Mining I. Alina Oprea Associate Professor, CCIS Northeastern University DS 4400 Machine Learning and Data Mining I Alina Oprea Associate Professor, CCIS Northeastern University January 10 2019 Class Outline Introduction 1 week Probability and linear algebra review Supervised

More information

What is Machine Learning? Machine Learning Fall 2018

What is Machine Learning? Machine Learning Fall 2018 What is Machine Learning? Machine Learning Fall 2018 1 Our goal today And through the semester What is (machine) learning? 2 Let s play a game 3 The badges game Attendees of the 1994 conference on Computational

More information

CPSC 340: Machine Learning and Data Mining. Fundamentals of Learning Fall 2018

CPSC 340: Machine Learning and Data Mining. Fundamentals of Learning Fall 2018 CPSC 340: Machine Learning and Data Mining Fundamentals of Learning Fall 2018 Admin Course webpage: www.ugrad.cs.ubc.ca/~cs340 Assignment 1 is due Friday: you should be almost done. Waiting list people:

More information

CS340 Machine learning Lecture 2

CS340 Machine learning Lecture 2 CS340 Machine learning Lecture 2 What is machine learning? ``Learning denotes changes in the system that are adaptive in the sense that they enable the system to do the task or tasks drawn from the same

More information

Machine Learning and Pattern Recognition Introduction

Machine Learning and Pattern Recognition Introduction Machine Learning and Pattern Recognition Introduction Giovanni Maria Farinella gfarinella@dmi.unict.it www.dmi.unict.it/farinella What is ML & PR? Interdisciplinary field focusing on both the mathematical

More information

Harivinod N Dept of CSE Vivekananda College of Engineering Technology, Puttur

Harivinod N Dept of CSE Vivekananda College of Engineering Technology, Puttur 15CS73, VTU CBCS Scheme By Dept of CSE Vivekananda College of Engineering Technology, Puttur What is Learning? Learning - improve automatically with experience Using past experiences to improve future

More information

Machine Learning Tom M. Mitchell Machine Learning Department Carnegie Mellon University. January 11, 2011

Machine Learning Tom M. Mitchell Machine Learning Department Carnegie Mellon University. January 11, 2011 Machine Learning 10-701 Tom M. Mitchell Machine Learning Department Carnegie Mellon University January 11, 2011 Today: What is machine learning? Decision tree learning Course logistics Readings: The Discipline

More information

Supervised Learning: The Setup. Machine Learning Fall 2017

Supervised Learning: The Setup. Machine Learning Fall 2017 Supervised Learning: The Setup Machine Learning Fall 2017 1 Last lecture We saw What is learning? Learning as generalization The badges game 2 This lecture More badges Formalizing supervised learning Instance

More information

Machine Learning: CS 6375 Introduction. Instructor: Vibhav Gogate The University of Texas at Dallas

Machine Learning: CS 6375 Introduction. Instructor: Vibhav Gogate The University of Texas at Dallas Machine Learning: CS 6375 Introduction Instructor: Vibhav Gogate The University of Texas at Dallas Logistics Instructor: Vibhav Gogate Email: Vibhav.Gogate@utdallas.edu Office: ECSS 3.406 Office hours:

More information

CS 2750: Machine Learning. Introduction. Prof. Adriana Kovashka University of Pittsburgh January 5, 2017

CS 2750: Machine Learning. Introduction. Prof. Adriana Kovashka University of Pittsburgh January 5, 2017 CS 2750: Machine Learning Introduction Prof. Adriana Kovashka University of Pittsburgh January 5, 2017 About the Instructor Born 1985 in Sofia, Bulgaria Got BA in 2008 at Pomona College, CA (Computer Science

More information

Introduction to Machine Learning & Its Application in Healthcare Lecture 4 Oct 3, 2018 Presentation by: Leila Karimi

Introduction to Machine Learning & Its Application in Healthcare Lecture 4 Oct 3, 2018 Presentation by: Leila Karimi Introduction to Machine Learning & Its Application in Healthcare Lecture 4 Oct 3, 2018 Presentation by: Leila Karimi 1 What Is Machine Learning? A branch of artificial intelligence, concerned with the

More information

Introduction to Machine Learning Stephen Scott, Dept of CSE

Introduction to Machine Learning Stephen Scott, Dept of CSE Introduction to Machine Learning Stephen Scott, Dept of CSE What is Machine Learning? Building machines that automatically learn from experience Sub-area of artificial intelligence (Very) small sampling

More information

CSE 258 Lecture 3. Web Mining and Recommender Systems. Supervised learning Classification

CSE 258 Lecture 3. Web Mining and Recommender Systems. Supervised learning Classification CSE 258 Lecture 3 Web Mining and Recommender Systems Supervised learning Classification Last week Last week we started looking at supervised learning problems Last week We studied linear regression, in

More information

Bias-Variance Tradeoff

Bias-Variance Tradeoff What s learning, revisited Overfitting Generative versus Discriminative Logistic Regression Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University September 19 th, 2007 Bias-Variance Tradeoff

More information

THE DESIGN OF A LEARNING SYSTEM Lecture 2

THE DESIGN OF A LEARNING SYSTEM Lecture 2 THE DESIGN OF A LEARNING SYSTEM Lecture 2 Challenge: Design a Learning System for Checkers What training experience should the system have? A design choice with great impact on the outcome Choice #1: Direct

More information

Introduction to Machine Learning

Introduction to Machine Learning Introduction to Machine Learning CS 586 Machine Learning Prepared by Jugal Kalita With help from Alpaydin s Introduction to Machine Learning and Mitchell s Machine Learning Machine Learning: Definition

More information

Welcome to CSCE 496/896: Deep Learning! Welcome to CSCE 496/896: Deep Learning! Override Policy. Override Policy. Override Policy.

Welcome to CSCE 496/896: Deep Learning! Welcome to CSCE 496/896: Deep Learning! Override Policy. Override Policy. Override Policy. Welcome to CSCE 496/896: Deep! Welcome to CSCE 496/896: Deep! Please check off your name on the roster, or write your name if you're not listed Indicate if you wish to register or sit in Policy on sit-ins:

More information

Class Overview and General Introduction to Machine Learning

Class Overview and General Introduction to Machine Learning Class Overview and General Introduction to Machine Learning Piyush Rai www.cs.utah.edu/~piyush CS5350/6350: Machine Learning August 23, 2011 (CS5350/6350) Intro to ML August 23, 2011 1 / 25 What is Machine

More information

Machine learning theory

Machine learning theory Machine learning theory Machine learning theory Introduction Hamid Beigy Sharif university of technology February 27, 2017 Hamid Beigy Sharif university of technology February 27, 2017 1 / 28 Machine learning

More information

Welcome to CMPS 142 Machine Learning

Welcome to CMPS 142 Machine Learning Welcome to CMPS 142 Machine Learning Instructor: David Helmbold, dph@soe.ucsc.edu Office hours: Tentatively after class Tu-Th 12-1:30. TA: Keshav Mathur, kemathur@ucsc.edu Web page: https://courses.soe.ucsc.edu/courses/cmps142/spring15/01

More information

CS Lecture 11. Basics of Machine Learning

CS Lecture 11. Basics of Machine Learning CS 6347 Lecture 11 Basics of Machine Learning The Course So Far What we ve seen: How to compactly model/represent joint distributions using graphical models How to solve basic inference problems Exactly:

More information

Machine Learning - Introduction

Machine Learning - Introduction Machine Learning - Introduction CSE 4309 Machine Learning Vassilis Athitsos Computer Science and Engineering Department University of Texas at Arlington 1 What is Machine Learning Quote by Tom M. Mitchell:

More information

Linear Regression: Predicting House Prices

Linear Regression: Predicting House Prices Linear Regression: Predicting House Prices I am big fan of Kalid Azad writings. He has a knack of explaining hard mathematical concepts like Calculus in simple words and helps the readers to get the intuition

More information

The Fundamentals of Machine Learning

The Fundamentals of Machine Learning The Fundamentals of Machine Learning Willie Brink 1, Nyalleng Moorosi 2 1 Stellenbosch University, South Africa 2 Council for Scientific and Industrial Research, South Africa Deep Learning Indaba 2017

More information

Introduction to Machine Learning

Introduction to Machine Learning Introduction to Machine Learning Hamed Pirsiavash CMSC 678 http://www.csee.umbc.edu/~hpirsiav/courses/ml_fall17 The slides are closely adapted from Subhransu Maji s slides Course background What is the

More information

What is Machine Learning?

What is Machine Learning? What is Machine Learning? INFO-4604, Applied Machine Learning University of Colorado Boulder August 29-31, 2017 Prof. Michael Paul Definition Murphy: a set of methods that can automatically detect patterns

More information

The Machine Learning Landscape

The Machine Learning Landscape The Machine Learning Landscape Vineet Bansal Research Software Engineer, Center for Statistics & Machine Learning vineetb@princeton.edu Oct 31, 2018 What is ML? A field of study that gives computers the

More information

CSE 190 Lecture 8. Data Mining and Predictive Analytics. Recommender Systems

CSE 190 Lecture 8. Data Mining and Predictive Analytics. Recommender Systems CSE 190 Lecture 8 Data Mining and Predictive Analytics Recommender Systems Why recommendation? The goal of recommender systems is To help people discover new content Why recommendation? The goal of recommender

More information

Machine Learning for NLP

Machine Learning for NLP Natural Language Processing SoSe 2014 Machine Learning for NLP Dr. Mariana Neves April 30th, 2014 (based on the slides of Dr. Saeedeh Momtazi) Introduction Field of study that gives computers the ability

More information

Data Classification: Advanced Concepts. Lijun Zhang

Data Classification: Advanced Concepts. Lijun Zhang Data Classification: Advanced Concepts Lijun Zhang zlj@nju.edu.cn http://cs.nju.edu.cn/zlj Outline Introduction Multiclass Learning Rare Class Learning Scalable Classification Semisupervised Learning Active

More information

SUPERVISED LEARNING. We ve finished Part I: Problem Solving We ve finished Part II: Reasoning with uncertainty. Part III: (Machine) Learning

SUPERVISED LEARNING. We ve finished Part I: Problem Solving We ve finished Part II: Reasoning with uncertainty. Part III: (Machine) Learning SUPERVISED LEARNING Progress Report We ve finished Part I: Problem Solving We ve finished Part II: Reasoning with uncertainty Part III: (Machine) Learning Supervised Learning Unsupervised Learning Overlaps

More information

36-350: Data Mining. Fall Lectures: Monday, Wednesday and Friday, 10:30 11:20, Porter Hall 226B

36-350: Data Mining. Fall Lectures: Monday, Wednesday and Friday, 10:30 11:20, Porter Hall 226B 36-350: Data Mining Fall 2009 Instructor: Cosma Shalizi, Statistics Dept., Baker Hall 229C, cshalizi@stat.cmu.edu Teaching Assistant: Joseph Richards, jwrichar@stat.cmu.edu Lectures: Monday, Wednesday

More information

Machine Learning in Patent Analytics:: Binary Classification for Prioritizing Search Results

Machine Learning in Patent Analytics:: Binary Classification for Prioritizing Search Results Machine Learning in Patent Analytics:: Binary Classification for Prioritizing Search Results Anthony Trippe Managing Director, Patinformatics, LLC Patent Information Fair & Conference November 10, 2017

More information

Machine Learning Basics

Machine Learning Basics Deep Learning Theory and Applications Machine Learning Basics Kevin Moon (kevin.moon@yale.edu) Guy Wolf (guy.wolf@yale.edu) CPSC/AMTH 663 Outline 1. What is machine learning? 2. Supervised Learning Regression

More information

Do not turn this page until you have received the signal to start. In the meantime, please read the instructions below carefully.

Do not turn this page until you have received the signal to start. In the meantime, please read the instructions below carefully. UNIVERSITY OF TORONTO FACULTY OF ARTS AND SCIENCE FINAL EXAMINATION, APRIL 2017 DURATION: 3 hours CSC 411 H1S Machine Learning and Data Mining Aids allowed: Non-programmable calculators and Aid sheets

More information

CSE 255 Lecture 7. Data Mining and Predictive Analytics. Recommender Systems

CSE 255 Lecture 7. Data Mining and Predictive Analytics. Recommender Systems CSE 255 Lecture 7 Data Mining and Predictive Analytics Recommender Systems Announcements Recommender systems are today (obviously) Assignment 1 will be out this week (I ll talk about it on Wednesday) It

More information

Python Machine Learning

Python Machine Learning Python Machine Learning Unlock deeper insights into machine learning with this vital guide to cuttingedge predictive analytics Sebastian Raschka [ PUBLISHING 1 open source I community experience distilled

More information

CSC 2515: Lecture 01: Introduction

CSC 2515: Lecture 01: Introduction CSC 2515: Lecture 01: Introduction Richard Zemel & Raquel Urtasun University of Toronto Sep 17, 2015 Zemel & Urtasun (UofT) CSC 2515: 01-Introduction Sep 17, 2015 1 / 50 Today Administration details Why

More information

Statistical Pattern Recognition

Statistical Pattern Recognition Statistical Pattern Recognition A Brief Overview of the course Hamid R. Rabiee Jafar Muhammadi, Nima Pourdamghani Spring 2012 http://ce.sharif.edu/courses/90-91/2/ce725-1/ Agenda What is a Pattern? What

More information

CSE 546 Machine Learning

CSE 546 Machine Learning CSE 546 Machine Learning Instructor: Luke Zettlemoyer TA: Lydia Chilton Slides adapted from Pedro Domingos and Carlos Guestrin Logistics Instructor: Luke Zettlemoyer Email: lsz@cs Office: CSE 658 Office

More information

Foundations of Machine Learning and Data Mining Rainer Marrone, Ralf Möller. Today s slides taken partly from E. ALPAYDIN

Foundations of Machine Learning and Data Mining Rainer Marrone, Ralf Möller. Today s slides taken partly from E. ALPAYDIN Foundations of Machine Learning and Data Mining Rainer Marrone, Ralf Möller Today s slides taken partly from E. ALPAYDIN 1 Lab Class and literature Thursday, 13:15 14:45, ES42 2589 Lab Class Fr 9:45-10:30,

More information

Machine Learning and Data Mining. Introduction. Kalev Kask 273P Spring 2018

Machine Learning and Data Mining. Introduction. Kalev Kask 273P Spring 2018 Machine Learning and Data Mining Introduction Kalev Kask 273P Spring 2018 Artificial Intelligence (AI) Building intelligent systems Lots of parts to intelligent behavior Darpa GC (Stanley) RoboCup Chess

More information

Machine Learning for Chemoinformatics An introduction

Machine Learning for Chemoinformatics An introduction Machine Learning for Chemoinformatics An introduction Francesca Grisoni University of Milano-Bicocca, Dept. of Earth and Environmental Sciences, Milan, Italy ETH Zurich, Dept. of Chemistry and Applied

More information

Introduction to Machine Learning NPFL 054

Introduction to Machine Learning NPFL 054 Introduction to Machine Learning NPFL 054 http://ufal.mff.cuni.cz/course/npfl054 Barbora Hladká hladka@ufal.mff.cuni.cz Martin Holub holub@ufal.mff.cuni.cz Charles University, Faculty of Mathematics and

More information

Machine Learning. Module 12

Machine Learning.   Module 12 Machine Learning http://datascience.tntlab.org Module 12 Today s Agenda How You're Already Using Machine Learning Models Overview of Statistical Analysis vs. Machine Learning Terminology differences Model

More information

Machine Learning. Introduction. Hamid Beigy. Sharif University of Technology. Fall 1395

Machine Learning. Introduction. Hamid Beigy. Sharif University of Technology. Fall 1395 Machine Learning Introduction Hamid Beigy Sharif University of Technology Fall 1395 Hamid Beigy (Sharif University of Technology) Machine Learning Fall 1395 1 / 15 Table of contents 1 What is machine learning?

More information

ID2223 Lecture 2: Distributed ML and Linear Regression

ID2223 Lecture 2: Distributed ML and Linear Regression ID2223 Lecture 2: Distributed ML and Linear Regression Terminology Observations. Entities used for learning/evaluation Features. Attributes (typically numeric) used to represent an observation Labels.

More information

August 29: Lecture 1

August 29: Lecture 1 August 29: Lecture 1 Machine Learning Fall 2011 Machine learning is a wide collection of concepts and techniques that allows machines to improve their performance at a given task, using 'experience' or

More information

Data Mining: Practical Machine Learning Techniques

Data Mining: Practical Machine Learning Techniques Artificial Intelligence Data Mining: Practical Machine Learning Techniques Dae-Won Kim School of Computer Science & Engineering Chung-Ang University AI Scope 1. Search-based optimization techniques for

More information

CS434 Machine Learning and Data Mining. Fall 2013

CS434 Machine Learning and Data Mining. Fall 2013 CS434 Machine Learning and Data Mining Fall 2013 1 Administrative Trivia Instructor: TA: Dr. Xiaoli Fern web.engr.oregonstate.edu/~xfern Office hour: one hour before class or by appointment Zahra Iman

More information

MACHINE LEARNING FOR DEVELOPERS A SHORT INTRODUCTION. Gregor Roth / 1&1 Mail & Media Development & Technology GmbH

MACHINE LEARNING FOR DEVELOPERS A SHORT INTRODUCTION. Gregor Roth / 1&1 Mail & Media Development & Technology GmbH MACHINE LEARNING FOR DEVELOPERS A SHORT INTRODUCTION Gregor Roth / 1&1 Mail & Media Development & Technology GmbH Software Engineer vs. Data Engineer vs. Data Scientist Software Engineer "builds applications

More information

Machine Learning Lecture 1: Introduction

Machine Learning Lecture 1: Introduction What is? Building machines that automatically learn from experience Sub-area of artificial intelligence (Very) small sampling of applications: Lecture 1: Introduction Detection of fraudulent credit card

More information

Fall 2015 COMPUTER SCIENCES DEPARTMENT UNIVERSITY OF WISCONSIN MADISON PH.D. QUALIFYING EXAMINATION

Fall 2015 COMPUTER SCIENCES DEPARTMENT UNIVERSITY OF WISCONSIN MADISON PH.D. QUALIFYING EXAMINATION Fall 2015 COMPUTER SCIENCES DEPARTMENT UNIVERSITY OF WISCONSIN MADISON PH.D. QUALIFYING EXAMINATION Artificial Intelligence Monday, September 21, 2015 GENERAL INSTRUCTIONS 1. This exam has 10 numbered

More information

Pattern Recognition Systems

Pattern Recognition Systems Pattern Recognition Systems Dr. Shuang LIANG School of Software Engineering TongJi University Fall, 2012 Today s Topics An example Pattern recognition systems The design cycle Introduction Pattern Recognition,

More information

Linear Regression. Chapter Introduction

Linear Regression. Chapter Introduction Chapter 9 Linear Regression 9.1 Introduction In this class, we have looked at a variety of di erent models and learning methods, such as finite state machines, sequence models, and classification methods.

More information

CS798: Selected topics in Machine Learning

CS798: Selected topics in Machine Learning CS798: Selected topics in Machine Learning Introduction Jakramate Bootkrajang Department of Computer Science Chiang Mai University Jakramate Bootkrajang CS798: Selected topics in Machine Learning 1 / 22

More information

Distributed Representations of Sentences and Documents. Authors: QUOC LE, TOMAS MIKOLOV Presenters: Marjan Delpisheh, Nahid Alimohammadi

Distributed Representations of Sentences and Documents. Authors: QUOC LE, TOMAS MIKOLOV Presenters: Marjan Delpisheh, Nahid Alimohammadi Distributed Representations of Sentences and Documents Authors: QUOC LE, TOMAS MIKOLOV Presenters: Marjan Delpisheh, Nahid Alimohammadi 1 Outline Objective of the paper Related works Algorithms Limitations

More information

MACHINE LEARNING WITH SAS

MACHINE LEARNING WITH SAS This webinar will be recorded. Please engage, use the Questions function during the presentation! MACHINE LEARNING WITH SAS SAS NORDIC FANS WEBINAR 21. MARCH 2017 Gert Nissen Technical Client Manager Georg

More information

ROBERTO BATTITI, MAURO BRUNATO. The LION Way: Machine Learning plus Intelligent Optimization. LIONlab, University of Trento, Italy, Apr 2015

ROBERTO BATTITI, MAURO BRUNATO. The LION Way: Machine Learning plus Intelligent Optimization. LIONlab, University of Trento, Italy, Apr 2015 ROBERTO BATTITI, MAURO BRUNATO. The LION Way: Machine Learning plus Intelligent Optimization. LIONlab, University of Trento, Italy, Apr 2015 http://intelligentoptimization.org/lionbook Roberto Battiti

More information

Lecture I Outline. Course information and details Why do machine learning? What is machine learning? Why now? Type of Learning

Lecture I Outline. Course information and details Why do machine learning? What is machine learning? Why now? Type of Learning Lecture I Outline Course information and details Why do machine learning? What is machine learning? Why now? Type of Learning Association Classification Three types: Linear, Decision Tree, and Nearest

More information

Handout 2 More Similarity Searching; Multidimensional Scaling

Handout 2 More Similarity Searching; Multidimensional Scaling Handout 2 More Similarity Searching; Multidimensional Scaling 36-350: Data Mining August 30, 2006 Reading: Principles of Data Mining, sec. 14.3 (skip 14.3.3 for now) and 14.4. Let s recap similarity searching

More information

Project organization. Exam Details Wed 3/7/18. Approximation-generalization tradeoff. Approximation-generalization tradeoff.

Project organization. Exam Details Wed 3/7/18. Approximation-generalization tradeoff. Approximation-generalization tradeoff. Project organization Exam Details Wed 3/7/18 Project proposals due March 14 (~1.5 weeks) I would like to make sure everyone has a team, so I want to add a new deadline By TODAY please go to the link posted

More information

Text Classification with Machine Learning Algorithms

Text Classification with Machine Learning Algorithms 2013, TextRoad Publication ISSN 2090-4304 Journal of Basic and Applied Scientific Research www.textroad.com Text Classification with Machine Learning Algorithms Nasim VasfiSisi 1 and Mohammad Reza Feizi

More information

Comparison of Classification Algorithms Using Machine Learning

Comparison of Classification Algorithms Using Machine Learning Comparison of Classification Algorithms Using Machine Learning Ankta Pal 1, Neelesh Shrivastava 2, Pradeep Tripathi 3 M.Tech Scholar, Department of Computer Science & Engineering, VITS Satna, (M.P), India,

More information

SB2b Statistical Machine Learning Hilary Term 2017

SB2b Statistical Machine Learning Hilary Term 2017 SB2b Statistical Machine Learning Hilary Term 2017 Mihaela van der Schaar and Seth Flaxman Guest lecturer: Yee Whye Teh Department of Statistics Oxford Slides and other materials available at: http://www.oxford-man.ox.ac.uk/~mvanderschaar/home_

More information

ECE521 Lecture10 Deep Learning

ECE521 Lecture10 Deep Learning ECE521 Lecture10 Deep Learning Learning fully connected multi-layer neural networks For a single data point, we can write the the hidden activations of the fully connected neural network as a recursive

More information

Introducing Machine Learning

Introducing Machine Learning Introducing Machine Learning What is Machine Learning? Machine learning teaches computers to do what comes naturally to humans and animals: learn from experience. Machine learning algorithms use computational

More information