Introduction to Computational Linguistics

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Introduction to Computational Linguistics"

Transcription

1 Introduction to Computational Linguistics Olga Zamaraeva (2018) Based on Guestrin (2013) University of Washington April 10, / 30

2 This and last lecture: bird s eye view Next lecture: understand precision & recall in detail Coming next week: N-grams, then CFG will also look in more detail Some more bird s eye topics later in the course 2 / 30

3 What is? Study of algorithms that: improve their performance at some task with experience Data ML Understanding Note: understanding is more general than e.g. linguistics or speech. (This is where my the distinction between and NLP comes in, and that s why NLP is more closely associated with ML.) 3 / 30

4 ML tasks: Classification From data to discrete labels Spam filtering Text classification Object detection Weather prediction (e.g. rain, snow...) Sentiment analysis etc. 4 / 30

5 ML tasks: Regression Predict a numeric value Stock market Weather prediction (temperature) Predict final scores given comments in the code :) 5 / 30

6 ML tasks: Similarity Finding data Given image, find similar ones Similar products, songs... Similar texts Similar words... 6 / 30

7 Clustering (Unsupervised learning) Group similar things together 7 / 30

8 Embedding Representing data (e.g. images) 8 / 30

9 Embedding Representing data (e.g. words) 9 / 30

10 Reinforcement Learning Training by feedback Have an agent: make sensor observations select action receive rewards compute a strategy to maximize expected rewards balance immediate reward and exploration pic from: 10 / 30

11 Neural Nets and Deep Learning picture from: 11 / 30

12 Data in ML is all about finding patterns in the Data typically, Big data is required Training data find patterns in the data train a function to minimize mistakes (learn) Development data Test data Tune the parameters Perform error analysis Never ever learn on the test data (why?) But even if you never look at your test data but keep using the same test data... The case of Wall Street Journal, section / 30

13 Decision function: separates data points I Train the function on the training data I Given a new ( test ) point, know which side of the DF I parameters, e.g. feature weights are optimized I which θ = P (heads ) makes the data HHHTT most probable I or: find such a vector w that the decision function φ(w1 f1, w2 f2, w3 f3 ) makes least mistakes I (f1 is the value for feature 1, e.g. yesterday s t ) pic from: Koprowski et al. (2012) 13 / 30

14 Features Extract informative features (e.g. whiskers length, ear size...) Turn them into numbers somehow How important is each feature? (weights) Given the below training dataset, how important is whiskers length? what about color? what about tail position? 14 / 30

15 Loss function On the training data, observe the true label (value) and penalize the mistakes 15 / 30

16 Bias-Variance Tradeoff 16 / 30

17 Fundamental questions in ML (according to Mitchel, 2017) How can computers improve performance through experience? Which theoretical laws govern learning systems? Think again about what NLP s fundamental questions are What about Linguistics fundamental questions? Acquisition (the Holy Grail, for some linguists?) 17 / 30

18 ML perspectives ML as optimization E.g. optimize a loss function to get better predictions ML as probabilistic inference E.g. derive a function that makes the data most probable (Recall MLE (maximum likelihood estimation)) ML as parametric programming E.g. Deep Learning networks instantiate a specific program out of a set of possible programs ML as evolutionary search :) Is evolution a ML phenomenon? Think again about the research question of NLP...we want to understand something about the world through language / 30

19 ML: Key results No free lunch...no system has any basis to reliably classify new examples that go beyond those it has already seen... Three sources of error: Bias, variance, and unavoidable error: Overfitting some probability of us being wrong When True error > Train error What is the relationship between True error and Test error? 19 / 30

20 Overfitting 20 / 30

21 Bayesian Networks and Graphical Models Discover some structure in and analyze complex data distributions 21 / 30

22 Discriminative and Generative models Generative: Learn joint distribution P(x,y) (from which conditional can be inferred) Need to make more assumptions Can generate data (x,y) Based on my generation assumptions, which category is most likely to generate this observation? Example: Naive Bayes classifier, HMM Discriminative: Learn conditional probability directly (P(y x) ) Need fewer constraints/assumptions Which class to predict given observation? Does not care about how the data was generated Example: Logistic Regression (Maximum Entropy classifier) While Generative models sound more generally useful, discriminative often perform better 22 / 30

23 Discriminative and Generative models x=1: cat goes outside; x=0: cat stays indoors y=1: cat catches mouse; y=0 cat does not catch mouse Observe the cat for 10 days and get the following data (x,y): (0,1), (0,0) (0,0) (1,0) (1,0) (1,0) (1,1) (1,1) (1,1) (1,1) Joint probability of both events happening: P(x,y): y=0 y=1 x= x= Conditional P(y x): choice of x value is fixed: y=0 y=1 x= x= Now suppose you want to artificially create more observations (e.g. for a computer game about a cat). If you generate N more observations using P(x,y), will you end up with the same probabilities of events? Can you use P(y x)? But how to determine how many times X was equal to 0 and to 1? 23 / 30

24 Deep Neural Networks A family of ML algorithms where simple units are combined to perform a larger computation Simultaneously train millions of parameters (for all simple units) Development in types of units used LSTM (Long Short Term Memory) units Structural questions are asked about the data But how well can we tell what is going on in the end? Specific architectures for specific problems Good performance...in domain How to generalize knowledge from here? Representation learning Learn new representation of data in hidden layers E.g. progress in relating text to images 24 / 30

25 Other issues PAC learning theory (upper error bounds) Ensemble learning Semi-supervised learning and Active learning Kernel methods (changing dimensionality of data) Reinforcement learning 25 / 30

26 Where is ML headed next? Will ML change the way we think about human learning? Human-machine (learning) interaction ML by reading Note that both directions involve natural language understanding 26 / 30

27 ML & NLP Natural Language Understanding (NLU) in demand in the industry At the same time, NLP is behind e.g. machine vision (in using deep learning) Researchers are after performance improvement on classic tasks, as well as defining new interesting tasks, as well as after understanding how learning works, through language input ML is the dominant paradigm in today s NLP 27 / 30

28 ML & ML is often employed for automatically tagging data e.g. to get access to larger annotated corpora (bootstrap from smaller sample) Is it likely to discover something linguistically valid about language via statistics? Definitely! But can we learn everything we want this way? The question of how learning happens is equally interesting to most linguists...however, most related questions go beyond well-defined linguistic theories Note that e.g. NLU as defined in NLP is not strictly speaking a linguistic task E.g. semantic theory is not trying to model world knowledge ML is not the dominant paradigm in Linguistics 28 / 30

29 Python libraries for ML comes with good documentation and (usually small) examples! analytics/ working with text data.html sample datasets and sample code are available 29 / 30

30 What you need to know ML fundamental questions Training, Dev, Test data Decision vs. Loss function (what are they for) Overfitting, Sources of error No free lunch Regression vs. Classification Optimization (why is it an important perspective) Features and feature weights (what is their role) Role of conditional probability (why is it used) Difference between conditional and joint probability in terms of generating data 30 / 30

Introduction to Machine Learning 1. Nov., 2018 D. Ratner SLAC National Accelerator Laboratory

Introduction to Machine Learning 1. Nov., 2018 D. Ratner SLAC National Accelerator Laboratory Introduction to Machine Learning 1 Nov., 2018 D. Ratner SLAC National Accelerator Laboratory Introduction What is machine learning? Arthur Samuel (1959): Ability to learn without being explicitly programmed

More information

Practical Advice for Building Machine Learning Applications

Practical Advice for Building Machine Learning Applications Practical Advice for Building Machine Learning Applications Machine Learning Fall 2017 Based on lectures and papers by Andrew Ng, Pedro Domingos, Tom Mitchell and others 1 This lecture: ML and the world

More information

Course Overview and Introduction CE-717 : Machine Learning Sharif University of Technology. M. Soleymani Fall 2012

Course Overview and Introduction CE-717 : Machine Learning Sharif University of Technology. M. Soleymani Fall 2012 Course Overview and Introduction CE-717 : Machine Learning Sharif University of Technology M. Soleymani Fall 2012 Course Info Instructor: Mahdieh Soleymani Email: soleyman@ce.sharif.edu Lectures: Sun-Tue

More information

Key Ideas in Machine Learning

Key Ideas in Machine Learning CHAPTER 14 Key Ideas in Machine Learning Machine Learning Copyright c 2017. Tom M. Mitchell. All rights reserved. *DRAFT OF December 4, 2017* *PLEASE DO NOT DISTRIBUTE WITHOUT AUTHOR S PERMISSION* This

More information

PAC Learning Introduction to Machine Learning. Matt Gormley Lecture 14 March 5, 2018

PAC Learning Introduction to Machine Learning. Matt Gormley Lecture 14 March 5, 2018 10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University PAC Learning Matt Gormley Lecture 14 March 5, 2018 1 ML Big Picture Learning Paradigms:

More information

CS 6375 Advanced Machine Learning (Qualifying Exam Section) Nicholas Ruozzi University of Texas at Dallas

CS 6375 Advanced Machine Learning (Qualifying Exam Section) Nicholas Ruozzi University of Texas at Dallas CS 6375 Advanced Machine Learning (Qualifying Exam Section) Nicholas Ruozzi University of Texas at Dallas Slides adapted from David Sontag and Vibhav Gogate Course Info. Instructor: Nicholas Ruozzi Office:

More information

CSE 446 Machine Learning

CSE 446 Machine Learning CSE 446 Machine What is Machine? Daniel Weld Xiao Ling Congle Zhang 1 2 Machine Study of algorithms that improve their performance at some task with experience Why? Data Machine Understanding Is this topic

More information

Bias-Variance Tradeoff

Bias-Variance Tradeoff What s learning, revisited Overfitting Generative versus Discriminative Logistic Regression Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University September 19 th, 2007 Bias-Variance Tradeoff

More information

Course Overview and Introduction CE-717 : Machine Learning Sharif University of Technology. M. Soleymani Fall 2014

Course Overview and Introduction CE-717 : Machine Learning Sharif University of Technology. M. Soleymani Fall 2014 Course Overview and Introduction CE-717 : Machine Learning Sharif University of Technology M. Soleymani Fall 2014 Course Info Instructor: Mahdieh Soleymani Email: soleymani@sharif.edu Lectures: Sun-Tue

More information

An Introduction to Machine Learning

An Introduction to Machine Learning MindLAB Research Group - Universidad Nacional de Colombia Introducción a los Sistemas Inteligentes Outline 1 2 What s machine learning History Supervised learning Non-supervised learning 3 Observation

More information

Machine Learning for Computer Vision

Machine Learning for Computer Vision Computer Group Prof. Daniel Cremers Machine Learning for Computer PD Dr. Rudolph Triebel Lecturers PD Dr. Rudolph Triebel rudolph.triebel@in.tum.de Room number 02.09.059 Main lecture MSc. Ioannis John

More information

The Fundamentals of Machine Learning

The Fundamentals of Machine Learning The Fundamentals of Machine Learning Willie Brink 1, Nyalleng Moorosi 2 1 Stellenbosch University, South Africa 2 Council for Scientific and Industrial Research, South Africa Deep Learning Indaba 2017

More information

Deep Learning for Computer Vision. commercial-in-confidence

Deep Learning for Computer Vision. commercial-in-confidence Deep Learning for Computer Vision Introduction to Computer Vision & Deep Learning Presented by Hayden Faulkner What Is Computer Vision? What is Computer Vision? Using computers to understand (process)

More information

Applied Machine Learning

Applied Machine Learning Applied Spring 2018, CS 519 Prof. Liang Huang School of EECS Oregon State University liang.huang@oregonstate.edu is Everywhere A breakthrough in machine learning would be worth ten Microsofts (Bill Gates)

More information

Structured Output Prediction

Structured Output Prediction Structured Output Prediction CS4780/5780 Machine Learning Fall 2011 Thorsten Joachims Cornell University Reading: T. Joachims, T. Hofmann, Yisong Yue, Chun-Nam Yu, Predicting Structured Objects with Support

More information

Machine Learning: Summary

Machine Learning: Summary Machine Learning: Summary Greg Grudic CSCI-4830 Machine Learning 1 What is Machine Learning? The goal of machine learning is to build computer systems that can adapt and learn from their experience. Tom

More information

Programming Social Robots for Human Interaction. Lecture 4: Machine Learning and Pattern Recognition

Programming Social Robots for Human Interaction. Lecture 4: Machine Learning and Pattern Recognition Programming Social Robots for Human Interaction Lecture 4: Machine Learning and Pattern Recognition Zheng-Hua Tan Dept. of Electronic Systems, Aalborg Univ., Denmark zt@es.aau.dk, http://kom.aau.dk/~zt

More information

Artificial Intelligence Introduction to Machine Learning

Artificial Intelligence Introduction to Machine Learning Artificial Intelligence Introduction to Machine Learning Artificial Intelligence Chung-Ang University Narration: Prof. Jaesung Lee Introduction Applications which Machine Learning techniques play an important

More information

Session 1: Gesture Recognition & Machine Learning Fundamentals

Session 1: Gesture Recognition & Machine Learning Fundamentals IAP Gesture Recognition Workshop Session 1: Gesture Recognition & Machine Learning Fundamentals Nicholas Gillian Responsive Environments, MIT Media Lab Tuesday 8th January, 2013 My Research My Research

More information

SUPERVISED LEARNING. We ve finished Part I: Problem Solving We ve finished Part II: Reasoning with uncertainty. Part III: (Machine) Learning

SUPERVISED LEARNING. We ve finished Part I: Problem Solving We ve finished Part II: Reasoning with uncertainty. Part III: (Machine) Learning SUPERVISED LEARNING Progress Report We ve finished Part I: Problem Solving We ve finished Part II: Reasoning with uncertainty Part III: (Machine) Learning Supervised Learning Unsupervised Learning Overlaps

More information

Class Overview and General Introduction to Machine Learning

Class Overview and General Introduction to Machine Learning Class Overview and General Introduction to Machine Learning Piyush Rai www.cs.utah.edu/~piyush CS5350/6350: Machine Learning August 23, 2011 (CS5350/6350) Intro to ML August 23, 2011 1 / 25 What is Machine

More information

Course Overview Introduction to Machine Learning. Matt Gormley Lecture 1 January 17, 2018

Course Overview Introduction to Machine Learning. Matt Gormley Lecture 1 January 17, 2018 10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University Course Overview Matt Gormley Lecture 1 January 17, 2018 1 WHAT IS MACHINE LEARNING?

More information

INTRODUCTION TO DATA SCIENCE

INTRODUCTION TO DATA SCIENCE DATA11001 INTRODUCTION TO DATA SCIENCE EPISODE 6: MACHINE LEARNING TODAY S MENU 1. WHAT IS ML? 2. CLASSIFICATION AND REGRESSSION 3. EVALUATING PERFORMANCE & OVERFITTING WHAT IS MACHINE LEARNING? Definition:

More information

Machine Learning: Preliminaries & Overview

Machine Learning: Preliminaries & Overview Machine Learning: Preliminaries & Overview Winter 2018 LOL What is machine learning? Textbook definitions of machine learning : Detecting patterns and regularities with a good and generalizable approximation

More information

Machine Learning. Lecture 1: Introduction to Machine Learning. Nevin L. Zhang

Machine Learning. Lecture 1: Introduction to Machine Learning. Nevin L. Zhang Machine Learning Lecture 1: Introduction to Machine Learning Nevin L. Zhang lzhang@cse.ust.hk Department of Computer Science and Engineering The Hong Kong University of Science and Technology This set

More information

Lecture 1: Introduction to Machine Learning

Lecture 1: Introduction to Machine Learning Statistical Methods for Intelligent Information Processing (SMIIP) Lecture 1: Introduction to Machine Learning Shuigeng Zhou School of Computer Science September 13, 2017 What is machine learning? Machine

More information

CSE 446 Sequences, Conclusions

CSE 446 Sequences, Conclusions CSE 446 Sequences, Conclusions Administrative Final exam next week Wed Jun 8 8:30 am Last office hours after class today Sequence Models High level overview of structured data What kind of structure? Temporal

More information

Introduction. Binary Classification and Bayes Error.

Introduction. Binary Classification and Bayes Error. CIS 520: Machine Learning Spring 2018: Lecture 1 Introduction Binary Classification and Bayes Error Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture They

More information

CS6375: Recap. Nicholas Ruozzi University of Texas at Dallas

CS6375: Recap. Nicholas Ruozzi University of Texas at Dallas CS6375: Recap Nicholas Ruozzi University of Texas at Dallas Supervised Learning Regression & classification Discriminative methods k-nn Decision trees Perceptron SVMs & kernel methods Logistic regression

More information

CS534 Machine Learning

CS534 Machine Learning CS534 Machine Learning Spring 2013 Lecture 1: Introduction to ML Course logistics Reading: The discipline of Machine learning by Tom Mitchell Course Information Instructor: Dr. Xiaoli Fern Kec 3073, xfern@eecs.oregonstate.edu

More information

Lecture 10 Summary and reflections

Lecture 10 Summary and reflections Lecture 10 Summary and reflections Niklas Wahlström Division of Systems and Control Department of Information Technology Uppsala University. Email: niklas.wahlstrom@it.uu.se SML - Lecture 10 Contents Lecture

More information

Course Overview and Introduction CE-717 : Machine Learning Sharif University of Technology. M. Soleymani Fall 2016

Course Overview and Introduction CE-717 : Machine Learning Sharif University of Technology. M. Soleymani Fall 2016 Course Overview and Introduction CE-717 : Machine Learning Sharif University of Technology M. Soleymani Fall 2016 Course Info Instructor: Mahdieh Soleymani Email: soleymani@sharif.edu Lectures: Sun-Tue

More information

Introduction to Machine Learning & Its Application in Healthcare Lecture 4 Oct 3, 2018 Presentation by: Leila Karimi

Introduction to Machine Learning & Its Application in Healthcare Lecture 4 Oct 3, 2018 Presentation by: Leila Karimi Introduction to Machine Learning & Its Application in Healthcare Lecture 4 Oct 3, 2018 Presentation by: Leila Karimi 1 What Is Machine Learning? A branch of artificial intelligence, concerned with the

More information

What is Machine Learning? Machine Learning Fall 2018

What is Machine Learning? Machine Learning Fall 2018 What is Machine Learning? Machine Learning Fall 2018 1 Our goal today And through the semester What is (machine) learning? 2 Let s play a game 3 The badges game Attendees of the 1994 conference on Computational

More information

Machine Learning Nanodegree Syllabus

Machine Learning Nanodegree Syllabus Machine Learning Nanodegree Syllabus Artificial Neural Networks, TensorFlow, and Machine Learning Algorithms Before You Start Prerequisites: In order to succeed in this program, we recommend having experience

More information

DS Machine Learning and Data Mining I. Alina Oprea Associate Professor, CCIS Northeastern University

DS Machine Learning and Data Mining I. Alina Oprea Associate Professor, CCIS Northeastern University DS 4400 Machine Learning and Data Mining I Alina Oprea Associate Professor, CCIS Northeastern University January 10 2019 Class Outline Introduction 1 week Probability and linear algebra review Supervised

More information

SB2b Statistical Machine Learning Hilary Term 2017

SB2b Statistical Machine Learning Hilary Term 2017 SB2b Statistical Machine Learning Hilary Term 2017 Mihaela van der Schaar and Seth Flaxman Guest lecturer: Yee Whye Teh Department of Statistics Oxford Slides and other materials available at: http://www.oxford-man.ox.ac.uk/~mvanderschaar/home_

More information

Machine Learning: CS 6375 Introduction. Instructor: Vibhav Gogate The University of Texas at Dallas

Machine Learning: CS 6375 Introduction. Instructor: Vibhav Gogate The University of Texas at Dallas Machine Learning: CS 6375 Introduction Instructor: Vibhav Gogate The University of Texas at Dallas Logistics Instructor: Vibhav Gogate Email: vgogate@hlt.utdallas.edu Office: ECSS 3.406 Office hours: M/W

More information

CSC 411: Introduction to Machine Learning

CSC 411: Introduction to Machine Learning CSC 411: Introduction to Machine Learning Lecture 1 - Introduction Roger Grosse, Amir-massoud Farahmand, and Juan Carrasquilla University of Toronto (UofT) CSC411-Lec1 1 / 28 This course Broad introduction

More information

(Refer Slide Time: 0:33)

(Refer Slide Time: 0:33) Machine Learning for Engineering and Science Applications. Professor Dr. Balaji Srinivasan. Department of Mechanical Engineering. Indian Institute of Technology, Madras. Overview of Machine Learning. We

More information

ECS171: Machine Learning

ECS171: Machine Learning ECS171: Machine Learning Lecture 1: Overview of class, LFD 1.1, 1.2 Cho-Jui Hsieh UC Davis Jan 8, 2018 Course Information Website: http://www.stat.ucdavis.edu/~chohsieh/teaching/ ECS171_Winter2018/main.html

More information

COMP 551 Applied Machine Learning Lecture 11: Ensemble learning

COMP 551 Applied Machine Learning Lecture 11: Ensemble learning COMP 551 Applied Machine Learning Lecture 11: Ensemble learning Instructor: Herke van Hoof (herke.vanhoof@mcgill.ca) Slides mostly by: (jpineau@cs.mcgill.ca) Class web page: www.cs.mcgill.ca/~hvanho2/comp551

More information

Text Classification and Convolutional Neural Networks

Text Classification and Convolutional Neural Networks Text Classification and Convolutional Neural Networks COSC 7336: Advanced Natural Language Processing Fall 2017 Some content on these slides was borrowed from J&M Today s lecture Text Classification: task

More information

M. R. Ahmadzadeh Isfahan University of Technology. M. R. Ahmadzadeh Isfahan University of Technology

M. R. Ahmadzadeh Isfahan University of Technology. M. R. Ahmadzadeh Isfahan University of Technology 1 2 M. R. Ahmadzadeh Isfahan University of Technology Ahmadzadeh@cc.iut.ac.ir M. R. Ahmadzadeh Isfahan University of Technology Textbooks 3 Introduction to Machine Learning - Ethem Alpaydin Pattern Recognition

More information

Machine Learning Tom M. Mitchell Machine Learning Department Carnegie Mellon University. January 11, 2011

Machine Learning Tom M. Mitchell Machine Learning Department Carnegie Mellon University. January 11, 2011 Machine Learning 10-701 Tom M. Mitchell Machine Learning Department Carnegie Mellon University January 11, 2011 Today: What is machine learning? Decision tree learning Course logistics Readings: The Discipline

More information

Fall 2015 COMPUTER SCIENCES DEPARTMENT UNIVERSITY OF WISCONSIN MADISON PH.D. QUALIFYING EXAMINATION

Fall 2015 COMPUTER SCIENCES DEPARTMENT UNIVERSITY OF WISCONSIN MADISON PH.D. QUALIFYING EXAMINATION Fall 2015 COMPUTER SCIENCES DEPARTMENT UNIVERSITY OF WISCONSIN MADISON PH.D. QUALIFYING EXAMINATION Artificial Intelligence Monday, September 21, 2015 GENERAL INSTRUCTIONS 1. This exam has 10 numbered

More information

Electronics & ICT Academy (Under Ministry of Electronics and Information Technology (MeitY), Govt. of India)

Electronics & ICT Academy (Under Ministry of Electronics and Information Technology (MeitY), Govt. of India) Electronics & ICT Academy (Under Ministry of Electronics and Information Technology (MeitY), Govt. of India) Indian Institute of Technology Guwahati, Guwahati, Assam, Pin 781039 Phone: +91-361-2583182/3009,

More information

ECE521 Lecture1. Introduction

ECE521 Lecture1. Introduction ECE521 Lecture1 Introduction Outline History of machine learning Types of machine learning problems What is machine learning? A scientific field is best defined by the central question it studies. The

More information

Welcome to CMPS 142 Machine Learning

Welcome to CMPS 142 Machine Learning Welcome to CMPS 142 Machine Learning Instructor: David Helmbold, dph@soe.ucsc.edu Office hours: Tentatively after class Tu-Th 12-1:30. TA: Keshav Mathur, kemathur@ucsc.edu Web page: https://courses.soe.ucsc.edu/courses/cmps142/spring15/01

More information

Lecture 2: Mixing Compositional Semantics and Machine Learning

Lecture 2: Mixing Compositional Semantics and Machine Learning Lecture 2: Mixing Compositional Semantics and Machine Learning Kyle Richardson kyle@ims.uni-stuttgart.de April 14, 2016 Plan main paper: Liang and Potts 2015 (conceptual basis of class) secondary: Mooney

More information

Machine Learning for Computer Vision

Machine Learning for Computer Vision Prof. Daniel Cremers Machine Learning for Computer PD Dr. Rudolph Triebel Lecturers PD Dr. Rudolph Triebel rudolph.triebel@in.tum.de Room number 02.09.058 (Fridays) Main lecture MSc. Ioannis John Chiotellis

More information

Machine learning theory

Machine learning theory Machine learning theory Machine learning theory Introduction Hamid Beigy Sharif university of technology February 27, 2017 Hamid Beigy Sharif university of technology February 27, 2017 1 / 28 Machine learning

More information

On Machine Learning. Aggelos K. Katsaggelos

On Machine Learning. Aggelos K. Katsaggelos On Machine Learning Aggelos K. Katsaggelos Joseph Cummings Professor Northwestern University Department of EECS Department of Linguistics Argonne National Laboratory NorthShore University Health System

More information

STA 414/2104 Statistical Methods for Machine Learning and Data Mining

STA 414/2104 Statistical Methods for Machine Learning and Data Mining STA 414/2104 Statistical Methods for Machine Learning and Data Mining Radford M. Neal, University of Toronto, 2014 Week 1 What are Machine Learning and Data Mining? Typical Machine Learning and Data Mining

More information

Machine Learning Nanodegree Syllabus

Machine Learning Nanodegree Syllabus Machine Learning Nanodegree Syllabus Artificial Neural Networks, TensorFlow, and Machine Learning Algorithms Before You Start Prerequisites: In order to succeed in this program, we recommend having experience

More information

Linear Models Continued: Perceptron & Logistic Regression

Linear Models Continued: Perceptron & Logistic Regression Linear Models Continued: Perceptron & Logistic Regression CMSC 723 / LING 723 / INST 725 Marine Carpuat Slides credit: Graham Neubig, Jacob Eisenstein Linear Models for Classification Feature function

More information

Lecture 1: Machine Learning Basics

Lecture 1: Machine Learning Basics 1/69 Lecture 1: Machine Learning Basics Ali Harakeh University of Waterloo WAVE Lab ali.harakeh@uwaterloo.ca May 1, 2017 2/69 Overview 1 Learning Algorithms 2 Capacity, Overfitting, and Underfitting 3

More information

Introduction to Machine Learning for NLP I

Introduction to Machine Learning for NLP I Introduction to Machine Learning for NLP I Benjamin Roth CIS LMU München Benjamin Roth (CIS LMU München) Introduction to Machine Learning for NLP I 1 / 49 Outline 1 This Course 2 Overview 3 Machine Learning

More information

Machine Learning for Computer Vision

Machine Learning for Computer Vision Prof. Daniel Cremers Machine Learning for Computer PD Dr. Rudolph Triebel Lecturers PD Dr. Rudolph Triebel rudolph.triebel@in.tum.de Room number 02.09.059 (Fridays) Main lecture MSc. Ioannis John Chiotellis

More information

Machine Learning: CS 6375 Introduction. Instructor: Vibhav Gogate The University of Texas at Dallas

Machine Learning: CS 6375 Introduction. Instructor: Vibhav Gogate The University of Texas at Dallas Machine Learning: CS 6375 Introduction Instructor: Vibhav Gogate The University of Texas at Dallas Logistics Instructor: Vibhav Gogate Email: Vibhav.Gogate@utdallas.edu Office: ECSS 3.406 Office hours:

More information

Harivinod N Dept of CSE Vivekananda College of Engineering Technology, Puttur

Harivinod N Dept of CSE Vivekananda College of Engineering Technology, Puttur 15CS73, VTU CBCS Scheme By Dept of CSE Vivekananda College of Engineering Technology, Puttur What is Learning? Learning - improve automatically with experience Using past experiences to improve future

More information

Era of AI (Deep Learning) and harnessing its true potential

Era of AI (Deep Learning) and harnessing its true potential Era of AI (Deep Learning) and harnessing its true potential Artificial Intelligence (AI) AI Augments our brain with infallible memories and infallible calculators Humans and Computers have become a tightly

More information

Statistical Approaches to Natural Language Processing CS 4390/5319 Spring Semester, 2003 Syllabus

Statistical Approaches to Natural Language Processing CS 4390/5319 Spring Semester, 2003 Syllabus Statistical Approaches to Natural Language Processing CS 4390/5319 Spring Semester, 2003 Syllabus http://www.cs.utep.edu/nigel/nlp.html Time and Location 15:00 16:25, Tuesdays and Thursdays Computer Science

More information

Unsupervised Learning: Clustering

Unsupervised Learning: Clustering Unsupervised Learning: Clustering Vibhav Gogate The University of Texas at Dallas Slides adapted from Carlos Guestrin, Dan Klein & Luke Zettlemoyer Machine Learning Supervised Learning Unsupervised Learning

More information

INTRODUCTION TO MACHINE LEARNING SOME CONTENT COURTESY OF PROFESSOR ANDREW NG OF STANFORD UNIVERSITY

INTRODUCTION TO MACHINE LEARNING SOME CONTENT COURTESY OF PROFESSOR ANDREW NG OF STANFORD UNIVERSITY INTRODUCTION TO MACHINE LEARNING SOME CONTENT COURTESY OF PROFESSOR ANDREW NG OF STANFORD UNIVERSITY IQS2: Spring 2013 Machine Learning Definition 2 Arthur Samuel (1959). Machine Learning: Field of study

More information

n Learning is useful as a system construction method n Examples of systems that employ ML? q Supervised learning: correct answers for each example

n Learning is useful as a system construction method n Examples of systems that employ ML? q Supervised learning: correct answers for each example Learning Learning from Data Russell and Norvig Chapter 18 Essential for agents working in unknown environments Learning is useful as a system construction method q Expose the agent to reality rather than

More information

MACHINE LEARNING. Slide adapted from learning from data book and course, and Berkeley cs188 by Dan Klein, and Pieter Abbeel

MACHINE LEARNING. Slide adapted from learning from data book and course, and Berkeley cs188 by Dan Klein, and Pieter Abbeel MACHINE LEARNING Slide adapted from learning from data book and course, and Berkeley cs188 by Dan Klein, and Pieter Abbeel Machine Learning?? Learning from data Tasks: Prediction Classification Recognition

More information

Midterm Exam Review Introduction to Machine Learning. Matt Gormley Lecture 14 March 6, 2017

Midterm Exam Review Introduction to Machine Learning. Matt Gormley Lecture 14 March 6, 2017 10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University Midterm Exam Review Matt Gormley Lecture 14 March 6, 2017 1 Reminders Midterm Exam

More information

CPSC 340: Machine Learning and Data Mining. Course Review/Preview Fall 2015

CPSC 340: Machine Learning and Data Mining. Course Review/Preview Fall 2015 CPSC 340: Machine Learning and Data Mining Course Review/Preview Fall 2015 Admin Assignment 6 due now. We will have office hours as usual next week. Final exam details: December 15: 8:30-11 (WESB 100).

More information

Postgraduate Certificate in Data Analysis and Pattern Recognition

Postgraduate Certificate in Data Analysis and Pattern Recognition Postgraduate Certificate in Data Analysis and Pattern Recognition 1 of Certificate: Postgraduate Certificate in Data Analysis and Pattern Recognition 1.1 of Award: Postgraduate Certificate in Data Analysis

More information

A Literature Review of Domain Adaptation with Unlabeled Data

A Literature Review of Domain Adaptation with Unlabeled Data A Literature Review of Domain Adaptation with Unlabeled Data Anna Margolis amargoli@u.washington.edu March 23, 2011 1 Introduction 1.1 Overview In supervised learning, it is typically assumed that the

More information

Machine Learning ICS 273A. Instructor: Max Welling

Machine Learning ICS 273A. Instructor: Max Welling Machine Learning ICS 273A Instructor: Max Welling Class Homework What is Expected? Required, (answers will be provided) A Project See webpage Quizzes A quiz every Friday Bring scantron form (buy in UCI

More information

Introduction to Machine Learning

Introduction to Machine Learning Introduction to Machine Learning CSCI 1950-F Instructors: Erik Sudderth & Mark Johnson Graduate TA: Deqing Sun Undergraduate TAs: Max Barrows & Evan Donahue Visual Object Recognition sky skyscraper sky

More information

Ensemble Learning CS534

Ensemble Learning CS534 Ensemble Learning CS534 Ensemble Learning How to generate ensembles? There have been a wide range of methods developed We will study to popular approaches Bagging Boosting Both methods take a single (base)

More information

Distributed Representations of Sentences and Documents. Authors: QUOC LE, TOMAS MIKOLOV Presenters: Marjan Delpisheh, Nahid Alimohammadi

Distributed Representations of Sentences and Documents. Authors: QUOC LE, TOMAS MIKOLOV Presenters: Marjan Delpisheh, Nahid Alimohammadi Distributed Representations of Sentences and Documents Authors: QUOC LE, TOMAS MIKOLOV Presenters: Marjan Delpisheh, Nahid Alimohammadi 1 Outline Objective of the paper Related works Algorithms Limitations

More information

Covariate Shift Consequences and good. practice Covariate shift, re-weight training data, active sampling. Joyce Wang Software Engineer Sep 2017

Covariate Shift Consequences and good. practice Covariate shift, re-weight training data, active sampling. Joyce Wang Software Engineer Sep 2017 Covariate Shift Consequences and good practice Covariate shift, re-weight training data, active sampling Joyce Wang Software Engineer Sep 2017 www.csiro.au Motivation Validation Accuracy = 0.96 Query Accuracy

More information

Lecture 7: Distributed Representations

Lecture 7: Distributed Representations Lecture 7: Distributed Representations Roger Grosse 1 Introduction We ll take a break from derivatives and optimization, and look at a particular example of a neural net that we can train using backprop:

More information

Deanonymizing Quora Answers

Deanonymizing Quora Answers Deanonymizing Quora Answers Pranav Jindal pranavj@stanford.edu Paranjape, Ashwin ashwinpp@stanford.edu 1 Introduction Quora is a knowledge sharing website where users can ask/answer questions with the

More information

ECE 6254 Statistical Machine Learning Spring 2017

ECE 6254 Statistical Machine Learning Spring 2017 ECE 6254 Statistical Machine Learning Spring 2017 Mark A. Davenport Georgia Institute of Technology School of Electrical and Computer Engineering Statistical machine learning How can we learn effective

More information

Multilingual. Language Processing. Applications. Natural

Multilingual. Language Processing. Applications. Natural Multilingual Natural Language Processing Applications Contents Preface xxi Acknowledgments xxv About the Authors xxvii Part I In Theory 1 Chapter 1 Finding the Structure of Words 3 1.1 Words and Their

More information

Introduction to Machine Learning (CSCI-UA )

Introduction to Machine Learning (CSCI-UA ) Introduction to Machine Learning (CSCI-UA.0480-007) David Sontag New York University Slides adapted from Luke Zettlemoyer, Pedro Domingos, and Carlos Guestrin Logistics Class webpage: http://cs.nyu.edu/~dsontag/courses/ml16/

More information

TTIC 31190: Natural Language Processing

TTIC 31190: Natural Language Processing TTIC 31190: Natural Language Processing Kevin Gimpel Winter 2016 Lecture 10: Neural Networks for NLP 1 Announcements Assignment 2 due Friday project proposal due Tuesday, Feb. 16 midterm on Thursday, Feb.

More information

I590 Data Science Onramp Basics

I590 Data Science Onramp Basics I590 Data Science Onramp Basics Data Science Onramp contains mini courses with the goal to build and enhance your data science skills which are oftentimes demanded or desired in data science related jobs.

More information

Final exam for CSC 321 April 11, 2013, 7:00pm 9:00pm No aids are allowed.

Final exam for CSC 321 April 11, 2013, 7:00pm 9:00pm No aids are allowed. Your name: Your student number: Final exam for CSC 321 April 11, 2013, 7:00pm 9:00pm No aids are allowed. This exam has two sections, each of which is worth a total of 10 points. Answer all 10 questions

More information

COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection.

COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection. COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection. Instructor: (jpineau@cs.mcgill.ca) Class web page: www.cs.mcgill.ca/~jpineau/comp551 Unless otherwise

More information

Andrew Ng. Nuts and bolts of building AI applications using Deep Learning. Andrew Ng. Trend #1: Scale driving Deep Learning progress

Andrew Ng. Nuts and bolts of building AI applications using Deep Learning. Andrew Ng. Trend #1: Scale driving Deep Learning progress Trend #1: Scale driving Deep Learning progress Nuts and bolts of building AI applications using Deep Learning Trend #2: The rise of end-to-end learning Learning with integer or real-valued outputs: Learning

More information

An Introduction to Deep Learning. Labeeb Khan

An Introduction to Deep Learning. Labeeb Khan An Introduction to Deep Learning Labeeb Khan Special Thanks: Lukas Masuch @lukasmasuch +lukasmasuch Lead Software Engineer: Machine Intelligence, SAP The Big Players Companies The Big Players Startups

More information

Machine Learning 101a. Jan Peters Gerhard Neumann

Machine Learning 101a. Jan Peters Gerhard Neumann Machine Learning 101a Jan Peters Gerhard Neumann 1 Purpose of this Lecture Statistics and Math Refresher Foundations of machine learning tools for robotics We focus on regression methods and general principles

More information

Statistical Pattern Recognition

Statistical Pattern Recognition Statistical Pattern Recognition A Brief Overview of the course Hamid R. Rabiee Jafar Muhammadi, Nima Pourdamghani Spring 2012 http://ce.sharif.edu/courses/90-91/2/ce725-1/ Agenda What is a Pattern? What

More information

Bootstrapping Dialog Systems with Word Embeddings

Bootstrapping Dialog Systems with Word Embeddings Bootstrapping Dialog Systems with Word Embeddings Gabriel Forgues, Joelle Pineau School of Computer Science McGill University {gforgu, jpineau}@cs.mcgill.ca Jean-Marie Larchevêque, Réal Tremblay Nuance

More information

Parameter and Structure Learning in Graphical Models

Parameter and Structure Learning in Graphical Models Advanced Signal Processing 2 SE Parameter and Structure Learning in Graphical Models 02.05.2005 Stefan Tertinek turtle@sbox.tugraz.at Outline Review: Graphical models (DGM, UGM) Learning issues (approaches,

More information

COMS 4771 Introduction to Machine Learning. Nakul Verma

COMS 4771 Introduction to Machine Learning. Nakul Verma COMS 4771 Introduction to Machine Learning Nakul Verma Machine learning: what? Study of making machines learn a concept without having to explicitly program it. Constructing algorithms that can: learn

More information

SNS College of Engineering. Machine Learning

SNS College of Engineering. Machine Learning SNS College of Engineering Machine Learning About subfield of Artificial Intelligence (AI) name is derived from the concept that it deals with construction and study of systems that can learn from data

More information

Regularization. INFO-4604, Applied Machine Learning University of Colorado Boulder. September 19, 2017 Prof. Michael Paul

Regularization. INFO-4604, Applied Machine Learning University of Colorado Boulder. September 19, 2017 Prof. Michael Paul Regularization INFO-4604, Applied Machine Learning University of Colorado Boulder September 19, 2017 Prof. Michael Paul Generalization Prediction functions that work on the training data might not work

More information

FNLP Lecture 23b Wrapping Up

FNLP Lecture 23b Wrapping Up FNLP Lecture 23b Wrapping Up Nathan Schneider 7 December 2016 1 In a nutshell We have seen representations, datasets, models, and algorithms for computationally reasoning about textual language in a data-driven

More information

Statistical Machine Learning (CSE 575)

Statistical Machine Learning (CSE 575) Statistical Machine Learning (CSE 575) About this Course The link between inference and computation is central to statistical machine learning, which combines the computational sciences with statistics.

More information

The Machine Learning Landscape

The Machine Learning Landscape The Machine Learning Landscape Vineet Bansal Research Software Engineer, Center for Statistics & Machine Learning vineetb@princeton.edu Oct 31, 2018 What is ML? A field of study that gives computers the

More information

Figures. Agents in the World: What are Agents and How Can They be Built? 1

Figures. Agents in the World: What are Agents and How Can They be Built? 1 Table of Figures v xv I Agents in the World: What are Agents and How Can They be Built? 1 1 Artificial Intelligence and Agents 3 1.1 What is Artificial Intelligence?... 3 1.1.1 Artificial and Natural Intelligence...

More information

Lecture 24 Wrapping Up

Lecture 24 Wrapping Up Lecture 24 Wrapping Up Nathan Schneider ENLP 30 April 2018 1 In a nutshell We have seen representations, datasets, models, and algorithms for computationally reasoning about textual language. Persistent

More information