IC 669: Introduction to Machine Learning. Lecture 1 Taesup Moon

Transcription

1 IC 669: Introduction to Machine Learning Lecture 1 Taesup Moon

2 Outline Examples of machine learning Course info Course outline - Supervised learning: classification, regression -Unsupervised learning: clustering, PCA, etc. -Learning theory: bias-variance tradeoff, generalization bound - Deep learning, Graphical models, HMM/LSTM, etc

3 Machine Learning is hot these days The core thing Google is working on is essentially Machine 2015 Eric Schimidt Chairman of Alphabet, Inc.

4 Machine Learning is hot these days Artificial Intelligence (Machine Learning) is a really exciting area of development. We have this goal in five to 10 years, we want to build computer systems, which can be better at main human senses than people New Delhi, 2015 Mark Zuckerberg CEO of Facebook

5 Machine Learning is hot these days We want to advance digital intelligence in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial initiative annoucement, 2015 Elon Musk CEO of Tesla, SpaceX

6 Definitions of machine learning Arthur Samuel (1959) - Machine learning is a field of study that gives computers the ability to learn without being explicitly programmed. Tom Mitchell (1998) - Machine learning is a well-posed learning problem: A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with E. Wikipedia - Machine learning explores the study and construction of algorithms that can learn from and make predictions on data. Tibshirani & Hastie (Statistical learning?) - Machine learning puts more emphasis on large-scale applications and prediction accuracy, whereas statistical learning emphasizes models, their interpretability, and precision and uncertainty - Essentially, the same! (Statisticians say that CS people did a better marketing with Machine Learning!)

7 Examples of machine learning Subway seat selection algorithm -

8 Examples of machine learning Spam mail filtering

9 Examples of machine learning Handwriting recognition

10 Examples of machine learning Recommendation system Slides from Alex Smola

11 Examples of machine learning Web search ads Slides from Alex Smola

12 Examples of machine learning Speech recognition Hello world Estimate Hello world

13 Examples of machine learning Self-driving cars

14 Examples of machine learning Robotics

15 Examples of machine learning Image captioning - g_computers_to_understand_pictures?language=en

16 Examples of machine learning Computer games

17 Examples of machine learning Go watch?v=subqykxvx0a a- go.html -March 9 th, 10 th, 12 th, 13 th and 15 L:A L:A L:A L:A L:A L:A - ICE Media = 5:0 Wall, = 4:1 Live = 3:2 YouTube = 2:3 Broadcast = 1:4 (planned) = 0:

18 Examples of machine learning What is Machine Learning? (by Oxford) - oad=10

19 Course info Course website Instructor: Taesup Moon - Tue 1:30pm-2:45pm, Wed 4:30pm-5:45pm

20 Course outline Preliminaries - Linear algebra, probability, optimization Supervised learning - Regression, classification (linear/nonlinear) Learning theory - Bias-variance tradeoff, VC bounds Unsupervised learning - Clustering (K-means), Dimensionality reduction (PCA) Advanced topics - Deep learning, Reinforcement learning

21 Preliminaries Linear algebra - Eigenvalue/vectors, singular-value decomposition (SVD) Probabilities - Random variables/vectors, expectation/covariance - Pdf/pmf, independence, Bayes rule Optimization - Convexity, gradient descent

22 Supervised learning Classification (Breast cancer prediction) Malignant? 1(Y) 0(N) Tumor Size Classification Discrete valued output (0 or 1) Tumor Size

23 Supervised learning Classification (Breast cancer prediction, more variables(features)) Age - Clump Thickness - Uniformity of Cell Size - Uniformity of Cell Shape Tumor Size

24 Supervised learning Classification à basically, learning boundaries -Logistic regression - Support Vector Machines (SVM) - K-Nearest Neighbors - Neural networks

25 Supervised learning Regression (Housing price prediction) 400 Price ($) in 1000 s Size in feet 2 Supervised Learning right answers given Regression: Predict continuous valued output (price)

26 Course outline Regression - Linear regression (OLS) - Regularized linear regression (Lasso, Ridge)

27 Supervised learning Starting point: Slides from Hastie&Tibshirani

28 Supervised learning Objective: Slides from Hastie&Tibshirani

29 Learning theory Bias-variance tradeoff Regularization and generalization bound - VC dimension, Hoeffding s bound Model selection

30 Unsupervised learning Clustering Slides from Alex Smola

31 Unsupervised learning Principal Component Analyses (dimensionality reduction) Slides from Alex Smola

32 Unsupervised learning Slides from Hastie&Tibshirani

33 Deep learning Neural networks - Convolutional neural networks - Recurrent neural networks

34 Sequence learning Hidden Markov Modeling, Long Short-Term Memory

35 Reinforcement learning Environment reacts! Making sequential decisions. (not too much focus) Ex) Chess, Go, Robots