IC 669: Introduction to Machine Learning Lecture 1 Taesup Moon
Outline Examples of machine learning Course info Course outline - Supervised learning: classification, regression -Unsupervised learning: clustering, PCA, etc. -Learning theory: bias-variance tradeoff, generalization bound - Deep learning, Graphical models, HMM/LSTM, etc
Machine Learning is hot these days The core thing Google is working on is essentially Machine Learning @BoxDev, 2015 Eric Schimidt Chairman of Alphabet, Inc.
Machine Learning is hot these days Artificial Intelligence (Machine Learning) is a really exciting area of development. We have this goal in five to 10 years, we want to build computer systems, which can be better at main human senses than people are. @IIT New Delhi, 2015 Mark Zuckerberg CEO of Facebook
Machine Learning is hot these days We want to advance digital intelligence in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial return. @OpenAI initiative annoucement, 2015 Elon Musk CEO of Tesla, SpaceX
Definitions of machine learning Arthur Samuel (1959) - Machine learning is a field of study that gives computers the ability to learn without being explicitly programmed. Tom Mitchell (1998) - Machine learning is a well-posed learning problem: A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with E. Wikipedia - Machine learning explores the study and construction of algorithms that can learn from and make predictions on data. Tibshirani & Hastie (Statistical learning?) - Machine learning puts more emphasis on large-scale applications and prediction accuracy, whereas statistical learning emphasizes models, their interpretability, and precision and uncertainty - Essentially, the same! (Statisticians say that CS people did a better marketing with Machine Learning!)
Examples of machine learning Subway seat selection algorithm - https://www.youtube.com/watch?v=ker5t8mni38
Examples of machine learning Spam mail filtering
Examples of machine learning Handwriting recognition
Examples of machine learning Recommendation system Slides from Alex Smola
Examples of machine learning Web search ads Slides from Alex Smola
Examples of machine learning Speech recognition Hello world Estimate Hello world
Examples of machine learning Self-driving cars
Examples of machine learning Robotics https://www.youtube.com/watch?v=rvlhmgqgdky
Examples of machine learning Image captioning - https://www.ted.com/talks/fei_fei_li_how_we_re_teachin g_computers_to_understand_pictures?language=en
Examples of machine learning Computer games https://www.youtube.com/watch?v=q70ulpjw3gk
Examples of machine learning Go https://www.youtube.com/ watch?v=subqykxvx0a https://deepmind.com/alph a- go.html http://spri.kr/post/14725 -March 9 th, 10 th, 12 th, 13 th and 15 th @1pm L:A L:A L:A L:A L:A L:A - ICE Media = 5:0 Wall, = 4:1 Live = 3:2 YouTube = 2:3 Broadcast = 1:4 (planned) = 0:5 14 10 6 1 3
Examples of machine learning What is Machine Learning? (by Oxford) - https://www.youtube.com/watch?v=f_uwkziaem0&spfrel oad=10
Course info Course website - http://mind.dgist.ac.kr/classes/ic669 - Instructor: Taesup Moon - Tue 1:30pm-2:45pm, Wed 4:30pm-5:45pm
Course outline Preliminaries - Linear algebra, probability, optimization Supervised learning - Regression, classification (linear/nonlinear) Learning theory - Bias-variance tradeoff, VC bounds Unsupervised learning - Clustering (K-means), Dimensionality reduction (PCA) Advanced topics - Deep learning, Reinforcement learning
Preliminaries Linear algebra - Eigenvalue/vectors, singular-value decomposition (SVD) Probabilities - Random variables/vectors, expectation/covariance - Pdf/pmf, independence, Bayes rule Optimization - Convexity, gradient descent
Supervised learning Classification (Breast cancer prediction) Malignant? 1(Y) 0(N) Tumor Size Classification Discrete valued output (0 or 1) Tumor Size
Supervised learning Classification (Breast cancer prediction, more variables(features)) Age - Clump Thickness - Uniformity of Cell Size - Uniformity of Cell Shape Tumor Size
Supervised learning Classification à basically, learning boundaries -Logistic regression - Support Vector Machines (SVM) - K-Nearest Neighbors - Neural networks
Supervised learning Regression (Housing price prediction) 400 Price ($) in 1000 s 300 200 100 0 0 500 1000 1500 2000 2500 Size in feet 2 Supervised Learning right answers given Regression: Predict continuous valued output (price)
Course outline Regression - Linear regression (OLS) - Regularized linear regression (Lasso, Ridge)
Supervised learning Starting point: Slides from Hastie&Tibshirani
Supervised learning Objective: Slides from Hastie&Tibshirani
Learning theory Bias-variance tradeoff Regularization and generalization bound - VC dimension, Hoeffding s bound Model selection
Unsupervised learning Clustering Slides from Alex Smola
Unsupervised learning Principal Component Analyses (dimensionality reduction) Slides from Alex Smola
Unsupervised learning Slides from Hastie&Tibshirani
Deep learning Neural networks - Convolutional neural networks - Recurrent neural networks
Sequence learning Hidden Markov Modeling, Long Short-Term Memory
Reinforcement learning Environment reacts! Making sequential decisions. (not too much focus) Ex) Chess, Go, Robots