Lecture 1: Introduction

Transcription

1 Administration CSED703R: Deep Learning for Visual Recognition (2017F) Lecture 1: Introduction Bohyung Han Computer Vision Lab. CSED703R: Deep Learning for Visual Recognition Instructor: Prof. Bohyung Han B4-123) Time & Location: TuTh 12:30 ~ 13:45 PM, B2-105 Office hour: by appointment Textbook (for reference) Deep Learning by I. Goodfellow, Y. Bengio, and A. Courville Neural Networks and Deep Learning by M. Nielsen Prerequisite Coursework: probability theory, linear algebra, computer vision Substantial programming experience including script languages in Unix/Linux/ MacOS environment 2 3 Class Coverage Introduction and preliminaries Unsupervised representation learning Convolutional Neural Networks (CNNs) Image classification, object detection and localization Visual tracking, action recognition and localization Semantic segmentation CNN optimization and analysis Recurrent Neural Networks (RNNs) Vision and languages Image caption generation Visual question answering Generative Adversarial Networks (GANs) Deep reinforcement learning Deep learning applications and others Grading Assignments (30%) Problem solving Programming projects Mid-term exam (20%) Presentations (10%) Final project (40%) Research project Final report Note: No Pass/Fail grading Individual percentages are subject to change. 4

2 Final Project Team organization: individual project Deliverables Demo, source code, and presentation Frequent intermediate reports Final report Guideline You should decide the theme of your project. Final report should adhere to the standard quality and format of reputable conferences and journals. Top venues in machine learning: ICML, NIPS, AISTATS, ICLR, JMLR Top venues in computer vision: CVPR, ICCV, ECCV, TPAMI, IJCV Course Policy Assignments submission Late assignments will be accepted for three days with score deduction. Programming platform TensorFlow Other platforms such as Caffe, Torch, Theano, and MatConvNet are also allowed but you should make sure to minimize grading complexity with proper documentation. Academic integrity Make sure to acknowledge the POSTECH academic integrity. Violating the academic integrity means the automatic failure (F) in this class with NO exception. 5 6 Course Policy Course identity This is NOT an introductory course in machine learning or deep learning. The major requirement of this course is the final project. The students willing or competent to do the project very seriously are recommended to take this course. The instructor has the right to evaluate students based only on the performance in the final project if necessary. Deep Learning 7 8

3 Pipeline of Visual Recognition Components of Standard Visual Recognition Data: images and videos Computer vision algorithms Representation of visual data Visual features Hand-crafted features: HOG, BoW, GIST, LBP, MSER, SIFT, SURF, Learned features: CNN, RNN, Auto-encoder Classifiers Discriminative methods: NN, SVM, random forest, boosting, Generative methods: naïve Bayes Features vs. classifiers Good features are key ingredients to recent progress in recognition. Various classification algorithms have been proposed so far. Features Classifiers 9 10 What is deep learning? A learning method to model high-level abstractions in data by using model architectures composed of multi-layer non-linear operations Representation learning A buzzword of neural network with many layers Deep Learning Applications Computer vision Natural language processing Speech recognition Bioinformatics Medical imaging And many others Deep Learning 11 12

4 Speech Recognition Machine Translation 13 [Johnson16] M. Johnson, M. Schuster, Q. V. Le, M. Krikun, Y. Wu, Z. Chen, N. Thorat, F. B. Viégas, M. Wattenberg, G. Corrado, M. Hughes, J. Dean: Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation. arxiv , Image Classification Object Detection R-CNN: regions with CNN features Supervised pretraining using large-scale data Domain-specific fine tuning Linear SVM applied to pool5, fc6, and fc7 Input image Extract region Compute CNN features Classification proposal Any proposal method Any architecture Softmax or SVM 15 Image Classification Top-5 Errors (%) [Girshick2014] R. Girshick, J. Donahue, S. Guadarrama, T. Darrell, J. Malik: Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation, CVPR

5 Semantic Segmentation Visual Tracking MDNet (Multi-Domain Network) Multi-domain learning Separating shared and domain-specific layers Input image Ground-truth FCN DeconvNet The Winner of Visual Object Tracking Challenge 2015 [Noh15] H. Noh, S. Hong, B. Han: Learning Deconvolution Network for Semantic Segmentation, ICCV [Nam16] H. Nam, B. Han: Learning Multi-Domain Convolutional Neural Networks for Visual Tracking. CVPR Face Verification Pose Estimation FaceNet [Schroff15] F. Schroff, D. Kalenichenko, J. Philbin: FaceNet: A Unified Embedding for Face Recognition and Clustering. CVPR 2015 DeepFace [Taigman14] Y. Taigman, M. Yang, M. Ranzato, L Wolf: DeepFace: Closing the Gap to Human-Level Performance in Face Verification. CVPR [Insafutdinov16] E. Insafutdinov, L. Pishchulin, B. Andres, M. Andriluka, B. Schiele: DeeperCut: A Deeper, Stronger, and Faster Multi-person Pose Estimation Model. ECCV

6 Image Caption Generation Image Question Answering Classification Network Dynamic Parameter Layer teddy bear What is in the cabinet? Parameter Prediction Network GRU GRU GRU GRU GRU GRU What is in the cabinet? Hashing Candidate Weights [Vinyals15] O. Vinyals, A. Toshev, S. Bengio, D. Erhan: Show and Tell: A Neural Image Caption Generator. CVPR [Noh15] H. Noh, P. H. Seo, B. Han: Image Question Answering using Convolutional Neural Network with Dynamic Parameter Prediction, arxiv: , Neural Artistic Style Image Generation Style1: The Starry Night Source Style2: The Scream [Gatys15] L. A. Gatys, A. S. Ecker, M. Bethge: A Neural Algorithm of Artistic Style. arxiv: , Generative Adversarial Networks (GANs) [Goodfellow14] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. C. Courville, Y. Bengio: Generative Adversarial Nets. NIPS

7 Deep Reinforcement Learning: Atari Games Deep Reinforcement Learning: AlphaGo [Minh2013] V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, M. A. Riedmiller: Playing Atari with Deep Reinforcement Learning. arxiv: , [Silver16] D. Silver et al.: Mastering the Game of Go with Deep Neural Networks and Tree Search. Nature,