Demystifying Deep Learning The MathWorks, Inc. 1

Transcription

1 Demystifying Deep Learning Application Engineer, MathWorks Benelux Application Engineer, MathWorks Switzerland 2015 The MathWorks, Inc. 1

2 Agenda What s Deep Learning and why should I care? A practical approach to Deep Learning (for images) Transfer the learning from an expert model to your own application Building a Deep Learning network from scratch Deep Learning for time series and text data Key learnings of the session and cool features 2

3 Deep learning with MATLAB is easy and accessible! Use Less Data and Less Time with Transfer Learning Automatically generate code! Image Pre-processing CNN Image Post-processing Prototype and productize your entire workflow on embedded GPUs Apply Deep Learning not only on images but also on text and time series data 3

4 Why Machine Learning or Deep Learning? Transformational technology Close (better) than human accuracy for specific tasks Performance scales with data It is hard to use, it is challenging Enables engineers, researchers and other domain experts to create products and applications with more built-in intelligence 4

5 Application Breadth Artificial Intelligence, Machine Learning and Deep Learning Artificial Intelligence Machine Learning Deep Learning 1950s Timeline Today 5

6 Machine Learning vs Deep Learning Machine Learning Features Training Data Deep Learning Classification Task End-to-end Learning is an iterative process 6

7 Deep Learning Common Workflow ACCESS AND EXPLORE LABEL AND PREPROCESS DEVELOP PREDICTIVE INTEGRATE MODELS WITH DATA DATA MODELS SYSTEMS Files Data Augmentation/ Transformation Hardware-Accelerated Training Desktop Apps Databases Labeling Automation Hyperparameter Tuning Enterprise Scale Systems Sensors Import Reference Models Network Visualization Embedded Devices and Hardware 7

8 Deep learning is usually implemented using a neural network architecture The term deep refers to the number of layers in the network the more layers, the deeper the network. Data flows through network in layers, which provide transformation of data Convolutional Neural Network 8

9 Convolutional Neural Network is a popular Deep Learning architecture Shallow Neural Network Convolutional Neural Network Input Layer Hidden Layer Output Layer Every input neuron Connects to every neuron in the hidden layer Local receptive fields connect to neurons in the hidden layer Translate across an image create a feature map efficiently with convolution Input a b c d e f g h i j k l Output aw + bx + ey + fz Kernel w y x z 9

10 Deep Learning: different types of network architectures Type Series Network Directly Acyclic Graph Network (DAG) Recurrent Neural Network (RNN) Basic Architecture Network Example AlexNet, VGG R-CNN (fast, faster), GoogLeNet, SegNet LSTM Application Object Recognition: Cars, Lane, Pedestrian Object Detection: Cars, Traffic Signs, Scene (Semantic Segmentation) Sequential data: time series, signals 10

11 Agenda What s Deep Learning and why should I care? A practical approach to Deep Learning (for images) Transfer the learning from an expert model to your own application Building a Deep Learning network from scratch Deep Learning for time series and text data Key learnings of the session and cool features 11

12 Deep Learning can be complex and challenging to apply Design Deep Learning & Vision Algorithms Main Challenges Handle large image sets Image labeling is tedious Have access to models Accelerate and Scale Training Main Challenges Capability of training with multiple GPUs Capability of training in the cloud High Performance Deployment Main Challenges Convert models to CUDA code Compress models to fit into embedded GPUs. 12

13 Object Recognition using Deep Learning in MATLAB Supervised Learning Stapler CoffeCoaster BLACK BOX Highlighter RubikCube AlexNet PRETRAINED MODEL Transfer Knowledge OfficeNet NEW MODEL MATLAB Support Package for USB Webcams 13

14 Where to start? Two Approaches for Deep Learning Train from Scratch Deep Neural Network, e.g. CNN Transfer Learning Fine - Tune 14

15 Why Perform Transfer Learning? Less data Less training time Leverage best network types from top researchers AlexNet PRETRAINED MODEL VGG-16 PRETRAINED MODEL ResNet PRETRAINED MODEL Reference models are great feature extractors Caffe M O D E L S GoogLeNet PRETRAINED MODEL TensorFlow/Keras M O D E L S 15

16 Transfer Learning Workflow Replace final layers Load pretrained network Early layers that learned low-level features (edges, blobs, colors) Last layers that learned task specific features New layers learn features specific to your data 1 million images 1000s classes Fewer classes Learn faster 16

17 Transfer Learning Workflow Load pretrained network Early layers that learned low-level features (edges, blobs, colors) Last layers that learned task specific features Replace final layers New layers to learn features specific to your data Training images Train network Training options 1 million images 1000s classes Fewer classes Learn faster 100s images 10s classes 17

18 Transfer Learning Workflow Deploy results Load pretrained network Early layers that learned low-level features (edges, blobs, colors) Last layers that learned task specific features Replace final layers New layers to learn features specific to your data Training images Train network Training options Predict and assess network accuracy Test images Car Probability Truck 1 million images 1000s classes Fewer classes Learn faster 100s images 10s classes Trained Network 98 % Bike 18

19 Transfer Learning Workflow Load pretrained network Replace final layers Train network Training images Predict and assess network accuracy Deploy results Early layers that learned low-level features (edges, blobs, colors) Last layers that learned task specific features New layers to learn features specific to your data Training options Test images Probability Car Truck 1 million images 1000s classes Fewer classes Learn faster 100s images 10s classes Trained Network 98 % Bike 19

20 Accelerate training and prediction! Deep Learning End-to-end Prediction Learning is an iterative process Alexnet vs Squeezenet More GPUs Whitepaper: Deep Learning in the Cloud 20

21 Frames per second Alexnet Inference on NVIDIA Titan Xp GPU Coder + TensorRT (3.0.1, int8) GPU Coder + TensorRT (3.0.1) GPU Coder + cudnn mxnet (1.1.0) TensorFlow (1.6.0) Batch Size CPU Intel(R) Xeon(R) CPU E GHz Testing platform GPU cudnn Pascal Titan Xp v7 21

22 Deep Learning is easy and accessible with MATLAB! IN OUT MKL-DNN TensorRT & cudnn Neon Design Deep Learning & Vision Algorithms Accelerate and Scale Training High Performance Deployment Highlights Highlights Highlights Datastores for large image sets Single line of code to: Automate compilation with Automate image labeling Accelerate training with GPU Coder Direct access to models within MATLAB with multiple GPUs or 1.4x speedup over C++ Caffe support packages Scale to clusters on Jetson TX2 Import Tensor Flow Keras and Caffe networks 22

23 Agenda What s Deep Learning and why should I care? A practical approach to Deep Learning (for images) Transfer the learning from an expert model to your own application Building a Deep Learning network from scratch Deep Learning for time series and text data Key learnings of the session and cool features 23

24 Deep Learning for Time Series, Sequences and Text Sentiment Analysis Speech-to-text Music Recognition Predictive Maintenance Text-to-Speech Movie Ranking Prediction Instant Translation 24

25 Deep Learning for Time Series Example: Seizure prediction (time series classification) Goal: Predict seizures in long-term Dataset: iegg time series Data size: 20GB Output: Classification before or between seizure Link for melbourne-university-seizure-prediction 25

26 Types of Datasets Numeric Data Time Series/ Text Data Image Data Long Short Term Memory (LSTM) LSTM or CNN Convolutional Neural Networks (CNN) Directed acyclic graph networks (DAG) 26

27 Deep Learning for Time Series CNN: Data for Time series = Pixel for Images

28 Deep Learning for Time Series Long Short Term Memory (LSTM) Network 28

29 Text Analytics Access and Explore Data Preprocess Data Develop Predictive Models Clean-up Text Convert to Numeric Media reported two trees blown down along I-40 in the Old Fort area. cat dog run two doc media report two tree blown down i40 old fort area doc Word Docs PDF s Text Files Stop Words Stemming Tokenization Bag of Words TF-IDF Word Embeddings LSTM Latent Dirichlet Allocation Latent semantic analysis 29

30 Key Takeaways: Deep Learning for Time Series and Text Applications Time series Text forecasting classification (Predictive Maintenance) classification (Sentiment Analysis, Tagging) clustering (Topic Modelling) Text Analytics: Prepend Text preprocessing Conversion to numeric 30

31 Agenda What s Deep Learning and why should I care? A practical approach to Deep Learning (for images) Transfer the learning from an expert model to your own application Building a Deep Learning network from scratch Deep Learning for time series and text data Key learnings of the session and cool features 31

32 Deep learning with MATLAB is easy and accessible! Use Less Data and Less Time with Transfer Learning Automatically generate code! Image Pre-processing CNN Image Post-processing Prototype and productize your entire workflow on embedded GPUs Apply Deep Learning not only on images but also on text and time series data 32

33 Algorithm Development, Testing & Verification Neural Network Toolbox GPU Coder Images 14x with MEX ~66 Fps (Tegra X1) 2x with MEX Semantic Scene Segmentation Acceleration (Classification) Deployment (Embedded) Time Series 33

34 Mini-batchable datastore Have a small number of large high-res images To train, need a large number of pairs of images 34

35 Ground truth labelling: attributes and sublabels 35

36 Visualize and understand the network architecture 19-Apr :00:00 Detect problems before wasting time training! Missing or disconnected layers, Mismatching or incorrect sizes of layer inputs, Incorrect number of layer inputs, Invalid graph structures. Download Support Package 36

37 Semantic Segmentation Fully convolutional networks (FCN) Segmentation Networks (SegNet) Other directed acyclic graph (DAG) Manage connections, add and remove layers Manage label data and evaluate performance >>help semanticseg 37

38 Algorithm Development, Testing & Verification Applicable to other sensors, e.g. LiDar Automated Driving System Toolbox Computer Vision System Toolbox Neural Network Toolbox Access Data Velodyne FileReader Preprocess and Label Object Classification PointNet Deep Neural Network Preprocessing Clustering LiDAR Annotation Ground Truth [Under Request] 38

39 Input LSTM Fully Connected Regression Layer LSTM for both Classification and Regression Time to failure Feature extraction Classification 40

40 Define new operations for deep networks with Custom Layers 41

41 THANK YOU! Use Less Data and Less Time with Transfer Learning Automatically generate code! Image Pre-processing CNN Image Post-processing Prototype and productize your entire workflow on embedded GPUs Apply Deep Learning not only on images but also on time series data 42