Practical Applications of Deep Learning

Transcription

1 Practical Applications of Deep Learning A hands-on MATLAB workshop Pitambar Dayal Abhijit Bhattacharjee Product Marketing Manager Application Engineer The MathWorks, Inc.

2

3

4 Agenda Introduction Exercise : Deep learning in lines of code Deep learning fundamentals Exercises and : Exploring pretrained networks/classifying handwritten digits Transfer learning Exercise : Creating a food classifier Deploying deep neural networks Conclusion

5 Deep Learning Applications Voice assistants (speech to text) Teaching character to beat video game Automatically coloring black and white images

6 What is Deep Learning?

7 What is Deep Learning? Subset of machine learning with automatic feature extraction Learns features and tasks directly from data More Data = better model Machine Learning Deep Learning Deep Learning

8 DL Applications include image classification, speech recognition, autonomous driving, etc. Detection of cars and road in autonomous driving systems Rain Detection and Removal Iris Recognition.% accuracy. Deep Joint Rain Detection and Removal from a Single Image" Wenhan Yang, Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan. Source: An experimental study of deep convolutional features for iris recognition Signal Processing in Medicine and Biology Symposium (SPMB), IEEE Shervin Minaee ; Amirali Abdolrashidiy ; Yao Wang; An experimental study of deep convolutional features for iris recognition

9 Deep Learning Models can Surpass Human Accuracy. Human Accuracy Source: ILSVRC Top- Error on ImageNet

10 Deep Learning Enablers Increased GPU acceleration Labeled public datasets World-class models AlexNet PRETRAINED MODEL ResNet PRETRAINED MODEL VGG-/ PRETRAINED MODEL GoogLeNet PRETRAINED MODEL Caffe MODEL IMPORTER TensorFlow-Keras MODEL IMPORTER

11 Deep Learning Datatypes Image Signal Numeric Text

12 Let s try it out! Open: DeepLearningInLines.mlx in folder -DeepLearningInLines

13 Deep learning is not complicated. It can be easy!

14 Deep Learning Uses a Neural Network Architecture Input Layer Hidden Layers (n) Output Layer

15 Thinking about Layers Layers are like blocks Stack on top of each other Replace one block with a different one Each hidden layer processes the information from the previous layer

16 Thinking about Layers Layers are like blocks Stack on top of each other Replace one block with a different one Each hidden layer processes the information from the previous layer Layers can be ordered in different ways

17 Convolutional Neural Networks (CNNs) Special layer combinations that make them great for image classification Convolution Layer Max Pooling Layer ReLU Layer

18 Convolution Layers Search for Patterns These patterns would be common in the number

19 All patterns are compared to the patterns on a new image. Pattern starts at left corner Perform comparison Slide over one pixel Reach end of image Repeat for next pattern

20 Convolution Layers Search for Patterns These patterns would be common in the number

21 Good pattern matching in convolution improves chances that object will classify properly This image would not match well against the patterns for the number zero It would only do very well against this pattern

22 Max Pooling is a down-sampling operation Shrink large images while preserving important information x filters Stride Length =

23 Rectified Linear Units Layer (ReLU) Typically converts negative numbers to zero

24 CNNs typically end with Layers Fully Connected Layer Looks at which high-level features correspond to a specific category Calculates scores for each category (highest score wins) Softmax Layer Turns scores into probabilities. Classification Layer Categorizes image into one of the classes that the network is trained on

25 Deep Learning Workflow Preprocess Data Repeat these steps until network reaches desired level of accuracy Define Layers Set training options Train the network Test/deploy trained network

26 Pretrained Networks Researchers created network architecture for classifying hundreds of objects MATLAB makes it easy to import these networks AlexNet, GoogLeNet, ResNet, VGG, Caffe models, TensorFlow-Keras models This is what we did for the peppers example! (AlexNet) AlexNet PRETRAINED MODEL ResNet PRETRAINED MODEL VGG-/ PRETRAINED MODEL GoogLeNet PRETRAINED MODEL Caffe MODEL IMPORTER TensorFlow-Keras MODEL IMPORTER

27 Import Models from Keras-Tensorflow and Caffe

28 Questions?

29 Let s try it out! Exercise: Work_ExploringPretrainedNetworks.mlx in folder -PretrainedModelExercise Exercise: MNIST_HandwritingRecognition.mlx in folder -MNISTExercise

30 Takeaways Pre-trained networks have a pre-determined layer order that makes them effective for classifying images Typically trained to classify lots of images Different networks yield different results Great starting point, but not consistently accurate We ll fix this later with transfer learning!

31 Takeaways Deep learning for image classification uses CNNs CNNs can have different combinations of initial layers but usually end with: Fully Connected Layer Softmax Layer Classification Layer Important factors that affect accuracy and training time Network architecture Initial learning rate

32 Two Approaches for Deep Learning. Train a Deep Neural Network from Scratch. Fine-tune a pre-trained model (transfer learning)

33 Transfer Learning Workflow Load pretrained network Early layers that learned low-level features (edges, blobs, colors) Last layers that learned task specific features Replace final layers New layers to learn features specific to your data Train network Training images Training options Predict and assess network accuracy Test images Deploy results Probability Boat Plane Car Train Trained Network million images s classes Fewer classes Learn faster s images s classes

34 Transfer Learning Workflow Step Load pretrained network Early layers learn lowlevel features (edges, blobs, colors) Last layers learn taskspecific features million images s classes

35 Transfer Learning Workflow Step Replace final layers New layers learn features specific to your data Load pretrained network Early layers that learned low-level features (edges, blobs, colors) Last layers that learned task specific features Fewer classes Learn faster million images s classes

36 Transfer Learning Workflow Step Train network Training images Training options Load pretrained network Early layers that learned low-level features (edges, blobs, colors) Last layers that learned task specific features Replace final layers New layers to learn features specific to your data s images s classes million images s classes Fewer classes Learn faster

37 Transfer Learning Workflow Step Predict and assess network accuracy Test images Load pretrained network Early layers that learned low-level features (edges, blobs, colors) Last layers that learned task specific features Replace final layers New layers to learn features specific to your data Train network Training images Training options Trained Network million images s classes Fewer classes Learn faster s images s classes

38 Transfer Learning Workflow Step Predict Deploy and results assess network accuracy Probability Test images Boat Load pretrained network Early layers that learned low-level features (edges, blobs, colors) Last layers that learned task specific features Replace final layers New layers to learn features specific to your data Train network Trained Network Plane Car Training images Train Training options Predict and assess network accuracy Test images million images s classes Fewer classes Learn faster s images s classes Trained Network

39 Transfer Learning Workflow Load pretrained network Early layers that learned low-level features (edges, blobs, colors) Last layers that learned task specific features Replace final layers New layers to learn features specific to your data Train network Training images Training options Predict and assess network accuracy Test images Deploy results Probability Boat Plane Car Train Trained Network million images s classes Fewer classes Learn faster s images s classes

40 Let s try it out! Exercise: Work_SeeFoodTransferLearning.mlx in folder -TransferLearningExercise

41 Takeaways Transfer Learning Replace last layers with our own layers Efficient way to modify pre-trained models to our needs Use an Image datastore when working with lots of images MATLAB lets you visualize activations in a network

42 Deep Learning Workflow Extends Beyond Training LABEL AND PREPROCESS DATA DEVELOP PREDICTIVE MODELS INTEGRATE MODELS WITH SYSTEMS Data Augmentation/ Transformation Hardware-Accelerated Training Desktop Apps Labeling Automation Hyperparameter Tuning Enterprise Scale Systems Import Reference Models Network Visualization Embedded Devices and Hardware

43 Automated Object Detection

44 Labeling Data Image Labeler App Object Detection Pixel Labeling Ground-truth Labeler App Videos for automated driving applications D Point Cloud Labeling with Semantic Segmentation Labeling Big Images

45 One Step Left Deployment! Access Data Preprocess Select Network Train Deploy Image Acquisition Image Processing Neural Network Parallel Computing GPU Coder Computer Vision System

46 GPU Coder Automatically generates CUDA Code from MATLAB Code can be used on NVIDIA GPUs

47

48 How fast is GPU Coder? Vision Algorithms compared to C on CPU Fog removal x speedup Frangi filter x speedup Distance transform x speedup Ray tracing x speedup SURF feature extraction x speedup

49 GPU Coder Demo Deploying our deep network on a GPU Open: GenerateGPUCode.mlx in folder -GPUCoder

50 Takeaways GPU Coder MATLAB supports entire deep learning workflow Generate code for various targets from MATLAB code GPU Coder+TensorRT fastest for series networks GPU Coder very fast for DAG networks

51 Deep Learning OnRamp Self-paced FREE course Hands-on experience Everything done in the browser

52 Resources Web Documentation Community File Exchange/GitHub Deep Learning OnRamp

53 POP QUIZ Results will be reported to your manager

54 What is the difference between Machine Learning and Deep Learning? A. Deep learning is machine learning done really far underground. B. I don t know, I didn t pay attention, I actually don t even work here, I just show up to these things. C. Machine learning requires manual feature extraction while deep learning automatically extracts features making it end-to-end learning

55 What is the difference between Machine Learning and Deep Learning? A. Deep learning is machine learning done really far underground. B. I don t know, I didn t pay attention, I actually don t even work here, I just show up to these things. C. Machine learning requires manual feature extraction while deep learning automatically extracts features making it end-to-end learning

56 Which of the following is not an application of deep learning? A. Image classification B. Speech recognition C. Automated driving D. Filtering applications like rain removal E. Recognizing people s faces on your phone s photo app F. Building a hotdog/not-hotdog classifier G. None of the above

57 Which of the following is not an application of deep learning? A. Image classification B. Speech recognition C. Automated driving D. Filtering applications like rain removal E. Recognizing people s faces on your phone s photo app F. Building a hotdog/not-hotdog classifier G. None of the above

58 Which of the following is NOT a layer in deep networks? A. Fully Connected Layer B. Softmax Layer C. Classification Layer D. Convolution Layer E. ReLu Layer F. MaxPooling Layer G. Banana Layer (classifies all objects as Banana)

59 Which of the following is NOT a layer in deep networks? A. Fully Connected Layer B. Softmax Layer C. Classification Layer D. Convolution Layer E. ReLu Layer F. MaxPooling Layer G. Banana Layer (classifies all objects as Banana)

60 What does the Fully Connected Layer do? A. Calculates a score for each category B. Plays a full game of Connect Four C. Saves you % or more on car insurance

61 What does the Fully Connected Layer do? A. Calculates a score for each category B. Ensures your layered sandwiches stay Fully Connected C. Saves you % or more on car insurance

62 How do we perform transfer learning? A. Change every other layer of our network to a softmax layer B. Transfer all data from the CPU to the GPU C. Load in a pre-trained network, modify the last few layers, and train it on our data.

63 How do we perform transfer learning? A. Change every other layer of our network to a softmax layer B. Transfer all data from the CPU to the GPU C. Load in a pre-trained network, modify the last few layers, and train it on our data.

64 What are three hyperparameters that have a major impact on training time and accuracy? A. Network Architecture B. Mini Batch Size C. Learning Rate D. Flux Capacitor

65 What are three hyperparameters that have a major impact on training time and accuracy? A. Network Architecture B. Mini Batch Size C. Learning Rate D. Flux Capacitor

66 What is loss? A. The opposite of a win B. The state or feeling of grief when deprived of someone or something of value C. A measurement of error between predicted labels and actual labels. Loss has an inverse relationship with score, and our goal is to minimize loss. D. All of the above

67 What is loss? A. The opposite of a win B. The state or feeling of grief when deprived of someone or something of value C. A measurement of error between predicted labels and actual labels. Loss has an inverse relationship with score, and our goal is to minimize loss. D. All of the above

68 Which of the following statements is false? A. MATLAB makes it easy to import pre-trained models through add-ons and model importers B. MATLAB supports the entire deep learning workflow including labeling, training, and deployment C. MATLAB has visual training plots that allow you to see accuracy and loss during training D. We do a great job of subtly marketing MATLAB s deep learning capabilities

69 Which of the following statements is false? A. MATLAB makes it easy to import pre-trained models through add-ons and model importers B. MATLAB supports the entire deep learning workflow including labeling, training, and deployment C. MATLAB has visual training plots that allow you to see accuracy and loss during training D. We do a great job of subtly marketing MATLAB s deep learning capabilities

70 Free Seminar: ADAS and Automated Driving Development Using MATLAB and Simulink

71 Questions?

72 Convolution Layer Core building block of a CNN Convolve the filters sliding them across the input, computing the dot product dot W sum dot W Intuition: learn filters that activate when they see some specific feature

73 Rectified Linear Unit (ReLU) Layer Frequently used in combination with Convolution layers Do not add complexity to the network Most popular choice: f x = max, x, activation is thresholded at

74 Pooling Layer Perform a downsampling operation across the spatial dimensions Goal: progressively decrease the size of the layers Max pooling and average pooling methods Popular choice: Max pooling with x filters, Stride = Max pooling Average pooling

75 Convolution

76 Convolution

77 - - Convolution

78 Convolution

79 Convolution

80 Convolution

81 Activation ReLU

82 Activation ReLU

83 Activation ReLU

84 Activation ReLU

85 Pooling

86 Pooling

87 Pooling

88 Pooling

89 Pooling