Deep Learning and Storage

Transcription

1 Keep Those GPUs Busy Deep Learning and Storage Igor Ostrovsky 1

2 THREE PILLARS OF DEEP LEARNING EXPERTISE TECHNIQUES & TOOLS COMPUTE FROM CPU TO GPU SERVERS DATA MASSIVE TRAINING SETS 2

3 THREE PILLARS OF DEEP LEARNING EXPERTISE TECHNIQUES & TOOLS COMPUTE FROM CPU TO GPU SERVERS DATA MASSIVE TRAINING SETS 3

4 DEEP LEARNING AND DATA 4

5 MODEL SIZE GROWTH BEN TAYLOR, ZIFF GB kb MB

6 BIG MODELS NEED BIG DATA SETS Dog Cat Dog or cat? Powerful models have many parameters Many parameters require big training sets 6

7 Performance DEEP LEARNING AND TRAINING SET OBSERVATION BY ANDREW NG, AI LEADER Deep learning Older learning algorithms Amount of data 7

8 TRAINING SETS FOR DEEP LEARNING 2011 flickr30k 2012 GBs TBs PBs 8

9 TRAINING SETS FOR DEEP LEARNING Synthetic data 2011 flickr30k 2012 GBs TBs PBs 9

10 We don t have better algorithms. We just have more data. PETER NORVIG Google Research Director 10

11 DEEP LEARNING INFRASTRUCTURE FOR TRAINING 11

12 INGEST COMPLEXITIES OF AI IN PRODUCTION CLEAN & TRANSFORM EXPLORE TRAIN From sensors, machines, & user generated Label, anomaly detection, ETL, prep, stage Quickly iterate to converge on models Run for hours to days in production cluster CPU Servers GPU Server GPU Production Cluster COPY & TRANSFORM COPY & TRANSFORM COPY & TRANSFORM 12

13 INGEST REAL WORLD PIPELINE IN AN AUTONOMOUS CAR COMPANY CLEAN, LABEL, RESIZE EXPLORE TRAIN INFERENCE IN VIRTUAL WORLD CPU Servers GPU Server GPU Production Cluster GPU Production Cluster 10 S OF PB COLD STORAGE 13

14 SOFTWARE PIPELINE EXAMPLE 14

15 WIDE RANGE OF NEEDS IN THE PIPELINE SIGNIFICANT CHALLENGE INGEST CLEAN & TRANSFORM EXPLORE TRAIN & VALIDATE Sensors, Synthetic CPU Servers GPU Server GPU Cluster 15

16 WHAT IS FLASHBLADE? 17 TB or 52 TB blades 1.5 M IOPS and 16 GB/s performance ELASTIC FABRIC is FAST AND SIMPLE and scales linearly NFS, S3, SMB, HTTP and 1.6 PB (3:1) N+2 redundancy POWER max 1850 WATT fully loaded 16

17 FROM THE EXPERTS Building and managing data pipelines is typically one of the most costly pieces of a complete machine learning solution. If your boss asks you, tell them that I said [to] build a unified data warehouse. Jeremy Hermann & Mike Del Balso Uber Machine Learning Platform Andrew Ng Former head of Baidu AI/Google Brain Nuts and Bolts of Applying Deep Learning 17

18 TRAINING BENCHMARKS 18

19 AI SYSTEMS DESIGN PATTERNS HOW MUCH PERFORMANCE PENALTY DUE TO SHARED STORAGE? FULL TRAINING WORKFLOW decode scale evaluate forward-propagation update back-propagation I/O CPU GPU BENCHMARK SETUP Setup #1: DGX-1 with 4x Local SSDs Setup #2: DGX-1 with 1x FlashBlade 19

20 IMAGENET TRAINING DGX-1, 8 x P100 Model Year Top-5 DGX-1 images/s AlexNet % 9968 Image classification challenge 1.28M labeled images 1000 categories VGG % 1093 Inception V % 1052 ResNet % 1542 ResNet %

21 TIME TO RESULTS TIME TO RESULTS TensorFlow ResNet-50 Training, 200kB Images 1.8 Hours process 10M images 1.8 Hours process 10M images 21 NVIDIA DGX-1 Local SSDs NVIDIA DGX-1 Pure FlashBlade

22 TIME TO RESULTS TIME TO RESULTS TensorFlow ResNet-50 Training, 200kB Images 0.9 Hours load 2TB 33% Faster 1.8 Hours process 10M images 1.8 Hours process 10M images 22 NVIDIA DGX-1 Local SSDs NVIDIA DGX-1 Pure FlashBlade

23 TIME TO RESULTS TIME TO FIRST RESULT TensorFlow ResNet-50 Training, 200kB Images 0.9 Hours load 2TB Instant 23 NVIDIA DGX-1 Local SSDs NVIDIA DGX-1 Pure FlashBlade

24 TIME TO RESULTS TIME TO FIRST RESULT TensorFlow ResNet-50 Training, 200kB Images [M]achine learning is [ ] an iterative process of running the learner, analyzing the results, modifying the data and/or the learner, and repeating. Pedro Domingos professor at University of Washington author of The Master Algorithm 0.9 Hours load 2TB Instant 24 NVIDIA DGX-1 Local SSDs NVIDIA DGX-1 Pure FlashBlade

25 CASE STUDIES 25

26 MAKING AUTONOMOUS CARS POSSIBLE BY 2021 Zenuity, a joint venture of Volvo and Autoliv, chose NVIDIA DGX-1 and Pure FlashBlade systems for their deep learning infrastructure. 26

27 10X FASTER INVESTMENT DECISIONS WITH PURE FLASHBLADE Our quants want to test a model, get the results, and then test another one all day long. 27 Gary Collier, co-cto, Man AHL

28 LESSONS FROM CUSTOMERS HOW IS STORAGE IMPORTANT? DATA COPY ELIMINATION Dramatically improve time to results & time to first result PERFORMANCE & SCALABILITY Support evolving data pipeline with varying I/O patterns SIMPLICITY Focus more on AI, less on infrastructure 28

29