Machine Learning and Privacy. Vitaly Shmatikov

Transcription

1 Machine Learning and Privacy Vitaly Shmatikov

2

3

4 Typical Task: Classification Training Set Query Classification result airplane automobile ship truck slide 4

5 Deep Neural Networks input output slide 5

6 activation signals Deep Neural Networks bias 1 w_0 neurons in the previous layer a_1 a_2 a_k w_1 w_2 w_k parameters f(wa) activation function next layer Learn parameters using Stochastic Gradient Descent (SGD) slide 6

7 Parameter Training using SGD w_13 w_11 w_12 w_23 w_21 w_22 w_33 w_31 w_32 w_41 w_42 w_43 w_1i w_2j w_3k w_4n Find parameters that minimize the classification error slide 7

8 Parameter Training using SGD ) 1Feed-Forward input output slide 8

9 Parameter Training using SGD ) 2Back-propagation error slide 9

10 Parameter Training using SGD ) 3Gradient Descent error E slide 10

11 Parameter Training using SGD Parameter Update Repeat for new batches of training data slide 11

12 2014 Users data Services Threats Collection of sensitive personal data Anonymization and re-identification Inference attacks Side channels slide 12

13 2018 Users data Machine learning Services Do trained models leak sensitive data? Is it possible to train a good model while respecting privacy of training data? Is it possible to keep the model itself private? slide 13

14 Model Inversion Fredrikson et al. Given an output of a machine learning model, infer something about the input unexpected attributes slide 14

15 Model Inversion in Action Model given patient s genome determine correct warfarin dosage Privacy breach : given patient s warfarin dosage, infer information about patient s genome What does this chart measure? slide 15

16 Does Inference Breach Privacy? Model training set slide 16

17 Recommended Reading Frank McSherry. Statistical inference considered harmful slide 17

18 Machine Learning as a Service Model Prediction API Training API Input from users, apps Classification DATA Sensitive! Transactions, preferences, online and offline behavior slide 18

19 Exploiting Trained Models Model Prediction API Training API Input from the training set Classification DATA Input not from the training set Classification recognize the difference slide 19

20 ML Against ML Model without knowing the specifics of the actual model! Prediction API Training API Input from the training set Classification DATA Train a model to Input not from the training set Classification recognize the difference slide 20

21 Training Attack Model using Shadow Models Target Model Shadow Model 1 Shadow Model 2 Shadow Model k Train 1 classification classification Test 1 Train 2 Test 2 Train k Test k classification IN OUT IN OUT IN OUT Train the attack model to predict if an input was a member of the training set (in ) or a non-member (out) slide 21

22 Training Data for Shadow Models Real: must be similar to training data of the target model (drawn from same distribution) Synthetic: sample feature values from (known) marginal distributions Synthetic: exploit target model Sample from inputs classified by the target model with high confidence Confidence of target model s predictions input space target s training inputs slide 22

23 Synthesizing Shadow Training Data Hill-climb the space of possible inputs to find those classified by the target model with high confidence Sample from these inputs to synthesize the training dataset for shadow models If many candidate inputs rejected by the target model, re-randomize some features and try again slide 23

24 Membership Inference Attack Input (data) Output (classes and confidence values) airplane automobile ship truck Was this image part of the training set? slide 24

25 Model Prediction API Training API target data record DATA Membership Inference Attack Was this record in the training set? Training Set slide 25

26 Minimum Attack Accuracy on 75% of classes Purchase Dataset Classify Customers slide 26

27 Next Step: Reconstruction Model Prediction API Training API Partial record????? DATA Auxiliary information, public databases, accidentally revealed data INFER hidden parts of the customer record Example: store purchases or mobile phone locations slide 27

28 Why Do These Attacks Work? Model Overfitted! Prediction API Training API Membership Inference Reconstruction DATA slide 28

29 Attack Success vs. Test-Train Gap More overfitted slide 29

30 Privacy : Does the model leak information about data in the training set? Learning : Does the model generalize to data outside the training set? Model training set Overfitting is the common enemy data universe slide 30

31 Does Inference Breach Privacy? SCIENCE! Model training set PRIVACY BREACH! Privacy breach = risk of membership: Gap between what can be inferred from the model about a member of the training set and an arbitrary input from the population slide 31

32 Non-Members Risk of membership Baseline (use statistics) Members of Training Set Purchase Dataset Classify Customers Google API slide 32

33 Future Modern machine learning is both a threat and an opportunity for data privacy For once, privacy and utility are not in conflict: overfitting is the common enemy Overfitted models leak training data Overfitted models lack predictive power Need generalizability and accuracy slide 33

34 Utility Privacy-preserving machine learning Privacy slide 34