Multivariate models and machine learning for fmri

Transcription

1 Multivariate models and machine learning for fmri Methods and Models in fmri, Jakob Heinzle Translational Neuromodeling Unit (TNU) Institute for Biomedical Engineering (IBT) University and ETH Zürich Many thanks to Sudhir Raman and Kay Brodersen for material 1 Translational Neuromodeling Unit

2 Overview Motivation Modelling Terminology Learning from data Multivariate Bayes in SPM Generative Embedding fmri Analysis and Classifcation 2

3 Why multivariate? Univariate approaches are excellent for localizing activations in individual voxels. * n.s. v 1 v 2 v 1 v 2 reward no reward

4 Why multivariate? Multivariate approaches can be used to examine responses that are jointly encoded in multiple voxels. n.s. n.s. v 1 v 2 v 1 v 2 v 2 orange juice apple juice v 1

5 A bit of history Multidymensional scaling Psychophysical rating fmri Two-dimensional projection of similarity measure for both psychophysical rating and fmri response. Edelman et al, Psychobiology, 1998 fmri Analysis and Classifcation 5

6 A bit of history Classification Studies Haxby et al, Science, 2001 fmri Analysis and Classifcation 6

7 A bit of history Classification Studies Kamitani and Tong, Nat Neurosci, 2005 fmri Analysis and Classifcation 7

8 Representational similarity analysis Idea: Compare the similarity of representations (correlation between activation patterns) between different stimuli. Allows for a comparison between monkey (neural firing pattern) and human (fmri activation patterns). Kriegeskorte et al, Neuron, 2008 fmri Analysis and Classifcation 8

9 Overview Motivation Modelling Terminology Learning from data Multivariate Bayes in SPM Generative Embedding fmri Analysis and Classifcation 9

10 Analysis steps Feature Extraction Classification Clustering Modelling Regression Inference Cross validation Performance Prediction Model Selection

11 Feature space F 1 F 2... F P S Features Data Points S S N Discrete Continuous

12 Feature selection for fmri multivariate analysis Different features answer different questions. Reducing the dimensionality might reduce noise, but could also reduce relevant information. Model parameters Mean values Raw data Correlations between regions Model Parameters, e.g. DCM fmri Analysis and Classifcation 12

13 Model selection - Generalizability Model Fit Model Complexity Bishop (2006), Pitt & Miyung (2002), TICS fmri Analysis and Classifcation 13

14 Encoding and decoding models condition stimulus response prediction error encoding model gg: XX tt YY tt decoding model h: YY tt XX tt context (cause or consequence) XX tt R dd BOLD signal YY tt R vv fmri Analysis and Classifcation 14

15 Modelling goals Prediction Y h X Predictive Density

16 Modelling goals Model Selection Sparse Coding Distributed Coding Model Evidence

17 Overview Motivation Modelling Concepts Learning From Data Multivariate Bayes in SPM Generative Embedding

18 Learning from data Supervised Learning Unsupervised Learning Reinforcement Learning Semi-supervised Learning Labels for training data are known! Labels for training data are NOT known!

19 Supervised learning Independent variables X Function - f Continuous dependent variable Y Categorical

20 Classification X Function - f Y Kernel Methods Support Vector Machines φφ Kernel Function K xx ii, xx jj = φφ xx ii. φφ xx jj Kernel methods for pattern analysis, Taylor, Cristianini, 2004

21 Gaussian Processes Other popular classifiers C. E. Rasmussen & C. K. I. Williams, Gaussian Processes for Machine Learning, the MIT Press, 2006, Deep Belief networks G.E. Hinton, S. Osindero, and Y. Teh, A fast learning algorithm for deep belief nets, Neural Computation, vol 18, 2006

22 Generative and Discriminative classifiers Generative classifiers Learn the parameters for the functions p(y) and p(x Y), e.g. Naïve Bayes Classifier Discriminative classifiers Learn the parameters for p(y X), e.g. logistic regression, SVM fmri Analysis and Classifcation 22

23 Cross-validation The generalization ability of a classifier can be estimated using a resampling procedure known as cross-validation. One example is 2-fold cross-validation: examples 1? training example 2?? test examples 3? ??... Model Selection Performance evaluation 1 2 folds Balanced Accuracy F1 Score performance evaluation fmri Analysis and Classifcation 23

24 Cross-validation Another commonly used variant is leave-one-out cross-validation. examples 1? training example 2?? test example 3? ? 100? folds performance evaluation In fmri often leave one-run-out fmri Analysis and Classifcation 24

25 Performance Single Subject Binomial Test pp = PP XX kk HH 0 = 1 BB kk nn, ππ 0 Brodersen et al. 2013, NeuroImage k=30!!! Cross-validated data are not necessarily binomially distributed Permutation tests are better!!! fmri Analysis and Classifcation 25

26 Performance Mulitple subjects Random effects Fixed effects Brodersen et al. 2013, NeuroImage fmri Analysis and Classifcation 26

27 Confounds GLM vs. MVPA Todd et al. 2013, NeuroImage fmri Analysis and Classifcation 27

28 Second level t-tests for accuracies? True β-values are normally distributed. True accuracies are not normal and truncated at chance. A possible solution is given by Allefeld et al. Allefeld et al. Neuroimage, 2016 fmri Analysis and Classifcation 28

29 Statistical testing with classification Within subjects: Permutation statistics Parametric tests ar not valid (assumptions not met), e.g. Biomialor t-test (c.f. Schreiber and Krekelberg, 2013). Across subjects: Assumptions for t-tests are not met Full Bayesian model (Bordersen et al. 2013, but assumptions are not met for CV) Use prevalence statistic proposed in Allefeld et al., 2016 fmri Analysis and Classifcation 29

30 Research questions for classification Overall classification accuracy accuracy 100 % Spatial deployment of discriminative regions 80% 50 % Left or right button? Truth or lie? Healthy or ill? classification task 55% Temporal evolution of discriminability Model-based classification accuracy 100 % 50 % Accuracy rises above chance Participant indicates decision within-trial time { group 1, group 2 } Pereira et al. (2009) NeuroImage, Brodersen et al. (2009) The New Collection fmri Analysis and Classifcation 30

31 Decoding «hidden» intentions searchlight approach Haynes et al., Current Biology, 2007 fmri Analysis and Classifcation 31

32 Decoding of free decisions Decoding of fingerpresses (red line). Participants freely choose timing and hand. Earliest information about left-right long before execution free will? Soon et al., Nat Neurosci, 2008 fmri Analysis and Classifcation 32

33 Decoding task preparation connectitivy based decoding SV-Classifier on connectivity graph (correlation) Discriminative maps Heinzle et al., J Neurosci, 2012 fmri Analysis and Classifcation 33

34 Unsupervised learning Building a representation of data Dimensionality Reduction Clustering Time series K-means Mixture models fmri Analysis and Classifcation 34

35 K-means clustering Cost function Algorithm 1. Initialize 2. Estimate assignments 3. Estimate cluster centroids 4. Repeat 2,3 until convergence Bishop PRML (2006) fmri Analysis and Classifcation 35

36 Clustering Mixture of Gaussians Bishop PRML (2006) fmri Analysis and Classifcation 36

37 Interpretation Cluster parameters Cluster 1 Cluster 2 Internal Criterion Model Evidence External Criterion - Purity Inferred Labels Subjects External Labels fmri Analysis and Classifcation 37

38 Motivation Modelling Learning from Data Multivariate Bayes in SPM Generative Embedding fmri Analysis and Classifcation 38

39 Encoding vs. Decoding models fmri Analysis and Classifcation 39

40 Encoding vs. Decoding models fmri Analysis and Classifcation 40

41 Coding Hypotheses Sparse vectors Spatial vectors Smooth vectors Distributed vectors Singular vectors of data UUUUVV TT = RRYY TT Support vectors UU = RRYY TT fmri Analysis and Classifcation 41

42 Coding Hypotheses Friston et al NeuroImage fmri Analysis and Classifcation 42

43 Solved with variational Bayes Friston et al NeuroImage fmri Analysis and Classifcation 43

44 Example Decoding of motion. Attention to motion dataset - Büchel & Friston 1999 Cerebral Cortex Experimental factors: 1. Photic 2. Motion 3. Attention Friston et al NeuroImage fmri Analysis and Classifcation 44

45 Friston et al NeuroImage fmri Analysis and Classifcation 45

46 Results Friston et al NeuroImage fmri Analysis and Classifcation 46

47 Multivariate Bayes in SPM Motion contrast(s) < < 3 SPMmip [-36, -87, -3] < SPM{T 338 } SPMresults:.\SPM-practical\attention\GLM Height threshold T = {p<0.05 (FWE)} log-evidence maximum p = % 500 distribution of weights frequency adjusted response partitions PPM: MVB_Motion (Motion) MVB_Motion (prior: sparse) target prediction prediction Posterior probabilities voxel-weight at maxima p( w > 0) location (x,y,z) weight (w) p = ,-90.0,-3.0mm q = ; p = ,-99.0,-3.0mm q = ; p = ,-99.0,3.0mm q = ; p = ,-90.0,9.0mm q = ; p = ,-75.0,-3.0mm q = ; p = ,-84.0,6.0mm q = ; p = ,-87.0,3.0mm q = ; p = ,-84.0,-6.0mm q = ; p = ,-81.0,-15.0mm q = ; p = ,-84.0,12.0mm q = ; p = ,-84.0,-3.0mm q = ; p = ,-75.0,3.0mm q = ; 506 observed voxels; 360 and scans predicted contrast SNR (variance) scans contrast fmri Analysis and Classifcation 47

48 Laminar activity related to novelty and episodic memory Maas et al Nature Communications fmri Analysis and Classifcation 48

49 Motivation Modelling Principles Learning from Data Multivariate Bayes in SPM Generative Embedding fmri Analysis and Classifcation 49

50 Classifying Groups of Subjects Voxel activity Connectivity Subject 1 Subject 2. Dynamic causal model (DCM) High dimensionality Unusual cluster distributions Lack of interpretation Subject 1 Subject 2. Subject N Classification Clustering Subject N Group 1 Group 2 fmri Analysis and Classifcation 50

51 Generative Embedding Brodersen et al. PLOS computation biology fmri Analysis and Classifcation 51

52 DCM for speech processing fmri Analysis and Classifcation 52

53 Working memory in Schizophrenia 41 Schizophrenia patients (DSM IV,ICD 10), 42 controls Visual numeric n-back working memory task 1 900ms ms Deserno et al (2012) The Journal of Neuroscience fmri Analysis and Classifcation 53

54 Model based clustering Brodersen et al 2014 Neuroimage fmri Analysis and Classifcation 54

55 Results healthy vs. schizophrenia patients Brodersen et al 2014 Neuroimage fmri Analysis and Classifcation 55

56 Within patients clustering Brodersen et al 2014 Neuroimage fmri Analysis and Classifcation 56

57 Be aware Interpretation of decoding or classification results is difficult. The decoded information must be in the data, but in what features exactly is often hard to find out fmri Analysis and Classifcation 57

58 Summary Summary Modelling Principles Learning from Data Multivariate Bayes in SPM Generative Embedding fmri Analysis and Classifcation 58

59 Acknowledgments Many thanks to K.E. Stephan, Sudhir S. Raman and K. Brodersen for sharing their teaching material. fmri Analysis and Classifcation 59