The Health Economics and Outcomes Research Applications and Valuation of Digital Health Technologies and Machine Learning Workshop W29 - Session V 3:00 4:00pm May 25, 2016 ISPOR 21 st Annual International Meeting Washington, DC Page 1 Discussion Leaders Machine Learning and Regression Models for Prediction: Statistical Background Mei Sheng Duh, MPH, ScD Managing Principal and Chief Epidemiologist Analysis Group, Inc. Ensembles for Health Economics Research Sherri Rose, PhD Associate Professor, Harvard Medical School Application of Machine Learning in a Real Time Big Data Setting Gigi Yuen-Reed, PhD Program Director, Data Science Solutions IBM Watson Health Innovation Uses of Machine Learning Approaches with Real World Evidence Steven Pashko, PhD Principal Offering Manager IBM Watson Health, Life Sciences Real-World Evidence Page 2 1
Machine Learning and Regression Models for Prediction: Statistical Background Presented by Mei Sheng Duh, MPH, ScD Analysis Group, Inc. BOSTON BEIJING CHICAGO DALLAS DENVER LOS ANGELES MENLO PARK MONTREAL NEW YORK SAN FRANCISCO WASHINGTON, DC Background Machine learning (ML) focuses on pattern recognition and computational learning, often used to create predictive algorithms or to make classifications based on data Unlike traditional methods, ML methods are capable of analyzing highdimensional data and exploring unknown patterns among covariates without substantial a priori knowledge of covariate relationships Page 4 Page 4 2
Types of ML Supervised Learning: True outcome is available, and the goal is to learn covariate patterns that predict the outcome Neural network Support vector machine (SVM) Random forest For discussion Unsupervised Learning: True outcome is unavailable, and the goal is to discover underlying patterns Reinforcement Learning: To learn a behavior by interacting in an environment and receiving rewards Page 5 Page 5 Neural Network A neural network is a non-linear network of neurons inspired by the human brain. It is configured for applications such as pattern recognition and data classification through a learning process The learning process involves adaptive adjustments to the connections between the neurons Page 6 Page 6 3
Support Vector Machine A SVM is a discriminative classifier formally defined by a separating line A non-linear boundary can be defined using kernels Page 7 Page 7 Random Forest A random forest aggregates thousands of decision trees to identify the most important predictors of an outcome and then predicts that outcome It also corrects for overfitting that can occur with a single decision tree Page 8 Page 8 4
Comparing Logistic Regression to Machine Learning Methods with a Data Example Goal: To predict which patients have liver cirrhosis using age and treatment duration as covariates Patient ID Is the patient a case? Age 1 Yes 31 60 2 No 54 45 3 Yes 65 73 4 Yes 43 32. Treatment duration (months) Page 9 Page 9 Logistic Regression with Main Effects A logistic regression model including only main effects of the two covariates. The predicted probability thresholds are formulated as: Figure 1. Contour mapping between input and output spaces for the logistic regression model including only main effects Page 10 Page 10 5
Logistic Regression with Interaction Effects A logistic regression model including both main and interaction effects of the two covariates. The predicted probability thresholds are formulated as: Figure 2. Contour mapping between input and output spaces for the logistic regression model including both main and interaction effects Page 11 Page 11 Neural Network A neural network model can classify cases and controls non-linearly and account for idiosyncrasies in the data Figure 3. Contour mapping between input and output spaces for the neural network model Page 12 Page 12 6
Support Vector Machine A SVM can output a non-linear boundary to classify cases and controls Legend: Neural network SVM Figure 4. Contour mapping between input and output spaces for the neural network model and a decision boundary for the SVM Page 13 Page 13 Random Forest A random forest can subdivide the data into regions that are roughly uniform in values (either cases or controls) First split by duration Third split by duration Legend: Neural network Random forest Second split by age Figure 5. Contour mapping between input and output spaces for the neural network model and random forest models Page 14 Page 14 7