Correlation of National Board of Medical Examiners Scores with United States Medical Licensing Examination Step 1 and Step 2 Scores

Transcription

1 Correlation of National Board of Medical Examiners Scores with United States Medical Licensing Examination Step 1 and Step 2 Scores Christopher M. Zahn, MD, Aaron Saguil MD, MPH, Anthony R. Artino Jr, PhD, Ting Dong, PhD, Gerald Ming, Jessica T. Servey, MD, Erin Balog, MD, Matthew Goldenberg, MD, and Steven J. Durning, MD, PhD for the Long Term Career Outcomes Study

2 Background USMLE Step 1 and 2 Clinical Knowledge (CK) Passage required for medical licensure Important medical school curricular outcome NBME Clinical Subject ( shelf ) Examination Often used as component of clerkship grade Limited evidence of correlation with Step 2 CK Department of Defense Graduate Medical Education Programs must rank students by November

3 Purpose Determine if NBME shelf examination performance at USU is correlated with USMLE Step 1 and 2 CK performance

4 Theoretical Framework Context specificity Application of knowledge is contextually dependent Something other than content drives performance Physician Similar Patients Different Conclusions Student Similar Tests Different Results

5 Conceptual Framework Uniformed Services have defined primary care needs: brigade, battalion, flight surgeons and general medical officers USU weaves primary care topics through the preclerkship and clerkship years

6 Hypotheses Individual shelf performance: small-tomoderate correlation with USMLE Step 1 and 2 CK performance Composites of primary care (FM, IM, PED) shelf performance and total (FM, IM, PED, GS, OB, PSY) shelf performance: moderate-tolarge correlations with USMLE performance

7 Methods Uniformed Services University (USU) of the Health Sciences F. Edward Hébert School of Medicine Long Term Career Outcome Study Traditional curriculum (2+2) at time of study Clerkships : family medicine (FM), internal medicine (IM), general surgery (GS), obstetrics and gynecology (OBG), pediatrics (PED), and psychiatry (PSY) Step 1 taken end of second year Step 2 CK taken fall of fourth year

8 Methods N = 507 from Classes of Variables NBME shelf Examination scores USMLE Step 1 and 2 CK scores Gender Cumulative basic science GPA Third-year clerkship GPA Analyses Descriptive Bivariate with Pearson correlations Multivariate with stepwise linear regression

9 Results-Descriptive Characteristic USU Total (%) Nation, 2010, % Students: no. 507 Gender Men: no. (%) 368 (73) 51.4 Women: no. (%) 139 (27) 48.6 Mean age (years) (median) Ethnicity African-American (%) 9 (1.8) 6.7 American Indian (%) 7 (1.4) 0.8 Asian/Pacific Islander (%) 67 (13.2) 20.3 Mexican-American/Hispanic (%) 22 (4.3) 6.4 Mean undergraduate GPA Mean MCAT sub-scores (bio/phys/verbal) 9.2/8.8/ /9.4/9.1

10 Results-Descriptive USU National Variable Mean (SD) Range Mean USMLE exams Step (17.0) Step (18.5) NBME subject exams Internal Medicine 74.2 (6.7) Family Medicine 71.1 (6.7) Pediatrics 73.0 (6.6) Surgery 70.9 (7.4) Obstetrics and Gynecology 70.4 (6.5) Psychiatry 73.8 (6.7)

11 Results-Pearson s Correlations Variable USMLE Step 1 USMLE Step 2 IM shelf FM shelf PED shelf GS shelf OBG shelf PSY shelf Secondyear GPA Thirdyear GPA USMLE Step USMLE Step IM shelf FM shelf PED shelf GS shelf OBG shelf PSY shelf Composite shelf Second-year GPA Third-year GPA * All values significant at the p < 0.01 level

12 Results-Multiple Linear Regression USMLE Step 1 USMLE Step 2 Clinical Knowledge Type of score Standardized coefficients P Standardized Coefficients P Step 1 IM FM PED R 2 Primary care NBME scores.31 < < < < < <

13 USMLE Step 1 USMLE Step 2 Clinical Knowledge Type of score Standardized coefficients P Standardized Coefficients P Step 2 IM FM PED GS OBG PSY NBME scores of Surgery, Obstetrics and Gynecology, and Psychiatry Subject Exams.20 < < < < < < < R Total model adjusted R

14 USMLE Step 1 USMLE Step 2 Clinical Knowledge Type of score Standardized coefficients P Standardized Coefficients P Step 3 IM FM PED GS OBG PSY MS2 Cum GPA (Step 1)/MS 3 GPA (Step 2) R 2 MS2 Cumulative GPA for Step 1 model MS3 GPA for Step 2 Clinical Knowledge Model < < < < < Total model adjusted R

15 Discussion NBME shelf exam scores moderately correlated with one another NBME shelf exam scores moderately-to-highly correlated with USMLE Step 1 and Step 2 CK scores Primary care composite shelf performance explains large portion of variance in Step 2 CK scores Total composite shelf performance explains majority of variance in Step 2 CK performance

16 Implications NBME shelf exam performance may be useful surrogate for USMLE Step 2 CK performance Useful in curriculum reform predicting need for pre -mediation Makes case for acting on poor performance on individual shelf

17 Limitations Single institution study Correlation vice causation Interpretation may differ depending on ordering of independent variables Timing and length of clerkships not studied

18 Future Research Multi-institution collaboration with LTCOS Prospective collection of data

19 QUESTIONS?