About the Panel Unintended Consequences of Best Intentions: The True Medical School Story? Daniel A. London, MS Mary C. Anderson, MD John D. Mahan, MD Jeffrey S. Zabinski, MSSA/MA Friday, April 10 th, 2015 Daniel London 5 th year medical student at the Cleveland Clinic Lerner College of Medicine Dr. Mary Anderson Assistant Dean, Professional Development, Rush Medical College (former) Dr. John Mahan Program Director, Pediatrics, Nationwide Children s Hospital Jeffrey Zabinski 4 th year medical student at Boonshoft School of Medicine, Wright State University The Two Problems 1. Qualities for success in the residency application process may appear and often are in conflict with the qualities we select for in medical school admissions, and this results in confusing messages and detrimental impacts on students 2. Solutions that rectify these conflicts and will benefit all parties from medical school admissions, through the medical school curriculum, and into residency training are needed Only the Best Intentions The vision of the AAMC and its members is a healthy nation and world in which: America's system of medical education prepares physicians and scientists to meet the nation's evolving health needs The nation's medical students residents, fellows, [and] faculty...are diverse and culturally competent Concern for compassion professionalism, and the public good guide the health care community https://www.aamc.org/about/ Only the Best Intentions The vision of the AAMC and its members is a healthy nation and world in which: America's system of medical education prepares physicians and scientists to meet the nation's evolving health needs The nation's medical students residents, fellows, [and] faculty...are diverse and culturally competent Concern for compassion professionalism, and the public good guide the health care community Only the Best Intentions The vision of the AAMC and its members is a healthy nation and world in which: America's system of medical education prepares physicians and scientists to meet the nation's evolving health needs The nation's medical students residents, fellows, [and] faculty...are diverse and culturally competent Concern for compassion professionalism, and the public good guide the health care community https://www.aamc.org/about/ https://www.aamc.org/about/ 1
Only the Best Intentions The vision of the AAMC and its members is a healthy nation and world in which: America's system of medical education prepares physicians and scientists to meet the nation's evolving health needs The nation's medical students residents, fellows, [and] faculty...are diverse and culturally competent Concern for compassion professionalism, and the public good guide the health care community https://www.aamc.org/about/ Only the Best Intentions The vision of the AAMC and its members is a healthy nation and world in which: America's system of medical education prepares physicians and scientists to meet the nation's evolving health needs The nation's medical students residents, fellows, [and] faculty...are diverse and culturally competent Concern for compassion professionalism, and the public good guide the health care community The mission [of medical school x]: To deliver outstanding medical education focused on patient care, research, and community service Our diverse students learn in a model, which promotes collaboration, accountability and respect We graduate physicians who are dedicated to the pursuit of excellence in clinical practice, research, and service https://www.aamc.org/about/ http://www.rushu.rush.edu Only the Best Intentions The vision of the AAMC and its members is a healthy nation and world in which: America's system of medical education prepares physicians and scientists to meet the nation's evolving health needs The nation's best medical laid schemes students residents, o' Mice fellows, an' Men, [and] faculty...are diverse and culturally competent Concern for compassion professionalism, Gang aft agley, and the public good guide the health Robert care Burns, community 1785 The mission [of medical school x]: To deliver outstanding medical education focused on patient care, research, and community service Our diverse students learn in a model, which promotes collaboration, accountability and respect We graduate physicians who are dedicated to the pursuit of excellence in clinical practice, research, and service https://www.aamc.org/about/ http://www.rushu.rush.edu Medical Education Environment AAMC predicts a shortage of 46,000 90,000 physicians by 2025 Call for 30% increase in medical student enrollment Indirect control Call for concomitant increase in GME spots No control due to Congressional cap on Medicare GME support https://www.aamc.org/newsroom/newsreleases/426166/20150303.html 30% Increase in Medical Students Met Number of Students Increasing Competitiveness of Medical School Admissions 50,000 45,000 40,000 35,000 30,000 25,000 20,000 15,000 10,000 5,000 0 50% 49% 48% 47% 46% 45% 44% 43% 42% 41% 40% Percentage Year Applicants Matriculants Acceptance Rate Results from the AAMC s 2013 Medical School Enrollment Survey Data adapted from Table 17: MCAT Scores and GPAs for Applicants and Matriculants to U.S. Medical Schools, 2003 2014 2
Average GPA Academic Profiles of Medical School Applicants and Matriculants by Year 3.75 3.70 3.65 3.60 3.55 3.50 3.45 Year Applicant Total GPA Matriculant Total GPA Data adapted from Table 17: Applicant MCAT Scores Total and MCAT GPAs for Applicants Matriculant and Matriculants Total MCAT to U.S. Medical Schools, 2003 2014 32 31 30 29 28 27 26 Average MCAT Selecting Students via Holistic Review Holistic review is a flexible, individualized way of assessing an applicant s capabilities by which balanced consideration is given to experiences, attributes, and academic metrics and, when considered in combination, how the individual might contribute value as a medical student and physician. Image and text taken from AAMC s Advancing Holistic Review Initiative Brochure Selecting Students via Holistic Review Holistic review is a flexible, individualized way of assessing an applicant s capabilities by which balanced consideration is given to experiences, attributes, and academic metrics and, when considered in combination, how the individual might contribute value as a medical student and physician. Selecting Students via Holistic Review Holistic review is a flexible, individualized way of assessing an applicant s capabilities by which balanced consideration is given to experiences, attributes, and academic metrics and, when considered in combination, how the individual might contribute value as a medical student and physician. Image and text taken from AAMC s Advancing Holistic Review Initiative Brochure Image and text taken from AAMC s Advancing Holistic Review Initiative Brochure Selecting Students via Holistic Review Holistic review is a flexible, individualized way of assessing an applicant s capabilities by which balanced consideration is given to experiences, attributes, and academic metrics and, when considered in combination, how the individual might contribute value as a medical student and physician. Selecting Students via Holistic Review Holistic review is a flexible, individualized way of assessing an applicant s capabilities by which balanced consideration is given to experiences, attributes, and academic metrics and, when considered in combination, how the individual might contribute value as a medical student and physician. Image and text taken from AAMC s Advancing Holistic Review Initiative Brochure Image and text taken from AAMC s Advancing Holistic Review Initiative Brochure 3
Selecting Students via Holistic Review Holistic review is a flexible, individualized way of assessing an applicant s capabilities by which balanced consideration is given to experiences, attributes, and academic metrics and, when considered in combination, how the individual might contribute value as a medical student and physician. Image and text taken from AAMC s Advancing Holistic Review Initiative Brochure 4
Critical Analysis and Reasoning Skills section Critical Analysis and Reasoning Skills section is designed to help medical schools assess how you think.no longer is it humanly possible to memorize every fact relevant to the practice of medicine. What is more important is an ability to think critically and to have the necessary reasoning skills to know where to seek answers and how to solve problems in the clinical environment. Critical Analysis and Reasoning Skills section is designed to help medical schools assess how you think.no longer is it humanly possible to memorize every fact relevant to the practice of medicine. What is more important is an ability to think critically and to have the necessary reasoning skills to know where to seek answers and how to solve problems in the clinical environment. Critical Analysis and Reasoning Skills section is designed to help medical schools assess how you think.no longer is it humanly possible to memorize every fact relevant to the practice of medicine. What is more important is an ability to think critically and to have the necessary reasoning skills to know where to seek answers and how to solve problems in the clinical environment. Summary of the Medical School Admissions Environment Ever increasing number of medical school applicants Ever increasing undergraduate GPAs and MCAT scores Promotion of holistic review and revamped MCAT to support selection of compassionate healers Number of PGY 1 Positions 28,000 26,000 24,000 22,000 20,000 18,000 16,000 14,000 12,000 10,000 PGY 1 Positions Offered and Filled are Increasing 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 Year Offered Filled (All) Filled (US Seniors) Data adapted from the NRMP s Main Residency Match Results 5
PGY 1 Positions Offered and Filled are Increasing Number Unfilled PGY 1 Positions are Decreasing and Unmatched US Seniors are Steady 2,300 2,100 1,900 1,700 1,500 1,300 1,100 900 700 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 Year Unfilled Positions Data adapted from the NRMP s Main Residency Match Results Unmatched US Seniors Unfilled PGY 1 Positions are Decreasing and Unmatched US Seniors are Steady Trends in Applications, Interviews, and Rankings from a Program Perspective 900 800 700 600 500 400 300 200 100 0 856 765 716 Number of Applications Number of Interviews 85 89 119 66 66 77 2010 2012 2014 Data adapted from the NRMP s Program Director Survey Number of Ranked Applicants Top 10 Interview Selection Factors 0% 20% 40% 60% 80% 100% Step 1 Score Letters of Recommendation MSPE Step 2 CK Score Personal Statement US Allopathic Graduate Clerkship Grades Gaps in Medical Education Honors in Clerkships Commitment to Specialty Class Rank Data adapted from the NRMP s Program Director Survey Percent Citing Each Factor 2008 2010 2012 2014 Increasing Use of USMLE Target Scores 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 57% 60% 63% 68% Percentage of Programs Using a Step 1 Target Score 41% 44% 48% 50% Percentage of Programs Using a Step 2CK Target Score 2008 2010 2012 2014 Data adapted from the NRMP s Program Director Survey 6
Top 10 Applicant Ranking Factors 2014 Top 10 Applicant Ranking Factors 0% 20% 40% 60% 80% 100% Interactions with Faculty Interpersonal Skills Interaction with House Staff Interactions with Residents Step 1 Score Letters of Recommendation Step 2CK Score MSPE Commitment to Specialty Professionalism 2014 Data adapted from the NRMP s Program Director Survey Summary of the Residency Selection Environment Ever increasing number of residency applicants Medical student perceptions of an increasingly competitive environment Increased number of applications per student/program Increasing use of screening measures and reliance on standardized numbers by residency programs to offer interviews Personal interactions matter most during interviews, but numbers still lurk Questions to Ponder Are the messages from stakeholders in sync? How are these trends changing the medical school experience? Is the current model sustainable? Does the entire spectrum of stakeholders in medical education UGME to GME to CME communicate enough? 7
Student A Art and literature college major Average GPA and MCAT Unsure of specialty at matriculation Participated in student faculty choir and some community service activities Average Step 1 and clinical performance Wants to match in Midwest because of strong family ties Considering radiology or ophthalmology Student B Varsity student athlete in college Above average GPA and MCAT Applied to 10 medical schools and accepted to multiple Always knew desire to specialize in orthopedic surgery Participated in orthopedic research and the orthopedic surgery student group Top quartile Step 1 scores Focused on receiving honors/as in clinical clerkships Completed three away rotations Couples matching with partner applying to internal medicine Student C Biology major Above average GPA; average MCAT Active in undergrad student government, holding elected positions Unsure of specialty at matriculation Leader in medical student government Leader in community service Below average Step 1 score Pass on pre clerkship grades Average clerkship grades Interested in dermatology Began research in dermatology 4 th year Student D Humanities major at a 'top 10 college Underrepresented minority Completed post Bac program (or Master's in Anatomy, or worked for 4 years after undergrad) Average/below average GPA and MCAT Below average grades One Step 1 failure then low pass Below average Step 2CK 2 pre clinical courses remediated Above average clinical performance with fantastic written evaluations Very active in global health/community service No research Wants to match into OB/GYN or emergency medicine Unintended Consequences of Best Intentions: The True Medical School Story? From Admission to Residency Application Applications Participated in AAMC sponsored Holistic Review workshop in 2011 Committee on Admissions Redefined criteria Redefined process Required orientation for all new and returning Screeners Interviewers CoA members 46 Admissions Philosophy Required experiences Attributes Experiences Journey Metrics Community Service More than 90 % of all RMC students participate in community service during medical school Health care experience Patient interactions important All students begin seeing patients during 2 nd week of medical school Ensure that applicants are prepared to begin seeing patients 47 48 8
Valuable experiences Examples Leadership Research Exposure to other backgrounds Elected positions Selected positions Clinical Basic Science Underserved Rural Student A Majored in arts and literature as undergrad Average GPA; average MCAT scores Participated in student faculty choir Participated in some community service activities Student B Above average undergrad GPA; above average MCAT scores Applied to 10 medical schools and accepted to multiple Varsity student athlete in college 49 50 Student C Undergad degree in Biology Above average GPA; average MCAT scores Active in student government, holding multiple elected positions as an undergrad Examples Student D Average/below average undergrad GPA Average/below average MCAT Underrepresented minority Completed post Bac program (or Master's in Anatomy, or worked for 4 years after undergrad) Undergraduate major in humanities at a 'top 10' undergraduate program 51 Advising for Residency application Begins in fall first year Workshops focus on interests and skills Introduction to match statistics Third year Workshops Discuss next steps in decision making Discuss match statistics Fourth year Discuss preferred specialty Discuss match statistics based on preferences 52 Opportunities Career Guidance Specialty Immersion Student lead Interest Groups Med College Programs Preceptor and clerkships Specialty days Specialty panel Workshop CiMexercises and assessments Advisor Student Affairs Dean 53 Top 10 Interview Selection Factors 0% 20% 40% 60% 80% 100% Step 1 Score Letters of Recommendation MSPE Step 2 CK Score Personal Statement US Allopathic Graduate Clerkship Grades Gaps in Medical Education Honors in Clerkships Commitment to Specialty Class Rank Percent Citing Each Factor 2008 2010 2012 2014 Data adapted from the NRMP s Program Director Survey 9
Unfilled PGY 1 Positions are Decreasing and Unmatched US Seniors are Steady Student A Average Step 1 performance Average clinical performance Wants to match in Midwest because of strong family ties Considering Radiology or Ophthalmology Student B Top quartile Step 1 scores Focused on receiving honors/as in clinical clerkships Participated in research in Orthopedic surgery and Orthopedic surgery student group Completed three visiting student electives in Orthopedic surgery Couples matching with female student applying to Student C Below average Step 1 score Pass on pre clerkship grades (Pass/Fail for pre clinical grades at medical school) Average clerkship grades Leader of student government, holding several elected positions Leader in community service Interested in Dermatology; began research in Dermatology during fourth year Student D Below average grades, one STEP 1 failure then low pass, below average STEP 2 Two pre clinical courses required remediation Above average clinical performance with fantastic written evaluations Very active in global health/community service No research Wants to match into OB/GYN or Emergency Medicine Underrepresented minority Internal Medicine 57 58 Residency Program Imperatives Unintended Consequences of Best Intentions: The True Medical School Story? Residency Program Director Perspectives John D Mahan MD 1. For many (but not all) disciplines, increased numbers of applicants. 2. Increased number of applications per applicant. 3. Greater diversity in qualities of applicant pool (great number of medical school, greater variability in clinical training/sites, greater emphasis by applicants/schools on non-academic accomplishments). 4. Greater understanding and emphasis on acquiring effective trainees: patient outcomes, quality/safety focus; ACGME CLER process; JC focus on hospital efforts 60 10
Residency Program Imperative Acquire the most effective residents who will take care of the hospital/institution s patients!!!! Defining and Obtaining Most Effective Residents Make good clinical decisions Care Communicate well Align with and support staff Improve institution How to achieve this is still more art than science! 61 Secondary Match Goals Prestige (highly regarded training program; high board pass rates) Faculty recruitment Faculty satisfaction Train future leaders and academicians/ future primary care providers Pipeline of future fellows/referring physicians What do Residency Programs Have to Rely Upon? Personal experience with applicants in clinical rotations Academics (USMLE scores; Grades; class rank) Basic science, clinical grades, grades in discipline MSPE language (descriptors; narrative variability) Personal statement Letters of recommendation Significant research accomplishments What do Residency Programs Have to Rely Upon? Significant volunteer accomplishments in medical school AOA, other honors Advanced Degree Underrepresented minority status Growing up in region of program Plans for subspecialty or academic career Performance on rotation in your program / at your institution NRMP Survey: Top 10 Interview Selection Factors 0% 20% 40% 60% 80% 100% Step 1 Score Letters of Recommendation MSPE Step 2 CK Score Personal Statement US Allopathic Graduate Clerkship Grades Gaps in Medical Education Honors in Clerkships Commitment to Specialty Class Rank Percent Citing Each Factor 2008 2010 2012 2014 Data adapted from the NRMP s Program Director Survey 11
What Do Residency Programs Rely Upon For Granting Interviews? USMLE Criteria for Interviews Do you utilize a USMLE threshold for offering interviews? 29.4% 68.0% Yes No 67 68 Ranking It is All About the Interview!!!! RANKING PREFERENCES: What is the most important factor that affects ranking position on the rank list USMLE scores 5.9% 88.2% 5.9% Class rank MSPE letter Basic science grades Clinical clerkships grades Grades in clinical rotation in discipline Personal statement Interview scores Significant research accomplishments Significant volunteer accomplishments in medical school AOA, other honors Advanced Degree Underrepresented minority status Growing up in region of program Plans for subspecialty or academic career Performance on rotation in your program / at your institution In Our Rank List The Ultimate Position Is Determined by These Factors Fit with residents in program Plans for subspecialty or academic career Growing up in region of program Underrepresented minority status Advanced Degree AOA, other honors Significant volunteer accomplishments in medical school Significant research accomplishments Interview scores/evaluations Personal statement Grades in clinical rotation in discipline Clinical clerkships grades Basic science grades MSPE letter Class rank USMLE scores 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0% 70 Do Any Of These Behaviors Affect Placement In The Rank List? Did rotation at your institution Expressed that the applicant will rank you highly Sent thank you note Came for a 2nd visit to your program Yes No 58% 42% 16% 84% 16% 84% 22% 78% Overarching Purpose in Constructing Your Rank List and Obtaining Your Residents 5.0% 84.0% 11.0% Best chances to achieve highest board pass rate Best quality residents Development of best subspecialists in your discipline Development of future academic scientists Development of future leaders in your discipline Promotion of primary care providers Provision of most diversity n residency program Fit with residents in program Other (please specify) 71 72 12
Decisions/Decisions Student A Average Step 1 performance Average clinical performance Wants to match in Midwest because of strong family ties Considering Radiology or Ophthalmology Student B Top quartile Step 1 scores Focused on receiving honors/as in clinical clerkships Participated in research in Orthopedic surgery and Ortho surgery student group Completed three visiting student electives in Orthopedic surgery Couples matching with female student applying to Internal Medicine Decisions/Decisions Student C Below average Step 1 score Pass on pre clerkship grades (Pass/Fail for pre clinical grades at medical school) Average clerkship grades Leader of student government, holding several elected positions Leader in community service Interested in Dermatology; began research in Dermatology during fourth year Student D Below average grades, one STEP 1 failure then low pass, below average STEP 2 Two pre clinical courses required remediation Above average clinical performance; fantastic evaluations Very active in global health/community service No research Wants to OB/GYN or Emer Med Underrepresented minority 73 74 Matching Into Residency: Program Director Final Thoughts Pre-work (homework on programs) cuts out needless over-applying Realism in goals and programs Remember energy and costs of many interviews Rely on knowledgeable advisors Likelihood of successful match in discipline Preferred number of applications Safe number of interviews and programs to rank INTERVIEW, INTERVIEW Rank based on your preferences That s why it is called a Match! A Student Perspective Jeffrey S. Zabinski, MSSA/MA 75 The Two Problems Constantly Applying 1. Qualities for success in the residency application process may appear and often are in conflict with the qualities we select for in medical school admissions, and this results in confusing messages and detrimental impacts on students 2. Solutions that rectify these conflicts and will benefit all parties from medical school admissions, through the medical school curriculum, and into residency training are needed Undergraduate Admissions Graduate School Post Bac Programs Medical School Admissions Job Applications Scholarships Research Grants Residency Applications Fellowships 13
The Information Environment Constantly Applying waste of time to apply with that score you can t apply MD you have zero chances unless you cured cancer, there s no way Undergraduate Admissions Graduate School Post Bac Programs Medical School Admissions IS THIS JUST A STORY OF NEEDING BETTER Scholarships ADVISING? Job Applications Research Grants Residency Applications Fellowships http://forums.studentdoctor.net/forums/pre medical allopathic md.10/ Undergraduate Admissions Must have high scores and be a good person Constantly Applying Medical School Admissions Are these just futurephysician checkpoints? Residency Applications Must have high scores and be a good person Solutions? Do we really need to change anything? Medical school admissions changes are the right changes, residency process should change to be in sync Residency application process trends are beneficial, medical school admissions should change to be in sync Student A Art and literature college major Average GPA and MCAT Unsure of specialty at matriculation Participated in student faculty choir and some community service activities Average Step 1 and clinical performance Wants to match in Midwest because of strong family ties Considering radiology or ophthalmology Student B Varsity student athlete in college Above average GPA and MCAT Applied to 10 medical schools and accepted to multiple Always knew desire to specialize in orthopedic surgery Participated in orthopedic research and the orthopedic surgery student group Top quartile Step 1 scores Focused on receiving honors/as in clinical clerkships Completed three away rotations Couples matching with partner applying to internal medicine Student C Biology major Above average GPA; average MCAT Active in undergrad student government, holding elected positions Unsure of specialty at matriculation Leader in medical student government Leader in community service Below average Step 1 score Pass on pre clerkship grades Average clerkship grades Interested in dermatology Began research in dermatology 4 th year Student D Humanities major at a 'top 10 college Underrepresented minority Completed post Bac program (or Master's in Anatomy, or worked for 4 years after undergrad) Average/below average GPA and MCAT Below average grades One Step 1 failure then low pass Below average Step 2CK 2 pre clinical courses remediated Above average clinical performance with fantastic written evaluations Very active in global health/community service No research Wants to match into OB/GYN or emergency medicine 14
The Two Problems Final Thoughts 1. Qualities for success in the residency application process may appear and often are in conflict with the qualities we select for in medical school admissions, and this results in confusing messages and detrimental impacts on students 2. Solutions that rectify these conflicts and will benefit all parties from medical school admissions, through the medical school curriculum, and into residency training are needed How Do We Bridge The Gap? 1. Qualities for success in the residency application process may appear and often are in conflict with the qualities we select for in medical school admissions, and this results in confusing messages and detrimental impacts on students 2. Solutions that rectify these conflicts and will benefit all parties from medical school admissions, through the medical school curriculum, and into residency training are needed The Match Draft Deciding who has the best intentions Change from the current 2 round Match setup to a multi round [5] program Limit the number of applications a student can submit per round Requires students to more honestly assess their competitiveness Decreases the number of applications a program will receive and have to sift through at one time Allows for more holistic review? Reduces stigma of SOAP as the process is repeated 4 more times 15