Departmental Summative Assessments REVIEW AND UPDATE Sara Fazio, MD Chair, PCE Steering Committee John L. Dalrymple, MD Chair, Assessment Committee Harvard Medical School
Grading in HMS Clerkships PCE Core Clerkships Many clinical assessments: EPAs, formative evals from faculty and residents, shelf scores, mini-cex, oral exams, clerkship-specific OSCEs a) formative narrative b) summative narrative (in MSPE) c) grade of Satisfactory/Unsatisfactory Advanced Electives and Sub-Internships º Continue to have current grades: Honors-with-Distinction; Honors, Pass, Unsatisfactory º Even though most students will have more of these than in the past, will these be enough for residency selection process? 2
Departmental Summative Assessment A Departmental Summative Assessment (DSA) provides a comprehensive and longitudinal assessment of a student s clinical skill development and achievement in a field highly relevant to the student s residency application. This summative assessment is based on at least two different clinical rotations that span the student s clinical curricular experience, from the PCE clerkships to at least one relevant advanced clinical elective or sub-i. Both the student s trajectory and the student s competency level at the time of residency application determine the DSA. 3
Departmental Summative Assessments - Three step Process Teaching Faculty Completes Student Performance Evaluation on OASIS during PCE rotations Submits formative feedback to CD Clerkship Director Sub-I/ACE Director Completes CD Summary Form; Final EPA checklist Compose Formative and Summative Narratives Assigns SATISFACTORY/UNSATISFACTORY Completes Summary Form; Final EPA checklist Compose Formative and Summative Narratives Assigns HD/H/P/U Departmental Assessment Committee Reviews all clinical evaluations (Yrs 2-3) Assigns HD/H/P/U (Departmental Summative Assessment) CDs + Sub-I course directors + select course directors from ACE 4
Departmental Summative Assessments - Key Elements DSAs assigned by student s chosen specialty º 8 DSAs reflect 8 core clerkships º Targeted survey of select PDs across country and HMS PDs º Up to two (2) DSAs for any student; IM for all students 5
DSAs Assigned by Specialty Specialty DSAs Specialty DSAs Anesthesiology Medicine Orthopedics Medicine, Surgery Child Neurology Medicine, Neurology Otolaryngology Medicine, Surgery Dermatology Medicine Pathology Medicine Emergency Medicine Medicine Pediatrics Medicine, Pediatrics Family Medicine Medicine, Primary Care PM&R Medicine General Surgery Medicine, Surgery Plastic Surgery Medicine, Surgery Internal Medicine - Categorical Medicine Psychiatry Medicine, Psychiatry Internal Medicine Medicine, Primary Care Radiation Primary Care Oncology Medicine Medicine-Pediatrics Medicine, Pediatrics Radiology Medicine, Radiology Neurology Medicine, Neurology Thoracic Surgery Medicine, Surgery Neurosurgery Medicine, Surgery Urology Medicine, Surgery OBGYN Medicine, OBGYN Vascular Surgery Medicine, Surgery Ophthalmology Medicine 6
Departmental Summative Assessments - Key Elements DSAs assigned by student s chosen specialty º 8 DSAs reflect 8 core clerkships º Up to two (2) DSAs for any student; IM for all students º Targeted survey of select PDs across country and HMS PDs Reporting º HD/H/P/U; Summer/Fall before graduation º Grade on MSPE and transcript; No narrative º Grades still given in all 3 rd /4 th year electives/sub-i s Components º PCE clerkship + sub-i + clin electives in a given field (e.g. IM, OB) º Assignment based on clinical content, not home department º Same inputs as current clerkship grade (+ others following PCE) º Formative and summative narratives from Clerkships 7
Departmental Summative Assessments - Key Elements (cont.) Components not included º Away electives; Non-clinical electives/research º If no further clinical work beyond PCE, then a DSA will NOT be given Criteria º Determined by Departmental Assessment Committees (DACs) º All inputs considered; no threshold or cut-offs (may contribute a %) º Not a weighted average; trajectory important; professionalism º Criteria-referenced not norm-referenced Departmental Assessment Committees º CDs (Clerkship Committees); Sub-I directors º Directors of Advanced clinical elective taken by many students º Fully supported by HMS central staff 8
Questions? Comments? 9
Thank you! 10
EPA Description Entrustable Professional Activities Mapped Across HMS Pre- PCE PCE Sub-I Post-PCE CLINICAL ELECTIVES Inpt Amb Consult 1A Gather a history X X X X X X X 1B Perform a physical examination X X X X X X X 2 Prioritize DDx following clinical encounter X X X X X X X 3 Recommend/interpret common Dx/screening tests X X X X X X 4 Enter and discuss orders and Rx X +/- +/- +/- X 5 Document clinical encounter in pt record X X X X X X X 6 Provide oral presentation of clin encounter X X X X X X X 7 Form clinical questions and retrieve evidence to advance pt care X X X X X X 8 Give/receive patient handover to transition care responsibility X +/- +/- +/- X 9 Collaborate as a member of an IP team X X X X X X 10 Recognize a pt requiring urgent/emergent care; initiate evaluation X X +/- +/- X 11 Obtain informed consent for tests/procedures X +/- +/- +/- X 12 Perform general procedures of a physician +/- X +/- +/- +/- X 13 ID system failures; contribute to culture of safety and improvement X +/- +/- +/- X Capstone 11
EPA Description Entrustable Professional Activities Mapped Across HMS Pre- PCE PCE Sub-I Post-PCE CLINICAL ELECTIVES Inpt Amb Consult 1A Gather a history X X X X X X X 1B Perform a physical examination X X X X X X X 2 Prioritize DDx following clinical encounter X X X X X X X 3 Recommend/interpret common Dx/screening tests X X X X X X 4 Enter and discuss orders and Rx X +/- +/- +/- X 5 Document clinical encounter in pt record X X X X X X X 6 Provide oral presentation of clin encounter X X X X X X X 7 Form clinical questions and retrieve evidence to advance pt care X X X X X X 8 Give/receive patient handover to transition care responsibility X +/- +/- +/- X 9 Collaborate as a member of an IP team X X X X X X 10 Recognize a pt requiring urgent/emergent care; initiate evaluation X X +/- +/- X 11 Obtain informed consent for tests/procedures X +/- +/- +/- X 12 Perform general procedures of a physician +/- X +/- +/- +/- X 13 ID system failures; contribute to culture of safety and improvement X +/- +/- +/- X Capstone 12
EPA Description Entrustable Professional Activities Mapped Across HMS Pre- PCE PCE Sub-I Post-PCE CLINICAL ELECTIVES Inpt Amb Consult 1A Gather a history X X X X X X X 1B Perform a physical examination X X X X X X X 2 Prioritize DDx following clinical encounter X X X X X X X 3 Recommend/interpret common Dx/screening tests X X X X X X 4 Enter and discuss orders and Rx X +/- +/- +/- X 5 Document clinical encounter in pt record X X X X X X X 6 Provide oral presentation of clin encounter X X X X X X X 7 Form clinical questions and retrieve evidence to advance pt care X X X X X X 8 Give/receive patient handover to transition care responsibility X +/- +/- +/- X 9 Collaborate as a member of an IP team X X X X X X 10 Recognize a pt requiring urgent/emergent care; initiate evaluation X X +/- +/- X 11 Obtain informed consent for tests/procedures X +/- +/- +/- X 12 Perform general procedures of a physician +/- X +/- +/- +/- X 13 ID system failures; contribute to culture of safety and improvement X +/- +/- +/- X Capstone 13