Update Milestone content review CCC: who, what, how Assessment tools
3
Milestones are progressive over time. There is no prescribed speed. Levels do not correspond to PGY or year in program.
Patient Care and Medical Knowledge Medical Knowledge Ischemic Heart Disease Congenital Heart Disease Cardiopulmonary Bypass, Myocardial End Stage Cardiopulmonary Disease Protection and Temporary Circulatory Support Valvular Disease Great Vessel Disease General Competencies Critical Care Professionalism Ethics and Values; Personal Accountability Esophagus Practice Based Learning and Improvement Learning; Research and Teaching Lung and Airway Interpersonal and Communication Skills Chest Wall/Pleura/Mediastinum Systems Based Practice Patient Safety; Resource Allocation; Practice Management
Patient Care and Medical Knowledge Medical Knowledge General Competencies Professionalism Ethics and Values; Personal Accountability Practice Based Learning and Improvement Learning; Research and Teaching Interpersonal and Communication Skills Systems Based Practice Patient Safety; Resource Allocation; Practice Management
Residents are not required or expected to reach Level 5 by the end of training. When selecting the best Level for a resident Selecting a middle box implies that all milestones in that level and in lower levels have been attained. Selecting between levels all milestones in lower levels have been attained as well as some milestones in the higher level(s). 7
Thoracic Surgery Reporting Milestones Timeline Jan-Mar 2014 -Form CCC, meet, review & selects assessment tools -Develop new tools to fill any assessment gaps Apr-Jun 2014 -CCC meets mo -Implement tools, Beta test, make adjustments We are here. Jul Nov 2014 Perform assessments and collect data Nov-Dec 2013 CCC meets and assigns Milestones July 1, 2014 TS Milestone Begin January 2015 1st Milestones document due to ACGME January 2014 April 2014 July 2014 October 2014 ACGME expects programs to implement Subspecialty Reporting Milestones for AY 2014-2015
Surgical Specialty Milestones Specialty # Milestones Faculty Development Congenital Cardiac 9 +++ General Surgery 16 0 Otolaryngology 17 0 Colorectal 20 0 Neurosurgery 24 0 Pediatric Surgery 24 0 Thoracic 26 +++ Plastics* 30 0 Vascular 31 0 Urology 32 + Orthopedic* 40 0 MEAN 24
CCC Team PD/ Assoc PD Clinical Chair/Chief Chief Residents Chair Program Coordinator Core Teaching/ Advisory Faculty
Limit the size (4-6) Broad representation of Disciplines, experience, locations Non-faculty (e.g. RN s, midlevels, perfusionists) Educators (PhD, RN)
Review/develop/implement assessment tools Review trainee portfolios Enforce completion of assessment data Review assessment data for accuracy Assign milestone levels for each trainee biannually Develop remediation strategies Review faculty and trainee surveys, peer evaluations
Review current assessment tools and find strengths and weaknesses Implement assessment tools that are practical with maximal faculty compliance Intervention strategies for struggling residents/remediation plans Provide career development
Everyone must buy in Improve timeliness and quality of evals/feedback All members need to be fair Implement assessment tools that are practical with maximal faculty compliance Intervention strategies for struggling residents/remediation plans Provide career development
Each CCC member is responsible for one trainee (their assigned mentee) best for small programs Each CCC member is responsible for one or more competencies for all trainees All CCC members review all assessment data for all trainees and come to milestone level consensus
Program Review and Development Mapping experiences Faculty/resident development Educate leadership Develop/Adopt Systems Pilot testing Study for validity Assessment Methods and Validation Data/ Assessment System Development Data manager Professional development Reporting mechanism Provide feedback Remediation Develop assessment curricula Modify educational experiences Summative Assessment and Feedback
Selecting/developing effective assessment tools Incorporating milestone assessments Subgrouping PGY years Faculty development Individualize curriculum Time commitment
Domain Ave Time (hrs/resident) Program Review/Devt 17.3 Assessment Methods/Validation 12.4 Data System Devt 6.6 Summative Assessment/Feedback 18.3 Total 54.6
Milestones and evaluation principles Integrating/synthesizing data Providing timely and appropriate feedback Thresholds of concern
Are meetings and practices peer protected Meeting minutes: Discoverable How will differences between CCC chair/ members /PD be managed What is the appeals process
Formative feedback and summative evaluation of trainees Resident promotion decisions Curriculum and educational program assessment and improvement Educational research
gmeclient milestones
Admin home User home
Admin home User home
#1: Dependent on your Institution s Graduate Medical Committee Being able to import and show summary data from evaluations/tools Generate simple counts of how many evaluators chose a certain answer to a certain question Connect evaluation questions to Milestones reports: by competency, unique questions, and/or directly verbatim
ITS Structured case discussion Oral exams Video assessment Simulation Moodle exams Direct observation Case logs Structured case discussion Oral exams Video assessments Simulation Chart review Case logs Patient outcomes Direct observation Chart review Pt survey Oral exams Video assessments Simulation Direct observation 360 assessment Chart review Structured case discussions Video assessments Direct observation Pt survey Video assessments Simulation
Record Review Chart Stimulation Recall Check List Global Rating Standardized Patients OSCE/ CASPE Simulations and Models 360 Global Rating Learning Portfolios ITE Mock Oral Exam Procedures and Case Logs Patient Survey
SESATS Moodle Courses TSC Curriculum and Modules Simulation / Video Assessment Database Patient Outcomes Observation of Patient Encounters Presentation skills Patient evaluation QI Review Residents as educator Chart audit
Implementation July 1, 2014 Documentation Jan 1, 2015 Program coordinator support is most critical Semi-annual assessment of residents by CCC Select and implement assessment tools Faculty buy-in and development Various methods in mapping and housing assessments to the Milestones in development
syang@jhmi.edu