PC004 BR11 Developing the Evaluator: a smart approach to getting SMART results Lauren D. Garlapo Accreditation Administrator Katherine R. Cich, M.M. Assistant DIO, Director of Accreditation Roseanne C. Berger, M.D. DIO, Senior Associate Dean
Disclosure No conflicts of interest to report
Goal and Objectives Goal: To develop Program Administrators to evaluate ACGMEaccredited programs Objectives: Review a program and determine the issue(s) that require further action Evaluate a program s Action Plan for adequacy Construct a committee response to elicit SMART action plans from the program
Icebreaker: participants will work in table groups and race other tables to complete an ACGME vocabulary-based timed crossword (5 minutes)
University at Buffalo GME 60 ACGME-accredited programs: 776 residents and fellows 9 affiliated training hospitals 1 Program Quality Review Subcommittee Manager on GME staff (with many additional accreditation responsibilities)
UB GME resident survey window April APEs conducted April-June ACGME survey results June YOU ARE HERE X ACGME resident survey window January- May Most residents and fellows graduate June AIR summary presented to UB GMEC January UB GME Life Cycle APE tools completed by programs August 1st AIR conducted by special committee December Dashboard created from ACGME survey, APE & ADS April-August Full Special Reviews conducted by Ad Hoc Committees November- January Focused Special Reviews conducted by PQRS October- November Focused Special Review activities determined by PQRS September
Program Quality Review Subcommittee We accomplish a great deal of our institutional oversight through this subcommittee of our GMEC Committee membership: Chair GME staff Program Directors Program Administrators Residents/Fellows Meet monthly (except January) 2-3 weeks before our GMEC monthly meetings
Program Quality Review Subcommittee Purpose I.B.6. The GMEC must demonstrate effective oversight of underperforming programs through a Special Review process. (Core) I.B.6.a) The Special Review process must include a protocol that: (Core) I.B.6.a).(1) establishes criteria for identifying underperformance; and, (Core) I.B.6.a).(2) results in a report that describes the quality improvement goals, the corrective actions, and the process for GMEC monitoring of outcomes. (Core) PQRS exists to help our programs achieve excellence beyond the requirements
Evaluating a small fellowship WORKSHOP ACTIVITIES
Small Fellowship Accreditation Status: continued accreditation Length of training: 3 years Number of fellows in the program: 5
The UB GME Dashboard
Activity #1: Perform a 2013-2014 Annual Review on a program using the institutional dashboard (5 minutes) What, if any, areas would you like to investigate further? Share responses (2 minutes)
Annual Review Results Focused Special Review: Board Performance GME survey shows minor problems, but has improved since last year (we ll leave them alone because they re improving!) PQRS notifies program and requests SMART action plans from the Program Director responding to Board Performance (Program Administrator is copied on communication)
SMART objectives (program action plans; PQRS responses) Specific Measurable Achievable Relevant Time-based
Activity #2: Review the fellowship s Action Plans and evaluate them (5 minutes) Are the Action Plans SMART? Is any additional information needed? Share responses (5 minutes)
APE Action Plan Examples Graduate Board Certification Planned action(s) to improve program performance Name(s) or title(s) of responsible parties Date by which action(s) must be completed Assign structured reading- each fellow will work closely with his/her mentor to meet on a monthly basis, discuss studying & reading from critical care text book in order to enrich their knowledge PICU Faculty 6/30/15 Specific Measurable Achievable Relevant Time-based Develop lectures- each fellow is responsible for presenting lectures NOT Specific NOT Measurable Achievable Relevant NOT Timebased
Activity #3: Construct a committee response to elicit SMART action plans from the program (5 minutes) Include: Specific request for missing information Reasonable response deadline Share responses (5 minutes)
ldgarlap@buffalo.edu http://www.smbs.buffalo.edu/gme