Annual Program Evaluation & Improvement Plan UWHC GMEOC Process & Documentation Requirements for 2009
Annual Program Evaluation & Improvement Plan Common Program Requirements Evaluation Section VC: Formal Systematic Evaluation of the Curriculum at Least Annually Effective Date: July 1, 2007
Program Must Monitor and Track Four Required Sets of Outcome Data Resident Performance Graduate Performance Faculty Development Program Quality Must include confidential evaluation from residents and faculty Must use these results and other data to improve the program
Annual Program Evaluation: Lessons Learned from 2008 Failure to include trainee(s) and faculty in process Failure to clarify purpose of meeting Differentiate between routine Education or Residency Committee Failure to ask for clinic data ahead of time Failure to review aggregate data relevant to a required topic
Annual Program Evaluation: Citations earned from 2008 The program must document formal, systematic evaluation of the curriculum at least annually. The program must monitor and track the following areas: a) resident performance, b) faculty development, c) graduate performance, including performance of graduates on the certification examination, and d) program quality (Program Requirement V.C.1.) (June 2009 letter) There is no documented evidence of annual meetings to review the resources available to the program and its effectiveness in meeting its goals and objectives. In addition, it is not clear that these meetings include both faculty and fellow representatives. (May 2009 letter) It is not evident that faculty provide formal written evaluations of the program on an annual basis. (May 2009 letter)
Annual Program Assessment Can be held at: Half/Full Day Retreat Faculty Meeting Meeting of Key Faculty from all Major Sites Must include residents/fellows Goals Analyze the results of data collected Discuss implications Set goals for improvement Develop action & implementation plan Record all the above
For Resident Performance 2008 Radiology Evaluation Outcome Documentation Summarized discussion on all points Identified areas of strength and weakness Targeted four issues to improve and act on Case presentation skills Ultrasound scanning skills Practice-based learning and improvement issues Systems-based knowledge topics Developed corresponding itemized Action Plan Ad hoc Committee to create potential solutions to improve case presentation skills for junior residents Creation of ultrasound scanning labs PQI projects to begin in fall with presentation of results in spring Ad hoc committee to investigate options for increasing resident involvement from a systems perspective
For Faculty Development Aggregate resident evaluations of Faculty teaching grouped by subspecialty Specific written comments by residents noted on evaluations Updated Faculty CVs Faculty CME Aimed at Teaching Other: Faculty Portfolios (Urology) Conference attendance (Orthopedic Surgery) Faculty survey of development needs (Orthopedic Surgery) Annual Workload Statements on GME participation Award or Promotion for Excellence in Education
2008 Anesthesiology Discussion & Plan Development Process 9 Working groups of 3-5 faculty & residents each focusing on priority issues and on assessing teaching six competencies Asked & answered: Provide an appraisal of the current situation explored by your group What are your specific plans for improvement? How will you track the progress of your proposed improvement plan? Group activities documented & presented Final formal approval process
When in Doubt, Ask Questions about Process? Consult GME Office on: issues requiring confidential treatment creating aggregate reports in E*Value appropriateness of using specific types of data completeness of documentation concreteness of action plan Use Annual Program Evaluation Report Template & model on examples of effective Annual Program Evaluation & Improvement Plans
Timelines Improvement Plan should be finalized by April for implementation by July, 2010 For 2009: Document Evaluation Plan Finalize Action Plan Submit Checklist and Report using Template to the GMEOC December 31 Deadline Purpose Common Program Requirement Institutional Review Requirement Continuous Quality Improvement Life-long Learning
Deliverables Formal written plan for program evaluation based on systematic data collection and analysis Formal written Improvement (Action) Plan that has buy-in from faculty and trainees Fully documented Evaluation and Action plan Checklist Report Template Samples of Effective Evaluation & Improvement Plans PDSA Cycle for continuous quality improvement GMEOC/DIO monitoring of annual process Progress on goals discussed at Annual Directors Meeting