Resident as Teacher: A Mutually Beneficial Arrangement APPD/COMSEP Pre-Course 10 April 2013 H. Barrett Fromme, MD, MHPE Kate Gibbs, MD Steve Paik, MD, EdM Shari Whicker, EdD, MEd
Disclosures We have no conflicts of interest to report
Objectives By the end of the pre-course, participants will be able to: Appreciate the importance of and need for Resident-As-Teacher (RAT) Curricula for graduate medical trainees Recognize the need for collaboration between clerkship and program directors in establishing and maintaining RAT curricula Discuss the relevant current literature on RAT programs in pediatrics, including program formats, resources and evaluation Utilize Kern s six-steps for curriculum development to plan/improve a RAT program Identify local and national partners and resources for collaboration in development and maintenance of RAT curricula
A Few Introductions
Time Course Outline Topic 2-2:10 Introduction to Course & Presenters 2:10-2:20 Introduction to Kern 2:20-2:55 Step 1: Problem Identification Step 2: Targeted Needs Assessment 2:55-3:35 Step 3: Goals and Objectives o 10 minute introduction o 20 minute small group o 10 minute report back 3:35-3:50 SESSION BREAK 3:50-4:35 Step 4: Educational Strategies o 10-15min introduction to types o 20-25 min small group o 10 min group share 4:35-5:00 Step 5: Implementation 5:00-5:30 Step 6: Evaluation o Intro to evaluation types/kirkpatrick o Small group breakout 5:30-5:45 Debrief and consult time 5:45-6pm Wrap-up
Kern s Six Steps
1. Problem Identification/General Needs Assessment 2. Targeted Needs Assessment 3. Goals & Objectives 4. Educational Strategies 5. Implementation Six Steps 6. Evaluation & Feedback Kern, Thomas & Hughs, 2009
Kern, Thomas & Hughs, 2009 Six Steps
Step 1: Problem Identification/General Needs Assessment Health Care/Other Problem Current Approach Ideal Approach
Content Step 2: Targeted Needs Assessment Targeted Learner Content Learning Environment Content Methods General Considerations Specific Methods Surveys
Step 3: Goals & Objectives Goals vs. Objectives Types of Objectives Learner Objectives Process Objectives Outcome Objectives
Step 4: Educational Strategies Choice of Educational Methods General Guidelines Methods for Achieving Cognitive vs. Affective vs. Psychomotor Objectives
Step 5: Implementation Logistics, Logistics, Logistics! Resources Personnel Time Facilities Funding/Costs Support Internal External
Step 5: Implementation Logistics, Logistics, Logistics! Administration Structure Communication Operations Scholarship Barriers Introducing the Curriculum Piloting Phasing Full implementation
Step 6: Evaluation Kirkpatrick s Levels of Evaluation Level 1: Reaction Level 2: Knowledge Level 3: Behavior Level 4: Impact
Step 1: Problem Identification and General Needs Assessment
ED-24 2012-2013 At an institution offering a medical education program, residents who supervise or teach medical students and graduate students and postdoctoral fellows in the biomedical sciences who serve as teachers or teaching assistants must be familiar with the educational objectives of the course or clerkship (or, in Canada, clerkship rotation) and be prepared for their roles in teaching and assessment.
Annotation The minimum expectations for achieving compliance with this standard are that: (a) residents and other instructors who do not hold faculty ranks (e.g., graduate students and postdoctoral fellows) receive a copy of the course or clerkship/clerkship rotation objectives and clear guidance from the course or clerkship/clerkship rotation director about their roles in teaching and assessing medical students and (b) the institution and/or its relevant departments provide resources (e.g., workshops, resource materials) to enhance the teaching and assessment skills of residents and other non-faculty instructors. There should be central monitoring of the level of residents' and other instructors' participation in activities to enhance their teaching and assessment skills.
The Literature Responsible for 19-40% of teaching (Rotenburg, 2000; Kerfoot, 2004; Bensinger, 2005) Expected and appreciated by attendings Learn better when they teach it (Weiss, 1998) Impact student career choice (Whittaker, 2005)
Current State of RAT 55+% of residencies (Morrison, 2001; Fromme, 2011) Large variation in quality/quantity Feedback and One-Minute Preceptor
Curricular Formats Lectures Small Group Discussions Role Play/Simulation Debriefing Videotape review Workshops (incl. all above) (Hill, 2009)
Curricular Formats (Fromme, 2011)
Curricular Topics One Minute Preceptor Feedback Bedside Teaching Teaching on the Admission Procedural Skills The Difficult Learner Team Leadership Learning Climate Understanding/Retention Lecture Skills Evaluating Learners Small Group Teaching (Fromme, 2011)
Evaluation Methods (Fromme, 2011)
Outcomes Improved learner evaluations Improved confidence in skills Improved self evaluations Variable improvement in behavior (Post, 2009; Hill, 2009)
Kirkpatrick Outcome Levels Hill, 2009
Current Limitations Single institutions Single specialties Heterogenous curricula Lower level outcomes
A Thought... Those who can do, those who can t teach... and those who can t teach, teach teaching.
Step 3: Goals & Objectives
Step 3: Goals & Objectives Goals vs. Objectives Types of Objectives Learner Objectives Process Objectives Outcome Objectives
Goals vs. Objectives Goals Communicate overall purposes of curricula Criteria against which curricular components can be judged Use general terms (learn, understand, etc.) Objectives Frame refinement of curricular content Guide selection of appropriate educational and evaluation methods Concrete statements that are tangible and measurable Answer: Who will do how much/how well of what by when Demonstrate if goals have been achieved (e.g. learners have learned, know or understand).
Learner Objectives Types of Objectives Cognitive (knowledge) Affective (attitudinal) Psychomotor (skill, competence, behavior, performance)
Bloom s Taxonomy of Cognitive Domain Objectives
Process Objectives Types of Objectives Implementation of the curriculum Individual Program
Outcome Objectives Health Health care Patient outcomes Types of Objectives
Small Group YOUR Turn! Write 1 broad educational goals Write > 1 educational objective of each type Goal: Learner (cognitive, affective, or psychomotor) Process Health, healthcare, or patient outcome Individual Aggregate or Program
Step 4: Educational Strategies
Educational Strategies Learning Concepts Learning climate Relevance Active Experiential Problem centered Feedback is given Influenced by a program s needs assessment and institutional resources Institutional resources do not need to limit implementation
Educational Strategies Align them with your goals and objectives and plans for evaluation Utilize strategies so that goals/objectives are accomplished Plan evaluation to measure if objectives were achieved Resident comfort/satisfaction while teaching Learner assessment of resident teaching skills Skill acquisition and utilization Acquisition evaluated at time of RAT program; utilization later (weeks/months not hours/days)
Educational strategies Content Specific curricular materials Giving feedback Learning climate Questioning as a teaching tool Diagnosing your learner Methods How is the content presented Didactic presentations, self-directed learning Using skills Direct observation with feedback
Educational Strategies Where to implement your RAT Noon conference Intermittent or longitudinal Workshop Resident retreat Teaching rotation Can be a component of the individualized curriculum Participation in already existing pre-clerkship or clerkship experiences
Educational strategies developing Self assessment and utilizing skills Video examples of teaching skills (gone well or awry) Role play Direct observation Example: during clerkship didactics or on rounds Participation in a medical school physical diagnosis course Participation in clerkship activities Completing an observed history and physical with a student Giving feedback after an oral presentation Review and give feedback on student progress notes
Teaching skill feedback Use of video In this video a resident is giving feedback to a medical student Residents are asked to answer the following questions Was this effective feedback? If so, why? If not, why not?
Step 5: Implementation
Implementation Existing institutional support Do other departments have a RAT program? Do you need buy-in at a department or institutional level? Identify stakeholders Identify resources Who? When? Where? How much?
Implementation Determine a reasonable timeline Pilot Could start with workshop or conference prior to a full block rotation Phase in Implement educational strategies (linked to your goals/objectives) and evaluation pieces together Reassess and evaluate successes and failures Full implementation Continue to reassess and evaluate successes/failures Consider mechanisms for scholarly dissemination
Barriers to implementation Time, time and time Space Teaching support Do the faculty need development of their teaching skills first? $$$$ Productivity expectations Administrative support
Step 6: Evaluation
The Four Levels Level I: Evaluate Reaction Level II: Evaluate Learning Change in Attitude Change in Knowledge or Skills Level III: Evaluate Behavior Level IV: Evaluate Results Change in system/organization Change in participants
Types of Assessments Used at Each Level Level 4 - Results Was it worth it? Level 3 - Behavior KSA being used? Level 2 Know/Skills/Attitudes Did they learn anything Level 1 - Reaction Was the environment suitable for learning? Type Summative Formative Summative Diagnostic Formative Summative Reaction Formative Form Departmental Evaluations Rotation ratings Program Evaluation Milestones (outcomes) OSTE Observations Evaluations OSTE Testing Self Assessment Reflective writing Survey Real-time Polling Quizzing
Reaction How favorably participants react to the RAT curriculum Collects reactions to instructors, material and learning environment Communicates to trainees that their feedback is valued Can provide quantitative information Can adjust to needs of the residents
Learning Knowledge Skills Attitudes
Learning Methods used to measure learning Interviews Post rotation written evaluation and plan Surveys Pre and Post Observations Practice Videotaping Combinations OSTEs
Behavior Transfer of knowledge, skills, and/or attitude to the real world Measure achievement of performance objectives
Behavior Observe performer, first-hand Survey and evaluate key people who observe performer (med students, interns, etc) Use checklists, questionnaires or OSTEs Have residents apply skills in different types of conference and evaluate
Results Assesses bottom line, final results Definition of results dependent upon the goal of RAT curriculum
Results Depends upon objectives ACGME LCME Residency Program Highlight Meet needs of the program/residents Culture Change
Recap Level I: Level II: Level III: Level IV: Evaluate Reaction Evaluate Learning Evaluate Behavior Evaluate Results
What Will You Do? Initiate Collaborate Disseminate
Take Home Points Follow Kern, and you are off to a good start Aim high for outcomes think of evaluation at the start Don t reinvent the wheel Find collaborators
Thanks! Barrett Fromme, The University of Chicago, hfromme@peds.bsd.uchicago.edu Kate Gibbs, Mt. Sinai School of Medicine, kathleen.gibbs@mssm.edu Steve Paik, Columbia University, sp793@columbia.edu Shari Whicker, Duke University, shari.whicker@duke.edu