THE ABCs OF MEDICAL EDUCATION - PDF Free Download

THE ABCs OF MEDICAL EDUCATION A guide to curriculum development & teaching in patient care environments for faculty members, residents, fellows, & other health professionals David Lee Gordon, M.D., FAAN, FANA, FAHA Professor & Chair Department of Neurology Kathryn G. and Doss Owen Lynn, M.D., Chair in Neurology The University of Oklahoma Health Sciences Center Excellence Collegiality Innovation

THE ABCs OF MEDICAL EDUCATION DISCLOSURES Under Accreditation Council for Continuing Medical Education guidelines disclosure must be made regarding relevant financial relationships with commercial interests within the last 12 months. David Lee Gordon, MD I have no relevant financial relationships or affiliations with commercial interests to disclose.

THE ABCs OF MEDICAL EDUCATION LEARNING OBJECTIVES At the end of this presentation, the learner will be able to: Define curriculum alignment and outline its effect on course administration and implementation Describe the 10 essential features (ABCs) of medical education Describe the 5 components (S.M.A.R.T.) of effective clinical teaching

EFFECTIVE MEDICAL EDUCATION: SIMPLE CONCEPT, COMPLEX EXECUTION The key to effective medical education is competency based learning, based on the principle of curriculum alignment in which the learning objectives, assessments, and learning experiences are all aligned The principle is simple and logical Implementation is complex A word on nomenclature: competency based learning = mastery learning; curriculum alignment = instructional alignment = blueprinting

BENJAMIN S. BLOOM (1913 1999) A GIANT OF EDUCATIONAL REFORM American educational psychologist Facilitated initiation of Head Start program Played major role in founding the International Association for the Evaluation of Educational Achievement Beginning in the 1950s, responsible for furthering the concepts of: Educational objective domains (Bloom s taxonomy) Instructional alignment Mastery learning

COMPETENCY BASED/MASTERY LEARNING: KEY EDUCATIONAL TERMS Learning objectives = pre specified, learner specific knowledge, skills, attitudes, & behaviors (KSAB)* that learners are expected to achieve Competencies (as used by ACGME) = educational program objectives (EPOs) = standardized categories of learning objectives = the categories of KSAB necessary to be a competent physician (ACGME & OUCOM have identified 6) Assessment = demonstration & evaluation of KSAB; proof that a learner has achieved an objective Curriculum alignment = instructional alignment = blueprinting = linking of objectives, assessments, and learning experiences to ensure learners achieve what is expected of them *Modified from Bloom s taxonomy of educational objectives into cognitive, psychomotor, & affective domains (Bloom et al. 1956)

COMPETENCY BASED/MASTERY LEARNING: KEY EDUCATIONAL TERMS Core content = essential KSAB that learners must acquire prior to graduation & that are necessary for success after graduation Entrustable professional activity (EPA) = a learner specific core clinical activity a learner should be able to perform independently; may be defined by a series of integrated learning objectives from multiple competencies Milestones = standardized learning objectives linked to a learner s stage in development; objective specific steps to competency Hidden curriculum = informal learning that differs from what is taught in the formal (declared) curriculum; may have a negative or positive influence on learners (from Dent & Harden 2013) Learning environment = the sum of internal (formal curriculum) & external (informal/hidden curriculum) circumstances & influences surrounding & affecting a person s learning; in particular the modeling of professionalism & its influence on the learning & professionalism of trainees

COMPETENCY BASED/MASTERY LEARNING: KEY PSYCHOLOGICAL TERMS Flow = flow channel = state of optimal experience = a state of enjoyment and maximal concentration that occurs as a result of participating in activities that one perceives as worth pursuing for their own sake, provide a sense of accomplishment, and lead to personal growth; activities that lead to flow : Provide a sense that one s skills are adequate to cope with the challenges at hand (i.e., learner s skills match challenge difficulty) Are part of a goal directed, rule bound action system that provides clear clues as to how well one is performing (i.e., include goals, structure, & feedback) M Csikszentmihalyi 1990 Deliberate practice = Focused, repetitive practice in which training (often designed & arranged by instructors) is focused on improving particular tasks necessary for improving performance & advancement to the level of expert; requires structure, feedback, & opportunity for refinements KA Ericsson et al. 1993, KA Ericsson 2008

COMPETENCY BASED/MASTERY LEARNING DEFINITION & TWO ESSENTIAL ELEMENTS Competency based learning refers to systems of instruction & assessment based on learners demonstrating that they have acquired the KSAB they are expected to acquire as they progress through their education. Ensures learners achieve what they are expected to achieve Provides detailed information about learner progress in order to identify academic strengths & weaknesses & specific concepts & skills learners have not yet mastered Equivalent terms are mastery, proficiency, outcome, performance, and standards based learning Competency Based Learning http://edglossary.org 2014 Per Bloom, the two essential elements of mastery learning are: Feedback ( feedback, corrective, enrichment process ) Instructional alignment TR Guskey 2007

COMPETENCY BASED CURRICULUM VS. TRADITIONAL CURRICULUM Unlike a traditional curriculum, in a competency based curriculum: Learner need at end of training, not teacher convenience, determines structure & implementation Focus is on performance & outcomes, not experience alone Guidelines & objectives drive learning, not local expertise Learning experiences are standardized & structured, utilize deliberate practice, & take into consideration learner flow Feedback is frequent & assessments are used as learning tools Learners are evaluated against a fixed standard or criterion rather than against each other Cooperation trumps competition among learners

COMPETENCY BASED LEARNING: RATIONALE IN MEDICINE Competent clinicians demonstrate certain core knowledge, skills, attitudes, & behaviors (KSAB) One can organize these core KSAB within standardized categories or (core) competencies Graduating students & trainees must demonstrate core KSAB appropriate for their level of training Clinicians work in teams & their success depends to a large degree on their ability to cooperate with others Competency based learning beginning with the end in mind ensures assessment of all core competencies (all core KSAB necessary to be a competent physician within the 6 ACGME competencies or medical school EPOs )

GME (RESIDENT & FELLOW) & UME (MEDICAL STUDENT) PROGRAM GOALS ACGME Outcome Project & OUCOM Educational Program Objectives Begin with the end in mind Stephen R. Covey Provide resources & opportunities to enable each trainee & student to achieve excellence in the 6 ACGME competencies or OUCOM educational program objectives (EPOs) ACGME Competency Patient care Medical knowledge Practice based learning & improvement* Interpersonal & communication skills Professionalism Systems based practice** OUCOM EPO Same Same Same Communication Same Same *PBLI = evidence based medicine, quality improvement, & research **SBP = cost effectiveness, team skills, patient safety, handovers

CURRICULUM ALIGNMENT: CONSEQUENCES Removes guesswork from learner studying Enhances learner focus on course material Removes subjectivity from grading Guarantees success The learner achieves what the course director determined learners should achieve ( you get what you reward ) Causes a domino effect with major implications for course administration Is the responsibility of the course director

CURRICULUM ALIGNMENT: THE FALLING DOMINOES Curriculum alignment is only effective if you: Notify learners & instructors of objectives in advance Teach to the test (ensure learning aligns w/objectives & assessments) Discourage teaching that contradicts objectives & assessments Exclude noncompliant & incompetent instructors Enforce standardized learning schedule Base assessments on standardized learning Create assessments internally Use criterion referenced grading (no bell shaped curve)

THE COURSE DIRECTOR & 7 HABITS OF HIGHLY EFFECTIVE PEOPLE* Private Victory: Path to Independence Public Victory: Path to Interdependence 1. Be proactive Plan & own the curriculum (leave nothing to chance) 2. Begin with the end in mind Start with national standards & objectives, align curriculum 3. Put first things first Emphasize core content, keep commitments 4. Think win/win Use criterion referenced grading (you get what you reward) 5. Seek first to understand then to be understood Value learner & instructor feedback (but follow selectively) 6. Synergize Ensure clinical experiences affirm standardized learning 7. Sharpen the saw Practice quality improvement (using outcomes & feedback) *Stephen R. Covey. The 7 Habits of Highly Effective People. Simon & Schuster. New York, NY. 1989, 2004.

THE ABCs OF MEDICAL EDUCATION: 10 ESSENTIAL FEATURES Alignment of curriculum Blended learning environment Core content Consistency Deliberate practice Experiences affirm curriculum Formative & summative assessments Grading criterion referenced Homogeneity Institutional support David Lee Gordon

A ALIGNMENT OF CURRICULUM Aligning objectives, assessments, & learning gives learners structure & direction Objectives determine assessments. Assessments drive learning. Objectives Assessments Learning Step 1 Step 2 Step 3 Objectives Assessments Learning 3 STEPS OF CURRICULUM DEVELOPMENT: DEFINE, DETERMINE, DELINEATE Define the target Determine the outcome Delineate the path Meaningful, practical, pertinent, assessable, comprehensive (KSAB in all competencies/epos ). Learner & objective specific evaluations. Formats vary based on corresponding objective. Standardized experiences. Consistent with objectives & assessments.

A ALIGNMENT OF CURRICULUM Continuously compare objectives & assessments and rewrite as necessary to ensure alignment Write objectives Ensure all assessments are covered in the objectives Ensure all objectives are assessable Create assessments ALWAYS write objectives with assessments in mind. ALWAYS write assessments with objectives in mind. ALWAYS link teaching to assessments ( teach to the test ).

A ALIGNMENT OF CURRICULUM Make learning objectives assessable Write learning objectives with assessments in mind objectives have no purpose without assessment Objectives must contain active, assessable verbs Assessments must be feasible Write learning objectives from perspective of learner not from perspective of instructor or program; for guidance, preface learning objectives with the following phrase: At the end of the experience/curriculum, the learner will be able to Write learning objectives for each learning experience and ensure experience affords opportunity to achieve objectives Note: Milestones are staged objectives

A ALIGNMENT OF CURRICULUM Use appropriate verbs when writing objectives Examples of INCORRECT learning objectives: Teach the management of stroke (instructor perspective) Understand/know the management of stroke (not assessable) Manage stroke (not feasible if learners are students) Counsel caregivers about brain death (not feasible if based on ward experience alone) Examples of CORRECT learning objectives: Describe the management of stroke List the components of stroke management Match objective verb to objective type: K Compare, describe, distinguish, explain, identify, list, name S Assess, demonstrate, detect, employ, evaluate, interpret AB Demonstrate, employ, implement, interpret, perform, use

B BLENDED LEARNING ENVIRONMENT Base learning experience & assessment strategy on objective type & learner level of training Objective type K lectures, small groups, readings, written tests, oral exams S patients, SP/OSCE, simulation, multimedia/web AB patients, SP/OSCE, readings, essays, discussions Level of training per Miller s Pyramid of Competence Does PATIENTS Performance assessment in vivo Observation, undercover SP/video, logs Shows How Knows How SIMULATION TEST Performance assessment in vitro OSCE, SP, simulator, case based test Clinical context based tests MCQ, essay, oral Knows TEST Factual tests MCQ, essay, oral GE Miller 1990, V. Wass et al. 2001 SP = standardized patient; OSCE = objective structured clinical exam

B BLENDED LEARNING ENVIRONMENT Gradually push learner up the flow channel CHALLENGE Putting learner into optimal learning environment (flow channel) requires appropriate blend of challenges matched to learner skill level and requires goals, structure, & feedback ANXIETY NOV INT SKILL LEVEL FLOW CHANNEL ADV BOREDOM The 2 essential components of mastery learning are also essential in flow theory: Goals & structure (curriculum alignment ) Feedback Gradually challenge difficulty: To enable learner s perceived skill level to keep pace and, thereby, For learner to remain in flow as s/he progresses from novice to intermediate to advanced (up the flow channel) NOV = Novice learner This process guarantees learners grow to INT = Intermediate learner their maximal skill level ADV = Advanced learner After M Csikszentmihalyi 1990

C 1 CORE CONTENT Point learners in the correct direction Learning objectives & curriculum alignment give learners direction assessing only core content ensures learners go in correct direction Varies with learner level & flow ( blended learning environment ) Base objectives & assessments on core content only Publicize this fact to learners & teachers Discourage teaching contrary to core content Encourage teaching supplemental to core content, but do not assess learners on supplemental content Assessing only core content facilitates learner flow, leads to improved: Retention of knowledge & skills Learner confidence Enthusiasm for the topic Comprehension of supplemental learning Providing skeleton enables learner to add flesh. Course director book editor (not just lecture organizer).

C 2 CONSISTENCY Align extramurally & intramurally Extramural consistency Base course (objectives) on consensus statement Obtain external review (esp. of objectives) Intramural consistency Three intramural dimensions Intradepartment multiple faculty/clinical sites (hidden curriculum) Interdepartment UME Year to year UME Requires extensive communication and cooperation among course directors and faculty and is most effective with institutional direction and oversight

D DELIBERATE PRACTICE Realize perfect practice makes perfect (Vince Lombardi 1913 1970) Experience itself teaches nothing (W. Edwards Deming, 1900 1993) The value of experience is not in seeing much but in seeing wisely. (Sir William Osler, 1849 1919) Deliberate practice = focused, repetitive practice in which training (often designed & arranged by instructors) is focused on improving particular tasks necessary for improving performance & advancement to the level of expert Essential components of deliberate practice are consistent with flow and competency based learning and include: Motivated & attentive learner (learner in flow ) Well defined task & goals (standardized curriculum, learning objectives) Appropriate level of difficulty (flow channel) Informative feedback from educational sources (supervised structure) Opportunities for repetition & refinements (simulation with feedback) Modified from Ericsson et al. 1993; Ericsson. 2008; McGaghie et al. 2011

D DELIBERATE PRACTICE Use simulation for practical implementation Clinical environment alone is not conducive to consistent & comprehensive implementation of deliberate practice Standardized curriculum that includes simulation is the only practical solution for implementation Standardized patients (SPs) Objective structured clinical exams (OSCEs) Low fidelity & high fidelity simulators Oral & written patient presentations Case based learning Role playing Gaming Multimedia videos, avatars, etc. Online delivery Simulation for deliberate practice is especially necessary when learner is at shows how level of competence in Miller s Pyramid for a specific objective, e.g., breaking bad news WC McGaghie et al. 2011

E EXPERIENCES AFFIRM CURRICULUM Create a positive learning environment & be S.M.A.R.T. S.M.A.R.T. Clinical Teaching The 5 components of effective teaching in patient care environments Set expectations provide structure & define responsibilities day 1 Model positive behavior create a positive hidden curriculum Affirm declared curriculum supplement, but never contradict Repeat feedback motivate & provide opportunity for improvement Target your audiences consider learner training level & flow S.M.A.R.T. specifics follow the ABCs of medical education at the end of this presentation David Lee Gordon

F FORMATIVE & SUMMATIVE ASSESSMENTS You can expect what you inspect (W. Edwards Deming 1900 1993) Assessment drives learning. Assessments are learning tools & teaching to the test is optimal. Assessment content BS Bloom 1968, V Wass et al. 2001, TR Guskey 2007 Base only on learning objectives to ensure learner growth & advancement up the flow channel Assessment features unrelated to content Type (formative/feedback & summative/graded) Validity (appropriateness) Reliability (consistency) Feasibility (practicality)

F FORMATIVE & SUMMATIVE ASSESSMENTS Give frequent, scheduled feedback & only test objectives Formative assessment Feedback without or with grade Intermediate gauge of learner s progress Summative assessment Graded testing without or with feedback Outcome measure of learner s KSAB acquisition Keys to effective implementation View assessments as learning tools Formative schedule frequent feedback opportunities Summative Make pertinent only assess core content/objectives Use criterion referenced (not norm referenced) grading Monitor results for course QI (curriculum evaluation) Note that a specific assessment may be both formative & summative After BS Bloom 1968; V Wass et al. 2001; TR Guskey 2007

F FORMATIVE & SUMMATIVE ASSESSMENTS Assess entrustable professional activities (EPAs) Use scale for formative & summative assessment of learner s ability to perform a core clinical activity (EPA) independently Entrustment (supervision) scale: 1. Not ready for entrustment 2. Ready for direct supervision 3. Ready for indirect supervision 4. Ready for unsupervised practice 5. Ready to supervise others After O ten Cate, lecture at ACGME conference 2015 & HC Chen et al. 2015

G GRADING CRITERION REFERENCED Use criterion referenced grading to assess competence NORM REFERENCED GRADING Comparison to other learners CRITERION REFERENCED GRADING Comparison to fixed standards (competence) Figures from TR Guskey 2007 Traditional curricula have norm referenced distribution of achievement Mastery learning curricula have criterionreferenced distribution of achievement In criterion referenced grading, the director predetermines standards for grades & the curve is shifted to the right, but there are still high, average, & low performers ( scalar or stepped grading).

G GRADING CRITERION REFERENCED Do not use norm referenced grading: it devalues teaching & inhibits cooperation & teamwork among learners Norm Referenced Grading Learner grade based on performance vs. performance of other examinees Purpose is to discriminate Normative (relative) Bell shaped curve Guaranteed some will fail Items assess broad content not known to learner in advance Inconsistent w/ curriculum alignment & competency based education (exam items not linked to objectives) Learners study independently, ignore what is taught by instructors Feedback not linked to performance Promotes competition and cooperation is seen as cheating Criterion Referenced Grading Learner grade based on performance vs. course director s standard/criterion Purpose is to assess competence Mastery (absolute) Bell shaped curve skewed to right Most perform well, all may pass Items assess learning objectives known to learner in advance Consistent w/ curriculum alignment & competency based education (exam items linked to objectives) Learners study what is taught by instructors Feedback linked to performance Promotes cooperation

G GRADING CRITERION REFERENCED You get what you reward. Is the college weeding out process (based on norm referenced grading) valid & effective? Pre Med / College UME (Medical School) GME (Residents/Fellows) Practitioner / Boards WEED OUT LEARNERS ESTABLISH LEARNER COMPETENCE WORK ALONE WORK IN TEAMS NORM REFERENCED ASSESSMENTS NORM & CRITERION REFERENCED ASSESSMENTS CRITERION REFERENCED ASSESSMENTS Reinforces Win/Lose Promotes competition Reinforces Win/Win Promotes cooperation Norm referenced grading in college & medical school promotes competition, but physicians practice in teams (systems) that necessitate cooperation (systems based practice).

H HOMOGENEITY Enforce standardized (equivalent) core curriculum for all learners Curriculum based on mastering core content & deliberate practice requires that ALL learners receive an equivalent homogeneous or standardized learning experience Standardized curriculum Is highly structured Requires mandatory participation & punctual attendance Takes precedence over other experiences (e.g., clinic) Requires full attention of learners (pagers & phones off) Accounts for majority of grade, not necessarily learner time Requires strong, well organized course director Does not preclude and actually aids supplemental learning by advanced learners ( enrichment process of Bloom)

I INSTITUTIONAL SUPPORT Provide protected time & authority for course director Recruit, assign, & incentivize faculty based on skills & interests, ensuring plan is financially viable (mission based hiring). Identify & support appropriate curriculum champion. Delegate (don t relegate) director who has: Both teaching & administrative/leadership skills Mettle & motivation Protected time & authority to exclude noncompliant/incompetent faculty Establish educational committees Protect didactic schedule in clinical rotations attendance w/o pagers Provide resources Director 10 50% FTE, coordinator 10 50% FTE depending on program size Simulation costs, e.g., SPs, OSCEs, skills center Instructors (faculty / residents / nurses / graduate students / et al.) Books, electronic tablets, travel, etc.

THE ABCs OF MEDICAL EDUCATION Absence of any of these essential features limits curriculum success Alignment of curriculum structure & direction Blended learning environment competence level & flow Core content correct direction, skeleton Consistency extramural & intramural Deliberate practice goals, structure, feedback, & simulation Experiences affirm curriculum learning environment, S.M.A.R.T. Formative & summative assessments feedback & tests drive learning Grading criterion referenced competence & cooperation Homogeneity standardized / equivalent core curriculum for all Institutional support director protected time & authority David Lee Gordon

S.M.A.R.T. CLINICAL TEACHING A guide to teaching in patient care environments for all clinical preceptors, including attendings, residents, fellows, & other health professionals Excellence Collegiality Innovation

S.M.A.R.T. CLINICAL TEACHING Create a positive learning environment & be S.M.A.R.T. Set expectations Model positive behavior Affirm declared curriculum Repeat feedback Target your audiences David Lee Gordon

S ET EXPECTATIONS Establish structure to facilitate learning On day 1, define: Roles of team members Goals & responsibilities of the learner(s) Your roles re: patient care, teaching, & team leadership Expectations of behavior and professionalism

M ODEL POSITIVE BEHAVIOR You are always on stage, a model of how to be a physician be aware of the hidden curriculum and your influence over it Hidden curriculum = informal learning that differs from what is taught in the formal or declared curriculum (may be positive or negative) Script your interactions use a standardized approach in ALL interactions with patients, caregivers, learners, health professionals Same approach every time with every patient, learner, & colleague AIDET is an example of such a standardized approach (see pocket card) Admit your own ignorance & mistakes & encourage others to do so Consciously fight unconscious biases (yours & others ) Types: appearance, behavior, age, gender, race, religion, ethnicity, etc. Activators: stress, time constraints, multitasking, need for closure, conformity Promote cooperation over competition (clinically & educationally) Avoid derogatory comments about others (spoken & written)

A FFIRM DECLARED CURRICULUM Clinical teaching is a component of the overall curriculum Be familiar with the content, learning objectives, and schedule of the declared (formal) curriculum Teach content consistent with the declared curriculum Never contradict the declared curriculum Clarify when teaching material supplemental to the declared curriculum

R EPEAT FEEDBACK Use feedback to: Motivate learner ( assessment drives learning ) Provide opportunity for refinements & improvement Give repetitive, task specific feedback (deliberate practice*) Preparation for rounds knowledge of patient & medical record Participation in rounds patient presentations, asking questions Progress notes meaningful and clear information Skills history, examination, image reading, handovers, etc. Give prompt feedback concerning potentially unprofessional behavior ( what you permit, you promote ) Complete formal evaluations: With learner present to provide timely feedback Including narrative of performance, using descriptive adjectives unique to learner s personality & performance avoid nondescriptive adjectives *Deliberate practice = focused, repetitive practice designed by instructors to improve performance of specific tasks necessary to advance to level of expert (KA Ericsson et al. 1993, KA Ericsson 2008)

T ARGET YOUR AUDIENCES CHALLENGE CHALLENGE FLOW ANXIETY ADV INT NOV BOREDOM SKILL LEVEL After M Csikszentmihalyi 1990 RETREAT STASIS GROWTH CONFIRMATION Keep in mind learner training level & flow ; maintain support challenge balance (see graphs) Flow = state of optimal experience in which learner perceives that his or her skills match challenge difficulty Use learner targeted teaching strategies Priming prepare learner for a task Modeling explain decisions & actions (think out loud) Personalization make situation pertinent for learner Anecdote tell stories consistent with evidence/best practice for improved understanding & retention Dogma maintain simplicity & consistency SUPPORT After DJ. Bower et al. 1998 NOV = novice, INT = intermediate, ADV = advanced (learners)

S.M.A.R.T. CLINICAL TEACHING CONCLUSION & TAKE HOME POINTS Create a positive learning environment & be S.M.A.R.T. Set expectations provide structure & define responsibilities day 1 Model positive behavior create a positive hidden curriculum Affirm declared curriculum supplement, but never contradict Repeat feedback motivate & provide opportunity for improvement Target your audiences consider learner training level & flow David Lee Gordon Use the S.M.A.R.T. pocket card to remind you of the 5 components of effective clinical teaching along with key educational terms, a description of AIDET, & a list of references

OU NEUROLOGY UME & GME EDUCATIONAL ACTIVITIES UME Medicine M1 thru M4 Physician associates Neuropsychology Nurse practitioner Nursing GME Neurology Medicine Neurosurgery Psychiatry OU & Griffin Pediatrics Neurology CNP fellows Neurology VN fellows Neurology ESN fellows Child psychiatry fellows Developmental pediatric fellows Genetics fellows Pulmonology fellows Neuroanesthesia fellows

OU NEUROLOGY EDUCATION: MEDICAL STUDENTS YR COURSE TOPICS DIRECTORS M1 Clinical Medicine I Neurologic exam Billings/Gordon M2 Clinical Medicine II Neurologic exam (Neuro Idol) Billings/Gordon Neuroscience Headache SPBL Neuroanatomy Neurophysiology Neuropathology Pathophysiology Clinical neurology & psychiatry Billings/Gordon Blair/Farrow/ Tucker Clinical Transitions Brain imaging Blevins/Gordon Weakness lesion localization M3 Clerkship Clinical neurology Gordon/Chappel M4 Electives Advanced clinical neurology Pendergraft Capstone Transitions in Care TBL Blevins/Gordon

OUCOM M2 NEUROSCIENCE COURSE 2015 AAMC Graduation Questionnaire for 2012 Courses % Students Rating Educational Quality Excellent 100.0 OUCOM ALL LCME 90.0 80.0 70.0 60.0 50.0 40.0 30.0 20.0 28.8 20.8 22.3 21.3 25.0 23.4 53.1 53.2 34.0 27.5 64.6 61.1 30.0 21.3 77.5 41.2 45.8 47.1 37.5 36.7 37.6 30.4 53.6 46.8 38.0 40.0 59.1 53.2 10.0 0.0

OUCOM M2 NEUROSCIENCE COURSE 2016 AAMC Graduation Questionnaire for 2013 Courses % Students Rating Educational Quality Excellent 100.0 OUCOM ALL LCME 90.0 80.0 70.0 60.0 50.0 40.0 30.0 20.0 31.5 32.3 22.3 25.8 24.5 16.0 51.1 54.1 40.2 34.8 63.8 61.4 63.4 57.3 33.6 37.1 41.9 46.5 47.5 30.2 38.1 33.3 58.3 53.8 41.5 33.8 60.3 59.7 10.0 0.0

OUCOM M2 NEUROSCIENCE COURSE 2015 Postcourse Student Evaluations, 7 point scale Course Ratings Quality of course... 5.99 Quality of neurology lectures 6.39 Course objectives clear.. 6.48 Exams reflected course objectives. 6.01 I learned a lot during the course..6.80 Neurology Lecturer Ratings Mean 6.39 Range..5.74 6.85 Faculty Arrington Battiste Chrusciel Farrow Gordon Herren Karunapuzha Vaughn Webb

AAN NEUROLOGY CLERKSHIP CORE CURRICULUM Begin with end in mind, focus on learners, emphasize core competencies Minimum body of clinical neurology skills and knowledge required of all graduating medical students, regardless of their eventual career path General principles and systematic approach to patients with neurologic symptoms and signs, rather than a large body of disease specific facts Specific conditions are emphasized only if they are common, illustrate essential concepts, or require urgent management Gelb et al., 2002

OU NEUROLOGY CLERKSHIP: GOALS / GLOBAL OBJECTIVES 1. Perform accurate & appropriate neurologic exam 2. Identify & describe significance of key neurologic findings 3. Describe pathophysiology, clinical course, & management of common neurologic conditions & key neurologic emergencies 4. Distinguish normal & abnormal CT & MRI scans of brain 5. Communicate effectively about neurologic patients Based on AAN core curriculum. Specific skills & cognitive learning objectives are matched to OUCOM s 6 educational program objectives..

OU NEUROLOGY CLERKSHIP: STANDARDIZED DIDACTIC CURRICULUM WEEK 1 AIDET & History Taking Skills Neurologic Communication Lesion Localization Essential Neurologic Exam Essential Neurologic Findings Brain Imaging Case Based Learning (4) SOAP Progress Note Feedback WEEKS 2 4 Patient Centered Articles Case Summary Training Aphasia & Coma SPs Ward Based Learning Interdisciplinary Team Ethics & Professionalism Case Based Learning (16) All supporting materials are on student website, http://hippocrates.ouhsc.edu/ Standardized didactic curriculum accounts for 21% of student s on campus time & 90% of student s grade

OU NEUROLOGY CLERKSHIP: STANDARDIZED DIDACTIC CURRICULUM Orientation History Taking Communication Localization Neuro Exam CBL Submit SICEF CBL Quiz CBL CBL Review Interdisciplinary Game & Quiz CBL Ward Based Learning CBL Quiz CBL OU Clinic OU Clinic CBL Aphasia SP Coma SP Course Review Case Summary Assessments Submit SICEF CBL = case based learning; SP = standardized patient; SICEF = student instructor contract & evaluation form Findings Quiz Imaging Quiz CBL Quiz SOAP Feedback CBL Quizzes x 2 Case Summary Feedback Ethics & Professionalism Discussion Final Exam OSCE Course Evals

OU NEUROLOGY CLERKSHIP: CLINICAL EXPERIENCE With student interests in mind, clerkship director assigns students to two 2 week rotations among: OU Admit Neurology OU Consult Neurology VAMC Neurology OU Child Neurology

OUCOM OKC NEUROLOGY CLERKSHIP Student Ratings Far Exceed National Average 100.0 90.0 80.0 70.0 60.0 50.0 40.0 30.0 20.0 94.5 95.3 97.0 98.8 100.0 96.2 100.0 92.0 73.1 73.4 71.6 71.6 73.3 75.8 76.1 61.4 62.0 61.2 61.3 63.3 48.7 37.5 31.4 31.7 05 06 07 08 09 10 11 12 13 14 15 16 AAMC Graduation Questionnaire % students rating education quality good or excellent OU OKC Neurology Clerkship All LCME Neurology Clerkships 2016 GQ reflects 2014 2015 clerkship New curriculum (clerkship 07 08) Perfect top box scores in 2013 & 2015

OUCOM OKC NEUROLOGY CLERKSHIP Every Year, Majority of Students Rate It Excellent 100.0 90.0 80.0 70.0 60.0 50.0 40.0 30.0 20.0 10.0 0.0 35.6 35.9 24.4 12.8 76.7 75.5 82.1 85.2 87.8 77.2 81.9 69.0 * 27.5 20.3 19.0 17.8 19.8 14.9 13.6 12.2 19.0 18.1 23.0 3.9 05 06 07 08 09 10 11 12 13 14 15 16 * AAMC Graduation Questionnaire % students rating education quality good or excellent % Excellent % Good *In 2016, Other LCME Neurology Clerkships: 39.4% excellent 36.7% good New curriculum (clerkship 07 08) 100% of students rated it good or excellent in 2013 & 2015

OUCOM OKC NEUROLOGY CLERKSHIP New Questions on 2016 AAMC Graduation Questionnaire % of students who responded good or excellent (top box) or excellent to the statements: Residents/Faculty provided effective teaching during the clerkship 100 90 80 70 60 50 40 30 20 10 0 91.1 79.2 % Top box Residents 56.3 45.8 % Excellent Residents 94.7 79.7 % Top box Faculty 71.7 42.3 % Excellent Faculty *% Students who felt faculty provided effective teaching in other OUCOM clerkships: 28.6, 29.2, 46.0, 51.3, 54.0, 59.3 * 2016 GQ Reflects 2014 2015 clerkship OU OKC All LCME

OUCOM STUDENTS MATCHING IN NEUROLOGY OR CHILD NEUROLOGY 10 8 6 6 8 6 10 8 9 7 6 4 2 0 1 2 0 3 2 1 0 4 P R O JE C TE D New clerkship curriculum

EFFECT OF COMPETENCY BASED CURRICULUM OUCOM MEDICAL STUDENT TEACHING AWARDS TEACHING AWARD Before 2007 After 2007 OUCOM Stanton L. Young Master Teacher Award (f. 1984) OUCOM Edgar W. Young Lifetime Achievement Award (f. 1987) OUCOM M2 Aesculapian Award To Faculty (f. 1962) OUCOM M3 Aesculapian Award To Resident (f. 1975) OUCOM M3 Aesculapian Award To Faculty (f. 2016) OUCOM M4 Aesculapian Award To Faculty (f. 1962) OUCOM Academy of Teaching Scholars Dewayne Andrews Excellence in Teaching M3 & M4 Students (f. 2014) AAN Clerkship Directors Teaching Award (f. 2011) AAN A.B. Baker Teacher Recognition Certificate Peggy W. Wisdom 1996 Herman E. Jones 2009 David Lee Gordon 2011 Herman E. Jones 2011 Herman E. Jones 2006 Anthony J. Vaughn 2011 Anthony J. Vaughn 2012 Anthony J. Vaughn 2013 Aaron K. Farrow 2014 Anthony J. Vaughn 2016 Gunter Haase 1964 Herman E. Jones 2009 David Lee Gordon 2012 David Lee Gordon 2014 Anthony J. Vaughn 2015 David Lee Gordon 2014 Anthony J. Vaughn 2014 Aaron K. Farrow 2015 ANA Distinguished Neurology Teacher Award David Lee Gordon, MD

OU NEUROLOGY RESIDENCY: PROGRAM GOALS Facilitated by ACGME Outcome, Milestones, & CLER Projects Begin with the end in mind Stephen R. Covey Provide resources & opportunities to enable each resident to achieve excellence in the 6 ACGME competencies Patient care Medical knowledge Practice based learning & improvement* Interpersonal & communication skills Professionalism Systems based practice** Assist each resident in achieving the ability to function independently as a neurologist by the time of graduation *PBLI = evidence based medicine, quality improvement, & research **SBP = cost effectiveness, team skills, patient safety, handovers CLER = Clinical Learning Environment Review

GME CURRICULUM DEVELOPMENT: LEARNING OBJECTIVE CATEGORIES Ensure that all areas of doctoring are covered COMPETENCIES / MILESTONES Domains of Doctoring Patient care Medical knowledge Practice based learning & improvement Interpersonal & communication skills Professionalism Systems based practice LEARNING ENVIRONMENT (CLER) CATEGORIES Learning Environment Patient safety Quality improvement Transitions in care Supervision Duty hours & fatigue Professionalism LEARNING OBJECTIVE TYPES Modified after Bloom Knowledge Skills Attitude Behavior

OU NEUROLOGY RESIDENCY: P2 4 ROTATIONS TEMPLATE (WEEKS PER YEAR) PGY 2 NSICU 8 OU Admit 8 OU Consult 8 VAMC 6 8 Neuropathology 4 Psychiatry (Jul Dec) 4 Subspecialty clinic (Jan Jun) 4 Night Float (VA/Child) 4 6 Leave 4 PGY 3 NSICU 4 Child 12 EEG/EMU 8 EMG 4 Elective 8 Night float (OUMC) 8 Night float (VA/Child) 4 Leave 4 PGY 4 NSICU 4 OU Admit 8 OU Consult 8 VAMC 8 Neuro ophtho 4 Elective 16 Leave 4 Residents are on weekly schedule rather than monthly schedule so we can preschedule vacations and overlap resident & attending schedules

OU NEUROLOGY RESIDENCY: DIDACTIC CURRICULUM TIMELINE There are conferences at 8A most days & 12P most Mon, Wed, Fri JUL AUG SEP OCT NOV DEC JAN FEB MAR APR MAY JUN Orientation P2 Essentials Outpatient Skills Level specific Readings & Quizzes (self study) Leadership Skills Finances/Regulations Intro Series Handovers Annual Conference Series with Journal Clubs (JAN FEB focus on RITE) Grand Rounds Doctoring Skills Advanced Neurologic Skills Academic Scholarship Program Path Review Morning Reports (Child & Adult) weekly Clinical Chief Rounds with Chair average biweekly Morbidity, Mortality, Quality Improvement (MMQI) Case Conferences monthly Resident Meetings monthly OUMC & VAMC Inpatient Service Meetings with QI Reports monthly PD Meetings P1 quarterly, P2 monthly, P3 quarterly, Administrative Chief Residents monthly Rotation specific Curricula Pathology, Psychiatry, Subspecialty Clinic, VA Rehabilitation

MEDICAL EDUCATION REFERENCES, 1 of 2 Bloom BS, Engelhart MD, Furst EJ, Hill WH, Krathwohl DR. Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. New York: David McKay Company.1956. Bloom BS. Learning for Mastery. Evaluation Comment 1968; 1(2):1 12. Bower DJ, Diehr S, Morzinski JA, Simpson DE. Support challenge vision: a model for faculty mentoring. Medical Teacher 1998;20:595 597. Chen HC, van den Broek WES, ten Cate O. The case for use of entrustable professional activities in undergraduate medical education. Acad Med 2015;90:431 436. Competency Based Learning. In S. Abbott (Ed.), The glossary of education reform. Retrieved from http://edglossary.org/competency based learning. 2014 Covey SR. The 7 Habits of Highly Effective People. New York: Simon & Schuster. 1989, 2004. Csikszentmihalyi M. Flow: The Psychology of Optimal Experience. New York: Harper & Row; 1990. Dent JA, Harden RM. A Practical Guide for Medical Teachers. 4 th ed. Edinburgh: Churchill Livingstone; 2013. Ericsson KA, Krampe RT, Tesch Römer C. The role of deliberate practice in the acquisition of expert performance. Psychological Review 1993;100:363 406. Ericsson KA. Deliberate practice and acquisition of expert performance: a general overview. Acad Emerg Med 2008;15:988 994.

MEDICAL EDUCATION REFERENCES, 2 of 2 Gelb DJ, Gunderson CH, Henry KA, et al. The Neurology clerkship core curriculum. Neurology 2002; 58:849 852. Guskey TR. Closing Achievement Gaps: Revisiting Benjamin S. Bloom s Learning for Mastery. Journal of Advanced Academics 2007; 19:8 31. Huston T. Teaching What You Don t Know. Cambridge, Mass.: Harvard University Press; 2009 LCME (Liaison Committee on Medical Education). Functions and Structure of a Medical School. Standards for Accreditation of Medical Education Programs Leading to the MD Degree. Standards and Elements Effective July 1, 2015. March 2014. McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does simulation based medical education with deliberate practice yield better results than traditional clinical education? A metaanalytic comparative review of the evidence. Acad Med 2011;86:706 711. Miller GE. The assessment of clinical skills / competence / performance. Acad Med 1990;65:S63 S67. Smith SR, Dollase RH, Boss JA. Assessing students performance in a competency based curriculum. Acad Med 2003;78:97 107. Ten Cate O. Entrustable professional activities as a framework for assessment. Lecture at the 2015 ACGME Annual Educational Conference. February 28, 2015. Wass V, van der Vleuten C, Shatzer J, Jones R. Assessment of clinical competence. Lancet 2001;357:945 949,

THE END Excellence Collegiality Innovation