Evaluating Knowledge and Communication Skills: The Oral Exam Faculty Development Workshop March 14, 2011
Mark Wolff, DDS, PhD Professor & Chair James M. Kaim, DDS Professor & Associate Chair David Hershkowitz, DDS Clinical Assistant Professor & Associate Chair Kenneth L. Allen, DDS, MBA Clinical Associate Professor
Outline: Overview Mark Wolff From Inception to Implementation Description Kenneth Allen Format & Forms Faculty David Hershkowitz Selection & How to Train and Standardize Real Session James Kaim Listen & Grade; Review Q & A
NEW YORK UNIVERSITY COLLEGE OF DENTISTRY
NEW YORK UNIVERSITY COLLEGE OF DENTISTRY Founded in 1865 Largest dental school in the United States 360,000 dental visits per year 1500 students from 55 countries Over 500 operatories 200 full time faculty 700 part time faculty Clinics open from 8:30 AM 8:00 PM 24 hour emergency coverage
ROSENTHAL CENTER FOR ESTHETIC DENTISTRY
Overview- Mark Wolff From Inception to Implementation As clinical educators, how often do we hear these questions? What did they teach this student in pre-clinic? This student may be smart but.. he can t think for himself he can t speak to patients I don t think he understands. We taught this in Basic Science, why doesn t he remember?
Overview- From Inception to Implementation How do we evaluate a student s knowledge Multiple choice examinations Essay examinations Portfolio papers Case based presentations Student self reflections Objective Standardized Clinical Examinations Daily clinical feedback of direct observations Clinical competency examinations And more
Overview- From Inception to Implementation How do we evaluate a student s knowledge Multiple choice examinations Essay examinations Portfolio papers Case based presentations Student self reflections Objective Standardized Clinical Examinations Daily clinical feedback of direct observations Clinical competency examinations And more
Overview From Inception to Implementation History of oral examination Socratic Medical Education Rarely Summative Never Comprehensive Apprentice in nature dependent on the master Continuous evaluation rarely high stakes
Overview From Inception to Implementation History of oral examination Medieval University Education Advancement Examination Summative Comprehensive High Stakes- often one examination per year
Overview From Inception to Implementation History of oral examination European Model of Graduate Education Summative Comprehensive High Stakes- often one examination per year
Overview From Inception to Implementation History of oral examination Graduate Education in the United States; Doctorate and some Masters Degrees The Oral Defense Summative Not Necessarily Comprehensive High Stakes- often one examination per degree
Overview From Inception to Implementation Why give an oral exam? What did they teach this student in pre-clinic? This student may be smart but.. he can t think for himself he can t speak to patients I don t think he understands.
Overview From Inception to Implementation Why give an oral exam? We have a large, diverse student population, Cultural Diversity poses challenges Students were not be evaluated on communication skills We were not challenging our students
Overview From Inception to Implementation Why give an oral exam? We have a large, diverse faculty, some teach with the Socratic method most are willing to offer appropriate feedback some are willing to award unsatisfactory grades
Overview From Inception to Implementation What were the challenges in deciding to administer oral examinations? Everything! scheduling students scheduling faculty scheduling rooms calibrating faculty standardizing faculty
Overview From Inception to Implementation What were the challenges in deciding to administer oral examinations? Everything! student buy in student training case selection and development evaluation rubric development and validation
Overview From Inception to Implementation So with all these challenges how did oral examinations come about? We decided to JUST DO IT!
Overview From Inception to Implementation TIMELINE APRIL Discussed concept Decided on feasibility- decided first exam in September Began deciding on the examination content Developed scheduling web site June Incorporated pass/fail requirement in D4 Comprehensive Care Course Informed class that there would be an oral exam
Overview From Inception to Implementation TIMELINE July Assigned faculty Ran faculty training on oral examination Ran mock examination on volunteer class officers August Completed work on multiple examinations Rework/design calibrate and prepare September LAUNCH
KENNETH ALLEN Description Format & Forms
F O R M A T: Beta testing 1 D4 Student 2 Trained and Standardized Faculty Added in 2010-11 1 D3 Observer 20 minutes to review *FMS *Medical History *Dental History and Chief Complaint 15 minute exam
T I M E L I N E 2:00 PM 2:20 PM 2:35 PM 2:40 PM 2:55 PM 3:00 PM 3:15 PM 3:20 PM 3:35 PM 3:40 PM Student 1 Review EXAM GRADE Student 2 Review EXAM GRADE Student 3 Review EXAM GRADE Student 4 Review EXAM GRADE Finalize Grades
Grades Posted on BlackBoard Comments added for those who get an F
Faculty and Students David Hershkowitz Select, Schedule, Standardize and Re-Standardize
Oral Exam Faculty Coverage 2010-11 All Exams are in Room 3W Monday Tuesday Wednesday Thursday 2:00-4:00 PM Fisher, S Godder, B Cammarata, R Congiusta, M Hertz, P Rosenblum, N Gluck, G DeBartolo, A Klaczany, G Soeprono, A Gross, H Glotzer, D Schwarz, J Weiss, J Maitland, R Mychajliw, P
Students D3 and D4 pre-assigned by academic affairs Curric. Year Student Name / ID e-mail Course ID / Section / Type Course Title Mon, 09/13/10 02:00 PM - 02:15 PM 1 xxxxxxxxxxxxxxxxx x 3CD-B x@nyu.edu D60.4508E - O079B Clin Oral Examination - D4 D4 2 xxxxxxxxxxxxxxxxx x 5CD-B x@nyu.edu D60.3508E - E012B Clin Oral Examination - D3 D3 02:20 PM - 02:35 PM 1 xxxxxxxxxxxxxxxxx x 5AB-B x@nyu.edu D60.3508E - E158B Clin Oral Examination - D3 D3 2 xxxxxxxxxxxxxxxxx x 5AB-B x@nyu.edu D60.4508E - O167B Clin Oral Examination - D4 D4 02:40 PM - 02:55 PM 1 Adams, Timothy x 5AB-B x@nyu.edu D60.3508E - E078B Clin Oral Examination - D3 D3 2 Nguyen, Diana K. x 5CD-B x@nyu.edu D60.4508E - O056B Clin Oral Examination - D4 D4 03:00 PM - 03:15 PM 1 Ghookasian, Miganoush x 3CD-B x@nyu.edu D60.4508E - O121B Clin Oral Examination - D4 D4 2 Park, Dong x 4AB-B x@nyu.edu D60.3508E - E034B Clin Oral Examination - D3 D3
T I M E L I N E 2:00 PM 2:20 PM 2:35 PM 2:40 PM 2:55 PM 3:00 PM 3:15 PM 3:20 PM 3:35 PM 3:40 PM Student 1 Review EXAM GRADE Student 2 Review EXAM GRADE Student 3 Review EXAM GRADE Student 4 Review EXAM GRADE Finalize Grades
2009-10 2010-11 100.0% Pass/Fail - Oral Exams 2009-11 80.0% 60.0% 40.0% 20.0% 0.0% 2009-10 2010-11 D4 D4 D3 No Show Rate Pass Rate
100% Faculty - Individual Pass Rates 80% 60% 40% 2009-10 2010-11 20% 0%
Faculty Retreat -
THE EXAM James Kaim
Simulated Session o Divide room into groups of 2 o Give out sample tests and evaluation forms o Play a recorded session o Attendees evaluate student o Attendees evaluate faculty o Compare faculty evaluations o Feedback from group
Case J SEX: M AGE: 66 Social Hx: Smokes 2 cigarettes/day. Moderate alcohol use. Brushes twice per day, flosses regularly. Uses Crest whitening toothpaste and Listerine mouthwash. Med Hx: Last MD visit was 1 year ago. BP normal. ASA 81mg daily. Rx Flomax. Dent Hx: Last DDS visit was 1 year ago for a checkup. CC: My bridge feels loose. Clinical Findings: Probing depth generalized 4 6 mm.
Case C SEX: M AGE: 22 Social Hx: Non smoker, weekend alcohol use limited to 1 or 2 beers. He brushes once/day in the evening, does not use floss but uses Scope mouthwash. Med Hx: Last MD visit was 8 months ago, BP 140/78. OTC medications vitamins. Dent Hx: Last DDS visit was 2 years ago. I don t like them picking at my teeth. Teeth are sensitive to hot and cold. CC: My back teeth get food stuck in them and it hurts when I chew on my upper right side. Clinical Findings: The gingival tissues readily bleed on probing in the posterior quadrants.
Case N SEX: F AGE: 58 Social Hx: Brushes once per day, flosses rarely. Does not smoke, drinks wine daily. Med Hx: Last MD visit was 1 year ago. BP normal. Rx Fosomax Dent Hx: Last DDS visit was 1 year ago for a checkup. CC: My teeth are ugly, can you fix them. Clinical Findings: Probing depth generalized 4 6 mm.
Case D SEX: F AGE: 52 Social Hx: Non smoker, limited alcohol use (less than 1 drink per day). She brushes twice/day in the evening, used floss once/day, no mouthwash. Med Hx: Last MD visit was 1 ½ years ago, BP normal. Broken hip last year repaired with a rod and pins. Dent Hx: Last DDS visit was 5 years ago and patient had extensive treatment. CC: I want to make sure I don t have cavities and I need a cleaning. Clinical Findings: The gingival tissues demonstrate bleeding on probing, pocket depths range from 3 6 mm.