The Evolution of Maintenance of Certification

10 0% Lo ng T erm Goal = 95% 90% Performance Im pr oveme nt P4P 80% F eed back Co llabo rative on Program Rep orts Re liability 70% 60% Pr oject Incep tion 50% 40% 30% % 10% 0% Web -bas ed Self - Registr y Man age me nt La unche d Colla bor ative Initia ted P4P Dead line "Per fect Care " In cluding F lu Sh ot Oct 03 Nov 03 Dec 0 3 Ja n 0 4 Feb 04 Mar 0 4 Apr 0 4 May 04 Jun 04 Ju l 04 Aug 04 Sep 04 Oct 0 4 Nov 04 Dec 0 4 Jan 05 Feb 05 Ma r 0 5 Apr 05 May 05 Jun 0 5 Jul 05 A ug 05 Se p 05 Oct 0 5 N ov 05 OVPCA Network Pr actice 1 Pra ctice 2 Pra ctice 3 Pra ctice 4 Practice 5 Pra ctice 6 Framework of the ABP Mission Founded 85 years ago at the dawn of the era of specialization The Evolution of Maintenance of Certification Keith Mann, MD, M.Ed. Professor of Pediatrics / University of Missouri Kansas City School of Medicine Associate Chair for Quality and Safety / Department of Pediatrics Chief Medical Quality & Safety Officer / Associate Executive Medical Director A nationally recognized way to make pediatric training, qualifications, and competencies clear to the public. Sole mission is to the public: To certify pediatricians based on standards of excellence that lead to high quality care. The ABP certification provides assurance to the public that a pediatrician fulfills the continuous evaluation requirements that encompass the six core competencies. [from the ABP Mission Statement] Professional Self-Regulation Reasons for Change in Certification Societies grant professional communities freedom from external regulation in return for their commitment to regulate their members' conduct. For its part, the profession must self-regulate in an open and rigorous fashion or it will lose the privilege, and this would be unfortunate for both society and for physicians. IOM reports Crossing the Quality Chasm and To Err is Human documenting the need for changes leading to improvement Health care research that uncovered wide gaps in the quality and cost of care for conditions known to have a best practice The public awareness about the quality gaps The public s demand for accountability from all involved in the profession Trust me, I am a physician. Cruess and Cruess, AMA Journal of Ethics JJ Cohen, JAMA A system based simply on a single or periodic tests of knowledge needed improvement! Show me the Data. Four Parts of Maintenance of Certification Part 1 - Professionalism C o n c e r n s Part 1: Professionalism Maintain unrestricted licensure Part 2: Lifelong Learning and Self-Assessment Identify knowledge gaps and work on them Part 3: Cognitive Knowledge Secure exam Part 4: Improvement in Practice Application of QI science and methods to any process intended to improve the health of children Current standard is unrestricted licensure Significant effort in the training space Teaching, Promoting and Assessing Professionalism Across the Continuum: A Medical Educator s Guide Part 2 Self-assessment in development Will be the subject of the annual American Board of Medical Specialties review for 17-18 1

Part 2 Self Assessment Learning activities that incorporate individual learning with individual assessment and feedback. ABP-developed activities (N=53) Subspecialty self assessments, topic specific self assessments, Decision Skills, General Pediatrics Knowledge Skills Assessment, and Question of the Week No additional charge for access or for CME Activities developed by other organizations (ex: AAP PREP) 72 listed on ABP website Outside organizations may charge for access and/or for CME New in 17: ACCME collaboration QOW Enhancements CME (!) As of January 17 10 MOC points + 10 CME hours per correct QOW answers Pearl available to all Archiving of retired items ACCME Collaboration January 17 Diplomates who participate in CME activities that meet ABP MOC standards will be able to earn ABP MOC credit. Activities must involve an assessment and feedback to the individual learner CME activity providers will apply through ACCME s automated system to be able to award MOC credit, based on their attestation that the activity meets ABP standards Trust and verify : ABP will accept the attestations, with random audits to assure adherence to standards ACCME collaboration with ABIM has resulted in >1000 new activities for Part 2 credit 1 CME Credit = 1 MOC II Point ACCME Collaboration Count of ActivityType Regularly Scheduled Series Journal-based CME Internet Live Course Internet Activity Enduring Material Enduring Material Course 0 50 100 150 0 250 300 First 6 months results: 489 MOC Part 2 activities registered 227 current, active activities 2590 diplomate completions 51 unique diplomates awarded credit 31,0 MOC Points awarded so far in 17 Part 3 - Cognitive Expertise Current Requirement for Part 3: Successfully pass a secure test of knowledge every 10 years in each area of certification. 1969 ABMS introduces Recertification 1980 1991 Closed Book (voluntary) 1993 02 1993-02 Open Book Exam (every 7 years) Although the MOC cycle is 5 years, a secure test of knowledge is only required every 10 years. Point of confusion: Exam cycles usually do not coincide with MOC points cycles! 03 present Secure Exam (every 10 years) 17 pilot: MOCA Peds continuous assessment 2

Concerns about the secure MOC exam Future of Testing Conference May 15 https://abpedsfoundation.org/ Walking in Two key : Should we move to remote proctoring for MOC Part 3? Should we allow for use of resources? Walking out ABA MOCA Minute MOCA-Peds Assessment of and for learning High level of diplomate engagement in development process including user and focus groups to guide development and evaluation MOCA-Peds Overview Jan-Mar Apr-Jun Jul-Sep Oct-Dec 1 2 3 4 plus 0-5 emerging topics (unscored) 40 learning objectives available prior to each year 2 per learning objective per year Some customization (inpatient, outpatient, or combined) Flexibility to answer within quarter; one-at-a-time or in batches Use of books, online references, resources is allowed, but should not be needed. Discussing, sharing of is NOT allowed. MOCA-Peds Pilot Open to diplomates with GP exam due in 17 ~5,000 out of 6,000 eligible signed up Pilot participants postpone (possibly forever) their secure exam Must answer all Meet the passing standard Participate in surveys Pilot will continue in 18 with the 17 participants and the addition of those with a GP exam due in 18 MOCA-Peds go-live planned for 19 Will include all Subspecialties by 22 Pilot Status Q1 Results 5,032 out of 5,081 logged in (99%) 4,993 completed all (98%) 2,914 comments on Average time per question 1 minute, 56 seconds ~39 minutes total for ***Quarter 2 Preliminary Results*** Only 10% drop in completion by end of 2 nd month. Over 500 mobile app user 3

70.0% 60.0% 61.6% Quarter 1 Results: MOCA Peds System Questions 57.9% 57.9% 58.2% 6% are now scoring less than 60% 49.2% 50.0% 46.3% 40.0% 30.0% 87.4% Strongly 79.8% Strongly 79.1% Strongly 80.6% Strongly 72.5% Strongly 59.5% Strongly Agree/Agree 25.8% Agree/Agree Agree/Agree Agree/Agree Agree/Agree Agree/Agree 21.9% 21.2% 22.4% 23.3% 21.6%.0% 14.9% 14.3% 11.6% 13.2% 11.5% 13.2% 9.4% 87.4% Strongly 79.7% Strongly 79.1% Strongly 80.6% Strongly 72.5% Strongly 59.5% 9.4% Strongly 10.0% Agree/Agree 7.4% Agree/Agree 6.4% Agree/Agree Agree/Agree 6.3% Agree/Agree Agree/Agree 2.2% 1.0% 1.3% 1.3% 1.7% 3.2% 4.5% 0.0% a. Questions aligned with b. Question difficulty wasc. Questions assessed my d. Questions were e. I had enough time to f. Questions were the learning objectives appropriate clinical judgment, going relevant to general answer each question relevant to my practice beyond factual recall pediatrics Strongly disagree Disagree Neither agree or disagree Agree Strongly agree a Source = MOCA Peds Quarter 1 Survey b Sample Size: n = 4,181 Participant Count 2,500 2,250 2,000 1,750 1,500 1,250 1,000 750 500 250 2,285 54.7% I did not prepare in advance 285 306 a Source = MOCA Peds Quarter 1 Survey b Sample Size: n = 4,181 Quarter 1 Results: Time Spent Studying in Advance of Taking MOCA Peds Questions 401 317 297 290 6.8% 7.3% 9.6% 7.6% 7.1% 6.9% <1 hour 1 to <2 hours 2 to <5 hours 5 to <10 hours Hours of Study Before Beginning 10 to hours > hours Count 2,000 1,750 1,500 1,250 1,000 750 500 250 803 19.2% I did NOT use resources during this quarter a Source = MOCA Peds Quarter 1 Survey b Sample Size: n = 4,181 Quarter 1 Results: Resource Used While Taking MOCA Peds Questions 1,782 42.6% 928 22.2% 668 16.0% With 1 to 5 With 6 to 10 With more than 10 Number of Questions Where Resources Were Used Life Circumstances/Tech Issues/Scoring Scoring process will drop the lowest 4 quarters of performance at the end of Year 4. (Diplomates will know at end of year 3 if they will successfully be meeting performance standard) Five-year MOC Cycle Year 1 Year 2 Year 3 Year 4 80 80 80 80 Because you can drop 4 quarters, TECHNICALLY you can drop all of year 4 and participate only Years 1-3. Am I meeting the performance standard at the end of Year 4? Year 5 Re-entry exam (if needed) You do not have to participate in MOCA- Peds until your next MOC 5-year cycle. 4

Part 3 Options (once MOCA-Peds adopted) MOCA-Peds Default with enrollment No additional fees Earn Part 2 points Proctored Exam Every 5-years at proctored site Additional fee to cover cost of seat fee and processing No Part 2 credit Subspecialty Rollout Schedule Year Subspecialties 19 CHAB, GAST, IDIS PULM, DBEH, NEON, NEPH 21 CARD, HMED, CRIT, ENDO 22 ADOL, EMER, HEMO, RHEU Maintaining Multiple Certificates Regardless of the number of certification held, diplomates will need to answer the same number of for a given discipline/area to maintain the certification for that area. ***Potential Concern*** 1. Will this be a burden on those that are maintaining multiple certificates? 2. Will this have impact on subspecialists maintaining in general pediatrics? Part 4: Improvement in Medical Practice ABMS 15 Standards: Activities result in improved population health outcomes, access to care, improved patient experience, increased value in health care system Encourage activities within the context of the health care team and system of practice Assure each diplomate has an adequate knowledge of QI science and practice ABP Part 4 credit the early days The options were: Online modules (PIMs or EQIPP), a long, complex and expensive application, or be a part of an organizational portfolio PIMs and EQIPP focused only on primary care pediatric offices So everyone chose handwashing because it was quick and easy, but often not very meaningful Application for independent projects was long and expensive only large hospitals with a QI staff could manage it No repeat credit for ongoing participation in large, more complex improvement efforts First Step in Engagement Pediatric Portfolio Sponsor Effort to reduce the application burden and costs for organizations Pediatric Portfolio Sponsor Program for organizations to approve and manage QI projects and award MOC IV points What can Portfolio Sponsors do? Evaluate their own quality improvement (QI) projects against the ABP standards Approve QI projects internally for MOC credit 5

Children s Mercy Current Portfolio 29 projects have been reviewed 24 projects have been approved 10 projects are active 5 projects pending approval 570 faculty eligible for MOC Part IV credit 342 faculty members have received credit 132 additional physicians have received credit What has changed? QI has become part of daily work in many settings ABP wants to recognize QI work already being done by diplomates ABP also wants to recognize that improvements made in all aspects of a pediatrician s work can contribute to improved child health Thus, new pathways for Part 4 credit were created starting in 15 New Part 4 Application Pathways Small Group Quality Improvement Projects (completed) 25 points Create your own QI Project This allows up to 10 diplomates to receive credit for a project they have already completed. The review and processing fee is $75 per project (not per diplomate) Use this pathway for small clinical projects and educational or advocacy based improvement efforts NCQA PCMH 40 points Pediatricians who can attest that they participated in QI activities as part of earning NCQA PCMH recognition can earn Part IV MOC points https://www.abp.org/news/moc-qi-credit-now-available-through-ncqa-pcmhrecognition QI Program Development 40 points This allows individual diplomates who lead large, usually institutional QI initiatives to receive credit for their leadership. QI Project applications from small groups Affectionately known as SQIPA Built for projects led by diplomates Up to 10 pediatricians can earn credit per project Simplified/streamlined QIPA application 8, directed to the physician project leader Application is submitted when the project is completed Credit awarded immediately upon approval Email and phone coaching available from ABP staff ABP Standards for project approval Identifies a measurable quality gap; has a defined aim Requires meaningful participation by pediatricians Structured project using accepted QI methodology Systematically test changes to improve care Documents measurement data over time and uses it for routine feedback to participants to see if improvement occurs Minimum of 3 cycles of data collection and review Meaningful involvement in improvement efforts In order to receive credit, each diplomate attests to meaningful involvement in the work, by meeting 4 criteria: Be intellectually engaged in planning and executing the project. Participate in implementing the project's interventions (the changes designed to improve care). Review data in keeping with the project's measurement plan. Collaborate actively by attending team meetings. 6

MOC for Residents/Fellows Residents can now earn Part 4 MOC credit during residency for meaningful participation in QI activities (just like a diplomate) PIMs, EQIPP, other online modules Approved QI projects in institutions and organizations Small group projects (including resident-led projects) Resident MOC credit will be in the bank for when they become certified. It will then be applied to their first MOC cycle Residents can access other ABP Part 2 activities (e.g. self assessments and QOW), but will not receive bankable credit for any Part 2 activities. Applies also to Fellows in ACGME approved training programs Keep in mind that many fellows are already full fledged diplomates No Duplication of Effort MOC points earned apply to ALL of a diplomate s certificates - each person has just one MOC cycle, even for the diplomates who have 4 ABP certificates. We re not sure why anyone maintains 4 certificates Reciprocity from other Boards: Diplomates of another American Board of Medical Specialties (ABMS) board who have met MOC requirements in their second specialty. Diplomates practicing in Canada who have met the Royal College of Physicians and Surgeons (RCPSC) MOC requirements. Diplomates who have completed 12 months of training in an ACGME non- Pediatric residency or any fellowship program earn 10 Part 2 and 10 Part 4 points for each year of training. 10,392 ABP diplomates received MOC Part 4 credit for QI projects initiated by the individual, team, network, or institution (ie, not online modules) in 16 Summary Number of Diplomates 100 10000 8000 6000 4000 00 0 3230 2696 7162 5411 3374 3216 2733 1784 18 10 11 12 13 14 15 16 ABP wants to make it easier to obtain MOC while upholding reasonable standards and maintaining the trust of the public Part 1: No changes at this time Part 2: Question of the Week, ACCME collaboration, MOCA Peds Part 3: MOCA Peds pilot Part 4: New pathways (NCQA, SQIPA, QI leadership) Standard QI project applications New pathways 7