Quality Improvement Science in Pediatric Urgent Care: Making it meaningful Lori Rutman, MD, MPH Assistant Professor, University of Washington Pediatric Emergency Medicine, Seattle Children s Hospital
Overview Using the Model for Improvement Plan-Do-Study-Act (PDSA) (repeat) Introduction to Statistical Process Control (SPC) Theory of Variation Shewhart (control) charts What are they, How to use them, Why to use them Examples
Using the Model for Improvement Information on Model for Improvement
Fundamental Principles for Improvement 1. Know why you need to improve 2. Have a feedback mechanism 3. Develop an effective change 4. Test a change before attempting to implement 5. Know when/how to make the change permanent HC Data Guide, p 4
Model for Improvement What are we trying to accomplish? How will we know that a change is an improvement? What change can we make that will result in improvement? Act Plan Study Do HC Data Guide, p 4
Why this Model for Improvement? Useful for both process and product improvement Applicable to all types of organizations Facilitates the use of teamwork Provides a framework for the use of statistical tools Encourages planning to be based on theory Emphasizes the iterative learning process Empowers people in the organization to take action
Aim statement Clearly written Aim statement Provides leadership understanding of purpose Assists with team selection Reduces variation from original purpose Defines magnitude of expected improvement Sets timeline
Family of Measures
Types of change 1 st order: Changes that are needed to keep the system performing at current levels 2 nd order: Changes that are needed to improve the system or create a new system Required in most improvement efforts Involve design/redesign of some aspect of the system Alter how the system works and what people do
2 nd order changes Developed by: Critical thinking about current system Learning from approaches in other organizations Using new technology Applying creative thinking methods
Case study: Morning meeting The director of an urgent care feels frustrated by recent difficulties expressed by staff regarding their daily morning meeting. Historically, this meeting was a time to discuss anticipated patient/clinic issues for the day and for staff to bring any quality/safety concerns to the director. Recently, the meetings have been running late due to multiple staff issues; some relevant to patient care and others not. At the last meeting, a physician stormed out in the middle of the meeting, saying we re wasting our time here.
Using the Model for Improvement What are we trying to accomplish? Redesign morning meeting to be more effective/timely. During next 4 weeks, redesign the morning meeting process to increase the number of topics covered (baseline 5) and end meeting within (or under) the allotted time 80% of the time.
Using the Model for Improvement How will we know that a change is an improvement? Monitor: Length of time of each meeting Number of items or topics covered Scale of 1 to 5, How was the meeting? assessment
Using the Model for Improvement What changes can we make that will result in improvement? Attendees at the next morning meeting brainstormed a list of changes they thought would result in improvement: Have fewer people in attendance Meet less often Prepare and use an agenda Give assignments to prepare for the meeting Quit having the meetings Limit the meeting time for each issue/patient End the meeting at 8AM no matter what Keep minutes and distribute for review and follow up Limit issues to critical ones (minor issues to be worked out by smaller groups)
The PDSA Cycle Plan, Do, Study, Act Also known as: Shewhart Cycle Deming Cycle Learning and Improvement Cycle Act Study Plan Do HC Data Guide, p 4
Act - What changes are to be made? - Next cycle? Study - Complete the analysis of the data - Compare data to predictions - Summarize what was learned Plan - Objective - Questions and predictions (Why?) - Plan to carry out the cycle (who, what, where, when) - Plan for Data collection Do - Carry out the plan - Document problems and unexpected observations - Begin analysis of the data HC Data Guide, p 9
Principles for Testing a Change Think a couple of Cycles ahead of the initial test Scale down, decrease time required for initial test Do not try to get buy-in for the test; use volunteers Be innovative to make the test feasible HC Data Guide, p 10
PDSA Cycles: Why Test? Increase degree of belief that change improvement Decide which changes will lead to improvement Evaluate how much improvement to expect Adapt proposed change to actual environment Evaluate cost implications/side effects of the change Experience the change prior to implementation
Back to the morning meeting
A: Director made some clarifications to new meeting process; prepared copies to hand out at the next meeting. PDSA Cycle 1 Act Plan P: Director designed a new meeting process on paper; predicted more effective meeting in half time. Sent to group for comments S: Reviewed feedback and concerns from group Study Do D: Took more than a week to get comments back from everyone
Repeated Use of the PDSA Cycle Changes That Result in Improvement Hunches Theories Ideas HC Data Guide, p 9
A: Staff member designed a standardized form for submitting topics for future meetings PDSA Cycle 2 Act Plan P: New meeting process reviewed (MWF, pre-submitted agenda, notes, timekeeper); plan to test for next meeting; all predicted short meeting S: Meeting very short but for wrong reason; quality ratings ranged from 1-3 Study Do D: Only 1 agenda topic presubmitted, group didn t understand how to submit topics
Repeated Use of the PDSA Cycle Changes That Result in Improvement Hunches Theories Ideas HC Data Guide, p 9
A: Everyone ready to commit to new process; agreed to continue to refine as necessary PDSA Cycle 3 Act Plan P: Submitted topics collected and agenda constructed day before the meeting; agenda distributed in advance so individuals could prepare S: Previously unable to cover more than 5 issues at a meeting; Quality ratings ranged from 4-5 s Study Do D: 15 agenda items covered. One topic not covered in allotted time, team forwarded to next meeting per process
Repeated Use of the PDSA Cycle Changes That Result in Improvement Hunches Theories Ideas HC Data Guide, p 9
A: Continue to reap benefits of new process: 60% reduction in meeting time, improved morale among staff who now have a way to get their issues on the agenda. PDSA Cycle 4 Act Plan P: New meeting process implemented across multiple subspecialty groups within the medical practice. Plans made to make the process sustainable. S:Minor changes made over next 6 months based on data collected and interprepted. Study Do D: Run charts of length of time, number of topics covered, quality rating developed and updated at end of each meeting
Morning meeting run charts HC Data Guide, p 9
Morning meeting discussion Starting with the Model for Improvement Answer 3 questions to focus the improvement effort Supporting change with data Use of run charts in real time to monitor successes/challenges Developing the change Be creative, think outside the box, poll people Testing the change Use multiple PDSA cycles; small scale changes for cycle 1 and ramp up Implementing a change Implementation cycle focuses on how to make change permanent The human side of change Engage all stakeholders early in process of change
Intro to SPC Walter Shewhart (1891 1967) W. Edwards Deming (1900-1993)
Shewhart s Theory of Variation Common Causes inherent in the system over time, affect everyone working in the system, and all outcomes of the system Common cause of variation Chance cause Stable process Process in statistical control Special Causes not part of the system all the time, do not affect everyone, arise because of specific circumstances Special cause of variation Assignable cause Unstable process Process not in statistical control
What is a Shewhart (Control) Chart? Statistical tool used to distinguish between variation in a measure due to common vs special causes Includes a center line and an upper and lower limit, called three sigma limits, which vary based on subgroup size Each type of chart has it s own formula with the most appropriate statistics used for that formula
Selecting the Type of Shewhart Chart Continuous data (Variable data) Weight, time, money, temperature, length, volume, workload or productivity (throughput) Classification data & Count data (Attribute data) Classification into 1 of 2 categories: harm/not harm conform/non-conform go/no-go pass/fail good/bad (occurrences plus non-occurrences) Count data focuses on attributes that occur that are unusual or undesirable: # Defects # mistakes # accidents (occurrences only)
Shewhart Charts
Three-Sigma Control Limits Shewhart called the control limits "three-sigma" control limits and gave a general formula to calculate the limits for any statistic. Let S be the statistic to be charted, then the centerline: CL = u s the upper control limit: UCL = u s + 3 * σ s the lower control limit: LCL = u s - 3 * σ s
Rationale for the use of 3-sigma limits The limits have a basis in statistical theory In most cases, use of the limits will approximately minimize the total cost due to overreaction and under reaction to variation in the process The limits protect the morale of workers in the process by defining the magnitude of the variation that has been built into the process HC Data Guide p. 115
Shewhart Charts
Interpreting Shewhart Charts Outside of limits: A data point that falls outside the limits on the chart, either above the upper limit or below the lower limit. Shift: Eight or more consecutive POINTS either all above or all below the mean. Trend: Six points all going up or all going down Two Out of Three: Two out of three consecutive points in the outer third of the chart. HC Data Guide p. 116
Rules for Special Cause
How to use Shewhart Charts? Example in Chapter 8, page 292-295 44
Why use Charts? PE, p.65 46
Same Before (8hrs) and After (3 hrs) Data Very different patterns on the run charts PE, p.66 47
Summary The Model for Improvement is a simple, powerful tool for accelerating improvement. Setting Aims Establishing Measures Selecting Changes PDSA cycles allow us to test changes in an iterative way, learning from our data from one cycle to the next. Statistical Process Control charts are useful tools for understanding the impact of our improvement efforts over time.
Examples
ED front-end redesign What are we trying to accomplish? Aim: Rapidly redesign and test a frontend system that reduces time to provider to 30 minutes or less. PE, p.65 50
Methods Model for new ED 5-day multidisciplinary workshop Rapid cycle trials
Lobby RN Elaine
Medication Intake Ashley
Care Team Swarm Elaine
Early Initiation Team Elaine
Results Current state Pilot 1 Pilot 2 N N/A 18 80 ED census 130 162 152 % admitted 20 25 20
Results Time (minutes) Cycle
Results % Patients Seen by Provider within 30 minutes 100% 93% 80% 64% 60% 40% 35% 20% 0% Current State Pilot 1 Pilot 2
Conclusions Rapid process improvement methodology (PDSA) was used to design and test a front-end ED system that reduced patient waiting time Lean principles may be employed to rapidly improve health care delivery systems
Asthma admission criteria Prolonged ED length of stay (LOS) for moderate to severe asthmatics who were ultimately admitted to the hospital Audit of asthmatics 10/2010-3/2011 ~90% with respiratory score 9-12 after 1- hour of therapy were admitted
Modified pathway What change can we make that will result in improvement?
Asthma admission criteria How will we know that a change is an improvement? Determine the impact of adding standardized, objective admission criteria after 1-hour of treatment on: ED LOS for admitted asthmatics Time to bed request Percent asthma admissions Inpatient length of stay
Balancing measures Percent asthma admissions Inpatient length of stay
Results Minutes 400 350 UCL ED Length of Stay for Admitted Patients with Asthma Baseline Uptake Sustained Improvement 300 281 266 250 LCL 249 200 150 Modified asthma pathway 100 Sep-10 Oct-10 Nov-10 Dec-10 Jan-11 Feb-11 Mar-11 Apr-11 May-11 Jun-11 Jul-11 Aug-11 Sep-11 Oct-11 Nov-11 Dec-11 Jan-12 Feb-12 Mar-12 Apr-12 May-12 Jun-12 Jul-12 Aug-12 Sep-12 Oct-12 Nov-12 Dec-12
Results Days 2.50 Inpatient Length of Stay 2.00 UCL 1.50 1.00 LCL 0.50 Modified asthma pathway - Sep-10 Oct-10 Nov-10 Dec-10 Jan-11 Feb-11 Mar-11 Apr-11 May-11 Jun-11 Jul-11 Aug-11 Sep-11 Oct-11 Nov-11 Dec-11 Jan-12 Feb-12 Mar-12 Apr-12 May-12 Jun-12 Jul-12 Aug-12 Sep-12 Oct-12 Nov-12 Dec-12
Results Minutes 250 Time to Bed Request 200 UCL 150 150 142 120 100 LCL 50 Modified asthma pathway 0 Sep-10 Oct-10 Nov-10 Dec-10 Jan-11 Feb-11 Mar-11 Apr-11 May-11 Jun-11 Jul-11 Aug-11 Sep-11 Oct-11 Nov-11 Dec-11 Jan-12 Feb-12 Mar-12 Apr-12 May-12 Jun-12 Jul-12 Aug-12 Sep-12 Oct-12 Nov-12 Dec-12
Results Percent 60% 50% 40% UCL Asthma Admissions? 30% 20% LCL 10% Modified asthma pathway 0% Sep-10 Oct-10 Nov-10 Dec-10 Jan-11 Feb-11 Mar-11 Apr-11 May-11 Jun-11 Jul-11 Aug-11 Sep-11 Oct-11 Nov-11 Dec-11 Jan-12 Feb-12 Mar-12 Apr-12 May-12 Jun-12 Jul-12 Aug-12 Sep-12 Oct-12 Nov-12 Dec-12
Results Percent 60% 50% 40% UCL Asthma Admissions (through March 2013)? 30% 20% LCL 10% Modified asthma pathway 0% Sep-10 Oct-10 Nov-10 Dec-10 Jan-11 Feb-11 Mar-11 Apr-11 May-11 Jun-11 Jul-11 Aug-11 Sep-11 Oct-11 Nov-11 Dec-11 Jan-12 Feb-12 Mar-12 Apr-12 May-12 Jun-12 Jul-12 Aug-12 Sep-12 Oct-12 Nov-12 Dec-12 Jan-13 Feb-13 Mar-13
What happened in Sept-Nov 2012? Seattle Source: http://nasa.gov
Results Period of increased asthma admissions
Results Percent Asthma Admissions 60% WA Wildfires (Sept-Oct'12) 50% UCL 40% 34% 30% 20% LCL 10% 0% Modified asthma pathway Sep-10 Oct-10 Nov-10 Dec-10 Jan-11 Feb-11 Mar-11 Apr-11 May-11 Jun-11 Jul-11 Aug-11 Sep-11 Oct-11 Nov-11 Dec-11 Jan-12 Feb-12 Mar-12 Apr-12 May-12 Jun-12 Jul-12 Aug-12 Sep-12 Oct-12 Nov-12 Dec-12 Jan-13 Feb-13 Mar-13
Conclusions Use of standardized, objective admission criteria for asthmatics after 1-hour of treatment led to: Decreased ED length of stay and time to bed request No change in percent of asthma admissions, inpatient length of stay Drilling down the special cause in percent asthma admissions revealed poor local air quality in the setting of wildfires as a potential explanation for the temporary increase in admissions.
Pediatr Emerg Care. 2015 Jun;31(6):395-8.
Resources Information on Model for Improvement
Thank you