Leveraging Designed Experiments for Success Scott C. Sterbenz, P.E. Six Sigma Master Black Belt, Ford Motor Company Technical Advisor, United States Bowling Congress
Presentation Outline 2 I. Introduction A. Ford Motor Company The One Ford Plan B. United States Bowling Congress Governance of the Sport of Bowling II. Lessons Learned in Successful DOE Application A. Get Creative with Your Response: Premature Bulb Failures Ford Motor Company B. The Response Needn t be Continuous Data: Carpet Quality Ford Motor Company C. Interactions Matter: Carpet Quality Ford Motor Company D. Expand the Range of Your Factors: Static Weight Study United States Bowling Congress E. Don t Forget About Center Points: Static Weight Study United States Bowling Congress III. Questions & Discussion
Ford Motor Company 3 The One Ford Plan: Lays a foundation for business success Focuses on working together to achieve profitable growth for all Facilitates leadership in four pillars for customer satisfaction and value
United States Bowling Congress 4 The United States Bowling Congress: Vision Create lifelong bowlers Mission Provide benefits and programs to enhance the bowling experience Equipment Specifications and Certification Department: Vision Uphold the credibility of bowling Leading source of technical information Mission Bring science, technology and bowling together Solve problems, answer questions, and implement specifications Expert technical services and sound statistical analyses
5 Leveraging Designed Experiments for Success #1: Get Creative With Your Response
Get Creative With Your Response 6 Training materials for designed experiments teach that the response should be: The KPOV of the process Directly related to the customer CTQ Selected from the C&E matrix, fishbone diagram, or y=f(x) cascade Generally, these guidelines are true, but sometimes yield a nonmeaningful measure.
Get Creative With Your Response 7 Practical Problem: Premature Bulb Failures Background: Warranty costs in 2005 were $2.7M, and increasing every model year Single highest warranty cost in Ford Motor Company Disagreement between vehicle subsystems about root cause Over-voltage Vehicle vibration Supplier quality
Get Creative With Your Response 8 Practical Problem: Premature Bulb Failures Plan: 2 5 full factorial DOE: Vibration input Voltage input Bulb supplier Filament orientation angle Filament centering Bench test: Typical customer usage cycle Twenty bulbs (replicates) What should the response be? Average time to failure Variance in time to failure Signal-to-noise ratio Is there something better?
Get Creative With Your Response 9 Practical Problem: Premature Bulb Failures Selected Responses: Reliability shape Reliability scale (B 63 Life) Analysis: Reliability Plots Constructed for Main Effects and Interactions Probability Plot of Effects to Illustrate Significance
Get Creative With Your Response 10 Practical Problem: Premature Bulb Failures Results: Cumulative savings since implementation (2008MY) = $5.6M Design rules for voltage regulation at incandescent lamps Implementation
11 Leveraging Designed Experiments for Success #2: The Response Needn t Be Continuous Data
The Response Needn t Be Continuous Data 12 Training materials for designed experiments teach that the response must be continuous data not attribute data. This guideline is true, but sometimes measurements are not possible to be continuous. Ratio or Interval Data (YES) Ordinal or Nominal Data (NO)
The Response Needn t Be Continuous Data 13 Practical Problem: Fiesta Carpet Quality Background: Critical vehicle launch for Ford Motor Company Largest threat to a quality launch Anticipated customer satisfaction concerns Ford and supplier at odds Competing responses brush marking and softness Cost versus quality Promises versus deliverables Brush Marks
The Response Needn t Be Continuous Data 14 Practical Problem: Fiesta Carpet Quality Plan: 2 6-1 fractional factorial DOE: Six factors Two center points Two replicates Evaluations: Five evaluators Brush marking and softness Likert scale Individual collected responses are attribute What can be done? Leverage replicates / multiple evaluators Transforms ordinal Likert scale Increases resolution from units digit to tenths digit Mimics continuous data
The Response Needn t Be Continuous Data 15 Practical Problem: Fiesta Carpet Quality Selected Responses: Average softness rating Average brush marking rating Analysis: Abbreviated DOE Matrix Shows Transformation of Likert Scale (Ordinal Data) Pareto Chart of Effects Illustrates Standard DOE Analysis
16 Leveraging Designed Experiments for Success #3: Interactions Matter
Interactions Matter 17 Training materials for designed experiments teach that three-way and higher interactions are rare (Sparsity of Effects Principle). Generally, this is correct. However, there are some cases where three-way interactions are not only present, but also very strong: Complex manufacturing processes Chemistry Psychology
Interactions Matter 18 Practical Problem: Fiesta Carpet Quality Selected Responses: Average softness rating Average brush marking rating Analysis: Minitab Display Available Designs Details Resolution of Designs Pareto Chart Shows Significant and Strong Three-Way Interactions
Interactions Matter 19 Practical Problem: Fiesta Carpet Quality Results: Full extent of interactions understood Fosters technical excellence Replication of knowledge Multi-Response Optimization Softness and brush marking Minitab Optimization Plot Illustrates Balance of Multiple Responses Elimination of Brush Marking; Softness Better Than Baseline
20 Leveraging Designed Experiments for Success #4: Expand the Range of Your Factors
Expand the Range of Your Factors 21 Training materials for designed experiments teach that factor levels should: Be wide enough to create a desired change in the response Go beyond typical limits in the process Not create unsafe or impossible conditions This guideline is absolutely correct: Don t be afraid to make bad parts Challenge the limits of the tools
Expand the Range of Your Factors 22 Practical Problem: Static Weight Study Background: 2007 study determined factors that affect ball motion on a lane High Influence coverstock Moderate Influence - core Low Influence static weights
Expand the Range of Your Factors 23 Practical Problem: Static Weight Study Background: 2007 study evaluated static weights within current specifications Bowling ball manufacturers requested removal of specification USBC concerned static weights were influential outside current specifications Negative Side Positive Side (±1 oz.) Bottom (Not Visible) Top (Grip Side) (±3 oz.) Finger Thumb (±1 oz.) Definition of Static Weights
Expand the Range of Your Factors 24 Practical Problem: Static Weight Study Plan: 2 6-1 fractional factorial DOE: Six factors Three static weights Core shape (intermediate diff.) Ball speed Rate of revolution Eight center points How wide should the levels be? USBC investigating completely removing static weight specification: Static weights set at maximum possible values Core shape is significant to ball motion Ball speeds and rates of revolution cover all bowling styles
Expand the Range of Your Factors 25 Practical Problem: Static Weight Study Selected Responses & Analysis: 19 measures characterize ball motion Collected from CATS (computer-aided tracking system) Transition points between phases Lengths of the phases Shape of the phases 23 Lane Sensors Track Ball Motion Regression Techniques Used to Characterize Ball Motion Mathematically
Expand the Range of Your Factors 26 Practical Problem: Static Weight Study Results: Anomalies discovered before the DOE was analyzed Residuals analysis from regression Undesirable 4 th phase of ball motion discovered Initial Results from Ball Motion Algorithm Residuals Analysis in Roll Phase Shows Missed Quadratic Term Correction Shows 4 th Phase of Ball Motion
Expand the Range of Your Factors 27 Practical Problem: Static Weight Study Results: 4 th phase is unpredictable Unfair advantage Athlete dissatisfaction Angle into Pins Augmented Angle into Pins Diminished
Expand the Range of Your Factors 28 Practical Problem: Static Weight Study Results: 4 th phase is unpredictable Unfair advantage Athlete dissatisfaction Normal / Expected Ball Motion 4 th Phase / Unexpected Ball Motion
29 Leveraging Designed Experiments for Success #5: Don t Forget About Center Points
Don t Forget About Center Points 30 Training materials for designed experiments teach to include center points: Increases power of the experiment Helps eliminate saturation Evaluates linearity of the response This guideline is absolutely correct: Can lead to use of Response Surface design More accurate modeling
Don t Forget About Center Points 31 Practical Problem: Static Weight Study Analysis: DOE analyzed without responses affected by 4 th phase Curvature was significant in 18 of 19 responses Analysis of Variance for A-Score (coded units) Source DF Seq SS Adj SS Adj MS F P Main Effects 6 0.0099790 0.00997901 0.00166317 2544.74 0.000 2-Way Interactions 15 0.0005627 0.00056270 0.00003751 57.40 0.000 3-Way Interactions 10 0.0000638 0.00006383 0.00000638 9.77 0.003 Curvature 1 0.0002570 0.00025705 0.00025705 393.30 0.000 Residual Error 7 0.0000046 0.00000458 0.00000065 Pure Error 7 0.0000046 0.00000457 0.00000065 Total 39 0.0108672 Typical Result Indicating Significance of Curvature
Don t Forget About Center Points 32 Practical Problem: Static Weight Study Analysis: Central Composite Design Reduced static weight levels: o Attempt elimination of 4 th phase o Widen specification, not elimination
Don t Forget About Center Points 33 Practical Problem: Static Weight Study Results: Non-linear effects confirmed 4 th phase of ball motion still present; direction and occurrence not predictable Effects of static weights within current specifications insignificant Static weight specification not changed Pareto of Effects Order of Effects and Non-Linearity Contour Plot Static Weights Within Specs Are Insignificant
Presentation Summary 34 Topic Get Creative with Your Response The Response Needn t Be Continuous Data Interactions Matter Expand the Range of Your Factors Don t Forget About Center Points Lessons Learned 1. Think beyond mean and standard deviation 1. Leverage replicates 2. Convert attribute data 1. Select proper design resolution 2. Improve process optimization 1. Discover what happens outside the typical inference space 1. Check linearity assumptions 2. Achieve greater knowledge with response surface methods
35 Leveraging Designed Experiments for Success Questions & Discussion