Leveraging Designed Experiments for Success

Transcription

1 Leveraging Designed Experiments for Success Scott C. Sterbenz, P.E. Six Sigma Master Black Belt, Ford Motor Company Technical Advisor, United States Bowling Congress

2 Presentation Outline 2 I. Introduction A. Ford Motor Company The One Ford Plan B. United States Bowling Congress Governance of the Sport of Bowling II. Lessons Learned in Successful DOE Application A. Get Creative with Your Response: Premature Bulb Failures Ford Motor Company B. The Response Needn t be Continuous Data: Carpet Quality Ford Motor Company C. Interactions Matter: Carpet Quality Ford Motor Company D. Expand the Range of Your Factors: Static Weight Study United States Bowling Congress E. Don t Forget About Center Points: Static Weight Study United States Bowling Congress III. Questions & Discussion

3 Ford Motor Company 3 The One Ford Plan: Lays a foundation for business success Focuses on working together to achieve profitable growth for all Facilitates leadership in four pillars for customer satisfaction and value

4 United States Bowling Congress 4 The United States Bowling Congress: Vision Create lifelong bowlers Mission Provide benefits and programs to enhance the bowling experience Equipment Specifications and Certification Department: Vision Uphold the credibility of bowling Leading source of technical information Mission Bring science, technology and bowling together Solve problems, answer questions, and implement specifications Expert technical services and sound statistical analyses

5 5 Leveraging Designed Experiments for Success #1: Get Creative With Your Response

6 Get Creative With Your Response 6 Training materials for designed experiments teach that the response should be: The KPOV of the process Directly related to the customer CTQ Selected from the C&E matrix, fishbone diagram, or y=f(x) cascade Generally, these guidelines are true, but sometimes yield a nonmeaningful measure.

7 Get Creative With Your Response 7 Practical Problem: Premature Bulb Failures Background: Warranty costs in 2005 were $2.7M, and increasing every model year Single highest warranty cost in Ford Motor Company Disagreement between vehicle subsystems about root cause Over-voltage Vehicle vibration Supplier quality

8 Get Creative With Your Response 8 Practical Problem: Premature Bulb Failures Plan: 2 5 full factorial DOE: Vibration input Voltage input Bulb supplier Filament orientation angle Filament centering Bench test: Typical customer usage cycle Twenty bulbs (replicates) What should the response be? Average time to failure Variance in time to failure Signal-to-noise ratio Is there something better?

9 Get Creative With Your Response 9 Practical Problem: Premature Bulb Failures Selected Responses: Reliability shape Reliability scale (B 63 Life) Analysis: Reliability Plots Constructed for Main Effects and Interactions Probability Plot of Effects to Illustrate Significance

10 Get Creative With Your Response 10 Practical Problem: Premature Bulb Failures Results: Cumulative savings since implementation (2008MY) = $5.6M Design rules for voltage regulation at incandescent lamps Implementation

11 11 Leveraging Designed Experiments for Success #2: The Response Needn t Be Continuous Data

12 The Response Needn t Be Continuous Data 12 Training materials for designed experiments teach that the response must be continuous data not attribute data. This guideline is true, but sometimes measurements are not possible to be continuous. Ratio or Interval Data (YES) Ordinal or Nominal Data (NO)

13 The Response Needn t Be Continuous Data 13 Practical Problem: Fiesta Carpet Quality Background: Critical vehicle launch for Ford Motor Company Largest threat to a quality launch Anticipated customer satisfaction concerns Ford and supplier at odds Competing responses brush marking and softness Cost versus quality Promises versus deliverables Brush Marks

14 The Response Needn t Be Continuous Data 14 Practical Problem: Fiesta Carpet Quality Plan: fractional factorial DOE: Six factors Two center points Two replicates Evaluations: Five evaluators Brush marking and softness Likert scale Individual collected responses are attribute What can be done? Leverage replicates / multiple evaluators Transforms ordinal Likert scale Increases resolution from units digit to tenths digit Mimics continuous data

15 The Response Needn t Be Continuous Data 15 Practical Problem: Fiesta Carpet Quality Selected Responses: Average softness rating Average brush marking rating Analysis: Abbreviated DOE Matrix Shows Transformation of Likert Scale (Ordinal Data) Pareto Chart of Effects Illustrates Standard DOE Analysis

16 16 Leveraging Designed Experiments for Success #3: Interactions Matter

17 Interactions Matter 17 Training materials for designed experiments teach that three-way and higher interactions are rare (Sparsity of Effects Principle). Generally, this is correct. However, there are some cases where three-way interactions are not only present, but also very strong: Complex manufacturing processes Chemistry Psychology

18 Interactions Matter 18 Practical Problem: Fiesta Carpet Quality Selected Responses: Average softness rating Average brush marking rating Analysis: Minitab Display Available Designs Details Resolution of Designs Pareto Chart Shows Significant and Strong Three-Way Interactions

19 Interactions Matter 19 Practical Problem: Fiesta Carpet Quality Results: Full extent of interactions understood Fosters technical excellence Replication of knowledge Multi-Response Optimization Softness and brush marking Minitab Optimization Plot Illustrates Balance of Multiple Responses Elimination of Brush Marking; Softness Better Than Baseline

20 20 Leveraging Designed Experiments for Success #4: Expand the Range of Your Factors

21 Expand the Range of Your Factors 21 Training materials for designed experiments teach that factor levels should: Be wide enough to create a desired change in the response Go beyond typical limits in the process Not create unsafe or impossible conditions This guideline is absolutely correct: Don t be afraid to make bad parts Challenge the limits of the tools

22 Expand the Range of Your Factors 22 Practical Problem: Static Weight Study Background: 2007 study determined factors that affect ball motion on a lane High Influence coverstock Moderate Influence - core Low Influence static weights

23 Expand the Range of Your Factors 23 Practical Problem: Static Weight Study Background: 2007 study evaluated static weights within current specifications Bowling ball manufacturers requested removal of specification USBC concerned static weights were influential outside current specifications Negative Side Positive Side (±1 oz.) Bottom (Not Visible) Top (Grip Side) (±3 oz.) Finger Thumb (±1 oz.) Definition of Static Weights

24 Expand the Range of Your Factors 24 Practical Problem: Static Weight Study Plan: fractional factorial DOE: Six factors Three static weights Core shape (intermediate diff.) Ball speed Rate of revolution Eight center points How wide should the levels be? USBC investigating completely removing static weight specification: Static weights set at maximum possible values Core shape is significant to ball motion Ball speeds and rates of revolution cover all bowling styles

25 Expand the Range of Your Factors 25 Practical Problem: Static Weight Study Selected Responses & Analysis: 19 measures characterize ball motion Collected from CATS (computer-aided tracking system) Transition points between phases Lengths of the phases Shape of the phases 23 Lane Sensors Track Ball Motion Regression Techniques Used to Characterize Ball Motion Mathematically

26 Expand the Range of Your Factors 26 Practical Problem: Static Weight Study Results: Anomalies discovered before the DOE was analyzed Residuals analysis from regression Undesirable 4 th phase of ball motion discovered Initial Results from Ball Motion Algorithm Residuals Analysis in Roll Phase Shows Missed Quadratic Term Correction Shows 4 th Phase of Ball Motion

27 Expand the Range of Your Factors 27 Practical Problem: Static Weight Study Results: 4 th phase is unpredictable Unfair advantage Athlete dissatisfaction Angle into Pins Augmented Angle into Pins Diminished

28 Expand the Range of Your Factors 28 Practical Problem: Static Weight Study Results: 4 th phase is unpredictable Unfair advantage Athlete dissatisfaction Normal / Expected Ball Motion 4 th Phase / Unexpected Ball Motion

29 29 Leveraging Designed Experiments for Success #5: Don t Forget About Center Points

30 Don t Forget About Center Points 30 Training materials for designed experiments teach to include center points: Increases power of the experiment Helps eliminate saturation Evaluates linearity of the response This guideline is absolutely correct: Can lead to use of Response Surface design More accurate modeling

31 Don t Forget About Center Points 31 Practical Problem: Static Weight Study Analysis: DOE analyzed without responses affected by 4 th phase Curvature was significant in 18 of 19 responses Analysis of Variance for A-Score (coded units) Source DF Seq SS Adj SS Adj MS F P Main Effects Way Interactions Way Interactions Curvature Residual Error Pure Error Total Typical Result Indicating Significance of Curvature

32 Don t Forget About Center Points 32 Practical Problem: Static Weight Study Analysis: Central Composite Design Reduced static weight levels: o Attempt elimination of 4 th phase o Widen specification, not elimination

33 Don t Forget About Center Points 33 Practical Problem: Static Weight Study Results: Non-linear effects confirmed 4 th phase of ball motion still present; direction and occurrence not predictable Effects of static weights within current specifications insignificant Static weight specification not changed Pareto of Effects Order of Effects and Non-Linearity Contour Plot Static Weights Within Specs Are Insignificant

34 Presentation Summary 34 Topic Get Creative with Your Response The Response Needn t Be Continuous Data Interactions Matter Expand the Range of Your Factors Don t Forget About Center Points Lessons Learned 1. Think beyond mean and standard deviation 1. Leverage replicates 2. Convert attribute data 1. Select proper design resolution 2. Improve process optimization 1. Discover what happens outside the typical inference space 1. Check linearity assumptions 2. Achieve greater knowledge with response surface methods

35 35 Leveraging Designed Experiments for Success Questions & Discussion