Advanced DOE for Improving a High Performance Racecar Scott Kowalski October 25, 2007 Minitab Inc.
Design of Experiments (DOE) Basic idea is to simultaneously study the impact of several factors on the response(s) of interest Sequential Approach from screening to optimization Interactions among factors are important Surfaces can be linear or quadratic
Guidelines for DOE State the problem and clearly define the objectives of the study Choose the factors to be studied and their levels Determine the responses and how to measure them Determine the appropriate experimental design
Guidelines for DOE Execute the design Statistically analyze the data Verify results using confirmatory runs Make recommendations
Racecar Experiment Introduction Wind tunnel experiment to characterize aerodynamic performance and develop improvements Typical factors are vehicle attitude, ride height, yaw angle, and vehicle geometry Responses include lift (downforce), drag, and lift to drag ratio Goal is often to minimize drag while maintaining a specified level of downforce
Racecar Experiment Introduction The experiment was performed in the wind tunnel at Langley Air-force Base (also used extensively for aircrafts) Four factors were considered • • • •
Front end height Rear end height Yaw angle Grill cover
Racecar
CL-rear CD
CL-front
Yaw
Grille with tape
Racecar Experiment Introduction They were concerned about possible curvature in the response A previous experiment had been performed (36 runs) • Replicated 24 design with 4 center points
Problems: • When either the front end or rear end height level is changed, then the other end also changes (needs adjusting) • Only way to change height is by shutting down the wind tunnel (30-45 minutes to get back to equilibrium ) • Other factors have easy to change levels (3 minutes) • (30 hours to complete the experiment)
Racecar Experiment Introduction Can we use some other advanced DOE tools to reduce the total time of the experiment? Can we collect more overall data using an advanced tool? How does using the advanced tool affect the analysis of the data? Answer to all 3 is YES!!!
Transition We will come back to the racecar example later We need to cover some general information about the advanced design that we will use
Treatments Factors have different levels used in the experiment Treatments: the combination of the factor levels used in the experiment Example: Temperature 100, 200 Material A, B, C • Treatments are • 100, A
200, A
100, B
200, B
100,C
200,C
DOE Basics Three Principles of DOE • Randomization—randomly assign the treatments to the units of interest • Replication--assign the treatments to more than one unit • Local Control—control for known sources of variation through blocking
We will focus on all three of these in some fashion
DOE Units There are two types of units in a DOE • Experimental Unit: the smallest unit to which a treatment can be applied independently of all other treatments • Observational Unit: the unit we take measurements on
Most times these units are the same It is important that we understand that experimental error comes from variation in running the same treatment on more than one experimental unit
DOE Units Consider an example of spraying pesticides on trees: two brands and two amounts with 2 replicates
DOE Units We spray the trees, then take several leaves from each tree and count the number of bugs
Thus we have two types of units • Experimental Unit: TREE Replicate • Observational Unit: LEAF Repeat
Randomization Let’s talk about the randomization principle • Randomization is done to “average” out the effects of lurking variables • A fundamental philosophy---textbooks assume for almost all techniques that the design is randomized • Most software for DOE automatically randomizes the runs • Unfortunately, random run order often results in changes to factor settings after each run for many of the factors in the DOE • What should be done then if, one or more of these factors cannot be easily or quickly changed?
Types of Factors Actually it is common in industry to have one or more factors that are not easily randomized Examples include temperatures, pressures, prototype factors and change over factors These factors are often called Hard-to-Change (HTC) factors while the rest of the factors in the design are referred to as Easy-to-Change (ETC) factors Many people ignore the impact of the HTC factors
Split-Plot Design Split-plot design and it originated in agriculture
I1
I2 I3 I2 I1 I3
F1 F2 F3 F1 F4 F3
Experimental Units:
F4 F1 F4 F3 F1 F2
Irrigation is Column
F2 F3 F2 F4 F2 F1 F3 F4 F1 F2 F3 F4
Fertilizer is Plot
Industrial Example 1 Printing Press Blanket (image carrier)
Blanket cylinder
Cylinder gap Paper
Impression cylinder
Industrial Example 2 Baking a cake • Oven Temperature, Egg Powder, Flour, Sugar • How do we conduct the experiment? • Mix up a cake with some level of Egg Powder, Flour, Sugar then bake it at a certain temperature • Notice this will take a long time to carry out
Another idea: fix the temperature at a level, then bake all the cakes involving Egg Powder, Flour and Sugar Then change the temperature and again bake all the cakes
Baking a Cake This looks something like High Temp Low Temp
The experimental unit for Temperature is the Oven The experimental unit for other factors is a Cake Cakes are observational units for Temperature
Baking a Cake We average the observational units (repeats) to get the response for the experimental unit Therefore to get the response for Temperature at High, we would average the 8 cakes involving the different combinations of Egg Powder, Flour, Sugar Hence, we only have 2 observations for Temperature: one at High and one at Low To get an estimate of error, we would need to run the Temperature at High twice and at Low twice
Baking a Cake In addition to two different experimental units, there is two randomizations
We randomly assign the Temperature to the oven, then randomly assign the cakes within the oven
Therefore, there are two error terms • One for testing Temperature • One for testing Egg Powder, Flour, Sugar and all the interactions
Baking a Cake The final design looks like High Temp Low Temp
Blocking Why is the design is not a block design? • A block is a collection of similar experimental units • Temperature is a factor applied to the experimental units • There is interest in the interactions with Temperature • The resulting design has two errors from two kinds of experimental units
However, we will be able to take advantage of the fact that it looks like a block in order to construct the design
Baking a Cake In Minitab, we construct the subplot design in “blocks� to get the right structure of the design Stat > DOE > Factorial > Create Factorial Design
Baking a Cake Fill in the ETC Factor Information
Baking a Cake It is common to rename blocks --- Rep To create the Temperature column • Calc > Make Patterned Data > Simple Set of Numbers
Baking a Cake The analysis involves a small trick to get the right error for Temperature since by default Minitab only has one error term Consider a simple One-Way ANOVA case The error is a nested term We use this knowledge to trick Minitab to get the correct error for Temperature
Baking a Cake Ignoring the two error terms • Use smaller error for Temp (Type I error) • Use larger error for other terms (Type II error) 1 Error
Incorrect
2 Errors
Correct
Temp
Signif.
Temp
Not
Egg
Signif.
Egg
Signif.
Flour
Signif.
Flour
Signif.
Sugar
Not
Sugar
Not
T*E
Not
T*E
Signif.
T*F
Signif.
T*F
Signif.
T*S
Not
T*S
Not
E*F
Not
E*F
Signif.
E*S
Not
E*S
Not
F*S
Not
F*S
Not
Back to the Racecar Experiment
CL-rear CD
CL-front
Yaw
Grille with tape
Racecar Experiment Recall that we have 4 factors • 2 HTC factors (front and rear heights) • 2 ETC factors (yaw and grill cover)
So a replicated 22 gives 8 runs for the heights (1 center point in the heights was included for a total of 9 runs) This means only changing the heights 9 times In each HTC run, 5 ETC combinations are carried out (22 plus one center run in Yaw and Grill Cover)
Racecar Experiment 1. randomly select the ride height factor levels of the car 2. at the factor levels from step 1, running all combinations of yaw and grille tape in random order 3. randomly selecting another ride height combination 4. again running all combinations of yaw angle and grille tape in random order 5. repeat the steps until all ride height combinations have been tested
Racecar Design Rep 1 Front RH − Rear RH −
Front RH − Rear RH +
Front RH + Rear RH −
Front RH + Rear RH +
+
+
+
+
Tape
Tape
Tape
Tape
−
−
−
− Yaw +
− Yaw +
− Yaw +
−
− Yaw +
Rep 2 Front RH − Rear RH −
− Yaw +
Front RH − Rear RH +
Front RH + Rear RH −
Front RH + Rear RH +
+
+
+
+
Tape
Tape
Tape
Tape
−
−
−
− Yaw +
− Yaw +
Front RH 0 Rear RH 0 + Tape − Yaw +
−
− Yaw +
−
Racecar Experiment This leads to a total of 45 runs But it only took about 10 hours to complete So more data, in about 1/3 of the time Wind tunnel time is very expensive so this was a huge savings Analysis is more complicated than Cake example
Racecar Experiment We have two error terms We also have center points But since we use ANOVA for the analysis, Minitab will think there are 3 distinct levels instead of 2 levels with center points We need to create a bunch of terms in the calculator (interactions and center points)
Racecar Analysis The analysis needs to be done in two stages • First is to do the HTC factor analysis using the means of the 5 ETC combinations from each of the 9 runs of the HTC factors • This gives the correct tests for the HTC factors • Second is to use a categorical factor with 5 levels (representing the 5 combinations of the HTC factors) to account for the correct SS and df from the HTC factors • Doing this gives along with the nested trick from earlier gives all the correct tests for the other terms
Racecar Coefficient Table 45-run split-plot Term
Coefficient
p-value
Constant
0.40117
FRH
0.00858
0.0000
RRH
0.00898
0.0000
FRHxRRH
0.00013
0.7247
yaw
-0.01167
0.0000
tape
-0.00494
0.0000
FRHxyaw
0.00047
0.0910
FRHxtape
-0.00016
0.5640
RRHxyaw
-0.00047
0.0910
RRHxtape
0.00078
0.0070
tapexyaw
-0.00056
0.0360
ssq
0.00058
0.0480
wsq
0.00031
0.5592
Summary DOE is a great tool for learning about and optimizing products/processes Many applications of DOE involve HTC factors Using a split-plot design saves time and money Analysis is more complicated MINITAB-16 will have 2-level split-plot designs
References Montgomery DC. Design and Analysis of Experiments, 6th ed., John J. Wiley & Sons, New York, 2004. Kowalski, S. M.; Parker, P. A.; and Vining, G. G. (2007). “Tutorial on Split-Plot Experiments”. Quality Engineering 19, pp. 1-16. Simpson, J. R.; Kowalski, S. M.; and Landman, D. (2004). “Experimentation With Randomization Restrictions: Targeting Practical Implementation”. Quality and Reliability Engineering International 20(5), pp. 481-495.
GRACIAS