Robust parameter design

A robust parameter design, introduced by Genichi Taguchi, is an experimental design used to exploit the interaction between control and uncontrollable noise variables by robustification -- finding the settings of the control factors that minimize response variation from uncontrollable factors.^[1] Control variables are variables of which the experimenter has full control. Noise variables lie on the other side of the spectrum, and while these variables may be easily controlled in an experimental setting, outside of the experimental world they are very hard, if not impossible, to control. Robust parameter designs use a naming convention similar to that of FFDs. A 2^{(m1+m2)-(p1-p2)} is a 2-level design where m1 is the number of control factors, m2 is the number of noise factors, p1 is the level of fractionation for control factors, and p2 is the level of fractionation for noise factors.

Effect Sparsity. Interactions can only significantly effect the response if at least one of the parent factors has an effect on the response.

Consider an RPD cake baking example from Montgomery (2005), where an experimenter wants to improve the quality of cake.^[2] While the cake manufacturer can control the amount of flour, amount of sugar, amount of baking powder, and coloring content of the cake; other factors are uncontrollable, such as oven temperature and bake time. The manufacturer can print instructions for a bake time of 20 minutes but in the real world has no control over consumer baking habits. Variations in the quality of the cake can arise from baking at 325^o instead of 350^o or from leaving the cake in the oven for a slightly too short or too long period of time. Robust parameter designs seek to minimize the effects of noise factors on quality. For this example, the manufacturer hopes to minimize the effects in fluctuation of bake time on cake quality, and in doing this the optimal settings for the control factors is required.

RPDs are primarily used in a simulation setting where uncontrollable noise variables are easily controlled. Whereas in the real world noise factors are hard to control, in an experimental setting control over these factors is easily maintained. For the cake-baking example, the experimenter can fluctuate bake time and oven temperature to understand the effects of such fluctuation that may occur when control is no longer in his hands.

Robust parameter designs very similar to fractional factorial designs (FFDs) in that the optimal design can be found using Hadamard matrices, principles of effect hierarchy and factor sparsity are maintained, and aliasing is present when full RPDs are fractionated. Much like FFDs, RPDs are screening designs and can provide a linear model of the system at hand. What is meant by effect hierarchy for FFDs is that higher-order interactions tend to have a negligible effect on the response.^[3] As stated in Carraway, main effects are most likely to have an effect on the response, then two-factor interactions, then three-factor interactions, and so on.^[4] The concept of effect sparsity is that not all factors will have an effect on the response. These principles are the foundation for fractionating Hadamard matrices. By fractionating, experimenters can form conclusions in fewer runs and with fewer resources. Oftentimes, RPDs are used at the early stages of an experiment. Because two-level RPDs assume linearity among factor effects, other methods may be used to model curvature after the number of factors has been reduced.

Construction of RPDs

Partial Design Pattern. Hadamard matrices can be normalized and fractionated to produce an experimental design.

Hadamard matrices are square matrices consisting of only + and -. If a Hadamard matrix is normalized and fractionated, a design pattern is obtained. However, not all designs are equal. This means that some designs are better than others, and specific design criteria is used to determine which design is best. After obtaining a design pattern, experimenters know to which setting each factor should be set. Each row in the pattern indicates a run, and each column indicates a factor. For the partial design pattern shown left, the experimenter has identified 7 factors that may have an effect on the response and hopes to gain insight as to which factors have an effect in 8 runs. In the first run, factors 1, 4, 5, and 6 are set to high levels while factors 2, 3, and 7 are set to low levels. Low levels and high levels are settings typically defined by the subject matter expert. These values are extremes but not so extreme that the response is pushed into non-smooth regions. After each run, results are obtained; and by fluctuating multiple factors in single runs instead of using the OFAT method, interactions between variables may be estimated as well as the individual factor effects. If two factors interact, then the effect one factor has on the response is different depending on the settings of another factor.

Fractionating Hadamard matrices appropriately is very time-consuming. Consider a 24-run design accommodating 6 factors. The number of Hadamard designs from each Hadamard matrix is 23 choose 6; that's 100,947 designs from each 24x24 Hadamard matrix. Since there are 60 Hadamard matrices of that size, the total number of designs to compare is 6,056,820. Fortunately, Leoppky, Bingham, and Sitter (2006) used complete search methodology and have listed the best RPDs for 12, 16, and 20 runs. Because complete search work is so exhaustive, the best designs for larger run sizes are often not readily available. In that case, other statistical methods may be used to fractionate a Hadamard matrix in such a way that allows only a tolerable amount of aliasing. Efficient algorithms such as forward selection and backward elimination have been produced for FFDs, but due to the complexity of aliasing introduced by distinguishing control and noise variables, these methods have not yet been proven effective for RPDs.^[5][6][7]

History and Design Criteria

To fully understand the design criteria, an understanding of history and fractional factorial designs is necessary. FFDs seek to understand which factors have an effect on a response and seek to optimize the response by finding the appropriate factor settings. Unlike RPDs, FFDs do not distinguish between control and noise variables.

Resolution and Minimum Aberration

In 2003, Bingham and Sitter^[8] defined maximum resolution and minimum aberration for 2-level fractional factorial designs. Resolution determines the worst amount of aliasing present, and aberration determines how much of that worst-case aliasing is present in the design. Resolution III designs alias main effects with two-factor interactions. Resolution IV designs alias main effects with three-factor interactions. Resolution V designs alias main effects with four-factor interactions. As the resolution increases, the level of aliasing becomes less serious because higher order interactions tend to have negligible effects on the response. Resolution measures regular designs; that is, effects are either fully aliased or not aliased at all. Consider the following statement, "Factor A is aliased with the two-factor interaction of factors BC." This means that if the two-factor interaction BC has an effect on the response, then the estimation of factor A's effect on the response is contaminated because factor A's effect cannot be distinguished from BC's effect. Clearly a resolution V design is preferred over a resolution IV design.

Designs of the same resolution are not always equal, and the knowledge of which type of aliasing is the worst involved is not enough to know which design is better. Instead further investigation of how much of the worst-case aliasing is needed. This idea is known as minimum aberration. Better designs contain the least amount of the worst-case aliasing. If designs D1 and D2 are both resolution V designs, but D1 has more instances of main effects aliased with 4-factor interactions, then D2 is the better design. D2 is the better design because there is a larger quantity of well-estimated effects.

Generalized Resolution and Generalized Minimum Aberration

Fontana, Pistone, and Rogantin ^[9] had created an indicator function for 2-level fractional factorial designs; and in 2003 Ye expanded the indicator function for regular and nonregular designs.^[10] In doing this, Ye established generalized resolution and generalized minimum aberration. Whereas regular designs are designs with run size equaling a power of two; nonregular designs can be any multiple of four. In nonregular designs, effects can be fully aliased, partially aliased, or not aliased at all. Generalized minimum aberration and generalized resolution take this partial aliasing into account.

Formally, Ye(2003) distinguishes between regular and nonregular designs and states that any polynomial function can be written as

F(x)= ∑_JϵPb_J X_J (x)= ∑_J∈PC∑_K∈PNb_J∪K X_J∪K (x), where b_L = 1 / 2^m ∑_x∈FX_L (x) and b₀ = n ⁄ 2^m.

If |b_J∪K ⁄ b₀ | = 1 then the design is regular; otherwise partial aliasing exists.

Priority of Effects adapted from Leoppky, Bingham, and Sitter (2006). RPDs must protect the estimation of top priority effects.

While Ye developed this indicator function, Bingham and Sitter were working on clarification of resolution and aberration for robust parameter designs. In 2006, Leoppky, Bingham, and Sitter published the extended word length pattern and indicator function for robust parameter designs. Because RPDs are concerned about minimizing process variation due to noise factors, the priority of effects changes from the hierarchy of effects of FFDs. Main effects are still the first priority, and two-factor interactions are still the second priority; but if any interactions have a control-by-noise (CN) interaction, then that interaction is increased by 0.5 on the priority scale. For example, a CCN three-factor interaction would be a priority 3 in a FFD because three-factor interactions are the third priority, two-factor interactions are the second priority, and main effects are the first priority. However, since RPDs are concerned about noise variables, the CCN interaction is a priority 2.5 effect. The CN interaction bumps the priority up by 0.5; so the traditional priority 3 minus the 0.5 for the CN interaction results in a 2.5 priority. A full table of priorities can be found in Leoppky, Bingham, and Sitter (2006).^[11]

Design Comparison

Further investigation of the principles introduced will provide a deeper understanding of design comparison.

For regular fractional factorial designs, the word length will determine what types of aliasing are present. For example, the word "2367" can be broken into aliasing structures as follows:

Aliasing Structure	Meaning of Aliasing Structure
2=367	The estimation of the effect of factor 2 on the response is aliased with the three-factor interaction of factors 3, 6, and 7.
3=267	The estimation of the effect of factor 3 on the response is aliased with the three-factor interaction of factors 2, 6, and 7.
6=237	If the three-factor interaction of factors 2, 3, and 7 have an effect on the response, the estimation of factor 6 on the response is contaminated.
7=236	No distinction can be made from the effect of factor 7 and the effect of the three-factor interaction 236.
23=67	Two factor interactions cannot be estimated accurately because they are aliased with other two-factor interactions.

The word 2367 is of length 4, and the worst-case aliasing is that main effects are aliased with three-factor interactions, and two-factor interactions are aliased with other two-factor interactions.

Word lengths become less simplistic when talking about RPDs due to the fact that the priority of effects has changed. Consider the word 23578 where factors 2, 3, and 5 are control variables and factors 7 and 8 are noise variables. The following aliasing strings can be derived from this word:

2=3578, 3=2578 5=2378 or C=CCNN
7=2358, 8=2357 or N=CCCN
23=578, 25=378, 35=278 or CC=CNN
27=358 and 28=357 or CN=CCN
235=78 or CCC=NN

Now that we can see what types of aliasing occurs, we must use Leoppky, Bingham, and Sitter's priority of effects to determine the worst amount of aliasing present. This means that any CN interaction bumps that priority up by 0.5; and the word length is obtained by summing each side of the aliasing string. The table below finds the sums for each aliasing type found in the word 23578.

Priority(C) = 1	Priority(CCNN) = 3	Sum = 4
Priority(N) = 1	Priority(CCCN) = 3.5	Sum = 4.5
Priority(CC) = 2	Priority(CNN) = 2.5	Sum = 4.5
Priority(CN) = 1.5	Priority(CCN) = 2.5	Sum = 4
Priority(CCC) = 3	Priority(NN) = 2	Sum = 5

Since lower sums indicate worse aliasing, this word has the worst-case aliasing of length 4. It is important to understand that in an FFD the differentiation between control and noise would not be taken into account, and this word would be of length 5; but RPDs are concerned with this distinction and even though the word appears to be length 5, design criteria determines priority 4. Now, assume design D1 contains only the word just analyzed (23578). If D1 was compared to D2, and the worst-case aliasing found in D2 was priority 3.5, then D1 would be the better design. If, however, the worst-case aliasing of D2 was priority 4, then minimum aberration must be taken into consideration. For each design, we would calculate the frequencies of each type of worst-case aliasing. The better design would be chosen as the design that minimizes the occurrence of worst-case aliasing. These frequencies can be organized using the extended word length pattern (EWLP).

Notation

The notion of minimum aberration can be understood from the definition provided in Leoppky, Bingham, and Sitter (2006):

For any two 2^{(m1+m2 )-(p1+p2)} fractional factorial robust parameter designs, D1 and D2, we say that D1 has less aberration than D2 if there exists an r such that, B_i (D1) = B_i (D2) for all i < r – 1 and B_r (D1) < B_r (D2). If no other design has less aberration than D1, then D1 is the minimum aberration fractional factorial robust parameter design.

Leoppky, Bingham, and Sitter (2006) also provide the RPD indicator function as:

For a given design, D, and a run, x∈D, define a contrast X_L(x) = ∏_l∈Lx_l on D, where L ∈ P and P is the set of all subsets of {1, 2, …, m}. Further, define P_C to be the set of all subsets of {1, 2,…, m} and P_N to be the set of all subset of {1, 2, …, m}, where an element of P is of the form L ≡ J ∪ K where J ∈ P_C and K ∈ P_N.

Extended Word Length Pattern

Bingham and Sitter (2006) generate the EWLP by providing the following concept:

Let F be a robust parameter design with indicator function F(x)= ∑_J∈PC∑_K∈PNb_J∪K X_J∪K (x), if b_J∪K≠ 0, then X_J∪K is a word of the design F with word length r + (1- |b_J∪K ⁄ b₀ |) / 2, where |b_J∪K ⁄ b₀ | is a measure of the degree of confounding for the word X_J∪K. Further let g_{r+l / 2t} be the number of words of length (r+l / 2t), where r = 2.0, 2.5, 3.0, … according to Table 2.1. Thus, the robust parameter design extended word length pattern is (g_2.0,…,g_{2.0+((t-1)) ⁄ 2t} ,…,g_m-1,…,g_{m+(t-1) ⁄ 2t}).

Consider designs D1 and D2 with the following EWLPs:

D1: [(0 0 3)(2 3 1)(2 5 5)]

D2: [(0 0 3)(2 4 0)(2 4 6)]

We can read an EWLP from left to right since the left side indicates the most serious level of aliasing, and the aliasing becomes less serious as we move to the right. D2 is the better design because there is one more occurrence of more serious aliasing than in D1.

Uses and Examples

Design of Experiments (DOE) is a fundamental part of experimentation, modeling, and simulation. Banks ^[12] states, "Experimental design is concerned with reducing the time and effort associated with simulating by identifying the information needed to be gathered from each simulation replication, how many replications need to be made, and what model parameter changes need to be compared." After a conceptual model has been implemented as a programmed model, DOE is necessary to perform experimentation and obtain simulation results in the most timely and cost-efficient manner. The following examples demonstrate situations where RPDs can be used to draw significant conclusions.

Example 1

Consider the permanent marker manufacturing example adapted from Brewer, Carraway, and Ingram (2010). The subject matter experts (SMEs) have recognized seven factors that may affect the quality of the marker: amount of ink, propanol content, butanol content, diaceton content, quality of container, humidity, and temperature. Amount of ink, propanol content, butanol content, diaceton content, and quality of container are determined by the manufacturer; humidity and temperature, while easily controlled in an experimental setting, cannot be controlled once the product has left the manufacturer's hands. Even if the manufacturer states to keep the marker temperature between 35 and 80 degrees Fahrenheit, consumers may be in 90 degree weather or take little note of the advice. This variation is uncontrollable and affects the consumers opinion of the product; therefore, the manufacturer wants the product to be robust to variations due to temperature.

To run every possible combination of factors would be 128 runs. However, by fractionating this matrix, the effects of factors can be seen in much fewer runs. Therefore, fractionating is less costly and less time consuming.

After the RPD has been created, the quality of permanent marker is tested at the end of each run. This is an example of live simulation because in order to test the quality of the marker, simulating the humidity and temperature of the real-world is necessary. The permanent marker manufacturing company opts to simulate high or low temperatures and humidity instead of traveling to specific locations where the marker may be used. The manufacturer saves time and money and gets close to the same effect as someone using the marker in extreme weather conditions or elsewhere.

Example 2

Imagine being hired as a store manager and wanting to increase efficiency of labor. You have noticed that the same number of people are staffed at all hours of the day, but the store is busier from noon until 3:30 pm and empty after 7:00 pm. You do not want to risk being understaffed, so you choose to simulate different scenarios to determine the best scheduling solution. Control factors that effect scheduling optimality may include number of people on a shift whereas uncontrollable factors may include weather and traffic flow.

A constructive model is implemented to understand the dilemma at hand, and an RPD is the method used to determine the settings of the control factors we need in order to minimize the effects of the noise factors. In other words, we can use an RPD to determine how many people are needed on each shift so that we are not understaffed or overstaffed regardless of the weather conditions or flow of traffic.

Analyzing RPDs

Because RPDs relate so closely to FFDs, the same analysis methods can be applied. ANOVA can be used to determine which factors are significant. Center points can be ran to determine if curvature is present. Many statistics software packages have split-plot designs stored and ready for analysis. RPDs are screening designs and are often used to reduce the number of factors that are thought to have an effect on the response.

References

1. Brewer, K., Carraway, L., and Ingram, D. (2010) "Forward Selection as a Candidate for Constructing Nonregular Robust Parameter Designs." Arkansas State University.
2. Montgomery, D. (2005), Design and Analysis of Experiments. 6th ed. Wiley.
3. Wu, C.F.J. and Hamada, M. (2000), Experiments: Planning, Analysis, and Parameter Design Optimization. Wiley.
4. Carraway, L. (2008). “Investigating the Use of Computational Algorithms for Constructing Non-Regular Robust Parameter Designs,” Masters Thesis, Arkansas State University.
5. Ingram, D. (2000), The construction of generalized minimum aberration designs by efficient algorithm. Dissertation, University of Memphis.
6. Ingram, D. and Tang, B. (2001), Efficient Computational Algorithms for Searching for Good Designs According to the Generalized Minimum Aberration Criterion, American Journal of Mathematical and Management Sciences, 21 325-344.
7. Ingram, D. And Tang, B. (2005), Construction of minimum G-aberration Designs via Efficient Computational Algorithms, Journal of Quality Technology, 37 101-114.
8. Bingham, D. and Sitter, R.R. (2003), Fractional Factorial Split-Plot Designs for Robust Parameter Experiments, Technometrics, 45 80-89.
9. Fontana, R. Pistone, G. and Rogantin, M.P. (2000), Classification of Two-Level Factorial Fractions, Journal of Statistical Planning and Inference, 87 149-172.
10. Ye, K.Q. (2003), Indicator Functions and Its Application in Two Level Factorial Designs, Annals of Statistics, 31 984-994.
11. Loeppky, J. L., Bingham, D. and Sitter R.R, (2006), Constructing Non-Regular Robust Parameter Designs, Journal of Statistical Planning and Inference, 136 3710-3729.
12. Banks (2010) C. M. Banks, “Introduction to Modeling and Simulation”, in J. A. Sokolowski and C. M. Banks (Editors), Modeling and Simulation Fundamentals: Theoretical Underpinnings and Practical Domains, John Wiley and Sons, Hoboken NJ, 2010.

Additional Readings

Box, G.E.P., (1988), Signal-to-Noise Ratios, Performance Criteria, and Transformations (with discussion), Technometrics, 30 1-40.

Box, G.E.P., Hunter, W.G., and Hunter, J.S. (1978), Statistics for Experimenters. Wiley.

Castillo, E. (2007), Process Optimization: A Statistical Approach. Springer.

Deng, L.Y. and Tang, B. (1999), Generalized Resolution and Minimum Aberration Criteria for Plackett-Burman and Other Non-regular Factorial Designs, Statisitca Sinica, 9 1071-1082.

Deng, L.Y. and Tang, B. (2002), Design Selection and Classification for Hadamard Matrices Using Generalized Minimum Aberration Criteria, Technometrics, 44 173-184.

Lawson, J. and Erjavec, J. (2001), Modern Statistics for Engineering and Quality Improvement. Duxbury.

Loeppky, J. (2004), Ranking Non-Regular Designs. Dissertation, Simon Fraser University.

Novosad, S. and Ingram, D. (2006), Optimal Non-regular Designs that Provide Alternative to the 16-Run and 32-Run Regular Fractional Factorial Designs. Arkansas State University, State University, AR.

Pistone, G. and Wynn, H.P. (1996), Generalized Confounding with Gröbner Bases, Biometrika, 83 653-666.

Taguchi, G. (1986), Introduction to Quality Engineering. New York: Quality Resources.

Tang, B. and Deng. L.Y. (1999), Minimum G2-aberration for Non-regular Fractional Factorial Designs, The Annals of Statistics, 27 1914-1926.

Wiley, A. and Ingram, D. (2007), Uncovering the Complex Aliasing Patterns of Some Non-regular Designs. Senior Honors Thesis, Arkansas State University, State University, AR.