Download Quantitative Factor and Orthogonal Polynomials | ISYE 6413 and more Exams Systems Engineering in PDF only on Docsity!
2.3. QUANTITATIVE FACTORS AND ORTHOGONAL POLYNOMIALS 53
This method is known to be generally the most effective among conserva- tive methods for the one-way ANOVA, that is, its Type II error is generally the smallest (or equivalently, its confidence bound is the tightest). It is recom- mended unless the critical value qk,N −k,α is not available. For the balanced one-way layout (i.e., ni = n), the experimentwise error rate for the Tukey method is exactly α. To prove this, note that
Prob (at least one pair is declared significantly different |H 0 )
= P rob
max i,j
|¯yi· − y¯j·| ˆσ
(1/n + 1/n)
qk,N −k,α|H 0
= P rob
max ¯yi· − min ¯yi· σ ˆ
1 /n
qk,N −k,α|H 0
= α. (2.27)
The last equality in (2.27) holds because under H 0 √ n(max ¯yi· − min ¯yi·)/σˆ
is the Studentized range statistic with parameters k and N − k. By solving |(¯yj· − y¯i·) − (τj − τi)| σ ˆ
1 /nj + 1/ni
qk,N −k,α
for τj − τi, the simultaneous confidence intervals for τj − τi are
¯yj· − y¯i· ±
qk,N −k,α ˆσ
nj
ni
for all i and j pairs. Since the Bonferroni method is conservative, the simulta- neous confidence intervals based on the Tukey method are shorter. For the pulp experiment, according to (2.26) at α = 0.05, 1 √ 2
qk,N −k, 0. 05 =
q 4 , 16 , 0. 05 =
By comparing 2.86 with the t values in Table 2.4, the Tukey method also iden- tifies that operators B and D are different. The 2.86 used here is smaller than the 3.008 of the Bonferroni method because the Bonferroni method is more con- servative. For multiple comparisons at the 0.05 level, the two methods reach the same conclusion, but the simultaneous confidence intervals for the Tukey method are shorter.
2.3 Quantitative Factors and Orthogonal Poly-
nomials
Mazumdar and Hoa (1995) performed an experiment which dealt with the laser- assisted manufacturing of a thermoplastic composite. The experimental factor
54 CHAPTER 2. EXPERIMENTS WITH A SINGLE FACTOR
Table 2.5: Strength Data, Composite Experiment Laser Power 40 W 50 W 60 W 25.66 29.15 35. 28.00 35.09 39. 20.65 29.79 35.
Table 2.6: ANOVA Table, Composite Experiment Degrees of Sum of Mean Source Freedom Squares Squares F laser 2 224.184 112.092 11. residual 6 59.422 9. total 8 283.
is laser power at 40, 50, and 60 watts. The response is interply bond strength of the composite as measured by a short-beam-shear test. The strength data for the composite experiment are displayed in Table 2.5. By treating the experimental design as a one-way layout, the ANOVA table for the experiment is computed and given in Table 2.6. The p value for the test of significance for the laser factor P rob(F 2 , 6 > 11 .32) is 0.009, where 11.32 =
- 09 / 9 .90 is the observed F statistic value Fobs. Thus, the experiment provides strong evidence that laser power affects the strength of the composite. Because the laser power factor has three levels in the experiment, its significance can be further studied by decomposing the sum of squares for the laser factor (with two degrees of freedom) into a linear component and a quadratic component. Suppose that a factor is quantitative and has three levels with evenly spaced values. For example, laser power in the composite experiment has evenly spaced levels 40, 50, and 60. Then, one can investigate whether the relationship between the factor and response is linear or quadratic over the three levels. Denote the response value at the low, medium, and high levels by y 1 , y 2 , and y 3 , respectively. Then the linear relationship can be evaluated using
y 3 − y 1 = − 1 y 1 + 0y 2 + 1y 3 ,
which is called the linear contrast. To define a quadratic effect, one can use the following argument. If the relationship is linear, then y 3 − y 2 and y 2 − y 1 should approximately be the same, i.e., (y 3 − y 2 ) − (y 2 − y 1 ) = 1y 1 − 2 y 2 + 1y 3 should be close to zero. Otherwise, it should be large. Therefore, the quadratic contrast y 1 − 2 y 2 + y 3
can be used to investigate a quadratic relationship. The linear and quadratic contrasts can be written as (− 1 , 0 , 1)(y 1 , y 2 , y 3 )T^ and (1, − 2 , 1)(y 1 , y 2 , y 3 )T^. The coefficient vectors (− 1 , 0 , 1) and (1, − 2 , 1) are called the linear contrast vector and the quadratic contrast vector. Two vectors u = (ui)l 1 and v = (vi)l 1 are said to be orthogonal if their cross product uvT^ =
∑l i=1 uivi^ = 0.^ It is easy to
56 CHAPTER 2. EXPERIMENTS WITH A SINGLE FACTOR
Table 2.7: Tests for Polynomial Effects, Composite Experiment Standard Effect Estimate Error t p value linear 8.636 1.817 4.75 0. quadratic − 0. 381 1.817 − 0. 21 0.
To see whether laser power has a linear and/or quadratic effect on strength, the linear model with linear and quadratic contrasts [i.e., (− 1 , 0 , 1)/
2 for lin- ear, (1, − 2 , 1)/
6 for quadratic] can be fitted and their effects tested for signif- icance. Thus, for the complete data set,
y = (25. 66 , 29. 15 , 35. 73 , 28. 00 , 35. 09 , 39. 56 , 20. 65 , 29. 79 , 35 .66)T
and the model matrix X is
X =
whose first, second and third columns need to be divided by 1,
2 and
6, re- spectively. The formulas in (1.42) and (1.48) are used to calculate the estimates and standard errors, which are given in Table 2.7 along with the corresponding t statistics. (The least squares estimate for the intercept is 31.0322.) The results in Table 2.7 indicate that laser power has a strong linear effect but no quadratic effect on composite strength. While the ANOVA in Table 2.6 indicates that laser power has a significant effect on composite strength, the additional analy- sis in Table 2.7 identifies the linear effect of laser power as the main contributor to the significance. Suppose that the investigator of the composite experiment would like to predict the composite strength for other settings of the laser power, such as 55 or 62 watts. In order to answer this question, we need to extend the notion of orthogonal contrast vectors to orthogonal polynomials. Denote the three evenly spaced levels by m − ∆, m, and m + ∆, where m denotes the middle level and ∆ the distance between consecutive levels. Then define the first-and second-degree polynomials
P 1 (x) = x − m ∆
P 2 (x) = 3
[(
x − m ∆
]
2.3. QUANTITATIVE FACTORS AND ORTHOGONAL POLYNOMIALS 57
It is easy to verify that P 1 (x) = − 1 , 0 , 1 and P 2 (x) = 1, − 2 , 1 for x equal to m − ∆, m, and m + ∆, respectively. These two polynomials are more general than the linear and quadratic contrast vectors because they are defined for a whole range of the quantitative factor and give the same values as the contrast vectors at the three levels of the factor where the experiment is conducted. Based on P 1 and P 2 , we can use the following model for predicting the y value at any x value in the range,
y = β 0 + β 1 P 1 (x)/
2 + β 2 P 2 (x)/
where
2 and
6 are the scaling constants used for the linear and quadratic contrast vectors and ≤ are independent N (0, σ^2 ). Because the y values are observed in the experiment only at x = m − ∆, m and m + ∆, the least squares estimates of β 0 , β 1 , and β 2 in (2.34) are the same as those using regression analysis with the X matrix in (2.31). For the composite experiment, fitting model (2.34) including the quadratic effect (that was not significant) leads to the prediction model:
Predicted strength = 31.0322 + 8. 636 P 1 (laser power)/
− 0. 381 P 2 (laser power)/
where the estimated coefficients 8.636 and − 0 .381 are the same as in Table 2.7. The model in (2.35) can be used to predict the strength for laser power settings in [40, 60] and its immediate vicinity. For example, at a laser power of 55 watts,
P 1 (55) =
P 1 (55) =
P 2 (55) = 3
[(
]
and 1 √ 6
P 2 (55) =
where m = 50 and ∆ = 10. Therefore at 55 watts
Predicted strength = 31.0322 + 8.636(0.3536) − 0 .381(− 0 .5103) = 34. 2803.
Using (2.35) to extrapolate far outside the experimental region [40, 60], where the model may no longer hold, should be avoided. The confidence interval and prediction interval for the predicted strength at 55 watts can be obtained in the same way as in the general regression setting of Section 1.6. Details will be left as an exercise. For k evenly spaced levels, orthogonal polynomials of degree 1,... , k − 1 can
2.5. ONE-WAY RANDOM EFFECTS MODEL 59
effects and model (2.1) is called a one-way fixed effects model. Consider now a situation, where there is a large pool (population) of operators. Assume that the four operators A, B, C and D are selected at random from this population to achieve the same objective as before, that is, to determine whether there are differences between operators in making the handsheets and reading their brightness. Now the interest no longer lies in the performances of the four specific operators, rather in the variation among all operators in the popula- tion. Because the observed data are from operators randomly selected from the population, the treatment (operator) effects are referred to as random effects. Thus, we arrive at a one-way random effects model
yij = η + τi + ≤ij , i = 1,... , k; j = 1,... , ni, (2.36)
where yij is the jth observation with treatment i, τi is the random effect as- sociated with the ith treatment effect such that the τi’s are independent and identically distributed as N (0, σ^2 τ ), the errors ≤ij are independent N (0, σ^2 ), τi and ≤ij are independent, k is the number of treatments, and ni is the number of observations with treatment i. Unlike the fixed effects model in (2.1), the cur- rent model has two error components τi and ≤ij to represent two sources of error. The corresponding variances σ τ^2 and σ^2 are also called variance components and the model (2.36) is also referred to as a one-way components of variance model. Unlike the fixed effects model, one need not estimate the τi in a random effects model, since the interest now lies in the entire population and making inference about the parameter σ^2 τ. Therefore, the null hypothesis (2.8) would now be replaced by the new null hypothesis
H 0 : σ^2 τ = 0. (2.37)
Note that, equations (2.4) and (2.5) and the decomposition of the corrected total sum of squares into two components SST r and SSE as given by (2.6) still holds good for a random effects model. The respective mean squares M ST r and M SE can be obtained similarly by dividing SST r and SSE by their respective degrees of freedom k − 1 and N − k. It can easily be verified that SSE/σ^2 ∼ χ^2 N −k. Further, if H 0 holds good, SST r/σ^2 ∼ χ^2 k− 1. Thus, under the null hypothesis σ τ^2 = 0, the test statistic F given by
F =
M ST r M SE
∑k i=1 ni(¯yi·^ −^ ¯y··) (^2) /(k − 1) ∑k i=
∑ni j=1(yij^ −^ ¯yi·) (^2) /(N − k)
follows an F distribution with parameters k − 1 and N − k. As before, the higher the computed value of F , the more inclined one would be to reject the null hypothesis. Thus, for the pulp experiment, the conclusions from the F test remains the same, and the ANOVA table is exactly the same as Table 2.3. The null hypothesis H 0 : σ^2 τ = 0 is thus rejected in favor of the alternative hypothesis H 1 : σ τ^2 > 0, and we conclude that there is a significant difference among the operators’ performances.
60 CHAPTER 2. EXPERIMENTS WITH A SINGLE FACTOR
Unlike the fixed effects model, the next step after rejection of H 0 will not be a multiple comparisons test, but estimation of the parameters σ^2 and σ τ^2. To perform this estimation, we need to derive the expected values of M ST r and M SE. The expected mean squares can also be used for sample size determina- tion as was done in Section 2.4. As in the case of a fixed effects model (**, refer to Section 2.4),
E(M SE) = σ^2 , (2.39)
and thus MSE can be taken to be an unbiased estimator of σ^2. To derive the expectation of M ST r, let us assume the simple case of equal replicates, where ni = n for all i = 1, 2 ,... , k. Since ¯yi. − y¯.. =
τi − ¯τ.
¯≤i. − ¯≤..
it follows that
SST r =
∑^ k
i=
n
y ¯i. − ¯y..
= n
{ (^) ∑k
i=
τi − ¯τ.
∑^ k
i=
¯≤i. − ¯≤.
∑^ k
i=
¯≤i. − ¯≤.
τi − ¯τ.
The cross product term has mean 0 (because τ and ≤ are independent). It can be shown that
E
{ (^) k ∑
i=
τi − ¯τ.
= (k − 1)σ τ^2 and E
{ (^) k ∑
i=
¯≤i. − ¯≤.
(k − 1)σ^2 n
Therefore
E(SST r) = n(k − 1)σ τ^2 + (k − 1)σ^2 ,
E(M ST r) = E
SST r k − 1
= σ^2 + nσ^2 τ. (2.40)
Thus, if σ^2 τ = 0, E(M ST r) and E(M SE) are the same, and the F ratio M ST r/M SE increases as the true value of σ^2 τ becomes larger. This also pro- vides an intuitive explanation of the hypothesis testing decision rule (i.e., reject H 0 when F is large) described earlier. From (2.39) and (2.40), an estimator of σ^2 τ is therefore given by
σˆ τ^2 = M ST r − M SE n
If the ni’s are unequal, n in (2.40) and (2.41) should be replaced by
n′^ =
k − 1
[ (^) k ∑
i=
ni −
∑k i=1 n 2 i ∑k i=1 ni
]
62 CHAPTER 2. EXPERIMENTS WITH A SINGLE FACTOR
Comparing this with (2.44) and recalling that M SE and ˆσ τ^2 are unbiased esti- mators of σ^2 and σ^2 τ respectively, an unbiased estimator of var(ˆη) is obtained as M ST r/nk. A 100(1 − α)% confidence interval for η is thus given by
ˆη ± tk− 1 , α 2
M ST r nk
In the pulp experiment, n = 5, k = 4, ηˆ = 60.40, M ST r = 0.447, and the 95% confidence interval for η is
5 × 4
2.6 Residual Analysis: Assessment of Model As-
sumptions
Before making inferences using hypothesis testing and confidence intervals, it is important to assess the model assumptions:
(i) Have all important effects been captured?
(ii) Are the errors independent and normally distributed?
(iii) Do the errors have constant (the same) variance?
We can assess these assumptions graphically by looking at the residuals
ri = yi − yˆi, i = 1,... , N,
where ˆyi = xTi βˆ is the fitted (or predicted) response at xi and xi is the ith row
of the matrix X in (1.36). Writing r = (ri)Ni=1, y = (yi)Ni=1, yˆ = (ˆyi)Ni=1 = Xβˆ, we have r = y − ˆy = y − Xβˆ. (2.45)
In the decomposition, y = ˆy + r, yˆ represents information about the assumed model, and r can reveal information about possible model violations. Based on the model assumptions it can be shown (the proofs are left as two exercises) that the residuals have the following properties:
(a) E(r) = 0 ,
(b) r and ˆy are independent, and
(c) r ∼ M N ( 0 , σ^2 (I − H)), where I is the N × N identity matrix and
H = X(XT^ X)−^1 XT
is the so-called hat matrix since ˆy = Hy, i.e., H puts the hat ∧^ on y.
