Download Assessing Model Term Significance: t-Test and ANOVA and more Study notes Statistics in PDF only on Docsity!
Using the t-Test for Model Coefficient Significance
In addition to the ANOVA methods covered in weeks 1 and 2, we can also use the t-test to assess the significance of our model terms or coefficients.
Assume a 2-input full factorial model of the form:
A hypothesis to test the significance of the regressors would be:
Since each of these coefficients is a linear sum (or average) of normally distributed random variables (the data), we can use a t-test on the hypothesis that the mean value of each of the coefficients is in fact zero.
For this test the t statistic is defined as
𝛽𝑖 𝑠/�𝑛 (^) 𝛽 where n is the number of degrees of freedom used to estimate. Since we use all of the tests
to do this,
NB: This ratio is called the “standard error” and we can write our statistic as :
The sample variance s^2 is the same as the mean square error for the ANOVA analysis, since our fixed effects model assumes a uniform variance. This leads to:
We can then find p-values for each term and or test these values against a critical value based on
significance α: t 1−α,ν where in our case ν is the d.o.f. used to find the MS E. ( ν = 2k^ (n-1) as per
slide 70.)
Example : 2^2 Problem – Regression Model and Coefficient Significance
Consider the following full factorial experiments with 3 replicates. (k=2, n=3)
Forming the X matrix for the regression (using the group sample means):
X=
and the output vector y =
From this we can get the regression model:.
After entering X and y into MATLAB, the solution is found with the command line:
beta=inv(X)*y:
and our regression model is
We will test the significance it three ways:
- t-test on the model coefficients
- ANOVA using contrasts
- ANOVA using the regression coefficients
ANOVA Test of Model Significance – Using Contrasts
The ANOVA table for using contrasts (as per slide 70, week 2) is shown to the right.
We can fill in the table if we find the contrasts for our problem; since we already have MS (^) E
To find the contrasts, we take the rows of our X matrix and associate them with the “standard” responses:
A B AB
a 30.860 1 1 - 1 - 1 b 30.340 1 - 1 1 - 1 ab 30.702 1 1 1 1
Multiplying the A, B and AB columns by the response column give the contrasts:
contrast A 0.
contrast B - 0.
contrast AB 0.
and the effects and corresponding regression coefficients are given by
A = 0.
B = - 0.
AB = 0.
The contrast-based ANOVA table is then:
SS MS F P-value A 0.0019 0.002 1.014 0. B 0.0657 0.066 34.603 0. AB 0.0273 0.027 14.358 0. Error 0.
Note that the P-values found here are identical to those from the t-Test approach.
ANOVA Test of Model Significance – Using Regression Coefficients
Finally, we can use the ANOVA table from slide 71 of week 2, and demonstrate a 3rd^ way of testing significance.
Given:
MS (^) E= 0.0019 and n=3 ; a =
we can quickly fill in the table to find:
SS d.o.f MS F p value SS0 1260.184 1 1260.2 663604 0 ss1 0.0019 1 0.002 1.019 0. ss2 0.0657 1 0.066 34.603 0. ss12 0.0273 1 0.027 14.378 0. error 8 0.
and confirm that the results are again the same.
