Download Two Independent Samples t test and more Lecture notes Statistics in PDF only on Docsity!
Two Independent Samples t test
Overview of Tests Presented
Three tests are introduced below: (a) t-test with equal variances, (b) t-test with unequal variances, and (c) equal variance test. Generally one would follow these steps to determine which t-test to use:
- Perform equal variances test to assess homogeneity of variances between groups,
- if group variances are equal, then use t-test with equal variances,
- if group variances are not equal, then use t-test with unequal variances.
These notes begin with presentation of t-test with equal variances assumed, next is information about testing equality of variances, then presented is the t-test with unequal variances.
1. Purpose
The two independent samples t-test enables one to determine whether sample means for two groups differ more than would be expected by chance. The independent variable is qualitative with two categories and the dependent variable must be quantitative (ratio, interval, or sometimes ordinal).
Example 1: Is there a difference in mean systolic blood pressure between males and females in EDUR 8131?
(Note: IV = sex [male vs. female], DV = blood pressure.)
Example 2: Does intrinsic motivation differ between students who are given an opportunity to provide instructional feedback and students who not given an opportunity to provide instructional feedback?
(Note: IV = feedback opportunity [yes vs. no], DV = intrinsic motivation. Background: Two weeks into a semester an instructor asked students to provide written feedback on instruction with suggestions for improvement.)
2. Steps of Hypothesis Testing
Like with the one-sample t test, the two-sample t test follows the same steps for hypothesis testing:
a. Define both H 0 and H 1 b. Set alpha (α, probability of a Type 1 Error) c. Identify decision rule (either for α, test statistic, or confidence interval) d. Calculate the test statistic (t ratio) e. Find df (degrees of freedom), confidence intervals, and p-values f. Present inferential and interpretation of results (in APA style)
3. Hypotheses
(a) Example 1 Is there a difference in mean systolic blood pressure between males and females in EDUR 8131?
Null Written The mean systolic blood pressure for males and females in EDUR 8131 is equal.
Symbolic H 0 : 1 = 2 or H 0 : 1 – 2 = 0
Alternative Non-directional Written The mean systolic blood pressure for males and females in EDUR 8131 differ.
Symbolic H 1 : 1 ≠ 2 or H 1 : 1 – 2 ≠ 0
(b) Example 2 Does intrinsic motivation differ between students who are given an opportunity to provide instructional feedback and students who not given an opportunity to provide instructional feedback?
Null Written There is no difference in mean intrinsic motivation between those given the opportunity to provide feedback and those not given the opportunity to provide feedback.
Symbolic H 0 : 1 = 2 or H 0 : 1 – 2 = 0
Alternative Non-directional Written Mean intrinsic motivation differs between those given the opportunity to provide feedback and those not given the opportunity to provide feedback.
Symbolic H 1 : 1 ≠ 2 or H 1 : 1 – 2 ≠ 0
4. Decision Rules
Decision rules for the two-sample t test are listed below.
(a) p-values and
As noted for the one-sample t test, the probability of committing a Type 1 error, α, is normally set to .10, .05, or .01.
Decision rule for p-values (this decision rule holds for all statistical tests in which p-values are present) states
If p ≤ reject H 0 , otherwise fail to reject H 0
5. t-test with Equal Variances: t-ratio, Assumptions, SEd, and df
There are several ways to calculate the standard error of the mean difference, SEd , in a two-group t-test. One approach assumes groups have equality of variances or homogeneous variances (i.e., ), and the second approach relaxes this assumption and allows for unequal group variances. The second approach is presented later in these notes.
(a) t-ratio Formula
Like the one-sample t test, the two-sample t-test forms a ratio of mean differences divided by the standard error of that difference:
( ̅ ̅ ) ( ) ̅̅
Since the difference between population means is expected to be 0.00, the component 1 - 2 drops from the equation leaving the reduced equation for the t-ratio:
̅ ̅ ̅̅
̅ ̅
(Note: If the mean difference expected in the null hypothesis does not have to be 0.00 but could be any expected difference [e.g., H 0 : 1 − 2 = 3.00]. While it is rare that one hypothesizes a mean difference value other than 0.00, be aware that the component 1 - 2 would not drop from the equation when something other than 0.00 is expected.)
(b) Assumptions
For the independent samples t-test to provide accurate calculated t-ratios, p-values, and confidence intervals, three assumptions are required:
- Scores sampled from population distributions that are normally distributed,
- population scores are independent,
- and both groups have population variances that are homogeneous, or equal (i.e., ),.
It is also important, as with all statistical estimates, that samples not contain influential outliers, otherwise all parameter estimates and inferential statistics can be inaccurate.
The first assumption, normal distribution, tends to be of little importance with larger samples due to the central limit theorem, so violations of this assumption often have little impact on calculated t-ratios, p-values, or confidence intervals.
The second assumption is important; if scores are not independent then the “independent samples t-test” is likely the wrong test to use. Depending upon the nature of lack of independence, other statistical procedures may be more appropriate such as the correlated samples t-test. Violation of this assumption will also lead to inaccurately calculated standard errors, t-ratios, p-values, and confidence intervals.
Violations to the third assumption can also impact calculated standard errors, t-ratios, p-values, and confidence intervals. However, when sample sizes are equal or approximately equal, or when sample sizes are large (whether equal or not), then violations of this assumption typically have little impact. However, for small sample sizes unequal variance can greatly affect p-values, t-ratios, and confidence intervals.
The t-test can be corrected for violation to the homogeneous variance assumption. This correction is discussed below in the presentation of t-test for unequal variances. The next section, however, shows how to calculate a t-ratio when variances are assumed equal or approximately equal.
(c) Standard Error of Difference, SEd (Equal Variances Assumed)
In most cases the following formulas for SEd and df will be appropriate when sample sizes are equal or approximately equal, or when variances are equal or approximately equal. When equal or approximately equal variances exist between the two groups, one may take the weighted mean of variances and this mean is called the pooled estimate of the population variance and is symbolized by (where p = pooled). A weighted mean is an average that takes into account sample sizes for each group such that the larger group’s variance will influence more the pooled variance estimate. The pooled variance is found by this formula:
Using the pooled variance, , the standard error of the mean difference , or (^) ̅ ̅ , is calculated as follows:
̅ ̅^ √(
) ( ) √^ ( )
(d) Degrees of Freedom, df (Equal Variances Assumed)
When the pooled estimate of the is employed, one may calculate degrees of freedom as
df (or ) = n 1 + n 2 – 2
6. Critical Values (Equal Variances Assumed)
If the two groups have approximately equal variances, or if they have approximately equal sample sizes (since equal sample sizes negate any problems due to lack of equal variances), then df = n 1 + n 2 – 2.
Like for the one-sample t-test, critical values can be found in a table of critical t-values for identified α and df. A table of critical t-values can be found on the course web page.
Example for Finding df
(a) Compare systolic blood pressure between males and females; there are seven males and seven females, what would be the critical t-value for this analysis if α = .05? What if α = .01?
df = n 1 + n 2 – 2 = 7 + 7 – 2 = 12
α = .05; critical t = ± 2. 18 α = .01; critical t = ± 3.
(d) Find or (^) ̅ ̅ , standard error of the mean difference:
√ ( ) √^ ( ) √^ ( ) √ (^ )
(e) Find the t-ratio, t:
̅ ̅
(f) Find df and critical values for whichever significance level (α) is specified if using t-values for hypothesis testing:
df = n 1 + n 2 – 2 = 7 + 7 – 2 = 12
α = .05; critical t = ± 2. α = .01; critical t = ± 3.
(g) Apply decision rule
| | | |
| | | |
Since the calculated t-value of 2.385 is larger than the critical t-value of 2.18 reject Ho and conclude that males have higher blood pressure than females since males’ blood pressure mean is higher.
Results from SPSS are presented below.
Figure 1 SPSS Output for Two-group t-test of Blood Pressure by Sex
Questions
(a) Why does SPSS show a t-value of -2.38 while the calculated t-value above was positive, t= 2.38? Does this affect inference and interpretation?
Reason – simply which group is listed first in the t-ratio formula. No – no, inference will remain the same (e.g., same p-value), interpretation – be clear which group has the higher mean.
(b) What is the p-value specified by SPSS for this t-test and what would be the p-value decision rule?
p = .034 (see “Sig. (2-tailed)” in SPSS output.
If p ≤ reject H 0 , otherwise fail to reject H 0
If .034 ≤ .05 reject H 0 , otherwise fail to reject H 0 (p is less than alpha, so reject Ho)
(b) Example 2
Does intrinsic motivation differ between students who are given an opportunity to provide instructional feedback to their instructor and students who not given an opportunity to provide instructional feedback? Intrinsic motivation is measured as a mean composite of several items each of which is on a scale that ranges from 1 = low to 5 = high. For example students rate the degree to which the subject matter is of interest to them, how easy it is for them to pay attention to the subject matter, and their ability to stay focused when studying this content.
Raw data and descriptive statistics for intrinsic motivation are presented below in Table 2.
Table 2 Intrinsic Motivation by Feedback Opportunity Feedback Opportunity: Yes
Feedback Opportunity: No 4.2 4.6 1.1 1. 4.5 2.3 3. 4.3 3.8 4. 3.9 4.8 2. 4.3 4.2 2. 3.9 4.9 1. M = 4.243 M = 3. s = 0.270 s = 1. n = 7 n = 12
If calculating by hand, follow these steps:
(a) Determine mean and sample size for both groups.
(b) Find s^2 , variances, for both groups (use of standard deviation rather than variance is a common error):
Figure 2 SPSS Output for Two-group t-test of Intrinsic Motivation by Feedback Opportunity
Question
What is the p-value specified by SPSS for this t-test and what would be the p-value decision rule?
SPSS calls the p-value for the t-test “Sig. (2-tailed)” and it equals 0.052 in this example.
If p ≤ reject H 0 , otherwise fail to reject H 0
If .052 ≤. 05 reject H 0 , otherwise fail to reject H 0 (since .052 is larger than .05 fail to reject Ho)
8. Results of Heterogeneity of Variance (Unequal Variances between Groups)
The formula for the standard of the mean difference, or (^) ̅ ̅, presented above assumes that both groups have the same or similar variances. If the variances are largely different, then the standard error for the mean difference can be over- or underestimated and result in incorrect inferential decisions due to inaccurate t-ratios, p-values, and confidence intervals.
Consider, for example, the SPSS t-test results for intrinsic motivation presented in Figure 3. Note that the variances for the two groups are 0.073 and 1.996 – the ratio of these two variances is 1.996 / 0.073 = 27.3, so one variance is 27x larger than the second variance.
Figure 3 SPSS Output for Two-group t-test of Intrinsic Motivation by Feedback Opportunity
When group variances are so different, and when sample sizes are unequal, the inequality of variances can produce misleading inferential results. For example, note the t-values and p-values for the two rows of t-test results in Figure 3. The top row assumes equal variances and the bottom row adjusts for unequal variances. With α = .05, these results lead to different inferential conclusions: fail to reject Ho if equal variances assumed and reject Ho if equal variances are not assumed.
If sample sizes are unequal, or not very similar, and if group variances are largely different (i.e., one variance is about twice the size of the other), then one should test for homogeneity of variances and determine whether inferential results and confidence intervals differ between the two t-test results.
9. Assessing Homogeneity of Variances
A first step in conducting a t-test should include determining whether the assumption of homogeneity of variances (equal group variances) holds. Below are two methods for testing this assumption.
(a) Levene’s Test for Equality of Variances (SPSS Default)
Levene’s test assesses whether group variances differ more than would be expected by chance. The null and alternative hypotheses are
where subscripts a and b represent group A and B. In words, Ho indicates that the two samples come from populations with equal variances.
Example 1: Blood Pressure
Do males and females appear to have similar variances for blood pressure?
Figure 4 shows SPSS results for Levene’s test (see area highlighted in red).
Figure 5 SPSS Output for Two-group t-test of Intrinsic Motivation by Feedback Opportunity
Since the groups appear to have unequal variances, one should use a t-test that corrects for unequal variances.
SPSS provides this correction with results denoted as “Equal variances not assumed” – see section highlighted in blue in Figure 5.
Note that with the corrected t-test (i.e., unequal variances t-test) results the inferential statistics are t = 2.718 and p = 0.018. Given these results, one would reject H 0 : 1 = 2 and conclude that Intrinsic Motivation does appear to differ between Instructional Feedback groups. This inference differs from the inference provided above when equal group variances were assumed.
(b) F-max Test: Useful for Hand Calculation Assessment of Homogeneity of Variance
If homogeneity of variances is not automatically tested by software employed, then one may assess by using the F-max test. This simple test requires only group variances and sample sizes. F-max is based upon the F distribution which is positively skewed, cannot be less than zero, and takes a different shape depending upon the degrees of freedom available. Examples of F distributions are presented in Figure 6.
Figure 6 Examples of Several F Distributions for Varying Degrees of Freedom
The F-ratio for the F-max test is calculated as
where is the group with the largest variance and is the group with the smallest variance. With all F tests there are two different degrees of freedom,
( ) ( ) ( ) ( )
One uses df 1 and df 2 to find critical F values against which are compared calculated F ratios for the F-max test.
Critical F values are located in Figure 7. To find critical values one matches the column (df 1 ) and row (df 2 ) and selects the F value that results. If the calculated df does not exist, round down to the nearest tabled df value.
Examples (a) df 1 = 20 and df 2 = 17; the critical F value is 1.86. (b) df 1 = 9 and df 2 = 55; since df 2 = 55 is not tabled select 50 for df 2 ; the critical F value is 1.76. (c) df 1 = 29 and df 2 = 221; there is no df 1 = 29 tabled so select df 1 = 24; and for df 2 = 221 select df 2 = 100; critical F value is 1.46.
Probability
F Values
df=2,10 df=4, df=6,40 df=8,
F-Distribution with Different Degrees of Freedom
The variance for males is 20.514^2 = 420.824 and the variance for females is 22.267^2 = 495.82.
The female variance is larger so it will be the numerator in the F-max calculation and also df 1 = 7 – 1 = 6. The denominator degrees of freedom is df 2 = 7 – 1 = 6.
The critical F value, for =.10, with df = 6,6 is 3.05.
Since 1.18 is less than 3.05 the decision is fail to reject therefore we conclude that the two groups come from populations with similar variances, given the data collected.
If the variances are different then usually one would proceed with the t-test formula that corrects for heterogeneous variances.
Example 2
Do the two groups of students involved in feedback participation have similar intrinsic motivation variances?
Table 4 Intrinsic Motivation by Feedback Opportunity Feedback Opportunity: Yes
Feedback Opportunity: No M = 4.243 M = 3. s = 0.270 s = 1. n = 7 n = 12
The group that provided feedback had an intrinsic motivation variance of 0.073 (n = 7) and the group who did not participate in feedback displayed an intrinsic motivation variance of 1.996 (n = 12).
The no feedback group’s variance is larger so it will be the numerator in the F-max calculation and that group‘s df 1 = 12 – 1 = 11. The feedback group’s variance will be the denominator in the F test and their corresponding degrees of freedom is df 2 = 7 – 1 = 6.
The critical F value, for =.10, with df = 11,6 is 2.92.
Since 11.285 is larger than 2.92 one must reject H 0 and conclude the intrinsic motivation variances for these two groups differ therefore the unequal variance formulas, the un-pooled variances, for the two-group t test must be used.
10. t-test with Unequal Variances: t-ratio, SEd, and df
When variances are unequal, one uses the standard formula for the t-ratio:
̅ ̅ ̅̅
̅ ̅
The difference from the pooled variance t-test is how the standard error of the mean difference is calculated; when unequal variances exist one should use separate variances to determine the standard error. The standard error of the difference, using separate variances rather than the pooled variance, is found by
̅ ̅^ √
The t ratio that results from employing the above standard error of the difference does not follow a t-distribution so p- values will be incorrect. One approach to addressing this deficiency is to use a correction to the degrees of freedom. The following formula represents the Satterthwaite approximated, or corrected, degrees of freedom, , reported by most software:
This formula usually produces degrees of freedom that have decimal fractions (i.e., not a whole number). Critical t tables only provide critical values for whole numbered degrees of freedom. If one is using statistical software to calculate the standard error and dfc then the decimal fraction is not an issue since the software should handle this information correctly. However, if one wishes to find critical t values from a t table using dfc, then the decimal fraction is problematic because tables for critical t values assume df are whole numbers.
Recommendations for using dfc with critical t tables:
replace dfc with , whichever is smaller, and use this value as df for finding critical t values round down the dfc to the nearest whole number – this method is recommended – then find critical t values
11. Example t-tests for Unequal Variances in Excel and SPSS
(a) Example 1: Blood Pressure
Is there a difference in mean systolic blood pressure between males and females in EDUR 8131? Raw data, descriptive statistics, and t-test results for both groups are provided below.
The t-test with unequal variances is not needed for these data, nevertheless these data will be used as an example of how to calculate the t-test for unequal variances.
α = .05; critical t = ± 2. α = .01; critical t = ± 3. 11
(g) Apply decision rule
| | | |
| | | |
Since the calculated t-value of 2.385 is larger than the critical t-value of 2.20 reject Ho and conclude that males have higher blood pressure than females since males’ blood pressure mean is higher.
Results from SPSS are presented below.
Figure 8 SPSS Output for Two-group t-test of Blood Pressure by Sex
Questions
- Why is the t-value obtained by hand calculation positive (2.385) and the one produced by SPSS negative (-2.384)?
In SPSS’s t-test command one has the option of identifying which group should be listed first in the t-test formula; if the group with the higher mean is listed first then the mean difference and t-ratio will be positive; if the group with the lower mean is listed first then the mean difference and t-ratio will be negative.
- Recall that for blood pressure data we found that group variances were not different which means the t-test with equal variances can be used. This also means the results for both equal and unequal t-tests should be similar. Do you notice any differences in t-test results between the equal and unequal t-test results reported by SPSS in Figure 8?
(b) Example 2: Intrinsic Motivation
Does intrinsic motivation differ between students who are given an opportunity to provide instructional feedback to their instructor and students who not given an opportunity to provide instructional feedback?
Raw data and descriptive statistics for intrinsic motivation are presented below in Table 2.
Table 2 (reposted) Intrinsic Motivation by Feedback Opportunity Feedback Opportunity: Yes
Feedback Opportunity: No 4.2 4.6 1.1 1. 4.5 2.3 3. 4.3 3.8 4. 3.9 4.8 2. 4.3 4.2 2. 3.9 4.9 1. M = 4.243 M = 3. s = 0.270 s = 1. n = 7 n = 12
If calculating by hand, follow these steps:
(a) First, perform F-max or Levene’s test to assess homogeneity of variances. If is rejected then proceed with the steps below for the unequal variances t-test; if If is not rejected, the use the pooled-samples (equal variances) t-test steps and formulas discussed above in section 7.
SPSS: Levene’s F = 10.71; p = 0.004 ; since p is less than α = .10 reject Ho Excel: F-max F = 27.39; critical F = 2.92; reject since calculated F is larger than critical F
(b) Determine mean and sample size for both groups.
(c) Find s^2 , variances, for both groups (use of standard deviation rather than variance is a common error):
(d) Find or (^) ̅ ̅ , standard error of the mean difference:
√ √^ √
(e) Find the t-ratio, t:
