Download Understanding Dependent-Samples t Test: Assessing Mean Differences in Paired Observations and more Schemes and Mind Maps Statistics in PDF only on Docsity!
UNDERSTANDING THE DEPENDENT-SAMPLES t TEST
A dependent-samples t test (a.k.a. matched or paired-samples, matched-pairs, samples, or subjects, simple repeated-measures or within-groups, or correlated groups) assesses whether the mean difference between paired/matched observations is significantly different from zero. That is, the dependent-samples t test procedure evaluates whether there is a significant difference between the means of the two variables (test occasions or events). This design is also referred to as a correlated groups design because the participants in the groups are not independently assigned. The participants are either the same individuals tested (assessed) on two occasions or under two conditions on one measure, or there are two groups of participants that are matched (paired) on one or more characteristics (e.g., IQ, age, gender, etc.) and tested on one measure.
HYPOTHESES FOR THE DEPENDENT-SAMPLES t TEST
Null Hypothesis: H 0 : μ 1 = μ 2 where μ 1 stands for the mean for the first
variable/occasion/events and μ 2 stands for the mean
for the second variable/occasion/event.
-or- H 0 : μ 1 – μ 2 = 0
If we think of the data as being the set of difference scores, the null hypothesis becomes
the hypothesis that the mean of a population of difference scores (denoted μ D or δ) equals
0. Because it can be shown that μ D = μ 1 – μ 2 , we can write H 0 : μ D = μ 1 – μ 2 = 0 or
( H 0 : δ = μ 1 – μ 2 = 0). The hypothesized population parameter, defined by the null
hypothesis will be δ = 0, where δ (delta) is defined as the mean of the difference scores
across the two measurements.
Alternative (Non-Directional) Hypothesis: Ha : μ 1 ≠ μ 2 -or- Ha : μ 1 – μ 2 ≠ 0
Alternative (Directional) Hypothesis: Ha : μ 1 < μ 2 -or- Ha : μ 1 > μ 2
( depending on direction ) NOTE: the subscripts (1 and 2) can be substituted with the variable/occasion/event
identifiers. For example: H 0 : μ pre = μ post Ha : μ pre ≠ μ post
ASSUMPTIONS UNDERLYING THE DEPENDENT-SAMPLES t TEST
- The dependent variable (difference scores) is normally distributed in the two conditions.
- The independent variable is dichotomous and its levels (groups or occasions) are paired, or matched, in some way (e.g., pre-post, concern for pay-concern for security, etc.).
When there is an extreme violation of the normality assumption or when the data are not of appropriate scaling, the Wilcoxon Matched-Pairs Signed Ranks Test should be used.
DEGREES OF FREEDOM
Because we are working with difference (paired) scores, N will be equal to the number of differences (or the number of pairs of observations). We will lose (restrict) one df to the mean and have N – 1 df. In other words, df = number of pairs minus the 1 restriction.
THE DEPENDENT-SAMPLES t TEST PAGE 2
EFFECT SIZE STATISTICS FOR THE DEPENDENT-SAMPLES t TEST
Cohen’s d (which can range in value from negative infinity to positive infinity) evaluates the degree (measured in standard deviation units) that the mean of the difference scores is equal to zero. If the calculated d equals 0, the mean of the difference scores is equal to zero. However, as d deviates from 0, the effect size becomes larger.
The d statistic may be computed using the following equation:
SD
Mean d = where the pooled Mean and the Std. Deviation are reported in the
SPSS output under Paired Differences
The d statistic can also be computed from the reported values for t (obtained t value) and N (the number of pairs) as follows:
N
t d =
So what does this Cohen’s d mean? Statistically, it means that the difference between the two sample means is (e.g., .52) standard deviation units (in absolute value terms) from zero, which is the hypothesized difference between the two population means. Effect sizes provide a measure of the magnitude of the difference expressed in standard deviation units in the original measurement. It is a measure of the practical importance of a significant finding.
SAMPLE APA RESULTS
Using an alpha level of .05, a dependent-samples t test was conducted to evaluate whether students’ performance using two methods of mathematics instruction differed significantly. The results indicated that the students’ average performance (score out of 10) using the first method of mathematics instruction ( M = 5.67, SD = 1.49) was significantly higher than their average performance using the second method ( M = 4.50, SD = 1.83), with t (29) = 2.83, p < .05, d = .52. The 95% confidence interval for the mean difference between the two methods of instruction was .32 to 2.01.
Note: there are several ways to interpret the results, the key is to indicate that there was a significant difference between the two methods at the .05 alpha level – and include, at a minimum, reference to the group means, effect size, and the statistical strand.
t (29) = 2.83, p < .05, d =. t Indicates that we are using a t -Test (29) Indicates the degrees of freedom associated with this t -Test 2.83 Indicates the obtained t statistic value ( tobt ) p < .05 Indicates the probability of obtaining the given t value by chance alone d = .52 Indicates the effect size for the significant effect (the magnitude of the effect is measured in standard deviation units)
THE DEPENDENT-SAMPLES t TEST PAGE 4
our example, tobt = 27.00 and tcv = 2.052, therefore, tobt > tcv – so we reject the null hypothesis and conclude that there is a statistically significant difference between the two conditions. That is, the average reaction time for the alcohol condition ( M = 42.07) was significantly longer (slower) than the average reaction time for the no alcohol condition ( M = 30.50).
METHOD THREE: examining the confidence interval and determining whether zero (the hypothesized mean difference) is contained within the lower and upper boundaries. If the confidence intervals do not contain zero – we reject the null hypothesis of no difference. If the confidence intervals do contain zero – we retain the null hypothesis of no difference. In our example, the lower boundary is 10.69 and the upper boundary is 12.45, which does not contain zero – therefore, we reject the null hypothesis of no difference. That is, the reaction time for the alcohol condition ( M = 42.07) was significantly longer (slower) than the reaction time for the no alcohol condition ( M = 30.50). Note that if the upper and lower bounds of the confidence intervals have the same sign (+ and + or – and –), we know that the difference is statistically significant because this means that the null finding of zero difference lies outside of the confidence interval.
As shown above, we reject the null hypothesis in favor of the alternative hypothesis. This indicates that the participant’s reaction time for the alcohol condition ( M = 42.07) was, on average, significantly higher (slower) than their reaction time for the no alcohol condition ( M = 30.50). Further this indicates that the mean difference (11.57) between the two conditions was significantly different from zero.
CALCULATING AN EFFECT SIZE: Since we concluded that there was a significant difference – we will need to calculate an effect size. Had we not found a significant difference – typically, no effect size would have to be calculated. A non-significant finding would have indicated that the participant’s reaction time in the two conditions only differed due to random fluctuation or chance – or that the mean difference was not significantly different from zero.
To calculate the effect size for our example, we can use either of two formulas:
For: 5. 1018519
- 268
SD
Mean d = 5.
where the pooled Mean and the Std. Deviation are reported in the SPSS output under Paired Differences
The d statistic can also be computed from the reported values for t (obtained t value) and N (the number of pairs) as follows:
For: 5. 1025204
- 2915026
N
t d = 5.
Dependent-Samples T-Test Example
Paired Samples Statistics
28
28
alcohol no_alcohol
Pair 1
Mean
N^
Std. Deviation
Std. Error
Mean
Paired Samples Correlations
28
.
.
alcohol & no_alcohol
Pair 1
N
Correlation
Sig. Paired Samples Test
.
27
.
alcohol - no_alcohol
Pair 1
Mean
Std. Deviation
Std. Error
Mean
Lower
Upper
95% ConfidenceInterval of the
Difference
Paired Differences
t^
df
Sig. (2-tailed)
Drop-Down Syntax for Dependent-Samples T-Test Example T-TEST PAIRS = alcohol WITH no_alcohol (PAIRED)/CRITERIA = CI(.95)/MISSING = ANALYSIS. Alternative Syntax for Dependent-Samples T-Test Example t-test / pairs = alcohol no_alcohol.
