Download Reliability - Mathematics and Statistics - Study Notes and more Study notes Mathematical Statistics in PDF only on Docsity!
1
The RELIABILITY procedure employs one of two different computing methods, depending upon the MODEL specification and options and statistics requested. Method 1 does not involve computing a covariance matrix. It is faster than method 2 and, for large problems, requires much less workspace. However, it can compute coefficients only for ALPHA and SPLIT models, and it does not allow computation of a number of optional statistics, nor does it allow matrix input or output. Method 1 is used only when alpha or split models are requested and only FRIEDMAN, COCHRAN, DESCRIPTIVES, SCALE, and/or ANOVA are specified on the STATISTICS subcommand and/or TOTAL is specified on the SUMMARY subcommand. Method 2 requires computing a covariance matrix of the variables. It is slower than method 1 and requires more space. However, it can process all models, statistics, and options. The two methods differ in one other important respect. Method 1 will continue processing a scale containing variables with zero variance and leave them in the scale. Method 2 will delete variables with zero variance and continue processing if at least two variables remain in the scale. If item deletion is required, method 2 can be selected by requesting the covariance method.
Notation
There are N persons taking a test that consists of k items. A score X (^) ji is given to the jth person on the ith item. i → items 1 2 … i^ … k (^1) X 11 X 12 X (^1) k P 1 M M j (^) X (^) ji Pj
M M N (^) X (^) N 1 X (^) N 2 X (^) Nk PN T 1 T 2 …^ Ti …^ Tk G
If the model is SPLIT, k 1 items are in part 1 and k 2 = k − k 1 are in part 2. If the number of items in each part is not specified and k is even, the program sets k 1 = k 2 = k 2. If k is odd, k 1 = 1 k + 16 2. It is assumed that the first k 1 items are in part 1.
W w (^) j j
N
=
∑ 1
Sum of the weights, where w (^) j is the weight for case j
P (^) j Xji i
k
=
∑ 1
The total score of the jth person
P (^) j = P (^) j k Mean of the observations for the^ jth person
Ti X (^) ji wj j
N
=
∑ 1
The total score for the ith item
G X (^) ji wj j
N
i
k
= =
∑ ∑ 1 1
Grand sum of the scores
G
G
Wk
Grand mean of the observations
Scale and Item Statistics—Method 1
Item Means and Standard Deviations
Mean for the i th Item
Ti =Ti W
For the split model:
Variance Part 1
S
W
p w^ j X^ W^ T j
N ji i
k i i
k 1 2 1 1 1
2 1 1
1 1
−
!
"
$
= = = #
∑ ∑ ∑
Variance Part 2
S
W
p w^ j X^ W^ T j
N ji i k
k i i k
k 2 2 1 1
2
1
2 1 1 1 1
!
"
$
= = + = + #
∑ ∑ ∑
Item-Total Statistics
Scale Mean if the i th Item is Deleted
M^ ~ M T
i =^ − i
Scale Variance if the i th Item is Deleted
S^ ~ S S cov X ,P i p i i (^2) = 2 + 2 − 2 1 6
where the covariance between item i and the case score is
cov X ,P W i P Xj ji w^ j T T j
N l i l
k 1 6 =^ −
∑= ∑=
Alpha if the i th Item Deleted
A k k i S^ l Si l
k
l i
= ≠
∑
1
Correlation between the i th Item and Sum of Others
R
X P S
i (^) S S
i i i i
cov , − ~
1 6 2
The ANOVA Table (Winer, 1971)
Source of variation
Sum of Squares df
Between people
P wj j k G Wk j
N 2 1
2
∑ −^ W^ −^1
Within people
w Xj ji P w k j
N
i
k j j j
N 2 1 1
2 = = = 1
∑ ∑ −^ ∑ W k 1 − (^16)
Between measures
Ti W G Wk i
k 2 1
2
∑ −^ k^ −^1
Residual w Xj ji P w^ k^ T^ W^ G^ Wk j
N
i
k j j j
N i i
k 2 1 1
2 1
2 1
2 = = = =
∑ ∑ −^ ∑ −^ ∑ −^1 W^ −^1 61 k^ −^16
Total w Xj ji G^ Wk j
N
i
k 2 1 1
2 = =
∑ ∑ −^ Wk^ −^1
where
D SS SS Wk
T G w P G
M T P X w G P w k G SS
w P G X T G
SS SS SS df W k
i i
k j j j
N
i i
k j ji j j
N j j j
N
j j j
N ji i i
k
!
"
$
!
"
$
!
"
$
= =
= = =
= =
∑ ∑
∑ ∑ ∑
∑ ∑
bet. meas bet. people
bet. meas
bal res nonadd
3 8 1^6
3 8 3 8
3 8 3 8
1 61 6
2 1
2 1
1 1
2 1
1 1
The test for nonadditivity is
F MS
MS
= nonadd df = W − k− − balance
21 ,^1 161 16
The regression coefficient for the nonadditivity term is
B^ $^ =M D,
and the power to transform to additivity is
p^ $^ = 1 −BG$
Scale Statistics
Reliability coefficient alpha (Cronbach 1951)
A k k
S
S
i i
k
p
=
∑
1
2 1 2
If the model is split, separate alphas are computed:
A k k
S
S
A k k
S
S
i i p
k
i i k p
k
1 1 1
2
1 2 1
2 2 2
2
2 2 1
1
1
2
=
= +
For Split Model Only
Correlation Between the Two Parts of the Test
R
S S S
S S
p p p p p
2 1 2 2 2
1 2
4 9
Equal Length Spearman-Brown Coefficient
Y
R
R
Scale Variance
S (^) p S (^) i v i
k ij
k
i j
k 2 2 1
= <
∑ ∑∑
If the model is split,
S S v
S S v
p i i
k ij
k
i j
k
p i i k
k ij j i
k
i k
k
1 2 2 1
2 2 2 1 1
1 1 1
1 1
= <
= + = + >
where the first k 1 items are in part 1.
Scale Statistics—Method 2
Alpha Model
Estimated Reliability
k k
S
S
i i
k
− (^) p
=
∑ 1
2 1 2
Standardized Item Alpha
k k
Corr 1 + 1 − 16 Corr
where
Corr = − ∑<∑
k k 1
rij
k
i j
k
1 6
Split Model
Correlation between Forms
v
S S
ij j k
k
i
k
p p
= = +
∑ ∑ 1
1
1 1 1 2
Guttman Split-Half
G
v
S
ij j k
k
i
k
p
= = =^ +
∑ ∑ 1
1
1 1 2
Alpha and Spearman-Brown equal and unequal length are computed as in method 1.
True Variance
TV
k k
vij
k
i j
k = = − ∑<∑
cov 2 1 16
Error Variance
EV = var −cov
Common Inter-Item Correlation
R^ $^ =cov var
Reliability of the Scale
A k k
S
S
i i
k
p
=
∑
1
2 1 2
Unbiased Estimate of the Reliability
A^ $ W^ A
W
1 6 1 6
where A is defined above.
Test for Goodness of Fit
χ 2
2 1 1
^
^
W
k k k
k k k W
1 6 L
1 6 1 6
1 6 1 6 1 6
log
where
L
V
k
df k k
= k − + +
− 3 var^ cov^8 3 var^0 5 cov 8
0 5
1 1
Log of the Determinant of the Unconstrained Matrix
log UC =logV
Log of the Determinant of the Constrained Matrix
log C log var cov var k cov
k = − + − ^
− 4 9 4 1 6 9
1 1
Strict Parallel (Kristof 1963)
Common Variance
CV
k Ti G i
k = + − =
var (^) ∑
1
3 8
Test for Goodness of Fit
χ 2
2 1 1
W (^) k k k k k k W
1 6 L
1 6 1 6 1 6 1 2 6 71 6
log
where
L
V
k k
T G
df k k
i i
k k
=
− var 1 cov var cov ∑
2 1
1 4 1 6 9 3 8
1 6
Log of the Determinant of the Unconstrained Matrix
log UC =logV
Log of the Determinant of the Constrained Matrix
log C log var k cov var cov k
Ti G i
k k = + − − + −
− (^1) ∑
2 1
1 4 1 6 9 3 8
Additional Statistics—Method 2
Descriptive and scale statistics and Tukey’s test are calculated as in method 1. Multiple R^2 if an item is deleted is calculated as
R^ ~
S V
i i i
i ii
2 2 2
2 (^1 ) = − = 1 −
ε (^) ε 4 9
Analysis of Variance Table
Source of variation
Sum of Squares df
Between people
W
k
S
k v k i v i
k ij i j
ij i j
!
"
$
= < <
(^1) ∑ ∑ ∑∑ ∑
2 1
1 6 1 6
W − 1
Within people
W k k S k i v^ W^ SS i
k ij i j
− − (^) − −
!
"
$
∑= ∑<∑ ##+^ −
(^1 1 ) 1
(^21) 1
1 61 6 (^1 6) bet. people (^) W k 1 − 16
Between measures
W T
k i T i
k i i
k 2 1 1
2 1 = =
∑ − ∑
k − 1
Residual W k k
S
k i v i
k ij i j
!
"
$
= < #
∑ ∑∑
2 1
1 61 6
1 W − 161 k − 16
Total Between SS + Within SS Wk − 1
Hotelling’s T 2 (Winer, 1971)
T 2 = W Y B ′ −^1 Y
Item Variance Summaries
Same as for item means excepts that Si^2 is substituted for Ti in all calculations.
Inter-Item Covariance Summaries
Mean = −
<
∑ ∑v
k k
ij i j 1 16
Variance = 1 1 1
2
2
k k
v k k ij v i j
ij − − (^) i j
!
"
$
< < # 1 6 ∑∑^1 6 ∑∑
Maximum = max i j, ij
v
Minimum = min i j, ij
v
Range = Maximum −Minimum
Ratio Maximum Minimum
Inter-Item Correlations
Same as for inter-item covariances, with vij being replaced by rij.
If the model is split, statistics are also calculated separately for each scale.
Intraclass Correlation Coefficients
Intraclass correlation coefficients are always discussed in a random/mixed effects model setting. McGraw and Wong (1996) is the key reference for this document. See also Shrout and Fleiss (1979).
In this document, two measures of correlation are given for each type under each model: single measure and average measure. Single measure applies to single measurements, for example, the ratings of judges, individual item scores, or the body weights of individuals, whereas average measure applies to average
measurements, for example, the average rating for k judges, or the average score for a k-item test.
One-Way Random Effects Model: People Effect Random
Let X (^) ji be the response to the i-th measure given by the j-th person, i = 1, …, k, j = 1, …, W. Suppose that X (^) ji can be expressed as X (^) ji = μ + pj + wji, where pj is the between-people effect which is normal distributed with zero mean and a variance of σ (^) p^2 , and wji is the within-people effect which is also normal distributed with zero mean and a variance of σ (^) w^2. Let MSBP and MSWP be the respective between-people Mean Squares and within- people Mean Squares. Formulas for these two quantities can be found on page 479 of SPSS 7.5 Statistical Algorithms by dividing the corresponding Sum of Squares with its degrees of freedom.
Single Measure Intraclass Correlation
The single measure intraclass correlation is defined as
ρ
σ ( ) (^1) σ σ
2 = (^2) + 2 p p w
Estimate
The single measure intraclass correlation coefficient is estimated by
ICC MS^ MS
MS k MS
BP WP BP WP
In general,
(^1) < à 1
k
ICC( ).
Confidence Interval
