Docsity
Docsity
Prepare for your exams
Prepare for your exams

Study with the several resources on Docsity

Earn points to download
Earn points to download

Earn points by helping other students or get them with a premium plan

Guidelines and tips
Guidelines and tips
Sell on Docsity
Sell on Docsity
Docsity AI
Docsity AI
Log inSign up

Prepare for your exams

Study with the several resources on Docsity

Find documents

Prepare for your exams with the study notes shared by other students like you on Docsity

Search for your university

Find the specific documents for your university's exams

Docsity AINEW

Summarize your documents, ask them questions, convert them into quizzes and concept maps

Explore questions

Clear up your doubts by reading the answers to questions asked by your fellow students

Earn points to download

Earn points by helping other students or get them with a premium plan

Share documents
download point icon20 Points

For each uploaded document

Answer questions
download point icon5 Points

For each given answer (max 1 per day)

All the ways to get free points
Get points immediately

Choose a premium plan with all the points you need

Study Opportunities

Choose your next study program

Get in touch with the best universities in the world. Search through thousands of universities and official partners

Community

Ask the community

Ask the community for help and clear up your study doubts

Free resources

Our save-the-student-ebooks!

Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors

Reliability, Interpretations of Reliability - Basic Statistics for Behavioral Sciences - Lecture Notes, Study notes of Statistics for Psychologists

English and Foreign Languages UniversityStatistics for Psychologists

Reliability, Consistency of Test Scores, Six Interpretations of Reliability, Three Situations Related to Reliability, Methods of Reliability Estimation, Internal Consistency Estimation are learning points of this lecture.

Typology: Study notes

2011/2012

Uploaded on 11/21/2012

ashakiran
ashakiran 🇮🇳

4.5

(27)

261 documents

1 / 4

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
10
Ch. 4. Reliability
I. Definition: Consistency of test scores.
II. Six interpretations of reliability
A. ρXX': the correlation between observed scores on parallel tests.
B. ρ²XX': the proportion of variance in X explained by a linear relationship with X'.
C. σ²T
ρXX' = ───
σ²X
(the ratio of true-score variance to observed score variance).
D. ρXX' = ρ²XT: the squared correlation between observed scores and true scores.
E. ρXX' = 1 - ρ²XE (supplementary to D).
F. σ²E
ρXX' = 1 - ──── (supplementary to C).
σ²X
III. Three situations related to reliability
A. If ρXX' = 1,
1. The measurement has no error (all E = 0).
2. X = T for all examinees.
3. σ²X = σ²T: all observed variance is true variance.
4. ρXT = 1: the correlation between observed scores and true scores is 1.
5. ρXE = 0: the correlation between observed scores and error scores is zero
because everything observed reflects true scores.
B. If ρXX' = 0,
1. Everything measured is error.
2. X = E for all examinees.
3. σ²X = σ²E: all observed variance is error variance.
4. ρXT = 0: the correlation between observed scores and true scores is 0.
5. ρXE = 1: the correlation between observed scores and error scores is one
because everything observed reflects error scores.
C. If 0 < ρXX' < 1,
1. Measurement includes some true scores and some error scores.
2. X = T + E
3. σ²X = σ²T + σ²E: observed variance includes some true variance and some
error variance.
4. Differences among observed scores reflect some true score difference and
some measurement error.
5. ρXT =
'XX
: the correlation between observed scores and true scores is
the square-root of reliability. (Interpretation D)
6. ρXE =
'
1XX
: the correlation between observed scores and error scores
is the square-root of 1 minus reliability.
Docsity.com
pf3
pf4

Partial preview of the text

Download Reliability, Interpretations of Reliability - Basic Statistics for Behavioral Sciences - Lecture Notes and more Study notes Statistics for Psychologists in PDF only on Docsity!

Ch. 4. Reliability

I. Definition: Consistency of test scores.

II. Six interpretations of reliability

A. ρXX': the correlation between observed scores on parallel tests. B. ρ²XX': the proportion of variance in X explained by a linear relationship with X'.

C. σ²T ρXX' = ─── σ²X (the ratio of true-score variance to observed score variance). D. ρXX' = ρ²XT: the squared correlation between observed scores and true scores. E. ρXX' = 1 - ρ²XE (supplementary to D).

F. σ²E ρXX' = 1 - ──── (supplementary to C). σ²X

III. Three situations related to reliability A. If ρXX' = 1,

  1. The measurement has no error (all E = 0).
  2. X = T for all examinees.
  3. σ²X = σ²T: all observed variance is true variance.
  4. ρXT = 1: the correlation between observed scores and true scores is 1.
  5. ρXE = 0: the correlation between observed scores and error scores is zero because everything observed reflects true scores. B. If ρXX' = 0,
  6. Everything measured is error.
  7. X = E for all examinees.
  8. σ²X = σ²E: all observed variance is error variance.
  9. ρXT = 0: the correlation between observed scores and true scores is 0.
  10. ρXE = 1: the correlation between observed scores and error scores is one because everything observed reflects error scores. C. If 0 < ρXX' < 1,
  11. Measurement includes some true scores and some error scores.
  12. X = T + E
  13. σ²X = σ²T + σ²E: observed variance includes some true variance and some error variance.
  14. Differences among observed scores reflect some true score difference and some measurement error.
  15. ρXT = (^) XX ' : the correlation between observed scores and true scores is the square-root of reliability. (Interpretation D)
  16. ρXE = (^1) XX ' : the correlation between observed scores and error scores is the square-root of 1 minus reliability.

σ²T ρXX' = ──── σ²X : Reliability is the ratio of true score variance to the observed score variance.

  1. The larger the reliability, the smaller the error variance ---> the more accurate the true score estimation.

IV. Three methods of reliability estimation ( ˆ^ XX ')

A. Test-retest estimation

  1. Testing the same examinees twice with the same test, then compute the correlation coefficient. --> ˆ^ XX '= rXX'.
  2. If X1 = a + bX2 with some variance in X1 and X2, then ˆ^ XX '= rXX' = 1. Assuming there is some variance among the observed scores, if the observed scores of all examinees from one test are perfectly linearly related to the observed scores from the second test, then the estimated reliability is 1.
  3. If b = 0 from X1 = a + bX2, then ˆ^ XX '= rXX' = 0.
  4. Carry-over effect between two tests. a) Remembering the answers from the first test (Overestimation). b) Variation in practice effect (Underestimation).
  5. Time interval between two tests is important because of the carry-over effect. B. Parallel-form estimation and alternate form estimation
  6. The correlation between test scores from two parallel tests. It is almost impossible to make two parallel tests in practice.
  7. Alternate form estimation has no proof that two tests are parallel. a) If TX = TZ but (^2) EX E^2 Z , then one test is less reliable then the other. b) If TX TZ, then two tests may measure two different traits. c) ρXZ = ρXX' if Z = bX' + a (b 0) although Z is not parallel to X. d) The alternate form reliability is valid if the alternate test (Z) is parallel to the original test (X) or if the alternate test (Z) is a linear function of the parallel test (Z = bX' + a).

C. Internal consistency estimation

  1. Split-Half test: one test is divided into two halves to avoid carry-over effect. Two subtests may be either parallel tests or τ-equivalent tests (odd-even, first-second half, matched random). a) It gives the reliability of one half of the whole test (underestimation). b) The reliability should be adjusted for the whole test.

c) If two subtests are parallel, use the Spearman-Brown formula,

  1. General case of the Spearman-Brown formula NρYY' ρXX' = ───────── 1 + (N-1)ρYY' where, X = ΣYi. and N = number of subtests or items (Yi). V. Confidence intervals for true scores A. Standard error of measurement From consequence 'M' it is shown that σ²E = σ²X(1 - ρXX'), and σE is called standard error of measurement. Thus, the estimation of σE,

(^) E, is sE. sE = sX 1 rXX '.

B. Confidence interval for true score x - zα/2sE T x + zα/2sE where, zα/2 is the z critical value for a given α.