IRT and Ability Scales: Monotonic Relationship and Estimation Methods, Study notes of Statistics for Psychologists

The relationship between ability and true scores in classical test theory (ctt) and item response theory (irt). It covers the general procedure for estimating true ability, the nature of ability scales in ctt and irt, and the transformation of ability scales. The document also explains the ratio-scale property and log-odds (logits) in irt.

Typology: Study notes

2011/2012

Uploaded on 11/21/2012

ashakiran
ashakiran šŸ‡®šŸ‡³

4.5

(27)

261 documents

1 / 3

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
28
Ch. 5: Ability Scale
I. Introduction
A. In CTT, X (number-right-score) is an unbiased estimate of a person’s true score
T,

E(X) T.
B. In IRT, a person’s ability, , is monotonically related to the person’s true score.
Monotonic relationship is a non-linear and strictly increasing relationship.
C. General procedure in estimating the true ability of a person.
1. Obtain item responses of 1s and 0s (binary item).
2. When item parameters are known, use one of the estimation methods
(MLE, BME, EAP, WLE).
3. When item parameters are unknown, use joint MLE or MMLE for item
parameters and then, use one of MLE, BME, EAP, and WLE.
4. The estimated or transformed

can be reported.
II. Nature of ability scale
A. In CTT,
1. E(X) T
2. X/N = proportion-correct in criterion-referenced test.
3. Norm-reference test: stanines, percentiles,
T-score, ACT, SAT, GRE.
4. X is test- and person-dependent.
B. In IRT
1. (ability, trait, or proficiency level) is test-independent.
2. may be an ordinal scale, but has a limited ratio scale interpretation.
III. Transformation of
A. Linear transformation of , b, and a.
Let * = , b* =
b
, and a* = a
/
, then 3-pl is,
P(
*
) = c +
)]()[(/
1
1
bDa
e
c
= c +
][/
1
1
bDa
e
c
= c +
= c +
)(
1
1
bDa
e
c
= P( ).
P( ) is invariant under the linear transformation of , b, and a. 
Indeterminancy. (Woodcock-Johnson’s scale p. 80).
B. Non-linear transformation of and b: partial ratio-scale interpretation only for
the Rasch model.
1. Let * =
D
e
and b* =
Db
e
where D = 1, a = 1.
2. P( *) =
)(
)(
1bD
bD
e
e
=
DbD
DbD
e
e
1
=
DbD
DbD
ee
ee
/1
/
Docsity.com
pf3

Partial preview of the text

Download IRT and Ability Scales: Monotonic Relationship and Estimation Methods and more Study notes Statistics for Psychologists in PDF only on Docsity!

Ch. 5: Ability Scale

I. Introduction

A. In CTT, X (number-right-score) is an unbiased estimate of a person’s true score

T,  E (X) T.

B. In IRT, a person’s ability, , is monotonically related to the person’s true score.

Monotonic relationship is a non-linear and strictly increasing relationship.

C. General procedure in estimating the true ability of a person.

  1. Obtain item responses of 1s and 0s (binary item).
  2. When item parameters are known, use one of the estimation methods

(MLE, BME, EAP, WLE).

  1. When item parameters are unknown, use joint MLE or MMLE for item

parameters and then, use one of MLE, BME, EAP, and WLE.

  1. The estimated or transformed

can be reported.

II. Nature of ability scale

A. In CTT,

  1. E (X) T
  2. X/N = proportion-correct in criterion-referenced test.
  3. Norm-reference test: stanines, percentiles,

T-score, ACT, SAT, GRE.

  1. X is test- and person-dependent.

B. In IRT

  1. (ability, trait, or proficiency level) is test-independent.
  2. may be an ordinal scale, but has a limited ratio scale interpretation.

III. Transformation of

A. Linear transformation of , b, and a.

Let * = , b* = b , and a* = a / , then 3-pl is,

P( *) = c + / [( )( )] 1

Da b e

c

= c + / [ ] 1

Da b e

c

= c + / .[ ] 1

Da b e

c

= c + ( ) 1

Da b e

c = P( ).

P( ) is invariant under the linear transformation of , b, and a. 

Indeterminancy. (Woodcock-Johnson’s scale p. 80).

B. Non-linear transformation of and b: partial ratio-scale interpretation only for the Rasch model.

  1. Let * =

D e and b* =

Db e where D = 1, a = 1.

2. P( *) = ( )

( )

D b

D b

e

e = D Db

D Db

e

e

1

= D Db

D Db

e e

e e

1 /

= Db Db D Db

D Db

e e e e

e e

/ /

= Db D

D

e e

e

b

P( *) =

b

(The first Rasch model)

If * = b*, then P( *) = .5.

Q( *) = 1 – P( *)

b

b

b

b

b

  1. The odds for success, O

O =

Q

P

b b

b

b

  1. Ratio of success odds for two examinees ( 1 *,^2 *)

Given Op1 =

b

, and Op2 =

b

2

1

2

1

p

p

O

O

If 1 * 2 2 *, then examinee 1 has twice the odds of examinee 2 in

answering the item correctly (ratio-scale property).

  1. Ratio of success odds for two items (b 1 *, b 2 *)

Oi1 =

b 1

, Oi2 =

b 2

, then

1

2

2

1

b

b

O

O

i

i .

If b 2 * = 2b 1 *, then item 1 has twice the odds of item 2 for an examinee to get the item correct (ratio-scale property).

  1. Log-odds (logits)

( 1 2 ) 2

1

2

1

2

1

* D

D

D

p

p e e

e

O

O

ln ( 1 2 ) 2

1 D O

O

p

p .

In 1-pl model, D is omitted (D=1.0).

ln 1 2 2

1

p

p

O

O

In the same way.

ln 2 1 2

1 b b O

O

i

i .

  1. Logits can also be obtained from the original model.

Pi( ) = ( )

( )

b

b

e

e , and Qi( ) = ( ) 1

b e