Download Ability and Item Parameter Estimation and more Study notes Statistics for Psychologists in PDF only on Docsity!
Ch. 3: Ability and Item Parameter Estimation
I. Item response A. An item response can be obtained from an examinee in such a way that: ui = 1 if the item is answered correctly, or 0 otherwise. B. A vector of item response from an examinee can be obtained in such a way that: u’ = [1, 1, 0] for 3 items. C. For this pattern of item response, we can compute the likelihood function.
L L( u | ) =
n
i
u i
u i Pi^ Q i 1
1
where u ’ = the row vector of obtained item response, ui = item response, and Qi = 1 – Pi.
e.g. For u ’ = [1, 1, 0], we can have L = P 1 *P 2 *Q 3 If P 1 = .50, P 2 = .40, and P 3 = .30, then, L = (.50)(.40)(.70) II. Ability Estimation A. For a given set of item responses, the main job of a psychometrician is to estimate the examinee’s true ability using the likelihood function of the response pattern. B. There are several estimation methods depending on the algorithm applied (MLE, BME, EAP, WLE). C. MLE
- The maximum value of the likelihood function can be obtained for the examinee’s true ability value of.
- Obtain the first derivative of the natural log of the likelihood function ( l ) to get the slope of the function.
l’ = ln L = i i i
i i i P Q
P '( u P )
where
i i i i
i i P c Q c
P Da P )( ) 1
- Set l’ to zero (the slope is set to zero), then solve the equation for. MLE.
- There is no closed form solution for the equation. Use numerical analysis methods.
- Either bi-sectional or Newton-Raphson method.
- When an examinee answers all items correctly or all items incorrectly, then ^ = - or +. Truncation is needed.
- Sometimes the maximum value does not exist. Aberrant responses.
*Bayes’ Rule
P(Ai|B) = ( )
P B
P Ai B
PB AP A PB AP A
PB AP A
(binary)
i i
i i
P B A P A
P B A P A
(polytomous)
PB AP AdA
P B Ai P Ai
( | ) ( )
(continuous)
E. EAP( ) = E[p( | u )]
= P ( | u ) d
Lu d
Lu d
, (^) P( | u ) = Lu d
Lu
( | ) ( )
No closed form solution for this equation. Approximation.
EAP( ) = q
k
k k
q
k
k k k
Lu X A X
Lu X A X X
1
1
( | ) ( )
where Xk = Gauss-Hermite quadrature points, and A(Xk) = the weight of G-H quadrature points. F. Bayesian Modal Estimate (BME) or Maximum A Posteriori (MAP): BME is the mode of posterior distribution (Samijima, 1969). G. Weighted Likelihood Estimate (WLE, Warm, 1989): Added a weighted function to MLE in order to reduce the bias of MLE. H. Summary of the ability estimation results (Kim & Nicewander, 1993)
- Basic facts a) As the a-parameter value increases, bias, ( )
, decreases and reliability increases. b) The closer between and b-parameter, the smaller the bias. c) Bias is large in the tails of the ability distribution.
- When item parameters are known, a) MLE and the number-right-score (z) are the most biased. b) EAP and WLE are least biased. c) MLE is outwardly biased (overestimates the upper tail and underestimates the lower tail). d) BME, EAP WLE, and z are inwardly biased.
- When item parameters are estimated, a) Errors in the estimation of item parameters do not have dramatic negative effects on ability estimation in terms of bias, standard error, and reliability. b) WLE was moderately affected by errors.
III. Item parameter estimation A. Likelihood function
L L( u | , a, b, c) =
n
i
u i
u i Pi^ Q i 1
1
B. Log likelihood function l = lnL = [uilnPi + (1-ui)lnQi]. C. Compute the first derivative of l with regard to each item parameter. D. Set each of the derivatives to zero and simultaneously solve the equations with 3 unknowns. E. The multivariate Newton-Raphson method will be used for each item.
F. Local independence is not required since we are estimating each item. Instead the independence of examinees’ responses is required. IV. Joint estimation of the item and ability parameters A. Joint MLE
- Stage 1. a) Ability parameters are computed form X/(N-X) for each examinee as a starting value (X = number-right-score for a given test). b) ln[X/(N-X)] for each examinee is standardized to set a (0, 1) distribution (indeterminancy). c) The standardized ln[X/(N-X)] values are treated as known ability values. Then, item parameters are estimated with the known ability values.
- Stage 2: Using the estimated item parameters we can estimate the ability parameter.
- May have the same MLE problems as in the ability estimation (All correctly and incorrectly answered items will have +- infinity for their ability estimation). B. Marginal MLE (MMLE)
- Assume a prior form (usually normal) of ability distribution.
- Based on the prior (marginal) distribution, we estimate the item parameters (solve some problems of the joint MLE).
- A large number of examinees is important for the prior ability distribution.
- MMLE was implemented in BILOG.
V. Item information function and standard error.
A. I( , ui ) i i
i P Q
P '^2
2 2
[ ][ 1 ]
Dai bi Dai b i i
i i c e e
Da c
( | )
Var
Inverse of the variance of
given.
B. SE(
I
Var.
