26
Ch. 4: Model-Data Fit
I. Introduction
A. IRT has different models (1, 2, 3 plm, and uni- and multidimensional models).
B. If a model does not fit the data, IRT will lose its advantages over CTT.
C. Three methods of checking the model-data fit.
1. Assumption checking.
2. Invariance checking.
3. Prediction checking.
II. Assumption checking
A. Unidimensionality checking
1. Factor analysis for a one-factor solution on inter-item correlation matrix
(tetrachoric). Looking for one dominant eigenvalue.
2. Local independence can be checked by investigating the variance-
covariance matrix of items.
3. The b-value can be checked from two different ability groups. If the b-
values are in linear relationship between the two groups, then the
unidimensionality assumption is met.
B. Equal discrimination index checking (a)
1. If the equal discrimination index assumption is violated, the Rasch model
(1-plm) is not valid for the data.
2. The item-test correlation distribution can be used. If it is homogeneous,
the a-value is equal.
C. Minimal guessing checking (c-parameter)
1. If the c-parameter is not minimal, the 3-pl model is valid.
2. If the performance of the low-ability students on the most difficult items is
close to zero, the c-parameter is minimal.
D. Non-speeded test checking
1. The variance of number of omitted items and the variance of number of
incorrectly answered items can be checked. If the ratio of two variances
(O/I) is close to zero, the assumption of non-speeded test is met.
2. The test scores of examinees under the specific time limit and without a
time limit can be compared. If they are similar to each other, the
assumption is met.
3. The percentage of examinees completing the test, percentage of examinees
completing 75% of the test, and the number of items completed by 80%
ofthe examinees can be reviewed. If nearly all examinees complete nearly
all of the items, speed is not an important factor.
III. Invariance checking
A. Checking ability parameter ( )
1. Make two tests with different b-values for a unidimensional ability item
bank.
2. Administer the two tests to a group of examines.
Docsity.com
