Download Analyzing Item Difficulty and Discrimination: Understanding Test Effectiveness and more Exercises Teaching method in PDF only on Docsity!
Module 4c:Analyzing Item Difficulty and
Discrimination
Passport to Great Teaching โ Creative Assessment Timothy S. Brophy
Item Difficulty and Discrimination
- Teachers who create tests for classroom use often seek to know how
effective their tests are
- Item analysis provides important information about how well items function
- Item difficulty helps us to know the degree to which students get the
answer correct
- Item discimination examines how the top scoring group of test takers
compare to the low scoring group
Calculating Item Difficulty Count item correctly. the total number of students answering each
For by the each total item, number divide of the students. number answering correctly
This answered gives (^) eachyou the item proportion correctly. of This students figure whois called the item's difficulty level. Caution: item, and The vice higher versa. the difficulty level the easier the
1
2
3
Item Discrimination Index (DI)
When we subtract the proportion of low scoring students who got the item right from the proportion of high scoring students who got it right, the remainder is the discrimination index
This is a measure of how well the item discriminates between the top scorers and the bottom scorers on the item
Calculating the DI
Step items 1. each Compute student each got student's correct.) score. (i.e., count the number of
Step 2. Divide the class into groups on the basis of total score
(e.g., low, or 50% 25% high high, and 50% 50% middle, low, or 25% 33% low, high, etc.). 33% middle, and 33%
Step middle 3. Thegroup high is notand needed low groups in computing must be the discrimination same size. The index.
(e.g., 25% high and 25% lowโ50% middle not used.)
Step scoring 4. For group each that item, got count the item the right. number of students in the high
Calculating the DI
Step total 5. number Divide thisin the number high scoring by the group, for each item.
Step each 6. item, Repeat this steps time 4 using and only 5 for the low scoring group.
Step the proportion 7. For each of item low subtract scoring students from the who proportion got the item of high right scoring The students remainder who gotis the it right. discrimination index.
4
5
6
Item 1. Calculating Difficulty and Discrimination
- Top scoring group (n = 25): 20 got it correct, or 80%
- Middle group (n = 50): 20 got it correct
- Lowest scoring group (n = 25): 15 got it correct, or 60%
- Difficulty level: Add the total who got the item correct โ
20+20+15 = 55, and divide by the number of students, 100
- 55/100 =.
- Discrimination Index: Subtract 80% โ 60% = 20%, for a DI of.
Score group Upper (n = 25) Middle (n = 50) Lower (n = 25) Item number n p n p n p 1 20 .80 20 15.
Item 2. Calculating Difficulty and Discrimination
- Top scoring group (n = 25): 15 got it correct, or 60%
- Middle group (n = 50): 30 got it correct
- Lowest scoring group (n = 25): 5 got it correct, or 20%
- Difficulty level: Add the total who got the item correct โ
15+30+5 = 50, and divide by the number of students, 100
- 50/100 =.
- Discrimination Index: Subtract 60% โ 20% = 40%, for a DI of.
Score group Upper (n = 25) Middle (n = 50) Lower (n = 25) Item number n p n p n p 2 15 .60 30 5.
Item 3. Calculating Difficulty and Discrimination
- Top scoring group (n = 25): 24 got it correct, or 96%
- Middle group (n = 50): 38 got it correct
- Lowest scoring group (n = 25): 12 got it correct, or 48%
- Difficulty level: Add the total who got the item correct โ
24+38+12 = 74, and divide by the number of students, 100
- 74/100 =.
- Discrimination Index: Subtract 96% โ 48% = 48%, for a DI of.
Score group Upper (n = 25) Middle (n = 50) Lower (n = 25) Item 3 number n 24 (^) p .96 (^) n 38 (^) p n 12 (^) p.
10
11
12
Keeping or Revising items
Keeping and revising items
When general an acceptability item meets the guidelines discrimination, for difficulty you should and keep the item
If guidelines, it falls outside you makethe a judgment revise, or rejectwhether it to keep,
Guidelines for Item Difficulty and
Discrimination Index statistics
Difficulty level: the standard range is .25โ. Discrimination index: the standard is .20 or higher Item 1 โ Difficulty, .55; DI,. Item 2 โ Difficulty, .50; DI,. Item 3 โ Difficulty, .74; DI,. Keep or reject?
13
14
15
