Docsity
Docsity
Prepare for your exams
Prepare for your exams

Study with the several resources on Docsity

Earn points to download
Earn points to download

Earn points by helping other students or get them with a premium plan

Guidelines and tips
Guidelines and tips
Sell on Docsity
Sell on Docsity
Docsity AI
Docsity AI
Log inSign up

Prepare for your exams

Study with the several resources on Docsity

Find documents

Prepare for your exams with the study notes shared by other students like you on Docsity

Search for your university

Find the specific documents for your university's exams

Docsity AINEW

Summarize your documents, ask them questions, convert them into quizzes and concept maps

Explore questions

Clear up your doubts by reading the answers to questions asked by your fellow students

Earn points to download

Earn points by helping other students or get them with a premium plan

Share documents
download point icon20 Points

For each uploaded document

Answer questions
download point icon5 Points

For each given answer (max 1 per day)

All the ways to get free points
Get points immediately

Choose a premium plan with all the points you need

Study Opportunities

Choose your next study program

Get in touch with the best universities in the world. Search through thousands of universities and official partners

Community

Ask the community

Ask the community for help and clear up your study doubts

Free resources

Our save-the-student-ebooks!

Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors

Understanding Item Analysis: Exam Stats - Discrimination, Difficulty, and Deviation, Lecture notes of Statistics

Ateneo de Davao University (ADDU)Statistics

This quick reference guide provides insights into various statistics generated by testing software, focusing on discrimination index, difficulty index, and standard deviation. Learn how to interpret these metrics to improve test questions and evaluate student performance. Canvas and examsoft are mentioned as examples.

Typology: Lecture notes

2021/2022

Uploaded on 08/05/2022

hal_s95
hal_s95 🇵🇭

4.4

(655)

10K documents

1 / 4

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
!
!
Quick&Reference&Guide&to&Item&Analysis&
!
Considerations&
When!using!the!statistics!generated!by!testing!software,!you!should!always!consider!the!size!of!
the!class.!!The!more!students!there!are,!the!more!accurate!the!statistics!will!be.!!Also,!as!the!
instructor,!you!will!be!the!best!judge!of!a!question’s!validity.!!Test!questions!serve!different!
purposes.!!The!statistics!should!just!be!used!to!guide!you!as!you!improve!your!tests.!
!
Terms&
Discrimination&index!(or!Point!Biserial!Index-PBI):!!A!discrimination!index!is!an!indicator!of!
whether!a!question!accurately!determines!student’s!mastery!of!a!concept.!!The!index!is!
calculated!by!comparing!whether!a!student!got!the!question!right!with!their!total!score.!The!
theory!is!that!a!question!that!is!a!good!discriminator!will!be!missed!by!weaker!students,!but!
that!better!students!will!get!that!question!right.!!An!index!of!<.2!generally!indicates!the!
question!should!be!revised.!!!
!
Canvas!Guide:!
Student!groups!are!generally!divided!as!the!top!27%,!the!middle!46%,!and!the!bottom!27%.!Ideally,!
students!who!did!well!on!the!exam!should!get!the!question!right.!If!students!do!well!on!the!overall!exam!
but!not!on!the!question,!the!question!itself!may!need!to!be!revised.!
!
Lower!discrimination!scores!(in!red)!are!scored!+0.24!or!lower;!good!scores!(green)!are!+0.25!or!higher.!
An!ideal!discrimination!index!shows!students!who!scored!higher!on!the!quiz!getting!the!quiz!question!
right,!students!who!scored!lower!on!the!quiz!getting!the!quiz!question!wrong,!and!students!in!the!middle!
range!on!either!side.!A!discrimination!index!of!zero!shows!all!students!getting!the!quiz!question!right!or!
wrong.!
!
!
!
ExamSoft!Guide:!
Discrimination&Index&(-1.00-1.00):!The!discrimination!index!of!a!question!shows!the!difference!in!
performance!between!the!upper!27%!and!the!lower!27%.!It!is!determined!by!subtracting!the!
difficulty!index!of!the!lower!27%!from!the!difficulty!index!of!the!upper!27%.!!A!score!close!to!0!
indicates!that!the!upper!exam!takers!and!the!lower!exam!takers!performed!similarly!on!this!
question.!As!a!discrimination!index!becomes!negative,!this!indicates!that!more!of!the!lower!
pf3
pf4

Partial preview of the text

Download Understanding Item Analysis: Exam Stats - Discrimination, Difficulty, and Deviation and more Lecture notes Statistics in PDF only on Docsity!

Quick Reference Guide to Item Analysis Considerations When using the statistics generated by testing software, you should always consider the size of the class. The more students there are, the more accurate the statistics will be. Also, as the instructor, you will be the best judge of a question’s validity. Test questions serve different purposes. The statistics should just be used to guide you as you improve your tests. Terms Discrimination index (or Point Biserial Index-PBI): A discrimination index is an indicator of whether a question accurately determines student’s mastery of a concept. The index is calculated by comparing whether a student got the question right with their total score. The theory is that a question that is a good discriminator will be missed by weaker students, but that better students will get that question right. An index of <.2 generally indicates the question should be revised. Canvas Guide: Student groups are generally divided as the top 27%, the middle 46%, and the bottom 27%. Ideally, students who did well on the exam should get the question right. If students do well on the overall exam but not on the question, the question itself may need to be revised. Lower discrimination scores (in red) are scored +0.24 or lower; good scores (green) are +0.25 or higher. An ideal discrimination index shows students who scored higher on the quiz getting the quiz question right, students who scored lower on the quiz getting the quiz question wrong, and students in the middle range on either side. A discrimination index of zero shows all students getting the quiz question right or wrong. ExamSoft Guide: Discrimination Index (-1.00-1.00) : The discrimination index of a question shows the difference in performance between the upper 27% and the lower 27%. It is determined by subtracting the difficulty index of the lower 27% from the difficulty index of the upper 27%. A score close to 0 indicates that the upper exam takers and the lower exam takers performed similarly on this question. As a discrimination index becomes negative, this indicates that more of the lower

performers got this question correct than the upper performers. As it becomes more positive, more of the upper performers got this question correct. Discrimination indexes (PBIs) are also generated for distractors. Distractors should have negative numbers and the correct answer should have a positive number. Standard Deviation : The standard deviation measures how different scores are among students. A low standard deviation indicates that students generally scored close to the mean, while a large standard deviation indicates that scores differed widely. A deviation < 1 is generally low. A deviation >= 1 is generally high. Difficulty Index : The difficulty index is a number between 0 and 1 that indicates how hard the question is. It is the proportion of students who answered correctly. You generally want a range of .3-.8 (so 30%-80% answered correctly), unless you have a very fundamental question where you want all students to answer correctly. Cronbach’s Alpha and KR- 20 : these are statistical tests for internal reliability. These generate a number between 0-1 and > .5 is ideal. As long as there is no missing data and partial credit is not given, both methods should produce the same result. Cronbach’s alpha is the more commonly used method, but is not reliable when data is missing. KR-20 is not appropriate when questions may allow partial credit. Canvas uses Cronbach’s Alpha and ExamSoft uses KR-

Quiz Statistics in Canvas First, go into the quiz and choose: Quiz statistics: The next two graphs illustrate several examples in Canvas. What do these graphs tell you about performance on these two tests?