Item analysis
Item analysis is the statistical review of each question in a test or survey scale to see how it performs: how hard or easy it is, and how well it separates people who score high overall from those who score low.
How it works
Two statistics do most of the work. The difficulty index is the proportion of respondents who answer an item correctly, or who agree with it on a scale; values between about 0.3 and 0.7 are usually considered useful, while items everyone passes or everyone fails tell the analyst little. The discrimination index compares the top-scoring group with the bottom-scoring group, and a good item is answered correctly by far more of the top group.
Items with low or negative discrimination are rewritten or removed. In survey research the same logic applies to scale items, alongside checks such as item-total correlation.
Difficulty p = correct answers / respondents; Discrimination D = (correct in top group - correct in bottom group) / size of one group
A worked example
A software company's certification exam is taken by 400 candidates. One item is answered correctly by 360, p = 0.90, too easy to be informative. Another has 70 of the top 100 scorers correct and 30 of the bottom 100, D = (70 - 30) / 100 = 0.40, a strong item.
Related
Automate this in Pulse
Respondents' explanations of a weak item, read in Pulse transcripts, show whether the wording or their reading of it caused the result item analysis flagged.