What coaching centres get wrong about question banks
Ten thousand questions is not an asset if none of them are tagged. A short guide to structuring a bank so it can actually be taught from.
We have migrated a lot of institute question banks now, and they nearly all arrive in the same condition: enormous, undifferentiated, and almost impossible to teach from. Ten thousand questions across forty spreadsheets, with no topic tags, inconsistent answer formats and no record of which paper any of them came from.
The institutes that own these banks describe them as their main asset. They are right that the content is valuable. They are wrong that the bank, in that state, is an asset, because nobody can find anything in it.
The test of whether a bank is usable
Ask one question: can a teacher produce a twenty-item paper on one specific sub-topic, at one specific difficulty, in under two minutes? If the answer is no, the bank is a storage archive rather than a teaching tool, regardless of how many items it holds.
The four tags that do the work
- Topic, at lesson granularity
Not Physics. Rotational dynamics.
- Difficulty, three levels
Three applied consistently beats ten applied by feel
- Source
Past paper, original or adaptation. Matters legally too
- Last reviewed
An item nobody has read since 2019 may test a dead syllabus
retrieval value per unit of tagging effort · editors' estimate
Topic at lesson granularity, difficulty on a three-point scale, source provenance, and a last-reviewed date. Each is scored here by how much retrieval value it adds relative to how much effort it costs a teacher to apply.
That is it. Four fields. Institutes routinely propose fifteen, and fifteen-field taxonomies do not survive contact with a busy teacher. The fields go unfilled and the bank ends up untagged anyway.
Explanations are not optional
An item with a correct answer and no explanation is worth a fraction of an item with both. The whole value of a wrong answer is the moment immediately after it, and a bank that cannot fill that moment is a bank that produces scores rather than learning.
We had eleven thousand questions and no explanations. Students would get one wrong and just move on. We were running a measurement service, not a teaching one.
Reviewing what the data flags
Once a bank is live, it starts telling you about itself. An item that most of a strong batch gets wrong is usually not a hard question. It is a badly worded one, or one with two defensible answers. Flagging those automatically and re-reviewing them is the cheapest quality improvement available to any institute.
| What the data shows | Usual cause | What to do |
|---|---|---|
| Strong students get it wrong at the same rate as weak ones | Two defensible answers, or an ambiguous stem | Rewrite the stem or remove the second defensible option |
| Almost everyone picks the same wrong option | That option is arguably correct, or the key is wrong | Check the key first, before rewriting anything |
| Almost everyone gets it right | The item is not discriminating | Keep it as a warm-up, exclude it from assessment sets |
| Answers cluster at random | Students are guessing because the stem is unreadable | Shorten the stem and check the translation if there is one |
You do not have to do this yourself
When institutes join us we do the import and the tagging, including from printed papers and scanned files, and route anything ambiguous to their teachers rather than guessing. It is included in the annual price, mostly because we learned that asking an institute to tag ten thousand questions before onboarding means onboarding never happens.
Frequently asked questions
- How granular should a topic tag be?
- At the level a teacher actually teaches in. A tag broader than a single lesson cannot be taught from, which is why Physics is useless and rotational dynamics is not.
- How many difficulty levels should we use?
- Three, applied consistently by a team that has agreed what each one means. Ten levels applied by feel produce less usable data than three applied by rule.
- Do we need to keep past-paper provenance?
- Yes, for two reasons. Pedagogically it tells you whether an item has been seen before, and legally it tells you what you are entitled to reproduce.
- Can you import from scanned or printed papers?
- Yes. That is the usual case rather than the exception, and anything ambiguous is routed to your teachers rather than guessed at.