Ranking questions: what they are and when to use them
A ranking question asks respondents to put a set of options in order of preference or importance — first, second, third, and so on. Use ranking when you need to force trade-offs and find true priorities, because unlike rating scales, ranking won't let people call everything "very important." The cost is more effort for respondents and trickier analysis: ranking gives you relative order, not how far apart the options are. Keep the list short (ideally 4–6 items), and don't ask people to rank things they can't meaningfully compare.
What is a ranking question?
In a ranking question, respondents arrange a list of options into an order — for example, dragging five features into the sequence that matters most to them. The output is an ordinal ranking per person, which you aggregate to see which options rise to the top across everyone. Ranking is a forced-choice format: every item gets a distinct position, so respondents have to decide what beats what.
Ranking vs rating: what's the difference?
This is the decision most people get wrong. A rating question asks how good or important each option is on its own, so a respondent can rate ten things "very important." A ranking question asks which option beats which, forcing trade-offs. If you need to know absolute levels (how satisfied, how important), rate. If you need priorities and trade-offs (what to build first, what matters most), rank.
| Question type | Tells you | Best for | Watch out for |
|---|---|---|---|
| Rating | Absolute level per item | Tracking, comparing items independently | Everything rated 'very important' |
| Ranking | Relative order across items | Forcing priorities and trade-offs | More effort; no sense of distance between ranks |
| MaxDiff (advanced) | Most vs least important across sets | Cleaner priorities with many items | More complex to set up and analyze |
When should you use a ranking question?
- Prioritizing a roadmap: "Rank these five features by how much you'd use them."
- Finding the top motivation when several sound appealing on a rating scale.
- Allocating limited resources where trade-offs are the whole point.
- Breaking ties when a rating question produced everything at the top of the scale.
When should you avoid it?
- Long lists. Ranking more than about six items is exhausting and produces sloppy answers near the bottom.
- Items people can't meaningfully compare (apples and oranges).
- When you need absolute levels — ranking can't tell you whether the top item is loved or merely least disliked.
- Mobile-heavy audiences if your tool's drag interaction is clumsy on small screens.
A worked example
Question: "Drag these into the order that matters most to you when choosing a survey tool." Options: price, ease of use, question types, branding control, response limits. A respondent ranks them 1–5. Across 200 respondents, you might find ease of use ranked first by most people, with price a close second and branding control consistently last. That ordering is the priority signal — but note it doesn't tell you how much more important ease of use is than price.
If everything keeps coming out "very important" on your rating questions, that's the signal to switch to ranking. Rating scales are prone to a ceiling effect where respondents inflate every item; ranking forces the trade-offs that reveal what people will actually choose when they can't have it all.
How do you analyze ranking data?
- Report the percentage who ranked each item first — the simplest, most intuitive headline.
- Use an average rank position per item (lower is better) to see the overall order, but remember the gaps between ranks aren't equal distances.
- Show a stacked bar of rank positions per item to reveal consensus versus disagreement.
- Watch base sizes if you compare segments; ranking data gets thin fast when sliced.
- Don't treat average rank as an interval score — it's ordinal, so a 1.5 vs 2.0 gap isn't a precise 'half a rank' of preference.
Building one is simple in most survey tools: in Formkii, the ranking question type lets respondents drag options into order on any device, and results export to CSV so you can compute first-choice percentages and average ranks in a spreadsheet.
Common mistakes
- Asking people to rank a list of 10+ items, which produces fatigue and random answers low in the list.
- Interpreting average rank as a precise score with equal spacing.
- Using ranking when you actually needed absolute importance — a rating would have answered the question.
- Forgetting to randomize the starting order, which can bias which items get dragged to the top.
Frequently asked questions
What is a ranking question in a survey?
A ranking question asks respondents to put a set of options in order of preference or importance, such as dragging features into the sequence that matters most to them. It's a forced-choice format that produces a relative ordering rather than an independent score for each item, which makes it good for revealing true priorities.
When should you use ranking instead of rating?
Use ranking when you need to force trade-offs and find priorities, especially if a rating scale produced everything at the top. Use rating when you need to know the absolute level of each item independently — how satisfied or how important — rather than just which item beats which.
How many items should a ranking question have?
Keep it short, ideally four to six items. Ranking long lists is tiring and the answers near the bottom become careless or random. If you have many options to prioritize, consider a MaxDiff approach or split the items into smaller, related groups.
How do you analyze ranking question results?
Report the share of respondents who ranked each item first as a clear headline, and use average rank position to see the overall order. Remember the data is ordinal, so the gaps between ranks aren't equal distances — show the distribution of rank positions rather than treating the average as a precise score.