How many responses do I need?
Graham uses this to recommend a target number of completed responses for each study. The goal is enough participants for stable, trustworthy groupings, without paying to recruit people you don't need.
Short answer
For most card sorts, aim for 20 to 30 completed responses.
- Below ~15, results get noisy and a few people can swing the groupings.
- Around 20 to 30 is the sweet spot, where groupings are stable and clear.
- Above ~30 you hit diminishing returns: the picture barely changes for the extra recruiting cost.
Why those numbers
Card sorting stabilizes with surprisingly few people:
- With about 15 participants, results correlate ~0.90 with what you'd get from a much larger group (Nielsen Norman Group guidance).
- At 20 to 30 participants, that correlation rises to ~0.93 to 0.98, then flattens (Tullis & Wood, 2004).
So 15 is a defensible floor for a quick, directional read; 30 is the practical ceiling for confidence on a single audience.
Adjust for the study
Start from the base and adjust:
| Situation | Recommendation |
|---|---|
| Open or hybrid sort (people make their own groups) | 20 to 30. Lean to 30 with a large card set (40+) or a varied audience. |
| Closed sort (validating your categories) | ~30, and up to ~50 for a high-stakes structure where you want extra confidence. |
| Quick, low-stakes, directional | 15 is enough. |
| Comparing audience segments (e.g. new vs. power users) | ~15 to 20 completed per segment. Multiply: 2 segments → ~30 to 40 total; 3 → ~45 to 60. |
Recruit more than your target
The target is completed responses. Some people will be screened out or drop off, so recruit a bigger pool:
- If your screener qualifies roughly half of people, recruit about 2× your target.
- Rule of thumb:
people to recruit ≈ target ÷ expected qualify-rate, then add ~10 to 15% for drop-off.
Example: target 30 completes, ~60% qualify → recruit ≈ 30 ÷ 0.6 ≈ 50, plus a buffer ≈ 55 to 60 invites.
Where to set it
Set the number in the study's Target responses field (see Creating a study). It drives percentage-based screener quotas and triggers the "study full" message once reached. Graham can suggest a value and explain the reasoning.
How to tell you've collected enough
Watch the within-group agreement in your results as responses come in. Once the agreement percentages and the main clusters stop shifting with each new batch, you've reached a stable picture, and more responses won't change the story.