Screening questions that actually filter
August 17, 2026
A screener has one job: let the right people through and keep the wrong people out. Most screeners fail at the second half, and the cost is invisible until you are halfway through analysis wondering why the findings feel thin.
This is a practical guide to writing screeners that hold up.
Why screeners leak
Any paid study creates an incentive to qualify. That is not dishonesty so much as ordinary human optimism: people round up, they interpret generously, and they notice which answer the question seems to want.
A leaky screener has recognisable symptoms. Participants who match on paper but have nothing to say. Sessions where the person clearly has not done the thing they claimed. Findings that contradict everything else you know.
Almost always, the questions were the problem.
The single biggest fix: stop telegraphing
Compare these two.
Weak: Do you manage a marketing budget of more than fifty thousand dollars per year?
Better: Roughly what annual marketing budget do you have direct sign-off on?
- No budget authority
- Under 10,000
- 10,000 to 49,999
- 50,000 to 249,999
- 250,000 or more
The first question announces the answer you want. The second requires the person to place themselves on a scale without knowing which band qualifies.
Any question phrased as "do you do X" where X is obviously the study topic will over-recruit. Rewrite it as a quantity, a frequency, or a selection from a list.
Use decoys in list questions
When you need users of a particular product or service, never list it alone.
Weak: Do you use a project management tool?
Better: Which of these have you used for work in the last month? Select all that apply.
- A named competitor
- Your target product
- Another named competitor
- A generic spreadsheet
- None of these
Two things happen. You stop signalling the right answer, and you learn something useful about the competitive set even from people who do not qualify.
Include at least one plausible option that nobody could genuinely be using, and treat selecting it as a disqualification. It quietly catches people ticking everything.
Ask the same thing twice
Place two questions that depend on the same underlying fact far apart in the screener, phrased differently. If someone says they switched banks in the last three months, and later gives a date range that contradicts it, that is a clean disqualification.
You do not need many of these. One well placed consistency check catches most careless answering.
Include one open question
A short free text question is the strongest anti-fraud tool available, and it costs the participant thirty seconds.
Good prompts:
- What is the most frustrating part of doing X in your role?
- Describe the last time you did X. What happened?
Someone who genuinely does the thing writes two specific sentences. Someone who does not writes something vague and generic. You can tell the difference in about three seconds of reading.
This also gives you an early read on how articulate a participant is, which matters if you are recruiting for interviews.
Separate must-have from nice-to-have
Every hard criterion multiplies your recruitment difficulty. Before finalising, sort each question into:
- Disqualifying. Wrong answer means they cannot take part.
- Quota. You want a mix, so track it but do not exclude on it.
- Context. Useful to know when reading their session, not a filter.
Teams routinely make criteria disqualifying when a quota would have done, then wonder why nobody qualifies. Age and gender are usually quotas. Category behaviour is usually disqualifying.
Keep it short
A screener over about eight questions starts losing good participants to fatigue. The people most likely to abandon a long screener are busy professionals, who are often exactly the people you are trying to reach.
If you cannot get under eight, your criteria list is too long. Go back to the previous section.
Rewrites worth stealing
Weak: Are you interested in fitness? Better: How many times did you exercise in the last seven days? (numeric)
Weak: Do you shop online frequently? Better: How many separate online orders did you place in the last thirty days? (numeric)
Weak: Are you a decision maker? Better: For a purchase of this type in your organisation, what is your role? (final approval, recommend, contribute, no involvement)
Weak: Do you speak Arabic and English? Better: Rate your comfort conducting a work conversation in each language. (per language, four point scale)
The pattern is consistent. Replace the yes-or-no with a number, a frequency, or a position on a scale.
Read the answers before you accept
The screener is a filter, not a decision. On UsersArabia you see every applicant's screener answers, along with their demographic profile and reputation from previous studies, and you accept the ones you want. That final human read catches what no question can.