Every study lives or dies on who takes it. Recruit the wrong people and it won't matter how good your questions are or how sharp your analysis is. You'll have a clean, rigorous study of the wrong population, which is worse than no study at all because it looks trustworthy.
The screener is the gate that decides who gets in. A good one filters for participants whose behavior matches your real users, without tipping them off to the "right" answers and without quietly skewing your sample. A bad one lets in professional survey-takers and people guessing their way past your filters. Here's how to write the good kind.
Screen for behavior, not just demographics
The most common screener mistake is leaning on demographics. Age, location, and job title are easy to ask, but they often have little to do with whether someone actually uses a product like yours. What you usually want is people with the right attitudes and behaviors.
Ask what people do, not just who they are. "How many times in the last month did you order groceries online?" tells you far more about fit for a grocery-app study than someone's age bracket. Behavioral screening gives you participants who relate to your product the way your real users do, which is the entire point. Use demographics to round out the sample, not as the primary filter.
Don't reveal what you are looking for
If participants can guess which answer gets them in, some of them will give it to you whether it's true or not, especially when an incentive is on the line. Two defenses:
- Keep the study's purpose ambiguous. Don't announce "we're looking for frequent users of budgeting apps" and then ask if they use budgeting apps frequently. You just told them the password.
- Hide the qualifying answer in a list. Surround the answer you want with plausible alternatives so it doesn't stand out. A question where the "right" option is obvious isn't a filter.
Avoid binary questions
A yes or no question gives an unqualified person a 50% chance of slipping through by guessing, which erodes the credibility of your whole sample. Use multiple-choice with several plausible options instead. It's harder to game, and it lets you capture specific behaviors and frequencies that a binary can't. Multiple-choice also evaluates instantly, which matters for unmoderated recruiting at scale.
Never use leading questions
A screener is not the place to validate your assumptions. Leading questions push people toward an answer and hand you either skewed data or the wrong participants. "How much do you enjoy using mobile banking apps?" presumes enjoyment and filters for agreeable people. Ask neutral questions about actual behavior and let the answers sort people honestly. This is the same discipline that protects the study itself, covered in how to conduct effective user research.
Eliminate conflicts of interest
People with industry insider knowledge skew results. Screen out participants who work, or whose close family works, in your industry or with competing products, unless that's specifically who you want. An insider reacts to your product with knowledge a real user wouldn't have, which contaminates the findings.
Keep it short and front-load the deal-breakers
Two practical rules:
- Five questions, maximum. Long screeners cause drop-off, and a participant who quits halfway is wasted recruiting spend. Ask only what you need to decide eligibility.
- Put elimination questions first. Ask the hard disqualifiers at the very start so unqualified people exit quickly, out of respect for their time and yours. Don't make someone answer four questions before the one that screens them out.
Test the screener before you launch
Before you open recruiting, run the screener against people you know you want and people you know you don't, and confirm each group lands in the right bin. If a known-bad tester qualifies, your filter has a hole. This quick check catches problems while they're cheap to fix, exactly like piloting a usability test.
It's also worth getting internal stakeholders to sign off on the screener up front. It tightens the criteria, and it means no one can wave away the findings later by questioning who you talked to.
A quick screener checklist
Before launch, confirm your screener:
- Filters on behavior and attitudes, not just demographics.
- Hides which answers qualify.
- Uses multiple-choice, not binary, for the filtering questions.
- Contains no leading questions.
- Screens out conflicts of interest.
- Is five questions or fewer, with disqualifiers first.
- Has been tested against known-good and known-bad profiles.
- Has stakeholder sign-off.
Where this fits at User Evaluation
User Evaluation lets you screen participants before they reach an AI-moderated interview, so the people who make it into your study match the audience you actually care about. Clean screening up front is what makes the depth and scale of automated interviews worth having, since even the best interview of the wrong person tells you nothing useful. For getting those participants quickly, see recruiting without a panel.
What this comes down to
A screener is the most leveraged few minutes in a study, because it decides whether everything downstream is built on the right people. Filter for behavior, hide the qualifying answers, avoid binary and leading questions, screen out insiders, keep it short with deal-breakers first, and test it before you launch. Get the gate right and the rest of your research is finally measuring what you think it's measuring.