How we keep research participants real, honest and paying attention

Research is only as good as the people in it. A study full of fake accounts, people who say whatever gets them in, or answers written by a chatbot will give you confident conclusions that happen to be wrong. This post explains every check we run to make sure the participants in your UsersArabia study are real, honest and paying attention.

The problem is getting harder

Research panels have always had to deal with three kinds of bad data:

• Fake or duplicate people. Invented identities, or one person running several accounts to collect more incentives.
• People who bend the truth to qualify. Often called professional respondents, they learn what screeners are looking for and answer accordingly.
• Low-effort or automated answers. Rushing through, pasting generic text, or handing the whole study to an AI tool.

The last one has changed fast. A peer-reviewed study in PNAS by Sean Westwood at Dartmouth built an autonomous AI respondent and ran it through 6,000 standard attention checks. It passed 99.8% of them. The trick questions most panels rely on no longer catch the thing they were designed to catch.

So we don't rely on any single check. We layer several, each aimed at a different threat, and we look at how they add up.

Three principles behind everything

• Low friction for honest people. Nothing here gets in the way of a genuine participant. There are no extra hoops at sign-up, and verification is optional.
• Flag, don't auto-ban. Every signal feeds a flag that a member of our team reviews. No algorithm removes anyone on its own.
• Signals add up. One odd signal is usually noise. Someone on holiday connects from a different country. Several signals together are a pattern worth a closer look.

Every study is reviewed before it goes live

Trust works both ways. Before any study is shown to participants, our team reviews it. Studies that aren't ready go back to the researcher with a note, and any credits reserved for incentives are returned. Participants only ever see studies a person has checked.

Optional ID verification, powered by Didit

Participants can verify their identity once, in about a minute, with a photo of their ID or passport. We offer it at the moment it matters, when someone applies to a study, and from their profile. It's never required, and choosing to skip it never stops anyone from applying.

Verified participants get a Verified badge, and they're shown first in the applicant list for every study. Researchers can also filter applicants to verified only, and by the nationality shown on the document.

Why we chose Didit

We looked for a verification partner that fits this region rather than one built for somewhere else. We chose Didit because:

• It reads the documents people here actually carry. Didit supports the Saudi national ID, the Iqama, GCC ID cards from the UAE, Qatar, Bahrain, Kuwait and Oman, and passports, in one flow. It covers more than 220 countries, so expat participants are covered too.
• Your ID stays with them, not us. The ID photo is handled by Didit. We never receive or store it. We keep only the result, the document type, the issuing country, the nationality, the year of birth, and a coded version of the document number that can't be turned back into the number.
• Security is independently checked. Didit reports SOC 2 Type II and ISO/IEC 27001 certification, and GDPR compliance.
• It's quick. Checks come back in seconds, so participants aren't left waiting.
• It lets us keep verification free for participants. Clear, per-check pricing with a free monthly allowance means we can offer it to everyone rather than charging for it.

We deliberately ask for a photo of the ID and not a selfie. Camera steps put many people off, and verification only helps if people are willing to do it.

Every result Didit sends us is signed, and we check the signature before trusting it, so nobody can fake a "verified" result.

One person, one account

Verification also lets us stop the same person from running several accounts:

• One ID, one account. Each ID document can be verified on one UsersArabia account only. If a second account tries to verify with the same document, it's held for review and both accounts are flagged. Neither shows the Verified badge until our team has looked.
• Shared devices. Didit tells us when the same device was used to verify another account. Families do share phones, so this is a flag for review rather than a block.
• Hidden connections. We flag verification through a VPN, Tor or a data-centre connection, which can hide where someone really is.
• Country checks. If someone verifies with a residence permit from one country while connecting from another, we take a look. We don't apply this to passports, because many people here live outside the country that issued theirs.
• Clusters. When several accounts verify from the same internet connection, we check them together. Mobile networks in the region share connections between many people, so a single match means nothing on its own.

Answers that add up

Professional respondents are hard to catch with any single question. They're much easier to catch over time.

Participant profiles use structured answers: country of residence, nationality, age, employment, occupation, industry, education, household and the devices people use. When a researcher writes a multiple-choice screener question, they can link it to a profile field. From then on we check automatically:

• Screener against profile. If the profile says teacher and the screener answer says nurse, we flag it.
• Answers across studies. If someone gave a different nationality or gender in an earlier screener, we flag it. For things that genuinely change, like a job, we only flag repeated switching.
• Profile against ID. For verified participants, we compare the nationality and age they give with their document. A mismatch here is serious, and removes the Verified badge until it's resolved.
• Profile changes. We keep a history of profile changes. Someone who changes the same field again and again is flagged.

People do change jobs and move cities, so none of this rejects anyone automatically. It tells our team where to look.

Limits on how often people take part

A panel where the same few people take every study gives you the same few opinions. By default, a participant can take part in up to five studies in 30 days, and must wait two weeks between two studies of the same type. If someone reaches a limit, we tell them the date they can apply again rather than leaving them guessing.

A quality score for every hosted response

For screeners and surveys that run on UsersArabia, we look at how the answers were given, not only what was answered. Each response gets a quality score from 0 to 100, with the reasons behind it in plain English. We look for:

• Speeding. Finishing in under a third of the typical time for that study, once it has enough responses to know what typical is, or faster than the questions could be read. We measure the time on our servers, so it can't be faked.
• Rushing. Answering most questions faster than they could be read.
• Pasted answers. Open answers that were mostly pasted in rather than typed.
• Text that wasn't typed. Answers that appear without being typed or pasted, which suggests a script filled them in.
• Machine-like typing. A perfectly even typing rhythm that no person produces.
• Leaving and pasting. Switching to another tab, then pasting an answer on return. It's a common sign of copying from an AI tool.
• Straight-lining. Giving the same rating to every scale question.

Researchers see the score and reasons on each applicant. Low scores are also flagged to our team.

This is about how answers are given. We record timing and counts, like how long a question took and how much text was pasted. We never record what people type, keystroke by keystroke.

Reputation that follows participants

When a study is completed, researchers can rate the participant from one to five stars and add a private note. Those ratings build into a reputation that other researchers see when that person applies to their study. Participants who show up prepared and give thoughtful answers stand out, and people who don't are easy to spot. Participants can see their own rating too, which is a good nudge to keep doing their best.

People make the final call

All of these signals feed one review queue that our team works through, most serious first. For each flag we can clear it as a false alarm, act on it, or pause or remove a participant from the panel. Every decision is logged. Participants can email us if they think we've got something wrong, and we'll take another look.

Incentives are paid for genuine participation. If we find that someone has broken our terms, for example by running duplicate accounts, we can withhold payment for participation that wasn't genuine.

What this means for you

If you're a researcher, you get participants who are more likely to be who they say they are, answers you can check at a glance, and a team behind the scenes keeping the panel clean. You can prioritise verified participants, filter by verified nationality, and use the quality score to decide which responses to trust.

If you're a participant, doing the right thing is rewarded. Verify once, keep your profile honest and take your time, and you'll be shown first to the researchers choosing who takes part.

We'll keep adding to this as the ways people try to game research change. If you'd like to see how it works for your next study, get in touch.