Confirmation bias can enter before drafting begins
A literature set can make a proposition look stronger when contradictory or less striking findings are searched less often, judged more harshly or quietly dropped. This is not always deliberate. Search language, database coverage, access, language competence and prior expectations shape what becomes visible.
Predefined rules cannot remove every influence, but they introduce a reviewable constraint. They require relevance to be justified independently of the result. The record later shows whether supportive and conflicting studies received the same treatment. It also makes limits arising from time, access and language explicit.
Derive criteria from the question
Start with population or material, concept, context, dates, publication type and any design needed to answer the question. Give each criterion an operational test. “High quality” is vague; “reports participant selection and data collection sufficiently to identify the analysed sample” is assessable. Keep relevance screening separate from quality appraisal.
State how language, publication date, peer review, grey literature, preprints and unavailable full texts are handled. A language limit may be practical but is not a marker of inferior research. A date threshold needs a substantive reason, such as a technological change. “Does not support the thesis” is never a legitimate exclusion.
Specify order. At title/abstract stage, use only information visible there. Uncertainty advances to full-text review rather than becoming a convenient rejection. Reapply criteria to the full text. One primary reason prevents multiple counting, while a note may retain secondary observations.
Test decisions without viewing result direction
- Freeze criteria, codes and search date before screening.
- Remove duplicates using stable bibliographic fields.
- Screen title and abstract only for predefined relevance.
- Move uncertain records to the next stage.
- Give every full-text exclusion one primary reason.
- Compare a sample between two reviewers where feasible.
- Appraise included evidence independently of result direction.
A useful challenge temporarily hides result fields while a second person sees only question, population, design and criteria. A different selection decision is discussed against the protocol. This is not full blinding of a review, but it can reveal explanations written around a preferred result.
A small dissertation may have one screener. In that case, record borderline decisions in more detail and revisit them later. Do not claim independent duplicate screening when it did not occur. Transparent limits are more credible than an invented review process.
Permit amendments without rewriting history
A criterion may prove unusable: perhaps abstracts do not reveal a concept, or a date limit removes foundational theory. Amend it openly with date, reason, affected records and a decision about rescreening earlier exclusions. Preserve the original version rather than silently backdating the new one.
An amendment deserves scrutiny when it appears only after an unwanted result and removes just that study. Ask whether the same rule would exclude favourable evidence. If so, apply it consistently. If not, improve its methodological rationale or reject the amendment.
Separate access problems from substantive exclusions. “Full text not obtained” does not mean “study ineligible.” Explore lawful library access or author manuscripts and report remaining gaps. Never invent results for an inaccessible source from an abstract or secondary citation.
Report selection together with exclusions
The methods section identifies databases, search date, search logic, screening stages, criteria version, number of reviewers and conflict process. A flow diagram can report counts; the exclusion log explains consequential full-text decisions. Personal judgements and identifying notes do not belong in it.
Before closing, filter open records, absent codes and amended criteria. Sample included and excluded titles and reapply the current rule. Search specifically for conflicting evidence and document the terms or citation routes used to find it.
Learn how to document inclusion and exclusion criteria for consistent decisions. Then use the literature matrix to compare included studies. Find further templates in the research methods hub, and consult the evidence-review foundation for tracing findings.