“Not significant” does not mean “no effect”
A statistical test may provide insufficient evidence against a null hypothesis under its assumptions. That does not establish the null as true. Small samples, imprecise measurement, high variability or unsuitable design can leave broad uncertainty. Report effect or association estimates, intervals, data quantity and methodological limits rather than a binary label alone.
A result pointing in the expected direction may still be inconclusive. Conversely, a precise estimate can exclude effects of practical importance even when described as negative. Interpretation begins with the question the design can answer. An observational analysis does not automatically establish causality, and an absent difference in one sample does not govern all contexts.
Use precise sentences rather than proof claims
Do not use “no evidence for” to disguise a design that provided little information. Report precision and relevant effect sizes. Equivalence or non-inferiority questions require suitable pre-specified approaches; an ordinary non-significant result does not answer them automatically.
Keep all relevant outcomes visible
Outcome switching occurs when planned outcomes are replaced, reframed or omitted after results are known. Compare the report with proposal, registration or analysis plan. Changes may be methodologically necessary, but timing, reason and effect should be explicit. Report the original outcome too or explain why it could not be analysed.
Avoid spin. An unfavourable primary result does not become positive because one secondary analysis, unplanned subgroup or selective phrase looks promising. Such observations can motivate further work. Label them exploratory, list material variants and keep the principal question at the centre of the results.
Negative cases in qualitative research
Qualitative projects do not use the same significance logic but still encounter disconfirming evidence. A negative case does not fit an emerging account. Ask whether category, case boundary or theory needs revision. Record when it appeared and how previous coding was revisited. One counterexample does not mechanically invalidate every interpretation, but it cannot disappear because it complicates the narrative.
Report variation, minority perspectives and conditions in which a pattern was absent. Do not choose quotations purely for vividness. A case matrix can connect main pattern, negative case, context and effect on interpretation while avoiding details that enable re-identification.
Failure, interruption and unusable data
A technical failure or inadequate measure differs from a negative substantive result. If the instrument cannot assess the question, the conclusion is “not reliably testable under these conditions”, not “no effect”. Explain failure, diagnosis, affected data and attempted repair. Preserve originals and do not silently delete failed runs.
Stopping early may be ethically or operationally appropriate. State the rule, decision and consequence. Do not invent a success story after the event. A well-documented failure can still expose a methodological boundary without supporting broader effectiveness claims.
Final audit for a balanced account
- Can every pre-specified primary question be found?
- Do estimates and uncertainty accompany test decisions?
- Are exclusions and missing data fully explained?
- Are exploratory analyses unmistakably labelled?
- Are contrary findings and negative cases interpreted?
- Is the conclusion no broader than design and evidence?
- Does the literature discussion include credible disagreement?
Ask a knowledgeable reader to examine only the abstract and result tables, then state the main finding. If their account conflicts with the discussion, narrative emphasis may be distorted. Correct structure and wording, not the result.
Check the title and headings too: definitive labels can overrule cautious paragraphs. Add the population, period or condition where it materially bounds the finding.