Funding-method-transparency matrix
Investigate funding without making insinuations
Search the publication, project page, annual report and relevant official transparency register. Organisation-wide funding is not always the same as sponsorship of the report. Ask whether a funder commissioned the project, supplied data, set terms of reference or had an editorial role. If no project disclosure is available, report an information gap rather than claiming that influence occurred.
Interests can also arise from institutional mission, party links, advisory work, campaigning or authors' positions. A viewpoint is not in itself a methodological flaw. The important questions are whether it is visible, whether competing interests are declared and whether outsiders can inspect the data and reasoning. Transparent interests and opaque dependencies should not be collapsed into the same judgement.
Classify the publication before assessing its method
A “report” might be a short policy brief, economic model, commissioned survey, evidence review or opinion essay. Work out which it is. Commentary need not satisfy the design of a representative survey, but it should not be cited as though it had done so. For surveys, examine population, recruitment, wording, weighting and field dates. For modelling, inspect assumptions, scenarios, validation and sensitivity. For case studies, ask why cases were selected and how far findings can travel.
Trace headline numbers to primary data. If the authors reanalyse official statistics, distinguish the official observations from their transformations and interpretation. When code or data are unavailable, look for sufficiently detailed definitions and calculations. Attractive graphics, footnote density and confident executive summaries do not compensate for missing methods.
Use comparison to explain disagreement
Set the report beside independent research, official statistics and, where useful, policy work with a different perspective. Agreement is not proof, and disagreement is not a refutation. Identify whether studies use different concepts, populations, time windows, baselines or modelling assumptions. These differences often explain divergent headlines more productively than guessing at motives.
Media visibility, citation counts and organisational rankings mostly indicate attention or reputation. They are weak substitutes for assessing one research design. A small organisation can publish reproducible analysis; a prominent institution can produce a document irrelevant to your exact question. Keep the unit of assessment at the publication level.
Separate evidence, interpretation and advocacy
Label sentences in the executive summary as empirical findings, interpretation or recommendation. Verify each empirical statement against the method and results. Test interpretations for plausible alternatives and acknowledged uncertainty. Recommendations introduce values: which outcomes matter, whose costs count and what trade-offs are acceptable. A model can inform those choices without logically determining them.
Watch for causal language attached to descriptive comparisons, broad claims drawn from selected cases and percentages without a clear denominator. Check whether uncertainty present in the technical appendix disappears in the press release. Cite the most precise source section, not simply the most accessible headline.
Use the study at the right level
A transparent think tank report may be valuable for definitions, original surveys, scenarios, stakeholder positions or novel analysis of public data. Attribute it precisely: “Under the institute's stated assumptions, the model estimates …” conveys more information than “research proves …”. For contested policy conclusions, use peer-reviewed and official evidence to establish the wider landscape.
Record author, organisation, title, document type, publication date, version, URL and access date. Preserve the methods appendix you used. End your evaluation by naming the specific claim that can be carried and its limit, rather than branding the entire organisation credible or not credible.
Preserve an audit trail
Save the report, project page and disclosures visible at the time of use. In one record, capture authors, version, funder statement, data source, methodological appendix and known corrections. For a model, note the decisive assumptions and their page locations. Ask a second reader to test whether the headline claim follows from the table and design without asking whether they agree politically. Turn disagreement into a specific methodological question.
Weight omissions by consequence rather than counting them. An unexplained sampling frame can undermine an estimate; an unnamed design contractor probably cannot. Even a methodologically limited brief may reliably document the organisation's stated position. Your evaluation should define that narrower use instead of converting every weakness into wholesale rejection.