Treat disagreement as methodological evidence
Different codes may expose ambiguous categories, inconsistent segment boundaries, missing context or competing theoretical readings. A log shows whether a problem belongs to one unusual passage or could affect the wider dataset. Simply replacing two answers with one shared value removes that diagnostic history.
Define both the coding unit and disagreement before coding begins. Under multiple coding, one researcher may apply two categories while another selects one. Two coders may choose similar meanings but mark different text spans. The record therefore keeps segmentation, assigned categories and rationale as distinct fields.
Capture reasons before discussing resolution
Each coder records a view before seeing the other’s conclusion. During discussion, both consult the segment, necessary surrounding context and current definition. They identify the textual cue behind the code and the plausible alternative they rejected. This creates a reasoned record rather than a memory of who persuaded whom.
Resolution may retain A, retain B, permit multiple codes, or classify the segment as unresolved. A brief reason is required. A project lead’s ruling is labelled adjudication, not mutual consensus. If a third reviewer decides, record their role, material available and decision basis.
Unresolved items can remain in a dedicated state when the methodology permits it. Forced certainty may obscure a legitimate interpretive boundary. Before analysis, explain how such items were handled and whether they were included. Never convert an unresolved code silently into missing data.
Separate case disposition from codebook revision
One difficult passage does not automatically justify a new category. First test whether the existing definition and anchor examples already resolve it. A substantive change creates a new codebook version with date, affected fields, rationale and responsible decision. The superseded version remains available.
The team then identifies previously coded material potentially affected by the revision. Applying a new definition only going forward can introduce inconsistency, while recoding the entire corpus may be unnecessary. Record the search rule and actual review scope. Do not claim a full retrospective check unless it occurred.
Anchor examples should avoid unnecessary personal information. An abstracted extract may preserve the distinction. If full context is methodologically necessary, restrict access and link it through a pseudonymous segment ID. Publication examples use authorised or appropriately anonymised passages.
Protect source data and decision records
A log can combine sensitive interview content, interpretation and identifiers. Include only the text anchor required for the decision. Names, contact details and rich biographical descriptions should not enter a broadly shared spreadsheet. Access and retention follow the project’s data-management plan and applicable privacy requirements.
Pseudonymisation is not anonymisation when a separate key enables reidentification. Protect that key and keep it outside the consensus log. An external reviewer does not automatically need the key or full transcript. Determine the minimum material and legitimate access route for the defined role.
The Research methods hub links related tools. The guide to creating a qualitative coding guide provides definitions and anchor examples. Exactly one foundation page is linked: transparent research foundations.
Use the log in reporting and audit
Before closing analysis, filter for unresolved cases, obsolete versions and missing rationales. Confirm that recorded rule changes were applied to the stated data scope. The methods section reports sampling, coding, disagreement handling and revision closely enough for readers to understand the logic without exposing confidential transcripts.
Do not select an agreement statistic after seeing which number looks favourable. If a coefficient belongs in the design, disclose unit, sample, missing-data treatment, multiple-code handling and computation. For interpretive approaches, a transparent account of disagreement may serve a different purpose from numerical agreement. Explain that choice rather than implying a universal threshold.
- Original codes and text anchors remain preserved.
- Every decision names the codebook version and rationale.
- Case resolution and general revision are distinct.
- Open cases are not silently overwritten.
- Personal detail is limited to what review needs.
- The report matches the work actually performed.
Archive the final log with the corresponding corpus and codebook versions. If a late change affects analysis, reopen only the identified scope and record it. This makes revision traceable without manufacturing certainty or agreement rules that the study never established.