Science and health reporting problems: practical fixes

What to take away

Every week a new study becomes a news story, and the same errors recur when it is explained to readers. Many science and health reporting problems are not arithmetic mistakes. They happen when a reporter changes the type of claim while compressing it: association becomes cause, a selected sample becomes "people," a laboratory response becomes a treatment, or a modeled scenario becomes a forecast.

These fixes work on two layers at once: the research notes behind a claim and the sentence readers finally receive.

Problem: the release becomes the evidence

A news release selects findings, quotes, and framing for an institution. It may be accurate, but it is not a substitute for the paper, protocol, tables, supplements, or regulator record.

Fix: build a claim ledger with three columns: the release claim, the underlying evidence, and the narrower statement the evidence supports. Copy the release claim, locate the underlying evidence, and write the narrower statement. Mark any claim that cannot be traced. The three columns come from the health and science claim checklist.

Problem: the comparison disappears

"Risk fell by 30 percent" is incomplete without the two rates, population, and period; the statistical claim checklist lists the other checks. "Patients improved" is incomplete without what happened under usual care, placebo, another treatment, or the prior baseline.

Fix: draft the result as a comparison before shortening it. If the source does not give the second quantity, ask for it or describe the missing comparison.

Problem: the causal verb grows during editing

An observational paper says "associated with." A news release says "linked to." The headline says "drives." Each step sounds sharper, but the final version makes a claim the design may not support.

Fix: keep a design-to-verb note beside the headline field. Require a methods editor to approve "caused," "prevented," or "led to" when the source did not use causal language backed by the design. The verb-to-design pairing sits in the scientific evidence comparison.

Observational research also has its own reporting guideline. The STROBE statement for observational studies sets out how such research is meant to be reported, which gives a reporter a fixed standard to hold a verb against.

Problem: the denominator changes

The story compares positive tests with all residents, adverse-event reports with doses, or survey respondents with registered users. That can create a rate whose parts do not describe the same population.

Fix: label every number with its unit in the notes. Write "47 reports," "47 people," or "47 visits," not just "47." Verify that numerator and denominator cover the same place, period, eligibility rules, and data system. Mismatched denominators also open the list of statistics reporting problems.

Problem: a surrogate replaces a patient outcome

A biomarker may help researchers study a disease or predict risk. A change in that marker is not automatically a change in symptoms, disability, hospitalization, or survival.

Fix: name the measured outcome first. Then explain why researchers use it and whether clinical benefit has been established separately.

Problem: outside comment is ornamental

A quote from a prominent scientist can look independent while adding little methodological scrutiny. The expert may work outside the relevant specialty, share an institutional relationship, or have a financial or advocacy interest.

Fix: ask the commentator to explain one design strength, one limit that changes interpretation, and the exact claim the data cannot support. Record relevant relationships and describe independence narrowly.

A 2017 CMAJ study of expertise and conflicts reviewed 104 comments by independent experts in news stories about studies from seven leading medical journals. About a quarter of the commenters had no relevant academic or clinical expertise, and conflicts of interest were common but seldom disclosed. That supports checking both qualities, but it does not justify assuming that any individual expert is conflicted or unqualified.

Problem: uncertainty is saved for the last paragraph

By the time a reader reaches a caveat, the headline and lead may have established a stronger conclusion. The correction has less force than the claim.

Fix: place the main boundary beside the result. Examples include "in mice," "in a preprint," "among 82 patients," or "under the model's central scenario." Keep the final limitations section for detail rather than first disclosure. Status-first labeling follows how to read a study or preprint, which puts review state beside the first result.

A worked example with an invented study

The study below is invented to show the fixes together. Suppose a press release says: "Coffee drinkers have 30 percent lower heart risk." The paper is an observational study of 4,000 adults followed for five years, and it has not yet been peer reviewed.

The corrected line reads: "In an unreviewed observational study of 4,000 adults, coffee drinkers had fewer heart events than non-drinkers, but the design cannot show coffee was the reason."

Problem: disagreement becomes a two-sided contest

Two quotes can create symmetry where the evidence is uneven. A credential alone does not show how much support a position has.

Fix: explain the basis of each claim. Compare study quality, relevant data, institutional review, and uncertainty. Do not count quotations as votes.

Problem: the update hides the original error

Quietly changing a number may fix the page while leaving syndicated copies, screenshots, and reader understanding untouched.

Fix: add a dated note when the correction changes meaning. State the old error, the correct information, and which parts of the story changed. Notify distribution partners when the error traveled beyond the page.

Common questions

Should a reporter ignore news releases?

No. They can point to papers, contacts, and institutional context. Treat them as leads and claims to verify.

Is one independent expert enough?

It depends on the claim and the available expertise. One suitable expert can expose a clear limit, while a disputed or cross-disciplinary question may require more review.

Can a small study be newsworthy?

Yes. Describe it as early or limited evidence, state the sample and design, and avoid a population-wide conclusion.

When does a correction need a note?

Use a note when the change affects meaning, magnitude, attribution, advice, or the reader's understanding of the evidence.