Nine statistics reporting problems and how to repair them

What to take away

• A percentage without a denominator is not ready to publish.
• A visible difference may be indistinguishable from sampling noise.
• Correlation alone cannot establish cause and effect.
• Rankings exaggerate small differences when uncertainty is hidden.
• Revised data require recalculation, not a quiet number swap.

Most numerical errors are not advanced mathematical failures. They are failures to define, compare, or qualify. The repairs below can be applied during reporting and editing.

1. Missing denominator

Problem: "Complaints rose 40 percent" appears without the old and new counts.

Repair: Give both values, period, covered population, and whether reporting rules changed. A rise from 5 to 7 is 40 percent but carries a different scale from a rise of thousands. The developing story context checklist makes redoing that arithmetic a standing item before publication.

2. Percent and percentage points mixed

Problem: A rate moves from 20 percent to 25 percent and the story calls it a 5 percent increase.

Repair: Write "up 5 percentage points, or 25 percent relative to the earlier rate."

3. Tiny subgroup treated as exact

Problem: A full-sample margin is attached to a subgroup with far fewer respondents.

Repair: Find subgroup sample size and design-based uncertainty. If precision is too weak, describe the result as inconclusive rather than forcing a ranking. Subgroup limits are covered step by step in reading a public opinion poll.

4. Point estimates turned into firm winners

Problem: Places are ranked 1 through 50 even though adjacent estimates overlap substantially.

Repair: Group estimates that cannot be distinguished, show uncertainty, or use ranges. Avoid "best" and "worst" when the evidence supports only broad tiers. The poll estimates comparison explains why adjacent point estimates so often overlap.

5. One-year movement called a trend

Problem: Two points define a trend without checking earlier years, revisions, seasonality, or method changes.

Repair: Plot a suitable series, mark breaks, and explain whether movement exceeds normal variation. Use "increased from last year" when that is all the evidence shows.

6. Correlation written as causation

Problem: Areas with higher Measure A also have higher Outcome B, so the article says A caused B.

Repair: Report the association and inspect study design, timing, confounding, reverse causation, and mechanism. NIST's scatter-plot guidance says an association in a plot cannot prove cause and effect. The NIST explanation of scatter plots and causality supports limiting the claim to an observed relationship unless other evidence establishes cause.

7. Chart scale drives the drama

Problem: A truncated vertical axis makes a small change look extreme, or uneven time intervals are spaced equally.

Repair: Choose a scale suited to the question, label it clearly, and show context. Include a zero baseline for bars because their lengths encode magnitude. A line chart may use a narrower range when readers can see the axis and the purpose is change, but test whether it distorts practical scale.

8. Summary statistic hides structure

Problem: An average is reported without examining distribution, outliers, or groups.

Repair: Plot the data and inspect median, spread, missingness, and subgroups. NIST describes graphical methods as tools for finding structure, testing assumptions, selecting models, spotting outliers, and identifying relationships. The NIST guide to the role of statistical graphics supports pairing numeric summaries with visual inspection rather than relying on one aggregate.

9. Revision handled as a cosmetic edit

Problem: A source revises the inputs, but the story swaps the number without checking derived percentages, rankings, charts, or conclusions.

Repair: Reproduce every dependent calculation, update graphics, compare the old and new finding, and add a correction if the earlier conclusion was materially wrong. Distribution-wide corrections follow the repair order in developing news briefing problems.

Repair sequence

1. Identify the source and data vintage.
2. Define population, measure, unit, and period.
3. Recalculate the claim.
4. inspect uncertainty and method changes.
5. Review chart and subgroup structure.
6. Narrow causal or ranking language.
7. Rewrite headline, summary, and correction note.

Common questions

Is a zero-based axis always required?

Bars generally need zero because length represents magnitude. Lines can use a narrower scale for change if the axis is clear and the design does not misstate practical size.

Can two data points show a change?

Yes. They show a difference between two periods. They are usually insufficient to establish a sustained trend.

Should every article include a chart?

No. Use a chart when it makes pattern, distribution, or comparison easier to see. A short table may be clearer for a few exact values.

What if revised data strengthen the original conclusion?

Update the values and revision note. Transparency applies even when the direction stays the same.