Statistics and polls reporting: Explain uncertainty

What to take away

In statistics and polls reporting, numbers can sharpen an explanation, but precision on the page is not the same as certainty in the evidence. A poll result, administrative count, modeled estimate, and audited total come from different processes. In a developing story, the article must tell readers which kind they are seeing and what it can—and cannot—establish.

Begin with the question the number answers. Then examine how the value was produced and how far it can be compared.

Photo and credit

The historical photograph illustrates in-person enumeration. It does not represent every modern census or survey method.

Identify the number

Use this six-part label before drafting:

Field	Question

If any field is missing, pause before using the value in a headline. Use this label as the quick check; the statistical claim checklist provides a fuller edit pass.

Count, estimate, rate, and index

A count is a recorded total under stated rules. It can still miss cases or include duplicates. An estimate uses observed information to infer a value. A rate relates events to a population or exposure. An index tracks relative change from a reference level and may not represent a dollar amount or count.

Do not convert among them casually. A 10 percent increase in a rate is not a 10 percentage-point increase. An index rising from 120 to 126 is a 5 percent rise, not six percent just because the index gained six points. The poll estimates comparison does the same sorting for survey numbers, margins, averages, and forecasts.

Read the method before the result

For a survey, record sampling frame, sample size, response method, field dates, weighting, question wording, order, population, and reported uncertainty. For public opinion polls, the American Association for Public Opinion Research (AAPOR) transparency standards provide a disclosure check. For administrative data, record collection purpose, reporting rules, duplicates, lag, missing fields, and later revisions.

The Census Bureau's handbook for journalists describes using American Community Survey material for geographic and population comparisons and notes that published estimates carry margins of error. The ACS journalism handbook page supports treating uncertainty as part of the reported estimate. Its guidance is specific to ACS products rather than every dataset.

For administrative files, GAO's guide to assessing data reliability frames the review as testing whether data are accurate, complete, and applicable for the intended use, drawing on existing information and professional judgment. It is written for auditors, so a desk can borrow the questions without claiming an audit.

Explain uncertainty without a statistics lecture

Yes. Put the estimate and its main uncertainty in one plain-language sentence, then add only limitations that could change how readers interpret it. For example:

The survey estimates support at 48 percent, with a reported margin of sampling error of plus or minus 3 percentage points for the full sample.

Then add limits that matter. The full-sample margin may not apply to a subgroup. It does not cover every risk from nonresponse, coverage, question wording, weighting, or a turnout model. State the limitations that could affect this result.

Avoid saying two values are different merely because their point estimates are unequal. Check whether the source supplies a significance test or enough information to perform the appropriate comparison.

Compare on four dimensions

Before writing "rose," "fell," "highest," or "lowest," check:

Inflation can make dollar values from different years misleading. Boundary changes can alter local totals without matching population movement. A new questionnaire can break a trend. Comparisons against a stated baseline are also how government policy reporting measures results.

Separate association from cause

Two measures moving together does not establish that one caused the other. State the association, consider timing and alternative explanations, and look for a research design capable of supporting causal inference.

Use "was associated with" when that is what the study reports. Reserve "caused" for evidence and methods that support the claim.

Report revisions

Store the release date, vintage, revision status, and next scheduled update. When a revised value changes the article's conclusion, recalculate and correct the story.

Routine revision is not evidence of misconduct. It may reflect later responses, benchmark data, seasonal factors, or a documented method change. Handling updates in a live story follows the practice in covering developing news stories.

Common questions

Is a census free from error because it aims to count everyone?

No. Coverage, reporting, processing, definitions, and timing can still affect the result. Read the program's quality documentation.

What is the difference between percent and percentage points?

A move from 40 percent to 50 percent is 10 percentage points and a 25 percent increase relative to the starting value.

Should an article round estimates?

Yes, to a level consistent with their accuracy and the source's practice. Extra decimal places can imply unsupported precision.

Can a reporter average several polls?

Only with a stated method that addresses dates, populations, modes, sample overlap, and weighting. A simple average can hide meaningful differences.