Does a study's sample size support the headline before it runs?

What to take away

• Read the methods and tables before the abstract's conclusion.
• Reconstruct the research question in plain language.
• Check enrollment, exclusions, comparison groups, and outcome definitions.
• Distinguish a preprint version from a peer-reviewed publication.
• Write only the claim the reported data can carry.

The abstract is a map, not the full evidence. It compresses choices about who entered the study, what researchers measured, how they handled missing data, and which results received attention. A reporter should rebuild those choices before turning the paper into a public claim. The surrounding editorial rules live in the science and health reporting guide.

Photo and credit

The image documents an earlier laboratory setting. It is not evidence about any study discussed below.

Step 1: identify the paper

Record the title, authors, institution, funder, journal or repository, version date, identifier, and access date. Look for a correction, retraction, linked protocol, statistical plan, data supplement, and later journal version.

A preprint is a public manuscript, not a failed journal article. It may later change, remain unpublished, or appear in a journal. Use the version date in the story so readers know which document was reviewed.

The National Library of Medicine's page on the NIH Preprint Pilot defines preprints as complete public drafts of scientific documents not yet certified by peer review, and it describes adding eligible NIH-funded preprints to PubMed Central with that status displayed. The definition is the useful part here; indexing does not upgrade a manuscript's review status.

Step 2: rewrite the research question

Use a four-part sentence:

Among [population], did [exposure or intervention], compared with [alternative or baseline], change [defined outcome] over [period]?

If the paper does not provide one of those parts, the reporting should not silently supply it. A study of hospital patients cannot automatically answer what happens in the general population. A six-week outcome does not establish a lasting effect.

Step 3: name the design

Find whether researchers assigned an intervention, observed an exposure, sampled a population once, followed a cohort, analyzed existing records, combined prior studies, or tested a laboratory mechanism.

The National Institutes of Health overview of clinical study designs and their limits explains that randomized controlled trials assign participants by chance and can support causal conclusions when well conducted. It contrasts those trials with observational studies, which can identify correlations but cannot by themselves show that one factor caused another.

That distinction does not rank every trial above every observational study. Execution, measurement, missing data, sample size, and relevance still matter.

Step 4: trace the participants

Write down:

1. how participants were recruited;
2. inclusion and exclusion rules;
3. the number assessed, enrolled, assigned, followed, and analyzed;
4. differences between groups at the start;
5. withdrawals, missing measurements, and stated reasons.

A large starting sample can become a much smaller analysis. Report the number attached to the result, not only the enrollment number in the press material.

Step 5: inspect the comparison

Ask what the comparison group received or experienced. "Usual care" can vary by place and time. A placebo may look or feel different from the treatment. A historical comparison may reflect changes unrelated to the intervention.

For observational research, list the major factors that could influence both exposure and outcome. Then check which factors the analysis measured and adjusted for. Adjustment can reduce a measured imbalance; it cannot remove an unmeasured one.

Step 6: define the outcomes

Separate the primary outcome from secondary, exploratory, subgroup, and post hoc results. Check whether the paper reports a laboratory marker, symptom score, diagnosis, hospitalization, death, or a combined outcome.

A composite can combine events of very different consequence. Report its parts when the balance matters. If the outcome changed during the study, find the protocol explanation and date.

Step 7: read the numbers in pairs

For each main result, record both groups, the effect estimate, uncertainty interval, units, and period. Then ask whether the result is precise enough and large enough to matter in practice. The poll estimates comparison applies the same pairing to survey numbers: a point value travels with its interval.

Do not translate "not statistically significant" into "no effect." The estimate may be imprecise. Likewise, statistical significance does not tell readers whether the size is medically important. The distinction gets a fuller treatment in reporting statistics and polls.

Step 8: test the authors' conclusion

Compare the conclusion with the registered question, primary outcome, and actual estimates. Mark every step where the paper shifts population, changes from association to causation, or treats a surrogate measure as a patient benefit.

Read the limitations, then add limitations visible in the methods. Authors may identify several constraints without explaining which one most changes interpretation.

Step 9: report the publication status

State "preprint" near the first description of the result, not only in a final caveat. If a peer-reviewed version appears, compare authors, sample, outcome definitions, estimates, tables, and conclusions. Update the story if any material point changed. Version-to-version comparison is a records habit; the record version desk case shows it on a contract instead of a paper.

Common questions

Is peer review a guarantee that a paper is correct?

No. It is a review process, not a certificate of truth. Reporters still need to inspect the methods, data presentation, corrections, and later evidence.

Should the abstract supply the headline number?

Only after the number is matched to the table, outcome definition, population, comparison, and analysis set.

What if the full paper is unavailable?

Report only what the accessible record supports. Ask the authors for the paper and methods, and tell readers when the evidence could not be fully inspected.

How much of the limitations section belongs in the article?

Include the limits that materially change what the result means, who it applies to, or how confident readers should be.