The Ulutas Medical Journal p-ISSN 2149-0430 e-ISSN 2149-388X

Common statistics mistakes in medical manuscripts

Last updated:

Many statistical problems in manuscripts are not advanced errors; they are basic choices that were never explained, or numbers that readers cannot check. This guide groups the most common problems under four headings and ends with a checklist based on the SAMPL guidelines (basic statistical reporting for biomedical journals), the ICMJE Recommendations and this journal's Instructions for Authors.

Choosing the wrong test

Every statistical test makes assumptions about the data. SAMPL asks authors to confirm that the data met the assumptions of each test, and gives three typical examples: skewed data should be analysed with non-parametric tests, paired data with paired tests, and relationships analysed with linear regression should actually be linear.

Typical errors in manuscripts:

  • Treating paired measurements as independent, for example comparing the same patients before and after treatment with an unpaired t test.
  • Using means and t tests for clearly skewed data such as length of stay or many laboratory values. SAMPL advises summarising non-normally distributed data with medians and interpercentile ranges or ranges.
  • Using the standard error to describe variability. SAMPL says not to use the standard error of the mean to show how spread out the data are; use the standard deviation, interpercentile range or range, and write "mean (SD)" rather than "mean ± SD".
  • Listing all the tests in one sentence ("t test, chi-square and Mann–Whitney tests were used") without saying which test was used for which comparison. SAMPL asks you to make this clear for each analysis.
  • Turning continuous variables into categories without explanation. SAMPL asks authors to report any such changes to the raw data, including collapsing continuous data into categories.

If you are unsure which test fits your design, ask a statistician before collecting data, not after the reviewers' comments arrive.

Multiple comparisons

Each test run at the 0.05 level carries a chance of a false-positive result. Bland and Altman showed in the BMJ that with 20 independent tests of true null hypotheses, the chance of at least one false "significant" result is about 64%. They also explain the Bonferroni method (dividing 0.05 by the number of tests) and its limits: it becomes too conservative when the tests are correlated, for example several outcomes measured in the same patients. SAMPL asks authors to state whether and how they adjusted for multiple comparisons, and to identify post-hoc analyses, including unplanned subgroup analyses, as exploratory. ICMJE similarly asks authors to distinguish prespecified from exploratory analyses.

In practice:

  • name one primary outcome in advance and base your main conclusion on it;
  • report how many comparisons you made, not only the ones with small P values;
  • if you adjusted, say which method and why, and remember that Bonferroni is not suitable for every design;
  • label subgroup findings as hypothesis-generating rather than as proof.

The ASA statement on P values reminds authors that proper inference requires full reporting and transparency, which rules out reporting only the analyses that reached significance.

Percentages without denominators

"Complications occurred in 25% of patients" is impossible to judge: is that 1 of 4 or 250 of 1,000? SAMPL asks authors to report numerators and denominators for all percentages, and to report total sample and group sizes for each analysis. The EASE guidelines for authors make the same point in plain language: when you report percentages, make clear what counts as 100%.

Related problems:

  • Too many decimals. SAMPL suggests rounding as much as is reasonable; a percentage based on 20 patients does not need two decimal places.
  • Denominators that change silently because of missing data. If 12 patients had no follow-up, say so and give the denominator actually used.
  • Rates without a time period or population unit. SAMPL asks you to name the type of rate or ratio, the time period it covers and any multiplier (for example "per 10,000").

A good format is "18/72 (25%)" in tables and "18 of 72 patients (25%)" in the text.

Reporting checklist

Before you submit, check your Statistical Analysis subsection and Results against this list:

  • Every method used is described, with enough detail for a knowledgeable reader with the data to check the results (ICMJE).
  • The statistical software is named, with its version (ICMJE; this journal).
  • The significance level is stated, and whether tests were one- or two-tailed (SAMPL; this journal).
  • Assumptions of each test were checked, and the test used for each comparison is clear (SAMPL).
  • Normally distributed data are given as mean (SD); skewed data as median with interpercentile range or range (SAMPL).
  • All percentages have numerators and denominators; group sizes are given for each analysis (SAMPL).
  • Main results give an effect size with a 95% confidence interval and an exact P value, not only "P<0.05" (ICMJE; SAMPL; this journal).
  • Adjustments for multiple comparisons are described, and exploratory analyses are labelled as such (SAMPL; ICMJE).
  • Missing data and their handling are reported (STROBE for observational studies).
  • The relevant EQUATOR reporting guideline (CONSORT, STROBE, STARD, PRISMA or CARE) has been followed, as the journal requires.

Our guide on P values and confidence intervals explains how to interpret and word these results.

Sources

All guides

WhatsApp Tell the Editor
Submit manuscript Contact