The same p-value, different effect sizes
The two results below are entirely fictional. They assume a two-sided normal-approximation test against zero, known standard errors and the same measurement unit. This is not a study finding; it explains why a p-value alone does not describe effect magnitude.

1. Write the estimated difference and its measurement unit first. Do not call a number important without identifying what it compares.
2. Record the standard error and model. A standard error is not the same thing as the standard deviation of individual observations.
3. For a zero difference, calculate z = estimate / standard error. The same ratio can arise from estimates with very different magnitudes.
4. Read the approximate interval alongside the design and practical context. The normal approximation used here is not appropriate for every dataset or study design.

Fictional A: Difference 0.2 units; standard error 0.1. z = 2; two-sided normal p ≈ 0.0455.
Fictional B: Difference 2 units; standard error 1. z = 2; the same p ≈ 0.0455. Both estimate and uncertainty are ten times A’s.
Approximate 95% intervals: Estimate ± 1.96 × standard error: A [0.004, 0.396], B [0.04, 3.96] units. These are Wald intervals under the stated assumptions.
Interpretation limit: The same p does not mean the same effect or practical importance. Nor is p the probability that the null hypothesis is true.
Result card: Comparison __; unit __; estimate __; standard error __; model __; interval __; practical meaning __; limitations __.

Fictional A: ____________________
Fictional B: ____________________
Approximate 95% intervals: ____________________
Interpretation limit: ____________________
Result card: ____________________
The ASA’s 2016 statement separates effect size from statistical significance. This new exercise is not attributed to earlier PiVOLKA authors and is not a clinical decision tool.

ASA — statement on statistical significance and p-values, 2016 — https://www.amstat.org/asa/files/pdfs/p-valuestatement.pdf