Put the method between the headline and the conclusion
On your first pass, make just five notes: the research question, how participants were selected, the groups compared, the outcome measured and the period studied. If the text does not provide one of these, do not assume it has been established. Keep the authors’ interpretation separate from what the table actually shows.
A reading table
| Area | Question to ask |
|---|---|
| Design | Observation or intervention? How was the comparison made? |
| Sample | Who was included and who was left out? |
| Measurement | Was the concept in the claim actually measured? |
| Result | In what units, and with what uncertainty, is the effect reported? |
| Transparency | Are the plan, data, code and conflicts of interest disclosed? |
A statistical threshold is not a decision by itself
The ASA statement on p-values emphasises that statistical significance is not a measure of effect size or the scientific importance of a result. Do not simply note “a significant result was found”. Record the size of the effect, the units of measurement and the limits of the study as well.
Finish with three separate statements
Write what the study shows, what you infer from it and what you do not yet know. Explicitly identify any step that generalises to another population or period. Being cautious about a study is not the same as dismissing it.
Before reading a percentage: who is in the denominator?
Before interpreting a survey percentage, identify which people entered the calculation. Those invited, those who responded and those who answered a particular question may be different groups. The following numbers are entirely fictional reading-exercise data, not ELYADAL participants or results from a real study.
Two correct calculations, two different questions
Suppose 200 people known to be eligible were invited to a survey. Of these, 120 responded and all 120 answered the question concerned. Thirty chose option A and 90 chose other options. No response is available for 80 people.
| Calculation | What can be said? |
|---|---|
| 30 / 120 = 25% | A quarter of respondents selected option A. |
| 30 / 200 = 15% | Among the 200 invitees, 15% were observed to answer A. The preferences of nonrespondents are unknown. |
| 80 / 200 = 40% | In this simple example, no response was obtained from 40% of invitees. |
Do not treat a missing response as a negative answer
You cannot automatically place the 80 nonrespondents in a group “against option A”. There is insufficient information to calculate the A preference of everyone invited. Real surveys can also include people of unknown eligibility and partial interviews. Do not use this example’s 120/200 ratio as a universal AAPOR formula for every survey. AAPOR requires disposition codes and the appropriate definition to be identified separately. A response rate alone does not establish the presence or size of nonresponse bias.
Your own reading note
- Find the number of people stated alongside the percentage and the exact wording of the question.
- Define the denominator: all invitees, participants or only those answering this question?
- Record how missing, partial and unknown-eligibility records were handled.
- State the result only for the group supported. “25% of respondents” and “one in four people” are different claims.
- Keep unknowns on a separate line; do not fill them with assumptions.
This note does not score the overall quality of a study. Sampling, question wording and measurement error must be examined separately. Use the blank worksheet below to check how different denominators give the same count different meanings.
Download the worksheet and example ↓
Source: AAPOR · Standard Definitions.
Sources and provenance
Source check / web edition: 14 October 2026. Historical documents retain their own original dates.