Critical thinking: from a claim to a researchable question

Make the claim more specific first

“This method is very successful” sounds open to discussion, but it is not measurable. Which method, for whom, for which outcome and over what period? Rewrite the statement using these four questions. Make clear whether your disagreement concerns the meaning of a word, the quality of the data or the interpretation drawn from a result.

Four stages of review

1. Claim: Write one testable statement. Pay particular attention to expressions such as “always”.
2. Source: Follow the news report to the research, and the research to the data it uses. A widely shared post is not independent confirmation.
3. Method: Read how people were selected, how the comparison was made and what was measured.
4. Limit: Write down the question these data do not answer. Acknowledging uncertainty does not make a claim worthless; it corrects its scope.

Apply the same standard to yourself

Do not set a lower threshold of evidence for a result you like and a higher one for a result you dislike. Put observations that support and weaken the claim into two separate columns. Deciding what would change your mind before you begin is a useful working discipline.

Preregistration aims to distinguish analyses planned in advance from analyses developed later. Registration does not automatically establish that the results are correct. It makes the relationship between the plan and what was done open to inspection.

A five-minute exercise

Choose a news item you encountered today. Write its actual claim rather than its headline. Find the link to the first source. Then answer: “What other explanation could fit this result?” On the last line, state the limits of the available evidence rather than a definitive verdict.