What is evidence? How to tell what supports a claim

Evidence is anything that makes a claim more or less likely to be true: an observation, a measurement, a record, a test result, someone's testimony. Nothing is evidence on its own; it is always evidence for or against a particular claim. Its strength depends on one question: how much more likely is this observation if the claim is true than if it is false? Strong evidence is hard to explain any other way.

In short

• Something is evidence for a claim when the claim is more likely to be true given it than without it. Evidence comes in degrees, not as yes or no.
• Evidence is strong when it would be unlikely if the claim were false. Anecdotes are weak because they fit many explanations.
• Kinds of evidence have typical strengths, from a single story up to a review of many careful trials, but how well a study was done matters as much as its kind.

What makes something evidence?

A thing is evidence for a claim when it shifts the odds that the claim is true. The Stanford Encyclopedia of Philosophy puts the usual modern definition simply: "If E is evidence for some hypothesis H, then E makes it more likely that H is true." It also traces the idea back to something older, a "reliable sign, symptom, or mark": smoke as a sign of fire, a rash as a sign of measles.

Take a wet street in the morning as evidence that it rained overnight. If it rained, the street would almost surely be wet, say 95% of the time. If it didn't, the street might still be wet from a street-cleaning truck or a burst pipe, say 10% of the time. The wet street is 95 ÷ 10 = 9.5 times more likely if it rained than if it didn't, so it is fairly strong evidence of rain. Now take a dry street: it would happen only 5% of the time after rain and 90% of the time without, so it is strong evidence the other way.

What makes evidence strong is not how vivid or convincing it feels, but how badly the alternatives explain it. That ratio, how likely the evidence is under the claim compared with under its rivals, is the heart of Bayes' theorem, which gives the exact rule for updating a belief when evidence arrives.

What kinds of evidence are there, and how strong is each?

In medicine and the sciences that study people, kinds of evidence are often ranked by how well they rule out other explanations. This hierarchy of evidence puts reviews of many randomized trials near the top and expert opinion and single cases near the bottom.

Kind of evidence	What it is	What it can show	What it can't show
Anecdote or testimonial	One person's experience: "I took it and felt better"	That something can happen; ideas worth testing	Whether it happens often, or whether something else caused it
Expert opinion	What a specialist believes, based on experience	A best guess where no studies exist	More than the evidence the expert is drawing on
Observational study	Researchers record what people already do and what happens to them	Patterns across many people; links worth taking seriously	Cause, because people who choose something differ in other ways
Randomized controlled trial	People are assigned to a treatment or a comparison by chance	Cause, because chance makes the groups alike in everything else	Effects in people unlike those studied, or effects too rare for the trial's size
Systematic review	A search for every good study on a question, combined	The overall answer across many studies, and how consistent it is	More than the studies it combines: a review of weak studies is weak

The step from observational study to trial is the step from correlation to causation. If coffee drinkers live longer, it could be the coffee, or it could be that healthy people drink more coffee. Assigning coffee at random breaks that link.

A ranking is a rule of thumb, not a law. A large, careful observational study can tell you more than a small, sloppy trial, and some questions can't be put to a trial at all: no one can randomly assign people to smoke for 30 years. Critics of strict hierarchies, quoted in the same overview, argue that they give trials too much authority. The kind of study is a starting point; how well it was done, and how many independent studies agree, settle the rest.

How strong is the evidence on a real claim?

Here is one claim worked through: taking vitamin C prevents colds. It was popularized by the chemist Linus Pauling in 1970, and it is still widely believed.

1. Pin down the claim. "Prevents colds" could mean fewer colds, shorter ones or milder ones. Each is a different claim with different evidence.
2. Ask what kind of evidence is on offer. "I take vitamin C every day and never get sick" is an anecdote. The person might be lucky, healthy for other reasons, or forgetting the colds they had.
3. Look for the best summary. A Cochrane systematic review, published in 2013 by Harri Hemilä and Elizabeth Chalker, combined the randomized trials.
4. Read the main result. The review pooled 29 trial comparisons with 11,306 participants. In the trials of ordinary people, 10,708 of them, taking vitamin C every day made almost no difference to how often they caught a cold: the risk ratio was 0.97, where 1 would mean no difference at all.
5. Check for groups where it differs. In 5 trials of 598 marathon runners, skiers and soldiers under heavy physical stress, the risk was about halved: a risk ratio of 0.48.
6. Check the other versions of the claim. In people taking it daily, colds were 8% shorter in adults and 14% shorter in children. For an adult's week-long cold, 8% is about 13 hours. Starting vitamin C once a cold had begun showed no consistent effect.

The verdict is not "true" or "false" but something more useful: for most people, daily vitamin C does not prevent colds; it shortens them a little; it may cut the risk for people under extreme physical strain. Good evidence usually turns a yes-or-no question into "how much, for whom, and how sure are we?"

This explains how to weigh evidence, using vitamin C as an example. It isn't medical advice; ask a doctor or pharmacist about supplements.

What do people get wrong about evidence?

• "There's no evidence for it" means it's false. Not always. Absence of evidence counts against a claim only if someone looked properly, with a test able to find the effect if it were there. A small study that finds nothing is weak evidence of nothing.
• Anecdotes aren't evidence. They are, but weak evidence. A doctor's report of an odd reaction to a new drug can be the first warning of a real problem. The mistake is treating a story as if it settled how common or how caused something is.
• One strong study settles it. Single results can be flukes. Confidence grows when independent studies, done different ways, keep finding the same thing.
• Evidence and proof are the same. In mathematics a proof makes a claim certain. In science and daily life, evidence only makes a claim more or less likely, which is why science stays open to revision.

Is evidence in court the same thing?

The idea is the same, but the rules are stricter. In a trial, evidence is what the court allows to be presented, in the form of testimony, documents and physical objects, under rules about how it was collected and kept. It is then weighed against a standard of proof that depends on the case: in many legal systems, criminal cases demand proof beyond reasonable doubt, while civil cases ask which side's account is more likely, as the overview of evidence describes. A court must reach a decision by a deadline; science can wait for more evidence.

Where does the idea lead next?

Evidence is what tests a hypothesis, and a claim that no possible evidence could count against is the problem falsifiability names. The philosophical questions behind it, what it means to know something and how evidence justifies belief, are the subject of epistemology. For the everyday habit of asking for evidence before believing, see critical thinking.

Check yourself

• Without looking back, explain why a wet street is evidence of rain, and what would make it weaker evidence.
• Why does a randomized trial show cause when an observational study usually can't?
• What three things did the vitamin C evidence say, and why is that more useful than "true" or "false"?

Sources

• Evidence — Thomas Kelly, Stanford Encyclopedia of Philosophy, revised 2014
• Vitamin C for preventing and treating the common cold — Harri Hemilä and Elizabeth Chalker, Cochrane, 2013
• Hierarchy of evidence — Wikipedia
• Evidence — Wikipedia