What is probability? Chance, odds and the law of large numbers

Probability is a number between 0 and 1 that says how likely an event is: 0 means it cannot happen, 1 means it is certain, and 0.5 means it happens half the time. When every outcome is equally likely, the probability of an event is the number of outcomes that count as the event, divided by the number of outcomes there are. A fair die lands on 6 with probability 1 in 6, about 0.17.

In short

• Probability runs from 0 (impossible) to 1 (certain), and the chances of all the possible outcomes add up to exactly 1.
• Three rules do most of the work: "not" subtracts from 1, "and" multiplies for independent events, and "or" adds once the overlap is taken out.
• Over many trials the share of successes settles near the probability, but nothing pushes a short run back into line.

What does a probability actually mean?

A probability is a number attached to an event, an outcome or set of outcomes of something uncertain, such as a coin toss, tomorrow's weather or a medical test. It can be written as a fraction (1/6), a decimal (0.167) or a percentage (16.7%). Mathematicians write it P(event), so P(heads) = 0.5 for a fair coin.

There are two everyday ways to read that number. The frequency reading says a probability is the share of times the event happens if you repeat the situation many times: a fair coin comes up heads in about half of a long run of tosses. The belief reading says a probability measures how strongly the evidence supports a claim, which is how a forecaster can say there is a 70% chance of rain tomorrow even though tomorrow happens only once. The Stanford Encyclopedia of Philosophy sets out these and other readings. They agree on the arithmetic, which the Russian mathematician Andrey Kolmogorov put on firm axioms in 1933, and differ on what the numbers are about. Bayes' theorem holds under every reading, but the belief reading is the one Bayesian statistics builds on, using the theorem to update a probability as evidence arrives.

The subject started with gambling. In letters in 1654, Blaise Pascal and Pierre de Fermat worked out how to split the stakes of a dice game interrupted before it finished, and the method they found for counting outcomes is still the first one taught.

How do you work out a probability? The rules, on two dice

Roll two fair dice, one red and one blue. Each die has 6 faces, so there are 6 × 6 = 36 possible results, from (1, 1) to (6, 6), all equally likely. Every rule below can be checked by counting those 36.

Question	Rule used	Working	Answer
The total is 7	Count the outcomes	(1,6), (2,5), (3,4), (4,3), (5,2), (6,1): 6 of 36	6/36 = 1/6, about 0.167
No six on either die	"And" for independent events: multiply	5/6 × 5/6	25/36, about 0.694
At least one six	"Not": subtract from 1	1 − 25/36	11/36, about 0.306
Both dice show six	"And": multiply	1/6 × 1/6	1/36, about 0.028
A total of 7, or a double	"Or", no overlap: add	6/36 + 6/36	12/36 = 1/3
The red die is even, or the total is 7	"Or", with overlap: add, then take away the overlap	18/36 + 6/36 − 3/36	21/36, about 0.583

The "at least one" row shows the most useful trick in the subject. Counting every way to get at least one six is fiddly; counting the ways to get none and subtracting from 1 is quick. The last row shows the commonest mistake. Adding 18/36 and 6/36 counts three outcomes twice, (2,5), (4,3) and (6,1), which are both "red die even" and "total 7", so they are taken out once.

The "and" rule multiplies only when the events are independent, meaning one tells you nothing about the other. The two dice don't influence each other, so multiplying is safe. Drawing two cards from one deck without putting the first back is different: the chance the second is an ace depends on whether the first one was.

What is the difference between probability and odds?

Odds compare the chances for and against, where probability compares the chances for with all the chances there are. A die lands on 6 with probability 1/6. The odds of a 6 are 1 to 5, because 1 outcome wins and 5 lose; bookmakers would say 5 to 1 against.

To turn a probability p into odds, divide p by 1 − p. A probability of 0.2 gives odds of 0.2 to 0.8, or 1 to 4. To turn odds of a to b into a probability, divide a by a + b: odds of 3 to 1 mean a probability of 3/4. The two match closely for rare events and drift far apart for common ones. A probability of 0.9 is odds of 9 to 1.

What is the gambler's fallacy?

The gambler's fallacy is the belief that a run of one outcome makes the opposite more likely next time, as if chance kept a ledger and had to even it up. Coins and dice have no memory. After five heads in a row, a fair coin's next toss is still heads with probability 0.5, because the tosses are independent.

The name "Monte Carlo fallacy" comes from 18 August 1913, when a roulette wheel at the Monte Carlo Casino landed on black 26 times in a row, and gamblers lost heavily betting that red was "due" (Wikipedia). A run like that is very unlikely in advance, about 1 in 68 million for 26 of either color on a single-zero wheel. Once 25 blacks have happened, though, the 26th spin is just one more spin.

What is the law of large numbers?

The law of large numbers says that as a chance experiment is repeated, the share of times an event happens gets closer and closer to its probability. Jacob Bernoulli proved the first version, published in 1713, after more than 20 years of work, and Siméon Denis Poisson gave it its name in 1837 (Wikipedia).

How close is close? For a fair coin, the typical distance of the share of heads from 50% is 50% divided by the square root of the number of tosses. That gives this pattern:

Tosses	Typical gap from 50% in the share of heads	Typical gap from half in the number of heads
10	about 16 percentage points (3 or 7 heads is common)	about 1.6 heads
100	about 5 points (say, 45 or 55 heads)	about 5 heads
10,000	about 0.5 points (say, 4,950 or 5,050 heads)	about 50 heads

Someone actually did the 10,000. The mathematician John Kerrich, interned in occupied Denmark during the Second World War, tossed a coin 10,000 times with a fellow internee and got 5,067 heads, a share of 50.67% (Wikipedia).

Read the two right-hand columns together, because they correct the gambler's fallacy. The share of heads closes in on 50%, but the raw gap between the number of heads and exactly half the tosses tends to grow, from about 2 to about 50. The share converges not because tails "catch up" but because each early lead is swamped by the thousands of tosses that follow. Ten tosses can easily give 70% heads. Ten thousand tosses almost never do.

Where does probability lead?

Probability is the base of statistics: the spread of outcomes around an average is what standard deviation measures, and the bell shape that sums of many chance events take is the normal distribution. Scientists use it to ask how surprising a result would be by luck alone, which is what a p-value reports. Physics uses it at its foundations: in quantum mechanics, the theory predicts only the probabilities of measurement results.

Puzzles show how easily intuition goes wrong. In the Monty Hall problem, the right answer, switching wins two times in three, depends entirely on what the host knows, which is a lesson in how new information changes a probability.

Check yourself

• Without the table, explain why the chance of at least one six on two dice is 1 minus 25/36.
• How would you turn a probability of 0.25 into odds, and back again?
• Why does the share of heads settle near 50% even though tails never need to catch up?

Sources

• Interpretations of Probability — Stanford Encyclopedia of Philosophy
• Two Basic Rules of Probability, in Introductory Statistics 2e — OpenStax, Rice University
• Law of large numbers — Wikipedia
• Gambler's fallacy — Wikipedia
• John Edmund Kerrich — Wikipedia