Free preview

74 lessons

Essential Probability & Statistics for ML

Free preview

Essential Probability & Statistics for ML · 74 lessons

Master the fundamentals

No surprise gaps

Actually remember it

Skip what you know

One subscription. All learning paths included.

Our content is best on a larger screen

Collisions: the birthday problem

The probability of no match

Explanation

The birthday problem is a famous probability puzzle: how many people must be in a room before it is more likely than not that two of them share a birthday? The answer surprises most people, and the method behind it applies well beyond birthdays.

We start with three people. How likely is it that no two of them share a birthday?

We take each person's birthday to be equally likely to fall on any of 365365 days, which is a close approximation to real birthdays.

An outcome lists the three birthdays in order, one per person, and a day can appear more than once. That is an ordered sample of size 33 from 365365 days with replacement, so

∣Ω∣=3653.|\Omega| = 365^3.

No match means the three birthdays are all different. The first person can have any of 365365 days, the second any of the 364364 left and the third any of the 363363 left. That is the count of ordered samples without replacement, so

P(no match)=365⋅364⋅3633653.P(\text{no match}) = \dfrac{365 \cdot 364 \cdot 363}{365^3}.

Nothing here depends on there being three people or 365365 days.

Probability of no match

Theorem

For kk people whose birthdays are equally likely to fall on any of mm days, with k≤mk \le m,

P(no match)=m(m−1)⋯(m−k+1)mk.P(\text{no match}) = \dfrac{m(m-1)\cdots(m-k+1)}{m^k}.

The same count applies whenever each of kk things is equally likely to land in any of mm places, such as people's birth months (m=12m = 12). In that general setting, a match is called a collision.

Example

Five people each pick a whole number from 11 to 2020 at random, every number equally likely. Let AA be the event that no two people pick the same number. Write P(A)P(A) as a ratio of counts.

Solution

Each person is equally likely to pick any number, so every outcome is equally likely and

P(A)=∣A∣∣Ω∣P(A) = \dfrac{\lvert A\rvert}{\lvert\Omega\rvert}

Here kk counts the people and mm the numbers:

  • k=5k = 5 people.
  • m=20m = 20 numbers.

An outcome lists the number each of the five people picks, in order, and a number can be picked more than once, so:

∣Ω∣=205|\Omega| = 20^5

The outcomes in AA are those where the five people pick five different numbers. The first person can pick any of the 2020 numbers, the second any of the 1919 left, and so on down to the fifth, who has 1616 left:

20⋅19⋅18⋅17⋅1620 \cdot 19 \cdot 18 \cdot 17 \cdot 16

Dividing the second count by the first:

P(A)=20⋅19⋅18⋅17⋅16205P(A) = \dfrac{20 \cdot 19 \cdot 18 \cdot 17 \cdot 16}{20^5}

Practice questions

4 questions

Four people are chosen at random, and each person's birthday is equally likely to fall on any of 365365 days. Which expression gives P(no shared birthday)P(\text{no shared birthday})?

Select the correct answer:

+ 3 more questions

A match with a given person

Explanation

We've seen how to find the probability that no two people in a group share a birthday. A different question is how likely it is that someone shares one particular person's birthday, say Ana's.

Picture Ana in an empty room, with the others arriving one at a time. For no one to share Ana's birthday, each new arrival only has to avoid Ana's day, so every one of them has 364364 of the 365365 days available.

Suppose the group has kk people: Ana and k−1k - 1 others. Ana's birthday is already set, so an outcome lists the birthdays of the k−1k - 1 others, and

P(no one shares Ana’s birthday)=364k−1365k−1=(364365)k−1.P(\text{no one shares Ana's birthday}) = \dfrac{364^{k-1}}{365^{k-1}} = \left(\dfrac{364}{365}\right)^{k-1}.

The bottom counts every outcome for the k−1k - 1 others, and the top counts the outcomes in which each of them avoids Ana's day.

By the complement strategy,

P(someone shares Ana’s birthday)=1−(364365)k−1.P(\text{someone shares Ana's birthday}) = 1 - \left(\dfrac{364}{365}\right)^{k-1}.

Nothing here depends on birthdays. Each person is a draw from 365365 days and Ana's is the first, so the same count works for any ordered sample.

A collision with the first draw

Theorem

In an ordered sample of size kk drawn with replacement from mm values, all samples equally likely, the probability that at least one of the later draws collides with the first is

P(a collision with the first draw)=1−(m−1m)k−1.P(\text{a collision with the first draw}) = 1 - \left(\dfrac{m-1}{m}\right)^{k-1}.

In words: each of the k−1k - 1 later draws has m−1m - 1 values out of mm that avoid the first draw's value, and a collision is the complement of all of them avoiding it.

Example

Leo and three friends compare birth months, each equally likely to be any of the 1212 months. Find the probability that at least one friend shares Leo's month.

Solution

Every outcome is equally likely, so we find the probability that no friend shares Leo's month by counting outcomes.

Leo's month is fixed, so an outcome lists the months of the three friends, each one of 1212:

∣Ω∣=123=1728|\Omega| = 12^3 = 1728

No friend shares Leo's month when each friend avoids that one month. Each friend has the same 1111 months left:

∣no friend shares Leo’s month∣=113=1331|\text{no friend shares Leo's month}| = 11^3 = 1331

Dividing the second count by the first:

P(no friend shares Leo’s month)=13311728P(\text{no friend shares Leo's month}) = \dfrac{1331}{1728}

By the complement strategy,

P(at least one friend shares Leo’s month)=1−13311728=3971728≈0.23.P(\text{at least one friend shares Leo's month}) = 1 - \dfrac{1331}{1728} = \dfrac{397}{1728} \approx 0.23.

Practice questions

4 questions

Ravi is in a room with 3030 other people, and each person's birthday is equally likely to fall on any of 365365 days. Which expression gives the probability that at least one of the others shares Ravi's birthday?

Select the correct answer:

+ 3 more questions

Any match versus a match with a given person

Explanation

Back to the puzzle: how many people does a room need before a shared birthday is more likely than not? More likely than not means a probability greater than 0.50.5, so we want the smallest number of people kk for which P(at least one match)>0.5P(\text{at least one match}) > 0.5.

How does a match between any two people compare with a match with Ana? By the complement strategy, P(at least one match)=1−P(no match)P(\text{at least one match}) = 1 - P(\text{no match}). The graph shows it in blue for each number of people kk, with a match with Ana in green.

102030405060700.250.50.751

Drag the red point k to change the number of people k.

k= 23 people
Any two people share a birthday: 0.507
Someone shares Ana's birthday: 0.059
A shared birthday is first more likely than not at k=23.

A match between any two people passes 0.50.5 at k=23k = 23. In the same group, the chance that someone shares Ana's birthday is only about 0.060.06.

Why the gap? To have no match with Ana, each new arrival only needs to avoid her one day, so every arrival has 364364 safe days. To have no match at all, each arrival must avoid every birthday already in the room, so the safe days shrink: 364364, then 363363, down to 343343 for the 2323rd person.

EventDays each person can have, in turnP(no match)P(\text{no match})
Any two people share a birthday365,364,363,…365, 364, 363, \dots365⋅364⋯(365−k+1)365k\dfrac{365 \cdot 364 \cdots (365 - k + 1)}{365^k}
Someone shares Ana's birthday364,364,364,…364, 364, 364, \dots(364365)k−1\left(\dfrac{364}{365}\right)^{k-1}
How these counts fit the sampling table

Details

Example

Zoe and four friends each spin a spinner with 1010 equal sections, numbered 00 to 99. Which is more likely: that at least one friend lands on Zoe's number, or that at least two of the five land on the same number?

Solution

For a match with Zoe, her number is fixed and each of the other 44 people has 99 numbers that avoid it:

P(at least one friend lands on Zoe’s number)=1−0.94=1−0.6561=0.3439P(\text{at least one friend lands on Zoe's number}) = 1 - 0.9^4 = 1 - 0.6561 = 0.3439

For a match between any two of the five, no match means five different numbers. The first person has 1010 numbers, the second the 99 left, and so on down to 66 for the fifth:

P(no match)=10⋅9⋅8⋅7⋅6105=30240100000=0.3024P(\text{no match}) = \dfrac{10 \cdot 9 \cdot 8 \cdot 7 \cdot 6}{10^5} = \dfrac{30240}{100000} = 0.3024

By the complement strategy,

P(at least one match)=1−0.3024=0.6976P(\text{at least one match}) = 1 - 0.3024 = 0.6976

A match between any two of the five is about twice as likely, because it can come from any of the (52)=10\binom{5}{2} = 10 pairs of people, not only the 44 pairs that include Zoe.

Practice questions

4 questions

A class has 2525 students, and each student's birthday is equally likely to fall on any of 365365 days. Which expression gives the probability that at least two students share a birthday?

Select the correct answer:

+ 3 more questions