Free preview

88 lessons

Essential Probability & Statistics for ML

Free preview

Essential Probability & Statistics for ML · 88 lessons

Master the fundamentals

No surprise gaps

Actually remember it

Skip what you know

One subscription. All learning paths included.

Our content is best on a larger screen

What is a random variable?

What a random variable is

Explanation

We often care about a number given by an experiment's outcome rather than the outcome itself, such as the total of two dice, how long a customer waits or a player's winnings.

Suppose a game pays £22 for each head in ten tosses. With events alone, describing the winnings needs eleven of them, "win £00", "win £22", up to "win £2020", and there is still no way to say the winnings are twice the number of heads. If we give the number of heads a name, HH, the winnings are 2H2H. A number given by the outcome in this way is called a random variable.

It is a third kind of object on a sample space, after outcomes and events.

Consider two coin tosses, with Ω={HH,HT,TH,TT}\Omega = \{HH, HT, TH, TT\}, and let XX be the number of heads. The diagram builds up each kind of object in turn, then runs the experiment.

Ω
HH
HT
TH
TT

Step 1: An outcome

The diagram draws each outcome as an oval inside Ω. An outcome is one element of Ω, one result of the experiment, such as HT.

Step 2: An event

An event is a set of outcomes. The event "exactly one head" is the set {HT,TH}.

Step 3: The probability P

P assigns each event a probability, so its one arrow starts from the whole set. For fair coins,

P({HT,TH})=12

Step 4: The random variable X

A random variable X assigns each outcome a number, so an arrow starts from every outcome. Here X counts heads.

Step 5: What is random?

Tossing the coins picks one outcome at random, with chances given by P. The arrows, the rule, never change.

Only the outcome is random. The value of X follows from it.
Toss the coins to pick an outcome.HH, so X=2HT, so X=1TH, so X=1TT, so X=0
Random variable

Definition

A random variable on a sample space Ω\Omega is a function X:Ω→RX: \Omega \to \mathbb{R}. It assigns each outcome ss one number X(s)X(s), the value of XX at ss. We write random variables with capital letters such as XX and YY.

A card's suit is not a number, but "11 if the card is a heart, 00 otherwise" assigns every card a number.

So a random variable is not an outcome, an event or a single number, but a rule that assigns a number to every outcome.

Example

A shop's records list every basket bought last week, and one basket is chosen at random. Which of these quantities are random variables?

  • The items in the basket.
  • The baskets that cost over £2020.
  • The total cost of the basket.
  • The number of items in the basket.

Solution

Each outcome is one of last week's baskets. For each quantity we ask what it gives each basket:

QuantityWhat it gives each basketRandom variable?
The items in the basketA list of goods, the outcome itselfNo
The baskets that cost over £2020A set of baskets, an eventNo
The total costOne number, its cost in poundsYes
The number of itemsOne number, its countYes

A random variable must give every basket one number, which is what the middle column checks. The total cost and the number of items vary because the basket is chosen at random: the rule is fixed, the basket is not.

Practice questions

4 questions

One card is drawn from a shuffled 5252-card deck. Which of these is a random variable?

Select the correct answer:

+ 3 more questions

The value of a random variable

Explanation

We've seen that a random variable assigns each outcome a number. The value X(s)X(s) is the number it assigns the outcome ss, and we find it by applying the rule to ss.

If XX counts the number of heads in two coin tosses, the outcome HTHT has one head, so X(HT)=1X(HT) = 1. Applying the rule to every outcome in the sample space Ω={HH,HT,TH,TT}\Omega = \{HH, HT, TH, TT\} in turn gives a table of the values of XX, with one row per outcome.

Several random variables can be defined on the same sample space. If YY counts the number of tails, its values form a second column of the same table:

Outcome ssNumber of heads, X(s)X(s)Number of tails, Y(s)Y(s)
HHHH2200
HTHT1111
THTH1111
TTTT0022

Example

A player tosses a coin three times, scoring 11 for each head and −1-1 for each tail. Give her total score WW on each outcome.

Solution

We start by listing the sample space. Each outcome records the three tosses in order, so

Ω={HHH,HHT,HTH,THH,HTT,THT,TTH,TTT}\Omega = \{HHH, HHT, HTH, THH, HTT, THT, TTH, TTT\}

Next we apply the rule to each outcome. For example, HHTHHT has two heads and one tail, so

W(HHT)=1+1+(−1)=1W(HHT) = 1 + 1 + (-1) = 1

Doing the same for every outcome gives the value of WW on each:

Outcome ssW(s)W(s)
HHHHHH33
HHTHHT11
HTHHTH11
THHTHH11
HTTHTT−1-1
THTTHT−1-1
TTHTTH−1-1
TTTTTT−3-3

Practice questions

4 questions

A die is rolled twice, and DD is the first number minus the second. Which of these is true?

Select the correct answer:

+ 3 more questions

Events described by a random variable

Explanation

Suppose we toss two fair coins, and let XX be the number of heads. How likely is exactly one head, that is, X=1X = 1?

It is tempting to write P(X)P(X) or P(1)P(1), but neither makes sense. PP gives probabilities to events, which are sets of outcomes. XX is a rule that gives each outcome a number, and 11 is a number, so neither is an event. We first have to find the event that "X=1X = 1" describes. The diagram shows how.

Ω
HH
HT
TH
TT
0
1
2
X

Step 1: How likely is X=1?

We want the probability that X=1, so we start from the value 1 on the line of X.

Step 2: Which outcomes give 1?

Two arrows end at 1. They start from HT and TH, the outcomes that X assigns the value 1.

Step 3: Gather the outcomes

Together they form an event, {HT,TH}. We write it {X=1}.

Step 4: Apply P

Now P can act: P(X=1) is the probability of the event {HT,TH}. For fair coins,

P(X=1)=P({HT,TH})=12
The event {X=x}\{X = x\}

Definition

For a random variable XX and a number xx, the event that XX takes the value xx is

{X=x}={s∈Ω:X(s)=x},\{X = x\} = \{s \in \Omega : X(s) = x\},

the set of outcomes that XX assigns the value xx. Lower-case xx stands for a number, usually one of the values of XX.

So {X=1}={HT,TH}\{X = 1\} = \{HT, TH\}. Its probability is written P(X=1)P(X = 1), without the inner braces, and read as "the probability that XX equals 11".

Inequalities work the same way. {X≥1}\{X \ge 1\} is the event of at least one head, {HH,HT,TH}\{HH, HT, TH\}, and its probability is written P(X≥1)P(X \ge 1).

Example

Two fair dice are rolled, and MM is the larger of the two numbers, or their shared value if they match. Find P(M=2)P(M = 2).

Solution

We label the dice, so each outcome is an ordered pair (a,b)(a, b) with die 11 showing aa and die 22 showing bb. The dice are fair, so the 3636 outcomes are equally likely.

We first list the event {M=2}\{M = 2\}, the outcomes that MM assigns the value 22. Neither die can show more than 22, and at least one must show 22, so

{M=2}={(1,2),(2,1),(2,2)}\{M = 2\} = \{(1, 2), (2, 1), (2, 2)\}

The event holds 33 of the 3636 equally likely outcomes, so by the naive definition of probability

P(M=2)=336=112P(M = 2) = \dfrac{3}{36} = \dfrac{1}{12}

So MM takes the value 22 with probability 112\dfrac{1}{12}.

Practice questions

4 questions

A coin is tossed three times, and XX is the number of heads. Which set is the event {X≥2}\{X \ge 2\}?

Select the correct answer:

+ 3 more questions