27  Introduction to Probability

Probability is the branch of mathematics that quantifies uncertainty. It provides the foundation on which all of inferential statistics is built: every confidence interval, hypothesis test, and predictive model ultimately rests on probability theory. This chapter introduces the basic language and rules of probability that later chapters on distributions, estimation, and hypothesis testing depend on.

27.1 Basic Terminology

• Random Experiment: A process whose outcome cannot be predicted with certainty, such as tossing a coin or observing whether a customer completes a purchase.
• Sample Space (\(S\)): The set of all possible outcomes of a random experiment. Tossing a coin has sample space \(S = \{Heads, Tails\}\).
• Event: Any subset of the sample space — one outcome or a combination of outcomes that is of interest.
• Probability: A number between 0 and 1 that expresses how likely an event is to occur; 0 means impossible, 1 means certain.

27.2 Approaches to Probability

Classical (Theoretical) Probability

Based on equally likely outcomes: \[P(E) = \frac{\text{Number of favourable outcomes}}{\text{Total number of possible outcomes}}\] Example: The probability of rolling a 4 on a fair six-sided die is \(1/6\).

Empirical (Relative Frequency) Probability

Based on observed data from repeated trials: \[P(E) = \frac{\text{Number of times } E \text{ occurred}}{\text{Total number of trials}}\] Example: If 45 out of 500 past customers returned a product, the empirical probability of a return is \(45/500 = 0.09\).

Subjective Probability

Based on personal judgement or expertise when neither equally likely outcomes nor historical data are available. Example: An analyst’s estimate that there is a 70% chance a new product launch succeeds.

27.3 Rules of Probability

Addition Rule

For any two events \(A\) and \(B\): \[P(A \cup B) = P(A) + P(B) - P(A \cap B)\] If \(A\) and \(B\) are mutually exclusive (cannot occur together), \(P(A \cap B) = 0\), so \(P(A \cup B) = P(A) + P(B)\).

Multiplication Rule

For any two events \(A\) and \(B\): \[P(A \cap B) = P(A) \times P(B \mid A)\] If \(A\) and \(B\) are independent (one does not affect the other), \(P(B \mid A) = P(B)\), so \(P(A \cap B) = P(A) \times P(B)\).

Complement Rule

\[P(A^c) = 1 - P(A)\] where \(A^c\) is the event that \(A\) does not occur.

Conditional Probability

The probability of \(B\) occurring given that \(A\) has already occurred: \[P(B \mid A) = \frac{P(A \cap B)}{P(A)}, \quad P(A) > 0\]

Bayes’ Theorem

Allows conditional probabilities to be reversed, updating the probability of a hypothesis given new evidence: \[P(A \mid B) = \frac{P(B \mid A)\, P(A)}{P(B)}\]

27.3.1 Worked Example

A company finds that 60% of its website visitors are on mobile devices, and 8% of mobile visitors make a purchase, compared with 5% of desktop visitors. What is the probability that a randomly chosen visitor is on mobile and makes a purchase?

Let \(M\) = visitor is on mobile, $P^* $ = visitor makes a purchase.

\[ P(M \cap P^*) = P(M) \times P(P^* \mid M) = 0.60 \times 0.08 = 0.048 \]

So there is a 4.8% chance that a randomly selected visitor is both on mobile and makes a purchase.

27.4 Probability Calculations in R and Python

Python

Transition to Probability Distributions

Individual probability rules describe single events. In practice, analysts usually work with a variable that can take many possible values, each with its own probability — for example, the number of defective items in a batch, or a customer’s spending amount. The next chapter introduces probability distributions, which describe exactly this.

Summary

Concept	Description
Foundations
Random Experiment	A process whose outcome cannot be predicted with certainty
Sample Space	The set of all possible outcomes of a random experiment
Event	Any subset of the sample space that is of interest
Probability	A number between 0 and 1 expressing how likely an event is to occur
Approaches
Classical Probability	Probability based on equally likely outcomes: favourable outcomes over total outcomes
Empirical Probability	Probability based on observed relative frequency across repeated trials
Subjective Probability	Probability based on personal judgement when data or equally likely outcomes are unavailable
Rules of Probability
Addition Rule	P(A or B) equals P(A) plus P(B) minus P(A and B)
Multiplication Rule	P(A and B) equals P(A) times P(B given A), simplifying to P(A) times P(B) when independent
Complement Rule	P(not A) equals 1 minus P(A)
Conditional Probability	The probability of B occurring given that A has already occurred
Bayes' Theorem	A rule for reversing conditional probabilities to update a hypothesis given new evidence
Special Event Types
Mutually Exclusive Events	Events that cannot occur together, so their joint probability is zero
Independent Events	Events where the occurrence of one does not affect the probability of the other