29 Sampling Distributions and the Central Limit Theorem
Inferential statistics uses a single sample to draw conclusions about an entire population. This is only possible because sample statistics — like the sample mean — follow predictable patterns of their own across repeated sampling. A sampling distribution describes exactly this: the distribution of a statistic (not of the raw data) computed from many different samples of the same size drawn from the same population.
29.1 What Is a Sampling Distribution?
Imagine repeatedly drawing a random sample of size \(n\) from a population, computing the sample mean \(\bar{x}\) each time, and plotting all of those means. That plot is the sampling distribution of the mean. It is distinct from both the population distribution (the spread of individual values) and any single sample’s distribution.
- Standard Error: The standard deviation of a sampling distribution, which measures how much a sample statistic is expected to vary from sample to sample.
- For the sample mean, the standard error is: \[ SE = \frac{\sigma}{\sqrt{n}} \] where \(\sigma\) is the population standard deviation and \(n\) is the sample size. Larger samples produce a smaller standard error — sample means from bigger samples cluster more tightly around the true population mean.
29.2 The Central Limit Theorem (CLT)
The Central Limit Theorem is one of the most important results in statistics. It states that:
As the sample size \(n\) increases, the sampling distribution of the sample mean approaches a normal distribution, regardless of the shape of the population distribution — provided the sample size is sufficiently large (a common rule of thumb is \(n \geq 30\)).
Formally, for a population with mean \(\mu\) and standard deviation \(\sigma\), the sampling distribution of \(\bar{x}\) approaches: \[ \bar{x} \sim N\left(\mu, \; \frac{\sigma}{\sqrt{n}}\right) \]
Why the CLT Matters
- It justifies using normal-distribution-based methods (z-tests, t-tests, confidence intervals) even when the underlying population data is skewed or non-normal.
- It explains why sample means are more stable and predictable than individual observations.
- It is the theoretical bridge between probability distributions (previous chapter) and the estimation and hypothesis-testing methods used throughout the rest of this book.
29.2.1 Illustrating the CLT by Simulation
A population of purchase amounts is strongly right-skewed (most purchases are small, a few are very large). If we repeatedly draw samples of size 40 from this population and plot the distribution of the 1,000 resulting sample means, that distribution will look approximately normal — even though the original population was skewed. This is demonstrated in code below by simulating from a heavily skewed (exponential) population.
29.3 Simulating the CLT in R and Python
29.4 Common Misunderstandings
- The CLT describes the distribution of the sample mean, not the distribution of individual observations — individual data can remain skewed even as the sampling distribution of the mean becomes normal.
- A “large enough” sample size depends on how skewed the population is; a rule of thumb like \(n \geq 30\) is a guideline, not a guarantee.
- The standard error decreases with the square root of \(n\), not proportionally to \(n\) — quadrupling the sample size only halves the standard error.
Transition to Confidence Intervals
Because the sampling distribution of the mean is approximately normal, an analyst can use a single observed sample mean and its standard error to build a range of plausible values for the true, unknown population mean. That range is a confidence interval, covered in the next chapter.
Summary
| Concept | Description |
|---|---|
| Foundations | |
| Sampling Distribution | The distribution of a sample statistic, such as the mean, across repeated samples of the same size |
| Standard Error | The standard deviation of a sampling distribution, measuring how much a statistic varies across samples |
| Standard Error of the Mean Formula | SE equals the population standard deviation divided by the square root of the sample size |
| The CLT | |
| Central Limit Theorem | As sample size increases, the sampling distribution of the mean approaches normal regardless of the population shape |
| CLT Sample Size Rule of Thumb | A commonly used minimum sample size of about 30 for the CLT approximation to hold reasonably well |
| Why the CLT Matters | It justifies normal-based inference methods even when the underlying population is not normal |
| Evidence and Caveats | |
| Sampling Distribution vs Population Distribution | The sampling distribution describes a statistic across samples, not the spread of individual values |
| Simulation Evidence for the CLT | Repeatedly sampling from a skewed population and averaging produces an approximately normal distribution of means |
| Common Misunderstanding: Individual vs Mean | The CLT concerns the sample mean's distribution, not the distribution of individual observations |
| Common Misunderstanding: Square-Root Scaling | The standard error shrinks with the square root of n, so quadrupling n only halves the standard error |