The Weak Law of Large Numbers, also known as Bernoulli?s theorem, states that if you have a sample of independent and identically distributed random variables, as the sample size grows larger, the sample mean will tend toward the population mean.
To put this in formal mathematical notation, it looks like this:
Weak Law of Large Numbers (Bernoulli?s theorem)As the sample size n grows to infinity, the probability that the sample mean x-bar differs from the population mean mu by some small amount epsilon is equal to 0.
We can prove this using Chebyshev?s inequality, which says the probability that a random variable X differs from its mean by some small constant k is less than or equal to the variance of X divided by the the square of the constant k.
Chebyshev?s Inequality
Since the random variable X and the constant k can be anything, we can replace X with the sample mean and replace k with epsilon.
Because we assumed that our sample contains independent and identically distributed random variables, we can simplify the right side of the equation.
Replacing the right side of Chebyshev?s inequality, we have the following.
As n tends to infinity, it follows that the right side of the inequality equals 0.
