Normal Distribution and Central Limit Theorem

If anyone asks me, What are the top 3 things that an analyst should know, Normal Distribution and Central Limit Theorem (CLT) would definitely be in the list! This is one of the very basic concepts of statistics (and a beautiful one!).

I have a separate article on Basic Statistical concepts. Do read this to understand the basics

Common Questions in Analytics Interviews - Part 2 | Statistics Basics

Now that you've got the basics of statistics, let's dive deeper.

---

What is a Normal Distribution?

Normal Distribution, also known as Bell Curve, is a Continuous Probability Distribution which is centred around its mean and tapers off symmetrically in both directions

The mathematical definition of Normal Distribution is

x: Independent Variable

f(x): The probability of occurence of x

μ: The mean of the distribution

𝜎: Standard Deviation of the distribution

Some of the special characteristics of this are

• Mean = Median = Mode

• It is symmetrical around its mean

• Majority of the observations are centred around the mean, and it tapers down on both the sides

  • 68% of the observations are present between 1 SD from the Mean (μ - 𝜎 , μ - 𝜎)

  • 95% of the observations are present between 2 SD from the Mean (μ - 2𝜎 , μ - 2𝜎)

  • 99% of the observations are present between 3 SD from the Mean (μ - 3𝜎 , μ - 3𝜎)

(This is an important characteristic of the the distribution)

• A special case of Normal Distribution with Mean = 0 and SD = 1 is called the Standard Normal Distribution

Now that we've got basic understanding of Normal Distribution, let's understand it's applications.

---

Central Limit Theorem (CLT)

This is one of the most beautiful theorems in Statistics.

CLT states that the Means of sufficiently large samples drawn from a population are Normally Distributed and the Mean of this distribution is similar to the Population Mean.

Let us understand this with an example.

You have been tasked with finding out the Average Weight of a person in India. How can you possibly find this?

Get the weight for each and every individual living in India. (In statistical terms, this is called the Population Mean)

Does that mean we meet every single person and measure his/her weight? Unless you have some superpowers, this approach is not feasible! So, how do we proceed?

Central Limit Theorem to the rescue!

1. Instead of getting the weights of everyone, we can get the weights of a randomly selected sample.

2. We need to ensure that each sample is large enough and randomly selected

3. Let's assume we took 10 samples with 100 points each i.e. we measure the weights of 1000 randomly picked people.

4. We calculate the Means of every sample and plot a histogram. Following is a sample chart of how it might look

5. Now, if we increase the samples to 100, calculate the means and plot it, following chart can be expected

6. To make our estimates even better, if we take 1000 samples.

You can see a trend, right?

Higher the number of samples, the Means of Samples tend to follow a normal distribution!

1. The Mean of this Sampling Distribution is equal to the Mean of the Population

2. SD of this Distribution is equal to SD of Population / Square Root of Sample Size

So, just by taking a small % of observations (1000 samples x 100 per sample = 1Mn observations) we are able to estimate the characteristics of the whole population (1 Bn+ people of India)

Interestingly, this Sampling Distribution holds true even if the underlying variable is NOT Normally Distributed.

---

That's all regarding the Normal Distribution and CLT. These simple concepts form the basis of modern Experimentation techniques and Hypothesis Testing, which are important things for an analyst to master.