Poisson limit theorem

The law of rare events or Poisson limit theorem gives a Poisson approximation to the binomial distribution, under certain conditions.^[1] The theorem was named after Siméon Denis Poisson (1781–1840).

Statement

If

$n \rightarrow \infty, p \rightarrow 0$, such that $np \rightarrow \lambda$

then

$\frac{n!}{(n-k)!k!} p^k (1-p)^{n-k} \rightarrow e^{-\lambda}\frac{\lambda^k}{k!}.$

Example

Suppose that in an interval [0, 1000], 500 points are placed randomly. Now what is the number of points that will be placed in [0, 10]?

The probabilistically precise way of describing the number of points in the sub-interval would be to describe it as a binomial distribution $p_n(k)$.

If we look here, the probability (that a random point will be placed in the sub-interval) is $p = 10/1000 = 0.01$. Here $n=500$ so $np=5$.

The probability that $k$ points lie in the sub-interval is

$p_n(k)=\frac{n!}{(n-k)!k!} p^k (1-p)^{n-k}.$

where: $p$ is the probability of falling with in the interval. $n!/(k! \cdot (n-k)!)$ gives the number of ways in which $k$ elements can be selected. $p^k$ gives the probability of the $k$ elements falling in the interval. $(1-p)^{n-k}$ counts the probability that ${n-k}$ elements fall outside of the interval

But using the Poisson Theorem we can approximate it as

$e^{-\lambda}\frac{\lambda^k}{k!} = e^{-5}\frac{5^k}{k!}.$

Proofs

According to factorial's rate of growth, we replace factorials of large numbers with approximations:

$\frac{n!}{(n-k)!k!} p^k (1-p)^{n-k} \rightarrow \frac{ \sqrt{2\pi n}\left(\frac{n}{e}\right)^n}{ \sqrt{2\pi \left(n-k\right)}\left(\frac{n-k}{e}\right)^{n-k}k!} p^k (1-p)^{n-k}.$

After simplifying the fraction:

$\frac{ \sqrt{2\pi n}\left(\frac{n}{e}\right)^n}{ \sqrt{2\pi \left(n-k\right)}\left(\frac{n-k}{e}\right)^{n-k}k!} p^k (1-p)^{n-k}\rightarrow \frac{ \sqrt{n}n^np^k (1-p)^{n-k}}{ \sqrt{n-k}\left(n-k\right)^{n-k}e^kk!}\rightarrow \frac{n^np^k (1-p)^{n-k}}{\left(n-k\right)^{n-k}e^kk!}.$

After using the condition $np \rightarrow \lambda$:

$\frac{n^np^k (1-p)^{n-k}}{\left(n-k\right)^{n-k}e^kk!} \rightarrow \frac{n^k\left(\frac{\lambda}{n}\right)^k (1-\frac{\lambda}{n})^{n-k}}{\left(1-\frac{k}{n}\right)^{n-k}e^kk!}=\frac{\lambda^k \left(1-\frac{\lambda}{n}\right)^{n-k}}{\left(1-\frac{k}{n}\right)^{n-k}e^kk!}\rightarrow\frac{\lambda^k \left(1-\frac{\lambda}{n}\right)^{n}}{\left(1-\frac{k}{n}\right)^{n}e^kk!}$

As $n\rightarrow \infty$, $\left(1+\frac{x}{n}\right)^n \rightarrow e^x$ so:

$\frac{\lambda^k \left(1-\frac{\lambda}{n}\right)^{n}}{\left(1-\frac{k}{n}\right)^{n}e^kk!}\rightarrow\frac{\lambda^k e^{-\lambda}}{e^{-k}e^kk!}=\frac{\lambda^k e^{-\lambda}}{k!}$

Q.E.D.

Alternative Proof

If we make the stronger assumption $np=\lambda$ (rather than $np\rightarrow \lambda$) then a simpler proof is possible without needing approximations for the factorials. Since $np=\lambda$, we can rewrite $p=\lambda / n$. We now have:

$\lim_{n\to\infty}\frac{n!}{(n-k)!k!} \left(\frac{\lambda}{n}\right)^k \left(1- \frac{\lambda}{n}\right)^{n-k} = \lim_{n\to\infty}\frac{n(n-1)(n-2)\dots(n-k+1)}{k!} \frac{\lambda^k}{n^k} \left(1- \frac{\lambda}{n}\right)^{n-k}$

Taking each of these terms in sequence, $n(n-1)(n-2)\dots(n-k+1)=n^k+O\left(n^{k-1}\right)$, meaning that $\lim_{n\to\infty} \frac{n(n-1)(n-2)\dots(n-k+1)}{n^k k!}= \frac{1}{k!}$.

Now $\left(1- \frac{\lambda}{n}\right)^{n-k}=\left(1- \frac{\lambda}{n}\right)^{n}\left(1- \frac{\lambda}{n}\right)^{-k}$. The first portion of this converges to $e^{-\lambda}$, and the second portion goes to 1, as $\lim_{n\to\infty}\left(1- \frac{\lambda}{n}\right)^{-k}=\lim_{n\to\infty}\left(1- 0\right)^{-k}=1$

This leaves us with $\frac{1}{k!}\lambda^k e^{-\lambda}$. Q.E.D.

Ordinary Generating Functions

It is also possible to demonstrate the theorem through the use of Ordinary Generating Functions (OGF). Indeed, the OGF of the binomial distribution is

$G_\mathrm{bin}(x;p,N) \equiv \sum_{k=0}^{N} \left[ \binom{N}{k} p^k (1-p)^{N-k} \right] x^k = \Big[ 1 + (x-1)p \Big]^{N}$

by virtue of the Binomial Theorem. Taking the limit $N \rightarrow \infty$ while keeping the product $pN\equiv\lambda$ constant, we find

$\lim_{N\rightarrow\infty} G_\mathrm{bin}(x;p,N) = \lim_{N\rightarrow\infty} \Big[ 1 + \frac{\lambda(x-1)}{N} \Big]^{N} = \mathrm{e}^{\lambda(x-1)} = \sum_{k=0}^{\infty} \left[ \frac{\mathrm{e}^{-\lambda}\lambda^k}{k!} \right] x^k$

which is the OGF for the Poisson distribution. (The second equality holds due to the definition of the Exponential function.)

See also

• De Moivre–Laplace theorem
• Le Cam's theorem

References

1. Papoulis, Pillai, Probability, Random Variables, and Stochastic Processes, 4th Edition