Cauchy distribution

"Lorentz distribution" redirects here. It is not to be confused with Lorenz curve or Lorenz equation.
Cauchy
Probability density function


The purple curve is the standard Cauchy distribution

Cumulative distribution function

Parameters x_0\! location (real)
γ > 0 scale (real)
Support \displaystyle x \in (-\infty, +\infty)\!
PDF \frac{1}{\pi\gamma\,\left[1 + \left(\frac{x-x_0}{\gamma}\right)^2\right]}\!
CDF \frac{1}{\pi} \arctan\left(\frac{x-x_0}{\gamma}\right)+\frac{1}{2}\!
Quantile x_0+\gamma\,\tan[\pi(F-\tfrac{1}{2})]
Mean undefined
Median x_0\!
Mode x_0\!
Variance undefined
Skewness undefined
Ex. kurtosis undefined
Entropy \log(\gamma)\,+\,\log(4\,\pi)\!
MGF does not exist
CF \displaystyle \exp(x_0\,i\,t-\gamma\,|t|)\!

The Cauchy distribution, named after Augustin Cauchy, is a continuous probability distribution. It is also known, especially among physicists, as the Lorentz distribution (after Hendrik Lorentz), Cauchy–Lorentz distribution, Lorentz(ian) function, or Breit–Wigner distribution.

The Cauchy distribution is often used in statistics as the canonical example of a "pathological" distribution since both its mean and its variance are undefined. (But see the section Explanation of undefined moments below.) The Cauchy distribution does not have finite moments of order greater than or equal to one; only fractional absolute moments exist.[1] The Cauchy distribution has no moment generating function.

The Cauchy distribution f(x; x_0,\gamma) is the distribution of the X-intercept of a ray issuing from (x_0,\gamma) with a uniformly distributed angle. Its importance in physics is the result of it being the solution to the differential equation describing forced resonance.[2] In mathematics, it is closely related to the Poisson kernel, which is the fundamental solution for the Laplace equation in the upper half-plane. In spectroscopy, it is the description of the shape of spectral lines which are subject to homogeneous broadening in which all atoms interact in the same way with the frequency range contained in the line shape. Many mechanisms cause homogeneous broadening, most notably collision broadening.[3]

It is one of the few distributions that is stable and has a probability density function that can be expressed analytically, the others being the normal distribution and the Lévy distribution.

History

Functions with the form of the Cauchy distribution were studied by mathematicians in the 17th century, but in a different context and under the title of the Witch of Agnesi. Despite its name, the first explicit analysis of the properties of the Cauchy distribution was published by the French mathematician Poisson in 1824, with Cauchy only becoming associated with it during an academic controversy in 1853.[4] As such, the name of the distribution is a case of Stigler's Law of Eponymy. Poisson noted that if the mean of observations following such a distribution were taken, the mean error did not converge to any finite number. As such, Laplace's use of the Central Limit Theorem with such a distribution was inappropriate, as it assumed a finite mean and variance. Despite this, Poisson did not regard the issue as important, in contrast to Bienaymé, who was to engage Cauchy in a long dispute over the matter.

Characterisation

Probability density function

The Cauchy distribution has the probability density function[1][5]

f(x; x_0,\gamma) = \frac{1}{\pi\gamma \left[1 + \left(\frac{x - x_0}{\gamma}\right)^2\right]} = { 1 \over \pi \gamma } \left[ { \gamma^2 \over (x - x_0)^2 + \gamma^2  } \right],

where x0 is the location parameter, specifying the location of the peak of the distribution, and γ is the scale parameter which specifies the half-width at half-maximum (HWHM), alternatively 2γ is full width at half maximum (FWHM). γ is also equal to half the interquartile range and is sometimes called the probable error. Augustin-Louis Cauchy exploited such a density function in 1827 with an infinitesimal scale parameter, defining what would now be called a Dirac delta function.

The maximum value or amplitude of the Cauchy PDF is 1/(π γ), located at x=x0.

It is sometimes convenient to express the PDF in terms of the complex parameter ψ = x0 + i γ:


f(x;\psi)=\frac{1}{\pi}\,\textrm{Im}\left(\frac{1}{x-\psi}\right)=\frac{1}{\pi}\,\textrm{Re}\left(\frac{-i}{x-\psi}\right)

The special case when x0 = 0 and γ = 1 is called the standard Cauchy distribution with the probability density function[6][7]

 f(x; 0,1) = \frac{1}{\pi (1 + x^2)}. \!

In physics, a three-parameter Lorentzian function is often used:

f(x; x_0,\gamma,I) = \frac{I}{\left[1 + \left(\frac{x-x_0}{\gamma}\right)^2\right]} = I \left[ { \gamma^2 \over (x - x_0)^2 + \gamma^2  } \right],

where I is the height of the peak. The three-parameter Lorentzian function indicated is not, in general, a probability density function, since it does not integrate to 1, except in the special case where  I = \frac{1}{\pi\gamma}.\!

Cumulative distribution function

The cumulative distribution function is:

F(x; x_0,\gamma)=\frac{1}{\pi} \arctan\left(\frac{x-x_0}{\gamma}\right)+\frac{1}{2}

and the quantile function (inverse cdf) of the Cauchy distribution is

Q(p; x_0,\gamma) = x_0 + \gamma\,\tan\left[\pi\left(p-\tfrac{1}{2}\right)\right].

It follows that the first and third quartiles are (x0−γ, x0+γ), and hence the interquartile range is 2γ.

For the standard distribution, the cumulative distribution function simplifies to arctangent function arctan(x):

F(x; 0,1)=\frac{1}{\pi} \arctan\left(x\right)+\frac{1}{2}

Entropy

The entropy of the Cauchy distribution is given by:

H(\gamma)=-\int_{-\infty}^\infty f(x;x_0,\gamma) \log( f(x;x_0,\gamma))\,dx=\log(4\pi\gamma)

The derivative of the quantile function, the quantile density function, for the Cauchy distribution is:

Q'(p; \gamma) = \gamma\,\pi\,{\sec}^2\left[\pi\left(p-\tfrac{1}{2}\right)\right].\!

The differential entropy of a distribution can be defined in terms of its quantile density,[8] specifically:

H(\gamma) = \int_0^1 \log\,(Q'(p; \gamma))\,\mathrm dp = \log(4\pi\gamma)

The Cauchy distribution is the maximum entropy probability distribution for a random variate X for which

E[\log(1+(X-x_0)^2/\gamma^2)]=\log(4)

or, alternatively, for a random variate X for which

E[\log(1+(X-x_0)^2)]=2\log(1+\gamma).

In its standard form, it is the maximum entropy probability distribution for a random variate X for which[9]

\operatorname{E}\!\left[\ln(1+X^2) \right]=\ln(4).

Properties

The Cauchy distribution is an example of a distribution which has no mean, variance or higher moments defined. Its mode and median are well defined and are both equal to x0.

When U and V are two independent normally distributed random variables with expected value 0 and variance 1, then the ratio U/V has the standard Cauchy distribution.

If X1, ..., Xn are independent and identically distributed random variables, each with a standard Cauchy distribution, then the sample mean (X1+ ... +Xn)/n has the same standard Cauchy distribution. To see that this is true, compute the characteristic function of the sample mean:

\phi_{\overline{X}}(t) = \mathrm{E}\left[e^{i\overline{X}t}\right]

where \overline{X} is the sample mean. This example serves to show that the hypothesis of finite variance in the central limit theorem cannot be dropped. It is also an example of a more generalized version of the central limit theorem that is characteristic of all stable distributions, of which the Cauchy distribution is a special case.

The Cauchy distribution is an infinitely divisible probability distribution. It is also a strictly stable distribution.[10]

The standard Cauchy distribution coincides with the Student's t-distribution with one degree of freedom.

Like all stable distributions, the location-scale family to which the Cauchy distribution belongs is closed under linear transformations with real coefficients. In addition, the Cauchy distribution is closed under linear fractional transformations with real coefficients.[11] In this connection, see also McCullagh's parametrization of the Cauchy distributions.

Characteristic function

Let X denote a Cauchy distributed random variable. The characteristic function of the Cauchy distribution is given by

\phi_X(t; x_0,\gamma) = \mathrm{E}\left[e^{iXt} \right ] =\int_{-\infty}^\infty f(x;x_{0},\gamma)e^{ixt}\,dx =  e^{ix_0t - \gamma |t|}.

which is just the Fourier transform of the probability density. The original probability density may be expressed in terms of the characteristic function, essentially by using the inverse Fourier transform:

f(x; x_0,\gamma) = \frac{1}{2\pi}\int_{-\infty}^\infty \phi_X(t;x_0,\gamma)e^{-ixt}\,dt \!

The nth moment of a distribution is the nth derivative of the characteristic function evaluated at t=0. Observe that the characteristic function is not differentiable at the origin: this corresponds to the fact that the Cauchy distribution does not have well-defined moments higher than the zeroth moment.

Differential equation

The pdf of the standard Cauchy distribution is a solution to the following differential equation:

\left\{\begin{array}{l}
\left(1+x^2\right) f'(x)+2 x f(x)=0, \\
f(1)=\frac{1}{2 \pi}
\end{array}\right\}

Explanation of undefined moments

Mean

If a probability distribution has a density function f(x), then the mean is

\int_{-\infty}^\infty x f(x)\,dx. \qquad\qquad (1)\!

The question is now whether this is the same thing as

\int_a^\infty x f(x)\,dx+\int_{-\infty}^a x f(x)\,dx.\qquad\qquad (2) \!

for an arbitrary real number a.

If at most one of the two terms in (2) is infinite, then (1) is the same as (2). But in the case of the Cauchy distribution, both the positive and negative terms of (2) are infinite. Hence (1) is undefined.[12]

Although we may take (1) to mean

\lim_{a\to\infty}\int_{-a}^a x f(x)\,dx, \!

and this is its Cauchy principal value, which is zero, we could also take (1) to mean, for example,

\lim_{a\to\infty}\int_{-2a}^a x f(x)\,dx, \!

which is not zero, as can be seen easily by computing the integral.

Various results in probability theory about expected values, such as the strong law of large numbers, will not work in such cases.[12]

Higher moments

The Cauchy distribution does not have finite moments of any order. Some of the higher raw moments do exist and have a value of infinity, for example the raw second moment:


\begin{align}
\mathrm{E}[X^2] & \propto \int_{-\infty}^\infty \frac{x^2}{1+x^2}\,dx = \int_{-\infty}^\infty 1 - \frac{1}{1+x^2}\,dx \\[8pt]
& = \int_{-\infty}^\infty dx - \int_{-\infty}^\infty \frac{1}{1+x^2}\,dx = \int_{-\infty}^\infty dx-\pi = \infty.
\end{align}

By re-arranging the formula, one can see that the second moment is essentially the infinite integral of a constant (here 1). Higher even-powered raw moments will also evaluate to infinity. Odd-powered raw moments, however, are undefined, which is distinctly different from existing with the value of infinity. The odd-powered raw moments are undefined because their values are essentially equivalent to \infty - \infty since the two halves of the integral both diverge and have opposite signs. The first raw moment is the mean, which, being odd, does not exist. (See also the discussion above about this.) This in turn means that all of the central moments and standardized moments are undefined, since they are all based on the mean. The variance — which is the second central moment — is likewise non-existent (despite the fact that the raw second moment exists with the value infinity).

The results for higher moments follow from Hölder's inequality, which implies that higher moments (or halves of moments) diverge if lower ones do.

Estimation of parameters

Because the parameters of the Cauchy distribution don't correspond to a mean and variance, attempting to estimate the parameters of the Cauchy distribution by using a sample mean and a sample variance will not succeed.[13] For example, if n samples are taken from a Cauchy distribution, one may calculate the sample mean as:

\bar{x}=\frac{1}{n}\sum_{i=1}^n x_i

Although the sample values xi will be concentrated about the central value x0, the sample mean will become increasingly variable as more samples are taken, because of the increased likelihood of encountering sample points with a large absolute value. In fact, the distribution of the sample mean will be equal to the distribution of the samples themselves; i.e., the sample mean of a large sample is no better (or worse) an estimator of x0 than any single observation from the sample. Similarly, calculating the sample variance will result in values that grow larger as more samples are taken.

Therefore, more robust means of estimating the central value x0 and the scaling parameter γ are needed. One simple method is to take the median value of the sample as an estimator of x0 and half the sample interquartile range as an estimator of γ. Other, more precise and robust methods have been developed [14][15] For example, the truncated mean of the middle 24% of the sample order statistics produces an estimate for x0 that is more efficient than using either the sample median or the full sample mean.[16][17] However, because of the fat tails of the Cauchy distribution, the efficiency of the estimator decreases if more than 24% of the sample is used.[16][17]

Maximum likelihood can also be used to estimate the parameters x0 and γ. However, this tends to be complicated by the fact that this requires finding the roots of a high degree polynomial, and there can be multiple roots that represent local maxima.[18] Also, while the maximum likelihood estimator is asymptotically efficient, it is relatively inefficient for small samples.[19][20] The log-likelihood function for the Cauchy distribution for sample size n is:

\hat\ell(\!x_0,\gamma\mid x_1,\dotsc,x_n) = - n \log (\gamma \pi) - \sum_{i=1}^n \log \left(1 + \left(\frac{x_i - x_0}{\gamma}\right)^2\right)

Maximizing the log likelihood function with respect to x0 and γ produces the following system of equations:

 \sum_{i=1}^n \frac{x_i - x_0}{\gamma^2 + [x_i - \!x_0]^2} = 0
 \sum_{i=1}^n \frac{\gamma^2}{\gamma^2 + [x_i - x_0]^2} - \frac{n}{2} = 0

Note that

 \sum_{i=1}^n \frac{\gamma^2}{\gamma^2 + [x_i - x_0]^2}

is a monotone function in γ and that the solution γ must satisfy

 \min |x_i-x_0|\le \gamma\le \max |x_i-x_0|.

Solving just for x0 requires solving a polynomial of degree 2n−1,[18] and solving just for γ requires solving a polynomial of degree n (first for γ2, then x0). Therefore, whether solving for one parameter or for both parameters simultaneously, a numerical solution on a computer is typically required. The benefit of maximum likelihood estimation is asymptotic efficiency; estimating x0 using the sample median is only about 81% as asymptotically efficient as estimating x0 by maximum likelihood.[17][21] The truncated sample mean using the middle 24% order statistics is about 88% as asymptotically efficient an estimator of x0 as the maximum likelihood estimate.[17] When Newton's method is used to find the solution for the maximum likelihood estimate, the middle 24% order statistics can be used as an initial solution for x0.

Multivariate Cauchy distribution

A random vector X = (X1, ..., Xk)′ is said to have the multivariate Cauchy distribution if every linear combination of its components Y = a1X1 + ... + akXk has a Cauchy distribution. That is, for any constant vector aRk, the random variable Y = a′X should have a univariate Cauchy distribution.[22] The characteristic function of a multivariate Cauchy distribution is given by:

\phi_X(t) =  e^{ix_0(t)-\gamma(t)}, \!

where x0(t) and γ(t) are real functions with x0(t) a homogeneous function of degree one and γ(t) a positive homogeneous function of degree one.[22] More formally:[22]

x_0(at) = ax_0(t),
\gamma (at) = |a|\gamma (t),

for all t.

An example of a bivariate Cauchy distribution can be given by:[23]

f(x, y; x_0,y_0,\gamma)= { 1 \over 2 \pi } \left[ { \gamma \over ((x - x_0)^2 + (y - y_0)^2 +\gamma^2)^{1.5}  } \right] .

Note that in this example, even though there is no analogue to a covariance matrix, x and y are not statistically independent.[23]

Analogous to the univariate density, the multidimensional Cauchy density also relates to the multivariate Student distribution. They are equivalent when the degrees of freedom parameter is equal to one. The density of a k dimension Student distribution with one degree of freedom becomes:

f({\mathbf x}; {\mathbf\mu},{\mathbf\Sigma}, k)= \frac{\Gamma\left(\frac{1+k}{2}\right)}{\Gamma(\frac{1}{2})\pi^{\frac{k}{2}}\left|{\mathbf\Sigma}\right|^{\frac{1}{2}}\left[1+({\mathbf x}-{\mathbf\mu})^T{\mathbf\Sigma}^{-1}({\mathbf x}-{\mathbf\mu})\right]^{\frac{1+k}{2}}} .

Properties and details for this density can be obtained by taking it as a particular case of the multivariate Student density.

Transformation properties

\frac{aX+b}{cX+d} ~ Cauchy\left(\frac{a\psi+b}{c\psi+d}\right)

where a,b,c and d are real numbers.

\frac{X-i}{X+i} ~ CCauchy\left(\frac{\psi-i}{\psi+i}\right)
where "CCauchy" is the circular Cauchy distribution.

Lévy measure

The Cauchy distribution is the stable distribution of index 1. The Lévy-Khintchine representation of such a stable distribution of parameter  \gamma is given, for  X \sim \textrm{Stable}(\gamma, 0, 0)\, by:

\mathbb{E}\left( e^{ i x X } \right) = \exp\left( \int_{ \mathbb{R} } (  e^{i x y } - 1  ) \Pi_{\gamma}(dy) \right)
where
\Pi_{ \gamma}(dy) = \left( c_{1, \gamma}   \frac{1}{ y^{ 1 + \gamma } } 1_{ \left\{ y > 0 \right\} } + c_{2, \gamma}  \frac{1}{ |y|^{ 1 + \gamma } } 1_{ \left\{ y < 0 \right\} } \right) dy

and  c_{1, \gamma}, c_{2, \gamma} can be expressed explicitly.[25] In the case  \gamma = 1 of the Cauchy distribution, one has  c_{1, \gamma} = c_{2, \gamma}  .

This last representation is a consequence of the formula

|x| = \int_{ \mathbb{R} } ( 1 - e^{i x y }  )  \frac{dy}{y^2}

Related distributions

Relativistic Breit–Wigner distribution

In nuclear and particle physics, the energy profile of a resonance is described by the relativistic Breit–Wigner distribution, while the Cauchy distribution is the (non-relativistic) Breit–Wigner distribution.

See also

References

  1. 1 2 3 N. L. Johnson, S. Kotz, and N. Balakrishnan (1994). Continuous Univariate Distributions, Volume 1. New York: Wiley., Chapter 16.
  2. http://webphysics.davidson.edu/Projects/AnAntonelli/node5.html Note that the intensity, which follows the Cauchy distribution, is the square of the amplitude.
  3. E. Hecht (1987). Optics (2nd ed.). Addison-Wesley. p. 603.
  4. Cauchy and the Witch of Agnesi in Statistics on the Table, S M Stigler Harvard 1999 Chapter 18
  5. Feller, William (1971). An Introduction to Probability Theory and Its Applications, Volume II (2 ed.). New York: John Wiley & Sons Inc. p. 704. ISBN 978-0-471-25709-7.
  6. Riley, Ken F.; Hobson, Michael P.; Bence, Stephen J. (2006). Mathematical Methods for Physics and Engineering (3 ed.). Cambridge, UK: Cambridge University Press. p. 1333. ISBN 978-0-511-16842-0.
  7. Balakrishnan, N.; Nevrozov, V. B. (2003). A Primer on Statistical Distributions (1 ed.). Hoboken, New Jersey: John Wiley & Sons Inc. p. 305. ISBN 0-471-42798-5.
  8. Vasicek, Oldrich (1976). "A Test for Normality Based on Sample Entropy". Journal of the Royal Statistical Society, Series B 38 (1): 54–59.
  9. Park, Sung Y.; Bera, Anil K. (2009). "Maximum entropy autoregressive conditional heteroskedasticity model" (PDF). Journal of Econometrics (Elsevier): 219–230. Retrieved 2011-06-02.
  10. Campbell B. Read, N. Balakrishnan, Brani Vidakovic and Samuel Kotz (2006). Encyclopedia of Statistical Sciences (2nd ed.). John Wiley & Sons. p. 778. ISBN 978-0-471-15044-2.
  11. F. B. Knight (1976). "A characterization of the Cauchy type". Proceedings of the American Mathematical Society 55: 130–135. doi:10.2307/2041858. JSTOR 2041858.
  12. 1 2 "Cauchy Distribution". Virtual Laboratories. University of Alabama at Huntsville. Retrieved September 12, 2014.
  13. Illustration of instability of sample means
  14. Cane, Gwenda J. (1974). "Linear Estimation of Parameters of the Cauchy Distribution Based on Sample Quantiles". Journal of the American Statistical Association 69 (345): 243–245. doi:10.1080/01621459.1974.10480163. JSTOR 2285535.
  15. Zhang, Jin (2010). "A Highly Efficient L-estimator for the Location Parameter of the Cauchy Distribution". Computational Statistics 25 (1): 97–105. doi:10.1007/s00180-009-0163-y.
  16. 1 2 Rothenberg, Thomas J.; Fisher, Franklin, M.; Tilanus, C.B. (1966). "A note on estimation from a Cauchy sample". Journal of the American Statistical Association 59 (306): 460463. doi:10.1080/01621459.1964.10482170.
  17. 1 2 3 4 Bloch, Daniel (1966). "A note on the estimation of the location parameters of the Cauchy distribution". Journal of the American Statistical Association 61 (316): 852855. doi:10.1080/01621459.1966.10480912. JSTOR 2282794.
  18. 1 2 Ferguson, Thomas S. (1978). "Maximum Likelihood Estimates of the Parameters of the Cauchy Distribution for Samples of Size 3 and 4". Journal of the American Statistical Association 73 (361): 211. doi:10.1080/01621459.1978.10480031. JSTOR 2286549.
  19. Cohen Freue, Gabriella V. (2007). "The Pitman estimator of the Cauchy location parameter" (PDF). Journal of Statistical Planning and Inference 137 (6): 1901. doi:10.1016/j.jspi.2006.05.002.
  20. Wilcox, Rand (2012). Introduction to Robust Estimation & Hypothesis Testing. Elsevier.
  21. Barnett, V. D. (1966). "Order Statistics Estimators of the Location of the Cauchy Distribution". Journal of the American Statistical Association 61 (316): 1205. doi:10.1080/01621459.1966.10482205. JSTOR 2283210.
  22. 1 2 3 Ferguson, Thomas S. (1962). "A Representation of the Symmetric Bivariate Cauchy Distribution". The Annals of Mathematical Statistics: 1256. doi:10.1214/aoms/1177704357. JSTOR 2237984.
  23. 1 2 Molenberghs, Geert; Lesaffre, Emmanuel (1997). "Non-linear Integral Equations to Approximate Bivariate Densities with Given Marginals and Dependence Function" (PDF). Statistica Sinica 7: 713738.
  24. 1 2 McCullagh, P., "Conditional inference and Cauchy models", Biometrika, volume 79 (1992), pages 247259. PDF from McCullagh's homepage.
  25. Kyprianou, Andreas (2009). Lévy processes and continuous-state branching processes:part I (PDF). p. 11.

External links

This article is issued from Wikipedia - version of the Thursday, May 05, 2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.