Deviance (statistics)

Not to be confused with Deviation (statistics).

In statistics, deviance is a quality-of-fit statistic for a model that is often used for statistical hypothesis testing. It is a generalization of the idea of using the sum of squares of residuals in ordinary least squares to cases where model-fitting is achieved by maximum likelihood.

Definition

The deviance for a model M0, based on a dataset y, is defined as:[1][2]

D(y) = -2 \Big( \log \big( p(y\mid\hat \theta_0)\big)-\log \big(p(y\mid\hat \theta_s)\big)\Big).\,

Here \hat \theta_0 denotes the fitted values of the parameters in the model M0, while \hat \theta_s denotes the fitted parameters for the "saturated model": both sets of fitted values are implicitly functions of the observations y. Here the saturated model is a model with a parameter for every observation so that the data are fitted exactly. This expression is simply 2 times the log-likelihood ratio of the reduced model compared to the full model. The deviance is used to compare two models in particular in the case of generalized linear models where it has a similar role to residual variance from ANOVA in linear models (RSS).

Suppose in the framework of the GLM, we have two nested models, M1 and M2. In particular, suppose that M1 contains the parameters in M2, and k additional parameters. Then, under the null hypothesis that M2 is the true model, the difference between the deviances for the two models follows an approximate chi-squared distribution with k-degrees of freedom.[2]

Some usage of the term "deviance" can be confusing. According to Collett:[3]

"the quantity -2 \log \big( p(y\mid\hat \theta_0)\big) is sometimes referred to as a deviance. This is [...] inappropriate, since unlike the deviance used in the context of generalized linear modelling, -2 \log \big( p(y\mid\hat \theta_0)\big) does not measure deviation from a model that is a perfect fit to the data." However, since the principal use is in the form of the difference of the deviances of two models, this confusion in definition is unimportant.

See also

Notes

  1. Nelder, J.A.; Wedderburn, R.W.M. (1972). "Generalized Linear Models". Journal of the Royal Statistical Society. Series A (General) 135 (3): 370–384. doi:10.2307/2344614. JSTOR 2344614.
  2. 1 2 McCullagh and Nelder (1989)
  3. Collett (2003)Template:Page 76

References

External links

This article is issued from Wikipedia - version of the Saturday, February 20, 2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.