Score test

Rao's score test, or the score test (often known as the Lagrange multiplier test in econometrics[1]) is a statistical test of a simple null hypothesis that a parameter of interest \theta is equal to some particular value \theta_0. It is the most powerful test when the true value of \theta is close to \theta_0. The main advantage of the Score-test is that it does not require an estimate of the information under the alternative hypothesis or unconstrained maximum likelihood. This constitutes a potential advantage in comparison to other tests, such as the Wald test and the generalized likelihood ratio test (GLRT). This makes testing feasible when the unconstrained maximum likelihood estimate is a boundary point in the parameter space.

Single parameter test

The statistic

Let L be the likelihood function which depends on a univariate parameter \theta and let x be the data. The score U(\theta) is defined as


U(\theta)=\frac{\partial \log L(\theta \mid x)}{\partial \theta}.

The Fisher information is[2]


\mathcal{I}(\theta) = - \operatorname{E} \left[\left. \frac{\partial^2}{\partial\theta^2} \log L(X;\theta)\right|\theta \right]\,.

The statistic to test \mathcal{H}_0:\theta=\theta_0 is 
S(\theta_0) = \frac{U(\theta_0)^2}{I(\theta_0)}

which has an asymptotic distribution of \chi^2_1, when \mathcal{H}_0 is true.

Note on notation

Note that some texts use an alternative notation, in which the statistic 
S^*(\theta)=\sqrt{ S(\theta) }
is tested against a normal distribution. This approach is equivalent and gives identical results.

As most powerful test for small deviations


\left(\frac{\partial \log L(\theta \mid x)}{\partial \theta}\right)_{\theta=\theta_0} \geq C

Where L is the likelihood function, \theta_0 is the value of the parameter of interest under the null hypothesis, and C is a constant set depending on the size of the test desired (i.e. the probability of rejecting H_0 if H_0 is true; see Type I error).

The score test is the most powerful test for small deviations from H_0. To see this, consider testing \theta=\theta_0 versus \theta=\theta_0+h. By the Neyman–Pearson lemma, the most powerful test has the form


\frac{L(\theta_0+h\mid x)}{L(\theta_0\mid x)} \geq K;

Taking the log of both sides yields


\log L(\theta_0 + h \mid x ) - \log L(\theta_0\mid x) \geq \log K.

The score test follows making the substitution (by Taylor series expansion)


\log L(\theta_0+h\mid x) \approx \log L(\theta_0\mid x) + h\times
\left(\frac{\partial \log L(\theta \mid x)}{\partial \theta}\right)_{\theta=\theta_0}

and identifying the C above with \log(K).

Relationship with other hypothesis tests

The likelihood ratio test, the Wald test, and the Score test are asymptotically equivalent tests of hypotheses.[3] When testing nested models, the statistics for each test converge to a Chi-squared distribution with degrees of freedom equal to the difference in degrees of freedom in the two models.

Multiple parameters

A more general score test can be derived when there is more than one parameter. Suppose that \hat{\theta}_0 is the maximum likelihood estimate of \theta under the null hypothesis H_0. Then


U^T(\hat{\theta}_0) I^{-1}(\hat{\theta}_0) U(\hat{\theta}_0) \sim \chi^2_k

asymptotically under H_0, where k is the number of constraints imposed by the null hypothesis and


U(\hat{\theta}_0) = \frac{\partial \log L(\hat{\theta}_0 | x)}{\partial \theta}

and


I(\hat{\theta}_0) = -E\left(\frac{\partial^2 \log L(\hat{\theta}_0 \mid x)}{\partial \theta \partial \theta'} \right).

This can be used to test H_0.

Special cases

In many situations, the score statistic reduces to another commonly used statistic.[4]

When the data follows a normal distribution, the score statistic is the same as the t statistic.

When the data consists of binary observations, the score statistic is the same as the chi-squared statistic in the Pearson's chi-squared test.

When the data consists of failure time data in two groups, the score statistic for the Cox partial likelihood is the same as the log-rank statistic in the log-rank test. Hence the log-rank test for difference in survival between two groups is most powerful when the proportional hazards assumption holds.

See also

References

  1. Bera, Anil K.; Bilias, Yannis (2001). "Rao's score, Neyman's C(α) and Silvey's LM tests: An essay on historical developments and some new results". Journal of Statistical Planning and Inference 97: 9–44. doi:10.1016/S0378-3758(00)00343-8. Engle, Robert F (1984) . Wald, Likelihood Ratio and Lagrange Multiplier tests in Econometrics. in Handbook of Econometrics, Volume II, Edited by Z. Griliches and M.D. Intriligator. Elsevier Science Publishers BV.
  2. Lehmann and Casella, eq. (2.5.16).
  3. Engle, Robert F. (1983). "Wald, Likelihood Ratio, and Lagrange Multiplier Tests in Econometrics". In Intriligator, M. D.; and Griliches, Z. Handbook of Econometrics II. Elsevier. pp. 796–801. ISBN 978-0-444-86185-6.
  4. Cook, T. D.; DeMets, D. L., eds. (2007). Introduction to Statistical Methods for Clinical Trials. Chapman and Hall. pp. 296–297. ISBN 1-58488-027-9.
This article is issued from Wikipedia - version of the Sunday, January 31, 2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.