Dixon's Q test
In statistics, Dixon's Q test, or simply the Q test, is used for identification and rejection of outliers. This assumes normal distribution and per Dean and Dixon, and others, this test should be used sparingly and never more than once in a data set. To apply a Q test for bad data, arrange the data in order of increasing values and calculate Q as defined:
Where gap is the absolute difference between the outlier in question and the closest number to it. If Q > Qtable, where Qtable is a reference value corresponding to the sample size and confidence level, then reject the questionable point. Note that only one point may be rejected from a data set using a Q test.
Example
Consider the data set:
Now rearrange in increasing order:
We hypothesize that 0.167 is an outlier. Calculate Q:
With 10 observations and at 90% confidence, Q = 0.455 > 0.412 = Qtable, so we conclude 0.167 is an outlier. However, at 95% confidence, Q = 0.455 < 0.466 = Qtable 0.167 is not considered an outlier. This means that for this example we can be 90% sure that 0.167 is an outlier, but we cannot be 95% sure.
McBane[1] notes: Dixon provided related tests intended to search for more than one outlier, but they are much less frequently used than the r10 or Q version that is intended to eliminate a single outlier.
Table
This table summarizes the limit values of the test.
Number of values: | 3 |
4 |
5 |
6 |
7 |
8 |
9 |
10 |
Q90%: |
0.941 |
0.765 |
0.642 |
0.560 |
0.507 |
0.468 |
0.437 |
0.412 |
Q95%: |
0.970 |
0.829 |
0.710 |
0.625 |
0.568 |
0.526 |
0.493 |
0.466 |
Q99%: |
0.994 |
0.926 |
0.821 |
0.740 |
0.680 |
0.634 |
0.598 |
0.568 |
See also
References
- ↑ Halpern, Arthur M. "Experimental physical chemistry : a laboratory textbook." 3rd ed. / Arthur M. Halpern , George C. McBane. New York : W. H. Freeman, c2006 Library of Congress
- R. B. Dean and W. J. Dixon (1951) "Simplified Statistics for Small Numbers of Observations". Anal. Chem., 1951, 23 (4), 636–638. Abstract Full text PDF
- Rorabacher, D.B. (1991) "Statistical Treatment for Rejection of Deviant Values: Critical Values of Dixon Q Parameter and Related Subrange Ratios at the 95 percent Confidence Level". Anal. Chem., 63 (2), 139–146. PDF (including larger tables of limit values)
- McBane, George C. (2006) "Programs to Compute Distribution Functions and Critical Values for Extreme Value Ratios for Outlier Detection". J. Statistical Software 16(3):1–9, 2006 Article (PDF) and Software (Fortan-90, Zipfile)
- Shivanshu Shrivastava, A. Rajesh, P. K. Bora (2014) "Sliding window Dixon's tests for malicious users' suppression in a cooperative spectrum sensing system" IET Communications, 2014, 8 (7)
External links
- Test for Outliers Main page of GNU R's package 'outlier' including 'dixon.test' function.
- Dixon's test in Communications Use of Dixon's test in Cognitive Radio Communications by Shivanshu Shrivastava