Automated quality control of meteorological observations

A meteorological observation at a given place can be inaccurate for a variety of reasons, such as a hardware defect. Quality control can help spot which meteorological observations are inaccurate.

One of the main automated quality control programs used today in the area of meteorological observations is the meteorological assimilation data ingest system (MADIS).^[1]

History

Weather observation quality control systems verify probability, history, and trends. One of the main and simplest forms of quality control is the check of probability.^[2] This check throws out impossible observations, such as the dew point being higher than the temperature or data outside acceptable ranges, such as temperatures over 200 degrees Fahrenheit. Another basic quality control check is to have the data compared to preset geographic extremes,^[3] perhaps combined with diurnal variations. However this only flags the data as uncertain because the station could be reporting correctly but there is no way to know. A better way is to correlate with previous observations as well as the other simple checks.^[4] This method uses one hour persistence to check the quality of the current observation. This method makes continuity of observations better since the system is able to make better judgments on whether the current observations are bad or not.

Current

Systems such as MADIS use a three-pronged approach.^[5] This approach is much better mainly because it has more information to compare the current observation to. The first part of the process is the limit check. As already described the program checks whether the observation is within predetermined limits that are set according to whether they can physically exist or not. The second part is the temporal check which compares the station to its closest surrounding stations. The third part is internal checking, which compares the observation to previous ones and sees whether it makes sense or not. It also takes into account present weather conditions so that the data is not considered bad just because the system is set for fair weather.

MADIS uses this current three-pronged approach for its quality control tests. They are organized into three different levels of checks.^[4] Level one is the validity tests, level two is the internal checks and also statistical spatial tests and level three is the spatial test. The level two statistical spatial test tests whether or not the station has failed any quality control check more than 75% of the time during the previous seven days. Once this has happened the station will continue to fail until it improves to failing only 25% of the time. The spatial check for the MADIS program also uses a reanalysis procedure. What this does is if there is a large difference between the station being checked and the station that it is being checked against then one of them is wrong. Instead of assuming that the station being checked is wrong, the program then moves on to the other stations that are near the one being checked. If the station that is being checked still is way off compared to most of the stations surrounding it then it is flagged as bad. However if the station is close to all of the other ones except for one then that one is bad.

References

1. Meteorological assimilation data ingest system (MADIS)
2. Gandin, L., 1988: Complex Quality Control of Meteorological Observations. Monthly Weather Review, 116, 1137–1156.
3. DeGaetano, A., 1997: A Quality-Control Routine for Hourly Wind Observations. Journal of Atmospheric and Oceanic Technology, 14, 308–317.
4. Miller, P. and S. Benjamin, 1992: A System for the Hourly Assimilation of Surface Observations in Mountainous and Flat Terrain. Monthly Weather Review, 120, 2342–2359.
5. Graybeal, D., A. DeGaetano, and K. Eggleston, 2004: Improved Quality Assurance for Historical Hourly Temperature and Humidity: Development and Application to Environmental Analysis. Journal of Applied Meteorology,43, 1722–1735.