A New Real-time Quality Control Algorithm for the AERI

 

Authors

Jonathan Gero — University of Wisconsin
Coda Phillips — University of Wisconsin, Madison
Alex Diebold — Space Science and Engineering Center
Denny John Hackel — University of Wisconsin

Category

Radiation

Description

While the AERI instruments are generally robust and operate reliably and continuously in the field, a small percentage of their data can be compromised and fail to meet quality control (QC) metrics. Bad data can arise from single event upsets, transient environmental forcing, or fault conditions that need to be corrected. It is obviously critical to identify compromised data in a dataset for accurate scientific analysis. Currently compromised data is identified by the mentor team, field technicians and the ARM Data Quality Office through the DQR process, which can be slow and time consuming. Furthermore, in certain cases expert knowledge on the operation of the AERI is required for nuanced scientific analysis. Our goal has been to make this process automated and real-time. We have developed a comprehensive suite of QC tests based on the 7 years of field diagnostics since the version 4 AERIs have been deployed by ARM. These include tests of the instrument state (hatch, scene mirror, blackbodies, laser, cooler), electronics, thermal stability as well as radiometric performance. The tests use time series analysis and statistical methods, such as outlier detection, power spectral analysis and signal cross correlation. The algorithm can be deployed to run in real-time in the field locally on the AERI, which facilitates remote monitoring and quick identification and resolution of fault conditions. Furthermore, the resulting quantitative QC mask lets novice investigators use AERI data with confidence, and allows advanced users to fine tune the mask to suit their needs. Overall, this QC algorithm represents a major leap forward in the scientific analysis and dissemination of AERI data. We will show the QC algorithms, their operational implementation, and case studies of their performance from recent campaigns such as AWARE and MARCUS.