Optimal Estimation retrievals and their uncertainties: What every atmospheric scientist should know



Maahn, Maximilian — University of Colorado
Turner, David D. — NOAA- Earth System Research Laboratory

Area of research

Cloud Processes

Journal Reference

Maahn M, D Turner, U Löhnert, D Posselt, K Ebell, G Mace, and J Comstock. 2020. "Optimal Estimation Retrievals and Their Uncertainties: What Every Atmospheric Scientist Should Know." Bulletin of the American Meteorological Society, preprint(2020), 10.1175/BAMS-D-19-0027.1. ONLINE.


This educational paper gives an overview on developing Bayesian Optimal Estimation retrievals, which are a key method for converting indirect remote-sensing measurements into atmospheric variables. We present two sample retrievals based on microwave radiometers and radars including all data and retrieval code. Using a special website, the readers can explore and modify the retrieval by themselves interactively.


By guiding the reader through all required steps for developing a retrieval, discussing the most important sources for uncertainties, and providing extensive documentation and sample code, we enable the readers to develop their own Optimal Estimation retrievals quickly. These can be applied, e.g., to the extensive ARM remote-sensing data sets.


Remote-sensing instruments are heavily used to provide observations for both the operational and research communities, but retrieval algorithms are necessary to convert the indirect observations into the variable of interest. It is critical to be aware of the underlying assumptions made by many retrieval algorithms, including that the retrieval problem is often ill-posed and that there are various sources of uncertainty that need to be treated properly. In short, the retrieval challenge is to invert a set of noisy observations to obtain estimates of atmospheric quantities. The problem is often complicated by imperfect forward models, imperfect prior knowledge, and by the existence of non-unique solutions. Optimal Estimation (OE) is a widely used physical retrieval method that combines measurements, prior information, and the corresponding uncertainties based on Bayes’ theorem to find an optimal solution for the atmospheric state. Here, we provide a novel Python library to illustrate the use of OE for inverse problems in atmospheric sciences. We introduce two example problems: How to retrieve drop size distribution parameters from radar observations and how to retrieve the temperature profile from ground-based microwave sensors. Using these examples, we discuss common pitfalls, how the various error sources impact the retrieval, and how the quality of the retrieval results can be quantified. The examples can be explored interactively with a web browser.