Lidar Cloud Detection with Fully Convolutional Networks
Authors
Erol Cromwell — Pacific Northwest National Laboratory
Donna M. Flynn — Pacific Northwest National Laboratory
Category
General topics – Clouds
Description
sgp30smplcmask1zwangC1.c1.20150101, first quarter of day, flipped. The top two images are the MPL attenuated backscatter profile and linear depolarization ratio, respectively, which are used for input for the FCN model. The middle left image is the hand-labeled cloud mask image we used to train our model. The middle right image is the MPL Cloud Mask product result from the backscatter. The bottom right image is the FCN model’s cloud mask output for the image. The bottom left image is the model’s per-pixel confidence in its cloud prediction.
The vertical distribution of clouds from active remote sensing instrumentation is a widely used data product from global atmospheric measuring sites. The presence of clouds can be expressed as a binary cloud mask and is a primary input for climate modeling efforts and cloud formation studies. The current MPL Cloud Mask operational product tends to oversample or over-represent the cloud layers. This translates as uncertainty for assessing the radiative impact of clouds and tracking changes in cloud climatologies. The Atmospheric Radiation Measurement (ARM) program has over 20 years of micro-pulse lidar (MPL) data and companion automated cloud mask product at the mid-latitude Southern Great Plains (SGP) atmospheric observatory.
Using this data, we apply a machine-learning approach to train a fully convolutional network (FCN) with semi-supervised learning to segment lidar imagery into geometric time-height cloud locations for the SGP site and MPL instrument. In our semi-supervised approach, we pre-train the classification layers of the FCN with “weakly labeled” lidar data. Then, we facilitate end-to-end “unsupervised” pre-training with the MPL Cloud Mask and transition to fully supervised learning with ground truth labeled data. The FCN achieved a recall and precision of 0.86 and 0.84, respectively, compared to the current cloud mask algorithm of 0.88 and 0.44.
One of the challenges of operational algorithms is how sensitive they are to changes in measurements in data quality. We plan to analyze and compare the FCN model’s and the MPL Cloud Mask product’s dependency on data quality by first identifying good and bad data epochs for the MPL instruments. Then, we will compare the performance of both methods on the different quality data. Additionally, we plan to use the FCN model’s prediction confidence to investigate how uncertainty changes with data quality.
