AI teaches itself to identify clouds
Submitter
Fast, Jerome D — Pacific Northwest National Laboratory
Area of research
Cloud Distributions/Characterizations
Journal Reference
Science
Clouds affect the Earth’s weather and climate by influencing light, heat, and moisture in the atmosphere. They come in many varieties with unique impacts on the atmosphere. Researchers strive to understand these nuances to accurately model weather and climate. Artificial intelligence (AI) can help study clouds by analyzing satellite images, but typical image recognition AIs require vast amounts of human-generated labels to train. In this work, researchers demonstrate an AI that can be taught to recognize cloud types by looking at millions of satellite images of clouds without requiring human input.
Impact
Clouds play a pivotal role in the Earth’s climate and weather. They form in a diversity of distinct patterns and types that can have profoundly different impacts on things like the water cycle and how radiant energy moves through the atmosphere. Earth-observing satellites have been capturing images for decades and these data sets offer a unique opportunity to study clouds and their impact on the Earth system. The massive quantity of satellite images can prohibit such research because there is far too much imagery for scientists to sort manually. Self-supervised AI can learn to interpret images without human input; in this work, researchers trained such an AI to differentiate between cloud types in satellite images. This AI will allow for a variety of scientific investigations into cloud properties and behavior. More broadly, self-supervised methods have not yet been widely adopted in the atmospheric sciences. This study demonstrates how those methods can be used to parse extraordinarily large data sets of satellite images.
Summary
AI developed in this research learns to generate a simple numerical representation of cloud images by performing what is essentially an image-matching task. It is trained on pairs of cloud images taken from nearby locations that are likely to contain the same cloud types. It is rewarded for learning similar representations of images with the same cloud type and differing representations for images with different cloud types. The model was evaluated on several data sets of cloud images that have been hand-labeled by past cloud classification studies and performed comparably to classification models that had been trained with human-generated labels. Researchers also demonstrated the model’s ability to generalize between satellite instruments and explored the internal representations of clouds learned by the model by forcing the model to hallucinate images based on its own parameters.
Demonstration of using a CNN for image search on a database of satellite cloud images. The “query image” column shows single images with a specific cloud type (row labels). Samples 1-5 are images found by the CNN that contain similar cloud types to the corresponding query image. Image from journal.