Developing E3SM: How to Fit a Planet Inside a Computer

 
Published: 30 July 2018

Ruby Leung, E3SM chief scientist.

The Earth was apparently losing water.
Ruby Leung, a scientist from the U.S. Department of Energy’s (DOE) Pacific Northwest National Laboratory (PNNL), and her team were baffled by their results. “We were seeing the sea level decreasing at an alarming rate,” she said.
Fortunately, they were only looking at a virtual Earth. They quickly realized there were errors in the earth system computer model they were developing. Scientists use these computer programs to visualize the present and see into the future, and they need to come as close as possible to modeling how Earth’s systems function in real life. Because the real Earth cycles water but never loses it, neither should the model.
Identifying the problem was simple. Fixing it wasn’t. There are hundreds of variables in the model that could affect its water cycle. Pinpointing the exact one and changing it without creating another inaccuracy can take hours or days of work. In this instance, the model wasn’t sending all the water runoff fluxes from the land into the ocean. In addition, the atmospheric model was losing a very tiny amount of water at each time step, which added up quite a lot. To fix the problem, Leung and colleagues changed the model to conserve water.
Solving this problem was just one of the many challenges the team faced as they developed DOE’s new software: the Energy Exascale Earth System Model (E3SM). By representing many of Earth’s systems and interactions in greater detail than ever before, they hope to help scientists better understand our planet today and in the future.


The E3SM project involves more than 100 scientists and engineers at multiple DOE national laboratories and universities, along with DOE programmatic contributions and collaborations. Process understanding provided by ASR, for example, helped constrain and guide researchers as they developed parameterizations of clouds and aerosols. Phase 2 of E3SM is expected to incorporate products developed under projects for the Climate Model Development and Validation (CMDV) activity, which is tasked to improve earth system model design and to develop scale-aware physics for earth systems and enhance model validation methods. Several of these projects require collaboration between E3SM, ASR, and the Atmospheric Radiation Measurement (ARM) user facility.


Understanding a Changing Earth

Earth always has been and always will be changing. But human activities are accelerating those changes, and there is general agreement that global temperatures are rising. While scientists can broadly predict how rising temperatures will affect our world, policy-makers and planners need to understand what will happen to individual cities and states. That’s where earth system models come into play.

2017 E3SM team photo.

DOE is interested in understanding how energy production could affect Earth’s systems and vice versa. While energy production can accelerate climate change, rising temperatures can also affect energy production and use. A lack of water can reduce a hydropower plant’s output or limit water available for cooling nuclear plants.
“DOE and its predecessors have always had a mission to understand the environmental consequences of energy production and use,” said David Bader, a scientist at DOE’s Lawrence Livermore National Laboratory and E3SM team lead. “We know that the climate and Earth system is not stationary. It changes. We need a way to understand the interactions.”
The scope of earth system models makes them unique. Scientists construct a variety of software programs to simulate different natural systems, but earth system models bring them all together. They demonstrate how these individual systems—the atmosphere, soil, ocean, ice sheets, and more—interact. They also incorporate many aspects of human activity, such as the products of energy production, water management and use, and agriculture management as well as other land-use or land-cover changes.
These combinations help scientists understand the current and future interplay between the natural world and human activities. To ensure a model is correctly representing the present, they compare modeled results to real-life observations. The predictions looking decades into the future help people planning bridges or power plants understand how their long-term choices could affect or be affected by these changes.

A Model That Revolves Around Energy

DOE’s new E3SM earth system model can simulate storms with surface winds faster than 150 miles per hour. This picture from the simulation shows how the storms affect the sea’s surface temperatures in ways that can influence future hurricanes.

While there’s only one Earth, there are many earth system models. The E3SM team members designed their model to answer questions relevant to DOE’s mission.
One of the holy grails when it comes to energy planning is being able to predict how and when rain, snow, and other precipitation fall over land. Climate and land use changes could increase the atmosphere’s ability to hold moisture and cause storms to occur more frequently and intensely. In contrast, it could also lead to more evaporation, resulting in more drought. Or most likely, both could occur in different places around the world.
The new model focuses on three areas that have significant effects on surface precipitation, winds, and temperature as well as energy production: the water cycle, the way different earth system components exchange biogeochemical fluxes, and the movement and melting of ice sheets.
“To predict precipitation, you need to really understand almost every other part of the earth system,” said Leung. “The atmosphere is playing the role of connecting everything. The atmosphere is not local.”
Combining these systems into one model and getting the necessary level of detail isn’t something you can do on your desktop computer. Instead, it requires the big machines — the supercomputers at DOE Office of Science user facilities. E3SM programmers wrote the software to take full advantage of these computers’ hardware and operating systems. The team is also designing it so they could upgrade the model to run on DOE’s future exascale computers. These computers will be 50 times more powerful than today’s fastest computers.
Read the full article on the Department of Energy Office of Science website.
# # #
This work was supported by the U.S. Department of Energy’s Office of Science, Office of Biological and Environmental Research as part of the Atmospheric System Research Program.