This work represents a new paradigm for climate model development: combining physically rigorous design with modern machine learning to create parameterizations that are accurate, stable, efficient, and require minimal training data.
Figure 1. High-resolution simulation of convection near the ocean surface. Top row, from left to right: buoyancy (how much a fluid parcel floats), vertical velocity. Bottom row, from left to right: horizontally-averaged temperature and salinity. Run with isotropic resolution of 0.5m on a GPU using Oceananigans.jl, the fastest ocean model ever.
The ocean is a fluid which is warm at the top and cold at the bottom, since cooler fluid is generally denser. The complex interaction between atmosphere, ocean, and ice means that the ocean is a turbulent fluid. Observe the video above, which shows a high-resolution simulation of ocean convection near the surface. Just as your coffee cools when left out, the ocean surface loses heat to the atmosphere. This creates cooler plumes which are denser and thus sink from the surface into the interior. You can see the vertical velocity plot, where blue means a downward motion, while red means an upward motion. This turbulent convection causes a well-mixed layer to form near the surface called the boundary layer. From the plots in the bottom row you can see the horizontally-averaged temperature and salinity fields—note the mixed layer near the surface which deepens with time.
Figure 2. High-resolution simulation of winds blowing over the ocean surface. Top row, from left to right: buoyancy (how much a fluid parcel floats), vertical velocity. Bottom row, from left to right: horizontally-averaged temperature and salinity. Run with isotropic resolution of 0.5m on a GPU using Oceananigans.jl, the fastest ocean model ever.
Now let’s look at this other animation above. In this simulation, we demonstrate a different process—wind-driven mixing. This is exactly the phenomenon where winds blow on the surface of the ocean, or when you blow on your hot coffee to cool it down. This also mixes the fluid and forms a mixed layer that deepens with time, similar to convection. However, if you compare the vertical velocity structure between wind- and convective-driven cases, you will find that their structures are quite different. In this wind-driven case, you see less of those large-scale plume structures that are present in convection. Even though their overall effects on the structure of the ocean surface may be similar, the governing physics are quite different.
Why is this important?
The ocean takes up a huge portion of the additional heat that was created from greenhouse effect due to its much larger heat capacity when compared to the atmosphere. The temperature difference between the ocean and atmosphere at the interface critically affects the rate of heat uptake, and that is primarily governed by the small scale processes I showed above. In order for us to understand and make accurate predictions about the future climate, getting these processes right are absolutely crucial! However, with the computational power that we have in the foreseeable future (think decades to centuries), we will never be able to have climate models which are high-resolution enough to capture these effects. The state-of-the-art ocean models have a resolution of around 10km, while you need around 1m resolution to be able to resolve these processes. Therefore, they have to be modeled, or parameterized as we call it in the climate community.
However, despite efforts for many decades, modeling convection and small-scale wind-driven mixing has proven to be a challenging task, especially considering the desired requirements of parameterizations which are: high numerical stability, low computational cost, and of course high accuracy.
Current climate models often predict mixed layer depth with errors comparable to the depth itself—particularly in the Southern Ocean and tropics where these processes are most critical.
Therefore, NORi is our effort in trying to tackle this issue again, but with newer techniques (AI) which has emerged over the past years.
What is NORi? 海苔 🍘?
NORi—Neural ODEs (Ordinary Differential Equations) + Richardson number (Ri) closure is a novel parameterization for ocean boundary layer turbulence that combines physics-based modeling with machine learning. We try to solve this problem by incorporating our existing knowledge about the dynamics of the processes using a physical model, and then delegating the remaining effects which we do not have a good handle of using neural networks. In particular, we use neural networks to capture entrainment, the anti-diffusive process where surface-driven turbulence penetrates into the stratified ocean interior. Entrainment happens in convection when the vertical plumes plunge toward the base of the mixed layer and overshoot into the ocean interior due to inertia, bringing colder fluid from the interior into the mixed layer at the same time (see Figure 1 at the beginning for entrainment in action).
If you look at the 3rd and 4th plots from the left in Figures 1 and 2, you’d see two different lines—large-eddy simulations and NORi. Large-eddy simulations (LES) are the horizontally-averaged values we get from running these high-resolution simulations, and NORi is our neural network model. You can see the two lines line up quite closely, meaning that NORi can accurately predict the processes we showed in the LES. In fact, in all of the training and validation cases we have tested it on, which are based on realistic conditions, NORi performs really well. We are confident that it is indeed capable of representing the entrainment physics which are challenging to model using physics-based equations.
Key Innovations
…can’t say too much right now. Will update once we submit the paper for publication!
The people who made this possible
This work would not have happened without these brilliant scientists and wonderful people: Ali Ramadhan, Andre Souza, Gregory L. Wagner, Simone Silvestri, John Marshall, Raffaele Ferrari.
Code and Resources
For now, I have released the training data that I’ve used to train the model. It contains more than 100 large-eddy simulations (LES) of ocean surface boundary layer mixing due to convection and wind stress just like the ones you see above. It uses the realistic seawater’s equation of state (TEOS-10), and covers many characteristic ocean states such as midlatitude Atlantic, equatorial Pacific and the Southern Ocean. It contains the horizontally-averaged vertical profiles of the fields. Feel free to play around with it.
This work is also enabled by Oceananigans.jl, which is currently the fastest ocean model on earth. It can do lots of things in general beyond the ocean. More specifically, it does large-eddy simulations and hydrostatic simulations of incompressible fluids really well. Give it a try, it is fast and friendly!
Paper link and code repository coming soon…