U-NO: : U-shaped Neural Operators

Abstract

Neural operators generalize classical neural networks to maps between infinite-dimensional spaces, e.g., function spaces. Prior works on neural operators proposed a series of novel methods to learn such maps and demonstrated unprecedented success in learning solution operators of partial differential equations. Due to their close proximity to fully connected architectures, these models mainly suffer from high memory usage and are generally limited to shallow deep-learning models. In this paper, we propose a U-shaped Neural Operator (U-NO), a U-shaped memory-enhanced architecture that allows for deeper neural operators. U-NOs exploit the problem structures in function predictions and demonstrate fast training, data efficiency, and robustness with respect to hyperparameter choices. We study the performance of U-NO on PDE benchmarks, namely, Darcy’s flow law and the Navier-Stokes equations. We show that U-NO results in an average of 26% and 44% prediction improvement on Darcy’s flow and turbulent Navier-Stokes equations, respectively, over the state-of-the-art. On the Navier-Stokes 3D spatiotemporal operator learning task, we show U-NO provides 37% improvement over the state-of-the-art methods.

Paper: U-NO: U-shaped Neural Operators

UNO_Tutorial.ipynb - A step-by-step tutorial for using and buidling U-NO. Link to Google colab

U-NO is now available on Neural Operator library. Quick Start

Requirements

PyTorch 1.11.0

Files

Files	Descriptions
integral_operators.py	Contains codes for Non-linear integral operators for 1D, 2D, and 3D functions.
UNO_Tutorial.ipynb	A tutorial on using the integral operators and U-NO.
Darcy Flow
darcy_flow_main.py	Script for loading data, training, and evaluating training UNO performing 2D spatial convolution for solving Darcy Flow equation.
darcy_flow_uno2d.py	UNO architectures for solving Darcy Flow equation.
train_darcy.py	Training routine for Darcy flow equations.
data_load_darcy.py	Function to load Darct-flow data.
Navier–Stokes
data_load_navier_stocks.py	Function to load Navier–Stokes data generated by data generator prodived
ns_uno2d_main.py	Script for loading data, training, and evaluating the UNO (2D) autoregressive in time for Navier–Stokes equation.
ns_train_2d.py	Training function for UNO(2D) in time for Navier–Stokes equation
navier_stokes_uno2d.py	UNO(2D) architecture in time for Navier–Stokes equation.
ns_uno3d_main.py	Script for loading data,training and evaluating the UNO(3D) performing 3D (spatio-temporal) convolution for Navier–Stokes equation.
navier_stokes_uno3d.py	UNO(3D) achitectures performing 3D convolution for Navier–Stokes equation.
ns_train_3d.py	Training function for UNO(3D) for Navier–Stokes equation.
Supporting Files
Data Generation	Folder contains scripts to generate data from Navier–Stokes equation and Darcy flow
utilities3.py	Contains supporting functions for data loading and error estimation.

Data

Link to two files containing 2000 simulations of the Darcy Flow equation: Google Drive Link

The Data Generator folder contains the script for generating a simulation of the Darcy Flow and Navier-Stocks equation.