Caltech Open-Sources FNO: A Deep Learning Method For Solving PDEs (Partial differential equations)

Caltech’s Dolcit group recently open-sourced FNO, Fourier Neural Operator, a deep-learning method for Solving the PDEs (Partial differential equations). FNO being three times faster than traditional solvers outperforms the existing deep-learning techniques for solving PDEs. FNO is used to speed up the calculations and weather predictions.

The model and experiments were described in a paper published on arXiv. For the family of PDEs that are invariant of mesh resolution, the neural network can represent solutions By learning a mapping from one function to another. 

By applying a Fourier transformation, the model efficiently calculates a global convolution. The error rates achieved by FNO are (comparatively) 30% lower on Navier-Stokes equations and 60% lower on the Darcy flow. 

PDEs find use in many engineering and physics areas as they describe a wide variety of phenomena, including fluid dynamics, heat transfer, and quantum mechanics. The solution of the PDE is a function, often of time and space. However, when we talk practically, there is no closed-form PDE solution, which forces the scientists and engineers to resort to numerical approximations, using finite difference methods (FDM)or finite element methods (FEM). A fine-grained mesh of discrete points of interest is created in the above Techniques, and the PDE’s behavior in a small neighborhood is analyzed around each mesh point for a short span of time. 

These methods have some disadvantages, which are as follow:

  1. The process is time-consuming.
  2. The process must be rerun if any changes are made to the parameters of the problem or the grid definition.

Deep-learning and Neural networks have shown excellent results in speeding up scientific simulations. The PDEs are often solved by producing a model that can quickly generate sample data for statistical analysis. The above is applied in the inverse problem where the final observations are given, and we try to determine a system’s initial conditions. The two previous deep-learning approaches are:

  1. Finite-dimensional operators: The models are not mesh-independent. They use convolutional neural networks (CNN) and produce a parameterized approximation of a solution.
  2. Neural FEM: They are mesh-independent but represent the solution for only a specific instance of a PDE. FEM has to be trained again in case the parameters are changed.

The team’s approach is to build a neural network that can learn the mapping between a PDE and its solution. The FNO work is built on the previous graph kernel network (GKN) paper presented by the team at the recent NeurIPS conference.