Fourier Neural Operators (FNOs) are an advanced framework in deep learning designed for solving PDEs by learning mappings between functional spaces. They extend neural network architectures to directly model complex operators, offering efficient and accurate solutions for high-dimensional problems, including stochastic partial differential equations (SPDEs). FNOs leverage the Fourier transform to capture and propagate information globally, making them especially powerful for tasks involving complex, multiscale behavior.

1. Concept of Neural Operators

• Operators vs. Functions: Traditional neural networks approximate functions $f: \mathbb{R}^n \to \mathbb{R}^m$ . Neural operators, in contrast, learn mappings between spaces of functions, such as $\mathcal{G}: L^2(\Omega) \to L^2(\Omega)$ . This allows them to solve entire families of PDEs with varying inputs rather than just specific instances.
• Objective: The goal of a neural operator like an FNO is to model the underlying transformation $\mathcal{G}$ that takes an input function (e.g., initial conditions or forcing terms) and maps it to the output solution.

2. Fourier Neural Operator Architecture

The Fourier Neural Operator applies the Fourier transform to learn and apply global convolutions efficiently, bypassing some limitations of traditional convolutional neural networks (CNNs) which are inherently local.

Key Components:

1. Fourier Transform Layer:

   • The FNO uses the Fast Fourier Transform (FFT) to map the input function $u(x)$ from physical space to Fourier space, representing it by its frequency components.
   • This enables the network to capture global interactions more effectively than spatial convolutions.

2. Spectral Convolution:

   • In Fourier space, a learned kernel $P(\xi)$ is applied directly to the frequency components $\hat{u}(\xi)$ :

   $\hat{v}(\xi) = P(\xi) \hat{u}(\xi),$

   where $\hat{u}(\xi)$ and $\hat{v}(\xi)$ are the Fourier transforms of the input and output, respectively.

3. Inverse Fourier Transform:

   • The modified frequency representation $\hat{v}(\xi)$ is transformed back to the physical space using the inverse FFT to produce the output function.

4. Iterative Layers:

   • The process is repeated over multiple layers to improve the expressiveness and accuracy of the learned mapping, resembling residual layers in standard deep networks.

3. Mathematical Formulation

Given an input function $u_0(x)$ , the FNO approximates the solution $u(x)$ by repeatedly applying a global convolution in Fourier space and transforming it back to physical space:

1. Initial Lift:

   • A linear layer maps the input $u_0$ to a higher-dimensional space:

   $u^{(0)}(x) = W_{\text{lift}} u_0(x).$

2. Fourier Convolution:

   • Each layer computes:

   $\hat{u}^{(k+1)}(\xi) = P^{(k)}(\xi) \hat{u}^{(k)}(\xi),$

   followed by:

   $u^{(k+1)}(x) = \text{IFFT}(\hat{u}^{(k+1)}(\xi)).$

3. Nonlinearity and Update:

   • The output is updated with an activation function and a bias term:

   $u^{(k+1)}(x) = \sigma(u^{(k+1)}(x) + b^{(k)}).$

4. Final Projection:

   • A final linear layer projects the result back to the desired output dimension:

   $u(x) = W_{\text{proj}} u^{(K)}(x).$

4. Advantages of Fourier Neural Operators

• Global Context: FNOs capture global dependencies through spectral convolution, making them ideal for problems with long-range interactions or periodic boundary conditions.
• Computational Efficiency: The FFT reduces the computational complexity of the convolution from $O(N^2)$ (for traditional convolutions) to $O(N \log N)$ , where $N$ is the number of data points.