Fourier Neural Operators (FNOs) have gained significant attention for their ability to solve partial differential equations (PDEs) and stochastic partial differential equations (SPDEs) efficiently. Unlike traditional neural network architectures that map finite-dimensional inputs to outputs, FNOs operate directly on functions and can generalize to solutions across different resolutions. This makes them powerful for learning the mappings in SPDEs where stochasticity and high-dimensional function spaces are involved.

1. Overview of Fourier Neural Operators (FNOs)

• Concept: FNOs are a type of operator-learning model designed to map between function spaces, making them well-suited for learning the solution operator of PDEs and SPDEs. They leverage the Fourier transform to capture and process global information, which enhances their ability to model complex and high-dimensional dynamics.
• Architecture: The core idea involves applying the Fourier transform to a function, processing the transformed representation, and then applying the inverse Fourier transform to map it back to the original domain.

2. Why Use FNOs for SPDEs?

• Efficiency in High Dimensions: Traditional numerical solvers become computationally prohibitive as the dimensionality of the problem increases. FNOs can handle high-dimensional problems more efficiently because they process the global structure of data in Fourier space.
• Stochastic Processing: FNOs can be adapted to handle stochastic processes by learning mappings that account for the randomness present in SPDEs.
• Resolution-Invariance: FNOs can generalize across different grid resolutions, making them adaptable for solving SPDEs at various scales.

3. Mathematical Foundation of FNOs

• Problem Setup: Consider an SPDE of the form:

$\frac{\partial u(t, x)}{\partial t} = \mathcal{L}u(t, x) + \sigma(u(t, x)) \dot{W}(t, x),$

where $u(t, x)$ is the solution, $\mathcal{L}$ is a differential operator, $\sigma$ is a noise coefficient function, and $\dot{W}(t, x)$ represents white noise.

• Neural Operator Framework:

$\mathcal{G}\theta: u_0(x) \mapsto u(t, x),$

where $\mathcal{G}\theta$ is a neural operator parameterized by $\theta$ , mapping initial conditions $u_0(x)$ to the solution $u(t, x)$ .

4. Fourier Transform in FNOs

• Fourier Transform: Converts a function from the spatial domain to the frequency domain:

$\hat{u}(k) = \mathcal{F}[u(x)],$

where $\hat{u}(k)$ is the Fourier transform of $u(x)$ , and $k$ represents the frequency components.

• Inverse Fourier Transform: Used to map processed frequency data back to the spatial domain:

$u(x) = \mathcal{F}^{-1}[\hat{u}(k)].$

5. Architecture of Fourier Neural Operators

1. Input Layer:
   • Takes a discretized representation of $u_0(x)$ or an intermediate function state.
2. Fourier Layer:
   • Applies the Fourier transform to map the input to the frequency domain.
3. Spectral Processing:
   • Processes the frequency representation using linear or non-linear transformations to modify the amplitude and phase of each component. This step often uses parameterized multipliers to learn how different frequencies should be weighted.
4. Inverse Fourier Transform:
   • Transforms the modified frequency data back to the spatial domain to produce an updated representation of the function.
5. Nonlinear Activation:
   • Applies a non-linear activation function (e.g., ReLU, tanh) to enhance expressivity.