Fourier Neural Operators (FNOs) are an advanced machine learning method designed to learn mappings between functions, making them well-suited for solving parameterized partial differential equations (PDEs). These operators leverage the power of Fourier transforms to efficiently capture global information and complex interactions in high-dimensional spaces. Here, I will outline the structure, working mechanism, and applications of FNOs in the context of parameterized PDEs.

1. What are Fourier Neural Operators?

• Definition: Fourier Neural Operators are neural network architectures that learn mappings between function spaces. They extend traditional neural network methods to solve PDEs by operating in the Fourier domain.
• Goal: To provide a framework that can learn solution operators for families of PDEs with varying parameters (e.g., coefficients, boundary conditions, or forcing terms).

2. Mathematical Background

• Parameterized PDEs: Consider a PDE of the form:

$\mathcal{L}_a u(x) = f(x), \quad x \in \Omega,$

where $\mathcal{L}_a$ is a differential operator that depends on a parameter $a$ , $u(x)$ is the solution, and $f(x)$ is the source term.

• Solution Operator: The goal is to learn an operator $\mathcal{G}$ such that:

$u(x) = \mathcal{G}(f, a)(x).$

3. Key Concepts in Fourier Neural Operators

• Fourier Transform: The core idea of FNOs is to represent functions in the frequency domain by applying the Fourier transform. This allows for efficient manipulation of global features in the data.
• Non-Local Information: Unlike traditional finite element or finite difference methods, which rely on local stencils, FNOs can capture non-local dependencies by working in the Fourier space, where interactions across the entire domain are easier to represent.

4. Architecture of Fourier Neural Operators

The FNO architecture typically involves the following steps:

1. Lifting Layer: Maps the input $f(x)$ from the input space to a higher-dimensional feature space:

$v_0(x) = P(f(x)),$ where $P$ is a linear or nonlinear transformation. 2. Fourier Layers: - Fourier Transform: Transform $v(x)$ into the Fourier space to obtain its frequency components:

$\\hat{v}(k) = \\mathcal{F}v,$
where  $k$  represents the frequency.
- **Kernel Multiplication**: Apply a learnable kernel  $\\hat{K}(k)$  in the Fourier space:

$\\hat{u}(k) = \\hat{K}(k) \\hat{v}(k).$
- **Inverse Fourier Transform**: Transform back to the spatial domain:

$u(x) = \\mathcal{F}^{-1}\\hat{u}.$

3. Nonlinear Activation: Apply a pointwise nonlinear activation to introduce nonlinearity:

$v_{i+1}(x) = \sigma(u(x)),$ where $\sigma$ is an activation function like ReLU or tanh. 4. Stacking Layers: Repeat the Fourier layer process for multiple iterations to capture complex relationships. 5. Projection Layer: Finally, project the output back to the original space to obtain the solution $u(x)$ .

5. Advantages of Fourier Neural Operators

• Efficiency in High Dimensions: FNOs perform well in high-dimensional spaces since the Fourier transform is computationally efficient, scaling better than grid-based methods.
• Global Receptive Field: The Fourier transform allows FNOs to capture long-range dependencies without increasing the model's complexity.