CAFE+FNO: Fourier Kernel Generation via Multiplicative Feature Composition

arXiv:2610.10105 · cs.LG, cs.NA, math.NA · Submitted 2026-10-07 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "CAFE+FNO: Fourier Kernel Generation via Multiplicative Feature Composition".

Jane: The gist The CAFE+FNO introduces explicit, learnable feature composition into Fourier kernel generation, which achieves lower mean errors on four out of five PDE benchmarks compared to existing variants.

Tom: First, who's behind it and why it matters.

Paper summary: Tom: To recap, "CAFE+FNO: Fourier Kernel Generation via Multiplicative Feature Composition" introduces an explicit way to compose features into Fourier kernel generation by incorporating Content-Aware Frequency Encoding plus, or CAFE+, into the existing AM-FNO framework.

Jane: Essentially, they propose that instead of just relying on subsequent neural networks to implicitly form frequency interactions, they build these interactions explicitly using Fourier–Chebyshev features combined with a Hadamard product.

Lu: The authors argue that this construction creates learnable interactions among spectral-coordinate features before the kernel prediction stage, which allows the model to retain amortized parameterization over all stored modes without needing extra frequency truncation.

Meng: They are focusing on shifting the focus from storing mode-specific kernel coefficients to approximating a frequency-to-kernel function that is shared across all modes, which is a key architectural change.

Lalam: The abstract states they achieve the lowest mean error on four out of five PDE benchmarks compared to existing variants, specifically showing error reductions relative to SirenFNO by eighteen point four–sixty point two percent on those tasks <ref:2610.10105#pg2,relative to SirenFNO by 18.4–60.2% on>.

Tom: So the main claim is that this method achieves lower mean errors on Darcy, NS, Burgers, and Airfoil when compared against SirenFNO on those specific problems.

Jane: But they also admit that SirenFNO performs better specifically on the ReacDiff benchmark, which is a good bit of nuance to include in any discussion about model performance.

Lu: The paper sets up the problem by considering an integral operator approximation, where they parameterize the solution operator G using a neural network and reduce discrepancy between predictions and observations.

Meng: They are looking at approximating this solution operator by mapping input functions to a hidden function and then repeatedly applying layers consisting of an integral operation.

Tom: That sets the stage for how they map input functions into a hidden space before the actual kernel generation part kicks in.

Jane: The whole point is that this shift toward explicit feature composition helps improve the representation used to generate mode-dependent complex kernels from spectral coordinates, according to what they describe on page two of THIS PAPER — CAFE+FNO: Fourier Kernel Generation via Multiplicative Feature Composition.

Conclusion: Tom: So, looking at the whole paper, "CAFE+FNO: Fourier Kernel Generation via Multiplicative Feature Composition," the authors are showing that explicit feature composition is a viable design choice for generating Fourier kernels in these operator models.

Jane: It boils down to incorporating CAFE+ into AM-FNO to construct those learnable interactions between spectral and coordinate features before the final kernel prediction step.

Lu: The implication is that we can improve the representation of the spectral coordinate itself, making it more structured for learning frequency interactions in a way that might be more efficient than relying on implicit MLPs to figure it out later.

Meng: For deployment, this means we have a model with better performance on most common benchmarks without necessarily increasing the complexity of how we store the modes themselves.

Lalam: Lalam is sure that this paper suggests a path toward better inductive biases in representation learning for these types of problems, because they explicitly define how those spectral features should interact.

Tom: So, the title points to this idea: Fourier Kernel Generation via Multiplicative Feature Composition, and the result is that combining Fourier–Chebyshev encoding with multiplicative composition leads to lower mean errors on four of five PDE benchmarks.

Hyungjoon Juen, Minwoo Shin

Department of Software, Yonsei University (Mirae Campus)

cs.LG, cs.NA, math.NA

Submitted: 2026-10-07

Updated: 2026-10-07

Code: https://github.com/fabsk101/CAFEPlusFNO

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 83/100

The gist: The gist The CAFE+FNO introduces explicit, learnable feature composition into Fourier kernel generation, which achieves lower mean errors on four out of five PDE benchmarks compared to existing

Key concepts

Content-Aware Frequency Encoding (CAFE+): This technique incorporates CAFE+ into the Fourier kernel generation of AM-FNO. It constructs learnable interactions among spectral-coordinate features before predicting the kernel, allowing for more nuanced feature combination than previous approaches.
Fourier–Chebyshev Feature Encoding: This involves defining Fourier features using Gaussian matrices and Chebyshev features using polynomials along each coordinate axis. These encoded features are then combined through linear branches and a Hadamard product to create a representation zℓ(ξk).
Multiplicative Frequency Composition: This stage feeds the representation into parallel branches, where each branch applies an affine transformation. Combining these outputs via a Hadamard product enables the network to selectively adjust frequency interactions, including constructing higher-order polynomial components.
Complex Fourier Kernel Generation: The final representation is passed to a complex matrix-valued kernel generator Mℓ implemented by an MLP. This generates the complex kernel Rℓ(k) from the learned feature representation zℓ(ξk), ensuring shared parameters across all Fourier modes.

Summary

The gist The CAFE+FNO introduces explicit, learnable feature composition into Fourier kernel generation, which achieves lower mean errors on four out of five PDE benchmarks compared to existing variants.

How it works

CAFE+FNO incorporates Content-Aware Frequency Encoding (CAFE+) into the amortized Fourier kernel generation of AM-FNO to construct learnable interactions among spectral-coordinate features before kernel prediction. This approach retains amortized parameterization over all stored modes without additional frequency truncation while introducing a learnable feature-composition stage before kernel prediction. The method constructs interactions within and across the two feature families by combining Fourier–Chebyshev features through parallel affine branches and a Hadamard product.

Fourier–Chebyshev Feature Encoding

The encoding process involves defining Fourier features using Gaussian matrices sampled from a standard normal distribution to map the normalized spectral coordinate to m pairs of Fourier features. These are defined as sin(2πω⊤i x), cos(2πω⊤i x) for M Fourier frequency vectors ωi. Alongside these, CAFE+ uses J Chebyshev polynomials of the first kind T0,..., TJ−1 along each coordinate axis to construct the Chebyshev features. The concatenated features pass through Nb linear branches whose outputs are combined using a Hadamard product to form zl(ξk).

Multiplicative Frequency Composition

The multiplicative composition stage involves feeding the representation γl(ξk) into Nb parallel branches, where each branch applies an affine transformation with a bias. The outputs of these branches are combined by a Hadamard product to yield zl(ξk). This composition allows the network to selectively adjust the frequency interactions needed for kernel generation, as products of branch outputs can include basis frequencies corresponding to sums and differences of encoding frequencies. Furthermore, products of Chebyshev components can construct higher-order polynomial components absent from the initial encoding or recombine lower-order components.

Complex Fourier Kernel Generation

The resulting representation zl(ξk) is passed to a complex matrix-valued kernel generator Ml, which is implemented using a single real-valued MLP fl: R dz → R 2d squared v. The kernel corresponding to Fourier mode k is then defined as Rl(k) = Ml zl(ξk) = Rel(k) + iIml(k). The same Ml and encoding parameters are shared across all Fourier modes within layer l, making the number of trainable parameters independent of the number of modes for a fixed architecture.

Experimental Results

The dense CAFE+FNO achieves the lowest mean error on Darcy, NS, Burgers, and Airfoil. Relative to SirenFNO, errors decrease by 43.7%, 60.2%, 29.5%, and 18.4% on these tasks. Ablation studies show that the combined Fourier–Chebyshev encoding yields lower mean errors than either single-basis variant on all five datasets. Finally, CAFE+FNO†, with learnable RFF bandwidth ρl, yields lower mean errors than the fixed bandwidth default on NS and ReacDiff.

Conclusion

The results support explicit spectral-coordinate feature composition as an effective design choice for Fourier kernel generation. The dense model achieves the lowest mean error on four of five benchmarks, while SirenFNO performs better on ReacDiff. Factorized variants use fewer parameters than dense CAFE+FNO, generally at some cost in accuracy. Future work will focus on extending CAFE+FNO to a broader range of PDE problems and higher-dimensional spatial domains.

Improvements for AI systems

Bold kernel generation via CAFE+FNO: The system can generate Fourier kernels for all grid modes by incorporating Content-Aware Frequency Encoding+ which combines Fourier–Chebyshev features through parallel affine branches and a Hadamard product, forming interactions within and across the two feature families. This allows the model to capture richer frequency interactions than standard FNO variants.
Amortized parameterization independence: The system can maintain a parameter count independent of the number of modes for a fixed architecture because each layer shares its generator across all modes, shifting focus from storing mode-specific coefficients to approximating a frequencyto-kernel function shared across all modes.
Enhanced spectral interaction modeling: The model can construct complex frequency interactions by allowing the representation to include basis frequencies corresponding to the sums ωl,i + ωl,j and differences ωl,i − ωl,j through the Hadamard product of parallel branches. This allows for selective adjustment of the frequency interactions needed for kernel generation.
Reduced parameter overhead via factorization: The system can achieve significant reductions in trainable parameters by employing CP, TT, or Tucker decompositions where factor generators use reduced dimensions (e.g., 16 Fourier frequency vectors and eight Chebyshev polynomials), leading to parameter counts up to 86.1%, although this is balanced against accuracy trade-offs.
Targeted error reduction on specific PDEs: The improved system can achieve a relative L2 error of as low as 3.843e-03 on the ReacDiff benchmark, demonstrating superior performance compared to SirenFNO (2.36 versus 3.80, ×10−3) for that specific problem.