Adaptive Local Frequency Filtering for Fourier-Encoded Implicit Neural Representations

arXiv:2604.02846 · cs.CV, eess.IV · Submitted 2026-04-03 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Adaptive Local Frequency Filtering for Fourier-Encoded Implicit Neural Representations".

Jane: Here's the gist of the paper. Fourier-encoded implicit neural representations have shown strong capability in modeling continuous signals from discrete samples The proposed method introduces a spatially varying parameteralpha(x) to modulate encoded Fourier components, enabling a smooth transition among low-pass, band-pass, and high-pass behaviors at different spatial locations Experiments on 2D image fitting, 3D shape representation, and sparse data reconstruction demonstrate that the proposed method consistently improves reconstruction quality and leads to faster optimization compared to fixed-frequency baselines The motivation stems from the fact that natural signals often exhibit spatially varying local spectra, where smooth regions are dominated by low-frequency components, whereas edges and fine structures require stronger high-frequency responses Applying a fixed Fourier basis uniformly is suboptimal because it cannot adapt to these local differences in frequency demand The authors propose that an effective representation should emphasize only the frequency components most relevant to the local signal structure, rather than activating a broad fixed frequency set everywhere Mechanism of Local Frequency Modulation The core mechanism involves introducing a spatially varying parameteralpha(x) to modulate the local frequency response of Fourier features This modulation is applied element-wise to the encoded Fourier feature vector gamma(x) by computing a filtered representation gamma'(x) through element-wise multiplication The filter produces channel-wise responses denoted as HB(alpha(x)) which are then used to compute the filtered Fourier representation gamma'(x) = HB(alpha(x)) ⊙ gamma(x) NTK-Inspired Interpretation of Learning Dynamics The authors analyze the method from the Neural Tangent Kernel (NTK) perspective to understand how it reshapes the effective kernel spectrum during training This analysis shows that the proposed filter rescales the contribution of each dyadic frequency scale by a factor H̄j(alpha(x))2, leading to a local effective eigenvalue approximation lambdaALj(x) ≈ H̄j(alpha(x))2lambdaj This suggests that higher-frequency kernel components receive larger effective weights in regions where the adaptive parameter favors them, which accelerates the learning of local details Empirical Validation and Performance Gains Experimental results on representative INR tasks demonstrate improvements in reconstruction quality and convergence speed over fixed-frequency baselines In 2D image fitting, the Ours-S variant achieved the best performance across PSNR, SSIM, and LPIPS metrics compared to PE-MLP, MFN, SIREN, and DINER For 3D shape representation with SDFs, the proposed method obtained the best average performance in both Chamfer distance and IoU across four tested shapes In sparse data reconstruction, the method successfully recovers overall scene structure even at 5% observation levels Limitations and Future Directions The proposed method has limitations, primarily depending on the resolution of the trainable parameter grid used to represent alpha(x) Furthermore, it introduces several task-dependent hyperparameters such as filter bandwidth B and encoding level L which may require tuning for different datasets Future work could explore adaptive grid allocation or automatic hyperparameter selection to reduce manual tuning The proposed method offers a practical way to enhance Fourier-encoded INRs with explicit local frequency control by adapting the encoding to spatially varying signal content These results suggest that explicit local frequency control is a promising direction for improving INR-based reconstruction methods in settings with spatially varying spectral content The paper concludes that adaptive local frequency modulation is a practical enhancement for Fourier-encoded INRs

Paper summary: Tom: Let’s talk about this paper now. "Adaptive Local Frequency Filtering for Fourier-Encoded Implicit Neural Representations." The core idea is that while Fourier-encoded networks can model continuous signals from discrete samples, they usually use a fixed set of frequencies across the entire spatial domain, which doesn't work well for signals where the frequency content changes depending on where you are.

Jane: That’s the problem. Natural signals often have spatially varying local spectra. Smooth areas are dominated by low frequencies, but edges and fine structures need stronger high-frequency responses in those specific locations. Conventional Fourier feature mappings fail to adapt their frequency focus locally, which slows down the learning process for those important local details.

Lu: The authors tackle this by introducing a spatially varying parameter alpha to modulate these encoded Fourier components. This allows the network to smoothly transition between low-pass, band-pass, and high-pass behaviors depending on the spatial location being processed. It’s about giving the network local frequency control.

Meng: So it’s not just one fixed frequency set; it’s a dynamic adjustment based on where you are in the input space. That sounds like it could be really powerful for modeling complex, heterogeneous data, but I need to know how much computational overhead that dynamic modulation adds during training and inference.

Lalam: The mechanism involves taking the encoded Fourier feature vector gamma and computing a filtered representation gamma prime through element wise multiplication with a filter HB of alpha, which results in channel wise responses HB of alpha being used to compute gamma prime equals HB of alpha dot product with gamma.

Tom: That’s the core math—element wise multiplication—which is the mechanism for this spatial modulation. The authors are using this setup to build a representation that is inherently aware of local spectral variations.

Jane: They aren't just patching a problem; they’re proposing an approach where the encoding itself becomes adaptive based on the signal's local characteristics, which is what makes it matter for modeling real-world continuous signals better.

Lu: And they aren't just showing off; they analyze this method from the Neural Tangent Kernel perspective to provide an NTK inspired interpretation of how this filter reshapes the effective kernel spectrum during training. It gives us a deeper understanding of why it works in terms of learning dynamics.

Meng: An NTK interpretation is interesting for understanding *why* it learns faster, but what does that actually mean for deployment? Does it make the model smaller or just more complex to manage on a server?

Lalam: The analysis shows that this filter rescales the contribution of each dyadic frequency scale by a factor H bar j of alpha squared, which leads to a local effective eigenvalue approximation lambda AL j x approximately equals H bar j of alpha squared lambda j. This means higher frequency kernel components get larger weights where the adaptive parameter favors them.

Tom: So, if we look at the numbers, it suggests that the network learns local details faster in regions where alpha is set up to emphasize those high frequencies. That's a tangible speedup mechanism they’re proposing.

Jane: It makes sense because natural signals are inherently non-uniform; you can’t treat every part of an image or shape the same way with a single frequency band. This method lets the network respect that spatial reality directly.

Lu: And empirically, they demonstrate on representative INR tasks that this method consistently improves both reconstruction quality and convergence speed compared to fixed-frequency baselines across different benchmarks like 2D image fitting and three dee shape representation with SDFs.

Meng: The empirical validation is encouraging, but I still need those real traffic data results. Until we see how it handles the noise and variability of production inputs, it’s just great lab performance.

Lalam: The results show that in 2D image fitting, the Ours-S variant achieved the best performance across PSNR, SSIM, and LPIPS metrics when compared against models like PE MLP and DINER <ref:2604.02846#pg2>.

Conclusion: Tom: So, looking at the paper "Adaptive Local Frequency Filtering for Fourier-Encoded Implicit Neural Representations," we see that the authors have successfully introduced a spatially varying parameter alpha to give Fourier-encoded networks explicit local frequency control over their feature components.

Jane: The implication is that for any task involving continuous signals, especially those with complex, localized spectral content like textures or fine structures, this method offers a way to tailor the network's frequency response precisely where it needs it most.

Lu: They’ve moved beyond just using fixed frequency sets across the whole input; they’ve shown that making the encoding itself adapt based on spatial location can lead to faster learning and better reconstruction quality in many complex modeling scenarios.

Meng: The main thing for us is that this suggests a path forward for making INR-based reconstruction methods more robust, provided we can figure out how to make the grid representation of alpha efficient enough for large scale deployment.

Tom: So, in simple terms, they’ve made these networks smarter by giving them a local knob to adjust their frequency focus based on the signal content at that specific spot. It’s a practical enhancement for improving reconstruction methods.

Jane: That’s right. It gives us a new tool to handle the inherent spatial variation in signals without having to rely on overly complex, fixed models everywhere.

Lu: Ultimately, it points toward a future where representation learning can be inherently more aware of spatial frequency demands, which is a big step for how we model continuous data.

Ligen Shia, Jun Qiub, Yuhang Zhengb, Zengyu Pangc, Chang Liub

College of Computer Science, Inner Mongolia University · Institute of Computational Imaging, Beijing Information Science and Technology University · School of Mathematics and Statistics, Wuhan University

cs.CV, eess.IV

Submitted: 2026-04-03

Updated: 2026-11-03

Comments: 10 pages, 5 figures

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 82/100

The gist: Fourier-encoded implicit neural representations have shown strong capability in modeling continuous signals from discrete samples and this paper proposes an adaptive local frequency filtering method…

Key concepts

Spatially Varying Parameter $\alpha(x)$
This is a function that changes its value based on the spatial location (x) within the input data. It acts as a control knob that dictates how much each frequency component of the Fourier encoding should be emphasized or suppressed at any specific point in space, enabling local adaptation.
Local Frequency Modulation
The core mechanism where $\alpha(x)$ is used to filter the encoded Fourier features. This involves multiplying the encoded features by a filter $H_B(\alpha(x))$, which selectively scales different frequency bands based on the local signal structure, ensuring that high frequencies are prioritized where needed.
NTK-Inspired Interpretation
Analyzing how the method affects training reveals that it rescales dyadic frequency scales. This means that higher-frequency components receive larger effective weights in areas where $\alpha(x)$ favors them, which effectively accelerates the learning process for fine local details during training.
Fourier-Encoded INR
This refers to a type of neural representation where the continuous function being modeled is decomposed into its constituent frequency components using Fourier analysis. The proposed method modifies this standard encoding by making it spatially aware, allowing the model to capture complex, spatially dependent spectral information more accurately.

Terminology

Summary

Fourier-encoded implicit neural representations have shown strong capability in modeling continuous signals from discrete samples and this paper proposes an adaptive local frequency filtering method to address spectral bias in these representations. p. 0

Where the paper says it

Adaptive Local Frequency Filtering for Fourier-Encoded Implicit Neural Representations Ligen Shia , Jun Qiub , Yuhang Zhengb , Zengyu Pangc and Chang Liub,∗ aCollege of Computer Science (College of Software), Inner Mongolia University, Hohhot, 010021, China b Institute of Computational Imaging, Beijing Information Science and Technology University, Beijing, 102206, China cSchool of Mathematics and Statistics, Wuhan University, Wuhan, 430072, China A R T I C L E I N F O Keywords: Implicit neural representation Adaptive filtering Spectral bias Neural tangent kernel A B S T R A C T Fourier-encoded implicit neural representations (INRs) have shown strong capability in modeling continuous signals from discrete samples. However, conventional Fourier feature mappings use a fixed set of frequencies over the entire spatial domain, making them poorly suited to signals with spatially varying local spectra and often leading to slow convergence of high-frequency details.

The proposed method introduces a spatially varying parameter alpha(x) to modulate encoded Fourier components, enabling a smooth transition among low-pass, band-pass, and high-pass behaviors at different spatial locations. p. 0
Where the paper says it

To address this issue, we propose an adaptive local frequency filtering method for Fourier-encoded INRs. The proposed method introduces a spatially varying parameter 𝛼(𝐱) to modulate encoded Fourier components, enabling a smooth transition among low-pass, band-pass, and high-pass behaviors at different spatial locations.

Problem Statement

Conventional Fourier feature mappings use a fixed set of frequencies over the entire spatial domain, which is poorly suited to signals with spatially varying local spectra and often leads to slow convergence of high-frequency details. Natural signals are often non-stationary and exhibit substantial spatial variation in their local spectral content, meaning smooth regions are usually dominated by low-frequency components, whereas edges, fine structures, and textured areas require stronger high-frequency responses. p. 0

Where the paper says it

However, natural signals are often non-stationary and exhibit substantial spatial variation in their local spectral content. Smooth regions are usually dominated by lowfrequency components, whereas edges, fine structures, and textured areas require stronger high-frequency responses.

Applying the same Fourier basis uniformly to all spatial locations is therefore suboptimal because it cannot adapt to these local differences in frequency demand. p. 0
Where the paper says it

Smooth regions are usually dominated by lowfrequency components, whereas edges, fine structures, and textured areas require stronger high-frequency responses. Applying the same Fourier basis uniformly to all spatial locations is therefore suboptimal, as it cannot adapt to these local differences in frequency demand [7, 8].

Methodology and Filter Design

The proposed method introduces an adaptive local frequency filter that acts directly on the encoded Fourier feature vector. This filter produces one response for each encoded channel, which is then element-wise multiplied with the encoded Fourier feature vector to create the filtered representation. p. 3

Where the paper says it

(6) denote the channel-wise filter responses, where 𝐶𝑁 = 2𝑑in𝐿 is the dimension of the encoded Fourier feature vector. The filtered Fourier representation is then defined as 𝛾 ′ (𝐱) = 𝐻𝐵(𝛼(𝐱)) ⊙ 𝛾(𝐱), (7) where ⊙ denotes element-wise multiplication.

The filter formulation is designed as the difference of two sigmoid functions to yield a smooth band-pass response with controllable center and bandwidth on the encoded-channel axis. p. 3
Where the paper says it

Design of the Learnable Filter 3. Filter Formulation To achieve explicit local frequency modulation, we design the filter as the difference of two sigmoid functions, which yields a smooth band-pass response with controllable center and bandwidth on the encoded-channel axis.

The adaptive parameter field alpha(x) is stored on a learnable grid and queried at each coordinate by multilinear interpolation to ensure spatial smoothness. p. 3
Where the paper says it

Therefore, the resulting response can still be interpreted as low-pass, band-pass, and high-pass behaviors. To ensure spatial smoothness, the parameter field 𝛼(𝐱) is stored on a learnable grid and queried at each coordinate by multilinear interpolation: 𝛼(𝐱) = (𝐱; 𝐀), (12) where 𝐀 denotes the learnable parameter grid and (⋅; 𝐀) denotes multilinear interpolation over the grid.

NTK-Inspired Analysis

The analysis from the Neural Tangent Kernel perspective provides an NTK-inspired view of how the filter reshapes the effective kernel spectrum during training. The local effective eigenvalue approximation is given by lambda AL(x) approx H̄ j(alpha(x)) squared lambda j, which suggests that the contribution of each dyadic frequency scale is reweighted by the factor H̄ j(alpha(x)) squared. p. 4

Where the paper says it

(17) This locally stationary approximation suggests that the adaptive filter rescales the contribution of each dyadic frequency scale by the factor 𝐻̄ 𝑗 (𝛼(𝐱))2 . If 𝜆𝑗 denotes the effective kernel eigenvalue associated with the 𝑗-th dyadic scale in the unfiltered case, then the corresponding local effective eigenvalue can be interpreted as 𝜆 AL 𝑗 (𝐱) ≈ 𝐻̄ 𝑗 (𝛼(𝐱))2𝜆𝑗 .

This local kernel reweighting view shows that in regions where alpha(x) activates encoded channels associated with higher dyadic frequency scales, the corresponding high-frequency kernel components receive larger effective weights, which can accelerate the learning of local details. p. 4
Where the paper says it

(18) provides an interpretable local view of how the proposed filter modifies learning dynamics. In regions where 𝛼(𝐱) activates encoded channels associated with higher dyadic frequency scales, the corresponding high-frequency kernel components receive larger effective weights, which can accelerate the learning of local details.

Empirical Validation

Experiments on 2D image fitting and 3D shape representation demonstrate that the proposed method consistently improves reconstruction quality and leads to faster optimization compared to fixed-frequency baselines. Quantitative results show that OursSine achieves the best performance across all three metrics, including PSNR, SSIM, and LPIPS in 2D image fitting. p. 7

Where the paper says it

9938 LPIPS↓ 2. 41e-3 The quantitative results in Table 3 show that OursSine achieves the best performance across all three metrics, including PSNR, SSIM, and LPIPS.

For 3D shape representation with SDFs, the proposed method obtains the best average performance in both Chamfer distance and IoU across the four shapes tested. p. 7
Where the paper says it

The proposed method achieves the best average performance in both Chamfer distance and IoU across the four shapes. In particular, it obtains the best Chamfer distance on Dragon and Lucy, and the highest IoU on Armadillo, Dragon, and Lucy.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Adaptive Local Frequency Filtering for Fourier-Encoded INRs allows models to emphasize relevant frequency components based on local signal content, meaning different regions can emphasize low-, band-, or high-pass behaviors according to their local spectral characteristics. This enables the AI system to better model non-stationary signals by tailoring its frequency response spatially.

  2. The method provides an NTK inspired interpretation where the effective kernel eigenvalue for each dyadic frequency scale is interpreted as lambda AL j (x) approximately equals H̄ j (alpha(x)) squared lambda j. This suggests that the proposed filter can be understood as a position-dependent mechanism that reshapes the local kernel spectrum and reduces spectral mismatch between fixed Fourier encoding and spatially varying signal content.

  3. The learned adaptive parameter map alpha x provides an intuitive visualization of spatial frequency preferences, showing higher values in regions with strong edges or dense textures, which helps explain the behavior of the model on non-stationary signals. This interpretability allows researchers to understand how the model adapts its local frequency response to image content.

  4. The system can achieve faster optimization by utilizing a method that exhibits faster early optimization and improved final reconstruction quality compared to fixed-frequency baselines, as demonstrated in 2D image fitting experiments where Ours-Sine achieved the best performance across PSNR, SSIM, and LPIPS metrics.

  5. For sparse data reconstruction, the AI system can recover substantially finer texture and edge details when only 35% of pixels are observed by imposing a total variation penalty on the adaptive parameter grid alpha x to encourage spatial smoothness under sparse supervision.

Abstract

Implicit neural representations (INRs) commonly employ Fourier features to alleviate spectral bias and represent high-frequency signals. Fourier encodings, however, use the same frequency representation across spatial locations despite the spatially varying frequency content of natural signals. We propose Adaptive Frequency Implicit Neural Representation (AFINR), a spatially adaptive frequency-conditioning framework for Fourier encoded INRs. A sampling compatible Fourier dictionary first defines the explicit frequency support, and a learnable spatial control field then modulates the admitted Fourier components before decoding. The control field is parameterized in the discrete cosine transform (DCT) domain to provide a compact and continuously queryable spatial representation. AFINR can be combined with different coordinate decoders, including ReLU, SIREN, CAFE, and CAFE+. Experiments on Kodak and DIV2K show consistent average improvements across these decoder families. In particular, AFINR-CAFE+ improves CAFE+ by 3.95 dB on Kodak and 3.50 dB on DIV2K. Ablation studies characterize the effects of adaptive-field parameterization, DCT coefficient retention, and frequency-response parameters, while learned spatial responses show position-dependent use of the admitted Fourier components.

Sources

Related papers