Frequency Domain Reservoir Computing

arXiv:2606.24969 · cs.LG · Submitted 2026-06-23 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Frequency Domain Reservoir Computing".

Jane: Here's the gist of the paper. Frequency Domain Reservoir Computing introduces FRESCO, an ESN architecture operating entirely in the frequency domain while avoiding domain-shift overheads to achieve O(N) complexity for dense, non-linear recurrent updates By employing a novel dimensional zero-padding input embedding, a packed frequency-domain readout, and a natively applied frequency-domain non-linearity, FRESCO drastically reduces computational costs and energy consumption of training and inference Furthermore, FRESCO matches the state-ofthe-art predictive performance on memory benchmarks, sequential classification, and multivariate long-horizon forecasting.

Tom: Pruning redundant reasoning tokens The paper addresses the quadratic sequence-length bottleneck of transformers by introducing Frequency Domain Reservoir Computing (FRESCO), an ESN architecture that operates entirely in the frequency domain while avoiding domain-shift overheads to achieve O(N) complexity for dense, non-linear recurrent updates.

Jane: This approach circumvents the input transformation bottleneck by lifting the 1D circular convolution formulation to 2D, enabling a novel dimensional zero-padding embedding.

Tom: By replacing the dense O(N 2) recurrent product with element-wise Hadamard multiplications between frequency-domain state and weight representations, the per-step cost of the recurrent update drops to O(N), even for a fully dense, non-linear reservoir.

Title and authors: Tom: We're looking at "Frequency Domain Reservoir Computing" by Klaus Schertler and Xiomara Runge. It sounds like they are taking the idea of reservoir computing and moving it entirely into the frequency domain to solve scaling issues.

Jane: That’s what it suggests, and the title makes it clear that they are aiming to eliminate some of those domain shift headaches that usually come with these types of models.

Lu: Basically, they are addressing that quadratic sequence-length bottleneck in transformers by introducing FRESCO, which is an ESN architecture operating entirely in the frequency domain >

Meng: So how does moving it to the frequency domain actually help beyond just changing the math? What’s the main practical hurdle they overcame?

Tom: They circumvented it by lifting that one dimensional circular convolution formulation to two dimensions, which allows them to use a novel dimensional zero padding embedding >

Jane: That embedding strategy is key because it means you don't need a full N point FFT anymore for your input transformation; you can use a single O of n log n FFT instead >

Lu: Because they replace the dense O of N squared recurrent product with element wise Hadamard multiplications between frequency domain state and weight representations, which drops the per step cost to O of n >

Meng: So it’s not just about the input transformation being faster, but it's about making sure that every single recurrent update is fast enough for large reservoir sizes >

Tom: Right. And they call this packed frequency-domain readout because they use the Hermitian symmetry of the reservoir state to get a contiguous real valued vector without needing an inverse FFT >

Jane: So that’s a nice win because it means no extra FFT steps after you finish the main computation, which really helps inference speed up >

The paper's summary: Tom: This paper, "Frequency Domain Reservoir Computing" is about taking the standard Echo State Network and reformulating it in the frequency domain to exploit those inherent efficiencies of that processing technique >

Jane: And they systematically avoid the usual slow Fast Fourier Transforms that are usually associated with these approaches, which is a major hurdle in this area >

Lu: They reformulate the standard ESN equation so you can use those frequency domain techniques while circumventing the costly FFTs usually associated with these approaches >

Meng: So what’s the actual mechanism for avoiding those FFT costs in this new formulation? Is it just clever input embedding?

Tom: The input embedding strategy is dimensional zero padding, which lets an input of size N times something be transformed using a single O of n log n FFT instead of a full N point FFT >

Jane: And that’s how they get the initial transformation step to be efficient enough for their needs because it replaces a full N point FFT with just that specific one >

Lu: The key is replacing the dense O of N squared recurrent product with element wise Hadamard multiplications between frequency domain state and weight representations, leading to the state update in the frequency domain given by equation four >

Meng: So they're basically saying you can do all your recurrent calculations just by doing simple element wise products now? That sounds like a huge reduction in complexity >

Tom: It is. And they show that this new formulation results in O of N complexity for dense, non linear recurrent updates, even for a fully dense reservoir >

Jane: So the overall message is that frequency domain processing provides a principled route to overcome the quadratic bottleneck and makes it efficient enough to scale properly >

The paper's improvements: Tom: They lay out several specific improvements they made to make this work better, and these are what really matter for engineering implementation >

Jane: What are the key things they did that move beyond just the basic O of N complexity claim? Are there structural changes in how you build the reservoir?

Lu: One improvement is dimensional zero padding, which we already touched on but it’s a specific strategy mapping low dimensional inputs into the frequency domain, eliminating standard transform bottlenecks >

Meng: And then they have this packed frequency-domain readout which is designed to be redundancy free and memory contiguous and FFT free >

Tom: That readout leverages the Hermitian symmetry of the reservoir state to give you a contiguous real valued vector without requiring inverse FFTs for training or inference >

Jane: So that's a nice practical detail because it means you’re not adding complexity after the main computation when you need to do your final prediction step >

Lu: Plus, they also introduce a structural mixing mechanism via a discrete circular shift to couple frequency components without sacrificing optimal scaling and enabling rich cross frequency harmonic mixing in the reservoir dynamics >

Meng: So that means it’s not just one trick, but they’re actively designing the dynamics for better feature representation during training >

Tom: Right. It sounds like they’s not just optimizing the speed; they's also improving how features are learned through this mixing mechanism >

Conclusion: Jane: So to wrap up, this paper on "Frequency Domain Reservoir Computing" shows that shifting computation to the frequency domain is a very sustainable alternative to more resource intensive sequence modeling approaches >

Tom: It proves that the mathematical elegance of the frequency domain offers a powerful and highly sustainable alternative for dense recurrent layers >

Lu: It achieves O of n complexity for dense, non linear recurrent updates by replacing the dense O of n squared recurrent product with element wise Hadamard multiplications between frequency domain state and weight representations >

Meng: From an engineering standpoint, they’re achieving massive inference speedups compared to standard ESNs and matching predictive accuracy on long horizon forecasting while consuming up to three orders of magnitude less energy >

Lalam: I'm confident that this work is very solid because the mathematical equivalence to spatial training through proposition four means it’s provably correct in that regard >

Jane: It just shows that the mathematical elegance of this approach is a powerful way to bypass those quadratic bottlenecks in sequence modeling >

Airbus Central Research & Technology · University of Pisa · Bielefeld University

cs.LG

Submitted: 2026-06-23

Updated: 2026-11-10

Comments: Accepted to NeurIPS 2026 (main track); camera-ready version

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 83/100

The gist: Frequency Domain Reservoir Computing introduces FRESCO, an ESN architecture operating entirely in the frequency domain while avoiding domain-shift overheads to achieve O(N) complexity for dense…

Key concepts

Frequency Domain Reservoir Computing (FRESCO)
FRESCO is a novel ESN architecture that performs all recurrent calculations within the frequency domain. It replaces the dense, quadratic recurrent product with simpler element-wise multiplications between frequency-domain states and weights, drastically reducing computational cost while preserving non-linear dynamics.
Dimensional Zero-Padding
This is an input embedding strategy that transforms an input of size Nx into a 2D representation. It allows the input to be processed using a single FFT operation of O(Nx log Nx) instead of a full N-point FFT, significantly lowering the computational cost during the initial transformation step.
Packed Frequency-Domain Readout
This readout method exploits the Hermitian symmetry of the reservoir state. It allows for extracting a contiguous, real-valued vector directly from the frequency domain without needing an inverse FFT, making inference faster and more efficient than traditional methods.
O(N) Complexity
The goal of FRESCO is to achieve linear complexity for recurrent updates. This is accomplished by using frequency domain techniques to replace the dense O(N^2) matrix-vector product with element-wise Hadamard multiplications, ensuring that the per-step cost scales linearly with the reservoir size N.

Terminology

Summary

Frequency Domain Reservoir Computing introduces FRESCO, an ESN architecture operating entirely in the frequency domain while avoiding domain-shift overheads to achieve O(N) complexity for dense, non-linear recurrent updates By employing a novel dimensional zero-padding input embedding, a packed frequency-domain readout, and a natively applied frequency-domain non-linearity, FRESCO drastically reduces computational costs and energy consumption of training and inference Furthermore, FRESCO matches the state-ofthe-art predictive performance on memory benchmarks, sequential classification, and multivariate long-horizon forecasting. p. 0

Where the paper says it

By employing a novel dimensional zero-padding input embedding, a packed frequency-domain readout, and a natively applied frequency-domain non-linearity, FRESCO drastically reduces computational costs and energy consumption of training and inference. Furthermore, FRESCO matches the state-ofthe-art predictive performance on memory benchmarks, sequential classification, and multivariate long-horizon forecasting, offering a scalable path forward for dense recurrent architectures.

Pruning redundant reasoning tokens

The paper addresses the quadratic sequence-length bottleneck of transformers by introducing Frequency Domain Reservoir Computing (FRESCO), an ESN architecture that operates entirely in the frequency domain while avoiding domain-shift overheads to achieve O(N) complexity for dense, non-linear recurrent updates. This approach circumvents the input transformation bottleneck by lifting the 1D circular convolution formulation to 2D, enabling a novel dimensional zero-padding embedding. p. 1

Where the paper says it

In this paper, we introduce frequency domain reservoir computing (FRESCO), an ESN architecture that operates entirely in the frequency domain and is designed for efficiency. FRESCO circumvents the input transformation bottleneck by lifting the 1D circular convolution formulation to 2D, enabling a novel dimensional zero-padding embedding.

By replacing the dense O(N 2) recurrent product with element-wise Hadamard multiplications between frequency-domain state and weight representations, the per-step cost of the recurrent update drops to O(N), even for a fully dense, non-linear reservoir. p. 1
Where the paper says it

This allows an input of size Nx to be transformed using a single O(Nx log Nx) FFT rather than a full N-point FFT. By replacing the dense O(N2 ) recurrent product with element-wise Hadamard multiplications between frequency-domain state and weight representations, the per-step cost of the recurrent update drops to O(N), even for a fully dense, non-linear reservoir.

p. 1
Where the paper says it

This allows an input of size Nx to be transformed using a single O(Nx log Nx) FFT rather than a full N-point FFT. By replacing the dense O(N2 ) recurrent product with element-wise Hadamard multiplications between frequency-domain state and weight representations, the per-step cost of the recurrent update drops to O(N), even for a fully dense, non-linear reservoir.

How the training signal is built

The core innovation of FRESCO is to i) reformulate the standard ESN of Eq. (1) in the frequency domain to exploit the inherent efficiencies of frequency-domain processing techniques, yet ii) systematically circumvent costly FFTs usually associated with these approaches. p. 3

Where the paper says it

3 FRESCO The core innovation of FRESCO is to i) reformulate the standard ESN of Eq. (1) in the frequency domain to exploit the inherent efficiencies of frequency-domain processing techniques, yet ii) systematically circumvent costly FFTs usually associated with these approaches.

p. 3
Where the paper says it

3 FRESCO The core innovation of FRESCO is to i) reformulate the standard ESN of Eq. (1) in the frequency domain to exploit the inherent efficiencies of frequency-domain processing techniques, yet ii) systematically circumvent costly FFTs usually associated with these approaches.

The input embedding strategy involves dimensional zero-padding, which allows an input of size Nx to be transformed using a single O(Nx log Nx) FFT rather than a full N-point FFT. p. 1
Where the paper says it

FRESCO circumvents the input transformation bottleneck by lifting the 1D circular convolution formulation to 2D, enabling a novel dimensional zero-padding embedding. This allows an input of size Nx to be transformed using a single O(Nx log Nx) FFT rather than a full N-point FFT.

The recurrent update is executed via element-wise products between frequency domain state and weight representations, leading to the state update in the frequency domain given by Eq. p. 1
Where the paper says it

This allows an input of size Nx to be transformed using a single O(Nx log Nx) FFT rather than a full N-point FFT. By replacing the dense O(N2 ) recurrent product with element-wise Hadamard multiplications between frequency-domain state and weight representations, the per-step cost of the recurrent update drops to O(N), even for a fully dense, non-linear reservoir.

(4).

Where the gains come from

The efficiency gains stem from several key architectural advantages detailed in FRESCO. These include:

  1. Frequency domain reservoir computing, which executes with O(N) complexity natively in the frequency domain. p. 0

    Where the paper says it

    To address this limitation, we introduce Frequency Domain Reservoir Computing (FRESCO), an ESN architecture operating entirely in the frequency domain while avoiding domain-shift overheads to achieve O(N) complexity for dense, non-linear recurrent updates. By employing a novel dimensional zero-padding input embedding, a packed frequency-domain readout, and a natively applied frequency-domain non-linearity, FRESCO drastically reduces computational costs and energy consumption of training and inference.

  2. Dimensional zero-padding, which reduces the FFT complexity from O(N log N) to O(Nx log Nx). p. 1

    Where the paper says it

    FRESCO circumvents the input transformation bottleneck by lifting the 1D circular convolution formulation to 2D, enabling a novel dimensional zero-padding embedding. This allows an input of size Nx to be transformed using a single O(Nx log Nx) FFT rather than a full N-point FFT.

  3. Packed frequency-domain readout, which exploits the Hermitian symmetry of the reservoir state to yield a contiguous, real-valued vector without requiring inverse FFTs. p. 1

    Where the paper says it

    Finally, a packed frequency-domain readout exploits the Hermitian symmetry of the reservoir state to yield a contiguous, real-valued vector. On this representation, ridge regression is provably equivalent to spatial-domain training, entirely removing the inverse FFT from both training and inference.

Performance and Stability

FRESCO maintains competitive performance by ensuring mathematical equivalence to spatial training through unitary transformations. Specifically, the packed frequency-domain readout is mathematically identical to training on the spatial states via Proposition 4, which states that performing ridge regression on the transformed features is equivalent to the original domain. p. 5

Where the paper says it

This mechanism executes training (via standard ridge regression) and inference (via a real-valued linear readout) natively on the frequency-domain representation, completely eliminating computational overhead while maintaining mathematical equivalence to a training process performed explicitly on the inverse-transformed reservoir states. This equivalence stems from the unitary nature of the normalized DFT: performing ridge regression on the flattened complex state vec(Rˆ ) is mathematically identical to training on the spatial states (Proposition 4, Appendix A.

Furthermore, stability conditions are characterized by propositions such as Proposition 3 (Necessary condition for FRESCO ESP), which provides a necessary condition for FRESCO to satisfy the Echo State Property. p. 6
Where the paper says it

Proposition 3 (Necessary condition for FRESCO ESP). A necessary condition for FRESCO to satisfy the Echo State Property is   Y N1 i=1 Y N2 j=1 |Wˆ r [i, j]|   1/N ≤ 1 α .

The experiments confirm that FRESCO achieves competitive predictive performance alongside substantial inference speedups over standard ESNs. p. 6
Where the paper says it

We use average pooling over the packed reservoir states to extract a single feature vector for the readout classifier (details in Appendix C. As detailed in Table 1, FRESCO achieves competitive predictive performance alongside substantial inference speedups over standard ESNs, reaching up to a factor of 26.

Conclusion

FRESCO demonstrates that the computational bottlenecks of dense recurrent layers can be elegantly bypassed by shifting computation to the frequency domain. It proves that the mathematical elegance of the frequency domain offers a powerful, highly sustainable alternative to more resource-intensive sequence modeling approaches. p. 8

Where the paper says it

Beyond sheer speed, it matches the predictive accuracy of heavily parameterized deep sequence models on complex, long-horizon multivariate forecasting while consuming up to three orders of magnitude less energy. Ultimately, this proves that the mathematical elegance of the frequency domain offers a powerful, highly sustainable alternative to more resource-intensive sequence modeling approaches.

Empirically, FRESCO achieves highly favorable scaling for both input embedding and recurrent updates, surpassing the inference speeds of standard ESNs. p. 8
Where the paper says it

While achieving strictly linear scaling requires constraining recurrent connectivities to circular convolutions, the absence of statistically significant accuracy differences across our classification benchmarks confirms that this structural prior does not systematically compromise expressivity. Empirically, FRESCO achieves highly favorable scaling for both input embedding and recurrent updates, surpassing the inference speeds of standard ESNs.

Beyond sheer speed, it matches the predictive accuracy of heavily parameterized deep sequence models on complex, long-horizon multivariate forecasting while consuming up to three orders of magnitude less energy. p. 8
Where the paper says it

Empirically, FRESCO achieves highly favorable scaling for both input embedding and recurrent updates, surpassing the inference speeds of standard ESNs. Beyond sheer speed, it matches the predictive accuracy of heavily parameterized deep sequence models on complex, long-horizon multivariate forecasting while consuming up to three orders of magnitude less energy.

This proves that the mathematical elegance of the frequency domain offers a powerful, highly sustainable alternative to more resource-intensive sequence modeling approaches >

--- Page 1 ---

Frequency Domain Reservoir Computing

Klaus Schertler

Airbus Central Research & Technology

Taufkirchen, Germany

klaus.schertler@airbus. p. 0

Where the paper says it

Frequency Domain Reservoir Computing Klaus Schertler Airbus Central Research & Technology Taufkirchen, Germany klaus. schertler@airbus.

com

Xiomara Runge

Airbus Central Research & Technology

Taufkirchen, Germany

xiomara.runge@airbus. p. 0

Where the paper says it

schertler@airbus. com Xiomara Runge Airbus Central Research & Technology Taufkirchen, Germany xiomara.

com

--- Page 2 ---

While the quadratic sequence-length bottleneck of transformers has fueled a resurgence in recurrent models, effectively capturing complex dynamics requires architectures that balance efficient training with highly expressive latent states. Echo State Networks (ESNs) offer a compelling approach by utilizing fixed recurrent weights to circumvent backpropagation through time, enabling a closed-form training solution. p. 0

Where the paper says it

it Abstract While the quadratic sequence-length bottleneck of transformers has fueled a resurgence in recurrent models, effectively capturing complex dynamics requires architectures that balance efficient training with highly expressive latent states. Echo State Networks (ESNs) offer a compelling approach by utilizing fixed recurrent weights to circumvent backpropagation through time, enabling a closed-form training solution.

However, achieving the expressivity needed for complex tasks demands large reservoirs, exposing an O(N 2) state-update bottleneck that prevents ESNs from matching the scale of contemporary recurrent models. p. 0
Where the paper says it

Echo State Networks (ESNs) offer a compelling approach by utilizing fixed recurrent weights to circumvent backpropagation through time, enabling a closed-form training solution. However, achieving the expressivity needed for complex tasks demands large reservoirs, exposing an O(N2 ) state-update bottleneck that prevents ESNs from matching the scale of contemporary recurrent models.

The per-step cost of a dense ESN scales as O(N 2) with the number of reservoir neurons, dominated by a recurrent matrix-vector product. The frequency domain offers a principled route to overcome this bottleneck because the 1D Discrete Fourier Transform (DFT) diagonalizes circulant weight matrices, collapsing the dense matrix-vector product to an element-wise Hadamard product (circular convolution theorem). p. 1

Where the paper says it

The frequency domain offers a principled route to overcome this bottleneck. Because the 1D Discrete Fourier Transform (DFT) diagonalizes circulant weight matrices, the dense matrix-vector product collapses to an element-wise Hadamard product (circular convolution theorem).

FRESCO circumvents the input transformation bottleneck by lifting the 1D circular convolution formulation to 2D, enabling a novel dimensional zero-padding embedding. This allows an input of size Nx to be transformed using a single O(Nx log Nx) FFT rather than a full N-point FFT. p. 1

Where the paper says it

FRESCO circumvents the input transformation bottleneck by lifting the 1D circular convolution formulation to 2D, enabling a novel dimensional zero-padding embedding. This allows an input of size Nx to be transformed using a single O(Nx log Nx) FFT rather than a full N-point FFT.

By replacing the dense O(N 2) recurrent product with element-wise Hadamard multiplications between frequency-domain state and weight representations, the per-step cost of the recurrent update drops to O(N), even for a fully dense, non-linear reservoir.

The paper establishes stability conditions for ESNs through the Echo State Property (ESP), which imposes constraints on the recurrence weights. The FRESCO variants satisfy the ESP under certain conditions, as shown in Proposition 2 (Sufficient condition for FRESCO ESP). p. 6

Where the paper says it

Proposition 2 (Sufficient condition for FRESCO ESP). If α · maxi,j |Wˆ r [i, j]| < 1, then FRESCO satisfies the Echo State Property.

The packed frequency-domain readout is designed to be redundancy-free and FFT-free, which matches the standard ESN approach in terms of computational speed. p. 1
Where the paper says it

• Packed frequency-domain readout: We design a redundancy-free, memory-contiguous, and FFT-free state readout, provably equivalent to spatial-domain training and inference. • High-efficiency: We show that FRESCO drastically reduces computational and energy footprints while delivering competitive results against established sequence models.

--- Page 3 ---

A dense ESN’s per-step complexity is dominated by these matrix-vector multiplications, scaling at O(N squared + NNx). Overcoming this quadratic bottleneck is the primary objective of FRESCO, while preserving the standard computational profile for both training and inference. p. 2

Where the paper says it

A dense ESN’s per-step complexity is dominated by these matrix-vector multiplications, scaling at O(N2 + NNx). Overcoming this quadratic bottleneck is the primary objective of FRESCO, while preserving the standard computational profile for both training and inference.

p. 2
Where the paper says it

A dense ESN’s per-step complexity is dominated by these matrix-vector multiplications, scaling at O(N2 + NNx). Overcoming this quadratic bottleneck is the primary objective of FRESCO, while preserving the standard computational profile for both training and inference.

A key stability requirement for ESNs is the Echo State Property (Definition 1), imposing a constraint on the recurrence weights. p. 2
Where the paper says it

A comparison of both architectures is shown in Figure 1. A key stability requirement for ESNs is the Echo State Property (Definition 1), imposing a constraint on the recurrence weights (Proposition 1).

p. 2
Where the paper says it

A comparison of both architectures is shown in Figure 1. A key stability requirement for ESNs is the Echo State Property (Definition 1), imposing a constraint on the recurrence weights (Proposition 1).

In practice, reservoirs are initialized by drawing Wr at random and then rescaling its elements to enforce a desired value of its spectral radius ρ(Wr). p. 2
Where the paper says it

A key stability requirement for ESNs is the Echo State Property (Definition 1), imposing a constraint on the recurrence weights (Proposition 1). In practice, reservoirs are initialized by drawing Wr at random and then rescaling its elements to enforce a desired value of its spectral radius ρ(Wr).

p. 2
Where the paper says it

A key stability requirement for ESNs is the Echo State Property (Definition 1), imposing a constraint on the recurrence weights (Proposition 1). In practice, reservoirs are initialized by drawing Wr at random and then rescaling its elements to enforce a desired value of its spectral radius ρ(Wr).

p. 2
Where the paper says it

A key stability requirement for ESNs is the Echo State Property (Definition 1), imposing a constraint on the recurrence weights (Proposition 1). In practice, reservoirs are initialized by drawing Wr at random and then rescaling its elements to enforce a desired value of its spectral radius ρ(Wr).

The core innovation of FRESCO is to i) reformulate the standard ESN of Eq. (1) in the frequency domain to exploit the inherent efficiencies of frequency-domain processing techniques, yet ii) systematically circumvent costly FFTs usually associated with these approaches. The FRESCO concept is outlined and compared to a standard ESN in Figure 1. p. 3

Where the paper says it

(1) in the frequency domain to exploit the inherent efficiencies of frequency-domain processing techniques, yet ii) systematically circumvent costly FFTs usually associated with these approaches. The FRESCO concept is outlined and compared to a standard ESN in Figure 1.

It comprises a novel method for efficiently embedding arbitrary inputs into the frequency domain, executing the recurrent updates in the frequency domain, and performing readouts without requiring subsequent FFTs on the reservoir state. p. 3
Where the paper says it

The FRESCO concept is outlined and compared to a standard ESN in Figure 1. It comprises a novel method for efficiently embedding arbitrary inputs into the frequency domain, executing the recurrent updates in the frequency domain, and performing readouts without requiring subsequent FFTs on the reservoir state.

The FRESCO architecture's advantages are summarized as follows:

• Frequency domain reservoir computing: We introduce FRESCO, a dense, non-linear ESN architecture executed with O(N) complexity natively in the frequency domain.

• Dimensional zero-padding: We propose a novel input embedding strategy mapping lowdimensional inputs into the frequency-domain, eliminating standard transform bottlenecks. p. 1

Where the paper says it

Our main contributions are summarized as follows: • Frequency domain reservoir computing: We introduce FRESCO, a dense, non-linear ESN architecture executed with O(N) complexity natively in the frequency domain. • Dimensional zero-padding: We propose a novel input embedding strategy mapping lowdimensional inputs into the frequency-domain, eliminating standard transform bottlenecks.

• Packed frequency-domain readout: We design a redundancy-free, memory-contiguous, and FFT-free state readout, provably equivalent to spatial-domain training and inference. p. 1

Where the paper says it

• Dimensional zero-padding: We propose a novel input embedding strategy mapping lowdimensional inputs into the frequency-domain, eliminating standard transform bottlenecks. • Packed frequency-domain readout: We design a redundancy-free, memory-contiguous, and FFT-free state readout, provably equivalent to spatial-domain training and inference.

--- Page 4 ---

A key stability requirement for ESNs is the Echo State Property (Definition 1), imposing a constraint on the recurrence weights. In practice, reservoirs are initialized by drawing Wr at random and then rescaling its elements to enforce a desired value of its spectral radius ρ(Wr). This requires an O(N 3) eigendecomposition, a bottleneck that scales poorly with reservoir size. p. 2

Where the paper says it

In practice, reservoirs are initialized by drawing Wr at random and then rescaling its elements to enforce a desired value of its spectral radius ρ(Wr). This requires an O(N3 ) eigendecomposition, a bottleneck that scales poorly with reservoir size.

--- Page 5 ---

A dense ESN’s per-step complexity scales at O(N squared + NNx). Overcoming this quadratic bottleneck is the primary objective of FRESCO, while preserving the standard computational profile for both training and inference. A key stability requirement for ESNs is the Echo State Property (Definition 1), imposing a constraint on the recurrence weights. In practice, reservoirs are initialized by drawing Wr at random and then rescaling its elements to enforce a desired value of its spectral radius ρ(Wr).

--- Page 7 ---

--- Page 8 ---

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Dimensional zero-padding allows an input of size Nx to be transformed using a single O(Nx log Nx) FFT rather than a full N-point FFT, drastically reducing computational costs compared to naive frequency-domain ESNs.

  2. The packed frequency-domain readout exploits the Hermitian symmetry of the reservoir state to yield a contiguous, real-valued vector, which is then used for ridge regression entirely avoiding inverse FFTs during training and inference.

  3. FRESCO (mix) introduces a structural mixing mechanism via a discrete circular shift to couple frequency components without sacrificing optimal scaling, enabling rich cross-frequency harmonic mixing in the reservoir dynamics.

  4. The architecture achieves O(N) complexity for dense, non-linear recurrent updates by replacing the dense O(N2) recurrent product with element-wise Hadamard multiplications between frequency-domain state and weight representations.

  5. The system can perform multivariate long-horizon forecasting on datasets like ETT and Weather while consuming up to three orders of magnitude less energy than deep sequence models, matching or exceeding their predictive accuracy.

Abstract

While the quadratic sequence-length bottleneck of transformers has fueled a resurgence in recurrent models, effectively capturing complex dynamics requires architectures that balance efficient training with highly expressive latent states. Echo State Networks (ESNs) offer a compelling approach by utilizing fixed recurrent weights to circumvent backpropagation through time, enabling a closed-form training solution. However, achieving the expressivity needed for complex tasks demands large reservoirs, exposing an O(N 2) state-update bottleneck that prevents ESNs from matching the scale of contemporary recurrent models. To address this limitation, we introduce Frequency Domain Reservoir Computing (FRESCO), an ESN architecture operating entirely in the frequency domain while avoiding domain-shift overheads to achieve O(N) complexity for dense, non-linear updates via structured circulant recurrence. By employing a novel dimensional zero-padding input embedding, a packed frequency-domain readout, and a natively applied frequency-domain non-linearity, FRESCO drastically reduces computational costs and energy consumption of training and inference. Furthermore, FRESCO achieves competitive predictive performance on memory benchmarks, sequential classification, and multivariate long-horizon forecasting, offering a scalable and energy-efficient path forward for dense recurrent architectures.

Sources

Related papers