Hardware Generation and Exploration of Lookup Table-Based Accelerators for 1.58-bit LLM Inference

arXiv:2604.25183 · cs.AR · Submitted 2026-04-28 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Hardware Generation and Exploration of Lookup Table-Based Accelerators for 1.58-bit LLM Inference".

Paper summary: Tom: So what they're saying is this framework lets you explore every point in the design space systematically and fairly compare different configurations >

Jane: It’s about having a common footing for evaluation, which was missing before because different accelerators were using inconsistent parameters >

Lu: The key contribution is that they present an open-source hardware generator written in Chisel that can produce the actual Register Transfer Level code for any parameter combination within that space >

Meng: From my side, it's about being able to generate something concrete from a set of design choices instead of just reading theoretical trade-offs >

Lalam: I see they are looking at four main design parameters: mu, which is the LUT group size, L, which is the number of parallel lookup tables you instantiate, K, which is the number of parallel fetchers per LUT, and finally the activation data type like INT8 or FP16 >

Tom: And they show how these choices affect things like spatial reuse and temporal reuse of intermediate sums in their architecture >

Jane: They also discuss how maximizing core size improves area density compared to highly tiled approaches, suggesting that bigger cores are often better for the physical layout >

Conclusion: Tom: So when you look at the title "Hardware Generation and Exploration of Lookup Table-Based Accelerators for one point five eight-bit LLM Inference," it sounds like this is really about creating a systematic way to build these specific types of AI hardware > <ref:2604.25183#pg1,Hardware Generation and Exploration of Lookup Table-Based Accelerators for 1.58>

Jane: It’s establishing a common basis for comparison, which is important because before this work, accelerators were often optimized based on guesswork rather than a model driven approach >

Lu: The implication is that if you want to design future ternary weight inference accelerators, you should use this methodology—the generator and the cost model—to explore the full architectural space systematically >

Meng: What this means for me at scale is that I can finally generate designs from a known set of parameters instead of starting from scratch with every new idea >

Lalam: I'm pretty sure that what they show is a reproducible methodology, and it provides concrete guidance for designing these kinds of accelerators >

Robin Geens, Joran Heldens, Joren Dumoulin, Marian Verhelst

MICAS, KU Leuven

cs.AR

Submitted: 2026-04-28

Updated: 2026-04-28

Comments: Presented as ISPASS 2026

DOI: 10.1109/ISPASS69572.2026.00048

Code: https://github.com/KULeuven-MICAS/ternary-lut-dse

License: http://creativecommons.org/licenses/by/4.0/

The gist: Ternary weight quantization offers a promising path to mitigate the memory bandwidth bottleneck in Large Language Model (LLM) inference.

Key concepts

Design Space Exploration
This involves systematically mapping out all possible configurations for a hardware design, such as parameters like tile size or activation data type. The paper uses a generator to create any valid configuration within this space, allowing researchers to find the most efficient architecture without relying on guesswork.
LUT-Based Accelerators
These are specialized hardware circuits that use lookup tables (LUTs) instead of traditional arithmetic units for matrix multiplication in LLM inference. The paper investigates how different parameters affect the performance and area of these LUT-based structures.
Analytical Cost Modeling
This is a mathematical model used to predict the hardware cost, specifically the area, based on design choices. It breaks down the total hardware into components like adders and multiplexers, providing formulas that help determine which design parameters lead to the smallest physical chip size.
Ternary Weight Quantization
This is a method of reducing the precision of model weights to three discrete levels instead of standard floating-point numbers. The paper focuses on how this specific quantization affects the efficiency and performance when implemented in hardware accelerators.

Terminology

Summary

Ternary weight quantization offers a promising path to mitigate the memory bandwidth bottleneck in Large Language Model (LLM) inference. p. 0

Where the paper says it

, BitNet b1. 58) offers a promising path to mitigate the memory bandwidth bottleneck in Large Language Model (LLM) inference.

This work formalizes the design space of ternary LUT-based accelerators and presents an open-source hardware generator coupled with an analytical cost model, validated against synthesis in TSMC 16nm technology. p. 0
Where the paper says it

Existing designs rely on heuristic parameter selection, lacking a systematic understanding of the architectural tradeoffs. This work addresses this gap by formalizing the design space of ternary LUT-based accelerators and presenting an opensource hardware generator coupled with an analytical cost model, validated against synthesis in TSMC 16nm technology.

Systematic Design Space Exploration

The paper addresses a gap in prior work by formalizing the design space of LUT-based architectures and presenting an open-source hardware generator capable of producing RTL for any parameter instantiation within this space. This framework enables rapid design space exploration but also establishes a common footing for fair cross-design evaluation, which was previously hindered by inconsistent instantiations across published accelerators. p. 0

Where the paper says it

This work addresses this gap by formalizing the design space of ternary LUT-based accelerators and presenting an opensource hardware generator coupled with an analytical cost model, validated against synthesis in TSMC 16nm technology. By spanning the full architectural space, this framework not only enables rapid design space exploration but also establishes a common footing for fair cross-design evaluation, which was previously hindered by inconsistent instantiations across published accelerators.

The key contributions include a hardware generator in Chisel to instantiate any point within the design space. p. 1
Where the paper says it

Using this unified framework, we contextualize existing architectures and extract practical guidelines to inform future ternary-weight inference accelerator design. The key contributions of this work are: • Hardware generator (Section III): We formalize the design space of ternary LUT-based matrix multiplication accelerators and propose an open-source hardware generator in Chisel to instantiate any point within this design space.

Architectural Trade-offs and Optimizations

The optimal architecture is fundamentally governed by the activation data type, with LUT-based reuse offering significant gains for high-cost arithmetic like FP16 but diminishing returns for small integer types. Furthermore, maximizing core size consistently improves area density compared to highly tiled approaches. p. 0

Where the paper says it

, FP16), it yields diminishing returns for small integer types. Furthermore, we show that maximizing core size consistently improves area density compared to highly tiled approaches.

The optimized designs achieve a 2. p. 0
Where the paper says it

Furthermore, we show that maximizing core size consistently improves area density compared to highly tiled approaches. Our optimized designs achieve a 2.

2× area reduction compared to multiplier-based baselines and correcting suboptimal parameters yields up to a 1. p. 0
Where the paper says it

2× area reduction compared to multiplier-based baselines. Moreover, by benchmarking state-of-the-art implementations against our model, we reveal that correcting suboptimal parameters yields up to a 1.

2× area improvement. p. 0
Where the paper says it

Moreover, by benchmarking state-of-the-art implementations against our model, we reveal that correcting suboptimal parameters yields up to a 1. 2× area improvement.

Analytical Cost Modeling

A companion analytical model was developed to estimate the area cost of the LUT-based architecture by subdividing the hardware design into submodules and deriving scaling equations for each module as a function of key design parameters. The total area is expressed as a weighted sum of main contributors: Area = aadd · (Build+cost + Accumulate+cost) + (amux + ainv) · MUXcost + areg · OutRegcost. p. 4

Where the paper says it

Analytical Hardware Cost Model The scaling equations provide only qualitative insight into how the architecture grows with group size and tile dimensions. To quantitatively estimate the required area for a given set of parameters and to identify efficient design points more precisely, the total area must be expressed as a weighted sum of the main contributors: Area = aadd · (Build+cost + Accumulate+cost) + (amux + ainv) · MUXcost + areg · OutRegcost (9) Here, the coefficients ax represent the technology-dependent area of the fundamental unit cells: • aadd: Area of a single scalar adder of chosen activation type.

These formulas reveal that an optimal µ must balance the opposing contributions of LUT Build and Accumulate hardware. p. 4
Where the paper says it

An optimal µ must therefore balance these opposing contributions. Analytical Hardware Cost Model The scaling equations provide only qualitative insight into how the architecture grows with group size and tile dimensions.

Empirical Validation and Findings

Synthesis results confirmed that both area and power metrics exhibit the same minima at optimal µ = 3 for a 32×32 configuration with FP16 adders. The cost model tracks synthesis results with high fidelity across all tile sizes and group sizes. p. 5

Where the paper says it

Figure 6 compares the analytical predictions against the synthesis data. The model tracks the synthesis results with high fidelity across all tile sizes and group sizes.

The analysis confirms that for FP16, a large improvement can be made by selecting µ > 1 because adders become significantly more expensive than MUX/inverter logic. p. 6
Where the paper says it

For INT8, the area difference between the µ = 1 case and optimal group size is negligible. In contrast, for FP16, a large improvement can be made by selecting µ > 1.

Comparative Performance Gains

The optimized LUT-based design achieves a 2.23× area reduction compared to the dequantization approach and a 1. p. 6

Where the paper says it

As summarized in Table IV, our optimized LUT-based design achieves a 2. 23× area reduction compared to the dequantization approach and a 1.

64× reduction compared to the optimized sign-flip baseline for FP16. p. 6
Where the paper says it

64× reduction compared to the optimized sign-flip baseline. This further confirms that for complex data types (FP16), a LUT-based approach yields substantial savings compared to a dedicated sign-flip unit.

This work establishes a common basis for fair comparison and offers concrete guidance for the design of future ternary-weight inference accelerators. p. 8
Where the paper says it

4× architectural improvement we demonstrate over a state-of-the-art design. With a publicly available generator and reproducible methodology, this work establishes a common basis for fair comparison and offers concrete guidance for the design of future ternary-weight inference accelerators.

p. 8
Where the paper says it

4× architectural improvement we demonstrate over a state-of-the-art design. With a publicly available generator and reproducible methodology, this work establishes a common basis for fair comparison and offers concrete guidance for the design of future ternary-weight inference accelerators.

The findings underscore that achieving state-of-the-art efficiency requires moving beyond heuristics to a model-driven design methodology. p. 0
Where the paper says it

2× area improvement. These findings underscore that achieving state-of-the-art efficiency requires moving beyond heuristics to a model-driven design methodology.

Optimal Geometry Determination

The optimal aspect ratio is strictly governed by the activation data type, with FP16 activations favoring elongated tiles where K > L·µ L·µ K. The paper concludes that maximizing core size is the optimal strategy for area efficiency as tile dimensions increase because overhead terms vanish. p. 6

Where the paper says it

As the tile dimensions n and m increase, these overhead terms vanish, leaving only the fundamental compute cost. Consequently, maximizing core size is the optimal strategy for area efficiency.

Conclusion on Design Methodology

This work establishes a systematic framework to explore the design space of ternary LUT-based accelerators, and by combining an open-source RTL generator with an analytical model validated through synthesis, it enables exploration of the full architectural space. This work establishes a common basis for fair comparison and offers concrete guidance for the design of future ternary-weight inference accelerators. The evaluation reveals that prior accelerators often deviate from these optimal points, explaining the 1. p. 8

Where the paper says it

These insights demonstrate that substantial area savings are achievable under matched throughput, but only when designs adopt suitable tile shapes, sufficient LUT fan-out, and activation types that justify ternary-weight arithmetic. Our evaluation reveals that prior accelerators often deviate from these optimal points, explaining the 1.

4× architectural improvement demonstrated over a state-of-the-art design. p. 8
Where the paper says it

Our evaluation reveals that prior accelerators often deviate from these optimal points, explaining the 1. 4× architectural improvement we demonstrate over a state-of-the-art design.

The paper provides a publicly available generator and reproducible methodology to guide the design of future ternary-weight inference accelerators. p. 8
Where the paper says it

4× architectural improvement we demonstrate over a state-of-the-art design. With a publicly available generator and reproducible methodology, this work establishes a common basis for fair comparison and offers concrete guidance for the design of future ternary-weight inference accelerators.

The paper is enabled by our open-source hardware generator available at https://github. p. 0
Where the paper says it

These findings underscore that achieving state-of-the-art efficiency requires moving beyond heuristics to a model-driven design methodology. This transition is enabled by our open-source hardware generator available at https://github.

p. 0
Where the paper says it

These findings underscore that achieving state-of-the-art efficiency requires moving beyond heuristics to a model-driven design methodology. This transition is enabled by our open-source hardware generator available at https://github.

com/KULeuven-MICAS/ternary-lut-dse. p. 0
Where the paper says it

This transition is enabled by our open-source hardware generator available at https://github. com/KULeuven-MICAS/ternary-lut-dse.

p. 0
Where the paper says it

This transition is enabled by our open-source hardware generator available at https://github. com/KULeuven-MICAS/ternary-lut-dse.

The paper is 2604. p. 0
Where the paper says it

To fully exploit these efficient quantization schemes, specialized hardware architectures are required. Lookup Table (LUT)-based architectures offer a viable arXiv:2604.

p. 0
Where the paper says it

To fully exploit these efficient quantization schemes, specialized hardware architectures are required. Lookup Table (LUT)-based architectures offer a viable arXiv:2604.

p. 0
Where the paper says it

To fully exploit these efficient quantization schemes, specialized hardware architectures are required. Lookup Table (LUT)-based architectures offer a viable arXiv:2604.

p. 0
Where the paper says it

To fully exploit these efficient quantization schemes, specialized hardware architectures are required. Lookup Table (LUT)-based architectures offer a viable arXiv:2604.

25183.

References

The references cited in the paper include works on BitNet [1] and BitNet b1.58 [2] regarding extreme weight-only quantization. p. 0

Where the paper says it

Consequently, the cost of fetching weights dominates, leading to underutilized compute hardware and limiting throughput by off-chip bandwidth. Extreme weight-only quantization formats, such as BitNet [1] and BitNet b1.

The paper also utilizes open-source floating-point units [17] to support diverse data types for the generator. p. 3
Where the paper says it

HARDWARE GENERATOR DESIGN To assess the impact of parameter choices, we developed a fully parametrizable RTL generator for LUT-based arithmetic. This generator is written in Chisel, an objectoriented hardware construction language, and utilizes opensource floating-point units [17] to support diverse data types.

The paper is enabled by our open-source hardware generator available at https://github.com/KULeuven-MICAS/ternary-lut-dse. The paper is 2604.25183.

Index Terms

The index terms for the work are Ternary Quantization, LUT-Based Matrix Multiplication, Hardware Acceleration, Large Language Models, and Design Space Exploration. The paper is 2604.25183.

Acknowledgments

The project has been partly funded by the European Research Council (ERC) under grant agreement No. 101088865, the European Union’s Horizon 2020 program under grant agreement No. p. 8

Where the paper says it

ACKNOWLEDGMENTS This project has been partly funded by the European Research Council (ERC) under grant agreement No. 101088865, the European Union’s Horizon 2020 program under grant agreement No.

101070374, the Flanders AI Research Program, Research Foundation Flanders (FWO) under grant No. p. 8
Where the paper says it

101088865, the European Union’s Horizon 2020 program under grant agreement No. 101070374, the Flanders AI Research Program, Research Foundation Flanders (FWO) under grant No.

1S37125N, and KU Leuven. p. 8
Where the paper says it

101070374, the Flanders AI Research Program, Research Foundation Flanders (FWO) under grant No. 1S37125N, and KU Leuven.

The paper is 2604.25183.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Systematic Design Space Exploration (DSE) enables rapid design space exploration by spanning the full architectural space of ternary LUT-based accelerators, providing an analytical and empirical model to guide design choices instead of relying on heuristic parameter selection.

  2. The system can achieve a 2.2 times area reduction compared to multiplier-based baselines for complex data types like FP16, and up to a 1.2 times area improvement by correcting suboptimal parameters, leading to substantial efficiency gains in inference hardware.

  3. The improved AI system can be specifically optimized for weight-only ternary quantization, minimizing memory bandwidth required during the memory-bound decode stage and simplifying arithmetic into conditional additions (+x, -x, or 0).

  4. By leveraging the analytical cost model derived from scaling equations (Eq. 5-8), the system can determine optimal tile shapes and group sizes for a given throughput target to maximize area efficiency across different activation data types.

  5. The system can be designed with an optimal aspect ratio based on activation type; for FP16, it favors elongated tiles where K > L times µ, while for INT8, it favors tiles where L times µ > K to minimize the accumulation cost term.

Abstract

Ternary weight quantization (e.g., BitNet b1.58) offers a promising path to mitigate the memory bandwidth bottleneck in Large Language Model (LLM) inference. However, conventional compute platforms lack native support for ternary-weight arithmetic, often relying on inefficient dequantization. Lookup table (LUT)-based hardware architectures provide an effective alternative by replacing multiplications with conditional additions, but their design space remains largely unexplored. Existing designs rely on heuristic parameter selection, lacking a systematic understanding of the architectural trade-offs. This work addresses this gap by formalizing the design space of ternary LUT-based accelerators and presenting an open-source hardware generator coupled with an analytical cost model, validated against synthesis in TSMC 16nm technology. By spanning the full architectural space, this framework not only enables rapid design space exploration but also establishes a common footing for fair cross-design evaluation, which was previously hindered by inconsistent instantiations across published accelerators. Using this framework, we challenge several assumptions and design choices in recent literature. We demonstrate that the optimal architecture is fundamentally governed by the activation data type: while LUT-based reuse offers significant gains for high-cost arithmetic (e.g., FP16), it yields diminishing returns for small integer types. Furthermore, we show that maximizing core size consistently improves area density compared to highly tiled approaches. Our optimized designs achieve a 2.2x area reduction compared to multiplier-based baselines. Moreover, by benchmarking state-of-the-art implementations against our model, we reveal that correcting suboptimal parameters yields up to a 1.2x area improvement.

Sources

Related papers