Hardware Generation and Exploration of Lookup Table-Based Accelerators for 1.58-bit LLM Inference
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "Hardware Generation and Exploration of Lookup Table-Based Accelerators for 1.58-bit LLM Inference".
Paper summary: Tom: So what they're saying is this framework lets you explore every point in the design space systematically and fairly compare different configurations >
Jane: It’s about having a common footing for evaluation, which was missing before because different accelerators were using inconsistent parameters >
Lu: The key contribution is that they present an open-source hardware generator written in Chisel that can produce the actual Register Transfer Level code for any parameter combination within that space >
Meng: From my side, it's about being able to generate something concrete from a set of design choices instead of just reading theoretical trade-offs >
Lalam: I see they are looking at four main design parameters: mu, which is the LUT group size, L, which is the number of parallel lookup tables you instantiate, K, which is the number of parallel fetchers per LUT, and finally the activation data type like INT8 or FP16 >
Tom: And they show how these choices affect things like spatial reuse and temporal reuse of intermediate sums in their architecture >
Jane: They also discuss how maximizing core size improves area density compared to highly tiled approaches, suggesting that bigger cores are often better for the physical layout >
Conclusion: Tom: So when you look at the title "Hardware Generation and Exploration of Lookup Table-Based Accelerators for one point five eight-bit LLM Inference," it sounds like this is really about creating a systematic way to build these specific types of AI hardware > <ref:2604.25183#pg1,Hardware Generation and Exploration of Lookup Table-Based Accelerators for 1.58>
Jane: It’s establishing a common basis for comparison, which is important because before this work, accelerators were often optimized based on guesswork rather than a model driven approach >
Lu: The implication is that if you want to design future ternary weight inference accelerators, you should use this methodology—the generator and the cost model—to explore the full architectural space systematically >
Meng: What this means for me at scale is that I can finally generate designs from a known set of parameters instead of starting from scratch with every new idea >
Lalam: I'm pretty sure that what they show is a reproducible methodology, and it provides concrete guidance for designing these kinds of accelerators >
Robin Geens, Joran Heldens, Joren Dumoulin, Marian Verhelst
MICAS, KU Leuven
cs.AR
Submitted: 2026-04-28
Updated: 2026-04-28
Comments: Presented as ISPASS 2026
DOI: 10.1109/ISPASS69572.2026.00048
Code: https://github.com/KULeuven-MICAS/ternary-lut-dse
License: http://creativecommons.org/licenses/by/4.0/
The gist: Ternary weight quantization offers a promising path to mitigate the memory bandwidth bottleneck in Large Language Model (LLM) inference.
Key concepts
- Design Space Exploration
- This involves systematically mapping out all possible configurations for a hardware design, such as parameters like tile size or activation data type. The paper uses a generator to create any valid configuration within this space, allowing researchers to find the most efficient architecture without relying on guesswork.
- LUT-Based Accelerators
- These are specialized hardware circuits that use lookup tables (LUTs) instead of traditional arithmetic units for matrix multiplication in LLM inference. The paper investigates how different parameters affect the performance and area of these LUT-based structures.
- Analytical Cost Modeling
- This is a mathematical model used to predict the hardware cost, specifically the area, based on design choices. It breaks down the total hardware into components like adders and multiplexers, providing formulas that help determine which design parameters lead to the smallest physical chip size.
- Ternary Weight Quantization
- This is a method of reducing the precision of model weights to three discrete levels instead of standard floating-point numbers. The paper focuses on how this specific quantization affects the efficiency and performance when implemented in hardware accelerators.
Terminology
Summary
Ternary weight quantization offers a promising path to mitigate the memory bandwidth bottleneck in Large Language Model (LLM) inference. p. 0 , BitNet b1. 58)
offers a promising path to mitigate the memory bandwidth bottleneck in Large Language Model (LLM) inference. Existing designs rely on heuristic parameter selection,
lacking a systematic understanding of the architectural tradeoffs. This work addresses this gap by formalizing the design space
of ternary LUT-based accelerators and presenting an opensource hardware generator coupled with an analytical cost
model, validated against synthesis in TSMC 16nm technology.Where the paper says it
Where the paper says it
Systematic Design Space Exploration
The paper addresses a gap in prior work by formalizing the design space of LUT-based architectures and presenting an open-source hardware generator capable of producing RTL for any parameter instantiation within this space. This framework enables rapid design space exploration but also establishes a common footing for fair cross-design evaluation, which was previously hindered by inconsistent instantiations across published accelerators. p. 0 This work addresses this gap by formalizing the design space
of ternary LUT-based accelerators and presenting an opensource hardware generator coupled with an analytical cost
model, validated against synthesis in TSMC 16nm technology. By spanning the full architectural space, this framework not
only enables rapid design space exploration but also establishes
a common footing for fair cross-design evaluation, which was
previously hindered by inconsistent instantiations across published accelerators. Using this unified framework, we contextualize
existing architectures and extract practical guidelines to inform future ternary-weight inference accelerator design. The key contributions of this work are:
• Hardware generator (Section III): We formalize the
design space of ternary LUT-based matrix multiplication accelerators and propose an open-source hardware
generator in Chisel to instantiate any point within this
design space.Where the paper says it
Where the paper says it
Architectural Trade-offs and Optimizations
The optimal architecture is fundamentally governed by the activation data type, with LUT-based reuse offering significant gains for high-cost arithmetic like FP16 but diminishing returns for small integer types. Furthermore, maximizing core size consistently improves area density compared to highly tiled approaches. p. 0 , FP16), it yields diminishing returns
for small integer types. Furthermore, we show that maximizing
core size consistently improves area density compared to highly
tiled approaches. Furthermore, we show that maximizing
core size consistently improves area density compared to highly
tiled approaches. Our optimized designs achieve a 2. 2× area
reduction compared to multiplier-based baselines. Moreover,
by benchmarking state-of-the-art implementations against our
model, we reveal that correcting suboptimal parameters yields
up to a 1. Moreover,
by benchmarking state-of-the-art implementations against our
model, we reveal that correcting suboptimal parameters yields
up to a 1. 2× area improvement.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Analytical Cost Modeling
A companion analytical model was developed to estimate the area cost of the LUT-based architecture by subdividing the hardware design into submodules and deriving scaling equations for each module as a function of key design parameters. The total area is expressed as a weighted sum of main contributors: Area = aadd · (Build+cost + Accumulate+cost) + (amux + ainv) · MUXcost + areg · OutRegcost. p. 4 Analytical Hardware Cost Model
The scaling equations provide only qualitative insight
into how the architecture grows with group size and tile
dimensions. To quantitatively estimate the required area for a
given set of parameters and to identify efficient design points
more precisely, the total area must be expressed as a weighted
sum of the main contributors:
Area = aadd · (Build+cost + Accumulate+cost)
+ (amux + ainv) · MUXcost + areg · OutRegcost (9)
Here, the coefficients ax represent the technology-dependent
area of the fundamental unit cells:
• aadd: Area of a single scalar adder of chosen activation
type. An optimal µ must
therefore balance these opposing contributions. Analytical Hardware Cost Model
The scaling equations provide only qualitative insight
into how the architecture grows with group size and tile
dimensions.Where the paper says it
Where the paper says it
Empirical Validation and Findings
Synthesis results confirmed that both area and power metrics exhibit the same minima at optimal µ = 3 for a 32×32 configuration with FP16 adders. The cost model tracks synthesis results with high fidelity across all tile sizes and group sizes. p. 5 Figure 6
compares the analytical predictions against the synthesis data. The model tracks the synthesis results with high fidelity
across all tile sizes and group sizes. For INT8, the area
difference between the µ = 1 case and optimal group size
is negligible. In contrast, for FP16, a large improvement can
be made by selecting µ > 1.Where the paper says it
Where the paper says it
Comparative Performance Gains
The optimized LUT-based design achieves a 2.23× area reduction compared to the dequantization approach and a 1. p. 6 As
summarized in Table IV, our optimized LUT-based design
achieves a 2. 23× area reduction compared to the dequantization approach and a 1. 64× reduction compared to the
optimized sign-flip baseline. This further confirms that for
complex data types (FP16), a LUT-based approach yields
substantial savings compared to a dedicated sign-flip unit. 4× architectural improvement we demonstrate over a state-of-the-art
design. With a publicly available generator and reproducible
methodology, this work establishes a common basis for fair
comparison and offers concrete guidance for the design of
future ternary-weight inference accelerators. 4× architectural improvement we demonstrate over a state-of-the-art
design. With a publicly available generator and reproducible
methodology, this work establishes a common basis for fair
comparison and offers concrete guidance for the design of
future ternary-weight inference accelerators. 2× area improvement. These findings underscore
that achieving state-of-the-art efficiency requires moving beyond
heuristics to a model-driven design methodology.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Optimal Geometry Determination
The optimal aspect ratio is strictly governed by the activation data type, with FP16 activations favoring elongated tiles where K > L·µ L·µ K. The paper concludes that maximizing core size is the optimal strategy for area efficiency as tile dimensions increase because overhead terms vanish. p. 6 As the tile dimensions n and m increase, these overhead
terms vanish, leaving only the fundamental compute cost. Consequently, maximizing core size is the optimal strategy
for area efficiency.Where the paper says it
Conclusion on Design Methodology
This work establishes a systematic framework to explore the design space of ternary LUT-based accelerators, and by combining an open-source RTL generator with an analytical model validated through synthesis, it enables exploration of the full architectural space. This work establishes a common basis for fair comparison and offers concrete guidance for the design of future ternary-weight inference accelerators. The evaluation reveals that prior accelerators often deviate from these optimal points, explaining the 1. p. 8 These insights demonstrate that substantial area savings
are achievable under matched throughput, but only when
designs adopt suitable tile shapes, sufficient LUT fan-out,
and activation types that justify ternary-weight arithmetic. Our evaluation reveals that prior accelerators often deviate
from these optimal points, explaining the 1. Our evaluation reveals that prior accelerators often deviate
from these optimal points, explaining the 1. 4× architectural improvement we demonstrate over a state-of-the-art
design. 4× architectural improvement we demonstrate over a state-of-the-art
design. With a publicly available generator and reproducible
methodology, this work establishes a common basis for fair
comparison and offers concrete guidance for the design of
future ternary-weight inference accelerators. These findings underscore
that achieving state-of-the-art efficiency requires moving beyond
heuristics to a model-driven design methodology. This transition
is enabled by our open-source hardware generator available at
https://github. These findings underscore
that achieving state-of-the-art efficiency requires moving beyond
heuristics to a model-driven design methodology. This transition
is enabled by our open-source hardware generator available at
https://github. This transition
is enabled by our open-source hardware generator available at
https://github. com/KULeuven-MICAS/ternary-lut-dse. This transition
is enabled by our open-source hardware generator available at
https://github. com/KULeuven-MICAS/ternary-lut-dse. To fully exploit these efficient quantization schemes, specialized hardware architectures are required. Lookup Table (LUT)-based architectures offer a viable
arXiv:2604. To fully exploit these efficient quantization schemes, specialized hardware architectures are required. Lookup Table (LUT)-based architectures offer a viable
arXiv:2604. To fully exploit these efficient quantization schemes, specialized hardware architectures are required. Lookup Table (LUT)-based architectures offer a viable
arXiv:2604. To fully exploit these efficient quantization schemes, specialized hardware architectures are required. Lookup Table (LUT)-based architectures offer a viable
arXiv:2604.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
References
The references cited in the paper include works on BitNet [1] and BitNet b1.58 [2] regarding extreme weight-only quantization. p. 0 Consequently, the cost of fetching
weights dominates, leading to underutilized compute hardware and limiting throughput by off-chip bandwidth. Extreme weight-only quantization formats, such as BitNet [1] and BitNet b1. HARDWARE GENERATOR DESIGN
To assess the impact of parameter choices, we developed a fully parametrizable RTL generator for LUT-based
arithmetic. This generator is written in Chisel, an objectoriented hardware construction language, and utilizes opensource floating-point units [17] to support diverse data types.Where the paper says it
Where the paper says it
Index Terms
The index terms for the work are Ternary Quantization, LUT-Based Matrix Multiplication, Hardware Acceleration, Large Language Models, and Design Space Exploration. The paper is 2604.25183.
Acknowledgments
The project has been partly funded by the European Research Council (ERC) under grant agreement No. 101088865, the European Union’s Horizon 2020 program under grant agreement No. p. 8 ACKNOWLEDGMENTS
This project has been partly funded by the European Research Council (ERC) under grant agreement No. 101088865,
the European Union’s Horizon 2020 program under grant
agreement No. 101088865,
the European Union’s Horizon 2020 program under grant
agreement No. 101070374, the Flanders AI Research Program, Research Foundation Flanders (FWO) under grant No. 101070374, the Flanders AI Research Program, Research Foundation Flanders (FWO) under grant No. 1S37125N, and KU Leuven.Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Systematic Design Space Exploration (DSE) enables rapid design space exploration by spanning the full architectural space of ternary LUT-based accelerators, providing an analytical and empirical model to guide design choices instead of relying on heuristic parameter selection.
-
The system can achieve a 2.2 times area reduction compared to multiplier-based baselines for complex data types like FP16, and up to a 1.2 times area improvement by correcting suboptimal parameters, leading to substantial efficiency gains in inference hardware.
-
The improved AI system can be specifically optimized for weight-only ternary quantization, minimizing memory bandwidth required during the memory-bound decode stage and simplifying arithmetic into conditional additions (+x, -x, or 0).
-
By leveraging the analytical cost model derived from scaling equations (Eq. 5-8), the system can determine optimal tile shapes and group sizes for a given throughput target to maximize area efficiency across different activation data types.
-
The system can be designed with an optimal aspect ratio based on activation type; for FP16, it favors elongated tiles where K > L times µ, while for INT8, it favors tiles where L times µ > K to minimize the accumulation cost term.
Abstract
Ternary weight quantization (e.g., BitNet b1.58) offers a promising path to mitigate the memory bandwidth bottleneck in Large Language Model (LLM) inference. However, conventional compute platforms lack native support for ternary-weight arithmetic, often relying on inefficient dequantization. Lookup table (LUT)-based hardware architectures provide an effective alternative by replacing multiplications with conditional additions, but their design space remains largely unexplored. Existing designs rely on heuristic parameter selection, lacking a systematic understanding of the architectural trade-offs. This work addresses this gap by formalizing the design space of ternary LUT-based accelerators and presenting an open-source hardware generator coupled with an analytical cost model, validated against synthesis in TSMC 16nm technology. By spanning the full architectural space, this framework not only enables rapid design space exploration but also establishes a common footing for fair cross-design evaluation, which was previously hindered by inconsistent instantiations across published accelerators. Using this framework, we challenge several assumptions and design choices in recent literature. We demonstrate that the optimal architecture is fundamentally governed by the activation data type: while LUT-based reuse offers significant gains for high-cost arithmetic (e.g., FP16), it yields diminishing returns for small integer types. Furthermore, we show that maximizing core size consistently improves area density compared to highly tiled approaches. Our optimized designs achieve a 2.2x area reduction compared to multiplier-based baselines. Moreover, by benchmarking state-of-the-art implementations against our model, we reveal that correcting suboptimal parameters yields up to a 1.2x area improvement.
Sources
- LUT-GEMM: Quantized Matrix Multiplication based on LUTs for Efficient Inference in Large-Scale Generative Language Models
- TeLLMe: An Energy-Efficient Ternary LLM Accelerator for Prefilling and Decoding on Edge FPGAs
- TernaryLLM: Ternarized Large Language Model
- Spectra: Surprising Effectiveness of Pretraining Ternary Language Models at Scale
- LLaVaOLMoBitnet1B: Ternary LLM goes Multimodal!
- Quantifying the Capabilities of LLMs across Scale and Precision
Related papers
- Embedded Arena: Iterative Optimization via Hardware Feedback
- WitCert: Sound Runtime Risk Observability and Gating for KV-Cache Quantization
- SparsePixels: Efficient Convolution for Sparse Data on FPGAs
- Golden Ruler: A Numeric Format Catalog with Bit-Exact Conformance Vectors for FP8, BF16, MXFP4, and Microscaling Formats
- AMMA: A Multi-Chiplet Memory-Centric Architecture for Low-Latency 1M Context Attention Serving
- RFChipAgent: Multi-Agentic AI Flow for Analog/RF Chip Design