ConvLUT: Rethinking Convolutional Architectures for Ultra-Low-Latency LUT-Based Inference

arXiv:2611.11507 · cs.LG · Submitted 2026-11-08 · Read on arXiv

cs.LG

Submitted: 2026-11-08

Updated: 2026-11-08

Code: https://github.com/mlcommons/tiny_results_v1.1

License: http://creativecommons.org/licenses/by/4.0/

The gist: Lookup-table (LUT)-based neural networks (NNs) offer a compelling path to ultra-low-latency inference on FPGAs.

Abstract

Lookup-table (LUT)-based neural networks (NNs) offer a compelling path to ultra-low-latency inference on FPGAs. However, existing LUT-based neural networks have largely been restricted to MLP-style topologies, leaving structured vision and audio tasks underexplored. We present ConvLUT, a family of fully unrolled LUT-mappable convolutional architectures targeting the MLPerf Tiny benchmarks. It is comprised of a standard convolutional model built around a hierarchical tree of LUTs and a depthwise separable model following the MobileNet design philosophy. These architectures are enabled by four key contributions: a unified LUT-based convolutional framework supporting standard, depthwise, and pointwise convolutions under explicit fan-in constraints; a LUT-tree that addresses the k 2c receptive field dimensionality bottleneck; a channel-partitioned LUT pooling layer that replaces conventional spatial pooling; and a hardware-aware iterative pruning strategy for stable convergence at scale. Evaluated on CIFAR-10, Visual Wake Words, and Keyword Spotting, ConvLUT achieves competitive accuracy with fully unrolled inference latencies below about 50 ns. On CIFAR-10 it reaches at least a 8844 times latency reduction over non-logic-based prior implementations, and outperforms existing MLP-style LUT-based models on the same task by at least 20 pp in test accuracy.

Sources

Related papers