FTerViT: Fully Ternary Vision Transformer
cs.CV
Submitted: 2026-05-20
Updated: 2026-11-02
Code: https://github.com/szymonrucinski/FTerViT
Terminology
Sources
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- ViT-1.58b: Mobile Vision Transformers in the 1-bit Era
- TerViT: An Efficient Ternary Vision Transformer
- TernaryCLIP: Efficiently Compressing Vision-Language Models with Ternary Weights and Distilled Knowledge
- Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
- DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
- The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
- Ternary Weight Networks
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- Quantizing deep convolutional networks for efficient inference: A whitepaper
- Do Not Wait: Learning Re-Ranking Model Without User Feedback At Serving Time in E-Commerce
- LLM Pruning and Distillation in Practice: The Minitron Approach
- Quantization-Aware Distillation for NVFP4 Inference Accuracy Recovery
- The Role of Masking for Efficient Supervised Knowledge Distillation of Vision Transformers
- MCUFormer: Deploying Vision Transformers on Microcontrollers with Limited Memory
- TinyFormer: Efficient Transformer Design and Deployment on Tiny Devices
- Low-Bit Quantization Favors Undertrained LLMs: Scaling Laws for Quantized LLMs with 100T Training Tokens
- Escaping the Big Data Paradigm with Compact Transformers
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs