I-Segmenter: Integer-Only Vision Transformer for Efficient Semantic Segmentation
cs.CV, cs.AI, cs.LG
Submitted: 2025-09-12
Updated: 2026-06-08
Comments: Accepted by the Journal of Systems Architecture
DOI: 10.1016/j.sysarc.2026.103888
Code: https://github.com/ms245755/I-segmenter
License: http://creativecommons.org/licenses/by/4.0/
The gist: Vision Transformers (ViTs) have recently achieved strong results in semantic segmentation, yet their deployment on resource-constrained devices remains limited due to their high memory footprint and…
Terminology
Abstract
Vision Transformers (ViTs) have recently achieved strong results in semantic segmentation, yet their deployment on resource-constrained devices remains limited due to their high memory footprint and computational cost. Quantization offers an effective strategy to improve efficiency, but ViT-based segmentation models are notoriously fragile under low precision, as quantization errors accumulate across deep encoder-decoder pipelines. We introduce I-Segmenter, the first fully integer-only ViT segmentation framework. Building on the Segmenter architecture, I-Segmenter systematically replaces floating-point operations with integer-only counterparts. To further stabilize both training and inference, we propose λ-ShiftGELU, a novel activation function that mitigates the limitations of uniform quantization in handling long-tailed activation distributions. In addition, we remove the L2 normalization layer and replace bilinear interpolation in the decoder with nearest neighbor upsampling, ensuring integer-only execution throughout the computational graph. Extensive experiments show that I-Segmenter achieves accuracy within a reasonable margin of its FP32 baseline (5.1 % on average), while reducing model size by up to 3.8x and enabling up to 1.2x faster inference with optimized runtimes. Notably, even in one-shot PTQ with a single calibration image, I-Segmenter delivers competitive accuracy, underscoring its practicality for real-world deployment.
Sources
- SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
- SegViT: Semantic Segmentation with Plain Vision Transformers
- SegViTv2: Exploring Efficient and Continual Semantic Segmentation with Plain Vision Transformers
- Per-Pixel Classification is Not All You Need for Semantic Segmentation
- Understanding and Overcoming the Challenges of Efficient Transformer Quantization
- A Survey of Techniques for Optimizing Transformer Inference
- MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer
- Rethinking Vision Transformers for MobileNet Size and Speed
- EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformers
- Mobile-Former: Bridging MobileNet and Transformer
- EasyQuant: An Efficient Data-free Quantization Algorithm for LLMs
- SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
- PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantization
- Mixed Non-linear Quantization for Vision Transformers
- Training data-efficient image transformers & distillation through attention
- Knowledge Distillation in Vision Transformers: A Critical Review
- RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformers
- ADFQ-ViT: Activation-Distribution-Friendly Post-Training Quantization for Vision Transformers
- Mix-QViT: Mixed-Precision Vision Transformer Quantization Driven by Layer Importance and Quantization Sensitivity
- Gaussian Error Linear Units (GELUs)
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs