Hibana: Fast Agentic Inference on Unified-Memory Devices
cs.CL, cs.AI
Submitted: 2026-11-07
Updated: 2026-11-07
Code: https://github.com/sharar-project/hibana
Project page: https://sharar-project.github.io
Terminology
Sources
- SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding
- LLM in a flash: Efficient Large Language Model Inference with Limited Memory
- Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignment
- Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
- Quamba: A Post-Training Quantization Recipe for Selective State Space Models
- Quamba2: A Robust and Scalable Post-training Quantization Framework for Selective State Space Models
- Generating Long Sequences with Sparse Transformers
- GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- MiniLLM: On-Policy Distillation of Large Language Models
- Fast Matrix Multiplications for Lookup Table-Quantized LLMs
- Lynx: Enabling Efficient MoE Inference through Dynamic Batch-Aware Expert Selection
- Distilling the Knowledge in a Neural Network
- SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengths
- LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
- Mixtral of Experts
- Scaling Laws for Neural Language Models
- Efficient Memory Management for Large Language Model Serving with PagedAttention
- Fast Inference from Transformers via Speculative Decoding
- Training-Free Loosely Speculative Decoding: Accepting Semantically Correct Drafts Beyond Exact Match
- EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data