Tug of Watts: A study on the Energy - Accuracy Trade-off of Large Language Models on Medical Question Answering Tasks
cs.CL, cs.AI
Submitted: 2026-11-02
Updated: 2026-11-02
Comments: Accepted to "Workshop on GREEN-AI: Grounding Research on Energy- and resource-Efficient Networks for sustainable AI", Co-Located with CIKM 2026, November 08, 2026, Rome, Italy
Code: https://github.com/ag-sc/autollm
License: http://creativecommons.org/licenses/by/4.0/
The gist: Large Language Models (LLMs) are increasingly used in medical question answering (QA) systems.
Terminology
Abstract
Large Language Models (LLMs) are increasingly used in medical question answering (QA) systems. While recent works aim at incrementally improving the downstream task accuracy, an empirical estimation of the models' energy consumption and the associated carbon footprints is often overlooked. In this study, we demonstrate how practitioners can estimate energy consumption and arrive at reliable emission estimates for medical QA tasks available in the OpenMedical-LLM leaderboard by investigating 13 LLMs from 3 model families ranging from 270M to 14B parameters. We discuss how one can critically interpret energy versus accuracy trade-offs for a set of models for a given task. With this work, we underline that reporting disjoint energy and accuracy metrics alone does not suffice for careful trade-off assessments, but it also demands systematic construction and interpretation of joint trade-off metrics. We envisage such studies aiding practitioners in adopting better and more efficient models for their task and helping them become active participants in building sustainable LLM pipelines.
Sources
- Capabilities of GPT-4 on Medical Challenge Problems
- MedGemma Technical Report
- BioMedLM: A 2.7B Parameter Language Model Trained On Biomedical Text
- Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Models
- Rethinking Parameter Counting in Deep Models: Effective Dimensionality Revisited
- Does Model Size Matter? A Comparison of Small and Large Language Models for Requirements Classification
- Smart but Costly? Benchmarking LLMs on Functional Accuracy and Energy Efficiency
- Qwen3 Technical Report
- The Llama 3 Herd of Models
- Gemma 2: Improving Open Language Models at a Practical Size
- Gemma 3 Technical Report
- Comparing energy consumption and accuracy in text classification inference
- Energy-Aware LLMs: A step towards sustainable AI for downstream applications
- Systematic Weight Evaluation for Pruning Large Language Models: Enhancing Performance and Sustainability
- From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inference
- Characterizing LLM Inference Energy-Performance Tradeoffs across Workloads and GPU Scaling
- Med42 -- Evaluating Fine-Tuning Strategies for Medical LLMs: Full-Parameter vs. Parameter-Efficient Approaches
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data