Efficient Task Adaptation in Large Language Models: A Survey of Weight-Based, Prompt-Based, and Embedding-Based Adaptations

arXiv:2610.00928 · cs.CL · Submitted 2026-10-01 · Read on arXiv

cs.CL

Submitted: 2026-10-01

Updated: 2026-10-01

Comments: Accepted by AACL-IJCNLP 2026 Main

Code: https://github.com/Jungwona/Awesome-Task-Adaptation-for-Large-Language-Models

License: http://creativecommons.org/licenses/by/4.0/

The gist: As large language models are increasingly deployed across diverse downstream tasks, efficient task adaptation has emerged as a central challenge.

Terminology

Abstract

As large language models are increasingly deployed across diverse downstream tasks, efficient task adaptation has emerged as a central challenge. In response, a wide range of task adaptation methods have been proposed, spanning parameter-efficient fine-tuning, in-context learning, and embedding-injection approaches. However, these lines of work have largely evolved within individual paradigms, leaving their cross-paradigm relationships and trade-offs underexplored, especially for recently emerging embedding-based adaptations. This survey presents a unified framework that categorizes task adaptation methods by where and how task information is encoded: model weights, input prompts, or injected task embeddings. We provide a comprehensive taxonomy that integrates these paradigms, analyze their key strengths and limitations to explain how different adaptation paradigms have evolved, clarify relationships across paradigms, and highlight open problems for future research.

Sources

Related papers