Emergent Abilities in Large Language Models: A Survey
cs.LG, cs.AI, cs.CL
Submitted: 2025-02-28
Updated: 2026-08-26
Code: https://github.com/SajjjadAyobi/PersianQA
License: http://creativecommons.org/licenses/by-sa/4.0/
The gist: Large Language Models (LLMs) are leading a new technological revolution as one of the most promising research streams toward artificial general intelligence.
Terminology
Abstract
Large Language Models (LLMs) are leading a new technological revolution as one of the most promising research streams toward artificial general intelligence. The scaling of these models, accomplished by increasing the number of parameters and the magnitude of the training datasets, has been linked to various so-called emergent abilities that were previously unobserved. These emergent abilities, ranging from advanced reasoning and in-context learning to coding and problem-solving, have sparked an intense scientific debate: Are they truly emergent, or do they simply depend on external factors, such as training dynamics, the type of problems, or the chosen metric? What underlying mechanism causes them? Despite their transformative potential, emergent abilities remain poorly understood, leading to misconceptions about their definition, nature, predictability, and implications. In this work, we shed light on emergent abilities by conducting a comprehensive review of the phenomenon, addressing both its scientific underpinnings and real-world consequences. We first critically analyze existing definitions, exposing inconsistencies in conceptualizing emergent abilities. We then explore the conditions under which these abilities appear, evaluating the role of scaling laws, task complexity, pre-training loss, quantization, and prompting strategies. Our review extends beyond traditional LLMs and includes Large Reasoning Models (LRMs), which leverage reinforcement learning and inference-time search to amplify reasoning and self-reflection. However, emergence is not inherently positive. As AI systems gain autonomous reasoning capabilities, they also develop harmful behaviors, including deception, manipulation, and reward hacking. We highlight growing concerns about safety and governance, emphasizing the need for better evaluation frameworks and regulatory oversight.
Sources
- A Theory for Emergence of Complex Skills in Language Models
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- Constitutional AI: Harmlessness from AI Feedback
- Mechanistic Interpretability for AI Safety -- A Review
- User Intent Recognition and Satisfaction with Large Language Models: A User Study with ChatGPT
- Could a Large Language Model be Conscious?
- States Hidden in Hidden States: Implicit Discrete State Representations Emerge in LLMs' Hidden States
- AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors
- Scaling Laws for Predicting Downstream Performance in LLMs
- Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
- CausalLM is not optimal for in-context learning
- A Survey on In-context Learning
- Understanding Emergent Abilities of Language Models from the Loss Perspective
- Demystifying Prompts in Language Models via Perplexity Estimation
- Large Language Model based Multi-Agents: A Survey of Progress and Challenges
- Deception Abilities Emerged in Large Language Models
- A Theory of Emergent In-Context Learning as Implicit Structure Induction
- Structured Prompting: Scaling In-Context Learning to 1,000 Examples
- Measuring Massive Multitask Language Understanding
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks