A Survey of LLM Prompt Datasets: Taxonomy, Linguistic Patterns, and Practical Uses
cs.LG, cs.CL
Submitted: 2025-10-10
Updated: 2026-08-27
Comments: Accepted to Findings of the Association for Computational Linguistics: EMNLP 2026. 30 pages
Code: https://github.com/tatsu-lab/stanford_alpaca
License: http://creativecommons.org/licenses/by/4.0/
The gist: We compile 129 public LLM prompt datasets with more than 1.22TB and more than 673M instances and organize them into a unified taxonomy.
Terminology
Abstract
We compile 129 public LLM prompt datasets with more than 1.22TB and more than 673M instances and organize them into a unified taxonomy. We use seven datasets for detailed analysis and identify lexical, syntactic, and semantic patterns that distinguish prompts from general text. We evaluate these features in prompt filtering, source domain routing, and elicited response quality analysis. A 63 dimensional linguistic feature set that can be extracted on a CPU achieves over 91% of the F1 of GPU sentence embeddings and reduces latency per request by nearly half. In cross dataset routing, sentence embeddings reach 0.74 Macro-F1 and a structural feature subset reaches 0.61, compared with the chance level of 0.20. Structural routing features remain negatively associated with elicited response quality after prompt length is controlled on UltraFeedback. The apparent positive association of type to token ratio becomes much smaller under the same control. These findings support prompt processing pipelines that combine efficient structural features with robust semantic routing. Our catalogue, annotations, and code are given at https://github.com/ymzhang-cs/prompt-dataset-analysis.
Sources
- OpenCodeInstruct: A Large-scale Instruction Tuning Dataset for Code LLMs
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs
- FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance
- PerceptionLM: Open-Access Data and Models for Detailed Visual Understanding
- UltraFeedback: Boosting Language Models with Scaled AI Feedback
- Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselves
- Scaling Synthetic Data Creation with 1,000,000,000 Personas
- LongForm: Effective Instruction Tuning with Reverse Instructions
- Bactrian-X: Multilingual Replicable Instruction-Following Models with Low-Rank Adaptation
- Datasets for Large Language Models: A Comprehensive Survey
- Beyond Prompt Content: Enhancing LLM Performance via Content-Format Integrated Prompt Optimization
- Crosslingual Generalization through Multitask Finetuning
- RouteLLM: Learning to Route LLMs with Preference Data
- PHYBench: Holistic Evaluation of Physical Perception and Reasoning in Large Language Models
- CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmark
- The Prompt Report: A Systematic Survey of Prompt Engineering Techniques
- Instruction Tuning for Large Language Models: A Survey
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks