DataWeave: Deploying Human-LLM Analytics for Exploratory Structured Data Analysis
cs.AI, cs.HC
Submitted: 2026-10-02
Updated: 2026-10-02
License: http://creativecommons.org/licenses/by/4.0/
The gist: Data journalism, the practice of using data analysis to surface newsworthy stories, depends increasingly on the ability of reporters and investigative journalists to uncover trends, disparities, and
Abstract
Data journalism, the practice of using data analysis to surface newsworthy stories, depends increasingly on the ability of reporters and investigative journalists to uncover trends, disparities, and accountability narratives. In practice, exploring large structured datasets remains slow and brittle: journalists must navigate hundreds of variables across many datasets over years, understand data coding conventions, and write non-trivial analysis code while hypotheses evolve. Although LLMs are often touted as "ask in English, get SQL/answers," real newsroom workflows expose recurring failures, e.g., schema mismatches and drift, misread domain semantics and units, and silent assumptions. We present DataWeave, a system that addresses these needs by combining conversational interaction, schema grounding, analytical planning, and executable query generation to support exploratory analysis over structured data. Rather than treating LLMs as autonomous answer engines, DataWeave frames them as interactive partners whose outputs can be inspected, corrected, and steered as hypotheses shift. We present a case study with professional journalists using our system to analyze the U.S. Department of Education's Integrated Postsecondary Education Data System (IPEDS), a high-stakes public dataset with substantial domain semantics and frequent schema updates. We also report how deployment experience and iterative refinement shaped the current DataWeave architecture and its analytical workflow. Our findings distill design principles and deployment lessons for trustworthy human-LLM collaboration in structured data analysis.
Sources
- GPT-4 Technical Report
- Gpt-4: A Review on Advancements and Opportunities in Natural Language Processing
- Training and Evaluating a Jupyter Notebook Data Science Assistant
- Is GPT-4 a Good Data Analyst?
- Conversational Challenges in AI-Powered Data Science: Obstacles, Needs, and Design Opportunities
- LIDA: A Tool for Automatic Generation of Grammar-Agnostic Visualizations and Infographics using Large Language Models
- OpenTab: Advancing Large Language Models as Open-domain Table Reasoners
- The Dawn of Natural Language to SQL: Are We Fully Ready?
- Is Self-Repair a Silver Bullet for Code Generation?
- InsightBench: Evaluating Business Analytics Agents Through Multi-Step Insight Generation
- PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models
- Data-Copilot: Bridging Billions of Data and Humans with Autonomous Workflow
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems