DrawOffer: Evidence-Grounded Longitudinal User-State Modeling for Career Support
cs.AI
Submitted: 2026-11-02
Updated: 2026-11-02
Comments: 9 pages, 4 tables. Evaluation code and aggregate results: https://github.com/ql-link/DrawOffer-Eval . Core engine private; not all experiments independently reproducible
Code: https://github.com/ql-link/DrawOffer-Eval
License: http://creativecommons.org/licenses/by/4.0/
The gist: Career-support assistants must distinguish user claims from evidence of current ability and withhold judgments when evidence is insufficient.
Terminology
Abstract
Career-support assistants must distinguish user claims from evidence of current ability and withhold judgments when evidence is insufficient. DrawOffer separates language-model observation extraction from deterministic inference over source-linked records to maintain uncertain ability estimates, experience assessments, and revisable issues. We evaluate 102 fictional personas with 580 scored checkpoints, a historical full-context and Mem0 comparison, and three auxiliary knowledge-tracing datasets. On 37 test personas, DrawOffer passes 169/210 checkpoints; checkpoint-defined issue precision and recall are 0.875 and 0.636. Across three development simulations, 87.0% of checkpoint verdicts agree, including persistent failures. A cache-only development ablation replacing answer-evidence requirements with confidence-only settlement reduces majority-vote passes from 306 to 293 of 370 and issue precision from 0.927 to 0.725. Equalizing positive source weights improves predictive metrics without changing total passes; removing decay gives mixed results. These findings support an exploratory systems contribution centered on evidence sufficiency. Developer-authored scenarios, unmatched baseline models, historical test exposure, and incomplete public artifacts limit generalization and independent reproduction.
Sources
- LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory
- Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
- Zep: A Temporal Knowledge Graph Architecture for Agent Memory
- A-MEM: Agentic Memory for LLM Agents
- Belief Memory: Agent Memory Under Partial Observability
- EdNet: A Large-Scale Hierarchical Dataset in Education
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents