ProfileFoundry: A Synthetic Person-Object Substrate for Privacy, Memory, and Tool-Use Evaluation in LLM Agent
cs.CL
Submitted: 2026-06-24
Updated: 2026-08-28
Code: https://github.com/selvamsriram/ProfileFoundry
License: http://creativecommons.org/licenses/by/4.0/
The gist: Foundation-model research increasingly needs data about people: user state, personal histories, relationships, contact-like fields, documents, and longitudinal updates.
Terminology
Abstract
Foundation-model research increasingly needs data about people: user state, personal histories, relationships, contact-like fields, documents, and longitudinal updates. Real user data is difficult to share, perturb, audit, or redistribute responsibly, while independently generated fake fields rarely preserve the cross-field and temporal consistency needed for controlled evaluation. We present ProfileFoundry, a deterministic generator and fixed reference release of 100,000 adult synthetic Person Objects across eight locales. Each object combines a typed current snapshot, household, family, and employer links, snapshot-aligned events, normalized relational views, and generation provenance. The release contains 709,228 events, 40,338 households, 52,491 employers, and 518,564 directed relationship edges. We report evidence in separate categories: selected population-marginal comparisons, per-object invariant checks, release-wide referential and temporal closure, and coincidence/provenance screens. A pilot case study, MatchDesk, uses certified coincidences, typed events, and history truncation to evaluate whether models distinguish corroborated identity evidence from underdetermined matches. ProfileFoundry is not a population-fidelity model, a rendered-text corpus, or a formal privacy mechanism. Instead, it is a responsible synthetic source layer for constructing downstream foundation-model evaluations involving memory, privacy, document understanding, record linkage, and agent state while keeping the synthetic person behind each artifact inspectable.
Sources
- Scaling Synthetic Data Creation with 1,000,000,000 Personas
- PIIBench: A Unified Multi-Source Benchmark Corpus for Personally Identifiable Information Detection
- Privasis: Synthesizing the Largest "Public" Private Dataset from Scratch
- LaMP: When Large Language Models Meet Personalization
- PersonaBench: Evaluating AI Models on Understanding Personal Information through Accessing (Synthetic) Private User Data
- LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering