Following the Flow the Law Requires: Training Voice Agents to Transfer to Unseen Regulated Procedures in Debt Collection

The gist The study demonstrates that a single small policy can be trained once and then follow procedures that were not seen during training, specifically improving documented procedure-following behaviour on procedures from an unseen lineage while maintaining strong grounded behaviour.

The Problem Addressed

Voice agents built on large language models (LLMs) must follow a conversational flow set by law in regulated applications like debt collection, where simply completing the task is insufficient The agent must confirm it is speaking to the debtor before mentioning the debt, identify itself and the creditor, state the amount, and avoid prohibited language Every creditor turns these rules into its own procedure and revises it An agent that says the right things in the wrong order is still non-compliant Neither of the two established ways of building such agents is satisfactory, as a free-form LLM agent cannot guarantee the flow Traditional flow-based chatbots and voicebots guarantee the flow by construction, but their hand-authored intent classifiers and state machines break on the variety of real calls The legal requirements are shared, but each creditor turns them into its own procedure, with its own disclosure order, tool names, payment-request limits A policy trained for one procedure must be retrained and revalidated when the procedure changes, and it does not carry over to a new client

The Proposed Method: FlowSpec

The central design principle is that the four consumers are not independently maintained artefacts but are four representations derived from the same machine-readable procedure (Fig. 1) Each deployment is represented by one JavaScript Object Notation (JSON) document loaded through a common interface A representative procedure contains 15 states, 7 tools, 15 customer events, 35 constraints, and a 57-entry utterance catalogue States define the dialogue flow and specify the utterances allowed in that phase and the tools that must run on entry Tools are explicitly declared in the procedure rather than assumed by deployment-specific code, specifying their name, argument schema, and gating conditions Speaking is represented as a call to a reply tool whose argument is a list of catalogue identifiers, which are declared as an enumeration containing exactly the catalogue entries available in that conversation This design ensures that an approved catalogue entry is the only text that can be spoken by construction, meaning compliance review applies to a finite catalogue of approved sentences

Training and Population Generation

The policy is fine-tuned on 2,506 conversations from 297 generated procedures derived from 7 base lineages Identifiers, tool names and flow structure are randomized per example while one complete lineage is held out The population of 340 procedures is generated with seed 42 using eight base lineages to construct the synthetic procedure population A mutation pass generates synthetic procedures by rewriting the procedure document rather than perturbing its rendered prompt Two properties are randomized independently for every generated conversation, including catalogue identifiers and their order, to encourage transfer across procedures The agent side of the conversation generator is deterministic with respect to the procedure, walking the state machine rather than using an LLM to generate agent behaviour Conversations that violate a constraint are rescored and removed before training, ensuring the resulting training data is guaranteed to follow the flow structure

Evaluation and Results

The evaluation separates checks into documented checks, which test behaviour determined by the procedure, and grounded checks, which test behaviour determined by the observed conversation Documented checks test behaviour whose expected condition is determined by the procedure, such as which tools a state requires and which disposition closes the call Grounded checks test behaviour whose expected condition is determined by the observed conversation, such as whether a tool argument matches what the caller said Before training, the policy passes 97–100% of grounded checks but only 67–71% of documented ones Training raises documented performance to 93–100%, including on the withheld lineage The main difference between the two groups is that the untrained policy already passes 97% and 100% of grounded checks in the related-lineage and whole-lineage groups, leaving relatively little room for improvement on those checks The largest measured improvement is therefore on documented checks, whose reference conditions are explicitly determined by the procedure Scenario-level pass rates show that across the three related-lineage deployments, 7 of 22 scenarios pass before training and all 22 pass after training On the whole-lineage holdout, the scenario-level pass rate increases from 8/21 to 18/21

Conclusion and Limitations

The results support the feasibility of training a single small policy across multiple procedures, but they do not establish that every component of the proposed design is necessary or that the result generalizes to arbitrary procedures Training therefore produces its largest measured improvement on behaviour whose expected condition is explicitly determined by the procedure The evaluation measures conformance to the formalised procedure rather than correctness against an independently constructed ground truth, and specification defects can remain invisible to the metric derived from that specification Conformance to FlowSpec is not legal compliance, as rules governing when a call is placed are outside the policy’s scope The study does not isolate the contribution of individual components of the training design, and it does not evaluate procedure revisions The results support a narrower conclusion than general-purpose procedure following, demonstrating transfer to the tested held-out lineage rather than establishing general transfer across arbitrary procedures Overall, the study demonstrates a concrete form of transfer where a single 9B policy can improve its documented procedure-following behaviour on procedures from an unseen lineage while maintaining strong grounded behaviour

Appendix A

Procedure Schema and Action Space. Six top-level keys are required: flow id, events, tools, states, faq routing and constraints The schema recognizes eleven constraint types, including max occurrences, once per call, forbid after event, require tool before end and tool pair States do not directly reference catalogue entries by their identifiers; instead, they reference a semantic fine state label that is resolved against the catalogue at rendering time The environment reads these declarations directly, so no company-specific tool branches are required The policy uses a structured action space throughout the call: at = (tool, arguments) Speaking is represented as a call to a reply tool whose argument is a list of catalogue identifiers

Appendix D

Specification Audit. Table VIII lists each defect found in the procedure definitions and their derived evaluation artefacts, its scope, and whether the derived metric exposed it Defect 7, a balance disclosed to an unverified caller, is the kind of disclosure that the Thai Debt Collection Act prohibits (Section I) The audit qualifies the scores in Section IV and does not change them Total 11 defects 8 exposed · 3 not exposed

Appendix C

Training Configuration and Seeding. The policy is Qwen3.5-9B with a rank-32 LoRA adapter (alpha = 64, dropout 0) on all linear layers Training runs for three epochs (468 steps) in bfloat16 on one H100 for about five hours; final training loss is 0.022 and validation loss 0.068 The population of 340 procedures is generated with seed 42 Decoding uses temperature 0, but the runtime is not fully deterministic (Section IV-A4)

Appendix B

The Evaluation Checks. Table VII lists the twelve checks, the class each is assigned according to the source of its reference condition The documented and grounded checks are aggregated in Table III Grounded denominators differ between arms because they count actions the policy emitted The two grounded checks, args and no off flow beats, are defined with respect to the structured action space in Section III-A3

Appendix C

Training Configuration and Seeding. The policy is Qwen3.

1. A small, self-hosted dialogue policy can be trained once and then follow regulated procedures it has never seen. The improved system can deploy a single 9B policy trained on a population of randomized procedures to reliably adhere to unseen regulatory flows, achieving documented check performance improvements from 67–71% to 93–100%.

2. A formulation of regulated voice-agent behaviour as procedure following, in which one machine-readable specification drives the instruction, the tool contract, the supervision and the tests. This allows for a single machine-readable specification (FlowSpec) to generate all necessary components—policy instruction, tool gating, supervision targets, and evaluation checks—ensuring consistency across training and deployment.

3. The policy selects canonical actions [a t = (tool, arguments)], while the environment maps those actions to the deploymentspecific tool names, limits, disclosure order, and closing rules defined by the procedure. The system will enforce compliance by ensuring an approved catalogue entry is the only text that can be spoken by construction, making prohibited language exclusion a structural guarantee rather than a probabilistic outcome.

4. The evaluation should be understood as measuring how consistently the policy follows the behaviour formalised by the specification. This mandates that legal compliance validation must include auditing both the policy's conformance to the specification, and whether the specification accurately represents the intended behaviour, preventing metric blind spots where a defect in the specification can propagate into both training and evaluation without necessarily appearing as a model error.

5. The main measured gain from training is concentrated in behaviour explicitly determined by the procedure. The system will prioritize training efforts on documented checks, whose expected conditions are determined by the procedure, as this component shows substantial improvement, while acknowledging that grounded checks begin near ceiling in this evaluation, leaving substantially less room for measurable improvement.

Voice agents built on large language models (LLMs) now handle customer calls, but in regulated applications completing the task is not enough: the agent must follow a conversational flow set by law. In debt collection, it must confirm it is speaking to the debtor before mentioning the debt, identify itself and the creditor, state the amount, and avoid prohibited language. Every creditor turns these rules into its own procedure and revises it. We ask whether a small, self-hosted policy can be trained once and then follow regulated procedures it has never seen. One machine-readable specification per deployment, FlowSpec, generates the policy instruction, gates the tools, supplies the supervision and defines the tests; the agent speaks only approved catalogue sentences. We fine-tune a 9-billion-parameter model on 2,506 conversations from 297 randomized procedures and evaluate a Thai outbound-call agent on four unseen deployment procedures (three debt collection, one non-debt), one from a wholly withheld lineage. Documented checks, fixed by the procedure, rise from 67--71 percents to 93--100 percents, while grounded checks, fixed by the conversation, already pass at 97--100 percents before training; scenario pass rate rises from 15/43 to 40/43. An audit of the shared specification finds 11 defects, 3 of which its own metric did not expose.