Withholding the Completing Chunk: Exact Release-Boundary Equivalence for Production Streaming Guardrails

This paper studies a deterministic construction for moderating streamed LLM output, where each output-danger signature is the conjunction of two lexical predicates. The core mechanism: before releasing each chunk, the guard scans the accumulated prefix and withholds the first chunk that makes both predicates observable.

Key results from the evaluation:

• Mechanism matrix (RQ1–RQ3): "All 32 pair trials were detected, all 32 labels matched the buffered scanner, and all 32 completing chunks were withheld. All eight single-predicate controls passed. The median number of fixture characters released before a block was 48, with a maximum of 73."

• Streaming-strategy comparison (RQ7): "The full-prefix and complete-buffer strategies detect every configured pair... Full-prefix scanning preserves streaming but repeatedly processes old text. The bounded window misses pairs separated by more context than it retains, while the chunk-local strategy also misses pairs fragmented across chunk boundaries. Specifically: Cumulative full prefix 128/128 (100.00%), 512-character rolling window 96/128 (75.00%), Chunk local 38/128 (29.69%), Complete buffer 128/128 (100.00%)."

• Human-derived safe-response subset (RQ4): The scanner flagged 0/338 responses, giving an observed conditional flag incidence of 0%. The two-sided Wilson 95% interval is 0%–1.12%.

• Contextual stress screen: "Four answers were flagged (0.0305%), all within 987 cybersecurity records... The review found defensive malware triage, detection rules, and threat-framework mapping that mentioned both lexical signals. These are true pair matches but contextual false positives."

• External unsafe-response scope check (RQ5): The fixed-pair scanner detected 0/394 responses (observed detection rate 0%; two-sided Wilson 95% interval 0%–0.97%).

• Calibrated semantic-guard comparison (RQ8): "The semantic guard classified 310/338 safe-subset responses as safe and flagged 28/338 (8.28%; Wilson 95% interval 5.79%–11.71%). It classified 202/394 unsafe-subset responses as unsafe (51.27%; Wilson 95% interval 46.35%–56.17%)."

• Scanner latency (RQ6): across 50 repetitions of a 16,384-character response, median scanner time ranged from 13.261 ms with 512-character chunks to 829.640 ms with eight-character chunks. The paper notes: The 16,384-character, eight-character condition requires 2,048 scans and exceeds 810 ms, roughly 1,017 times the one-off buffered scan.

• Concurrent throughput: Aggregate throughput remains approximately 75 responses per second as concurrency increases.

Formal properties established:

• Proposition 1 (completing-chunk non-release): no character of xt is released by the guard when the first chunk completing a pair is scanned before release.

• Proposition 2 (buffered/streaming terminal-label parity): the stream's first non-empty label is equal to D(Pt) at its trip prefix.

• Proposition 3 (released-prefix invariant): Before blocking, Ri = Pi. After blocking at t, Ri = Pt−1 for every i ≥ t.

Key limitations acknowledged:

• The construction is exact only with respect to its fixed predicates; it does not estimate general harmful-output recall, infer intent, or prevent release of the first predicate before the second appears.

• The first member of a pair may be released before the second arrives.

• The evaluation covers exactly four signature families and cannot be reported as general harmful-output recall.

• The naive implementation's quadratic cost motivates incremental automata or bounded moderation windows.

Main conclusion: "Pair-completion scanning is a simple way to restore parity between buffered and streamed enforcement for a fixed lexical policy, provided that the application can tolerate moderation chunks large enough to control repeated-scan cost. It is not a reason to replace semantic moderation, authorization checks, red-team testing, or careful stream integration."

Improvements to AI systems:

1. Add a deterministic lexical pair-completion guard layer to streaming LLM output pipelines. The improved system can withhold any chunk that completes a configured conjunction of two lexical predicates, ensuring no streamed response ever exposes both signals together, regardless of chunk boundaries.

2. Implement full-prefix scanning with incremental state to eliminate quadratic re-scan cost. The improved system can detect all configured pair completions in streaming output with 100% recall (128/128 in tests) while maintaining low latency, avoiding the 75% recall of rolling windows and 29.69% recall of chunk-local scans.

3. Use buffered/streaming terminal-label parity as a formal invariant for moderation consistency. The improved system can guarantee that the first non-empty moderation label in a stream matches what a complete-buffer scan would produce, enabling reliable audit trails and deterministic enforcement across deployment modes.

4. Add a bounded-window fallback with explicit miss reporting for resource-constrained environments. The improved system can trade recall for latency (e.g., 512-character window at 75% recall) while logging missed pairs, allowing operators to tune window size against known false-negative rates.

5. Integrate the scanner as a pre-filter before semantic moderation to reduce false positives. The improved system can flag only exact lexical pair completions (0% false positives on safe subsets, 0.0305% on contextual stress screens) before passing survivors to a calibrated semantic guard, cutting the semantic guard’s 8.28% false-positive rate on safe responses.

6. Enable chunk-size adaptive control based on measured scanner latency. The improved system can dynamically select chunk sizes (e.g., 512 characters for 13 ms scans vs. 8 characters for 830 ms scans) to meet latency budgets while preserving detection guarantees, using the paper’s latency model (roughly linear in scan count).

7. Add a release-prefix invariant checker for post-hoc verification. The improved system can assert that before blocking, released prefix equals scanned prefix, and after blocking, released prefix remains fixed at the pre-block state—enabling automated tests for moderation correctness in streaming deployments.

8. Provide a configuration interface for multiple predicate families with per-family detection rates. The improved system can report exact detection rates for each configured pair (e.g., 100% for all 32 tested pairs) and expose Wilson confidence intervals, allowing operators to reason about coverage without overclaiming general harmful-output recall.