How a shared state is described determines whether AI agents synchronize

arXiv:2608.06968 · physics.soc-ph, cs.AI, cs.CY · Submitted 2026-08-07 · Read on arXiv

Research Center for Advanced Science and Technology, The University of Tokyo · Department of Aeronautics and Astronautics, School of Engineering, The University of Tokyo

physics.soc-ph, cs.AI, cs.CY

Submitted: 2026-08-07

Updated: 2026-09-23

Code: https://github.com/tkEzaki/observation-maps-llm-agents

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 100/100

Terminology

Summary

The paper investigates whether the way a physical state is encoded into text for a language-model agent — which the authors call the observation map — changes the collective dynamics of a population of such agents, even when the underlying physical state is held fixed. The central claim is that state encodings therefore form part of a model-dependent effective interaction law, not a neutral interface. The authors note that Language-model agents never act directly on an environment; they act on an encoded description of it, and that while single-turn prompt sensitivity is established, what is not is whether it survives feedback, so that a population driven by one encoding ends in a qualitatively different state than the same population driven by another.

The study uses a deliberately minimal circular synchronization assay. Agent i has an unwrapped phase xi(t), a wrapped phase θi(t) = xi(t) mod 2π, and a fixed natural increment ωi. At each step, a language model sees only a text description of the other agents' relative phases and returns fi(t) ∈ −1, 0, +1 (advance, stay, retard), with no goal and no instruction to synchronize. The deterministic engine then applies:

xi(t + 1) = xi(t) + ωi + Kfi(t),

where K sets how far a single action shifts the phase. The model was not given K, absolute phase, agent identity, time, history, or a preferred state. Each call is stateless, the coupling lives only in the shared deterministic engine, and all three payloads are computed from the same 24-bin measurement of the same field, so anything that separates the conditions must pass through the action distribution.

The three encodings were:

  • Moments: the first three circular moments of the relative-phase field, summarizing mean direction and concentration at three harmonic orders.

  • Centers: a 24-bin histogram labelled by bin center.

  • Intervals: the same 24 bin masses labelled by bin interval to six decimals — differing from centers only in serialization, since both carry identical masses.

Experiments used N = 17 agents over T = 100 steps at couplings K ∈ −0.15, 0, 0.08, 0.15, six physical seeds matched across encodings, one fixed natural-increment vector reused everywhere, and the model gpt-5.4-mini. All 122,400 calls returned valid actions; the independent inference unit is the physical seed. The macroscopic observables include the standard Kuramoto order parameters r1(t), r2(t), r3(t), and the descriptive contrast Q2 = r2 − r1 distinguishing two-cluster from one-cluster states.

At K = 0.08, the mean final polar order r1 was 0.999 for moments, 0.710 for centers, and 0.382 for intervals; at K = 0.15, respectively 0.996, 0.727, and 0.507. Under the prespecified criterion r1(T) ≥ 0.9, moments locked in all six seeds at both positive couplings, whereas centers and intervals locked in none. All six paired seeds favoured moments over each histogram encoding (p = 0.03125, two-sided exact sign test). Intervals more often retained two-cluster structure: Q2 averaged 0.128 at K = 0.08 and 0.213 at K = 0.15 for intervals, versus −0.004 and −0.013 for moments.

The K = 0 condition is an exact negative control: the coupling term vanishes, so all three encodings produced identical r1, r2, r3 trajectories from each shared seed even though their action distributions differed. This excludes mismatched initial conditions and encoding-specific engines. At K = −0.15, all three stayed low in polar order but remained active, with mean signed actions differing in sign: −0.335 for moments, −0.033 for centers, and +0.062 for intervals — suppression of polar order rather than inactivity.

To measure the microscopic response operator alone, the authors presented synthetic fields under each encoding and estimated the probability of the three actions from repeated queries: p R(ρ) = [p R(−1ρ), p R(0ρ), p R(+1ρ)]. Sharpening the same single-peaked field did not move the three encodings together: moments ended as a purely directional controller that always acted, intervals as a two-lobed response with little net direction, and centers as a reversed directional response with a net retarding bias. These characterizations reproduced across two separately acquired batches. Across two-peaked, asymmetric, antipodal, and sparse few-peer fields, the encodings continued to differ in activity, signed bias, and total-variation distance. Even for an antipodal field, where the one-cycle component a1 collapses for all three encodings, they still differed in the full action distribution, with moments switching from always acting to mostly abstaining. Sweeping the directional imbalance ε, moments was almost inactive at balance but acted already at the smallest nonzero imbalance tested, ε = 0.02 — sharp but grid-limited, not discontinuous.

Forty-eight fields that arose during the collective runs were selected under a fixed rule, balanced across six trajectory-state types and source encodings. Each was re-encoded and presented under all three encodings to GPT, 32 responses in two blocks (4,608 valid calls). The same field did elicit different action distributions, separated by a mean pairwise total-variation distance of 0.344. None of 5,000 relabellings of the presented-encoding labels produced a separation this large (p = 0.0002, the resolution floor). The separation was 3.76 times the test–retest block noise (0.344 versus 0.092). Crucially, centers and intervals — information-matched, differing only in how the same 24 bin masses are labelled — separated by 0.290 (95% CI 0.220–0.364), 3.2 times that block noise, so serialization moves the operator with the retained information fixed.

Two further prespecified tests on source-dependent modulation — whether fields generated under different encodings draw different average actions (p = 0.078) and whether re-encoding's effect depends on the source encoding (p = 0.072) — did not cross the prespecified threshold and remain unresolved.

On the same 48-field panel, claude-haiku-4-5-20251001 and gemini-3.5-flash showed a re-encoding effect on the action distribution (p = 0.0002 for each). The authors then ran a second matched collective experiment in Claude, reusing the six GPT seeds. At K = 0.15, moments locked in 0/6 seeds and the histogram encodings in 6/6; at K = 0.08 the counts were 0/6, 2/6 and 5/6. Exact within-seed permutations over all 46,656 label reassignments gave p = 0.00103 (terminal-lock score) and p = 0.000386 (final-order score), with means ordered moments < centers < intervals. "Both families therefore showed a collective encoding effect, but the encoding-to-locking relationship reversed: moments produced synchronization in GPT but only partial order in Claude, and the histogram encodings the opposite. None of the three tested encodings was uniformly optimal across GPT and Claude." Prespecified replication gates A and B passed (encoding has an effect; the effect is qualitative), while Gate C failed — the GPT map was not reproduced.

Three GPT inputs carried the same circular-moment values presented differently: original narrative, a table layout, and a version with added task-irrelevant text. Even with identical numbers, presentation mattered: the mean pairwise distance among the three versions was 0.311 (p = 0.0002), against a within-version block noise of 0.069. Reformatting as a table moved the operator by 0.145 (2.1× noise, interval overlapping test–retest range), while adding task-irrelevant context moved it by 0.414 — exceeding the 0.344 between different encodings. The padding condition changed context volume, value position, and length together, so it is a compound manipulation. Two supplementary controls argued against a simple prompt-length account: "a monotone length account predicts that whichever histogram prompt lies farther in characters is also farther in response; the opposite happened at both anchors, and a second control found almost no change in response distance when the character gap was multiplied by 2.6 to 7.8."

The authors asked whether a cheap statistical model could replace the language model inside collective simulation. The moments, centers, and intervals branches all met their prespecified in-domain (compressibility) criteria, but "only the revised moments model cleared the closed-loop support stage… and was evaluated prospectively. For centers and intervals the nearby training fields either failed to reproduce the collective-field response or disagreed with one another, so both branches stopped under their prespecified rules before any prospective acquisition. For moments, a revision separating exact-balance abstention from active fields was frozen and then evaluated prospectively against 44 new collective runs, where it improved average prediction of microscopic actions and of final collective order without being uniformly superior on every endpoint."

The authors conclude that our results identify the observation map as a constituent of the effective policy a language-model agent implements, stronger than an open-loop prompt-sensitivity result because the interface change propagated through feedback to select different outcomes. They note the cross-family reversal is central: the simple reading of the GPT experiment, that moment compression is intrinsically synchronization-promoting, is falsified by Claude under the same design. This connects to the classical point that the observation model is part of the policy and to performative prediction, since a small change in a stochastic action distribution may disappear, accumulate or redirect the states agents observe later.

Limitations include: the collective experiments used a minimal, stateless, three-action system at a single size N = 17 with a small coupling grid and fixed horizon; Collective dependence was tested in GPT and Claude, the presentation controls only in GPT, and all three tested models were lower-cost, non-frontier offerings queried under one prompt contract at a fixed temperature. Practically, the authors recommend that the observation serializer is a versioned component: it belongs in what an agent evaluation reports, and must be revalidated with the model, in the closed loop in which both will operate.

Improvements for AI systems

Here are specific improvements to AI systems suggested by the paper, and what the improved systems can do.

  1. Treat the observation serializer as part of the policy, not as a neutral input pipeline.

The paper shows that changing only how a physical state is serialized into text changes collective outcomes, even when the underlying state and information content are identical. AI systems should therefore version and track the observation map alongside model weights, prompts, and environment code.

  1. Validate observation encodings in closed-loop, multi-agent settings, not just single-turn benchmarks.

Single-turn prompt sensitivity does not predict whether an encoding effect survives feedback. The paper found that encoding differences were amplified through collective dynamics, causing one encoding to synchronize while another did not. Evaluation pipelines must include closed-loop simulations to catch these emergent effects.

  1. Do not assume a universally optimal encoding, even for the same model family or task.

The GPT and Claude experiments produced opposite encoding-to-synchronization relationships. Systems should not hard-code best prompt or serializer formats across models; instead, they should search or adapt the observation map per model and deployment context.

  1. Include exact negative controls and information-matched ablations when testing interface changes.

The K = 0 control and the centers-vs-intervals comparison isolate encoding effects from confounds like initial conditions or retained information. AI evaluation suites should include such controls to distinguish genuine policy changes from spurious prompt-length or content artifacts.

  1. Add presentation and formatting as explicit experimental variables, with task-relevant information held fixed.

The paper found that reformatting identical numerical content (table vs. narrative) and adding task-irrelevant text changed the agent's response operator. AI systems should track formatting context, and interface designers should treat layout and padding as tunable parameters with measurable behavioral impact.

  1. Build and evaluate cheap surrogate models using closed-loop support criteria, not just in-domain fit.

The surrogate analysis showed that a statistical model could reproduce open-loop responses but still fail to reproduce collective dynamics. Improved systems should validate surrogates against closed-loop field distributions before using them to replace expensive language models in simulation.

  1. Measure and report full action distributions, not just point predictions.

The paper shows that different encodings can produce different abstention rates, signed biases, and response sharpness even when mean behavior looks similar. AI systems that output action probabilities should be evaluated on distributional properties such as total-variation distance, activity level, and bias.

  1. Use permutation and exact-significance tests on matched seeds to detect interface effects.

Because language-model outputs are stochastic and pairwise-matched seeds are needed, the paper's within-seed permutation tests provide reliable evidence. Improved evaluation systems should use paired-seed designs and exact permutation tests rather than pooled comparisons.

  • Detect emergent synchronization, polarization, or collapse caused solely by the observation format, before deployment, by running closed-loop simulations over multiple matched seeds and encodings.

  • Avoid false conclusions from single-turn evaluations by re-testing observation maps under feedback, where small action-distribution shifts can accumulate into qualitatively different collective states.

  • Automatically select or recommend an observation serializer for a given model family and task, rather than reusing one from a different model or application.

  • Quantify the sensitivity of multi-agent outcomes to serialization choices (bin labels, table vs. narrative, added context), and report these sensitivities alongside performance metrics.

  • Identify when a cheap surrogate model is safe to use for multi-agent simulation, based on whether it reproduces the closed-loop field distribution rather than only open-loop accuracy.

  • Flag risky presentation changes — such as adding task-irrelevant text or changing number formatting — that shift the effective policy even though the underlying information is unchanged.

  • Produce evaluation reports that version the observation map, allowing reproducible comparisons across models, prompts, and serializers, so that same physical state, different dynamics failures are caught early.

Abstract

Language-model agents act on state encodings of their environment, yet these are treated as interchangeable interfaces. Using pretrained language models, we designed a circular-synchronization experiment applying a state-encoding intervention while holding the physical system fixed: each agent sees only a summary of its neighbours' relative phases and chooses to advance, stay or retard. Encoding that state as low-order circular moments rather than as a histogram selected different collective outcomes. In GPT the moment encoding synchronized the population in 6/6 seeds and the histogram encodings in 0/6; the effect replicated in Claude but reversed direction. Replaying identical fields shifted each agent's advance/stay/retard probabilities far beyond within-encoding repeat variation, in GPT, Claude and Gemini; in GPT, presentation alone shifted the operator with the moment values fixed. State encodings therefore form part of a model-dependent effective interaction law, not a neutral interface.

Sources

Related papers