Toward Operationalizing Rasmussen in Evolving Software Systems
summary
The gist
Software operations increasingly rely on SLOs, traces, deployment specifications, and change events, yet dashboards and thresholding practices often expose share-like operational signals as separate…
In short
The paper proposes an automated drift monitor for software operations that maps changing artifacts into a stable compositional state using lineage-aware canonical groups. This system uses isometric log-ratio coordinates to detect boundary-directed drift, allowing operators to distinguish between benign changes and movement toward critical policy limits.
Key concepts
- Compositional Data Analysis (CoDA)
- This statistical framework is used because operational signals like remaining reliability margin are inherently compositional—parts sum up to a whole. CoDA helps analyze these parts together rather than treating them as independent numbers, which is crucial for accurately assessing system state under change.
- Isometric Log-Ratio (ILR) Map
- The ILR map provides a coordinate system for the compositional space. It allows the monitor to represent complex relationships between different operational parts in a way that preserves geometric distances, making it possible to track drift direction and proximity to policy boundaries effectively.
- Lineage-Aware Stabilization
- This technique addresses architectural churn like service splits or renames. By maintaining a lineage map, the system groups evolving components into stable canonical sets. This ensures that dashboard signals remain comparable even as the underlying software structure changes.
- Boundary-Directed Drift Diagnostics
- Instead of simple thresholds, this method looks for drift along a specific direction toward a defined policy boundary. It calculates 'step-to-boundary' to find the minimum change required to cross a safety limit, providing actionable insights into imminent risks.
Terminology used across episodes
This episode discusses
- Toward Operationalizing Rasmussen in Evolving Software Systems · Paper Radio
- Evaluating Asynchronous Semantics in Trace-Discovered Resilience Models: A Case Study on the OpenTelemetry Demo
- Anomaly Detection for Compositional Data using VSI MEWMA control chart
The paper
Toward Operationalizing Rasmussen in Evolving Software Systems · Read on arXiv
Anatoly A. Krasnovsky
Innopolis University
Rasmussen's qualitative drift model explains how successive, locally successful adaptations under efficiency pressures can move a system toward unacceptable operation. Brooker and Hochstein applied this perspective conceptually to software operations without specifying a quantitative procedure for observing drift. We take a step in this direction and propose a runtime observer that quantitatively operationalizes the model's observable part for evolving software. The proposed observer is intended to give engineers evidence for pausing or revising adaptation before declared operating limits are crossed. It tracks how activity or resource use is distributed across components and types of work. Measurement history quantifies accumulated change and recent movement toward or away from current limits. Assuming that direction and limits remain unchanged, the proposed observer calculates the first constraint ahead and the amount of further change required to reach it. Measurements remain comparable across supported component changes, while policy revisions are distinguished from changes in execution. Preliminary runtime studies demonstrated these distinctions under annotated software changes and revised retry limits. A separate multivariate exponentially weighted moving average (MEWMA) experiment detected CPU redistribution earlier with log-ratio coordinates than with raw shares, motivating investigation of representation-dependent observation. Further work will ground constraints in service outcomes, investigate runtime-model acquisition, and test whether decisions informed by observed drift reduce violations at acceptable useful-work loss and intervention cost.
Transcript
Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.
Rosa: We're looking at the paper: "Toward Operationalizing Rasmussen in Evolving Software Systems".
Dev: Here's the gist of the paper. Software operations increasingly rely on SLOs, traces, deployment specifications, and change events, yet dashboards and thresholding practices often expose share-like operational signals as separate scalar panels or baseline distances <ref:2602.05483#pg2> This paper proposes an automated design for a drift monitor that maps changing software artifacts into a stable compositional monitoring state with lineage-aware policy boundary diagnostics <ref:2602.05483#pg2> Rasmussen’s dynamic safety model frames safety as a moving target where systems operate under competing gradients and local adaptations can produce slow drift toward failure as boundaries of safe operation are approached <ref:2602.05483#pg2> Software operations is a natural target because services evolve continuously, are governed by explicit policies such as SLOs and error-budgets, and emit rich telemetry <ref:2602.05483#pg2> Teams often reason about multiobjective operational state as shares like remaining reliability margin across SLOs or engineering effort across work categories <ref:2602.05483#pg2> Applying Euclidean thresholds to such closed data can conflate safe redistribution with risk accumulation because signals form compositional views where relative structure is meaningful <ref:2602.05483#pg2> Artifact-Derived Model Discovery The monitor pipeline involves a sequence of steps where the artifact stream is mapped to a compositional state and then analyzed for drift <ref:2602.05483#pg2> The extraction operator yields a lightweight model Mt = Extract(At), which can include an SLO inventory, service graph, request-class partition, and policy-constraint set <ref:2602.05483#pg2> The process is defined by the pipeline At → Mt → Φ(Mt, At) → xt → x˜t → ILR(x˜t) → report <ref:2602.05483#pg2> The monitor requires a current part inventory, telemetry-to-part mapping, lineage metadata, policy constraints where available, and health signals to function effectively <ref:2602.05483#pg2> Simplex Geometry and Lineage Stabilization The strictly positive composition with D parts lies in the simplex SD−1 = ∪R D >0: Íi xi = 1 <ref:2602.05483#pg3> Aitchison operations define its geometry, and an isometric log-ratio (ILR) map is used as a coordinate representation of that geometry <ref:2602.05483#pg4> Raw ILR coordinates are not directly comparable when services split or SLOs are revised, necessitating a lineage map pit: 1,..., D(t) → 1,..., K from current parts to stable canonical groups <ref:2602.05483#pg4> The monitor maintains x˜t,k = ∑︁ i:pit (i)=k xt,i, x˜t ∈ SK−1 <ref:2602.05483#pg4> This lineage aggregation is an operational stabilization layer that keeps dashboard-level signals comparable under churn while retaining leaf-level subcompositions for drill-down <ref:2602.05483#pg4> Drift Diagnostics on the Simplex One-step drift in the stabilized state is modeled as x˜t+1 = x˜t ⊕ (beta ⊙ g˜t) ⊕ eta˜t <ref:2602.05483#pg5> In ILR coordinates, this becomes additive: z˜t+1 = z˜t + betau˜t + epsilont, where u˜t = ILR(g˜t) <ref:2602.05483#pg5> Only the product betau˜t is identifiable from observed changes, making the primary directional signal the smoothed direction uˆt = Δz˜t /∥Δz˜t ∥ <ref:2602.05483#pg5> A simple barrier B(x˜t) = −ÍK k=1 log x˜t,k diverges as any part approaches zero Policy boundaries are encoded as inequalities hj (x) ≤ 0, which should be represented as log-ratios for ratio constraints like F /R ≤ τ A complementary diagnostic is step-to-boundary along the smoothed drift direction to seek the smallest λ > 0 such that ILR−1(z˜t+λuˆt) ∉ Ω Instantiations and Sanity Check Results The synthetic sanity check over 300 trajectories of length 100 demonstrated that boundary-relevant drift should be separable from benign redistribution Under injected F /R drift, the boundary monitor detected all crossings with a median lead time of 13 windows Controlled split/merge churn resulted in a stable canonical signal with a median maximum error of 0 This confirms that lineage-aware aggregation preserved the canonical signal when lineage was exact The evaluation tested three falsifiable claims, including H1 that exogenous shocks induce non-zero mean drift directions in balance space Positioning and Conclusion The novelty of this work lies in reducing an evolving software architecture—renames, splits, merges, and policy revisions—to a stable compositional monitoring state with lineage-aware policy boundary diagnostics <ref:2602.05483#pg2> The proposed monitor reports drift direction, step-to-boundary, attribution via top-k balances mapped back to the extracted model, and model-health indicators such as ct and mother t This framework links safety-science drift models with artifact-derived observability for evolving systems <ref:2602.05483#pg2> The goal is to discover an operational model from artifacts, map telemetry to a compositional state, stabilize that state under churn through lineage-aware canonical groups, and report log-ratio coherent, boundary-aware drift diagnostics with model-health gating <ref:2602.05483#pg2> The paper concludes by proposing this method as an automated runtime drift observability under evolving part inventories <ref:2602.05483#pg2> References
1: John Aitchison.
Dev: The Statistical Analysis of Compositional Data.
Rosa: Chapman & Hall
2: Anatoly A.
Title and authors: Rosa: So we're looking at this paper by Anatoly Krasnovsky called "Toward Operationalizing Rasmussen in Evolving Software Systems." It’s trying to solve a problem where we have all these metrics like SLOs and traces, but the dashboards just throw all these signals into separate panels, and that can actually cause problems when things are changing.
Dev: Exactly. The core idea here is taking those shifting operational signals—things like remaining reliability margin or engineering effort—and turning them into something stable where we can actually track drift toward failure, not just noise from normal business changes.
Taro: I'm curious if this is something that actually works when the software architecture itself is constantly changing, like services splitting or getting renamed. Because that's usually where our simple thresholding breaks down.
Rosa: That’s what it tackles directly with this paper. It proposes an automated design for a drift monitor that maps those changing software artifacts into a stable compositional monitoring state using lineage-aware policy boundary diagnostics.
Dev: It sounds like they are moving away from just looking at single numbers and trying to understand the whole structure of the system through these measurements.
The paper's summary: Rosa: This paper describes a pipeline where you take the artifact stream—the traces, metrics, deployment specifications—and run it through an extraction operator to get a lightweight model at each time step. This model might include an SLO inventory or a service graph, and then they map all that telemetry into a positive composition.
Dev: So instead of just looking at one metric like error budget share across different services, they are creating this whole structure where everything relates back to these defined parts and policies. It’s about getting the data into a shape that respects how the system is actually organized.
Taro: I've seen work on modeling software effort distribution before, but mapping continuous telemetry directly onto a compositional state seems like a big step when you have things like service splits happening while you’re running the analysis.
Rosa: The paper emphasizes that this process involves updating lineage and then using canonical groups to stabilize things through renames or merges. This keeps the dashboard signals comparable even when the underlying architecture is churning.
Dev: So the main takeaway is they are building a system that extracts structure from chaos—the constant architectural change—to create a stable view where we can see if we’re drifting toward a policy boundary, which is what Rasmussen's safety model suggests.
The paper's improvements: Rosa: One of the main things they suggest is moving beyond simple Euclidean thresholds. They use something called an isometric log-ratio map to represent the geometry of this compositional state, which is much better than raw numbers when you have these kinds of relationships.
Dev: And that leads to their stabilization layer where they keep a lineage map and aggregate parts into canonical groups, which is supposed to make sure that even if a service gets renamed or split, the dashboard level signal stays consistent.
Taro: I wonder about the boundary diagnostics part. They talk about step-to-boundary analysis, which sounds like they are trying to figure out how close we are to actually hitting a hard policy limit in this new space.
Rosa: That’s right. They model drift in this stabilized state additively, and only the product of the change and the direction you're moving is what they can reliably identify as a directional signal.
Dev: And they use log-ratios to encode policy constraints, like a ratio constraint such as 'failures divided by requests is less than some threshold,' so that when things go wrong, it shows up in those coordinates too.
Conclusion: Rosa: So we've seen how this framework takes the constant churn of software architecture—renames and splits—and turns it into a stable compositional view using lineage aggregation to keep dashboards comparable.
Dev: The paper's main contribution seems to be linking that compositional structure directly to safety science concepts like drift and boundary diagnostics, giving us a way to see not just that something is wrong, but in what direction it’s moving relative to the policy constraints.
Taro: For someone focused on resilience testing, the synthetic sanity checks are important; they showed that boundary-relevant drift can be separated from benign redistribution even under injected failures.
Rosa: It suggests we can build an automated system that reports on drift direction and step-to-boundary metrics, which is really helpful for SRE teams trying to decide when to intervene.
Dev: Overall, this paper proposes a method for discovering an operational model from the artifacts themselves, mapping telemetry into a compositional state, stabilizing that state under architectural churn with lineage awareness, and reporting those coherent log-ratio boundary diagnostics.
Taro: It’s about giving us a way to observe the system's safety margin as it evolves through software changes.
Rosa: That’s the core idea behind this paper "Toward Operationalizing Rasmussen in Evolving Software Systems." We'll be looking at how this stuff plays out next time.
More episodes
- 2608.13438-ContactGuard: Pre-Contact Execution Monitoring with Action-Conditioned Latent World Models
- 2610.34593-Future Prediction as Training-Only Video Supervision for Pretrained VLA Policies
- 2610.35385-EmbodiedLab: Beyond Manipulation, Benchmarking Embodied Agents That Act to Know
- 2610.36378-Data-Driven Hamiltonian for Neural Optimal Control
- 2610.37275-A Heuristic for Scheduling Autonomous Vehicles at Intersections
- 2610.37288-Dynamic Coordination of Autonomous Vehicles at Intersections
- 2610.37632-Real-Time Coordination of Autonomous Vehicles at Intersections
- 2611.13779-Softness is something you also need! Why compliance helps control: information metrics of a passive walker with compliant ankle
- 2611.13411-Residual-Corrected Physics-Informed Neural Network KKL Observers with Input-to-State Stability Guarantees
- 2611.14488-Is Your Robot's Memory Just a Clock? Temporal Shortcuts in Memory-Augmented VLA Policies