Towards Reliable Local Security Agents: Verifiable Post-Training for Linux Privilege Escalation

arXiv:2603.17673 · cs.CR, cs.AI · Submitted 2026-03-18 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: We're looking at the paper: "Towards Reliable Local Security Agents".

Paper summary: Elias: To understand what they did, we look at the two stages they propose. First, supervised fine-tuning on traces collected from procedural privilege escalation environments to give the model grounded exploit knowledge.

Nadia: And then there's stage two, reinforcement learning with verifiable rewards trained in procedural environments to teach it how to deploy that knowledge efficiently under budget.

Priya: I wonder if those reward terms are what really make the difference, because they mentioned isolating two design choices: the supervised fine-tuning trace collection regime crossed with reasoning format and the reward terms layered on top of the verifiable outcome reward.

Elias: That's what they focused on in their analysis. They found that for supervised fine-tuning, unguided long reasoning traces were best because they are strongest on procedural holdout success, which means you get traces that reflect the deployment-time state distribution.

Nadia: But for reinforcement learning, they chose a reward function called outcome plus cost because it keeps things stable across the training set when looking at the holdout data. They specifically mentioned that price cost, not rounds, was what they found was important for RL.

Priya: So what does this actually mean for the kind of security agent we're trying to build? It implies that getting good performance isn't just about feeding it lots of raw data; it’s about carefully selecting the right training traces and setting up a reward structure that reflects real-world constraints.

Elias: Precisely. The paper demonstrates that this two-stage approach yields a 4B open-weight agent called PrivEscLLM 4B that achieves ninety-three point three percent success within twenty rounds on a static benchmark <ref:2603.17673#pg1>.

Nadia: And they compare it to something like Claude Opus at most budgets, trailing it at round twenty, but leading at round five with seventy eight point three percent success versus Claude's sixty six point seven percent <ref:2603.17673#pg2>.

Priya: That performance gap is significant when you think about the economics. They note that the expected inference cost per successful escalation drops by more than eighty times compared to those frontier models, and this post-training effect amortizes after roughly seven hundred successful escalations at round twenty against Claude Opus four point seven <ref:2603.17673#pg2>.

Elias: It positions PrivEscLLM 4B in that upper left corner of the cost reliability frontier, suggesting it’s competitive while running on a single consumer GPU, which is a big deal for local deployment <ref:2603.17673#pg2>.

Conclusion: Nadia: Looking at this paper, "Towards Reliable Local Security Agents: Verifiable Post-Training for Linux Privilege Escalation," the authors are essentially showing how to make small language models useful for security agents without relying on expensive cloud services.

Elias: The core message is that capable security agents need not depend on cloud API access. They achieved this by focusing on a two-stage post-training recipe, which they claim is the transferable contribution that actually carries the gains <ref:2603.17673#pg2>.

Priya: So, in simpler terms for the person listening who might just be driving or cooking, it means you can take a small open model and give it this specific training recipe to make it a reliable tool for testing security flaws locally.

Nadia: That’s the point. They used Linux privilege escalation as their testbed because the task is verifiable and demands multi-step reasoning, which really tests how well the agent actually uses tools <ref:2603.17673#pg2>.

Elias: The authors are proposing a method that turns a small model into an executable agent by using an agent harness that exposes tools and records full interaction traces <ref:2603.17673#pg3>.

Priya: The limitation they point out is that the evidence is limited to one setting and one base architecture, Qwen three four B, so they admit cross-family generality is still open. They also noted that Docker group escape shows a specific limit where failure analysis is more detailed than just failing to recognize the vulnerability <ref:2603.17673#pg3>.

Nadia: So, while it’s a solid recipe for this task, they are being careful about how far it applies outside of this specific Linux privilege escalation setting. It’s a strong piece of work showing how to approach security agents locally and reliably <ref:2603.17673#pg1>.

TU Wien

cs.CR, cs.AI

Submitted: 2026-03-18

Updated: 2026-11-17

Comments: Accepted at the 42nd Annual Computer Security Applications Conference (ACSAC 2026). Artifacts Available, Functional, and Reproduced

Code: https://github.com/sailab-vienna/privesc-llm

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 78/100

The gist: LLM agents are becoming increasingly important in the security domain, but leading systems are often closedsource, cloud-based…

Key concepts

Two-Stage Post-Training Recipe
This is a method involving two distinct steps: first, supervised fine-tuning (SFT) on expert traces to teach the model exploit knowledge; second, reinforcement learning with verifiable rewards (RLVR) to teach it how to deploy that knowledge efficiently under budget.
Agent Harness
This is the runtime layer that converts a language model into an executable agent. It exposes specific tools and records full interaction traces, allowing the model to interleave reasoning with tool actions during execution.
Verifiable Rewards (RLVR)
A reward function used in reinforcement learning that ensures training stability by basing rewards on verifiable outcomes. The authors found that using an OUTCOME+COST reward structure proved effective for teaching the model efficient deployment strategies.
Deployment-Time State Distribution
This refers to the distribution of states where the exploit is actually deployed during runtime. The authors found that SFT traces reflecting this state distribution are strongest for training because they teach the model how to succeed in real-world scenarios.

Terminology

Summary

LLM agents are becoming increasingly important in the security domain, but leading systems are often closedsource, cloud-based, hard to reproduce or use with sensitive code

The core contribution of this work is a two-stage post-training recipe that transforms a small local language model into a reliable security agent for Linux privilege escalation This method demonstrates that capable security agents need not depend on cloud API access The transferable contribution is the part of the pipeline that actually carries the gains.

System Architecture and Task Definition

The study focuses on Linux privilege escalation as a representative setting to systematically study local model training, because this task is both automatically verifiable and requires multi-step interactive reasoning The agent harness provides the runtime layer that turns a language model into an executable agent, exposing tools and recording full interaction traces Success in this setting is binary because it makes outcome verification clean The task requires a real tool-use loop where interleaving reasoning with actions and deciding when to call tools are part of the policy itself.

How the training signal is built

The proposed recipe involves two distinct stages: supervised fine-tuning (SFT) on filtered teacher traces, followed by reinforcement learning with verifiable rewards (RLVR) trained on procedural environments Stage 1 involves SFT on expert traces to give the model grounded exploit knowledge Stage 2 applies RLVR, teaching it to deploy that knowledge efficiently under budget.

Where the gains come from

The performance of the model is determined by isolating two design choices: the SFT trace-collection regime crossed with reasoning format, and the reward terms layered on top of the verifiable outcome reward. For SFT, unguided long-reasoning traces were selected because they are strongest on procedural holdout success Table 9> Finding (RQ1): SFT traces should reflect the deployment-time state distribution. p. 6 For RL, the reward function selected was OUTCOME+COST because it maintains stability on the holdout across training Finding (RQ2). p. 6 Price cost, not rounds. p. 7

Performance and Economic Comparison

The resulting model, PrivEsc-LLM 4B, reaches 93.3% success within 20 rounds on the held-out static benchmark This performance trails Claude Opus 4. p. 07 at most budgets but leads at r=5 with 78. p. 93% success The expected inference cost per successful escalation decreases by more than 80× compared to frontier models. p. 0 Post-training amortizes after roughly 700 successful escalations at r=20 against Claude Opus 4. p. 87 PrivEsc-LLM 4B sits in the upper-left of the cost–reliability frontier. p. 8

Limitations and Future Directions

The evidence is limited to one setting and one base architecture, Qwen3 4B, so cross-family generality remains open The procedural generators cover common misconfiguration families but not the long tail of real-world escalation paths. Docker group escape is the concrete instance where failure analysis shows a more specific limit than failure to recognize the vulnerability. p. 7 The Gemma 4 31B result on Docker scenario is noted as suggesting a capability gap in the 4B base model. p. 9

Conclusion

We presented a leakage-controlled post-training recipe for turning a small open-weight language model into a reliable tool-using security agent for Linux privilege escalation The recipe and protocol behind these numbers are the contribution. This approach shows that small local models can approach frontier-model reliability on interactive security tasks when training preserves the deployment-time tool interface and uses automatically verifiable outcomes. p. 9 The recipe is a natural candidate for other verifier-friendly security tasks. p. 9

Ethical Considerations

We mitigate dual-use risk by restricting scenarios to already documented Linux misconfigurations and known privilege-escalation paths The experiments do not interact with live third-party systems, collect personal data, or involve human subjects. Responsible disclosure to vendors is not applicable because the paper does not introduce new vulnerabilities or exploit primitives.

Acknowledgments

This work was supported by the Vienna Science and Technology Fund (WWTF) under project BREADS.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Post-training recipe for small local models can be applied to complex security tasks like Linux privilege escalation by using a two-stage process: supervised fine-tuning on traces from procedural environments followed by reinforcement learning with verifiable rewards. This turns a small local language model into a reliable security agent that is budget efficient and competitive with frontier models.

  2. The agent can perform multi-step interactive reasoning, such as enumerating attack surfaces and executing targeted exploits, under strict resource constraints by being trained on procedurally varied training environments. Success is measured by the budgeted reliability P(Hroot <= r), allowing for operation under a fixed round budget of inference calls rather than a best-of-k retry metric.

  3. The system can be designed with an agent harness that defines a clear action space using native tool calling, including exec command and test credentials, ensuring that the agent must interleave reasoning with actions to reach its goal. This structure ensures behavioral differences reflect the policy itself rather than interface inconsistencies between data collection and evaluation.

  4. The reinforcement learning phase can utilize a reward function such as OUTCOME+COST, which prices LLM and tool latency directly, allowing for cost-effective operation under a fixed budget of inference rounds. This enables the agent to reach high success rates while maintaining an expected inference cost that is significantly lower than frontier API systems.

  5. The resulting local agent can achieve reliability comparable to Claude Opus 4.7 on tight budgets, such as reaching 93.3% success within 20 rounds, and the total training and inference cost amortizes after roughly 700 successful escalations against API-served models.

Sources

Related papers