Where LLM Agents Get Stuck Debugging Databases: Access More Than Reasoning
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "Where LLM Agents Get Stuck Debugging Databases".
Jane: Here's the gist of the paper. Large language model (LLM) coding agents are increasingly asked not only to write code but to debug running systems from live runtime data The agent’s wrong hypotheses were consistent with the evidence it could see; they were not fabrications We argue that the next step for production-debugging agents is a forensics layer that gives the agent the access it lacks inside existing security controls The core finding regarding agent limitations Access to production state was identified as the most pervasive bottleneck on an LLM coding agent’s diagnosis An access gap was present in all five concurrency-dependent incidents In one of them it was the only limit In the other four it combined with a reasoning error, and in each of those the error was corrected only through production evidence that the agent could not obtain directly The three incidents that did not depend on concurrency needed nothing beyond what the agent already had Analysis of incident patterns The study analyzed eight database incidents over a ten-day window, where six of them converged on the locking and transactional-consistency layer Table 2 summarizes the result The matrix splits cleanly along concurrency The three incidents that did not depend on concurrency needed nothing the agent did not already have Implications for future tooling and research The paper proposes a concrete agenda for tool builders, including a safe database forensics API, a lock-and-deadlock flight recorder, and a production replay harness The agent’s largest gap is that it cannot ask the production database anything beyond a read-only projection Standardizing such an API for agents is a concrete deliverable Limitations and future work The study is limited by being a retrospective field observation of a single agent on a single system Future work suggests turning the observation into a controlled comparison, crossing model capability with access The most direct next step is to turn our observation into a controlled comparison, crossing model capability with access The reasoning × access matrix makes the pattern explicit: every concurrency-dependent incident had an access gap, and no incident was limited by reasoning alone This finding changes where the next investment should go The implication runs against the assumption that the model is the ceiling The first bottleneck of today’s debugging agents is access, and it is the one that tooling can remove Page 5 --- Access More Than Reasoning The agent’s largest gap is that it cannot ask the production database anything beyond a read-only projection: “I do not have access to the database nor exec in prod” What it needs is a forensics API: a read-only surface exposing pg locks, pg stat activity, query plans, and row-level state under the safety guarantees that already protect production, but rich enough to answer “which statement holds this lock?” This is the runtime counterpart of what SWE-agent calls the agent–computer interface (Yang et al., 2024), and of Panda’s grounding and affordance components (Singh et al., 2024) Page 8 --- Access More Than Reasoning The matrix splits cleanly along concurrency The three incidents that did not depend on concurrency needed nothing the agent did not already have Page 9 --- Access More Than Reasoning The reasoning × access matrix makes the pattern explicit: every concurrency-dependent incident had an access gap, and no incident was limited by reasoning alone This finding changes where the next investment should go The implication runs against the assumption that the model is the ceiling Page 9 --- Access More Than Reasoning The agent’s diagnostic reasoning was largely sound: it formed plausible hypotheses, noticed its own contradictions, and retired wrong hypotheses through measurement Yet every incident whose root cause depended on runtime concurrency began with a false start The agent had the entire codebase, yet for these failures the code was not enough Page 9 --- Access More Than Reasoning The agent cannot run queries against the production database, so it must infer row-level state from logs: “I do not have access to the database nor exec in prod” Locks and deadlocks are transient by nature: by the time the agent looks, the lock is gone, and the deadlock detail arrives truncated “the DETAIL is STILL truncated...
Tom: at the SOURCE” A deadlock happens when two transactions each hold a lock the other needs The error’s DETAIL is the lock graph It names which transaction is waiting for which lock and which transaction holds it, and it is precisely what identifies the deadlock cycle Page 9 --- Access More Than Reasoning The information was not absent; it was out of reach Even obtaining access cost a deploy A full deadlock report existed in the Postgres server’s own log—which has log lock waits enabled and logs the complete deadlock report—but the agent had access only to the application log Page 9 --- Access More Than Reasoning The agent cannot run queries against the production database, so it must infer row-level state from logs It must guess, ship, and watch Each wrong hypothesis then costs a full deploy–observe cycle before it can be refuted The indexed Datadog logs that would settle a diagnosis sit behind a login the agent cannot pass “requires login” Page 9 --- Access More Than Reasoning A false start, as we use the term, is a hypothesis consistent with everything the agent had seen that turns out to be wrong A false start, as we use the term, is a hypothesis consistent with everything the agent had seen that turns out to be wrong A false start, as we use the term, is a hypothesis consistent with everything the agent had seen that turns out to be wrong Page 9 --- Access More Than Reasoning The two call for different remedies: hallucination calls for better grounding, and false starts call for better access This reading is consistent with evidence that current LLMs detect data races and deadlocks reliably when the whole program is in view Our agent could read the whole repository, but at about 406,000 lines it had to choose what to read Page 9 --- Access More Than Reasoning Within the incidents, our agent’s wrong hypotheses were not fabrications; they were explanations consistent with the evidence it could see Within the standard architecture of LLM-based agents (planning, memory, tools, action) (Wang et al., 2024a), all of these strands point to the tool and perception layer, at least as much as to the reasoning core The agent cannot run queries against the production database Page 9 --- Access More Than Reasoning The agent’s wrong hypotheses were not fabrications; they were explanations consistent with the evidence it could see Within the standard architecture of LLM-based agents (planning, memory, tools, action) (Wang et al., 2024a), all of these strands point to the tool and perception layer, at least as much as to the reasoning core The agent cannot run queries against the production database Page 9 --- Access More Than Reasoning The agent’s wrong hypotheses were not fabrications; they were explanations consistent with the evidence it could see Within the standard architecture of LLM-based agents (planning, memory, tools, action) (Wang et al.
Paper summary: Tom: So Jane, we're looking at this paper now called "Where LLM Agents Get Stuck Debugging Databases: Access More Than Reasoning." Basically, they’re testing how well these AI coding agents can fix problems in a live production database. The big idea is that the agent isn't failing because its reasoning is bad; it’s failing because it doesn't have the right access to what’s actually happening inside the system.
Jane: Exactly. The paper says that these agents are being asked not just to write code, but to debug systems using live runtime data, and they observed a specific issue with an agent called Claude Code diagnosing eight database incidents in a message dispatch engine. The core claim is that when the root cause depended on runtime concurrency—that’s when things happen in real time with multiple things happening at once—the agent announced a cause, tried to ship a fix, and then walked it back later.
Lu: What’s really interesting from the abstract is how they mapped each of those incidents to what the agent was missing. They found that access to production state was the most pervasive bottleneck for these agents, and this missing access included things like live lock views, untruncated deadlock reports, read-only execution privileges, and an explicit environment context.
Meng: So it sounds like the model itself isn't the problem; it’s what the model is allowed to see or touch. I mean, if we give a model all the code in a repository, it can reason through logic pretty well, but if it can't query the actual database state, how can it fix concurrency issues?
Lalam: Lalam here. The paper confirms that access to production state was the most pervasive bottleneck for these LLM coding agents (Where LLM Agents Get Stuck Debugging Databases: Access More Than Reasoning). They found an access gap present in all five incidents whose root cause depended on runtime concurrency, and that this missing access was the most pervasive bottleneck (reference <ref:2611.17304#pg1>).
Tom: And what they found is that this access gap showed up alone in one of the incidents, but in four of them, it combined with a reasoning error. That means sometimes the agent had a wrong idea, and then it needed that production evidence to correct itself—evidence it just couldn't get directly.
Paper summary: Jane: That’s a crucial detail. It suggests that when an agent is wrong, its reasoning isn't necessarily flawed; it’s often just based on incomplete information from the perspective of what the agent can see. The agent had the whole codebase, but for those failures, the code wasn't enough (reference <ref:2611.17304#pg9>).
Lu: It points toward a need for a forensics layer. The paper suggests that instead of just improving the reasoning core of the model, we need to give it access it lacks inside existing security controls. They propose things like a safe database forensics API, and even something like a lock-and-deadlock flight recorder (reference <ref:2611.17304#pg2>).
Meng: From an engineering standpoint, that’s what we need to build. Right now, if I want to debug a live issue at scale, I can't just ask the production database questions; I have to rely on logs or guess how the row state is changing. The paper highlights that the agent can't run queries against production because it doesn't have that direct access (reference <ref:2611.17304#pg9>).
Lalam: Lalam confirms that the agent’s largest gap is being unable to ask the production database anything beyond a read-only projection; it says, "I do not have access to the database nor exec in prod" (reference <ref:2611.17304#pg2>). What it needs is this forensics API that exposes things like pg locks and query plans under safety guarantees (reference <ref:2611.17304#pg5>).
Tom: So the research question they ask is really about whether the limiting factor for an LLM coding agent’s diagnosis of production database incidents is its reasoning or its access (reference <ref:2611.17304#pg2>). Their answer, based on this field observation, leans heavily toward access (reference <ref:2611.17304#pg9>).
Jane: And that finding shifts where we should invest our time and resources. It goes against the idea that the model itself is the ceiling for what these agents can do (reference <ref:2611.17304#pg9>). The implication is clear: we need to focus on providing better tools, like standardizing this forensics API for agents, because that’s a concrete deliverable (reference <ref:2611.17304#pg5>).
Lu: What the paper does well is making the pattern explicit using that reasoning times access matrix (reference <ref:2611.17304#pg9>). It shows that every concurrency-dependent incident had an access gap, and no incident was limited by reasoning alone (reference <ref:2611.17304#pg9>).
Meng: I agree with Lu on the pattern recognition part. My concern is the operational cost of building this access layer. The paper mentions that even obtaining that full deadlock report existed in the Postgres server’s own log, but because the agent only had access to the application logs, it couldn't see that detail (reference <ref:2611.17304#pg9>).
Paper summary: Tom: It shows how much information is out of reach even when it exists on the system. The agent has to guess, ship a fix, and watch—which costs a full deploy-and-observe cycle every time it guesses wrong (reference <ref:2611.17304#pg9>). That’s not scalable for production debugging.
Jane: So what does this mean for someone just listening to the show? It means that when we talk about AI fixing real system bugs, we have to stop assuming the model is smart enough to see everything in production automatically (reference <ref:2611.17304#pg9>). We have to acknowledge that the next big investment isn't just a bigger language model, but better tools that give the agent visibility into the live state of things.
Lu: The authors also point out their own limitation, which is important for us to keep in mind. They note that this study was a retrospective field observation of one agent on one system (reference <ref:2611.17304#pg3>). They suggest turning this observation into a controlled comparison, crossing model capability with access, as the most direct next step (reference <ref:2611.17304#pg3>).
Meng: That makes sense. We can’t just rely on one study to tell us everything about what an agent *can* do at scale. It has to be a controlled comparison that tests model capability alongside access capabilities (reference <ref:2611.17304#pg3>).
Lalam: Lalam is pretty sure of this direction too. The paper’s conclusion on reasoning versus access makes the pattern explicit, and that finding changes where the next investment should go (reference <ref:2611.17304#pg9>). The first bottleneck for today’s debugging agents is access, and it is the one that tooling can remove (reference <ref:2611.17304#pg9>).
Tom: So we're moving away from just chasing better reasoning models and toward building infrastructure that gives the agent real eyes inside the production environment. That’s a tangible engineering problem they’ve pointed out (reference <ref:2611.17304#pg5>). It moves debugging from pure guesswork to evidence-based diagnosis, even if the evidence comes from a specialized API (reference <ref:2611.17304#pg5>).
Jane: Right. We’re moving from hoping the model gets it right to making sure the model can see what it needs to get it right (reference <ref:2611.17304#pg9>). This paper, "Where LLM Agents Get Stuck Debugging Databases: Access More Than Reasoning," really hammers home that for AI agents working with live systems, the missing link isn't always the logic; sometimes it's just the right permission.
Conclusion: Tom: So, we're wrapping up on "Where LLM Agents Get Stuck Debugging Databases: Access More Than Reasoning." Basically, this paper shows that for AI coding agents trying to fix live database problems, the real blocker isn't just their brains; it’s what they are allowed to look at.
Jane: Right. The authors are pointing out that these agents have a huge gap because they can't ask the production database anything beyond just reading data. It says they don't have direct access to things like live locks or the full deadlock reports.
Lu: What this means is that when you have concurrency issues, which is when multiple things happen at once, the agent usually fails because it’s missing that real-time state information. The paper shows a pattern where every single concurrency incident had this access gap.
Meng: From my side, it's about deployment costs. If the agent can't just query the database state directly, it has to infer everything from application logs or guess row states. That means every wrong hypothesis requires a full deploy and observation cycle before you can even test something new.
Lalam: I agree with Meng on that cost issue. The research shows that while the AI’s reasoning might be sound, it gets stuck because the evidence needed to correct a false start isn't accessible to it within the existing security controls.
Tom: So, in simple terms, this paper is saying we need to build a forensics layer for these agents. It suggests that giving an agent access to things like pg locks and query plans is more important than just making its language model smarter about logic.
Jane: Exactly. The implication for the world is that the next big investment isn't necessarily in bigger models, but in better tools—specifically, standardized APIs that give agents the right kind of visibility into production systems.
Lu: The authors themselves admit they did this as a field observation on one system, so their next step is to run controlled comparisons between different models and different access levels. They aren't claiming this is a final answer yet.
Meng: I like that honesty in the conclusion. It keeps it grounded in reality for what we can actually build right now instead of making big claims about what future models will do automatically.
Lalam: And so, the core finding remains: reasoning times access is a matrix where every concurrency failure hits an access gap, and that’s where we need to focus our engineering efforts.
Gustavo Pinto
Federal University of Parı (UFPA)
cs.SE
Submitted: 2026-11-12
Updated: 2026-11-12
Comments: 12 pages, 2 figures, 2 tables. Submitted to the Journal of Systems and Software
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 89/100
The gist: Large language model (LLM) coding agents are increasingly asked not only to write code but to debug running systems from live runtime data The agent’s wrong hypotheses were consistent with the…
Key concepts
- Access Gap
- The agent cannot query the production database directly; it can only see what's exposed through existing security controls. This lack of direct access prevents it from verifying hypotheses about complex runtime issues like locking and transactional consistency.
- Forensics API
- A proposed tool that gives the agent read-only access to critical database information, such as lock states and query plans. This would allow the agent to answer specific questions about why a transaction is blocked or what locks are held, moving beyond just reading logs.
- False Start
- A hypothesis generated by the agent that seems plausible based on available code but turns out to be incorrect when tested against runtime evidence. These false starts often stem from the inability to inspect real-time database state, leading to costly debugging cycles.
- Reasoning vs. Access Bottleneck
- The study proves that for concurrency-dependent incidents, the limitation is access, not reasoning. The agent's ability to form plausible hypotheses is sound; it fails when it cannot obtain the necessary runtime data to confirm or deny those hypotheses.
Terminology
Summary
Large language model (LLM) coding agents are increasingly asked not only to write code but to debug running systems from live runtime data The agent’s wrong hypotheses were consistent with the evidence it could see; they were not fabrications We argue that the next step for production-debugging agents is a forensics layer that gives the agent the access it lacks inside existing security controls
The core finding regarding agent limitations
Access to production state was identified as the most pervasive bottleneck on an LLM coding agent’s diagnosis An access gap was present in all five concurrency-dependent incidents In one of them it was the only limit In the other four it combined with a reasoning error, and in each of those the error was corrected only through production evidence that the agent could not obtain directly The three incidents that did not depend on concurrency needed nothing beyond what the agent already had
Analysis of incident patterns
The study analyzed eight database incidents over a ten-day window, where six of them converged on the locking and transactional-consistency layer Table 2 summarizes the result The matrix splits cleanly along concurrency The three incidents that did not depend on concurrency needed nothing the agent did not already have
Implications for future tooling and research
The paper proposes a concrete agenda for tool builders, including a safe database forensics API, a lock-and-deadlock flight recorder, and a production replay harness The agent’s largest gap is that it cannot ask the production database anything beyond a read-only projection Standardizing such an API for agents is a concrete deliverable
Limitations and future work
The study is limited by being a retrospective field observation of a single agent on a single system Future work suggests turning the observation into a controlled comparison, crossing model capability with access The most direct next step is to turn our observation into a controlled comparison, crossing model capability with access
Conclusion on reasoning versus access
The reasoning × access matrix makes the pattern explicit: every concurrency-dependent incident had an access gap, and no incident was limited by reasoning alone This finding changes where the next investment should go The implication runs against the assumption that the model is the ceiling The first bottleneck of today’s debugging agents is access, and it is the one that tooling can remove
--- Page 5 ---
Access More Than Reasoning
The agent’s largest gap is that it cannot ask the production database anything beyond a read-only projection: “I do not have access to the database nor exec in prod” What it needs is a forensics API: a read-only surface exposing pg locks, pg stat activity, query plans, and row-level state under the safety guarantees that already protect production, but rich enough to answer “which statement holds this lock?” This is the runtime counterpart of what SWE-agent calls the agent–computer interface (Yang et al. p. 7 ” What it needs
is a forensics API: a read-only surface exposing pg_locks,
pg_stat_activity, query plans, and row-level state under
the safety guarantees that already protect production, but
rich enough to answer “which statement holds this lock? ”
This is the runtime counterpart of what SWE-agent calls
the agent–computer interface (Yang et al. ”
This is the runtime counterpart of what SWE-agent calls
the agent–computer interface (Yang et al. , 2024), and of
Panda’s grounding and affordance components (Singh et al.Where the paper says it
Where the paper says it
--- Page 8 ---
Access More Than Reasoning
The matrix splits cleanly along concurrency The three incidents that did not depend on concurrency needed nothing the agent did not already have
--- Page 9 ---
Access More Than Reasoning
The reasoning × access matrix makes the pattern explicit: every concurrency-dependent incident had an access gap, and no incident was limited by reasoning alone This finding changes where the next investment should go The implication runs against the assumption that the model is the ceiling
--- Page 9 ---
Access More Than Reasoning
The agent’s diagnostic reasoning was largely sound: it formed plausible hypotheses, noticed its own contradictions, and retired wrong hypotheses through measurement Yet every incident whose root cause depended on runtime concurrency began with a false start The agent had the entire codebase, yet for these failures the code was not enough
--- Page 9 ---
Access More Than Reasoning
The agent cannot run queries against the production database, so it must infer row-level state from logs: “I do not have access to the database nor exec in prod” Locks and deadlocks are transient by nature: by the time the agent looks, the lock is gone, and the deadlock detail arrives truncated “the DETAIL is STILL truncated... at the SOURCE” A deadlock happens when two transactions each hold a lock the other needs The error’s DETAIL is the lock graph It names which transaction is waiting for which lock and which transaction holds it, and it is precisely what identifies the deadlock cycle
--- Page 9 ---
Access More Than Reasoning
The information was not absent; it was out of reach Even obtaining access cost a deploy A full deadlock report existed in the Postgres server’s own log—which has log lock waits enabled and logs the complete deadlock report—but the agent had access only to the application log
--- Page 9 ---
Access More Than Reasoning
The agent cannot run queries against the production database, so it must infer row-level state from logs It must guess, ship, and watch Each wrong hypothesis then costs a full deploy–observe cycle before it can be refuted The indexed Datadog logs that would settle a diagnosis sit behind a login the agent cannot pass “requires login”
--- Page 9 ---
Access More Than Reasoning
A false start, as we use the term, is a hypothesis consistent with everything the agent had seen that turns out to be wrong A false start, as we use the term, is a hypothesis consistent with everything the agent had seen that turns out to be wrong A false start, as we use the term, is a hypothesis consistent with everything the agent had seen that turns out to be wrong
--- Page 9 ---
Access More Than Reasoning
The two call for different remedies: hallucination calls for better grounding, and false starts call for better access This reading is consistent with evidence that current LLMs detect data races and deadlocks reliably when the whole program is in view Our agent could read the whole repository, but at about 406,000 lines it had to choose what to read
--- Page 9 ---
Access More Than Reasoning
Within the incidents, our agent’s wrong hypotheses were not fabrications; they were explanations consistent with the evidence it could see Within the standard architecture of LLM-based agents (planning, memory, tools, action) (Wang et al., 2024a), all of these strands point to the tool and perception layer, at least as much as to the reasoning core The agent cannot run queries against the production database
--- Page 9 ---
Access More Than Reasoning
The agent’s wrong hypotheses were not fabrications; they were explanations consistent with the evidence it could see Within the standard architecture of LLM-based agents (planning, memory, tools, action) (Wang et al., 2024a), all of these strands point to the tool and perception layer, at least as much as to the reasoning core The agent cannot run queries against the production database
--- Page 9 ---
Access More Than Reasoning
The agent’s wrong hypotheses were not fabrications; they were explanations consistent with the evidence it could see Within the standard architecture of LLM-based agents (planning, memory, tools, action) (Wang et al.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Implement a forensics layer that gives agents access to production state by combining a safe interrogation API, a lock-and-deadlock flight recorder, and a production replay harness. This restores the agent’s access without lowering existing security controls.
-
Develop an environment where agents are evaluated under partial observability rather than only on how hard a fault is to explain. This allows researchers to separate failures of reasoning from failures of access at scale.
-
Create a mechanism that forces agents to record each hypothesis and the evidence that retired it, transforming them into an auditable hypothesis ledger. This makes the false-start chain auditable and teaches the agent which classes of database failure to stop guessing about.
-
Build tooling that allows an agent to test a hypothesis against production before deploying, such as dry-run execution or EXPLAIN against a live replica. This collapses the guess-ship-observe loop from “ship and watch” to “ask and answer.”
-
Design an agent harness tuned for low-access debugging that treats a hypothesis as a firstclass object, persisted with its evidence across turns, preferring richer, safer read-only primitives over raw log search.
Abstract
Large language model (LLM) coding agents are increasingly asked not only to write code but to debug running systems from live runtime data. We report a field observation of one such agent (Claude Code) as it diagnosed eight database incidents in a production message-dispatch engine. In every incident whose root cause depended on runtime concurrency, the agent announced a cause, shipped a fix, and later walked it back, and each wrong guess cost a deploy-and-observe cycle. Mapping each incident to what the agent lacked, we find that access to production state was the most pervasive bottleneck. The missing access included live lock views, untruncated deadlock reports, read-only execution, and an explicit environment context. An access gap was present in all five concurrency-dependent incidents: alone in one, and in four combined with a reasoning error such as anchoring on an early cause, incomplete exploration of the configuration, or a misreading of the code's transaction semantics. The three incidents that did not depend on concurrency needed nothing beyond what the agent already had. Within the incidents, the agent's wrong hypotheses were consistent with the evidence it could see; they were not fabrications. We argue that the next step for production-debugging agents is a forensics layer that gives the agent the access it lacks inside existing security controls. Such a layer would combine a safe interrogation API, a lock-and-deadlock flight recorder, and a production replay harness.
Related papers
- ArkEval: Benchmarking and Evaluating Automated CodeRepair for ArkTS
- TestPrism: Rethinking Test Evaluation Beyond a Single Reference
- Mining Workflow Graphs for Black-Box Boundary Testing of Conversational LLM Agents
- Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
- GitSkills: A Dataset of Agent Skills on GitHub
- For Coding Agents, the Repository Matters