Cognitive Tours: Making Understanding a Deliverable in AI-Assisted Development

summary

The gist

Generative AI and agentic development are accelerating software development, leading to concerns about technical debt Cognitive debt is the absence or erosion of a team’s shared understanding where…

In short

Cognitive Tours is a tool designed to combat technical and intent debt in AI-driven development by creating curated narratives of system features. It uses 'waypoint' links, human commentary, and AI auditing skills to help developers document the reasons behind decisions, implementation choices, and uncertainties.

Key concepts

Cognitive Debt
This is the erosion of a team's shared understanding where no one can confidently explain how a system works or predict the impact of making changes. It means code might be technically sound but difficult for maintainers to comprehend.
Intent Debt
This refers to the lack of clear goals, constraints, and rationale explaining what a system is intended to do and how it should evolve. A working implementation might fail if the original requirements were never clearly articulated or documented.
Waypoint
A waypoint is an important anchor point or landmark for understanding, such as a system goal or an unresolved question. Each waypoint has a description and can link to project artifacts like source files, serving as a building block in the overall tour.
Tour of Waypoints
A tour is a shared artifact that records the narrative of how waypoints are connected. It includes author information and last change dates, allowing team members to follow the story and question decisions through curated paths.

Terminology used across episodes

This episode discusses

The paper

Cognitive Tours: Making Understanding a Deliverable in AI-Assisted Development · Read on arXiv

Margaret-Anne Storey

University of Victoria, Canada

Generative AI and agentic coding are accelerating software development, leading to concerns about technical debt. But two less visible forms of debt, cognitive debt and intent debt, are emerging as key risks to software health. Cognitive debt is the absence or erosion of a team's shared understanding where no one can confidently explain how a system works or predict the impact of a change. Intent debt is the lack of clearly articulated goals, constraints, and rationale that explain what the system is for and guide how it should evolve. This paper presents Cognitive Tours, a tool designed to mitigate and repair cognitive and intent debt. Cognitive Tours is a web-based prototype for creating and following curated paths connecting user needs, design decisions, code, tests, and open questions. AI-assisted skills help audit existing waypoints and tours for possible problems or staleness and suggest candidates for new ones; humans decide what matters, verify relationships, and construct the narrative. This division of work deliberately preserves productive friction: the sensemaking involved in selecting, explaining, and revising a tour, which is intended to foster and share understanding. Our demonstration uses a tour of Cognitive Tours itself to introduce the tool and illustrate its design decisions and interaction mechanisms. A proposed evaluation agenda concludes the paper. A demo is available at https://cognitive-tours-demo.vercel.app/

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Cognitive Tours: Making Understanding a Deliverable in AI-Assisted Development".

Jane: Here's the gist of the paper. Generative AI and agentic development are accelerating software development, leading to concerns about technical debt Cognitive debt is the absence or erosion of a team’s shared understanding where no one can confidently explain how a system works or predict the impact of a change Intent debt is the absence of clearly articulated goals, constraints, and rationale that explain what the system is for and guide how it should evolve This paper presents Cognitive Tours, a tool designed to mitigate and repair cognitive and intent debt Cognitive Debt and Intent Debt Cognitive debt is defined as the absence or erosion of a team’s shared understanding where no one can confidently explain how a system works or predict the impact of a change Intent debt is described as the loss or absence of explicit goals, constraints, and rationale that explain what the system is for and guide how it should evolve A program may be technically sound and of high quality, but be difficult for its maintainers to explain or know how it can be safely evolved Furthermore, a working and plausible implementation may not satisfy a requirement that was never adequately articulated or externalized Cognitive Tours provides a mechanism for developers relying on AI and/or agents to hand craft curated journeys through the reasons, decisions, implementation, evidence, and uncertainties that help tell the story behind a feature or task Cognitive Tours Mechanism Cognitive Tours relies on three core concepts: a waypoint, a relationship or link between two waypoints, and a tour of waypoints A waypoint is defined as a selected anchor or landmark that the developer considers important for understanding, such as a system goal or an unresolved question Waypoints have both a description and optional commentary, and they can link to existing project artifacts like specifications or source files Named relationships explain why nodes are connected, for example, a feature addresses a user need Tours are shared artifacts that record the author and when it was last changed, allowing team members to follow and question the narrative Docent annotations provide human-curated commentary between waypoints, explaining what matters and how waypoints connect AI-Supported Auditing and Human Sensemaking Cognitive Tours uses AI-assisted skills to help maintain the waypoint graph and its curated routes by auditing individual waypoints and entire tours Four Claude Code skills accompany the tool, including auditwaypoints, which finds waypoint descriptions that no longer match their files The two audit skills report findings one at a time, such as a waypoint description that no longer matches its linked file The two suggest skills rank recent work worth recording and supply only the fact fields, followed by questions for the author The author decides which findings to act on and writes the descriptions and narratives, deliberately retaining manual curation work Evaluation and Future Work The paper plans to compare human authored, AI-generated and partially AI-generated/audited code explanations and tours in terms of individual and team understanding Assessments will consider orientation, behavioral prediction, explanation quality, knowledge acquisition and knowledge transfer The authors also plan to assess how cognitive tours can foster a culture of understanding across a team through curation and navigation Finally, they will assess how writing a tour enhances the author’s own understanding, not only that of its readers The authors acknowledge that tours and links may become stale, AI suggestions may be inaccurate, and curated paths may omit important steps The main aim behind Cognitive Tours is not to provide explanations of code but rather to author deliberate accounts of what matters for developers This approach introduces costs and risks, such as stale links or inaccurate AI suggestions, but the authors hypothesize that this productive friction supports the development of human understanding The tool has been used by the author during its development and provided cognitive support across sessions over six months This experience showed the value in slowing down to record decisions and choices that may matter in the future The authors plan to compare different types of code explanations and tours to one another in terms of understanding References

Paper summary: Tom: So, wrapping up "Cognitive Tours: Making Understanding a Deliverable in AI-Assisted Development," the paper is focused on creating these deliberate accounts of what matters for developers, rather than just writing explanations of code.

Jane: The authors are proposing this approach introduces risks like stale links or inaccurate AI suggestions but they hypothesize that this productive friction actually supports the development of human understanding over time.

Lu: This tool is designed to mitigate and repair cognitive and intent debt by providing curated paths connecting user needs, design decisions, code, tests, and open questions.

Meng: It’s a mechanism where you can rely on AI for scaffolding but the developer must be responsible for actually understanding the system through these tours.

Lalam: So it’s about authoring these deliberate accounts of what matters to build that shared context between people in the team.

Tom: The paper explores how we can use AI and agent skills to audit those waypoints and tours, but the human role remains central to deciding what needs recording and how those pieces connect.

Jane: It’s a framework for making understanding something you actively build, not something you passively hope for when tools like generative AI are accelerating development.

Lu: They are comparing different types of code explanations to see which ones lead to better individual and team understanding in terms of orientation and prediction.

Meng: The implication here is that if we want software health to stay good as AI gets more involved, we need a way to actively manage the shared knowledge base.

Lalam: It shifts the focus from just having a working implementation to making sure that implementation makes sense within the larger context of what was decided.

Conclusion: Tom: So we're wrapping up this on Cognitive Tours, looking at what those authors really want us to take away from this paper about making understanding a deliverable in AI-assisted development.

Jane: Basically, they’re not just writing explanations of code anymore; they’re building these deliberate accounts of what truly matters for the developers building the system.

Tom: Right, it's about authoring these structured narratives instead of just hoping the code makes sense on its own. They call this Cognitive Tours because it tackles that technical debt, that gap in team understanding where nobody can confidently explain how a system works or what will happen if you change something.

Lu: The core idea is using waypoints—these anchors like a specific system goal or an unresolved question—and linking them with relationships and tours to build a story around the decisions, the implementation, and even the uncertainties.

Meng: But here's where it gets interesting for us in production; they use AI skills to audit those waypoints, finding things that don't match what’s actually in the files. It’s like an AI assistant checking if our documentation or our notes are still accurate against the actual code.

Jane: And that auditing process isn't just a dry check; it involves human curation—the author has to decide which findings are important and write the descriptions themselves, deliberately keeping that manual work in place.

Tom: So they’re betting that this friction, this deliberate slowing down to record decisions, is what actually builds the shared understanding across the team over time. It’s not just about getting an answer; it's about building a better way for the whole team to know what they know.

Lu: They’re planning to compare different kinds of code explanations—human-written versus AI-generated versus partially audited—to see which ones actually improve how well people understand things, like predicting behavior or transferring knowledge.

Jane: It suggests that if we want AI tools to help us build better software, we need a way to actively manage the context and narrative, not just rely on the tool spitting out facts.

Tom: It’s about shifting from assuming understanding comes automatically with good code quality to actually making understanding a part of the development process itself.

Jane: And that leads us to thinking about how we can structure these tours so they're useful for real-world teams facing this kind of complexity.

More episodes

← Home