Caprese: Change Impact Recommendation Using History and Runtime Analysis
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.
Tom: We're looking at the paper: "Caprese: Change Impact Recommendation Using History and Runtime Analysis".
Jane: Here's the gist of the paper. Change impact recommendation identifies program entities potentially affected by source code changes given a change set of modified program entities, change impact analysis identifies an impact set of potentially affected entities, while change impact recommendation prioritizes them for inspection, modification, or regression testing.
Tom: CAPRESE is an open-source framework for functionlevel change impact recommendation in Node.js given a change set of modified functions, CAPRESE produces a ranked impact set, termed the Candidate Inspection Set (CIS), through three modes: repository-based recommendation, using confidence-guided mining of recurring, change-set-relevant cochange patterns of varying lengths; dynamic recommendation, capturing dependencies involving calls, callbacks, events, timers, and shared state; and hybrid recommendation.
Jane: Framework Architecture CAPRESE is an automated framework for function-level change impact recommendation in JavaScript given a change set of modified functions, source code, tests, and repository history, CAPRESE produces a ranked Candidate Inspection Set (CIS) of functions potentially impacted by the changes.
Tom: It supports three modes: repository-based recommendation from historical co-change patterns, dynamic recommendation from execution-derived dependencies, and hybrid recommendation combining both sources of evidence.
Paper summary: Tom: So we’ve walked through 'Caprese: Change Impact Recommendation Using History and Runtime Analysis'. The paper introduces CAPRESE as an open-source framework for functionlevel change impact recommendation in Node JavaScript.
Jane: The authors are Sadjad Tavakoli, Saba Alimadadi, and Simon Fraser University researchers. They present three modes: repository-based recommendation using historical co-change patterns, dynamic recommendation from execution dependencies like calls and events, and a hybrid mode combining both.
Lu: The big point they make is that you can get a ranked candidate inspection set by combining these two sources of evidence. They show this works well across ten open-source applications and one hundred selected change sets.
Meng: It sounds like the implication is that developers could use this to proactively find risks in their code before deploying major updates, rather than just waiting for runtime errors.
Tom: That’s right, it’s about finding those potential breakage points earlier in the development cycle by ranking them by how likely they are to be impacted. It moves the recommendation process from a vague guess to a ranked set based on quantifiable evidence.
Jane: What this means for us is that we can use historical context and actual program behavior together to make smarter decisions about what needs testing or inspection when we modify our code.
Lalam: It’s about using both the past structure of the code and its current execution flow to guide our impact assessment, giving us a more comprehensive view than either method could offer alone.
Conclusion: Tom: So, we're wrapping up 'Caprese: Change Impact Recommendation Using History and Runtime Analysis'. The authors are Sadjad Tavakoli, Saba Alimadadi, and Simon Fraser University researchers.
Jane: They built this framework to rank functions that might get broken when you change your code in Node JavaScript by using two different types of clues.
Tom: Exactly. It mixes looking at what developers have changed in the past with actually watching how the program runs right now.
Lu: What they did is create a ranked list, called a Candidate Inspection Set, which tells you which functions you should look at first for testing or changing them.
Meng: So instead of guessing or just running everything on every change, you get a prioritized list of what matters most. That sounds like it could save some serious debugging time when you're shipping updates.
Lalam: It’s about using both the history—like seeing how two functions always change together—and the runtime data—like tracking which calls happen and which shared data is touched—to make that ranking as accurate as possible.
Tom: So, it’s less about finding one perfect clue and more about combining these different signals to get a better picture of the impact.
Jane: Right. It shows that historical patterns and real-time behavior actually complement each other really well when you're trying to predict where your code might fail.
Lu: The evaluation showed that most of the functions they ranked highly were only found by one method, but when they combined them, you got a much more complete set of candidates.
Meng: But what about the cost? How much time does this take to run? I need to know if it’s something I can actually put into my CI/CD pipeline without slowing everything down too much.
Lalam: The dynamic analysis part, which tracks calls and state changes, takes about ninety-five point four seven seconds on average, but they say that time can be reused across different change sets.
Tom: That’s a concrete number. So we’re looking at a tool that gives you a ranked list based on both past habits and current execution flow, with measurable run times.
Jane: And it suggests that for developers building complex Node applications, this could be the first step in making sure they know exactly what to test when they make a modification.
Simon Fraser University · BC, Canada
cs.SE
Submitted: 2026-11-11
Updated: 2026-11-13
Code: https://github.com/SEatSFU/caprese
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 69/100
The gist: Change impact recommendation identifies program entities potentially affected by source code changes given a change set of modified program entities…
Key concepts
- Repository-based recommendation
- This mode analyzes functions that frequently change together across different versions of the code. It looks for recurring patterns in how these functions are modified and maps those patterns back to the specific functions involved, suggesting they should be inspected if a change set is applied.
- Dynamic recommendation
- This method instruments an application during testing to track real-time dependencies. It captures how functions interact at runtime through calls, callbacks, events, timers, and shared data access. This helps identify dependencies that might not be obvious from just looking at the code history.
- Hybrid Recommendation Strategy
- This strategy combines results from both historical and dynamic analyses. Candidates found by both methods are ranked based on their runtime dependency distance first. Unique candidates from each analysis are then interleaved, creating a comprehensive, prioritized list of potential impact areas.
- Candidate Inspection Set (CIS)
- The CIS is the final output of CAPRESE: a ranked set of functions that are potentially impacted by the proposed code changes. It serves as a prioritized list for developers to decide which functions need manual inspection, modification, or regression testing.
Terminology
Summary
Change impact recommendation identifies program entities potentially affected by source code changes given a change set of modified program entities, change impact analysis identifies an impact set of potentially affected entities, while change impact recommendation prioritizes them for inspection, modification, or regression testing. p. 0 INTRODUCTION
Understanding the downstream effects of source code
changes is essential to software maintenance. Given a change
set of modified program entities, change impact analysis
identifies an impact set of potentially affected entities, while
change impact recommendation prioritizes them for inspection, modification, or regression testing [1]–[4]. Historical
co-changes and runtime dependencies provide complementary
evidence, particularly in JavaScript, where dynamic and asynchronous behaviour complicates dependency analysis. We present CAPRESE, an open-source framework for functionlevel change impact recommendation in Node. js applications. Given a change set of modified functions, CAPRESE
produces a ranked impact set, termed the Candidate Inspection Set (CIS), through three modes: repository-based recommendation, using confidence-guided mining of recurring,
change-set-relevant co-change patterns of varying lengths;
dynamic recommendation, capturing dependencies involving
calls, callbacks, events, timers, and shared state; and hybrid
recommendation, combining both.Where the paper says it
Where the paper says it
Where the paper says it
Framework Architecture
CAPRESE is an automated framework for function-level change impact recommendation in JavaScript given a change set of modified functions, source code, tests, and repository history, CAPRESE produces a ranked Candidate Inspection Set (CIS) of functions potentially impacted by the changes. It supports three modes: repository-based recommendation from historical co-change patterns, dynamic recommendation from execution-derived dependencies, and hybrid recommendation combining both sources of evidence. p. 0 Given a change set of modified functions, CAPRESE
produces a ranked impact set, termed the Candidate Inspection Set (CIS), through three modes: repository-based recommendation, using confidence-guided mining of recurring,
change-set-relevant co-change patterns of varying lengths;
dynamic recommendation, capturing dependencies involving
calls, callbacks, events, timers, and shared state; and hybrid
recommendation, combining both. Its modular implementation
supports reuse of historical and runtime analyses across change
sets.Where the paper says it
Repository Analysis Details
The repository-based mode analyzes evolutionary coupling functions that repeatedly change together may require joint inspection. CAPRESE independently mines recurring functionlevel co-change patterns associated with the input change set. p. 1 Repository-Based Recommendation
The repository-based mode analyzes evolutionary coupling:
functions that repeatedly change together may require joint inspection. Unlike transaction-overlap approaches such as TARMAQ [6], CAPRESE independently mines recurring functionlevel co-change patterns associated with the input change set. CAPRESE represents each commit as a
transaction of modified functions. It uses RefDiff’s [9] code
structure trees (CSTs) to identify changes between consecutive
revisions and extends this analysis to map the detected changes
to their enclosing functions. Unlike pairwise co-change analysis, this search discovers patterns of varying lengths that overlap with subsets of the change
set, without enumerating all patterns across the repository. For a pattern P, support(P) is the number of transactions
containing all its functions. For a pattern P, support(P) is the number of transactions
containing all its functions. Confidence measures how frequently the complete pattern occurs relative to its change-set
functions:
confidence(P) = support(P)
support(Pc)
, (1)
where Pc contains the functions of P belonging to the change
set.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Dynamic Analysis Mechanism
The dynamic mode instruments the application using NodeProf2 and executes its tests to collect runtime traces. These traces are processed into a directed dependency graph whose vertices represent functions, with special handling for file-level execution and classes. p. 1 Dynamic Dependency Recommendation
The dynamic mode instruments the application using NodeProf2
and executes its tests to collect runtime traces. These
traces are processed into a directed dependency graph whose
vertices represent functions, with special handling for file-level
execution and classes. Dependency extraction. CAPRESE infers dependencies
from function calls and callbacks, event registration and
emission, timers, and reads and writes to shared state. CAPRESE infers dependencies
from function calls and callbacks, event registration and
emission, timers, and reads and writes to shared state. Edges
represent possible directions of change propagation rather than
merely call directions.Where the paper says it
Where the paper says it
Where the paper says it
Hybrid Recommendation Strategy
The hybrid mode combines candidates from both analyses, prioritizing functions identified by both. Shared candidates are ranked by increasing runtime dependency distance, breaking ties by decreasing historical support and then confidence. p. 1 Hybrid Recommendation
The hybrid mode combines candidates from both analyses,
prioritizing functions identified by both. Shared candidates are
ranked by increasing runtime dependency distance, breaking
ties by decreasing historical support and then confidence. Shared candidates are
ranked by increasing runtime dependency distance, breaking
ties by decreasing historical support and then confidence. Dynamic-only candidates are ranked by dependency distance,
while history-only candidates are ranked by support and then
confidence. Dynamic-only candidates are ranked by dependency distance,
while history-only candidates are ranked by support and then
confidence. The two exclusive lists are interleaved after the
shared candidates, preserving recommendations unique to either analysis.Where the paper says it
Where the paper says it
Where the paper says it
Evaluation Findings
On average across projects, 77.4% of the relevant functions among the top 60 hybrid recommendations are identified exclusively by one analysis, demonstrating substantial complementarity. p. 0 Evaluation on ten open-source applications and 100 selected
change sets shows that, on average across projects, 77. 4% of the
relevant functions among the top 60 hybrid recommendations are
identified exclusively by one analysis, demonstrating substantial
complementarity. 4% of the
relevant functions among the top 60 hybrid recommendations are
identified exclusively by one analysis, demonstrating substantial
complementarity. Dynamic recommendation achieves the highest
mean precision among the top five recommendations (0. Dynamic recommendation achieves the highest
mean precision among the top five recommendations (0. 65),
while repository-based recommendation performs best when
the ranking is extended to 60 recommendations (0.Where the paper says it
Where the paper says it
Where the paper says it
Practical Insights
Manual examination identified relevant coupling captured through history but missed by runtime analysis, including duplicated or semantically similar code, dependencies mediated by external resources, shared identifiers, and unexecuted code. Conversely, historical analysis may introduce false positives through incidental co-changes, obsolete relationships, or bidirectional coupling that obscures asymmetric dependencies. p. 3 Practical Insights and Limitations
Manual examination identified relevant coupling captured
through history but missed by runtime analysis, including duplicated or semantically similar code, dependencies mediated
by external resources, shared identifiers, and unexecuted code. Conversely, historical analysis may introduce false positives
through incidental co-changes, obsolete relationships, or bidirectional coupling that obscures asymmetric dependencies. Execution cost. On an Apple M1 system, CAPRESE generated ranked recommendations in 3. On an Apple M1 system, CAPRESE generated ranked recommendations in 3. 09 seconds on average,
excluding dynamic dependency extraction, compared with
2. 66 seconds for TARMAQ. Dynamic dependency extraction,
including test execution, required 95. Dynamic dependency extraction,
including test execution, required 95. 47 seconds on average
and can be reused across change sets.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Conclusion
We presented CAPRESE, an open-source framework integrating function-level co-change pattern mining and runtime dependency analysis for Node.js change impact recommendation. p. 0 Caprese: Change Impact Recommendation Using
History and Runtime Analysis
Sadjad Tavakoli
Magellan AI
BC, Canada
sadjad tavakoli@sfu. ca
Saba Alimadadi
Simon Fraser University
BC, Canada
saba@sfu. js change impact recommendation. Evaluation on ten applications demonstrates substantial
complementarity between historical and runtime evidence,
although hybrid ranking does not consistently outperform the
individual modes. Evaluation on ten applications demonstrates substantial
complementarity between historical and runtime evidence,
although hybrid ranking does not consistently outperform the
individual modes. The released tool and evaluation artifacts
support further research on impact recommendation, including
improved hybrid ranking, broader runtime dependency coverage, and additional JavaScript ecosystems.Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Improved recommendation capabilities through hybrid ranking by combining historical co-change patterns and execution-derived dependencies to identify potentially affected functions.
-
Enhanced ability to capture indirect interactions beyond conventional call graphs by inferring dependencies from function calls, callbacks, events, timers, and shared state during dynamic analysis.
-
More precise identification of relevant functions by leveraging the
confidence-guided mining of recurring, change-set-relevant cochange patterns of varying lengths
in the repository-based mode. -
Better prioritization of inspection candidates by ranking them based on their shortest dependency distance from the change set in the dynamic recommendation mode, capturing proximity effects.
Abstract
Change impact recommendation identifies program entities potentially affected by source code changes. Historical co-changes and runtime dependencies provide complementary evidence, particularly in JavaScript, where dynamic and asynchronous behaviour complicates dependency analysis. We present Caprese, an open-source framework for function-level change impact recommendation in Node.js. Given a change set, it produces a ranked impact set of potentially affected functions. It supports repository-based recommendation through confidence-guided mining of recurring, change-set-relevant co-change patterns of varying lengths; dynamic recommendation through execution-derived dependencies involving calls, callbacks, events, timers, and shared state; and hybrid recommendation integrating both analyses. Its modular implementation supports reusable analyses. Evaluation on ten open-source applications and 100 selected change sets shows that, on average across projects, 77.4% of the relevant functions among the top 60 hybrid recommendations are identified exclusively by one analysis, demonstrating substantial complementarity. Dynamic recommendation achieves the highest mean precision among the top five recommendations (0.65), while repository-based recommendation performs best when the ranking is extended to 60 recommendations (0.42). Hybrid recommendation outperforms TARMAQ on average but does not consistently outperform the individual modes. Caprese is publicly available with source code, documentation, and evaluation artifacts.
Related papers
- ArkEval: Benchmarking and Evaluating Automated CodeRepair for ArkTS
- TestPrism: Rethinking Test Evaluation Beyond a Single Reference
- Mining Workflow Graphs for Black-Box Boundary Testing of Conversational LLM Agents
- Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
- GitSkills: A Dataset of Agent Skills on GitHub
- For Coding Agents, the Repository Matters