Caprese: Change Impact Recommendation Using History and Runtime Analysis

arXiv:2611.14937 · cs.SE · Submitted 2026-11-11 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: We're looking at the paper: "Caprese: Change Impact Recommendation Using History and Runtime Analysis".

Jane: Here's the gist of the paper. Change impact recommendation identifies program entities potentially affected by source code changes given a change set of modified program entities, change impact analysis identifies an impact set of potentially affected entities, while change impact recommendation prioritizes them for inspection, modification, or regression testing.

Tom: CAPRESE is an open-source framework for functionlevel change impact recommendation in Node.js given a change set of modified functions, CAPRESE produces a ranked impact set, termed the Candidate Inspection Set (CIS), through three modes: repository-based recommendation, using confidence-guided mining of recurring, change-set-relevant cochange patterns of varying lengths; dynamic recommendation, capturing dependencies involving calls, callbacks, events, timers, and shared state; and hybrid recommendation.

Jane: Framework Architecture CAPRESE is an automated framework for function-level change impact recommendation in JavaScript given a change set of modified functions, source code, tests, and repository history, CAPRESE produces a ranked Candidate Inspection Set (CIS) of functions potentially impacted by the changes.

Tom: It supports three modes: repository-based recommendation from historical co-change patterns, dynamic recommendation from execution-derived dependencies, and hybrid recommendation combining both sources of evidence.

Paper summary: Tom: So we’ve walked through 'Caprese: Change Impact Recommendation Using History and Runtime Analysis'. The paper introduces CAPRESE as an open-source framework for functionlevel change impact recommendation in Node JavaScript.

Jane: The authors are Sadjad Tavakoli, Saba Alimadadi, and Simon Fraser University researchers. They present three modes: repository-based recommendation using historical co-change patterns, dynamic recommendation from execution dependencies like calls and events, and a hybrid mode combining both.

Lu: The big point they make is that you can get a ranked candidate inspection set by combining these two sources of evidence. They show this works well across ten open-source applications and one hundred selected change sets.

Meng: It sounds like the implication is that developers could use this to proactively find risks in their code before deploying major updates, rather than just waiting for runtime errors.

Tom: That’s right, it’s about finding those potential breakage points earlier in the development cycle by ranking them by how likely they are to be impacted. It moves the recommendation process from a vague guess to a ranked set based on quantifiable evidence.

Jane: What this means for us is that we can use historical context and actual program behavior together to make smarter decisions about what needs testing or inspection when we modify our code.

Lalam: It’s about using both the past structure of the code and its current execution flow to guide our impact assessment, giving us a more comprehensive view than either method could offer alone.

Conclusion: Tom: So, we're wrapping up 'Caprese: Change Impact Recommendation Using History and Runtime Analysis'. The authors are Sadjad Tavakoli, Saba Alimadadi, and Simon Fraser University researchers.

Jane: They built this framework to rank functions that might get broken when you change your code in Node JavaScript by using two different types of clues.

Tom: Exactly. It mixes looking at what developers have changed in the past with actually watching how the program runs right now.

Lu: What they did is create a ranked list, called a Candidate Inspection Set, which tells you which functions you should look at first for testing or changing them.

Meng: So instead of guessing or just running everything on every change, you get a prioritized list of what matters most. That sounds like it could save some serious debugging time when you're shipping updates.

Lalam: It’s about using both the history—like seeing how two functions always change together—and the runtime data—like tracking which calls happen and which shared data is touched—to make that ranking as accurate as possible.

Tom: So, it’s less about finding one perfect clue and more about combining these different signals to get a better picture of the impact.

Jane: Right. It shows that historical patterns and real-time behavior actually complement each other really well when you're trying to predict where your code might fail.

Lu: The evaluation showed that most of the functions they ranked highly were only found by one method, but when they combined them, you got a much more complete set of candidates.

Meng: But what about the cost? How much time does this take to run? I need to know if it’s something I can actually put into my CI/CD pipeline without slowing everything down too much.

Lalam: The dynamic analysis part, which tracks calls and state changes, takes about ninety-five point four seven seconds on average, but they say that time can be reused across different change sets.

Tom: That’s a concrete number. So we’re looking at a tool that gives you a ranked list based on both past habits and current execution flow, with measurable run times.

Jane: And it suggests that for developers building complex Node applications, this could be the first step in making sure they know exactly what to test when they make a modification.

Simon Fraser University · BC, Canada

cs.SE

Submitted: 2026-11-11

Updated: 2026-11-13

Code: https://github.com/SEatSFU/caprese

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 69/100

The gist: Change impact recommendation identifies program entities potentially affected by source code changes given a change set of modified program entities…

Key concepts

Repository-based recommendation
This mode analyzes functions that frequently change together across different versions of the code. It looks for recurring patterns in how these functions are modified and maps those patterns back to the specific functions involved, suggesting they should be inspected if a change set is applied.
Dynamic recommendation
This method instruments an application during testing to track real-time dependencies. It captures how functions interact at runtime through calls, callbacks, events, timers, and shared data access. This helps identify dependencies that might not be obvious from just looking at the code history.
Hybrid Recommendation Strategy
This strategy combines results from both historical and dynamic analyses. Candidates found by both methods are ranked based on their runtime dependency distance first. Unique candidates from each analysis are then interleaved, creating a comprehensive, prioritized list of potential impact areas.
Candidate Inspection Set (CIS)
The CIS is the final output of CAPRESE: a ranked set of functions that are potentially impacted by the proposed code changes. It serves as a prioritized list for developers to decide which functions need manual inspection, modification, or regression testing.

Terminology

Summary

Change impact recommendation identifies program entities potentially affected by source code changes given a change set of modified program entities, change impact analysis identifies an impact set of potentially affected entities, while change impact recommendation prioritizes them for inspection, modification, or regression testing. p. 0

Where the paper says it

INTRODUCTION Understanding the downstream effects of source code changes is essential to software maintenance. Given a change set of modified program entities, change impact analysis identifies an impact set of potentially affected entities, while change impact recommendation prioritizes them for inspection, modification, or regression testing [1]–[4].

CAPRESE is an open-source framework for functionlevel change impact recommendation in Node. p. 0
Where the paper says it

Historical co-changes and runtime dependencies provide complementary evidence, particularly in JavaScript, where dynamic and asynchronous behaviour complicates dependency analysis. We present CAPRESE, an open-source framework for functionlevel change impact recommendation in Node.

js given a change set of modified functions, CAPRESE produces a ranked impact set, termed the Candidate Inspection Set (CIS), through three modes: repository-based recommendation, using confidence-guided mining of recurring, change-set-relevant cochange patterns of varying lengths; dynamic recommendation, capturing dependencies involving calls, callbacks, events, timers, and shared state; and hybrid recommendation. p. 0
Where the paper says it

js applications. Given a change set of modified functions, CAPRESE produces a ranked impact set, termed the Candidate Inspection Set (CIS), through three modes: repository-based recommendation, using confidence-guided mining of recurring, change-set-relevant co-change patterns of varying lengths; dynamic recommendation, capturing dependencies involving calls, callbacks, events, timers, and shared state; and hybrid recommendation, combining both.

Framework Architecture

CAPRESE is an automated framework for function-level change impact recommendation in JavaScript given a change set of modified functions, source code, tests, and repository history, CAPRESE produces a ranked Candidate Inspection Set (CIS) of functions potentially impacted by the changes. It supports three modes: repository-based recommendation from historical co-change patterns, dynamic recommendation from execution-derived dependencies, and hybrid recommendation combining both sources of evidence. p. 0

Where the paper says it

Given a change set of modified functions, CAPRESE produces a ranked impact set, termed the Candidate Inspection Set (CIS), through three modes: repository-based recommendation, using confidence-guided mining of recurring, change-set-relevant co-change patterns of varying lengths; dynamic recommendation, capturing dependencies involving calls, callbacks, events, timers, and shared state; and hybrid recommendation, combining both. Its modular implementation supports reuse of historical and runtime analyses across change sets.

Repository Analysis Details

The repository-based mode analyzes evolutionary coupling functions that repeatedly change together may require joint inspection. CAPRESE independently mines recurring functionlevel co-change patterns associated with the input change set. p. 1

Where the paper says it

Repository-Based Recommendation The repository-based mode analyzes evolutionary coupling: functions that repeatedly change together may require joint inspection. Unlike transaction-overlap approaches such as TARMAQ [6], CAPRESE independently mines recurring functionlevel co-change patterns associated with the input change set.

It uses RefDiff’s code structure trees (CSTs) to identify changes between consecutive revisions and extends this analysis to map the detected changes to their enclosing functions. p. 1
Where the paper says it

CAPRESE represents each commit as a transaction of modified functions. It uses RefDiff’s [9] code structure trees (CSTs) to identify changes between consecutive revisions and extends this analysis to map the detected changes to their enclosing functions.

For a pattern P, support(P) is the number of transactions containing all its functions. p. 1
Where the paper says it

Unlike pairwise co-change analysis, this search discovers patterns of varying lengths that overlap with subsets of the change set, without enumerating all patterns across the repository. For a pattern P, support(P) is the number of transactions containing all its functions.

Confidence measures how frequently the complete pattern occurs relative to its change-set functions. p. 1
Where the paper says it

For a pattern P, support(P) is the number of transactions containing all its functions. Confidence measures how frequently the complete pattern occurs relative to its change-set functions: confidence(P) = support(P) support(Pc) , (1) where Pc contains the functions of P belonging to the change set.

Dynamic Analysis Mechanism

The dynamic mode instruments the application using NodeProf2 and executes its tests to collect runtime traces. These traces are processed into a directed dependency graph whose vertices represent functions, with special handling for file-level execution and classes. p. 1

Where the paper says it

Dynamic Dependency Recommendation The dynamic mode instruments the application using NodeProf2 and executes its tests to collect runtime traces. These traces are processed into a directed dependency graph whose vertices represent functions, with special handling for file-level execution and classes.

CAPRESE infers dependencies from function calls and callbacks, event registration and emission, timers, and reads and writes to shared state. p. 1
Where the paper says it

Dependency extraction. CAPRESE infers dependencies from function calls and callbacks, event registration and emission, timers, and reads and writes to shared state.

Edges represent possible directions of change propagation rather than merely call directions. p. 1
Where the paper says it

CAPRESE infers dependencies from function calls and callbacks, event registration and emission, timers, and reads and writes to shared state. Edges represent possible directions of change propagation rather than merely call directions.

Hybrid Recommendation Strategy

The hybrid mode combines candidates from both analyses, prioritizing functions identified by both. Shared candidates are ranked by increasing runtime dependency distance, breaking ties by decreasing historical support and then confidence. p. 1

Where the paper says it

Hybrid Recommendation The hybrid mode combines candidates from both analyses, prioritizing functions identified by both. Shared candidates are ranked by increasing runtime dependency distance, breaking ties by decreasing historical support and then confidence.

Dynamic-only candidates are ranked by dependency distance, while history-only candidates are ranked by support and then confidence. p. 1
Where the paper says it

Shared candidates are ranked by increasing runtime dependency distance, breaking ties by decreasing historical support and then confidence. Dynamic-only candidates are ranked by dependency distance, while history-only candidates are ranked by support and then confidence.

The two exclusive lists are interleaved after the shared candidates, preserving recommendations unique to either analysis. p. 1
Where the paper says it

Dynamic-only candidates are ranked by dependency distance, while history-only candidates are ranked by support and then confidence. The two exclusive lists are interleaved after the shared candidates, preserving recommendations unique to either analysis.

Evaluation Findings

On average across projects, 77.4% of the relevant functions among the top 60 hybrid recommendations are identified exclusively by one analysis, demonstrating substantial complementarity. p. 0

Where the paper says it

Evaluation on ten open-source applications and 100 selected change sets shows that, on average across projects, 77. 4% of the relevant functions among the top 60 hybrid recommendations are identified exclusively by one analysis, demonstrating substantial complementarity.

Dynamic recommendation achieves the highest mean precision at Top-5 and Top-20 (0. p. 0
Where the paper says it

4% of the relevant functions among the top 60 hybrid recommendations are identified exclusively by one analysis, demonstrating substantial complementarity. Dynamic recommendation achieves the highest mean precision among the top five recommendations (0.

65 and 0.47, respectively). Repository-based recommendation performs best when the ranking is extended to 60 recommendations (0. p. 0
Where the paper says it

Dynamic recommendation achieves the highest mean precision among the top five recommendations (0. 65), while repository-based recommendation performs best when the ranking is extended to 60 recommendations (0.

42).

Practical Insights

Manual examination identified relevant coupling captured through history but missed by runtime analysis, including duplicated or semantically similar code, dependencies mediated by external resources, shared identifiers, and unexecuted code. Conversely, historical analysis may introduce false positives through incidental co-changes, obsolete relationships, or bidirectional coupling that obscures asymmetric dependencies. p. 3

Where the paper says it

Practical Insights and Limitations Manual examination identified relevant coupling captured through history but missed by runtime analysis, including duplicated or semantically similar code, dependencies mediated by external resources, shared identifiers, and unexecuted code. Conversely, historical analysis may introduce false positives through incidental co-changes, obsolete relationships, or bidirectional coupling that obscures asymmetric dependencies.

Execution cost for CAPRESE generated ranked recommendations in 3. p. 3
Where the paper says it

Execution cost. On an Apple M1 system, CAPRESE generated ranked recommendations in 3.

09 seconds on average, excluding dynamic dependency extraction. p. 3
Where the paper says it

On an Apple M1 system, CAPRESE generated ranked recommendations in 3. 09 seconds on average, excluding dynamic dependency extraction, compared with 2.

Dynamic dependency extraction required 95. p. 3
Where the paper says it

66 seconds for TARMAQ. Dynamic dependency extraction, including test execution, required 95.

47 seconds on average and can be reused across change sets. p. 3
Where the paper says it

Dynamic dependency extraction, including test execution, required 95. 47 seconds on average and can be reused across change sets.

Conclusion

We presented CAPRESE, an open-source framework integrating function-level co-change pattern mining and runtime dependency analysis for Node.js change impact recommendation. p. 0

Where the paper says it

Caprese: Change Impact Recommendation Using History and Runtime Analysis Sadjad Tavakoli Magellan AI BC, Canada sadjad tavakoli@sfu. ca Saba Alimadadi Simon Fraser University BC, Canada saba@sfu.

Evaluation on ten applications demonstrates substantial complementarity between historical and runtime evidence, although hybrid ranking does not consistently outperform the individual modes. p. 3
Where the paper says it

js change impact recommendation. Evaluation on ten applications demonstrates substantial complementarity between historical and runtime evidence, although hybrid ranking does not consistently outperform the individual modes.

The released tool and evaluation artifacts support further research on impact recommendation, including improved hybrid ranking, broader runtime dependency coverage, and additional JavaScript ecosystems. p. 3
Where the paper says it

Evaluation on ten applications demonstrates substantial complementarity between historical and runtime evidence, although hybrid ranking does not consistently outperform the individual modes. The released tool and evaluation artifacts support further research on impact recommendation, including improved hybrid ranking, broader runtime dependency coverage, and additional JavaScript ecosystems.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Improved recommendation capabilities through hybrid ranking by combining historical co-change patterns and execution-derived dependencies to identify potentially affected functions.

  2. Enhanced ability to capture indirect interactions beyond conventional call graphs by inferring dependencies from function calls, callbacks, events, timers, and shared state during dynamic analysis.

  3. More precise identification of relevant functions by leveraging the confidence-guided mining of recurring, change-set-relevant cochange patterns of varying lengths in the repository-based mode.

  4. Better prioritization of inspection candidates by ranking them based on their shortest dependency distance from the change set in the dynamic recommendation mode, capturing proximity effects.

Abstract

Change impact recommendation identifies program entities potentially affected by source code changes. Historical co-changes and runtime dependencies provide complementary evidence, particularly in JavaScript, where dynamic and asynchronous behaviour complicates dependency analysis. We present Caprese, an open-source framework for function-level change impact recommendation in Node.js. Given a change set, it produces a ranked impact set of potentially affected functions. It supports repository-based recommendation through confidence-guided mining of recurring, change-set-relevant co-change patterns of varying lengths; dynamic recommendation through execution-derived dependencies involving calls, callbacks, events, timers, and shared state; and hybrid recommendation integrating both analyses. Its modular implementation supports reusable analyses. Evaluation on ten open-source applications and 100 selected change sets shows that, on average across projects, 77.4% of the relevant functions among the top 60 hybrid recommendations are identified exclusively by one analysis, demonstrating substantial complementarity. Dynamic recommendation achieves the highest mean precision among the top five recommendations (0.65), while repository-based recommendation performs best when the ranking is extended to 60 recommendations (0.42). Hybrid recommendation outperforms TARMAQ on average but does not consistently outperform the individual modes. Caprese is publicly available with source code, documentation, and evaluation artifacts.

Related papers