How Quickly Do Development Teams Update Their Vulnerable Dependencies?

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: We're looking at the paper: "How Quickly Do Development Teams Update Their Vulnerable Dependencies?".

Elias: Here's the gist of the paper. The gist The study proposes two novel metrics, Mean-Time-To-Update for dependencies (MTTUdep) and Mean-Time-To-Remediate for vulnerable dependencies (MTTRdep), to measure the responsiveness of development teams in keeping their software packages updated and address the limitations of existing metrics.

Nadia: Industry practitioners are increasingly concerned with software that contains vulnerable versions of third-party dependencies that are included both directly and transitively Industry practitioners are increasingly concerned with software that contains vulnerable versions of third-party dependencies that are included both directly and transitively > The majority of codebases had vulnerable dependencies for more than two years despite the availability of a fixed version

2: ,

3: Novel Metrics Design The authors propose two novel metrics, MTTUdep and MTTRdep, designed to overcome limitations in existing metrics We propose two novel metrics, Mean-Time-To-Update for dependencies (MTTUdep) and Mean-Time-To-Remediate for vulnerable dependencies (MTTRdep) that overcome the limitations of existing metrics > MTTUdep of a package is the weighted aggregated time the package uses an outdated direct dependency version in its lifetime T T Udep(pi, pj) = P Pt wtdt t wt (2) The design choices for these metrics involve splitting dependency relations into multiple intervals based on when the package or the dependency has a newer release We begin by examining each of the relationships from a temporal perspective and split each relation into multiple intervals based on when the package or the dependency has a newer release (major, minor, or patch release) > We mark as “updated”=false if the resolved dependency version for that interval does not match the highest available version of the dependency at that interval Empirical Study and Findings The empirical study was conducted using 163, 207 packages in npm (117, 129), PyPI (42, 777), and Cargo (3, 301) We conduct an empirical study using 163, 207 packages in npm (117, 129), PyPI (42, 777), and Cargo (3, 301) and characterize how the ecosystems differ in MTTUdep and MTTRdep We found that most packages have a relatively fast dependency update practice We also found that older packages tend to have higher MTTUdep and MTTRdep values Proxy Analysis and Package Characteristics The study performed a proxy analysis to explore if MTTUdep can serve as a proxy for MTTRdep when sufficient vulnerability data are not available The results of the TOSTs show statistical equivalence between MTTUdep and MTTRdep (p 0.05) The regression with Wald Test returned a positive and also statistically significant coefficient, indicating that, on average, when MTTUdep increases by one unit, MTTRdep also increases by 0.69 The sensitivity analysis returns GF I = 0.83, AGF I = 0.67, and NF I = 0.83 This result indicates that MTTUdep does not provide a good fit to MTTRdep Correlation analysis showed weak or negligible correlations between the tested factors and MTTUdep and MTTRdep The regression analysis for MTTUdep as the dependent variable yielded an R2 value of 0.039, indicating that 3.9% variation in the dependent variable can be explained by the model Practical Implications The practical implications suggest that developers should use MTTRdep when available and only MTTUdep as a proxy for MTTRdep with caution when MTTRdep is not available Our results indicate that improving general dependency update frequency likely improves security as well A mixed strategy with automatic minor and patch updates, alongside manual major version updates, could be the most effective strategy to keep the MTTUdep and MTTRdep minimal Future Research Directions Future research can explore additional factors influencing dependency update practices, such as the severity of vulnerabilities, organizational policies, or developer incentives Expanding this analysis to other ecosystems, combining with transitive dependencies, and incorporating qualitative insights from developers could further refine our understanding of dependency updatedness The code and data for the analysis in this paper are all available in our replication package in Zenodo

17: > The code and data for the analysis in this paper are all available in our replication package in Zenodo

17: Page 1 --- The gist The study proposes two novel metrics, Mean-Time-To-Update for dependencies (MTTUdep) and Mean-Time-To-Remediate for vulnerable dependencies (MTTRdep), to measure the responsiveness of development teams in keeping their software packages updated and address the limitations of existing metrics.

Paper summary: Nadia: So, wrapping up this paper on "How Quickly Do Development Teams Update Their Vulnerable Dependencies?", the authors introduce MTTUdep and MTTRdep to better measure how fast teams are keeping their software packages updated.

Elias: They found that while most packages update dependencies relatively quickly, older packages tend to have higher values for both metrics.

Priya: The key finding is that MTTUdep can only be used as a proxy for MTTRdep when vulnerability data isn't available, and they didn't find strong statistical evidence supporting that link.

Nadia: So, the paper suggests a mixed strategy where developers use MTTRdep when possible and treat MTTUdep with caution as an indicator of overall responsiveness.

Elias: It really shifts the focus from just finding vulnerabilities to understanding the process behind those updates so we can actually improve it systematically.

Priya: For someone listening, this means if you're a security researcher, you have a better way to gauge team behavior, and for a developer, it gives them concrete guidance on how to set up their update processes.

Nadia: It’s about making the process of keeping software secure more transparent through these specific measurements.

Conclusion: Nadia: So we’re wrapping up this look at "How Quickly Do Development Teams Update Their Vulnerable Dependencies?" and looking at who actually wrote it.

Elias: The authors are focusing on building two new ways to measure how fast teams respond to security updates, which they call MTTUdep and MTTRdep.

Nadia: They’re trying to fix the old metrics because those old ones didn't account for things like floating versions or prioritizing newer updates.

Elias: Exactly, they designed these new metrics so the measurements reflect the real speed of a development team’s update habits better than before.

Nadia: The study used a huge amount of code across npm PyPI and Cargo to see how different ecosystems behave in terms of these update times.

Elias: What they found was that most packages are actually updating their dependencies pretty quickly, which is good news for the security community.

Nadia: But the paper also showed that older packages tend to have higher MTTUdep and MTTRdep values, so age still matters a bit.

Elias: And then they did this interesting proxy analysis where they tried to see if MTTUdep could stand in for MTTRdep when vulnerability data is scarce.

Nadia: Well, the results showed that there isn't really strong statistical proof that you can just use MTTUdep as a reliable stand-in for MTTRdep.

Elias: The regression with the Wald Test gave a positive coefficient, and the sensitivity analysis showed it’s not a good fit for what MTTRdep is measuring.

Nadia: So, the main implication is that you should use MTTRdep when you have that specific vulnerability data available and treat MTTUdep with a lot of caution otherwise.

Elias: They suggest that improving general update frequency across the board is likely to boost security, but doing so smartly—maybe mixing automatic minor updates with manual major ones—is probably the best path forward.

Nadia: It’s about getting a clearer picture of the process, not just counting how many vulnerabilities are out there.

Elias: Right, and that leads us next to figuring out what other factors might actually be slowing down those update cycles.