RCOMPSs: A Scalable Runtime System for R Code Execution on Manycore Systems

summary

The gist

RCOMPSs presents RCOMPSs, a scalable runtime system that enables efficient parallel execution of R applications on multicore and manycore systems <ref:2505.06896#pg2> This paper introduces RCOMPSs…

In short

RCOMPSs is a scalable runtime system that allows R applications to run efficiently in parallel across manycore and multicore systems. It introduces a dynamic, task-based model where users write sequential code by annotating functions as tasks. The runtime automatically manages task dependencies and schedules execution across available resources, enabling efficient parallel computing for statistical workflows.

Key concepts

Dynamic, Task-Based Programming Model
This model allows users to write R code in a simple, sequential style. Instead of manually managing parallel threads or processes, the user marks specific functions as 'tasks.' The runtime then takes over to handle the complex details of when and where these tasks should run based on dependencies and available hardware resources.
Task Dependency Graph
The runtime automatically builds a map showing how different tasks rely on each other. If Task B needs the output of Task A, the graph shows this relationship. This graph is crucial because it tells the scheduler which tasks can run concurrently and which must wait for others to finish before they can start.
Strong and Weak Scalability
Scalability measures how well RCOMPSs performs when you increase the number of cores or nodes. Strong scalability means performance increases significantly as you add more cores on one machine, while weak scalability shows how performance holds up when distributing tasks across many different machines.

Terminology used across episodes

This episode discusses

The paper

RCOMPSs: A Scalable Runtime System for R Code Execution on Manycore Systems · Read on arXiv

Xiran Zhang, Javier Conejero, Sameh Abdulah, Jorge Ejarque, Ying Sun, Rosa M. Badia, David E. Keyes, Marc G. Genton

King Abdullah University of Science and Technology (KAUST) · Barcelona Supercomputing Center (BSC)

R has become a cornerstone of scientific and statistical computing due to its extensive package ecosystem, expressive syntax, and strong support for reproducible analysis. However, as data sizes and computational demands grow, native R parallelism support remains limited. This paper presents RCOMPSs, a scalable runtime system that enables efficient parallel execution of R applications on multicore and manycore systems. RCOMPSs adopts a dynamic, task-based programming model, allowing users to write code in a sequential style, while the runtime automatically handles asynchronous task execution, dependency tracking, and scheduling across available resources. We present RCOMPSs using three representative data analysis algorithms, i.e., K-nearest neighbors (KNN) classification, K-means clustering, and linear regression and evaluate their performance on two modern HPC systems: KAUST Shaheen-III and Barcelona Supercomputing Center (BSC) MareNostrum 5. Experimental results reveal that RCOMPSs demonstrates both strong and weak scalability on up to 128 cores per node and across 32 nodes. For KNN and K-means, parallel efficiency remains above 70% in most settings, while linear regression maintains acceptable performance under shared and distributed memory configurations despite its deeper task dependencies. Overall, RCOMPSs significantly enhances the parallel capabilities of R with minimal, automated, and runtime-aware user intervention, making it a practical solution for large-scale data analytics in high-performance environments.

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: We're looking at the paper: "RCOMPSs: A Scalable Runtime System for R Code Execution on Manycore Systems".

Jane: Here's the gist of the paper. RCOMPSs presents RCOMPSs, a scalable runtime system that enables efficient parallel execution of R applications on multicore and manycore systems <ref:2505.06896#pg2> This paper introduces RCOMPSs, a scalable runtime system that enables efficient parallel execution of R applications on multicore and manycore systems <ref:2505.06896#pg2>.

Tom: Experimental results reveal that RCOMPSs demonstrates both strong and weak scalability on up to 128 cores per node and across 32 nodes <ref:2505.06896#pg2>.

Jane: The Core Concept of RCOMPSs RCOMPSs adopts a dynamic, taskbased programming model, allowing users to write code in a sequential style, while the runtime automatically handles asynchronous task execution, dependency tracking, and scheduling across available resources <ref:2505.06896#pg2>.

Tom: It is built on top of the COMPSs framework and introduces a new binding for R that allows developers to write sequential R code while transparently benefiting from parallel execution <ref:2505.06896#pg2>.

Paper summary: Tom: So looking at the whole picture of "RCOMPSs: A Scalable Runtime System for R Code Execution on Manycore Systems," what’s the title and who wrote this paper really telling us about its place in the field?

Jane: The authors, including King Abdullah University of Science and Technology and Barcelona Supercomputing Center researchers, are showing how you can extend existing parallel programming concepts to fit the R language ecosystem.

Lu: What they’re really emphasizing is that this system abstracts away complex parallelization tasks like thread management and data distribution, allowing developers to focus on building the actual scientific application.

Meng: From my side, what this suggests is that if we can get a tool like this running efficiently on our current infrastructure, it could significantly reduce the engineering time needed to scale up R-based models for production environments.

Tom: It shows that even when dealing with manycore systems and complex data management, you don't always need to dive into low-level parallel programming details yourself.

Jane: The implication is that R can become a more viable language for large-scale scientific computing because the barrier to entry for writing truly parallel code is lowered substantially.

Conclusion: Tom: So we've seen how RCOMPSs lets you write sequential R code but get parallel execution for free on these big computer systems like manycore setups, and now we're getting to the end of this paper.

Jane: Yeah, what is the main point here, Tom? Like, why does this system actually matter if it just makes things run a little faster?

Tom: Well, it’s about taking that task-based parallel execution idea from other systems and making it work smoothly inside R itself without forcing you to rewrite every single function.

Lu: What they're really saying is they built this runtime on top of the COMPS framework and added a new way for R users to mark functions as tasks, which the system then manages automatically.

Meng: From my side, it's about reducing the headache of manually juggling threads or distributed memory calls when you just want to run a standard statistical model in R.

Lalam: The authors are showing that this system works on systems up to one hundred twenty-eight cores per node and across thirty-two nodes, which is pretty big for a runtime system.

Tom: Right, so the main thing is they proved it can scale up without you having to manage all those low-level details yourself.

Jane: So if I'm a data scientist who just wants to run my analysis in R, this means I can use more powerful hardware without needing a team of parallel programming experts?

Tom: Exactly. It moves the complexity from the user to the system, letting you focus on your math and your data instead of how the computer is physically wired.

Lu: The implication is that we can get R code running in a much more efficient way across these manycore architectures than before, which opens up new kinds of applications.

Meng: I'm still watching how it handles real-world data bottlenecks, though; does this runtime actually keep up when the data itself gets huge?

Tom: That’s where we’ll get to next; we need to look at their performance numbers on actual algorithms like linear regression and KNN.

More episodes

← Home