Programmable GPU-Accelerated dApps for Real-Time Edge Inference and ISAC on NVIDIA Aerial Testbed
summary
The gist
The paper presents a programmable, opensource framework for processing PHY/MAC signals through real-time…
In short
This work introduces an open-source framework allowing real-time AI processing of radio signals on edge RAN infrastructure using NVIDIA GPUs. It enables Integrated Sensing and Communication (ISAC) by processing physical layer data to perform accurate, real-time localization of moving objects, overcoming challenges in data exposure and high-accuracy sensing.
Key concepts
- Programmable Framework
- A flexible, open-source software structure designed to interface with edge radio hardware. It allows developers to build custom Artificial Intelligence (AI) applications that run directly on the GPU for processing radio signals, offering a standardized way to deploy complex AI tasks in RAN environments.
- Integrated Sensing and Communication (ISAC)
- A technology that combines communication functions with sensing capabilities within the same radio system. This framework specifically uses communication signals to perform real-time sensing tasks, such as accurately locating moving entities, by analyzing the physical layer data.
- cuSense dApp
- A specific application developed within the framework designed for ISAC. It consumes uplink channel estimates from 5G networks and uses a neural network to infer the precise position of a moving person, achieving localization errors as low as 77 cm.
- GPU-accelerated Inference
- The process of using powerful Graphics Processing Units (GPUs) to run complex AI models very quickly. This framework leverages GPUs to process raw PHY/MAC signals in real-time, ensuring low latency and high throughput necessary for tracking applications.
Terminology used across episodes
This episode discusses
- Programmable GPU-Accelerated dApps for Real-Time Edge Inference and ISAC on NVIDIA Aerial Testbed · Paper Radio
The paper
Programmable GPU-Accelerated dApps for Real-Time Edge Inference and ISAC on NVIDIA Aerial Testbed · Read on arXiv
Davide Villa, Mauro Belgiovine, Nicholas Hedberg, Michele Polese, Chris Dick, Tommaso Melodia
NVIDIA Corporation · Institute for Intelligent Networked Systems
Transcript
Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.
Tom: We're looking at the paper: "Programmable GPU-Accelerated dApps for Real-Time Edge Inference and ISAC on NVIDIA Aerial Testbed".
Jane: Here's the gist of the paper. The paper presents a programmable, opensource framework for processing PHY/MAC signals through real-time, GPU-accelerated Artificial Intelligence (AI) applications on edge Radio Access Network (RAN) infrastructure to enable Integrated Sensing and Communication (ISAC) on NVIDIA Aerial Testbed Framework Introduction The work introduces a programmable, opensource framework for processing PHY/MAC signals through real-time, GPU-accelerated Artificial Intelligence (AI) applications on the edge Radio Access Network (RAN) infrastructure Building on the Open RAN dApp architecture, the framework interfaces with a GPU-accelerated gNB based on NVIDIA Aerial Testbed (ATB), feeding PHY/MAC data to custom AI logic with a framework overhead of 150 µs, multiple inference engines, and support for several AI backends The framework is released as open source, providing a reference design for future AI-native RANs and ISAC applications Key Challenges Addressed The paper addresses two critical challenges in enabling real-time ISAC with GPU-accelerated dApps on the NVIDIA Aerial Testbed (ATB) 5G Stack Challenge number 1 is the Lack of Real-Time Data Exposure, where RAN control and observability remain limited to near- or non-real-time scales and focused on the control plane Challenge number 2 is Real-time, High-Accuracy Estimates, which involves extracting accurate sensing estimates from communication signals that are high-dimensional, noisy, and contain strong static multipath components Framework Design Components
Title and authors: Tom: So we're looking at this paper now titled "Programmable GPU-Accelerated dApps for Real-Time Edge Inference and ISAC on NVIDIA Aerial Testbed." This seems to be about taking the concepts of Integrated Sensing and Communication, or ISAC, and putting them onto the edge Radio Access Network infrastructure using programmable AI applications running on a GPU.
Jane: Right, it sounds like they're talking about making sense of both what the network is communicating with and what it’s sensing using these GPU-accelerated dApps. The paper introduces this framework as a way to do that, building on the Open RAN dApp architecture.
Lu: What I find interesting from the title is that they aren't just talking about a theoretical idea; they are presenting a programmable, open source framework for processing PHY and MAC signals in real time using AI on edge RAN infrastructure. It’s about making these complex sensing tasks happen right there on the hardware.
Meng: From my side, I'm more interested in how much overhead this framework adds. The paper mentions it has a framework overhead of one hundred fifty microseconds, and they support multiple inference engines, which tells me it needs to be pretty efficient for deployment at scale.
Lalam: It seems like the core idea is connecting this framework to a GPU-accelerated gNB based on the NVIDIA Aerial Testbed, which is essentially an AI-enabled platform for testing AI-RAN research. This setup lets them feed PHY and MAC data directly into custom AI logic.
Tom: Exactly. The main challenge they are tackling here is enabling real time ISAC with GPU accelerated dApps on the NVIDIA Aerial Testbed, specifically focusing on two things: first, getting real time data exposure because control and observability in RAN are usually near or non real time; and second, achieving real time high accuracy estimates when you're dealing with communication signals that are noisy and have strong static multipath components.
Jane: That sounds like a really tricky problem because CSI measurements are high dimensional, noisy, and they have these strong static multipath parts that you really need to separate from the changes caused by something moving. How does this framework actually help solve that challenge?
Lu: The paper suggests several improvements to address these challenges. First, they introduce a cuSense dApp which can consume uplink DMRS channel estimates from a five g NR deployment and use a neural network to infer the position of a moving person with an error of seventy seven centimeters. That’s the concrete application they’re showing.
Title and authors: Meng: Seventy seven centimeters is specific, but what about the infrastructure? The paper also mentions that the framework lets dApps access PHY and MAC data via shared memory buffers and an E3 based control or data plane interface, which they call zero copy data sharing between RAN and dApps. That sounds like a big win for performance.
Lalam: And they have this architecture that supports multiple E3 Agents running within the gNB, with an E3 Manager that can independently register with multiple agents to create different subscriptions, like at different sampling rates or telemetry streams. That suggests a lot of flexibility in how you can use the system.
Tom: So we've covered the framework setup and some specific applications on the ATB. The paper also details that they support three inference engine configurations: Triton C API engine, Triton gRPC engine, and an embedded Python engine, supporting backends like PyTorch, LibTorch, ONNX Runtime, and TensorRT. That shows they’re giving users a lot of choice in terms of how the AI model actually runs.
Jane: The processing pipeline for their cuSense dApp is also pretty detailed; it goes through three stages: first background environment characterization which is offline, then temporal noise reduction and feature normalization which happens online, and finally NN based location estimation in Triton during online operation. That’s a very structured approach to handling the data.
Lu: And they quantify the performance with a total end-to-end latency of approximately two point three milliseconds for that cuSense dApp inference pipeline running on the GH200 with MIG, which they say is well within real time requirements for tracking applications. That puts a hard number on how fast this can actually run.
Meng: Two point three milliseconds is very fast, but we have to consider what that means when you try to deploy this at scale across many gNBs. What happens when the traffic volume ramps up significantly? Does that latency hold steady, or does it degrade under heavy load?
Tom: That’s a fair question, Meng. The paper focuses on the inference pipeline performance under those specific conditions on the GH200 with MIG, but scaling across a whole network is another layer of complexity they will have to address later. However, for tracking applications specifically, that speed is what makes it viable.
Jane: So for someone who doesn't read a lot about RAN stack specifics, what does this really mean for them? It means they can use their communication signals not just to send data but also to sense their surroundings in real time with good accuracy because of this framework.
Title and authors: Lu: It moves the capability from just communication to integrated sensing directly on the edge, using AI models that are trained and run right there in the network environment where the data is generated.
Meng: From an engineering standpoint, it’s about decoupling the AI logic from custom embedded kernels, which gives developers more flexibility than being locked into a specific hardware implementation. That modularity is what makes me think it could actually be deployable on commercial gear sooner.
Lalam: I just want to emphasize that they've made this open source, so anyone can build on top of this reference design for future AI native RANs and ISAC applications. It’s a blueprint for what the future might look like in this space.
Tom: So, to wrap up: the paper "Programmable GPU-Accelerated dApps for Real-Time Edge Inference and ISAC on NVIDIA Aerial Testbed" shows a programmable, open source framework that uses real time GPU accelerated AI applications on edge RAN infrastructure to enable ISAC. They address the lack of real time data exposure and high accuracy estimates by introducing things like the cuSense dApp and zero copy data sharing.
Jane: It’s about taking complex communication signals, separating the static noise from what's moving, and using a structured pipeline with specific inference engines to get accurate location estimates very quickly on the NVIDIA Aerial Testbed.
Lu: The key is that they show a path to submeter accuracy with an end-to-end latency of two point three milliseconds on hardware like the GH200. It’s about demonstrating that this level of performance is achievable for tracking applications.
Meng: What I'd say is that the modularity and the way it handles different inference engines give us a good starting point to think about how we can make this robust enough for real traffic, not just test scenarios on a specific accelerator.
Lalam: And because they open source it, it means we can all start building on top of this foundation for future work in AI native RANs. It’s a practical design that's ready to be used right now.
Tom: That’s the gist of it: a solid, programmable framework that bridges the gap between communication and sensing using GPU accelerated dApps on the Aerial Testbed. We'll take a quick break, and after this, we'll talk about how these AI models are actually trained on real-world RAN traffic.
The paper's summary: Tom: So, to wrap up what we just talked about, this paper shows they built a whole framework that lets you run AI applications in real time on edge radio access network stuff using a GPU on the Aerial Testbed to do integrated sensing and communication.
Jane: Yeah, it’s like they took the raw signal data from the phone—the PHY and MAC layers—and fed it right into these custom AI models running on a specialized chip, making sure this whole process happens in less than two point three milliseconds.
Lu: What’s really interesting is that they solve the problem of getting real time data exposure, which is usually a huge headache because RAN stuff isn't always ready for AI to look at instantly.
Meng: I’m thinking about the deployment side here, how you run this on real hardware. The paper mentions they use shared memory buffers and a zero copy data sharing thing between the radio network and the dApps, which sounds like it cuts out a ton of overhead that usually slows things down when you try to scale up.
Tom: Exactly. And they give us these options for running the AI models—they have three different inference engine configurations, like Triton C API or embedded Python, so you’re not locked into just one way to run your model on the hardware.
Jane: And this cuSense dApp they built? It takes those raw uplink measurements and it figures out where a moving person is with an error of seventy seven centimeters. That’s a concrete result, showing that this setup can actually do meaningful sensing tasks.
Lu: The three-stage pipeline for that dApp is also smart: they characterize the environment first offline, then they clean up the noise and normalize it online, and finally the neural network does the location estimation during online operation. It’s a very structured way to handle messy real world data.
Meng: That sounds complicated to build correctly on production gear. What breaks is usually when you try to push that complexity onto something that has to run reliably for hours under heavy traffic, not just for a quick test run.
Tom: That’s the million dollar question, Meng. But the paper shows this framework is open source, which means anyone can take this blueprint and build on it for future AI native RANs and integrated sensing applications.
Jane: It really changes how we think about 5G networks moving forward—it’s not just about sending data anymore; it’s about making the network actively sense its surroundings with high accuracy, right there at the edge <ref:2512.06493#pg2>.
Lu: The implication is that for things like tracking or autonomous vehicles, you can get those precise location estimates quickly enough to make a real difference in how those systems operate.
Meng: It makes sense if you can get a reliable estimate that fast, but we still need to know what happens when the traffic gets really high and the network starts feeling stressed.
Tom: That’s definitely the next step for anyone trying to move this from testbed success to something you could actually put in front of customers.
The paper's improvements: Tom: So, we’ve seen how they set up the framework on the Aerial Testbed, and now let's talk about what they actually improved in their design to make it work better than just a basic setup.
Jane: Right, so they introduced several specific upgrades that tackle those hard problems we talked about earlier—like making sure the data moves without copying it unnecessarily.
Lu: They focused on this cuSense dApp which is really clever because it doesn't just look at raw signals; it takes the uplink measurements and specifically removes those static multipath components before running the neural network.
Meng: That removal of static multipath is a big deal because those things are usually what make sensing estimates so noisy, so if you clean that up before feeding it to the AI, you should get a much better position estimate.
Tom: And they made the data sharing way more efficient by using shared memory buffers and an E3 based interface to achieve zero copy data sharing between the radio network and these dApps. That means less delay when moving that information around.
Jane: It’s also about giving developers flexibility with three different inference engine configurations, like Triton C API or embedded Python, so they can choose the best way to run their specific AI model on the hardware.
Lu: The pipeline for that cuSense dApp is structured in three stages: background environment characterization offline, then temporal noise reduction online, and finally the neural network location estimation running online in Triton. It’s a very logical flow.
Meng: That sounds like a lot of moving parts to get right on production hardware. What breaks at scale is often when you can't easily update those components without taking the whole system down, so modularity matters for deployment.
Tom: They addressed that by allowing multiple E3 Agents to run inside the gNB, and an E3 Manager that can register with them independently to create different subscriptions with different sampling rates or telemetry streams. That’s a lot of control for a single radio unit.
Jane: This architecture is powerful because it lets you tailor the sensing data stream exactly how you need it, whether you want high frequency updates or just occasional status checks.
Lu: And they quantified the speed really well, showing that the total end-to-end latency for that cuSense dApp inference pipeline on a GH200 with MIG hits about two point three milliseconds. That’s a very tight number for real time tracking applications.
Meng: Two point three milliseconds is fast, but I still need to know if that speed holds up when the network starts getting really busy and the traffic ramps up significantly.
Tom: That’s exactly what you need to look at next, Meng. The paper shows it works well for those specific tracking tasks on that particular hardware setup.
Conclusion: Tom: So, to wrap up this talk about "Programmable GPU-Accelerated dApps for Real-Time Edge Inference and ISAC on NVIDIA Aerial Testbed," we’ve seen how they built a framework that lets us run AI applications in real time on edge radio access network stuff using a GPU on the Aerial Testbed to do integrated sensing and communication.
Jane: It really shows that we can get complex tasks like figuring out where something is moving, using communication signals, done at the edge with very low latency.
Lu: The implication is that we move away from just sending data and start using the network itself as a sensor in real time.
Meng: For deployment, it gives us a solid reference design for building things modularly instead of being stuck with some locked-down custom code on every piece of gear.
Lalam: I’m confident this framework is the right direction because it directly addresses the gap between how we communicate and how we sense, and the results they show on that hardware are very concrete.
Tom: The speed numbers, like two point three milliseconds for the cuSense dApp inference pipeline, are what really make me excited about shipping something practical.
Jane: It’s about moving this capability from just a research concept to something that could actually be put into real-world tracking applications without needing massive data centers.
Lu: The open source nature means everyone can take this blueprint and build on it for future AI native RAN systems, which is a big win for the whole community.
Meng: I still need to know how much that runs in a live environment with heavy traffic, but having this tested on the Aerial Testbed gives us a really good starting point for figuring out those scaling issues.
Tom: Exactly. The next big question is taking this proven concept and figuring out how to make it robust enough to handle real network stress.
Jane: And after we talk about what makes this framework work, we’ll look at how these AI models are actually trained on real world RAN traffic, which is the next logical step in making this technology useful.
More episodes
- 2505.16254-Human-AI Writing as Distributed Cognition: Reassessing Collaborative Writing Theories in the Age of LLMs
- 2506.10913-Choreographic Quick Changes: First-Class Location (Set) Polymorphism
- 2507.14812-A Black-Box Approach for Exogenous Replenishment in Online Resource Allocation
- 2507.15448-Frames over finite fields with respect to the Galois inner product and their connections to self-orthogonal and LCD codes
- 2508.21572-NewsReX: An Open-Source Multi-Framework for Neural News Recommendation
- 2509.17028-Impact of Packetization on Network Calculus Analysis
- 2510.17342-AoA Services in 5G Networks: A Framework for Real-World Implementation and Systematic Testing
- 2511.04088-Efficient and rate-optimal list-decoding in the presence of minimal feedback
- 2511.17264-Notes on Stack Machines and Quantum Stack Machines
- 2601.15318-On the closest balanced game