Real-Time 2D LiDAR Object Detection Using Three-Frame RGB Scan Encoding

summary

The gist

Indoor service robots need perception that is robust, more privacy-friendly than RGB video, and feasible on embedded hardware<ref:2602.02167#pg2> This work presents a camera-free 2D LiDAR object…

In short

This work proposes a camera-free 2D LiDAR object detection pipeline for indoor robots. It encodes three consecutive LiDAR scans into an RGB-like tensor to capture short-term motion and viewpoint changes. This compact input is fed into a YOLOv8n model, achieving high accuracy in simulation and running in real time on embedded hardware like the Raspberry Pi 5.

Key concepts

2D LiDAR Scan Rasterization
A raw 2D LiDAR scan consists of angle and distance measurements. Rasterization converts this sparse data into a fixed-size binary image, typically 64x360 pixels. This process maps the angular measurements to columns and the distance measurements to rows, creating a grid representation suitable for image processing.
Temporal Data Encoding
To capture motion cues without needing robot pose alignment, three consecutive scans are stacked as three separate RGB channels. This creates a compact input tensor where each pixel location across the 64x384 grid holds the scan data from time t-2, t-1, and t. This stacking allows the model to infer movement over a short time window.
YOLOv8n for Embedded Feasibility
The detection model used is YOLOv8n, chosen because it is lightweight with approximately 3 million parameters. This small size makes the model feasible for deployment on resource-constrained embedded CPUs, such as the Raspberry Pi 5. It processes the compact encoded input efficiently to achieve real-time performance.
Compact Input Tensor (64x384x3)
The final input format is a compact tensor created by stacking three consecutive rasterized scans. This structure preserves the angular layout of the LiDAR data while adding temporal depth via the three channels. This avoids complex occupancy-grid construction, resulting in a small, efficient input size for standard CNN detectors.

Terminology used across episodes

This episode discusses

The paper

Real-Time 2D LiDAR Object Detection Using Three-Frame RGB Scan Encoding · Read on arXiv

Soheil Behnam Roudsari, Alexandre S. Brandao, Felipe N. Martins

Institute of Engineering Hanze University of Applied Sciences · Nucleo de Especializacção em Robótica, Departamento de Engenharia ElƔtrica, Universidade Federal de VicƓsa

Indoor service robots need perception that is robust, more privacy-friendly than RGB video, and feasible on embedded hardware. We present a camera-free 2D LiDAR object detection pipeline that encodes short-term temporal context by stacking three consecutive scans as RGB channels, yielding a compact YOLOv8n input without occupancy-grid construction while preserving angular structure and motion cues. Evaluated in Webots across 160 randomized indoor scenarios with strict scenario-level holdout, the method achieves 98.4% mAP@0.5 (0.778 mAP@0.5:0.95) with 94.9% precision and 94.7% recall on four object classes. On a Raspberry Pi 5, it runs in real time with a mean post-warm-up end-to-end latency of 47.8ms per frame, including scan encoding and postprocessing. Relative to a closely related occupancy-grid LiDAR-YOLO pipeline reported on the same platform, the proposed representation is associated with substantially lower reported end-to-end latency. Although results are simulation-based, they suggest that lightweight temporal encoding can enable accurate and real-time LiDAR-only detection for embedded indoor robotics without capturing RGB appearance.

DOI: 10.1109/SAS70013.2026.11744174

Transcript

Introduction to the show: ident: Electrical Engineering. The day in electrical engineering: signals, systems and the hardware that carries them.

Rosa: I'm Rosa.

Dev: And I'm Dev, here with Taro.

Rosa: We're looking at the paper: "Real-Time 2D LiDAR Object Detection Using Three-Frame RGB Scan Encoding".

Dev: Here's the gist of the paper. Indoor service robots need perception that is robust, more privacy-friendly than RGB video, and feasible on embedded hardware<ref:2602.02167#pg2> This work presents a camera-free 2D LiDAR object detection pipeline that encodes short-term temporal context by stacking three consecutive scans as RGB channels, yielding a compact YOLOv8n input without occupancy-grid construction while preserving angular structure and motion cues<ref:2602.02167#pg2> The method achieves 98.4% mAP@0.5 (0.778 mAP@0.5:0.95) with 94.9% precision and 94.7% recall on four object classes in simulation<ref:2602.02167#pg2> On a Raspberry Pi 5, it runs in real time with a mean post-warm-up end-to-end latency of 47.8 ms per frame, including scan encoding and postprocessing<ref:2602.02167#pg2> System Overview We propose an efficient object detection pipeline based entirely on 2D LiDAR, designed for real-time execution on embedded CPUs<ref:2602.02167#pg4> Compared with prior LiDAR–YOLO pipelines that first construct occupancy grids, our approach encodes scans directly into a compact RGB-style tensor that can be processed by standard CNN detectors<ref:2602.02167#pg4> The core idea is to stack three consecutive 2D scans as the three channels of a small raster image<ref:2602.02167#pg4> This preserves the ordered angular structure of the scan while injecting short-term temporal cues (motion and viewpoint change) without requiring odometry alignment<ref:2602.02167#pg4> The resulting input is compact (64 × 384 × 3) and avoids the preprocessing cost and large input sizes typical of occupancy-grid pipelines<ref:2602.02167#pg4> Temporal Data Encoding A 2D LiDAR scan is a sequence of angle–range measurements where θi is the bearing and ρi is the measured distance<ref:2602.02167#pg4> Rasterization involves converting each scan into a sparse binary raster It ∈

0, 255: 64×360, where columns represent angle bins and rows represent discretized range bins<ref:2602.02167#pg4> The angular bin index is computed as ab = mod(round(θ

◦: ), 360), (1)i.e., angles are rounded to the nearest degree and wrapped modulo 360<ref:2602.02167#pg4> For the range axis, we discretize into 64 bins with a fixed full-scale value Rmax = 4.0 m<ref:2602.02167#pg4> The range-bin index is computed as rb = clip(ρ · 63 / Rmax, 0, 63, (2)where ρ is in meters<ref:2602.02167#pg4> To encode short-term temporal context, we maintain a first-in, first-out (FIFO) buffer of three consecutive rasters at times t − 2, t − 1, and t<ref:2602.02167#pg4> These are stacked into an RGB tensor I RGB t ∈

0, 255: 64×384×3: I RGB t (r, a,:) = It−2(r, a), It−1(r, a), It(r, a)<ref:2602.02167#pg4> This encoding injects motion cues and short-term viewpoint variation without scan matching or pose alignment<ref:2602.02167#pg4> Where the gains come from The main contributions include a compact multi-frame 2D LiDAR encoding that stacks three consecutive scans as RGB channels, injecting short-term temporal cues without pose alignment<ref:2602.02167#pg2> The model training uses YOLOv8n, chosen for embedded feasibility (approximately 3M parameters), with a fixed input resolution of 64 × 384<ref:2602.02167#pg5> Training was performed on a server with an NVIDIA H100 GPU (80 GB VRAM) for 120 epochs<ref:2602.02167#pg5> Comparison with Prior Work Table III summarizes reported results from Najem et al.

3: alongside our method<ref:2602.02167#pg6> Because Najem et al.

Dev: evaluate on real data with manual labels while our results are simulation-based with automatic labels, accuracy metrics are not directly comparable<ref:2602.02167#pg6> However, the occupancy-grid pipeline in

3: reports substantially higher end-to-end latency on Raspberry Pi 5 than our compact scan encoding This should be interpreted cautiously because the datasets, labeling regimes, software stacks, and timing protocols differ<ref:2602.02167#pg6> Inference Time Analysis We measure end-to-end CPU latency with batch size 1 and FP32 inference, including scan encoding, the YOLOv8n forward pass, and non-maximum suppression (NMS)<ref:2602.02167#pg6> On Raspberry Pi 5, the pipeline achieves a mean end-to-end latency of 47.8 ms per frame<ref:2602.02167#pg6> This meets the real-time target while leaving computational headroom for other onboard modules<ref:2602.02167#pg6> Limitations and Threats to Validity This study is evaluated in simulation; real 2D LiDAR sensors introduce noise, missing returns, and calibration effects that may reduce performance under domain shift<ref:2602.02167#pg6> Labels are derived from simulator metadata, which removes annotation ambiguity and may inflate reported metrics relative to manually labeled real-world datasets<ref:2602.02167#pg6> Temporal stacking is performed without pose alignment and may degrade under fast robot rotation or highly dynamic scenes<ref:2602.02167#pg6> Finally, while 2D LiDAR avoids RGB appearance capture, geometric signals may still enable behavioral inference in some deployments<ref:2602.02167#pg6> This paper presented a privacy-friendly, real-time 2D LiDAR object detection pipeline for indoor service robots<ref:2602.02167#pg6> The proposed approach achieves 98.4% mAP@0.5 (0.778 mAP@0.5:0.95) with 94.9% precision and 94.7% recall on four indoor object classes<ref:2602.02167#pg2> Future work will evaluate sim-to-real transfer on physical robots with real 2D LiDAR sensors, including robustness to noise, missing returns, and calibration effects<ref:2602.02167#pg6> We also plan to extend the object set (including dynamic agents), study failure modes under fast rotations where unaligned temporal stacking may degrade, and explore deployment optimizations such as quantization and hardware-accelerated runtimes to further reduce latency and power consumption<ref:2602.02167#pg6> REFERENCES

1: C.-Y.

Rosa: Wang, H.-Y.

Paper summary: Rosa: Thinking about the title "Real-Time 2D LiDAR Object Detection Using Three-Frame RGB Scan Encoding" and who wrote it, this work by Soheil Behnam Roudsari, Alexandre S <ref:2602.02167#pg1,Real-Time 2D LiDAR Object Detection Using Three-Frame RGB Scan Encoding>. Brandao, and Felipe N. Martins really focuses on making perception practical for indoor service robots <ref:2602.02167#pg1>.

Dev: It’s about taking the raw 2D LiDAR measurements and turning them into a format that looks like an RGB image with a little bit of time baked in by stacking three consecutive scans <ref:2602.02167#pg4>.

Taro: The implication for us is that we can design detection systems that are leaner, fitting on smaller chips, without needing those heavy preprocessing steps they are trying to get rid of <ref:2602.02167#pg4>.

Rosa: It’s about achieving a strong result with ninety-eight point four percent mean average precision at a five percent threshold on four object classes in simulation, which is quite impressive for this kind of setup <ref:2602.02167#pg1>.

Dev: But we must remember the caveats they laid out, because they said that since the method is evaluated in simulation, it might not perform as well when you introduce real-world noise or calibration errors from a physical sensor <ref:2602.02167#pg6>.

Taro: And also that the unaligned three-frame stacking might break down if the robot has fast rotations or encounters very dynamic things in its path <ref:2602.02167#pg3>.

Rosa: So, this paper shows a way to encode short-term temporal context from LiDAR scans to get a compact input for deep learning detectors, aiming for real-time performance on devices like the Raspberry Pi five <ref:2602.02167#pg6>.

Dev: It’s not a complete solution for deployment yet because of those real sensor issues and the limitations on rapid motion, but it's a solid step toward making perception more privacy-friendly and hardware-feasible for indoor robotics <ref:2602.02167#pg1>.

Conclusion: Rosa: So, we’ve been looking at this paper titled "Real-Time 2D LiDAR Object Detection Using Three-Frame RGB Scan Encoding <ref:2602.02167#pg1,Real-Time 2D LiDAR Object Detection Using Three-Frame RGB Scan Encoding>." Basically, they took a standard 2D LiDAR and turned its measurements into something that looks like an RGB image using three consecutive scans stacked together <ref:2602.02167#pg1>.

Dev: Right, so they're trying to make object detection work on embedded hardware by encoding the temporal context—that short-term motion or viewpoint change—into those three channels without needing any complex alignment from odometry.

Taro: It’s interesting because it avoids the whole heavy process of building those occupancy grids that most people use with LiDAR, which is usually super slow and memory hungry.

Rosa: Exactly, and they claim this compact input format, which is small—sixty-four by three hundred eighty-four by three—can run on things like a Raspberry Pi five in real time.

Dev: Forty-seven point eight milliseconds per frame including everything from encoding the scan to the final detection step. That’s tight for real robot control, but it meets the target for processing speed.

Taro: I'm still thinking about how that temporal stacking works when things move really fast or if the robot suddenly rotates quickly; that unaligned stacking could definitely cause some fuzzy detections in those dynamic situations.

Rosa: Well, the authors say it gets ninety-eight point four percent mean average precision at a five percent threshold across four object classes in their simulation tests. That’s a solid number for something running on limited hardware.

Dev: But we have to remember, that’s simulation data, so we gotta be careful about how well it transfers when you move this onto the physical world where real sensor noise and calibration errors kick in.

Taro: Yeah, and the labels they use are from simulator metadata, which means the accuracy numbers might be inflated compared to what we see on actual robots with manually labeled data.

Rosa: It shows that even without using cameras for appearance information, you can use temporal data from LiDAR to get a compact representation that’s viable for indoor service robots.

Dev: So it’s a promising way to build privacy-friendly perception systems, but the next big test is definitely seeing how this holds up when we move out of the simulator and into messy real-world environments.

More episodes

← Home