Design and Evaluation of a Touchscreen-Based Teleoperation Interface for Robotic Manipulators
summary
The gist
Intuitive teleoperation interfaces are crucial for the safe and effective operation of robotic manipulators in challenging environments This study designs and evaluates a novel touchscreen…
In short
This study developed and tested a novel touchscreen teleoperation interface for robotic manipulators to enable more natural surface interaction. The interface maps finger movements directly to robot motion, allowing finer velocity control and integrating visualization. Results showed participants completed tasks faster, achieved better coverage on complex paths, and experienced lower cognitive load compared to traditional joystick controls.
Key concepts
- Touchscreen Teleoperation Interface
- A novel control system where an operator interacts with a touchscreen by moving their finger. This finger movement is directly translated into the robot's motion, aiming to make remote operation feel more intuitive and direct, similar to touching the object itself.
- Hybrid Impedance-Admittance Architecture
- A control design that combines two methods: impedance for tracking desired paths (like tangential movement) and admittance for regulating contact forces (like normal force). This hybrid approach ensures precise motion while maintaining stable, repeatable contact during surface tasks.
- Normal-Force Regulation via Admittance Loop
- The system uses an admittance loop to manage the force perpendicular to the surface. Instead of manually adjusting pressure, this loop converts errors in measured contact force into specific motion commands for the robot, ensuring consistent and safe interaction with surfaces.
- Cognitive Load (NASA-TLX)
- A measure used to quantify how mentally demanding a task is for the operator. The study found that switching from a joystick to the touchscreen reduced this mental effort significantly, making the operation less taxing for the user.
Terminology used across episodes
This episode discusses
- Design and Evaluation of a Touchscreen-Based Teleoperation Interface for Robotic Manipulators · Paper Radio
- Towards Universal Shared Control in Teleoperation Without Haptic Feedback
- Towards Accessible Robot Control: Comparing Kinesthetic Teaching, SpaceMouse Teleoperation, and a Mixed Reality Interface
- Human-centric telerobotics: investigating users' performance and workload via VR-based eye-tracking measures
The paper
Design and Evaluation of a Touchscreen-Based Teleoperation Interface for Robotic Manipulators · Read on arXiv
Juan Jose Garc ía Cardenas, Alperen Kenan, Hamidreza Raei, Paul Bremner, Manuel Giuliani, Arash Ajoudani, Ádiana Tapus
ENSTA Institut Polytechnique de Paris · University of the West of England
Intuitive teleoperation interfaces are crucial for the safe and effective operation of robotic manipulators in challenging environments. In the nuclear industry, surface contact tasks such as swab sampling require precise path and force tracking, obstacle avoidance, and sustained operator attention, which conventional joystick interfaces struggle to support effectively. This study designs and evaluates a novel touchscreen teleoperation interface that maps continuous finger movements directly to robotic manipulator motions, provides finer velocity control, and integrates control with visualization, enabling more natural, precise, and intuitive surface interaction than conventional controllers. A comparative user study with 20 participants evaluated task performance and workload using the proposed touchscreen, a conventional joystick, and a single-click autonomous mode. Tasks simulated realistic surface manipulation using a Franka Emika Panda arm, remotely controlled from another country. Kinematic, physiological, and behavioral data were recorded to comprehensively assess task performance, cognitive load, and operator trust across each control condition. Participants completed teleoperation tasks more efficiently and accurately with the touchscreen interface, achieving a 53.5% reduction in completion time (median: 2.50 vs. 5.38 min), higher in-area coverage on the sinusoidal path (90.7% vs. 84.1%), and lower overshoot on both path geometries compared with the joystick. Cognitive load, quantified via NASA-TLX (0-100), decreased from joystick to touchscreen (mean TLX 52 to 43; -9 points, -17.3%) and was lowest under the autonomous one-click mode (31; -21 points vs. joystick, -40.4%; -12 vs. touchscreen, -27.9%). This research presents an easy-to-implement touchscreen interface that improves performance in teleoperated surface tasks while reducing cognitive load.
DOI: 10.1109/RO-MAN72286.2026.11714783
Transcript
Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.
Rosa: I'm Rosa.
Dev: And I'm Dev, here with Taro.
Rosa: We're looking at the paper: "Design and Evaluation of a Touchscreen-Based Teleoperation Interface for Robotic Manipulators".
Dev: Here's the gist of the paper. Intuitive teleoperation interfaces are crucial for the safe and effective operation of robotic manipulators in challenging environments This study designs and evaluates a novel touchscreen teleoperation interface that maps continuous finger movements directly to robotic manipulator motions, provides finer velocity control, and integrates control with visualization, enabling more natural, precise, and intuitive surface interaction than conventional controllers Participants completed teleoperation tasks more efficiently and accurately with the touchscreen interface, achieving a 53.5% reduction in completion time (median: 2.50 vs.
Rosa: 5.38 min), higher in-area coverage on the sinusoidal path (90.7% vs.
Dev: 84.1%), and lower overshoot on both path geometries compared with the joystick Cognitive load, quantified via NASA-TLX (0–100), decreased from joystick to touchscreen (mean TLX 52 to 43; -9 points, -17.3%) and was lowest under the autonomous one-click mode (31; -21 points vs.
Rosa: joystick, -40.4%; -12 vs.
Paper summary: Rosa: So we're looking at this paper called Design and Evaluation of a Touchscreen-Based Teleoperation Interface for Robotic Manipulators. Basically it tackles the problem that right now operating robotic arms in tricky places requires too much skill and attention with the old joystick controllers.
Dev: Right, so what's the main idea here? What is this paper trying to show us about making these interfaces better?
Rosa: It designs a new touchscreen interface that maps your finger movements directly to how the robot moves. The claim is that this gives you much finer control over speed and lets you visualize things on screen, making surface interaction more natural and precise than what's available now Dev That sounds like it could really simplify tasks like swabbing or sampling in places where you can’t be there physically.
Taro: I wonder if that direct mapping is actually practical when you need to deal with real-world constraints, like latency and bandwidth? Does this interface actually work reliably outside of a perfectly controlled lab setting?
Dev: That's a good question, Taro. The authors designed the control architecture using a hybrid impedance-admittance scheme to handle those exact issues. They use Cartesian impedance for tracking the tangential motion—that's the movement along the surface—and an admittance loop for the normal direction, which is how they regulate that contact force Rosa That means while you touch to move along a path, the system automatically manages how hard it pushes against whatever surface it's touching.
Taro: So if you are swabbing and you accidentally push too hard, this system should compensate by regulating the force rather than just letting your input dictate a sudden impact? What happens when the world misbehaves and that force regulation needs to happen really fast?
Dev: The paper mentions they put a distal six degrees of freedom force torque sensor right at the robot's wrist, which closes that force loop locally. This way, the crucial normal-force regulation is decoupled from how slow your network connection is Rosa It also has this surface-lock behavior where if the measured normal force exceeds a certain threshold, it switches from free mapping to sticking the tool onto the surface Taro That sounds like a safety feature for when things get messy.
Rosa: And they quantified how much better it is using this setup. They compared three conditions: a conventional Xbox joystick, this new touchscreen interface, and an autonomous single-click mode Dev The results show that using the touchscreen cut down the task completion time by fifty-three point five percent, going from a median of five point three eight minutes with the joystick to two point five zero minutes with the new interface Taro That's a big jump in efficiency for repetitive tasks.
Dev: And it wasn't just speed, though. They also looked at how much mental effort the operator had to put in, which they measured using something called NASA-TLX scores Rosa The cognitive load dropped from an average score of fifty-two with the joystick down to forty-three with the touchscreen interface, that’s a nine point drop Taro
Paper summary: Taro: A nine point drop in mental load is significant for sustained remote operation. What about when the system is just running on its own, that single-click mode? Does it feel less stressful than even the new touchscreen setup?
Dev: It does. The autonomous one-click mode had the lowest cognitive load score of thirty-one Rosa That was a twenty point drop compared to the joystick and a twenty-seven point drop compared to this touchscreen interface Taro So, for someone just watching or supervising, that single-click approach seems much less taxing on their brain.
Rosa: It really shows how intuitive these mappings can be when they target surface contact specifically, unlike many previous touch interfaces that focused more on just moving the end effector in free space Dev The study suggests this particular design targets reducing operator burden by letting you emulate direct physical interaction with the environment Taro It’s about making the control feel more like you're actually touching things.
Dev: That's the core of it. They coupled that touch mapping with a control scheme that enforces a desired normal-force profile while keeping the tangential tracking compliant Rosa The whole goal was to reduce manual force modulation during tasks like swabbing because the robot handles that part autonomously Taro But they did flag a limitation: this interface primarily targets contact-rich surface manipulation, and it might not be as robust for purely free-space positioning or things where you need incredibly stable contact under really poor communication conditions Dev
Rosa: So, to summarize this paper Design and Evaluation of a Touchscreen-Based Teleoperation Interface for Robotic Manipulators, the authors are showing that by mapping finger movements directly to robot motion and using a hybrid control scheme with force regulation, you can achieve more precise surface interaction with significantly reduced operator workload Taro The implication is that for industries like nuclear or complex surface inspection where precision and sustained attention matter, this could make remote operation much more viable than current setups Dev
Taro: It shifts the focus from just getting a point to making the entire remote interaction process smoother for the human operator Rosa It’s less about fighting the machine and more about smoothly guiding it through a physical task. That seems like a big shift in how we design these systems, right?
Dev: Exactly. The paper highlights that this system reduces completion time by fifty-three point five percent on tasks like the sinusoidal path, which is hard to get with joysticks Taro It also shows that when you factor in the cognitive load reduction, especially with the autonomous mode, it becomes a much more practical tool for real-time application Rosa
Rosa: So yeah. This paper suggests that intuitive mapping combined with intelligent force control is a way forward for making robotic manipulators work better on surfaces where things are tricky Dev We'll keep an eye on how they tackle those real operational constraints in future work, because that’s where the next challenge will be Taro
Conclusion: Rosa: So we've been looking at this paper, "Design and Evaluation of a Touchscreen-Based Teleoperation Interface for Robotic Manipulators." Basically, they’re showing how mapping finger movements directly to robot motions can make operating these arms much easier on the human side.
Dev: Right. They built a touchscreen interface that maps continuous finger movements straight to the manipulator's motion, and it gives you better velocity control and integrates visualization. It aims for more natural surface interaction.
Taro: So, what's the big picture here? Is this just a fancy way to move the arm around in a lab setting, or does it actually matter when we take these systems out into rough environments?
Rosa: Well, it’s about reducing operator workload. The study shows that participants using this touchscreen interface were much faster at tasks. They saw a fifty-three point five percent reduction in completion time compared to the old joystick controllers.
Dev: That speed increase is significant, but the paper also looked at how much mental effort it took to do the job, quantified by NASA TLX scores. The load dropped by nine points when switching from the joystick to this touchscreen setup.
Taro: I'm interested in what happens when things go wrong out there. The authors mentioned they designed a hybrid impedance-admittance control architecture to handle latency and force regulation simultaneously. How does that actually behave when the connection between you and the robot is shaky?
Rosa: They addressed that by putting a force torque sensor right at the robot's wrist so they could regulate contact forces locally, even if the network link was slow. They also have this surface-lock behavior where if it touches too hard, it switches to sticking to the surface instead of just pushing harder.
Dev: So they decoupled the normal force regulation from network latency by putting that sensor close to the action. That’s a clever way to build a safety case for real-time control, even in noisy environments.
Taro: What does this mean for someone who only listens? It means we are moving away from needing perfect, continuous joystick input and toward something more tactile and automated during surface tasks.
Rosa: It shifts the focus from just controlling every single joint manually to guiding the interaction with the surface itself through touch. This suggests that for complex, contact-rich manipulation, intuitive interfaces might be a key way to make remote work practical instead of just theoretical.
More episodes
- 2608.13438-ContactGuard: Pre-Contact Execution Monitoring with Action-Conditioned Latent World Models
- 2610.34593-Future Prediction as Training-Only Video Supervision for Pretrained VLA Policies
- 2610.35385-EmbodiedLab: Beyond Manipulation, Benchmarking Embodied Agents That Act to Know
- 2610.36378-Data-Driven Hamiltonian for Neural Optimal Control
- 2610.37275-A Heuristic for Scheduling Autonomous Vehicles at Intersections
- 2610.37288-Dynamic Coordination of Autonomous Vehicles at Intersections
- 2610.37632-Real-Time Coordination of Autonomous Vehicles at Intersections
- 2611.13779-Softness is something you also need! Why compliance helps control: information metrics of a passive walker with compliant ankle
- 2611.13411-Residual-Corrected Physics-Informed Neural Network KKL Observers with Input-to-State Stability Guarantees
- 2611.14488-Is Your Robot's Memory Just a Clock? Temporal Shortcuts in Memory-Augmented VLA Policies