TacMind: Benchmarking and Modeling Tactile Interaction States for Robotic Manipulation
cs.RO
Submitted: 2026-10-30
Updated: 2026-10-30
Comments: Project Page: https://tacmind-bench.github.io/
Code: https://github.com/dexmal/opendm
Project page: https://tacmind-bench.github.io
License: http://creativecommons.org/licenses/by/4.0/
The gist: Tactile sensing provides robots with direct observations of contact physics, local geometry, and interaction dynamics, making it a vital source of feedback for contact-rich manipulation.
Terminology
Abstract
Tactile sensing provides robots with direct observations of contact physics, local geometry, and interaction dynamics, making it a vital source of feedback for contact-rich manipulation. However, existing tactile manipulation studies are often developed and evaluated within specific task settings, making it difficult to systematically compare how tactile information contributes under different interaction conditions. In this work, we develop a benchmark in simulation comprising 128 tasks organized around three complementary evaluation regimes: Precise, Recognition, and Dynamic. The benchmark further includes controlled static and dynamic task variants and 12 perturbation settings for evaluating robustness under changes in visual and physical conditions. These settings motivate studying tactile information as a temporally evolving interaction representation rather than only as an instantaneous observation. We therefore propose an action-conditioned latent predictive world model that integrates tactile history and learns a tactile state through prediction of future tactile latents. Building on this representation, we design a model that conditions action generation on the learned tactile state and uses predicted future tactile information to refine candidate actions through predictive tactile reflection. Experiments on the proposed benchmark show consistent gains from incorporating tactile information across the evaluated regimes. Project Page: https://tacmind-bench.github.io/
Sources
- UniVTAC: A Unified Simulation Platform for Visuo-Tactile Manipulation Data Generation, Learning, and Benchmarking
- Dexterity from Touch: Self-Supervised Pre-Training of Tactile Representations with Robotic Play
- 3D-ViTac: Learning Fine-Grained Manipulation with Visuo-Tactile Sensing
- ViTacWorld: Scaling Visuo-Tactile World Models for Contact-Rich Robot Manipulation
- $\pi_{0.5}$: a Vision-Language-Action Model with Open-World Generalization
- SoftVTBench: A Safety-Aware Visuo-Tactile Benchmark for Physically Constrained Robotic Manipulation of Deformable Objects
- ManiSkill-ViTac 2025: Challenge on Manipulation Skill Learning With Vision and Tactile Sensing
- LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning
- Spiral RoPE: Rotate Your Rotary Positional Embeddings in the 2D Plane
- ManiFeel: Benchmarking and Understanding Visuotactile Manipulation Policy Learning
- LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels
- TacEx: GelSight Tactile Simulation in Isaac Sim -- Combining Soft-Body and Visuotactile Simulators
- 3DThinkVLA: Endowing Vision-Language-Action Models with Latent 3D Priors via 3D-Thinking-Guided Co-training
- cuRobo: Parallelized Collision-Free Minimum-Jerk Robot Motion Generation
- $N_0$-VTLA: Scaling Vision-Tactile-Language-Action Model with Latent Tactile Tokens
- Reactive Diffusion Policy: Slow-Fast Visual-Tactile Policy Learning for Contact-Rich Manipulation
- Rotating without Seeing: Towards In-hand Dexterity through Touch
- MimicTouch: Leveraging Multi-modal Human Tactile Demonstrations for Contact-rich Manipulation
- TacForeSight: Force-Guided Tactile World Model for Contact-Rich Manipulation
Related papers
- Rethinking Actions: Continuity-Aware Priors for World-Model Planning
- MOSAIC: Modular Obstacle and Spatial-Aware Framework for Manipulation In Cluttered Scenes
- SAGAS: Semantic-Aware Graph-Assisted Stitching for Offline Temporal Logic Planning
- DexSeed: Expanding Human Hand-Object Interactions into Robot Demonstrations for Dexterous Policy Learning
- LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning
- CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation