Apple: Toward General Active Perception via Reinforcement Learning

arXiv:2505.06182 · cs.RO, cs.LG · Submitted 2025-05-09 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: We're looking at the paper: "Apple: Toward General Active Perception via Reinforcement Learning".

Dev: Here's the gist of the paper. Active perception is framed as a supervised learning problem where an agent must actively gather data from an interactive environment to infer properties like object classes.

Rosa: APPLE is a novel framework that jointly trains a transformer-based perception module and decision-making policy with a unified optimization objective to learn how to actively gather information.

Dev: This approach is designed to be general and not limited to specific tasks, allowing it to be applied across a wide range of active perception problems.

Rosa: Problem Formulation Active perception is formally defined as a special case of a Partially Observable Markov Decision Process (POMDP) where the agent must act under uncertainty to actively reduce ambiguity about a target property.

Paper summary: Rosa: So what they're actually doing here is defining this objective function J of pi as something that maximizes the expected discounted return based on this combined reward structure >

Dev: The gradient of that objective function splits into two parts: a policy gradient and a negative supervised prediction loss gradient >

Taro: That’s interesting because it means they can optimize the agent's behavior using policy gradient methods while also using the prediction loss to guide learning about what properties are actually there >

Rosa: The total reward is defined as r of ht, asterisk yt, at and yt as the RL reward minus a differentiable prediction loss >

Dev: This structure allows them to optimize things like the agent's behavior based on both how well it’s making decisions and how accurately it predicts what’s around it >

Taro: It moves away from just maximizing information gain in a task-specific way, aiming for something more general across different problems >

Rosa: The main idea is that this joint training helps the agent learn a way to actively gather data across many different active perception scenarios rather than being limited to one specific task >

Dev: It’s designed to be general and not limited to specific tasks, allowing it to be applied across a wide range of active perception problems >

Taro: But they're still dealing with the fact that active perception is usually about sensing something local, which is different from vision where you can get wider information coverage from one observation >

Rosa: So while tactile sensing poses distinct challenges because each contact only gives a small glimpse of the environment, this framework tries to solve that locality issue using reinforcement learning >

Dev: They look at related work like vision and touch sensing, but they point out that most methods are tailored to a specific objective or use greedy information-gain heuristics >

Taro: The paper suggests there isn't really an active perception method shown to work on a wide range of problems yet because the existing ones are too task-specific >

Rosa: So APPLE is trying to bridge that gap by using this joint training signal to learn a more versatile way to perceive and interact with the world >

Dev: The structure of the reward function, r of ht, asterisk yt, at and yt, is what allows it to handle both the uncertainty about what’s hidden and the actions taken >

Taro: It's like they are teaching the agent how to play a game where winning means finding information you didn't know was there before >

Rosa: So for someone just listening, it means we might see agents that can actively search for things in physical spaces without being explicitly told what the goal is for every single step >

Conclusion: Rosa: So to wrap up on Apple: Toward General Active Perception via Reinforcement Learning, the paper is about this novel framework that uses reinforcement learning to address a wide range of different active perception problems >

Dev: The authors are aiming for generality by jointly training a transformer-based perception module and the decision-making policy with a unified objective function >

Taro: It’s trying to move beyond methods that are tailored to specific tasks, which is what they see as a gap in the existing research area >

Rosa: The main implication is that this approach could be applied across many different active perception problems, not just one or two specific ones like shape estimation or grasping >

Dev: It suggests a way to learn how to actively gather information in an environment without having to manually design a completely new algorithm for every single problem >

Taro: It’s about learning the skill of active perception itself rather than just optimizing the final outcome for one fixed goal >

Rosa: The authors also show that their approach can be compared against other methods like APPLE-SAC and APPLE-CrossQ, which are variants based on actor critic methods >

Dev: They found that APPLE-CrossQ can be more robust to new environments without needing extensive hyperparameter tuning compared to the APPLE-SAC variant >

Taro: So while they've shown promise across five benchmarks including classification, volume estimation, and localization tasks, there's still a limitation mentioned about the high amount of training data required for tactile perception tasks >

Rosa: That’s true; they state it needs up to five million steps for those tactile perception tasks because of the combination of the transformer architecture and RL optimization >

Dev: And their future work will focus on improving sample efficiency, perhaps by leveraging pre-trained transformer models or looking at multi-modal integration with vision and touch >

Taro: The paper shows that while the core concept is strong for generalization, making it practical for real-world deployment across all environments is still a question >

Tim Schneider, Cristiana de Farias, Roberto Calandra, Liming Chen, Jan Peters

Department of Computer Science, TU Darmstadt, Germany · LIRIS, CNRS UMR5205, Ecole Centrale de Lyon, France · LASR Lab & CeTI, TU Dresden, Germany · DFKI · Hessian.AI · RIG · Centre for Cognitive Science

cs.RO, cs.LG

Submitted: 2025-05-09

Updated: 2026-05-11

Comments: 27 pages; 21 figures; accepted at the Fourteenth International Conference on Learning Representations (ICLR 2026)

Journal ref: International Conference on Learning Representations (ICLR 2026)

Code: https://github.com/ax-ml/jax

Project page: https://timschneider42.github.io/apple

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 85/100

The gist: Active perception is framed as a supervised learning problem where an agent must actively gather data from an interactive environment to infer properties like object classes.

Key concepts

Active Perception
This is when an agent must actively gather information from its environment to reduce uncertainty about a hidden property, like an object's class. It's treated as a decision-making problem where the agent chooses actions not just to move, but specifically to learn what it needs to know.
POMDP
Partially Observable Markov Decision Process is the formal mathematical framework used here. It describes a situation where the agent cannot see the true state of its environment perfectly. The hidden state includes both what is directly observable and unknown properties, requiring strategic actions under uncertainty.
Unified Optimization Objective
APPLE combines two main goals: maximizing expected discounted return from an RL policy and minimizing a prediction loss for inferred properties. This joint training allows the agent to learn how to gather information (prediction loss) while simultaneously learning the best way to act based on that gathered knowledge (RL reward).
Prediction Loss ($\ell$)
This is a differentiable loss function used during training that measures how well the agent's prediction of an unknown property matches the actual environment state. It guides the agent to actively seek information, as minimizing this loss encourages the agent to explore and gather data that helps refine its understanding.

Terminology

Summary

Active perception is framed as a supervised learning problem where an agent must actively gather data from an interactive environment to infer properties like object classes. p. 3

Where the paper says it

However, unlike in classical supervised learning, we assume that the agent is not simply presented with a static data point as input, but rather with an interactive environment that it can actively gather data from. , the agent could be presented with an object and has to decide actively how to examine it to extract the information it needs.

APPLE is a novel framework that jointly trains a transformer-based perception module and decision-making policy with a unified optimization objective to learn how to actively gather information. p. 0
Where the paper says it

To address this gap, this work introduces APPLE (Active Perception Policy Learning) – a novel framework that leverages reinforcement learning (RL) to address a range of different active perception problems. APPLE jointly trains a transformer-based perception module and decision-making policy with a unified optimization objective, learning how to actively gather information.

This approach is designed to be general and not limited to specific tasks, allowing it to be applied across a wide range of active perception problems. p. 0
Where the paper says it

By design, APPLE is not limited to a specific task and can, in principle, be applied to a wide range of active perception problems. We evaluate two variants of APPLE across different tasks, including tactile exploration problems from the Tactile MNIST benchmark.

Problem Formulation

Active perception is formally defined as a special case of a Partially Observable Markov Decision Process (POMDP) where the agent must act under uncertainty to actively reduce ambiguity about a target property. The problem is framed such that the hidden state decomposes into h˜t = (ht, ∗yt), where ht is the remainder of the hidden state without the ground truth property value. p. 4

Where the paper says it

Preprint decomposes into h˜ t = (ht, ∗ yt ), where ht is the remainder of the hidden state without the ground truth property value. Additionally, the agent’s action space contains not only control actions at but also a current estimate yt of the desired environment property.

The action space decomposes into a˜t = (at, yt), meaning that the agent predicts the desired environment property at every step. p. 4
Where the paper says it

Additionally, the agent’s action space contains not only control actions at but also a current estimate yt of the desired environment property. In other words, the action space decomposes into a˜t = (at, yt), meaning that the agent predicts the desired environment property at every step.

The overall reward function r˜ consists of two parts: a differentiable prediction loss l and an RL reward r. p. 4
Where the paper says it

, a desired finger movement, which is communicated to the agent’s motor controllers. The overall reward function r˜ consists of two parts: a differentiable prediction loss ℓ and an RL reward r.

Training Signal Construction

The objective function J(π) is formulated to maximize the expected discounted return based on the combined reward structure. The gradient of the objective function J(πθ) decomposes into a policy gradient and a negative supervised prediction loss gradient. p. 4

Where the paper says it

As can be seen in Eq. (3), the gradient of the objective function J(πθ) decomposes into a policy gradient and a negative supervised prediction loss gradient.

This decomposition allows for optimization via policy gradient methods augmented by the prediction loss gradient. p. 5
Where the paper says it

Preprint to dynamically recompute the total reward when evaluating the Bellman residual, yielding Lcritic = ED [ 1 2 (Qθ(o0∶t, at) − (rt − ℓπθ ( ∗ yt , o0∶t) + γEπθ [Qθ¯(o0∶t+1, at+1)]))2 ] . During the policy update, the policy gradient is augmented by the prediction loss gradient.

The total reward is defined as r˜(ht, ∗yt, at, yt) = r(ht, at) − l(∗yt, yt). p. 5
Where the paper says it

Preprint to dynamically recompute the total reward when evaluating the Bellman residual, yielding Lcritic = ED [ 1 2 (Qθ(o0∶t, at) − (rt − ℓπθ ( ∗ yt , o0∶t) + γEπθ [Qθ¯(o0∶t+1, at+1)]))2 ] . During the policy update, the policy gradient is augmented by the prediction loss gradient.

Method Variants and Comparison

Two variants of APPLE are proposed based on actor-critic methods: APPLE-SAC and APPLE-CrossQ. These methods utilize off-policy RL to estimate the policy gradient in Eq. p. 4

Where the paper says it

3 DERIVING APPLE-SAC AND APPLE-CR O S SQ We use RL-based techniques to estimate the policy gradient in Eq. Here we propose two variations of APPLE based on two actor-critic methods: SAC Haarnoja et al.

(3). CrossQ is similar to SAC but drops target networks and uses BatchRenorm Ioffe layers for stabilization. p. 4
Where the paper says it

To stabilize the training of the Q-networks, SAC deploys target networks, which slowly track the actual Q-networks. CrossQ is similar to SAC but drops the target networks and instead uses BatchRenorm Ioffe (2017) layers in the Q-network to stabilize their training.

Experiments show that APPLE-CrossQ can be more robust to new environments without hyperparameter tuning compared to APPLE-SAC. p. 9
Where the paper says it

In contrast, we employ off-policy methods (APPLE-SAC and APPLE-CrossQ), enabling sample reuse – a critical factor in active perception, where supervised learning benefits from multiple passes over the same data. Both APPLE-SAC and APPLE-CrossQ perform comparably on environments they have been tuned on, but APPLE-CrossQ has proven more robust to new environments without hyperparameter tuning and offers a clear computational advantage.

Empirical Evaluation Across Tasks

APPLE was evaluated across five benchmarks including classification, volume estimation, and localization tasks. The experiments on the tactile domain showed that APPLE consistently outperforms baselines like HAM and APPLE-RND across CircleSquare, TactileMNIST classification, TactileMNISTVolume regression, and Toolbox pose estimation. p. 8

Where the paper says it

5 DISCUSSION The results in Section 4 show that APPLE successfully learns exploration strategies across diverse active perception tasks. We evaluated APPLE on the CircleSquare classification task and three simulated tactile benchmarks: TactileMNIST (digit classification), TactileMNISTVolume (volume estimation), and Toolbox (pose estimation).

For instance, in the Toolbox task, APPLE-CrossQ reached a final accuracy of 1. p. 7
Where the paper says it

APPLE-RND, achieves a 68% accuracy, highlighting the need for active perception in the CircleSquare task. Despite extensive tuning of the learning rate and β parameter with HEBO and training for 10M steps, we could not find a configuration for which HAM reached a prediction accuracy beyond random guessing.

9cm and 13○ on average while HAM required a large amount of samples to learn an effective policy. p. 6
Where the paper says it

While HAM is generally able to solve this task (see Section C. 5 for a longer run), it requires a large amount of samples to learn an effective policy.

Limitations and Future Directions

A primary limitation of APPLE is its reliance on large amounts of training data, requiring up to 5M steps for the tactile perception tasks. This high data requirement stems from the combination of a transformer-based architecture and RL-based policy optimization. p. 5

Where the paper says it

2, the agent receives this exact combination of tactile images and sensor positions. To efficiently process this data in the policy and Q-networks, we use an architecture similar to the Video-Vision-Transformer (ViViT) Arnab et al.

Future work will focus on improving sample efficiency by leveraging pre-trained transformer models or exploring multi-modal integration with vision and touch. p. 9
Where the paper says it

These results demonstrate APPLE’s versatility across diverse tasks, including both classification and regression. Future work will focus on improving the sample efficiency of APPLE, extending to more realistic applications such as in-hand pose estimation and texture recognition, and exploring multi-modal integration with vision and touch.

The framework supports multi-fingered robotic hands and multi-modal perception but its practical scalability remains an open question. p. 9
Where the paper says it

Future work will focus on improving the sample efficiency of APPLE, extending to more realistic applications such as in-hand pose estimation and texture recognition, and exploring multi-modal integration with vision and touch. Another potential future research avenue will be deploying APPLE on a real robotic system.

Hyperparameter Optimization

Extensive hyperparameter searches using the HEBO Cowen-Rivers et al. (2022) Bayesian optimizer were performed to tune parameters for APPLE-SAC and APPLE-CrossQ. p. 8

Where the paper says it

It is important to note that we again did not tune any of these methods on this task and instead relied on hyperparameters optimized for TactileMNIST. While tuning APPLE-SAC on this task can lead to stronger performance, these results indicate that APPLE-CrossQ might be more robust w.

The search provided valuable insights for APPLE-SAC and APPLE-CrossQ, demonstrating greater robustness when transferred to unseen environments. p. 1
Where the paper says it

This formulation enables adaptability across different tasks by making minimal assumptions about the nature of the underlying POMDP. • Comprehensive empirical evaluation of two method variants, based on SAC and CrossQ, on classification, volume estimation, and localization tasks, demonstrating that APPLE can discover active exploration policies without task-specific heuristics.

Applying APPLE-CrossQ’s vision-encoder hyperparameters to CircleSquare produced no measurable degradation in performance. p. 8
Where the paper says it

It is important to note that we again did not tune any of these methods on this task and instead relied on hyperparameters optimized for TactileMNIST. While tuning APPLE-SAC on this task can lead to stronger performance, these results indicate that APPLE-CrossQ might be more robust w.

The final hyperparameter settings selected by HEBO summarized the findings for the methods. p. 6
Where the paper says it

For regression tasks, we report average error and final error. The hyperparameters of all methods are tuned with HEBO Cowen-Rivers et al.

Connection to Intrinsic Reward Methods

APPLE is concerned with learning hidden properties of the environment within an episode rather than using intrinsic reward bonuses to enhance policy learning across episodes. The exploration procedure of APPLE takes place within each episode, while intrinsic reward methods occur across episodes. These two concepts are fully orthogonal and could potentially be combined to speed up learning progress. p. 3

Where the paper says it

In the remainder of this section, we formally define our problem and derive the Active Perception Policy Learning (APPLE) framework. We propose two variants of APPLE, based on SAC Haarnoja et al.

Role of Prediction Loss in Downstream Tasks

The prediction loss can help in the presence of another downstream task by using a tracking-aware loss function l((∗y, wt), yt) = wtMSE(∗yt, yt). This allows only successful tracking steps to be considered in the prediction loss. p. 4

Where the paper says it

Computing the gradient of J(πθ) now yields ∂ ∂θ J(πθ) = E pθ (h, ∗ y,o,a) [ ∂ ∂θ lnπθ(a ∣ o) ∞ ∑ t=0 γ t r˜(ht, ∗ yt , at, yt)] ´ ¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¸¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¶ policy gradient − E pθ ( ∗ y,o) [ ∞ ∑ t=0 γ t ∂ ∂θ ℓπθ ( ∗ yt , o0∶t)] ´ ¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¸¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¶ prediction loss gradient . (3) Intermediate steps can be found in Section G.

Incorporating label and loss-function information into the reward can guide agents toward discovering relevant information for solving downstream tasks like finding a tool from a toolbox. The regular agent significantly outperformed the agent relying on pure RL in this variant of CircleSquareHideAndSeek. p. 1
Where the paper says it

Ideally, the formulation of such methods should be agnostic to the underlying task, enabling transfer across modalities and problem settings. Thus, in this work, we ask the following: can we design a principled RL-based algorithm for discovering active perception policies by relying only on a ground-truth label and a differentiable loss during training?

Model Architecture Influence

The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. p. 9

Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

p. 9
Where the paper says it

Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks. 6 CONCLUSION We introduced APPLE, a framework combining reinforcement learning and transformer-based models for active tactile perception.

The transformer-based architecture outperformed the LSTM architecture when used with a 20% of the image width input for CircleSquare. p. 5
Where the paper says it

2, the agent receives this exact combination of tactile images and sensor positions. To efficiently process this data in the policy and Q-networks, we use an architecture similar to the Video-Vision-Transformer (ViViT) Arnab et al.

The off-policy algorithms like APPLE-CrossQ demonstrated superior sample efficiency compared to PPO variants despite using the same model for policy and class predictions. p. 9
Where the paper says it

(2020), which failed to learn an effective strategy even on the Circle-Square toy task, defaulting to predicting the mean class despite extensive tuning using HEBO. This points to a fundamental limitation of HAM in this setting, namely that HAM relies on on-policy RL, which discards samples after a single update, limiting sample efficiency.

The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

p. 8
Where the paper says it

6 show that APPLE-CrossQ reaches a final accuracy of 1. 9cm and 13○ on average, while APPLE-SAC and the random baseline APPLE-RND stagnate at much lower accuracies.

The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. The final prediction accuracies show that APPLE-CrossQ outperforms APPLE-PPO by orders of magnitude in sample-efficiency. The choice of model can have a significant influence on APPLE’s overall performance, as shown by the ablation study comparing transformer and LSTM models. Th…

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Improved AI systems can learn active perception policies for tactile sensing by jointly training a transformer-based perception module and a decision-making policy with a unified optimization objective, learning how to actively gather information. This allows the system to address various active perception problems without being limited to specific tasks or strong assumptions, as stated in the abstract.

  2. The system can perform classification, regression, and pose estimation tasks on tactile data by formulating active perception within the setting of partially observable Markov decision processes (POMDPs), where it must act under uncertainty to actively reduce ambiguity about a target property. This approach allows the framework to be general enough to extend across diverse problems without requiring task-specific exploration heuristics.

  3. The system can discover active exploration policies by relying only on a ground-truth label and a differentiable loss during training, as the objective is framed as minimizing a loss l(yt, y∗t) between its current prediction yt and the ground-truth label y∗t. This enables the learning of an agent's ability to actively gather data from an interactive environment.

  4. The system can utilize off-policy reinforcement learning methods like APPLE-SAC and APPLE-CrossQ, which are noted to be more robust to new environments without hyperparameter tuning compared to on-policy methods like HAM. This suggests a more practical framework for deploying active perception in real robotic systems.

  5. The system can learn complex exploration strategies, such as finding the handle of a wrench and then sliding along it for orientation disambiguation in pose estimation tasks, demonstrating that learned behaviors can be intuitive from a human perspective. This capability is shown to be effective across different environments like the Toolbox task.

Abstract

Active perception is a fundamental skill that enables us humans to deal with uncertainty in our inherently partially observable environment. For senses such as touch, where the information is sparse and local, active perception becomes crucial. In recent years, active perception has emerged as an important research domain in robotics. However, current methods are often bound to specific tasks or make strong assumptions, which limit their generality. To address this gap, this work introduces APPLE (Active Perception Policy Learning) - a novel framework that leverages reinforcement learning (RL) to address a range of different active perception problems. APPLE jointly trains a transformer-based perception module and decision-making policy with a unified optimization objective, learning how to actively gather information. By design, APPLE is not limited to a specific task and can, in principle, be applied to a wide range of active perception problems. We evaluate two variants of APPLE across different tasks, including tactile exploration problems from the Tactile MNIST benchmark. Experiments demonstrate the efficacy of APPLE, achieving high accuracies on both regression and classification tasks. These findings underscore the potential of APPLE as a versatile and general framework for advancing active perception in robotics. Project page: https://timschneider42.github.io/apple

Sources

Related papers