EN / EO

PROPRA

Self-Supervised Anchoring of Fingertip Sensing to Proprioception and Proactive Actions for Robot Imitation Learning

PROprioceptive-and-PRoactive Anchoring

Tomohiro Motoda1, Masaki Murooka2, Keisuke Shirai1, Hanbit Oh1, Ryoichi Nakajo1, Shotaro Miwa1, Roman Mykhailyshyn1, Hugo Duarte1, and Yukiyasu Domae1

1 National Institute of Advanced Industrial Science and Technology (AIST), Japan
2 CNRS-AIST Joint Robotics Laboratory (JRL), Japan

arXiv/Code (soon)

Teaser video static/videos/teaser.mp4

Fingertip sensors speak only in bursts: proximity just before contact, tactile just after. PROPRA anchors each of them to something the robot always has — its current state and the actions it is about to execute.

Abstract

Robotic imitation learning often relies on external cameras, yet local interaction cues such as object proximity, contact onset, and grasp state are difficult to observe near the fingertips because of occlusion and limited temporal resolution. We study how to effectively incorporate complementary fingertip sensing into imitation learning using pressure-sensitive tactile and reflective proximity sensors, along with pretrained sensor encoders.

The two modalities provide information at different manipulation phases: proximity sensing is informative before contact, whereas tactile sensing becomes informative after contact. However, naively adding these signals to a policy does not consistently improve performance and can even underperform vision-only policies, suggesting that sparse, phase-dependent sensor signals are difficult to exploit from limited demonstrations.

We therefore propose a proprioception-anchored pretraining method, PROprioceptive-and-PRoactive Anchoring (PROPRA), which independently aligns each fingertip sensor history with proprioceptive and action segments. This provides a continuously available sensorimotor reference, allowing each sensor to be aligned independently during its informative phases.

Experiments on real-world manipulation tasks show that our pretraining method improves average success rates over vision-only policies and image-anchored pretraining baselines. Representation analysis further shows that it preserves richer information about pre-contact states, enabling more effective use of complementary fingertip sensing.

Method

Two pretraining schemes. (a) Image-anchored alignment ties sensor histories to image embeddings at the current time step. (b) PROPRA ties each sensor history to an anchor built from the current state and the forthcoming action chunk, with no direct sensor-to-sensor objective.
(a) Image-anchored pretraining lines each sensor up with what the camera sees now. (b) PROPRA lines it up with the robot’s own state and its next actions — a reference that exists at every step.

Each fingertip history — 0.5 s of 3 tactile and 6 proximity channels — gets its own encoder. The anchor is the current end-effector pose and gripper opening plus the next 10 actions: defined at every step, and needing no camera. A contrastive loss pulls each sensor embedding toward that anchor; the encoders then go into a diffusion policy and the anchor is dropped.

Results

Four tasks on a UR5e, 20 demonstrations each. PROPRA beats vision-only policies and image-anchored pretraining on average success rate, and keeps more pre-contact information in the proximity representation. The numbers are in the paper.

Video

Supplementary video static/videos/supplementary.mp4

BibTeX

@misc{motoda2026propra,
      title={Self-Supervised Anchoring of Fingertip Sensing to Proprioception and Proactive Actions for Robot Imitation Learning}, 
      author={Tomohiro Motoda and Masaki Murooka and Keisuke Shirai and Hanbit Oh and Ryoichi Nakajo and Shotaro Miwa and Roman Mykhailyshyn and Hugo Duarte and Yukiyasu Domae},
      year={2026},
      eprint={2609.29822},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2609.29822}, 
  }