Fiber Optic Sensing Glove for High Performance Dexterous Manipulation Capture

J.D. Peiffer^1,2*, Taylor Niehues¹, Li Guan¹, Ziyi Kou¹, Ergys Ristani¹

¹Meta Reality Labs ²Northwestern University

^*Work done during an internship at Meta

IROS 2026 · Pittsburgh

arXiv Poster

Our fiber optic sensing glove (blue) keeps tracking whole-hand pose through occlusions that break marker-based motion capture (green) — with <5 mm fingertip error and no drift or magnetic interference.

4.9 mm

fingertip error during manipulation

60 Hz

real-time registration + IK

34%

of mocap frames occluded — the glove keeps tracking

~2 hours

5-subject dexterous manipulation dataset

Abstract

Capturing hand pose during dexterous manipulation remains difficult: vision-based methods degrade under occlusion and challenging lighting, while sensorized gloves, though occlusion-free, are prone to drift and magnetic interference and rarely match motion-capture accuracy.

We introduce a fiber optic sensing glove for full hand pose tracking that targets these failure modes, using multi-core shape-sensing fibers that capture each fiber’s full 3D shape rather than curvature alone. A novel pipeline registers each reconstructed fiber shape to a common hand reference frame, and a new inverse-kinematics solver reconstructs full hand pose at 60 Hz using curve constraints.

Benchmarked on a 2-hour dataset of dexterous object manipulation tasks across 5 subjects, the glove achieves 7.2 mm mean fingertip position error against motion capture ground truth, reduced to 4.9 mm by a one-time factory calibration of the fiber routing hub that transfers across users and sessions. These capabilities enable high-fidelity data capture and bimanual virtual teleoperation — both essential to advancing the robotics field.

Glove Design

Each finger is instrumented with a 125 µm multi-core optical fiber containing three sensing cores with 26 FBGs each, spaced every 1 cm. Each FBG reflects a wavelength that shifts with local strain; combining the three cores gives local curvature and bending direction, which are integrated into a 3D fiber shape at 1 mm resolution and 60 Hz. Because curvature comes from differential strain between cores, common-mode effects such as temperature largely cancel.

Fibers are fixed at a 3D-printed fixture near the fingernail but otherwise slide freely through stitched dorsal fabric loops, minimizing interference with natural motion. On the back of the hand, a routing hub guides each fiber through a tunnel of known geometry to a shared outlet; these tunnels are what let us register every fiber into a common reference frame.

Sensing glove with routing hub, routing tunnels, optic fibers, distal fixtures and fabric routing loops

Hand Pose Reconstruction

Shape registration. Each fiber shape is first reconstructed in its own local frame. We slide a window along the fiber to find the segment whose curvature best matches the routing-hub tunnel for that digit, then solve a closed-form Procrustes alignment to place the fiber in the hub frame. This runs independently for every fiber at every time step.

Inverse kinematics. We fit a scaled Momentum hand model (22 joint angles plus wrist pose) to the registered fibers using fingertip position and distal-direction constraints, point-to-spline constraints along each finger, dorsum constraints tied to the routing hub, temporal regularization, and joint limits, solved with a Gauss–Newton optimizer. Registration takes ~5 ms and IK ~0.1 ms per sample — well within the 16.7 ms budget for 60 Hz.

Shape registration of fibers from local frame to routing hub frame, followed by inverse kinematics with spline and distal direction constraints

Tracking Accuracy

Five participants each completed two sessions (with a full doff-and-don in between) of range-of-motion poses and functional manipulation tasks: buzz wire, box and blocks, cup stacking, in-hand object rotation, the 9-Hole Peg Test, and object squeezing. Ground truth came from 20 OptiTrack cameras and markers on the glove. We report Mean Keypoint Position Error over 21 hand landmarks (MKPE) and over fingertips only (F.MKPE).

Index and thumb fingertip position traces for mocap and glove, with the glove continuing to track during marker occlusion

Fingertip traces during the 9-Hole Peg Test (left) and in-hand object rotation (right). The glove keeps tracking through a period of mocap marker occlusion (grey).

Metric	Fiber-to-Marker Residual (mm)		MKPE (mm)		F.MKPE (mm)
Metric	Range of Motion	Manipulation	Range of Motion	Manipulation	Range of Motion	Manipulation
Base	3.3	3.4	5.3	5.9	6.5	7.2
Tunnel Aligned	1.7	2.1	4.4	4.4	4.5	4.9

One-Time Tunnel Alignment

The dominant source of residual error is fiber-to-hub registration, and it is largely a consistent spatial bias. A single rigid transform applied to the routing-hub tunnel templates corrects it, cutting fingertip error by about 2 mm and reducing error for every subject and session. In a preliminary analysis, an alignment estimated from a different participant and session performed within 0.1 mm of a matched one, so it can be treated as a one-time factory calibration.

Ground truth (green), original registration (red), and aligned registration (blue).

Histogram of fingertip landmark errors per subject and session before and after tunnel alignment

Robust to Occlusion

During manipulation, 34% of mocap frames had at least one marker occluded — and these poses are not extreme: simply grasping a coffee mug handle is enough to break marker-based tracking. Because the glove senses shape directly, it produces consistent hand poses through both self-occlusion and object occlusion.

Glove, glove reconstruction, and marker-based mocap under self occlusion and object occlusion

Real-Time Bimanual Teleoperation

Attaching a 6-DoF tracker to each routing hub places both hands in a shared world frame. Combined in real time with the glove's finger pose estimates, this lets a user open and close a virtual jar in VR with minimal effort — a step toward retargeting the glove for robotic teleoperation and collecting contact-rich demonstrations for robot learning.

Real hands (top left), glove reconstruction (bottom left), and the user opening a jar in VR (right).

BibTeX

@misc{peiffer_fiber_2026,
  title     = {Fiber {Optic} {Sensing} {Glove} for {High} {Performance} {Dexterous} {Manipulation} {Capture}},
  author    = {Peiffer, J. D. and Niehues, Taylor and Guan, Li and Kou, Ziyi and Ristani, Ergys},
  year      = {2026},
  month     = aug,
  publisher = {arXiv},
  doi       = {10.48550/arXiv.2608.24572},
  url       = {http://arxiv.org/abs/2608.24572},
  note      = {arXiv:2608.24572 [cs.RO]}
}