RTFF: Random-to-Target Fabric Flattening Policy using Dual-Arm Manipulator

Kai Tang1,2,†, Dipankar Bhattacharya1,2,†, Hang Xu1,2, Fuyuki Tokuda1,2, Norman C. Tien1,2, Kazuhiro Kosuge1,2
† These authors contributed equally to this work.
1The University of Hong Kong
2Centre for Transformative Garment Production
arXiv Code Coming soon

Abstract

Robotic fabric manipulation in garment production for sewing, cutting, and ironing requires reliable flattening and alignment, yet remains challenging due to fabric deformability, effectively infinite degrees of freedom, and frequent occlusions from wrinkles, folds, and the manipulator’s End‑Effector (EE) and arm. To address these issues, this paper proposes the first Random‑to‑Target Fabric Flattening (RTFF) policy, which aligns a random wrinkled fabric state to an arbitrary wrinkle‑free target state. The proposed policy adopts a hybrid Imitation Learning–Visual Servoing (IL–VS) framework, where IL learns with explicit fabric models for coarse alignment of the wrinkled fabric toward a wrinkle‑free state near the target, and VS ensures fine alignment to the target. Central to this framework is a template‑based mesh that offers precise target state representation, wrinkle‑aware geometry prediction, and consistent vertex correspondence across RTFF manipulation steps, enabling robust manipulation and seamless IL–VS switching. Leveraging the power of mesh, a novel IL solution for RTFF—Mesh Action Chunking Transformer (MACT)—is then proposed by conditioning the mesh information into a Transformer-based policy. The RTFF policy is validated on a real dual‑arm tele‑operation system, showing zero‑shot alignment to different targets, high accuracy, and strong generalization across fabrics and scales.

Supplemental Video

BibTeX

@misc{tang2025,
      title={RTFF: Random-to-Target Fabric Flattening Policy using Dual-Arm Manipulator}, 
      author={Kai Tang and Dipankar Bhattacharya and Hang Xu and Fuyuki Tokuda and Norman C. Tien and Kazuhiro Kosuge},
      year={2025},
      eprint={2510.00814},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2510.00814}, 
}