IROSA: Interactive Robot Skill Adaptation using Natural Language

View PDF HTML (experimental) Abstract:Foundation models have demonstrated impressive capabilities across diverse domains, while imitation learning provides principled methods for robot skill adaptation from limited data. Combining these approaches holds significant promise for direct application to robotics, yet this combination has received limited attention, particularly for industrial deployment. We present a novel framework that enables open-vocabulary skill adaptation through a tool-based architecture, maintaining a protective abstraction layer between the language model and robot hardware. Our approach leverages pre-trained LLMs to select and parameterize specific tools for adapting robot skills without requiring fine-tuning or direct model-to-robot interaction. We demonstrate the framework on a 7-DoF torque-controlled robot performing an industrial bearing ring insertion task, showing successful skill adaptation through natural language commands for speed adjustment, trajectory correction, and obstacle avoidance while maintaining safety, transparency, and interpretability. Comments: Accepted IEEE Robotics and Automation Letters (RA-L) journal, 8 pages, 5 figures, 3 tables, 1 listing. Code available: this https URL Subjects: Robotics (cs.RO); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG) Cite as: arXiv:2603.03897 [cs.RO] (or arXiv:2603.03897v3 [cs.RO] for this version) https://doi.org/10.48550/arXiv.2603.03897 arXiv-issued DOI via DataCite Journal reference: IEEE Robotics and Automation Letters (RA-L), 2026 Related DOI: https://doi.org/10.1109/LRA.2026.3671560 DOI(s) linking to related resources Submission history From: Markus Knauer [view email] [v1] Wed, 4 Mar 2026 09:54:09 UTC (6,504 KB) [v2] Fri, 13 Mar 2026 09:00:04 UTC (4,277 KB) [v3] Thu, 16 Apr 2026 15:37:03 UTC (6,504 KB)