Control interfaces for human-robot collaboration in extended reality

Details

Supervisors

Ronsse, Renaud ; Macq, Benoît ; El Hafi, Lotfi ; Garcia Ricardez, Gustavo Alfonso

Faculty

Ecole polytechnique de Louvain

Degree label

Master [120] : ingénieur civil électromécanicien, à finalité spécialisée: mécatronique

Abstract

enNowadays, robots and automated machines are deployed in many environments, such as industries and domestic households. Current developments are aligned with initiatives such as Society 5.0 and the Moonshot R&D Program. A complete fusion of the physical and cybernetic spaces, while focusing on human needs and overcoming the physical, spatial and temporal limitations of our bodies, is promoted by these projects to solve our society’s ethical and economic issues. Despite notable progress in service robotics, public acceptance of autonomous robots remains a strong barrier for development, due to their cost, the lack of transparency in their decisions, as well as their unreliability in unpredictable environments. Human-Robot Collaboration (HRC) through semi-autonomous control provides a middle ground, offering human guidance in tasks of high computational complexity. However, conventional control methods are still often inaccessible to novice users. Extended Reality (XR) presents promising tools to augment robots and their surroundings, allowing users to visualise overlaid robots’ information in real-world environments and interact with augmented elements, using an XR headset as one of the possible augmenting devices. Although this technology induced the development of XR-based control methods that have shown promising results in enhancing efficiency in HRC tasks, these new control methods remain unintuitive for non-expert users. Among the existing XR control devices associated with such interfaces, XR motion controllers and hand-tracked inputs are two common techniques. The XR pen, a pen-like XR controller, is another type of device that has shown great potential in improving the intuitiveness of XR-based control interfaces, however, it has been rarely included in recent comparative studies for robot control in XR. This thesis presents a novel interface using the XR pen device to control a semi-autonomous mobile robot in Augmented Reality (AR), by selecting navigation goals. When using the interface, the operator points at a position in the room, selects it and drags an augmented arrow (that appeared at the selection position) to the orientation he/she wants the robot to be when it has reached that destination. Additionally, two other interfaces have been developed using the same framework for two other XR control devices: the XR motion controllers and the operator’s hands being tracked as XR controllers. To assess the performance and users’ perceptions of these XR control interfaces, and of the XR pen in particular, this thesis presents a comparative study in which these three XR-based control interfaces are evaluated against a conventional computer-based control interface, used as a baseline in this study. The results show that the XR pen significantly outperforms the other XR-based and computer-based control devices in task performance, but does not improve the user experience compared to the other devices.