Post doc - Remote technician assistance via multimodal interfaces F/M

about the role

Your role is to carry out a Post PhD on the theme of Remote Assistance of a technician via multimodal interfaces as part of the 5GMetaverse project financed by the BPI.

The ambition of the 5GMetaverse project is to enable the emergence of innovative products, services and solutions based on 5G that will revolutionize industrial services, particularly in terms of remote collaboration and teleworking in the industrial world. Today, augmented reality (AR), virtual reality (VR) and the Metaverse make it possible to interconnect a physical space with its digital model in real time. The synchronization achieved through the digital twin enables interactions performed in real or virtual space to impact both environments, enabling different users to communicate and collaborate remotely. In the context of the augmented technician, remote experts can assist field technicians in assembly, maintenance or inspection operations. The democratization of VR and AR peripherals, the development of low-latency, high-speed networks and the growth of the Internet of Things in the industrial sector are all factors that will increase interest in and enhance these remote assistance solutions.

The goal of the Post PhD is to explore new metaphors for interaction, visualization and remote control, enabling a user to assist a field technician. Recent work has shown the limitations of current remote assistance systems in terms of the modalities exploited. Compared to the state of the art, the aim is to propose multimodal metaphors, i.e. exploiting AR but also ambient interactions through the digital twin, based on the various sensors and actuators in the real world (presence sensors, screens, lighting systems, robots, drones, etc.).  This type of multimodal metaphors can have multiple advantages, such as eliminating the need for fragile and expensive AR equipment and leaving the operator's hands free, acting on the technician's environment rather than just guiding it, and finally making human-machine interactions more reliable and removing communication ambiguities. The key will be to find the right combination of modalities to deliver these benefits.

In the context of this Post PhD you will contribute to the proposal of new multimodal remote assistance metaphors, the development the associated software components and the evaluation of their impact on improving communication and collaboration from a user point of view.