Fluently - the essence of human-robot interaction

Informations projet

Fluently

N° de convention de subvention: 101058680

DOI

10.3030/101058680

Projet clôturé

Date de signature de la CE 30 Mai 2022

Date de début 1 Juin 2022

Date de fin 31 Mai 2025

Financé au titre de

• Digital, Industry and Space

Coût total

€ 11 200 141,55

Contribution de l’UE

€ 9 176 043,50

Coordonné par

REPLY DEUTSCHLAND SE
Germany

Periodic Reporting for period 1 - Fluently (Fluently - the essence of human-robot interaction)

Période du rapport: 2022-06-01 au 2023-11-30

Résumé du contexte et des objectifs généraux du projet

Fluently leverages the latest advancements in AI-driven decision-making process to achieve true social collaboration between humans and machines while matching extremely dynamic manufacturing contexts.
The project results will be: 1) Fluently Smart Interface unit and 2) the Robo-Gym. The Fluently Smart Interface unit features: 1) interpretation of speech content, speech tone and gestures, automatically translated into robot instructions, making industrial robots accessible to any skill profile; 2) assessment of the operator’s state through a dedicated sensors’ infrastructure that complements a persistent context awareness to enrich an AI-based behavioral framework in charge of triggering the generation of specific robot strategies; 3) modelling products and production changes in a way they could be recognized, interpreted and matched by robots in cooperation with humans.
Robots equipped with Fluently will constantly embrace humans’ physical and cognitive loads, but will also learn and build experience with their human teammates to establish a manufacturing practice relying upon quality and wellbeing. The bond between human and robot is personalized, and it is established during a preliminary training at the Robo-Gym, the first European hub for human-robot interactive training, where human and robot learn from each other a common work practice. Fluently will be validated by demonstration on three full scale pilots characterized by various level of automation, production dynamism and complexity of the manufacturing decision making process.
The total project value required to commercialize the Fluently device and Robo-Gym concept is 18.8 M EUR. Forecast for EBITDA in 2028 is 16.3 M EUR, payback in 2028, with 13% gross profit and ROI 91% in line with sectorial performances. The new turnover generated requires at least 325 workers employed along the Fluently supply chain. Fluently will bring together 21 key industrial and academic stakeholders from 13 countries.

Travail effectué depuis le début du projet jusqu’à la fin de la période considérée dans le rapport et principaux résultats atteints jusqu’à présent

FLUENTLY proposes a radically new vision – we call Ethical Manufacturing – aiming at preserving the central role of humans, thus nourishing workers’ competencies and skills while reducing the cognitive and physical load triggered by production practices.
Fluently main achievements so far are:
- The development of FLUENTLY Smart Interface unit designed to enable frictionless use of robots. This includes the Human-Fluently AI based wearable device for the operator and the Robot-Fluently hardware/software extension, providing cognitive/behavioural capacities to industrial robots and allowing them to understand, communicate and learn from humans.
- FLUENTLY defined, designed, and developed the Artificial Intelligence functionalities that are needed to ensure better and more natural communication and cooperation between the human operator and the robot.
- Moreover FLUENTLY developed a conceptual framework highlighting the importance of human-centered variables when designing training to facilitate the acquisition of knowledge and skills required to collaborate with robots.
- The Robo-Gym has been set-up, with two main purposes: it will be a place where Fluently adopters, who are common workmen, will be trained and taught to use robots and to collaborate with them, and at the same time it will be the place where Machine Learning models and more in general AI functionalities will be trained, optimized and tested with real people and industrial tasks.

Progrès au-delà de l’état des connaissances et impact potentiel prévu (y compris l’impact socio-économique et les conséquences sociétales plus larges du projet jusqu’à présent)

The Fluently solution and related training experience is uniquely positioned in the current technical and industrial spectrum as a result of the following set of key competitive leverages:

- Industrial Smart Interface unit with AI dedicated processors.
Fluently would set a ground-breaking product streamline for manufacturing integrating AI processors and communication interfaces to humans and robots in a compact device available for industrial adoption.

- AI for human-robot interaction and industrial robotics applications.
Fluently proposes a complete solution, integrating human-robot interaction in complex decision-making industrial processes based on personalized ML and multimodal sensor analysis.

- Training of humans with robots.
Fluently offers an immersive training ecosystem for human-robot collaboration, where the robots that will operate in the field will raise awareness about people, so that personalized behaviours can be shaped to enable people’s wellbeing together with production optimization.

- Mapping to cognitive/physical status.
Fluently targets the harnessing of a selection of physiological and behavioural signals generated by the human (i.e. voice tone, heartbeat, brain activity and eye movements) in response to specific stimuli. In a behavioural model of the human, these signals, only partially conscious, can be treated as objective predictors as they can potentially disclose precious and reliable information about the human experience. No industrial robot is currently capable of interpreting such a context and deliberate in autonomy to target quality and productivity while caring for the human workers.

H-Fluently software modules and communication with other devices.

Robo-Gym Dexterity and Application areas.

R-Fluently software modules and communication with other devices.