Fluently - the essence of human-robot interaction

Project Information

Fluently

Grant agreement ID: 101058680

DOI

10.3030/101058680

Project closed

EC signature date 30 May 2022

Start date 1 June 2022

End date 31 May 2025

Funded under

• Digital, Industry and Space

Total cost

€ 11 200 141,55

EU contribution

€ 9 176 043,50

Coordinated by

REPLY DEUTSCHLAND SE
Germany

Periodic Reporting for period 1 - Fluently (Fluently - the essence of human-robot interaction)

Reporting period: 2022-06-01 to 2023-11-30

Summary of the context and overall objectives of the project

Fluently leverages the latest advancements in AI-driven decision-making process to achieve true social collaboration between humans and machines while matching extremely dynamic manufacturing contexts.
The project results will be: 1) Fluently Smart Interface unit and 2) the Robo-Gym. The Fluently Smart Interface unit features: 1) interpretation of speech content, speech tone and gestures, automatically translated into robot instructions, making industrial robots accessible to any skill profile; 2) assessment of the operator’s state through a dedicated sensors’ infrastructure that complements a persistent context awareness to enrich an AI-based behavioral framework in charge of triggering the generation of specific robot strategies; 3) modelling products and production changes in a way they could be recognized, interpreted and matched by robots in cooperation with humans.
Robots equipped with Fluently will constantly embrace humans’ physical and cognitive loads, but will also learn and build experience with their human teammates to establish a manufacturing practice relying upon quality and wellbeing. The bond between human and robot is personalized, and it is established during a preliminary training at the Robo-Gym, the first European hub for human-robot interactive training, where human and robot learn from each other a common work practice. Fluently will be validated by demonstration on three full scale pilots characterized by various level of automation, production dynamism and complexity of the manufacturing decision making process.
The total project value required to commercialize the Fluently device and Robo-Gym concept is 18.8 M EUR. Forecast for EBITDA in 2028 is 16.3 M EUR, payback in 2028, with 13% gross profit and ROI 91% in line with sectorial performances. The new turnover generated requires at least 325 workers employed along the Fluently supply chain. Fluently will bring together 21 key industrial and academic stakeholders from 13 countries.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

FLUENTLY proposes a radically new vision – we call Ethical Manufacturing – aiming at preserving the central role of humans, thus nourishing workers’ competencies and skills while reducing the cognitive and physical load triggered by production practices.
Fluently main achievements so far are:
- The development of FLUENTLY Smart Interface unit designed to enable frictionless use of robots. This includes the Human-Fluently AI based wearable device for the operator and the Robot-Fluently hardware/software extension, providing cognitive/behavioural capacities to industrial robots and allowing them to understand, communicate and learn from humans.
- FLUENTLY defined, designed, and developed the Artificial Intelligence functionalities that are needed to ensure better and more natural communication and cooperation between the human operator and the robot.
- Moreover FLUENTLY developed a conceptual framework highlighting the importance of human-centered variables when designing training to facilitate the acquisition of knowledge and skills required to collaborate with robots.
- The Robo-Gym has been set-up, with two main purposes: it will be a place where Fluently adopters, who are common workmen, will be trained and taught to use robots and to collaborate with them, and at the same time it will be the place where Machine Learning models and more in general AI functionalities will be trained, optimized and tested with real people and industrial tasks.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The Fluently solution and related training experience is uniquely positioned in the current technical and industrial spectrum as a result of the following set of key competitive leverages:

- Industrial Smart Interface unit with AI dedicated processors.
Fluently would set a ground-breaking product streamline for manufacturing integrating AI processors and communication interfaces to humans and robots in a compact device available for industrial adoption.

- AI for human-robot interaction and industrial robotics applications.
Fluently proposes a complete solution, integrating human-robot interaction in complex decision-making industrial processes based on personalized ML and multimodal sensor analysis.

- Training of humans with robots.
Fluently offers an immersive training ecosystem for human-robot collaboration, where the robots that will operate in the field will raise awareness about people, so that personalized behaviours can be shaped to enable people’s wellbeing together with production optimization.

- Mapping to cognitive/physical status.
Fluently targets the harnessing of a selection of physiological and behavioural signals generated by the human (i.e. voice tone, heartbeat, brain activity and eye movements) in response to specific stimuli. In a behavioural model of the human, these signals, only partially conscious, can be treated as objective predictors as they can potentially disclose precious and reliable information about the human experience. No industrial robot is currently capable of interpreting such a context and deliberate in autonomy to target quality and productivity while caring for the human workers.

H-Fluently software modules and communication with other devices.

Robo-Gym Dexterity and Application areas.

R-Fluently software modules and communication with other devices.