Periodic Reporting for period 1 - FORTIS (Multi-Modal and Multi-Aspect Holistic Human-Robot Interaction)
Okres sprawozdawczy: 2024-01-01 do 2024-12-31
FORTIS aims to develop a multi-modal, human-centric HRI solution, enabling robots to seamlessly collaborate with humans. Unlike traditional automation, FORTIS fosters an intelligent, adaptive approach, integrating cognitive, emotional, and physical aspects of interaction. The project envisions an advanced Human-Robot Collaboration (HRC) ecosystem, where robots understand and respond to human behavior, enhancing workplace productivity and safety. To validate its technology, FORTIS will conduct large-scale demonstrations in real-world environments, including construction, infrastructure services, and manufacturing. Additionally, the project will explore civilian interaction scenarios in healthcare, public spaces, and retail, ensuring broad applicability and acceptance of HRI technologies. Finally, FORTIS seeks to ensure scalability and replicability by developing an open and modular framework for seamless integration with existing industrial and service-oriented robotic platforms. It will also establish guidelines and best practices for the adoption of human-centric HRI solutions, ensuring long-term sustainability and widespread implementation.
FORTIS is set to deliver transformative impacts across key areas:boosting workforce productivity and reducing labor shortages; minimizing injuries and fatigue by offloading physically and cognitively demanding tasks and advancing human-aware AI, cognitive robotics, and intelligent perception.
FORTIS integrates SSH to:
- Conduct human-centered design studies optimizing HRC.
- Evaluate emotional and psychological impacts of robotic interactions.
- Ensure ethical AI practices, including bias mitigation and transparency.
FORTIS is set to deliver transformative impacts across key areas:boosting workforce productivity and reducing labor shortages; minimizing injuries and fatigue by offloading physically and cognitively demanding tasks and advancing human-aware AI, cognitive robotics, and intelligent perception.
FORTIS integrates SSH to:
- Conduct human-centered design studies optimizing HRC.
- Evaluate emotional and psychological impacts of robotic interactions.
- Ensure ethical AI practices, including bias mitigation and transparency.
The main achievements for this period can be summarized as follows:
• Establishment of project management and governance structures.
• Completion of stakeholder and requirements analyses to define project needs and key performance indicators.
• Development of the FORTIS system architecture and technical planning to guide solution development.
• Initiation of dissemination activities to enhance project visibility and stakeholder engagement.
• Specification of the open calls management.
• Establishment of project management and governance structures.
• Completion of stakeholder and requirements analyses to define project needs and key performance indicators.
• Development of the FORTIS system architecture and technical planning to guide solution development.
• Initiation of dissemination activities to enhance project visibility and stakeholder engagement.
• Specification of the open calls management.
During this first reporting period, FORTIS has focused on the initial architecture and design of its toolkits, as well as the analysis of pilot use cases and stakeholder requirements. These foundational activities are essential for the next phase, which will concentrate on the implementation and integration of the first release of the FORTIS toolkits, followed by testing and validation in pilot environments.
In WP1 (Requirements Analysis and Planning), the project has successfully completed a comprehensive stakeholder and technical requirements analysis, documented in the FORTIS Requirements Analysis Report (D1.1). This deliverable consolidates insights from industry, academia, and end-users, ensuring alignment with real-world needs. Additionally, the project has defined the first version of the system architecture and technical specifications (D1.2) detailing the modular structure and data flow between components. To guide development efforts, a technical activities planning document (D1.3) has been prepared, establishing the roadmap for implementation, integration, and pilot demonstrations.
In WP2 (Human and AI Perspective), significant progress has been made in designing human-centric AI models that will enable robots to understand human behavior, cognitive states, and physical activities. The Human-Centred Data Collection Toolkit has been defined, integrating wearable and non-wearable sensors, such as IMUs, EEG, EDA, RGB cameras, LiDAR, and microphones, to gather multimodal data for human perception modeling. Initial research has also been conducted on AI models for real-time cognitive and physical state monitoring, laying the groundwork for adaptive human-robot interaction. Furthermore, the Multi-Modal Adaptive Communication Toolkit has been conceptualized, allowing robots to interact with humans through speech, gestures, and visual cues, ensuring intuitive and seamless collaboration.
In WP3 (Robotic and AI Perspective), the project has advanced in sensor integration and robotic perception, selecting and implementing LiDAR, stereo cameras, and depth cameras into ROS-enabled robotic platforms. The first version of the SLAM-based localization system has been developed, providing real-time navigation and mapping capabilities in dynamic environments. Additionally, the Robotic Intelligibility Toolkit has been designed to enable robots to adjust their behavior based on human interactions, improving predictability and safety. Another key achievement is the definition of the Multi-Robot Dynamic Reconfigurability Toolkit, which leverages AI-driven task allocation and coordination models to facilitate multi-robot collaboration in industrial settings.
In WP5 (Demonstration and Validation), the project has conducted a detailed analysis and refinement of the use cases for its three pilot sites in construction, infrastructure maintenance, and manufacturing. Initial pilot site visits have provided valuable insights into real-world requirements and constraints, guiding the technical developments in WP2 and WP3. A demonstration and validation plan (D5.1 - draft version) has been prepared, defining the evaluation methodology and key performance indicators (KPIs) for assessing the impact of the FORTIS toolkits in operational environments.
With the foundational architecture and design completed, the next phase of FORTIS will focus on the implementation and integration of the first release of the toolkits, followed by testing and validation in pilot environments.
To maximize its impact and ensure successful market adoption, FORTIS must address several key challenges. Further research and development will be essential to enhance robot learning, behavioral adaptation, and decision-making capabilities. Large-scale demonstration and validation activities in real-world environments will be critical for fine-tuning performance and ensuring compliance with industry standards. Additionally, access to market opportunities and financing will be necessary to drive commercialization, requiring strong partnerships with industry leaders, investors, and regulatory bodies. The project must also establish a clear intellectual property (IP) and exploitation strategy, securing patents and fostering technology transfer mechanisms to accelerate adoption. Furthermore, contributions to standardization and regulatory frameworks will ensure that FORTIS aligns with emerging EU policies on AI and robotics, facilitating wider acceptance and integration into industrial workflows. By addressing these factors, FORTIS aims to become a benchmark solution for human-centric AI-driven robotics, bridging the gap between cutting-edge research and real-world applications.
In WP1 (Requirements Analysis and Planning), the project has successfully completed a comprehensive stakeholder and technical requirements analysis, documented in the FORTIS Requirements Analysis Report (D1.1). This deliverable consolidates insights from industry, academia, and end-users, ensuring alignment with real-world needs. Additionally, the project has defined the first version of the system architecture and technical specifications (D1.2) detailing the modular structure and data flow between components. To guide development efforts, a technical activities planning document (D1.3) has been prepared, establishing the roadmap for implementation, integration, and pilot demonstrations.
In WP2 (Human and AI Perspective), significant progress has been made in designing human-centric AI models that will enable robots to understand human behavior, cognitive states, and physical activities. The Human-Centred Data Collection Toolkit has been defined, integrating wearable and non-wearable sensors, such as IMUs, EEG, EDA, RGB cameras, LiDAR, and microphones, to gather multimodal data for human perception modeling. Initial research has also been conducted on AI models for real-time cognitive and physical state monitoring, laying the groundwork for adaptive human-robot interaction. Furthermore, the Multi-Modal Adaptive Communication Toolkit has been conceptualized, allowing robots to interact with humans through speech, gestures, and visual cues, ensuring intuitive and seamless collaboration.
In WP3 (Robotic and AI Perspective), the project has advanced in sensor integration and robotic perception, selecting and implementing LiDAR, stereo cameras, and depth cameras into ROS-enabled robotic platforms. The first version of the SLAM-based localization system has been developed, providing real-time navigation and mapping capabilities in dynamic environments. Additionally, the Robotic Intelligibility Toolkit has been designed to enable robots to adjust their behavior based on human interactions, improving predictability and safety. Another key achievement is the definition of the Multi-Robot Dynamic Reconfigurability Toolkit, which leverages AI-driven task allocation and coordination models to facilitate multi-robot collaboration in industrial settings.
In WP5 (Demonstration and Validation), the project has conducted a detailed analysis and refinement of the use cases for its three pilot sites in construction, infrastructure maintenance, and manufacturing. Initial pilot site visits have provided valuable insights into real-world requirements and constraints, guiding the technical developments in WP2 and WP3. A demonstration and validation plan (D5.1 - draft version) has been prepared, defining the evaluation methodology and key performance indicators (KPIs) for assessing the impact of the FORTIS toolkits in operational environments.
With the foundational architecture and design completed, the next phase of FORTIS will focus on the implementation and integration of the first release of the toolkits, followed by testing and validation in pilot environments.
To maximize its impact and ensure successful market adoption, FORTIS must address several key challenges. Further research and development will be essential to enhance robot learning, behavioral adaptation, and decision-making capabilities. Large-scale demonstration and validation activities in real-world environments will be critical for fine-tuning performance and ensuring compliance with industry standards. Additionally, access to market opportunities and financing will be necessary to drive commercialization, requiring strong partnerships with industry leaders, investors, and regulatory bodies. The project must also establish a clear intellectual property (IP) and exploitation strategy, securing patents and fostering technology transfer mechanisms to accelerate adoption. Furthermore, contributions to standardization and regulatory frameworks will ensure that FORTIS aligns with emerging EU policies on AI and robotics, facilitating wider acceptance and integration into industrial workflows. By addressing these factors, FORTIS aims to become a benchmark solution for human-centric AI-driven robotics, bridging the gap between cutting-edge research and real-world applications.