Achieving Complex Collaborative Missions via Decentralized Control and Coordination of Interacting Robots

Project Information

Co4Robots

Grant agreement ID: 731869

DOI

10.3030/731869

Project closed

EC signature date 26 October 2016

Start date 1 January 2017

End date 30 June 2020

Funded under

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Total cost

€ 3 820 956,25

EU contribution

€ 3 820 956,25

Coordinated by

KUNGLIGA TEKNISKA HOEGSKOLAN
Sweden

Periodic Reporting for period 2 - Co4Robots (Achieving Complex Collaborative Missions via Decentralized Control and Coordination of Interacting Robots)

Reporting period: 2018-07-01 to 2020-06-30

Summary of the context and overall objectives of the project

Imagine a future social/industrial facility where robots have been deployed to provide services and accomplish everyday tasks like object handling/transportation, or pickup and delivery operations. In this context, different robots with different actuation, manipulation and perception capabilities must be coordinated to achieve various complex tasks that require collaborative actions with each other and with humans. Thus, the supervision and coordination of the overall heterogeneous system mandates a decentralized framework that integrates high-level task-planning, low-level motion control and robust, real-time robot perception. From the World Robotics Survey, indoor logistics have become a major market. Sales of professional service robots in the logistics segment registered a growth of 28% in 2014 with the value of sales increasing to USD 2.2 billion. This shows the opportunity for European companies to address this area. Current practice in coordination of robotic teams is mostly based on offline, centralized planning and related tasks are almost exclusively fulfilled in a predefined manner. Only a little room is allowed for real-time and coordinated decentralized actions. We argue that this does not utilize the capabilities of multi-robot systems to operate efficiently in a dynamic environment. In most cases, sudden changes in the environment, the type of assigned tasks, and the need for coordination, would cause the system to halt, ask for a human intervention and restart. Thus the need for a framework for decentralized, real-time, automated task (re)-planning is evident.

Based on the above, the goal of Co4Robots is to build: (i) a systematic real-time decentralized methodology to accomplish complex mission specifications given to a team of potentially heterogeneous robots; (ii) a set of control schemes appropriate for the mobility and manipulation capabilities of the considered robotic platforms, their types and dynamics, the unexpected and sudden changes in the environment, and the presence of humans; (iii) a set of perceptual capabilities that enable robots to localize themselves and estimate the state of their highly dynamic environment, in the presence of strong interactions and in a collaborative manner; and (iv) their corresponding systematic integration at both the conceptual and the software implementation levels.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

During the first period (Months 1-18), a set of strategies for manipulation and mobility have been developed. We tackled Load Exchange among heterogeneous agents, with a vision-based grasping and motion control scheme, a planning algorithm and an optimal grasping selection scheme. Regarding Cooperative Load Transportation, the provided solutions include human-robot collaboration in obstacle-cluttered environments via implicit communication and a compliance control for human–robot cooperative manipulation. Regarding Cooperative Motion Planning, the contributions include navigation algorithms, decentralized and reconfigurable control schemes, and a decentralized real-time planning framework for heterogeneous agent teams. In addition, robot perception algorithms were developed to enable the robot to detect and track objects and humans and recognize human gestures.

In the second period (Months 19-42), we proposed efficient approaches to deal with the issues in cooperation and coordination. Regarding manipulation and mobility, we tackled the problem of cooperative load transportation/manipulation among heterogeneous agents to achieve the physical interaction with users, operators or other system entities, studied the case of partially known object dynamics and obstacle-cluttered environments, and proposed control modules to ensure effective operation via the tasks to be executed. Regarding decentralized real-time planning, we proposed decentralized online abstraction scheme to capture agents’ motion capabilities, combined static and dynamic quantization to refine the abstraction dynamically, and applied the refinement technique to deal with motion planning under non-holonomic and geometric constraints. Regarding decentralized real-time perception, we developed robot perception algorithms to solve the perception problems in the collaboration of multiple agents for certain goals. Regarding software platform and applications, we developed a platform to integrate all the software and to evaluate all the key objectives, and instantiated and customized the outcomes to specific demonstrators.

Overall, the goal of Co4Robots has been achieved. We finished 7 work packages with 28 tasks, and distributed 3 demonstrations and 25 deliverables. Commercial relevance and exploitation potential have been considered constantly by Bosch and its subsidiary Bosch Rexroth. The Co4Robots activities prepared AI-based solutions for self-organization and learning of cooperative systems in the Factory of the Future, addressed the increasing complexity of intralogistics systems and the resulting need for systematic development procedures and scaling algorithmic solutions. The results of Co4Robots can be treated as “door-openers” for future intralogistics systems, which is shown in the filed patent applications.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

Although the most employed communication form in multi-agent systems is explicit, increasing the number of robots requests complex protocols to deal with crowded bandwidth. Despite the increased difficulty, our goal is to design an autonomous system with multiple heterogeneous robots and to solve tasks using interactions among them, both explicitly at a symbolic level and implicitly at the low level. Towards this direction, innovative motion and manipulation control schemes were designed and implemented, in order to deal with the model uncertainty in the interaction dynamics and the environment, extending thus greatly the current state of the art in robust motion planning, collision avoidance and manipulation control. The developed control approaches exhibit increased level of robustness and guarantee collision avoidance as well as satisfaction of the goals, obeying simultaneously operational constraints and providing the targeted system abilities required in cooperative tasks among heterogeneous robots. Moreover, LTL captures many temporal tasks, but not explicit time bounds. We thus aimed for automated synthesis under local MITL tasks that involve quantitative time bounds. A novelty is the assignment of specifications to the team in a partial decentralization manner. The perception goal was to aid interaction and collaboration among robots/humans and to enhance the performance in tasks such as SLAM and object/agent detection and tracking. Therefore, we increased the perception ability of each robotic platform and allowed for enhanced overall perception of the environment via robots and humans in collaboration. To do so, we developed and integrated various vision technologies, including 3D pose estimation and gesture recognition, which performed on board the robots and in real time. Software engineering plays a key role in securing the new technology’s affirmation by making it pervasive and ubiquitous. We have used Model-Driven Engineering, a key technology at a mature level in other domains but not in robotics.

Co4Robots logo and image of 2nd MStone Demo meeting experiments

Co4Robots logo and image of 1st MStone Demo meeting experiments

Co4Robots logo and image of 3rd MStone Demo meeting experiments