CoreSense: A Hybrid Cognitive Architecture for Deep Understanding

Project Information

CoreSense

Grant agreement ID: 101070254

DOI

10.3030/101070254

EC signature date 13 June 2022

Start date 1 October 2022

End date 30 September 2026

Funded under

• Digital, Industry and Space

Total cost

€ 5 409 839,38

EU contribution

€ 5 409 839,38

Coordinated by

UNIVERSIDAD POLITECNICA DE MADRID
Spain

Periodic Reporting for period 2 - CoreSense (CoreSense: A Hybrid Cognitive Architecture for Deep Understanding)

Reporting period: 2023-10-01 to 2025-03-31

Summary of the context and overall objectives of the project

Autonomy is a hot word these days. Not only due to Tesla’s cars or the surge of automated drone wars. Robots and artificial intelligences are taking responsibilities in performing tasks commonly done by humans. Robots that think like humans -cognitive robots- are augmenting their autonomy, enabling them to deployments in increasingly open-ended environments and for many classes of tasks. This offers enormous possibilities for improvements in human economy and wellbeing.

However, this deployment of autonomous robots also poses strong risks that are difficult to assess and control by humans. The trend towards increased autonomy conveys augmented problems concerning reliability, resilience, and trust when robots deal with complex or novel situations. If you have a task that want to be done, and you are not inside a well-organised factory, you don’t delegate it to a robot.

The essence of the problem is that robots do not understand well. They don’t understand well what you say, they don’t understand well what they see, they don’t understand well themselves nor what to do when things are not as expected. This is a problem that artificial intelligence approaches based on machine learning are not addressing well. We can see that in the many childish errors, hallucinations and fake arguments that they produce.

An improvement in the capability of understand of autonomous robots is really needed. The CORESENSE project tries to approach a solution to this need by developing a solid theory of understanding and software assets to endow robots with this capability. In the project we apply the results in three real robot demonstrations to augment flexibility of manufacturing robots, augment resilience and group cohesion of drone teams doing inspections, and to augment human alignment of social robots.

In summary, we will develop a cognitive architecture for autonomous robots based on a formal concept of understanding, supporting value-oriented situation understanding and self-awareness to improve robot flexibility, resilience and explainability.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

During this first reporting period the project did exploratory and recapitulation work concerning the core themes: understanding, awareness, cognition, and model-based engineering of robot control software. We elaborated initial theories and definitions and set the ground for formal work. Also during this time, the teams in charge of the applications testbeds worked in aligning their ideas and ways of working to the approach and terminology of the project theory. At the end of the first year we have reached a good situation concerning cooperation and sharing of points of view concerning the project contents and approach to effective improvement of robot operations.

As a result of this work, the project has reached a point where the three separate research threads in theory, technology and application have come together into a shared “understanding of understanding” in autonomous robots. After this first year of work:

1. We now know better what are the needs for deeper “understanding” in the three testbed applications -manufacturing, inspection and social robots- and have defined the key performance indicators to test CoreSense value in augmenting the efficacy of those systems.
2. We know better the problem of what it means to "understand" in a general sense, not only restricted to understanding natural human language or classifying objects in images that are the common uses of “understanding” in artificial intelligence. We also have an initial formal definition of understanding to be used in the implementation of core mechanisms.
3. We have established the core concept of the cognitive architecture in its foundational part: the infrastructural function layer. We have gathered a collection of specific requirements for the architecture from the project objectives, the development process, the testbeds and the context of its potential use.
4. We have analysed the spectrum of cognitive capabilities from several cognitive architectures, from the functional needs of the testbeds and from what cognitive capabilities are available in the ROS domain.
5. We have analysed the model-based engineering process in robotics and established a prototype lifecycle and a roadmap for a ROS-oriented toolchain development.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

After the initial work performed, it is too early to talk about results that can have immediate impact. We can only elaborate on the potential impact of the project if objectives are achieved.

The project has a dual objective: science and technology. A science of understanding would help design and build better artificial intelligence systems in terms of performance -better capability of doing things- and robustness – better capability of tolerating disturbance. Thanks to the reflective character of the technology, the systems could generate self-awareness, leading to better resilience and dependability.

To achieve these impacts, it is necessary that the technology finds its path to the real use. For this the project is engaged with the ROS community. ROS is the platform of election for new software developments in robotics. Finding a place in that ecosystem would maximise the potential impact of the results.


To be effective in the medium-long terms, it will be necessary to find extra funding or industrial collaboration to maintain the product line.