Mind and Time: Investigation of the Temporal Traits of Human-Machine Convergence

Project Information

TIMESTORM

Grant agreement ID: 641100

DOI

10.3030/641100

Project closed

EC signature date 24 November 2014

Start date 1 January 2015

End date 30 June 2018

Funded under

• EXCELLENT SCIENCE - Future and Emerging Technologies (FET)

Total cost

€ 2 892 500,00

EU contribution

€ 2 892 500,00

Coordinated by

IDRYMA TECHNOLOGIAS KAI EREVNAS
Greece

Periodic Reporting for period 3 - TIMESTORM (Mind and Time: Investigation of the Temporal Traits of Human-Machine Convergence)

Reporting period: 2017-07-01 to 2018-06-30

Summary of the context and overall objectives of the project

The TimeStorm project has been funded by the EU FETPROACT-2-2014 invited innovative and high impact research under the topic "Knowing, doing, being: cognition beyond problem solving". The goal of the TimeStorm project entitled "Mind and Time: Investigation of the Temporal Traits of Human-Machine Convergence" is to examine the role of time in multi-agent collaboration, considering particularly the case of daily human-robot interaction.
More specifically, the increasing need for robots smoothly integrated into our daily lives assumes focused exploration of the temporal aspects which are innately present in human machine interaction. In contrast to humans, ordinary computational systems cannot efficiently handle time, an issue that significantly harnesses fluency in short-term human-robot interaction and long-term human-robot symbiosis.
TimeStorm postulates that sense of time acts as a neuro-cognitive 'glue' that integrates processes from different cognitive modalities, resulting in more complete and powerful intelligent systems. In that sense, the equipment of artificial agents with temporal cognition establishes a new framework for the investigation and integration of "knowing", "doing", and "being" in artificial systems.
TimeStorm aims at the multidisciplinary investigation of time perception in order to extract working principles that enable implementing a computational architecture, analogous to natural temporal cognition, in robotic systems. To this end, TimeStorm explores the temporal aspects of multi-modal cognition in the context of real-world scenarios. The project aims at implementing a new generation of autonomous robots perfectly situated both in space and time depending on the goals and the wider collaborative framework. The achievement of TimeStorm goals will be demonstrated in the context of assistive human-robot interaction, in a kitchen environment.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

The TimeStorm project develops, for the first time, artificial time perception capacities that will provide robots with human comparable temporal cognition, enabling them to consider the three main temporal views that living organisms develop on the world, namely the future, the present and the past. All three of them are essential for effectively addressing the long-lasting and continuous aspects of human-robot symbiotic interaction in natural and unstructured contexts.

Towards the development of not only embodied but also “entimed” robotic systems, TimeStorm has investigated multiple, largely unexplored aspects of mind-time interaction have been investigated that include, among others, the perception of long time intervals, the coordination of multi-agent behavior for the synergetic accomplishment of tasks, the perception of time in social setups, and second person perspective on sense of time. Along this line, neuro-psychological investigation of brain mechanisms devoted in the processing of time successfully completed and key findings have provided guidelines for the implementation of computational models.

Computational modeling work has been particularly effective in incorporating time in robotic cognitive systems. A number of models have been implemented that accomplish the time-enriched representation of knowledge amenable to be searched and recalled at future times, the active and time-informed perceptuo-motor coordination of robot activities with third-party procedures in social environments and the long and short-term action planning of robot actions effectively adjusted to unexpected real-world events.

Early embodiment of partial models have been already achieved and the beneficial effect of robotic temporal cognition on human-robot synergies and time-critical multi-agent scenarios has been demonstrated. In parallel, major endeavors have already started in integrating and practically testing the implemented models. Interestingly, the embodiment of implemented models in robots collaborating with humans in home-like realistic setups, has shown that humans appreciate and consider much more natural and productive the collaboration with time-informed robotic systems in comparison to the ordinary systems.

A new FET-funded project, namely EnTiment has recently commenced, which capitalizes on the successful research and development activities on artificial time perception pursued within TimeStorm, in order to verify and substantiate the innovation potential of key TimeStorm research results turning them into a genuine social and economic innovation. In short, ENTIMENT aims to assess and deploy a novel Temporal Cognition Toolbox (TCT) that greatly facilitates the development of time-aware robots, capable to engage in prolonged, symbiotic interaction with humans.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The development of TimeStorm is in agreement to the information provided in the DoA, with respect to the expected impacts by the project. To maximize the impact of TimeStorm, the consortium has so far implemented the following concrete actions:

TimeStorm presence on the web. The TimeStorm website is regularly enriched with new material that summarizes recent research achievements which maximize the visibility of the project and assist the project goals and experimental results be appreciated by scientific community and the wider public.

Scientific publications. The project beneficiaries have published extensively in the most relevant scientific journals in the field. Selected relevant articles are given below:
• Damsma, A., & van Rijn, H. (2017). Pupillary response indexes the metrical hierarchy of unattended rhythmic violations. Brain and cognition, 111, 95-103.
• Droit-Volet, S. (2016). Emotion and implicit timing. PlosOne. DOI: 10.1371 / journal. pone. 0158474.
• Droit-Volet, S. & Berthon, M. (2017). Emotion and Implicit timing: the arousal Effect. Frontiers in Psychology.
• Droit-Volet, S., Trahanias, P., & Maniadakis, M. (2017). Passage of time judgments in everyday life are not related to duration judgments except for long durations of several minutes. Acta Psychologica, 73, 116-121.
• Halbersma, G., & Van Rijn, H., (2016). An Evaluation of the Effect of Auditory Emotional Stimuli on Interval Timing. Timing and Time Perception.
• Linares, D., Cos, I. & Roseboom, W., (2016). Adaptation for multisensory relative timing, Current Opinion in Behavioral Sciences, 8, pp. 35-41.
• Mandery C., Terlemez O., Do M., Vahrenkamp N., and Asfour T., “Unifying representations and large-scale whole-body motion databases for studying human motion,” IEEE Transactions on Robotics, vol. 32, no. 4, pp. 796–809, 2016
• M. Maniadakis, E. Hourdakis, P. Trahanias, Time-informed task planning in multi-agent collaboration, Cognitive Systems Research, 2017.
• M. Maniadakis, & P. Trahanias (2015). Integrated Intrinsic and Dedicated Representations of Time: A Computational Study Involving Robotic Agents, Timing & Time Perception, 3 (3-4), pp. 246 – 268.
• K. Nikiforou, P. Mediano, M. Shanahan. An investigation of the dynamical transitions in harmonically driven random networks of firing-rate neurons. Cognitive Computation, 2017. doi:10.1007/s12559-017-9464-6
• W. Roseboom, Z. Fountas, K. Nikiforou, D. Bhowmik, M. Shanahan, A. K. Seth. A functioning model of human time perception. 2017.
• Van Rijn, H.,(2016). How Memory Mechanisms Influence Interval Timing: A Review, Current Opinions in Behavioral Sciences.

Moreover a number of papers acknowledging TimeStorm have been presented in leading international scientific conferences.

timestorm-scheme.png