Periodic Reporting for period 2 - MEDIATOR (MEdiating between Driver and Intelligent Automated Transport systems on Our Roads)
Periodo di rendicontazione: 2020-11-01 al 2022-04-30
Vehicle automation has a substantial potential to improve road safety, since it reduces the influence of human fallibility. At the same time, vehicle automation is likely to introduce new risks, particularly during the transition phase to full automation when the task of the driver changes from an active to a more passive, supervisory role. This could result in reduced attention and situational awareness, and mode confusion, making the driver unfit to take over control when needed.
MEDIATOR aims to develop an intelligent ‘mediator’ support system, enabling safe, real-time switching between the human driver and automation. The Mediator system continuously and in real time monitors and weighs the information about the driving context, the driver state, as well as the automation status. By weighing these elements, the Mediator system will determine how and to what extent the human or the automation should be in control, and ensures safe, comfortable, real-time switching between the two.
MEDIATOR aims to develop an intelligent ‘mediator’ support system, enabling safe, real-time switching between the human driver and automation. The Mediator system continuously and in real time monitors and weighs the information about the driving context, the driver state, as well as the automation status. By weighing these elements, the Mediator system will determine how and to what extent the human or the automation should be in control, and ensures safe, comfortable, real-time switching between the two.
MEDIATOR booked significant progress in the first two phases of the project; setting the functional requirements for the Mediator system and, based on the system design developed, implement this system in various prototypes for evaluation in the third phase of the project.
The first phase started with the elaboration of general principles of the Mediator system. Use cases were developed to focus further development of the system and a framework was developed addressing the levels of automation from a human centred perspective. The Mediator system functional requirements for all system components were defined based on state-of-art knowledge and knowledge gained from research conducted in the project.
In the second phase the technical design for the Mediator system was defined. Subcomponents were built and integrated into different prototypes.
The four main areas of research and development are: estimate and predict human performance, estimate and predict automation performance, central decision logic and human machine interface design.
Human performance
To intervene in time, the Mediator system must be able to predict that a driver is likely to become unfit to drive or to feel uncomfortable before it affects driving performance. It must also predict when a currently unfit or uncomfortable driver can be fit again to take over when the automation becomes unfit. Predicting the near future situation means that fitness (fatigue and distraction) and comfort cannot be detected directly, e.g. from drivers steering behaviour, as most currently available applications do. Hence, algorithms to detect early-stage driver fatigue and distraction irrespective of driver control input have been developed. For predicting when a driver would become uncomfortable, the work resulted in a description of typical uncomfortable driving situations, as well as initial research into possible methods for detecting upcoming discomfort in real time and personalizing comfort predictions.
Automation performance
To quantify the automation fitness, an automation fitness scale was introduced. The automation state is complementary to and mirrors the driver state component in assessing the “fitness to drive”. Using collected and annotated data for the driving automation system to be assessed, with the goal to correlate both the automation indicators and the driving context with the number of occurrences of system deactivations/overrides/fallback initiations per time unit normalized on the automation fitness scale.
Central decision logic
Central Decision Logic’s role is to recommend switches between the various available automation levels (with different levels of urgency), trigger corrective actions to attempt to improve fitness of a degraded driver (based on detection of distraction or fatigue or discomfort), enforce emergency actions, and provide additional supportive information related to this role to the Human Machine Interface.
An Artificial Intelligence (AI) decision making algorithm is responsible for deciding whether and when an action (such as a transfer of control, etc.) should be executed. An execution & monitoring algorithm is responsible for executing the actions by the AI in close collaboration with the HMI, and monitor whether they are successful.
Human machine interface (HMI)
The HMI of the Mediator system focuses on both preventing unsafe and uncomfortable driving situations to arise and intervening when needed. For prevention, an analysis of the literature resulted in an overview of possible HMI designs that can prevent task-induced fatigue and distraction, the main challenges of assisted driving and conditional automation. A separate literature study and a series of online experiments focused on preventing the main challenges of conditional and high-level automation like mode confusion and overreliance. Providing continuous and peripherally visible information on automation reliability and remaining time in current driving mode proved to be a viable strategy.
Prototypes for evaluation
Subcomponents of the Mediator system were integrated in five different prototypes; two vehicles, two driving simulators and one computer simulation. In the next phase of the project all prototypes will be used in numerous trials to test and validate the Mediator system. Based on the results, the potential safety benefits and other related societal benefits of the Mediator system will be estimated, and guidelines, protocols and recommendations that help further exploitation of the Mediator system and similar support systems will be defined.
Exploitation strategy
To optimise the opportunities for commercial implementation of the Mediator system, an exploitation plan was developed and opportunities were monitored during the course of the project. Based on the definition of the main potentially exploitable results (hardware, software, knowledge) and a related SWOT analysis, each of the project partners identified its own exploitation opportunities. As elements of the Mediator system are expected also to be of value for other transport modes, a similar exploitation plan was prepared for these other modes.
The first phase started with the elaboration of general principles of the Mediator system. Use cases were developed to focus further development of the system and a framework was developed addressing the levels of automation from a human centred perspective. The Mediator system functional requirements for all system components were defined based on state-of-art knowledge and knowledge gained from research conducted in the project.
In the second phase the technical design for the Mediator system was defined. Subcomponents were built and integrated into different prototypes.
The four main areas of research and development are: estimate and predict human performance, estimate and predict automation performance, central decision logic and human machine interface design.
Human performance
To intervene in time, the Mediator system must be able to predict that a driver is likely to become unfit to drive or to feel uncomfortable before it affects driving performance. It must also predict when a currently unfit or uncomfortable driver can be fit again to take over when the automation becomes unfit. Predicting the near future situation means that fitness (fatigue and distraction) and comfort cannot be detected directly, e.g. from drivers steering behaviour, as most currently available applications do. Hence, algorithms to detect early-stage driver fatigue and distraction irrespective of driver control input have been developed. For predicting when a driver would become uncomfortable, the work resulted in a description of typical uncomfortable driving situations, as well as initial research into possible methods for detecting upcoming discomfort in real time and personalizing comfort predictions.
Automation performance
To quantify the automation fitness, an automation fitness scale was introduced. The automation state is complementary to and mirrors the driver state component in assessing the “fitness to drive”. Using collected and annotated data for the driving automation system to be assessed, with the goal to correlate both the automation indicators and the driving context with the number of occurrences of system deactivations/overrides/fallback initiations per time unit normalized on the automation fitness scale.
Central decision logic
Central Decision Logic’s role is to recommend switches between the various available automation levels (with different levels of urgency), trigger corrective actions to attempt to improve fitness of a degraded driver (based on detection of distraction or fatigue or discomfort), enforce emergency actions, and provide additional supportive information related to this role to the Human Machine Interface.
An Artificial Intelligence (AI) decision making algorithm is responsible for deciding whether and when an action (such as a transfer of control, etc.) should be executed. An execution & monitoring algorithm is responsible for executing the actions by the AI in close collaboration with the HMI, and monitor whether they are successful.
Human machine interface (HMI)
The HMI of the Mediator system focuses on both preventing unsafe and uncomfortable driving situations to arise and intervening when needed. For prevention, an analysis of the literature resulted in an overview of possible HMI designs that can prevent task-induced fatigue and distraction, the main challenges of assisted driving and conditional automation. A separate literature study and a series of online experiments focused on preventing the main challenges of conditional and high-level automation like mode confusion and overreliance. Providing continuous and peripherally visible information on automation reliability and remaining time in current driving mode proved to be a viable strategy.
Prototypes for evaluation
Subcomponents of the Mediator system were integrated in five different prototypes; two vehicles, two driving simulators and one computer simulation. In the next phase of the project all prototypes will be used in numerous trials to test and validate the Mediator system. Based on the results, the potential safety benefits and other related societal benefits of the Mediator system will be estimated, and guidelines, protocols and recommendations that help further exploitation of the Mediator system and similar support systems will be defined.
Exploitation strategy
To optimise the opportunities for commercial implementation of the Mediator system, an exploitation plan was developed and opportunities were monitored during the course of the project. Based on the definition of the main potentially exploitable results (hardware, software, knowledge) and a related SWOT analysis, each of the project partners identified its own exploitation opportunities. As elements of the Mediator system are expected also to be of value for other transport modes, a similar exploitation plan was prepared for these other modes.
Work already substantially progressed beyond the state of the art by extending knowledge on predicting degraded performance by driver fatigue and distraction, by explicitly defining the role of automation status in elaborating the concept of the Mediator system, and by taking user acceptance as the central element of the HMI design.
Until the end of the project, the scientific basis for the decisions of the Mediator system will be elaborated for specific use cases and applied in laboratory and in-vehicle prototypes. The prototypes currently are being evaluated by computer simulation, in driving simulators and on-road. Guidelines for measuring degraded human performance, protocols for low-cost laboratory system testing, and recommendations on legal and regulatory aspects will be developed to assist the automotive industry in further developing the Mediator system principles, and tailor it to their exact needs.
Progress has also been made towards the realisation of most of the expected potential impacts. The ultimate impact of a Mediator system will be on the number of road casualties. The size of this impact (and related societal impacts) will be estimated in the last phase of the project based on the effects on driving behaviour as assessed in the evaluation studies.
Until the end of the project, the scientific basis for the decisions of the Mediator system will be elaborated for specific use cases and applied in laboratory and in-vehicle prototypes. The prototypes currently are being evaluated by computer simulation, in driving simulators and on-road. Guidelines for measuring degraded human performance, protocols for low-cost laboratory system testing, and recommendations on legal and regulatory aspects will be developed to assist the automotive industry in further developing the Mediator system principles, and tailor it to their exact needs.
Progress has also been made towards the realisation of most of the expected potential impacts. The ultimate impact of a Mediator system will be on the number of road casualties. The size of this impact (and related societal impacts) will be estimated in the last phase of the project based on the effects on driving behaviour as assessed in the evaluation studies.