In this part, you will find all the achievements thanks to the SME 1 Grants. It was an opportunity to bring some improvements to the tools that you will see in this document
1. Face2Pain: a first pain scoring algorithm with Automatic facial recognition of pain
Face2Pain is a protoype developed at Lucine since 2017 which uses deep learning toward estimation of pain intensity from the face.
We try to solve a regression problem (the output is a "pain level" on a scale from 0 to infinity). We predict a frame-wise pain level from the facial expression.
This approach tries to make up for the lack of available "pain face" data by leveraging a model which has already been trained toward a similar goal.
Our neural network was trained from the UNBC McMaster University dataset. It contains videos of 129 patients with shoulder pain who are asked to do physical exercise that causes varying degrees of pain.
This first algorithm can detect the level of suffering expressed by a person. The size of the data set used for training being relatively small, the results are impacted by different biases. For example, a smile will tend to be considered as a grin of suffering because the algorithm has not been trained to differentiate between the two.
To improve the performance of this algorithm, Lucine's data-scientists are working on several avenues:
• Training on larger, more representative datasets.
• Use of several modalities as input signals: face video, posture video, sound track, physiological signals, ...
• Improved learning algorithms and better processing of temporality.
2. Face2Pain improvements since SME 1
Our work has allowed us to customize Face2Pain to achieve better results.
The first one was to modernize the architecture of the model used for the fine tuning phase. Initially based on the VGG16 network, we used the FaceNet network based on the Inception ResNet V1 networks. The results obtained are relatively similar but with a much lower training time.
The second is to combine the use of dataset with a much larger dataset for the fine tuning phase.
SME 1 : Creation of correct tools
Pain recognition involves identifying different actions from video clips where the action may or may not be performed throughout the video. Tools are then needed to "localize" the actions, with action localization in videos involving finding both the spatial and temporal extent of an action, i.e. both where and when an action takes place. We can therefore apply these kinds of methods to detect "unusual" events, such as the appearance of pain
Lucine's scientific hypothesis is to consider the expression of pain as an action and to apply the findings of this field of research to our industrial goal.
Aria and Artemis science: tools for data acquisition
In order to build a relevant dataset, Lucine has deployed a first web application named Artemis Science and is currently developing a mobile application named Aria.
1. Artemis Science
As part of her research on pain assessment and the implementation of digital treatment, Lucine needs as much data as possible to understand the behaviour of a patient in pain.
During an acquisition session we will obtain one or more videos of the patient (front, side, depth camera), some physiological signals (electrocardiogram, Electro dermal activity, Electroencephalogram, etc) as well as a certain amount of information about the patient and his state of health, which will be called metadata.
In this context, Lucine has deployed a system capable of retrieving a maximum amount of data from different sources. This data is formatted and stored in a homogenous way so that it can be used for others purposes.
2. Aria
Aria will allow the acquisition of less structured data, captured during the daily life of users: at home, at work, etc
These data are generally of poorer quality but this allows to avoid the bias induced by the presence of a scientist and to have a much higher frequency of capture.
Aria consists in a mobile app dedicated to:
• Primary purpose : Collecting data which will feed the training of facial and postural recognition of pain algorithms.
Data can be provided by patients, but also by healthy volunteers. Indeed, healthy volunteers’ data are essential to improve and to challenge algorithms ‘results. Besides, a key challenge of Aria is to be enough convincing, enough playful or useful to persuade healthy people to share data.
• Secondary purpose : Federating and retaining cohorts for clinical studies of pain
