Periodic Reporting for period 1 - d3pm (Sonio: Deep-Learning for Detection and Diagnostic of Prenatal Malformations)
Reporting period: 2023-01-01 to 2023-12-31
Sonio’s mission is to contribute to access to high-quality care for women and children worldwide thanks to artificial and collective intelligence.
On the mother's side, too many women still die due to pregnancy complications. This is true in low and middle-income countries (LIMICS) but also in high-income countries (HICS) such as the USA where maternal mortality has increased significantly in recent years. The main culprits are haemorrhage or other complications related to childbirth. In France, it has been estimated that 58% of maternal deaths were probably or possibly avoidable.
On the foetus side, according to EUROCAT network, 2,7% of the foetus have at least one congenital malformations. Most of the studies conclude that 50% of congenital malformations are not detected antenatally and that these screening rates vary a lot depending on the geographic location (i.e physicians expertise). An undiagnosed congenital malformation constitutes a greater risk of mortality for the newborn, especially, abnormalities requiring specific and urgent care at birth (e.g. transposition of the great arteries). Ultrasound is both relatively inexpensive (portable probes) and allows access to a large part of the foetal anatomy and the detection of obstetric complications. The limit is therefore rather access to clinical expertise in LIMICS and HICS rather than the technical improvement of ultrasound.
Foetal ultrasound challenges: very high number of potential malformations accessible in all foetal anatomical regions (more than 400), knowing that these malformations can be isolated or part of a poly-malformation syndrome (9000 syndromes referenced in Orphanet, including 800 accessible on foetal ultrasound). In this context, learned societies have produced increasingly comprehensive and demanding protocols putting more pressure on practitioners. The latest recommendations from ISUOG (the main international learned society of obstetric and foetal medicine) of 2022 include 20 mandatory ultrasound plans showing 42 anatomical structures (while there were only 32 in 2011). As a result, doctors may forget the obligatory American planes, leading to underdiagnosis. Even when the examination is exhaustive as to which American plane to take, a congenital anomaly can go unnoticed by not detecting abnormal ultrasound signs on the image. In this context, artificial intelligence is a tool of choice for strengthening the skills of health practitioners. It has been shown that using a checklist can improve exam pass rates from 49% to 97% [1]. However, checklists are little used in practice because their use is cumbersome and time-consuming. Sonio can democratise and generalise the use of a checklist. AI also has the potential to provide clinical and diagnostic information by detecting abnormal ultrasound signs on an image that may be related to a congenital malformation, recommending the next abnormalities to check for, and providing an opinion on phenotype and genotype.
[1] beyond ultrasound first forum on improving the quality of ultrasound imaging in Obstetrics and Gynecology, B.Benacerraf and al. https://onlinelibrary.wiley.com/doi/full/10.1002/jum.14504
On the mother's side, too many women still die due to pregnancy complications. This is true in low and middle-income countries (LIMICS) but also in high-income countries (HICS) such as the USA where maternal mortality has increased significantly in recent years. The main culprits are haemorrhage or other complications related to childbirth. In France, it has been estimated that 58% of maternal deaths were probably or possibly avoidable.
On the foetus side, according to EUROCAT network, 2,7% of the foetus have at least one congenital malformations. Most of the studies conclude that 50% of congenital malformations are not detected antenatally and that these screening rates vary a lot depending on the geographic location (i.e physicians expertise). An undiagnosed congenital malformation constitutes a greater risk of mortality for the newborn, especially, abnormalities requiring specific and urgent care at birth (e.g. transposition of the great arteries). Ultrasound is both relatively inexpensive (portable probes) and allows access to a large part of the foetal anatomy and the detection of obstetric complications. The limit is therefore rather access to clinical expertise in LIMICS and HICS rather than the technical improvement of ultrasound.
Foetal ultrasound challenges: very high number of potential malformations accessible in all foetal anatomical regions (more than 400), knowing that these malformations can be isolated or part of a poly-malformation syndrome (9000 syndromes referenced in Orphanet, including 800 accessible on foetal ultrasound). In this context, learned societies have produced increasingly comprehensive and demanding protocols putting more pressure on practitioners. The latest recommendations from ISUOG (the main international learned society of obstetric and foetal medicine) of 2022 include 20 mandatory ultrasound plans showing 42 anatomical structures (while there were only 32 in 2011). As a result, doctors may forget the obligatory American planes, leading to underdiagnosis. Even when the examination is exhaustive as to which American plane to take, a congenital anomaly can go unnoticed by not detecting abnormal ultrasound signs on the image. In this context, artificial intelligence is a tool of choice for strengthening the skills of health practitioners. It has been shown that using a checklist can improve exam pass rates from 49% to 97% [1]. However, checklists are little used in practice because their use is cumbersome and time-consuming. Sonio can democratise and generalise the use of a checklist. AI also has the potential to provide clinical and diagnostic information by detecting abnormal ultrasound signs on an image that may be related to a congenital malformation, recommending the next abnormalities to check for, and providing an opinion on phenotype and genotype.
[1] beyond ultrasound first forum on improving the quality of ultrasound imaging in Obstetrics and Gynecology, B.Benacerraf and al. https://onlinelibrary.wiley.com/doi/full/10.1002/jum.14504
Our scientific efforts financed by EIC have gone in three different main directions:
- improve the completeness of prenatal ultrasound examinations by building an AI tool able to recognize the different US planes and structures (Sonio Detect)
- improve the rate of abnormal ultrasound signs discovery at foetal ultrasound by building an image-based AI having high specificity and sensitivity on most important abnormal ultrasound signs (Sonio Suspect)
- improve the rate of prenatal rare disease diagnosis by providing an AI tool able to recommend next images to take or next anomaly to look for in order to avoid incorrect conclusion of an isolated malformation when it is in fact a rare disease (Sonio Expert).
We managed to obtain a FDA clearance in June 2023 for Sonio Detect V1 and to submit a V2 with a wider perimeter in January 2024. We went from a V1 with 13 foetal US views and 15 anatomical structures to a V2 with 26 Views and 53 structures. Especially the V2 is much more comprehensive on first trimester foetal ultrasound which is known to be more challenging. Achieving such performance can be done in a supervised learning framework by collecting a large amount of data that correctly represents the variability of clinical practice and labelling it with domain experts. Building this database is tedious, time-consuming and necessitates constant back and forth between AI scientists and physicians. Optimising image preprocessing, detection parameters, model architecture have also been constant activities of the R&D team.
With all these efforts, Sonio Detect has demonstrated high performance in the set objectives of detecting standard views and anatomical structures for quality assessment purposes. But our investigations have also shown the limits of Sonio Detect, mainly trained on normal images (congenital malformations only represent 3% of births), when it is tested on pathological images. This is a clear and completely new demonstration of the interest of the Sonio Suspect that we are covering with funding from the EIC. Sonio Detect is already proving effective on certain pathologies such as “Agenesis of the corpus callosum” or “Persistent left superior vena cava” where a standard structure is either surnumerary or absent. Though Sonio Detect could also be prone to hallucinations and falsely detect absent anatomical structures in the context of other pathologies like renal agenesis or some cardiopathies where Sonio Detect may clear some images because the deviation that makes them pathological is too subtle to be detected. Our latest achievements have been the collection of large pathological databases, the annotation of these images and clips with their ultrasound semiology, as well as the training of Deep-Learning models on pathological images and normal images to better control these hallucinations.
Finally we made some notable contributions to the effort of building an AI tool able to assist the physicians when trying to do prenatal diagnosis of a rare disease. The study we have performed in 2022 for the CE mark of Sonio Expert has been published in 2023 in the journal UOG. Several independent studies building their own databases of rare disease cases with their foetal phenotype are currently performed and will be published in 2024. This is an important point, as it has been repeatedly demonstrated that, in the field of rare diseases, the reproducibility of studies on the performance of AI software is poor.
- improve the completeness of prenatal ultrasound examinations by building an AI tool able to recognize the different US planes and structures (Sonio Detect)
- improve the rate of abnormal ultrasound signs discovery at foetal ultrasound by building an image-based AI having high specificity and sensitivity on most important abnormal ultrasound signs (Sonio Suspect)
- improve the rate of prenatal rare disease diagnosis by providing an AI tool able to recommend next images to take or next anomaly to look for in order to avoid incorrect conclusion of an isolated malformation when it is in fact a rare disease (Sonio Expert).
We managed to obtain a FDA clearance in June 2023 for Sonio Detect V1 and to submit a V2 with a wider perimeter in January 2024. We went from a V1 with 13 foetal US views and 15 anatomical structures to a V2 with 26 Views and 53 structures. Especially the V2 is much more comprehensive on first trimester foetal ultrasound which is known to be more challenging. Achieving such performance can be done in a supervised learning framework by collecting a large amount of data that correctly represents the variability of clinical practice and labelling it with domain experts. Building this database is tedious, time-consuming and necessitates constant back and forth between AI scientists and physicians. Optimising image preprocessing, detection parameters, model architecture have also been constant activities of the R&D team.
With all these efforts, Sonio Detect has demonstrated high performance in the set objectives of detecting standard views and anatomical structures for quality assessment purposes. But our investigations have also shown the limits of Sonio Detect, mainly trained on normal images (congenital malformations only represent 3% of births), when it is tested on pathological images. This is a clear and completely new demonstration of the interest of the Sonio Suspect that we are covering with funding from the EIC. Sonio Detect is already proving effective on certain pathologies such as “Agenesis of the corpus callosum” or “Persistent left superior vena cava” where a standard structure is either surnumerary or absent. Though Sonio Detect could also be prone to hallucinations and falsely detect absent anatomical structures in the context of other pathologies like renal agenesis or some cardiopathies where Sonio Detect may clear some images because the deviation that makes them pathological is too subtle to be detected. Our latest achievements have been the collection of large pathological databases, the annotation of these images and clips with their ultrasound semiology, as well as the training of Deep-Learning models on pathological images and normal images to better control these hallucinations.
Finally we made some notable contributions to the effort of building an AI tool able to assist the physicians when trying to do prenatal diagnosis of a rare disease. The study we have performed in 2022 for the CE mark of Sonio Expert has been published in 2023 in the journal UOG. Several independent studies building their own databases of rare disease cases with their foetal phenotype are currently performed and will be published in 2024. This is an important point, as it has been repeatedly demonstrated that, in the field of rare diseases, the reproducibility of studies on the performance of AI software is poor.
This year 2023, funded by EIC, allowed us to obtain FDA clearance of Sonio Detect V1 for prenatal practitioners. This reporting solution ensures that we have a clear go-to-market, in line with our US commercial launch roadmap in Q1 2024. CE marking and European commercial launch are expected in Q2 and Q3 2024.
This quality control tool should help reduce obstetric complications and congenital malformations missed antenatally, leading to an increased risk of maternal and neonatal death or disability. The new version of Sonio Detect and Sonio Suspect, which presents perimeter and workflow features never achieved in this field, has the potential to meet these challenges. This universal solution can be deployed regardless of the ultrasound device used by the physicians.
This quality control tool should help reduce obstetric complications and congenital malformations missed antenatally, leading to an increased risk of maternal and neonatal death or disability. The new version of Sonio Detect and Sonio Suspect, which presents perimeter and workflow features never achieved in this field, has the potential to meet these challenges. This universal solution can be deployed regardless of the ultrasound device used by the physicians.