Periodic Reporting for period 1 - AVATAR (Unlocking the Power of Medical Images for Surgeons)
Reporting period: 2023-06-01 to 2024-05-31
Surgeons rely on medical images to make difficult decisions that affect patient survival and treatment outcomes. Images that are difficult to interpret can result in unnecessarily invasive procedures and follow-up surgeries, especially in the use cases targeted by AVATAR: orthopedic, oncology and vascular surgery.
Current surgery preparation requires numerous discussions between surgeons and radiologists. Lacking the chance to talk to the radiologist, the surgeon will tend to opt for more radical (invasive) procedures to secure clear margins for oncology surgery, or to access the target zone in vascular surgery. Surgeons may have a 3D reconstruction or 3D printing at their disposal, but these lack the intricate details (e.g. soft tissues and blood vessels) required for full visual appreciation. Surgeons need more streamlined methods to prepare and perform surgeries. This will allow them to reduce errors associated due to imaging limitations and to increase hospital productivity.
The AVATAR surgery solution will be the first photorealistic Extended Reality (XR: the umbrella term for Virtual + Augmented Reality) viewer for 3D medical images (MRI, CT-scans). Its breakthrough human-in-the-loop machine learning algorithms join perception and contextual knowledge to radically simplify use, improve productivity and enhance surgeon decision making. Visualizing muscles and bones, for example, allows better planning of orthopedic surgeries. Visualizing plaque in the lumen of blood vessels allows selection of the correct arteries for endovascular surgeries.
Our advanced rendering displays lossless 3D representations (“avatars”) for surgeons. This enables them to modify and optimize their patient representations in real-time (e.g. highlight or remove organs) for a visually comfortable and actionable visualization experience.
The AVATAR surgery solution is a groundbreaking productivity tool for surgeons and medical specialists. It is the first photorealistic XR viewer of 3D medical images, based on innovative machine learning algorithms and data visualization techniques, with the express objective of enhancing medical decision making.
Through high-fidelity local or Cloud-based render streaming, AVATAR will be the first solution to meet surgeon expectations for seamless preoperative and intraoperative data integration. Our Clinical Applications will assist surgeons in planning and implementing specific procedures.
For hospitals struggling to manage growing demand for surgery, AVATAR promises to provide valuable productivity gains without investing in new equipment or personnel.
AVATAR is also a powerful tool in communication with patients, engaging them with a more visual and intuitive approach during the patient journey.
Current surgery preparation requires numerous discussions between surgeons and radiologists. Lacking the chance to talk to the radiologist, the surgeon will tend to opt for more radical (invasive) procedures to secure clear margins for oncology surgery, or to access the target zone in vascular surgery. Surgeons may have a 3D reconstruction or 3D printing at their disposal, but these lack the intricate details (e.g. soft tissues and blood vessels) required for full visual appreciation. Surgeons need more streamlined methods to prepare and perform surgeries. This will allow them to reduce errors associated due to imaging limitations and to increase hospital productivity.
The AVATAR surgery solution will be the first photorealistic Extended Reality (XR: the umbrella term for Virtual + Augmented Reality) viewer for 3D medical images (MRI, CT-scans). Its breakthrough human-in-the-loop machine learning algorithms join perception and contextual knowledge to radically simplify use, improve productivity and enhance surgeon decision making. Visualizing muscles and bones, for example, allows better planning of orthopedic surgeries. Visualizing plaque in the lumen of blood vessels allows selection of the correct arteries for endovascular surgeries.
Our advanced rendering displays lossless 3D representations (“avatars”) for surgeons. This enables them to modify and optimize their patient representations in real-time (e.g. highlight or remove organs) for a visually comfortable and actionable visualization experience.
The AVATAR surgery solution is a groundbreaking productivity tool for surgeons and medical specialists. It is the first photorealistic XR viewer of 3D medical images, based on innovative machine learning algorithms and data visualization techniques, with the express objective of enhancing medical decision making.
Through high-fidelity local or Cloud-based render streaming, AVATAR will be the first solution to meet surgeon expectations for seamless preoperative and intraoperative data integration. Our Clinical Applications will assist surgeons in planning and implementing specific procedures.
For hospitals struggling to manage growing demand for surgery, AVATAR promises to provide valuable productivity gains without investing in new equipment or personnel.
AVATAR is also a powerful tool in communication with patients, engaging them with a more visual and intuitive approach during the patient journey.
In the project's first year, we invented a cloud architecture capable of managing demanding volume rendering algorithms and executing machine-learning-based image analysis. We have developed a robust segmentation algorithm for shoulder bones, a demanding application for which no public model nor dataset was available. We have combined those to develop a web-based surgery planning software capable of displaying volume renderings of soft tissues and of the inside of bones, with a Medtech partner developing shoulder implants. This key milestone, the Clinical App Shoulder, will be presented in the progress meeting.
We have developed key technological components that will be embedded in our future products. We figured out how to optimize ray tracing algorithms to implement them in demanding cloud computing contexts. These photorealistic volume renderings, including the modelling of the reflection of light, have been proven useful in a number of surgical applications. We have invented a solution to create “light renderings” while preserving an exploitable quality and interactivity. This solution is being patented and could solve the challenge of disseminating XR experiences from the cloud anywhere: phones, tablets, autonomous VR or AR headsets
We have developed key technological components that will be embedded in our future products. We figured out how to optimize ray tracing algorithms to implement them in demanding cloud computing contexts. These photorealistic volume renderings, including the modelling of the reflection of light, have been proven useful in a number of surgical applications. We have invented a solution to create “light renderings” while preserving an exploitable quality and interactivity. This solution is being patented and could solve the challenge of disseminating XR experiences from the cloud anywhere: phones, tablets, autonomous VR or AR headsets
AVATAR will have a positive impact for the healthcare market by improving hospital productivity and increasing the market’s ability to meet growing demand for surgeries. By promoting less invasive surgeries, avoiding secondary operations and fostering faster patient recovery, hospitals can provide service to a larger number of patients in the same amount of time in their existing facilities.
At the same time, AVATAR will have an impact on the upskilling of new surgeons, helping them understand the intricacies of medical images in months instead of years, which will ensure a steady supply of radiology competent surgeons in the future.
To achieve this goal, democratization and access are key. In this regard, our light rendering approach cracks the unsolved problem of exploiting volume rendering level of image quality while keeping portability and access.
For MedTech companies, AVATAR provides a key component to emergent medical technologies that require advanced imaging. This will have a positive impact on the market by allowing MedTechs to develop and commercialize new-generation solutions to meet the needs of the healthcare industry.
The product brAIn Shoulder Positioning (the Clinical App Shoulder) is the first web-based surgery planning software capable of displaying volume renderings of soft tissues and of the inside of bones.
At the same time, AVATAR will have an impact on the upskilling of new surgeons, helping them understand the intricacies of medical images in months instead of years, which will ensure a steady supply of radiology competent surgeons in the future.
To achieve this goal, democratization and access are key. In this regard, our light rendering approach cracks the unsolved problem of exploiting volume rendering level of image quality while keeping portability and access.
For MedTech companies, AVATAR provides a key component to emergent medical technologies that require advanced imaging. This will have a positive impact on the market by allowing MedTechs to develop and commercialize new-generation solutions to meet the needs of the healthcare industry.
The product brAIn Shoulder Positioning (the Clinical App Shoulder) is the first web-based surgery planning software capable of displaying volume renderings of soft tissues and of the inside of bones.