Final Report Summary - SCOLIO-SEE (3D-IMAGE PROCESSING SYSTEM FOR HELPING PHYSICIANS IN THE DIAGNOSIS AND MONITORING OF SCOLIOSIS)
Executive Summary:
Any change that has a positive impact on citizens’ health is relevant for all the European government agencies and companies involved in the medical and health sector.
Although it is known and recognized that the repeated usage of x-rays in the medical imaging context, has negative long-term effects on the patients’ health, it remains nevertheless a common practice to identify and diagnose the internal causes of a trauma. The X-Rays permit to assess at a time the causes of the pain and to respond of the need for surgery that in itself is also a risk.
The x-ray technology is part of the many human paradoxes, in the same way as our usage of fossil energies, a problem to a solution, the fossil fuel emission control, the Kyoto protocol. In the medical context, the sanitary protocols as to radiation exposure are controlled by safety procedures based on the dose rate (Sv), the time the body is exposed to that dose. To put the problem into perspective, a spinal X-ray correspond to the radiation per hour detected at Fukushima site (+1 day after the disaster), around 1.4 millisieverts (mSv), when the average dose of natural background radiation per person per years (varies widely) is around the 2.2 millisieverts (mSv) .
Whereas the usage of fossil energies is widely disputed and identified as one of the main causes of the climatic and environmental changes that will, directly or indirectly affect all the aspects of our social environment. On a scale not so far from this, it remains that the radiations from the Medical Sector represent 14% of the total amount of emitted radiations on the earth and the remaining 85% are from natural radiation .
From a technical and physical point of view, the reduction of emitted radiation by radiation-free system usage, presents the technical challenge of trying to lower our exposure energy band, while maintaining information content of an X-ray scan.
The SCOLIO-SEE project contributes to European policy objective Council Directive 97/43/Euratom of 30 June 1997 on health protection of individuals against the dangers of ionizing radiation in relation to medical exposure, and repealing Directive 84/466/Euratom by developing a technology to reduce or substitute radioactive techniques in the detection of scoliosis in children. The development of this technology as applied to healthy diagnostic or monitoring choices will encourage the development of radiation-free systems and contribute to the healthy living agenda.
Project Context and Objectives:
SCOLIOSIS is a three-dimensional deformity of the human spinal column, characterized by a side-to-side curvature of the spine and usually combined to rotation of the vertebrae. There are around 1.8 million people with scoliosis throughout the EU and presents at any age, but it is most common in children over 10 years of age, and is of paramount importance for the health and self-esteem of juveniles.
SCOLIOSIS diagnosis begins with the inclinometer, a tool designed to measure the trunk asymmetry, if there is a suspicion of scoliosis, generally, an X-ray is prescribed to establish the underlying cause to the curvature. By measuring the Cobb Angle , the golden standard metric, the physicians quantifies the scoliosis grade.
SCOLIO-SEE is a system that should facilitate the diagnostic and the monitoring of the spine; while reducing radiation exposure needs.
The SCOLIO-SEE approach is constructed around the extraction of the shared information between; radiological acquisitions; topographical acquisitions from the external surface of patient back; a set of physical measurements acquired manually; a set of processed parameters, in order to know if the spine behavior can be inferred, from the free-radiation acquisitions, spaced in time; in complement of an artificial neural network (ANN) working over the whole data of patients participating to the Scolio-SEE project
The SCOLIO-SEE project is subject to the ethical rules of data protection and privacy.
Project Results:
The SCOLIO-SEE project is divided into 4 main components, connected between them by a set of main objectives and rules oriented around the scoliosis metric, the Cobb Angle.
• The SCOLIO-SEE database gathering all the data from the patients participating to the SCOLIO-SEE project
• The artificial neural network (ANN) responsible of the prediction of the Cobb Angle from a set of parameters, of metrics, measured and extracted manually or semi-automatically from the patients data.
• The spine curve extraction from radiological acquisition and its fusion/mapping over its corresponding topographical acquisition.
• The registration and matching between two topographical acquisitions, spaced in time.
All the others come from the derivation of those components.
The Proposal
There are concepts in the SCOLIO-SEE proposal.
For example, if we could have also access to topographical information from the bending acquisition, given that the new prototype could be used during the generation of the database. The new structural support prototype for the 3D imager, a DIERS Formetric System provided by DIERS and equipped with a rotating arm, manipulated by a clinician. An evaluation performed by ISICO Clinic has allowed to show that the prototype was not sufficiently agile for being able to acquire with accuracy the Adam's forward bend test, in comparison with the quality of a standing acquisition.
Dictum proprium Dr. Stefano Negrini
• Medical Doctor, Specialist in Physical and Rehabilitation Medicine
• Associate Professor, University of Brescia, Italy
• Scientific Director, ISICO (Italian Scientific Spine Institute), Milan, Italy
• Scientific Referent, Care & Research Institute Don Gnocchi, Rovato (Bs), Italy
• Chief-Editor, European Journal of Physical and Rehabilitation Medicine
• President of SOSORT
The forward bending position (Adam's test) allows to reach a diagnosis of scoliosis deformity through the detection of the hump or prominence, even if it is not possible to define the origin of the deformity without a radiographic exam. In fact, while bending anteriorly the spine is gently extended and any scoliotic posture without deformity is totally solved, while the deformity reveals itself without allowing the spine to de-rotate completely and showing the hump/prominence. While standing there is a rotation, but this could appear also in postural abnormalities without any deformity: we suppose that this adjunct could greatly increase the diagnostic accuracy of the exam.
The Scolio-SEE database can start collecting the needed data from the feedbacks given by ISICO in its evaluation, DIERS and NEO have already started to work in the Next version of the prototype: The bending acquisition should be driven by the patient data or directly by the patient?
AUTH and ISICO have made a fantastic work, collecting and archiving the medical information from more than 150 patient(s). The role of the ISICO clinic, has been fundamental during the whole project; giving an outstanding scientific and medical support in the arcane of the scoliosis; a natural and ethical approach which involves the patients in the project, as well as the extra work provided, such as the classification and annotation of the radiological database. The SCOLIO-SEE project has received a favorable evaluation from the Ethical Committee of Milano.
The SCOLIO-SEE consortium would like to greatly thank again the parents and the children who have participated in the project, providing metrics, topographical and radiological acquisitions, which are their private data. We would like also to thank Dr. Lilian Mitrou, advisor in data protection, who made sure that the consortium complies with the ethical rules of data protection and privacy.
AUTH has driven with efficiency the SCOLIO-SEE ANN, the artificial neural network responsible of the prediction of the Cobb Angle from a set of parameters (63), of metrics, from the patients database. AUTH has developed an ANN that predicts the Cobb Angle with an error inferior to 10 degrees, for 100% of the 30 patients participating, and an error inferior to 5 degrees for 87 % of the patients of the 30 patients participating in the initial dataset and this from different subsets of parameters. Furthermore during the final ANN evaluation procedure the ANN achieved a Cobb Angle prediction with an error inferior to 5 degrees for a 91, and 96 % (depending in the different parameters – metrics subset that was used) of the 22 patients participating in the evaluation stage. The results confirm and improve those from the work initiated by Jaremko et al , by the usage of an ANN over data from the back surface, which predict the Cobb Angle with an error in the order of 10 degrees, for 84% of the 46 patients participating.
From SCOLIO-SEE ANN, a parameter detaches itself from the lot of parameters, since it is present in most of the subset configurations of parameters, as an indicator of prediction. This parameter comes from ISICO, it is already a metric since it relies on the integration of a path (2D or 3D) passing through the spine; the question asked is more about the domain of this metric: probabilistic, frequential or spatial. It's seems already to be a very good metric.
Since the results from the automatic spine curve extraction are not always sufficiently accurate for being exploitable as information, the scoliosis clinician user can intervene to either adjust the automatically extracted spine curve, or manually identify it on the X-ray by choosing points along the spine. On the other hand, the fusion with the topographical acquisition is automatically performed in an accurate and quick way (takes less than five seconds).
GEO provided a software that semi-automatically identifies the spine curve from an X-ray of any kind and resolution and then merges the extracted spine curve with the 3D model of the back produced by Formetric. Information is stored into a file for being then read by the next modules of SCOLIO-SEE SYSTEM in order to link and parametrize the spine curve with the external surface deformation.
Regarding the topographic space, ATEKNEA has developed a Local and Non-Rigid Deformation model, in order to capture all the Local deformations in a non-rigid way between topographical acquisitions, spaced in time, such that the information can be treated as a neural layer in 2D, of nodes without apparent connections between them, and from which a sub-layers filtration process, the tesselation, is responsible to build the most informative network from the set of all the initial and possible networks, following a Non-injective and Non-surjective recursive process.
From two topographical acquisitions, A and B; A provides the information path to B; B then checks the content of this information, the intent of A with its own content, and provides a path update to A, where A will now evaluate the intent of B until they converge to a steady state, A is mostly agree with the intent of B and B is mostly agree with the intent of A, A and B share a commune intent for rendering the information, following the Principle of superposition and the premise of a lattice-based cryptography structure .
For this last objective, a software solution, developed by ATEKNEA, presents the visual and numerical results generated by running the Local and Non-Rigid deformation model according to patient topographical data. We have also reviewed and compared our results with the papers from Berg et Al and Ang et Mitchell ; In comparison, our model provides the Local aspect to the Non-Rigid deformation, given that it is simply not based on the minimization or maximization of a function, such as the root-mean-square error (RMSE), also presented as an integral metrics.
The Final System
What's an x-ray toy model?
Based on the result of those first but principal objectives, a SCOLIO-SEE Contingency plan has been established in order to generate the SCOLIO-SEE Demonstrator, a software solution, developed by ATEKNEA, following the DIERS coding conventions and the processing of data in parallel, based on a GPU implementation.
The SCOLIO-SEE Demonstrator allows to MONITOR the deformation of the internal spine; from the motions of deformations registered locally between topographical acquisitions, spaced in time; and from the parametrization of the spine from the radiological acquisitions, once fused, mapped over the topographical acquisition. By the bias of Kinematics we provide motion to the spine kinetics, enclosed inside a topographical and parametric space. We can infer the deformation observed by the spine.
The X-Ray Toy Model
What if the Scolio-SEE Demonstrator and its Local and Non-Rigid Deformation model, are both numerically and sufficiently accurate to provide the spine motion?
Does it mean that the parametric space captures sufficiently well the underlying causes, responsible of the spine shape. We recall here that the parametric space is built from the same Local and Non-Rigid deformation model, except that we provide it a topographical acquisition and its flopped version, the chiral evaluation.
We designed an X-Ray Toy Model with the following components: A topographical acquisition placed at the center of the toy model. Two interleaved ring , controlling the noise shaping of the topographical data, we want to analyse a set of high frequency patterns; the poisson sampling acting as a high pass filter, filtering the noise, the «decay» from the high frequency data and providing the sampling pattern to run the Local and Non-Rigid deformation model.
We can finally execute a spectral and spatial analysis of the topographical acquisition by iterating over the spectrum of the first ring. We add a black box inside the kernel that allows to record the superposition activity for all the nodes situated over the topographic acquisition grid. And we classify this activity into four learning groups according to the kernel structure.
We present in this final report, the results from this simulation.
PHOTO I - On the left, the normal surface representation – On the right, the mean curvature representation – Courtesy of DIERS
PHOTO II - On the left, the raw data from the Black Box - On the right, the most shared features from the Black Box
The X-Ray Toy Model is run the commutative ring of prime numbers modulo 255 – The template size is of 3x3 – The surface coverage is adaptive
PHOTO III - Two sets of special features extracted from the Black Box – The green and black points form a set of features, for which we know if they pertains to the right or left part of the acquisition - The magenta and red points form a set of features, for which the model is unable to know if they pertains to the right or left part of the acquisition.
PHOTO IV - On the right, the radiological acquisition and the vertebrae annotation performed by ISICO - On the left, in green, the extracted spine curve fused manually with the topographical acquisition, in red and black, the features extracted by the x-ray toy model
(All the photos are separate documents attached to this Final Report)
Potential Impact:
Project logo, project dissemination materials and photographs illustrating and promoting the work of the project are attached to this Report as PDF. A promotional video (related to D9.7 - Demonstration Showcase) can be found in the project Youtube channel: https://www.youtube.com/watch?v=xOoE7Lm2Xqc
List of Websites:
http://www.scolio-see.eu
List of contact details is attached as PDF.
Any change that has a positive impact on citizens’ health is relevant for all the European government agencies and companies involved in the medical and health sector.
Although it is known and recognized that the repeated usage of x-rays in the medical imaging context, has negative long-term effects on the patients’ health, it remains nevertheless a common practice to identify and diagnose the internal causes of a trauma. The X-Rays permit to assess at a time the causes of the pain and to respond of the need for surgery that in itself is also a risk.
The x-ray technology is part of the many human paradoxes, in the same way as our usage of fossil energies, a problem to a solution, the fossil fuel emission control, the Kyoto protocol. In the medical context, the sanitary protocols as to radiation exposure are controlled by safety procedures based on the dose rate (Sv), the time the body is exposed to that dose. To put the problem into perspective, a spinal X-ray correspond to the radiation per hour detected at Fukushima site (+1 day after the disaster), around 1.4 millisieverts (mSv), when the average dose of natural background radiation per person per years (varies widely) is around the 2.2 millisieverts (mSv) .
Whereas the usage of fossil energies is widely disputed and identified as one of the main causes of the climatic and environmental changes that will, directly or indirectly affect all the aspects of our social environment. On a scale not so far from this, it remains that the radiations from the Medical Sector represent 14% of the total amount of emitted radiations on the earth and the remaining 85% are from natural radiation .
From a technical and physical point of view, the reduction of emitted radiation by radiation-free system usage, presents the technical challenge of trying to lower our exposure energy band, while maintaining information content of an X-ray scan.
The SCOLIO-SEE project contributes to European policy objective Council Directive 97/43/Euratom of 30 June 1997 on health protection of individuals against the dangers of ionizing radiation in relation to medical exposure, and repealing Directive 84/466/Euratom by developing a technology to reduce or substitute radioactive techniques in the detection of scoliosis in children. The development of this technology as applied to healthy diagnostic or monitoring choices will encourage the development of radiation-free systems and contribute to the healthy living agenda.
Project Context and Objectives:
SCOLIOSIS is a three-dimensional deformity of the human spinal column, characterized by a side-to-side curvature of the spine and usually combined to rotation of the vertebrae. There are around 1.8 million people with scoliosis throughout the EU and presents at any age, but it is most common in children over 10 years of age, and is of paramount importance for the health and self-esteem of juveniles.
SCOLIOSIS diagnosis begins with the inclinometer, a tool designed to measure the trunk asymmetry, if there is a suspicion of scoliosis, generally, an X-ray is prescribed to establish the underlying cause to the curvature. By measuring the Cobb Angle , the golden standard metric, the physicians quantifies the scoliosis grade.
SCOLIO-SEE is a system that should facilitate the diagnostic and the monitoring of the spine; while reducing radiation exposure needs.
The SCOLIO-SEE approach is constructed around the extraction of the shared information between; radiological acquisitions; topographical acquisitions from the external surface of patient back; a set of physical measurements acquired manually; a set of processed parameters, in order to know if the spine behavior can be inferred, from the free-radiation acquisitions, spaced in time; in complement of an artificial neural network (ANN) working over the whole data of patients participating to the Scolio-SEE project
The SCOLIO-SEE project is subject to the ethical rules of data protection and privacy.
Project Results:
The SCOLIO-SEE project is divided into 4 main components, connected between them by a set of main objectives and rules oriented around the scoliosis metric, the Cobb Angle.
• The SCOLIO-SEE database gathering all the data from the patients participating to the SCOLIO-SEE project
• The artificial neural network (ANN) responsible of the prediction of the Cobb Angle from a set of parameters, of metrics, measured and extracted manually or semi-automatically from the patients data.
• The spine curve extraction from radiological acquisition and its fusion/mapping over its corresponding topographical acquisition.
• The registration and matching between two topographical acquisitions, spaced in time.
All the others come from the derivation of those components.
The Proposal
There are concepts in the SCOLIO-SEE proposal.
For example, if we could have also access to topographical information from the bending acquisition, given that the new prototype could be used during the generation of the database. The new structural support prototype for the 3D imager, a DIERS Formetric System provided by DIERS and equipped with a rotating arm, manipulated by a clinician. An evaluation performed by ISICO Clinic has allowed to show that the prototype was not sufficiently agile for being able to acquire with accuracy the Adam's forward bend test, in comparison with the quality of a standing acquisition.
Dictum proprium Dr. Stefano Negrini
• Medical Doctor, Specialist in Physical and Rehabilitation Medicine
• Associate Professor, University of Brescia, Italy
• Scientific Director, ISICO (Italian Scientific Spine Institute), Milan, Italy
• Scientific Referent, Care & Research Institute Don Gnocchi, Rovato (Bs), Italy
• Chief-Editor, European Journal of Physical and Rehabilitation Medicine
• President of SOSORT
The forward bending position (Adam's test) allows to reach a diagnosis of scoliosis deformity through the detection of the hump or prominence, even if it is not possible to define the origin of the deformity without a radiographic exam. In fact, while bending anteriorly the spine is gently extended and any scoliotic posture without deformity is totally solved, while the deformity reveals itself without allowing the spine to de-rotate completely and showing the hump/prominence. While standing there is a rotation, but this could appear also in postural abnormalities without any deformity: we suppose that this adjunct could greatly increase the diagnostic accuracy of the exam.
The Scolio-SEE database can start collecting the needed data from the feedbacks given by ISICO in its evaluation, DIERS and NEO have already started to work in the Next version of the prototype: The bending acquisition should be driven by the patient data or directly by the patient?
AUTH and ISICO have made a fantastic work, collecting and archiving the medical information from more than 150 patient(s). The role of the ISICO clinic, has been fundamental during the whole project; giving an outstanding scientific and medical support in the arcane of the scoliosis; a natural and ethical approach which involves the patients in the project, as well as the extra work provided, such as the classification and annotation of the radiological database. The SCOLIO-SEE project has received a favorable evaluation from the Ethical Committee of Milano.
The SCOLIO-SEE consortium would like to greatly thank again the parents and the children who have participated in the project, providing metrics, topographical and radiological acquisitions, which are their private data. We would like also to thank Dr. Lilian Mitrou, advisor in data protection, who made sure that the consortium complies with the ethical rules of data protection and privacy.
AUTH has driven with efficiency the SCOLIO-SEE ANN, the artificial neural network responsible of the prediction of the Cobb Angle from a set of parameters (63), of metrics, from the patients database. AUTH has developed an ANN that predicts the Cobb Angle with an error inferior to 10 degrees, for 100% of the 30 patients participating, and an error inferior to 5 degrees for 87 % of the patients of the 30 patients participating in the initial dataset and this from different subsets of parameters. Furthermore during the final ANN evaluation procedure the ANN achieved a Cobb Angle prediction with an error inferior to 5 degrees for a 91, and 96 % (depending in the different parameters – metrics subset that was used) of the 22 patients participating in the evaluation stage. The results confirm and improve those from the work initiated by Jaremko et al , by the usage of an ANN over data from the back surface, which predict the Cobb Angle with an error in the order of 10 degrees, for 84% of the 46 patients participating.
From SCOLIO-SEE ANN, a parameter detaches itself from the lot of parameters, since it is present in most of the subset configurations of parameters, as an indicator of prediction. This parameter comes from ISICO, it is already a metric since it relies on the integration of a path (2D or 3D) passing through the spine; the question asked is more about the domain of this metric: probabilistic, frequential or spatial. It's seems already to be a very good metric.
Since the results from the automatic spine curve extraction are not always sufficiently accurate for being exploitable as information, the scoliosis clinician user can intervene to either adjust the automatically extracted spine curve, or manually identify it on the X-ray by choosing points along the spine. On the other hand, the fusion with the topographical acquisition is automatically performed in an accurate and quick way (takes less than five seconds).
GEO provided a software that semi-automatically identifies the spine curve from an X-ray of any kind and resolution and then merges the extracted spine curve with the 3D model of the back produced by Formetric. Information is stored into a file for being then read by the next modules of SCOLIO-SEE SYSTEM in order to link and parametrize the spine curve with the external surface deformation.
Regarding the topographic space, ATEKNEA has developed a Local and Non-Rigid Deformation model, in order to capture all the Local deformations in a non-rigid way between topographical acquisitions, spaced in time, such that the information can be treated as a neural layer in 2D, of nodes without apparent connections between them, and from which a sub-layers filtration process, the tesselation, is responsible to build the most informative network from the set of all the initial and possible networks, following a Non-injective and Non-surjective recursive process.
From two topographical acquisitions, A and B; A provides the information path to B; B then checks the content of this information, the intent of A with its own content, and provides a path update to A, where A will now evaluate the intent of B until they converge to a steady state, A is mostly agree with the intent of B and B is mostly agree with the intent of A, A and B share a commune intent for rendering the information, following the Principle of superposition and the premise of a lattice-based cryptography structure .
For this last objective, a software solution, developed by ATEKNEA, presents the visual and numerical results generated by running the Local and Non-Rigid deformation model according to patient topographical data. We have also reviewed and compared our results with the papers from Berg et Al and Ang et Mitchell ; In comparison, our model provides the Local aspect to the Non-Rigid deformation, given that it is simply not based on the minimization or maximization of a function, such as the root-mean-square error (RMSE), also presented as an integral metrics.
The Final System
What's an x-ray toy model?
Based on the result of those first but principal objectives, a SCOLIO-SEE Contingency plan has been established in order to generate the SCOLIO-SEE Demonstrator, a software solution, developed by ATEKNEA, following the DIERS coding conventions and the processing of data in parallel, based on a GPU implementation.
The SCOLIO-SEE Demonstrator allows to MONITOR the deformation of the internal spine; from the motions of deformations registered locally between topographical acquisitions, spaced in time; and from the parametrization of the spine from the radiological acquisitions, once fused, mapped over the topographical acquisition. By the bias of Kinematics we provide motion to the spine kinetics, enclosed inside a topographical and parametric space. We can infer the deformation observed by the spine.
The X-Ray Toy Model
What if the Scolio-SEE Demonstrator and its Local and Non-Rigid Deformation model, are both numerically and sufficiently accurate to provide the spine motion?
Does it mean that the parametric space captures sufficiently well the underlying causes, responsible of the spine shape. We recall here that the parametric space is built from the same Local and Non-Rigid deformation model, except that we provide it a topographical acquisition and its flopped version, the chiral evaluation.
We designed an X-Ray Toy Model with the following components: A topographical acquisition placed at the center of the toy model. Two interleaved ring , controlling the noise shaping of the topographical data, we want to analyse a set of high frequency patterns; the poisson sampling acting as a high pass filter, filtering the noise, the «decay» from the high frequency data and providing the sampling pattern to run the Local and Non-Rigid deformation model.
We can finally execute a spectral and spatial analysis of the topographical acquisition by iterating over the spectrum of the first ring. We add a black box inside the kernel that allows to record the superposition activity for all the nodes situated over the topographic acquisition grid. And we classify this activity into four learning groups according to the kernel structure.
We present in this final report, the results from this simulation.
PHOTO I - On the left, the normal surface representation – On the right, the mean curvature representation – Courtesy of DIERS
PHOTO II - On the left, the raw data from the Black Box - On the right, the most shared features from the Black Box
The X-Ray Toy Model is run the commutative ring of prime numbers modulo 255 – The template size is of 3x3 – The surface coverage is adaptive
PHOTO III - Two sets of special features extracted from the Black Box – The green and black points form a set of features, for which we know if they pertains to the right or left part of the acquisition - The magenta and red points form a set of features, for which the model is unable to know if they pertains to the right or left part of the acquisition.
PHOTO IV - On the right, the radiological acquisition and the vertebrae annotation performed by ISICO - On the left, in green, the extracted spine curve fused manually with the topographical acquisition, in red and black, the features extracted by the x-ray toy model
(All the photos are separate documents attached to this Final Report)
Potential Impact:
Project logo, project dissemination materials and photographs illustrating and promoting the work of the project are attached to this Report as PDF. A promotional video (related to D9.7 - Demonstration Showcase) can be found in the project Youtube channel: https://www.youtube.com/watch?v=xOoE7Lm2Xqc
List of Websites:
http://www.scolio-see.eu
List of contact details is attached as PDF.
