Periodic Reporting for period 1 - TreatStroke (arTificial neuRal nEtwork prediction of hAemorrhagic Transformation in STROKE.)
Période du rapport: 2017-09-29 au 2018-09-28
Short door-to-needle times are critical in Acute Ischemic Stroke (IS), as stroke progresses over time and hence treatment benefits are time dependent as well. State-of-the-art diagnosis of IS is based on Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) scanning. Current imaging allows assessment of irreversibly damaged tissue and surrounding tissue suffering reduced blood delivery, which is still viable albeit non-functional. Early identification of this salvageable tissue is crucial in management and treatment of patients suffering from acute IS to avoid further tissue injury. However, accurately determining the volume of salvageable tissue requires assessment by expert stroke radiologists and is time consuming – with a median door-to-needle time of 20 min, but normally reaching up to 60 min for more than 80% of the cases . In addition, IS assessment and decision-to-treat is highly subjective, i.e. different radiologists will present different results – a phenomenon known as inter-rater variability. A longstanding challenge in predicting tissue outcome is the well-known heterogeneity in IS progression. Whereas previous models implicitly assume a fixed progression pattern for all patients, we have developed an adaptive approach, which acknowledges the heterogeneity in IS progression.
Cercare Medical is developing a novel fully automated personalized clinical decision support system (CDSS) for optimal treatment of acute IS patients – the HTSafe solution under development at CS. This novel CDSS will be built using our recently developed technology – the AI-based Personalised Risk Map. This system automatically compares the MRI scan of an acute patient to thousands of scans of previous patients with a known outcome. Effectively this technology reduces the multitude of acute scan images to one single risk map, which is a high-resolution image of the most likely brain tissue outcome (infarct or tissue salvage) based on all previously monitored patients. Based on the predicted outcome, HTSafe must be capable of determining whether treatment should be offered through drug-based (anticoagulant) therapy or mechanical thrombolytic devices, if any. In particular, if the solution finds a risk of inducing a haemorrhage, i.e. so-called a haemorrhagic transformation (HT), only supportive treatment should be given. A solution capable of providing an accurate assessment of this situation is very valuable.
The objective of the project is to enable prediction of the feared hemorrhagic transformation based on acute imaging to assist physicians in selecting the optimal therapeutic strategy
Cercare Medical is developing a novel fully automated personalized clinical decision support system (CDSS) for optimal treatment of acute IS patients – the HTSafe solution under development at CS. This novel CDSS will be built using our recently developed technology – the AI-based Personalised Risk Map. This system automatically compares the MRI scan of an acute patient to thousands of scans of previous patients with a known outcome. Effectively this technology reduces the multitude of acute scan images to one single risk map, which is a high-resolution image of the most likely brain tissue outcome (infarct or tissue salvage) based on all previously monitored patients. Based on the predicted outcome, HTSafe must be capable of determining whether treatment should be offered through drug-based (anticoagulant) therapy or mechanical thrombolytic devices, if any. In particular, if the solution finds a risk of inducing a haemorrhage, i.e. so-called a haemorrhagic transformation (HT), only supportive treatment should be given. A solution capable of providing an accurate assessment of this situation is very valuable.
The objective of the project is to enable prediction of the feared hemorrhagic transformation based on acute imaging to assist physicians in selecting the optimal therapeutic strategy
We consider a dataset with 249 patients showing symptoms of acute ischemic stroke. All patients were scanned using perfusion-weighted CT. A total of 154 of the patients had a lesion on the 24-hours follow-up scan and was therefore included in this study. The total number of slices with a lesion was 1434 and the mean number of voxels in these lesions was 4028 (min: 10, max: 40748). The total number of slices affected by hemorrhage was 224 with a mean number of voxels of 883 (min: 1, max: 14954) and 39 patients in total.
For classification tasks like this, convolutional neural networks (CNN) are currently the go-to method in the field. They have the advantage of being able to include spatial information and multiple image-biomarkers and they have enough representational power to find and account for the expected complex interplay between image-biomarkers and the heterogeneity in stroke patients. We modified the VGG CNN architecture1 with 21 layers (in the 13 layers with convolutions we added a batch normalization2, we changed the downsampling to convolutional downsampling and edited the input layer to take 10 image-biomarkers instead of the usual three color channels used for image classification).
In order to account for the very unbalanced number of slices and voxels with lesion and hemorrhage and to counteract overfitting of the network, we used data augmentation to mimic more data. This was done by vertical flipping of the slices and rotation with an angel of ±15◦. Furthermore, we choose to train on patches which is 64x64 voxels randomly sampled subsets of the 128x128 voxels in the original data. This allowed the same slice to be in the data set several times. Several different data distributions were evaluated and we ended up with a 50/50 split of patches with lesion and hemorrhage. Additionally, we trained the networks using mini-batch stochastic gradient descent to stabilize the training process, obtain convergence faster and avoid too much adaption to the training set as the entire data set is not available to the network at once.
As input image-biomarkers, we used plain CT, the mean raw image and the parametric version of the following perfusion image-biomarkers: mean capillary transit time, cerebral blood flow, cerebral blood volume, delay, cerebral transit time heterogeneity, oxygen extraction function, cerebral metabolism of oxygen and relative transit time heterogeneity. The parameters were calculated as in Nielsen et. al.3 . The target was the follow-up lesion delineated on the 24-hours follow-up scan by an expert neuroradiologist.
To evaluate the model, we calculated sensitivity and specificity for several thresholds for the 296 slices in the test set with either a lesion or a hemorrhage (28 slices). Furthermore, we used area under the receiver operation characteristics curve (AUC) which has the advantage of being threshold-independent and can be interpreted as the probability of slice with hemorrhage receiving a higher haemorrhage-risk score than a slice with lesion.
The performance of the model in independent test data is characterized by an AUC of 67%. With a threshold of 0.6 the sensitivity is 0.82 the specificity is 0.48. In this case 23 out of 28 hemorrhages were correctly detected by the algorithm while only 5 were missed. On the other hand the algorithm overestimates the actual number of hemorrhages at this threshold. The specificity clearly improves with higher thresholds, at the cost of a drop in sensitivity, however with a threshold around 0.65 predictive value is demonstrated in sensitivity as well as specificity.
The number of patients is relatively low and more importantly the area of the hemorrhage is typically very small. The latter results in considerable class imbalance and lack of informative ‘positive’ cases. A collaboration has been established with the Stanford Department of Neuroradiology – Neuroimaging and Neurointervention and in this collaboration CT as well as MRI scans in acute stroke patients will be pooled allowing assessment of the risk of hemorrhagic transformation in a much larger cohort, building on the approaches developed in this project.
For classification tasks like this, convolutional neural networks (CNN) are currently the go-to method in the field. They have the advantage of being able to include spatial information and multiple image-biomarkers and they have enough representational power to find and account for the expected complex interplay between image-biomarkers and the heterogeneity in stroke patients. We modified the VGG CNN architecture1 with 21 layers (in the 13 layers with convolutions we added a batch normalization2, we changed the downsampling to convolutional downsampling and edited the input layer to take 10 image-biomarkers instead of the usual three color channels used for image classification).
In order to account for the very unbalanced number of slices and voxels with lesion and hemorrhage and to counteract overfitting of the network, we used data augmentation to mimic more data. This was done by vertical flipping of the slices and rotation with an angel of ±15◦. Furthermore, we choose to train on patches which is 64x64 voxels randomly sampled subsets of the 128x128 voxels in the original data. This allowed the same slice to be in the data set several times. Several different data distributions were evaluated and we ended up with a 50/50 split of patches with lesion and hemorrhage. Additionally, we trained the networks using mini-batch stochastic gradient descent to stabilize the training process, obtain convergence faster and avoid too much adaption to the training set as the entire data set is not available to the network at once.
As input image-biomarkers, we used plain CT, the mean raw image and the parametric version of the following perfusion image-biomarkers: mean capillary transit time, cerebral blood flow, cerebral blood volume, delay, cerebral transit time heterogeneity, oxygen extraction function, cerebral metabolism of oxygen and relative transit time heterogeneity. The parameters were calculated as in Nielsen et. al.3 . The target was the follow-up lesion delineated on the 24-hours follow-up scan by an expert neuroradiologist.
To evaluate the model, we calculated sensitivity and specificity for several thresholds for the 296 slices in the test set with either a lesion or a hemorrhage (28 slices). Furthermore, we used area under the receiver operation characteristics curve (AUC) which has the advantage of being threshold-independent and can be interpreted as the probability of slice with hemorrhage receiving a higher haemorrhage-risk score than a slice with lesion.
The performance of the model in independent test data is characterized by an AUC of 67%. With a threshold of 0.6 the sensitivity is 0.82 the specificity is 0.48. In this case 23 out of 28 hemorrhages were correctly detected by the algorithm while only 5 were missed. On the other hand the algorithm overestimates the actual number of hemorrhages at this threshold. The specificity clearly improves with higher thresholds, at the cost of a drop in sensitivity, however with a threshold around 0.65 predictive value is demonstrated in sensitivity as well as specificity.
The number of patients is relatively low and more importantly the area of the hemorrhage is typically very small. The latter results in considerable class imbalance and lack of informative ‘positive’ cases. A collaboration has been established with the Stanford Department of Neuroradiology – Neuroimaging and Neurointervention and in this collaboration CT as well as MRI scans in acute stroke patients will be pooled allowing assessment of the risk of hemorrhagic transformation in a much larger cohort, building on the approaches developed in this project.
In this work we showed that we are able to create an algorithm which goes beyond state-of-the-art by uitilizing microvascular and microvascular (CTH) information which can provide information beyond traditional flow (such as oxygenation status) which is highly relevant for hemorrhagic transformation. Furthermore we showed that this data can be obtained within the realm of routine advanced CT and MRI acute stroke workups. The next step will be to develop a fully automated risk analysis for HT in the individual patient and test its use in a large clinical dataset.