Final Report Summary - SKINDETECTOR (Application of the innovative data fusion based non-invasive approach for management of the diabetes mellitus)
Executive Summary:
The SkinDetector project is a Research for SMEs (R4S) project framed in the FP7 European commission projects.
The SkinDetector prototype system allows acquiring information using three different types of sensors that have to be properly combined to facilitate the medical doctors a final and accurate diagnosis. These three sensors/technologies are: thermography camera, digital dermatoscope and phased array ultrasound device.
The feet skin temperature can be inspected by a thermography camera. The thermal comparation of both feet will highlight the feet sole areas in where the temperature different is higher than a threshold using isothermal lines, fig1. The thermography system was validated on healthy and non healthy patients (clinic trials).
A set of optical filters, fig4, have been designed and manufactured to improve the quality of the dermatoscope picture and the visualization of the skin markers that are needed to evaluate or diagnostic the skin lesions. Additionally, a colour and size analysis of the skin lesion can be also carried out by means of an specific software developed within the Skindetector project, fig5. The Skindetector dermatoscope system was tested and validated on healthy and non-healthy patients (clinic trials), fig3.
The goal of the ultrasound analysis is to measure the skin thickness of the feet. An ultrasound scanning device prototype was designed, manufactured and tested. Fig6. Additionally, a 8 channel ultrasound front end, a phase coherence algorithm and a high speed data PCIe connection were also designed, implemented and tested, fig 9 and 10. According to the advices and suggestions of an external auditor, the ultrasound system was validated on special phantoms having the curvature of metatarsus like the real diabetic foot and multi-layered structure of the particular region in order to mimic wound, callus and ulcer, fig 11. and also on the real samples of the multi-layered biological tissue (fresh meat of pork) due to healthy and safety issues and its current TRL (Technology Readiness Level) level. The interfaces between fibers of the muscles and fat regions were clearly indicated in ultrasonic image.
The 3D beamforming algorithms, data visualisation and analysis software were also successfully executed and finalize, fig 8, 11, 12 and 13.
Project Context and Objectives:
The SkinDetector project is a Research for SMEs (R4S) project framed in the FP7 European commission projects.
A person is said to suffer from Diabetes Mellitus (DM) or Diabetes when one’s own body does not produce insulin which is needed to control blood sugar levels (Type 1) or when the body does produce sufficient insulin but cannot not use the insulin produced properly i.e. insulin resistant (Type 2). Another type of diabetes is Gestational diabetes mellitus that occur when pregnant women have high blood glucose levels due to hormones produced in pregnancy which could lead to Type2 diabetes. There are other more rare types of diabetes too.
The aim of this project is to develop a tool for early detection, accurate diagnosis and monitoring that will reduce the microvascular (the damage of small arteries) and macrovascular (damage to large arteries) that arise from complications of diabetes mellitus.
The project aims to develop a portable set of equipment that can be used at a general practitioner’s surgery and/or local primary care clinics or even at a patient’s home. Furthermore with the capabilities of the modern telemedicine, the possibility of analysing the clinical investigation results via a tele-consultation (with tertiary level specialist physicians) and making available a prognosis and a treatment plan is another possible outcome. Thus this has much benefit to rural or underserved areas that perform clinical investigation of patients with Diabetes Mellitus but may not have a resident specialist.
The system will be capable of measuring the external temperature of the patient’s feet using a Thermographic camera, take pictures of the complications using a digital dermatoscope “Smartscope” and also be able to acquire ultrasonic data, by means of a high frequency phased array system and a linear phased array probe that would be positioned over the Skin (e.g. feet) manually but would have automatic linear movement of the arrays to produce a complex ‘picture’ of data; this collective data would allow clinicians/physicians to assess the severity of the skin complication in one consultation (visit); thereby helping with early ‘one-stop’ accurate diagnosis and a treatment plan.
Diabetes mellitus (DM) is an incurable chronic disease associated with invariable hyperglycemia and carbohydrate and fat dysmetabolism, resulting in multiple organ damage. The diabetic foot is the main cause of hospitalization and amputation among DM patients. Neuropathy and angiopathy are considered to be major risk factors for diabetic foot ulceration. According to studies, usually foot temperature increases prior to the formation of ulcers, while reduced temperature correlates with the degree of the severity of peripheral circulatory disorders and neuropathy. In another cases, we can observe increasing ischemia of foot tissues which also leads to irreversible damage and subsequently amputation.
On development of the collective system consisting of a thermographic camera, dermatoscope and ultrasonic transducers, the Skin Detector system will be tested on Skin Phantoms and on a real patient clinical study with 150DM patients with no ulcers, 150 DMs with ulcers and 100 control ‘patients’. We believe this will lead to the possibility of early diagnostics of the diabetic foot and its complications in the skin and soft tissues thereby helping to attest further more severe progression of DM. Scientifically, the results of the study will be useful for the evaluation of the sensitivity and specificity of non-invasive physical examination techniques in detecting clinical markers important for the diagnostics of the diabetic foot and its complications and thus prevent debilitating amputations.
Project Results:
From the beginning of the project, the project partners made a great effort developing the Skindetector system and fulfil the project requirements. An early clinical diagnosis of the diabetic foot and its complications in the skin and soft tissues thereby helping to attest further more severe progression of Diabetis mellitus. Some of the most important consequences of Diabetis mellitus such as, risk of skin infections, ulceration and even amputation in diabetes patients, may directly impact on the patient life changing their habits and quality of life.
At this stage, all internal project delays have been properly addressed by the project partners without affecting the overall project schedule and all deliverables have been officially and successfully reported (submitted). All technical tasks have been also successfully executed and all project milestones achieved.
WP1: Project coordination and management
All WP1 deliverables have been successfully submitted: D1.1 D1.2 D1.3 and D1.4
From the beginning of the project, the project coordinator and project partners have executed all technical tasks in order to fulfil the project requirements and specifications. All tasks have been properly synchronized and coordinated in order to go ahead with the development process of the project and to avoid unnecessary delays. All the internal delays raised up during the project have been properly managed to do not affect the evolution and development process of other internal tasks and/or the overall project schedule. Several project, teleconference and telephone meetings have been organized and scheduled. A fluid communication process between the partners has been completely necessary in order to finalize all technical tasks and to fulfil the project goals on time. The project coordinator have supported the rest of the SMEs in terms of financial reporting process and coordination activities and the SMEs have supported the RTDs in order to complete the technical work. No consortium modifications have been done and no project extensions have been needed.
As it was previously described, all technical and management deliverables have been successfully reported (submitted) in line with the official project deadline. The official progress reports (reporting period 1 and 2, and final progress report) and the additional progress reports requested by the project officer and reviewer have been submitted on time in line with the project deadlines.
WP2: Project specifications (Definition of requirements)
All WP2 deliverables have been successfully submitted: D2.1 D2.2 and D2.3 and the milestone MS1 achieved.
The Overall system initial requirements, detailed subsystem specifications and the specification revision and final agreement were successfully defined, agreed by all partners and submitted during the first reporting period. These project specifications have been used for the project partners as baselines for the technical development.
WP3: Development of the special optical hardware (digital dermatoscope and thermography device)
All WP3 deliverables have been successfully submitted: D3.1 D3.2 and D3.3 and the milestones MS2 and MS3 achieved.
During Work Package 3, a series of prototypes expanding the capabilities of the Smartscope’s SK1 module have been developed. 3D computer aided design tools were used coupled with optical simulations to redesign the module and additive manufacturing was used to produce them. The electronics had to be changed as well and new PCB boards were designed, manufactured and assembled. Medical trials using the prototypes were conducted and helped understand which of the prototypes were most useful. Final, production grade designs were then generated for future use by the end user. An additional design was also generated to tackle the need for higher magnifications utilizing a microscope objective lens. Fig 3 and 4.
A MATLAB program was successfully developed to perform object segmentation and analysis in an image and specifically in images acquired from a dermatoscope. The program runs quickly and is easy to use something particularly helpful to general practitioners and local primary care clinics that evaluate skin spots and perform diagnoses. Moreover, it gives a precise colour analysis and size of the object analysed. The segmentation and analysis algorithms have been successfully demonstrated on ink areas and were refined afterwards using real skin spots. The algorithm used can detect several objects in the same image and can also detect details within regions, requiring only a minimum level of in homogeneity between objects. As skin images can present in homogeneity, two solutions were developed and reported. Fig 5.
A set of mathematical algorithms and specific software were designed, codified and tested to measure, using a thermography camera, the temperature different between the patient feet according to a threshold previously defined by the medical doctors. The highlighted areas, using isotherms lines, will be analysed in detail by the doctor by means of the other two Skindetector techniques (Visual -Dermatoscope- and ultrasound inspection). In order to ensure a good quality of the inputs RAW data (thermography pictures), the camera has to be fixed on a tripod system, designed and manufactured with the project frame, and the patient has to be conformable placed on a massage table to ensure a constant distance between the camera and the patient feet. Both feet have to be immobilized during the thermography picture acquisition process in order to improve the quality of the results. Fig 1 and 2.
WP4: Development of the special ultrasonic hardware:
All WP4 deliverables have been successfully submitted: D4.1 D4.2 D4.3 D4.4 and D4.5 and the milestone MS4 achieved.
A prototype 8 channel ultrasound front end have been developed within this work package. Through iterative design and continuous effort, a fully functional prototype was designed, manufactured, assembled, tested and reported. The 8 channel board is a fully parameterized 8 channel pulse/receive system utilizing MAX4940 high voltage digital quad pulsers and an AD9279 octal analog to digital converter (ADC). The system can drive piezoelectric transducers with tunable high voltage pulses and captures the response signals coming from the same piezo elements. The data is acquired and sent directly to the DDR3 memory located on the FPGA board. All the configurations of the design’s operation are done through AXI interfaces, increasing the reusability and versatility of the FPGA design. The work in this task covers all the base work needed to be done if a company wanted to produce a new ultrasonic device based on latest integrated circuit components and the latest FPGAs. Fig 9.
A high speed data connection solution was developed to enable ultrasound devices to rapidly transfer data to a host pc. To be more precise a solution to transfer data between an FPGA card and a host computer running a Microsoft Windows operating system was developed. The solution can achieve transaction speeds above 1300MB/s if operated in FIFO mode. When the transaction takes place between the DDR3 memory of the FPGA board we can achieve write speeds of 800MB/s and read speeds of 533MB/s. An ultrasound data generator simulating an overload of the data transfer design was also developed. In this case the speeds fall to 650MB/s for the write and 450MB/s for the read. Even the slower speeds achieved in the task are an order of magnitude higher than any ultrasonic device currently in the market or officially announced by key suppliers worldwide. Scaling up the channel count the solution can allow a 128 channel ultrasound system to operate and transfer 16KSamples long Full Matrix Capture measurements (12bit/sample) at a theoretical pulse repetition frequency of 150Hz. After carrying out several lab tests, a transaction speeds of 2000 Mbytes has been achieved. Fig 10.
The phase coherence processing algorithm was also developed. Phase coherence processing is a signal processing technique used for adaptive beamforming applications. It is adaptive in the sense that it utilizes information from the signals as they are received in order to choose their processing factors. The phase coherence factors have been proposed in order to improve lateral resolution, suppress side and grating lobes and enhance signal-to-noise ratio. They evaluate the variance in phases of echo signals received by individual transducer elements after delay compensation has been applied. The phase coherence factor suppresses speckle echoes because phases of speckle echoes fluctuate due to their mutual interference. The development was done using the Matlab programming environment. Its performance was tested using simulated signals generated by CIVA™ and then polluted with artificial speckle noise.
A ultrasound scanning device prototype was also designed, manufactured, assembled and tested to ensure smooth mechanical movements of the phasedArray probe from Imasonic and a very high measurement repeatability and resolution. The inspected area is equal to 50x40mm and its resolution is <=0.05mm. The transducers movement is generated by a miniature stepper motor and driven by a lead screw. The control board and limit sensors are installed on board. An embedded software was designed and codified to implement the application state machine. Fig 6.
The integration activities and validation tests were carried out to verify the performances of the scanning device using the SITAU PRT device from Dasel. According to the results of these tests, an iterative redesign and manufacturing process was carried out. In this same line, the 8 channel ultrasound front end, the high speed communication PCIe connexion and the phase coherence algorithm were also tested and validated after several tests performed by the project partners. Fig 7.
WP 5: Development of the sophisticated software:
All WP5 deliverables have been successfully submitted: D5.1 D5.2 D5.3 and D5.4 and MS5 achieved.
The Matlab based control software (routines) for ultrasound system control and data acquisition was developed, user manual was prepared. The importance of 3D beamforming algorithm for data processing after gathering from the multi-channel phased array unit is related to improving quality of the visualized ultrasonic image, signal to noise ratio and spatial resolution. In order to achieve these objectives it has been chosen to use the selected methods Synthetic Aperture Focusing Technique (SAFT) and Total Focusing Method (TFM). Both methods were proposed, analysed and implemented in order to reconstruct the high resolution images of FBH and SDH (elastomer sample).
3D beamforming (SAFT and TFM) software engine for GPUs processing was developed and implemented into environment of computational package “Matlab R2013a”. Additionally, testing of computation performance of the “Matlab” based GPU processor for SAFT and TFM ultrasound processing was performed using CPU resources only and the combined resources of CPU - GPU (heterogeneous system). System parameters used for testing: CPU 2 x Intel Xeon E5606 (2.13 GHz), RAM 64 GB, GPU GeForce GTX 680 2GB.
The results of the processing time dependence on number of the grid elements (with or without GPU) in the cases of SAFT, TFM and creation of the C-scan images (SAFT processing) were obtained. It was observed a clear sharp “jump” of processing time of heterogeneous system (CPU-GPU), which reaches the computation time of single CPU performance. It was observed that there is sufficiently “small” region, where single CPU process data faster than heterogeneous system (CPU-GPU), but only until the number of ROI grid elements is sufficiently “low” (until reach value of 3e4 – 8e4).
On the other hand, the algorithms for image enhancement, image segmentation (Otsu, K-means), rendering and 3D image creation were developed. The Matlab based routines for 3D data fusion, visualization and analysis software (measurements, rotation), were integrated and user manual were developed. Application of the different non-invasive imaging techniques (thermography, digital optical dermatoscopic imaging and ultrasonic techniques) gives the additional information about the lesions of the skin affected by diabetus mellitus. For that purpose interactive graphical interfaces of 3D ultrasonic data navigation, analysis and fusion with optical / thermography / 3D camera images were also developed.
From the novel volumetric imaging of the ultrasonic data (demonstration was performed on special phantoms) medical doctors are getting more informative image for screening and early stage diagnosis: spatial 3D distribution of the non-homogeneity (lesion width, length and depth), highlighted contour and thickness of the adjacent layers of superficial tissue lesions.
WP6: Integration of the overall system:
All WP6 deliverables have been successfully submitted: D6.1 D6.2 and D6.3 and the milestone MS6 achieved.
Within this work package and in line with the engineering design and manufacturing processes during the prototype stages, several tests were carried out to calibrate the Skindetector systems and to validate their performances. A detailed explanation of these validation and calibration tests can be found into the deliverable D6.2.
Additionally, within the deliverable D6.1 the end user can see a detailed description about the integration activities that were carried out and how each of the Skindetector subsystems (Thermography, dermatoscope and ultrasound) have to be managed and operated (Technical documents and manuals of the application). Fig 11.
Also, as part of the normal engineering process and taking into account the results from the internal and official tests, several design and manufacturing iteration processes were executed in order to resolve the prototype technical issues raised up. All these technical issues, and their corrective actions, were described in the deliverable D6.3.
WP7: medical validation:
All WP7 deliverables have been successfully submitted: D7.1 D7.2 and D7.3.
During the SkinDetector framework clinical study (approved by Kaunas Regional Biomedical Research Ethics Committee), the project partners investigated skin temperature (separately in 8 anatomic points and mean temperature in both feet), epidermal thickness (separately in 5 points and mean thickness), and the skin surface of diabetic feet (n =28) and controls (n =13). Also, we examined 10 diabetic patients with type 2 diabetes mellitus and foot ulcer. The following diagnostic techniques were used: 1) Infrared camera (temperature sensitivity - 0.05 °C, resolution - 320x240 pixels, emission coefficient - 0.98) ; 2) 22 MHz ultrasound (Taberna pro Medicum); 3) Dermatoscope Optomed (x 6 magnification) with addition modules of blue, red, green and infrared light filters. Also we have determined the sensitivity and specificity of the thermography program for the evaluation of temperature differences between both feet.
For the clinical validation of the ultrasound system, an external auditor analysed the need of clinical validations of all 3 systems considering the TRL (Technology Readiness Level) of each system individually. Auditor identified that the Phased Array Ultrasonic Imaging Device (Prototype V1.0 2014) does not require clinical evaluation performing clinical investigations (trial) due to the fact that prototype is not near commercialisation stage (TRL 8) as compare to the other two SkinDetector System. Phased Array unit system requires more detailed clinical validation after optimisation of current prototype by conducting further demonstration/research and to incorporate the results of demonstrations into a prototype etc. The external auditor suggested that SkinDetector consortium carry on more pre-clinical trial studies such as clinical literature evaluation and validation in action on skin phantom, fresh meat without clinical trial on human body.
Parallel we investigated 30 melanocytic nevi before excision and after this procedure using dermatoscope with addition modules of 4 light filters. From medical point it has been concluded, that skin temperature of the feet can be effective inspected by a SkinDetector thermography system. A digital dermatoscope allows for a better visual inspection of the feet and ulcer. According to the literature review and performed clinical study using commercialised ultrasound device in similarly frequency as created prototype in project, ultrasound gives accurate measuring the skin thickness of the feet. All biophysical parameters are very important skin markers in the formation of diabetic foot ulcer. The results of medical validation were documented in deliverables of the project (D7.1 Practical guidelines on the use of SkinDetector; D7.3 Feasibility and effectiveness of the SkinDetector system).
WP8: Training:
All WP8 deliverables have been successfully submitted: D8.1 and D8.2.
During the project period the training programme for SkinDetector system was prepared and presented in practical workshop which was held in Kaunas, 24th October, 2014. The seminar was organized by LUHS and KTU UI, fig 14. All project partners were officially informed and invited during the official meeting celebrated at Dasel facilities on 25th June, 2014, Madrid, Spain. The aim of the programme was to promote the developed SkinDetector technology by involving the end-users – physicians and project partners. The training programme was introduced according to the terms of the project and documented within the deliverable D8.1 and D8.2 . In the organized SkinDetector seminar, 30 medical specialists (dermatologists, endocrinologists, vascular surgeons) and family doctors participated.
The official project video, produced as part of the exploitation and dissemination activities, introduces and presents from a practical point of view, how the SkinDetector system have to be managed and operated. The dermatoscope and the thermography systems are tested on real patients and the ultrasound scanning device, due to the final decision from the ethics committee, was demonstrated on the diabetic foot phantom.
WP9: Dissemination and exploitation:
All WP9 deliverables have been successfully submitted: D9.1 D9.2 D9.3 D9.4 and D9.5 and the milestone MS7 achieved.
The overall exploitation and dissemination planning in SkinDetector has been geared towards maximising the economic benefit to the SMEs as quickly as possible following the end of the project. In order to reach the wider audience outside the project, the consortium has developed various dissemination means, such as project website, NDT wiki, video and flyers. Partners had also disseminated the project in their own websites. During the two years of the project, SkinDetector was presented in 14 conferences and workshops, 6 publications were written.
It is important to highlight the results of SkinDetector clinical study were officially presented at 12th Congress of Baltic Association of Dermatovenereologists (30-31 May, 2014, Riga, Latvia) and during the meeting of the project (on 25th June, 2014, Madrid, Spain).
The official project video was produced and it will be reproduced from the official project website.
Dissemination and exploitation activities will continue after completion of the SkinDetector project. SMEs are still planning to actively attend various events and disseminate the results of the project as well as they are planning to contact the manufacturers/end-users in order to explore the exploitation possibilities.
Potential Impact:
The final outcome of the project is the ‘Skin Detector’ prototype that enables early clinical diagnosis of the diabetic foot and its complications in the skin and soft tissues thereby helping to attest further more severe progression of Diabetis mellitus. The potential impacts of the SkinDetector system is to minimize/avoid the risk of skin infections, ulceration and even amputation in diabetes patients.
Scientifically, the results of the clinical study validated the sensitivity and specificity of non-invasive physical examination techniques in detecting, in less than 30 minutes, clinical markers important for the diagnostics of the diabetic foot and its complications. These markers will be analyzed by the medical doctors in order to produce a final diagnostic.
In order to reach the wider audience outside the project, the consortium has developed various dissemination means, such as project website, NDT wiki, video and flyers. Partners had also disseminated the project in their own websites. During the two years of the project, SkinDetector was presented in 14 conferences and workshops, 6 publications were written. The final results of SkinDetector clinical study were officially presented at 12th Congress of Baltic Association of Dermatovenereologists (30-31 May, 2014, Riga, Latvia) and during the meeting of the project (on 25th June, 2014, Madrid, Spain).
List of Websites:
http://www.skindetector.eu/(odnośnik otworzy się w nowym oknie)
http://www.ndtwiki.com/index.php/SkinDetector(odnośnik otworzy się w nowym oknie)
The SkinDetector project is a Research for SMEs (R4S) project framed in the FP7 European commission projects.
The SkinDetector prototype system allows acquiring information using three different types of sensors that have to be properly combined to facilitate the medical doctors a final and accurate diagnosis. These three sensors/technologies are: thermography camera, digital dermatoscope and phased array ultrasound device.
The feet skin temperature can be inspected by a thermography camera. The thermal comparation of both feet will highlight the feet sole areas in where the temperature different is higher than a threshold using isothermal lines, fig1. The thermography system was validated on healthy and non healthy patients (clinic trials).
A set of optical filters, fig4, have been designed and manufactured to improve the quality of the dermatoscope picture and the visualization of the skin markers that are needed to evaluate or diagnostic the skin lesions. Additionally, a colour and size analysis of the skin lesion can be also carried out by means of an specific software developed within the Skindetector project, fig5. The Skindetector dermatoscope system was tested and validated on healthy and non-healthy patients (clinic trials), fig3.
The goal of the ultrasound analysis is to measure the skin thickness of the feet. An ultrasound scanning device prototype was designed, manufactured and tested. Fig6. Additionally, a 8 channel ultrasound front end, a phase coherence algorithm and a high speed data PCIe connection were also designed, implemented and tested, fig 9 and 10. According to the advices and suggestions of an external auditor, the ultrasound system was validated on special phantoms having the curvature of metatarsus like the real diabetic foot and multi-layered structure of the particular region in order to mimic wound, callus and ulcer, fig 11. and also on the real samples of the multi-layered biological tissue (fresh meat of pork) due to healthy and safety issues and its current TRL (Technology Readiness Level) level. The interfaces between fibers of the muscles and fat regions were clearly indicated in ultrasonic image.
The 3D beamforming algorithms, data visualisation and analysis software were also successfully executed and finalize, fig 8, 11, 12 and 13.
Project Context and Objectives:
The SkinDetector project is a Research for SMEs (R4S) project framed in the FP7 European commission projects.
A person is said to suffer from Diabetes Mellitus (DM) or Diabetes when one’s own body does not produce insulin which is needed to control blood sugar levels (Type 1) or when the body does produce sufficient insulin but cannot not use the insulin produced properly i.e. insulin resistant (Type 2). Another type of diabetes is Gestational diabetes mellitus that occur when pregnant women have high blood glucose levels due to hormones produced in pregnancy which could lead to Type2 diabetes. There are other more rare types of diabetes too.
The aim of this project is to develop a tool for early detection, accurate diagnosis and monitoring that will reduce the microvascular (the damage of small arteries) and macrovascular (damage to large arteries) that arise from complications of diabetes mellitus.
The project aims to develop a portable set of equipment that can be used at a general practitioner’s surgery and/or local primary care clinics or even at a patient’s home. Furthermore with the capabilities of the modern telemedicine, the possibility of analysing the clinical investigation results via a tele-consultation (with tertiary level specialist physicians) and making available a prognosis and a treatment plan is another possible outcome. Thus this has much benefit to rural or underserved areas that perform clinical investigation of patients with Diabetes Mellitus but may not have a resident specialist.
The system will be capable of measuring the external temperature of the patient’s feet using a Thermographic camera, take pictures of the complications using a digital dermatoscope “Smartscope” and also be able to acquire ultrasonic data, by means of a high frequency phased array system and a linear phased array probe that would be positioned over the Skin (e.g. feet) manually but would have automatic linear movement of the arrays to produce a complex ‘picture’ of data; this collective data would allow clinicians/physicians to assess the severity of the skin complication in one consultation (visit); thereby helping with early ‘one-stop’ accurate diagnosis and a treatment plan.
Diabetes mellitus (DM) is an incurable chronic disease associated with invariable hyperglycemia and carbohydrate and fat dysmetabolism, resulting in multiple organ damage. The diabetic foot is the main cause of hospitalization and amputation among DM patients. Neuropathy and angiopathy are considered to be major risk factors for diabetic foot ulceration. According to studies, usually foot temperature increases prior to the formation of ulcers, while reduced temperature correlates with the degree of the severity of peripheral circulatory disorders and neuropathy. In another cases, we can observe increasing ischemia of foot tissues which also leads to irreversible damage and subsequently amputation.
On development of the collective system consisting of a thermographic camera, dermatoscope and ultrasonic transducers, the Skin Detector system will be tested on Skin Phantoms and on a real patient clinical study with 150DM patients with no ulcers, 150 DMs with ulcers and 100 control ‘patients’. We believe this will lead to the possibility of early diagnostics of the diabetic foot and its complications in the skin and soft tissues thereby helping to attest further more severe progression of DM. Scientifically, the results of the study will be useful for the evaluation of the sensitivity and specificity of non-invasive physical examination techniques in detecting clinical markers important for the diagnostics of the diabetic foot and its complications and thus prevent debilitating amputations.
Project Results:
From the beginning of the project, the project partners made a great effort developing the Skindetector system and fulfil the project requirements. An early clinical diagnosis of the diabetic foot and its complications in the skin and soft tissues thereby helping to attest further more severe progression of Diabetis mellitus. Some of the most important consequences of Diabetis mellitus such as, risk of skin infections, ulceration and even amputation in diabetes patients, may directly impact on the patient life changing their habits and quality of life.
At this stage, all internal project delays have been properly addressed by the project partners without affecting the overall project schedule and all deliverables have been officially and successfully reported (submitted). All technical tasks have been also successfully executed and all project milestones achieved.
WP1: Project coordination and management
All WP1 deliverables have been successfully submitted: D1.1 D1.2 D1.3 and D1.4
From the beginning of the project, the project coordinator and project partners have executed all technical tasks in order to fulfil the project requirements and specifications. All tasks have been properly synchronized and coordinated in order to go ahead with the development process of the project and to avoid unnecessary delays. All the internal delays raised up during the project have been properly managed to do not affect the evolution and development process of other internal tasks and/or the overall project schedule. Several project, teleconference and telephone meetings have been organized and scheduled. A fluid communication process between the partners has been completely necessary in order to finalize all technical tasks and to fulfil the project goals on time. The project coordinator have supported the rest of the SMEs in terms of financial reporting process and coordination activities and the SMEs have supported the RTDs in order to complete the technical work. No consortium modifications have been done and no project extensions have been needed.
As it was previously described, all technical and management deliverables have been successfully reported (submitted) in line with the official project deadline. The official progress reports (reporting period 1 and 2, and final progress report) and the additional progress reports requested by the project officer and reviewer have been submitted on time in line with the project deadlines.
WP2: Project specifications (Definition of requirements)
All WP2 deliverables have been successfully submitted: D2.1 D2.2 and D2.3 and the milestone MS1 achieved.
The Overall system initial requirements, detailed subsystem specifications and the specification revision and final agreement were successfully defined, agreed by all partners and submitted during the first reporting period. These project specifications have been used for the project partners as baselines for the technical development.
WP3: Development of the special optical hardware (digital dermatoscope and thermography device)
All WP3 deliverables have been successfully submitted: D3.1 D3.2 and D3.3 and the milestones MS2 and MS3 achieved.
During Work Package 3, a series of prototypes expanding the capabilities of the Smartscope’s SK1 module have been developed. 3D computer aided design tools were used coupled with optical simulations to redesign the module and additive manufacturing was used to produce them. The electronics had to be changed as well and new PCB boards were designed, manufactured and assembled. Medical trials using the prototypes were conducted and helped understand which of the prototypes were most useful. Final, production grade designs were then generated for future use by the end user. An additional design was also generated to tackle the need for higher magnifications utilizing a microscope objective lens. Fig 3 and 4.
A MATLAB program was successfully developed to perform object segmentation and analysis in an image and specifically in images acquired from a dermatoscope. The program runs quickly and is easy to use something particularly helpful to general practitioners and local primary care clinics that evaluate skin spots and perform diagnoses. Moreover, it gives a precise colour analysis and size of the object analysed. The segmentation and analysis algorithms have been successfully demonstrated on ink areas and were refined afterwards using real skin spots. The algorithm used can detect several objects in the same image and can also detect details within regions, requiring only a minimum level of in homogeneity between objects. As skin images can present in homogeneity, two solutions were developed and reported. Fig 5.
A set of mathematical algorithms and specific software were designed, codified and tested to measure, using a thermography camera, the temperature different between the patient feet according to a threshold previously defined by the medical doctors. The highlighted areas, using isotherms lines, will be analysed in detail by the doctor by means of the other two Skindetector techniques (Visual -Dermatoscope- and ultrasound inspection). In order to ensure a good quality of the inputs RAW data (thermography pictures), the camera has to be fixed on a tripod system, designed and manufactured with the project frame, and the patient has to be conformable placed on a massage table to ensure a constant distance between the camera and the patient feet. Both feet have to be immobilized during the thermography picture acquisition process in order to improve the quality of the results. Fig 1 and 2.
WP4: Development of the special ultrasonic hardware:
All WP4 deliverables have been successfully submitted: D4.1 D4.2 D4.3 D4.4 and D4.5 and the milestone MS4 achieved.
A prototype 8 channel ultrasound front end have been developed within this work package. Through iterative design and continuous effort, a fully functional prototype was designed, manufactured, assembled, tested and reported. The 8 channel board is a fully parameterized 8 channel pulse/receive system utilizing MAX4940 high voltage digital quad pulsers and an AD9279 octal analog to digital converter (ADC). The system can drive piezoelectric transducers with tunable high voltage pulses and captures the response signals coming from the same piezo elements. The data is acquired and sent directly to the DDR3 memory located on the FPGA board. All the configurations of the design’s operation are done through AXI interfaces, increasing the reusability and versatility of the FPGA design. The work in this task covers all the base work needed to be done if a company wanted to produce a new ultrasonic device based on latest integrated circuit components and the latest FPGAs. Fig 9.
A high speed data connection solution was developed to enable ultrasound devices to rapidly transfer data to a host pc. To be more precise a solution to transfer data between an FPGA card and a host computer running a Microsoft Windows operating system was developed. The solution can achieve transaction speeds above 1300MB/s if operated in FIFO mode. When the transaction takes place between the DDR3 memory of the FPGA board we can achieve write speeds of 800MB/s and read speeds of 533MB/s. An ultrasound data generator simulating an overload of the data transfer design was also developed. In this case the speeds fall to 650MB/s for the write and 450MB/s for the read. Even the slower speeds achieved in the task are an order of magnitude higher than any ultrasonic device currently in the market or officially announced by key suppliers worldwide. Scaling up the channel count the solution can allow a 128 channel ultrasound system to operate and transfer 16KSamples long Full Matrix Capture measurements (12bit/sample) at a theoretical pulse repetition frequency of 150Hz. After carrying out several lab tests, a transaction speeds of 2000 Mbytes has been achieved. Fig 10.
The phase coherence processing algorithm was also developed. Phase coherence processing is a signal processing technique used for adaptive beamforming applications. It is adaptive in the sense that it utilizes information from the signals as they are received in order to choose their processing factors. The phase coherence factors have been proposed in order to improve lateral resolution, suppress side and grating lobes and enhance signal-to-noise ratio. They evaluate the variance in phases of echo signals received by individual transducer elements after delay compensation has been applied. The phase coherence factor suppresses speckle echoes because phases of speckle echoes fluctuate due to their mutual interference. The development was done using the Matlab programming environment. Its performance was tested using simulated signals generated by CIVA™ and then polluted with artificial speckle noise.
A ultrasound scanning device prototype was also designed, manufactured, assembled and tested to ensure smooth mechanical movements of the phasedArray probe from Imasonic and a very high measurement repeatability and resolution. The inspected area is equal to 50x40mm and its resolution is <=0.05mm. The transducers movement is generated by a miniature stepper motor and driven by a lead screw. The control board and limit sensors are installed on board. An embedded software was designed and codified to implement the application state machine. Fig 6.
The integration activities and validation tests were carried out to verify the performances of the scanning device using the SITAU PRT device from Dasel. According to the results of these tests, an iterative redesign and manufacturing process was carried out. In this same line, the 8 channel ultrasound front end, the high speed communication PCIe connexion and the phase coherence algorithm were also tested and validated after several tests performed by the project partners. Fig 7.
WP 5: Development of the sophisticated software:
All WP5 deliverables have been successfully submitted: D5.1 D5.2 D5.3 and D5.4 and MS5 achieved.
The Matlab based control software (routines) for ultrasound system control and data acquisition was developed, user manual was prepared. The importance of 3D beamforming algorithm for data processing after gathering from the multi-channel phased array unit is related to improving quality of the visualized ultrasonic image, signal to noise ratio and spatial resolution. In order to achieve these objectives it has been chosen to use the selected methods Synthetic Aperture Focusing Technique (SAFT) and Total Focusing Method (TFM). Both methods were proposed, analysed and implemented in order to reconstruct the high resolution images of FBH and SDH (elastomer sample).
3D beamforming (SAFT and TFM) software engine for GPUs processing was developed and implemented into environment of computational package “Matlab R2013a”. Additionally, testing of computation performance of the “Matlab” based GPU processor for SAFT and TFM ultrasound processing was performed using CPU resources only and the combined resources of CPU - GPU (heterogeneous system). System parameters used for testing: CPU 2 x Intel Xeon E5606 (2.13 GHz), RAM 64 GB, GPU GeForce GTX 680 2GB.
The results of the processing time dependence on number of the grid elements (with or without GPU) in the cases of SAFT, TFM and creation of the C-scan images (SAFT processing) were obtained. It was observed a clear sharp “jump” of processing time of heterogeneous system (CPU-GPU), which reaches the computation time of single CPU performance. It was observed that there is sufficiently “small” region, where single CPU process data faster than heterogeneous system (CPU-GPU), but only until the number of ROI grid elements is sufficiently “low” (until reach value of 3e4 – 8e4).
On the other hand, the algorithms for image enhancement, image segmentation (Otsu, K-means), rendering and 3D image creation were developed. The Matlab based routines for 3D data fusion, visualization and analysis software (measurements, rotation), were integrated and user manual were developed. Application of the different non-invasive imaging techniques (thermography, digital optical dermatoscopic imaging and ultrasonic techniques) gives the additional information about the lesions of the skin affected by diabetus mellitus. For that purpose interactive graphical interfaces of 3D ultrasonic data navigation, analysis and fusion with optical / thermography / 3D camera images were also developed.
From the novel volumetric imaging of the ultrasonic data (demonstration was performed on special phantoms) medical doctors are getting more informative image for screening and early stage diagnosis: spatial 3D distribution of the non-homogeneity (lesion width, length and depth), highlighted contour and thickness of the adjacent layers of superficial tissue lesions.
WP6: Integration of the overall system:
All WP6 deliverables have been successfully submitted: D6.1 D6.2 and D6.3 and the milestone MS6 achieved.
Within this work package and in line with the engineering design and manufacturing processes during the prototype stages, several tests were carried out to calibrate the Skindetector systems and to validate their performances. A detailed explanation of these validation and calibration tests can be found into the deliverable D6.2.
Additionally, within the deliverable D6.1 the end user can see a detailed description about the integration activities that were carried out and how each of the Skindetector subsystems (Thermography, dermatoscope and ultrasound) have to be managed and operated (Technical documents and manuals of the application). Fig 11.
Also, as part of the normal engineering process and taking into account the results from the internal and official tests, several design and manufacturing iteration processes were executed in order to resolve the prototype technical issues raised up. All these technical issues, and their corrective actions, were described in the deliverable D6.3.
WP7: medical validation:
All WP7 deliverables have been successfully submitted: D7.1 D7.2 and D7.3.
During the SkinDetector framework clinical study (approved by Kaunas Regional Biomedical Research Ethics Committee), the project partners investigated skin temperature (separately in 8 anatomic points and mean temperature in both feet), epidermal thickness (separately in 5 points and mean thickness), and the skin surface of diabetic feet (n =28) and controls (n =13). Also, we examined 10 diabetic patients with type 2 diabetes mellitus and foot ulcer. The following diagnostic techniques were used: 1) Infrared camera (temperature sensitivity - 0.05 °C, resolution - 320x240 pixels, emission coefficient - 0.98) ; 2) 22 MHz ultrasound (Taberna pro Medicum); 3) Dermatoscope Optomed (x 6 magnification) with addition modules of blue, red, green and infrared light filters. Also we have determined the sensitivity and specificity of the thermography program for the evaluation of temperature differences between both feet.
For the clinical validation of the ultrasound system, an external auditor analysed the need of clinical validations of all 3 systems considering the TRL (Technology Readiness Level) of each system individually. Auditor identified that the Phased Array Ultrasonic Imaging Device (Prototype V1.0 2014) does not require clinical evaluation performing clinical investigations (trial) due to the fact that prototype is not near commercialisation stage (TRL 8) as compare to the other two SkinDetector System. Phased Array unit system requires more detailed clinical validation after optimisation of current prototype by conducting further demonstration/research and to incorporate the results of demonstrations into a prototype etc. The external auditor suggested that SkinDetector consortium carry on more pre-clinical trial studies such as clinical literature evaluation and validation in action on skin phantom, fresh meat without clinical trial on human body.
Parallel we investigated 30 melanocytic nevi before excision and after this procedure using dermatoscope with addition modules of 4 light filters. From medical point it has been concluded, that skin temperature of the feet can be effective inspected by a SkinDetector thermography system. A digital dermatoscope allows for a better visual inspection of the feet and ulcer. According to the literature review and performed clinical study using commercialised ultrasound device in similarly frequency as created prototype in project, ultrasound gives accurate measuring the skin thickness of the feet. All biophysical parameters are very important skin markers in the formation of diabetic foot ulcer. The results of medical validation were documented in deliverables of the project (D7.1 Practical guidelines on the use of SkinDetector; D7.3 Feasibility and effectiveness of the SkinDetector system).
WP8: Training:
All WP8 deliverables have been successfully submitted: D8.1 and D8.2.
During the project period the training programme for SkinDetector system was prepared and presented in practical workshop which was held in Kaunas, 24th October, 2014. The seminar was organized by LUHS and KTU UI, fig 14. All project partners were officially informed and invited during the official meeting celebrated at Dasel facilities on 25th June, 2014, Madrid, Spain. The aim of the programme was to promote the developed SkinDetector technology by involving the end-users – physicians and project partners. The training programme was introduced according to the terms of the project and documented within the deliverable D8.1 and D8.2 . In the organized SkinDetector seminar, 30 medical specialists (dermatologists, endocrinologists, vascular surgeons) and family doctors participated.
The official project video, produced as part of the exploitation and dissemination activities, introduces and presents from a practical point of view, how the SkinDetector system have to be managed and operated. The dermatoscope and the thermography systems are tested on real patients and the ultrasound scanning device, due to the final decision from the ethics committee, was demonstrated on the diabetic foot phantom.
WP9: Dissemination and exploitation:
All WP9 deliverables have been successfully submitted: D9.1 D9.2 D9.3 D9.4 and D9.5 and the milestone MS7 achieved.
The overall exploitation and dissemination planning in SkinDetector has been geared towards maximising the economic benefit to the SMEs as quickly as possible following the end of the project. In order to reach the wider audience outside the project, the consortium has developed various dissemination means, such as project website, NDT wiki, video and flyers. Partners had also disseminated the project in their own websites. During the two years of the project, SkinDetector was presented in 14 conferences and workshops, 6 publications were written.
It is important to highlight the results of SkinDetector clinical study were officially presented at 12th Congress of Baltic Association of Dermatovenereologists (30-31 May, 2014, Riga, Latvia) and during the meeting of the project (on 25th June, 2014, Madrid, Spain).
The official project video was produced and it will be reproduced from the official project website.
Dissemination and exploitation activities will continue after completion of the SkinDetector project. SMEs are still planning to actively attend various events and disseminate the results of the project as well as they are planning to contact the manufacturers/end-users in order to explore the exploitation possibilities.
Potential Impact:
The final outcome of the project is the ‘Skin Detector’ prototype that enables early clinical diagnosis of the diabetic foot and its complications in the skin and soft tissues thereby helping to attest further more severe progression of Diabetis mellitus. The potential impacts of the SkinDetector system is to minimize/avoid the risk of skin infections, ulceration and even amputation in diabetes patients.
Scientifically, the results of the clinical study validated the sensitivity and specificity of non-invasive physical examination techniques in detecting, in less than 30 minutes, clinical markers important for the diagnostics of the diabetic foot and its complications. These markers will be analyzed by the medical doctors in order to produce a final diagnostic.
In order to reach the wider audience outside the project, the consortium has developed various dissemination means, such as project website, NDT wiki, video and flyers. Partners had also disseminated the project in their own websites. During the two years of the project, SkinDetector was presented in 14 conferences and workshops, 6 publications were written. The final results of SkinDetector clinical study were officially presented at 12th Congress of Baltic Association of Dermatovenereologists (30-31 May, 2014, Riga, Latvia) and during the meeting of the project (on 25th June, 2014, Madrid, Spain).
List of Websites:
http://www.skindetector.eu/(odnośnik otworzy się w nowym oknie)
http://www.ndtwiki.com/index.php/SkinDetector(odnośnik otworzy się w nowym oknie)
