Periodic Reporting for period 1 - DigiSocket (Digitalization of prosthetic socket personalization workflows trough artificial intelligence)
Okres sprawozdawczy: 2020-10-01 do 2022-05-31
Despite several innovations in the field – usually at hefty prices preventing wide adoption – the customer journey within upper limb prosthetics remains dominated by technologies which are decades old, such as simple myoelectric grippers and plaster-casted limb sockets. For the latter, the current approach involves an initial cast of the patient’s residual limb made by wrapping plaster bandage around it. From this positive model, a test socket made from a thermoplastic is made, which is repeatedly heated, cut and reformed to get an intimate fit, in a laborious process not always resulting in a comfortable socket (cf. Fig. 1). Besides its limitations on optimizing the fit to the patient stump – hence ultimately compromising its comfort, the conventional “plaster-casting” approach is both laborious and costly, thus providing sizeable room for improvements. While previous digitizing prosthetics workflows have been attempted, they primarily rely on 3D scanning based on handheld line laser scanning. In particular, the latter has been taken as a major bottleneck for widespread adoption, as the required CAD design/manufacturing skills to operate the existing software tools are perceived by prosthetists as requiring a steep learning curve. Moreover, the handheld devices, even though easily transportable, typically require one scanner per prosthetic workshop – which creates a significant investment overhead in highly fragmented markets where are typically 1-2 prosthetists per workshop (e.g. USA). There is thus a latent business opportunity for Hy5 to provide a steppingstone to digitize the industry towards eliminating pain points and disrupt today’s value chain. Indeed, a cost-effective and user-friendly solution to enable the digitalization of the workflow for limb socket personalization would be a major milestone in the field, providing a major tailwind for Hy5 to gain visibility at global scale, while expanding its business across the prosthetics value chain.Through this grant, Hy5 aims to develop the world’s first smartphone-enabled, AI-driven, cost-effective and user-friendly digital workflow for limb socket automated design – DigiSocket – providing a cornerstone to shift the current paradigm in this sector and beyond.
From the knowledge gathered through conversations with Hy5-partnered certified orthotist / prosthetists (CPOs), as well as her own academic background, the IA composed an initial summarizing report on the current state of socket creation, along with the considerations and cautions for digitizing the socket creation process. From this distillation, the IA worked with senior management to put together a plan of action for the project. As the CPO feedback to the original project plan also highlighted significant challenges, these were also taken under consideration. From the IA’s findings and CPO feedback, it became apparent that developing a scanning algorithm to the quality that is required for this application was beyond the timeline of the project, the management greenlighted a thorough testing plan of external scanning resources toward use in the DigiSocket project. The IA then developed an experimental plan towards identifying an existing mobile scanning phone application that would be compatible with scanning residual limbs in varying conditions with high accuracy. The IA also developed an initial schematic for marking the residual limb such that the subcutaneous bony morphology and pressure points would be apparent in the 3D digital model.
The IA compiled a long list of existing phone scanning apps was assessed for basic technical experimental protocol to determine the quality of the scans on complete limbs, particularly the algorithm adjustments for skin and body hair. The long list of scanning phone apps was pared down to a short list for full experimental testing on residual limbs. The experimental testing of various existing mobile phone scanning applications took place over the course of several months. This iterative testing approach involved a variety of experimental conditions to assess scanning performance across different lighting conditions, skin types, users, external limb covers, phone cameras, limb morphology marking techniques, and limb sizes. The apps were also graded on their scan output file format, as the 3D model and associated texture file had to be compatible with Hy5 modelling systems. From the extensive testing described above, one phone app stood out as the most suitable scanning solution. This app was available across phone operating systems, and met the testing requirements for flexibility in accommodating various scanning environmental conditions and patient variability. Its algorithm is already used in facial scanning, so meets the topographical resolution requirements for prosthetic socket development. The experimental testing resulted in ‘best practices’ recommendations for residual limb scanning. These include placing a white or light-coloured sock over the residual limb prior to marking the bony morphology and pressure points, scanning in a well-lit room to minimize shadowing on the limb, and moving at a consistent distance around the surface of the residual limb during image capture, among others, which have been fully summarized in internal Hy5 documentation. Finally, the project concluded with scan post-processing protocol development towards creation of a 3D digital model.
The IA compiled a long list of existing phone scanning apps was assessed for basic technical experimental protocol to determine the quality of the scans on complete limbs, particularly the algorithm adjustments for skin and body hair. The long list of scanning phone apps was pared down to a short list for full experimental testing on residual limbs. The experimental testing of various existing mobile phone scanning applications took place over the course of several months. This iterative testing approach involved a variety of experimental conditions to assess scanning performance across different lighting conditions, skin types, users, external limb covers, phone cameras, limb morphology marking techniques, and limb sizes. The apps were also graded on their scan output file format, as the 3D model and associated texture file had to be compatible with Hy5 modelling systems. From the extensive testing described above, one phone app stood out as the most suitable scanning solution. This app was available across phone operating systems, and met the testing requirements for flexibility in accommodating various scanning environmental conditions and patient variability. Its algorithm is already used in facial scanning, so meets the topographical resolution requirements for prosthetic socket development. The experimental testing resulted in ‘best practices’ recommendations for residual limb scanning. These include placing a white or light-coloured sock over the residual limb prior to marking the bony morphology and pressure points, scanning in a well-lit room to minimize shadowing on the limb, and moving at a consistent distance around the surface of the residual limb during image capture, among others, which have been fully summarized in internal Hy5 documentation. Finally, the project concluded with scan post-processing protocol development towards creation of a 3D digital model.
The DigiSocket project sought to provide a novel, fully digitized workflow for the current nearly century-old plaster-caster based method for prosthetic socket design. This would have considerable impact for the prosthetic community and bring about the next evolution in socket creation. Our findings are that this is a necessary and advantageous pursuit, which should continue to production beyond this proof-of-concept development.