Skip to main content

A VR based system to allow matching of an optimum interface to a User of Assistive Technology


Overview The "Matching" work in I-Match concentrates on the development of techniques to optimise the selection and matching of an Assistive Technology (AT) device to AT users. This matching of device to user can be supported by measuring both the functional characteristics of the device and capturing the characteristics and the skills of the users. Correctly matching appropriate Assistive Technology (AT) devices and AT interfaces to users with disabilities is a key aspect involved in their eventual take up and adoption of Assistive Technologies. Another factor in their continuous use relates to the trial-ability of the AT devices, with studies showing a higher adoption and use if the users were able to experiment and train with the devices before adoption. One of the principle objectives of I-Match has been to use State of the Art measurement and simulation technologies to enable people with functional impairments to choose the optimum interface systems for AT devices. To this end a virtual reality environment was developed so the users can evaluate and test several devices, including a MANUS, and HANDY-1 and a powered wheelchair with various interface controls e.g. Haptic device, Switches, and Joysticks etc. Also in line with this objective software was developed, for use with a haptic PHANToM® device, that would provide a series of tests and simulations that would allow gauging of upper limb performance. The output from these simulations, tests and training sessions can be used to generate reports that could be used by Assistive Technology Assessors to recommend an appropriate interface device for a user. This also involves the development of standardised tests and testing procedures for evaluating the user experience with the devices. Comparison of this new method of measuring upper limb performance with traditional measures like Fugl-Meyer, Barthen and the Rivermead motor assessment score are also an aspect to this study. The final aspect of the project is the development of a decision support system that will aid Assistive Technology Assessors in recommending an appropriate device to the user. This will take the form of two systems, a Case based reasoning system and a rule based system decision support system. Case-Based Decision Support System The system uses the libraries, CBML and Fionn, developed by the Machine Learning Group in Trinity College Dublin using the Java programming language; the I-Match CBR system of Assistive Technologies therefore employs Java as the development platform to facilitate integration of these components. The Graphical User Interface, GUI, uses the Java Swing API system. The I-Match CBR System of Assistive Technologies has four principle components; the Case Base Manager, the Database Manager, the GUI and Feature Selection Component. These systems use the Fionn Framework and the CBML system. The FIONN Framework is designed for the development and testing of Machine Learning algorithms. Included in FIONN is support for all the basic aspects of a CBR system. FIONN works with an XML system called Case Based Mark-up Language, CBML, which represents cases in the system. Fionn is independent of the classifier or evaluation technique used; this allows various classifiers to be changed e.g. from a k-NN classifier to using a Naive Bayes classifier. The CBML system comprises two aspects. The first CBML format developed in XML that represents cases in a standardised and independent manner. The second aspect of the CBML System is an API developed for manipulation of CBML files. The first aspect of CBML, namely the CBML XML files are independent of application code, so any system that can read XML can use CBML. The CBML API is an implementation to simplify the job of working with CBML files. Findings The system was evaluated on a device selection task, which involved selecting a joystick for a user from two choices. The evaluation was done on a test set of 26 users with the system recommending a device for each user using a leave-one-out methodology. While the evaluation was successful with approximately 80% accuracy this was achieved with considerable effort expended on collecting good quality training cases. Our conclusion is that simply collecting the 'naturally occurring' cases in a clinic and using these, as training data for a decision support would yield poor quality solutions. Because of this, we do not propose to continue with this decision support application at this stage. Future work at TCD is unlikely as the PI responsible for the work is no longer at the institution. It is possible that the work could be continued by another I-Match partner.
The I-MATCH Upper Limb Evaluation Tool is a set of computer-based tests using Haptic interface, Assistive Technology devices or standard PC equipment as input devices. These tests are used to provide an objective measurement of the performance of the subject's upper limbs. Each test is conducted in a Virtual Reality setting and enables online recording of a set of parameters that are used to assess the user upper limb ability. The tests include: - Peg-In-Hole, a test inspired by the Nine Hole Peg Test (NHPT), which is used in clinical assessment. The Peg-in-hole assessment is essentially a 3D task presented in a virtual space. It has a simple geometry and objective. It consists of a large virtual table with two cylindrical holes indented on it, one small cylindrical peg that is to be placed in the holes and alternately moved between them using the hand. - Circular tracking: tracking a circle clockwise and counter-clockwise - Linear tracking: the user is required to follow a line to its edge and backwards. The test is repeated in six directions. - Target tracking: The patient's task is to move toward the target from the current position in the most natural way and then to hit it. When completing the current move, the destination point becomes a starting point and a new destination point is generated somewhere else. This procedure is then repeated ten times in total. - Labyrinth: In this test, a complex and movement demanding virtual environment, representing a labyrinth and aligned with the person s frontal plane is presented to the patient. The patient can move the pointer, represented with a ball, through the labyrinth in three dimensions. The visual and tactile information is fed back to the person using the computer display and haptic interface (in case one is used). The person can feel all the reactive forces occurring in ball contact with the labyrinth walls and can get the realistic impression as he/she was interacting with the real environment. This test can capture the finger dexterity, the forearm and shoulder movement abilities. - Maximal force: This task measures the capacity of exerting forces in six evenly distributed directions. The I-MATCH Haptic Evaluation Tool generates reports, which include a graphical representation of the measured parameters for each test subject. This provides the Technology Providers with immediate visual information about the ability of the test subject. A cut-down version of the Haptic Evaluation Tool, which does not have the ability to generate reports or log patient data, is freely available on the I-Match website.
A result of the project is the I-match database of the physical characteristics of Assistive Technology Input Devices largely joysticks and mice. Following a detailed exercise in defining the important physical variables a web based database of available input devices allows a Technology Provider (TP) to select an input device according to physical size, range of motion and forces required for operation. While this database was initially hosted using project resources, it then became necessary to ensure its availability beyond the end of the I-match project. This was achieved by migration to the SIVA portal hosted permanently by Fondazione Don Gnocchi in Italy that has a long experience in creation and management of Assistive Technology databases. The experience so far with the Siva Portal (the Italian Portal on assistive technology, administrated on behalf of the Italian Ministry of Welfare - has clearly demonstrated that running such a database is very onerous. In fact its practical use substantially depends on its constant updating, especially because this is a field with constant evolution. Updating requires the establishment of a permanent editorial team, which was considered inapplicable due lack of resources. For this reason, the integration of information inside of I-match database into the Siva Portal was proposed. The solution seemed particularly advantageous also because the Siva Portal is integrated inside the European Assistive Technology Information Network EASTIN ( and this gives visibility in Europe to I-match information. EASTIN has been realised by six European national information providers in order to achieve an original trans-national platform able to share knowledge and expertise as well as to allow for "horizontal" searches across all databases. 118 new devices were entered in the Siva portal from the I-match database, distributed as follows: 29 devices in class 12.24.03; 5 devices in 24.10.06; 82 devices in 24.09.18; 2 devices in 24.18.06. As the two databases use two different classifications (The Siva Portal uses the ISO 9999:2002 standard classification that has a functional approach while the I-match database uses a technical approach) there is not a perfect overlapping among the categories of the two databases. 11 new firms were introduced in the Siva portal database. Not all devices could be transferred to the Siva Portal: ergonomic mice and industrial joystick are not real AT devices. However, information about six such devices was not lost, as it was introduced in a different section of the Siva Portal ("ideas"=. 36 new advanced search characteristics were added to three divisions of the ISO classification. I-match technical characteristics were reduced in number and simplified to allow non-technical people, such as people with disabilities and their families, to easily understand the information delivered. The results obtained are: 1. Information will be accessible in Italian and English through the assistive technology official Portal of Italian ministry of welfare; this ensures wide visibility and regular exploitation in congresses and exhibitions 2. Information will be accessible all over Europe through the EASTIN network 3. The new advanced search characteristics will be an incentive for manufacturers to supply more detailed information, and for database national providers to spread it. 4. Information will be regularly updated by the Portal providers and by manufacturers themselves through a validated process. The key innovative features of the result are: - A first practical attempt of standardization of interface devices description giving a definite mask for introducing data with technical definition. - Using advanced search options makes it easier to find information (in this field the number of similar devices is constantly increasing). - Using the Interface Test System realised during the project gives the opportunity to have validated information on technical measurements. The end users of this result are: - People with disabilities: they will be able to access more comprehensive information on this category of assistive devices. - Rehabilitation services: they will be able to access easily understandable technical information, even in case the have little knowledge of this field - Assessment services: they will be able to access complete and updated information supplied in a consistent way to allow for an easy comparison. - Manufacturers: they will have a free way to supply information to a wider public; they will have the opportunity to request tests to measures their devices characteristics. We can say that the final solution is better than the state-of-art as regards AT interfaces databases because no other database allows for an advanced search inside these categories with so much technical information. This has been obtained without affecting the cost of maintaining the already existing AT databases.
The I-MATCH Wheelchair Simulation Software refers to a software simulation of a powered wheelchair. Accurate kinematics & dynamics models have been developed, to provide a realistic simulation of the movement of the powered wheelchair. The powered wheelchair simulation takes place in a Virtual Environment which represents a familiar home setting. Modules for supporting standard PC equipment (keyboard, mouse, standard game joystick) as input devices for the simulations have been developed. Further to the standard PC input devices, the software supports certain wheelchair control input devices, which may be connected to a PC through an interface box. The simulation guides the user through the execution of specially designed exercises, which may assess the user's ability in different wheelchair control scenarios (plain forward/backward movement, turning, avoiding obstacles). Each exercise is set in the context of an everyday task the user may have to execute, such as for example driving to the kitchen to grab a glass of water. The virtual environment provided in the Wheelchair Simulation was designed to be as familiar as possible with a real setting, where the user is able to: - Safely train her / him in a virtual environment in using AT equipment with a real AT control device. - Test different AT control devices, to select the most suitable. Furthermore, user sessions are recorded in a database and measurements are obtained, such as the time elapsed and the number of collisions for traversing a path using the wheelchair and other relevant indicators. Reports can be automatically created, however the complete raw data remain at the disposal of the expert user, to extract any kind of useful information he/she may desire. Moreover, the software simulation may be parameterised to realistically simulate a range of powered wheelchairs, including variants with front or rear drive, different mass, dimensions. In this way, it is possible to simulate an array of AT equipment and aid users in the training and selection process, as well as serve as many users as possible.