Final Report Summary - AISENSE (Human-Computer Interaction and Computer Vision for Improving Healthy Living of Elderly through Exer-gaming)
Aging is a major problem of many European countries characterized by a decrease in fertility and mortality rate, and a higher life expectancy among Europeans. The rising number of retirees needing assistance from social security systems makes it necessary to focus resources rather on prevention than in treatment of illnesses, to allow a longer independent life in the home environment. Physical inactivity is a major risk factor for developing chronic diseases such as coronary artery disease as well as obesity, cholesterol and diabetes. Even modest levels of physical activity could act as a great prevention strategy against the development of any chronic disease and elderly people can gain significant health benefits with a moderate amount of physical activity on a daily basis. Therefore it is really important for seniors to include physical activity as part of a regular routine. Since professional assistance is mostly unaffordable, at least on large scales, an interesting emerging technology to overcome these inconveniences has been the use of modern gaming technology. There is a new trend in the way physical exercise can be incorporated into a fitness training program, namely the “exergaming”. Exergaming is a form of exercise through the use of video games whose main focus is the improvement and promotion of physical health of individuals. Individuals can train themselves through a video game by physically interacting with its content. The trainee can manipulate a virtual character on the game screen through his body movements, imitating this way a plethora of sport activities, like walking or running
Usually, exergames are games full of exercises which would have a positive effect on balance ability of the players. However, most games are not developed for the elderly. Sometimes, games could be boring, or too fast and difficult to operate for them. Game scenarios composed of very simple set of instructions asking elderly to stand upright or sit down in a rehabilitation game might lead elderly people get bored very quickly and quit exercising. The exergaming system in this project aims to engage elderly people in mild exercising that wouldn’t extremely physically challenge them and entertain them while assessing their physical performance. Hence, Dr. Yalcin collaborated with professionals in the areas of Neurology and Physical Training in order to develop an exer-gaming scenario (AISENSE Exergaming System) that will be engaging, entertaining, increasing physical activity levels and has the capability of sustaining the attention of the subject.
The AISENSE project is developed around the idea of motivaving the player to move around the room searching for some objects and performing certain tasks with those objects. The exergaming scenario involves recognizing certain objects in a room equipped with depth sensors and cameras. The player is rewarded/penalized depending on whether he successfully accomplishes the subtasks of the game. The physical performance of the player is assessed both by evaluating his body pose throughout the exergaming duration as well as evaluating the measurements taken from a smartphone fastened securely on the body of the subject acting as a biosensor. Since the overall objective of AISENSE is to build an exergaming system that allows elderly people to naturally interact with a virtual environment through their body movements and assess their physical performance, the scenario is devised to engage elderly people in mild exercising that wouldn’t extremely physically challenge them and entertain them while assessing their physical performance.
Two critical components of AISENSE Exergaming system are human activity recognition and 3-D object recognition. A human-computer interaction needed for the AISENSE Exergaming system and it is achieved through recognition of the body movements of the subject using multiple numbers of vision sensor platforms which have conventional cameras and depth sensors on them. Human activity recognition has many applications in computer vision, including personal assistive robotics and smart homes/environments. Due to the large temporal and spatial variations in actions performed by humans, human action recognition has been a long-standing challenge. Throughout AISENSE project, Dr. Yalcin developed a method that recognizes certain human activities based on a motion descriptor that uses 3D human skeleton data. Motion descriptors (SHOJD) are defined using the 3D distance between the most frequent key poses that occur throughout the action that is intended to be recognized. Motion features are then fed into an artificial neural network for classification. Experimental results indicate that the SHOJD based human action recognition system is robust with high recognition rate.
The objects of interest in the exergaming environment are detected by the state-of-art 3-D object recognition algorithms. The research is focused on the developing of algorithms that allow different types of sensors to detect various aspects of the activity in the environment collectively. The AISENSE exergaming scenario is devised such that its objectives would establish a basis for developing state of art computer vision algorithms. One of the major challenges of the AISENSE project has been ensuring that spatial and temporal features obtained from multiple set of sensors to be processed simultaneously and developing fusion algorithms to combine the results obtained from different modalities and to improve the recognition rate by strengthening performance of each sensor by the results of the other sensors.
Many undergradute and graduate students have completed their thesis with themes that are subtopics of AISENSE project, learning to use point cloud library (PCL), OpenCV and OpenNI libraries for 3-d scene understanding, 3-d object recognition with depth sensors. Dr. Yalcin has been guiding them to develop algorithms for 3D object recognition, human body recognition and gesture recognition using RGB-D data acquired from depth sensors. Research carried out through AISENSE project complements the high quality research carried out in the department, which includes the computer vision, pattern recognition, robotics areas as well as areas like signal processing, partial diffential equations, time series analysis, and numerical methods due to her Electrical Engineering and Applied Mathematics background. Dr. Yalcin has been collaborating with other young faculty both in Electrical Engineering and Computer Engineering departments to apply for project proposals to be submitted to several national research funds to develop user-friendly multimodal human-computer interaction techniques such as body, face and facial gesture recognition for an assisted living smart space of the elderly.
Outreach activities through AISENSE project encompassed broad interactions with the general public in order to promote the communication between the scientific community and the general public and increase awareness of science. Part of the outreach activities was implemented through TUBITAK (The Scientific and Technological Research Council of Turkey). AISENSE project increased public awareness in nearby local hospitals and the municipality about improving the life quality of elderly people with exergaming.
Dr. Yalcin continues to work on the research initiated by AISENSE project to have an exergaming environment where different components of the exergaming system are working seamlessly together.
Usually, exergames are games full of exercises which would have a positive effect on balance ability of the players. However, most games are not developed for the elderly. Sometimes, games could be boring, or too fast and difficult to operate for them. Game scenarios composed of very simple set of instructions asking elderly to stand upright or sit down in a rehabilitation game might lead elderly people get bored very quickly and quit exercising. The exergaming system in this project aims to engage elderly people in mild exercising that wouldn’t extremely physically challenge them and entertain them while assessing their physical performance. Hence, Dr. Yalcin collaborated with professionals in the areas of Neurology and Physical Training in order to develop an exer-gaming scenario (AISENSE Exergaming System) that will be engaging, entertaining, increasing physical activity levels and has the capability of sustaining the attention of the subject.
The AISENSE project is developed around the idea of motivaving the player to move around the room searching for some objects and performing certain tasks with those objects. The exergaming scenario involves recognizing certain objects in a room equipped with depth sensors and cameras. The player is rewarded/penalized depending on whether he successfully accomplishes the subtasks of the game. The physical performance of the player is assessed both by evaluating his body pose throughout the exergaming duration as well as evaluating the measurements taken from a smartphone fastened securely on the body of the subject acting as a biosensor. Since the overall objective of AISENSE is to build an exergaming system that allows elderly people to naturally interact with a virtual environment through their body movements and assess their physical performance, the scenario is devised to engage elderly people in mild exercising that wouldn’t extremely physically challenge them and entertain them while assessing their physical performance.
Two critical components of AISENSE Exergaming system are human activity recognition and 3-D object recognition. A human-computer interaction needed for the AISENSE Exergaming system and it is achieved through recognition of the body movements of the subject using multiple numbers of vision sensor platforms which have conventional cameras and depth sensors on them. Human activity recognition has many applications in computer vision, including personal assistive robotics and smart homes/environments. Due to the large temporal and spatial variations in actions performed by humans, human action recognition has been a long-standing challenge. Throughout AISENSE project, Dr. Yalcin developed a method that recognizes certain human activities based on a motion descriptor that uses 3D human skeleton data. Motion descriptors (SHOJD) are defined using the 3D distance between the most frequent key poses that occur throughout the action that is intended to be recognized. Motion features are then fed into an artificial neural network for classification. Experimental results indicate that the SHOJD based human action recognition system is robust with high recognition rate.
The objects of interest in the exergaming environment are detected by the state-of-art 3-D object recognition algorithms. The research is focused on the developing of algorithms that allow different types of sensors to detect various aspects of the activity in the environment collectively. The AISENSE exergaming scenario is devised such that its objectives would establish a basis for developing state of art computer vision algorithms. One of the major challenges of the AISENSE project has been ensuring that spatial and temporal features obtained from multiple set of sensors to be processed simultaneously and developing fusion algorithms to combine the results obtained from different modalities and to improve the recognition rate by strengthening performance of each sensor by the results of the other sensors.
Many undergradute and graduate students have completed their thesis with themes that are subtopics of AISENSE project, learning to use point cloud library (PCL), OpenCV and OpenNI libraries for 3-d scene understanding, 3-d object recognition with depth sensors. Dr. Yalcin has been guiding them to develop algorithms for 3D object recognition, human body recognition and gesture recognition using RGB-D data acquired from depth sensors. Research carried out through AISENSE project complements the high quality research carried out in the department, which includes the computer vision, pattern recognition, robotics areas as well as areas like signal processing, partial diffential equations, time series analysis, and numerical methods due to her Electrical Engineering and Applied Mathematics background. Dr. Yalcin has been collaborating with other young faculty both in Electrical Engineering and Computer Engineering departments to apply for project proposals to be submitted to several national research funds to develop user-friendly multimodal human-computer interaction techniques such as body, face and facial gesture recognition for an assisted living smart space of the elderly.
Outreach activities through AISENSE project encompassed broad interactions with the general public in order to promote the communication between the scientific community and the general public and increase awareness of science. Part of the outreach activities was implemented through TUBITAK (The Scientific and Technological Research Council of Turkey). AISENSE project increased public awareness in nearby local hospitals and the municipality about improving the life quality of elderly people with exergaming.
Dr. Yalcin continues to work on the research initiated by AISENSE project to have an exergaming environment where different components of the exergaming system are working seamlessly together.
