POPULate AsymmeTric mobile gamEs

Periodic Reporting for period 1 - POPULATE (POPULate AsymmeTric mobile gamEs)

Reporting period: 2015-01-01 to 2016-06-30

To increase the adoption of innovative IT solutions by creative industries SMEs, it is necessary to make those technologies easily usable by creators themselves. To transform a technology into a tool that fits into the creative and production process requires a good understanding of user needs and a collaborative iterative design cycle between creative industry end-users and IT companies. In artistic fields, to transform an idea or a concept into an artistic production depends on the skills and expertise of the artist. In the field of digital content creation, a huge gap exists between the concept and the final result, and several technologies should be used in the production pipeline. Keeping the creative power in the hands of artists requires the development of tools that are both technology-driven and adapted to the creative processes. One of the objectives of the POPULATE proposal was to allow game designers to easily populate mobile games with digital creatures.

The video game industry has a high potential of being a driving force for introducing new ways for users of technology to interact with wearable devices, as well as encourage game developers to create experiences using innovative hardware. Indicators are that video games will progressively increase until 2020, with the biggest revenue from the mobile games category - more than 40% from the total of $115 billion. The objectives of the POPULATE project were dual: use connected objects as input and output peripherals of a video game and allow a collective interaction of different players with an enriched game content executed on a mobile device. These objectives can enrich the user experience of mobile games in several manners. Firstly, mobile games are today limited by the size and interactive capabilities of the mobile device itself while performances are continuously increased: HD images, enhanced sound system, 3D GPU capabilities, multi-core CPU. Mobile games are most of the time limited to individual use, but can also use a big TV screen as a second screen, which allows mobile users to share their content to their neighbors. Connected objects are starting to become more widespread among users of technology; a global wearables forecast estimates that 411 million connected objects worth $34 billion will be sold in 2020. This type of devices is also providing more and more interaction capabilities. The POPULATE project took advantage of all those capabilities to increase the gameplay of mobile games and to transform a simple mobile phone or tablet into an aggregator of different input/output peripherals, as in console applications. Another dimension that was increased by the POPULATE project is the density of dynamic content by integrating real-time crowd simulation functionality.

The POPULATE project accomplished the creation of a prototype platform for developing a new genre of asymmetric mobile games, that will be more captivating, collaborative and interactive than previous mobile games.

The POPULATE project combines a multi-device approach with 3D crowd generation and advanced behavior management. Using crowd generation and behaviour management, the prototype runs on smartphones and/or tablet devices using smart-glasses as controllers and head-ups display. It is also used as the perfect tool to study the psychology of gamers.
AMA has taken advantage of all technical capabilities to transform a simple smartphone into an aggregator of different input/output peripherals. By exploring the possibilities of multi-device interaction, AMA created a game reference for smartphones and smart-glasses and generated a game development framework dedicated to smart-glasses and asymmetric mobile games.
The POPULATE project developed a prototype for a multiplayer asymmetric mobile game. The prototype is a 1 vs. 1 asymmetric multiplayer experience that offers radically different gameplay for the players: for the Explorer (player 1) - action adventure game and, for the Defender (player 2), it will be a strategy game, where the Defender will control a crowd formed by minions.
AMA has created a gameplay where the smart-glass is used as a main controller, and the focus is on the crowd AI, making this the central features to the gameplay.
The gameplay opposes a defending player (Defender or God) that can use minions, shoot and cast spells, to the Explorer who can shoot and cast spells.
The Defender use, as main control method, the smart-glass (camera control, shooting and casting are performed using the smart-glass sensor fusion plus the smart-glass display is used as Heads-Up Display to stream vital information needed by the Defender) and the smartphone or tablet is used only to display the actual gameplay and to issue commands to the controlled minion groups.
The minions are spawned at fixed events, and can receive specific orders from the Defender via the smartphone or tablet touchscreen.
The Explorer is controlled in a more traditional way by using a smartphone/tablet (character control and gameplay display) but, depending on Smartglasses hardware capabilities, the entire Explorer's gameplay can be available directly from the Smartglass: the camera will be fully controlled by Smartglass sensors (Accelerometer and Gyroscope fusion) and the gameplay is displayed on the Smartglass screen. The movement and shooting is controlled with additional joystick connected through Bluetooth.
From hardware point of view, AMA has created a framework compatible with all smart-glasses based on Android OS (that meets the framework requirements).
In order to generate a population of autonomous characters with just a few steps for a video game, Golaem developed within the POPULATE project a dedicated solution. The crowd simulation module is an easy-to-use and efficient solution for game designers to populate and manage the population simulation of a video game for smartphones and tablets. The developed module, Golaem for Unity, is a plugin used by AMA, to populate the game populations. AMA's designers author populations using the native Unity tools for animation & path-finding/navigation, and Golaem's integrated behavior editor and placement tools in order to create complex behaviors. The designed behaviors are then executed on a dedicated runtime C# plugin. This editor is a production tool for creating and managing a behavior graph by connecting multiple behaviors to complement the built-in elementary behaviors. The current version of the plugin is a prototype, and as explained in deliverable D7.2 after an industrialization process, it will be proposed as a commercial product.
The population tool provides an easy way to place characters in the environment, handling the repartition of types and avoiding placement inside obstacles. AMA's developers used the Golaem for Unity module to build complex behaviors for crowds and also individuals, as they wanted the characters to react as a group in some situations and as a person in others. All this led to having diversity in the way the entities are being placed. It also helped to visualize and adapt the behavior and the decision flow at runtime.
This solution has the potential of reducing the time spent on simulating digital characters behaviors and moreover, one get the desired behaviors with only one solution.
The Graphics, Vision and Visualisation Group (GV2) of Trinity College Dublin (TCD) studied the perception of gamers to ensure maximum enjoyment and interactivity within the game. TCD recorded and analysed the players behaviour to examine the influence of the appearance and animation realism on the player’s decision. The aim of the research conducted by TCD was to evaluate the appeal of the virtual characters, the impact of realism and behavior patterns, as well as the perception of personality of virtual characters through a series of perceptual experiments. Eight experiments have been conducted by TCD within the POPULATE project:
1. Appearance of self-avatar faces: Recent advances in facial tracking technologies have allowed virtual faces to be controlled and animated in real-time, which has potential for use in future video-games. We designed an experiment to test the effect of different appearance styles (realistic or cartoon) and different levels of animation realism (complete or reduced) on the way participants engage with an animated virtual face that represents them in a game-scenario. Our results suggest that high levels of face ownership and agency towards the virtual face can be induced through real-time mapping of the facial movements. This study provides valuable insights regarding engagement with self-avatars that are animated through facial tracking. Although the POPULATE project does not include any facial tracking in the game design, this study gave us a first insight that effectively controlling a virtual character (or not), might affect the way we perceive virtual characters.
2. Impact of controlling or observing a virtual human on its level of acceptance: Recent advances in graphics have allowed game and movie producers to display high levels of rendering and animation realism with virtual characters closely resembling real people. However, audience reaction has not always been favorable towards photo-realistic humans, which has been attributed to a feeling of fear of the unknown or discomfort. In our experiment, we tested a hypothesis that the level of discomfort will be lessened if the user feels as though they have control over that character’s movements. In other words, the character being controlled cannot produce any unexpected behaviours, as the user is controlling its movements. We ran a series of experiences where participants controlled or observed a range of characters with different levels of appearance realism. Our results found differences in how realism was perceived in characters that were controlled or observed, indicating that artists/developers should carefully assess the level of realism when creating both first-person and third-person views. Furthermore, in immersive virtual reality the perceived realism of observed characters was higher than controlled, whereas the opposite was true (perceived realism was higher for controlled) when characters were viewed on a regular tv. This result highlights the fact that different parameters are needed for realism when characters are viewed across different platforms. Finally, no differences were found for affinity indicating that the level of appeal of a character does not change when you control it.
3. Impact of the level of realism of characters across different platforms: In this experiment, we tested if character realism was perceived differently across different platforms (in this case, we chose immersive virtual reality and a regular tv screen). The same character was rendered in different styles and we found that the most realistic render style was perceived significantly more appealing that the other two styles, which suggests that realism is a positive choice for virtual characters in immersive environments. Although the art assets for the POPULATE project have been already defined, manipulations that could enhance realism of the NPC, such as improving material realism, are still possible. This could enhance the overall experience of the user, since the NPCs (e.g. minions, question characters) could be tuned to be perceived equally realistic as the PC (Explorer).
4. Impact of dimensions of human facial features on the perception of personality: TCD conducted an exploratory study in order to retrieve insights about the way certain facial features affect the perceived personality of very abstract virtual faces. In particular, the effect of head and eye shape, and eye size on the perceived personality were tested. Interestingly, findings show that the same rules for real human faces do not apply for the perception of abstract faces, and in some cases are the complete reverse. These results provide a better understanding of the perception of abstract virtual faces, and a starting point for the creation of guidelines for how to portray personality using minimal facial cues.
5. Investigate how quest characters with different facial features can practically affect players’ decisions: we wanted to investigate whether perception of a virtual character could actually influence player’s choices and actions within a game. For this, we created a short online video game, where participants were presented with four different moral dilemmas, each presented by one character, and were asked by the character to solve the situation. We used the most aggressive, least aggressive, most trustworthy, and least trustworthy characters from the previous study. Our results show that perceived personality factors of a character can influence moral choices in games. We found that characters significantly influenced decisions in indirect harm moral dilemmas, with the aggressive-looking character triggering more utilitaristic choices and the least aggressive-looking character priming more deontological choices. These findings could be used to create more challenging moral dilemma scenarios in video games, by selectively employing characters likely to bias decisions towards the less-common response behavior. The results of this experiment were used in the final prototype to choose the most appropriate quest character facial features.
6. Investigate how different appearances and/or behaviours of the enemy crowd will affect player’s moves and decisions: In this experiment, we used the prototype game and investigated different appearances and behaviours of the crowds. Users interacted with 3 different versions of the crowd, where the characters had different appearance ranging from highly to low aggressiveness. We were interested to determine the effect on gameplay depending on the appearance of the crowd, while the behaviour remained the same. We hypothesized that the characters with the aggressive appearance would cause panic in the player response and would result in the player displaying more signs of panic and having less success in the game. Our results showed that the opposite was true. Although participants perceived the more aggressive crowd as being more difficult to fight against, game statistics showed that they died more often in the game when interacting with the less aggressive, friendlier-looking crowd.This implies that the heightened sense of panic when viewing the aggressive-looking crowd actually helped gamers to perform better, while they under-estimated the less aggressive-looking crowd.
7. User enjoyment during Gameplay: We took an early prototype of the POPULATE project to GDC 2016, looking for first impressions and feedback on the idea of the project. Conference attendees that passed-by our booth were invited to listen to the idea and try the prototype. A set of questions was administered to participants that were willing to provide us with some feedback. The questions were focused on the intuitiveness of the user interaction, the enjoyment of the game, the game aesthetics, and the overall opinion about it. In general, attendees reported enjoying trying the demo, they found the scenario interesting and challenging and they liked the graphics. Most of the criticism was about the controls of the game, since people did not have the time to train with this new smart-glass technology.
8. Navigation Experiment: The purpose of this preliminary experiment was to explore navigation strategies employed by users in three-dimensional spaces, when using different interaction techniques. The techniques included a typical mobile game setup, with on-screen controls, a gamepad controlled setup, and a mixture of mobile and smart-glass control setup. We investigated how input technologies influence participants’ strategy and ability to navigate and interact with objects in three-dimensional spaces. Overall, users preferred using the gamepad controller for moving forward, and selecting objects. For turning, the smart-glass and gamepad were equally liked, with the on-screen controls liked the least. For moving forward and selecting objects, the smart-glass setup was liked the least. It is possible that participants preferred the gamepad since this is the method of navigation they are most used to from playing video games, it takes longer to get used to the on-screen controls and the smart-glass. The prototype smart-glass (Google glass) had problems during the experiment as it overheated quite often, this may have affected users perception of it. Our results provide guidelines for improvements regarding the use of smart-glass as an input device for mobile video games.

Scientific impact

Through our novel experiments, we have gained new insights and guidelines for game designers and developers for the creation of appealing and engaging virtual characters. Our results also provide new insights into human perception and how humans engage with virtual characters ranging from highly abstract to photorealistic characters. More specifically, we have advanced the state of the art in the area by presenting new results on the effect of animation and appearance realism on user enjoyment and the use of physiological reactions to games. We have also presented new findings for interaction with virtual characters in Immersive Virtual Reality and provided a better understanding of the perception of abstract virtual faces. Our studies were the first to investigate the effect of a self-virtual face using facial tracking, providing valuable insights for game designers. Furthermore, we presented the first study to determine the effect of facial feature alterations on character traits for highly abstract characters. Finally, our work was the first to investigate if there is a difference in co-presence, realism and affinity when controlling or observing a virtual avatar. Our studies and results have already been published in high-impact conferences, with further submissions expected to high-impact journals.

Socio-economic impact

The POPULATE Project addresses to a new category of technology users, as well as creative industries SMEs that can benefit from a game development platform. According to a Nielsen 360° Gaming Report , the virtual and augmented reality devices that recently entered the gaming scene have the potential to change the way users play games, as well as the way in which other types of consumers relate to various types of entertainment.

So far, very few games are really dedicated to this increasingly requested type of connected device, smart-glasses, and no frameworks are available till this current date. As shown in a report published by a specialised non-profit organisation, shipments of smart-glasses will hit 1 billion around 2020, surpassing the expected shipments of mobile phones by 2025. We will witness in 2019 the beginning of the mainstream period for this type of devices. The POPULATE prototype has the advantage of being an already developed platform that will meet the needs of the wearables manufacturers in 2017, when most of the smart-glasses industry is starting to settle on strategies and discuss possibilities.
Furthermore, no game is making the link between smart-glasses and the most widespread mass market console in the world: smartphones and tablets. The POPULATE prototype is the foundation for a development platform that will enable game developers to create new games where the smart-glasses are the central focus or the smart-glasses are having an important role in the game mechanics. The platform will ease up the development of games designed specifically for smart-glasses or games that incorporate them in a certain degree.
POPULATE Project managed to create the first multiplayer asymmetric mobile game prototype and has the potential to define a new genre in mobile gaming.
Asymmetric gameplays are the new trend in video games but so far it is limited to some proprietary hardware. The most known is Nintendo’s Wii U (Rayman Legends, ZombiU). Even though video games studios that create unique looking games are daring to dip their toes into gaming’s final frontier, asymmetrical multiplayer, only some new PC or console games include gentle forms of asymmetrical multiplayer (Left 4 Dead). Regarding this trend on mobile devices, some asymmetric mini-games have been proposed between tablet or smartphone and Xbox One (such as Ubisoft’s Assassin’s creed 4 iOS and Android companion or GTA 5: IFruit). However, they are seen as nice to have “companions” and marketing tools for big video game blockbuster. With its asymmetric gameplay, the POPULATE prototype brings a new experience by using innovative non-proprietary hardware.
The crowd simulation module of the POPULATE project, developed by Golaem, is conceived as a prototype for a future Golaem for Games product for the Unity game engine. After an industrialization process, it will be released as a commercial product on the Unity store. As it provides innovative solutions without any concurrent solution, we expect having a big impact on the Unity game developer community.

populate.png

Periodic Reporting for period 1 - POPULATE (POPULate AsymmeTric mobile gamEs)

Share this page

Download