Skip to main content

Emotional Machines: The Technological Transformation of Intimacy in Japan

Periodic Reporting for period 2 - EMTECH (Emotional Machines:The Technological Transformation of Intimacy in Japan)

Reporting period: 2019-02-01 to 2020-07-31

EMTECH examines the relationship between aesthetic works on human–machine interaction (HRI) in Japan and advances in artificial emotional intelligence. EMTECH is a genuinely interdisciplinary collaboration between the humanities (cultural and literary studies, Japanese studies), social sciences (anthropology), the affective sciences, and the “hard” sciences (robotics, engineering, and affective computing).

Through analysis of new storytelling techniques, virtual spaces, and ethnographic observations of HRI in homes and public spaces, EMTECH’s purpose is to collect qualitative data on our expanding capacities to use and relate emotionally to technology. The project's ultimate aim is to examine how humans create affective bonds to (emotionally intelligent) machines and determine what implications this has on social issues in the private and public spheres in everyday life in Japan. EMTECH promises to produce empirical evidence of how software with the capacity to read affective feedback in humans is not reproducing traditional Japanese culture but in fact transforming it by offering novel strategies for coping with stress, loneliness, depression, and the loss of traditional forms of intimacy. By monitoring the effects of the recent introduction of machines equipped with emotionally sensitive software into Japanese homes, EMTECH documents the technological transformation of intimacy within sites such as the everyday domestic spaces of the new Japanese family.

In Section A, narratives/fiction depicting interactions between humans and machines or humans and new technologies are dissected in order to show how intimacy/distance between humans and new technologies, or the future relationships between humans and machines, are imagined. In addition, new technologies from emotion research are utilized. By employing affect-sensing devices and state-of-the-art affect analysis software, new insights on the affective reception of texts are gained. Quantitative data on affect processes are collected and, subsequently, used in interviews to generate qualitative data that ultimately shed light on the emotional evaluation of text reception. Section B also deals with interactions between technology and culture. Its research focus examines the ways in which humans create affective bonds, either to or by using new technologies. It investigates how humans form intimate relationships to virtual characters (e.g. holograms) or to characters in computer games, and how humans interacting in a virtual space (e.g. as avatars or in chat rooms) establish relationships among themselves. What repercussions such 'virtual relationships' might have on the formation of communities in real life, or what degree of influence they have on social interaction, traditional social structures, or human attachment capacities, is of further interest. Section C investigates, in the context of field research in Japan, how humans in public and private spaces develop various 'relationships' with machines of different types. The coexistence and interactions of humans and machines are researched through participatory observations and interviews. Moreover, this focus studies how people’s acceptance and assessment of machines may change over the course of months or years. The aim is to see if and how intimacy between humans and machines is generated and what repercussions this has on social environments.
(A) In order to distinguish novel HRI in Japanese (popular) culture, EMTECH collected material and compiled and analysed an extensive list of literary texts, movies/anime, manga, theatre plays and games released since the 1980s. Section A also uses affect-sensing devices to gain novel insights into the emotional status (affects and feelings) of Japanese test subjects when it comes to quantitative and qualitative data and narratives of feelings. These experiments are situated in the fields of the affective sciences and empirical studies of literature.

(B) EMTECH also includes an anthropological study of “Japanese romance gaming culture” (digital boyfriends, holographic spouses, AR/VR partners) and of the spread of affective devices in Japan (matrimonial intercession robots, match-making software). During fieldwork, EMTECH firstly established contact with romance device users and became familiar with their habits, rituals and terminology; secondly, recruited a pool of participants for a longitudinal follow-up study of the players’ community and individual practices, and, thirdly, started in-depth fieldwork on the manufacturers and service providers of emotional commodities. EMTECH met with famous romance app makers including Voltage, Cybird, and Idea Factory to conduct interviews with scenario writers, producers and sales representatives (investigating games’ history, business model, settings, characters, plotlines, design, genre, etc.). EMTECH also interviewed roboticists, electro-communication artists and engineers specialized in emotions.

(C) EMTECH performed a long-term ethnographic study of families living with companion robots in Japan. This empirical research followed Japanese families living with the emotional robot Pepper to investigate how humans integrate robots into their everyday life, how they communicate with them, and to what extent robots are accepted as companions or even family members. EMTECH identified different strategies to cope with the idiosyncrasies/limitations of the robot, ranging from a loss of acceptance to increasing humanization of the machine.

To initiate fruitful dialogue about the function and influence of HRI for society, the responsibility of roboticists for their inventions, and to show processes of mutual interdependence, EMTECH brought together researchers from various disciplines with roboticists and artists for two international conferences and a workshop. EMTECH also shares research insights with the non-scientific public via public presentations, TV interviews, newspaper articles, and social media. EMTECH's activities to date have led to the publication of several articles in peer-reviewed journals, magazines and scientific blogs.
In the remaining research period, EMTECH will continue to deepen scientific and public knowledge on the mechanisms of intimacy-building between humans and machines.

Further results are expected regarding the cultural narratives that have influenced roboticists to make the machines they create emotionally intelligent. The project will answer why so many machines are designed as (communication) 'partners/companions'.
EMTECH also expects to contribute to determining the limits of measuring/quantifying feelings by conducting field research in Japan that applies technologically enhanced reader-response theories. Affect-sensing technology will facilitate further insight into human feelings.

In the second half of the project, further technologies will be brought into focus, such as so-called holographic assistants. It will be shown how intimacy is created in the context of artificial emotional intelligence products, and to what degree their acceptability is affected if they are non-corporeal. Moreover, the project will explore the impact of extended interactions with digital entities on interpersonal bonding capacities. EMTECH will continue to collect data on the narrative structures of games and carry out further game-based surveys (comparative studies based on game play experiences), while transcribing, processing and analysing the interviews conducted with members of the Japanese gaming scene. A new series of semi-structured interviews targeting both the consumers and the makers of emotional commodities is planned. Supplementary data will come from on-site observations (at game events, players’ interactions outdoor and indoor, game factories, etc.).

Researching how people live with robots in a private space, and how dynamics within a family are transformed when a new robotic family member is added, is significant not only from a macrostructural societal angle, but also as regards the extension of human capacities to create new forms of relationship and structures of feelings. By continuing with fieldwork in Japan and conducting participant observation of individuals interacting with additional emotionally intelligent devices, such as Lovot, in domestic settings, EMTECH will further examine the impact of emotional machines on human relationships, especially on intimacy and empathy.

When it comes to high-tech products and their application, Japan is generally considered to be at the cutting edge, and so often serves as a trial balloon for trends. After a delay of up to several years these trends typically arrive in Europe and other parts of the world (e.g. exemplified by the widespread adoption of internet-ready smartphones). Thus, EMTECH’s approach may give European researchers a head start, enabling them to study newly emerging phenomena in Japan before they become relevant in Europe.

EMTECH will continue to share the outcomes of its research with both the scientific community and the public by publishing articles and monographs, and future conferences and workshops will serve as a platform for further interdisciplinary and cross-cultural dialogue in HRI.
Fieldwork with robot Pepper in Japanese families.
EMTECH members at Pepper Atelier Akihabara with Softbank, Tokyo.