Skip to main content

Language-Affect Interface in Parent-Infant Communication

Periodic Reporting for period 1 - MultiPic (Language-Affect Interface in Parent-Infant Communication)

Reporting period: 2019-01-15 to 2021-01-14

Language, gesture, and affect are the three musketeers of parent-child interaction. Although affect, our experience with emotions, plays a key role in communication, its developmental trajectory remains unclear. How do they perceive emotions from own or other cultures, how does language interact with affect perception, and how do infants’ backgrounds play a role?

This project examines first-year infants’ affect development and its interaction with language using a preferential looking paradigm where infants perceive videos of happy/angry expressions from various cultures along with languages that match or mismatch with the corresponding culture.

The main project objectives are:
1. To examine infants’ cross-age perception of positive and negative emotions expressed by mothers from different cultures.
2. To explore whether languages in the auditory modality modulate infants’ visual perception.
3. To investigate whether infants’ linguistic and cultural experiences alter their perception.

The main findings are:
1. First-year infants can discriminate the subtle cultural differences embedded in emotions.
2. Information (language) in the auditory modality positively modulates infants’ visual preferences. For instance, native language enhances infants’ preference for emotions expressed by familiar race/native culture.
3. Infants raised in relatively more diverse environments show stronger preferential-looking patterns than those growing up in monolingual/uni-cultural environments.

Europe and the world are changing more than ever towards a complex social environment, with interactions between languages and cultures. The outcomes of this project enhance European scientific excellence by contributing to a better understanding of infant culture-specific affectual development in various linguistic contexts. By learning these outcomes, caretakers are expected to change their attitude towards more acceptance of living in a diverse linguistic and cultural environment.
1. These tasks were performed in chronological order:
A. Literature review, Institute introduction, Ethics application
B. Experimental design, Stimuli recording
C. Babylab establishment, Testing environment setup, Hardware and software setup
D. Protocol setup, Infant recruitment, Data collection
E. Data Analysis
F. Experimental Report in manuscripts, workshops, conferences and public dissemination
Note: The pandemic greatly affected project data collection as travelling of both participants and experimenter become difficult.
Regardless, significant data have been collected with the help of the host organization.

2. Results overview
A. Direct to the project, our major preliminary findings are:
- Infants are sensitive to cross-cultural facial expressions in the first year of life. They can tell the differences in emotions expressed between Norwegian and Japanese mothers.
- Infants prefer happy facial expressions from their native culture, exhibiting a familiarity effect. Infants also prefer angry facial expressions from a foreign culture, exhibiting a novelty effect.
- Infants can match language with emotions. They prefer Norwegian emotions when Norwegian language is displayed, and Japanese emotions when Japanese language is displayed.
- Infants from bilingual and bicultural backgrounds show different preferential looking patterns from their monolingual peers.

B. Indirect to the project, I have conducted work in these areas:
- Bilingualism – I further explored research in bilingualism and published articles in this area.
- Infant development – I studied infant cognitive development from an angle different from emotion perception but following the same overarching principle in infant and early childhood development, on how experience shapes infant gender-specific toy preferences.
- International collaborations – I have worked and am working with researchers across the globe on numerous infant research projects.

3 Results exploitation and dissemination
A. During the project:
- Four articles were published or accepted on well-known journals (e.g. Infancy, Developmental Science).
- Four conference proceedings were published.
- Two book chapters and one editorial article were published.
- Two colloquium presentations were given before the pandemic at University of Stockholm and Fudan University, respectively.
- I was also invited to present at three institutes, including Thammasat University, Western Sydney university and University of Oslo.
- Twelve conference presentations were conducted (e.g. Society for the Neurobiology of Language).

B. After the project:
- Three journal articles are currently under review or revision. Seven more articles and one book are in preparation apart from the upcoming publications of the current project.
- Three talks have been scheduled for academic and dissemination purposes, one of which will be offline (Early Literacy Conference, Sydney, Australia) and the other two online (Summer schools, Potsdam, Germany).
- Results will also be disseminated in the unit Psychology: Behavioural Science I plan to coordinate in the section of developmental psychology.
The project fills in three critical areas beyond the state of the art.
1. Cross-culture emotion perception in infancy
2. Effects of speech on infant emotion perception
3. Effects of multilingual, multiracial, and multicultural exposure on infant emotion perception

The corresponding expected results are:
1. Infant show familiarity effects towards positive emotion expressed in experienced culture, and novelty effects towards negative emotion expressed in novel culture.
2. Speech information presented in auditory modality alters infants’ emotion perception patterns. Infants associate native speech with experienced cultural emotion, and non-native speech with novel cultural emotion.
3. Multilingual, multiracial, and multicultural experiences enhance infants’ emotion perception and make them less biased towards emotions expressed in experienced or novel cultures.

The potential socio-economic impacts are:
1. As infants are sensitive to subtle differences in facial expressions across cultures, early exposure to variations of cultural emotions should be considered in the first thousand days after birth when experience is considered critical for early childhood learning and development.
2. As speech and emotion perception interact, infant developmental and education programs should target simultaneous multi-modality functions rather than focusing on input with unimodality.
3. As infants’ multilingual, multiracial, and multicultural experiences positively modulate their emotion perception, such experiences should be considered beneficial and should not be avoided along the developmental trajectory since the beginning of life.

Some existing societal disseminations include:
Media Broadcasts include relevant twitters from the International Congress of Infant Studies (2020) and MARCS BabyLab podcasts with Kinderling Radio (2019). Public lectures include an online lecture for International Mother Language Day (2021) and a workshop The Multilingual Mind Across the Lifespan held in Litteraturhuset, Oslo, Norway (2020). One article is published in The New York Times involving a project I am participating in, "In Quarantine, Kids Pick Up Parents’ Mother Tongues" (2020). An additional six articles were published in Mandarin on Wechat Public Channel.