Periodic Reporting for period 1 - REaCHeS (References to Environs are Coordinated to be Heard and Seen (REaCHeS): an investigation of multimodal spatial referencing in Eastern Chatino)
Berichtszeitraum: 2019-08-01 bis 2021-07-31
People interact in the world, and often make an effort to draw one another’s attention to places, objects and actors within it. To draw attention toward an object or target is to indicate that target, an act that can be performed with speech (e.g.. using words like ‘this’ or ‘that’, ‘here’ or ‘there’) and with gestures (e.g. pointing with the hands and/or head). Targets can be indicated using a single strategy in isolation: a word like ‘there’ with no accompanying gesture, or a silent point, may be enough to effectively draw a person’s attention toward the target. Yet in natural discourse people are overwhelmingly likely to combine speech and gesture to indicate.
It is clear that indicating is typically performed by combining speech and gesture. Yet prior research in linguistics, psychology, and anthropology has very often explored indicating as though it were performed using speech alone. The few studies that have broken this trend, and that have asked how people combine gestures and speech to indicate, have been performed in laboratory settings, where research participants were asked to point out objects in ways that did not closely mirror natural interaction. These studies were nearly always focused on English, and looked narrowly at what factors caused people to use words like ‘this’ or ‘that’ without considering the host of indicating terms (or demonstrative expressions) that exist in the world’s languages, or the many strategies for pointing that are attested the world over. As a consequence, we know very little about how indicating is performed in real-world settings, using the rich set of strategies that are available across the world’s languages and across many cultures’ conventions for gesturing.
The REaCHeS project has been unique in: (1) studying the complex, multi-channel act of pointing, (2) in real-world settings, (3) with speakers of an under-documented and typologically distinctive language: Quiahije Chatino, a Zapotecan language spoken in Oaxaca, Mexico.
It is clear that indicating is typically performed by combining speech and gesture. Yet prior research in linguistics, psychology, and anthropology has very often explored indicating as though it were performed using speech alone. The few studies that have broken this trend, and that have asked how people combine gestures and speech to indicate, have been performed in laboratory settings, where research participants were asked to point out objects in ways that did not closely mirror natural interaction. These studies were nearly always focused on English, and looked narrowly at what factors caused people to use words like ‘this’ or ‘that’ without considering the host of indicating terms (or demonstrative expressions) that exist in the world’s languages, or the many strategies for pointing that are attested the world over. As a consequence, we know very little about how indicating is performed in real-world settings, using the rich set of strategies that are available across the world’s languages and across many cultures’ conventions for gesturing.
The REaCHeS project has been unique in: (1) studying the complex, multi-channel act of pointing, (2) in real-world settings, (3) with speakers of an under-documented and typologically distinctive language: Quiahije Chatino, a Zapotecan language spoken in Oaxaca, Mexico.
1. Research activities, organized within work packages
The MSCA Research Fellow worked in the Lund University Humanities Lab under the supervision of Professor Marianne Gullberg, with periodic supervision and feedback from Associate Professor Niclas Burenhult. The Fellow also worked extensively with Associate Professor Emiliana Cruz, a collaborator at CIESAS Mexico City, and was given the status of guest researcher at CIESAS through the sponsorship of Prof. Cruz. The positive working relationships established and maintained at Lund University and at CIESAS Mexico City allowed the Fellow to meet the ambitious goals of the project, and facilitated the accomplishments identified in this report.
1.1 WP 1: First Study -- Locating Landmarks in Natural Discourse
The goal for the first work package was to assemble an on-site research team for data collection in Oaxaca, Mexico, to oversee this team to collect, transcribe, and analyze data for a first study of multimodal indicating strategies for Chatino speakers, and to write and publish a paper summarizing the study results.
All milestones and deliverables for WP1 were completed. These included:
M1: Train Chatino-speaking research assistants.
M2: Collect discourse data.
M3: Train Lund-based RAs.
D1: Upload video data from the first study to an open access repository.
Mesh, K., Gullberg, M., and Cruz, E. (2020). REaCHeS Dataset. Lund University Corpus Server. http://hdl.handle.net/10050/00-0000-0000-0004-1F68-A@view
D2: Submit one publication to a high-impact venue.
Mesh, K., Cruz, E., van de Weijer, J., Burenhult, N., and Gullberg, M. (2021.) Effects of scale on multimodal deixis: Evidence from Quiahije Chatino. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2020.584231
D3: Report on ethics and data management.
1.2 WP2: Second Study: Targeted Experimental Approach Using a Referential Communication Task [This was replaced, due to restrictions from the COVID-19 pandemic, with a study of visual attention during question-answer sequences]
The goal for the second work package was to return to the field site in Oaxaca, Mexico, to complete a more targeted experiment. The COVID-19 pandemic made all travel to Mexico impossible. Instead, the Fellow returned to the original dataset collected in Year 1 and developed a research question that could be addressed with additional coding of the originally collected video data.
All milestones and deliverables for WP2 were completed in modified form to suit the constraints of the COVID-19 pandemic. These included:
M4:. End of experimental data collection.
M5: (Re)train RAs
D4: Upload experimental datasets to an online repository (the original dataset was updated with new metadata and annotations).
D5: Publish results. [Paper submitted; currently under review.]
Mesh, K. Gullberg, M. and Cruz, E. (Submitted). When attentional and politeness demands clash: The case of mutual gaze and chin pointing in Quiahije Chatino. https://psyarxiv.com/yuxt8
In addition, the Fellow completed activities exceeding the stated expectations, including the publication of an additional research paper.
Mesh, K. (2021). It’s as far as the arm can raise: Pointing height marks target distance among the San Juan Quiahije Chatino. Lingua. 259: 103099.
https://doi.org/10.1016/j.lingua.2021.103099
2. Training and of the two way transfer of knowledge between the Fellow and the host institution
2.1 The Fellow performed trainings for the following organizations:
(1) Adult education programme of San Juan Quiahije; (2) Multimodality and Coexpressivity in Linguistic Interaction in Mesoamerica, a workshop funded through UC-MEXUS CONACYT; (3) Lund University
2.2. The Fellow gave 3 lectures and public talks in university settings over the course of the fellowship period.
2.3. The Fellow participated in the the research activities of the hosting institution, Lund University, including the monthly or fortnightly meetings of 4 research groups.
3. The fellow served as an advisor and mentor in the following capacities: (1) Dissertation co-supervisor for a Ph.D student at the University of Goettingen (Student: Rehana Omardeen); (2) Mock examiner in preparation for a doctoral defense at Stockholm University (Defendant: Josefina Safar).
The MSCA Research Fellow worked in the Lund University Humanities Lab under the supervision of Professor Marianne Gullberg, with periodic supervision and feedback from Associate Professor Niclas Burenhult. The Fellow also worked extensively with Associate Professor Emiliana Cruz, a collaborator at CIESAS Mexico City, and was given the status of guest researcher at CIESAS through the sponsorship of Prof. Cruz. The positive working relationships established and maintained at Lund University and at CIESAS Mexico City allowed the Fellow to meet the ambitious goals of the project, and facilitated the accomplishments identified in this report.
1.1 WP 1: First Study -- Locating Landmarks in Natural Discourse
The goal for the first work package was to assemble an on-site research team for data collection in Oaxaca, Mexico, to oversee this team to collect, transcribe, and analyze data for a first study of multimodal indicating strategies for Chatino speakers, and to write and publish a paper summarizing the study results.
All milestones and deliverables for WP1 were completed. These included:
M1: Train Chatino-speaking research assistants.
M2: Collect discourse data.
M3: Train Lund-based RAs.
D1: Upload video data from the first study to an open access repository.
Mesh, K., Gullberg, M., and Cruz, E. (2020). REaCHeS Dataset. Lund University Corpus Server. http://hdl.handle.net/10050/00-0000-0000-0004-1F68-A@view
D2: Submit one publication to a high-impact venue.
Mesh, K., Cruz, E., van de Weijer, J., Burenhult, N., and Gullberg, M. (2021.) Effects of scale on multimodal deixis: Evidence from Quiahije Chatino. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2020.584231
D3: Report on ethics and data management.
1.2 WP2: Second Study: Targeted Experimental Approach Using a Referential Communication Task [This was replaced, due to restrictions from the COVID-19 pandemic, with a study of visual attention during question-answer sequences]
The goal for the second work package was to return to the field site in Oaxaca, Mexico, to complete a more targeted experiment. The COVID-19 pandemic made all travel to Mexico impossible. Instead, the Fellow returned to the original dataset collected in Year 1 and developed a research question that could be addressed with additional coding of the originally collected video data.
All milestones and deliverables for WP2 were completed in modified form to suit the constraints of the COVID-19 pandemic. These included:
M4:. End of experimental data collection.
M5: (Re)train RAs
D4: Upload experimental datasets to an online repository (the original dataset was updated with new metadata and annotations).
D5: Publish results. [Paper submitted; currently under review.]
Mesh, K. Gullberg, M. and Cruz, E. (Submitted). When attentional and politeness demands clash: The case of mutual gaze and chin pointing in Quiahije Chatino. https://psyarxiv.com/yuxt8
In addition, the Fellow completed activities exceeding the stated expectations, including the publication of an additional research paper.
Mesh, K. (2021). It’s as far as the arm can raise: Pointing height marks target distance among the San Juan Quiahije Chatino. Lingua. 259: 103099.
https://doi.org/10.1016/j.lingua.2021.103099
2. Training and of the two way transfer of knowledge between the Fellow and the host institution
2.1 The Fellow performed trainings for the following organizations:
(1) Adult education programme of San Juan Quiahije; (2) Multimodality and Coexpressivity in Linguistic Interaction in Mesoamerica, a workshop funded through UC-MEXUS CONACYT; (3) Lund University
2.2. The Fellow gave 3 lectures and public talks in university settings over the course of the fellowship period.
2.3. The Fellow participated in the the research activities of the hosting institution, Lund University, including the monthly or fortnightly meetings of 4 research groups.
3. The fellow served as an advisor and mentor in the following capacities: (1) Dissertation co-supervisor for a Ph.D student at the University of Goettingen (Student: Rehana Omardeen); (2) Mock examiner in preparation for a doctoral defense at Stockholm University (Defendant: Josefina Safar).
Impacts anticipated from the REaCHeS project include:
Increased understanding of the factors influencing when and how communicators combine speech and gesture to indicate targets in their surrounding environment (an impact of the first study, performed in WP1);
A richly supported understanding of how speakers manage visual attention toward their own multimodal messages, and of how listeners demonstrate their attention to those messages (an impact of the second study, performed in WP2);
Increased research attention toward the smaller, typologically distinct languages that provide a valuable lens on the diversity of human communication strategies (an impact of the two planned studies, and the additional research paper produced by the Fellow during the fellowship period)
Increased community buy-in for Chatino literacy programs in the focal research community, San Juan Quiahije, Oaxaca, Mexico. The transcription training provided to research assistants who supported the project has resulted in increased enthusiasm for writing in Quiahije Chatino -- an especially welcome impact from the project.
Increased understanding of the factors influencing when and how communicators combine speech and gesture to indicate targets in their surrounding environment (an impact of the first study, performed in WP1);
A richly supported understanding of how speakers manage visual attention toward their own multimodal messages, and of how listeners demonstrate their attention to those messages (an impact of the second study, performed in WP2);
Increased research attention toward the smaller, typologically distinct languages that provide a valuable lens on the diversity of human communication strategies (an impact of the two planned studies, and the additional research paper produced by the Fellow during the fellowship period)
Increased community buy-in for Chatino literacy programs in the focal research community, San Juan Quiahije, Oaxaca, Mexico. The transcription training provided to research assistants who supported the project has resulted in increased enthusiasm for writing in Quiahije Chatino -- an especially welcome impact from the project.