Marjut Johansson

Communicative machines and AI-mediated interaction

AI-mediated communication has impacted the everyday lives of ordinary people in many ways. In AI-based communication we may engage in speaking or listening with “talking machines.” These are independently functioning machines, such as delivery robots or vacuum cleaners, that may have the capacity to express utterances through a voice interface. Certain machines based on voice-user interfaces (VUIs), such as Siri, Alexa, and Google Assistant, perform different tasks at command. Social robots, used in locations such as museums and schools, have been tested in experiments in which they engage in embodied interactions with human users (cf. Tuncer et al. 2024).  While these are usually short-term situated tasks in everyday life, all these encounters with material artifacts that process and produce speech are becoming ordinary situations of interaction.  The generative AI impacts language learning and teaching as well and has already resulted in many  experiments and developments  (cf. Kern 2024).

In my talk, my aim is to examine (dis)embodied interaction with “communicative machines” (Hepp & al. 2024). Specifically, I will look into verbal interactions between humans and a social robot in foreign language oral learning situation. My presentation will focus on preliminary experimentations with oral task designs prompted with ChatGPT. First, I will investigate the (recipient) design of turns, and second, I will examine how these turns can be understood from an interactional perspective, and third, how human participants align with or disalign from them. The data come from the RoboLang research project, in which we have analyzed foreign language learning focusing on repair, learner-robot relationship, and progressivity in the interaction learners have with social robots (Honkalammi et al. 2022, Peura & Johansson 2022, Jakonen et al. 2023). In these situations the learning tasks have been based on ready-made software or the researcher has programmed the tasks themselves.

In a recent review of EM/CA based studies on AI-mediated interaction, researchers found three main themes: openings and closings, miscommunication and non-verbal aspects of communication (Mlynář et al. 2024). When we engage in AI-mediated communication, one intriguing concern is the verbal interaction what kind of participant is AI. I will focus on AI as an interactional partner and its effect on humans in the (foreign language) interaction. The human–machine interaction can be characterized as asymmetrical, as the interlocutors do not (usually) share common ground (Clark 1996). This means that human participants enact the meaning in these conversations (Johansson 2021).

References

Clark, Herbert 1996. Using Language, Cambridge: CUP.

Hepp, Andreas, Bolin, Göran, Guzman, Andrea L., Loosen, Wiebke 2024. Mediatization and Human-Machine Communication: Trajectories, Discussions, Perspectives. Human-Machine Communication. Special Edition, Volume 7, 2024. https://doi.org/10.30658/hmc.7.1

Honkalammi, Hilla-Marja, Veivo, Outi & Johansson, Marjut 2022. Advice-giving between Young learners in robot-assisted language learning. In: Proceedings of the Conference : Human Perspectives on Spoken Human-Machine Interaction. https://freidok.uni-freiburg.de/data/223816

Jakonen Teppo, Veivo, Outi, Mutta, Maarit, Maijala, Minna, Honkalammi, Hilla-Marja, Johansson, Marjut 2023. ‘Am I saying it wrong?’ Progressivity-related troubles and instructional opportunities in child-robot L2 interaction. Prologi – Journal of Communication and Social Interaction, 20(1)

Johansson, Marjut 2021. Talking with a chatbot: Simulated understanding of human-chatbot communication? In: Johansson, Marjut, Tanskanen Sanna-Kaisa & Chovanec, Jan (Eds.) 2021. Analyzing Digital Discourses. Between Convergence and Controversy. Cham: Palgrave MacMillan. https://doi.org/10.1007/978-3-030-84602-2

Kern, Richard 2024. Twenty-first century technologies and language education: Charting a path forward. The Modern Language Journal, 108.2 (2024): 515–533. https://doi.org./10.1111/modl.12924

Mlynář, Jakub, de Rijk, Lynn, Liesenfeld, Andreas, Stommel, Wyke, Albert, Saul. 2024.  AI in situated action: a scoping review of ethnomethodological and conversation analytic studies. AI & SOCIETY, https://doi.org/10.1007/s00146-024-01919-x

Peura, Liisa and Johansson, Marjut 2022. A Friend or a Machine? Study on the Child-Robot Relationship in a Foreign Language Class of Young Learners. In: Hakli, R. & al. (Eds.) Social Robots in Social Institutions. Frontiers of Artificial Intelligence and Applications. Amsterdam: IOS Press, 165–173.

Tuncer, Sylvaine, Licoppe, Christian, Luff, Paul, Heath, Christian 2024. Recipient design in human–robot interaction: the emergent assessment of a robot’s competence. AI & Society. AI & SOCIETY (2024) 39:1795–1810. https://doi.org/10.1007/s00146-022-01608-7