Le Colloque > Résumés > Schires Marc

Within the field of teaching and learning additional languages (LX), a large body of papers are published on Generative Artificial Intelligence (GenAI) since 2022 (Babarti et al., 2024; Ahmed et al., 2023; Xing, 2023). However, the few empirical studies to have investigated the impact of ChatGPT on students' language skills have mainly relied on emic data from questionnaires, interviews and focus group discussions (Monika & Suganthan, 2024; Alm & Ohashi, 2024). Our study explores the direct impact of GenAI on student writing through the examination of linguistic aspects such as lexical frequency and range, parts of speech and participle clauses as a possible example of an AI-favoured grammatical construction (Authors et al., 2024). 

Students were required to submit a collaborative project relating to their discipline including a 400-word synthesis of chosen texts. The study is guided by three main research questions: (1) Which tools (especially GenAI) do the participants use to write their assignments, and what do they use them for? (2) What proportion of the groups use GenAI to write their assignments, as self-declared and as identified by an AI detection tool? (3) How does ChatGPT impact their writing, especially the final synthesis? The participants were 753 students at Université de Lorraine enrolled in humanities and social sciences; 190 out of 245 assignments were exploitable, and analysed using different tools including Python, AntConc, ChatGPT 4.0 and GPTzero.

The results show that about a quarter of students explicitly acknowledged using GenAI, but initial comparisons with AntConc found no obvious differences between declared users and non-users. The texts were then imported into an AI detector, which produced rather different groupings; separating the extremes uncovered substantial differences. First, while learner-produced texts are relatively heavy on common items, AI generates lexically richer texts. Second, there is a high percentage of present participle clauses (cVing: Comma plus Verb in –ING form) in the assignments, confirming our assumption that GenAI tools impact student writing by “injecting” those patterns in their writing. Although cVing structures were found in both GPT and non-GPT groups (560 occurrences), results show assignments containing no cVing structures in both groups, but the GPT groups' assignments contained a higher proportion of cVing (274), meaning the 34.7% of the GPT group employed almost half (49.0%) of the overall use of cVing compared to the 65.3% of the assignments that claimed not to use ChatGPT (51%).

Clearly there is a need for further research to grasp the impact of GenAI tools on students' Lx writing, with the aim of exploring other levels of lexis, grammar and discourse. Finally, we discuss pedagogical implications and further questions related to the impact of GenAI in the field of Applied Linguistics.

References

Authors. (2024, in press).

Ahmed, S. M. A. A., Taha, A. R. A., Hussain, S., & Hayat, A. (2023). Enhancing the teaching and learning of English for specific purposes (ESP) with ChatGPT. International Journal of Technology and Education Research, 1(3).

Alm, A., & Ohashi, L. (2024). A worldwide study on language educators' initial response to ChatGPT. Technology in Language Teaching & Learning, 6(1), 1–23. https://doi.org/10.29140/tltl.v6n1.1141

Babarti, S. E., Colin de le Higuera, & Arnold Magdelaine. (2024). Uses and perceptions of ChatGPT by higher education students. ACM New York, 1–11.

Monika, M., Suganthan, C. (2024). A study on analyzing the role of ChatGPT in English acquisition among ESL learners during English language classroom. Bodhi International Journal of Research in Humanities, Arts and Science, 8(2), 75–84. http://dx.doi.org/10.13140/RG.2.2.28252.56961

Xing, R. (2023). Advancements in English listening education: Chat GPT and convolutional neural network integration. Journal of Pedagogical Research, 7(5), 280–290. https://doi.org/10.33902/JPR.202323980