Journal of Digital Convergence (디지털융복합연구)

The Society of Digital Policy and Management (한국디지털정책학회)
권호 표지
DOI QR코드

DOI QR Code

The Effects of Chatbot on Grammar Competence for Korean EFL College Students

한국 대학생 영어학습자들의 문법 습득에 있어 챗봇의 효과

  • Ahn, Soojin (Liberal Art Institute, Incheon National University)
  • 안수진 (인천대학교 기초교육원)
  • Received : 2022.01.01
  • Accepted : 2022.03.20
  • Published : 2022.03.28
Download PDF
⟨ Previous Next ⟩

Abstract

The purpose of this study was to test whether or not the AI chatbot is effective in acquiring target grammar for Korean EFL college students: prepositions and articles. A quasi-experiment was conducted with 46 first-year students taking part in a required English course. They were randomly divided into two groups: the experimental and control groups (23 students for each, respectively). The experimental group was engaged in six chat sessions with a chatbot over 6 weeks. A pretest and a posttest were used to examine the effectiveness of the chatbot by comparing any changes made in error frequencies of the target grammar in participants' English compositions. The results show that after a conversation with the chatbot, the experimental group significantly reduced the mean of omission errors in both prepositions and articles. To have a great effect in other error categories, chatbot feedback needs to be improved to reduce short responses or inaccurate utterances of students and induce them to actively participate in the conversation.

본 연구의 목적은 인공지능 챗봇이 한국 대학생 영어학습자들에게 전치사와 관사 두 목표 문법을 습득하는데 효과적인지 실험하기 위한 것이었다. 필수 영어 과목을 수강한 46명의 1학년 학생들을 대상으로 하였다. 참여자들은 실험군과 통제군. 두 그룹으로 무작위로 선별되었다 (각 그룹 23명). 실험 그룹은 6주 동안 챗봇과 6번의 채팅 세션을 가졌다. 사전·사후 실험을 통해 영어 작문에서 목표 문법의 오류 빈도의 변화를 비교함으로써 효과를 검토했다. 실험결과는 챗봇과의 대화 후에 실험군이 전치사와 관사 모두에서 누락 오류의 평균을 유의하게 줄였다는 것을 보여줬다. 다른 오류 범주에서 큰 효과를 거두기 위해서는 학생들의 단답형 또는 부정확한 답변을 줄이고 챗봇과의 대화에 적극적으로 참여하도록 유도하는 챗봇 피드백이 개선되어야 한다.

Keywords

References

  1. C. Leacock, M. Chodorow, M. Gamon & J. Tetreault. (2014). Automated Grammatical Error Detection for Language Learners. Morgan & Claypool publishers.
  2. A. Kerly, P. Hall & S. Bull. (2006). Bring Chatbots into Education: Towards Natural Language Negotiation of Open Learner Models. Knowledge-based Systems, 20, 177-185. https://doi.org/10.1016/j.knosys.2006.11.014
  3. A. Shevat. (2017). Designing Bots: Creating Conversational Experiences. Sebastopol, California, United States of America: O'Reilly Media, Inc.
  4. D. K. Shin. (2019). Exploring the Feasibility of AI Chatbots as a Tool for Improving Learners' Writing Competence of English. Korean Journal of Teacher Education, 35(1), 41-55. DOI:https://doi.org/10.14333/KJTE.2019.35.1.41
  5. T. Nghi, T. H. Phuc & N. T. Thang. (2019). Applying AI Chatbot for Teaching a Foreign Language: An Empirical Research. International Journal of Scientific & Technology Research, 8(12), 897-902. http://www.ijstr.org/final-print/dec2019/Applying-Ai-Chatbot-For-Teaching-A-Foreign-Language-An-Empirical-Research-.pdf
  6. H. Yang, H. Kim, D. K. Shin & J. H. Lee. (2019). A Study on Adopting AI-Based Chatbot in Elementary English-Speaking Classes. Multimedia-Assisted Language Learning 22, 184-205. https://doi.org/10.15702/MALL.2019.22.4.184
  7. Y. Xu, D. Wang, P. Collins, H. Lee & M. Warschauer. (2021). Same Benefits, Different Communication Patterns: Comparing Children's Reading with a Conversational Agent vs. a Human Partner. Computers & Education, 161, 104059. https://doi.org/10.1016/j.compedu.2020.104059
  8. O. W. Kwon, K. S. Lee., Y. K. Kim & Y. Lee. (2015). GenieTutor: A Computer Assisted Second Language Learning System Based on Semantic and Grammar Correctness Evaluations, Helm, F., Bradley, L., & Thouesny, S. (Eds) Proceedings of the EUROCALL (pp. 330-335), Dublin: Research-publishing.net. Retrieved from https://files.eric.ed.gov/fulltext/ED564216.pdf
  9. N. Y. Kim. (2019). A Study on the Use of Artificial Intelligence Chatbots for Improving English Grammar Skills. Journal of Digital Convergence, 17(8), 37-46. DOI: https://doi.org/10.14400/JDC.2019.17.8.037
  10. N. Haristiani, A. A. Danuwijaya, M. M. Rifa'i & H. Sarila. (2019). Gengobot: A Chatbot-Based Grammar Application on Mobile Instant Messaging as Language Learning Medium. Journal of Engineering Science and Technology, 14(6), 3158 - 3173. Retrieved from https://jestec.taylors.edu.my/Vol%2014%20issue%206%20December%202019/14_6_7.pdf
  11. L. Fryer & R. Carpenter. (2006). Bots as Language Learning Tools. Language Learning and Technology. Language Learning & Technology, 10(3), 8-14. https://www.researchgate.net/publication/233816040_Bots_as_Language_Learning_Tools
  12. Y. Zhao & C. Lai. (2008). Technology and Second Language Learning: Promises and Problems, Parker, L. (Ed.) Technology-mediated Learning Environments for Young English Learners (pp. 167-205), London, UK: Lawrence Erlbaum Associates.
  13. S. Corder. (1973). Introducing Applied Linguistics. Harmondsworth: Penguin.
  14. A. A. Khansir. (2012). Error Analysis and Second Language Acquisition. Theory and Practice in Language Studies, 2(5), 1027-1032. https://doi.org/10.4304/tpls.2.5.1027-1032
  15. E. Ruane, R. Young & A. Ventresque. (2020). Training a Chatbot with Microsoft LUIS: Effect of Intent Imbalance on Prediction Accuracy, Proceedings of the 25th International Conference on Intelligent User Interfaces Companion (pp. 63-64). Retrieved from https://dl.acm.org/doi/10.1145/3379336.3381494
  16. L. K. Fryer, M. Ainley, A. Thompson, A. Gibson & Z. Sherlock. (2017). Stimulating and Sustaining Interest in a Language Course: An Experimental Comparison of Chatbot and Human Task Partners. Computers in Human Behavior, 75, 461-468. https://doi.org/10.1016/j.chb.2017.05.045
  17. J. Hill, W. R. Ford & I. G. Farreras. (2015). Real Conversations with Artificial Intelligence: A Comparison Between Human-Human Online Conversations and Human-Chatbot Conversations. Computers in Human Behavior, 49(3), 240-250.