Journal of Digital Convergence (디지털융복합연구)

The Society of Digital Policy and Management (한국디지털정책학회)
권호 표지
DOI QR코드

DOI QR Code

The influence of users' satisfaction with AWE on English learning achievement through self-efficacy: using PLS-SEM

영어 자동쓰기평가(AWE) 사용만족도가 자기효능감을 매개로 학업성취감에 미치는 영향: PLS-SEM 모델 분석

  • Joo, Meeran (College of Liberal Arts, Dankook University)
  • 주미란 (단국대학교 자유교양대학)
  • Received : 2021.07.01
  • Accepted : 2021.09.20
  • Published : 2021.09.28
Download PDF
⟨ Previous Next ⟩

Abstract

The purpose of this study is to identify the influence of users' satisfaction with the Automatic Writing Evaluation(AWE) on learners' sense of learning achievement through self efficacy in English writing class. AWE is a tool that automatically provides feedback on writing outputs by AI technology. College students were asked to write essays for each topic and use AWE to get feedback on their drafts, and finally revise them referring to the feedback. A questionnaire survey was conducted for the data collection. The data was analyzed using SPSS, and smart PLS-SEM along with bootstrapping techniques, The results of the study reveal the followings: 1) the convenience and usefulness of AWE had a positive effect on the willingness to reuse it; 2) the satisfaction with AWE had a positive effect on self-efficacy; 3) self-efficacy had a positive effect on learning achievement in terms of emotional and linguistic aspects. With the development of the 4th industry and A.I. technology, it is recommended to introduce new materials or programs such as AWE in English education.

이 연구의 목적은 영어쓰기 교과목에서 자동쓰기평가(AWE) 프로그램의 사용자 만족도가 영어쓰기 자기효능감을 매개로 학습자의 학습성취감에 미치는 영향을 알아보기 위한 것이다. AWE는 쓰기 결과물에 대해 인공지능 기술에 의해 자동으로 피드백을 제공하는 프로그램이다. 영어쓰기 교과목을 수강하는 대학생을 대상으로 각 주제별로 작문을 하고 AWE 프로그램을 사용하여 피드백을 받은 후 그것을 참고하여 최종 수정본을 제출하도록 하였다. 설문지를 통해 수집된 데이터(n=99)를 SPSS, Smart PLS-SEM으로 분석하였다. 연구결과, 첫째, AWE의 사용 편의성과 유용성은 재사용 의지에 긍정적 영향을 미치는 것으로 나타났다. 둘째, AWE 사용 만족도는 영어쓰기 자기효능감에 긍정적 영향을 미치는 것으로 나타났다. 셋째, 영어쓰기 자기효능감은 언어적, 정서적 측면에서 학업 성취감에 긍정적 영향을 미치는 것으로 나타났다. 4차 산업 및 인공지능 기술 발달에 따라 영어교육에 AWE와 같은 새로운 학습재료 도입을 권장한다.

Keywords

References

  1. D. Ferris. (2006). Does Error Feedback Help Student Writers? New Evidence on the Short- and Long-term Effects of Written Error Correction. In K. Hyland, & F. Htland (Eds.), Feedback in Second Language Writing: Contexts and Issues (pp. 81-104). New York, NY: Cambridge University Press.
  2. K. Hyland & F. Hyland. (2019). Feedback in Second Language Writing: Contexts and Issues. Cambridge University.
  3. I. Leki. (1991). The Preferences of ESL Students for Error Correction in College-level Writing Classes. Foreign Language Annals, 24(3), 203-218. https://doi.org/10.1111/j.1944-9720.1991.tb00464.x
  4. S. Dikli. (2006). An Overview of Automated Scoring of Essays. The Journal of Technology, Learning and Assessment, 5(1), 4-35.
  5. G. L. Parra, & S. X. Calero. (2019). Automated Writing Evaluation Tools in the Improvement of the Writing Skill. International Journal of Instruction, 12(2), 209-226. https://doi.org/10.29333/iji.2019.12214a
  6. M. Stevenson & A. Pakhti. (2014). The Effects of Computer-generated Feedback on the Quality of Writing. Assessing Writing, 19, 51-65. https://doi.org/10.1016/j.asw.2013.11.007
  7. J. Wilson, & R. Roscoe. (2019). Automated Writing Evaluation and Feedback: Multiple Metrics of Efficacy. Journal of Educational ComputingResearch, https://doi.org/10.1177/0735633119830764.
  8. P. Wang. (2015). Effects of an Automated Writing eEaluation Program: Student Experiences and Perceptions. Electronic Journal of Foreign Language Teaching, 12(1), 79-100.
  9. W. Koh. (2017). Effective Applications of Automated Writing Feedback in Process-based Writing Instruction. English Teaching, 72(3), 91-118. https://doi.org/10.15858/engtea.72.3.201709.91
  10. Y. Lee. (2017). A Case Study on the Long-term Effect of Automated Writing Evaluation Feedback on Writing Development. Journal of the Korea English Education Society, 16(4), 105-130. https://doi.org/10.18649/jkees.2017.16.4.105
  11. Y. Lee. (2010). Effect of Automated Writing Evaluation Feedback on Korean University Students' Revision Behavior. Foreign Languages Education, 27(4), 1-22. https://doi.org/10.15334/FLE.2020.27.4.1
  12. K. Si, D. Park, & H. Lim. (2014). Applicabilities of Automated Short-answer Scoring to Large-scale English Writing Tests. The Journal of Curriculum and Evaluation, 17(2), 71-97. https://doi.org/10.29221/jce.2014.17.2.71
  13. Y. Lee. (2016). Investigating the Feasibility of Generic Scoring Models of E-rater for TOEFL iBT Independent Writing Tasks, English Language Teaching, 28(1), 101-122.
  14. C. F. E. Chen & W. Y. E. C. Cheng (2008). Beyond the Design of Automated Writing Evaluation: Pedagogical Practices and Perceived Learning Effectiveness in EFL Writing Classes. Language Learning & Technology, 12(2), 94-112.
  15. J. Li, S. Link, & V. Hegelheimer. (2015). Rethinking the Role of Automated Writing Evaluation (AWE) Feedback in ESL Writing Instruction. Journal of Second Language Writing, 27, 1-18. https://doi.org/10.1016/j.jslw.2014.10.004
  16. S. Link, M. Mehrzad, & M. Rahimi. (2020). Impact of Automated Writing Evaluation on Teacher Feedback, Student Revision, and Writing Improvement. Computer Assisted Language Learning, 33, 1- 30. https://doi.org/10.1080/09588221.2020.1743323.
  17. Z. Lu, X. Li, & Z. Li. (2015). AWE-based Corrective Feedback on Developing EFL Learners' Writing Skill. In F. Helm, L. Bradley, M. Guarda, & S. Thouesny (Eds.), Critical CALL-Proceedings of the 2015 EUROCALL Conference Padova, Italy. (pp. 375-380). Dublin: Reserach-publishing.net.
  18. A. Bandura. (1995). Exercise of Personal and Collective Efficacy in Changing Societies. Self-efficacy in Changing Societies, 1-45, https://doi.org/10.1017/CBO9780511527692.003.
  19. A. Kim. (2004). Self-efficacy and Academic motivation. The Korean Journal of Educational Methodology Studies, 16(2), 1-38. https://doi.org/10.3946/kjme.2004.16.1.1
  20. Chen & Lin. (2009). Academic Stress Inventory of Students at Universities and Colleges of Technology. World Transactions on Engineering and Technology Education, 7(2), 157-162.
  21. S. I. Han, & S. Lee. (2012). Influence of undergraduate students' English Proficiency and Active Class Participation on Academic Self-Efficacy, Class Satisfaction, and Academic achievement. The Journal of Yeolin Education, 20(2), 103-125.
  22. L. S. Teng, P. P. Sun, & L. Xu. (2018). Conceptualizing Writing Self Efficacy in English as a Foreign Language Contexts: Scale Validation Through Structural Equation Modeling. TESOL Quarterly, 52(4), 911-942. https://doi.org/10.1002/tesq.432
  23. J. C. Anderson & D. W. Gerbing. (1998). Structural Equation Modeling in Practice: A Review and Recommended Ttwo-step Approach. Psychological Bulletin 103, 411-423. https://doi.org/10.1037//0033-2909.103.3.411
  24. W. W. Chin. (2010). How to Write Up and Report PLS Analyses. In V. E. Vinzi, W. W. Chin, J. Henseler, & H. Wan, (Eds.), Handbook of Partial Least Squares: Concepts, Methods and Applications. (pp. 655-690). Springer, Heidelberg.
  25. L. A. Clark, & D. Watson. (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7(3), 309-319. https://doi.org/10.1037/1040-3590.7.3.309
  26. R. Li, Z. Meng, M. Toan, Z. Zhang, C. Ni, & W. Xiao. (2019). Examining EFL Learners' Individual Antecedents on the Adoption of Automated Writing Evaluation in China. Computer Assisted Language Learning, 32(7), 784-804. https://doi.org/10.1080/09588221.2018.1540433
  27. H. C. Liao. (2016). Enhancing the Grammatical Accuracy of EFL Writing by Using an AWE-assisted Process Approach. System, 62, 77-92. https://doi.org/10.1016/j.system.2016.02.007.