Journal of Internet of Things and Convergence (사물인터넷융복합논문지)

The Korea Internet of Things Society (한국사물인터넷학회)
권호 표지
DOI QR코드

DOI QR Code

Development of checklist questions to measure AI core competencies of middle school students

중학생의 AI 핵심역량 측정을 위한 체크리스트 문항 개발

  • Eun Chul Lee (Dept. of Child Education, Baekseok University) ;
  • JungSoo Han (Dept. of Computer Engineering, Baekseok University)
  • 이은철 (백석대학교 사범학부) ;
  • 한정수 (백석대학교 컴퓨터공학부)
  • Received : 2024.05.20
  • Accepted : 2024.06.10
  • Published : 2024.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

This study was conducted with the purpose of developing a checklist of questions to measure middle school students' AI capabilities. To achieve the goal of the study, literature analysis and question development Delphi survey were used. For literature analysis, two domestic studies, five international studies, and the Ministry of Education's curriculum report were collected through a search. The collected data was analyzed to construct core competency measurement elements. The core competency measurement elements are understanding of artificial intelligence (5 elements), artificial intelligence thinking (5 elements), utilization of artificial intelligence (4 elements), artificial intelligence ethics (6 elements), and artificial intelligence social-emotion (6 elements). elements). Considering the knowledge, skills, and attitudes of the constructed measurement elements, 31 questions were developed. The developed questions were verified through the first Delphi survey, and 10 questions were revised according to the revision opinions. The validity of 31 questions was verified through the second Delphi survey. The checklist items developed in this study are measured by teacher evaluation based on performance and behavioral observations rather than a self-report questionnaire. This has the implication that the level of reliability of measurement results increases.

본 연구는 중학생의 AI 역량 측정을 위한 체크리스트 문항을 개발하는 목적으로 수행되었다. 연구의 목적을 달성하기 위해서 문헌 분석과 문항개발 델파이 조사를 사용하였다. 문헌 분석을 위해 검색을 통해 국내 연구 2편, 국외 연구 5편, 교육부의 교육과정 보고서를 수집하였다. 수집된 자료를 분석해서 핵심역량 측정 요소를 구성하였다. 핵심역량 측정 요소는 인공지능의 이해(5개 요소), 인공지능 사고(5개 요소), 인공지능 활용(4개 요수), 인공지능 윤리(6개 요소), 인공지능 사회-정서(6개 요소)로 구성하였다. 구성된 측정 요소의 지식과 기능 그리고 태도를 고려하여, 31개 문항을 개발하였다. 개발된 문항은 1차 델파이 조사를 통해서 검증하였고, 수정의견에 따라 10개의 문항을 수정하였다. 2차 델파이 조사를 통해서 31개 문항의 타당성을 검증하였다. 본 연구에서 개발한 체크리스트 문항은 자기보고식 설문이 아닌 수행 및 행동 관찰을 기반으로 교사의 평가에 의해서 측정된다. 이에 측정 결과가 신뢰할 수 있는 수준이 높아진다는 시사점을 가지고 있다.

Keywords

References

  1. E.C.Lee and J.S.Han, "Research on AI education content system composition for early childhood education," Journal of Internet of Things and Convergence, Vol.9 No.5, pp.31-37, 2023. https://doi.org/10.20465/KIOTS.2023.9.5.031
  2. E.C.Lee and Y.S.Pyun, "Research on the composition of AI core competency elements for early childhood AI education," Journal of Internet of Things and Convergence, Vol.9 No.5, pp.55-60, 2023. https://doi.org/10.20465/KIOTS.2023.9.5.055
  3. O.Almatrafi and A.Johri and H.Lee, "A systematic review of AI literacy conceptualization, constructs, and implementation and assessment efforts (2019-2023)," Computers and Education Open, Vol.6, pp.1-20, 2024. https://doi.org/10.1016/j.caeo.2024.100173
  4. I.T.Sanusi and S.A.Olaleye and F.J.Agbo and T.K.F.Chiu, "The role of learners' competencies in artificial intelligence education," Computers and Education: Artificial Intelligence, Vol.3, pp.1-10, 2022. https://doi.org/10.1016/j.caeai.2022.100098
  5. Ministry of Education, "Practical (Technical Home) Information Science curriculum,"Ministry of Education, 2022.
  6. Ministry of Education, "Opening the era of 1:1 personalized education with artificial intelligence (AI) digital textbooks,"Ministry of Education, 2023.
  7. D.T.K.Ng and J.K.L.Leung and S.K.W.Chu and M.S.Qiao, "Conceptualizing AI literacy: An exploratory review," Computers and Education: Artificial Intelligence, Vol.2, pp.1-11, 2021. https://doi.org/10.1016/j.caeai.2021.100041
  8. S.K.Jo and M.S.Choi, "Modeling core competencies for elementary artificial intelligence education," Core Competency Education Research, Vol.7 No.1, pp.43-75,2022. https://doi.org/10.52616/JCCER.2022.7.1.43
  9. J.H.Choi and Y.S. Jeon, "Development and validation of a digital literacy competency assessment tool to foster digital competency in the artificial intelligence era," Research in University Teaching and Learning, Vol.16 No.3, pp.95-122, 2023.
  10. S.C.Kong and M.Y.W.Cheung and O.Tsang, "Developing an artificial intelligence literacy framework: Evaluation of a literacy course for senior secondary students using a project-based learning approach," Computers and Education: Artificial Intelligence, Vol.6, pp.1-11, 2024. https://doi.org/10.1016/j.caeai.2024.100214
  11. M.Y.Ryu and S.K.Han, "The Study on Test Standard for Measuring AI Literacy," Journal of the Korea Computer and Information Society, Vol.28 No.7, pp. 39-46, 2023.
  12. M.C.Laupichler and A.Aster and N.Haverkamp and T.Raupach, "Development of the "Scale for the assessment of non-experts' AI literacy" - An exploratory factor analysis," Computers in Human Behavior Reports, Vol.12, pp.1-10, 2023. https://doi.org/10.1016/j.chbr.2023.100338
  13. T.K.F.Chiu and Z.Ahmad and M.Ismailov and I.S.Sanusi, "What are artificial intelligence literacy and competency? A comprehensive framework to support them," Computers and Education Open, Vol.6, pp.1-9, 2024. https://doi.org/10.1016/j.caeo.2024.100171
  14. A.Carolus and M.J.Koch and S.Straka and M/E.Latoschik and C.Wienrich, "MAILS - Meta AI literacy scale: Development and testing of an AI literacy questionnaire based on well-founded competency models and psychological change- and meta-competencies," Computers in Human Behavior: Artificial Humans, Vol.1 No.2, pp.1-10, 2023. https://doi.org/10.1016/j.chbah.2023.100014
  15. N.Knoth and M.Decker and M.C.Laupichler and M.Pinski and N.Buchholtz and K.Bata and B.Schultz, "Developing a holistic AI literacy assessment matrix - Bridging generic, domain-specific, and ethical competencies," Computers and Education Open, Vol.6, pp.1-14, 2024. https://doi.org/10.1016/j.caeo.2024.100177
  16. M.C.Laupichle and A.Aster and T.Raupach, "Delphi study for the development and preliminary validation of an item set for the assessment of non-experts' AI literacy," Computers and Education: Artificial Intelligence, Vol.4, pp.1-10, 2023. https://doi.org/10.1016/j.caeai.2023.100126