Journal of The Korean Association of Information Education (정보교육학회논문지)

Korea Association of Information Education (한국정보교육학회)
권호 표지
DOI QR코드

DOI QR Code

Development and validation of heuristics for usability improvements on web-based diagnostic assessment management system for primary-middle schoolers

초·중학생 대상 웹기반 진단평가관리시스템 사용성 개선을 위한 휴리스틱스 개발 및 검증

  • Cha, Hyunjin (ON Education Design Research Institute) ;
  • Hwang, YunJa (Center for Innovative Engineering Education, Dankook University)
  • 차현진 (온교육디자인연구소) ;
  • 황윤자 (단국대학교 공학교육혁신센터)
  • Received : 2017.11.06
  • Accepted : 2017.12.01
  • Published : 2017.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Preparing the 4th industrial revolution age, a diagnostic assessment management system which can collect nationwide data has been implemented and utilized for the development of the evidence-based educational policy. This study aims to develop heuristics to help to improve usability of such systems, which are utilized by primary and middle school community including teachers and students in the nationwide. To achieve the objective, previous studies related to the development of heuristics in the educational contexts as well as basic heuristics from Nielsen were reviewed, and usability problems of the web-based diagnostic-supplement system being currently utilized, as a sample system, were analyzed. Then, the first version of heuristics was developed. The developed heuristics were validated and revised through 2 rounds of the delphi methods with 15 experts. This study has an implication with preparing foundations on improving usability of such diagnostic assessment management system utilized by school community at the nationwide in educational contexts.

4차 산업혁명 시대를 맞이하여 증거기반의 교육정책을 위해, 전국단위의 진단평가 데이터를 수집하기 위한 시스템을 구축 활용하고 있다. 본 연구는 이러한 전국 단위 초 중생이 활용하고 있는 진단평가관리시스템의 사용성 개선을 도울 수 있는 휴리스틱스 개발을 목적으로 하였다. 이를 위해 Nielsen의 기본 휴리스틱스를 비롯하여 교육적 목적에 종속적으로 개발된 휴리스틱스에 대한 선행연구를 기반으로 유사시스템인 진단 보정학습시스템의 사용성 문제를 분석함으로써 휴리스틱스 초안을 개발하였다. 이는 델파이 방법을 통해 15명의 전문가에 의해 2회의 타당화 과정을 수행함으로써 수정 보완되었고 최종안이 도출되었다. 본 연구는 교육 맥락에서 사용성 평가도구를 제공함으로써 전국단위의 관계자(교사 등)와 학생들이 활용하는 시스템의 사용성 개선을 위한 기반을 마련하였다는데 의의가 있다.

Keywords

References

  1. Adams Becker, S., Cummins, M., Davis, A., Freeman, A., Hall Giesinger, C., and Ananthanarayanan, V. (2017). NMC Horizon Report: 2017 Higher Education Edition. Austin, Texas: The New Media Consortium.
  2. Ahn, M.L, Noh, S.J., Kim, S.M. (2009). Research paper of Web accessibility.
  3. Albion, P. R. (1999). Heuristic evaluation of educational multimedia: From theory to practice. In 16th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education, ASCILITE, 9-15. Brisbane, Austrlia.
  4. Alsumait, A. a., & Al-Osaimi, A. (2010). Usability heuristics evaluation for child e-learning applications. Journal of Software, 5(6).
  5. Baker, K., Greenberg, S., Gutwin, C. (2002). Empirical Development of a Heuristic Evaluation Methodology for Shared Workspace Groupware, CSCW'02, November 16-20, 2002, New Orleans, Louisiana, USA.
  6. Blecken, A., Marx, W., (2010). Usability Evaluation of a Learning Management System, Proceedings of the 43rd Hawaii International Conference on System Science.
  7. Bonham, S.W., Titus, A., Beichner, R.J., Martin, L. Education Research using Web-based Assessment Systems, Journal of Research on Computing in Education, 2000.
  8. Cha, H.J., Ahn, M.L. (2009). Cyber Home Learning system diagnosis-prescription system usability evaluation and improvements on learning experience, HCI Conference 2009, 876-883.
  9. David, H. R.,, Meyer, A., Strangman, N., & Rappolt, G., Teaching every student in the digital age: Universal Design for Learning, The Association for supervision and curriculum design(ASCD), from http://www.ascd.org/publications/books/101042.aspx
  10. Ericsson, K. A., & Charness, N. (1994). Expert performance: Its structure and acquisition. American psychologist, 49(8), 725-747. https://doi.org/10.1037/0003-066X.49.8.725
  11. Fluck, A., Pullen, D. and Harper, C. (2009). Case study of a computer based examination system, Australasian Journal of Educational Technology, Vol. 25. No. 4. pp. 509-523
  12. Grant, J. S., & Davis, L. L. (1997). Selection and use of content experts for instrument development. Research in Nursing & Health, 20(3), 269-274. https://doi.org/10.1002/(SICI)1098-240X(199706)20:3<269::AID-NUR9>3.0.CO;2-G
  13. Jeon, S.J., Kim, H.S., (2016). A study on improvement of web-based diagnosis-supplement system for basic academic skills, Journal of The Korean Association of Information Education,, 20(5), 487-498. https://doi.org/10.14352/jkaie.20.4.487
  14. Jeong, Y.S., (2008). Digital textbook usability principle development, Journal of Computer Science Education, 11(30), 13-20.
  15. Korea Education & Research Information Service (2017). The educational use of IT-Convergent technology in 4th industrial revolution era. KERIS Research Report RR 2016-7.
  16. Korea Education & Research Information Service (KERIS). (2016). Whitepaper 2016,.
  17. Kwon, K.S., Park, J.S., Goo, C.D.(2014). Development of Agendas by using educational big data: Focused on analysis of SNS, KERIS, Research report KR 2014-10.
  18. Lim, C.L,, Song, H.D., Lee, Y.K,, Lee. Y.T. (2009) A study on the developing and applying digital textbook platform usability evaluation tools, Educational Technology Research, 25(4), 125-155.
  19. Mtebe, J. S., Kissaka, M. M. (2015), Heuristics for Evaluating Usability of Learning Management Systems in Africa, In IST-Africa Conference 2015, 1-13, IEEE
  20. Mtebe, J. S. (2015). Heuristics for Evaluating usability of learning management systems in Africa, IIMC International Information Management Corporation
  21. Nielsen, J. (1993). Usability Engineering, Academic Press Limited, London.
  22. Nielsen, J. (1995). 10 Usability Heuristics for User Interface Design, Nielsen Norman Group. https://www.nngroup.com/articles/ten-usability-heuristics/
  23. Nokelainen, P. (2006). An empirical assessment of pedagogical usability criteria for digital learning material with elementary school students. Educational Technology & Society, 9(2). 178-197.
  24. Park, C.H., Kim, G.T., Lee, B.H., Kang, T.H., Jeong, G.H., (2014). A study on analysis of online basic academic skills improvement system status and development of policy plans, KERIS, Research report KR 2014-6
  25. P21(Paternership for 21st century learning), Framework for 21st century learning, 2017, from http://www.p21.org/our-work/p21-framework
  26. Preece, J., Rogers, Y., Sharp, H., (2015). Interaction Design: beyound human-computer interaction, 4th ed. John Wiley & Sons Ltd. West Sussex:UK.
  27. Seo, Y.S. (2007). Development of usability evaluation tools for web-based learning site. PhD thesis.
  28. Silius, K. & Tervakari, A.-M. (2003). The Usefulness of Web-based Learning Environments. The Evaluation tool into the Portal of Finnish Virtual University. in International Conference on Network Universities and E-Learning. Proceedings of mENU. Valencia, Spain, V. Penarrocha.
  29. Sim, G., and Read, J. C., (2015). Using computer-assisted assessment heuristics for usability evaluations, British journal of educational technology, 47(4).
  30. Sim, G., Read, J.C., Cockton, G. (2009). Evidence based Design of Heuristics for Computer Assisted Assessment, IFIP Conference on Human-Computer Interaction, pp. 204-216.
  31. Song, H.D., & Park, H.J. (2009). Exploring factors that affect the usability of digital textbook based on affordance perspectives, Korean society for educational technology, 25(3), 135-155.
  32. Squires, D. and J. Preece, (1999). Predicting quality in educational software: Evaluating for learning, usability and the synergy between them. Interacting with Computers, Vol. 11. pp. 467-483. https://doi.org/10.1016/S0953-5438(98)00063-0
  33. W3C. Web Content Accessibility Guidelines, 2017 (WCAG) 2.0, from https://www.w3.org/TR/WCAG20/
  34. Zhenming, Y., Liang, Z., Guohua, Z. (2003). A Novel Web-based Online Examinations System for Computer Science Education, 33rd ASEE/IEEE Frontiers in Education Conference, November 5-8.

Cited by

  1. 웹기반 기초학력 진단-보정학습 시스템의 사용자 친화적인 개선 프로토타입 개발 vol.22, pp.1, 2019, https://doi.org/10.32431/kace.2019.22.1.006