International Journal of Computer Science & Network Security

International Journal of Computer Science & Network Security (국제컴퓨터통신보호논문지학회)
권호 표지
DOI QR코드

DOI QR Code

UML-ITS Usability Evaluation of Intelligent Tutoring System

  • Received : 2023.03.05
  • Published : 2023.03.30
Download PDF
⟨ Previous Next ⟩

Abstract

The most effective tutoring method is one-on-one, face-to-face in-person human tutoring. However, due to the limited availability of human tutors, computer-based alternatives have been developed. These software based alternatives are called Intelligent Tutoring Systems (ITS) which are used to tutor students in different domains. Although ITS performance is inferior to that of human teachers, the field is growing and has recently become very popular. User interfaces play key role in usability perspective of ITS. Even though ITS research has advanced, the majority of the work has concentrated on learning sciences while mostly disregarding user interfaces. Because of this, the present ITS includes effective learning modules but a less effective interface design. Usability is one approach to gauge a software's performance, while "ease of use" is one way to assess a software's quality. This paper measures the usability effectiveness of an ITS which is designed to teach Object-Oriented (OO) analysis and design concepts using Unified Modeling Language (UML). Computer Supported Usability Questionnaire (CSUQ) survey was conducted for usability evaluation of UML-ITS. According to participants' responses to the system's usability survey, all responses lie between 1 to 3 scale points which indicate that the participants were satisfied and comfortable with most of the system's interface features.

Keywords

References

  1. Mousavinasab, E., Zarifsanaiey, N., R. Niakan Kalhori, S., Rakhshan, M., Keikha, L., & Ghazi Saeedi, M. (2021). Intelligent tutoring systems: a systematic review of characteristics, applications, and evaluation methods. Interactive Learning Environments, 29(1), 142-163 (2021) https://doi.org/10.1080/10494820.2018.1558257
  2. Mark, M. A., & Greer, J. E. (1993). Evaluation methodologies for intelligent tutoring systems. Journal of Artificial Intelligence in Education, 4, 129-129 (1993).
  3. Paviotti, G., Rossi, P. G., and Zarka, D. Intelligent tutoring systems: an overview. Pensa Multimedia, 1-176 (2012).
  4. Miller, J. R. The role of human-computer interaction in Intelligent Tutoring Systems. Foundations of intelligent tutoring systems, 143-189 (1988).
  5. Walker, E., Rummel, N., Walker, S. and Koedinger, K. R., Noticing Relevant Feedback Improves Learning in An Intelligent Tutoring System for Peer Tutoring, Proceedings of 11th International Conference on Intelligent Tutoring Systems. Springer, Berlin, Heidelberg, pp. 222 - 232 (2012).
  6. Duangnamol T., Suntisrivarporn B., Supnithi T. and Ikeda M., Circuitously Collaborative Learning Environment to Enhance Metacognition, Proceedings of International Conference on Computers in Education, Japan: Asia-Pacific Society for Computers in Education, pp.1-4 (2014).
  7. Arroyo, I., Woolf, B. P., and Shanabrook, D. Casual collaborations while learning mathematics with an intelligent tutoring system. In Collaborative Learning Environments Workshop at ITS (2012).
  8. Haq, I., Anwar, A., Basharat, I. and Sultan, K., Intelligent Tutoring Supported Collaborative Learning (ITSCL): A Hybrid Framework, International Journal of Advanced Computer Science and Applications, 11(8), pp. 523-535 (2020) https://doi.org/10.14569/IJACSA.2020.0110866
  9. Chughtai, R., Zhang, S., & Craig, S. D. Usability evaluation of intelligent tutoring system: ITS from a usability perspective. In Proceedings of the human factors and ergonomics society annual meeting Vol. 59, No. 1, pp. 367-371, (2015).
  10. Lin, H. C. K., Wu, C. H., & Hsueh, Y. P. The influence of using affective tutoring system in accounting remedial instruction on learning performance and usability. Computers in Human Behavior, 514-522 (2014).
  11. Koscianski, A., & Zanotto, D. D. C. F. A Design Model for Educational Multimedia Software. Creative Education, 5(23), 2003, (2014).
  12. ERICSSON, K. A. and SIMON, H. A, Protocol Analysis: Verbal Reports as Data, Cambridge, MA: The MIT Press (1984).
  13. Matarasso, A. K., Rieke, J. D., White, K., Yusufali, M. M., and Daly, J. J. Combined real-time fMRI and real time fNIRS brain computer interface (BCI). Training of volitional wrist extension after stroke, a case series pilot study. PloS one, 16(5), e0250431 (2021).
  14. Siyam, N., & Abdallah, S. A Pilot Study Investigating the Use of Mobile Technology for Coordinating Educational Plans in Inclusive Settings. Journal of Special Education Technology, 01626434211033581 (2021).
  15. Lewis, J., Measuring Perceived Usability: The CSUQ, SUS, and UMUX, International Journal of Human-Computer Interaction, 34(12), pp.1148-1156 (2018). https://doi.org/10.1080/10447318.2017.1418805
  16. Lewis, J. R. Measuring perceived usability: SUS, UMUX, and CSUQ ratings for four everyday products. International Journal of Human-Computer Interaction, 35(15), 1404-1419 (2019). https://doi.org/10.1080/10447318.2018.1533152
  17. Abrejo, S., Kazi, H., Rahman, M. U., Baloch, A., & Baig, A. Learning from Peer Mistakes: Collaborative UML-Based ITS with Peer Feedback Evaluation. Computers, 11(3), 30 (2022).
  18. Cortina, J. M, What is Coefficient Alpha? An Examination of Theory and Applications, Journal of Applied psychology, 78(1), pp. 98 (1993).
  19. Schmitt, N., Uses And Abuses of Coefficient Alpha, Psychological Assessment, 8(4), pp. 350-360 (1996). https://doi.org/10.1037/1040-3590.8.4.350
  20. Lewis, J. R., Utesch, B. S. and Maher, D. E., UMUX-LITE-When there's no time for the SUS, In Proceedings of CHI 2013, Paris, France: Association for Computing Machinery, pp. 2099-2102 (2013).
  21. Sauro, J. and Lewis, J. R., When designing usability questionnaires, does it hurt to be positive? Proceedings of CHI 2011 Vancouver, Canada: ACM, pp. 2215-2223 (2011).
  22. Sauro, J. and Lewis, J. R., Quantifying The User Experience: Practical Statistics for User Research (2nd ed.). Cambridge, MA: Morgan Kaufmann. (2016).
  23. Lewis, J. R. and Sauro, J., Revisiting the Factor Structure of the System Usability Scale, Journal of Usability Studies, 12(4), 183 (2017).