DOI QR코드

DOI QR Code

Co-designing Human-Centered Learning Analytics: A Case Study

인간 중심 학습분석학을 위한 공동 설계 사례 연구

  • Received : 2023.04.19
  • Accepted : 2023.06.21
  • Published : 2023.06.30

Abstract

This study aims to specifically describe a co-design procedure for implementing human-centered learning analytics using the case study approach. Learning design and digital learning tools including dashboard for learning analytics were developed for teaching and learning settings in elementary and secondary schools. The learning engineering team that participated in the co-design process was comprised of six educational field experts, two educational administrators, five researchers, and three technicians. The co-design procedure included(1)Understanding Together, (2) Creating Together (Phase 1), (3) Applying Together, and (4) Creating Together(Phase 2). In the first phase, the needs and expectations of the learning engineering team members were identified and aligned through literature review and a co-design workshop. In the second phase, a co-design workshop was conducted to build learning scenarios that would serve as the foundation for learning design recommendations for learning analytics. In the third phase, the scenarios were implemented in a classroom context and the results of the implementation were examined. Lesson plans and learning tools for learning analytics were prototyped during the final phase. The implications of these findings for human-centered learning analytics practice were discussed.

본 연구는 인간 중심 학습분석학을 실현하기 위한 공동 설계 과정을 구체적으로 기술하는데 그 목적이 있다. 연구방법으로서 사례 연구 방법을 적용하였으며, 학습분석학을 초중등 교수·학습 상황에 적용하기 위한 학습설계안과 각 항목에 따른 대시보드 등의 프로토타입 도출 과정을 상세히 밝혔다. 공동 설계 과정에 참여한 학습 엔지니어링 팀은 6명의 교육현장 전문가, 2명의 교육행정가, 5명의 연구자, 3명의 기술자로 총 16명이다. 공동 설계 과정은 함께 이해하기, 함께 생산하기(1차), 함께 적용하기, 함께 생산하기(2차)의 과정을 거쳤다. 함께 이해하기 단계에서는 문헌 연구와 공동 설계 워크숍을 통해 학습 엔지니어링 팀 구성원의 요구와 기대를 확인하고 조율하였다. 함께 생산하기(1차) 단계에서는 공동 설계 워크숍을 통해 학습분석학을 위한 학습설계안의 기초가 될 학습 시나리오를 개발하였다. 함께 적용하기 단계에서는 학교 교육 현장에 적용하고 그 결과를 분석하였다. 마지막으로 함께 생산하기(2차) 단계에서는 학습분석학을 위한 학습설계안과 각 실행장면별 대시보드의 프로토타입을 개발하였다. 이를 바탕으로 인간 중심 학습분석학의 실천을 위한 시사점을 논의하였다.

Keywords

Acknowledgement

본 연구는 서울특별시교육청교육연구정보원의 지원을 받아 수행되었음

References

  1. Ahmad, A., Schneider, J., Griffiths, D., Biedermann, D., Schiffner, D., Greller, W., & Drachsler, H. (2022), Connecting the dots - A literature review on learning analytics indicators from a learning design perspective, Special Issues Article in Journal of Computer Assisted Learning. 
  2. Ahn, H. S., & Koo, Y. R. (2020), Developing Co-Design tool kit for Co-Creation with consumers for designing public policies and services from the view of service design, Journal of the Korean Society of Design Culture, 26(2), 293-308.  https://doi.org/10.18208/ksdc.2020.26.2.293
  3. Ahn, J., Campos, F., Hays, M., & DiGiacomo, D. (2019), Designing in context: reaching beyond usability in learning analytics dashboard design, Journal of Learning Analytics, v6.n2, 70-85. 
  4. Arnold, K. E., & Pistilli, M. D. (2012), Course signals at Purdue: Using learning analytics to increase student success, Proceedings of the 2nd International Conference on Learning Analytics and Knowledge, Canada, 267-270. 
  5. Bakharia, A., Corrin, L., de Barba, P., Kennedy, G., Gasevic, D., Mulder, R., Williams, D., Dawson, S., & Lockyer, L. (2016), A conceptual framework linking learning design with learning analytics, Proceedings of the 6th International Conference on Learning Analytics and Knowledge, United Kingdom, 329-338. 
  6. Bodily, R., Ikahihifo, T. K., Mackley, B., & Graham, C. R. (2018), The design, development, and implementation of student-facing learning analytics dashboards, Journal of Computing in Higher Education, 30(3), 572-598. 
  7. Buckingham Shum, S., Ferguson, R., & Martinez-Maldonado, R. (2019). Human-centred learning analytics, Journal of Learning Analytics, 6(2), 1-9. 
  8. Blumenstein, M. (2020). Synergies of learning analytics and learning design: A systematic review of student outcomes, Journal of Learning Analytics, 7(3), 13-32.  https://doi.org/10.18608/jla.2020.73.3
  9. Brown, T. (2010). Design thinking for social innovation, Development Outreach, 12(1), 29-43.  https://doi.org/10.1596/1020-797X_12_1_29
  10. Chatti, M. A., Muslim, A., Guesmi, M., Richtscheid, F., Nasimi, D., Shahin, A., & Damera, R. (2020). How to design effective learning analytics indicators? A human-centered design approach. In Addressing Global Challenges and Quality Education: 15th European Conference on Technology Enhanced Learning, EC-TEL2020, Heidelberg, Germany, September 14-18, 2020, Proceedings 15 (pp. 303-317). Springer International Publishing. 
  11. Crowe, S., Cresswell, K., Robertson, A., Huby, G., Avery, A., & Sheikh, A. (2011), The case study approach, BMC Medical Research Methodology, 11, 100(2011). 
  12. Dollinger, M., & Lodge, J. M. (2018), Co-Creation strategies for learning analytics, Proceedings of the 8th International Conference on Learning Analytics and Knowledge, 97-101. 
  13. Dollinger, M., Liu, D., Arthars, N., &Lodge, J. M. (2019), Working together in learning analytics towards the Co-Creation of value, Journal of Learning Analytics, 6(2), 10-26. 
  14. Few, S. (2013), Information dashboard design: Displaying data for at-a-glance monitoring(2nded.) Burlingame, CA: Analytics Press. 
  15. Guba, E., & Lincoln, Y.(1981). Effective evaluation. San Francisco CA: Jossey Bass. 
  16. Holstein, K., McLaren, B. M., & Aleven, V. (2019). Co-designing a real-time classroom orchestration tool to support teacher-AI complementarity. Journal of Learning Analytics, 6(2), 27-52. 
  17. Jo, I. H.(2015), Learning analytics and learning design and the development of the horizon of convergence, The Korean Society for Educational Technology Educational Technology International, 2015(2), 422-43. 
  18. Jo, I. H., Park, Y. J., & Kim. J. H.(2019). Understanding learning analytics. Korea: Pakyoungsa 
  19. Kim, J. H., Jo, I. H., & Park, Y. J.(2016). Effects of learning analytics dashboard: analyzing the relations among dashboard utilization, satisfaction, and learning achievement, Asia Pacific Education Review, 17, 13-24  https://doi.org/10.1007/s12564-015-9403-8
  20. Johnson, R. R.(1998) User-centered technology: A rhetorical theory for computers and other mundane artifacts. NewYork: State University of New York Press. 
  21. Shin, J. S. (2022, December 25), The era of public-private coopertaion in education, lay the ladder between public and private education with edutech, Sedaily. https://www.sedaily.com/NewsView/26F1ESYJ0A 
  22. Koper, R., & Tattersall, C.(2005). Learning design: A handbook on modelling and delivering networked education and training. Germany: Springer Science & Business Media. 
  23. Law, N., & Liang, L.(2020). A multilevel framework and method for learning analytics integrated learning design. Journal of Learning Analytics, 7(3), 98-117.  https://doi.org/10.18608/jla.2020.73.8
  24. Lim, L., Dawson, S., Joksimovic, S., & Gasevic, D. (2019). Exploring students' sensemaking of learning analytics dashboards: Does frame of reference make a difference?. In Proceedings of the 9th international conference on learning analytics & knowledge, 250-259. 
  25. Mohamed Amine Chatti, Arham Muslim, Mouadh Guesmi, Florian Richtscheid, Dawood Nasimi, Amin Shahinn, & Ritesh Damera, How to design effective learning analytics indicators? A human-centered design approach, European Conference on Technology Enhanced Learning, 303-317, 2020 
  26. Norman, D. A.(2013). The design of everyday things: revised and expanded edition. NewYork: Basic Book. 
  27. Okada, A., Shum, S. B., & Sherborne, T. (2014), Knowledge Cartography: Software Tools and Mapping Techniques: Springer London. 
  28. Pozdniakov, S., Martinez-Maldonado, R., Tsai, Y. S., Cukurova, M., Bartindale, T., Chen, P., Marshall, H., & Gasevic, D. (2022). The question-driven dashboard: how can we design analytics interfaces aligned to teachers' inquiry?. , LAK22: 12th International Learning Analytics and Knowledge Conference, 175-185. 
  29. Prieto-Alvarez, C. G., Martinez-Maldonado, R., &Anderson, T. D.(2018). Co-designing learning analytics tools with learners, Learning Analytics in the Classroom. UK: Routledge. 
  30. Rose, C. P., McLaughlin, E. A., Liu, R., & Koedinger, K. R.(2019). Explanatory learner models: Why machine learning (alone) is not the answer. British Journal of Educational Technology, 50(6), 2943-2958.  https://doi.org/10.1111/bjet.12858
  31. Seoul Metropolitan office of education(2015), Study on learning analytics model and expansion plan. Seoul Metropolitan office of education. 
  32. Shum, B. S., Ferguson, R., & Martinez-Maldonado, R. (2019). Human-centred learning analytics. Journal of Learning Analytics, 6(2), 1-9. 
  33. Shum, S. B.(2012). Learning analytics. UNESCO Institute for Information Technologies in Education, Moscow: Russian Federation 
  34. Siemens, G. (2012). Learning analytics: envisioning a research discipline and a domain of practive. In Proceedings of the 2nd international conferecne on learning analytics and knowledge, USA, 4-8
  35. Verbert, K., Duval, E., Klerkx, J., Govaerts, S., & Santos, J. L. (2013). Learning analytics dashboard applications, American Behavioral Scientist, 57(10), 1500-1509.  https://doi.org/10.1177/0002764213479363
  36. Weick, K. E., Sutcliffe, K. M., & Obstfeld, D. (2005). Organizing and the process of sensemaking. Organization science, 16(4), 409-421.  https://doi.org/10.1287/orsc.1050.0133
  37. Wise, Alyssa Friend & Jung, Y.(2019). "Teaching with analytics: towards a situated model of instructional decision-making." Journal of Learning Analytics 6.2, 53-69. 
  38. Choi, Y. S. & Bae, H. B. (2019). Analysis of single-case studies in Korean family therapy research-Focusing on Articles in 「the Journal of Korean Academy of Family Therapy」 and 「Family and Family Therapy」-, Family and Family Therapy, 27(1), 181-211.  https://doi.org/10.21479/KAFT.2019.27.1.181