Journal of The Korean Association For Science Education (한국과학교육학회지)

The Korean Association for Science Education (한국과학교육학회)
권호 표지
DOI QR코드

DOI QR Code

Analysis of the Abstract Structure in Scientific Papers by Gifted Students and Exploring the Possibilities of Artificial Intelligence Applied to the Educational Setting

과학 영재의 논문 초록 구조 분석 및 이에 대한 인공지능의 활용 가능성 탐색

  • Received : 2023.12.04
  • Accepted : 2023.12.18
  • Published : 2023.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

This study aimed to explore the potential use of artificial intelligence in science education for gifted students by analyzing the structure of abstracts written by students at a gifted science academy and comparing the performance of various elements extracted using AI. The study involved an analysis of 263 graduation theses from S Science High School over five years (2017-2021), focusing on the frequency and types of background, objectives, methods, results, and discussions included in their abstracts. This was followed by an evaluation of their accuracy using AI classification methods with fine-tuning and prompts. The results revealed that the frequency of elements in the abstracts written by gifted students followed the order of objectives, methods, results, background, and discussions. However, only 57.4% of the abstracts contained all the essential elements, such as objectives, methods, and results. Among these elements, fine-tuned AI classification showed the highest accuracy, with background, objectives, and results demonstrating relatively high performance, while methods and discussions were often inaccurately classified. These findings suggest the need for a more effective use of AI, through providing a better distribution of elements or appropriate datasets for training. Educational implications of these findings were also discussed.

본 연구는 영재학교 학생들의 논문 초록의 구조를 파악하여 그 특성을 분석하고, 인공지능을 활용하여 초록을 구성하는 여러 요소를 추출하여 그 성능을 비교함으로써 과학영재교육에서 인공지능의 활용 가능성을 모색하는 것을 목적으로 하였다. 이에 따라 S 영재학교의 2017~2021년의 5년간 졸업 논문 263건을 대상으로 초록에 포함된 배경, 목적, 방법, 결과, 논의의 빈도나 유형이 어떠한지 분석하고 이를 파인튜닝 및 프롬프트를 활용한 인공지능을 활용한 분류 방법을 통해 그 정확도를 평가하였다. 연구 결과, 영재 학생들이 작성한 과학 논문의 초록 요소의 출현 빈도는 목적, 방법, 결과, 배경, 논의(D)의 순이었고, 목적, 방법, 결과 등 초록에서 필수적으로 포함되어야 하는 요소를 모두 담은 경우는 전체 57.4%에 불과하였다. 인공지능을 활용한 이러한 요소를 분류한 결과, 파인튜닝을 이용한 경우가 가장 정확도가 높았으며 5가지 요소 중 배경, 목적, 결과는 비교적 높은 성능을 보였으나 방법, 논의에 대해서는 정확히 분류하지 못하는 경우가 많 았다. 이러한 결과는 여러 요소의 분포 비율이나 학습을 위한 적절한 데이터셋이나 정보를 제공해 인공지능을 활용해야 보다 효과적인 수단으로 활용될 수 있음을 의미하며, 이에 대한 교육적 시사점을 제시하였다.

Keywords

References

  1. Bazerman, C. (1992). From cultural criticism to disciplinary participation: Living with powerful words. Writing, teaching, and learning in the disciplines, 61-68. 
  2. Bingyu, Z., & Arefyev, N. (2022). The document vectors using cosine similarity revisited. Retrieved Nov 29, 2023 from https://arxiv.org/abs/2205.13357 
  3. Can, S., Karabacak, E., & Qin, J. (2016). Structure of Moves in Research Article Abstracts in Applied Linguistics. Publications, 4(3), 23. MDPI AG. Retrieved from http://dx.doi.org/10.3390/publications4030023. 
  4. Chang, W. (2016). A Study on the Structure and Content of Abstracts: Focused on Journal of the Korean Society for Information Management. Journal of the Korean Society for Information Management, 33(3), 107-131.  https://doi.org/10.3743/KOSIM.2016.33.3.107
  5. Chung, H. (2019). A Teaching Plan of Thesis Abstracts Writing for Engineering Students. Journal of Education & Culture, 25(3), 411-427.  https://doi.org/10.24159/joec.2019.25.3.411
  6. Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018). BERT: Pre-training of deep bidirectional transformers for language understanding. Retrieved Nov 26, 2023 from https://arxiv.org/abs/1810.04805 
  7. Geloebiowski, Z. (2009). Prominent Messages in Education and Applied Linguistics Abstracts: How Do Authors Appeal to their Prospective Readers? Journal of Pragmatics, 41, 753-769.  https://doi.org/10.1016/j.pragma.2008.10.009
  8. Jescovitch, L. N., Scott, E. E., Cerchiara, J. A., Merrill, J., Urban-Lurain, M., Doherty, J. H., & Haudek, K. C. (2020). Comparison of machine learning performance using analytic and holistic coding approaches across constructed response assessments aligned to a science learning progression. Journal of Science Education and Technology, 30, 150-167. 
  9. Jho, H. (2021). The Directions and Challenges of Science Education Based on the Prediction of Future Education and Schools. Journal of Research in Curriculum Instruction, 25(1), 61-78. 
  10. Jho, H. (2022). Understanding of generative artificial intelligence based on textual data and discussion for its application in science education. Journal of the Korean Association for Science Education, 43(3), 307-319.  https://doi.org/10.14697/JKASE.2023.43.3.307
  11. Jho, H., & Lee, B. (2022). Clustering Science Gifted Students' Graduation Theses Based on Machine Learning. Journal of Science Education for the Gifted, 14(1), 13-22. 
  12. Jo, K. (2020). Analysis of Research Paper Introductions Written by Science-gifted Students. Journal of Science Education for the Gifted, 12(2), 127-136.  https://doi.org/10.29306/jseg.2020.12.2.127
  13. Jo, K. (2021). Analysis of Conclusion Section of Scientific Research Papers Written by Gifted Students. Journal of Science Education for the Gifted, 13(3), 172-185.  https://doi.org/10.29306/jseg.2021.13.3.172
  14. Kaufman, J. C., Gentile, C. A., & Baer, J. (2005). Do gifted student writers and creative writing experts rate creativity the same way? Gifted Child Quarterly, 49(3), 260-265.  https://doi.org/10.1177/001698620504900307
  15. Keys, C. W. (1999). Revitalizing instruction in scientific genres: Connecting knowledge production with writing to learn in science. Science Education, 83(2), 115-130.  https://doi.org/10.1002/(SICI)1098-237X(199903)83:2<115::AID-SCE2>3.0.CO;2-Q
  16. Ki m, D., & Park, J. (2015). Development of A Checkli st for Helpi ng Students' Open Scientific Inquiry Report Writing. Journal of the Korean Association for Science Education, 35(6), 1075-1083.  https://doi.org/10.14697/jkase.2015.35.6.1075
  17. Kim, J. (2019). An Analysis on Structure of Moves in Abstracts for Korean Education. The Korean Journal of Literacy Research, 10(2), 377-400.  https://doi.org/10.37736/kjlr.2019.04.10.2.377
  18. Lee, H., & Shim, K. (2010). Study on Writing a Scientific Paper of University Students. Biology Education, 38(4), 599-610.  https://doi.org/10.15717/BIOEDU.2010.38.4.599
  19. Lin, Chin-Yew. (2004). ROUGE: A package for automatic evaluation of summaries. Paper presented. In Text summarization branches out. Paper presented at the 42nd Annual Meeting of the Association for Computational Linguistics, Barcelona, Spain, Jul. 21-26. 
  20. Mitrovic, S., & Henning, M. (2015). Summarizing citation contexts of scientific publications. Paper presented at the International Conference of the Cross-Language Evaluation Forum for European Languages, Toulouse, France, Sepp. 8-11. 
  21. Moon, H., & So, J. (2023). Comparing the Characteristics of Science Papers Written by Science Gifted Students and General High School Students. Journal of Science Education for the Gifted, 15(2), 269-278. 
  22. Mun, J., Kim, M., Kim, W., Mun, K., & Kim, S. (2020). Analysis on the titles of science research reports of high school students: Focusing on the errors in writing titles. The Journal of Learner-Centered Curriculum and Instruction, 20(8), 453-471. 
  23. Shim, K., Son, J., Lee, B., Lee, Y., & Han, H. (2021). A Study on the Mid to Long Term Development Plan for Gifted Science Academy and Science High School. CR 2021-24, Jincheon: Korean Educational Development Institute. 
  24. Son, J. (2009). The study of scientifically gifted students scientific thinking and creative problem solving ability through science writing. Journal of Science Education for the Gifted, 1(3), 21-32. 
  25. Song, J., Kang, S., Kwak, Y., Kim, D., Kim, S., Na, J., ... Joung, Y. (2019). Contents and features of 'Korean Science Education Standards (KSES)' for the next generation. Journal of the Korean Association for Science Education, 39(3), 465-478. 
  26. Weil, B. H. (1970). Standards for writing abstracts. Journal of the American Society for Information Science, 21(5), 351-357.  https://doi.org/10.1002/asi.4630210507
  27. Wilson, C. D., Haudek, K. C., Osborne, J. F., Buck, B., Z. E., Cheuk, T., Donovan, B. M., Stuhlsatz, M. A. M., Santiago, M. M., & Zhai, X. (2023). Using automated analysis to assess middle school students' competence with scientific argumentation. Journal of Research in Science Teaching, 1-32. https://dx.doi.org/10.1002/tea.21864 
  28. Zhai, X., Haudek, K. C., & Ma, W. (2023). Assessing argumentation using machine learning and cognitive diagnostic modeling. Research in Science Education, 53, 405-424.