Development and Validation of a Learning Progression for Astronomical Systems Using Ordered Multiple-Choice Items

순위 선다형 문항을 이용한 천문 시스템 학습 발달과정 개발 및 타당화 연구

  • Received : 2014.08.26
  • Accepted : 2014.12.26
  • Published : 2014.12.31


This study sought to investigate learning progressions for astronomical systems which synthesized the motion and structure of Earth, Earth-Moon system, solar system, and the universe. For this purpose we developed ordered multiple-choice items, applied them to elementary and middle school students, and provided validity evidence based on the consequence of assessment for interpretation of learning progressions. The study was conducted according to construct modeling approach. The results showed that the OMCs were appropriate for investigating learning progressions on astronomical systems, i.e., based on item fit analysis, students' responses to items were consistent with the measurement of Rasch model. Wright map analysis also represented that the assessment items were very effective in examining students' hypothetical pathways of development of understanding astronomical systems. At the lower anchor of the learning progression, while students perceived the change of location and direction of celestial bodies with only two-dimensional earth-based view, they failed to connect the locations of celestial bodies with Earth-Moon system model, and they could recognized simple patterns of planets in the solar system and milky way. At the intermediate levels, students interpreted celestial motion using the model of Earth rotation and revolution, Earth-Moon system, and solar system with space-based view, and they could also relate the elements of astronomical structures with the models. At the upper anchor, students showed the perspective change between space-based view and earth-based view, and applied it to celestial motion of astronomical systems, and they understood the correlation among sub-elements of astronomical systems and applied it to the system model.


learning progressions;astronomical systems;astronomical thinking;Rasch model;validity evidence


  1. Achieve, Inc. (2013). Next Generation Science Standards. Achieve Inc. On behalf of the twenty-six states and partners that collaborated on the NGSS.
  2. AERA, APA, & NCME. (1999). Standards for educational and psychological testing. Washington, DC: AERA.
  3. Alonzo, A. C., & Steedle J. T. (2009). Developing and assessing a force and motion learning progression. Science Education, 93(3), 389-421.
  4. Ben-Zvi Assaraf, O., & Orion, N. (2005). Development of system thinking skills in the context of Earth System education. Journal of Research in Science Teaching, 42, 518-560.
  5. Ben-Zvi Assaraf, O., & Orion, N. (2010). System thinking skills at the elementary school level. Journal of Research in Science Teaching, 47, 540-563.
  6. Berland, L.K., & McNeill, K.L. (2010). A learning progression for scientific argumentation: Understanding student work and designing supportive instructional contexts. Science Education, 94, 765-793.
  7. Black, P. & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education, 5, 7-74.
  8. Black, P., Wilson, M., & Yao, S. Y. (2011). Road maps for learning: A guide to the navigation of learning progressions. Measurement: Interdisciplinary Research & Perspective, 9, 71-123.
  9. Bond, T. G., & Fox, C. M. (2007). Applying the Rasch model: Fundamental measurement in the human sciences (2nd edition). New York, NY: Routledge.
  10. Boone, W. J., Staver, J. R., & Yale, M. S. (2014). Rasch analysis in the human sciences. New York, NY: Springer.
  11. Briggs, D. C., & Alonzo, A. C. (2012). The psychometric modeling of ordered multiple-choice item responses for diagnostic assessment with a learning progression. In A. C. Alonzo & A. W. Gotwals (Eds.), Learning progressions in science: Current challenges and future directions (pp. 293-316). Rotterdam, The Netherlands: Sense Publishers.
  12. Briggs, D. C., Alonzo, A. C., Schwab, C., & Wilson, M. (2006). Diagnostic assessment with ordered multiple-choice items. Educational Assessment, 11(1), 33-63.
  13. Corcoran, T., & Silander, M. (2009). Instruction in high schools: The evidence and the challenge. The Future of Children: America's High Schools, 19, 157-183.
  14. Corcoran, T., Mosher, F. A., & Rogat, A. (2009). Learning progressions in science: An evidence based approach to reform. Consortium for Policy Research in Education Report #RR-63. Philadelphia, PA: Consortium for Policy Research in Education.
  15. Duncan, R. G., Rogat, A. D., & Yarden, A. (2009). A learning progression for deepening students' understandings of modern genetics across the 5th-10th grades. Journal of Research in Science Teaching, 46, 655-674.
  16. Duschl, R. A., & Grandy, R. (Eds.) (2008). Teaching scientific inquiry: Recommendations for research and implementation. Rotterdam, The Netherlands: Sense Purblishers.
  17. Duschl, R., Maeng, S., & Sezen, A. (2011). Learning progressions and teaching sequences: A review and analysis. Studies in Science Education, 47, 123-182.
  18. Furtak, E. M. (2012). Linking a learning progression for natural selection to teachers' enactment of formative assessment. Journal of Research in Science Teaching, 49(9), 1181-1210.
  19. Gotwals, A. W., & Songer, N. B. (2013). Validity evidence for learning progression-based assessment items that fuse core disciplinary ideas and science practices. Journal of Research in Science Teaching, 50, 597-626.
  20. Ji, E., & Chae, S. (2000). Theory and practices of Rasch model. Seoul: Kyoyook-Kwahak-Sa.
  21. Jin, H., & Anderson, C. W. (2012). A learning progression for energy in socio-ecological systems. Journal of Research in Science Teaching, 49, 1149-1180.
  22. Maeng, S., Seong, Y., & Jang, S. (2013). Present states, methodological features, and an exemplar study of the research on learning progressions. Journal of the Korean Association for Science Education, 33, 161-180.
  23. Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47(2), 149-174.
  24. National Research Council (NRC). (2006). Learning to think spatially. Washington, DC: National Academies Press.
  25. National Research Council. (2007). Taking science to school: Learning and teaching science in grades K-8. Washington, DC: National Academies Press
  26. National Research Council. (2012). A framework for k-12 science education: Practices, crosscutting concepts, and core ideas. Washington, DC: National Academies Press.
  27. Neumann, K., Viering, T., Boone, W. J., & Fischer, H. E. (2013). Towards a learning progression of energy. Journal of Research in Science Teaching, 50, 162-188.
  28. Orion, N., & Basis, T. (2008). Characterization of High School Students' System Thinking Skills in the Context of Earth Systems. Presented in the 2008 NARST Annual Meeting. March, 2008. Baltimore, U.S.A.
  29. Plummer, J. D. (2014). Spatial thinking as the dimension of progress in an astronomy learning progression. Studies in Science Education, 50(1), 1-45.
  30. Plummer, J. D., & Krajcik, J. (2010). Building a learning progression for celestial motion: Elementary levels from an Earth-based perspective. Journal of Research in Science Teaching, 47(7), 768-787.
  31. Schwarz, C. V., Reiser, B. J., Davis, E. A., Kenyon, L., Acher, A., Fortus, D., Shwartz, Y., Hug, B., & Krajcik, J. (2009). Developing a learning progression for scientific modeling: Making scientific modeling accessible and meaningful for learners. Journal of Research in Science Teaching, 46, 632-654.
  32. Seong, T. (2002). Validity and Reliability. Seoul: Hakjisa.
  33. Seong, T., & Si, K. (2006). Reasearch Methodology. Seoul: Hakjisa.
  34. Shea, N. A., & Duncan, R. G. (2013). From theory to data: The process of refining learning progressions. Journal of the Learning Sciences, 22, 7-32.
  35. Songer, N. B., & Gotwals, A. W. (2012). Guiding explanation construction by children at the entry points of learning progressions. Journal of Research in Science Teaching, 49, 141-165.
  36. Stevens, S., Delgado, C., & Krajcik, J. S. (2010). Developing a hypothetical multi-dimensional learning progression for the nature of matter. Journal of Research in Science Teaching, 47, 687-715.
  37. Wilson, M. (2005). Constructing measures: An item response modeling approach. Mahwah, NJ: Lawrence Erlbaum Associates.
  38. Wilson, M. (2009). Measuring progressions: Assessment structures underlying a learning progression. Journal of Research in Science Teaching, 46, 716-730.
  39. Wilson, M. (2012). Responding to a challenge that learning progressions pose to measurement practice. In A. C. Alonzo & A. W. Gotwals (Eds.), Learning progressions in science: Current challenges and future directions (pp. 317-343). Rotterdam, The Netherlands: Sense Publishers.
  40. Wright, B. D., & Linacre, J. M. (1994). Reasonable mean-square fit value. Rasch Measurement Transactions, 8(3), 370.

Cited by

  1. Cross-Sectional Item Response Analysis of Geocognition Assessment for the Development of Plate Tectonics Learning Progressions: Rasch Model vol.35, pp.1, 2015,
  2. The Effects of an Instruction Using Geologic Planar Figures on High School Students’ Ability of Spatial Visualization and Geologic Spatial Ability vol.36, pp.3, 2015,
  3. An Analysis of Systems Thinking Revealed in Middle School Astronomy Classes: The Case of Science Teachers’ Teaching Practices for the Unit of Stars and Universe vol.36, pp.6, 2015,
  4. Effects of Systems Thinking on High School Students’ Science Self-Efficacy vol.37, pp.3, 2016,
  5. A Proposal of Curriculum and Teaching Sequence for Seasonal Change by Exploring a Learning Progression vol.39, pp.3, 2018,
  6. Exploring 6th Graders Learning Progression for Lunar Phase Change: Focusing on Astronomical Systems Thinking vol.39, pp.1, 2018,


Supported by : 한국연구재단