Journal of Fisheries and Marine Sciences Education (수산해양교육연구)

The Korean Society for Fisheries and Marine Sciences Education (한국수산해양교육학회)
권호 표지
DOI QR코드

DOI QR Code

Detecting Differential Item Functioning based on Gender: Field of Mathematics in the TIMSS 2007

초등학생의 성별에 따른 차별기능문항 분석: 수학 과학 성취도 국제비교연구(TIMSS) 2007 수학영역을 중심으로

  • Received : 2017.03.09
  • Accepted : 2017.04.17
  • Published : 2017.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

This study investigated not only the existence of differently functioned item due to gender but also domain. In this study, the randomly selected data of TIMSS 2007, which consist of 681 male and 646 women, were analyzed. To detect differently functioned items, this study employed Raju method. For Raju method, three-parameter logistic model was selected. Signed and unsigned area between two item characteristic curve were measured within the real ability range. An item which was detected commonly SA and UA area in Raju method was defined as a differently functioned item. As a result of this study, six items among twenty seven items of mathematics in the TIMSS 2007 were differently functioned item. Five items among those six items, were in favor of boys and one item was in favor of girls. Number, Geometric Shapes and Measures, and Applying were in favor of boys. but Data Display, Reasoning were in favor of girls. The conclusion of this study was summarized as existing differently functioned items in TIMSS 2007 and difference between favorable domain based gender. Finally, it is desirable to consider the differently functioned items by relating those item content for improving the test reliability of TIMSS 2007.

Keywords

References

  1. Ahn, Chang-kyu & Cha, Kyung-Ok(1984). A Comparison of Four Statistical Procedures for Detecting Test Item-Bias. Social Survey Research, 3, 29-49.
  2. Angoff, W. H.(1982) Use of item difficulty and discriminant indices for detecting item bias. ed. by Berk, R. A. In Handbook of method for detecting test bias, 96-116. Baltimore, London: The Johns Hopkins University Press,
  3. Angoff, W. H. & Ford, S. F.(1973). Item-race interaction on a test of scholastic aptitude. Journal of Educational Measurement, 10(2), 95-106 https://doi.org/10.1111/j.1745-3984.1973.tb00787.x
  4. Baker, F. B., & Al-Karni, A.(1991). EQUATE: Computer program for equating two metrics in item response theory [Computer program]. Madison WI: University of Wisconsin, Laboratory of Experimental Design.
  5. Camilli, G.(1979). A critique of the chi-square method assessing item bias. Unpublished paper, Laboratory of Education Research, University of Colorado at Boulder.
  6. Chin, Sujung & Seong, Tae-je(2004). An Exploratory Study on Item-format Related DIF with MH and SIBTEST techniques. Journal of Educational Evaluation, 17(2), 215-236.
  7. Choo, Jeong-A & Seong, Tae-je(1993). Detecting gender differently functioned items of the 4th and 5th tryouts of College Scholastic Ability Test by Mantel-Haeszel and Raju Methods. Journal of Educational Evaluation, 6(2), 259-286.
  8. Choo, Jeong-A.(1993). Detecting gender differently functioned items of the tryouts of College Scholastic Ability Test. Master's Thesis, Ehwa Womans University, Seoul.
  9. Cleary, T. A. & Hilton, T. L.(1968). An Investigation of Item Bias. Educational and Psychological Measurement, 28, 61-75. https://doi.org/10.1177/001316446802800106
  10. Cook, L. L..Dorans, N. J. & Eignor, D. R.(1988). An assessment of the dimensionality of three SAT-Verbal test editions. Journal of Educational Statistics, 13, 19-43. https://doi.org/10.3102/10769986013001019
  11. Doolittle, A. E.(1984). Interpretation of differential item performance accompanied by gender differences in academic background. Paper presented at the annual meeting of the American Educational Research Association, New Orleans.
  12. Doolittle, A. E. & Cleary, T. A.(1987). Gender-based differential item performance in mathematics achievement items. Journal of Educational Measurement, 24(2), 157-166 https://doi.org/10.1111/j.1745-3984.1987.tb00271.x
  13. Educational Research Institute of Seoul National University(1998). The encyclopedia of education. Seoul: Hawoo.
  14. Faggen-Steckler, J..K. A. McCarthy, & C. K. Tittle(1974). A quantitative method for measuring sex bias in standardized test. Journal of Educational Measurement, 11, 151-161. https://doi.org/10.1111/j.1745-3984.1974.tb00987.x
  15. Greaud, V. A.(1987). Investigation of the unidimensionality assumption of item response theory. Master's Thesis, The Johns Hopkins University, Baltimore, Maryland.
  16. Haley, D. C.(1952). Estimation of the dosage ,ortality relationship when the dose is subject to error(Technical Report No. 15). Stanford CA: Stanford University, Applied Mathematics and Statistics Laboratory.
  17. Hambleton, R. K. & Swaminathan, H.(1985). Item responsw theory : Principles and application. Boston: Kluwer Boston
  18. Hambleton, R. K.(1989). Principles and selected applications of item response theory. In R. L. Linn (ed.), Educational measurement(3rd ed., pp. 147-200). New York: MacMillan
  19. Hambleton, R. K..Swaminathan, H. & Rogers, H. J.(1991). Fundmentals of item response theory. CA: SAGE Publications.
  20. Hambleton, R. K..Swaminathan, H., & Rogers, J. H.(1991). Fundamentals of item response theory. Newbury Park, CA: Sage.
  21. Hanson, B. A.(1988). Uniform DIF and DIF defined by differences in item response function. Journal of Educational and Behavioral Statistics, 23(3), 244-253. https://doi.org/10.3102/10769986023003244
  22. Harnish, D. L.(1994). Performance assessment in review : New direction for assessment student understanding. International Journal of Educational Research, 21(3), 341-350 https://doi.org/10.1016/S0883-0355(06)80023-6
  23. Hatti, J. A.(1984). An empirical study of various indices for determining unidimensionality. Multivariate Behavioral Research, 19, 49-78. https://doi.org/10.1207/s15327906mbr1901_3
  24. Holland, P. W. & Thayer, D. T.(1988) Differential item performance and the Mentel-Haenszel procedure. In H. Wainer & H, I, Braun(Eds.). Test validity(pp. 129-145). Hillsdale, NJ: Lawrence Erlbaum.
  25. Holweger, N. & Weston, T.(1998). Differential item functioning: an applied comparison of the item characteristic curve method with the logistic discriminant function method. (ERIC Document Reproduction Service No. ED 422362)
  26. Jensen, A. R.(1969). How much can we boost IQ and Scholistic achievement?. Harvard Educational Review, 39, 81-83.
  27. Kim, S. H. & Cohen, A. S.(1991a). IRTDIF: A comparision of two area measures for detecting differential item function. Applied Psychological Measurement, 15, 269-278. https://doi.org/10.1177/014662169101500307
  28. Kim, S. H. & Cohen, A. S.(1991b). IRTDIF: A computer program for IRT differential item functioning[Computer program]. Madison WI: University of Wisconsin.
  29. Kim, Shin-young(1993). An Empirical Study for Validity of Mantel-Haenszel Method. Journal of Educational Evaluation, 6(1), 59-90.
  30. Kim, Shin-young(2001). Methods of Analyzing Differently functioned items. Seoul: kyoyookkwahaksa.
  31. Kim, Sukwoo.(1991). Gender and OTL effects on mathematics achievement for U.S. SIMS 12th grade students. Journal of Educational Evaluation, 4, 32-58.
  32. Kim, Sukwoo. & Park, Haejin.(1994). A Study of Differential Item Functioning based on Gender and Major. Journal of Pedagogy in Pusan, 7, 45-64.
  33. Ko, Jung Hwa.Do, Jong Hoon, & Song, Miyoung.(2008). An Analysis of the Gender Difference in National Assessment of Educational Achievement of Mathematics. The Journal of Educational Research in Mathematics, 18(2), 179-200.
  34. Lee, Dae Sik & Kim, Su Mi.(2003). Elementary School Students' and Teachers' Responses on Sex Differences in Mathematics Learning. The Journal of Elementary Education, 16(1), 297-315.
  35. Lee, Myung-Ae(2010). DIF Identification via Hierarchical Nonlinear Model. Journal of Educational Evaluation, 23(1), 171-190.
  36. Lee. Young(2002). Detecting Differential Item Functioning based on Gender and Major : Verbal Area in the tryout of College Scholastic Ability Test. Master's Thesis, Pusan National University, Busan.
  37. Lehman, J. D.(1986). Opportunity to learn and differential item functioning. Unpublished doctoral dissertation, University of Valifornia, Los Angeles.
  38. Linn, R. L. & Harnish, D. L.(1981) Interactions between item content and group membership on achievement test items. Journal of Educational Measurement, 18, 109-118. https://doi.org/10.1111/j.1745-3984.1981.tb00846.x
  39. Lord, F. M.(1980). Applications of item response theory to practical testing problems. Hilldale, NJ: Lawrence Erlbaum.
  40. Lord, F. M. & Novick, M. R.(1968). Statistic theories of mental test scores. Reading, MA: Addison-Wesley.
  41. Mellenbergh, G. J.(1982). Contingency table models for assessing item bias. Journal of Educational Statistics, 7, 105-108. https://doi.org/10.3102/10769986007002105
  42. Mislevy, R. J. & Bock, R. D.(1990). BILOG 3 for windows: Item analysis and test scoring with binary logistic models[Computer program]. Mooresville IN: Scientific Software Inc.
  43. Muthen, B..Kao, C.-F. & Burstein, L.(1988). Instructional sensitivity in mathematics achievement test item: Application of a new IRT-based detection technique. Forthcoming in Journal of Educational Measurement.
  44. Muthen, B. O. & Muthen, L. K.(2001). Mplus: Statistical analysis with latent variables[Computer program and manual]. Los Angeles: Statmodel.
  45. Noh, Un-Kyung(2007). Detecting Gender-related Differential Item Functioning on the Spatial Ability Test in the Aptitude Battery for Middle School Student. Master's Thesis, Ehwa Womans University, Seoul.
  46. Peterson, N. S..Kolen, M. S. & Hoover, H. D.(1989). Scaling, norming and equating. In Educational Measurement(3rd). Ed by Linn, R. L.. New York: Macmillian Publishing Company.
  47. Potenza, M. T. & Dorans, N. J.(1995). DIF Assessment for Polytomously Scored Item: A Framework for Classification and Evaluation. Applied Psychological Measurement, 19(1), 23-37. https://doi.org/10.1177/014662169501900104
  48. Raju, N. S.(1988). The area between item characteristic curves. Psychometrika, 53, 495-502. https://doi.org/10.1007/BF02294403
  49. Roussos, L. A. & Stout, W. F.(1996). Simulation studies of the effects of small sample size and studied item parameters on SIBTEST and Mantel-Haenszel type I error performance. Journal of Educational Measurement, 32(2), 215-230.
  50. Runder, L. M.(1977). An evaluation of select approaches for biased item identification. Unpublished doctoral dissertation. Catholic University of America, 18-27.
  51. Scheuneman, J. D.(1975). A new method of assessing bias in test items. Paper presented at annual meeting of American Educatioal Research Association. Washington D. C..
  52. Seong, Tae-je(1993). A Comparative Study between Raju and Mantel-Haeszel Methods for Detecting Differential Item Function. Journal of Educational Evaluation, 6(1), 91-120.
  53. Shealy, R. T. & Stout, W. F.(1993). An Item Response Model for Test Bias and Differential Test Functioning. Hillsdale, NJ: Erlbaum.
  54. Song, Miyoung(2001). Detection of gender-related DIF and comparison of DIF procedures in a performance assessment. doctoral Thesis, Ehwa Womans University, Seoul.
  55. Stocking, M. L. & Lord, F. M.(1983). Developing a common metric in item response theory. Applied Psychological Measurement, 7, 201-210. https://doi.org/10.1177/014662168300700208
  56. Stout, W.(1987). A nonparametric approach for assessing latent trait unidimensionality. Psychometrika, 52, 589-617. https://doi.org/10.1007/BF02294821
  57. Swaminathan, H. & Rogers, H. J.(1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361-370. https://doi.org/10.1111/j.1745-3984.1990.tb00754.x
  58. Williams, R. L.(1971). Abuse and misuse of testing black children. The Counseling Psychologists, 2, 62-73 https://doi.org/10.1177/001100007100200314