Safety and Health at Work

Occupational Safety and Health Research Institute, Korea Occupational Safety and Health Agency (산업안전보건연구원)
권호 표지
DOI QR코드

DOI QR Code

Reliability and Validity of a Nationwide Survey (the Korean Radiation Workers Study)

  • Lee, Dalnim (Laboratory of Radiation Epidemiology, National Radiation Emergency Medical Center, Korea Institute of Radiological and Medical Sciences) ;
  • Lim, Wan Young (Laboratory of Radiation Epidemiology, National Radiation Emergency Medical Center, Korea Institute of Radiological and Medical Sciences) ;
  • Park, Soojin (Laboratory of Radiation Epidemiology, National Radiation Emergency Medical Center, Korea Institute of Radiological and Medical Sciences) ;
  • Jin, Young Woo (Laboratory of Radiation Epidemiology, National Radiation Emergency Medical Center, Korea Institute of Radiological and Medical Sciences) ;
  • Lee, Won Jin (Department of Preventive Medicine, Korea University College of Medicine) ;
  • Park, Sunhoo (Laboratory of Radiation Epidemiology, National Radiation Emergency Medical Center, Korea Institute of Radiological and Medical Sciences) ;
  • Seo, Songwon (Laboratory of Radiation Epidemiology, National Radiation Emergency Medical Center, Korea Institute of Radiological and Medical Sciences)
  • Received : 2021.01.06
  • Accepted : 2021.07.21
  • Published : 2021.12.30
Download PDF
⟨ Previous Next ⟩

Abstract

Background: This study aimed to evaluate the reliability and validity of the self-administered questionnaire for Korean radiation workers. Methods: From May 24, 2016, to June 30, 2017, 20,608 participants completed the questionnaire, providing information on sociodemographics, lifestyle, work history and practices, medical radiation exposure, and medical history, which was linked to the National Dose Registry and the National Cancer Registry. The validity of the questionnaire was evaluated using the responses of 20,608 workers, and reliability was evaluated using the responses of 3043 workers who responded to the survey twice. Results: Responses concerning demographic characteristics and lifestyle showed reliability with a moderate-to-high agreement (kappa: 0.43-0.99), whereas responses concerning occupation and medical radiation exposure had a wide range of agreement (kappa: 0.05-0.95), possibly owing to temporal variability during employment. Regarding validity, responses to the question about the first year of employment had an excellent agreement with the national registry (intraclass correlation coefficient = 0.9); however, responses on cancer history had a wide range of agreement (kappa: 0.22-0.85). Conclusion: Although the reliability and validity of the questionnaire were not distinguished by demographic characteristics, they tended to be low among participants whose occupational radiation exposure was minimal. Overall, the information collected can be reliable for epidemiological studies; however, caution must be exercised when using information such as medical exposure and work practices, which are prone to temporal variability.

Keywords

Acknowledgement

This work was supported by the Korea Institute of Radiological and Medical Sciences, funded by the Nuclear Safety and Security Commission, Republic of Korea (grant 50091-2020).

References

  1. Gillham B. Developing a questionnaire. Bloomsbury Publishing; 2008. p. 1-14.
  2. Bowling A. Mode of questionnaire administration can have serious effects on data quality. J Publ Health 2005;27:281-91. https://doi.org/10.1093/pubmed/fdi031
  3. Althubaiti A. Information bias in health research: definition, pitfalls, and adjustment methods. J Multidiscip Healthc 2016;9:211-7. https://doi.org/10.2147/JMDH.S104807
  4. Nieuwenhuijsen MJ. Design of exposure questionnaires for epidemiological studies. Occup Environ Med 2005;62:272-80. https://doi.org/10.1136/oem.2004.015206
  5. Doll R. Hazards of ionising radiation: 100 years of observations on man. Br J Canc 1995;72:1339-49. https://doi.org/10.1038/bjc.1995.513
  6. Kim MJ, Cha ES, Ko Y, Chun BC, Lee WJ. Reliability of self-reported questionnaire on occupational radiation practices among diagnostic radiologic technologists. Am J Ind Med 2017;60:377-85. https://doi.org/10.1002/ajim.22704
  7. Bhatti P, Doody MM, Preston DL, Kampa D, Ron E, Weinstock RW, Simon S, Edwards AA, Sigurdson AJ. Increased frequency of chromosome translocations associated with diagnostic x-ray examinations. Radiat Res 2008;170:149-55. https://doi.org/10.1667/RR1422.1
  8. Kim HN, Jeong M, Park ES, Suh SJ, Jin Y-W. Reliability of a questionnaire in an epidemiological study for nuclear power plants workers in Korea. Kor J Occup Environ Med 2010;22:122-8 [in Korean]. https://doi.org/10.35371/kjoem.2010.22.2.122
  9. Azizova TV, Day RD, Wald N, Muirhead CR, O'Hagan JA, Sumina MV, Belyaeva ZD, Druzhinina MB, Teplyakov II , Semenikhina NG, Stetsenko LA, Grigoryeva ES, Krupenina LN, Vlasenko EV. The "clinic" medical-dosimetric database of Mayak production association workers: structure, characteristics and prospects of utilization. Health Phys 2008;94:449-58. https://doi.org/10.1097/01.HP.0000300757.00912.a2
  10. Kudo S, Nishde A, Ishida J, Yoshimoto K, Furuta H, Kasagi F. Direct risk comparison between radiation and smoking on cancer mortality among nuclear workers in Japan (J-EPISODE). Jpn J Health Phys 2020;55:32-9. https://doi.org/10.5453/jhps.55.32
  11. Park S, Seo S, Lee D, Park S, Jin Y-W. A Cohort study of Korean radiation workers: baseline characteristics of participants. Int J Environ Res Publ Health 2020;17:2328. https://doi.org/10.3390/ijerph17072328
  12. Seo S, Lim WY, Lee DN, Kim JU, Cha ES, Bang YJ, Lee WJ, Park S, Jin YW. Assessing the health effects associated with occupational radiation exposure in Korean radiation workers: protocol for a prospective cohort study. BMJ Open 2018;8:e017359. https://doi.org/10.1136/bmjopen-2017-017359
  13. Radiation and Health Study [Internet]. Radiation and health study among radiation workers in Korea. Available from: http://www.rhs.kr/method/overview.asp. [in Korean].
  14. National Cancer Institute: Division of Cancer Epidemiology & Genetics [Internet]. U.S. Radiologic Technologists Study: Archive Questionnaire Surveys. Available from: https://radtechstudy.nci.nih.gov/questionnaires.html.
  15. Chiang I, Jhangiani R, Price P. Reliability and validity of measurement. Res Methods Psychol 2015:96-102. 2nd ed. Victoria, B.C.: BCcampus.
  16. Warrens MJ. Cohen's kappa is a weighted average. Stat Methodol 2011;8:473-84. https://doi.org/10.1016/j.stamet.2011.06.002
  17. Cohen J. Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit. Psychol Bull 1968;7:213-20. https://doi.org/10.1037/h0069083
  18. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159-74. https://doi.org/10.2307/2529310
  19. White E, Armstrong BK, Saracci R. Principles of exposure measurement in epidemiology: collecting, evaluating and improving measures of disease risk factors. Oxford University Press; 2008.
  20. Fleiss J. Reliability of measurement. Des Anal Clin Exp 1986:1-32.
  21. Lang TA, Lang T, Secic M. How to report statistics in medicine: annotated guidelines for authors, editors, and reviewers. ACP Press; 2006.
  22. Kang YJ, Myong JP, Eom H, Choi B, Park JH, L Kim EA. The current condition of the workers' general health examination in South Korea: a retrospective study. Ann Occup Environ Med 2017;29:6. https://doi.org/10.1186/s40557-017-0157-0
  23. Feinstein AR, Cicchetti DV. High agreement but low kappa: I. The problems of two paradoxes. J Clin Epidemiol 1990;43:543-9. https://doi.org/10.1016/0895-4356(90)90158-L
  24. Brower PS, Attfield MD. Reliability of reported occupational history information for US coal miners, 1969-1977. Am J Epidemiol 1998;148:920-6. https://doi.org/10.1093/oxfordjournals.aje.a009718
  25. Hobson AJ, Sterling DA, Emo B, Evanoff BA, Sterling CS, Good L. Validity and reliability of an occupational exposure questionnaire for parkinsonism in welders. J Occup Environ Hyg 2009;6:324-31. https://doi.org/10.1080/15459620902836856
  26. Rosenberg CR. An analysis of the reliability of self reported work histories from a cohort of workers exposed to polychlorinated biphenyls. Br J Ind Med 1993;50:822-6.
  27. van der Gulden JW, Jansen IW, Verbeek AL, Kolk JJ. Repeatability of self-reported data on occupational exposure to particular compounds. Int J Epidemiol 1993;22:284-7. https://doi.org/10.1093/ije/22.2.284
  28. Blair A, Tarone R, Sandler D, Lynch CF, Rowland A, Wintersteen W, Steen WC, Samanic C, Dosemeci M. Reliability of reporting on life-style and agricultural factors by a sample of participants in the Agricultural Health Study from Iowa. Epidemiology 2002;13:94-9. https://doi.org/10.1097/00001648-200201000-00015
  29. Engel LS, Keifer MC, Thompson ML, Zahm SH. Test-retest reliability of an icon/ calendar-based questionnaire used to assess occupational history. Am J Ind Med 2001;40:512-22. https://doi.org/10.1002/ajim.1119
  30. Duell EJ, Millikan RC, Savitz DA, Schell MJ, Newman B, Tse CJ, Sandler DP. Reproducibility of reported farming activities and pesticide use among breast cancer cases and controls. A comparison of two modes of data collection. Ann Epidemiol 2001;11:178-85. https://doi.org/10.1016/S1047-2797(00)00208-8
  31. Zajacova A, Dowd JB. Reliability of self-rated health in US adults. Am J Epidemiol 2011;174:977-83. https://doi.org/10.1093/aje/kwr204
  32. Cho S, Shin A, Song D, Park JK, Kim Y, Choi JY, Kang D, Lee JK. Validity of self-reported cancer history in the health examinees (HEXA) study: a comparison of self-report and cancer registry records. Canc Epidemiol 2017;50:16-21. https://doi.org/10.1016/j.canep.2017.07.010
  33. Bergmann MM, Calle EE, Mervis CA, Miracle-McMahill HL, Thun MJ, Heath CW. Validity of self-reported cancers in a prospective cohort study in comparison with data from state cancer registries. Am J Epidemiol 1998;147:556-62. https://doi.org/10.1093/oxfordjournals.aje.a009487
  34. Navarro C, Chirlaque MD, Tormo MJ, Perez-Flores D, Rodriguez-Barranco M, Sanchez-Villegas A, Agudo A, Pera G, Amiano P, Dorronsoro M, Larranaga N, Quiros JR, Ardanaz E, Barricarte A, Martinez C, Sanchez MJ, Berenguer A, Gonzalez CA. Validity of self reported diagnoses of cancer in a major Spanish prospective cohort study. J Epidemiol Commun Health 2006;60:593-9. https://doi.org/10.1136/jech.2005.039131
  35. Inoue M, Sawada N, Shimazu T, Yamaji T, Iwasaki M, Sasazuki S, Tsugane S. Validity of self-reported cancer among a Japanese population: recent results from a population-based prospective study in Japan (JPHC Study). Canc Epidemiol 2011;35:250-3. https://doi.org/10.1016/j.canep.2010.12.002
  36. Nuclear safety and security commission. Korea institute of nuclear safety & Korea institute of nuclear nonproliferation and control. 2018 Nucl Saf Yearbk 2019 [in Korean].
  37. Craighead WE, Nemeroff CB. The concise corsini encyclopedia of psychology and behavioral science. Wiley; 2004.
  38. Brackbill Y. Test-retest reliability in population research. Stud Fam Plan 1974;5:261-6. https://doi.org/10.2307/1964896
  39. Stein AD, Courval JM, Lederman RI, Shea S. Reproducibility of responses to telephone interviews: demographic predictors of discordance in risk factor status. Am J Epidemiol 1995;141:1097-105. https://doi.org/10.1093/oxfordjournals.aje.a117375
  40. Alwin DF. Margins of error: a study of reliability in survey measurement. Wiley; 2007.