The Korean Journal of Applied Statistics (응용통계연구)

The Korean Statistical Society (한국통계학회)
권호 표지
DOI QR코드

DOI QR Code

Mode effects in concurrent mixed-mode surveys

병행적 혼합조사의 모드효과 분석

  • Baek, Jeeseon (Methodology Division, Statistical Research Institute, Statistics Korea) ;
  • Min, Kyung A (Methodology Division, Statistical Research Institute, Statistics Korea)
  • 백지선 (통계청 통계개발원 조사연구실) ;
  • 민경아 (통계청 통계개발원 조사연구실)
  • Received : 2016.03.29
  • Accepted : 2016.07.07
  • Published : 2016.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

Mixed-mode (MM) designs in which data are collected by different modes in one design have become increasingly popular. An MM data collection has several advantages such as reductions of coverage error, non-response and cost. However, MM designs may introduce mode effects that are confounded by selection effects and measurement effects, which can make MM data quality poor. In order to investigate mode effects, SRI implemented a concurrent mixed-mode experiment in 2014 where respondents could choose between a self-administrated Web survey and a self-administrated paper survey. This paper separately estimates selection effects and measurement effects. We found that measurement effects on some items are large.

사회환경 변화와 정보기술 발전에 따라 한 조사에 다양한 조사모드(survey mode)를 활용하는 혼합조사(mixed-mode survey)가 확대되고 있으며, 포함오차 축소, 비용절감, 응답률 향상 등의 장점을 가진다. 그러나 혼합조사는 선택효과와 측정효과가 혼재된 모드효과 발생으로 조사품질에 대한 우려가 있다. 통계개발원은 이러한 혼합조사의 모드효과를 파악하기 위하여 2014년에 병행적 혼합모드 시험조사를 수행하였다. 본 연구는 시험조사 자료에 대해 자기기입식 종이조사와 웹조사 간 모드효과를 선택효과와 측정효과로 분리하여 계량적으로 분석하였다. 선택계층의 효과를 분리하였을 때, 조사항목에 따라 과대 또는 과소 응답하는 측정효과가 존재하는 것을 확인하였다.

Keywords

References

  1. Bowling, A. (2005). Mode of questionnaire administration can have serious effects on data quality, Journal of Public Health, 27, 281-291. https://doi.org/10.1093/pubmed/fdi031
  2. Cannell, C., Miller, P., and Oksenberg, L. (1981). Research on interviewing techniques, Sociological Methodology, 389-437.
  3. De Leeuw, E. (2005). To mix or not to mix? Data collection modes in surveys, Journal of Official Statistics, 21, 1-23.
  4. Dillman, D. A., Phelps, G., Tortora, R., Swift, K., Kohrell, J., Berck, J., and Messer, B. L. (2009). Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (IVR), and the internet, Social Science Research, 38, 3-20.
  5. Groves, R. (1989). Survey Errors and Survey Costs, Wiley, New York.
  6. Groves, R., Fowler, F. J., Couper, M. P., Lepkowski, J. M., Singer, E., and Tourangeau, R. (2009). Survey Methodology, Wiley, New York.
  7. Groves, R. and Kahn, R. (1979). Surveys by Telephone: A National Comparison with Personal Interviews, Academic Press, New York.
  8. Jackle, A., Roberts, C., and Lynn, P. (2008). Assessing the effect of data collection mode on measurement, Institute for Social & Economic Research.
  9. Tourangeau, R., Rips, L. J., and Rasinski, K. A. (2000). The Psychology of Survey Response, Cambridge University press, Cambridge.
  10. Vannieuwenhuyze, J., Loosveldt, G., and Molenberghs, G. (2010). A method for evaluating mode effects in mixed-mode surveys, Public Opinion Quarterly, 74, 1027-1045. https://doi.org/10.1093/poq/nfq059
  11. Voogt, R. J. J. and Saris, W. E. (2005). Mixed mode designs; finding the balance between nonresponse bias and mode effects, Journal of Official Statistics, 21, 367-387.
  12. Weisberg, H. F. (2005). The Total Survey Error Approach: A Guide to the New Science of Survey Research, University of Chicago, Chicago.