Asian Journal for Public Opinion Research

Center for Asian Public Opinion Research & Collaboration Initiative (충남대학교 아시아여론조사학회)
권호 표지
DOI QR코드

DOI QR Code

RDD with Follow-Up Texting: A New Attempt to Build a Probability-Based Online Panel in South Korea

  • Received : 2023.05.17
  • Accepted : 2023.08.17
  • Published : 2023.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

Conducting face-to-face surveys is difficult and cost prohibitive, necessitating a new attempt to build a probability-based panel in South Korea. Since 99.9% of adult Koreans own a mobile phone, mobile phone numbers provide a viable sampling frame. Random digit dialing (RDD) surveys were conducted August-December 2021. Of the 288,056 valid phone numbers dialed, 13,655 respondents between the ages of 19 and 69 completed a phone survey. These respondents were later invited by text message to join a panel; 3,202 of these (23.4% or 1.2% based on the number initially contacted) joined the panel. When compared to official government statistics like resident registration data, the census, or the Social Survey, this new probability-based panel can be said to be representative of the Korean population on the basis of age, gender, location, marital status, and household size after weighting is applied. However, even after weighting, panel members are more educated than the general population, white-collar workers and self-employed people are overrepresented, and blue-collar workers are underrepresented. As of February 2023, this panel has grown to 10,471 participants with plans to continue to invite more panel members in the same way. Based on the comparisons in this paper, we can regard this panel as a cost-effective, probability-based panel that may be used for various kinds of public opinion research, by researchers both within and outside of Korea. As we continue to refine and grow this panel, we hope it will become more widely used by researchers as well as provide a model for those building similar panels in other countries.

Keywords

Acknowledgement

This work was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (Grant No. NRF-2022S1A5C2A02093620).

References

  1. AAPOR. (2023). Standard definitions: Final dispositions of case codes and outcome rates for surveys. https://aapor.org/wp-content/uploads/2023/05/StandardsDefinitions-10th-edition.pdf
  2. Alanya, A., Wolf, C., & Sotto, C. (2015). Comparing multiple imputation and propensity-score weighting in unit-nonresponse adjustments a simulation study. Public Opinion Quarterly, 79(3), 635-661. https://doi.org/10.1093/poq/nfv029
  3. Amaya, A., Hatley, N. and Lau, A. (2021). Measuring the Risks of Panel Conditioning in Survey Research. Pew Research Center. https://www.pewresearch.org/methods/2021/06/09/measuring-the-risks-ofpanel-conditioning-in-survey-research/
  4. Blom, A. G., Bosnjak, M., Cornilleau, A., Cousteaux, A.-S., Das, M., Douhou, S., & Krieger, U. (2016). A comparison of four probability-based online and mixed-mode panels in Europe. Social Science Computer Review, 34(1), 8-25. https://doi.org/10.1177/0894439315574825
  5. Cho, S. K., LoCascio, S. P., and Kim, S. (2021) Confirming the continued representativeness of an online/telephone panel using equivalence testing. Asian Journal for Public Opinion Research, 9(2), 188-211. http://dx.doi.org/10.15206/ajpor.2021.9.2.188
  6. Cho, S. K., LoCascio, S. P., Lee, K.-O., Jang, D.-H., & Lee, J. M. (2017). Testing the representativeness of a multimode survey in South Korea: Results from KAMOS. Asian Journal for Public Opinion Research, 4(2), 73-87. https://doi.org/10.15206/ajpor.2017.4.2.73
  7. Cho, S. K. & Oh, S. H. (2021). Webgiban seongeoyeolonjosaui jaengjeomgwa sinloeseong jego bang-an yeongu [A study on the issues of web-based election opinion survey and how to improve its reliability]. 2021 National Election Commission. https://www.nesdc.go.kr/portal/bbs/B0000002/view.do?nttId=2140&menuNo=200468
  8. Gallup Korea (2022). 2012-2022 smartphone sayonglyul & brand, smartwatch, museon-earphone-e daehan josa [2012-2022 smartphone & brand, smart watch, wireless earphone U&A survey. Gallup Report]. https://www.gallup.co.kr/gallupdb/reportContent.asp?seqNo=1309
  9. Gummera, T., and Rossmann, J. (2019). The effects of propensity score weighting on attrition biases in attitudinal, behavioral, and socio-demographic variables in a short-term web-based panel survey. International Journal of Social Research Methodology, 22(1), 81-95. https://doi.org/10.1080/13645579.2018.1496052
  10. Han, S. K. (2012). Honhabsig josawa webpanel-ui (yeot-eun) bichgwa (jit-eun) geuneul [Prospects and problems of the recent developments in social survey in Korea: Mixed-mode survey and volunteer web panel]. Survey Research, 13(3), 1-31.
  11. Hur, M. & Cho, S. K (2010). Jawonja panel-e uihan internet josaui seonghyangjojeong gajunghwa [Propensity adjustment weighting of the Internet survey by volunteer panel]. Survey Research 11(2), 1-28.
  12. Kaczmirek, L., Phillips, B., Pennay, D.W., Lavrakas, P.J., and Neiger, D. (2019). Building a probability-based online panel: Life in AustraliaTM. Australian National University Centre for Social Research & Methods (CSRM). https://csrm.cass.anu.edu.au/sites/default/files/docs/2019/8/19-050-MPONLINE-Publish.pdf
  13. Keeter, S. (2019). Growing and Improving Pew Research Center's American Trend Panel. Pew Research Center. https://www.pewresearch.org/methods/2019/02/27/growing-andimproving-pew-research-centers-american-trends-panel/
  14. Lee, S. (2006). Propensity score adjustment as a weighting scheme for volunteer panel web surveys. Journal of Official Statistics, 22(2), 329-340.
  15. Lee, K., & Jang, D. H. (2009). Internet seongeojosa-eseo seonghyang-gajungmohyeong jeog-yongsalye [Applying propensity score adjustment on election web surveys]. Survey Research, 10(3). 21-36.
  16. Soest A. V., Schonlau, M., and Kapteyn, A. (2007). Are "webographic" or attitudinal questions useful for adjusting estimates from web surveys using propensity scoring? Survey Research Methods, 1(3), 155-163. https://doi.org/10.2139/ssrn.1006108
  17. Vehovar, V. & Manfreda, K. L. (2008). Overview: Online surveys. In N. G. Fielding, R. M. Lee, & G. Blank, (Eds.), The SAGE Handbook of Online Research Methods (2nd Edition). SAGE Publications, 143-161.
  18. Yoon, J. S. (2022). Smartphone penetration rate as share of the population in South Korea from 2015 to 2025. Statista. https://www.statista.com/statistics/321408/smartphone-user-penetration-insouth-korea/
  19. Yoon, J. S. (2021). South Korea's landline telephone penetration 2011-2018. Statista. https://www.statista.com/statistics/284204/south-korea-mobile-phoneinternet-user-penetration/