Asian Journal for Public Opinion Research

Center for Asian Public Opinion Research & Collaboration Initiative (충남대학교 아시아여론조사학회)
권호 표지
DOI QR코드

DOI QR Code

Evaluating the Quality of Public Services Through Social Media

  • Received : 2021.03.20
  • Accepted : 2021.07.28
  • Published : 2021.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

Public services need to be evaluated regularly to identify areas that need further improvement. Data collection via Twitter is affordable and timely, so it has the potential to be utilized to evaluate the quality of public service. This study utilizes tweets mentioning three service units of the provincial government of Jakarta and applies both sentiment analysis and topic classification to predict a rating/score of public service quality. The research goal is to examine if the evaluation of public services based on social media data is possible. The findings indicate that the use of Twitter has an advantage in terms of sample size and variety of opinions. Tweets can be translated into scores as well. Nonetheless, the representativeness issue and the predominance of complaint tweets can affect the reliability of the results.

Keywords

References

  1. Aggarwal, C. C. (2016). Outlier analysis (2nd ed.). Springer. https://doi.org/10.1007/978-1-4614-6396-2
  2. Akinboade, O. A., Kinfack, E. C., & Mokwena, M. P. (2012). An analysis of citizen satisfaction with public service delivery in the Sedibeng district municipality of South Africa. International Journal of Social Economics, 39(3), 182-199. https://doi.org/10.1108/03068291211199350
  3. Aliyah Salsabila, N., Ardhito Winatmoko, Y., Akbar Septiandri, A., & Jamal, A. (2019). Colloquial Indonesian Lexicon. Proceedings of the 2018 International Conference on Asian Language Processing, IALP 2018, 226-229. https://doi.org/10.1109/IALP.2018.8629151
  4. Altammami, S. H., & Rana, O. F. (2017). Topic identification system to filter Twitter feeds. Proceedings - 2016 3rd International Conference on Soft Computing and Machine Intelligence, ISCMI 2016, 206-213. https://doi.org/10.1109/ISCMI.2016.14
  5. Anaman, K. A., Quaye, R., & Amankwah, E. (2017). Evaluation of Public Weather Services by Users in the Formal Services Sector in Accra, Ghana. Modern Economy, 08(07), 921-945. https://doi.org/10.4236/me.2017.87065
  6. Aziz, M. N., Firmanto, A., Miftah Fajrin, A., & Hari Ginardi, R. V. (2018). Sentiment analysis and topic modelling for identification of government service satisfaction. Proceedings - 2018 5th International Conference on Information Technology, Computer and Electrical Engineering, ICITACEE 2018, 125-130. https://doi.org/10.1109/ICITACEE.2018.8576974
  7. Bai, C., & Ye, C. (2005). Construct a public service quality evaluation model: Evidence from Chinese public service industry. 2005 International Conference on Services Systems and Services Management, Proceedings of ICSSSM'05, 1, 508-513. https://doi.org/10.1109/ICSSSM.2005.1499525
  8. Barbera, P., & Rivero, G. (2015). Understanding the political representativeness of Twitter users. Social Science Computer Review, 33(6), 712-729. https://doi.org/10.1177/0894439314558836
  9. Barbu, A., & Isaic-Maniu, A. (2011). Data collection in Romanian market research: A comparison between prices of PAPI, CATI and CAWI. Management & Marketing, 6(3), 349-364.
  10. Bencke, L., Cechinel, C., & Munoz, R. (2020). Automated classification of social network messages into smart cities dimensions. Future Generation Computer Systems, 109, 218-237. https://doi.org/10.1016/j.future.2020.03.057
  11. Benmansour, N. A. (2019). Citizens and expatriates satisfaction with public services in Qatar - evidence from a survey. International Journal of Social Economics, 46(3), 326-337. https://doi.org/10.1108/IJSE-03-2018-0118
  12. Bradshaw, S., & Howard, P. N. (2019). The global disinformation disorder: 2019 global inventory of organised social media manipulation. University of Oxford. https://comprop.oii.ox.ac.uk/wpcontent/uploads/sites/93/2019/09/CyberTroop-Report19.pdf
  13. Burnap, P., Colombo, G., Amery, R., Hodorog, A., & Scourfield, J. (2017). Multi-class machine classification of suicide-related communication on Twitter. Online Social Networks and Media, 2, 32-44. https://doi.org/10.1016/j.osnem.2017.08.001
  14. Chamby-Diaz, J. C., & Bazzan, A. L. C. (2019). Identifying traffic event types from Twitter by multi-label classification. Proceedings - 2019 Brazilian Conference on Intelligent Systems, BRACIS 2019, 806-811. https://doi.org/10.1109/BRACIS.2019.00144
  15. Chang, J., Rosenn, I., Backstrom, L., & Marlow, C. (2010). EPluribus: Ethnicity on social networks. Proceedings of the Fourth International AAAI Conference on Weblogs and Social Media, 18-25.
  16. Chodzaza, G. E., & Gombachika, H. S. H. (2013). Service quality, customer satisfaction and loyalty among industrial customers of a public electricity utility in Malawi. International Journal of Energy Sector Management, 7(2), 269-282. https://doi.org/10.1108/IJESM-02-2013-0003
  17. Citrin, J., & Green, D. P. (1986). Presidential leadership and the resurgence of trust in government. British Journal of Political Science, 16(4), 431-453. https://doi.org/10.1017/S0007123400004518
  18. Corallo, A., Fortunato, L., Matera, M., Alessi, M., Camillo , A., Chetta, V., Giangreco, E., & Storelli, D. (2015). Sentiment analysis for government: An optimized approach. In P. Perner (Ed.), Machine Learning and Data Mining in Pattern Recognition. MLDM 2015. Lecture notes in computer science, 9166, pp. 98-112. Springer International Publishing. https://doi.org/10.1007/978-3-319-21024-7_7
  19. Cui, W. (2009). A scientific inspection of public service of new countryside in China. Proceedings of the 2009 6th International Conference on Service Systems and Service Management, ICSSSM '09, 363-367. https://doi.org/10.1109/ICSSSM.2009.5174909
  20. Davis, D. H. (2017). Is Twitter a generalizable public sphere? A comparison of 2016 presidential campaign issue importance among general and Twitter publics. Proceedings of the 8th International Conference on Social Media & Society, 1-5. https://doi.org/10.1145/3097286.3097317
  21. Denhardt, R. B., & Denhardt, J. V. (2000). The new public service: Serving rather than steering. Public Administration Review, 60(6), 549-559. https://doi.org/10.1111/0033-3352.00117
  22. Ganu, G., Elhadad, N., & Marian, A. (2009). Beyond the stars: Improving rating predictions using review text content. WebDB, 9, 1-6.
  23. Ganu, G., Kakodkar, Y., & Marian, A. (2013). Improving the quality of predictions using textual information in online user reviews. Information Systems, 38(1), 1-15. https://doi.org/10.1016/j.is.2012.03.001
  24. Georgiadou, E., Angelopoulos, S., & Drake, H. (2020). Big data analytics and international negotiations: Sentiment analysis of Brexit negotiating outcomes. International Journal of Information Management, 51(November), 102048. https://doi.org/10.1016/j.ijinfomgt.2019.102048
  25. Gojali, S., & Khodra, M. L. (2016). Aspect based sentiment analysis for review rating prediction. 4th IGNITE Conference and 2016 International Conference on Advanced Informatics: Concepts, Theory and Application, ICAICTA 2016, 1-6. https://doi.org/10.1109/ICAICTA.2016.7803110
  26. Han, J., Kamber, M., & Pei, J. (2012). Data mining: Concepts and techniques (3rd ed.). Morgan Kaufmann Publishers.
  27. Heiervang, E., & Goodman, R. (2011). Advantages and limitations of web-based surveys: Evidence from a child mental health survey. Social Psychiatry and Psychiatric Epidemiology, 46, 69-76. https://doi.org/10.1007/s00127-009-0171-9
  28. Hu, X., & Liu, H. (2012). Text analytics in social media. In C. C. Aggarwal & C. Zhai (Eds.), Mining Text Data, 53(9), 385-414. Springer US. https://doi.org/10.1007/978-1-4614-3223-4_12
  29. Hutchinson, A. (2020). Who's complaining, and what are the most common reasons for calling out businesses on social? [Infographic]. Retrieved August 7, 2021 from https://www.socialmediatoday.com/news/whos-complaining-and-what-arethe-most-common-reasons-for-calling-out-bus/570076/
  30. Karimi, F., Wagner, C., Lemmerich, F., Jadidi, M., & Strohmaier, M. (2016). Inferring gender from names on the web: A comparative evaluation of gender detection methods. Proceedings of the 25th International Conference Companion on World Wide Web, 53-54. https://doi.org/http://dx.doi.org/10.1145/2872518.2889385
  31. Kaur, W., Balakrishnan, V., Rana, O., & Sinniah, A. (2019). Liking, sharing, commenting and reacting on Facebook: User behaviors' impact on sentiment intensity. Telematics and Informatics, 39, 25-36. https://doi.org/10.1016/j.tele.2018.12.005
  32. Kaura, V. (2013). Antecedents of customer satisfaction: A study of Indian public and private sector banks. International Journal of Bank Marketing, 31(3), 167-186. https://doi.org/10.1108/02652321311315285
  33. Koto, F., & Rahmaningtyas, G. Y. (2017). Inset lexicon: Evaluation of a word list for Indonesian sentiment analysis in microblogs. 2017 International Conference on Asian Language Processing (IALP), 2018-Janua(December), 391-394. https://doi.org/10.1109/IALP.2017.8300625
  34. Kroh, M., Ludtke, D., Duzel, S., & Winter, F. (2016). Response error in a web survey and a mailed questionnaire: The role of cognitive functioning. SOEPpapers on Multidisciplinary Panel Data Research, 888. https://doi.org/10.2139/ssrn.2920616
  35. Kwak, J. A., & Cho, S. K. (2018). Analyzing public opinion with social media data during election periods: A selective literature review. Asian Journal for Public Opinion Research, 5(4), 285-301. https://doi.org/10.15206/ajpor.2018.5.4.285
  36. Lailiyah, M., Sumpeno, S., & Purnama, I. K. E. (2017). Sentiment analysis of public complaints using lexical resources between Indonesian sentiment lexicon and sentiwordnet. 2017 International Seminar on Intelligent Technology and Its Application: Strengthening the Link Between University Research and Industry to Support ASEAN Energy Sector, ISITIA 2017 - Proceeding, 2017-Janua, 307-312. https://doi.org/10.1109/ISITIA.2017.8124100
  37. Levi, M., & Stoker, L. (2000). Political trust and trustworthiness. Annual Review of Political Science, 3, 475-507. https://doi.org/10.1146/annurev.polisci.3.1.475
  38. Li, Y., & Shang, H. (2020). Service quality, perceived value, and citizens' continuoususe intention regarding e-government: Empirical evidence from China. Information and Management, 57(3), 103197. https://doi.org/10.1016/j.im.2019.103197
  39. Liu, B. (2012). Sentiment analysis and opinion mining. Synthesis Lectures on Human Language Technologies, 5(1), 1-184. https://doi.org/10.2200/S00416ED1V01Y201204HLT016
  40. Madan, A., Arora, R., & Roy, N. R. (2018). Sentiment analysis of Indians on GST. In B. Panda, S. Sharma, & N. Roy (Eds.), Data Science and Analytics. REDSET 2017. Communications in Computer and Information Science, 799, 568-575. https://doi.org/10.1007/978-981-10-8527-7_47
  41. Martin-Domingo, L., Martin, J. C., & Mandsberg, G. (2019). Social media as a resource for sentiment analysis of airport service quality (ASQ). Journal of Air Transport Management, 78(January), 106-115. https://doi.org/10.1016/j.jairtraman.2019.01.004
  42. McGregor, S. C. (2019). Social media as public opinion: How journalists use social media to represent public opinion. Journalism, 20(8), 1070-1086. https://doi.org/10.1177/1464884919845458
  43. Medaglia, R., & Zheng, L. (2016). Characterizing government social media research: Towards a grounded overview model. 2016 49th Hawaii International Conference on System Sciences (HICSS), 2016-March, 2991-2999. https://doi.org/10.1109/HICSS.2016.375
  44. Mishler, W., & Rose, R. (2001). What are the origins of political trust? Testing institutional and cultural theories in post-communist societies. Comparative Political Studies, 34(1), 30-62. https://doi.org/10.1177/0010414001034001002
  45. Mislove, A., Lehmann, S., Ahn, Y.-Y., Onnela, J.-P., & Rosenquist, J. N. (2011). Understanding the demographics of Twitter users. Proceedings of the International AAAI Conference on Web and Social Media.
  46. Monett, D., & Stolte, H. (2016). Predicting star ratings based on annotated reviews of mobile apps. Proceedings of the 2016 Federated Conference on Computer Science and Information Systems, FedCSIS 2016, 8(June 2015), 421-428. https://doi.org/10.15439/2016F141
  47. Murphy, J., Link, M. W., Childs, J. H., Tesfaye, C. L., Dean, E., Stern, M., Pasek, J., Cohen, J., Callegaro, M., & Harwood, P. (2014). Social media in public opinion research: Executive summary of the AAPOR task force on emerging technologies in public opinion research. Public Opinion Quarterly, 78(4), 788-794. https://doi.org/10.1093/poq/nfu053
  48. OECD. (2017). Government at a Glance 2017. https://doi.org/https://doi.org/https://doi.org/10.1787/gov_glance-2017-en
  49. Pratama, T., & Purwarianti, A. (2017). Topic classification and clustering on Indonesian complaint tweets for Bandung government using supervised and unsupervised learning. Proceedings - 2017 International Conference on Advanced Informatics: Concepts, Theory and Applications, ICAICTA 2017, July 2016. https://doi.org/10.1109/ICAICTA.2017.8090981
  50. Psomas, E. (2020). Determining the impact of service quality on citizens' satisfaction and the role of citizens' demographics. The case of the Greek citizen's service centers. TQM Journal. https://doi.org/10.1108/TQM-12-2019-0274
  51. Qi, B., Costin, A., & Jia, M. (2020). A framework with efficient extraction and analysis of Twitter data for evaluating public opinions on transportation services. Travel Behaviour and Society, 21(May), 10-23. https://doi.org/10.1016/j.tbs.2020.05.005
  52. Rekha, V., Raksha, R., Patil, P., Swaras, N., & Rajat, G. L. (2019). Sentiment analysis on Indian government schemes using Twitter data. 2019 International Conference on Data Science and Communication, IconDSC 2019, 1-5. https://doi.org/10.1109/IconDSC.2019.8817036
  53. Salim, M., Peng, X., Almaktary, S., & Karmoshi, S. (2017). The impact of citizen satisfaction with government performance on public trust in the government: Empirical evidence from urban Yemen. Open Journal of Business and Management, 5(2), 348-365. https://doi.org/10.4236/ojbm.2017.52030
  54. Sayyadiharikandeh, M., Varol, O., Yang, K. C., Flammini, A., & Menczer, F. (2020). Detection of novel social bots by ensembles of specialized classifiers. Proceedings of the 29th ACM International Conference on Information & Knowledge Management, 2725-2732. https://doi.org/10.1145/3340531.3412698
  55. Seki, Y. (2016). Use of Twitter for analysis of public sentiment for improvement of local government service. 2016 IEEE International Conference on Smart Computing, SMARTCOMP 2016, 1-3. https://doi.org/10.1109/SMARTCOMP.2016.7501726
  56. Singh, P., Sawhney, R. S., & Kahlon, K. S. (2018). Sentiment analysis of demonetization of 500 & 1000 rupee banknotes by Indian government. ICT Express, 4(3), 124-129. https://doi.org/10.1016/j.icte.2017.03.001
  57. Sokolowska, M., Mazurek, M., Majer, M., & Podpora, M. (2019). Classification of user attitudes in Twitter -beginners guide to selected machine learning libraries. IFAC-PapersOnLine, 52(27), 394-399. https://doi.org/10.1016/j.ifacol.2019.12.692
  58. Sprout Social. (2017). Call-out culture: People, brands & the social media power struggle. https://media.sproutsocial.com/uploads/2017/08/2017-q3-sproutsocial-index.pdf
  59. Steinmetz, S., Bianchi, A., Tijdens, K., & Biffignandi, S. (2014). Improving web survey quality. A data quality perspective. John Wiley & Sons Ltd.
  60. Susilawati, E. (2016). Public services satisfaction based on sentiment analysis: Case study: Electrical services in Indonesia. 2016 International Conference on Information Technology System and Innovation (ICITSI), 1-6. https://doi.org/10.1109/ICITSI.2016.7858241
  61. Tala, F. Z. (2003). A Study of Stemming Effects on Information Retrieval in Bahasa Indonesia. Master's project, Universiteit van Amsterdam. https://eprints.illc.uva.nl/id/eprint/740/1/MoL-2003-02.text.pdf
  62. Twitter. (2021). Search API. Twitter. https://developer.twitter.com/en/docs/twitterapi/premium/search-api/guides/operators
  63. United Nations Department of Economic and Social Affairs. (2018). United Nations EGovernment Survey 2018: Gearing E-Government to Support Transformation Towards Sustainable and Resilient Societies. United Nations. https://publicadministration.un.org/egovkb/Portals/egovkb/Documents/un/2018-Survey/E-Government Survey 2018_FINAL for web.pdf
  64. United Nations Department of Economic and Social Affairs. (2020). E-Government Survey 2020 - Digital Government in the Decade of Action for Sustainable Development: With Addendum on COVID-19 Response. United Nations. https://publicadministration.un.org/egovkb/en-us/Reports/UN-E-GovernmentSurvey-2020
  65. ur Rahim, F., & Shirazi, N. S. (2018). Fiscal decentralization and citizen's satisfaction from local public service delivery in Pakistan. International Journal of Ethics and Systems, 34(1), 122-142. https://doi.org/10.1108/IJOES-04-2017-0066
  66. Valliant, R., & Dever, J. A. (2011). Estimating propensity adjustments for volunteer web surveys. Sociological Methods and Research, 40(1), 105-137. https://doi.org/10.1177/0049124110392533
  67. Zhang, L., Hall, M., & Bastola, D. (2018). Utilizing Twitter data for analysis of chemotherapy. International Journal of Medical Informatics, 120, 92-100. https://doi.org/10.1016/j.ijmedinf.2018.10.002
  68. Zhu, Q. (2018). Classification of trending topics in Twitter. Proceedings - 2018 International Conference on Computational Science and Computational Intelligence, CSCI 2018, 274-277. https://doi.org/10.1109/CSCI46756.2018.00060