International Journal of Computer Science & Network Security

International Journal of Computer Science & Network Security (국제컴퓨터통신보호논문지학회)
권호 표지
DOI QR코드

DOI QR Code

Sentiment Analysis of COVID-19 Vaccination in Saudi Arabia

  • Sawsan Alowa (Department of Information Technology, King Saud University) ;
  • Lama Alzahrani (Department of Information Technology, King Saud University) ;
  • Noura Alhakbani (Department of Information Technology, King Saud University) ;
  • Hend Alrasheed (Department of Information Technology, King Saud University)
  • Received : 2023.02.05
  • Published : 2023.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

Since the COVID-19 vaccine became available, people have been sharing their opinions on social media about getting vaccinated, causing discussions of the vaccine to trend on Twitter alongside certain events, making the website a rich data source. This paper explores people's perceptions regarding the COVID-19 vaccine during certain events and how these events influenced public opinion about the vaccine. The data consisted of tweets sent during seven important events that were gathered within 14 days of the first announcement of each event. These data represent people's reactions to these events without including irrelevant tweets. The study targeted tweets sent in Arabic from users located in Saudi Arabia. The data were classified as positive, negative, or neutral in tone. Four classifiers were used-support vector machine (SVM), naïve Bayes (NB), logistic regression (LOGR), and random forest (RF)-in addition to a deep learning model using BiLSTM. The results showed that the SVM achieved the highest accuracy, at 91%. Overall perceptions about the COVID-19 vaccine were 54% negative, 36% neutral, and 10% positive.

Keywords

References

  1. Cascella, M.; Rajnik, M.; Aleem, A.; Dulebohn, S.C.; Di Napoli, R. Features, evaluation, and treatment of coronavirus (COVID-19). In StatPearls; StatPearls Publishing: Treasure Island (FL), US, 2022.
  2. Zu, Z.Y.; Jiang, M.D.; Xu, P.P.; Chen, W.; Ni, Q.Q.; Lu, G.M.; Zhang, L.J. Coronavirus disease 2019 (COVID-19): A perspective from China. Radiology 2020, 296, E15-E25, doi:10.1148/radiol.2020200490.
  3. Guner, R.; Hasanoglu, I.; Aktas, F. COVID-19: Prevention and control measures in community. Turk J Med Sci 2020, 50, 571-577, doi:10.3906/sag2004-146.
  4. Ebrahim, S.H.; Ahmed, Q.A.; Gozzer, E.; Schlagenhauf, P.; Memish, Z.A. Covid-19 and community mitigation strategies in a pandemic. BMJ 2020, 368, m1066, doi:10.1136/bmj.m1066.
  5. Lester, G.V.; Brock Baskin, M.E.; Clinton, M.S. Employer-sponsored benefits in the United States: The past, present, and future. Compensation & Benefits Review 2021, 53, 24-42, doi:10.1177/0886368720947609.
  6. Cajner, T.; Figura, A.; Price, B.M.; Ratner, D.; Weingarden, A. Reconciling unemployment claims with job losses in the first months of the Covid-19 Crisis. Board of Governors of the Federal Reserve System 2020, doi:10.17016/FEDS.2020.055.
  7. Alliheibi, F.M.; Omar, A.; Al-Horais, N. Opinion mining of Saudi responses to COVID-19 vaccines on Twitter. IJACSA 2021, 12, doi:10.14569/IJACSA.2021.0120610.
  8. Alliheibi, F.M.; Omar, A.; Al-Horais, N. Opinion mining of Saudi responses to COVID-19 vaccines on Twitter. IJACSA 2021, 12, doi:10.14569/IJACSA.2021.0120610. Schaffer DeRoo, S.; Pudalov, N.J.; Fu, L.Y. Planning for a COVID-19 vaccination program. JAMA 2020, 323, 2458-2459, doi:10.1001/jama.2020.8711.
  9. Jung, J. Preparing for the coronavirus disease (COVID-19) vaccination: Evidence, plans, and implications. J Korean Med Sci 2021, 36, doi:10.3346/jkms.2021.36.e59.
  10. Strodel, R.; Dayton, L.; Garrison-Desany, H.M.; Eber, G.; Beyrer, C.; Arscott, J.; Rubenstein, L.; Sufrin, C. COVID-19 vaccine prioritization of incarcerated people relative to other vulnerable groups: An analysis of state plans. PLoS One 2021, 16, e0253208, doi:10.1371/journal.pone.0253208.
  11. Hardeman, A.; Wong, T.; Denson, J.L.; Postelnicu, R.; Rojas, J.C. Evaluation of health equity in COVID-19 vaccine distribution plans in the United States. JAMA Network Open 2021, 4, e2115653, doi:10.1001/jamanetworkopen.2021.15653.
  12. Hardeman, A.; Wong, T.; Denson, J.L.; Postelnicu, R.; Rojas, J.C. Evaluation of health equity in COVID-19 vaccine distribution plans in the United States. JAMA Network Open 2021, 4, e2115653, doi:10.1001/jamanetworkopen.2021.15653.
  13. Barry, M.; BaHammam, A.S. COVID-19 vaccine in the Kingdom of Saudi Arabia: A true operation warp speed. J Nat Sci Biol Med 2021, 4, 92, doi:10.4103/jnsm.jnsm_8_21.
  14. Alfatease, A.; Alqahtani, A.M.; Orayj, K.; Alshahrani, S.M. The impact of social media on the acceptance of the COVID-19 vaccine: A cross-sectional study from Saudi Arabia. Patient Prefer Adherence 2021, 15, 2673-2681, doi:10.2147/PPA.S342535.
  15. Almaghaslah, D.; Alsayari, A.; Kandasamy, G.; Vasudevan, R. COVID-19 vaccine hesitancy among young adults in Saudi Arabia: A cross-sectional webbased study. Vaccines 2021, 9, 330, doi:10.3390/vaccines9040330.
  16. Islam, M.S.; Kamal, A.-H.M.; Kabir, A.; Southern, D.L.; Khan, S.H.; Hasan, S.M.M.; Sarkar, T.; Sharmin, S.; Das, S.; Roy, T.; et al. COVID-19 vaccine rumors and conspiracy theories: The need for cognitive inoculation against misinformation to improve vaccine adherence. PLoS One 2021, 16, e0251605, doi:10.1371/journal.pone.0251605.
  17. Manguri, K.; Ramadhan, R.; Mohammed Amin, P. Twitter sentiment analysis on worldwide COVID-19 outbreaks. Kurdistan J J Appl Res 2020, 54-65, doi:10.24017/covid.8.
  18. Alamoodi, A.H.; Zaidan, B.B.; Zaidan, A.A.; Albahri, O.S.; Mohammed, K.I.; Malik, R.Q.; Almahdi, E.M.; Chyad, M.A.; Tareq, Z.; Albahri, A.S.; et al. Sentiment analysis and its applications in fighting COVID-19 and infectious diseases: A systematic review. Expert Syst Appl 2020, 114155, doi:10.1016/j.eswa.2020.114155.
  19. Chandra, R.; Krishna, A. COVID-19 sentiment analysis via deep learning during the rise of novel cases. PLoS One 2021, 16, e0255615, doi:10.1371/journal.pone.0255615.
  20. Hung, M.; Lauren, E.; Hon, E.; Birmingham, W.; Xu, J.; Su, S.; Hon, S.; Park, J.; Dang, P.; Lipsky, M. Social network analysis of COVID-19 sentiments: Application of artificial intelligence. J Med Inter Res 2020, 22, e22590, doi:10.2196/22590.
  21. Samuel, J.; Ali, G.G.Md.N.; Rahman, Md.M.; Esawi, E.; Samuel, Y. COVID-19 public sentiment insights and machine learning for tweets classification. Information 2020, 11, 314, doi:10.3390/info11060314.
  22. Alanezi, M.A.; Hewahi, N.M. Tweets Sentiment Analysis During COVID-19 Pandemic. In Proceedings of the 2020 International Conference on Data Analytics for Business and Industry: Way Towards a Sustainable Economy (ICDABI), October 2020, pp. 1-6.
  23. Du, J.; Xu, J.; Song, H.-Y.; Tao, C. Leveraging machine learning-based approaches to assess human papillomavirus vaccination sentiment trends with Twitter data. BMC Medical Inform. Decis. Mak 2017, 17, 69, doi:10.1186/s12911-017-0469-6.
  24. Raghupathi, V.; Ren, J.; Raghupathi, W. Studying public perception about vaccination: A sentiment analysis of tweets. Int J Environ Res Public Health 2020, 17, 3464.
  25. Salathe, M.; Khandelwal, S. Assessing vaccination sentiments with online social media: Implications for infectious disease dynamics and control. PLoS Comput Biol 2011, 7, e1002199, doi:10.1371/journal.pcbi.1002199.
  26. Cotfas, L.-A.; Delcea, C.; Roxin, I.; Ioanas, C.; Gherai, D.S.; Tajariol, F. The longest month: Analyzing COVID-19 vaccination opinions dynamics from tweets in the month following the first vaccine announcement. IEEE Access 2021, 9, 33203-33223, doi:10.1109/ACCESS.2021.3059821.
  27. Praveen, S.; Ittamalla, R.; Deepak, G. Analyzing the attitude of Indian citizens towards COVID-19 vaccine - A text analytics study. Diabetes Metab Syndr: Clin Res Rev 2021, 15, 595-599, doi:10.1016/j.dsx.2021.02.031.
  28. Nurdeni, D.A.; Budi, I.; Santoso, A.B. Sentiment Analysis on Covid-19 Vaccines in Indonesia: From the Perspective of Sinovac and Pfizer. In Proceedings of the 2021 3rd East Indonesia Conference on Computer and Information Technology (EIConCIT), Indonesia, April 2021, pp. 122-127.
  29. Shahriar, K.T.; Islam, M.N.; Anwar, Md.M.; Sarker, I.H. COVID-19 analytics: Towards the effect of vaccine brands through analyzing public sentiment of tweets. Inform Med Unlocked 2022, 31, 100969, doi:10.1016/j.imu.2022.100969.
  30. Alabrah, A.; Alawadh, H.M.; Okon, O.D.; Meraj, T.; Rauf, H.T. Gulf countries' citizens' acceptance of COVID-19 vaccines-A machine learning approach. Mathematics 2022, 10, 467, doi:10.3390/math10030467.
  31. Khan, I.U.; Aslam, N.; Chrouf, S.; Atef, I.; Merah, I.; AlMulhim, L.; AlShuaifan, R. Computational intelligence-based model for exploring individual perception on SARS-CoV-2 vaccine in Saudi Arabia. Comput Intell Neurosci 2022, e6722427, doi:10.1155/2022/6722427.
  32. Zacharias, C.; Poldi, F. TWINT-Twitter Intelligence Tool 2021.
  33. Bird, S.; Klein, E.; Loper, E. Natural Language Processing With Python, 1st ed.; O'Reilly, 2009; ISBN 978-0-596-51649-9.
  34. Su, Y.; Venkat, A.; Yadav, Y.; Puglisi, L.B.; Fodeh, S.J. Twitter-based analysis reveals differential COVID-19 concerns across areas with socioeconomic disparities. Comput Biol Med. 2021, 132, 104336, doi:10.1016/j.compbiomed.2021.104336.
  35. Abu Farha, I.; Magdy, W. Mazajak: An Online Arabic Sentiment Analyser. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, Association for Computational Linguistics: Florence, Italy, August 2019, pp. 192-198.
  36. Al-Smadi, M.; Al-Ayyoub, M.; Jararweh, Y.; Qawasmeh, O. Enhancing aspectbased sentiment analysis of Arabic hotels' reviews using morphological, syntactic and semantic features. Inf Process Manag 2019, 56, 308-319, doi:10.1016/j.ipm.2018.01.006.
  37. Eliguzel, N.; Cetinkaya, C.; Dereli, T. Comparison of different machine learning techniques on location extraction by utilizing geo-tagged tweets: A case study. Adv Eng Inform 2020, 46, 101151, doi:10.1016/j.aei.2020.101151.
  38. Alhajji, M.; Khalifah, A.A.; Aljubran, M.; Alkhalifah, M. Sentiment analysis of tweets in Saudi Arabia regarding governmental preventive measures to contain COVID-19. Preprints 2020, 2020040031, doi:10.20944/preprints202004.0031.v1.
  39. Alyami, S.N.; Olatunji, S.O. Application of support vector machine for Arabic sentiment classification using Twitter-based dataset. J Info Know Mgmt 2020, 19, 2040018, doi:10.1142/S0219649220400183.
  40. Dey, S.; Wasif, S.; Tonmoy, D.S.; Sultana, S.; Sarkar, J.; Dey, M. A Comparative Study of Support Vector Machine and Naive Bayes Classifier for Sentiment Analysis on Amazon Product Reviews. In Proceedings of the 2020 International Conference on Contemporary Computing and Applications (IC3A), February 2020, pp. 217-220.
  41. Xu, F.; Pan, Z.; Xia, R. E-commerce product review sentiment classification based on a naive Bayes continuous learning framework. Inf Process Manag 2020, 57, 102221.
  42. Daeli, N.O.F.; Adiwijaya, A. Sentiment Analysis on movie reviews using information gain and K-nearest neighbor. J Data Inf Sci 2020, 3, 1-7.
  43. Troussas, C.; Virvou, M.; Espinosa, K.J.; Llaguno, K.; Caro, J. Sentiment Analysis of Facebook Statuses Using Naive Bayes Classifier for Language Learning. In Proceedings of the IISA 2013, IEEE, 2013, pp. 1-6.
  44. Parveen, H.; Pandey, S. Sentiment Analysis on Twitter Data-Set Using Naive Bayes Algorithm. In Proceedings of the 2016 2nd International Conference on Applied and Theoretical Computing and Communication Technology (ICATCCT), IEEE, 2016, pp. 416-419.
  45. Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V.; et al. Scikit-Learn: Machine learning in Python. J Mach Learn Res 2011, 12, 2825-2830.
  46. Vangara, R.V.B.; Thirupathur, K.; Vangara, S.P. Opinion mining classification using naive Bayes algorithm. Int J Eng Innov Technol (IJITEE) 2020, 9, 495- 498. https://doi.org/10.35940/ijitee.E2402.039520
  47. Lorena, A.C.; Jacintho, L.F.O.; Siqueira, M.F.; Giovanni, R.D.; Lohmann, L.G.; de Carvalho, A.C.P.L.F.; Yamamoto, M. Comparing machine learning classifiers in potential distribution modelling. Expert Syst Appl 2011, 38, 5268- 5275, doi:10.1016/j.eswa.2010.10.031.
  48. Zulqarnain, M.; Ghazali, R.; Mazwin, Y.; Rehan, M. A Comparative review on deep learning models for text classification. Indones J Electr Eng 2020, 19, doi:10.11591/ijeecs.v19.i1.pp325-335.
  49. Yu, Y.; Si, X.; Hu, C.; Zhang, J. A review of recurrent neural networks: LSTM cells and network architectures. Neural Comput 2019, 31, 1235-1270, doi:10.1162/neco_a_01199.
  50. Rhanoui, M.; Mikram, M.; Yousfi, S.; Barzali, S. A CNN-BiLSTM model for document-level sentiment analysis. Mach Learn Knowl Extr 2019, 1, 832-847, doi:10.3390/make1030048.
  51. Greff, K.; Srivastava, R.K.; Koutnik, J.; Steunebrink, B.R.; Schmidhuber, J. LSTM: A search space odyssey. IEEE Trans Neural Netw Learn Syst 2017, 28, 2222-2232, doi:10.1109/TNNLS.2016.2582924.
  52. Hameed, Z.; Garcia-Zapirain, B. Sentiment classification using a single-layered BiLSTM model. IEEE Access 2020, 8, 73992-74001, doi:10.1109/ACCESS.2020.2988550.
  53. Chawla, N.V.; Bowyer, K.W.; Hall, L.O.; Kegelmeyer, W.P. SMOTE: Synthetic minority over-sampling technique. J Artif Intell Res 2002, 16, 321-357, doi:10.1613/jair.953.
  54. Al-Azani, S.; El-Alfy, E.-S.M. Using word embedding and ensemble learning for highly imbalanced data sentiment analysis in short Arabic text. Procedia Comput Sci 2017, 109, 359-366, doi:10.1016/j.procs.2017.05.365.
  55. Obiedat, R.; Qaddoura, R.; Al-Zoubi, A.M.; Al-Qaisi, L.; Harfoushi, O.; Alrefai, M.; Faris, H. Sentiment analysis of customers' reviews using a hybrid evolutionary SVM-based approach in an imbalanced data distribution. IEEE Access 2022, 10, 22260-22273, doi:10.1109/ACCESS.2022.3149482.
  56. SMOTE - Version 0.10.0. Dev0 Available online: https://imbalancedlearn.org/dev/references/generated/imblearn.over_sampling.SMOTE.html (accessed on 3 July 2022).
  57. Carneiro, T.; Medeiros Da NoBrega, R.V.; Nepomuceno, T.; Bian, G.-B.; De Albuquerque, V.H.C.; Filho, P.P.R. Performance analysis of Google colaboratory as a tool for accelerating deep learning applications. IEEE Access 2018, 6, 61677-61685, doi:10.1109/ACCESS.2018.2874767.
  58. Wadhe, A.A.; Suratkar, S.S. Tourist Place Reviews Sentiment Classification Using Machine Learning Techniques. In Proceedings of the 2020 International Conference on Industry 4.0 Technology (I4Tech), February 2020, pp. 1-6.
  59. Zeng, Y.; Jiang, K.; Chen, J. Automatic Seismic Salt Interpretation with Deep Convolutional Neural Networks. In Proceedings of the Proceedings of the 2019 3rd International Conference on Information System and Data Mining - ICISDM 2019, ACM Press: Houston, TX, USA, 2019, pp. 16-20.
  60. Alkaff, M.; Rizky Baskara, A.; Hendro Wicaksono, Y. Sentiment Analysis of Indonesian Movie Trailer on YouTube Using Delta TF-IDF and SVM. In Proceedings of the 2020 Fifth International Conference on Informatics and Computing (ICIC), November 2020, pp. 1-5.
  61. Srivastava, N.; Hinton, G.; Krizhevsky, A.; Sutskever, I.; Salakhutdinov, R. Dropout: A simple way to prevent neural networks from overfitting. J Mach Learn Res 2014, 15, 1929-1958.
  62. Ding, B.; Qian, H.; Zhou, J. Activation Functions and Their Characteristics in Deep Neural Networks. In Proceedings of the 2018 Chinese Control And Decision Conference (CCDC), June 2018, pp. 1836-1841.
  63. Jiao, Y.; Du, P. Performance Measures in evaluating machine learning based bioinformatics predictors for classifications. Quant Biol 2016, 4, 320-330, doi:10.1007/s40484-016-0081-2.
  64. Novakovic, J.D.; Veljovic, A.; Ilic, S.S.; Papic, Z.; Milica, T. Evaluation of classification models in machine learning. Theo Comput Math with Appl 2017, 7, 39-46.
  65. Patino, C.M.; Ferreira, J.C. Internal and external validity: Can you apply research study results to your patients? J Bras Pneumol 2018, 44, 183, doi:10.1590/S1806-37562018000000164.
  66. Alfageeh, E.I.; Alshareef, N.; Angawi, K.; Alhazmi, F.; Chirwa, G.C. Acceptability of a COVID-19 vaccine among the Saudi Population. Vaccines (Basel) 2021, 9, 226, doi:10.3390/vaccines9030226.
  67. فريق بوابة وزارة الصحةMinistry of Health - Kingdom of Saudi Arabia. Available online: https://www.moh.gov.sa/en/Pages/Default.aspx (accessed on 18 February 2022).
  68. Alahmed, M. عاجل: وزير الصحة السعودي يتلقى الجرعة الاولى من لقاح كرونا," صحيفة وقع الإلكترونية Urgent: The Saudi Minister of Health receives the first dose of the Corona vaccine. .صحيفة وقع الإلكترونية 2020 
  69. Al Hadath Saudi Health approves AstraZeneca and Moderna vaccines after Pfizer. Available online: shorturl.at/gpsz3 (accessed on 29 July 2022).
  70. Galluccio, B. New "Double Mutant" Coronavirus Variant Discovered in India. Available online: https://www.iheart.com/content/2021-03-25-new-double-mutant-coronavirus-variant-discovered-in-india/ (accessed on 29 July 2022).
  71. Singh, K. WHO Rules out New "hybrid" COVID Variant in Vietnam - Nikkei Reuters.Com. Reuters 2021.
  72. Ministry of Education Ministry of Education - Kingdom of Saudi Arabia Available online: https://www.moe.gov.sa/en/pages/default.aspx (accessed on 18 February 2022).
  73. Zarkachat. The Ministry of Education in Saudi Arabia forms a committee to discuss the return of the study in person. Zarkachat 2021.
  74. Ministry of Interior Date for Lifting Travel Suspension for Citizens, Opening All Land, Sea, Air Outlets in Full, Postponed to May 17. Available online: https://bit.ly/3gZa5eI (accessed on 19 February 2022).
  75. Asharq Al-Awsat Saudi Arabia: Vaccination Needed to Enter Govt, Private Offices. Available online: https://english.aawsat.com/home/article/2980751/saudi-arabia-vaccination-needed-enter-govt-private-offices (accessed on 29 July 2022).
  76. Arabian Business Saudi Arabia to Require Covid-19 Vaccine to Enter "all Places in All Regions" - Arabian Business Available online: https://www.arabianbusiness.com/industries/healthcare/466621-saudi-arabiato-require-covid-19-vaccine-to-enter-all-places-in-all-regions (accessed on 29 July 2022).
  77. Alam, K.N.; Khan, M.S.; Dhruba, A.R.; Khan, M.M.; Al-Amri, J.F.; Masud, M.; Rawashdeh, M. Deep learning-based sentiment analysis of COVID-19 vaccination responses from Twitter data. Comput Math Methods Med 2021, e4321131, doi:10.1155/2021/4321131.