Journal of Information Technology Applications and Management

Korea Data Strategy Society (한국데이터전략학회)
권호 표지
DOI QR코드

DOI QR Code

Comparative Study of Prediction Performance and Variable Importance in SEM-ANN Two-stage Analysis

SEM-ANN 2단계 분석에서 예측성능과 변수중요도의 비교연구

  • Sun-Dong Kwon (MIS Department in Chungbuk National University) ;
  • Yi Zhao (MIS Department in Chungbuk National University) ;
  • Hua-Long Fang (College of Liberal Arts, Cheongju University)
  • Received : 2024.01.30
  • Accepted : 2024.02.14
  • Published : 2024.02.29
Download PDF
⟨ Previous Next ⟩

Abstract

The purpose of this study is to investigate the improvement of prediction performance and changes in variable importance in SEM-ANN two-stage analysis. 366 cosmetics repurchase-related survey data were analyzed and the results were presented. The results of this study are summarized as follows. First, in SEM-ANN two-stage analysis, SEM and ANN models were trained with train data and predicted with test data, respectively, and the R2 was showed. As a result, the prediction performance was doubled from SEM 0.3364 to ANN 0.6836. Looking at this degree of R2 improvement as the effect size f2 of Cohen (1988), it corresponds to a very large effect at 110%. Second, as a result of comparing changes in normalized variable importance through SEM-ANN two-stage analysis, variables with high importance in SEM were also found to have high importance in ANN, but variables with little or no importance in SEM became important in ANN. This study is meaningful in that it increased the validity of the comparison by using the same learning and evaluation method in the SEM-ANN two-stage analysis. This study is meaningful in that it compared the degree of improvement in prediction performance and the change in variable importance through SEM-ANN two-stage analysis.

Keywords

Acknowledgement

This work was supported by a funding for the academic research program of Chungbuk National University in 2022. This work was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF-2022S1A3A2A01089239).

References

  1. Alam, M. M. D., Alam, M. Z., Rahman, S. A., and Taghizadeh, S. K., "Factors influencing mHealth adoption and its impact on mental well-being during COVID-19 pandemic: A SEM-ANN approach", Journal of Biomedical Informatics, Vol. 116, 2021, p. 103722.
  2. Alharbi, A., and Sohaib, O., "Technology readiness and cryptocurrency adoption: PLS-SEM and deep learning neural network analysis", IEEE Access, Vol.9, 2021, pp. 21388-21394.
  3. Chan, F. T. and Chong, A. Y., "A SEM- neural network approach for understanding determinants of interorganizational system standard adoption and performances", Decision Support Systems, Vol. 54, No. 1, 2012, pp. 621-630.
  4. Chong, A. Y. L., "A two-staged SEM-neural network approach for understanding and predicting the determinants of m-commerce adoption", Expert Systems with Applications, Vol. 40, No. 4, 2013, pp. 1240-1247.
  5. Chong, A. Y. L. and Bai, R., "Predicting open IOS adoption in SMEs: An integrated SEM-neural network approach", Expert Systems with Applications, Vol. 41, No. 1, 2014, pp. 221-229
  6. Cohen, J. E., "Statistical Power Analysis for the Behavioral Sciences", Hillsdale, NJ: Lawrence Erlbaum Associates, Inc, 1988.
  7. Jiang, G. B. and Kwon, S. D., "A SEM-ANN Two-step Approach for Predicting Determinants of Cloud Service Use Intention", Journal of Information Technology Applications & Management, Vol. 30, No. 6, 2023, pp. 91-111.
  8. Hair, Jr, J. F., Sarstedt, M., Matthews, L. M., and Ringle, C. M., "Identifying and treating unobserved heterogeneity with FIMIX-PLS: part I-method", European Business Review, Vol. 28, No. 1, 2016, pp. 63-76.
  9. Lau, A. J., Tan, G. W. H., Loh, X. M., Leong, L. Y., Lee, V. H., and Ooi, K. B., "On the way: Hailing a taxi with a smartphone? A hybrid SEM-neural network approach", Machine Learning with Applications, Vol. 4, 2021, p. 100034
  10. Leong, L. Y., Hew, T. S., Lee, V. H., and Ooi, K. B., "An SEM-artificial-neural-network analysis of the relationships between SERVPERF, customer satisfaction and loyalty among low-cost and full-service airline", Expert Systems with Applications, Vol. 42, No. 19, 2015, pp. 6620-6634
  11. Lo, P. S., Dwivedi, Y. K., Tan, G. W. H., Ooi, K. B., Aw, E. C. X., and Metri, B., "Why do consumers buy impulsively during live streaming? A deep learning-based dual-stage SEM-ANN analysis", Journal of Business Research, Vol. 147, 2022, pp. 325-337
  12. Nguyen, P. H., Tsai, J. F., Lin, M. H., and Hu, Y. C., "A hybrid model with spherical fuzzy-ahp, pls-sem and ann to predict vaccination intention against COVID-19", Mathematics, Vol. 9, No. 23, 2021, p. 3075.
  13. Priyadarshinee, P., Raut, R. D., Jha, M. K., and Gardas, B. B., "Understanding and predicting the determinants of cloud computing adoption: A two staged hybrid SEM-Neural networks approach", Computers in Human Behavior, Vol. 76, 2017, pp. 341-362
  14. Li, Q., "The Effect of Negative News on Consumer's Value Perception and Risk Perception", Master's thesis, Chungbuk National University, South Korea, 2018.
  15. Rehman, I. H. U., Ahmad, A., Akhter, F., and Aljarallah, A., "A dual-stage SEM-ANN analysis to explore consumer adoption of smart wearable healthcare devices", Journal of Global Information Management (JGIM), Vol. 29, No. 6, 2021, pp. 1-30.
  16. Scott, J. E., and Walczak, S., "Cognitive engagement with a multimedia ERP training tool: Assessing computer self-efficacy and technology acceptance", Information & Management, Vol. 46, No. 4, 2009, pp. 221-232.
  17. Sharma, A., Dwivedi, Y. K., Arya, V., and Siddiqui, M. Q., "Does SMS advertising still have relevance to increase consumer purchase intention? A hybrid PLS-SEM-neural network modelling approach", Computers in Human Behavior, Vol. 124, 2021, p. 106919.
  18. Sharma, S. K., Gaur, A., Saddikuti, V., and Rastogi, A., "Structural equation model (SEM)-neural network (NN) model for predicting quality determinants of e-learning management systems", Behaviour & Information Technology, Vol. 36, No. 10, 2017, pp. 1053-1066.
  19. Sharma, S. K., Joshi, A., and Sharma, H., "A multi-analytical approach to predict the Facebook usage in higher education", Computers in Human Behavior, Vol. 55, 2016, pp. 340-353.
  20. Sharma, S. K., Sharma, H., and Dwivedi, Y. K., "A hybrid SEM-neural network model for predicting determinants of mobile payment services", Information Systems Management, Vol. 36, No. 3, 2019, pp. 243-261.
  21. Sohaib, O., Hussain, W., Asif, M., Ahmad, M., and Mazzara, M., "A PLS-SEM Neural Network Approach for Understanding Cryptocurrency Adoption", IEEE Access, Vol. 8, No. 1, 2020, DOI:10.1109/ACCESS.2019.2960083.
  22. Sternad Zabukovsek, S., Kalinic, Z., Bobek, S., and Tominc, P., "SEM-ANN based research of factors' impact on extended use of ERP systems", Central European Journal of Operations Research, Vol. 27, 2019, pp. 703-735
  23. Tan, G. W. H., Ooi, K. B., Leong, L. Y., and Lin, B., "Predicting the drivers of behavioral intention to use mobile learning: A hybrid SEM-Neural Networks approach", Computers in Human Behavior, Vol. 36, 2014, pp. 198-213
  24. Xu, Y., Zhang, W., Bao, H., Zhang, S., and Xiang, Y., "A SEM-neural network approach to predict customers' intention to purchase battery electric vehicles in china's Zhejiang province", Sustainability, Vol. 11, No. 11, 2019, pp. 3164.