Quality Dimensions Affecting the Effectiveness of a Semantic-Web Search Engine

검색 효과성에 영향을 미치는 시맨틱웹 검색시스템 품질요인에 관한 연구

  • Published : 2009.03.31

Abstract

This paper empirically examines factors that potentially influence the success of a Web-based semantic search engine. A research model has been proposed that shows the impact of quality-related factors upon the effectiveness of a semantic search engine, based on DeLone and McLean's(2003) information systems success model. An empirical study has been conducted to test hypotheses formulated around the research model, and statistical methods were applied to analyze gathered data and draw conclusions. Implications for academics and practitioners are offered based on the findings of the study. The proposed model includes three quality dimensions of a Web-based semantic search engine-namely, information quality, system quality and service quality. These three dimensions each have measures designed to collectively assess the respective dimension. The model is intended to examine the relationship between measures of these quality dimensions and measures of two dependent constructs, including individuals' net benefit and user satisfaction. Individuals' net benefit was measured by the extent to which the user's information needs were adequately met, whereas user satisfaction was measured by a combination of the perceived satisfaction with search results and the perceived satisfaction with the overall system. A total of 23 hypotheses have been formulated around the model, and a questionnaire survey has been conducted using a functional semantic search website created by KT and Hakia, so as to collect data to validate the model. Copies of a questionnaire form were handed out in person to 160 research associates and employees working in the area of designing and developing semantic search engines. Those who received the form, 148 respondents returned valid responses. The survey form asked respondents to use the given website to answer questions concerning the system. The results of the empirical study have indicated that, of the three quality dimensions, information quality was found to have the strongest association with the effectiveness of a Web-based semantic search engine. This finding is consistent with the observation in the literature that the aspects of the information quality should serve as a basis for evaluating the search outcomes from a semantic search engine. Measures under the information quality dimension that have a positive effect on informational gratification and user satisfaction were found to be recall and currency. Under the system quality dimension, response time and interactivity, were positively related to informational gratification. On the other hand, only one measure under the service quality dimension, reliability was found to have a positive relationship with user satisfaction. The results were based on the seven hypotheses that have been accepted. One may wonder why 15 out of the 23 hypotheses have been rejected and question the theoretical soundness of the model. However, the correlations between independent variables and dependent variables came out to be fairly high. This suggests that the structural equation model yielded results inconsistent with those of coefficient analysis, because the structural equation model intends to examine the relationship among independent variables as well as the relationship between independent variables and dependent variables. The findings offer some useful implications for owners of a semantic search engine, as far as the design and maintenance of the website is concerned. First, the system should be designed to respond to the user's query as fast as possible. Also it should be designed to support the search process by recommending, revising, and choosing a search query, so as to maximize users' interactions with the system. Second, the system should present search results with maximum recall and currency to effectively meet the users' expectations. Third, it should be capable of providing online services in a reliable and trustworthy manner. Finally, effective increase in user satisfaction requires the improvement of quality factors associated with a semantic search engine, which would in turn help increase the informational gratification for users. The proposed model can serve as a useful framework for measuring the success of a Web-based semantic search engine. Applying the search engine success framework to the measurement of search engine effectiveness has the potential to provide an outline of what areas of a semantic search engine needs improvement, in order to better meet information needs of users. Further research will be needed to make this idea a reality.

Keywords

References

  1. Albertoni, R., Bertone, A., and Martino,M.D., "Semantic Web and Information Visualization," Proceedings of the $1^{st}$ ltalian Semantic Web Workshop, Ancona, Italy, 2004
  2. Bangyong L., Jie, T., and Juanzi, L., ''Association Search in Semantic Web: Search + Inference," Proceedings of WINW 2005, Chiba,Japan, 2005
  3. Belkin, N., "Anomalous states of knowledge as a basis for information retrieval," ,Cnadian Journal of Information Science, Vol. 5, 1980, pp.133-143
  4. Bonino, D. and Corno, F., "Ontology Driven Semantic Search," WSEAS Transaction on Information Science and Application, Issue 6, Vol. 1, 2004, pp. 1597-1605
  5. Colucci, S., Noia, T.D., and Sciascio, E.D.,"A semantic-based fully visual application for matchmaking and query refinement in B2C e-marketplaces," Proceedings of ICEC '06, Freericton, Canada, 2006 https://doi.org/10.1145/1151454.1151489
  6. D' Ambra, J. and Rice, R.E., "Emerging factors in user evaluation of the World Wide Web," Information and Management, Vol. 38, No. 6, 2001, pp. 373-384 https://doi.org/10.1016/S0378-7206(00)00077-X
  7. DeLone, W.H. and E.R. McLean, "Information systems success: the quest for the dependent variable,'’ Information Systems Research, Vol. 3, No. 1, 1992, pp. 60-92 https://doi.org/10.1287/isre.3.1.60
  8. DeLone, W.H and McLean, E.R, ''The Delone and McLean Model of Information Systems Success: A Ten-Year Update," Journal of Management Information Systems, Vol. 19, No.4, 2003, pp. 9-30
  9. Dong-il Han, Sang-Bum Ha, Ho-Jun Choi, "Fox Service: An Implementation Case of Ontology-based Search Agent," Journal of Korea Information Science Society(in Korean), Vol. 24, No. 4, 2006, pp. 75-81 https://doi.org/10.1109/MDM.2006.92
  10. Dong-il Han, Eunjoo Lee, "Exploring the Costs and Benefits of Internet Search from the Online Customers' Perspective: Implications for the Consumer Adoption of the Semantic Web-Based Search Engines," Management Education Review(in Korean), Vol. 11, No. 1, 2007, pp. 101-123
  11. Feldman, S. and Sherman C, "The High Cost of Not Finding Information," An IDC White Paper, 2001
  12. Flavian, C and Gurrea, R., "The choice of digital newspapers: influence of reader goals and user experience," Internet Research, Vol. 16, No. 3, 2006, pp. 231-247 https://doi.org/10.1108/10662240610673673
  13. Fornell, C and Larcker, D, "Evaluating structural equation model with unobservable and measurement error, " Journal of Marketing Research, Vol. 18, 1981, pp. 39-50 https://doi.org/10.2307/3151312
  14. Gehrke, D. and Turban, D., "Determinants of successful website design: relative importance and recommendations for effectiveness," Proceedings of the 32th Hawaii International Conference in System Sciences,1999, pp. 1-8 https://doi.org/10.1109/HICSS.1999.772943
  15. Guha, R. McCool, R. and Miller, E.,"Semantic Search," Proceedings of WWW 2003, May 20-24, ACM Press, Budapest, Hungary, 2003
  16. Hirsh, S. and Dinkelacker, J., Seeking information in order to produce information: an empirical study at Hewlett Packard Labs," Journal of the American Society for Information Science and Technology, 2004 https://doi.org/10.1002/asi.20024
  17. Liu, C and Arnett, K.P., "Exploring the factors associated with Web site success in the context of electronic commerce," Information and Management, Vol. 38, 2000, pp. 23-33 https://doi.org/10.1016/S0378-7206(00)00049-5
  18. Makela, E., Hyvonen, E., and Saarela, S., "Ontogator - A Semantic View-Based Search Engine Services for Web Applications Proceedings of the 5th International Semantic Web Conference 2006, ISWC 2006, Athens, GA, U5A, 2006, pp. 847-860
  19. Molla, A. and Licker, P.5., "E-commerce systems success: An attempt to extend and respecify the DeLone and McLean model of 15 success," Journal of Electronic Commerce Research, Vol. 2, No. 4, 2001, pp. 1-11
  20. Morville, P., "Ambient Findability: What We Find Changes Who We become," O'Reilly, www.oreilly.com, 2005
  21. Negash, S.,Ryan, T., and Igbaria, M, "Quality and effectiveness in Web-based customer support systems," Information and Management, Vol. 2029, 2002, pp.1-12 https://doi.org/10.1016/S0378-7206(02)00101-5
  22. Oddy, R., "Information retrieval through man-machine dialogue," Journal of Documentations, Vol. 33, 1977, pp. 1-14 https://doi.org/10.1108/eb026631
  23. Parasurman,A., Zeithaml, V.A., and Berry,L.L., "SERVQUAL: A Multiple-Item Scale for Measuring effectiveness," MIS Quarterly, Vol. 19, No. 2, 1988, pp.173-187 https://doi.org/10.2307/249687
  24. Pitt, L.F., Watson, R.T., and Kavan, CB., "Service quality: a measure of information systems effectiveness," MIS Quarterly, Vol. 19, No. 2, 1995, pp. 173-187 https://doi.org/10.2307/249687
  25. Richa, Schwabe, D., and Aragao, M.P., "A Hybrid Approach for Searching in the Semantic Web," Proceedings of WWW 2004, New York, USA, 2004
  26. Seddon,P.B. and Kiew, M.Y., "A partial test and development of DeLone and McLean model of IS success," Australian Journal of Information Systems, Vol. 4, No. 1, 1996, pp. 90-104
  27. Sun-Young Heo Eun-Gyung Kim, "A Study on Ontology-Based Semantic Search System," Proceedings of the 21" Korea Information Processing Society Conference, June 2007, pp. 263-466
  28. Sure, Y. and Iosif, V.,First Results of a Semantic Web Technologies Evaluation," DOA'02, 2002
  29. TimBerners-Lee,J.H. and O. Lassila. "The Semantic Web," Scientific American, 2001
  30. Wissbrock, F., Information Need Assessment in Information Retrieval; Beyond Lists and Queries," Proceedings of the 27th,German Conference on Artificial Intelligence, University of Ulm, Germany, 2004