Journal of the Korean Society for information Management (정보관리학회지)

Korean Society for Information Management (한국정보관리학회)
권호 표지
DOI QR코드

DOI QR Code

Users' Relevance Criteria in Universal Search in Korea : An Exploratory Study

통합 검색 환경에서 이용자 적합성 판단 기준에 관한 탐색적 연구

  • Received : 2012.05.17
  • Accepted : 2012.06.14
  • Published : 2012.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

This study is an exploratory research on the user relevance criteria in Korean search service environments that provide integrated search results. Data were collected from 10 participants using a semi-structured interview technique. The participants conducted a web search using integrated search services, such as Naver or Daum on a self-selected topic. They were asked to judge the relevance of retrieved documents and to report their relevance criteria. As a result, the research indicated 8 user-defined relevance and non-relevance criteria. The research shows that specificity and richness are the two most important criteria yet, the user's relevance criteria have not changed much despite the change in search environment.

본 연구는 한국 통합 검색 환경에서의 이용자 적합성 판단 기준에 관한 탐색적 연구이다. 이를 위해 10명의 참가자들을 대상으로 반구조화(semi-structured) 인터뷰를 수행하여 데이터를 수집하였다. 참가자들은 네이버, 다음 등과 같은 통합 검색 환경에서 본인들이 관심 있거나 필요로 하는 다양한 검색을 수행하고, 그 과정에서 문서가 적합한지와 그 판단 기준에 대해 기술하였다. 연구 결과 8개의 적합성 판단 기준과 비적합성 판단 기준, 그리고 검색 환경이 변화하여도 이용자가 적합성을 판단하는 기준들이 크게 변화하지는 않지만 데이터 증가와 이용자 요구의 고도화로 특수성과 구체성이 중요한 적합성 판단 기준으로 부각되는 점을 발견하였다.

Keywords

References

  1. 박소연, 이준호 (2008). 주요 검색 포탈들의 통합 검색 서비스 비교 평가. 한국도서관․정보학회지, 39, 265-278.(Park, Soyeon, & Lee, Joon-Ho. (2008). Comparative Evaluation of the Unified Search Services Provided by Major Korean Search Portals. Journal of Korean Library and Information Science Society, 39, 265-278.)
  2. Barry, C. L. (1994). User-defined relevance criteria: An exploratory study. Journal of the American Society for Information Science, 45, 149-159. https://doi.org/10.1002/(SICI)1097-4571(199404)45:3<149::AID-ASI5>3.0.CO;2-J
  3. Barry, C. L., & Schamber, L. (1998). Users' criteria for relevance evaluation: A cross-situational comparison. Information Processing & Management, 34, 219-236. https://doi.org/10.1016/S0306-4573(97)00078-2
  4. Bateman, J. (1998). Changes in relevance criteria: A longitudinal study. In: ASIS Proceedings. 1998, 23-32.
  5. Bilal, D. (2000). Children's use of the Yahooligans! web search engine: I. Cognitive, physical and affective behaviors on fact-based search tasks. Journal of the American Society for Information Science, 51, 646-65. https://doi.org/10.1002/(SICI)1097-4571(2000)51:7<646::AID-ASI7>3.0.CO;2-A
  6. Borlund, P. (2003). The concept of relevance in IR. Journal of the American Society for Information Science and Technology, 54, 913-925. https://doi.org/10.1002/asi.10286
  7. Choi, Y., & Rasmussen, E. M. (2002). Users' relevance criteria in image retrieval in American history. Information Processing & Management, 38, 695-726. https://doi.org/10.1016/S0306-4573(01)00059-0
  8. Cooper, W. S. (1971). A definition of relevance for information retrieval. Information Storage and Retrieval, 7, 19-37. https://doi.org/10.1016/0020-0271(71)90024-6
  9. Cosijn, E., & Ingwersen, P. (2000). Dimensions of relevance. Information Processing & Management, 36, 533-550. https://doi.org/10.1016/S0306-4573(99)00072-2
  10. Cuadra, C. A., & Katter, R.V. (1967a). Experimental studies of relevance judgments: Final report. Volume 1: Project summary (TM-3520/001/00). Santa Monica, CA: System Development Corp.
  11. Cuadra, C. A., & Katter, R.V. (1967b). Opening the black box of "relevance". Journal of Documentation, 23, 291-303. https://doi.org/10.1108/eb026436
  12. Fitzgerald, M. A., & Galloway, C. (2001). Relevance judging, evaluation, and decision making in virtual library: A descriptive study. Journal of the American Society for Information Science and Technology, 52, 989-1010. https://doi.org/10.1002/asi.1152
  13. Freund, L. (2008). Exploiting task-document relations in support of information retrieval in the workplace. PhD thesis, University of Toronto.
  14. Froehlich, T. J. (1994). Relevance reconsidered: Towards an agenda for the 21st century: Introduction to special topic issue on relevance research. Journal of the American Society for Information Science, 45, 124-134. https://doi.org/10.1002/(SICI)1097-4571(199404)45:3<124::AID-ASI2>3.0.CO;2-8
  15. Greisdorf, H. (2003). Relevance thresholds: A multi-stage predictive model of how users evaluate information. Information Processing & Management, 39, 403-423. https://doi.org/10.1016/S0306-4573(02)00032-8
  16. Hirsh, S. G. (1999). Children's relevance criteria and information seeking on electronic resources. Journal of the American Society for Information Science, 50, 1265-1283. https://doi.org/10.1002/(SICI)1097-4571(1999)50:14<1265::AID-ASI2>3.0.CO;2-E
  17. Hjørland, B., & Christensen, F. S. (2002). Work tasks and socio-cognitive relevance: A specific example.
  18. Krippendorf, K. (1980). Content analysis: An introduction to its methodology. Beverly Hills, CA: Sage.
  19. Maglaughlin, K. L., & Sonnewald, H. (2002). User perspective on relevance criteria: A comparison among relevant, partially relevant, and not-relevant. Journal of the American Society for Information Science and Technology, 53, 327-342. https://doi.org/10.1002/asi.10049
  20. Mizzaro, S. (1997). Relevance: The whole history. Journal of the American Society for Information Science, 48, 810-832. https://doi.org/10.1002/(SICI)1097-4571(199709)48:9<810::AID-ASI6>3.0.CO;2-U
  21. Moon, I., & Woyke, E. (2006, January 30). NHN: The little search engine that could. Businessweek.
  22. Park, T. K. (1993). The nature of relevance in information retrieval: An empirical study. Library Quarterly, 63, 318-351. https://doi.org/10.1086/602592
  23. Park, T. K. (1994). Toward a theory of user-based relevance: A call for a new paradigm of inquiry. Journal of the American Society for Information Science, 45, 135-141. https://doi.org/10.1002/(SICI)1097-4571(199404)45:3<135::AID-ASI3>3.0.CO;2-1
  24. Park, H. (1997). Relevance of science information: Origins and dimensions of relevance and their implications to information retrieval. Information Processing & Management, 33, 339-352. https://doi.org/10.1016/S0306-4573(96)00072-6
  25. Rees, A. M., & Schultz, D. G. (1967). A field experiment approach to the study of relevance assessments in relation to document searching, 2. Cleveland, OH: Center for Documentation and Communication Research, School of Library Science, Case Western Reserve University.
  26. Saracevic, T. (1975). Relevance: A review of and a framework for the thinking on the notion in information science. Journal of the American Society for Information Science, 26, 321-343. https://doi.org/10.1002/asi.4630260604
  27. Saracevic, T. (2007). Relevance: A review of the literature and a framework for thinking on the notion in information science. Part III: Behavior and effects of relevance. Journal of the American Society for Information Science and Technology, 58, 2126-2144. https://doi.org/10.1002/asi.20681
  28. Savolainen, R., & Kari, J. (2006). User-defined relevance criteria in web searching. Journal of Documentation, 62, 685-707. https://doi.org/10.1108/00220410610714921
  29. Schamber, L. (1991). Users' criteria for evaluation in a multimedia environment. Proceedings of the American Society for Information Science, Washington, DC, 126-133. Medford, NJ: Learned Information, Inc.
  30. Schamber, L. (1994). Relevance and information behavior. In M.E. Williams (Ed.), Annual review of information science and technology (ARIST), 3-48. Medford, NJ: Learned Information, Inc.
  31. Schamber, L., & Bateman, J. (1996). User criteria in relevance evaluation: Toward development of a measurement scale. Proceedings of the 59th Annual Meeting of the American Society for Information Science, 33, 218-225. Medford, NJ: InformationToday.
  32. Schamber, L., Eisenberg, M. B., & Nilan, M. S. (1990). A re-examination of relevance: Toward a dynamic, situational definition. Information Processing & Management, 26, 755-775. https://doi.org/10.1016/0306-4573(90)90050-C
  33. Taylor, A. R., Cool, C., Belkin, N. J., & Amadio, W. J. (2007). Relationships between categories of relevance criteria and stage in task completion. Information Processing & Management, 43, 1071-1084. https://doi.org/10.1016/j.ipm.2006.09.008
  34. Wang, P., & Soergel, D. (1998). A cognitive model of document use during a research project: Study I. Document selection. Journal of the American Society for Information Science, 49, 115-133. https://doi.org/10.1002/(SICI)1097-4571(199802)49:2<115::AID-ASI3>3.0.CO;2-T
  35. Wang, P., & White, M. D. (1999). A cognitive model of document use during a research project: Study II. Decisions at the reading and citing stages. Journal of the American Society for Information Science, 50, 98-144. https://doi.org/10.1002/(SICI)1097-4571(1999)50:2<98::AID-ASI2>3.0.CO;2-L
  36. Xu, Y., & Chen, Z. (2006). Relevance judgmen t- What do information consumers consider beyond topicality? Journal of the American Society for Information Science and Technology, 57, 961-973. https://doi.org/10.1002/asi.20361
  37. Yang, M., & Marchionini, G. (2004). Exploring users' video relevance criteria - A pilot study. Proceedings of the 67th annual meeting of the American Society for Information Science and Technology (ASIST '04), 229-238. Medford, NJ: Information Today.