DOI QR코드

DOI QR Code

Rutgers Information Retrieval Evaluation Project on IR Performance on Different Precision Levels

럿거스 정보검색 평가 프로젝트에 관한 연구

  • Lee, Hyuk-Jin (Texas Woman's University) ;
  • Belkin Nicholas J. (Texas Woman's University) ;
  • Krovitz Bob (Texas Woman's University)
  • Published : 2006.06.01

Abstract

The purpose of this study is to investigate what level of difference in precision would be significantly perceived by a human user of an information retrieval system. Not many researches have been conducted with regards to this issue in information retrieval field. Despite the non-significant results, there were several interesting findings in recognizing different levels of precision rates. The correctness of relevance task had little to do with the taken time for the task. In addition, the strong relationship between the subjects' topic familiarity and rate of correct judgments is one of the most interesting results in this study. It turned out that the subjects have more difficulty in a situation they have to judge between the two lists having more non-relevant documents than in a situation they do between the lists haying more relevant documents. Finally, the serious influence from the first top N documents in a list for relevance judgment task has been confirmed.

이 논문의 주요목적은 정보이용자들이 어떤 수준의 정확률 차이에서 유의미하게 차이를 인지하는지를 알아보고자 하는 것이다. 그에 관련한 몇 가지 흥미 있는 결과가 도출되었다. 그 외에 적합성 판정은 이용자의 판정시간과 관계가 없는 것으로 나타났다. 그리고 주제에 대한 이용자의 배경지식과 적합성 판정의 관계가 두드러졌다. 또한, 적합 문서의 숫자가 적었을 때 이용자들은 적합성 판정에 더욱 어려움을 겪었다. 마지막으로, 검색결과리스트중 상위 N 문서의 적합성 판정에 대한 중요성을 확인할 수 있었다.

Keywords

References

  1. ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR' 97), 236-245
  2. NIST Special Publication 500-236. 1995. The Fourth Text REtrieval Conference (TREC-4) : the proceedings of the fourth Text REtrieval Conference (TREC-4) (Gaithersburg, Maryland, November 1-3.)
  3. NIST Special Publication 500-240. 1997. The Sixth Text REtrieval Conference (TREC-6) : the proceedings of the sixth Text REtrieval Conference (TREC-6). (Gaithersburg, Maryland, November 19-21.)
  4. Saracevic, T. 1969. 'Comparative effect of titles, abstracts and full texts on relevance judgments.' Proceedings of the American Society for Information Science, 6, 293-299
  5. Sormunen, E. 2002. 'Liberal Relevance Criteria of TREC - Counting on Negligible Documents? In Beaulieu, M. et al. (Eds.)' Proceedings of the Twenty-Fifth Annual ACM SIGIR Conference on Research and Development in Information Retrieval (Tampere, Finland August 11-15), Special Issue of SIGIR Forum, 36, 324-330
  6. Veerasamy, A. & Heikes, R. 1997. Effectiveness of a graphical display of retrieval results. In Belkin, N.J. et a1., (Eds.) : Proceedings of the 20th Annual International

Cited by

  1. TEMPORARY REMOVAL: Human information seeking behaviour and its impact on personalised information retrieval: An advanced analysis and literature review 2016, https://doi.org/10.1016/j.ijinfomgt.2016.05.018
  2. Relevance: A review of the literature and a framework for thinking on the notion in information science. Part III: Behavior and effects of relevance vol.58, pp.13, 2007, https://doi.org/10.1002/asi.20681