Journal of the Korean Society for information Management (정보관리학회지)

Korean Society for Information Management (한국정보관리학회)
권호 표지
DOI QR코드

DOI QR Code

Multi-faceted Citation Analysis for Quality Assessment of Scholarly Publications

학술논문 품질평가를 위한 다방면 인용분석방식

  • Yang, Ki-Duk (Keimyung University. Department of Library and Information Science) ;
  • Meho, Lokman (American University of Beirut. Department of Political Science and Public Administration)
  • Received : 2011.05.15
  • Accepted : 2011.06.03
  • Published : 2011.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

Despite the widespread use, critics claim that citation analysis has serious limitations in evaluating the research performance of scholars. First, conventional citation analysis methods yield one-dimensional and sometimes misleading evaluation as a result of not taking into account differences in citation quality, not filtering out citation noise such as self-citations, and not considering non-numeric aspects of citations such as language, culture, and time. Second, the citation database coverage of today is disjoint and incomplete, which can result in conflicting quality assessment outcomes across different data sources. This paper discuss the findings from a citation analysis study that measured the impact of scholarly publications based on the data mined from Web of Science, Scopus, and Google Scholar, and briefly describes a work-in-progress prototype system called CiteSearch, which is designed to overcome the weaknesses of existing citation analysis methods with a robust citation-based quality assessment approach.

인용분석은 학자들의 연구실적 평가에 가장 많이 사용되는 방법 중 하나이지만 비평가들은 오늘날의 인용분석 자료와 방법론에 근본적인 문제가 있다고 주장한다. 전통적 인용분석 방식은 인용품질과 인용소음뿐만 아니라 언어, 시간, 문화와 같은 비수치적인 요소들을 고려하지 않아 단순하고 그릇된 평가를 가져올 수 있으며, 적용 범위가 각각 다르고 불완전한 인용 데이터베이스들은 충돌적인 인용분석결과를 초래하기 쉬울 수 있다. 이러한 문제들을 해결하려면 포괄적인 인용데이터를 다 방면과 다 방식으로 분석하는 새로운 인용분석연구가 필요하다. 본 논문은 Web of Science, Scopus와 Google Scholar를 비교 분석한 연구의 결과를 논의하며 기존의 인용분석 방법의 약점을 극복하기 위해 설계한 CiteSearch라는 프로토타입 시스템을 간략하게 설명한다.

Keywords

References

  1. Aksnes, D. W. and R. E. Taxt. 2004. "Peer reviews and bibliometric indicators: A comparative study at a Norwegian university." Research Evaluation, 13(1): 33-41. https://doi.org/10.3152/147154404781776563
  2. Bakkalbasi, N., K. Bauer, J. Glover, and L. Wang. 2006. "Three options for citation tracking: Google Scholar, Scopus and Web of Science." Biomedical Digital Libraries, 7. https://doi.org/10.1186/1742-5581-3-7
  3. Bar-Ilan, J. 2006. "An ego-centric citation analysis of the works of Michael O. Rabin based on multiple citation indexes." Information Processing and Management, 42(6): 1553-1566. https://doi.org/10.1016/j.ipm.2006.03.019
  4. Bar-Ilan, J. 2008. "Which h-index? - A comparison of WoS, Scopus and Google Scholar." Scientometrics, 74(2): 257-271. https://doi.org/10.1007/s11192-008-0216-y
  5. Bar-Ilan, J., M. Levene, and A. Lin. 2007. "Some measures for comparing citation databases." Journal of Informetrics, 1(1): 26-34. https://doi.org/10.1016/j.joi.2006.08.001
  6. Bauer, K. and N. Bakkalbasi. 2005. "An examination of citation counts in a new scholarly communication environment." D-Lib Magazine, 11(9).
  7. Bergstrom, C. T. 2007. "Eigenfactor: Measuring the value and prestige of scholarly journals." College & Research Libraries News, 68(5).
  8. Cronin, B. 1984. The Citation Process: The Role and Significance of Citations in Scientific Communication. London: Taylor Graham.
  9. Egghe, L. 2006. "Theory and practise of the g-index." Scientometrics, 69(1): 131-152. https://doi.org/10.1007/s11192-006-0144-7
  10. Giustini, D. 2006. "Blog posting at UBC Academic Search-Google Scholar Blog." [cited 2011.05.05]. .
  11. Hirsch, J. E. 2005. "An index to quantify an individual's scientific research output." Proceedings of the National Academy of Sciences of the United States of America, 102(46): 16569-16572. https://doi.org/10.1073/pnas.0507655102
  12. Holden, G., G. Rosenberg, and K. Barker. 2005. "Bibliometrics: A potential decision making aid in hiring, reappointment, tenure and promotion decisions." Social Work in Health Care, 41(3-4): 67-92. https://doi.org/10.1300/J010v41n03_03
  13. Holmes, A., and C. Oppenheim. 2001. "Use of citation analysis to predict the outcome of the 2001 Research Assessment Exercise for Unit of Assessment (UoA) 61: Library and Information Management." Information Research, 6(2).
  14. Jacso, P. 2005. "As we may search-comparison of major features of the Web of Science, Scopus, and Google Scholar citation-based and citation-enhanced databases." Current Science, 89(9): 1537-1547.
  15. Kleinberg, J. 1998. "Authoritative sources in a hyperlinked environment." Proceeding of the 9th ACM-SIAM Symposium on Discrete Algorithms, 668-677.
  16. Kousha, K., and M. Thelwall. 2006. "Sources of Google Scholar citations outside the Science Citation Index: A comparison between four science disciplines." In Book of Abstracts, 9th International Science & Technology Indicators Conference, Leuven, Belgium: 72-73.
  17. Lewison, G. 2001. "Evaluation of books as research outputs in history of medicine." Research Evaluation, 10(2): 89-95. https://doi.org/10.3152/147154401781777051
  18. MacRoberts, M. H., and B. R. MacRoberts. 1996. "Problems of citation analysis." Scientometrics, 36(3): 435-444. https://doi.org/10.1007/BF02129604
  19. Marchionini, G., P. Solomon, C. Davis, and T. Russell. 2006. "Information and library science MPACT: A preliminary analysis." Library & Information Science Research, 28(4).
  20. Martin, B. R. 1996. "The use of multiple indicators in the assessment of basic research." Scientometrics, 36(3): 343-362. https://doi.org/10.1007/BF02129599
  21. Meho, L. I., and K. Yang. 2007. "Impact of data sources on citation counts and rankings of LIS faculty: Web of Science vs. Scopus and Google Scholar." Journal of the American Society for Information Science and Technology, 58(13): 2105-2125. https://doi.org/10.1002/asi.20677
  22. Moed, H. F. 2005. Citation Analysis in Research Evaluation. Springer, Berlin.
  23. Nisonger, T. E. 2004. "Citation autobiography: An investigation of ISI database coverage in determining author citedness." College & Research Libraries, 65(2): 152-163. https://doi.org/10.5860/crl.65.2.152
  24. Norris, M., and C. Oppenheim. 2007. "Comparing alternatives to the Web of Science for coverage of the social sciences' literature." Journal of Informetrics, 1(2): 161-169. https://doi.org/10.1016/j.joi.2006.12.001
  25. Noruzi, A. 2005. "Google Scholar: The New Generation of Citation Indexes." Libri, 55(4): 170-180. https://doi.org/10.1515/LIBR.2005.170
  26. Seglen, P. O. 1998. "Citation rates and journal impact factors are not suitable for evaluation of research." Acta Orthopaedica Scandinavica, 69(3): 224-229. https://doi.org/10.3109/17453679809000920
  27. van Raan, A. F. J. 1996. "Advanced bibliometric methods as quantitative core of peer-review based evaluation and foresight exercises." Scientometrics, 36(3): 397-420. https://doi.org/10.1007/BF02129602
  28. Vaughan, L., and D. Shaw. 2008. "A new look at evidence of scholarly citation in citation indexes and from web sources." Scientometrics, 74(2): 317-330. https://doi.org/10.1007/s11192-008-0220-2
  29. West, J. D., C. T. Bergstrom, and T. C. Bergstrom. 2010. "Big Macs and Eigenfactor Scores: Don't Let Correlation Coefficients Fool You." [online]. [cited 2011.05.05]. .

Cited by

  1. Bibliometric Approach to Research Assessment: Publication Count, Citation Count, & Author Rank vol.1, pp.1, 2013, https://doi.org/10.1633/JISTaP.2013.1.1.2