Journal of the Korean BIBLIA Society for library and Information Science (한국비블리아학회지)

Korean Biblia Society for Library and Information Science (한국비블리아학회)
권호 표지
DOI QR코드

DOI QR Code

A Study on the Peer Review Activity of Domestic Researchers in International Journals: Focused on Publons

국내 연구자의 국제 학술지 동료 심사 활동에 관한 연구 - Publons를 중심으로 -

  • 조재인 (인천대학교 문헌정보학과)
  • Received : 2022.02.07
  • Accepted : 2022.02.28
  • Published : 2022.03.30
Download PDF
⟨ Previous Next ⟩

Abstract

As a new academic publication model is attempted to improve the transparency, efficiency, and speed of scientific knowledge production and distribution, the open peer review platform for verification and openness of peer review history is also activated. Publons is a global platform for tracking, validating, disclosing, and recognizing the peer-reviewed histories of more than 3 million researchers worldwide. This study analyzed the review activities of 579 researchers from domestic universities who are actively reviewing international journals through Publons. As a result of the analysis, first, researchers from domestic universities who actively review international academic journals were found to be in the fields of medicine and electrical and electronics, and in most fields, assistant professors or higher with high WOS indexed research papers are participating. Second, there was a long-tail phenomenon in which a small number of reviewers with extremely high number of review papers existed in all academic fields, and there was no significant difference in the number of review papers and review report length depending on the nationality, academic status, and age of the reviewers. Lastly, although there was a weak correlation between the amount of papers reviewed by reviewers and the number of published papers, it was found that researchers with an extremely large number of reviews do not necessarily produce as many research papers.

과학지식 생산 유통의 투명성, 효율성, 신속성을 제고하기 위하여 새로운 학술 출판 모델이 시도되면서, 동료 심사 이력의 검증과 개방을 위한 피어 리뷰 플렛폼도 활성화되고 있다. Publons는 3백만명 이상의 전 세계 연구자들의 동료 심사 이력을 추적해 검증하고 공개할 뿐 아니라 업적으로 인정하기 위한 글로벌 플렛폼이다. 본 연구는 Publons를 활용해 국제 학술지 심사 위원으로 활동하는 국내 대학 소속 연구자 579명을 선별해 동료 심사 활동을 분석하였다. 분석 결과 첫 번째, 국제 학술지에서 활발한 심사 활동을 하는 국내 연구자는 공학, 의약학 분야에 많으며, 대부분의 분야에서 높은 국제 학술지 출판 실적을 보유한 조교수급 이상의 교수들인 것으로 나타났다. 두 번째, 모든 학문 분야에서 극단적으로 높은 심사 논문수를 보이는 소수의 심사자가 존재하였으며, 심사자의 신분, 나이, 국적에 따라서 심사 논문수와 심사 보고서 길이에 유의미한 차이는 존재하지 않았다. 마지막으로 심사자의 심사 논문수와 SCI 급 출판 논문수 사이에는 약한 상관성(r = 0.311)이 존재했지만, 많은 연구실적을 보유한 연구자가 그만큼 많은 국제 학술지 심사 활동을 한다고 말하기 어려운 것으로 분석되었다.

Keywords

Acknowledgement

본 논문은 2021년도 인천대학교 자체연구비 지원으로 수행되었음.

References

  1. Black, N., Van Rooyen, S., Godlee, F., Smith, R., & Evans, S. (1998). What makes a good reviewer and a good review for a general medical journal?. Journal of the American Medical Association, 280, 231-233. https://doi.org/10.1001/jama.280.3.231
  2. Bornmann, L. & Daniel, H. D. (2008). Selecting manuscripts for a high-impact journal through peer review: a citation analysis of communications that were accepted by angewandte chemie international edition, or rejected but published elsewhere. Journal of the American Society for Information Science and Technology, 59(11), 1841-1852. https://doi.org/10.1002/asi.20901
  3. Clarivate (2018, February 26). It's not the size that matters. Available: https://clarivate.com/blog/its-not-the-size-that-matters/
  4. Clarivate Analytics (2020). Publons Reviewer Connect - ScholarOne. Available: https://static1.squarespace.com/static/576fcda2e4fcb5ab5152b4d8/t/5e7dceb25ae8b93895717126/1585303221283/Reviewer+Connect+in+ScholarOne+QRG_Final.pdf
  5. cOAlition S (2019). Guidance on the Implementation of Plan S. Available: https://www.coalition-s.org/wp-content/uploads/2020/09/271118_cOAlitionS_Guidanc e_annotated.pdf
  6. Dunne, M. (2019). Computer Generated Papers as a New Challenge to Peer Review. Master of Science in Technical Communication, Montana Tech. Available: https://digitalcommons.mtech.edu/cgi/viewcontent.cgi?article=1205&context=grad_rsch
  7. Evans, A. T., McNutt, R. A., Fletcher, S. W., & Fletcher, R. H. (1993). The characteristics of peer reviewers who produce good-quality reviews. Journal of General Internal Medicine, 8(8), 422-428. https://doi.org/10.1007/BF02599618
  8. Falkenberg, L. J. & Soranno, P. A. (2018). Reviewing reviews: an evaluation of peer reviews of journal article submissions, Limnology and Oceanography Bulletin, 27(1), 1-5. https://doi.org/10.1002/lob.10217
  9. Gasparyan, A. Y. & Kitas, G. D. (2012). Best peer reviewers and the quality of peer review in biomedical journals, Croatian Medical Journal, 53(4), 386-389. https://doi.org/10.3325/cmj.2012.53.386
  10. Glonti, K., Boutron, I., Moher, D., & Hren, D. (2019). Journal editors' perspectives on the roles and tasks of peer reviewers in biomedical journals: a qualitative study, BMJ Open, 9(11), e033421. http://dx.doi.org/10.1136/bmjopen-2019-033421
  11. Goldstein, S. (2019). Publons peer evaluation metrics are not reliable measures of quality or impact. Evidence Based Library And Information Practice, 14(3), 153-155. https://doi.org/10.18438/eblip29579
  12. Kliewer, M. A., Freed, K. S., DeLong, D. M., Pickhardt, P. J., & Provenzale, J. M. (2005). Reviewing the reviewers: comparison of review quality and reviewer characteristics at the American journal of roentgenology. American Journal of Roentgenology, 184(6), 1731-1735. https://doi.org/10.2214/ajr.184.6.01841731
  13. Ortega, J. L. (2017). Are peer-review activities related to reviewer bibliometric performance? a scientometric analysis of publons. Scientometrics, 112, 947-962. https://doi.org/10.1007/s11192-017-2399-6
  14. Ortega, J. L. (2019). Exploratory analysis of publons metrics and their relationship with bibliometric and altmetric impact. Aslib Journal of Information Management, 71(1), 124-136. https://doi.org/10.1108/AJIM-06-2018-0153
  15. Patterson, M. & Harris, S. (2009). The relationship between reviewers' quality-scores and number of citations for papers published in the journal Physics in Medicine and Biology from 2003-2005. Scientometrics, 80(2), 343-349. https://doi.org/10.1007/s11192-008-2064-1
  16. Pautasso, M. & Schafer, H. (2009). Peer review delay and selectivity in ecology journals. Scientometrics, 84(2), 307-315. https://doi.org/10.1007/s11192-009-0105-z
  17. Reilly, L. (2021). What are Scored Publications?. Publons. Availabe: https://publons.freshdesk.com/support/solutions/articles/12000081238-what-are-scoredpublications
  18. Ross-Hellauer, T., Deppe, A., & Schmidt, B. (2017). Survey on open peer review: attitudes and experience amongst editors, authors and reviewers. PLoS ONE 12(12), e0189311. https://doi.org/10.1371/journal.pone.0189311
  19. Schriger, D. L., Kadera, S. P., & Von Elm, E. (2016). Are reviewers' scores influenced by citations to their own work? an analysis of submitted manuscripts and peer reviewer reports. Annuals of Emergency Medicine, 67(3), 401-406. https://doi.org/10.1016/j.annemergmed.2015.09.003
  20. Swiontkowski, M. (2019). Publons: the next step in reviewer recognition. Journal of Bone and Joint Surgery, 101(13), 1137. https://doi.org/10.2106/JBJS.19.00481
  21. Teixeira da Silva, J. A. (2020). Are negative reviews, predatory reviewers or failed peer review rewarded at Publons?. International Orthopaedics (SICOT), 44, 2193-2194. https://doi.org/10.1007/s00264-020-04587-w
  22. Thomas, P. R. & Watkins, D. S. (1998). Institutional research rankings via bibliometric analysis and direct peer review: a comparative case study with policy implications. Scientometrics, 41(3), 335-355. https://doi.org/10.1007/BF02459050
  23. Ule, J. (2020). Open access, open data and peer review. Genome Biol, 21, 86. https://doi.org/10.1186/s13059-020-02005-3
  24. Wiechert, K., Chapman, J. R., & Wang, J. C. (2018). Recognizing our experts: global spine journal partners with publons to establish reviewers' platform. Global Spine Journal, 8(3), 217. https://doi.org/10.1177/2192568218773367
  25. Wilkinson, J. & Down, P. (2018). Publons: releasing the untapped power of peer review for universities. Insights, 31, 20. http://doi.org/10.1629/uksg.407
  26. Wolfram, D., Wang, P., Hembree, A., & Park, H. (2020). Open peer review: promoting transparency in open science. Scientometrics, 125, 1033-1051. http://doi.org/10.1629/uksg.407
  27. Yankauer, A. (1990). Who are the peer reviewers and how much do they review?. Journal of the American Medical Association, 263(10), 1338-1340. https://doi.org/10.1001/jama.1990.03440100042005
  28. Zong, Q., Fan, L., Xie, Y., & Huang, J. (2020). The relationship of polarity of post-publication peer review to citation count: evidence from publons. Online Information Review, 44(3), 583-602. https://doi.org/10.1108/OIR-01-2019-0027
  29. Sato, S. (2014). New issues concerning peer review. Current Awareness, 321. https://current.ndl.go.jp/ca1829
  30. Sato, S. (2016). Peer review problems and countermeasures. Information Science and Technology, 66(3), 115-121.