국가연구개발사업의 과학적 성과분석을 위한 새로운 계량지표 개발에 관한 연구

Developing Bibliometric Indicators for Analysis & Evaluation of National R&D Programs

최근 연구개발 투자규모가 급속히 증가함에 따라 정부는 R&D사업의 투자효율성을 높이기 위한 다양한 정책을 구사하고 있다. 구체적으로 R&D사업에 대한 성과평가를 강화하고 평가결과를 예산의 조정 및 배분에 반영하는 등 성과중심의 평가체계를 정착시키기 위해 노력하고 있다. 이처럼 국가연구개발사업에 대한 평가가 성과중심으로 체계화되기 위해서는 R&D 성과물을 과학적 객관적으로 분석할 수 있는 성과지표의 개발이 필요하다. 이에 본 연구에서는 전 부처 연구개발사업에 공통으로 적용할 수 있는 성과지표인 SCI논문지표를 바탕으로 교육과학기술부 3대 연구개발사업에 대한 구체적인 사례분석을 실시하여 이들 사업이 우리나라 전체 R&D에서 차지하는 질적 양적 위상을 파악하고자 하였다. 이러한 분석을 위해 SCI 관련 기존지표와 더불어 4개의 신규지표를 소개하였고 신규지표들의 타당성 조사를 위해 모의실험을 실시하였다. 그 결과 보완된 순위보정영향력지수(ordinal rank normalized impact factor, omIF)가 분야간 비교 시 가장 적절한 성과지표인 것으로 분석되었다.

Science and technology (S&T) is one of the most important elements in a nation's competitiveness. In an effort to strengthen their national competitiveness, all countries are focusing on upgrading the level of eir S&T. With these factors in mind, Korea has increased its support of national research and development (R&D). In recent years, this added support has resulted in an increased interest in the effectiveness of R&D. We have made continuous efforts to enhance the accountability and effectiveness of R&D by strengthening performance evaluation and considering R&D evaluation results during the budget review (appropriation) process. In order to change to a performance based system, we need to develop objective and scientific indicators to measure and evaluate the quality of the research performance of R&D programs. One of the primary research outcomes is publications. The impact factor of publications is widely used to evaluate overall journal quality and the quality of the papers published therein. However, the use of impact factors has been criticised because they can vary greatly when works from different subject areas are compared. In order to overcome this limitation, we have developed three kinds of qualitative indicators, which are functions of the impact factor. Two of these qualitative indicators, Modified Rank Normalized Impact Factor and Ordinal Rank Normalized Impact Factor, are based on order statistics (rank) for all journals from a specific specialty. The third qualitative indicator, Relative Field Impact Factor, uses the average impact factor of all journals within a subject category. We also suggest a quantitative indicator, Percentage of Contribution. In this study, we suggest 4 indicators and use them to evaluate the performance of outcomes from three R&D programs supported by the Ministry of Education, Science & Technology. We also perform a simulation study to verify the effectiveness of the proposed indicators. It can be shown that the proposed Ordinal Rank Normalized Impact Factor is the most reliable and effective indicator for comparing research performance across subject categories. However, we recommend using previous indicators in combination with the proposed indicators in this study for the research evaluation of R&D programs.