• 지식경영연구
• /
• 제10권2호
• /
• pp.65-84
• /
• 2009

The purpose of this study is to develop Meta Evaluation Indicators for Defense R&D Programs in Korea. At first, the four components of this meta evaluation model were designed, which are evaluation context, evaluation input, evaluation process and evaluation outcome. And fifty two indicators for this meta evaluation were developed by experts who performed evaluations for Defense R&D Programs. The reliability of components and items was verified by Cronbach's ${\alpha}$ coefficient. It was over 0.6 in all areas. The validity of components and items was verified by Factor Analysis. Analytic Hierarchy Process method was used in assigning the evaluation weight. The survey of twenty two evaluators participated in Defense R&D Programs showed that the Consistency Ratio was under 0.1 in evaluation components and items. In this study, an objective and reasonable set of Defense R&D Meta Evaluation Indicators was developed to increase the responsibility of Defense R&D Programs and improve the quality of evaluation results.

• 공학교육연구
• /
• 제16권3호
• /
• pp.28-41
• /
• 2013

The purpose of this study was to develop the Meta-Evaluation Criteria which can comprehensively and systematically evaluate and improve the Accreditation Evaluation System for Engineering Education. This will provide useful implications for establishing value, direction, and improvement remedy for Engineering Educations Accreditation System of Korea. In order to achieve the purpose of this study, Meta-Evaluation Criteria was developed to comprehensively and systematically assess and analyze the Accreditation Evaluation System of Engineering Education. The research methodology used to study these subjects: literature review, interviews with experts, Delphi survey (three times), survey, analysis of narrative opinion and secondary source. The results and conclusions of this study can be summarized as follows: First, the final Meta-Evaluation Criteria were developed so as to comprehensively and systematically assess and analyze the Accreditation Evaluation System of Engineering Education. The criteria's validity and reliability were also identified. Specific details are as follows: In a draft plan of the Meta-Evaluation Criteria, Meta-Evaluation concept was defined as evaluation on the whole range of Accreditation Evaluation System of Engineering Education. Meta-Evaluation Criteria was designed to be based on a systematic approach and applies the phased approach to the lower component to reflect evaluation's characteristics. Then validity and reliability of the developed draft plan was verified by calculating Content Validity Ratio (CVR), Degree of Consensus, Degree of Convergence and Cronbach's alpha. The final developed Meta-Evaluation Criteria obtaining validity and reliability were composed of 5 evaluation areas (environment, input, performance, result, utilization), 15 evaluation items, 68 evaluation indicators.

• 기술혁신학회지
• /
• 제12권1호
• /
• pp.113-136
• /
• 2009

국가 연구개발사업의 효율성과 생산성을 제고하기 위하여, 정부에서는 '국가연구개발사업 등의 성과 평가 및 성과관리에 관한 법률'을 제정하여 정부 각 부처의 연구개발 사업 성과의 효율성을 증대시키고 있다. 본 논문은 국방연구개발사업 자체평가시스템 측면에서 메타평가를 실시하기 위하여 요인분석(Factor Analysis)을 통한 계층분석적 의사결정기법(AHP, Analytic Hierarchy Process)을 사용한 평가지표를 개발한 결과이다. 국방연구개발사업의 메타평가를 위한 평가요소로서 평가상황, 평가투입, 평가수행, 평가결과 등 4요소로 구분하여 평가지표를 개발하였다. 평가항목 선정을 위하여 평가요소 및 평가지표간 요인분석을 실시하여 Cronbach's ${\alpha}$ 값을 측정한 결과, 평가상황요소는 0.877, 평가투입요소는 0.755, 평가결과요소는 0.792, 평가결과요소는 0.906으로 내적일관성을 고려한 신뢰성은 있는 것으로 나타났다. 그리고 평가요소 및 평가항목별 가중치 산정은 AHP기법을 사용하여 분석하였다. 분석결과 조사자의 전체 일관성 지수는 0.09로 조사자의 일관성은 있는 것으로 분석되었다.

• 산업경영시스템학회지
• /
• 제32권3호
• /
• pp.59-70
• /
• 2009

This study is the result of meta evaluation for the self evaluation of defense R&D programs in Korea by using meta evaluating indicators. The overall meta evaluation result of defense R&D programs gained 74.3 points out of 100, so it was evaluated as 'Good'. But it demonstrated that further improvement for overall system of defense R&D programs evaluation is required. And especially, it demonstrated that more alternatives are necessary in order to improve the utilizations and the feedbacks of evaluation results. The evaluation context component gained 80.2 points out of 100, so it was evaluated as 'Very Good'. The evaluation input component gained 73.1 points out of 100, so it was evaluated as 'Good'. The evaluation process component gained 74.8 points out of 100, so it was evaluated as 'Good'. And the evaluation outcome component gained 69.0 points out of 100, so it was evaluated as 'Good'. Basic model of meta evaluation was derived from the literature review and brain storming. And this meta evaluation model was determined by adopting the result of experts who performed evaluations for defense R&D programs in recent years. The reliability of components and items was verified by Cronbach's a coefficient. It was over 0.6 in evaluation components and items. And the reliability of evaluation context was 0.877, that of evaluation input was 0.755, that of evaluation process was 0.755, that of evaluation output was 0.755 respectively. From the analysis, it is attempted to identify possible problems and to find out the ways of improvements related to the self evaluation system of defense R&D programs. The ultimate objective of this study is to manage the programs effectively and improve the reliability and the objectiveness of the defense R&D programs.

• 기술혁신연구
• /
• 제15권1호
• /
• pp.87-113
• /
• 2007

For the realization of the national self-defense, the trend of the defense R&D projects becomes global and diverse with the increase of their budget. Thus, for the effective execution of the budget and the successful performance of the projects, the importance of selection, management and evaluation of the R&D projects is emphasized. This study suggests the applicability of a meta-evaluation model for the realization of the successful defense R&D project evaluation system. This study focused on the development of the meta-evaluation model design which can be applicable to the national defense R&D projects. Using the proposed meta-evaluation model, we propose a technique to enhance the fairness and the reliability of the national defense R&D projects evaluation system.

• 기술혁신학회지
• /
• 제18권4호
• /
• pp.539-560
• /
• 2015

국가연구개발사업 성과평가제도는 성과와 예산의 유기적인 연계를 강화하고 있다. 본 연구는 국가연구개발사업에 관한 국회의 확정예산을 대상으로 상위평가결과에 영향을 미치는 선행요인을 예산규모, 사업수행기간, 사업의 지속성, 자체평가결과로 분류하고 이들 요인과 상위평가결과와 예산증감율의 관계를 분석하였다. 분석 결과, 예산규모, 사업수행기간, 자체평가 결과는 상위평가결과에 영향을 주는 것으로 나타났다. 또한, 자체평가결과가 예산의 증감에 영향을 주는 상위평가 결과에 가장 큰 영향을 주었다. 본 연구의 결과는 국가연구개발성과평가제도가 국가연구개발예산의 관계를 실증분석을 통해 제시했다는 측면에서 의의를 가진다.

• 한국벤처창업학회:학술대회논문집
• /
• 한국벤처창업학회 2009년도 통합학술대회
• /
• pp.5-24
• /
• 2009

This study is the result of meta evaluation for the self evaluation of defense R&D programs in Korea. The overall meta evaluation result of defense R&D programs gained 74.3 points out of 100, so it was evaluated as 'Good'. But it demonstrated that further improvement for overall system of defense R&D programs evaluation is required. And especially, it demonstrated that more alternatives are necessary in order to improve the utilizations and the feedbacks of evaluation results. The evaluation context component gained 80.2 points out of 100, so it was evaluated as 'Very Good'. The evaluation input component gained 73.1 points out of 100, so it was evaluated as 'Good'. The evaluation process component gained 74.8 points out of 100, so it was evaluated as 'Good'. From the analysis, it is attempted to identify possible problems and to find out the ways of improvements related to the self evaluation system of defense R&D Programs.

• 기술혁신학회지
• /
• 제17권4호
• /
• pp.703-732
• /
• 2014

국가연구개발사업에 대한 메타평가는 과학기술수요와 목표와 연계된 국가전체의 연구개발시스템의 틀 안에서 구축되어야 한다. 연구개발사업평가시스템은 투입, 수행, 산출, 활용의 전 과정으로 이해될 수 있다. 전문가 24명이 참여한 전문가 델파이기법을 통하여 24개의 평가지표가 개발되었다. 메타평가 모델은 중요한 관점 8개의 하위시스템을 포함하고 있다. 8개 하위시스템에는 기획의 적실성, 정보의 충실성, 평가투입에서 평가자의 적정성, 평가수행과정과 방법의 적절성, 평가활용보고와 응용의 유용성 등이 포함된다. 메타평가모형의 활용방법으로서 평정점수의 평균은 5점척도 기준으로 2.73이며, 이는 평가점수의 중앙값 3.0보다 낮게 나타났다. 평가결과의 환류는 연구개발프로그램의 개선에 특별히 강화될 것을 제언한다.

• 대한수학교육학회지:학교수학
• /
• 제13권3호
• /
• pp.469-484
• /
• 2011

본 연구는 수학 학습의 정의적 영역에서 메타-정의의 개념을 구체화하고 명확히 하기 위해 메타-정의의 유형을 구분하고, 실제 사례를 보여주었다. 메타-정의는 정의의 모니터링, 감정의 평가, 감정의 조절, 정의의 활용 4가지로 구분되었고, 모니터링 후 평가, 조절, 활용이 시간적 순서로 이루어지는 것을 볼 수 있었다. 감정을 평가하고 조절하는 데 수학과 수학 학습에 대한 신념이 중요한 잣대로 작용하였으며, 메타-정의는 정의를 조절하고 학습 전략을 선정하는 등 수학 학습에서 중요한 역할을 하였다.

• 한국수학교육학회지시리즈A:수학교육
• /
• 제58권4호
• /
• pp.519-530
• /
• 2019

선행연구에 의하면 수학 학습활동에서 인지적, 정의적 요소들 사이의 상호작용에 기반하는 메타정의는 메타인지와 유사한 방식으로 학습자의 수학적 능력과 긴밀한 역학적 관련성을 유지한다. 본 연구에서는 이러한 특성을 현상학적으로 파악하기 위하여 초등학교 5학년 수학 영재아의 소집단 문제해결 사례를 메타정의적 관점에서 분석하였다. 그 결과 수학 영재아의 인지적, 정의적 특성이 메타정의적 활동을 통해 문제해결 활동에 나타나고 있음을 알 수 있으며, 특히 문제해결자의 정의적 역량은 정서나 태도 형태의 메타정의로 문제해결 활동에 작용함을 알 수 있었다.