• Journal of Information Technology Applications and Management
• /
• 제19권2호
• /
• pp.59-78
• /
• 2012

The purpose of this longitudinal case study is to shed light on the reliability problems of IS performance evaluation by analyzing the discrepancy between ex ante evaluation in 2011 and ex post evaluation in 2012. Through an information system development project of a public enterprise, the gap between ex ante and ex post evaluation was ascertained and the causes as to why such gap occurs and the success factors that can solve the problems were derived. The ex ante evaluation of IS performance was performed based on both IS success model and BSC model and the ex post evaluation was carried out at the time after the target system has been built up by applying the evaluation measures and process that are similar to the ex ante evaluation. In the ex ante evaluation the business performance improved by the target system was estimated to 18.2%. On the other hand, it was seen as being low at 15.2% in the ex post evaluation and the differences were found to be statistically significant in 6 out of 10 measures. The reasons as to why such gap occurs were diagnosed as being of 2 types : (1) changes in the evaluation psychology according to the differences in the evaluation objectives; and (2) excessive expectation levels of the target system that is formed at the time of the ex ante evaluation. In other words, the users who have excessive expectations tend to overestimate in the ex ante evaluation and, in the ex post evaluation, tend to underestimate lower than the actual performance mainly due to disappointment on the results that do not meet the early expectations. As solutions to overcome the reliability problem of ex ante evaluation, 3 factors were derived : (1) the temperance of excessive expectation levels of the users; (2) a clear definition of the scope and functionality of the target system; and (3) actual commitments for the evaluations of IT performance.

• Journal of Information Technology Applications and Management
• /
• 제15권1호
• /
• pp.203-224
• /
• 2008

The purpose of this study is to investigate the evaluation perspectives of IT business value and empirically analyze the differences of evaluation perspectives between ex ante evaluation and post implementation evaluation, and among various information system types. 4 evaluation perspectives which have been used in the previous studies and in the field of industry practices are financial, customer, internal process, and growth, which are based on the BSC model. Data of 98 information systems have been collected from 89 companies including manufacturers, banks, insurances, and stock trading companies. As results of multi-way MANOVA test, 3 out of 8 hypothesis have been accepted statistically. While the order of importance of 4 perspectives in ex ante evaluation is customer, financial, internal process, and growth, the order of post implementation evaluation appears to be much different : that is, internal process, customer, financial, and growth. These findings provide insights for both IT practitioners and researchers.

• Asian Journal of Innovation and Policy
• /
• 제9권3호
• /
• pp.281-307
• /
• 2020

A variety of approaches are being applied to improve the existing ex-ante evaluation by expert panels in publicly funded R&D. While the objective evaluation criteria are constantly being improved to screen and select the superior projects, alternative approaches such as random prioritization and logical modeling are also underway to overcome the conservative bias of reviewers and to secure disruptive innovation. This study intends to find critical implications for ex-ante evaluation of public R&D system from the comparison of Indonesia and Korea. For the comparative analysis, literature review and expert in-depth interviews are conducted on the national R&D system and the selection evaluation process. In Korea, the selection criteria of projects are legally promulgated for establishing an objective evaluation system, and at the program level, the major considerations in the planning process are specified by Presidential Decree. On the other hand, while Indonesia conducts R&D in 47 strategic fields largely by public research institutes (PRI) based on the non-competitive government contributions. This study draws out implications of institutionalizing the planning process at the level of program, and of increasing the ratio of contract-based competitive funding at the level of project in the national R&D portfolio.

• 수산경영론집
• /
• 제41권1호
• /
• pp.1-24
• /
• 2010

The main goal of the study is to propose an objective and standardized evaluation system of stock enhancement programs. In order to achieve this goal, the study first suggested the need for stock enhancement program evaluation system through the review of current status and problems. Second, the study identified possible problems of the existing stock enhancement program evaluation by reviewing domestic and foreign evaluation systems. Finally the study proposed a new evaluation system and implementation plan of it. This study also classified the program evaluation criteria into ex-ante evaluation and ex-post evaluation according to the evaluation point in time, and applied the economic, political and technical feasibility tests into the evaluation of the stock enhancement program in order to solve the current problems of the evaluation. The evaluation process of the stock enhancement program is composed of an evaluation system design, estimation of weights using the analytical hierarchy process, design of estimation standard, conversion of scores and final summary of the evaluation. The central government takes the lead in the evaluation of the regional (metropolitan city or province) projects and the regional government is in charge of the evaluation of the local (city or county) projects. For the implementation of the ex-ante evaluation, either the regional or local governments ask for the evaluation and then submit an evaluation plan and other necessary documents to the upper level governments. The ex-post evaluation is then carried out by the upper level governments.

• 기술혁신학회지
• /
• 제8권3호
• /
• pp.1175-1198
• /
• 2005

The objective of the study is to verify the relationship between technology valuation indexes and corporate's performance in ventures by business operating periods. The result of the study is expected to be useful in loan evaluation, investment decision, internal management decision making and business improvement. The results of study is as follows. First, in early stage ventures, we find that three major valuation index(technology feasibility, economic efficiency, productivity) are significant ex-ante variables which are discriminating between firms' going concern and firms' failure. Second, in growth stage ventures, we find that three major valuation index(business feasibility, general marketability, technology marketability) are significant ex-ante variables which are discriminating between firms' going concern and firms' failure. Third, in early stage ventures, we find that at least thirty-eight minor valuation index elements are significant ex-ante variables which are discriminating between firms' going concern and firms' failure and in growth stage ventures, thirty-one minor valuation index elements are significant in various analysis' results.

• STI Policy Review
• /
• 제4권2호
• /
• pp.41-54
• /
• 2013

R&D is inherently an uncertain endeavor, yet now more than ever those performing R&D with public funding are called upon to clarify the utility of their research. Calls for public accountability are mounting with the increase in constraints on government budgets due to the recent worldwide economic recession, in response to which both policymakers and researchers pay much more attention to rigorously assessing publicly funded R&D. A key issue complicating R&D evaluation in these circumstances is how to adequately account for the nature and degree of risk involved in a given R&D program or project. This study deliberates on certain issues involving the measurement of ex-ante risk in public R&D evaluation: (i) information asymmetry between R&D sponsors and performers, (ii) ambiguity in the measurement of returns in both prospective and retrospective evaluation, and (iii) the dilemma between measurement error and omitted variable bias for empirical estimation of R&D performance. The study then presents an analysis of hypothetical evaluation results that apply risk-relevant weights to the annual evaluation outcomes of South Korea's national R&D programs funded during 2006~2012. In this counterfactual re-evaluation of public R&D program performance, high-risk R&D programs turn out to receive higher evaluation than non-high-risk programs. The current study suggests that R&D evaluation ignoring ex-ante risk is not only conceptually invalid since R&D activities are intrinsically uncertain endeavors, but unfair as R&D performers are asked to be accountable for the results that were in fact out of their reach.

• 지식경영연구
• /
• 제7권1호
• /
• pp.49-63
• /
• 2006

The objective of the study is to verify the discriminatory power of technology innovation in predicting Early Stage Ventures' success or failure. To accomplish this objective, we test early stage ventures'(Firm's period is below 3 years)technology innovation and performance. The result of the study is expected to be useful in loan evaluation, investment decision, internal management decision making and business improvement. The results of study is as follows. First, Early Stage Ventures' technology innovation power is composed of 4 major indexes(technology, marketability, manufacturing infra and economic feasibility). Second, we find that thirty-seven minor indexes are significant ex-ante variable which are discriminating between firms' success and failure in Early Stage Ventures. Also thirty-seven minor indexes explain 57.2% of the total variance. This explainable power of these indexes is similar to that of the existing 58 index elements. Finally, we find that the most important technology innovation power of Early Stage Ventures' is economic feasibility.

• 한국전자거래학회지
• /
• 제20권4호
• /
• pp.241-256
• /
• 2015

기업의 정보 보안을 보장하기 위해, 많은 정부가 다양한 보안 관련 대책을 의무화해오고 있다. 하지만 이러한 보안 대책의 실행에 앞서, 이의 잠재적 효과성을 분석하는 연구는 부족한 실정이다. 본 연구는 안전벨트의 착용 법제화가 자동차 사고 사망자 감소에 미치는 효과에 대한 연구를 응용하여, 정부의 다양한 보안 대책의 사전 효과성 평가가 가능한 모형을 개발하는 것을 목적으로 한다. 또한, 인터넷 진흥원의 정보보호실태조사(기업편) 데이터를 개발된 모형에 적용하여 어떠한 보안대책의 법제화가 사회 및 산업 전반의 보안위험을 줄이는데 효과적인지 사전평가를 시행하였다. 그 결과 보안교육의 법제화가 다른 보안대책에 비해 효과적임을 확인하였다.

• 기술혁신학회지
• /
• 제17권2호
• /
• pp.289-308
• /
• 2014

본 연구는 최근 이슈가 되고 있는 대형 정부 R&D사업의 사전적 분석 및 의사결정에 있어서 이에 특화된 위험요인에 대한 분석은 아직 활발하게 진행되지 않았다는 문제 인식에서 진행되었다. 대형 R&D 사업의 사전평가 단계에서 제도적으로 고려되는 위험요인에 대한 중요도 인식에 어떠한 요인들이 영향을 미치는지를 실증 분석하여, 향후 대형 R&D 사업의 특성에 따라 위험요인에 대한 고려를 차별화하는 근거를 탐색해 보고자 한다. 위험요인별 중요도 인식에 영향을 주는 인자는 전문가 특성으로부터 도출하였다. 전문가 특성은 전문가가 연구하는 연구개발 단계, 연구 분야, 업무 경험, 경력 등의 다양한 변수들로 구성하였다. 분석 결과, 기획위험에 대한 중요도 인식은 전문가의 소속기관의 특성에 따라 차이를 보이고, 기술위험에 대한 중요도 인식은 연구단계 중 기초연구가 상대적으로 중요성이 떨어지는 것으로 조사되었다. 시장위험은 기술분야 중 IT 분야에 보다 더 중요하게 인식되는 것으로 조사되었다. 정치적 위험은 대학에 소속된 전문가는 중요도를 낮게 인식하고 있었고, 연구개발사업 참여자 경험자는 법적위험과 정치적 위험이 더 중요하다고 인식하였다. 이해관계자 위험에 대한 인식 정도는 전문가 특성에 크게 영향을 받지 않는 것으로 분석되었다.

• Journal of Electrical Engineering and Technology
• /
• 제9권3호
• /
• pp.796-802
• /
• 2014

South Korea is recently under serious situation in supplying electricity with enough power reserve. A single fault of power plant at a peak-load time may lead to a total blackout for whole area connected by a single electric grid and isolated from other grids. Despite of the seriousness of blackout, however, there are scarce studies with ex-ante analysis of the economic costs from blackout. In order to evocate the seriousness, we calculate the economic costs for both industrial and household sectors with using some survey data and statistical methodologies. As a result, total economic costs are 39.23 trillion KRW (35.83 trillion KRW for industrial sector, 3.40 trillion KRW for household sector).