• Journal of Information Technology Applications and Management
• /
• v.19 no.2
• /
• pp.59-78
• /
• 2012

The purpose of this longitudinal case study is to shed light on the reliability problems of IS performance evaluation by analyzing the discrepancy between ex ante evaluation in 2011 and ex post evaluation in 2012. Through an information system development project of a public enterprise, the gap between ex ante and ex post evaluation was ascertained and the causes as to why such gap occurs and the success factors that can solve the problems were derived. The ex ante evaluation of IS performance was performed based on both IS success model and BSC model and the ex post evaluation was carried out at the time after the target system has been built up by applying the evaluation measures and process that are similar to the ex ante evaluation. In the ex ante evaluation the business performance improved by the target system was estimated to 18.2%. On the other hand, it was seen as being low at 15.2% in the ex post evaluation and the differences were found to be statistically significant in 6 out of 10 measures. The reasons as to why such gap occurs were diagnosed as being of 2 types : (1) changes in the evaluation psychology according to the differences in the evaluation objectives; and (2) excessive expectation levels of the target system that is formed at the time of the ex ante evaluation. In other words, the users who have excessive expectations tend to overestimate in the ex ante evaluation and, in the ex post evaluation, tend to underestimate lower than the actual performance mainly due to disappointment on the results that do not meet the early expectations. As solutions to overcome the reliability problem of ex ante evaluation, 3 factors were derived : (1) the temperance of excessive expectation levels of the users; (2) a clear definition of the scope and functionality of the target system; and (3) actual commitments for the evaluations of IT performance.

• Journal of Information Technology Applications and Management
• /
• v.15 no.1
• /
• pp.203-224
• /
• 2008

The purpose of this study is to investigate the evaluation perspectives of IT business value and empirically analyze the differences of evaluation perspectives between ex ante evaluation and post implementation evaluation, and among various information system types. 4 evaluation perspectives which have been used in the previous studies and in the field of industry practices are financial, customer, internal process, and growth, which are based on the BSC model. Data of 98 information systems have been collected from 89 companies including manufacturers, banks, insurances, and stock trading companies. As results of multi-way MANOVA test, 3 out of 8 hypothesis have been accepted statistically. While the order of importance of 4 perspectives in ex ante evaluation is customer, financial, internal process, and growth, the order of post implementation evaluation appears to be much different : that is, internal process, customer, financial, and growth. These findings provide insights for both IT practitioners and researchers.

• Asian Journal of Innovation and Policy
• /
• v.9 no.3
• /
• pp.281-307
• /
• 2020

A variety of approaches are being applied to improve the existing ex-ante evaluation by expert panels in publicly funded R&D. While the objective evaluation criteria are constantly being improved to screen and select the superior projects, alternative approaches such as random prioritization and logical modeling are also underway to overcome the conservative bias of reviewers and to secure disruptive innovation. This study intends to find critical implications for ex-ante evaluation of public R&D system from the comparison of Indonesia and Korea. For the comparative analysis, literature review and expert in-depth interviews are conducted on the national R&D system and the selection evaluation process. In Korea, the selection criteria of projects are legally promulgated for establishing an objective evaluation system, and at the program level, the major considerations in the planning process are specified by Presidential Decree. On the other hand, while Indonesia conducts R&D in 47 strategic fields largely by public research institutes (PRI) based on the non-competitive government contributions. This study draws out implications of institutionalizing the planning process at the level of program, and of increasing the ratio of contract-based competitive funding at the level of project in the national R&D portfolio.

• The Journal of Fisheries Business Administration
• /
• v.41 no.1
• /
• pp.1-24
• /
• 2010

The main goal of the study is to propose an objective and standardized evaluation system of stock enhancement programs. In order to achieve this goal, the study first suggested the need for stock enhancement program evaluation system through the review of current status and problems. Second, the study identified possible problems of the existing stock enhancement program evaluation by reviewing domestic and foreign evaluation systems. Finally the study proposed a new evaluation system and implementation plan of it. This study also classified the program evaluation criteria into ex-ante evaluation and ex-post evaluation according to the evaluation point in time, and applied the economic, political and technical feasibility tests into the evaluation of the stock enhancement program in order to solve the current problems of the evaluation. The evaluation process of the stock enhancement program is composed of an evaluation system design, estimation of weights using the analytical hierarchy process, design of estimation standard, conversion of scores and final summary of the evaluation. The central government takes the lead in the evaluation of the regional (metropolitan city or province) projects and the regional government is in charge of the evaluation of the local (city or county) projects. For the implementation of the ex-ante evaluation, either the regional or local governments ask for the evaluation and then submit an evaluation plan and other necessary documents to the upper level governments. The ex-post evaluation is then carried out by the upper level governments.

• Journal of Korea Technology Innovation Society
• /
• v.8 no.3
• /
• pp.1175-1198
• /
• 2005

The objective of the study is to verify the relationship between technology valuation indexes and corporate's performance in ventures by business operating periods. The result of the study is expected to be useful in loan evaluation, investment decision, internal management decision making and business improvement. The results of study is as follows. First, in early stage ventures, we find that three major valuation index(technology feasibility, economic efficiency, productivity) are significant ex-ante variables which are discriminating between firms' going concern and firms' failure. Second, in growth stage ventures, we find that three major valuation index(business feasibility, general marketability, technology marketability) are significant ex-ante variables which are discriminating between firms' going concern and firms' failure. Third, in early stage ventures, we find that at least thirty-eight minor valuation index elements are significant ex-ante variables which are discriminating between firms' going concern and firms' failure and in growth stage ventures, thirty-one minor valuation index elements are significant in various analysis' results.

• STI Policy Review
• /
• v.4 no.2
• /
• pp.41-54
• /
• 2013

R&D is inherently an uncertain endeavor, yet now more than ever those performing R&D with public funding are called upon to clarify the utility of their research. Calls for public accountability are mounting with the increase in constraints on government budgets due to the recent worldwide economic recession, in response to which both policymakers and researchers pay much more attention to rigorously assessing publicly funded R&D. A key issue complicating R&D evaluation in these circumstances is how to adequately account for the nature and degree of risk involved in a given R&D program or project. This study deliberates on certain issues involving the measurement of ex-ante risk in public R&D evaluation: (i) information asymmetry between R&D sponsors and performers, (ii) ambiguity in the measurement of returns in both prospective and retrospective evaluation, and (iii) the dilemma between measurement error and omitted variable bias for empirical estimation of R&D performance. The study then presents an analysis of hypothetical evaluation results that apply risk-relevant weights to the annual evaluation outcomes of South Korea's national R&D programs funded during 2006~2012. In this counterfactual re-evaluation of public R&D program performance, high-risk R&D programs turn out to receive higher evaluation than non-high-risk programs. The current study suggests that R&D evaluation ignoring ex-ante risk is not only conceptually invalid since R&D activities are intrinsically uncertain endeavors, but unfair as R&D performers are asked to be accountable for the results that were in fact out of their reach.

• Knowledge Management Research
• /
• v.7 no.1
• /
• pp.49-63
• /
• 2006

The objective of the study is to verify the discriminatory power of technology innovation in predicting Early Stage Ventures' success or failure. To accomplish this objective, we test early stage ventures'(Firm's period is below 3 years)technology innovation and performance. The result of the study is expected to be useful in loan evaluation, investment decision, internal management decision making and business improvement. The results of study is as follows. First, Early Stage Ventures' technology innovation power is composed of 4 major indexes(technology, marketability, manufacturing infra and economic feasibility). Second, we find that thirty-seven minor indexes are significant ex-ante variable which are discriminating between firms' success and failure in Early Stage Ventures. Also thirty-seven minor indexes explain 57.2% of the total variance. This explainable power of these indexes is similar to that of the existing 58 index elements. Finally, we find that the most important technology innovation power of Early Stage Ventures' is economic feasibility.

• The Journal of Society for e-Business Studies
• /
• v.20 no.4
• /
• pp.241-256
• /
• 2015

In order to ensure that all firms are cyber-secure, many governments have started to enforce the implementation of various security measures on firms. Prior to the implementation, however, it is vague whether government enforced security measures will be effective for mitigating cyber-security risks. By applying a method for estimating the effectiveness of a mandatory seatbelt law in reducing fatalities from motor vehicle accidents, this study develops an ex ante evaluation method that can approximate the effectiveness of a government enforced security measure in reducing country-wide or industry-wide cyber-security risks. Using data obtained from the Korean Internet and Security Agency, this study then explores how to employ the developed method to assess the effectiveness of a specific security measure in mitigating cyber-security risks, if enforced by the government, and compares the effectiveness of various security measures. The comparison shows that compulsory security training has the highest effectiveness.

• Journal of Korea Technology Innovation Society
• /
• v.17 no.2
• /
• pp.289-308
• /
• 2014

We analyze the risk factors identified for ex-ante evaluation regarding government R&D programs. In particular, we empirically examined to what extent each risk factor is perceived to be important by the stage of R&D, technology field, researcher's career and experience. The empirical results show that technology risk is perceive to be less important for basic research, while market risk more perceived to be important in the IT field. Political risk is perceived to be less important for the university faculty, while researchers with government R&D project experience gave more value to legal risk factor.

• Journal of Electrical Engineering and Technology
• /
• v.9 no.3
• /
• pp.796-802
• /
• 2014

South Korea is recently under serious situation in supplying electricity with enough power reserve. A single fault of power plant at a peak-load time may lead to a total blackout for whole area connected by a single electric grid and isolated from other grids. Despite of the seriousness of blackout, however, there are scarce studies with ex-ante analysis of the economic costs from blackout. In order to evocate the seriousness, we calculate the economic costs for both industrial and household sectors with using some survey data and statistical methodologies. As a result, total economic costs are 39.23 trillion KRW (35.83 trillion KRW for industrial sector, 3.40 trillion KRW for household sector).