Journal of Convergence for Information Technology (융합정보논문지)

Convergence Society for SMB (중소기업융합학회)
권호 표지
DOI QR코드

DOI QR Code

Halo Effect in Evaluating Government Funded Art Programs: The Case of Local Representative Performing Art Festivals

정부지원 공연예술행사 평가의 후광효과: 지역대표공연예술제 성과관리 체계를 중심으로

  • Cho, Mun-Seok (Department of Public Administration, Hansung University) ;
  • Oh, Jae-Rok (Department of Public Administration, Jeonju University)
  • Received : 2019.07.15
  • Accepted : 2019.08.20
  • Published : 2019.08.28
Download PDF
⟨ Previous Next ⟩

Abstract

This research empirically investigated halo effect in evaluating culture and art performance program. We diagnosed halo effect by using correlation analysis, factor analysis, and regression model on results and scores of fifteen evaluation indicators within three categories for the 107 Local Representative Performance Art Festivals in 2014 and 2015. The results indicates strong possibility of halo effect in culture and art performance evaluation. The correlation coefficients between evaluation indicators is higher than 0.5 and factor structure does not match with evaluation categories in both years. Scores in categories and standard deviations also are also significantly correlated with each other. The results implies that more sophisticated standard, diversification of evaluator, education, and meta-anlysis are need to control halo effect.

이 연구의 목적은 문화예술행사 평가 과정에서 제기되는 평가 오류의 문제 중 하나인 후광효과를 실증적으로 분석하여 문제점을 진단하고 개선 방향을 제시하는 것이다. 이 연구는 2014년과 2015년 진행된 107개 지역대표공연예술제 평가 결과를 대상으로 3개 평가 영역, 15개 평가지표의 평가 결과에 대한 상관관계분석, 요인분석, 회귀분석을 통해 후광효과를 진단하였다. 분석 결과 평가지표 간의 상관계수의 평균은 2014년 0.676, 2015년 0.584로 나타났으며, 요인구조와 평가 영역은 일치하지 않았다. 회귀분석 결과 타 영역의 평가점수와 표준편차는 기획, 집행, 결과영역의 평가 결과에 영향을 미치는 것으로 나타났다. 이러한 평가결과의 왜곡은 평가의 신뢰성과 수용성을 낮추므로 합리적이고 객관적 평가를 위해 평가기준의 정교화, 평가지표별 평가자 다양화, 평가자 대상 사전교육, 메타평가의 실행을 통해 후광효과를 통제할 필요성을 제기한다.

Keywords

JKOHBZ_2019_v9n8_123_f0001.png 이미지

Fig. 1. Research Model

Table 1. Performing Art Program Budget

JKOHBZ_2019_v9n8_123_t0001.png 이미지

Table 2. Performance Indicator and Scores (2015)

JKOHBZ_2019_v9n8_123_t0002.png 이미지

Table 3. Descriptive Statistics

JKOHBZ_2019_v9n8_123_t0003.png 이미지

Table 4. Descriptive Statistics of Correlation Coefficients between Adjusted Scores of Indicators

JKOHBZ_2019_v9n8_123_t0004.png 이미지

Table 5. Summary of Factor Analysis Results

JKOHBZ_2019_v9n8_123_t0005.png 이미지

Table 6. Regression Results with Performance Scores on Plan as Dependent Variable

JKOHBZ_2019_v9n8_123_t0006.png 이미지

Table 7. Regression Results with Performance Scores on Implementation as Dependent Variable

JKOHBZ_2019_v9n8_123_t0007.png 이미지

Table 8. Regression Results with Performance Scores on Program Performance as Dependent Variable

JKOHBZ_2019_v9n8_123_t0008.png 이미지

References

  1. H. Y. Jang. (2016). Analysis on the Effect of Characteristic of Policy on K-PART Focused on Culture & Art Policy. The Korean Association for Policy Sturies, 25(2), 91-124.
  2. H. J. Ju, H. S. Ha & C. H. Jeong. (2016). The Study for Sustainability Securement of Culture and Arts Promotion Fund Program Evaluation: The Limitation and Reform of Program Evaluation System. Korean Comparative Government Review, 20(1), 75-99. https://doi.org/10.18397/kcgr.2016.20.1.75
  3. I. J. Kim & J. S. Jeon. (2009) A Study on Problem and Improvement of Management Performance Evaluation System for the Local Public Corporations with Meta Evaluation. The Korean Journal of Local Government Studies, 13(1), 133-148.
  4. C. K. Lee & S. R. Choi. (2010). An Analysis on Rating Errors by Relative Performance Evaluation for Public Enterprise. Korean Journal of Public Administration, 48(1), 97-119.
  5. C. M. Lee & I. Kim. (2016). The Influence of Service Contracting-out and Managerial Characteristics of Culture & Arts Center on Service Quality. The Korean Journal of Local Government Studies, 20(1), 525-553. https://doi.org/10.20484/klog.20.1.22
  6. J. H. Lee. (2018) Performance Evaluation of the Arts Organization: Focusing on Developing Alternative Evaluation Indicators of National Theater of Korea, The Korean Public Administration Journal, 27(1), 209-243.
  7. S. J. Han. (2014) A Study on the Evaluation of the Effectiveness of Current Government Cultural Policies: Focused on the Importance-Performance Analysis. Korean Journal of Local Government & Administration Studies, 28(4), 209-231. https://doi.org/10.18398/kjlgas.2014.28.4.209
  8. W. J. Baumol & W. G. Bowen. (1966). Performing Arts: The Economic Dilemma. New York: Twenty Century Fund.
  9. R. Eisenberger & S. Armeli. (1997). Can Salient Reward Increase Creative Performance without Reducing Intrinsic Creative Interest? Journal of Personality and Social Psychology, 72, 652-663. https://doi.org/10.1037/0022-3514.72.3.652
  10. B. S. Frey. (1999). State Support and Creativity in the Arts: Some New Consideration. Journal of Cultural Economics, 23(1), 71-85. https://doi.org/10.1023/A:1007518203490
  11. W. J. Baumol & W. G. Bowen. (1965). On the Performing Arts: The Anatomy of Their Economic Problems. The American Economic Review, 55(1/2), 495-502.
  12. G. Evans. (2000). Measures for Measure: Evaluating Performance and the Arts Organization. Studies in Culture, Organizations, and Societies, 6(2), 243-266. https://doi.org/10.1080/10245280008523549
  13. S. Tospler. (2001). Culture, Commerce, and Civil Society. Administration & Society, 33(5), 511-532.
  14. J. S. Jim. (2008). Reflection on Public Support for the Arts: A Case Study on the Gyeonggi Cultural Foundation's Arts Support Programs. The Journal of Cultural Policy, 20, 163-191
  15. J. H. Roh & J. W. Choi. (2012). Performance Appraisal System of Employees in Culture and Art Organizations: The Case of National Gugak Center. Korean Public Personnel Administration Review, 11(2), 103-127.
  16. C. K. Lee & S. R. Choi. (2009). Halo Effect in Performance Evaluation on Public Enterprise. Korean Public Administration Review, 43(3), 151-172.
  17. I. J. Kim & J. S. Jeon. A Study on Problem and Improvement of Management Performance Evaluation System for the Local Public Corporations with Meta Evaluation. The Korean Journal of Local Government Studies, 13(1), 133-148.
  18. E. L. Thorndike. (1920). A Constant Error in Psychological Ratings. Journal of Applied Psychology, 4(1), 25-29. https://doi.org/10.1037/h0071663
  19. W. H. Cooper. (1981). Ubiquitous Halo. Psychological Bulletin, 90, 218-244. https://doi.org/10.1037/0033-2909.90.2.218
  20. J. Crocker. (1981). Judgement of Covariation by Social Perceivers. Psychological Bulletin, 90, 272-292. https://doi.org/10.1037/0033-2909.90.2.272
  21. J. M.. Feldman. (1981). Beyond Attribution Theory: Cognitive Processes in Performance Appraisal. Journal of Applied Psychology, 66, 127-148. https://doi.org/10.1037/0021-9010.66.2.127
  22. S. A. Fisicaro & E. L. Charles. (1990). Implications of Three Causal Models for the Measurement of Halo Error. Applied Psychological Measurement, 14(4), 419-429. https://doi.org/10.1177/014662169001400407
  23. T. H. Feeley. (2002). Evidence of Halo Effects in Student Evaluations of Communication Instruction. Communication Education, 51(3), 225-236. https://doi.org/10.1080/03634520216519
  24. I. S. Han. (2016). Validity and Reliability of 360 Degree Feedback of Public Enterprise in Korea. Korean Public Administration Review, 50(2), 131-153. https://doi.org/10.18333/KPAR.50.2.131
  25. S. A. Fisicaro & R. J. Vance. (1994). Comments on the Measurement of Halo. Educational and Psychological Measurement, 54, 366-371. https://doi.org/10.1177/0013164494054002010
  26. W. C. Borman. (1979). Format and Training Effects on Rating Accuracy and Rater Errors. Journal of Applied Psychology, 64, 410-421. https://doi.org/10.1037/0021-9010.64.4.410
  27. G. S. Roh, M. C. Kim & I. H. Kim. (2004). Effect Analysis Focused on Region Economic in Culture Content Industry-Case Analysis of Culture Industry in Jeju Region. Journal of Digital Convergence, 2(1), 9-23.
  28. S. M. Kim. (2016). Study on Development of Crafts Cultural Industry-Central Region of South Korea Craft Industrial as Center. Journal of Digital Convergence, 14(5), 385-390. https://doi.org/10.14400/JDC.2016.14.5.385
  29. I. K. Bae. (2018). The Effects of Cultural Arts on the Improvement of Happiness Index of Seoul Citizens. Journal of Digital Convergence, 16(11), 89-96. https://doi.org/10.14400/JDC.2018.16.11.089
  30. H. W. Chang & Y. H. Song. (2018) Actions of the Department in Context of Higher Education Evaluation Policy-A Case on the Christian Education of A University. Journal of Convergence for Information Technology, 8(3), 153-160. https://doi.org/10.22156/CS4SMB.2018.8.3.153
  31. H. S. Lee. (2014). Analysis of Operational Competency of Public Culture & Arts Institution. Journal of Digital Convergence, 12(11), 23-32. https://doi.org/10.14400/JDC.2014.12.11.23