Development of Quality Management Indicators for Educational Institutions

Abstract

The purpose of this study was to develop indicators for quality management of educational institutions, and to achieve this, literature analysis and expert interviews were conducted. Through literature analysis, the components of TQM were derived focusing on factors to improve the quality of education, engineering education accreditation standards, and a representative education accreditation system, were considered. Additionally, seven areas as well as 32 indicators required for education quality management, were derived by comparing the EFQM excellence model and the MBNQA education model, applied for quality management of companies and institutions. By comparing and synthesizing these results, a draft was developed for the quality management index of educational institution. Next, opinions on correction and supplementation of quality management indicators derived from literature analysis, were collected from five education experts. From the comparisons and integration of these results, eight criteria (leadership, strategy, customers, people, facilities and environment, curriculum management, curriculum improvement, and performance management) and 34 indicators, were proposed for quality management indicators for educational institutions. Curriculum management, people, and performance management criteria were considered more significantly in quality management of educational institutions, and several implications are suggested based on the study results.

1. Introduction

Along with the rapid environmental changes like the opening of the education market, the importance of the quality is emphasized in the field of education. In other words, the efforts to apply Total Quality Management (TQM) to education are increasing under the circumstances that the quality is the core competency deciding the competitiveness and strategic advantage of all organizations [1]. For this reason, educational organizations around the world have shown interest in the quality management systems [2].

For academic organizations, the accreditation system is the representative example of how quality management is being applied to educational institutions: univerities, the birthplace of the higher education, have adopted accreditation system to the various academic fields starting with the engineering. For example, many universities in Korea already have implemented multiple accreditation systems such as Accreditation of Engineering Education, Accreditation of Medical Education, and Accreditation of Business Education. The quality of education is being managed through these system.

For business organizations, several efforts has been made to improve the quality of education: EFQM (European Foundation for Quality Management) Excellence Model has been applied in Europe and MBNQA (The Malcolm Baldrige National Quality Award) has been established in the United States. However, the quality management efforts mainly center around universities [3] and quality management standard has not been introduced actively for enterprises or public educational institutions yet.

As for the evaluation of education, performance-oriented evaluation is primarily applied rather than quality-oriented evaluation in business and public organization. The Four-Level Evaluation Model proposed by Kirkpatrick is mainly applied to evaluate educational performance: the model emphasizes the participants’ reaction, achievement, and behavioral change after education. This is related to the fact that the ultimate goal of education in the business and public organizations is the performance improvement of the organizations.

Therefore, to analyze the performance of education, the results obtained after providing education are analyzed and education is improved based on the analysis results. But this result-oriented performance evaluation sometimes causes the distorted phenomenon that education only focuses on the numeric results. It is not very desirable to focus only on education results too much especially when it is hard to analyze how much the performance of education contributes to the performances of the enterprises and institutions.

Quality management is a management philosophy for continuous improvement to provide quality products and services beyond the desires of customers [4,5]. It emphasizes the management of overall processes as well as results [4][6]. In this respect quality management differs from performance evaluation. If educational institutions apply quality management, they will not only obtain results but suggest quality management standard, collect and analyze data during the process, and then feed them back into the information for continuous improvement. As a result of this quality management, students will be provided with high quality education and institutions could improve the performance. To improve the quality of education, we analyze quality-related literature and develop quality control indicators for educational institutions. Therefore, the purpose of this study is to develop quality management indicators which could support effective quality management for the education.

2. Literature review

2.1 Total Quality Management

The concept of quality management started and advanced from the manufacturing industry and expanded to the service industry, then with the recognition of its importance, it has been applied to education field like schools. The early concept of quality management was proposed from the producer-centered viewpoint and it meant the managing for manufacturing standardized products [7]. Since the emergence of “customer-centered quality concept”, however, the concept of the quality management shifted from producer-centered to customer- centered, and the meaning also changed to the management for producing products and services satisfying the desires of customers [8]. Today, quality management is generally called as total quality management (TQM). TQM is the concept emphasizing that to produce products and services meeting the desires of customers, management should pay attention not just to one aspect like a production process, but to overall aspects such as leadership, organization, system, and environment [2,8]. That is, all the areas and parts of enterprises and institutions require total systems to manage and improve qualities to provide the products and services satisfying customers and in this respect, TQM could be considered as an advanced concept of quality management.

Researchers who stress the application of TQM suggested the factors for successful establishment and administration of TQM. Table 1 organized the Research studies integrating previous studies and suggesting these factors. [9] analyzed the study of [10] and the quality criteria for the Malcolm Baldrige National Quality Award and then suggested management behaviors, a strategy for TQM, organization for TQM, communication for TQM, training for TQM, employee involvement, process management and systems, management of process quality, and quality technologies. [6] reviewed and integrated widely promoted TQM related approaches including Deming’s 14 points, the Juran trilogy (quality planning, quality control, and quality improvement), and Crosby’s 14 quality steps and proposed the twelve TQM factors: committed leadership, adoption and communication of TQM, closer customer relationships, closer supplier relationships, benchmarking, increased training, open organization, employee empowerment, zero-defects mentality, flexible manufacturing, process improvement, and measurement. [5] reviewed and analyzed factors suggested by[10-12] and proposed success factors for TQM implementation: leadership, organization, education, quality in design, quality in suppliers, quality in process, fact-based management, human resource management, customer focus, and tools and techniques. [4] reviewed previous studies and proposed management leadership, training, employee relations, quality data & reporting, supply quality management, product/service design, process management, inventory management, quality performance, and financial & market performance as TQM factors. [2] also integrated previous studies and suggested TQM factors: excellent quality information system, professional qualifications of members, consumer-centered desire satisfaction, leadership of executives, empowerment, continuous improvement, active participation of every sector. Based on the factors suggested by more than two research studies, TQM factors were drawn and similar components were integrated and proposed as a single factor. As a result, twelve TQM critical factors were drawn as follow: leadership, quality information system, process/departmental quality management, education, employee management and empowerment, customer- centered, organization, communication, active participation, high quality technology, continuous improvement, and performance management.

Table 1. TQM factor studies

2.2 Accreditation of Education

Accreditation of education is a leading effort to enhance the quality of college education in terms of quality management. In Korea, since the implementation of Accreditation of Engineering Education in 2001, accreditation of education has spread through the introduction of accreditation in various fields including Accreditation of Business Education, Accreditation of Medical Education, and Accreditation of Veterinary Medical Education. This study will review accreditation of education focused on the Accreditation of Engineering Education which has been successfully introduced and taken root for a long time both internationally and domestically.

Accreditation of Engineering Education introduced for quality management of engineering education in the United States where college establishment is comparatively free. In Korea, it was introduced in earnest with the approval of establishment of the Accreditation Board for Engineering Education of Korea as a corporation in June 2000[13]. As of 2016, 560 programs of 85 colleges have applied Accreditation of Engineering Education. The Accreditation Board for Engineering Education of Korea proposed four objectives of accreditation. First, Accreditation of Engineering Education ensures the graduates with the completion of accredited program are ready to be put in actual engineering sites. Second, Accreditation of Engineering Education identifies whether an institution and departmental engineering programs respectively meet the accreditation criteria. Third, Accreditation of Engineering Education promotes the introduction of new and innovative ways of engineering education, provides guidelines for engineering education program, and responds for consultation. Fourth, Accreditation of Engineering Education contributes to promote advancement of engineering education and to produce competent manpower in engineering technology required for industries and society. In other words, the main goals of Accreditation of Engineering Education are suggesting education program standard and guidelines for engineering and related education in college, performing accreditation and consultation, and through these, promoting advancement of engineering education and producing competent manpower in engineering technology required for industries and society.

The essential part of Accreditation of Engineering Education is Accreditation of Engineering Education criteria and this is the basis of the implementation of quality management. Therefore, it is important to examine the composition of the accreditation criteria and contents of individual criteria for drawing implication of education quality management. Accreditation of Engineering Education criteria consists of 1) Educational objectives of the program, 2) Learning outcomes, 3) Curriculum, 4) Students, 5) Faculty, 6) Educational Environment, 7) Program improvement, and 8) Program criteria. Table 2 describes this criteria. It is significant that Accreditation of Engineering Education suggested specific standard for education quality management and influenced on the spread of accreditation of education to other fields of education.

Tables 2. Accreditation of Engineering Education criteria

2.3 Quality Management Model for Institutions

As quality management models for institutions, EFQM Excellence Model and Malcolm Baldrige National Quality Award model will be studied in this paper. EFQM Excellence Model is used mainly in Europe and Malcolm Baldrige National Quality Award model is mainly applied in the United States and propose separated quality management model for institutions in education sector.

2.3.1 EFQM Excellence Model

EFQM Excellence Model is a quality management model proposed by EFQM (European Foundation for Quality Management) and have applied by institutions in Europe. More than 30,000 enterprises and non-profit organizations in Europe are applying EFQM Excellence Model [14]. EFQM defined Excellence as doing one’s best and claimed that EFQM Excellence Model is applicable to an organization to increase the productivity and efficiency and to develop human resources so the organization can do its best. EFQM Excellence Model is divided into Enablers which are related with what and how an organization does for doing its best and Results which are about what an organization can achieve in strategic point. Enablers include leadership, people, strategy, partnerships and resources, and process, products, and services. Results include people results, customer results, society results, and business results.

For each criterion of Enablers and Results, 1~5 indicators (criterion parts) were developed and proposed. Table 3 shows these criteria and indicators (from 2013 EFQM Excellence Model). Organization can be evaluated with an overall score from 0 to 1000 with EFQM criteria and each criterion is allocated with a weighting. 50% of weighting is allocated to Enablers and Results respectively and the 15% of weighting which is comparatively higher is allocated respectively to customer results and business results.

Figure 1. EFQM Excellence Model. Source: EFQM (2013)

In relation to the validity and effectiveness of EFQM Excellence Model, [15] claimed that managers of an organization could aware of the role of self-assessment in the establishment of strategy and business planning process of the organization and identify the areas for improvement with analyzing organizational performance through carrying out self-assessment. [16] empirically demonstrated the validity of EFQM Excellence Model. They proposed the result of the factor analysis of EFQM model criteria and demonstrated that Enabler criteria influence on Result criteria by providing model suitability. Additionally in a comparative study of EFQM Excellence Model and a performance management tool, EFQM Excellence Model was suggested more accurate and advanced tool [17]. Thus, EFQM Excellence Model is used by various institutions for quality management in work-field and its effectiveness also has been proved in research studies.

2.3.2 The Malcolm Baldrige National Quality Award (MBNQA) model

The Malcolm Baldrige National Quality Award (MBNQA) began in the United States to set the quality criteria by sector, examine it, and present an award to improve the competitiveness of the nation. MBNQA model has been considered one of the suitable model to assess the management quality of enterprises and used in the world and with its introduction, The Korean National Quality Award has been administered since 1994 in Korea [18]. MBNQA is not just for presenting quality award but has been developed as a quality management model applicable for various institutions in society. In 1998, the President Clinton of the United States of America signed a bill allowing the MBNQA to expand into education and health care sectors and the award in education sector has been administered since 1999 [19].

MBNQA model (from MBNQA model 2015) consists of seven criteria (categories) of leadership, strategy, customers, measurement/analysis/knowledge management, workforce, operations, and results and each criterion includes indicators (criteria items). Fig. 2 shows MBNQA education model (Education Criteria for Performance Excellence) and the indicators are described in Table 3. Allocated point for each criterion varies from 85 to 450 and total point of criteria is 1000. This scoring system is designed for the examination of the National Quality Award and because allocated point for each criterion is big, detailed scoring guidelines for assessment are also presented. For its application for qualitative research, survey questions are developed based on each criterion and Likert scale is used for measure.

Figure 2. MBNQA education model (Education Criteria for Performance Excellence. Source: Baldrige Excellence Framework 2015-2016)

Table 3. EFQM Excellence model and MBNQA Education Criteria for Performance Excellence

Previous research studies on MBNQA Education Criteria for Performance Excellence was reviewed. [20] conducted survey of members of 15 United Arab Emirates (UAE) universities and colleges and identified leadership as a key driver and significant linkage between organizational outcomes and all criteria. From the case study of Abbott school district, [21] analyzed that MBNQA Education Criteria for Performance Excellence guided continuous improvement efforts of educational organizations. Research studies applying MBNQA Education Criteria for Performance Excellence was conducted in Korea as well. [19] surveyed elementary and secondary school teachers to analyze the cause and effect relationship between MBNQA Education Criteria for Performance Excellence and education performance. The result reported that leadership of school principal significantly influences on school’s strategic planning, human resources, and measurement/analysis/knowledge management and measurement/analysis/knowledge management influences on school’s strategic planning, human resources, and customers significantly. It also found the significant relationship between school organization and customers. In addition, [22][19] set hypotheses on the MBNQA Education Criteria for Performance Excellence and empirically demonstrated them. BNQA Education Criteria for Performance Excellence is used as basic material for various research including analysis of its application effectiveness, making hypotheses and cause and effect relationship analysis, as well as applying for quality management.

The comparison study of two quality management models evaluated EFQM Excellence model as more suitable quality management indicators for educational institution in terms of concreteness. Indicators of EFQM Excellence model are described more understandable, could be used as a survey tool because each criterion includes 4~5 indicators, and are simple for assessment. In this respect, MBNQA Education Criteria for Performance Excellence was integrated into EFQM Excellence model and education quality management model is proposed in Table 4. Customers criterion proposed in MBNQA Education Criteria for Performance Excellence was added to EFQM Excellence model and results criterion was comprehensively suggested like MBNQA Education Criteria for Performance Excellence reflecting criteria of EFQM Excellence model.

3. Methods

3.1 Literature review

The main method of this study is literature analysis. By analyzing the components of TQM, the education certification system and the education quality management model, areas and indicators related to the quality management model of civil servant education were derived. Based on the results of literature analysis, a draft of the quality management index for educational institution was developed.

3.2 Expert interview

For the draft quality management model for educational institution developed based on the results of literature analysis, revision opinions were investigated for civil servant education experts, and the quality management index for educational institution was developed by reflecting them. The experts participating in interviewers are education experts with more than 10 years of experience working or consulting at education institutions with doctoral degrees in education. (Table 4) shows the characteristics of the participants in the expert interview.

Table 4. Characteristics of the Expert participants

4. Discussion

4.1 Literature review results

TQM factors (Table 1) and Education Quality Management Model (Table 5) were drawn from literature review. And Accreditation of Engineering Education criteria which is a typical accreditation of education was also analyzed. The results comparison in table 4 found that TQM factors and Accreditation of Engineering Education criteria were included in Education Quality Management Model criteria. However, it was identified that compare to the partnerships and resources criterion and process, products, and services criterion of Education Quality Management Model, corresponding criteria of TQM factors and Accreditation of Engineering Education criteria were more detailed. It was because Accreditation of Engineering Education criteria was focused on education program and significant criteria in education field like educational objectives, curriculum, and students were presented in detail. Because this study was about developing quality management indicators for educational institution, emphasis on education was required and educational objectives, curriculum, and students criteria and their contents needed to be considered proactively. Therefore, Quality Management Model for Educational Institution was drawn focusing on the criteria of Education Quality Management Model, provided that criteria were detailed and the titles of criteria were revised reflecting Accreditation of Engineering Education criteria. Table 6 shows the result.

Table 5. Comparison of Education Quality Management Criteria and Indicators

Table 6. Criteria and Indicators of Quality management for Educational Institution

As a result of the comparison in Table 5, ‘Leadership’, ‘Strategy’, and ‘Customers’ criteria and their indicators proposed in Education Quality Management Model were included in Quality Management Model for Educational Institution. But the indicator 6e. Managing customer relationships of process, products, and services criterion in Education Quality Management Model was shifted to customers criterion after it was conclude to be more suitable for customers criterion. ‘People’ criterion was also established centered on the indicator of Education Quality Management Model and identified that employee management and empowerment, active participation, education, communication factors proposed in TQM factors were included. Because faculty is a part of the people of educational institution, faculty criterion of Accreditation of Engineering Education criteria was included in people criterion. Indicator 4f. Reward and recognition of people on education improvement was composed reflecting the contents of the improvement of education program criteria emphasized in accreditation criteria. Partnership and resources criteria of Education Quality Management Model included facilities and environment criterion and program improvement criterion of Accreditation of Engineering Education criteria. Terms of facilities and environment criterion and program improvement criterion were understood better than those of partnership and resources criteria. And quality information system factor and continuous improvement factor of TQM corresponding to program improvement criterion have been emphasized. Thus, ‘Facilities and environment’ criterion and ‘Program improvement’ criterion were established separately and considering the term curriculum is more frequently used in educational institutions than program, the title of the program improvement criterion was modified to ‘Curriculum improvement’ criterion. Process, products, and services criterion of Education Quality Management Model included process/departmental quality management and high quality technology criteria of TQM factors and this criterion corresponded to educational objectives, curriculum, and students criteria of Accreditation of Engineering Education criteria. Because proposed criteria titles in Accreditation of Engineering Education criteria were emphasizing education, they seemed to fit to the criteria titles of quality management for educational institution and the reviews for indicators of corresponding criteria showed that curriculum management could include criteria titles. Therefore, ‘Curriculum management’ was concluded as a criterion title and quality management indicators were proposed based on the contents suggested in Accreditation of Engineering Education criteria. However, high quality technology of TQM factors was added as a factor for the indicator 6i. High quality technology for curriculum planning, development, operation, and evaluation.

4.2 Expert interview results

The following (Table 6) summarizes the opinions of experts on the quality management area and indicators for educational institution derived based on the results of literature research. Results criterion of Education Quality Management Model corresponded to performance management criterion of TQM factors and learning outcomes criterion of Accreditation of Engineering Education criteria. Because performance was also the factor to be managed, performance management was set as a criterion title in the respect of quality management and sub-factors were identified based on the indicators proposed in Education Quality Management Model.

5. Conclusions

The purpose of this study was developing quality management indicators for educational institution. Through literature review, criteria and indicators for quality management were decided. The results are as following.

The eight quality management criteria for educational institution are leadership, strategy, customers, people, facilities and environment, curriculum management, curriculum improvement, and performance management. These criteria were deducted based on TQM factors, accreditation of education which is represented with Accreditation of Engineering Education criteria, EFQM Excellence Model, and MBNQA Education Criteria for Performance Excellence. These eight criteria show that not only the criteria that directly influence on education like curriculum management and curriculum improvement, but other various criteria such as leadership, strategy, customers, people are also required for quality management. TQM emphasizes comprehensive attention and management not just on a part of production process but on leadership, organization, system, environment and so forth [2]. This is also applicable to education quality management. This study results suggest various considerable criteria including leadership, people, and facilities and environment which should be considered in addition to education for quality management of educational institution.

Among the quality management criteria for educational institution, the criteria which give high weight on the quality management indicators in order are curriculum management, people, and performance management criteria. The fact that curriculum management criterion puts highest weight on quality management indicators relates with that the key product of education is curriculum. That is because quality of education is ultimately recognized and judged with curriculum.

The results of this study have following implications. First, this study result could contribute to activate the discussion for quality management of educational institution. Quality management is a management philosophy of process and result for the improvement of institutional service quality and suggests practical way of management as well. But the drivers are hardly prepared yet to recognize the importance of quality management in educational institution and push for it. The result of this study could help start putting efforts on the quality management in educational institution in terms of suggesting the importance, related factors, and critical criteria to consider for quality management. Educational institutions should provide education that meets the needs of students, who are consumers, but also consider aspects related to educational quality [23]. This study has the significance of improving the quality of educational institutions, not just the contents of education, in order to improve the quality of education.

Second, through advancing this study results, the quality management process and system for educational institution could be established and the actions for their expansion and settlement could be made. This study comprehensively compared and reviewed quality management factors, accreditation of education, and institution quality management models and suggested quality management criteria and indicators for educational institution. Based on the results of this study, institutions could start to discuss what kind of actual process and system should be established and with the application of these results, quality management in educational institution could take root and expand. Education programs in the UK are meticulously put into practice, particularly through assessment[24], to help students develop their skills. And the importance of research on educational quality and satisfaction among participants in the educational field is increasing [25]. Therefore, it is necessary to continuously monitor and improve education quality.

This study has limitations that the proposed quality management indicators for educational institution were deducted from literature reviews and validity test was not conducted. Thus, following suggestions are proposed.

First, the research for identifying significant criteria among the proposed quality management criteria for educational institution should be conducted. A study on MBNQA Education Criteria for Performance Excellence considered leadership the most significant [26] and the other study considered workforce and operation criteria more important [3]. Therefore, management in various criteria is required for quality management. If further research based on this study results identifies more significant criteria for quality management of educational institution, the criteria to concentrate on for quality management could be suggested.

Second, the validity test of quality management indicators for educational institution is required. In this study, the quality management indicators were deducted from literature reviews. Thus, empirical validation work should be conducted to generalize the indicators.

Third, a guideline on how each proposed criterion should work in actual education field should be suggested. For this, establishment of professional conference for the quality management system for educational institution is required and through this, the measures for actual application in field should be drawn up.