I. Introduction
Although DAPA aims to achieve budget reduction for the public and provide the best administrative service to the companies and military agencies while procuring military supplies, promoting defense businesses, and acquiring optimal weapon systems, there have been some problems in their IT service management:
- Due to the insufficient integrated management system based on the individual monitoring systems for each system and task, rapid response to system incidents is difficult.
- Information is not shared efficiently as their countermeasures and system recovery methods often depend on an individual’s experience or expertise. Moreover, the instructions for dealing with incidents are mainly managed with documents.
Proper business process and IT service system should be established together with an efficient IT process to maintain or improve the quality of service or develop a new business model [1-2]. In such regard, ITSM (IT Service Management) is the key to the solution. ITIL (IT Infrastructure Library)-based ITSM supports efficient system management without complicating the service process by providing guidance on how to achieve the management objectives strategically while consistently improving the quality of service. Since such ITSM system is expected to be able to provide high-level service to customers who are utilizing IT services in the military acquisition sector, this study attempted to analyze the level of satisfaction with ITSM application in the sector and proposed the development direction for the future [3-4].
As a result of applying the concept of ITSM to these problems, some improvements have been made so that, after analyzing them, an attempt will be made to redefine the future performance goals and present the possibility of continuous development of IT service in military acquisition programs considering the factors involved in achieving satisfactory service. Therefore, this study described the system proposed by a small defense company in the Republic of Korea.
In this study, processes and function automation were implemented with focus on IT services and service desk based on ITIL v2 in order to establish, develop, and operate the processes needed to provide services to customers effectively.
As such, this study is meaningful since it checks the importance of integrated management and satisfaction with various IT services based on those processes.
Due to the advancement of ITIL establishment, however, ITIL v3 is changing according to business value by strengthening the service life cycle viewpoint with focus on business value (ranging from IT planning to service design, operation, improvement, and strategy), strengthening IT governance support, and defining IT service as an asset that provides utility and assurance for business.
The recently released ITIL v4 focused on adaptation, adjustability, customization, and flexibility in IT service management. ITIL v4 expanded the existing concept with focus on value creation, process automation, and performance improvement, and it is increasing the importance of governance, reliability, stability, information security, and data security.
On the other hand, this study conducted research on the integrated management of IT services provided by Korea's national defense acquisition system, which was developed based on ITIL v2 as the existing system, and investigated the level of satisfaction with services with focus on ensuring that it can be used for understanding the necessity of the system and its advancement in the future.
II. Related Research
In the domestic ITSM market, with the announcement of ITILv3, the IT Governance field is expected to be added to the existing service desk and service level management fields operated with ITILv2. The expansion to IT Governance, which will include project and portfolio management, signifies the further advance of ITSM [5], [6]. Major target companies are those that have introduced ITSM systems into their business for more than two or three years, accounting for about 20~30% of all companies adopting ITSM. Thus, the demand for achieving IT Governance is expected to grow continuously for the ITSM market; based on such forecast, all the companies mentioned earlier are focusing on their customer requirements to increase customer satisfaction with their ITSM systems as well as seeking maximum return on their investment [7], [8].
In summary, we would like to emphasize again that the main contribution of this study is that it has proven, through scientific analysis, that the overall satisfaction level increased by managing the problems associated with service inconveniencies or operation efficiencies of the previous distributed systems.
On the other hand, ITIL v4 was officially released in the first quarter of 2019. ITIL has not been changed significantly since the last update in 2011 save for minor changes, but its framework is changed to a large extent [9], [10].
As the characteristic of the new version, the existing concept is expanded with focus on value creation, process automation, and performance improvement while maintaining the existing framework of ITIL v3. In particular, it is noteworthy that enterprise collaboration and communication are encouraged, and changes are made to accommodate agile and DevOps (Development Operations) in the ITIL framework [15]-[19].
2.1. History of ITIL
Let us look into the root of ITIL before describing what agile and DevOps are and why international IT service standards accommodate such culture [10], [11].
The CCTA (Central Communications and Telecommunications) Agency in the UK devised a set of recommendations for "a practical, convenient framework for businesses to define, plan, support, and provide IT services” in the 1980s. ITITL v1, which began as a collection of books on how to manage and implement IT services based on the CCTA recommendations, was composed of more than 30 books and published from 1986 to 1996 [12], [13].
Published between 2000 and 2001, ITIL v2 is a set of books on the process guidelines related to various areas of IT such as management, application, and service. The books were combined and published as eight books. CCTA in the UK was integrated into the OGC (Office of Government Commerce) in April 2001 [14].
OGC released “ITIL v3” in May 2007, which is currently known as the ITIL 2007 edition. ITIL v3 consists of 26 processes and functions and covers five key processes.
On the other hand, ITIL was updated again in July 2011. The guidelines, which clearly defined the processes that have not been defined at the time, were included, and the problems of the previous versions (lack of consistency and various errors) were fixed before completing the current ITIL v3.
2.2. Agile and DevOps
The Agile methodology is a software development methodology that is a compromise between unplanned development methods and excessively planned development methods. The waterfall development method, a traditional development method, was borrowed from the construction field; it consists of requirement definition → analysis → design → implementation → test → execution [15]-[17].
One of the important weaknesses of Agile Development when dealing with high-integrity software system is that it is lack of proven method in dealing with nonfunctional requirements [18]. Other method shows [19] a promising opportunity to improve Agile Development [20] from the requirements aspect.
Nonetheless, the waterfall development method has a disadvantage, i.e., it is difficult to change in line with the customer’s requirements because the next phase cannot be started before the previous step is completed, and the development output can be checked in the later development phase only.
On the other hand, the agile development method is used to develop software gradually and repetitively like an assembly line in the manufacturing industry. There is a higher possibility of obtaining needed software at the end, because development is conducted by repeating short-term development and receiving feedback repeatedly, which is called “Iteration.”.
In other words, unlike the previous methodologies that took the lead through planning, the agile development methodology creates prototypes ceaselessly at regular intervals without development based on forecast and develops large-scale software by adding and modifying the basic requirements from time to time as the need arises.
DevOps is a compound word consisting of software development and operation and can be defined as any development environment or culture that emphasizes communication, collaboration, and integration between software developers and IT professionals. It is a culture wherein interdependent response between software development organizations and operation organizations is emphasized, and organizations develop and deploy software products and services within a short period of time. This culture enables actual task performers to concentrate on service improvement activities at a higher level by developing and deploying services as soon as possible through the automation of tasks that were already proven and repeated by the organization through the integration of development and operation. The ultimate goal of DevOps is to increase efficiency and raise effectively the standard of the service provided by the enterprise quickly from the perspective of the customers who actually use the service [20], [21].
2.3. Impact of ITIL v4
Agile and DevOps are still unfamiliar to or rejected by the Korean public sector and institutions. Likewise, many persons in charge and developers in corporations consider Agile to be an impractical or an unrealistic concept that does not match the reality of Republic of Korea.
On the other hand, ITIL has been continually developed mainly by foreign joint ventures since its introduction in Korea in 2000. Governments and many companies are using the ITSM solution that implements the ITIL v3 concept. The ITSM currently used by many institutions has its limitations, however.
Most ITILs started as a form of SLA (Service Level Agreement) requested by the user to manage effectively the IT infrastructure in the early days. It began by applying first the matter that can be applied in the IT infrastructure environment, which is easy to measure and manage, because it was difficult to find SLA-based cases in Korea. Focusing on infrastructure-focused cases, infrastructureoriented indicator management, and SLA measurement items since then, the ITSM solution was also developed in line with such matters.
Given this situation, even the ITSM solutions introduced by organizations that manage infrastructures and services in an integrated manner are concentrating on infrastructures. Many IT managers are having difficulty managing software or services using ITSM, because software control knowhow or indicators are not applied. The same is true for most private companies.
The biggest difficulty lies in managing quick and various requests for service improvement or change using the existing ITIL. The decision-making process became complex as the focus of ITIL is the CAB (Change Advisory Board), which is inevitably accompanied by the problem of taking a long time before end-user demands are reflected to real services.
Second is the difference in viewpoint. When an IT service is developed and launched, operation is performed. The purpose of development is to modify and patch quickly when a problem occurs, but that of operation is to provide stable services. As a result, operation tries to avoid changes because it can be difficult if a server fails due to change. Such difference in point of view inevitably leads to conflict.
Third is the lack of flexibility. Recently, communication with IT departments has become increasingly more important as more departments need to access IT and IT systems from the service aspect. Nonetheless, the existing ITIL v3 has limitations in reflecting this requirement flexibly.
Fourth, introducing an ITSM that complies with ITIL v3 costs a lot. Since raising quality by improving the way IT services are managed does not come cheap, it was difficult for companies with insufficient financial resources to adopt and manage ITSM. As a result, conglomerates and SMBs could not use ITSM universally. There was also the problem of insufficient experience in best practices and better management indicators, which could be obtained if ITSM was used in many places [21].
If ITIL v4 including Agile and DevOps is released as a solution to cope with these difficulties, the Korean government and many enterprises are expected to follow suit. Accordingly, many ITSM solutions will implement the Agile and DevOps concept.
If it is done, it will be a solution to the current ITIL v3-based operations that currently experience difficulties. Software development and operations will be managed by the ITIL management tools applied with ITIL v4, instead of a separate project management tool. Literally, DevOps, development and operation, will be linked as one to provide continuous, fast services.
2.4. Definition of ITSM
ITSM is an IT service management framework that aims to prevent the deterioration of service quality due to diversified IT management environments, lack of standard management system, and unstable system operation and provide the details of the standard IT service management process, organization, and technology based on ITIL, the best operation/management practice for securing higherlevel service quality and competitiveness [22-24].
As the best practice of ITSM, ITIL is a reference of the IT-related standard operation process for companies. ITIL originated from the 45 books first published in 1986 by British government agency CCTA, currently known as OGC to provide guidance for systematically managing the information infrastructures of all government agencies.
Figure 1 presents the elements making up an ITSM system, the most important of which is the IT process, which is the process of providing and supporting IT service. The next most important part involves manpower and organization. It is essential that manpower with the skills and abilities necessary for providing optimal IT service be fostered/secured, that an organization be established with them, and that an appropriate role be assigned to each of them.
Fig. 1. ITSM Process.
The technology used here refers to the tool or solution required to provide optimal IT service; without such element, the effects of ITSM cannot be realized properly. Recently, many solution companies have released solutions related to ITSM. Figure 2 shows ITSM System Layer.
Fig. 2. ITSM System Layer.
2.5. ITSM Process Model
In examining the IT process models for ITSM, some of the major models include CobiT (Control Objectives for Information and related Technology), ITIL, and CMM (Capability Maturity Model). Among them, the one most closely related to ITSM and which has relatively detailed contents is ITIL. The core elements of ITSM are the service delivery and service support processes, which are considered to be the two major components of the service management presented by the ITIL framework as part of ITIL as a set of best practices (i.e., standardized best practices of IT resources, organization, and operations for IT service management). The service delivery process sets the level of IT service to allow a company to maintain it to achieve its business goals, whereas the service support process consists of support processes for service provision. The major processes are described as follows.
2.5.1. ITIL and eSCM
Although the models for ITSM include CMMI (Capability Maturity Model Integration), MOF (Managed Object Format), ITIL (IT Infrastructure Library), eSCM (eSourcing Capability Model), etc., only ITIL and eSCM will be introduced in this paper. ITIL is recognized as the industry standard.
2.5.2. ITIL (IT Infrastructure Library)
First developed by CCTA (Central Computer and Telecommunication Agency; UK), ITIL (IT Infrastructure First developed by CCTA, ITIL is a library that has been published in book form and which contains practices associated with IT service operation. Although ITIL had focused on the operation of IT infrastructure in the early stage, it has been expanded by including a series of best practices for managing IT services after its effectiveness was validated by several organizations and it was placed under the supervision of OGC. It is primarily divided into two domains: Service Delivery, which defines all types of processes necessary for the contract and monitoring of the service level agreed upon with customers; and Service Support for the operation and support of service delivery processes.
The respective service modules of the former and the latter are not operated or applied separately but can be completed by the overall flow and circulation of processes. ITIL-based certification (BS15000) can only be given by BSI; even though OGC currently has the right, many IT companies including MS (Microsoft), HP (Hewlett-Packard Co.), IBM (International Business Machines Co.), CA (CA Technologies), etc. are participating in its development and the supplementary work for each domain, based on the principle of making it applicable to and independent of the client-server environment and e-business environment of any company. ITILv3 is currently being adopted, and it is possible to acquire ISO90012 (International Standardization Organization) or BS15000 (British Standard 15000) certification if a certain service adopts this standard [22]-[24].
2.5.3. eSCM (eSourcing Capability Model)
eSCM was originally developed to assess the competency of IT outsourcing service providers objectively and improve the quality of service. At the same time, it is the model for evaluating the capability of IT service providers. eSCM was developed by CMU (Canadian Maritime Union) after benchmarking the existing ISO9000, MB (Malcolm Baldridge), COBIT (Control Objectives for Information and Related Technologies), ITIL, etc. It consists of some of the major issues pertaining to outsourcing tasks, such as proposal, negotiation, contract, service design, construction, service provision, and service level management, and the definitions of 85 practices in 10 areas (i.e., knowledge management, manpower management, performance management, relationship management, risk management, contract management service design & establishment, service provision, service transfer). Similar to CMM (Capability Maturity Model), it adopts five competency levels for certification by CMU [25]-[27].
2.5.4. Comparison between ITIL and eSCM
As the working-level practices for service quality improvement for IT organizations, ITIL places emphasis on the basic service operation and support tasks necessary for ensuring the internal stability of the IT-related department/section of an IT company to improve their IT service quality. In contrast, as the first specialized certification standard for IT outsourcing services, eSCM has as its main interest the management of the relationship between the IT outsourcing provider and customer and service proposal/negotiation/contract/design and construction/provision/support until the end of the contract. Table 1 shows the comparisons. In short, while eSCM provides the guidelines for the pre-business phase, ITIL specifies the service design, establishment, and provision stages [28], [29].
Table 1. Comparison with eSCM.
Recently, the concept of ITSM has become a popular solution for issues pertaining to corporate IT service management; it has become clear that most of the company businesses categorically depend on the quantity, quality, and availability of services provided and supported by the ICT infrastructure when IT infrastructure is used widely.
Therefore, the insufficient working capital and lack of flexibility of SMEs in terms of R&D have always been the major issue especially when it comes to developing new but innovative weapons systems requiring cutting-edge technologies. Their limited service and support to the customers often cause them to fall way behind large and multinational corporations. Still, it does not mean that their potential should be disregarded, as history shows that small but innovative companies can be quite successful in the hitech field. Their technical or technological capability is still vital to the military industry as it can break fresh ground for the future with novel concepts. With just a little support, the entire military industry can enjoy the benefits of partnering with SMEs when developing new and advanced weaponry or weapons systems.
2.6. Information Service Quality Measurement
2.6.1. Service Quality
The definition of service varies by each researcher, but we can see that each researcher makes a variety of definitions, with all researchers agreeing on the intangibility and characteristics consumed at the same time as production. Kim [30] and Han [31] have noted that a service is defined as an intangible asset that is produced by the service provider by providing part of the work or produced by the service provider and the customer interacting to meet the needs of the customer.
An et al. [32] defined the service as follows: it defined a service as an intangible activity that provides customers with some satisfaction with performance but cannot be owned or stored and transported.
According to the Parasuraman, Zeithaml and Berry [33], the concept of quality of service is harder to assess than quality of product, the perception of quality of service comes as a result of comparisons between consumers' expectations and actual performance of services, and the assessment of the quality of service by consumers also includes an assessment of the delivery process of services, including service performance. Grönroos [34] explained that "Perceived service quality" is perceived as a difference between expected and experienced quality and noted that the quality of two dimensions of process quality and resulting quality is formed through a filter called the image of the service provider or service provider.
Parasuraman, Zeithaml and Berry [33] have developed the SERQUAL model. The model consists of the service quality into 22 categories of five dimensions: tangible, reliable, responsive, confident, and empathy. Meanwhile, Rust and Oliver [35] have proposed that service quality in three dimensions: service product, service delivery and service environment. Lee and Ra [36] highlighted that service quality in three dimensions is referred to as process quality, result quality and environmental quality, and can be evaluated as a very logical and comprehensive model from an intuitive or practical point of view.
2.6.2. Service Value
Ettinger [37] said that in order to establish a successful strategy, consumers should evaluate the service value recognized by consumers, saying that the service value recognized by consumers is directly affected by the quality of the services received. Lee [38] pointed out that most studies show that quality of service affects the value of service, including that of quality of service affects the value of service. Conversely, Parasuranman et al. [39] noted that system availability, the nature of system quality during service quality, is not significant to perceived value, namely service value. Most studies have pointed out that the higher the quality of the service, the higher the value of the service, but that the relationship between the two is not significant. Therefore, it is meaningful to analyze whether ITSM's quality of service factors have a significant positive impact on the value of the service.
2.6.3. Service Satisfaction
Richard, L.O [40] attributed satisfaction to emotional variables formed by prior factors and quality of service to the recognition of the recognition of excellence in the individual attributes of services. Choi and Kang [41] studied and analyzed that customer satisfaction of hospitality companies (hotels, restaurants, lodging, etc.) does not have a significant positive impact on financial performance, but customer satisfaction of non-return companies (banking, hospital, distribution, etc.) has a significant impact on financial performance.
2.6.4. Service trust
Cha [42] pointed out that trust builds a lasting relationship when the customer is positive or emotionally trusted with the employee, and that a successful relationship with the service provider and the customer strengthens the relationship by allowing the service provider to trust in the customer's interests and to immerse itself in that relationship. Kim [43] noted that service confidence is being used as a concept similar to belief, conviction, expectation, consistency, safety, and so on, but various concepts are established according to the perspective, circumstances, and targets of the study on trust, which in marketing is 'trusted, belief, expectation, or certainty' and secondly 'convincing the other's dependence on the other. Service trust can be seen as being formed when the performance expected in advance by actual perceptual performance is reached. By summing up the preceding studies, a customer confidence is the same concept as service confidence and the customer confidence term is judged by service trust and research analysis was conducted.
2.6.5. Relationship among service confidence, value, and satisfaction
Through consideration of various research literature, we reviewed the identification of the relationship among service confidence, value and satisfaction. In particular, Park analyzed that the value of a service does not directly affect service confidence, whereas it affects service confidence through an intermediary variable called service satisfaction [44]. Kwon et al. [45] demonstrated that visual value and investment value had a positive impact on trust, the higher the value perceived, the higher the service satisfaction, and the more satisfied the customer, the higher the confidence. Through consideration of various research literature, the service value and the service satisfaction have been identified as variables that affect the service confidence. In this study, we verify whether ITSM's service value has a significant positive impact on service confidence, along with research and analysis of whether service satisfaction factors have a significant positive impact on service confidence [46]-[51].
III. ITSM Military Service
Figure 3 shows the comprehensive status board which is used as a window to achieve smooth communications. The integrated operation management system provides the IT service management status through a digital dashboard after establishing linkages between the service desk, service level management system, integrated resource management system, and integrated security management system. The weekly failures, entire structure compositions, weekly failure developments, weekly failure handlings, and weekly change-processing rates are presented with graphs on the board and the failure status can be retrieved based on aspecific condition such as the name of the institution or task entered arbitrarily. Additionally, detailed information such as the status for processing requests, failure registration information, failure result information of particular failure is provided.
Fig. 3. ITSM Military Service comprehensive status board.
In Figure 4, the basic data for calculating service levels provides the individual baselines for the number of failures, target-time compliance rates, rate of operation, security breach incidents, and user satisfaction based on the (SLA) Service Level Agreements. The information collected by the integrated operation management system for the service levels is used to calculate individual service levels based on the service target, operation & performance information, and service level management agreement. The resulting service levels are then provided to the user/operator as the information with which he/she can assess and manage his/her own levels.
Fig 4. Military Service Level Agreements (MSLA)
IV. Performance Evaluation: Concepts of Service Satisfaction and Analysis of Hypothetical Models
This study analyzed the characteristics of empirical samples by using the Mahalanobis distance to check the sincerity of 155 responses to the survey and used 154 data, excluding 1 response, for the analysis. Table 2 shows the characteristics of the survey samples.
Table 2. General Characteristics of Survey Subjects.
In terms of the position of respondents of the survey, those working in the administrative division and having experience with ITSM services accounted for the highest portion. The analysis of the survey subjects showed that 84% were middle managers or higher-ranked, confirming the validity of the response.
Table 3. Results of Reliability Analysis.
All items had Cronbach's alpha coefficient of 0.6 or higher; thus, they were confirmed to have internal consistency.
Reliability means that “the same results are obtained when the measurement target is measured multiple times, and there is internal consistency among the items constituting an indicator.” We used Cronbach's alpha coefficient to analyze and confirm the internal consistency of reliability and the SPSS program for statistical analysis and confirmation. Cronbach's alpha coefficient is used to improve the measurement reliability by identifying items with low internal consistency and removing them from the measurement. The alpha coefficient is a value between 0 and 1, and a high value means high reliability. A value between 0.8 and 0.9 is generally considered highly reliable, and a value between 0.6 and 0.7 is generally considered reliable. Items with alpha coefficient of 0.6 or lower are considered to have no internal consistency. Table 3 shows results of reliability analysis.
V. Performance Evaluation: Feasibility Analysis
5.1. Exploratory Factor Analysis
Table 4 shows the result of the exploratory factor analysis of the measured items. The first exploratory factor analysis of ITSM's service quality factor showed sufficient total load value of 67.133%, but the second exploratory factor analysis was performed since system quality factors 4, 5, and 6 were not grouped and were consequently removed although system quality factors 1, 2, and 3 were grouped. As a result, the total load value increased to 72.191%, confirming that the same grouping of the hypothetical study model was possible.
5.2. Fit Analysis of Individual Factors
The fit analysis of individual ITSM service quality factors analyzes the similarity between the study model and individual factors. The high fit range of an individual factor means that the study model represents the factor well. The analysis results indicate high convergent validity of the two relationships. Table 5 shows the chi-square and RMR values used for the model fit analysis.
The analysis of the chi-squared values checks the fit with the measured item of the questionnaire. This study performed the fit analysis, but the result was meaningless since the chi-squared p-value was significant. As such, we performed the fit analysis with RMR, and it showed that the quality factor was fit when the RMR value was 0.08 or lower. Since all RMR values of the factors are in the range of 0.08 or lower, we conducted the study by reflecting the first and additional exploratory factor analyses and excluding human quality factor 6, system quality factor 4, system quality factor 5, system quality factor 6, result quality factor 4, and result quality factor 5.
5.3. Fit Analysis of All Factors
We analyzed the factor load with the analysis result of the convergent validity of the potential factors to check the fit of all factors of the hypothetical study model.
Table 6 shows the result of the first-factor load. A factor load square of 0.5 or higher mostly indicates important convergent validity, and that between 0.4 and 0.5 indicates convergent validity.
Table 4. Results of Exploratory Factor Analysis.
“Factor extraction method: Principle component analysis Rotation method: Varimax with Kaiser normalization” The measured items of the service human quality factor, system quality factor, and resultant quality factor of ITSM with low component value were removed to ensure the consistency of the service factors of the study mode
Table 5. Results of Fit Analysis.
*: The analysis results are the values after removing the questions that show a low level of overlapping.
Table 6. Result of 1st Factor Load Analysis.
The most desirable range is 0.7 or higher, but it is not specified. We conducted the second-factor load analysis after excluding human quality 6, which showed load square of 0.4 or lower, system quality factors 4, 5, and 6, and result quality factors 4 and 5, which were not grouped in the factor analysis. Table 7 shows the result.
The results of the second-factor load analysis showed that the standardized factor load squares were 0.4 or higher, confirming the impact of the convergent validity.
We then checked the discriminant validity. We compared the AVE of a potential factor with the correlation squared of the potential factor and considered the existence of discriminant validity if the AVE was larger than the correlation squared.
Table 7. Result of 2nd-Factor Load Analysis.
VI. Hypothetical Model and Its Analysis for Research on ITSM Service Satisfaction
The trend of recent research works on public institutions is to assess not only the quality of the service provision process but also the result. Thus, a statistical method often used in the field of public service at present has been used to perform scientific analysis and obtain objective results. This study divided the service quality of ITSM into “process quality” and “resulting (outcome) quality”: human quality and system quality define process quality since the service provision process is processed by people and internal system.
The hypotheses and models below were established on the premise that each element of the ITSM service quality would significantly affect the service value.
Hypotheses for the components of service quality and value.
[Hypothesis H1]
H1.1： Human quality will have a significant positive impact on service value.
H1.2： System quality will have a significant positive impact on service value.
H1.3： The resulting quality will have a significant positive impact on service value.
Hypotheses for the components of service quality and satisfaction.
[Hypothesis H2]
H2.1: Human quality will have a significant positive (+) influence on service satisfaction.
H2.2: System quality will have a significant positive (+) influence on service satisfaction.
H2.3: The resulting quality will have a significant positive (+) influence on service satisfaction
The hypotheses have been formulated based on the view that service value affects service satisfaction. Likewise, based on the views that service value has an influence on the reliability of service, and that service satisfaction has a positive influence on reliability, the following hypotheses and model have been established:
(3) Hypotheses for service value, satisfaction, and reliability
[Hypothesis H3, 4, 5]
H3： Service value will have a significant positive impact on service satisfaction.
H4： Service value will have a significant positive impact on service reliability.
H5： Service satisfaction will have a significant positive impact on service reliability.
A comprehensive hypothetical model for the correlations between the variables of components of service quality is shown in Figure 5.
Fig. 5. Hypothetical Model.
Meanwhile, the measurement items were set by considering the items related to actual IT service requests or support after checking the existing research works on the ITSM service satisfaction, and a Likert 7-point scale was used. Table 8 shows measurement items for the ITSM service satisfaction research hypotheses.
Table 8. Measurement Items for the ITSM Service Satisfaction Research Hypotheses.
VII. Analysis of Service Satisfaction and Discussion
ITSM achieves consolidated management for the IT services supporting an acquisition system, and no outside connections can be established with such ITSM. Issues pertaining to the D2B can be addressed to System Q and A or a Call Center for problem-solving. In other words, internal staff can take the necessary measures for problems by directly connecting with ITSM.
We believe that satisfaction with the system can be used for system advancement and for checking organizational performance regarding the systems that use similar public institutions.
7.1. Discussion
The DAPA is operating its IT service with the aim of providing the best services for both corporations and military in terms of service efficiency or budget reduction for the procurement of military supplies, promotion of the defense industry, and acquisition of the best weapon systems. The obstacles in the information system lead to budget loss or hamper securing the feasibility of informatization businesses; in extreme cases, they can cause heavy losses by crippling the business. Such obstacles also delay the provision of information services, thereby not only raising the dissatisfaction level of users but also causing loss of credibility and efficiency. These problems have been gradually improved since the introduction of ITSM, and the significance of this study lies in realizing the best administrative services by presenting the effects of the IT service factors on service quality through analysis/research on the ITSM operation. The level of prompt, accurate provision of the required information through the introduction of various types of systems affects the credibility of the systems as well as the reliability of the organizations or agencies providing the services. The study reconfirmed the necessity of ITSM and the service quality factors largely affecting the credibility of the services.
7.2. Hypothesis Testing
By conducting a survey on the satisfaction or reliability of ITSM service, hypothesis testing was performed for the major items affecting the service.
While a service factor such as human or output (resulting) quality has a direct impact on the service value, system quality affects it indirectly without any direct impact.
Service satisfaction is tantamount to reliability of service through human and output qualities. The prerequisite for reliable ITSM service is service satisfaction on which human and output qualities have a direct influence. Due to the characteristics of the military, service satisfaction has an indirect rather than direct influence on system quality.
The fact that the factors of system quality such as design, page speed, search speed, etc. do not have a direct influence on service value or the characteristics of the closed world of the military organization where emphasis is placed on the continuity of business can be recognized through hypothesis testing.
To test the hypotheses established in this study, 320 survey responses were collected, and 154 valid ones were checked with Mahalanobis distance to investigate the correlations between ITSM service quality, value, satisfaction level, and reliability and each variable.
The correlations and the discriminant validity between the potential factors were determined. Discriminant validity is determined by comparing the AVE (Average Variance Extracted with the square of the correlation; if the value AVE is greater than the latter, the validity is considered to be affirmative. The arithmetic expression and the results are shown below. Table 9 shows the Square of Correlations and Discriminant Validity. Table 10 presents the Result of Hypotheses Testing.
\(AVE = \frac { \sum_{i=1}^n \lambda i 2 } {\pi}\) (1)
where λ𝒾 = A standardized factor loading of the first measurement variable and Π = No. of measurement variables.
Table 9. Square of Correlations and Discriminant Validity.
*The AVE values are highlighted in bold.
The results of hypotheses analysis can be summarized as follows. First, concerning the correlation between the components of service quality and service value, human quality and resulting quality were determined to have a significant positive influence on service value.
Second, among the service quality components, human quality, system quality, and resulting quality had no significant direct influence on service satisfaction. Such result may be attributed to the fact that, regardless of these three qualities, only rapid failover or particular problemsolving service was required in terms of operation (business) continuity or service availability; even considering the service satisfaction was deemed to be an obstacle. This could be the typicalness of the organizations or members involved in the military acquisition programs that mandate them to perform special tasks in a high-security environment. Nevertheless, it was possible to find an indirect influence on service satisfaction through the intermediate variable “Service value.” This supports the existing research result that service satisfaction holds a higher position in terms of quality than service value considering the fact that the accumulated service values lead to service satisfaction in the end. Third, for the correlations between service value, service satisfaction, and service reliability, service value had a direct influence on service satisfaction, which in turn had a direct influence on service reliability.
Table 10. Result of Hypotheses Testing.
Nonetheless, service value was determined not to affect service reliability directly. Fourth, the results showed that, due to the particular characteristics of DAPA’s organization, members, and operation as well as unique security-oriented characteristic, the nature of pursuing efficiency in business through operational continuity and availability does not have a direct influence on service reliability but actually wields an indirect impact on it. As mentioned above, the structural equation model presented in this study has been verified as an adequate model. The verification results of the structural hypotheses related to the model are shown in Figure 6.
Fig. 6. Result of Verification for the Hypothetical Model.
We believe that satisfaction with the system can be used for system advancement and for checking organizational performance regarding systems that use similar public institutions.
VIII. Conclusion and Implications
This study confirmed the effect of ITSM service in military acquisition agency on increasing the efficiency, and this section discusses the implications through the empirical study of the ITSM service quality factors.
First, we can confidently say that this study broadened various academic study domains regarding the effects of ITSM on increasing efficiency and ITSM service quality factors. There have been many studies on the effect of ITSM and ITSM service quality factors for services in the private sector and applications of various study results. Nonetheless, studies of the military sector are limited due to security, and it is difficult to use data for the same reason. The most important implication of this study is that it attempted to apply various studies for the private sector to the military sector. The potential benefits of ITSM include targeting enhanced organizational performance through organizational specialization and outsourcing by adopting the IT business processes and securing the transparency, fairness, and reliability of the specialized military acquisition administration.
Second, this study expanded the analyzed service quality factors and examined the relationships of various service quality factors that are not standardized service quality factors to reflect the special attributes of the military acquisition agency. It also reported results that were different from the results of the studies in the existing literature.
Third, this study is meaningful since it is a study of the adoption of ITSM for the methodical management of the IT system using the public data by utilizing the limitation from the special nature and closed limitation of security, although there is shortage of data related to the special attribute of the military acquisition agency. It is also a systematic study on the impact of service quality on service values and satisfaction with service.
Fourth, the results of this study provide a useful reference to ITSM service providers and organization managers. We can confidently say that providing the necessary and available information services increases the efficiency of business continuity and availability, thereby possibly serving as one of the practical measures to enhance the service reliability of the agency and increasing the transparency, fairness, and efficiency of the organizational culture.
Abbreviations
DAPA: Defense Acquisition Program Administration
ITSM: Information Technology Service Management
ITIL: IT Infrastructure Library
DevOps: Development Operations
CCTA: Central Communications and Telecommunications Agency
OGC: Office of Government Commerce
SLA: Service Level Agreement
CAB: Change Advisory Board
CobiT: Control Objectives for Information and related Technology
CMM: Capability Maturity Model
CMMI: Capability Maturity Model Integration
MOF: Managed Object Format
eSCM: eSourcing Capability Model
MS: Microsoft Co.
HP: Hewlett-Packard Co.
IBM: International Business Machines Co.
CA: CA Technologies Co.
ISO90012: International Standardization Organization
BS15000: British Standard 15000
CMU: Canadian Maritime Union
MB: Malcolm Baldridge
COBIT: Control Objectives for Information and Related Technologies
SLA: Service Level Agreements
MSLA: Military Service Level Agreements
AVE: Average Variance Extracted
Acknowledgements
This work was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF-2017S1A5B8060156)
References
- Dimou I., Wietgrefe H., Van Selm M., and Janovszki L., "Demonstration of a Cross Security Domain Service Management Capability for Federated Missions," in Proceedings of IEEE Military Communications Conference (MILCOM), pp. 184-191, 2014.
- Je S. M., "Zigbee-based Local Army Strategy Network Configurations for Multimedia Military Service," Journal of Multimedia Information System, vol. 6, no. 3, pp. 131-138, 2019. https://doi.org/10.33851/JMIS.2019.6.3.131
- Steinberg Randy A., Measuring ITSM. Trafford Publishing, 2013.
- Lee S., Woo H., and Shin Y., "High performance data exchange architecture design and implementation for software development between disconnected networks," in Proceedings of the 7-th International Conference on Advanced Information Management and Service (ICIPM), pp. 27-32, 2011.
- Abraham Davila., Rosanna Janampa., Paula Angeleri., and Karin Melendez., "ITSM model for very small organisation: an empirical validation," IET Software, pp. 1, 2019.
- Manuel Mora., Fen Wang., Jorge Marx Gomez., and Oswaldo Diaz., "A Comparative Review on the Agile Tenets in the IT Service Management and the Software Engineering Domains," in Proceedings of International Conference on Software Process Improvement, Springer, Cham, pp. 102-115, 2019.
- Lee S., Woo H., and Shin Y., "Study on Personal Information Leak Detection Based on Machine Learning," Advanced Science Letters, vol. 23, no. 12, pp. 12818-12821, 2017. https://doi.org/10.1166/asl.2017.10906
- Lee Y. W., "An Empirical Study on Evaluation Criteria of Projection Mapping Videos in Terms of Public Design," Journal of Korea Multimedia Society, vol. 18, no. 8, pp. 950-959, 2015. https://doi.org/10.9717/kmms.2015.18.8.950
- McNaughton Blake, Pradeep Ray, Lundy Lewis, "Designing an evaluation framework for IT service management," Information & Management, vol. 47, no. 4, pp. 219-225, 2010. https://doi.org/10.1016/j.im.2010.02.003
- Heikkinen Sanna, and Marko Jantti, "Studying Continual Service Improvement and Monitoring the Quality of ITSM," in Proceedings of International Conference on the Quality of Information and Communications Technology, Springer, Cham, pp. 193-206, 2019.
- Manuel Mora., Mahesh Raisinghani., Rory V. O'Connor., Jorge Marx Gomez., and Ovsei Gelman., "An extensive review of IT service design in seven international ITSM processes frameworks: Part I," International journal of information technologies and systems approach (IJITSA), IGI global, vol. 7, no. 2, pp. 83-107, 2014. https://doi.org/10.4018/ijitsa.2014070105
- Abir El Yamami., Khalifa Mansouri., Mohammed Qbadou., and El Hossein Illoussamen., "Introducing ITIL framework in small enterprises: tailoring ITSM practices to the size of company," International Journal of Information Technologies and Systems Approach (IJITSA), vol. 12, no.1, pp. 1-19, 2019.
- Kim A., Lee S., Lee H., and Shin Y., "Fair resource allocation scheme using cooperative game theory in layer 1 VPN," in Proceeding of the 8th International Conference on Computing Technology and Information Management (NCM and ICNIT), vol. 1, pp. 7-10, 2012.
- Varga Sergio, Gilmar Barreto, Paulo D. Battaglin, "Increasing Information Systems Availabiliy Through Accuracy, Awareness, Completeness and Manageability of ITSM," in Proceeding of the 14th Iberian Conference on Information Systems and Technologies (CISTI), pp. 1-4, 2019.
- Kim S. K., "A Study on the Effect of Blockchain on Personal Information Protection," Journal of Multimedia Information System, vol. 6, no. 3, pp.125-130, 2019. https://doi.org/10.33851/JMIS.2019.6.3.125
- Freddy Aquino., Diego Pacheco., Paula Angeleri., Rosanna Janampa., Karin Melendez., Abraham Davila., "Information Technology Service Management Processes for Very Small Organization: A Proposed Model," in Proceedings of International Conference on Software Process Improvement, Springer, pp. 55-68, 2018.
- Stanley J. E., Mills R. F., Raines R. A., and Baldwin, R. O., "Correlating network services with operational mission impact," in Proceedings of IEEE Military Communications Conference (MILCOM), pp. 162-168, 2005.
- R. Chapman, N. White, and J. Woodcock, "What can agile methods bring to high-integrity software development?" Communications of the ACM, vol. 60, no. 10, pp. 38-41, 2017. https://doi.org/10.1145/3133233
- Sun Peng, Jingwei Yang, Hua Ming, and Carl K. Chang, "A Multi-layered Desires Based Framework to Detect Users' Evolving Non-functional Requirements," in Proceedings of IEEE 42-nd Annual Computer Software and Applications Conference (COMPSAC), vol. 1, pp. 28-37, 2018.
- Ramsden Graham, "Agile ITSM Transformation," ITSM Press, pp. 1-176, 2018.
- HT Jung, Data net http://www.datanet.co.kr/news/articleView.html?idxno=131756 (accessed on 22 October 2019) (In Korean).
- Hochstein Axel, Rüdiger Zarnekow, Walter Brenner, "ITIL as common practice reference model for IT service management: formal assessment and implications for practice," in Proceedings of IEEE International Conference on e-Technology, e-Commerce and e-Service, pp. 704-710, 2005.
- Seo Y. S., and Bae D. H., "On the value of outlier elimination on software effort estimation research," Empirical Software Engineering, vol. 18, no. 4, pp. 659-698, 2013. https://doi.org/10.1007/s10664-012-9207-y
- Tan Wui-Gee, Aileen Cater-Steel, Mark Toleman, "Implementing IT service management: a case study focussing on critical success factors," Journal of Computer Information Systems, vol. 50, no. 2, pp. 1-12, 2009.
- Winniford MaryAnne, Sue Conger, Lisa Erickson-Harris, "Confusion in the ranks: IT service management practice and terminology," Information systems management, vol. 26, no. 2, pp. 153-163, 2009. https://doi.org/10.1080/10580530902797532
- Iden Jon, and Tom Roar Eikebrokk, "Implementing IT Service Management: A systematic literature review," International Journal of Information Management, vol. 33, no. 3, pp. 512-523, 2013. https://doi.org/10.1016/j.ijinfomgt.2013.01.004
- Braun Christian, Robert Winter, "Integration of IT service management into enterprise architecture," in Proceedings of the 2007 ACM symposium on Applied computing, pp. 1215-1219, 2007.
- M. W. Johnson.; A. Hately.; B. A. Miller.; R. Orr., "Evolving standards for IT service management," IBM Systems Journal, vol. 46, no. 3, pp. 583-597, 2007. https://doi.org/10.1147/sj.463.0583
- Antoni Lluís Mesquida., Antonia Mas., Esperança Amengual., Jose A. Calvo-Manzano., "IT Service Management Process Improvement based on ISO/IEC 15504: A systematic review," Information and Software Technology, vol. 54, no. 3, pp. 239-247, 2012. https://doi.org/10.1016/j.infsof.2011.11.002
- Kim J. H., "A Study on the Determinants of Service Quality and Customer Loyalty Concerning the Police Service," Kyonggi University, 2005.
- Han J. J., and Han K. S., "A Study on Service Quality Factors of Logistic Catalog Management System Affecting the Service Trust," Journal of Digital Convergence, vol. 11, no. 11, pp. 257-263, 2013. https://doi.org/10.14400/JDPM.2013.11.11.257
- Ahn K. H., Ha Y. W., and Park H. S., "Marketing Principle," Hakhyun Press, 2008.
- Parasuraman A., Zeithaml V. A., and Berry L. L., "A conceptual model of service quality and its implications for future research," Journal of marketing, vol. 49, no. 4, pp. 41-50, 1985. https://doi.org/10.1177/002224298504900403
- Gronroos C., "A service quality model and its marketing implications," European Journal of Marketing, vol. 18, no. 4, pp. 36-44, 1984. https://doi.org/10.1108/EUM0000000004784
- Rust R. T., and Oliver R. L., "Service quality: insights and managerial implications from the frontier," Service quality: New directions in theory and practice, pp. 1-19, 1994.
- Lee Y. J., and Ra S. N., "The Process of Changing the Service Quality Assessment System for Korean Companies," Seoul National University Press. Seoul, 2006.
- Ettinger J. W., "Consumer-perceived value: the key to a successful business strategy in the healthcare marketplace," Journal of the American Geriatrics Society, vol. 46, no. 1, pp. 111-113, 1998. https://doi.org/10.1111/j.1532-5415.1998.tb01024.x
- Lee G. C., "The effects of service quality on service value, customer satisfaction, behavior intentions in the traditional market: Focusing on the moderating effect of perceived pleasure and merchant consciousness," Doctoral dissertation, Konyang University, 2011.
- Parasuraman A., Zeithaml V. A., and Malhotra A., "ES-QUAL: A multiple-item scale for assessing electronic service quality," Journal of service research, vol. 7, no. 3, pp. 213-233, 2005. https://doi.org/10.1177/1094670504271156
- Janis Dietz, Satisfaction: a behavioral perspective on the consumer, Journal of Consumer Marketing, Emerald Group Publishing, vol. 14, pp. 408, 1997.
- Choi M. S., and Kang K. H., "The effect of customer satisfaction on firm performance in America - The moderating effect of economic condition," International Journal of Tourism and Hospitality Research, vol. 31, no. 9. pp. 71-84, 2017. https://doi.org/10.21298/IJTHR.2017.09.31.9.71
- Cha Y. G., "A study on the Factors and Model of Hotel Relationship Marketing," Journal of Industrial Economics and Business, vol. 13, no. 2, pp. 259-291, 2000.
- Kim Y. H., "Effects of Trust on Organizational Citizenships Behavior and Learning by Sharing," Korean Journal of Business Administration, vol. 38, pp. 987-1002, 2003.
- Park J. H., and Joo S. H., "Effects of Export-Supporting Service Quality on Service Value, Customer Responses in the field of Defense Industry," The Quarterly Journal of Defense Policy Studies, vol. 25, no. 1, pp. 207-252, 2009. https://doi.org/10.22883/JDPS.2009.25.1.008
- Kwon S. H., Kim T. U., and Lee Y. K., "The roles of customer's perceived value, satisfaction, trust and their relationship with loyalty in internet shopping environment," Korean Management Science Review, vol. 20, no. 1, pp. 149-163, 2003.
- Mayerl Christian, Tobias Vogel, Sebastian Abeck, "SOA-based integration of IT service management applications," in Proceedings of IEEE International Conference on Web Services (ICWS'05), pp. 785-786, 2005.
- Bartolini Claudio, Mathias Salle, David Trastour, "IT service management driven by business objectives an application to incident management," in Proceedings of the 10th IEEE/IFIP Network Operations and Management Symposium (NOMS), pp. 45-55, 2006.
- Paul P. Maglio., Savitha Srinivasan., Jeffrey Thomas Kreulen., J.C. Spohrer., "Service systems, service scientists, SSME, and innovation," Communications of the ACM, vol. 49, no. 7, pp. 81-85, 2006. https://doi.org/10.1145/1139922.1139955
- Grönroos Christian, "From scientific management to service management: a management perspective for the age of service competition," International Journal of Service Industry Management, vol. 5, no. 1, pp. 5-20, 1994. https://doi.org/10.1108/09564239410051885
- Woohan Chul, Lee Sang Chun, Cho Sung Kak, Leesang Do, and Cho Yong Gun, "An Analysis on ITSM application to improve IT Service in defense acquisition," Proceedings of Fall Conference KICS, 527-528, 2009. (In Korean)
- Woohan Chul, Kang Seok Jung, Lee Bum Koo, "An Analysis on ITSM application to improve the effectiveness of IT Service In Defense Acquisition SectorsAnalysis of Factors for Strengthening Submarine Industry Basis," Proceedings of Institute of Control, Robotics and Systems, 249-258, 2012.