Breaking the Myths of the IT Productivity Paradox

Abstract

IT is the key enabler of digital economy. Appropriate usage of IT can provide a strategic competitive advantage to a firm in a dynamic competitive environment. However, there has been a continuing debate on whether IT can actually enhance the productivity of firms. This concept is called IT productivity paradox. In this study, we analyzed the causality among appropriate indicators to demonstrate the real impact of IT on productivity. The 12,100 sample data from 2011 were used for analysis. As expected, the results indicated that mobile device usage, website adoption, e-commerce, open source, cloud computing, and green computing positively influence IT productivity. This unprecedented large-scale analysis can provide clarification regarding the ambiguous causal mechanism between IT usage and productivity.

1. Introduction

Since the emergence of computers in the 1980s, corporate investment in the information technology (IT) sector has been rapidly increasing. However, skepticism regarding IT productivity has not yet been addressed [1]. Many skeptics of IT productivity argue that there is no correlation between investment in information systems and IT productivity. Solow [2] mentioned this argument in his statement, “You can see the computer age everywhere but in the productivity statistics,” in the New York Times. Regardless of the IT productivity paradox debates, investment in IT has been increasing continuously, as shown in Fig. 1.

Fig. 1.Growth in the production of computers and related services

According to skeptics of IT productivity, due to drastic price reductions and rapid supply, IT has become common and ubiquitous. In other words, IT has become a basic commodity and is not regarded as proprietary technology that provides a company’s with a competitive advantage, rather, IT is considered as an infrastructural technology, such as electricity, railroads, and the telephone, which anyone can access [3].

In contrast, advocates of IT productivity are of the opinion that the paradox has occurred due to several misconceptions regarding IT productivity. First, they argue that there has been mismeasurement of the inputs and outputs of IT investment have been inaccurately measured. Second, they emphasize that a time lag in the return on investment (ROI) may exist. Third, the “redistribution of profits” may have been ignored. Moreover, the complementary effect of IT might have been ignored when evaluating the productivity of IT. Finally, they raised the issue that the mismanagement or improper use of IT has been neglected [4].

To resolve these problems, many studies have measured IT productivity in various ways. DeLone and McLean [5] argued that user intention and satisfaction could be appropriate dependent variables to measure IT productivity. They proposed the IS (information systems) success model which states that the net benefits from IT investment can be influenced by the information quality, system quality, and service quality of IT. Moreover, to prevent the mismeasurement of IT productivity, SERVPERF and SERVQUAL were developed [6]. SERVPERF focuses on performance-based measurements to measure IT productivity, while SERVQUAL measures IT productivity based on the perceived quality of IT service.

However, although these measurement methods are well accepted and considered to be robust, they still have several limitations. The IS success model limits the effect of IT to its use or the satisfaction obtained from its use. The model does not provide an accurate measurement of the ROI, which can show whether IT investment has actually influenced a company’s competitive advantage. Additionally, SERVPERF and SERVQUAL are limited because they rely on user perception for the measurement of IT productivity. Although SERVPERF uses performance-based measurements, IT performance is measured based on the perceptual responses of IT system (or service) users. This is because SERVPERF and SERVQUAL was established based on the IS success model. Moreover, since the IS success model only considers continuously used IT, it cannot measure the productivity of new and emerging IT systems. Hence, to measure the actual ROI of IT, this study proposes new concept of IT productivity evaluation. In this study, we measured IT productivity using two constructs, such as product/service performance and business process performance, concurrently. To enhance the generalizability and the explanatory power of the empirical statistical results, data was collected at the national level. This study aims to address the IT productivity paradox and propose new solutions for this problem, which has been unresolved for the past 30 years.

Additionally, many studies measured IT productivity by focusing on the usefulness or accessibility of IT itself [4, 7, 8, 9]. Thus, according to this type of measurement, IT systems that are more advanced and more useful have a greater positive influence on productivity. Examining the influence of IT excellence on productivity can result in an inaccurate measurement of IT ROI. Moreover, more advanced IT systems do not always induce higher firm productivity. For example, the famous technology acceptance model (TAM) was developed to investigate the reasons for the low usage rate of advanced IT systems. There have been many cases where an actual improvement in IT productivity was not achieved even with advanced IT capabilities. To resolve these issues, this study focuses on verifying the influence of IT type, rather than IT capabilities, on IT productivity. By classifying IT into six categories, computer usage, Internet usage, e-commerce (purchase or sale), possession of the website, cloud computing, and open source software, we can verify the different influences on IT productivity based on the IT type, and maximize the accuracy of research findings. Furthermore, by adopting this method, the effect of IT on a company’s performance can be examined regardless of IT performance.

 

2. Literature Review

2.1 IT productivity paradox

Solow, who won the Nobel Economics Prize in 1987, led the speculation regarding the “IT productivity paradox.” He pointed out that there was a discrepancy between IT investment and the output derived from such investment at the national level [10, 11]. In 1987, in a New York Times book review, Solow stated, “You can see the computer age everywhere but in the productivity statistics.” This triggered controversy regarding the ripple effect of IT investment [2]. Consequently, the IT productivity paradox became a major research theme. Many scholars addressed different aspects of the IT productivity paradox. Although Solow and many scholars who agreed with him did not present any substantial evidence, the overall economic indicators supported his argument. The productivity rate of the United States (U.S.) decreased by 0.7% from 1973 to 1997 even with consistent IT investment. The total factor productivity also decreased from 1.75% to 0.32% during this period [12]. This discrepancy between economic indicators and IT investment can be explained as follows.

First, of the input and output of IT investment could have been mismeasured. To analyze the influence of IT investment on a firm’s productivity accurately, the measurement of the IT investment and the firm’s productivity must be precise. However, existing measurements, such as SERVPERF and SERVQUAL, have certain limitations. Second, a time lag can exist between the time of the IT investment and the verification of its effect. When this time lag is not considered, the analyzed relationship between the IT investment and productivity becomes meaningless [13]. Third, productivity improvement may not be completely dependent on IT. Since productivity improvement is determined by various factors, such as environmental change and base material price fluctuation, a simple comparison between overall productivity of the firm and IT investment can result in making a hasty generalization [14]. Moreover, there is a claim that the IS productivity paradox occurs as a result of “applying the national economic indicators into a specific industry.” [15]. In other words, an aggregation effect can occur. For example, although a company may experience increased productivity due to IT investment, national economic indicators may still decrease because of overall low performing companies. The “redistribution of profits” could be the reason for the IS productivity paradox [12]. Furthermore, when IT is used as a supplement to an existing business process, and is not implemented as the key competitive advantage of the firm, IT can help in improving the current business process but its effect may not appear immediately in the firm’s overall productivity such as increase of sales. Hence, the complementary effect of IT has often been ignored. Finally, IT investment can be effective only when there is an appropriate change in the business process, such as business process reengineering (BPR). To absorb the potential productivity efficacy of IT, organizational process changes, such as training employees in the use of a new IT system and re-organization of the current business process to maximize the IT functionality, have to be implemented. Additionally, to mitigate the resistance to the rapid changes of implementing the new IT system into the existing fixed process, mobilizing employees by providing the motivation for these changes and creating corporate norms is necessary.

Roach [16] claimed that IT investment in the service sector does not affect the increase in output at an organization level of analysis. Berndt and Morrison [17] and Morrison [18] supported the IT productivity paradox with the finding that the productivity increase caused by IT investment is less than the cost of purchasing and maintaining the IT system. However, many studies refuted the IT productivity paradox by the results of their analysis by increasing the number of sample companies and elaborating on their measurement methodology [19, 20, 21, 22, 23, 24]. In particular, Brynjolfsson and Hitt [21] used data regarding the productivity increase over 4-7 years of IT investment to solve the time lag problem. Gilchrist et al.’s [25] study, which focused only on the manufacturing industry, shows that IT investment positively influences the increase in total factor productivity and labor productivity. Moreover, Oliner and Sichel [26] refuted the paradox with a finding that the 1.04% increased labor productivity from 1991~1995 to 1996~1999 resulted from IT investment [24]. Baily and Lawrence [27] supported that IT led to an increase in productivity by analyzing the growth rate of industrial productivity. They show that the productivity of the entire U.S. economy increased since 1995, and that the IT industry had the greatest productivity increase, and the productivity increase was accelerated in the service industry such as wholesale and retail trade as well as finance industry. They emphasized that the differences in the productivity increase pattern among industries was due to different IT investments adopted in their business processes. Additionally, McKinsey Global Institute (MGI) stated in their “2001 U.S. Productivity Growth” report that a “new economy” in the late 1990s formed only in six industries and that IT was the one of major factors of productivity improvement in those industries [28]. Similarly, in a 2002 report, McKinsey claimed that, although the effect of IT investment on productivity differs by industry type, IT has a certain positive impact on productivity. MGI proposed that IT investment should be accompanied by business process innovation and IT capabilities to experience productivity improvement. They mentioned that there is no “silver bullet” [29], which indicates that IT is not magical tool but can become a magical tool with appropriate usage.

However, once again, disputes over the IT productivity paradox is were sparked by. Carr’s [3] article “IT Doesn’t Matter” was published in Harvard Business Review. Carr argued that IT no longer provides a competitive advantage due to the commoditization of IT [3]. He stated that IT has become infrastructural technology, like railroads and electricity. This argument was immediately refuted by Intel’s CEO, Craig Barrett, and the president of Deloitte Touche, John Hagel. Hagel refuted Carr’s argument [30] by stating that the influence of IT on performance improvement was unlike that of technologies such as railroads and electricity. He mentioned, as an example, mainframes whose client-server architecture was changed for better flexibility and performance. In other words, he argued that IT can still be a major aspect of a firm’s comparative advantage since IT is consistently improving. Over the past 30 years, disputes regarding IT productivity or the strategic usefulness of IT have continued. This is because previous research studies limited the unit of analysis at an organizational or industrial level. Therefore, reliable and valid empirical analysis using nationwide data is more necessary than ever.

2.2 IS success model, SERVPERF, and SERVQUAL

To measure influence of IT investment on productivity, management information systems (MIS) researchers have developed and introduced various measurement tools. Representative performance measurement models are the IS success model by DeLone and McLean [8], SERVQUAL [18], and SERVPERF [9]. Based on their own previous research about information system quality, DeLone and McLean’s proposed model focuses on its outcomes in the scope of an organizational unit. In this model, individual perception toward information and system quality influences system “use” and “user satisfaction” from the system, which influences the individual impact of IT systems. Finally, the organizational impact is influenced by such individual impact. Ten years after their initial research, in 2003, DeLone and McLean [8] added “service quality” and “intention to use” to improve the strength of their model. They argued that service quality has a positive influence on “intention to use” and “user satisfaction.” Through the “intention to use” and “user satisfaction,” the net benefit of IT systems was determined. However, it is problematic to view user satisfaction and intention as being related to the overall net benefit of an organization, since “intention to use” and “user satisfaction” are merely perceptions about the IT system in an organization. Hence, “intention to use” and “user satisfaction” may only influence the effectiveness or outcome of the IT system itself and not the overall net benefit of an organization. The effectiveness or outcome of an IT system cannot be equated with the overall net benefit of an organization. In other words, even with an immense effectiveness or outcome, an IT system may not always induce an improvement in the net benefit of an organization. Thus, the IS success model may have proved a positive correlation between investment in IT systems and the effectiveness or outcome of IT systems but failed to prove the actual causal relationship between IT investment and net benefit of the organization. Therefore, this study aims to test the actual causal relationship between the IT investment and productivity improvement at the organizational level.

After the IS success model was introduced, SERVQUAL [7] was developed to measure IT productivity by measuring the consistency between users’ expectation on service quality and their perception after their service (IT system) use. SERVQUAL measures service quality in five dimensions: reliability, assurance, tangibles, empathy, and responsiveness. However, Cronin and Taylor [9] claimed that the key concept of SERVQUAL, which is a disparity between perception and expectation, is problematic. They emphasized that service quality and service satisfaction are not mutually exclusive. Moreover, they argued that SERVPERF, which is the perception of service (IT system) performance, is a more valid measure of IT productivity. SERVPERF measures the outcome of service but only measured user perception after system use. As SERVQUAL emphasized, individuals’ expectations regarding IT systems may differ. When using SERVPERF, there is a problem caused by the absence of natural origin, that is, since individuals’ origin differ from each other, their perception cannot be compared. Even SERVQUAL does not have the natural origin problem since it considers users’ expectations as the origin for each user and measures its gap from the perception of use (to make the SERVQUAL measure an interval scale for comparison); they have this limitation in common with the IS success model. Since all measures of SERVQUAL are perceptual measurements of users, SERVQUAL could not analyze the actual causal relationship between the investment on IT systems and improvement in the organization’s productivity. Furthermore, both SERVQUAL and SERVPERF can only analyze the influence of the existing IT system, and not newly introduced IT systems.

To resolve such problems, this study aims to measure the ROI of IT and verify its impact properly by using national data rather than merely limiting the study to certain systems or a particular industry.

 

3. Research Model

Although the IT productivity paradox has not yet been resolved, Solow’s [2] argument has been refuted by many studies [4, 7, 9]. Such refutations may not be needed since we live in an IT centric world. The emergence of the computer altered the basis of industry similar to, or more than, the emergence of the steam engine. However, this may not be evident in the “productivity statistics” mentioned by Solow [2], since they had productivity mismeasurement problems. In other words, Solow’s productivity statistics, which was developed prior to emergence of the computer, did not consider the characteristics of IT [4]. As mentioned earlier, many previous studies measured only the impact of certain IT devices or systems to measure the effect of IT or IS [4, 7, 8, 9]. These studies were limited to measuring the impact of the excellence of a specific IT device or system. Accordingly, by generalizing IT rather than limiting it to a specific system and by dividing it into general IT types, such as computers, Internet, and mobile devices, this study examines the impact of the use of each IT type on IT productivity. Fuhrer et al. [31] argued that outcomes can change depending on the IT device, which supports the following hypotheses of this study.

Regarding the measurement of outcomes, traditional measurements, such as product/service performance, are problematic since such measurements only reflect the IT system’s influence on product or service improvement. IT systems influence the overall business process from enhancing a simple document to supporting an organizational strategic decision. Accordingly, this study classified outcomes into two separate constructs: traditional product/service performance and business process performance. Based on this, the following hypotheses were established.

Like measuring productivity simply based on product/service performance, measuring the various IT device types according to IT usage has similar problems. A single IT device may have various functionalities. For example, a computer can be used for calculation and for an Internet search. Therefore, considering how an IT system is used is necessary. To address this problem, this study divides IT functionality into website adoption, e-commerce (purchasing), e-commerce (sales), open source software usage, and cloud computing usage. E-commerce, in particular, is classified into purchasing and sales separately since there are cases when IT is mainly used for only one area, purchasing or sales, according to the company’s characteristics. For example, regarding traditional supply chains, companies that produce raw materials focus their IT power mainly on purchasing. Conversely, for a retailer, IT power is mainly focused on sales. Of course, retailers and all other companies in the supply chain, such as manufacturers, make purchases as well as sales in their business activity. However, they focus their resources in line with their key strategic points; through this, the ratio of the impact on productivity is changed [32]. Accordingly, the following hypotheses were established.

Moreover, it is obvious that the enhancement of a business process leads to improvements in the product/service that is produced by a company. The positive correlation between business processes and products/services performance has been verified by many studies. Accordingly, the following hypothesis was established.

Based on the above hypotheses, the research model was established as shown in Fig. 2.

Fig. 2.Research model

 

4. Research Methods

4.1 Measurements

The Korean government has been continually monitoring the IT usage of companies and its influence, presented in the annual Yearbook of Information Society Statistics since 1999. The Yearbook, which is published by the Ministry of Science, is an official national statistics report that includes various information and communications technology (ICT) indicators. The National Information Society Agency (NIA) is responsible for the collection and analysis of corresponding data. Additionally, this official government statistics report, Information Society Statistics, is used as the base data for various reports in various international organizations, such as the Organization for Economic Cooperation and Development (OECD), the United Nations Conference on Trade and Development (UNCTAD), and the International Telecommunication Union (ITU). To gain the public’s confidence and achieve generalizability of this study, we used the same data that was used in the 2011 Yearbook of Information Society Statistics.

The population of this data comprises every establishment in South Korea, which includes 3,335,470 establishments. An establishment refers to a single economic unit under single ownership and/or single regulation but which differs from a company, which refers to a legal unit for producing goods and services. Sampling was conducted using stratified multi-stage sampling. For the stratification index, industry type and size (number of workers) was used. For industry type, the International Standard Industrial Classification of All Economic Activities (ISIC) Rev.4 was used, and size was classified according to the OECD’s standard. During the stratified multi-stage sampling, 14,200 samples were selected for the survey. After eliminating neglectful and invalid data, 12,100 data samples were used for further analysis and the effective response rate was found to be 85.3%. The characteristics of sample group are presented in Table 1 and Table 2.

Table 1.Respondent demographics by industry

Table 2.Respondent demographics by establishment size

To enhance survey quality, a face-to-face interview survey was conducted to minimize non-sampling error. The interview survey was conducted with IT managers and managers of general affairs. For establishments that could not participate in a face-to-face interview survey, telephone and e-mail surveys were conducted. In particular, for the annual regular survey target (every establishment with over 1,000 workers and some establishments with over 250 workers), an online survey was conducted to increase the accessibility and effectiveness of the survey. Prior to the survey, the purpose and objective of the survey, the survey method, and other required knowledge was provided to the interviewers. Moreover, the interviewers took a trial survey themselves to reduce possible errors or problems. Since this survey was an obligation for every target establishment, non-response was not allowed and hot-deck Imputation, which is the UNCTAD recommended method for handling missing data, was used for non-response despite multiple attempts.

The collected surveys were assessed for appropriateness through three stages of abnormality tests: review by the surveyor at the site, error checking and revising by a regional office, and quality verification by the head office of the research company. Additionally, a research management agency, NIA, re-verified the results obtained from the previous stages.

The overall survey procedure is described in Fig. 3. The definitions of some concepts and detailed survey items are presented in Table 3.

Fig. 3.Survey verification procedure

Table 3.Conceptual definitions

Since nearly all measurements (except independent variables) are objective measurements, such as computer possession status, a conceptual definition may not be needed but conceptual definitions are still provided (Table 3) to clarify any ambiguity regarding technical terms. The measurement items are presented in Table 4.

Table 4.Measurement items

4.2 Analysis and results

To secure validity of the measurement tool, the OECD Business ICT Usage Index by the Economic Analysis and Statistics Division (EASD) was used to measure each indicator. These measurements are listed in the EASD, the OECD Fact book: Economic, Environmental and Social Statistics, and the OECD Telecommunications and Internet Statistics. Moreover, these measurements are used by the UN-affiliated International Telecom Union, Department of Economic and Social Affairs, and by UNCTAD.

In terms of statistical measurement reliability, sufficient reliability was achieved as the Cronbach’s alpha values of the measurement tools were all 0.7 or above.

Statistical analyses were performed using the statistical software IBM Statistical Package for the Social Sciences (SPSS), version 19.00. SPSS was chosen because it is the most suitable and commonly used software for regression analysis.

The analyses were evaluated by examining the explained variances (R2) for the research model and the path coefficients of the independent variables. The results of the entire research model are presented in Fig. 4. Our research model accounts for 17.2% of the variances in product/service performance and 12.2% of the variances in business process performance.

Fig. 4.Research model results

The results of analyses (Table. 5) show that all the hypotheses are supported, except for H1a, H1b, H2a, H2b, and H4c. As expected, mobile device usage (H1c, t = 4.88, p < 0.001), website adoption (H3a, t = 10.07, p < 0.001), e-commerce (sales) (H3b, t = 12.66, p < 0.001), e-commerce (purchases) (H3c, t = 3.18, p < 0.01), open source software usage (H3d, t = 9.52, p < 0.001), cloud computing usage (H3e, t = 3.39, p < 0.001), and business process performance (H5, t = 30.41, p < 0.001) showed significant positive relationships with product/service performance. Additionally, mobile device usage (H2c, t = 10.02, p < 0.001), website adoption (H4a, t = 6.97, p < 0.001), e-commerce (sales) (H4b, t = 9.95, p < 0.001), open source software usage (H4d, t = 18.08, p < 0.001), and cloud computing usage (H4e, t = 5.59, p < 0.001) showed significant positive relationships with business process performance.

Table. 5.Summary of results

 

5. Discussion and Conclusions

5.1 Implications

Based on the government’s statistical data, which has the public’s confidence, this research empirically tests the influence of IT investment on organizational productivity, such as product/service performance and business process performance. Furthermore, this study used 12,100 nationwide organizational level samples, which are not normally seen in regular research, to improve the generalizability of the results. Moreover, this study endeavors to make several important contributions to the existing IT productivity paradox literature, using nationwide organizational level samples. Although many studies tried to resolve the IT productivity paradox debate, they had the limitation of measuring the productivity of certain IT systems only. This may be the major reason that the IT productivity paradox is unresolved. However, this study analyzes nationwide organization level data to solve this problem. Classifying IT by device type and functionality helps in avoiding bias due to data collection from certain systems and enhancing the overall productivity by general IT investment.

The results of this study imply that computer and Internet usage are not related to enhancing a firm’s competitive advantage. According to arguments by Solow [2] and Carr [3], contrary to our expectations, computer and Internet usage is not influenced by either product/service performance or business process performance. On the other hand, mobile device usage, website adoption, and many other IT adoptions have a positive influence on organizational productivity. This result indicates that computers and the Internet do not influence organizational productivity since they became commoditized. However, many other ITs are still considered as technical progress, offering a comparative advantage to an organization. Based on these results, this study proposed a new solution for resolving the IT productivity paradox debate, which has been unresolved for over 30 years.

Interestingly, there is no significant relationship between e-commerce usage in purchases and business process performance. This indicates that e-commerce usage for purchases does not improve the current business process performance. These results can be attributed to the current business situation. Since many current businesses, other than small retail businesses, do not conduct online company-to-company purchases, the relationship between e-commerce usage in purchases and business process performance may appear insignificant.

Additionally, the following contributions were expected, since this study aims to resolve the limitations of SERVPERF and SERVQUAL. First, SERVPERF and SERVQUAL are limited because they rely on user perception for measurement. However, this study measures the actual objective usage of IT and its influence to clarify the ambiguous relationships between the usage of various ITs and organizational productivity. Hence, regardless of IT excellence, the actual influence of IT investment could be assessed. Moreover, many previous studies measured IT productivity by focusing on the usefulness or availability of ITs [4, 7, 8, 9]. Analyzing IT ROI according to the IT excellence is problematic. To resolve these issues, this study proposed new concepts for IT productivity evaluation: product/service performance and business process performance. With a more objective evaluation for IT productivity, future studies may be able to resolve the conflicting debate regarding the IT productivity paradox and overcome the limitations of SERVPERF and SERVQUAL.

This study offers some implications for chief information officers (CIOs) and related practitioners. First, since each IT type has a different influence on productivity, this study can act as guide for CIOs or IT managers to choose various strategic IT investments. Further, while IT functionalities are more important for enhancing overall organizational productivity, CIO or IT managers have to choose appropriate IT functionalities for their organization regardless of the IT device.

5.2 Limitations and future research

Despite the many achievements of this research, there are a few limitations of this study. First, most of our research data scales are binary. This can limit the accurate measurement of the research variables. Although the measurement items used in this study fit the standards of the OECD and are widely used for various public reports, future studies should develop multiple measurement items and diversify the measurement scales through a Likert scale to ensure greater validity and reliability. This study is a cross-sectional study. To address the time lag issues, a longitudinal study is needed. Through a longitudinal study, as time passes by, the altered influence on IT productivity can be assessed. To conduct a longitudinal study, future researchers should avoid collecting data from firms that undergo radical changes such as M&A.