1. Introduction
Over the past decade, the research community has made major inroads in building common human computer interaction (HCI) design patterns, resulting in a lingua franca for user interface (UI) design. These patterns are increasingly important for designers as a vehicle for mediating between HCI and other software engineering practices, but there is still some room for improving them to maximize their utility to designers. Today’s software applications introduce challenges that also call for HCI patterns capable of organizing the interaction on modern interfaces for example, mobile platforms.
Beyond UI, HCI practitioners advocate a user-centered design (Norman, 1986) approach which includes a set of activities for building interactive systems with user involvement in all the development stages. According to this model, if a UCD process is identified as a being necessary, developers must determine who is to use the product and for what purpose, in addition to what other requirements a successful product must fulfil. Developers also need to evaluate design alternatives, create design solutions, and evaluate their usability with real users. The notion of design as it is defined in user-centered approaches refers to UI design or interaction design, not to software design as conceived from a software architecture perspective. Therefore, developers must integrate this UCD process into a particular software engineering process to build systems with the required quality attributes, including usability.
What extent has it been adopted after several years of UCD teaching and dissemination? Within the system development, process is it integrated? The main issues are social and organizational not the deficit of human-computer interaction (HCI) methods or expertise, as Donald Norman pointed out at a Bay-SIGCHI meeting in 1993. On our research agenda, organizational issues are high, relating to process development as John (2004) acknowledged, more than 10 years later.
Although the integration of UCD process into software engineering process is required, a few researches pay their attention to this integration (Vredenburg et al., 2002; Mao et al., 2005; Gulliksen et al., 2004; Venturi et al., 2006). However, these studies prioritized the organizational factors for UCD success and especially, they suggest no theoretical framework to examine the organizational factors for the successful adoption of UCD.
Given this situation in the research gap, the purpose of this study is to integrate organizational factors into UCD process. For this research purpose, this study will investigate the organizational factors which influence people behaviours in the context of UCD practice. This study presents organizational culture, organizational learning and change management as these organizational factors. Especially, this study is to investigate how change management influences the relationship between the organizational culture/learning and UCD performance.
2. Literature Review
The organizational impact of UCD success measures and UCD practice as perceived by practitioners were investigated by Vredenburg et al. (2002) and Mao et al. (2005). The practitioners were asked on a 7-point Likert scale issues to assess the organizational impact in their questionnaire. And, the development process type in the organization, the methods and techniques, the degree of user involvement and organizational factors were investigated by Gulliksen et al. (2004). In most software development phases, the level of user involvement is constant; practitioners focus on design activities, management support and those methods relatively informal were rated as effective and early involvement of UCD rated important as organizational factors. To learn the kind of must be tackled organizational issues, the state of UCD adoption was investigate by Venturi et al. (2006). Targeted at UCD practitioners, Web survey was designed and carried out by them. Several organizational factors of their management and UCD practitioners should be considered based on the survey. UCD should be supported by higher management and part of the business strategy. They must set usability goals through competitive analysis. If the goals are reached or exceeded, the practitioners should be rewarded.
However, although these studies suggested that UCD should be supported by higher management and part of the business strategy and if goals are reached or exceeded the practitioners should be rewarded, their arguments were just derived from the result of simple survey targeted at UCD practitioners. They prioritized the organizational factors for UCD success and especially, they suggest no theoretical framework to examine the organizational factors for the successful adoption of UCD.
Organizational culture is generally understood as the social glue that holds organizational members together and expresses the values, social ideals, and beliefs that members share (Schein, 1985). Therefore, a firm’s commanding culture influences on how its employees perceive events through its values and operating beliefs(O’Reilly & Chatman, 1996; Kotter & Heskett, 1992; Wilson 2001; Denison & Mishra, 1995) and their behavior(Rosen, 1995; Schein, 1985; Barney, 1986).
3. Methods and Hypothesis
3.1. Research Model
The research model of this study is shown as below [Figure 1].
[Figure 1] Research Model
3.2. Hypotheses
Organizational culture refers to the working environment in an SW development organizations (Netta, 2010). UCD practices such as user observation, focus groups, or even social networks can help elicit proper user needs, paper prototyping or storyboarding can drive development and heuristic evaluations, and usability testing can help develop software applications that are more focused on real users. A lot of innovation, team effort and group discussion are required for all of these practices. Hofstede et al. (1993) highlighted two types of cultures: closed and open. The decisions to the lower levels are dictated directly without considering the observations and the views of most employees and made at the higher levels in closed organizational cultures. On the basis of employee involvement and discussions, open organizational cultures make decisions in contrast. UCD practices require the employees have the chance participating in discussions under the culture of openness, have the power to express their views. For example, one of the critical process elements is variability management which require business unit and the development unit for active involvement in various parts of the organization, to specify areas for expansion in the UCD. An organizational culture supporting innovation, sharing of experiences, teamwork and learning has relatively greater potential institutionalizing UCD practices.
[Hypothesis 1] Open organizational culture will be positively associated with the performance of UCD.
Organizational learning is the process which members respond to the changes by detecting errors in the internal & external environments of the organization (Argyris, 1977), which they correct then to maintain the central features of the organization. Learning is defined as insights to change their behaviors and actions and a process that individuals gain new knowledge (Marquardt & Reynolds, 1994). Learning is defined as practice and encompassing acquisition of new skills, methodologies, values and attitudes which they are constantly changing and necessary to live in a world (Hames, 1994). Organizations can remember incidents from the past that may influence future actions and do learn from the experiences (Lyles, 1994).
In the UCD practices, we can classify the organizational learning into two domains (Argyris, 1977): external and internal. External learning involves necessary knowledge about users, competitors, external environments and market, which can help us make competitive product concept by exploiting the product characteristics. This learning helps capturing major market share of competitive products by organization. On the other hand, Internal learning requires acquiring ideas for process improvement which is transferring and sharing a UCD methodology and a comprehending the cross functional requirements of individuals, groups and the organization. Learning, especially for attempting to institutionalize UCD practices by organizations, is a continuous process. Learning from mistakes and experience further facilitates improvement in particular in the UCD practices.
[Hypothesis 2] The performance of UCD will be positively associated with external and internal organizational learning.
Organizational change considered as movement to some future or target state from the present state of the organization (Beckhard & Harris, 1987). Change management is defined as a systematic and structured, providing a conceptual framework encompasses people, politics, process and strategy (Todd, 1999). Organizational change shows a diversity of the organization in its environment and the interaction of the human and technical activities in the organization (Cao et al., 2000). In the context of UCD, organizational change is necessary to align the software process with the design loops needed to properly address the user experience (Tom, 2001). Successful implementation of UCD methodology depends on how people perceive the change ultimately. When UCD methodology is introduced to an organization, a certain degree of resistance is quite normal. But, if people understand the change is positive and their best interest as well as the organization, this resistance will disappear. Therefore, effective change management partly depends on how the strategy is communicated with the people who have responsibility for the implementation.
In the above, this study suggested that organizational culture and organizational learning influence the performance of UCD positively. Organizational culture is defined as a thinking system (O’Reilly & Chatman, 1996; Kotter & Heskett, 1992; Wilson, 2001). Organizational learning is defined as a learning system (Argyris, 1977; Marquardt & Reynolds, 1994; Hames, 1994). They have a common thing that they are unique capabilities which are formulated for a long time and very hard to manage directly. However, change management is defined as a management system (Todd, 1999), which influences how people in organization feel UCD practices. Although some organizations have low capabilities, they can improve their own capabilities for the performance by making them understand that is a positive change and is related with their best interest, as well as the organization, appreciation and reward system, promotion strategy, job security and competitive compensation through well-defined job placement. Therefore, when the organizations introduce a new methodology such as UCD, they cannot change organizational culture and organizational learning in essence, but they can improve them for the performance of UCD by proper and effective change management.
[Hypothesis 3] Change management will strengthen the positive relationship between open organizational culture and the performance of UCD.
[Hypothesis 4] Change management will strengthen the positive relationship between external and internal organizational learning and the performance of UCD.
3.3. Data Collection
This study conducted a surveys of 112 UCD practitioners in 52 companies that attended in HCI Korea 2014, which was held in High1 resort in Korea from 12 to 14, Feb, 2014. The organizations differed in range and size from medium to small-scale, which, in Korea, is generally supposed that the number of employees in small scale organization is under 100 and that in a medium scale organization is from 100 to 500. The sample includes 19 small-sized companies (36%) and 33 medium-sized companies (64%). In terms of employees number, the organization size is important which is based on total employees working in various departments of the organization. This study requested the companies under study to distribute the questionnaire within R&D departments. The target respondents are described as follows.
3.4. Measurement
This study used four separate items to measure the each variable of organizational culture, organizational learning, change management and UCD performance. This study changed the questionnaire of organizational culture, organizational learning, and change management that were used by the study of Ahmed et al. (2009) which investigated the organizational factors for the successful adoption of software product line (SPL). The reason that this study changed and used the questionnaire of Ahmed et al. (2009) is that because both SPL and UCD are new methodologies in the software engineering, the questionnaire of the organizational factors for SPL was customized for the context of the adoption of new methodology in the software engineering, so they are suiting for those for the adoption of UCD. And, UCD performance means how much UCD is used and is measured by the questionnaire that Vredenburg et al. (2002) used.
This study used five-point scale of the study of Ahmed et al. (2009) but changed five-point scale into seven-point scale. It is because the questionnaire that Vredenburg et al. (2002) was measured in seven-point scale. For collecting measures for these variables, the items were designed specifically and they are labeled from 1 through 20 sequentially, as in Appendix-I.
3.5. Reliability and validity analysis for the measuring instrument
In empirical studies, reliability and validity analysis are integral features. Reliability means the measurement reproducibility and validity means agreement between the measured value and its true value. It is designed specifically for this empirical investigation to conduct reliability and validity analysis by using the most common approaches in empirical studies. The multiple-item measurement scales’ reliability of the organizational culture, organizational learning, change management and UCD performance was evaluated by using internal consistency analysis. Using coefficient alpha of Cronbach (1951), we performed internal-consistency analysis.
[Table 2] reports the coefficient of alpha ranges from 0.77 to 0.89 in the reliability analysis. The coefficient of reliability 0.70 or higher for a measuring instrument is satisfactory (Nunnally & Bernste, 1994). Van and Ferry (1980) suggest that 0.55 or higher reliability coefficient and Osterhof (2001) suggests 0.60 or higher is satisfactory. Therefore, we can determine all variable items were reliable, in this empirical investigation.
[Table 2] > Coefficient Alpha & Principal Component Analysis of Variables
The content validity of the items observed in this study included in each organizational culture, organizational learning and change management, following the recommendations of Cronbach (1971) and Straub (1989), included possible items in the variable scales by a comprehensive literature survey. The proposed scales were reviewed by psychological and statistical experts. After conducting pilot tests, based on the suggestions of respondents led to modifications in the variable items, it improved the validity of the content.
Convergent validity occurs highly correlate when the scale items in a given construct move in the same directions (Campbell & Fisk, 1959). The principal component analysis (Comrey & Lee, 1992) performed and reported for organizational culture, organizational learning, change management and UCD performance in [Table 1] provide a measure of convergent validity. Using principal component analysis, Eigen value (Kaiser, 1970) and scree plot (Cattel, 1966) are used as reference points observing the construct validity. This study used Eigen value, known as Kaiser Criterion (Stevens, 1986), which means any component having an Eigen value greater than one was retained. It is revealed that the variables are completely formed a single factor in Eigen value analysis. There is shown clearly that the cut-off at the first component in scree plots. Therefore, it is sufficient that the convergent validity. The reliability and validity analysis measurements used in this study showed that the measurement procedures required level of psychometric properties.
[Table 1] Target Respondents Description
3.6. Data analysis techniques
This study used various statistical analysis techniques to analyze and check the significance of the [Hypothesis 1] - [Hypothesis 4]. We divided in three phases of the data analysis activity, initially. Parametric statistics and normal distribution tests and are conducted in Phase-I. Non-parametric statistics conducted in Phase-II. We used non-parametric and parametric statistical methods to reduce the threats of external validity due to small sample size. The factors using standard deviation, mean, kurtosis and skewness techniques of the normal distribution of all the organizational were tested and found, with some exceptions for the normal distribution, all these tests range were acceptable. Before performing statistical analysis, there is some modification of the data from respondents. Because the three independent variables and the dependent variable’s measuring instrument all had multiple items. To obtain a composite score for the measure, we added their ratings before statistical analysis. Based on the all the respondents data, we did the statistical analysis. This study conducted tests for hypotheses [Hypothesis 1]-[Hypothesis 4] using multiple regression analysis.
4. Results
[Table 3] displays the results of the regression analysis. First, this study insert control variables in the Model 1. The results show when UCD is introduced doesn’t influence UCD performance. But the smaller organizations, the better UCD performances. In the model 2, the results show that both of organizational culture and organizational learning have positive effects on UCD performance as expected. In this empirical investigation, this study found a positive association between organizational culture and the performance of UCD. In open environment, employees can express their suggestions and opinions, if an organization encourages new thinking and innovation. Without much resistance, new idea is assimilated into the current process readily, which is expected positive impact to the organizational goals. This results are from overall conducive working culture. To participate in discussions the employees having the chance and having the power expressing their views, the UCD requires openness culture.
[Table 3] Regression results
p < .05 *, p < .01 **
Another key to success is organizational learning. Organizational learning have a positive impact of on the overall performance of UCD according to this empirical investigation. Both employees and organization itself are included in organizational learning. This study can classify organizational learning in case of UCD into two domains: internal and external. Therefore, learning is a continuous process attempt to institutionalize UCD, especially for organizations. Therefore, [Hypothesis 1] and [Hypothesis 2] are supported.
And the results of model 3 show that change management strengthens the positive relationship between organizational learning and UCD performance but has no effect on the relationship between organizational culture and UCD performance. Change management is a kind of organizational management system which is manageable unlike organizational culture and organizational learning. Therefore, this study suggested that some organizations have low capabilities, they can improve them for the performance of UCD by making proper and effective change management. This empirical investigation finds change management strengthens the positive relationship between organizational learning and the performance of UCD, while has no modulating effect on the relationship of organizational culture-UCD performance. The results show that organizational learning for UCD can be improved by change management, while organizational culture for UCD cannot. Because organizational culture has been associated with the unique quality, the people and style of the organization largely, it may not be managed by management system such as promotion strategy, appreciation and reward system, well-defined job placement, competitive compensation and job security. However, organizational learning is a kind of practice of new methodologies, skills, attitudes, and values necessary to live in a world that is constantly changing and a kind of process to learn from their experiences and can remember incidents from the past that may influence future actions (Hames, 1994; Lyles, 1994). Although it is a unique capability which is formulated for a long time, it may be changeable for the strategy by some human resource management or process management for change. Therefore, is supported, while [Hypothesis 3] is not supported.
5. Discussions and Conclusions
We provide an opportunity to investigate the association between the organizational factors and UCD performance empirically. We provide the first empirical support and evidence for the theoretical foundations that, in the institutionalization of UCD line, the organizational factors play a critical role within an organization. In addition to their efforts in software engineering, the organization has to deal with multiple organizational factors in the adoption of UCD. In institutionalizing UCD in an organization, this results have the potential out of this approach to achieve maximum benefits.
It is the first kind of study to conducted and reported in the area of UCD. This research will enable organizations to the better understanding of the effectiveness for the relationships of organizational factors and UCD. For the significant impact of organizational factors, this research also reinforces current perceptions on successful institutionalization of UCD. In order to institutionalize this concept the organizations in the business of UCD need to consider their efforts and multiple key organizational factors to develop software. For the process assessment by developing Process Maturity Model of UCD is necessary in the future. In evaluating the organizational dimension of UCD process maturity, this work will provide the empirical justification to include these organizational factors.
The first observable limitation of this study is small sample size. In the software industry, not a lot of the organizations have launched and institutionalized this concept of UCD and a relatively young concept in software development. So there was a limitation collecting data from the software industry, which leads to small sample size. In terms of number of organizations and respondents for this small sample size, there is a potential threat, in this study, as the external validity.
The second is bias in decision-making. The bias still is a core issue, although using multiple respondents within the same organization for this matter. And in order to reduce the overestimate or underestimate filling in questionnaires by the human tendency, we asked the respondents for their organization to consult major sources of data, like models, plans, documents and actors, before responding to a particular item. The measurement is based on the subjective assessment of an individual largely, still the items were designed using accepted psychometric principles. This study helps to understand the organizational dimension of UCD and contributes significantly in the area of UCD, besides its general and specific limitations.