Cultural Issues in Offshore Teams: A Categorization based on Existing Studies

Abstract

Cultural and personal issues resulting from dispersed teams are considered to be serious barriers to form trust and organize effective agile teams. However, apart from separate, reported evidence of such issues from work experience, there has been no theoretical classification in literature. This paper provides a list and analysis of common challenges mainly resulting from cultural differences and barriers in Agile Software Development (ASD) offshore teams. The data source comprise articles published in IEEE, mostly of conferences related to ASD. Among the articles, papers with concrete evidence of Agile Methods (AM) implementation were selected. The results show that despite the relative significance of such issues, ASD adopters typically still rely on their own experience, and creativity rather than using well-defined methods. Moreover, this study reveals that the notion of trust, as discussed in the literature, mainly refers to maintaining the pace of communication, which is the focal point in ASD.

1. Introduction

 Agile Software Development (ASD) has been attempted in various areas from initially small collocated teams to large distributed ones. Agile Methods (AM) are inherently based on face-to-face relationships and requires high-level mutual understanding and trust. As a result, they (i.e. AM) definitely have their own challenges in offshore settings in that distance is a factor. This study particularly intended to investigate those cultural and personal issues that usually result from social culture and cultural differences in geographically distributed environments.

 The first study objective is to provide a categorization of these issues in terms of problem-solution pairs (i.e. a categorized list containing problems along with their corresponding solution(s) - if any) as the problems are faced and, the solutions are applied in real cases. These issues have been classified according to their similarities and differences. Second, along with these pairs of problems and solutions, their situational and environmental conditions are also systematically collected and preserved in the presentation. Thus, practitioners will be able to relate to the information provided for their own purposes. The results are presented in an innovative tabular form so the various data dimensions are preserved and presented concise as much as possible. Sections 2, 3 and 4 explain the data gathering and codifying techniques. Sections 5 and 6 present the results and further analysis.

2. Motivation and Research Questions

2.1 ASD in Distributed Settings

 The main incentives for off-shore development environments are access to a knowledgeable and experienced workforce and reductions in development costs. However, there are also challenges. Historically speaking, ASD emphasizes face-to-face relationships and close, continuous follow-up meetings in order to progress and control the pace of work. All these requisites are challenged in/by geographically distributed settings. More specifically, cultural issues are of interest here. These issues are often communicated in umbrella terms, e.g. cultural barriers/challenges. As a general piece of knowledge, there is always a chance to encounter concrete evidence of such kind of issues in work experience reports and case studies. Consequently, the main motivation here is to use such resources to study these issues.

 In actuality, solutions for such cultural issues have emerged for instance using communication tools to conduct meetings or scheduling work shifts according to (differences in) time zones and so on. These solutions inevitably require specific work practices and culture. The understanding based on a previous study [31] implies there are some major challenges with even using such replacement practices, among which and considering the current topic, the most frequent one is trust. Hence, this study is also intended to help understand how these cultural issues may interrupt the normal pace of work in offshore environments. Additionally, it is aimed to determine how such new emergent solutions may coexist, resolve or be disturbed by cultural issues. In the next subsections, these intentions are translated into separate, clear research questions.

 Since this study is exploratory in nature, it does not begin with hypotheses. Rather, the initial idea for this study is that cultural differences and barriers have specific negative effects on the AM implementation process anyhow.

2.2. Inception of Research

 To identify cultural issues and corresponding, adopted solutions that result from/overcome geographical distribution in ASD, a literature search (based on guidelines from [28]) was conducted mostly on work experience reports and case studies of ASD implementation.

2.2.1 The study is aimed to answer the following questions:

 RQ1: What are the challenges reported in ASD offshore teams that are cultural in nature or can be considered to be caused by cultural issues?

 RQ2: Which of these challenges are attributable to the limitations of ASD (as initially proposed for collocated teams) and which ones are attributable to distant workplaces in general?

 RQ3: What are the emergent solutions that have been used to overcome the difficulties (i.e. resulting from RQ2)?

 RQ4: How is applying such solutions (obtained via RQ3) effective and/or may have side-effects?

2.2.2. Search Clauses

 The search process was designed in two stages. First, the following search string was suggested to select ASD/AM-related papers.

 Search String 1: (Agile AND (Software OR Method OR Development)) in [Abstract].

 The results of the first search were treated separately to exclude unsuitable items, as described in the next section. The results were maintained as a general repository of papers for a set of relevant, past and future, studies. The second stage, which was solely performed for the purpose of the current study, entailed the following string to search among the papers obtained in the first stage.

 Search String 2: (Distributed OR Global OR Offshore OR Dispersed Teams OR Remote Teams) in [Whole Text].

 It is noted that, the second search was done manually on existing papers. Then, as a third stage, namely data extraction, all the papers resulting from the second stage were read thoroughly to extract cultural issues, whether of individual, team or organizational origin.

3. Data Collection

3.1 Search Sources and Criteria of Search

 This paper mostly relies on work experience reports to show how cultural issues have been surfacing and, what challenges practitioners face. Thus, IEEE publishing is considered suitable for this purpose because it contains the most extensive amount of work experience reports (identified by the first author through a past study [31]), mostly in the form of conference papers.

 As further justification for selecting IEEE, a pilot search of five other databases performed from February 2003 to March 2014 was performed. This pilot search was initially conducted for any items relevant to ASD based on title and abstract searches (see the first search string in the previous section). In IEEE alone, the number of papers retrieved papers was more than from the four other databases altogether (ACM, Science Direct, Emerald and Taylor): 2856 compared to 1273 (550, 312, 181 and 230 respectively). In the second phase, the items were filtered out by reviewing their title, abstract and occasionally the whole text. It was first assured that that the items were in the form of scholarly articles and technical reports rather than workshops, posters, manuals, books, etc. Second, the items were in English. Third, they had to be clearly related to ASD rather than topics like agile manufacturing and management. Consequently, the result set from IEEE was reduced to 660. Springer articles were also searched. Though, due to the inability to search for abstracts in the search engine, i.e. the Springer link, a second manual search was performed on entire texts. The final result set was 228. Considering the fact that a majority of IEEE papers are work experience reports or case studies, for the purpose of this study, it was conceived that the IEEE result set would be adequate. It should also be noted that this study focuses on work experience, which is treated as the primary data. As such, considering that work experience reports have been solely published in IEEE, the choice of IEEE is even more perceptible.

3.2 Data Collection: Steps and Results

 In the third filtration step, the 660 papers were revised and their whole text was searched for exact keywords: “offshore/dispersed/remote teams,” “global software engineering” and “distributed teams/development” (see search string 2 in the previous section). As a result, 124 papers were deemed relevant to offshore/distributed ASD settings. Among these and based on an analysis of the paper’s texts (detailed in section 5), 25 papers [1-25] were selected and referenced in the results section.

3.3 Search Extension Considerations

 As the data was slightly outdated since the first attempt, a complementary search was conducted for papers from 2014 and onward. This effort covered all new works from the same IEEE publication source, including conferences and journals. As emphasized before, mostly work experience reports and case studies published in conferences on ASD and GSC (i.e. Global Software Engineering) were sought.

 The data obtained in this complementary stage was used to corroborate the results from the original stage search. Consequently, this relatively lower number of data was used to identify and extract new categories of issues. However, it is was mostly utilized to corroborate the general categories from the original stage, which were trust, cultural differences and general cultural problems, and eventually communication and personal conflict issues.

 A complementary search was conducted on January 28, 2018. By searching the same keywords stated previously, 744 papers (2014: 152, 2015: 178, 2016: 225, 2017: 180 and 2018: 9) were found and examined based on their titles, abstracts and keywords. It is worth mentioning again the number for 2017-18 included only papers available on IEEE Xplore as of the beginning of 2018. As a result, the current number for 2017-18 would be slightly more than the aforementioned number. For example, the papers from 2017 became 202 on March 31, 2018 with the same search criteria.

 In the next step, a total of 35 papers (2014: 12, 2015: 11, 2016: 9 and 2017: 3 and 2018: 0) were found based on the full text review considering their subjects and relevancy. In the final step of this stage and by searching the problem keywords, 21 final papers [33-53] (2014: 8, 2015: 6, 2016: 4 and 2017: 3) were selected for inclusion in the results.

4. Data Extraction and Analysis Processes

4.1. Method Foundation

 Some conceptions and techniques of the grounded theory method (GTM) from [26] and [27] were used. This encouraged extracting concepts from data, classifying those concepts into higher levels of abstraction, finding properties and dimensions for the proposed classes and finally, striving to relate, merge or divide and, overall improve these classes based on continuous comparison.

 Although this article is not a grounded theory study, some GTM techniques for data coding have been used. As Glaser and Strauss explicitly mentioned, their method may be used with any form of data, whether quantitative or qualitative “since the process of generating theory is independent of the kind of data used” ([26] pg. 18). Therefore, as long as the theory is grounded in data it is a grounded theory somehow, even if it does not fully match all the steps and conditions customary of GTM studies. In our case, the evidence collected from work experiences and case studies is a form of valid data. Therefore, they underwent a process of codification.

 After selecting the appropriate phrases (as described later), three analysis stages were performed according to GTM, namely: data coding, second, category classification and finally, the extraction of notions, relationships, properties and dimensions. In this study, the third analysis stage of extracting notions, etc. was limited to pinpointing simple causal relationships between problems/solutions and their consequences. The analysis also includes the associations between problems/solutions and their properties and descriptions and possibly their types and variations. These patterns of relations and properties were simply found based on aggregations of (similar) data. Therefore, if there was more evidence for a certain topic, the corresponding category could be distinguished. Besides, these patterns contain causal relationships only on the basis that in the selected papers it is explicitly mentioned that, for instance, a solution was adopted for a given problem, or these were the consequences of an exact problem or solution.

The results were considered useful because they are based on a vast amount of experience (from IEEE publications). The results on their own are not necessarily subject to verification because the study deals with past experiences as they occurred (“…theory based on data can usually not be completely refuted by more data or replaced by another theory” [26], pg. 4). However, the results of this study may be very easily used to hypothesize new statements that inherently stand out from the current study itself. As a result, such hypotheses will certainly be subject to further verification.

4.2 Data Extraction Approach

 As indicated previously, the relevant phrases were extracted if they were related to reported problems that occurred in geographically distributed settings. All the encountered problems and solutions were collected whether they are obvious results of the factor of distance/distribution or not. Nonetheless the effect of distance was further analyzed based on the provided descriptions and context.

 The process of extracting and coding the data incorporates the following steps, although occasionally more cycles and cross checks can be performed.

 First, phrases that include any problem encountered in the given offshore settings was selected from the texts. A problem was chosen if the sentences directly indicate the problem and/or included terms like barriers, problems, challenges, etc. Thereafter, the problems’ effects within the same text were sought. The effects might be direct (disturbing) consequences or (negative) influences on other factors in the course of development and usage. Additionally, the solutions provided (if they match the same problems) were also considered. In the next step, the solutions served as a base to determine their own results in terms of their effectiveness (i.e. resolutions), new problems resulting from the solutions or their negative side-effects on other environmental parameters. In this manner of data extraction, not only were causal relationships sought, but insight on the (most) proactive factors was provided in terms of how they interact to facilitate or impede the resolution of a certain problem.

 Fig. 1 presents a conceptual model constructed with the semantic relationships between the presumed notions: problems, solutions, causes, side-effects, etc. This conceptual model served as an initial map to search for and relate data in terms of these presumed notions. These relationships are typically causal (from left to right). However, this is not always the case (e.g. between problems and their roots, and between solutions and their prerequisites). Indeed, this order (left to right) generally refers to the way data was sought. For instance, the problems were first pinpointed within the text, then additive information was sought e.g. the problems’ roots, consequences and solutions. Subsequently, the prerequisites, resolutions and side-effects of the same solutions were sought. The same order was used to summarize the results in tabular form (Tables 1-13) from top to bottom (from the problem to its description and causes) and left to right (from the problem to its subclasses, to the solution, to the effects/side-effects). Even though each paper was processed separately, it was also intended to adequately match the problems mentioned in a given paper with the solutions found in other papers based on the situational and contextual similarities between the papers.

 

Fig. 1. Conceptual relationships between phrases sought

4.2.1 Performing the Search for the Problems and their Descriptions (RQ1, 2)

 As stated, the selected set of papers was searched for any definite phrases or paragraphs that contained explicit references to problems that occurred during AM application (in distributed/offshore settings) and relate to cultural issues. As a result, 25 papers were considered. In those papers, 55 issues were selected in terms of barriers and challenges resulting from cultural differences. These issues are related to cultural topics e.g. human behaviors pertaining to a work culture, organizational/team culture, political barriers, cultural differences and so on. The results are presented in Fig. 2.

4.2.2 Performing the Search for Solutions and their Effects (RQ 3, 4)

 To answer RQ3 and RQ4, the solutions provided as well as their effects (if any) were searched for in this study. To be more accurate, solutions were preferably sought for each reported problem within the same (inclusive) paper; although, this approach might limit the opportunity to relate any give problem to the solutions from other papers.

 To classify the solutions, a distinction was made between practices and general suggestions. Practice here refers to specific solutions (in terms of the application of methods, experiences, etc.) so that their application was clearly expected to improve or resolve the given problematic situation. General suggestions by their nature are vaguer (for instance, “try good communication to improve trust”). This classification means was improved by constantly comparing items found. For instance, for trust as a general category practices were divided into three subcategories: (efforts to maintain) continuous communication, facilitating communication, e.g. using communication tools, and other techniques for the purpose of improving communication. Thus, the categorization was gradually enhanced based on newly emerging data evidence.

4.3 Classifying and Presenting Data

 As long as the results emerged, they were formulated into captions, phrases and sometimes (sets of) sentences. Then those captions and phrases were entered into a mind map software, in which each extracted problem was initially classified with its effects and solutions. Under each solution node in the map, their positive or negative effects were classified. Under each problem, its effects, the sub-classes, properties, roots and information implying the situation in which it occurred were also included. For each solution, besides its positive or negative effects, it was determined whether the description provided a general suggestion/guideline, if it was well-defined (e.g. in terms of a method or model) or may be merely understood as a “practice.” Moreover, any extra information available about the prerequisites and situation in which the solutions are applicable was also mentioned.

 Afterward, the initial classification was modified and improved twice in order to provide a more balanced structure with better clarity and meaningful connections, mostly in terms of is-a, have-the-property-of and a-result-of relationships. In doing so, in the first step the problems and solutions were added under each main title (e.g. trust, cultural differences and barriers, personal issues). However, the relationship between any specific problem/solution and its effects were maintained. In the second step, the structure was improved in terms of the generalization/specialization relationships based on the properties found for the problems and solutions.

4.4. Presentation of Results

 The results are presented in tabular form. Therefore, the mind map with the first and second levels are shown in Fig. 2 was transformed into tables (Tables 1-13). The presentation includes a list of problems and solutions found plus extra information, e.g. problem sub-classes descriptions, properties and roots/causes, and solution types, results, side-effects and situations in which they have been applied. In the process of forming the tables, the entire results underwent one more improvement semantically as well as for the sake of readability. This form of presentation (i.e. tabular) is hopefully sufficient to show the multidimensional aspect of the results.

 

Fig. 2. Classification of issues in the first two levels

5. Results

5.1 Trust Category

 As stated above, the classification map details are opened up in Tables 1-13. Each phrase in the tables is accompanied by a reference to the inclusive paper. Alongside the tables’ captions and headers, supportive information for reading the tables is given as follows. A table headed may come along with a category selected for that table (e.g. solutions that are only practices or only general suggestions). The two- or three-column tables are read from the left cell to the right. The items are vertically separated by lines and/or bullet points. When a bullet point is used, the heading title belongs to all the subsequent bullets (even after separating lines). In the case of a chain of four sequential related items, the third and fourth items in the chain come in an extra two-column table. As an exception, the problems’ descriptions/properties, roots/causes and comments are in the same column right after the problem title, whether for main or sub-class problems. An extra explanation for each table is given next.

 Table 1 describes trust as a problem, including its descriptions, properties, and roots and causes as elaborated in the literature (references are included within the table). The general suggestions to resolve/mitigate the problem are mentioned in the third column. More specifically, two variations of this problem and the associated solutions are mentioned in terms of suggestions and practices.

Table 1. Problems associated with trust, variations, sub-classes, consequences and solutions

 

 Table 2 is an extension of Table 1 and the focus is on solutions that may promote trust by means of improving mutual understanding among Agile team members. A specific side-effect of such suggestions is also mentioned (according to [18])

Table 2. Problems associated with trust, variations, sub-classes, consequences and solutions

 

 Table 3 has the same purpose as Table 2, expect it regards solutions (suggestions) that improve trust through communication and collaboration. Some positive and negative effects are also mentioned. Moreover, in two cases, prerequisites for adopting the solutions are also introduced according to [20]. Under the label resolution, more clarifications are provided about why and how the suggestion (i.e. having good communication) might be useful. Under the label conjunct problem, another issue that shares the same suggestion (knowledge sharing [18]) is mentioned.

Table 3. Solutions (general suggestions) and their effects associated with trust for the category: communication and collaboration

 

 Table 4 is similar to Table 3 in all aspects but with a focus on how to maintain a regular pace of communication and collaboration within Agile teams. The same is true for Table 5 which addresses ways to improve communication by using tools and methods that facilitate and simplify it. Table 6 lists solutions to advance trust by means of improving mutual understanding.

Table 4. Solutions (general suggestions) and effects associated with trust for the category: Communication and collaboration

 

Table 5. Solutions (general suggestions) and effects associated with trust for the category: Communication and collaboration

 

Table 6. Solutions (proposed/applied practices) for the category: Improving mutual understanding.

 

5.2 Cultural Differences Category

 Tables 7-9 address cultural differences as a core issue in offshore settings. Table 7 summarizes the problem descriptions and properties, and its known reported sub-classes as well as the suggested solutions. It is notable that the initial list of sub-classes (from “differences in cultural norms…” to “communication between client and vendor…”) has no extra details for each subclass. In the same way, the third column (effects/solutions) is associated with the general problem i.e. cultural differences and all its sub-classes. However, column 2 (from the row “different management style” and afterward) provides the problem sub-classes along with their specific details in the third column correspondingly. This is followed by the roots and causes in terms of causes that possibly lead to cultural differences. In column 3, more information is provided regarding these causes e.g. suggestions on how to avoid them. Two other minor problems (distinguished from the cultural differences problem) are identified in separate rows that are (difference in) time zone and second-class syndrome. Nonetheless, these two are also closely related to cultural differences. Table 8 is an extension of Table 7 in that more general suggestions are described along with the side-effects in column 2 (virtual column 4 if offsetting form Tables 7 that happens to be the column next to the solutions column). Table 9 is similar to Table 8 but includes practices instead of general suggestions.

Table 7. Problems associated with cultural differences, variations, sub-classes, consequences and solutions.

 

Table 8. Solutions (general suggestions) for cultural differences and their effects

 

Table 9. Solutions for cultural differences and their effects-, category of Proposed/Applied

 

5.3 Culture (General) Category

 Tables 10-11 are allotted to the problem culture in general (exempt from including cultural differences). Table 10 summarizes the problem culture (general) and three others i.e. understanding Agile (culture), (lack of) understanding other people and political barriers. Among them, the second (understanding Agile) is detailed with three more sub-classes as described in [23, 24]. The organizational change sub-class here refers to the general notion of the resistance of an existing organizational culture to understand the Agile culture effectively and efficiently. However, more details of this particular issue have not been found in the studied literature considering the method under study. Table 11 is an extension of Tables 10 in that one solution for the problem culture (general) is suggested along with its side-effects (in column 2; virtually equivalent to column 4 by offsetting from Table 10).

Table 10. Problems associated with culture (general), variations, sub-classes, consequences and solutions

Table 11. Solutions (general suggestions) for the cultural (general) problems and effects

5.4 Personal Issues Category

 Tables 12-13 present the problems related to personal issues category. This general category encompasses three other problems which are lack of commitment, personal issues and (lack of or insufficient) motivation. Four additional sub-classes (of problems 1, 3 and 4) as well as extra details and general suggestions are also summarized in Table 12. Finally, Table 13 is an extension of Table 12 in that a general solution for the first, general problem (personal issues) and its one side-effect are mentioned.

Table 12. Problems associated with personal issues/conflicts, variations, sub-classes, consequences and solutions.

Table 13. Solutions (general suggestions) for personal issues and effects

6. Interpretation and Discussion

 A comparison between the surfacing problems is depicted in Fig. 3. As illustrated, trust and cultural differences are two main categories of problems in distributed and off-shore teams. Considering the difficulties arising from distance, this seems plausible. Personal conflicts and issues appear to be more obvious but of lesser significance in agile teams compared to the issues resulting from physical distance i.e. cultural difference and trust.

 The two sets of data (the first from 2003-2014 with 25 articles and the second from 2014-2018 with 21 articles) are almost the same in size. However, as seen in Fig. 4, the number of problems surfacing in the second set is much lower. This is partially as a result of the less exploratory and more corroborative nature of the second data collection phase. At the same time Fig. 4 nonetheless implies that although much more attention has been directed to this subject within the last four years, relatively fewer cultural problems have surfaced and been reported in this period of time. This assertion is lightly substantiated by Fig. 5 in terms of comparing the problems, solutions and effects with each other. The relative number of solutions and effects has been gradually growing in recent years, whereas the relative number of problems does not show the same increase rate. In any case, the results are noteworthy despite not being conclusive. It is also noted that Fig. 5 does not include all the references cited in the 13 result tables, because some of their information cannot be definitely categorized in the three categories (problems, solutions and effects).

 

Fig. 3. Number of problems (including the problems, causes, symptoms and descriptions) over the entire search period (Jan. 2003-Feb. 2018). The same article may have been counted more than once if it was used repeatedly to identify a problem, symptom or cause.

 

Fig. 4. Number of articles referenced as sources for identifying and describing problems, segregated in two data collection groups: exploring phase and corroborating phase. Each article referenced for each category (trust, cultural differences, culture and personal conflicts) was counted only once although articles may be repeated considering the presence of four categories.

 

Fig. 5 - Relative number of references to problems, solutions and effects/side-effects, segregated by years of publication of the inclusive article. Thus, for each article and category (i.e. problems, solutions or effects), the references (in all 13 tables) were counted and then aggregated based on the years when the inclusive articles were published.

 The lack of the possibility for face-to-face relationships and cultural differences in offshore teams are both major roots of problems that are generally recognized and categorized as trust. In the given context, trust appears to be an inclusive notion that may incorporate different aspects and meanings. Indeed, the “trust” (as a problem) category here refers to the problems associated with losing or lacking trust, or surfacing dis/mistrust issues. Trust is considered as a crucial factor in maintaining the pace of (continuous) communication as is the case and required in ASD. The role of trust implicitly relies on existing definitions (e.g. as quoted in [20]), including influencing other parties for the purpose of project governance, sharing any assumed benefits and interests among stakeholders or facilitating communication. Barriers resulting from cultural differences are also notable. A distinction should be made between the problem of cultural differences as barriers (to understand each other) and other problems resulting from the (negative) consequences of cultural differences. For the purpose of this study, all the problematic cultural differences were considered and gathered as quoted by the authors. For instance, a work culture might be argumentative (and then problematic) depending on whether it is (relatively) more hierarchical than collaborative. Differences in time zones and work shifts may specifically result in various cultural barriers as well. Consequently, it can be argued that as long as team members distributed geographically are able to communicate effectively, regularly and understandably, and the trust factor can be established and maintained, such difficulties in cultural differences and barriers would be minimal.

 Considering the fact that communication and collaboration are the essence of AM, this result may be interpreted from several angles. First, this account shows that continuous collaboration is the core of ASD; hence as long as this quality is maintained and continued, the expected AM results are achievable. Second, it may be stated that in contrast, much fewer agile teams are able to accomplish their missions if there is any disruption in their inner continuous communication as a result of cultural differences/barriers for instance. Finally, the results show the problems of cultural nature may not be resolved only by means of technical suggestions (e.g. how to do pair programming). In dealing with such problems, agile practitioners usually rely on their experience, common sense and innovation rather than using theoretical and conceptual models or formal methods. Nevertheless, such models and methods may be of assistance in terms of not beginning to do everything from scratch.

7. Limitations and Further Studies

7.1 Limitations

 Apart from possible theoretical and methodical flaws, the main limitation of this type of research is the requirement for an adequate amount of evidence. It is clear from the results that this amount in many problem/solution categories is not sufficient to conclusively decide for instance if a certain solution is effective for a given purpose or what the exact properties of a certain problem are. This is because researchers must depend on the data available in existing literature. In any event, this study indicates the current state of knowledge as is experienced.

7.2 Further studies

 The proposed study method is straightforward and simple, and we have applied it in several past and current studies, some of which are not yet published. However, it is recommended that researchers are careful with how they utilize existing literature so the evidence collected is suitable for the results sought. Here we looked through reported evidence of problems in actual cases and how the proposed solutions have been applied, whether successfully or not. As such, it was possible to generalize both the concepts and the (causal) relationships as well as their inclusive situations. Different studies may need different approaches regarding data collection and extraction. Moreover, it is important to determine to what degree predefined categories may be used to classify data. We generally used common sense and basic technical knowledge to begin with and then permitted the categories to emerge by themselves and through continuous comparison. However, it is argued that sometimes it is preferable to begin with pre-given classes and categories (e.g. from past studies). As the data source for the current study, work experience reports are preferred. Case studies might have the same value, but the data may possibly be considered less raw. Besides, saving the introduction, method description and conclusion sections, only the result section of case studies is usually of interest for this type of research.

8. Conclusion

 This study arguably suggests a few potential improvements to literature-based studies. First, in this study a search approach was used implying a wider data area initially by applying more general keywords (here e.g. ASD and AM) and then manually searching entire texts. Although this approach is more time consuming due to the manual steps, the researchers had a greater chance of finding relevant data. To analyze and interpret the results, some general guidelines and techniques were adopted from GTM, including classifying data, producing abstract categories, continuously comparing and etc. The results are able to show some a number of causal relationships (here between problems and their effects, and between solutions and their outcomes). These causal relationships were strengthened by accumulating the number of occurrences, as more aggregation leads to greater certainty about each causal relationship. In this study it was also attempted to identify the situational settings of each problem and solution. These situational settings were represented in tabular form to some extent. Therefore, it is suggested that readers actively consider these environmental and situational settings in studying each case (i.e. problem). Moreover, the resulting categorizations with inner relationships may deemed as a theoretical basis for further theorization (see [27] pg. 3).