Assessing Knowledge Structures for Public Research Institutes

Abstract

This study uses a network approach to investigate the structural characteristics of sub-organizations within public research institutes in order to obtain their implications for organizational structures. We construct a network based on research similarities between sub-organizations because sub-organizations generally build their own research portfolios. We examine how sub-units are organized based on their structural features. The structural features are compared between three public research institutes in different countries: the Korean the Government-funded Research Institutes (GRIs), the Max-Planck-Gesellschaft in Germany, and the National Laboratories (NLs) in the United States. The structural comparison helps to identify organizational characteristics and to differentiate between them. We found little common ground in the research areas between the GRIs because individual sub-organizations have distinct research portfolios. Therefore, the organizational hierarchy of research in the GRIs is less matured than it is in other public research institutes. This study suggests that the GRIs need to establish integrated strategies in order to strengthen the common knowledge base.

Introduction

The capability to efficiently manage an organizational research portfolio has been given greater emphasis than ever before. In this context, the effective management of an institutional research portfolio is viewed as a dynamic capability, which means “the ability to integrate, build, and reconfigure internal and external competencies to address rapidly change environments” (Floricel and Ibanescu 2008, Teece et al. 1997). A research portfolio is sometimes required to renew, coordinate, or regulate pace in response to environmental changes. However, research portfolios are difficult to govern because they tend to have intricate structures. Moreover, research portfolios need to adapt continuously to challenges of diverse societal expectations, demands for relevance and efficiency, and fast-changing technologies. In the case of Public Research Institutions (PRIs), which are an association of research institutes rather than a single organization, changes in research portfolios are even more complicated. Therefore, in order to enhance the capacity to manage portfolios in the PRIs, it is necessary to understand the interdependent research structures between sub-organizations.

The OECD (2011) defines PRIs as “government research laboratories and establishments engaged in activities such as administration, health, defence and cultural services, public hospitals and clinics, technology centres and science parks.” PRIs are more inclined to disseminate their findings for innovation than in the case of industrial research, which focuses on development. PRIs prospered and rose to prominence with military and civil applications after World War II in many OECD countries (OECD 2011). They spanned almost all areas in which governments were involved, and continued growing until the 1960s. After PRIs began to decline in the 1970s, their contributions to innovation and technological development were called into question in the 1980s. This led to a decrease in public spending from the government sector. In recent years, most PRIs have undergone transformations, including modernization, efficiency imperatives, and the promotion of collaboration with industry. Moreover, with recent advances in the understanding of national innovation systems (Lundvall 2007, Sharif 2006), the role of PRIs is now recognized as preventing systematic failures, which impede the efficiency of Research and Development (R&D). The “Triple Helix” model of university–industry–government relationships (Leydesdorff 2003) has shed light on PRIs as one of the major domains. Despite varying functions of PRIs within a national context, general PRIs conduct research in the public domain with significant governmental support (Bozeman 1987).

A similar situation occurred in Korea (Yim 2005), where the imperatives of Science and Technology (S&T) development, led by the government, have been prominent aspects in the country’s industrial development (Arnold 1988, Kim 1995). In the 1960s, Korea imported almost all its industrial technologies from foreign companies, including American, Japanese, and European firms. However, since the 1970s, the Korean government has made an effort to develop indigenous S&T capabilities. Through massive governmental impetus, Korea has achieved remarkable development in S&T, despite the relatively a short period of only four decades. As a part of the strategies used to foster indigenous technologies, Government-funded Research Institutes (GRIs) were established in 1966. The nation’s first GRI was the Korea Institute of Science and Technology (KIST). Their goal at that time was to reduce the national dependence on foreign technologies.

Along with changes in the economic and social requirements, the mission of GRIs needs to be adjusted (Lee et al. 1991, Yim 2005). Research conducted by GRIs has covered a wide spectrum, including the electronics industry (Mazzoleni and Nelson 2005), materials science, physics, computer science, and chemistry (Albuquerque 2001). As Korea entered a post-catch-up era after the 1990s, GRIs began focusing on large-scale; high risk, high reward research; public welfare technologies; and practical problems that were rarely handled by universities and industries. Yang and Jung (2014) reported that recent studies by GRIs have occurred mainly in the fields of chemistry and chemical, mechanical, and civil engineering. Future roles of GRIs were proposed based on lessons from other nations, in particular, the United States, the United Kingdom, Germany, and Japan (Cho et al. 2007).

At the same time, an effective GRI system has become increasingly essential because of management problems originating from inefficiency and the thoughtless establishment of units. In 1999, the governance of GRIs was restructured into a research council system based on the Max-Planck-Gesellschaft (MPG) model in Germany. As a result, based on their research areas and problems, twenty-seven GRIs were placed within the two research councils: the Korea Research Council for Fundamental Science and Technology (KRCF) and the Korea Research Council for Industrial Science and Technology (ISTK). These GRIs have been managed by a single research council as of 2014, and are still playing an important part in the national innovation system (Park and Leydesdorff 2010, Ye et al. 2013).

Each GRI has its own technical plans. However, the sum of knowledge created by individual GRIs is insufficient to estimate all knowledge produced by GRIs. Rather, the knowledge created by GRIs should be interpreted as an interplay between the achievements of sub-organizations. In this regard, Hong (2011) emphasized that research by GRIs should adapt to socio-technical environmental conditions, and should be accompanied by quantitative methods in order to build specific plans, such as bibliographic analyses. Nevertheless, there is a lack of such an effort. In order to address these needs, we explore the knowledge structure of GRIs.

In general, each sub-organization has built its own research portfolio based on their academic mission. Managers (i.e. research councils or ministries), need to steer organizational research by consolidating individual research portfolios. This combined understanding would help to not only diagnose scientific contributions, but also to plan for future research from a holistic view. Hence, this study investigates organizational research structures using an inter-portfolio network. This network approach expedites the response to changing demands on research organization. Here, we use the scientific outputs for eighteen years (1995–2012) to determine the research territory of each sub-organization. Then, we use a network derived from research similarities to describe the academic relations between sub-organizations, and conduct a structural investigation on the network, including nodal classification. Furthermore, we identify organizational characteristics and differentiate between GRIs and other PRIs, namely, the MPG and the National Laboratories (NLs) in the United States.

Our major findings show that research similarities between GRIs are relatively low because individual sub-organizations have their own research areas. However, a few disciplines generate most of the outputs: Chemistry; Chemical, Mechanical, and Civil Engineering; and Math and Physics. Categorizing nodes based on their structural position reveals that GRIs evolve with low levels of structural hierarchy. We suggest that GRIs create an integrated plan to improve the common academic ground between sub-organizations. The results of this study can guide governing bodies in developing strategies to increase the efficiency of organizational management. This paper opens with the motivation of this study and its purpose. The remainder of this paper is structured as follows. The second section explains the methods used, namely how the data were collected and which analytical methods are applied. In the third section, we explain our results. Then, the final section concludes the paper.

 

Methodology

This section describes the construction method of a research similarity network based on sub-organizational research portfolios and their structural properties. A sub-organizational research portfolio is identified by scientific outputs. Here, we adopt a map of science as a journal title-based classification system that maps a bibliographic record to a research area. From similarity measures between sub-organizational portfolios, pairs of sub-organizations with similar knowledge bases are extracted to generate a network. We call the network as a research similarity network in this study. The criteria of nodal categorization according to structural position follow after the explanation of structural properties. Most of the analyses and visualizations in this study were conducted within the R environment (R Core Team 2015), aided by add-on packages, such as ggplot2 (Wickham 2009) and igraph (Csardi and Nepusz 2006).

Data Collection and Portfolio Identification

Bibliographic data associated with sub-organisational scientific outputs were gathered by each research institution: the GRIs, the MPG, and the NLs. We used Thomson Reuters Web of Knowledge as a source of bibliographic data. We consider both scientific articles and conference proceedings as scientific outputs, and publications for an eighteen-year period (1995-2012) are collected for this study. Only affiliations written in English are dealt with.

Each collected bibliographic record is classified into a research area. To accomplish that, we employ a map of science. The map, the UCSD map of science (Borner et al. 2012), is organized into two levels: discipline and sub-discipline. The map can distinguish 554 sub-disciplines within 13 disciplines corresponding to journal titles. In this study, a certain quantity at the discipline level means the sum of the quantity in sub-disciplines belonging to the disciplines. The reason why we use the map here is its applicability to various data sources: The map can categorize most journal titles indexed in Web of Knowledge and Scopus; and it also accepts keyword for the disciplinary classification. Including fractions of a sub-discipline to discipline(s) provided by the map also make easy to identify research area. Additionally, in order to ensure organizational persistence in a sub-discipline, we discard sub-disciplines detecting publications below than twenty.

Network Construction and Structural Properties

We measure similarities between research portfolios mainly by adopting methods from information retrieval, such as “inverse frequency factors” and “second-order cosine similarities.” The weighting method of inverse frequency factors is employed to differentiate unique research from prevalent research. The cosine similarity is a popular measure in bibliographic analyses (Eck and Waltman 2009), and is also called Salton’s measure, an equivalence index, and an Ochiai coefficient. In order to enhance the accuracy of the similarity, we draw a network from the second-order cosine similarities between research portfolios. Colliander and Ahlgren (2012) reported that first-order similarities focus directly on comparing just two portfolios, whereas second-order approaches quantify how similar two given portfolios are to other portfolios. Such methods for measuring similarities in research profiles were originally proposed by García et al. (2012), but their methods were applied to a journal title-based profile system. In the latter context, a small number of matches can determine the research similarity when comparing organizations with different academic backgrounds. Thus, they need to determine research similarities at higher levels of categories by consolidating the journal levels. This method of constructing a research network based on disciplinary similarities was applied in the study of Yang and Jung (In Press).

In addition, for ease of analysis, it is necessary that we condense the all-to-all similarity into significant measurements. Here, we derive a backbone network among research similarities using the Maximum Spanning Tree (MST) algorithm (Kruskal 1956). The MST in this study filters out low similarities using Prim’s algorithm (Prim 1957). The extracted network has the characteristics of a tree-like structure that is acyclic and has N - 1 links, where N is the number of nodes. Therefore, the research similarity network in this study refers to the tree-like structure, a result of the MST algorithm. Consequently, in the network, nodes represent sub-organizations, and a pair of sub-organizations sharing common knowledge foundations are connected.

Latora and Marchiori (2001) argued that interpreting the complex dynamics of a system starts by characterizing the structural properties of the system. In this regard, we evaluate several structural properties of a research similarity network by organization. Here, the number of nodes N shows the size of a network. The diameter of a network refers to the longest shortest path between all pairs of nodes as well as the width of the network. We also measure centrality, one of the most studied concepts in network analysis, to measure the influence of a node, including the degree, closeness, and betweenness centralities (Borgatti et al. 2009, Freeman 1977, Freeman 1978, Freeman et al. 1979). The assortativity coefficient refers to the connectivity correlations between nodes having the most prolific disciplines in common.

Community Structure of Inter-portfolio Network

Generally, the roles in a network refer to classes of structural equivalent nodes (Lorrain and White 1971). Here, the role of node is yielded by two properties (Guimera and Amaral 2005, Guimera et al. 2007): the relative within-module degree and the participation coefficient. The relative within-module z measures a node’s connectivity to other nodes in its module. The participation coefficient P quantifies the connectedness with different modules. The within-module degree z for node i (zi) is defined as follows:

where is the number of links of node i to nodes in module s, si is the module to which node i belongs, and chevrons < ⋯ >j∈s denote averages of terms inside the brackets for all nodes in module s. The participation coefficient P of node i (Pi) ) is also measured in the following manner:

Here, is the total degree of node i, and NM is the number of non-empty modules. When the links of node i are spread uniformly across modules, the participation coefficient is closer to one. In contrast, a zero participation coefficient means that all links of node i are subordinated to its own module. As a result, depending on the relative within-module and the participation coefficient, the connective pattern of a node is classified into one of seven roles, as listed in Table 1. This cartographic method was also applied to the following academic achievements: collaborations (Velden et al. 2010, Vonortas 2012, Vonortas and Okamura 2013) and patent citations (Vonortas 2012).

Table 1Classification of nodes according to their connective pattern

Nodes within the same module obtained from a discovery algorithm are more densely connected to each other than are linkages between modules with the rest of the network. Deeming that inter-community links show high values in betweenness centrality (Bavelas 1948, Freeman 1977), Newman and Girvan (2004) proposed an iterative method for detecting community. Their method identifies an inter-community link, successively removes that inter-community link, and leaves isolated communities. The community detection is employed here to determine the sub-organizational connectedness within and between modules.

 

Results

This section explains the results of the research on the structures of PRIs. In order to determine a research portfolio, bibliographic data are mapped into sub-disciplines according to the UCSD map of science. As an example, Figure 1 depicts the disciplinary mapping result of KIST: each circle corresponds to a sub-discipline, where the sub-organization has articles; its size is proportional to the number of documents; and its color is determined by the discipline to which the sub-discipline belongs. The sub-organizational research territory is visualized by Sci2 (Sci2 Team 2009), a mapping toolset for bibliometric studies. Here, we consider records that the UCSD map can recognize. After refining the bibliographic data, 59,333 articles for twenty-six GRIs are utilized in this study. The World Institute of Kimchi (WiKim) is excluded from our analysis owing to a lack of records. In the same way, we captured 85,540 scientific articles for fifty-eight sub-organizations in the MPG, and research portfolios of seventeen NLs are composed of 192,544 documents.

Figure 1Thematic categorization of KIST on UCSD map of science (Yang and Jung 2014)

We determine the prolific research areas as shown in Table 2, which lists the disciplinary composition of individual PRIs. A significant number of articles come from Math and Physics in both the NLs and the MPG, whereas Chemistry is the greatest discipline in the GRIs. In addition to Chemistry, scientific outputs in the GRIs are centralized within Chemical, Mechanical, and Civil Engineering. Moreover, considerable GRI publications are generated in Math and Physics, as in other PRIs, but the proportion in the former is lower than the latter. Then, outputs applicable to Humanities are hardly observed in all PRIs. We can find some disciplines, a trace of documents are published commonly, such as those of Social Sciences and Health Professionals. Of course, the difference between disciplines in the number of publications can originate from academic practices in each field, but the research interest of an organization can affect the difference.

Table 2Disciplinary composition by organization

Figure 2Similarity heat map of the NLs

The performance of a research organization involves the collective outputs produced by individual sub-organizations and organizational studies depend on each other. In order to deepen the understanding of thematic relationships between sub-organizations, we investigate the network structures of the PRIs. Here, we measure the research similarities between sub-organizational portfolios. The distribution of the NLs among the PRIs is illustrated as a heat map in Figure 3. The NLs have a left-skewed distribution, whereas the distributions of the GRIs and the MPG have a long tail in right extremes. In order to clearly demonstrate the biased distributions of the research similarities, we specify the descriptive statistics in Table 3, such as the mean, variance, quantiles, skewness, and kurtosis. The high value of the mean show that most NLs have similar research portfolios. On the other hand, right-skewed distributions of the research similarities are also observed in other PRIs. Thus, these all-to-all networks need be reconstructed with strong similarities to facilitate our investigation.

Figure 3Inter-portfolio network of the GRIs

Table 3Descriptive statistics of similarity distributions

We induce research similarity networks, with the case of the GRIs shown in Figure 4. Each node represents the disciplinary composition of the portfolio as a pie chart. Here, we magnify the node labelled as KIST in Figure 4. In the figure, the sub-organizational names are labelled, and the size of a name increases proportionally with the number of publications. The link thickness is determined by the value of similarity. The structural properties of the similarity networks are listed in the Table 4. In individual similarity networks, the number of nodes N corresponds to the number of sub-organizations. The number of connections is equals to N – 1, following from the definition of the tree structure. The number of components is one because the MST generates a connected network. Then, the clustering coefficient is zero because the tree structure denotes an acyclic network. The diameter of the MPG is relatively short, taking into account the nodes. This signifies that research units in the MPG are closely arranged, while each unit has its own research area, unlike the NLs. In terms of the number of connections, there are two nodes with maximum degree in the GRIs: KIST and the Korea Institute of Machinery & Materials (KIMM). We observe five NLs of degree three (the maximum value): Oak Ridge National Laboratory, Argonne National Laboratory, Brookhaven National Laboratory, SLAC National Accelerator Laboratory, and Pacific Northwest National Laboratory. Two units in the MPG are connected with five neighbors: the Max Planck Institute of Biochemistry and the Max Planck Institute of Molecular Cell Biology and Genetics. Finally, disciplinary assortativity quantifies the extent to which nodes are connected to nodes with similar features. According to the assortative coefficient values in the GRIs and the MPG, multiple pairs of connected nodes share the same most prolific discipline.

Figure 4Research similarity network and nodal role of the MPG

Table 4Structural properties of inter-portfolio networks

Units under a PRI are organized in different ways with respect to roles, and we reveal the sub-organizational role based on their structural position. Figure 5 exhibits the results of the role classification in a research similarity network of the MPG. Each node represents a unit in the MPG. Its color is determined by the most productive discipline, and its size increases with the number of publications. The color of the nodal border changes according to the role. We detect three types of role in the MPG: two kinds of peripherals and a hub. Interestingly, hub nodes are found only in the MPG. The results of the role assignment are enumerated in Table 5. We find both ultra-peripheral nodes (R1) and peripheral nodes (R2) in all PRIs. Most sub-organizations are classified into R1, which means all links are within their own module. The nodes belonging to R1 are composed of a small number of sub-disciplines, connected to adjacent neighbors with low degree. Compared to R1, a smaller number of nodes are assigned to R2, which strengthen the solidity in a research similarity network by binding together different modules. We categorize five GRIs into R2: Korea Atomic Energy Research Institute (KAERI), Korea Basic Science Institute (KBSI), National Fusion Research Institute (NFRI), Electronics and Telecommunications Research Institute (ETRI), and KIMM. In the GRIs, R2 is the highest level in the cartographic hierarchy. Two NLs fall within R2: Brookhaven National Laboratory and Pacific Northwest National Laboratory. Nodes within R2 are connected to nodes in the same module with a high degree, and to nodes in different modules at a weak level. Satellite connectors (R3) are found only in the NLs, despite being minority nodes: Argonne National Laboratory and Oak Ridge National Laboratory. Even though R3 connects different modules, it is not necessary in order to be a prolific lab. Two units in the MPG are provincial hubs (R5): the Max Planck Institute of Biochemistry and the Max Planck Institute for Human Cognitive and Brain Sciences. The nodes classified under R5 hold an important position in a research structure, but do not have to publish a large number of articles.

Table 5Role assignment of research similarity networks

 

Conclusions

This study structurally investigates public research institutes around the GRIs. In recent years, there has been an increasing demand for the GRIs to work together to promote innovation. For example, the research council is operating an inter-disciplinary research group where researchers gather to solve technological difficulties, irrespective of their affiliation, in order to rapidly fulfil industrial needs. However, high barriers between the GRIs hinder their collaboration. The statistics on the research similarities between GRIs show that it is difficult to harmonize GRI structures because individual GRIs conducting their research independently in different fields. Thus, this study can give a direction for organizational restructuring and formulating policies.

According to our findings, the research structure of the GRIs shows a poor hierarchical structure. There are no hub GRIs because their MST forms a boughless tree. From a managerial perspective, this structure is more difficult to manage than are the two PRIs. Most nodes are at equivalent levels on the periphery. In that case, every node would be an intervention point because sub-organizations hardly share intellectual common ground. In addition, the research structure of GRIs may be vulnerable to research failure in any of the sub-organizations, which would split a network into pieces. Therefore, our findings show that governing bodies (i.e. research councils or ministries) should focus on enhancing structural robustness by strengthening the common knowledge base between sub-organizations. That would improve the structural efficiency of managing the GRIs. Throughout the PRIs, we observed only four types of roles among seven categories. The reason is that the classification was applied to a skeletal structure. In addition, nodes in a research network are restricted in terms of spreading their connections across modules, because most sub-organizations are mission-led, with the exception of multi-disciplinary research units. This study utilizes scientific outputs for an eighteen year period, but we can explore the evolution of research in a PRI by dividing the data set into several equal-sized intervals. This issue remains as a topic for future work.