Discovering AI-enabled convergences based on BERT and topic network

Abstract

Various aspects of artificial intelligence (AI) have become of significant interest to academia and industry in recent times. To satisfy these academic and industrial interests, it is necessary to comprehensively investigate trends in AI-related changes of diverse areas. In this study, we identified and predicted emerging convergences with the help of AI-associated research abstracts collected from the SCOPUS database. The bidirectional encoder representations obtained via the transformers-based topic discovery technique were subsequently deployed to identify emerging topics related to AI. The topics discovered concern edge computing, biomedical algorithms, predictive defect maintenance, medical applications, fake news detection with block chain, explainable AI and COVID-19 applications. Their convergences were further analyzed based on the shortest path between topics to predict emerging convergences. Our findings indicated emerging AI convergences towards healthcare, manufacturing, legal applications, and marketing. These findings are expected to have policy implications for facilitating the convergences in diverse industries. Potentially, this study could contribute to the exploitation and adoption of AI-enabled convergences from a practical perspective.

1. Introduction

The emergence of artificial intelligence (AI) technologies has blurred the boundaries of many domains. With the progress of the fourth industrial revolution, diverse convergences have been triggered by blurred boundaries over areas. Due to their interdisciplinary nature, these AI-enabled convergent areas are likely to have diverse interactions and prompt the continuous evolutions, influencing researchers and practitioners [1, 2]. Therefore, it is necessary to understand the occurrence and emergence of these changes. [3].

The identification of emerging AI convergences and their changes are of our interest. AI convergences have recently become the frontier of digital transformation involving the joint evolution of industry and academia and influencing day-to-day life. Thus, this study attempts to provide a comprehensive and empirical approach to AI convergences [1–4]. Particularly, the study posits that tracking the emergence of AI convergences might be difficult because various domains are interactively participating and continuously changing. This necessitates the examination of trends and their convergences over diverse domains. For this reason, a specific understanding of the adoption and exploitation of these convergences is needed. However, unfortunately, previous studies have been limited to comprehensively examining and predicting AI convergence with active interactions of various sub-areas.

This study attempts to use a data-driven approach not specific to certain area or subject to any biases. We examine the evolution of AI convergence by applying bidirectional encoder representations from transformers (henceforth, BERT) with its topic modeling application [5, 6]. Execution of topic modeling could contribute to the detection emerging topics in AI-convergent areas. The shortest path investigation is also consecutively conducted on topic networks, and emerging convergences could be investigated.

‘Through this analysis we aimed to understand various opportunities and their evolution, it was expected that various opportunities and their evolution could be effectively understood. This paper is structured as follows: Section 2 reviews the related literature and methodologies and introduces the proposed framework. Section 3 explains the methodology in detail, Section 4 is devoted to the explanation of the analytical results, and Sections 5 and 6 comprise the discussion and conclusions, respectively, with possible directions for further study.

2. Literature Review and Research Issues

2.1 Emergence of AI and its Convergences

AI has been attracting significant interest from practitioners and researchers in various domains [7]. There could be different approaches to implementing and utilizing AI from the perspective of concepts, techniques, and possible applications. Previous studies have specifically started from a technological aspect and focused on implementing AI technologies [8]. Interestingly, a considerable amount of research is now focusing on the identification of its spread and the establishment of a strategy for effective implementation [9].

Meanwhile, AI has become the primary locus for the confluence of diverse technological and industrial needs with cutting-edge computing technology derived from information and communication technology (ICT). Interest in AI is hard-wired into the evolution of innovation ecosystems, where the active interactions of various domains are fully concentrated. That is, AI-based problem solving could shape its industrial landscape and applications to respond to global AI-based technological needs [10]. Borges et al. (2021) introduced the perspective that AI could grow out of the implementation of artificial neural network models, which began in the 1950s with ideas centered on implementing classification algorithms. AI concepts were subsequently combined with developments in digital transformation to create fast-evolving AI techniques for various domains.

The emergence of AI seems to have heretofore relied on advancements in ICT [7]. Like Shao et al. (2022) pointed out, the use and popularity of AI, especially the deep learning-related approach, has been rapidly increasing [8]. Thus, AI plays a critical role in leading industrial application and digital transformation. Findings from previous studies have indicated that AI accelerated ICT alignment with business landscape and also suggested a framework for measuring knowledge gaps and implications for sustainable development [9].

Along with the technological advancement of AI, this research also focuses on the research streams associated with the study of concepts and issues associated with AI. Current research on AI requires new avenues for exploration and exploitation [9]. Trending AI research is expected to contribute to an important aspect of its further improvements. Much research is likely to occur in various fields, such as energy, business, organization and education [11-14]. As evidenced by previous studies, AI seems to evolve with efforts to increase the competitiveness of its associated domains through a more practical and problem-solving perspective.

2.2 Tracking AI Convergences

As abovementioned, AI has rapidly grown and achieved success over various domains [1–4]. Its prevalence could be attributed to its interdisciplinarity and continuous evolution [7]. Understanding this interdisciplinary aspect of AI is likely to be a focus of future work to expand its applications to various data-driven problems. In particular, the interdisciplinary area could experience a rapid change and sudden development as Lee (2022) indicated and this development could affect academia as well as industries. Such interdisciplinary areas have attracted considerable interest among researchers in many ways [15].

Therefore, it is significant to discover how such interdisciplinary area like AI has been experiencing various changes. Unfortunately, for AI, tracing the evolutionary trajectory of interdisciplinary aspects might be challenging as many interdisciplinary domains contribute towards vigorous interactions and changes. To elaborate on this, Langer et al. (2021) further analyzed the stakeholders of AI and its ecosystem in terms of an explanation using an explicit model that emphasizes the presence of interdisciplinary opportunities [16].

However, previous studies may lack a systematic understanding of active interactions and their evolutionary trajectory with AI among domains. In addition, AI convergence is continuously changing and associated with diverse domains. Therefore, it is necessary to systematically track the increasingly complex evolving landscapes of AI convergence. As the exploitation and influence of AI convergence are accelerating, it becomes necessary to examine and forecast its trends and latent issues [17]. Such efforts further help practitioners and researchers to effectively and efficiently advance AI applications in their respective domains. Furthermore, convergences with AI have generated important leverages for the pursuit of innovation in diverse ways. Based on the abovementioned perspective, this study attempted to identify emerging topics of AI convergence, their networks and future trends. The evolutionary trajectory of AI convergent domains could subsequently be considered a basis for further research and development avenues.

Diverse bibliometric approaches can be used to track AI convergence based on relevant documents. For example, some studies [18, 19] have addressed the emerging issues and its convergence to uncover the latent aspects of certain domains. Moreover, considerable effort has been made to predict the emergence of important topics [20]. In that context, this study attempts to effectively analyze the domains newly converging with AI. The evolution and interdisciplinary changes in AI convergence might in turn benefit one by comprehensively identifying research issues and their altering trends. Our research could contribute to the establishment of data-driven problem solving backed by AI.

2.3 Research Issues

As provided, AI-related areas are now emerging and could bring significant leverage for triggering data-centered innovation. Thus, understanding and utilizing those emergences and changes from convergence with AI could become an important issue. As one of the ways to pursue AI convergences, this study attempts to examine the emerging issues, their relations, and their recent trends to achieve further opportunities for AI convergence. The proposed approach in this paper is relatively unique in comparison with previous approaches of predicting convergences. We expect that the proposed analysis could identify ways to effectively explore and exploit AI convergences.

As abovementioned, AI is likely to converge along various areas with unprecedented progress. These convergences of AI among diverse domains could require more specific understanding for further utilization. With the blurred boundaries between several domains, one could interpret and adopt such AI convergences with a systematic understanding of their changes. It is expected that data-driven examination could play a critical role in not stuck on specific sides. Subsequently, AI-associated evolutions among emerging industries could develop appropriate knowledge to respond to global needs, and the related changes could be utilized in a wide range of domains.

Unfortunately, previous studies lack a comprehensive understanding and prediction of AI convergence where various sub-areas vigorously interact with each other. With regard to this, efforts have been made using numerous techniques in the past to identify emerging areas. However, these studies might lack an integrated perspective, particularly for accelerating AI convergences. Therefore, the present study proposes a new analysis to identify and predict emerging convergences with AI, while also providing its empirical application to AI.

Identifying emerging AI convergence and their evolution is of significant interest. To address this, the present study proposed the shortest path-based analysis of the AI convergence of interdisciplinary areas. This method is also based on bibliometric data using natural language processing, the BERT language model and machine learning techniques. Through the proposed method, one could expect to understand the possible converging of various opportunities and their further developments. Therefore, many researchers and practitioners are likely to focus on AI applications in other areas, which are regarded as the interactions for triggering further evolutions.

3. Data and Methodology

This paper attempts to analyze the occurrences of AI enabled convergences over various domains. The empirical approach used the SCOPUS database with its export functionality. Specifically, we exported 3,597 abstracts of journal articles and conference proceedings on AI convergences by using keywords such as “AI,” “deep learning” and “artificial intelligence” through SCOPUS from 2017–2021 (as of April 2022). As this study used a search strategy with general keywords, it outputted a wide range of results and enabled the tracking of relevant research in various areas. In that sense, the findings were expected to reveal the convergence of AI with other domains. In order to properly have the empirical evidence, we newly put forth the analysis on AI enabled convergences, and the approach relies on the joint use of natural language processing, BERT and topic network. The proposed framework is as shown in Fig. 1.

Fig. 1. Proposed Framework.

For the natural language processing (henceforth, NLP) process, the abstracts went through stemming and stop words removal. Through the analysis, BERT and hierarchical DBSCAN clustering (henceforth, HDBSCAN) were deployed as they enabled researchers to discover latent topics from unstructured text in a better manner in comparison to previous methods. BERT could learn deep bi-directional representations on large scale corpus. BERT has recently been exploited for processing and analyzing natural languages due to its ability to represent large-scale texts into the probabilities of their tokens using deep learning techniques [5]. BERT has shown relatively good performances in many NLP problems over years. Such pretrained model is useful to represent a word and sentence with more accurate embeddings. It therefore has wide applications, and BERTopic, which is a BERT-based topic discovery technique [21], was used in the present study to track AI enabled convergences. BERTopic relies on distilBERT among several pretrained models from Hugging face since distilBERT is fast and effectively working.

That is, its established bidirectional aspect enables us to model the features from given data efficiently, and, interestingly, BERT could be fine-tuned on specific problems. Through BERTopic, the research abstracts are well represented with a total of 512 dimensions. BERT provides embedding of words, and these are further exploited to discover the topics from AI convergences. The pretrained BERT was fine-tuned to the research abstracts on AI convergence using a sentence transformer. The BERT-based topic discovery technique was subsequently deployed with a minimum topic size of 10. Further, clustering was applied for the discovered topics. HDBSCAN is a clustering algorithm [22] that is applied with Uniform Manifold Approximation and Projection (UMAP) embedding [23] based on cosine similarity. It is widely known that UMAP technique could reduce the dimension of data and fit well with topic discovery.

Finally, based on the discovered topics, we build the similarity-based topic network by using topic embeddings for discovering AI convergences. The topic network can be established based on topic embeddings. Each node on such topic network represents the discovered topic, and each link on the network stands for the relationship between nodes with the cosine similarity exceeding the threshold. This topic network is established on the embeddings derived from BERT. These embeddings are known to well represent the given texts, and the similarity could be effectively calculated from the results. Though, most of previous studies focused on predicting such edges on the network, this paper assumes the occurrence of AI convergences by connecting the related topics on the topic network using Dijkstra algorithm, which finds the shortest path on the network.

In that context, we concentrate on the structure of topic network. For the interdisciplinary area, there exist complicated interactions among diverse domains. Such aspect can be represented as a topic network where each node is fully connected to other nodes. It is possible to transform this characteristic of relationships into the multi-graph containing multiple edges between nodes. In addition, those edges are established based on the similarity. As this paper regards the similarity over 99.5th percentile is only important, that similarities between nodes over threshold only regarded as the meaningful links for building the network of discovered topics. Particularly, after excluding zero similarities, the non-zero similarities are of our interest to binarize edges. Among the non-zero similarities, we attempted to find the point where the similarity value is suddenly increased, and that point is regarded as the threshold. After comparing the average value of similarity percentiles with the interval of 0.5%, the average similarity of 99.5th percentile is relatively greater than that of 99th percentile by about 30%. So, 99.5th percentile is chosen for binarizing the similarities.

For the conciseness and simplicity of research, multi-graph is then converted to weighted graph. The number of multiple edges between nodes is modified as the weight of single converged edge. Precisely, this paper considers that if the nodes are closer, then, they have more possibilities of convergences. Dijkstra algorithm is one of the widely used shortest path algorithm, and it is consecutively deployed on the converted weighted graph for finding the shortest path between all pairs of nodes. Eventually, through the discovered shortest path, it is expected to predict the occurrence of convergences of AI related topics.

4. Results

4.1 Topic Discovery & Clustering

Targeted abstracts of research articles and conference proceedings were collected from SCOPUS over various domains. The BERT-based topic discovery discovered 70 topics out of 3,597 studies on AI. Firstly, this study applied BERT to investigate topics from AI associated research. The publicly available, pretrained BERT model is installed and research abstracts are applied to be fine-tuned. Specifically, the embeddings from BERT are transformed into the topics based on UMAP and HDBSCAN. For effectively finding the relevant topics, the dimensionality of BERT embeddings is reduced into 5 as UMAP could well maintain the local structure of BERT embeddings. Then, those UMAP result is further grouped into 8 clusters with HDBSCAN. Following this, a topic model was conducted based on the abstracts. As shown in Table 1, these findings ranged from AI techniques to industrial applications.

Table 1. Discovered Topic Label

Our findings clearly show that the recent research on AI largely relied on its industrial and practical applications. For instance, the discovered topics range over various domains, namely drug discovery, medical application, marketing, advanced network, security, manufacturing application, material engineering and financial and legal applications. COVID 19 related topics also appeared for diagnosis and spread analysis. There are also topics on data engineering, GPU computing, explainable AI(XAI) and reinforcement learning.

Fig. 2 depicts the distance map between topics. As shown in Fig. 2, some of the discovered topics were similar and grouping them provided a better understanding of AI enabled convergences. Using HDBSCAN clustering, the entire 70 topics were grouped into eight clusters, which were well interpreted with the relevant topics as shown in Fig. 3.

Fig. 2. Distance Map of Discovered topics on AI convergences.

Fig. 3. Clustering analysis for topics on AI convergences using HDBSCAN.

These clusters were edge computing, biomedical algorithms, predictive defect maintenance, cancer image analysis, medical applications, brain damage, fake news detection with block chain and XAI, and COVID 19 applications. In particular, the COVID 19 related cluster included topics on AI diagnosis, AI assistant for COVID 19 patients care, and drug development. This could imply that AI convergence actively occurred in the perspective of data driven problem solving over many areas. In addition, fake news-related clusters appeared to be important. Associated topics were fake news, malicious attack detection, explainable AI, and legal AI. The findings showed that AI convergence could also focus on cyber security. Remaining clusters suggested emerging topics on AI convergence, such as autonomous driving, crack inspection, or defect inspection.

4.2 Predicting Convergences

In this section, we analyze the emergence of AI convergences over the topic network. Though previous studies mostly utilized the link prediction, this study deployed the shortest path algorithm by focusing on the current topic network. This approach exploits robust aspects of analysis as it is highly based on an already established network. The distance of Dijkstra’s shortest path ranges from 0.995 to 4.997, and the average distance is calculated as 2.2317. Each shortest path represents the possibility of convergences between topics. The distance between each pair of nodes is inversely proportional to convergence possibility.

Some pairs of nodes with distance below 1 are chosen for further analysis. Based on top 15 shortest paths on each node of topic network, these results of nodes are provided as the most possible technological convergences. By filtering the shortest paths, we narrow down to top 15 possible convergences of discovered topics as shown in Table 2.

Table 2. Predicted Convergences

Table 2 suggests some candidates of AI convergences. Findings show that the most possible candidates of predicted convergences occur over various domains, such as Healthcare, Dental care, Surgery, Legal application, Bio-medical research, Security, Manufacturing, and Computing techniques. Specifically, for instance, this finding suggests that AI convergences are likely to occur with 6G networking and medical applications. In addition, AI computing seems to be associated with Healthcare area. Such findings provide that AI convergences are likely to concentrate on those major industries among various areas. In addition, the application area could be associated with the technical aspect of AI, and such relatedness is highly expected to be strengthened.

5. Discussion

There has been increasing interest in research on AI convergences for obtaining the competitiveness and continuous development. The qualitative approach used in the present study enabled it to successfully examine the emerging convergences and analyze their evolution. The findings of this study provide noteworthy insights for practitioners and researchers in AI-associated areas. In particular, this study contributes to discussions on research agendas and their possible convergence with other areas.

First, this study derived emerging issues regarding AI and its related domains. In this context, a new analysis is proposed to discover insights to effectively accelerate the uptake of AI. Furthermore, our findings could provide potential policy implications to researchers and practitioners in related areas by clarifying the emerging convergent areas for further development. Those could correspond to recent calls for further efforts to facilitate the evolutions of industries in the future. Examining the latent trends in AI convergence is expected to facilitate the development of interdisciplinary areas.

Second, this study focused on embedding-based topic convergences for further development while some previous studies mostly considered the citations or co-occurrences of classification codes to identify convergences. Such studies might lack an intuitive interpretation. Third, this study empirically applies the proposed analysis to a fast-evolving area, AI convergence. Specifically, AI is technology-driven in diverse areas; therefore, the convergence corresponds well to the research scope. While related issues are continuously changing, predicting their occurrence with the shortest path could discover the emergence of new areas. Therefore, grafting AI convergence into various industries can play an important role.

6. Conclusions

Currently, AI and its related areas are technology-driven; thus, the convergences on AI fit the research interest. This study offered insights for practitioners and researchers in AI-related areas. To be more elaborative, this study contributed towards discussions on the development of AI-related convergences and their agendas. The findings of this study proposed the application of BERT-based topic discovery and its joint use with clustering. As a result, these findings identified some emerging areas associated with AI and could extend relevant policy implications.

In this context, the findings of this study also corresponded to recent calls to further our understanding of AI convergence. In particular, this study focused on discovered topic as a recent trend of convergence, whereas some previous studies mostly focused on certain AI techniques of AI applications. Eventually, it is expected that this study will lead to a deeper understanding of AI and AI-enabled changes in a unique manner. However, this study was limited as it partially considered academic perspectives on AI convergence. Additionally, the proposed method smoothened the temporal aspects of AI convergence, which were likely to include research issues. These limitations should be addressed in future studies.

Finally, our findings indicated that research on AI convergence is currently insufficient because of its ever-changing behavior. That is, the proposed application and findings could bring a perspective of meta-understanding for newly generated AI convergence. This could broaden the research and practical perspectives for emerging industries and economic growth for sustainable development. Consequently, these findings are useful for researchers, practitioners, and policymakers to enhance their engagement in developing better AI convergence.