Applying a Novel Neuroscience Mining (NSM) Method to fNIRS Dataset for Predicting the Business Problem Solving Creativity: Emphasis on Combining CNN, BiLSTM, and Attention Network

Abstract

인공지능 기술이 발달하면서 뉴로사이언스 마이닝(NSM: NeuroScience Mining)과 AI를 접목하려는 시도가 증가하고 있다. 나아가 NSM은 뉴로사이언스와 비즈니스 애널리틱스의 결합으로 인해 연구범위가 확장되고 있다. 본 연구에서는 fNIRS 실험을 통해 확보한 뉴로 데이터를 분석하여 비즈니스 문제 해결 창의성(BPSC: business problem-solving creativity)을 예측하고 이를 통해 NSM의 잠재력을 조사한다. BPSC는 비즈니스에서 차별성을 가지게 하는 중요한 요소이지만, 인지적 자원의 하나인 BPSC의 측정 및 예측에는 한계가 존재한다. 본 논문에서는 BPSC 예측 성능을 높이는 방안으로 CNN, BiLSTM 그리고 어텐션 네트워크를 결합한 새로운 NSM 기법을 제안한다. 제안된 NSM 기법을 15만 개 이상의 fNIRS 데이터를 활용하여 유효성을 입증하였다. 연구 결과, 본 논문에서 제안하는 NSM 방법이 벤치마킹한 알고리즘(CNN, BiLSTM)에 비하여 우수한 성능을 가지는 것으로 나타났다.

With the development of artificial intelligence, efforts to incorporate neuroscience mining with AI have increased. Neuroscience mining, also known as NSM, expands on this concept by combining computational neuroscience and business analytics. Using fNIRS (functional near-infrared spectroscopy)-based experiment dataset, we have investigated the potential of NSM in the context of the BPSC (business problem-solving creativity) prediction. Although BPSC is regarded as an essential business differentiator and a difficult cognitive resource to imitate, measuring it is a challenging task. In the context of NSM, appropriate methods for assessing and predicting BPSC are still in their infancy. In this sense, we propose a novel NSM method that systematically combines CNN, BiLSTM, and attention network for the sake of enhancing the BPSC prediction performance significantly. We utilized a dataset containing over 150 thousand fNIRS-measured data points to evaluate the validity of our proposed NSM method. Empirical evidence demonstrates that the proposed NSM method reveals the most robust performance when compared to benchmarking methods.

I. Introduction

Efforts to merge neuroinformatics with artificial intelligence have increased in tandem with the development of AI [1]. Neuroscience is the study of the nervous system from a scientific perspective [2]. It is an interdisciplinary field that integrates computer science in order to comprehend the fundamental and emergent aspects of neurosystems. Due to ongoing efforts to integrate neuroscience and AI, data collecting techniques such as classical data mining in neuroinformatics are now essential for analyzing neuroscience data in business analytics. NSM, also known as neuroscience mining, is a multidisciplinary field that bridges the gap between computational neuroscience and corporate analytics [3]. Notwithstanding, the creative potentials of NSM have never been investigated. Moreover, in order to distinguish themselves from rivals, modern businesses are eager to develop their Business Problem Solving Creativity (BPSC).

BPSC is a special form of imagination. As opposed to conventional creativity, BPSC enables us to handle the issues inherent to business operations. Due to the fact that creativity is an intangible cognitive resource that is difficult to imitate, assessing it has long been viewed as a highly unstructured decision-making activity [4].

In this context, in order to contribute to the NSM and creativity investigations, we present a unique mechanism for integrating deep learning models and attention mechanisms with the objective of more accurately predicting BPSC. Therefore, we utilize the collected fNIRS (funtional near-infrared spectroscopy) experiment dataset to predict the BPSC [5]. This study is, to the best of our knowledge, a pioneering effort to predict the BPSC within the setting of the NSM.

Our proposed mechanism is theoretically novel in that it exploits the robustness of deep learning models to extract a set of critical features from a large number of neuroscience experiment datasets and then assigns attention weights to a handful of relatively more important features in order to improve performance [6]. The fundamental function of the attention mechanisms utilized in our proposed NSM approach is to improve BPSC prediction. Consequently, our proposed research question can be summarized as follow:

RQ: The new incorporation of deep learning models and attention mechanisms can enhance the performance of NSM for BPSC prediction tasks?

The BPSC dataset was garnered from the neuroscience experiments conducted in fNIRS. Estimating cortical hemodynamic activity in response to neural activity, fNIRS measures brain activity. Convolutional Neural Networks (CNN) and Bidirectional Long-Short-Term Memory (BiLSTM) are among the deep learning models utilized to answer the aforementioned RQ.

II. Previous Studies

1. Neuroscience Mining with AI

Conventional neuroscience approaches like Generalized Linear Model(GLM) and Analysis Of Variance(ANOVA) were unable to cover complex patterns included in the data. As GLM has a limitation provoked by loss of data, temporal complexity was lost during it’s own process. On the other hand, NSM allows us to preserve temporal complexity during process. In this regard, NSM is a critical method to be adopted specifically in measuring cognitive methods to be adopted in the business analytics area. Along with the increasing number of insights emerge from integrating neuroinformatics and business analytics, importance of NSM has highly increased in recent days.

NSM is a technique for acquiring data in the field of neuroinformatics. According to the current state of artificial intelligence, Hadeel et al. (2018) conducted a data mining study on Clustering functional magnetic resonance imaging (fMRI) data with a robust unsupervised learning algorithm for neuroscience. Clustering techniques have been used to estimate brain activity in fMRI research [13]. In this regard, the authors implemented a Robust Growing Neural Gas (RGNG) algorithm and compared it to a Growing Neural Gas (GNG) algorithm that had never been used in a medical application before. The proposed algorithm could be used to define the center positions of an output cluster corresponding to the minimal MDL value.

Table 1. Recent Studies Applying AI with Neuroscience

According to recent trends, deep learning outperformed other machine learning methods in terms of extracting features from raw input data and providing superior classification results. Dharmendra et al. (2021) conducted a review on how to classify neuroscience problems using a deep learning framework [14]. The authors used deep learning methods including CNN and LSTM to classify EEG signals with a focus on epilepsy, sleep stage disorders, and mental stage disorders.

The research findings indicate that deep learning methods are a promising option for future researchers conducting classification research on a variety of neuroscience samples, including EEG signals.

Asjid and Jawad (2019) used DNN (deep neural networks) and CNN to improve drowsiness detection using fNIRS data [15]. The drowsy and alert states were classified using DNN. CNN was used to train and test the models on color map images to determine the best channels for detecting brain activity. In the experiment, the CNN architecture model achieved an average accuracy of 99.3 percent.

2. Business Problem Solving Creativity (BPSC)

In contrast to ordinary creativity, business problem solving creativity (BPSC) is constrained by conditions that must be met in real-world situations. To begin, it must be applicable to real-world situations. Unlike conventional creativity, which focuses exclusively on originality and imagination, BPSC is required to provide appropriate and creative alternatives while taking into account the organization's environment. In this regard, BPSC exemplifies practical creativity by taking into account the context of their own organization in order to generate competitive advantage. According to BPSC's characteristic of requiring creative competitive advantage, it must be creative enough to enable subjects to defeat their competitors and provide an appropriate solution to the problems of organization is facing with [16].

Fig. 1. The Architecture of the Proposed NSM Method

Fig. 2. (a) fNIRS device source detector; (b) 30mm sensor; (c)30mm sensors with prefrontal brodmann area

III. Dataset

1. fNIRS Data

Data used in the experiment was obtained by using Functional Near Infrared Spectroscopy (fNIRS). fNIRS measures brain activity by estimating cortical hemodynamic activity in response to neural activity. Along with EEG, fNIRS is one of the most frequently used non-invasive neuroimaging techniques in portable settings. The fNIRS has been shown to accurately reflect mental workload during various tasks [5], also it could be considered an effective method for assessing participants' creativity in a variety of situations. As seen in the Figure. 1, the fNIRS raw data consisted of neuroimages showing brain activity, including oxygen saturation, of the frontal lobe of the cerebrum. From the data of 27 subjects, this study predicts the subject's BPSC by analyzing 68 channels (30mm) and 152,945 measurements (see. figure 2). We trained and tested by 10-cross-validation with the implemented dataset. OBELAB's NIRSIT transcriptional measurement equipment was being used to measure each participants' fNIRS.

2. BPSC Extraction

As a target variable, the BPSC was calculated as follows. Each participant was given a scenario containing a business problem, and they were tasked with creating a cognitive map of the scenario. Experts converted each participant's cognitive map into a score out of 10, leading to the BPSC score. Then, for the sake of computational clarity, the BPSC score was grouped by quartile. Consequently, the BPSC is used as a target variable, which is comprised of four classes based on the quartile of the BPSC score such as very high, high, low and very low.

IV. Methodology

1. Deep learning in neuroscience

Convolutional Neural Network (CNN) is a class of Artificial Neural Network (ANN) in deep learning. It is used mainly for image and video recognition, image classification/segmentation, natural language processing. CNN is regularized version of multilayer perceptrons. As it is the multilayer perceptrons, each neuron in one layer is connected to all neurons in the next layer. BiLSTM (Bidirectional long-short term memory) is the process of making neural network with including the sequence information in both directions backwards (future to past) or forward (past to future). BiLSTM is used primarily on natural language processing. Both CNN and BiLSTM are robust and effective method to be adopted in neuroscience study [17,18].

Fig. 3. Implemented Attention Mechanism

2. Attention Mechanism

Our proposed NSM method is based on the integration of CNN, BiLSTM, and an attention network. The attention mechanism is a method for simulating a human's cognitive attention in an artificial neural network. It enables prediction methods to place greater emphasis on a small but significant subset of input data. For this purpose, the attention mechanism uses gradient descent to train a model to determine which subset of data contains more vital information than others, based on the context. Figure 3 depicts the architecture of the implemented attention mechanism.

3. The Proposed NSM Method

Although conventional deep learning models have achieved many state-of-art results in prediction tasks, they will be able to make better prediction by adopting attention mechanism. The architecture of the proposed NSM method is shown in Fig. 1. After putting fNIRS data consists of 48 channels, the input data flows into each feature extraction layers. After obtaining the output data from feature extraction layers, we add an attention network layer to calculate global weights. After the implemented attention mechanism calculation, DNN with 6 layers makes prediction to classify four classes of BPSC such as very high, high, low and very low.

V. Results and Concluding Remarks

In comparison to other benchmarking methods, our proposed NSM method yields more robust and significantly improved performance, as summarized in Table 2.

As shown in the Table 2, the proposed NSM Method which is based on the integration of CNN, BiLSTM, and attention network made better performance than other implemented methods. In the overall accuracy score, the proposed NSM method shows better accuracy score in every validation sets. Furthermore, average performances of the proposed NSM Method are higher than CNN (increment: 4.43; 4.02; 4.40; 4.68), and BiLSTM (increment: 14.95; 12.65; 14.86; 15.61)

Table 2. BPSC predicting performance

1) Validation : Number of 10-K-Fold Cross validation

2) Acc : Accuracy (%)

3) Pre : Precision (%)

4) Rec : Recall (%)

5) F1 : F1 score (%)

To begin, by incorporating an attention network into CNN and BiLSTM, the RQ was clearly addressed, and the performance of predicting BPSC based on the fNIRS experiment dataset was significantly improved.

Second, our proposed NSM's improved performance was robust even when compared to benchmarking methods such as CNN and BiLSTM.

Third, the findings suggest that incorporating an attention mechanism into multiple deep learning models may help to improve NSM results.

In summary, based on the empirical findings in Table 2, we can conclude that the proposed NSM method, in which the attention network is added to multiple deep learning models such as CNN and BiLSTM, has the potential to improve BPSC predictive accuracy when using the fNIRS dataset. The followings are possible future research topics:

Firstly, our findings are currently limited to fNIRS data from a relatively small number of subjects (less than 30).

Second, beyond fNIRS, the proposed NSM method should be applied to more diverse neuroscience methods such as EEG, ECG, Eye-tracking, and fMRI, among others.

Third, the proposed NSM method can be generalized to include more advanced deep learning models such as RNN, XGBoost, snapshot ensembles, fast geometric ensembles, and HAN (hierarchical attention network), among others.