Advanced insider threat detection model to apply periodic work atmosphere

Abstract

We developed an insider threat detection model to be used by organizations that repeat tasks at regular intervals. The model identifies the best combination of different feature selection algorithms, unsupervised learning algorithms, and standard scores. We derive a model specifically optimized for the organization by evaluating each combination in terms of accuracy, AUC (Area Under the Curve), and TPR (True Positive Rate). In order to validate this model, a four-year log was applied to the system handling sensitive information from public institutions. In the research target system, the user log was analyzed monthly based on the fact that the business process is processed at a cycle of one year, and the roles are determined for each person in charge. In order to classify the behavior of a user as abnormal, the standard scores of each organization were calculated and classified as abnormal when they exceeded certain thresholds. Using this method, we proposed an optimized model for the organization and verified it.

1. Introduction

 As the information system evolves, organizations will continue to be exposed to various threats. Detecting and blocking attacks from the outside is important, but theft by insiders called "insider threat" is also one of the key challenges faced by organizations. We must address such malicious behavior, because it is greatly affecting and threatening organizations.

 The subject of insider threat is an area of great interest. An insider can be defined as a "system user who is entitled to certain rights and use" [1], a "user who is legally authorized to access or determine one or more assets in an organization's structure" [2], and “An authorized user who performs unauthorized actions that loses control of computational resources” [3]. Malicious insiders are "individuals who have access to an organization's network, system, or data, or who intentionally exceed or misuses access in a way that negatively affects confidentiality, integrity, or the organization's information or information system security issues" [4]. As such, research is currently underway worldwide to prevent threats from human resources as opposed to threats caused by system flaws or vulnerabilities.

 According to the 'Vormetric Insider Threat Report' [5] in 2015, 93% of the 408 respondents surveyed indicated that their organizations were vulnerable to insider threats. Examples of insider threats that have gained global attention include the cases of Bradley Manning and Edward Snowden. Bradley Manning, a former army intelligence analyst in Iraq from 2009 to 2010, withdrew hundreds of thousands of classified documents, diplomatic documents, and videos related to the Afghan and Iraqi wars from the U.S. military network and posted them on a website [6]. Edward Snowden, a computer technician in the U.S. who worked for the Central Intelligence Agency (CIA) and the National Security Agency (NSA), released various confidential NSA documents in an effort to inform the public [7].

 Looking at these incidents suggests that insider threats are a threat to not only businesses but also governments and military organizations, as massive national losses could result from information leakage depending on the sizes and impacts of the organizations involved. It is crucial for government agencies to proactively detect and respond to insider threats in a timely manner. The process of detecting insider threats differs depending on the nature and function of the organization. Public organizations, military organizations, and other organizations of a public nature often repeat similar tasks each year [8]. This paper suggests a model with which to detect insider threats based on the user's behavior, limited to organizations that repeat tasks at regular intervals. Further, while various models to detect insider threats are currently being researched, studies that have been validated using actual data are rare. Therefore, this paper validates this model by applying data from an actual public institutions.

 The rest of this paper is as follows. The related work on insider threat detection based on user behavior and machine learning is presented in Chapter 2. Details of the advanced insider threat detection model proposed in this paper are described in Chapter 3. The results of applying the proposed model to an actual organization are shown in Chapter 4. Finally, Chapter 5 concludes this paper.

2. Related Work

 In this section, we focus on the existing studies on insider detection based on user behavior and insider threat detection using machine learning. As the number and variety of information systems are continually increasing, it is impossible to create a normal pattern of all actions in order to detect insider threats based on scenarios or rules. Researchers have thus proposed models that can detect and prevent insider threats in advance based on an insider's psychological expression or behavior based on the records remaining in the active area of the organization. For example, Myers et al. [9] considered using a web server log file to detect malicious insiders trying to exploit internal systems. Eldardiry et al. [10] divided the activity area of the insider into the system log-on, the storage medium, the file, the HyperText Transfer Protocol (HTTP), and the e-mail domain. Philip et al. [11] detected insider threats by calculating the activity score, separating all activities performed within the organization into users and roles, and creating a tree-structured profile that makes it easy to compare one’s activities with those of their other peers. Magklaras et al. [12] developed an insider threat detection tool that measures the threat levels that might originate from specific insiders based on a specific user behavior profile. Brdiczka et al. [13] developed an architecture for detecting insider threats by combining personal psychological profile information with a structural anomaly detection model, and examined ways to improve the detection accuracy. Okolica et al. [14] conducted a Probabilistic Latent Semantic Indexing (PLSI) technique with which to potentially detect employees related to insider threats using indexes related to internal data leakage. Liu et al. [15] proposed the Sensitive Information Dissemination Detection (SIDD) framework by applying statistical and signal processing techniques for traffic flow. Nurse et al. [16] presented a framework for marking the malicious threats and human factors of an attacker, as well as descriptions of the potential attacks that may occur. Matthews et al. [17] developed an active indicator using stimuli that triggered a characteristic response from the insider and detected insider threats. Maasberg et al. [18] analyzed the relationship between dark triad personality traits and insider threats. Kauh et al. [19] developed an Indicator-Based Behavior Ontology (IB2O) with which to detect potential threats based on behavioral ontology in the early stages of social networks and corporate networks.

 As the size of the system grows with the incoporation of technology such as big data and cloud, the number of accumulated logs and the complexity of the analysis have increased, along with the difficulty in detecting insider threats. In order to solve these problems, studies using machine learning in the field of insider threat are being conducted. Parveen et al. [20] used unsupervised learning-based compression-based techniques to model common user behavior patterns and achieved high accuracy in classifying data streams for insider threats with unusual patterns. Parveen et al. [21] proposed an efficient approach for identifying internal threats and hiding non-ideal activities from internal threats using ensemble-based stream mining, unadjusted learning, and graph-based approaches to large data streams. Wangyan et al. [22] proposed an insider threat detection model that works by applying the cloud file share access data to three unsupervised learning algorithms to create indicators for outliers. Tuor et al. [23] developed a model for analyzing the system log in real time and scoring the abnormal behaviors of individual users. Kim et al. [24] used the Markov chain model to classify their state according to user behavior over time, and to detect insider threats using machine learning algorithms.

3. Methodology

3.1 Pre-processing

 Determine if the organization is suitable for use with this model and perform data preprocessing. It is imperative to determine whether this model is available for the organization. This model is intended for organizations that repeat similar types of tasks at given intervals. A public institution or army is a prime example of such an organization. The activities of organizations such as startup companies may vary from year to year, so this model cannot be used to effectively detect insider threats in such situations. Therefore, before using this model, it should be verified that the organization is suitable for the model.

 In general, the collected data cannot be used as is. The data collected should be verified and corrected to make it usable. Keyword extraction, categorization, and normalization are carried out to convert data into a format in which it can be processed.

3.2 Feature Selection

 Select various feature evaluator algorithms and search method algorithms. Feature selection is a process that involves identifying the subset of data that shows the best performance from the original data by selecting related attributes that have the most significant effect on detection or prediction among a large amount of data [25]. If there is an unrelated attribute without a feature selection process, there is a high possibility that the classification is only appropriate for the experimental data, and that it will be erroneously determined when actual new data is added. In addition, it has been proven to be efficient by extracting only closely related attributes in data mining and machine learning problems, thereby improving the analysis speed by reducing the dimension of data [26, 27]. Therefore, the efficiency of this model can be improved by using the feature selection algorithm.

 The feature selection algorithm is divided into a feature evaluator algorithm and a search method algorithm [28]. In this step, the desired feature evaluator algorithms and search method algorithms are selected in order to perform the next step.

3.3 Unsupervised Learning

 Select various unsupervised learning algorithms to apply to the model. Unsupervised learning [29] is a machine learning method that aims to classify unlabeled data and combine it into groups with similar functions. In general, the accuracy of supervised learning is higher than that of unsupervised learning. The reason for using unsupervised learning with this model is that unsupervised learning is more universal; in order to use supervised learning, it is necessary to detect a case in which insider detection occurs at a certain level or higher in the relevant organization. However, due to the nature of insider threats, only a few cases have been found in general organizations. Therefore, this model is employed in many organizations by using unsupervised learning.

 In this study, clustering is used as a representative model of non-instructional learning. Clustering [30] collects data based on high similarity and applies the autonomous learning algorithm to the selected attributes of unlabeled data in the grouping process in order to obtain classified results. In this step, we select the desired clustering algorithms and proceed to the next step.

3.4 Optimization

 Apply all of the combinations of feature selection algorithms, unsupervised learning algorithms, and standard scores to the data and derive an optimized model based on the priority criteria. The proposed insider threat detection model identifies the optimal model that changes the attribute selection algorithm, non-supervised learning algorithm, and standard score in 3D, as shown in Fig. 1.

 

Fig. 1. Optimized model for insider threat detection

3.4.1 Criteria for insider threat

 In this model, the standard score (z score) is used as a criterion for dividing the abnormal pattern of the system user and the normal pattern. The standard score [31] is a non-dimensional value that shows a statistically normal distribution as well as the position of each case on the standard deviation from the mean. The standard score is obtained using the following formula.

\(z=\frac{x-\mu}{\sigma}\)       (1)

 As shown in Fig. 2, numbers exceeding a certain standard score are classified as instances of abnormal behavior.

 

Fig. 2. Insider threat classification

 \(x\): The number of times a task person has performed a particular action

 \(μ\): Average number of times a person has been in charge of a specific action

 \(σ\): the standard deviation of the number of times a person has been in charge of a particular action

 The search order is changed according to the above process, and the maximum and minimum standard scores are set to 2.0 (97.7%) and 1.2 (84.1%), respectively.

3.4.2 Priority criteria for optimization models

 At the end of each cycle of the process, a model is formed, and it is determined whether the result of the verification step is an optimized model based on the priority criteria presented in Table 1.

Table 1. Priorities in determining optimized model

 

 Scores of True Positive, True Negative, False Positive, and False Negative are required to calculate Accuracy, Area Under the Curve (AUC), and True Positive Rate (TPR). True Positive is a correct detection in the clustering algorithm when a worker classified into the abnormal cluster has exhibited an abnormal job pattern on a certain standard score basis. False Negative is a case in which a clerk classified as having a the normal business pattern in the clustering algorithm shows an abnormal business pattern according to the standard score. False Positive is a mistake in the clustering algorithm when a person classified as showing an abnormal business pattern shows a normal working pattern according to a certain standard score standard. True Negative is the correct detection in which the clusters are classified according to the normal task pattern in the clustering algorithm and the standard score is normal. These are described in Table. 2.

Table 2. Confusion matrix

 

 Accuracy is one of the key measures for assessing whether the classification is correct [32]. Accuracy indicates how much the true positive and true negative account for all scores, where the closer the ratio is to 1, the better the classification is evaluated. Accuracy is calculated using the following formula.

\(\mathrm{ Accuracy }=\frac{T P+T N}{T P+F P+F N+T N}\)       (2)

 The ROC Curve is widely used to determine the efficiency of diagnostic methods, and is a suitable way for visualizing the performance of the classifier in order to select the appropriate operating point or decision threshold [33]. The ROC curve shows sensitivity and specificity in a two-dimensional plane. Sensitivity refers to the True Positive Rate (TPR) as a measure of how well a model detects abnormal users, based on a certain standard score. Specificity is represented by 1-FPR (False Positive Rate), which is a numerical value of whether the model selects a normal user among the normal users. TPR and FPR are calculated using the following formulas.

\(T P R=\frac{T P}{T P+F N}\)       (3)

\(F P R=\frac{F P}{F P+T N}\)       (4)

 AUC is the abbreviation for Area Under the ROC Curve, which refers to the area under the curve [33]. The reference line shows 0.5 in Fig. 3, which represents a meaningful test when the curve is to the left of the reference line. Curve A is more suitable as a diagnostic model based on the fact that it has a larger area than curve B on the lower right.

 

Fig. 3. Example of ROC

4. Case Study

4.1 Pre-processing

 The developed model was applied to a public institution in South Korea. This model is not targeted toward all organizations, but it is necessary to validate it, because it targets organizations that are repeatedly working at regular intervals. The organization applied to this model is divided into groups A, B, and C (A is a higher group of B and B is a higher group of C.) The tasks of each group are mutually complementary, and they repeat every year. The detailed business processes are described in Table 3.

Table 3. Processes between each group for one year

 

 This data consists of a log of approximately four years from 2014 to 2017. Fig. 4 shows the number of workers that have accessed sensitive information on a monthly basis over the four-year period. Although there are some differences, it can be seen that the monthly access frequency exhibits a constant pattern every year. Because the business process is assigned to each group in a cycle of one year, the model can be used to evaluate threats to insiders that fall outside of the business pattern. In September and October, the portion of task force team was so high that it did not repeat the work at regular intervals and was excluded at a later stage.

 

Fig. 4. Number of sensitive information accesses over four years (monthly)

 Since this data contains sensitive information, it was analyzed by a relevant worker in charge of the institution. The data used in this study are log records of user-specific sensitive information access from 2014 to 2017. The actual data consists of 88410 records (2014: 22,067, 2015: 20,782, 2016: 22,077, 2017: 23,484). Each record consists of access time, access IP, access method, and accessor ID, as shown in Table 4.

Table 4. Log Format of the data

 

In order to analyze the work pattern in the business process over a one-year cycle, the four-year dataset is divided into months. As the sensitive information access method is limited to inquiry, downloading, and output, the pattern for each user is converted into a continuous data form which is easy to analyze. In order to analyze the successive work patterns by year, we excluded the task force team.

 In order to identify the difference in the access amount of sensitive information by the characteristic of annual business process, the values of the average and standard deviation of downloading, output, and number of inquiries are extended to the attributes as shown in the table below. The standard deviation was used to determine the deviation from the mean, which is the representative value, in order to determine the reference value.

Table 5. Extended Attributes

 

 In this study, these attributes are used as features and become the input values of the feature selection algorithm in the next step.

4.2 Feature Selection

 We used Weka, a machine learning tool, in this study, so Weka's typical feature selection algorithms were applied. The feature evaluation algorithms used include CfsSubsetEval, SymmetricalUncert AttributeEval, CorrelationAttributeEval, InfoGainAttributeEval, and GainRatioAttributeEval. The search method algorithms used are BestFirst, Greedy Stepwise, and Ranker. The details of these algorithms are presented in Table 6.

Table 6. Feature Selection Algorithms [34]

 

4.3 Unsupervised Learning

 In this step, we used algorithms provided by Weka among the representative clustering algorithms. The corresponding algorithms are the Expectation-Maximization algorithm (EM), K-Means, Canopy, and Density-based algorithms. The details of these algorithms are presented in Table 7. The means and standard deviations of clusters are obtained through unsupervised learning. We then use the mean and standard deviation to determine the z value for each record.

Table 7. Unsupervised Learning Algorithms [35]

 

4.4 Optimization

 Accuracy, AUC, and TPR were calculated for all of the cases of feature selection (Feature Evaluator, Search Method) algorithms, Unsupervised learning algorithms, and Z-score (1.2 ~ 2.0) using the SPSS statistical tool. Based on the priority criteria, the most optimized model was derived on a monthly basis. The details are presented in Table 8.

 For every month except January, accuracy is 0.9 or more, and the AUC has a value of 0.5 or more above the reference line in all months, with March and June each showing a maximum of 1.0. TPR shows the detection rates for insiders with abnormal patterns, which are 100% for 3, 4, 5, 6, 8, and November. Looking at the graph, we can see that the monthly accuracy, TPR, and AUC are universally constant, and we can see that the accuracy and TPR are 0.9 or higher when looking at the average. The overall score is summarized in Fig. 5.

Table 8. Evaluation results of the proposed insider threat detection model (monthly)

 

 

Fig. 5. Evaluation results of the proposed insider threat detection model(monthly)

4.5 Validation

 In order to show that the proposed model is valid, 80% of the data is extracted at random, and the optimization model is constructed. Then, we applied the remaining 20% of the data to the optimization model. This procedure was repeated ten times, with the detailed results shown in Fig. 6.

 

Fig. 6. Result of Optimized Insider Threat Detection Model Using Verification Data

 Overall, the values of monthly accuracy, AUC, and TPR are stable. In particular, it can be seen that the verification results are constant in March, April, May, August, and November. From the verification results, we can see that the values of monthly accuracy, AUC, and TPR are similar to or better than the values shown in the optimization model. The results show that the average for each accuracy, AUC, and TPR of the verification results is similar to that shown in the optimization model. Further, this result indicates lower overall AUC and higher accuracy than the latest studies. (Lo et al. [36]: Accuracy 69%, Yuan et al. [37]: AUC 0.9449)

5. Conclusion

 Insider threats do not happen very often, but many organizations are fearful of such threats they can cause huge damage if they do occur. However, there are few practical examples of insider threats and few effective tools to detect such threats. In order to solve this problem, we developed an Insider threat detection model. Since there are many ways to detect Insider threats according to the characteristics of the organization, this model focuses on the general characteristics of repeating tasks in a certain cycle and makes them available to the concerned organizations. The inspirations that the work process of the organization under study is periodic and the role of the person in charge is fixed have become the basis of the model. The model is applicable to organizations that repeat tasks at specific intervals, such as military or public agencies. In this model, standard scores were used to classify abnormal users for insider threat detection. After repeatedly adjusting the standard scores and performing various feature selection algorithms and Unsupervised learning algorithms, an optimized model was derived by comparing Accuracy, AUC, and TPR across each result.

 We applied this model to the data of actual public institutions. The data includes key features such as access frequency, IP address, and access time based on four years of logs. We presented a model that is optimized for the data by using various feature selection algorithms provided by Weka as well as unsupervised learning. In order to verify this optimized model, 80% of the data was used to generate the model; the remaining 20% of the data was applied to this model, which was then proven to be effective.

 The quality of the feature is good in the case study, but it has a limitation in that the amount of features is small. Therefore, in future research, a big data analysis processing system which can process large logs should be applied to collect and analyze logs generated from a complex area such as a system user's terminal, e-mail, and web record. Therefore, more accurate insider threat detection models can be proposed.