An Anomalous Behavior Detection Method Using System Call Sequences for Distributed Applications

Abstract

Distributed applications are composed of multiple nodes, which exchange information with individual nodes through message passing. Compared with traditional applications, distributed applications have more complex behavior patterns because a large number of interactions and concurrent behaviors exist among their distributed nodes. Thus, it is difficult to detect anomalous behaviors and determine the location and scope of abnormal nodes, and some attacks and misuse cannot be detected. To address this problem, we introduce a method for detecting anomalous behaviors based on process algebra. We specify the architecture of the behavior detection model and the detection algorithm. The anomalous behavior detection and analysis demonstrate that our method is a good discriminator between normal and anomalous behavior characteristics of distributed applications. Performance evaluation shows that the proposed method enhances efficiency without security degradation.

1. Introduction

With the popularity and deepening of the network, distributed computing has to expand from its origins in shared-memory computing and local area networks to a wider context [1]. A distributed application is a program distributed on independent computers that exchange information with individual nodes by message passing to accomplish a common task. Compared to the traditional applications, distributed applications have many advantages such as providing increasingly complex functionality and acceptable performance that require parallelizing their operations on individual nodes. Unfortunately, a large complex behavior patterns appear in the distributed applications due to the nodes changing frequently [2]. Nowdays, distributed systems have become an increasingly important platform for providing multiple services, for example, distributed systems form the basis for critical infrastructure in different domains such as finance, medicine, aeronautics, telephony, and Internet services. The quality of such distributed systems is often crucial [3]. However, the development process of distributed applications is difficult and inefficiency. Its development cycle is long, and it is difficult to avoid and find implicit errors and defects. Meanwhile, hackers continually explore and develop new methods to perform attacks on distributed systems. Therefore, security for distributed applications is a concern, and providing safe, efficient parallel implementations of distributed applications remain a challenge.

There are many different security paradigms to protect distributed applications from some attacks. Irfan Gul and M. Hussain proposed an efficient multi-threaded distributed cloud IDS model to handle large scale network access traffic, avoiding sophisticated distributed intrusion attacks like Distributed Denial of Service (DDOS) [4]. Collberg et al. present a new general technique for protecting clients in distributed systems against Remote Man-at-the-end (R-MATE) attacks [5]. Idrees et al. proposed framework is an amalgamation of some of the existing state-of-the-art intrusion detection and prevention technologies for detection and prevention of known and unknown network and cloud computing vulnerabilities [6]. Many distributed real-time systems are often safety-critical and need to be certified, however their certification is hard due to their distributed nature. Meseguer et al. presented a formal model transformation that maps a synchronous design to reduce the design and verification complexities of achieving virtual synchrony [7]. Several assumptions have usually been overemphasized in the above security paradigms: That security policy can be distinctly and correctly specified, that distributed applications can be correctly implemented, and that systems can be correctly configured. In fact, distributed applications are not static: their distributed nodes being continually changed by connecting or disconnecting; Applications are added and removed, and configurations are changed. Anomalous behaviors in distributed applications are often hard to find. Many anomalous behaviors reflect discrepancies between a system’s behavior and the programmer’s assumptions about that behavior.

In this paper, we can improve security through detecting anomalous behaviors based on process algebra. This method is based on the assumption that distributed applications can be correctly designed, but that violations of security policy can be detected by monitoring and analyzing software behaviors. The basic steps are as follows: We obtain control flow graphs (CFG) of individual nodes by static binary code analysis. Then we transform the control flow graphs into the corresponding process expressions automatically by using the technique in our previous literature [8], and rewrite process expressions by eliminating the non determinism and adding concurrency operators. Finally, we construct a behavior detection model of distributed applications and give the detection algorithm.

The contributions can be summarized to the following four points.

1) The interactions and concurrent behaviors of distributed applications can be described accurately. We apply process algebra to distributed applications behavior modeling, which is a profitable security paradigms attempt.

2) We propose the concurrent mechanism in the distributed applications that is structured as synchronization process set and running state set. This reduces the complexity of behavior analysis and detection.

3) Our model only maintains the linear list, and does not use the backtracking algorithm because the non determinism of process expressions was eliminated. This reduces the runtime overheads.

4) We provide a formal analysis method for developers that are familiar or unfamiliar to distributed applications. And we also provide an anomalous behavior detection method for the system maintenance personnel.

The structure of this paper is as follows. In Section 2, we review the previous work in analysis and detection methods for distributed applications. Section 3 introduces process algebra. Section 4 specifies the construction process of our model. The experimental evaluation is discussed in Section 5, and we conclude the paper in Section 6.

 

2. Related Work

Applications use system calls to gain access to functions from an operating systems kernel. Therefore, it is theoretically possible to detect when a hacker may be exploiting a program by analyzing system call patterns of an application [9]. Since the original development of a model that takes advantage of the system call sequence for normal behavior of a program was presented by Forrest et al. [10-11], many scholars have researched behaviors using the system call. By dynamic training or static analysis, scholars use the system call analysis for behavior analysis and detection. These techniques can be divided into three categories: system call short sequences [11-13], automata [14-17], and the Virtual Path [18]. Of these techniques, modeling based on system call short sequences is efficient and can be implemented easily. However, this method is imprecise, and these intrusion detection models are much more prone to false positives. Compared with short sequences, branch and loop structures of programs can be expressed. Modeling based on automata improves the precision of behavior modeling and reduces the false positive rate. Unfortunately, these models still have some limitations. For example, impossible paths, prohibitively high space–time complexity, and they are unsuitability for analyzing concurrent behaviors.

There are many different analysis and detection methods for distributed systems. Moshirpour et al. proposed a method for detecting emergent behavior, which is an important issue in distributed systems' design [19]. Distributed and concurrent object-oriented applications are difficult to analyze due to the complexity of their concurrency, communication, and synchronization mechanisms. Din C. C. et al. established a proof system for partial correctness reasoning based on communication histories and class invariants, which allow components to be analyzed independently of their environment [3]. Yang Fan et al. designed an aspect-oriented programming language based on distributed tuple spaces to enforce security policies for distributed systems [20]. To address the problem of testing the large-scale network services for complex performance problems and configuration errors, Gupta et al. presented an approach to test distributed systems in which they multiplex all of the nodes [21]. As modern data centers run a variety of applications, detecting failures in distributed systems have limited scalability, or have results that are hard to interpret. Tan et al. presented a light-weight technique to quickly detect performance problems in distributed systems using only correlations of OS metrics [22]. Rohr et al. introduced a workload intensity sensitive timing behavior analysis method for distributed multi-user systems, which consider inter-dependencies between concurrent execution operations within a distributed system to reduce the standard deviation for succeeding analysis steps [23]. Moshirpour et al. proposed the utilization of an ontology-based approach to detect emergent behavior in distributed systems by a set of message sequence charts [24]. These methods solve some problems in distributed systems at a certain aspect.

In this paper, we address a distributed application that is composed of multiple computing subjects that accomplish computing tasks by cooperating with each other. The monitoring node in this paper refers to the deployment node of a distributed application—for instance, an application in a process space or an application deployed on a host. These nodes have a unified synchronous clock, so we can determine behavior traces by the occurrence order.

 

3. Process Algebra

Process algebra is a mathematical tool used for depicting concurrent systems [25-26], and is used for researching concurrent, distributed, interactive systems [27]. The "process" mentioned in process algebra refers to the behavior patterns that are shown by the distributed applications. That consists of a series of actions and the operators that are subordinate to the limited action set. This paper uses process algebra to describe the behaviors of distributed applications. We extract a common subset, the basic component of process algebra, containing the sequential composition operator (.), alternative composition operator (+), and parallel composition operator (||A). Let A be a finite set of synchronous actions (A). The syntax specifications are defined as follows:

P::=|√|α.P|P1 + P2|P1||AP2

Their corresponding meanings are as follows:

1) stands for the process down time, no action is performed. √ stands for process terminated successfully.

2) α.P stands for prefix action α, then transformed into process P . Actions in this paper are the same as actions in CCS [28], divided into action ( α ) and co-action ( ), obviously . The prefix action α can have parameters.

3) P1 + P2 stands for the choice of P1 or P2, according to the process subordinated by the following actions.

4) P1||AP2 means that if action ( α ) in P1 and co-action ( ) in P2 are subordinated to set A , then P1 and P2 execute synchronously, while any other actions are executed asynchronously. After P1 and P2 executing, the actions are replaced by τ(α).

Definition 1 Guarded Expression. The process expression begins with the prefix action. e.g., P = α.Q.

Definition 2 Behavior Trace. Suppose the process P can be defined as a finite state transition of the form:

< α1,α2,...αn> is the behavior trace of process P. The set of all possible behavior traces is denoted by traces(P).

Definition 3 Process Equivalence. If P ， Q are two different processes ， and ( traces(P) = traces(Q) ), i.e., traces(P) ⊆ traces(Q) and traces(Q) ⊆ traces(P), then these are denoted as process P ， Q equivalent.

The process algebra is used for describing behaviors of distributed applications in this paper. We detect behaviors base on process expressions; thus, the process equivalence is based on the behavior trace. If two processes have the same behavior trace, they are considered equivalent. This also meets the requirements for behavior detection. However, it differs from equivalence based on mutual simulation of CCS [28]. It also differs from equivalence based on refusal sets in CSP [29].

 

4. Core Mechanisms

The architecture of our model consists of a modeling unit and a detection unit, shown in Fig. 1.

Fig. 1.Architecture of the model

The input to the modeling unit is the binary code of distributed applications, used to generate CFGs of the monitoring nodes. The modeling unit contains the process for building the normal behavior database. Process expressions are obtained from the corresponding CFGs. We rewrite the process expressions to accurately describe the interaction behaviors and concurrent behaviors of distributed applications. The rewritten process expressions are deemed as normal behaviors, saved in files *.pe. The process migration rules are deduced from concurrent calculus rules and are saved in files *.pm, used for detecting anomalous behaviors. The input to the detection unit is the system call sequences monitored at runtime, used as the detection source. The states of distributed applications constitute running state set rsSet. We use the files *.pe, files *.pm, and rsSet to detect the system call sequences extracted at runtime. Once an anomalous behavior is detected, the model will alert and determine the location and scope of abnormal nodes.

4.1 Generating Process Expressions

Based on IA32 platforms running the Linux OS, we use a bank queue management system as an example. The distributed systems consist of get-ticket client, call-ticket clients, and a server. The server maintains a user queuing list and listens to clients to create a thread for each connection. When the get-ticket client sends a request, the server adds the request information to the end of the queuing list. When the call-ticket clients send a request, the server takes out the head of the queuing list and deals with the new business. The processing procedure must be locked because there can be multiple call-ticket clients. We compile the example to binary code and use the EEL [15-16] method to generate the CFGs for each function of monitoring nodes. We eliminate the edges ε of CFG using the previously reported reduction algorithm [30], merging each function CFGs of monitoring nodes into a global CFG. We capture part of the work flow in Fig. 2. The two actions connected by the red dotted line are complementary actions.

Fig. 2.A CFG for part of the work flow

We denote the CFG as G = {V , E}, where V denotes vertices and E denotes the directed edge that is marked with system calls. We store the CFG with an adjacency list, which keeps the system calls on the edge. We obtain the corresponding adjacency list for the CFG of the server, as shown in Fig. 3.

Fig. 3.The adjacency list of the CFG in Fig. 2

The algorithm that transforms the CFGs into the corresponding process expression is given below.

1) All the vertices v ∈ V are denoted by process label.

2) If vi, vj ∈ V and vi reaches vj exist, there exists an output edge e ∈ E , which generates the process expression. vi = e.vj.

3) If on the vertices vi ∈ V exist many output edges n(n 2), then each output edge defined in (2) generates a process expression, connected using the alternative composition operator: vi = ej1.vj1 + … + ejn.vjn.

4) If the vertices vi ∈ V do not contain an output edge, that information is denoted by the successful terminated process mark √.

We obtain the process expressions of the server based on the above algorithm.

4.2 Rewriting Process Expressions

4.2.1 Eliminating Nondeterminism

The concept “nondeterminism” describes a finite automaton that exists in several states at the same time. Similarly, if the process expression contains an item that has the form α.P + a.Q , it is nondeterminist. This kind of process expression can cause backtracking in the detection phase, reducing detection efficiency. Fortunately, we use an ingenious way to eliminate the nondeterminism.

According to definition 3, we know the left distributive law of alternative composition operators is established, i.e., α.P + a.Q = α.(P + Q) . Thus we use P + Q as a new process to eliminate nondeterminism (examples below).

We eliminate nondeterminism by adding a new expression, S8. The effectiveness is similar to the automata theory; however, our method is more intuitive and easy to implement.

4.2.2 Adding Concurrency Operators

The function CFGs cannot express concurrency, so we must rewrite the process expressions to describe the concurrent characteristics of distributed applications.

4.2.2.1 Structure Synchronous Actions Set

According to the concurrent operator P1||AP2 in Section 3, we must determine synchronous actions set A and its complementary actions. In order to accurately depict the behaviors of system calls, we need to capture and analyze arguments. For instance, we can determine whether the send and recv belong to the same socket channel. Then we can determine whether they are complementary actions. From 324 system calls (Linux-2.6.18 kernel), we extract the system calls that cause synchronous operation and analyze their arguments. Some of them are listed in Table 1, where the pairs accept and connect, send and recv are complementary actions.

Table 1.Arguments capture and analysis for system calls

We capture the arguments by analyzing the assembly code; for example.

From the above assembly code, we know the parameters sockfd of send and recv are from [EBP-18h]. So send and recv pass messages to others by the same socket channel, and we mark srcType and desType as c and s respectively: send(c,s), recv(c,s). Because the distributed nodes are being continually changed by connecting or disconnecting, c and s are used to distinguish the different applications, such as a server application or a client application. Moreover, we use them during the modeling phase instead of specific parameter values such as [EBP-18h] for two reasons: (1) We only need to know the relation between parameters during the modeling phase rather than specific values. (2) The specific values can only be determined at runtime. Obviously, we cannot expect programmers to provide specific values because they are too much. However, the relation of parameters is not determined only by analyzing the assembly code. The best way to see the relation is to study the program’s behaviors in correct execution. In other words, we can statistically “learn” the relation between parameters through training.

4.2.2.2 Adding Concurrency Operation

Concurrency operators are added to process expressions in the following situations:

1) Situation 1. The operations can create a new multi-process or multi-thread. When the multi-processes or multi-threads such as fork and vfork appear in the application, we should analyze jump sentences such as JLE and JNE and change their alternative composition operators to parallel composition operators (‘+’→’||A’).

For instance, in pthread_create of process expression S1 in section 4.2.1, the alternative composition operators in S8 are replaced with parallel composition operators: S8=S0||AS2=accept.S1||Arecv.S3.

2) Situation 2. The operations are used for the synchronization of multi-threads.

(a) The mutual exclusion operations appear in system calls, such as lock, pthread_mutex_lock, unlock, and pthread_mutex_unlock.

(b) The condition variables operations appear in system calls, such as lock, pthread_mutex_lock, wait, signal, unlock, and pthread_mutex_unlock.

(c) The read or write lock operations appear in system calls, such as rlock and wlock.

The method of rewriting a process expression is to create a process expression for a), b), c)—for instance, —and let it run concurrently with an access process, such as P||A Slock, as detailed in our previous publication [8].

(d) Parent and child process, the main thread or child thread synchronization operation:

In order to recycle the resources of a zombie process, after calling the child process exit, the parent process must call wait, wait3, wait4, waitpid, pthread_join, etc. Thus, we must add a synchronization mark: We insert an action sigi after wait and insert a co-action after system call exit (i is a positive integer, the identification number). Meanwhile, we add the new inserted action into the synchronous action set A.

3) Situation 3. The operations are used for interaction in distributed monitoring nodes. For example, in the queue management system in section 4.1, after obtaining the process expressions of each monitoring node, we combine the process expressions into the whole process expression by parallel composition operators, i.e., PQMS = G0||A G0||A S0.

4.3 Process Migration Rules

According to section 3, the sequential composition operator and alternative composition operator can migrate as . This gives us migration rule 1:

Migration rule 1:

According to migration rule 1, the following migration can lead to system halt, causing an abnormality.

Path abnormality 1: If

Based on definition 3, the concurrent relation laws are given as follows. We divide the concurrent relations into binary concurrent relations and multiplex concurrent relations.

1) Binary concurrent relations: The relation of two guarded expressions is connected with a parallel composition operator, such as α.P||A b.Q. According to the relation of the prefix action a, b, and synchronous set A, we present six laws for parallel compositions.

Law 1 (zero element) is a zero element; i.e.,

Law 2 (identity) √ is an identity; i.e., P||A √ = P.

According to Laws 1 and 2, we can obtain

Law 3 If then

Law 4 (synchronous abnormality) If , then α.P||A b.Q = .

According to laws 3 and 4, we know the actions in synchronous set A cannot be executed independently. Thus, they must be executed synchronously with corresponding co-actions.

Law 5 If a ∉ A and b ∈ A , then a.P||A b.Q = a.(P||A b.Q).

Law 6 If a,b ∉ A, then a.P||A b.Q = a.(P||A b.Q) + b.(a.P||A Q).

Laws 5 and 6 indicate that actions outside the set A execute asynchronously, and that parallel composition operators (||A) can be ultimately transformed into alternative composition operators (+).

2) Multiplex concurrent relations: The relations of guarded expressions are connected with multiple composition operators, such as (.), (+), and (||A).

Law 7 If

Law 7 indicates that synchronous operations will be performed preferentially when the synchronization condition is met. It is then easy to reach the following conclusion.

Law 8 If

Law 8 indicates that if the process a.P and a.Q are competitive for , they must be concurrent with separately.

The other forms of multiplex concurrent relations can be summed up by the above two relations. Law 7 and law 8 constitute the minimum complete law set of multiplex concurrent relations.

Next we give the other migration rules. According to law 3, law 7, and law 8, we know if then

Thus we can obtain migration rule 2 by combining this with migration rule 1.

Migration rule 2: If

According to law 5, we know that if a ∉ A and b ∈ A , then .

According to law 6, we know that if a,b ∉ A, then and .

According to migration rules 1 and 2, the following migration can lead to system halt, causing an abnormality.

Path abnormality 2: If ai ≠ c and bi ≠ c, then

4.4 Running State Set

We denote the part of the process expressions left of the equal sign as a process label to describe the process state. To record the running states of the application, we construct the running state set rsSet, consisting of distributed applications state appState, thusly: rsSet={appState}, appState={appType,riLink}. Here, appType is the application type identifier, and riLink is the pointers that point to running instance ri. ri={instanceID,proLabelSet,next}. Here, instanceID is the running instance identifier, and each running instance has a unique identifier that corresponds with the IP address and port number of the deployment node. proLabelSet is the process state set of the current running instance, consisting of process label. An application can have multiple running instance ri, and next represents the pointers pointing to the next ri that has the same appType. The construction algorithm is as follows.

(a) Initialize the rsSet so it is empty.

(b) If the process expression contains (||A), then we add the process label on both sides of the (||A) into rsSet.

The rsSet will be updated continually when detecting a system call syscall during the detection phase.The updating algorithm is as follows.

Input: the awaiting detection system call syscall; running state set rsSet.

Output: running state set rsSet.

Algorithm description:

The migration method in the algorithm is given in Section 5.2, shown in Fig. 4.

Fig. 4.The behavior detection flow chart

 

5. Anomalous Behavior Detection and Analysis

In this section, we report the results of behavior detection. The performance of the proposed method is evaluated experimentally.

5.1 Extraction of System Call Sequences

There are two situations for extracting system call sequences.

1) When detecting the behaviors of a monitoring node, such as the get-ticket client, the system call sequences at runtime are marked as awaiting detection actions.

2) When detecting the interaction behaviors of multiple monitoring nodes, if the monitoring nodes have a global clock, then the system call sequences of monitoring nodes will combine into one sequence according to the global clock. If these monitoring nodes do not have a global clock, then the events of distributed applications that happen later may be marked with an earlier time tag. We use the time stamp ordering method proposed by Lamport [2] to combine the events into one sequence by logical time sequence.

5.2 Detecting Anomalous Behavior

5.2.1 Behavior Detection Algorithm

The primary steps of the detection algorithm are shown in Fig. 4.

Its main idea is described as follows.

Input: the action sequences extracted according to the methods in Section 5.1;

Output: some alarm information, such as detection log, abnormal system call, and the current rsSet.

1) Initialize rsSet and read the action sequences in turn for detecting.

2) Query the running state set rsSet on the basis of system call information and obtain the running state of distributed applications. Then query the normal behavior database (files *.pe), finding the process expressions associated with the system call.

3) Match the system call with the prefix action of process expressions. If match is successful, then proceed to 4). Otherwise, alert and save the information.

4) Migrate the running state according to the migration rules (files *.pm) and update the migrated state into the running state set rsSet.

5) Read out the next system call and go to 2).

5.2.2 Behavior Detection Analysis

We analyze the queue management system of a bank in Fig. 2 and capture the system call sequences for get-ticket client, call-ticket clients, and server at runtime, as shown in Fig. 5. We capture system calls and their parameters according to Table 1. Then, we analyze the IP address and port information of the local and destination hosts and make them correspond with the running instance identifier, replacing the original parameters of system calls with the running instance identifier, as shown in Fig. 5. We bind a system call to its appType and instanceID, for example, {accept(2,1),S,1}. That means the appType of the system call accept is S. Similarly, the instanceID of the system call accept is 1. The instanceID of the running instance identifier that interacts with the system call accept is 2.

Fig. 5.System call sequences at runtime

We use the method in Section 5.1 to obtain the system call sequences as follows.

{connect(1),G,2},{accept(2,1),S,1},{pthread_create,S,1},{send(1),G,2},{recv(2),S,1}, {connect(1), C,3}, {accept(3,1),S,1},{pthread_create,S,1},{send(2),S,1},{recv(1),G,2},{send(1),C,3},{recv(3),S, 1},{lock(mut),S,1}, {send(3),S,1}, {recv(1),C,3}, {unlock(mut),S,1} …

The above system call sequences are read in turn, matching with the process expressions ( PQMS = G0||A C0||A S0 ) of the queue management system. Meanwhile, process migrates according to migration rules. The behavior detection procedures are described in Fig. 6. We use the method in Section 4.2.2.1 to obtain the parameters’ relation of process expressions in Fig. 6. The numbered arrows in the figure stand for detection steps. The detection steps are given for the first 10 times. For example, for the first detection read out {connect(1),G,2}, the type identifier of its application is G (get-ticket client), and the current running instance identifier is 2. The initial state is rsSet={{G，{2，{G0}} },{C,{3,{ C0}} }, {S,{1,{ S0}} } }, and thus the process expression G0 is associated with the system call connect(1). According to Fig. 4, we know the awaiting detection action connect(1) belongs to the synchronous actions set A . Therefore, the next system call accept(2,1) needs be read out, so the process migrates according to migration rule 2; i.e., G0 migrates to G1 and S0 migrates to S1, updating rsSet. Then go to step 2. The red number in the box is the running instance identifier.

Fig. 6.Behavior detection procedures

Initial state: rsSet={{G，{2，{G0}} },{C,{3,{ C0}} }, {S,{1,{ S0}} } };

The matching procedure of system calls is described as follows.

1) Query rsSet and determine the process expression associated with the systemcall. Does the systemcall name match the prefix action name of the process expression？ If the match is successful, then turn to (2).

2) Match the parameters of system calls with the parameters of process expressions. For instance, for {connect(1),G,2}, its appType is G, the instanceID of running instance identifier that interacts with it is 1, and its appType is S. Therefore, it successfully matches with connect(g,s). Note: While application types, such as G and S, are not case sensitive, we define the parameters of system calls with capital letters and define the action parameters of process expressions with lower-case letters.

If the system call sequences all match correctly, then the behavior is normal. If the matching procedure causes one of the abnormalities in Section 4.4, then an alert is generated. Moreover, according to the abnormal system call and the current rsSet, we can determine the associated process expression of the abnormal system call and the interactive processes, and then we can determine the location and scope of abnormal anomalous behavior nodes.

If a new node is connected during the detection phase, then we add its state into the running state set rsSet. For instance, when a call-ticket node number 4 connects with the server, rsSet={{G, {2, {G1}} },{C,{(3,{ C2}),(4,{C1})} }, {S,{1,{ S0, S4 }} } }. If a node is disconnected, then its state will be deleted from the running state set rsSet. In this way, the dynamic change of nodes will be dealt with well.

5.2.3 Detecting Intrusion

The sample code for the main function in the server is given in Section 5.2.2, whose function is used to invoke call-ticket or get-ticket. If the first character is a carriage return, then the call-ticket handling function is summoned. If the first character is A, then the get-ticket handling function is invoked. However, if the data packets from the get-ticket client were intercepted by anomalous behavior—such as if their character A is sullied by a carriage return character—the server remains rigid in determining their behavior according to the character. Although such an intrusion can happen, our model is effective in detecting it. Under these circumstances, our model can capture the system call sequences as follows.

…{send(1),G,2},{recv(2),S,1},{connect(1),C,3},{accept(3,1),S,1},{pthread_create,S,1},{lock(mut),S,1},{send(2),S,1},{recv(1),G,2},{unlock(mut),S,1} …

When {lock(mut),S,1} is detected during the detection phase, it is easy to know the running state of the server is S3 from Fig. 6. It can successfully match with the prefix action lock(m). Thus, the process migrates to S4 and then matches with the next system call {send(2),S,1} and prefix action send(s,c). Here the running instance identifier that interacts with system call send is 2, and the corresponding appType is G rather than C. From Fig. 6 we can see how noticeable intrusions are, and how easy anomalies are to detect.

5.3 Anomalous Behaviors Detection Analysis

A brief discussion on the relationship of several common attack types and exception types is as follows.

1) Code injection attacks.

Code injection attacks are the attackers from local or remote to insert dangerous executable shell code into an address space of the process, and then by some means to modify control flow of the process, make the process execute this shell code, eventually achieving the purpose of attacking behavior. Such as the buffer overflow attacks, the format string misuse and double-free. These kinds of attacks will trigger the path abnormality 1 and the path abnormality 2.

2) Impossible path attacks.

Impossible paths exist when multiple different call sites to the same target procedure exist. In order to achieve this kind of attack, the attackers need to construct the call stack to save function return address, and they may implement by injecting code. Once the impossible paths occurred, they will trigger path abnormality 1.Moreover, they may be trigger the other abnormities supposing that the attackers achieve their purposes indirectly by injecting code.

3) Mimicry attacks.

Mimicry attacks are that the attackers modify the parameters of system call using the rightful system call sequences to achieve the attack purpose. Our model only analyses the interactive action parameters of system call sequences, thus it can not detect the mimicry attacks. We will research on a wider range of data flow analysis to overcome this shortcoming in the future.

4) Denial of Service attacks / Distributed Denial of Service attacks.

According to the action mechanism, denial of service attacks / distributed denial of service attacks can be divided into resource depletion denial of service attacks and denial of service attacks based on the abnormality. The former refer to the attackers exhaust system resources by a large number of inputs, for example, setting up a large number of network connections, forcing process processing large files, etc. These kinds of attacks typically applied to distributed denial of service attacks, and they can not trigger the abnormality, thus our model can not detect these kinds of attacks. However, the latter depend on the specific defect of the process, for instance, infinite loops owing to the integer overflow. These kinds of attacks need to change the control flow of process to achieve the attack purpose, and they can trigger the path abnormality 1 and the path abnormality 2, thus they can be detected by our model.

5) The man-in-the-middle attacks.

The man-in-the-middle attack (MITM) requires an attacker to have the ability to both monitor and alter or inject messages into a communication channel. The attacker can intercept all relevant messages passing between the two victims and inject new ones. These kinds of attacks can cause the actions of the relevant nodes do not synchronize, and trigger the synchronous abnormality (Law 4).

6) Man-at-the-end attacks.

Man-at-the-end (MATE) attacks occur in settings where an adversary has physical access to a device and compromises it by tampering with its hardware or software. Remote man-atthe-end (R-MATE) attacks occur in distributed systems where un-trusted clients are in frequent communication with trusted servers over a network, and malicious users can get an advantage by compromising an un-trusted device. These kinds of attacks also need to change the control flow of process to achieve the attack purpose, and they can trigger the path abnormality 1 and the path abnormality 2, thus they can be detected by our model.

5.4 Comparative Study of Previous Approaches

The existing security paradigms proposed for distributed applications are aimed at some specific attacks, such as DOS/DDOS attacks and Remote Man-at-the-end (R-MATE) attacks. A comparative study on previously schemes is presented in Table 2.

Table 2.Comparative study of previous approaches

We also compare the precision of the above schemes and the result is presented in Table 3.

Table 3.Comparison of detection precision for schemes

As can be seen from Table 3, the precision of our scheme is better than the other schemes. But our scheme is better suited for an application to distributed applications that pay close attention to perform orders and logical, such as mission critical system.

5.5 Performance Evaluation

5.5.1 Theory Analysis

We adopt two algorithm methods to search the adjacency list: depth-first and breadth-first. No matter which algorithm is used, the time complexity of the adjacency list is the same: O(n+e), where n denotes the number of vertices and e denotes the number of adjacency edges. Obviously, the space complexity of the adjacency list is also O(n+e). If we store a set of m concurrent processes, the space and time complexity is O(m). Thus, the total space and time complexity is O(n+e+m); that is to say, the proposed method has linear space–time complexity. Moreover, the running costs do not increase significantly with the increasing of application scale.

5.5.2 Experimental Analysis

We use Hadoop for the test objects and deploy a Hadoop computer cluster. Next we investigate a benchmark test case for WordCount, Sort and Pi, using 3, 6, 9, and 12 Slave nodes, as well as the Master node. We record the cost of time and space that the process of modeling and detection incurs. Our runtime environment is as follows: one server running Linux OS (rhel-server-5.4) on an Intel 2 GHZ six-core CPU with 8GB of RAM, as the Master node, and twelve hosts running Linux OS (rhel-server-5.4) on Intel 1.73 GHZ dual core CPUs with 2 GB of RAM, as the Slave nodes. We use the properties of three benchmark test cases listed in Table 4.

Table 4.The properties of benchmark test case

We obtain the time and space overhead in the modeling phase, as shown in Fig. 7. Similarly, the time and space overhead in the detection phase are shown in Fig. 8 and Fig. 9 respectively. Program execution times are to record in seconds. The base execution time has no modeling and detection operation. Percentages compare against base execution.

Fig. 7.The time and space overhead in the modeling phase

Fig. 8.The time overhead in the detection phase

Fig. 9.The space overhead in the detection phase

As can be seen from experimental results, the time and space overheads of our model increase linearly in the detection phase, which basically was anticipated. The time and space overhead of the model are high in the modeling phase because of performing a large number of calculations. However, the time and space overhead of our detection model are satisfactory in the detection phase.

 

6. Conclusions

In this paper, the problem of providing safe, efficient parallel implementations of distributed applications is investigated. We presented a method for anomalous behavior detection using system call sequences. Normal behaviors were defined in terms of system call sequences executed by running privileged processes. We use process algebra to describe behaviors of distributed applications and detect behaviors based on process expressions. Our profiles of normal behaviors, which were generated using process expressions rewritten by eliminating nondeterminism and adding concurrency operators, were precise and complete. The architecture of our model was specified, using a bank queue management system as an example to describe the procedure of the detection algorithm. Behavior detection and analysis results show that our method is a good discriminator between normal and anomalous behavior characteristics of distributed applications. Performance evaluation shows that the proposed method enhances efficiency without security degradation.