A survey on cooperative fault-tolerant control for multiagent systems

Abstract

Complexity science is a new stage in the development of systems science that is the frontier areas of contemporary scientific development. Complexity science takes complex systems as the research object, which has attracted widespread attention from researchers in the fields of economy, control, management, and society. In recent years, with the rapid development of science and technology and people's deepening understanding for the theory of complex systems, the systems are no longer an object with a single function, but the systems are composed of multiple individuals with autonomous capabilities through cooperative and cooperation, namely multi-agent system (MAS). Currently, MAS is one of the main models for studying such complex systems. The intelligent control is to break the traditional multi-agent fault-tolerant control (FTC) concept and produce a new type of compensation mechanism. In this paper, the applications of fault-tolerant control methods for MASs are presented, and a discussion is given about development and challenges in this field.

1. Introduction

With the rapid development of computer technology, communication technology, and intelligent algorithms, as well as the widespread application of artificial intelligence technology in the country’s "air-space integration" or "new infrastructure" strategies, artificial intelligence has become a research hotspot in the field of science and technology [1-3]. Among them, distributed artificial intelligence is an important research direction of artificial intelligence, and the concept of agent comes from the idea of distributed artificial intelligence [4]. Therefore, the rise of artificial intelligence technology has brought new opportunities and challenges to the development of multi-agent systems (MASs). MAS is a comprehensive complex system. "Integration emergence" is the core of complex system collaboration, that is, it can form a new large system through the interaction between different agents through distributed, self-organization, and adaptability. The selection and elimination of "burst" and "fusion" can give birth to some new attributes or laws at a higher level [5]. Cooperative control based on MASs has the huge advantage of "emergence of integration". Its modeling and control problems have received extensive attention from scholars in the fields of mathematics, control, communication, biology, and computers [6], thereby becoming a very important and popular research field. Especially, in the field of control, as the complexity of the intelligent body system gradually increases, new requirements are put forward for its control technology, especially the requirements for the cooperative control technology of MASs are constantly increasing. Based on "inverse entropy" behavior, multi-agent cooperative control is mainly embodied in the following three aspects, as shown below:

(1) Improve fault tolerant performance: During the strike mission of the weapon formation system, any weapon in the formation fails or is attacked by the enemy. Through the cooperative mechanism, other weapons and equipment can quickly and accurately substitute the position of the destroyed weapon to achieve the established formation. In [7], authors propose a novel adaptive finite time prescribed performance fault tolerant control method for spacecraft with actuator faults, uncertainties and disturbances, which greatly improves the accuracy and speed of fault estimation. For the actual motion of the wheeled mobile robot, this paper proposed a fault-tolerant tracking control approach based on MPC and intermediate estimator (IE) that the observer matching condition need not be satisfied [8]. At the same time, through the implementation of a cooperative combat plan, complementary advantages are realized, and the survivability and cooperative combat capabilities in a complex and highly confrontational battlefield environment are improved. Additionally, in [9], for the issue of fault estimation and fault-tolerant control, authors propose a hybrid fault tolerant controller based on the dynamic event-triggered mechanism. And it can effectively decrease the impact of unknown faults on system performance. In [10], for the specific restrictions in the case of vehicle platoon faults, this paper proposes a novel prescribed time performance recovery fault tolerant control method to ensure nominal platoon performance under multiple faults. With the development of intelligent technology, the weapon formation system will become one of the main modes of attacking targets, effectively improving the reliability of air operations.

(2) Improve tracking efficiency: In the process of cooperative tracking of the target by the weapon formation system, the enemy target can be remotely located, which effectively reduces the risk of manned and aircraft detection and positioning. The weapon system has the characteristics of stealth and low detection. It can penetrate into enemy targets for close reconnaissance, monitor and track multiple targets at close range, and use triangulation to detect enemy target radiation sources or use external radiation sources to detect them. It performs precise positioning. In order to avoid damage to a single weapon and equipment, each weapon and equipment is equipped with passive detection equipment to achieve precise positioning and precise tracking of enemy targets.

(3) Improve combat effectiveness: A weapon cooperative formation system composed of weapons and equipment with different combat missions performs special tasks. The weapon formation system is based on a network communication platform and is connected to the ground command center through a communication link to realize battlefield information sharing, so that it can respond quickly and effectively improve combat effectiveness. Meanwhile, in the cooperative formation operations of the weapon system, the "leader-follower" distributed mode is used for formation flight. The leader and the follower maintain real-time communication to realize the real-time transmission and exchange of battlefield situation information. To achieve the maximization of information sharing, it is conducive to the decision-making of the ground station. The multi-agent systems are shown in Fig. 1:

Fig. 1. Different types of Multi-agent system

2. Situation analysis

2.1 Progress analysis of cooperative fault tolerant

MAS cooperative control has a wide range of applications and broad development prospects in the fields of aerospace, military, transportation, industry and civil life. However, how to interact and share information between agents in a complex cooperative situation and complete specific tasks cooperatively is a prerequisite for these applications. For the cooperative formation of MASs, it is a system in which multiple independent agents cooperate with each other to produce similar behaviors and complete special tasks through a cooperative mechanism [11]. The cooperative control of MAS is based on the consistency theory. Because of its application value, it has attracted extensive attention from scholars in many disciplines. The use of cooperative control algorithms has been a hot research topic in the past ten years, and many good research results have also been produced [12-14]. However, most scholars focus on algorithms and consensus theory. In [12], it gives a valuable review on the theory of consistency, but a more meaningful contribution lies in the following literature. Ref. [13] considered the problem of adaptive fault-tolerant synchronization control of a class of complex dynamical networks. However, other cooperative control technologies and applications are not mentioned. Ref. [14] studies the adaptive decentralized fault-tolerant tracking control problem for a class of uncertain interconnected nonlinear systems with unknown strong interconnections. Moreover, some cooperative algorithms and applications are not considered. At present, related research results are shown in Refs. [15-21]. The research on cooperative control of MASs includes many aspects, including flocking [15], swarming control [16], rendezvous [17], cooperative tracking [18], formation reconstruction [19], fault tolerant [20] and consistency problem [21] etc. Among them, consensus problem is the basis for studying other problems.

At present, the important challenges facing the cooperative control of multi-agent systems are as follows:

(1) In cooperative control, it is necessary to design control targets for MASs, rather than control targets for a single system. In addition, the MASs and the control goal of a single agent need to be balanced, that is, to meet cooperative control.

(2) Regarding the mismatch of communication bandwidth between agents, limited signal quality and unstable transmission. In addition, under the influence of strong external interference, how to consider the security design of communication interaction and sharing between agents.

(3) The real-time and timeliness of information interaction between agents should be considered in the design. It should be accurately modeled and signal quantized in the controller design to avoid its impact on system stability. Otherwise, unnecessary chattering will occur, which will undermine the stability of the entire system.

(4) The design of an agent controller in a restricted situation should include highperformance fault-tolerant algorithms, which are implemented through software redundancy, because hardware redundancy is not suitable for small agents, so the fault-tolerant control design of the agent itself is a difficult task. Therefore, based on the above challenges, this article divides the task execution process of multi-agents into two stages: the assembly process and the tracking process. The agent assembling process is the first, and the tracking process is closely followed. Among them, cooperative fault tolerant is mainly studied in the assembly stage, and cooperative tracking is mainly studied in the tracking process. The two stages or processes are closely cooperative, and the performance of the multi-agent cooperative formation can be optimized. The purpose of the first stage control is to eliminate faults and interferences in the cooperative process, and the purpose of the second stage control is that the system gathers a prescribed formation with a time-varying reference trajectory, and then tracks the target in this formation, thereby realizing multi-agents. The purpose of distributed cooperative tracking. Specific indicators of operational effectiveness, as shown below:

(1) Military force indicators are the basis for assessing combat effectiveness, including the number of combat units, the degree of modernisation of equipment and the quality of personnel.

(2) Combat force indicators assess the combat capability of combat units, including fire strike capability, tactical manoeuvre capability, information technology application capability and so on. In addition, command indicators assess the accuracy and timeliness of command decision-making, including the degree of perfection of the command system, the quality and experience of the commanders, and so on.

(3) The security indicators assess the ability to guarantee the combat capability of the unit, including the ability to provide logistical support, communication support, and medical support.

(4) Capabilities against specific threats, including counter-terrorism capabilities, antimissile capabilities, cyber warfare capabilities, etc.

2.2 Fault analysis of cooperative fault tolerant

Modern technical systems rely on complex control systems to meet performance and safety requirements. System faults will not only lead to system quality and performance degradation, but even bring disasters, and ultimately destroy the entire system. According to the different characteristics of the faults, the faults can be divided into the following three categories, as shown in Table 1. And the comparison of different control methods in the same fault situation are given as shown in Table 2. The detailed categories are as follows:

Table 1. Common fault types and models

Table 2. Comparison of different control methods in the same fault situation

(1) System faults: The system's own faults mainly include actuator faults, sensor faults and component faults according to the location of the fault. Among them, the actuator fault means that the actuator used to execute the control command in the control loop is stuck, constant gain change or constant deviation, and the control command cannot be executed correctly, which is specifically expressed as the difference between the input command of the actuator and its actual output. Sensor fault refers to the stuck, constant gain change or constant deviation of the sensor used to detect the measured sensor in the control loop, and the measured information cannot be accurately obtained. The specific manifestation is the difference between the measured value of the object variable and its actual value. Component fault refers to the abnormality of some components or even subsystems in the controlled object, which makes the entire system unable to perform the established functions normally. At present, in the research of multi-agent systems, the research on actuator fault is the most abundant, followed by the related research on sensors, and the research on component fault is relatively small.

(2) Coupling fault: The coupling fault of the system mainly refers to the behavior that affects the coupling mechanism. System coupling faults mainly include network communication faults and mechanical hinge faults. They have their applicable physical system objects. Network communication faults are mainly caused by data packet loss, bandwidth changes, abnormal delays, intermittent communication interactions, etc., which usually exist in aircraft formations [36], robot formations [37], tethered satellite formation models [38], aerospace in MASs such as the machine model [41]. Mechanical hinge faults are mainly caused by faults in interconnected systems with direct physical links, usually in motor roller systems, robotic arm systems [39], electromechanical system models [40], ship lifting systems [42], and other multiple intelligences in the MASs.

(3) Other faults: Other faults mainly include gap faults, compound faults and minor faults. Among them, gap fault refers to the occurrence of faults randomly and irregularly, compound fault refers to the simultaneous existence of multiple faults. Minor faults refer to faults that have minimal impact on the system at the initial stage of occurrence and can hardly be detected.

According to different forms of occurrence, it can be divided into additive faults and multiplicative faults. Among them, additive fault mainly refers to the unknown input acting on the system, which is zero when the system is running normally. Its appearance will cause the system output to change independently of the known input. Multiplicative faults mainly refer to changes in certain parameters of the system, which can cause changes in the output of the system, and these changes are also affected by known inputs. According to the time of occurrence, it can be divided into sudden change fault and slow change fault. Among them, sudden fault refers to the sudden large deviation of the parameter value, which cannot be monitored and predicted in advance. Slowly changing faults refer to faults in which parameter values change slowly with the passage of time and changes in the environment. In summary, the common fault types and common fault models are shown in Table 1. At present, the main problems of MAS cooperative fault tolerant research are shown in Fig. 2.

Fig. 2. The main key issues of multi-agent cooperative fault tolerant research

2.3 Methods analysis of cooperative fault tolerant

The multi-agent research methodologies are as follows:

(1) MAS cooperative FTC based on adaptive technology: As a modern control technology [43], adaptive control can estimate the system's uncertain parameters online, so the control law designed with adaptive technology can realize the system's uncertainty robust control of parameters, and this technology has been widely used in multi-agent fault-tolerant controllers. The key technologies are as follows:

① Redistribution of control laws. The basic idea is to calculate the required control laws under various faults offline and store them in the computer. Then, according to the latest fault information provided by the online FDD, the controller is selected and switched. However, this fault-tolerant method requires high accuracy and real-time performance of the FDD, otherwise any misdiagnosis may cause the fault-tolerant control of the system to fail.

② Reconfiguration Design (Reconfiguration Design). The core is to adjust the structure and parameters of the control system either on-line or off-line based on the diagnosis of faults by the FDD unit. The most common design methods for control law reconfiguration use feature structure configuration methods based on FDD diagnosis.

③ Model-following restructuring control (Mode l-Following). Adopting the idea of model reference adaptation makes the output of the controlled process always track the output of the reference model. This kind of fault-tolerance does not need the fault detection and separation (FDD) subsystem, and when a fault occurs, the actual controlled process changes accordingly, and the control law is adjusted accordingly to ensure that the controlled object tracks the output of the reference model.

(2) MASs cooperative FTC based on sliding mode control (SMC) technology: sliding mode variable structure control is a modern control technology [54]. Because of its strong robustness, a controller designed with sliding mode technology can realize system verification. Robust control for determining parameters, and this technology has been widely used in multi-agent fault-tolerant controllers.

(3) Observer technology is a modern control technology [60]. Because it can estimate system uncertain parameters online, a fault-tolerant controller designed with observer technology can realize the system's online estimation of uncertain parameters. In the body fault-tolerant controller, it has been widely used.

(4) Cooperative FTC of MASs based on neural network technology: neural network technology can realize the prediction and approximation of the uncertainty of unknown nonlinear functions through learning and training, especially radial basis neural network (RBFNN), which has no other forward network. The best approximation performance and global optimal characteristics.

(5) MASs cooperative FTC based on event trigger technology: Event trigger can effectively determine the communication time, save communication resources, reduce communication burden, effectively reduce controller updates, and save control resources.

(6) Cooperative FTC of MASs based on fuzzy logic technology: Fuzzy logic technology has the function of identifying unknown nonlinear functions, and can also be used to solve uncertain problems caused by system faults or external disturbances.

(7) Tracking control based on adaptive technology: For systems with unknown but constant or slowly changing parameters in the system model, in order to further improve the tracking accuracy of the system, adaptive control can be used to self-correct the parameters to achieve the desired tracking effect and maintain better performance. Meanwhile, combined with other control methods, the system can achieve fast and accurate tracking.

(8) Cooperative tracking control based on finite/fixed time adaptive control technology: For finite/fixed time control system optimization, it is the time-optimal control method. In addition to the advantages of optimal convergence performance, the finite/fixed-time controller has a fractional power term, which makes the finite/fixed-time closed-loop control system have better robustness and anti-disturbance. Therefore, it has important practical and theoretical research significance. Its core challenges are shown in Fig. 3.

Fig. 3. Key technical problems to be solved by multi-agents

3. Main results

3.1 Actuator fault and its fast response performance constraints

An important issue for MASs is security and reliability. In actual engineering, with the complexity and continuous expansion of the network scale of MASs, which are becoming more and more susceptible to faults, and the probability of faults in the system is increasing. In most cases, system faults can cause loss of system performance or stability or even serious damage to the system, especially in systems where safety is critical. Faults in MASs can generally be divided into actuator faults, network topology faults, sensor faults, and other component faults according to the location of occurrence.

Among them, actuator faults will cause deviations between the actual output and the control signal, thereby reducing system performance or making the system unstable. In order to solve this problem, how to design a reasonable fault-tolerant controller is an important research problem in the research of cooperative control of MASs. Secondly, most scholars on the research of actuator fault tolerant are FTC methods designed by adaptive, SMC or other technologies. These methods can only ensure that the agent achieves fault tolerant, and less consider the actuator fault tolerant. The operation quickly affects the problem.

Therefore, the controller designed at this time can only theoretically show that the MAS is globally asymptotically stable, and sometimes its convergence time is very long, or even infinite, and lacks the ability to respond quickly to the fault tolerant process of the actuator. This infinite convergence time is not suitable for MASs that require high timeliness and rapid response. Therefore, the design of MASs FTC that can achieve quick response to actuator fault within a limited time is another problem that needs to be solved in this research field.

3.2 Unknown nonlinear dynamic uncertainty

The dynamic model in the MAS causes the measurement accuracy to change due to the actuator fault, which leads to the existence of unknown and uncertain nonlinear system dynamics. Compared with the linear system dynamics, certain assumptions need to be met, such as the non-linear dynamics need to meet the Lipschitz condition, the existence of an upper bound function, and so on. Additionally, nonlinear multi-agent systems have the following characteristics:

① Non-linear dynamical models. This suggests that the interactions between the intelligences are difficult to be treated linearly.

② There are interactions and interference between multiple intelligences. This leads to the fact that the behaviours of the intelligences are not independent but influence each other.

③ The complexity of the system is high. The interactions and interference between the intelligences cause the system to exhibit a high degree of uncertainty and complexity. The control method of nonlinear multi-agent systems. Aiming at the characteristics of nonlinear multi-agent systems, researchers have proposed the following control methods:

① Centralised control. This control method concentrates all the information in the system in a central node, which controls the whole system. The advantages are that it is tolerable easy to implement and system performance is easy to optimise. However, this method has problems such as the risk of single point of failure and uneven computational load.

② Decentralised control. This control method assigns the control algorithm to each intelligent body node in the system, and each node controls only the nearby neighbour nodes. Advantages The advantages are high fault tolerance and balanced computational load, but the system performance is difficult to optimise.

③ Cooperative control. This control method is based on decentralised control with the addition of collaborative information exchange between intelligences to achieve optimal system performance. This method can be further subdivided into transfer function-based control methods and controller-based control methods.

Common MASs are mainly drones, unmanned ships, robots, and satellites. Their actual system models often contain nonlinear and uncertain parameters and other highly uncertain systems. How to ensure that the complex MAS is under the constraints of model uncertainty, parameter uncertainty, and unknown control direction, achieving high stability, high precision, and quick speed convergence, which is the focus and difficulty of studying the problem of cooperative tracking of MASs. Therefore, it is more challenging to solve the cooperative tracking problem of MASs with unknown or uncertain nonlinear dynamics.

3.3 Complex communication constraints

In the tracking phase of the formation keeping process, the MASs is a form of network structure, so the data transmission between the elements in its feedback control loop is realized through the network communication link. Due to the limited capacity of network transmission channels, signal quantization is inevitable. After the signal is quantized, there is a signal quantization error, which causes the traditional controller that does not consider the signal quantization, and the system performance is degraded or even unstable when faced with the quantization error.

In addition, In the multi-agent systems, the transmission bit rate, communication bandwidth, and network resources of the communication network are limited, and with the expansion of the system scale, the amount of information transmitted in the network increases, resulting in the system is prone to communication delays, data packet loss and other undesirable phenomena, and these undesirable networked phenomena lead to untimely reception of information, discontinuity, missing and inaccuracy, which reduces the performance of the collaborative control, and even leads to the multi-agent systems destabilisation and inability to complete the assigned tasks.

Moreover, due to the simultaneous existence of actuator fault and quantification, the traditional adaptive controller design cannot guarantee the convergence of the trajectory of the system. Therefore, how to design an effective quantization controller to compensate for the effects of quantization errors and consider actuator faults and quantization phenomena at the same time, so how to formulate a quantization parameter adjustment strategy, and construct and design an adaptive law to compensate the actuators at the same time There is still a lack of effective solutions to the effects of faults and quantification errors. Therefore, this is a challenging problem.

In the process of keeping track of the formation of MASs, the relative distance will continue to change, so the communication network has strong randomness, and the data is faced with bandwidth limitation, delay, packet loss and noise pollution during the transmission process. A variety of uncertain interferences, all of which increase the difficulty of theoretical research. How to ensure that the complex MASs achieve high stability, high precision, and high-speed convergence under the limited communication conditions as much as possible is another key and difficult point in the study of cooperative tracking of multi-agent systems.

3.4 Prescribed performance problems under compound constraints

How to ensure that the MAS is under compound constraints (there are faults, nonlinear dynamic uncertainties and communication limitations, etc.), establish a nonlinear model with disturbances, design corresponding control laws, and meet specific preset performances, Improve the transient performance of the fault-tolerant process of the MAS and the steadystate performance of the tracking process to ensure that the MAS still has high safety performance in the event of actuator fault and limited communication, and achieves high stability and high precision , High-speed convergence is the focus and difficulty of studying the problem of cooperative fault-tolerant tracking of MASs.

4. Conclusion and future challenges

Aiming at the above-mentioned MAS fault tolerant problem, using advanced control theory and artificial intelligence technology, under the guidance of system engineering thought, from the perspective of MAS cooperative integration, the research work is determined as how to integrate fault tolerant and tracking control into the agent system In the cooperative process, how to improve the fault tolerant performance and target tracking accuracy of the system under the constraints of actuator fault, unknown and uncertain nonlinearity, limited communication, and prescribed performance, so as to realize the cooperative of MASs in the assembly and tracking process The stability, safety and efficiency of control are difficult problems that need to be solved urgently. At the same time, the overcoming of this problem has very important practical significance and engineering application value for promoting the development of the fully autonomous cooperative technology of my country's unmanned intelligent systems.

In a complex confrontational environment, collaborative fault-tolerant control of a MAS with compound constraints is an arduous task with high complexity, strong coupling, and dynamic uncertainty. Existing literature mainly solves the problem of collaborative fault tolerant of MASs under actuator failure, unknown nonlinear dynamics, limited communication, and prescribed performance. Although some results have been achieved and some scientific problems have been solved, these methods have ignored the influence of secondary factors. At the same time, the methods and control strategies proposed are not optimal, and there are still many problems that need further research and solution, as follows:

(1) Integrated control scheme for MAS fault diagnosis and fault-tolerant control. At present, few real-time detection and intelligent diagnosis units are introduced into collaborative faulttolerant control schemes. Research on how to integrate MAS fault detection, diagnosis and fault-tolerant control schemes deserves further exploration.

(2) The design of MAS collaborative fault-tolerant tracking control scheme lacks the understanding of MAS directional fixation. Conduct research on security control issues under malicious attacks in a given topology.

(3) The MAS currently being studied is large in scale and highly coupled, and its collaborative fault tolerant and tracking control effects have only been verified through Lyapunov stability and simulation experiments, and its timeliness and decentralization have not been experimentally verified. Due to the limited processing speed of the current on-board computers carried by the agents and being limited to laboratory platforms, it is impossible to carry out research on real-time interaction and sharing of data in the process of fault tolerant and tracking control of MAS swarm.