Adversarial Detection with Gaussian Process Regression-based Detector

Abstract

Adversarial attack is a technique that causes a malfunction of classification models by adding noise that cannot be distinguished by humans, which poses a threat to a deep learning model. In this paper, we propose an efficient method to detect adversarial images using Gaussian process regression. Existing deep learning-based adversarial detection methods require numerous adversarial images for their training. The proposed method overcomes this problem by performing classification based on the statistical features of adversarial images and clean images that are extracted by Gaussian process regression with a small number of images. This technique can determine whether the input image is an adversarial image by applying Gaussian process regression based on the intermediate output value of the classification model. Experimental results show that the proposed method achieves higher detection performance than the other deep learning-based adversarial detection methods for powerful attacks. In particular, the Gaussian process regression-based detector shows better detection performance than the baseline models for most attacks in the case with fewer adversarial examples.

1. Introduction

In recent years, the development of machine learning, especially deep learning, and its applications have been actively studied [1]-[5]. The deep learning model has been applied tovarious fields, such as image classification [6]-[10], natural language processing [11]-[14], semantic analysis [15][16], and object detection [17][18] and shows high performance as astate-of-the-art technique. Although deep learning is applied in many areas, the advers arialattacks that have recently been proposed raise questions about the reliability of the deeplearning model.

The adversarial attack is a technique that changes the results of a classification orregression model by mixing perturbations that are imperceptible to human in the input data of the model. Recently, adversarial attack methods that deceive the image classification neural network model have been actively studied [19]-[23]. For a given natural image x that has noperturbation, the adversarial attack produces an image x' that is visually similar, but has adifferent classification result. The x' is called an adversarial example. By creating advers arialexamples through the attack, attackers can mislead the neural network model. Fig. 1 at right shows an image in which the adversarial attack is applied to the image at left recognized as & ld quo;panda&rd quo; by the image classification model. The two images are not distinguished by the human eye, but the image classification model recognizes the right image as “gibbon”, ratherthan “panda”.

Fig. 1. An adversarial image that has imperceptible perturbation added to the natural image [19].

If a deep learning model is applied to a major part of the system, the adversarial attack can lead to serious problems in system security. For example, when an adversarial attack is applied to the deep learning model that is part of an autonomous vehicle, the model interprets the current scene differently, and a fatal accident may occur. To prevent this, several methodshave recently been proposed to protect against adversarial attack [24][25]. Adversarial defenseaims to get the result of x for a given adversarial example x'. Many adversarial defensemethods have been proposed, such as increasing the robustness of the model by augmenting the training data of the classification model [19], or creating a more robust network by distilling the original classification model [24]. However, few defense methods that caneffectively defend against various and powerful attacks have yet been proposed.

Other researchers proposed adversarial detection instead of adversarial defense, which determine whether a given image is an adversarial example or not [26]-[29]. If the detectordetermines that the given image is an adversarial example, the image classification model can prevent the attack, by rejecting the classification task of the adversarial image. From the viewpoint of the service system with a deep learning model, it is only necessary to reject the input causing the malfunction of the system, without having to give an accurate result for that. Adversarial detection can initially block the provision of information about the decision boundary of the deep learning model to the attacker. Therefore, the attacker may receivelimited information about the model, making it more difficult for the attacker to perform amore sophisticated attack [30].

However, many of the detection methods already proposed are the deep neural network-structured model, which requires a large number of adversarial examples to train. From the point of view of the system to which the deep learning model is applied, it is moreeffective to perform detection with only a few adversarial examples, since advers arialexamples of a number of attacks can be secured. In particular, applying an attack that generates adversarial examples using gradients of the model can generate the advers arialexample that deceives not only the classifier model but also the detector, and the detection method applied to the model can be useless [30]. Therefore, adversarial detection toeffectively prevent adversarial attack requires the following characteristics:

1. Detection should be performed with high performance, even with a small number of advers arial examples.

2. Adversarial detection methods should be non-differentiable, so that gradient-based attacks cannot be applied to detection methods.

In this paper, we propose an efficient detection method for adversarial example. The proposed method consists of two steps. First, the intermediate feature values generated by the classification model are extracted for a given image. Second, the intermediate feature information is used to determine whether the image is an adversarial example by applying Gaussian process regression [31]. Gaussian process regression measures the correlationinformation of given data into a covariance matrix, and performs regression based on it. Therefore, classification for detecting adversarial examples can be performed effectively withonly a small amount of given data. In addition, the output function for the input is non-differentiable, because the Gaussian process regression trains the probabilistic distribution of the output using the observed data. Therefore, the secondary adversarial attack on Gaussian process regression does not work. Experimental results show that our detection model has good results with fewer adversarial examples than other neural network-based detectors. In particular, for powerful attacks with high attack success rates, our detection model outperforms the baseline model. In addition, the proposed model showed betterdetection performance than the baseline model for the whole attacks, in the case where thelearning data is extremely small.

Section 2 describes the adversarial attack, adversarial detection, and the Gaussian process regression used in the proposed method. Section 3 describes the proposed method, while Section 4 presents experiments and results to verify the performance of our detection method. Section 5 concludes the paper.

2. Related Work

2.1 Adversarial Attack

The basic purpose of the adversarial attack is to create an example with a minimal perturbation that looks similar to a natural image, but causes the target model to be misclassified. Theadvers arial attack has been actively studied, especially for deep learning models that perform image classification. For a given deep learning model , an adversarial example for thenatural image is generated by the following constrained optimization problem [32]:

(1)

where, and denote output classes of the model for and , respectively, and denotes the distance between two images. That is, the adversarial attack is an optimization problem that minimizes the size of the perturbation mixed to the natural image , under the condition that the classification result by the model is different from the naturalimage. From the viewpoint of the decision boundary of the model, the adversarial example is adata point that belongs to a different class from the natural image, but that is located very close to the decision boundary of the natural class. This is because although the adversarial example is located at a very small perturbation distance from the natural image, the output class by the model is different [33].

Recently, various types of adversarial attack have been proposed by researchers. Good fellow et al. [19] introduced the Fast Gradient Sign Method (FGSM), an attack that uses the gradient value of the loss function of a model for a given natural image. The FGSM is expressed as follows:

(2)

where, denotes the loss function of the model (e.g. cross-entropy), and is epsilon, which indicates the size of the perturbation mixed in the natural image. Since the gradient of the model contains the direction information to the decision boundary of the true class for agiven image, FGSM creates an adversarial example by adding a perturbation of magnitude ε to the image in the opposite direction of the gradient generated by the model. Fig. 1 shows the image generated by the FGSM.

Kurakin et al. [20] proposed an iterative version of the FGSM, the Basic Iterative Method (BIM). BIM is an attack that generates adversarial examples by repeatedly applying FGSM assmall steps, and is also called Iterative FGSM (I-FGSM). BIM is expressed as follows:

(3)

BIM carries out a search on the assumption that an adversarial example exists in thee-neighborhood based on the natural image. BIM can also perform a targeted advers arialattack that causes the output class of the adversarial example to be a specific target class. This method is an Iterative Target Class Method, expressed by the following equation:

(4)

If BIM is performed during sufficient iterations, it is observed that the adversarial examplegenerated by this attack can always have a target class as an output by the model.

Papernot’s Jacobian-based Saliency Map Attack [21] performs targeted attacks through the advers arial saliency map. The saliency map represents the influence of each input feature on the output of the model for a given image, and is computed through a Jacobian matrix, as follows:

(5)

Based on the saliency map, JSMA generates adversarial examples by changing only thosefeatures with a high saliency map value in the natural image, that is, those having a large influence on output determination. The model can easily be deceived with only a small perturbation generated through JSMA. Experimental results show that the attack rate of 97 % is achieved by modifying only 4.02 % of the input feature. However, it has the disadvantagethat the attack time is long, because of the high computational cost in updating the saliency map.

Moosavi-Dezfooli et al. [22] proposed the DeepFool attack that updates the perturbationvector for the natural image every iteration, and performs the algorithm, until the result image is misclassified for the first time. The DeepFool attack generates an adversarial example by adding noise iteratively using a perturbation vector with the direction of the nearest decision boundary from the given image, based on the assumption that an adversarial example exists near the decision boundary of the model.

Carlini and Wagner [23] proposed the C&W attack that is a kind of targeted attack, and has better performance than the other attacks proposed so far. They set the loss function that is lowon the adversarial example and high on the natural image, and perform an adversarial attack by minimizing it. The strongest L2 loss among the losses they searched for is [34]:

(6)

where, the adversarial example, and and are parameters.

In addition to the attacks mentioned above, various attack methods have been proposed that modify existing attacks, or have new methods. The proposed adversarial attacks areapplicable to different models in various data, and show high attack success rates. Thequestion about the reliability of the deep learning model is raised through adversarial attackstudies. Since a fundamentally robust deep learning model for such adversarial attacks has notyet been proposed, an additional method that can defend attacks is needed, such as advers arial defense or adversarial detection.

2.2 Adversarial Detection

Powerful adversarial attacks with various algorithms have been proposed, and thus in order todefend against such attacks, an adversarial defense has been proposed. The basic purpose of the adversarial defense is to allow a given model to produce a true label for the advers arialexample input. Goodfellow et al. [19] proposed a typical defense method called advers arialtraining. It is a kind of data augmentation that adds adversarial examples to the training data of the model. Adversarial training successfully defended against the adversarial attack method that used the augmentation, but failed to effectively defend against other attacks. In particular, advers arial training cannot effectively defend against the secondary attack that attacks the model with the defense method as a new target network. Adversarial training, as well as various adversarial defense methods, have been proposed, but most of them have shown high performance only for specific attacks, and it is impossible to defend against the newly proposed, powerful attacks.

Recently, adversarial detection has been studied to prevent adversarial attack by a rathersimple method, instead of adversarial defense. Adversarial detection is a technique for judging whether a given image is an adversarial example, and has been recently studied withadvers arial defense. From the perspective of a system with a deep learning model, it is possible to further secure the system by rejecting the adversarial example input, after determining through adversarial detection whether the given input is an adversarial example. Because the advers arial example is a data point located near the decision boundary of the model, rejecting the result for the adversarial example can minimize the propagation of information about the criteria that the model determines the output for a given input to the attacker. This makes it more difficult for an attacker to perform more sophisticated attacks that can deceive the target model with higher probability [30].

Various adversarial detection methods have been proposed by many researchers. Grosse et al. [26] proposed a modification of adversarial retraining to detect the adversarial example. For a classification model with result classes, a new th class corresponding to the advers arial image is added to perform detection, and the model is trained using natural images and adversarial images. If the existing training dataset of the model is , thenew training set of the model is as follows:

(7)

where, denotes the adversarial example generated by applying a specific attack to the modeltrained with . Adversarial retraining showed good detection performance for the MNIST dataset but showed poor detection performance of 70 % detection rate and 40 % falsepositive rate for the CIFAR10 dataset [30].

Metzen et al. [27] proposed a detector that performs detection using the output value of the inner convolution layer of the classification model. Fig. 2 shows that the proposed detector is adeep learning model that consists of convolution layers and max pooling layers. Metzen & rsquo;sdetection method showed high detection accuracy in experiments using the MNIST and CIFAR10 datasets.

Fig. 2. The deep learning structured adversarial detector proposed by Metzen et al. [27].

Several adversarial detection methods have been proposed that use deep neural networks structured detectors, such as the detector proposed by Metzen, and most of them show high detection performance [28][29]. However, such a deep neural network-based detection method has the disadvantage that it requires a large number of adversarial examples to train the detector. From the perspective of a system with a deep learning model, the number of advers arial examples acquired is equal to the number of attacks that the system received. Inother words, the system is able to defend the attack with high performance through a deepneural network-based detector only after obtaining enough information about the attack by receiving a large number of attacks. Therefore, the deep learning-based adversarial detection method cannot be said to effectively prevent the adversarial attack.

2.3 Gaussian Process Regression

Gaussian process is a random process in which every finite collection of random variables has a multivariate normal distribution. Gaussian process regression is a technique to infer the mean and variance of the whole data range based on the observed data, by defining therelationship between the data using the characteristics of the data, assuming the distribution of the data follows the Gaussian process [31][35]-[37]. Assuming for the function of , the log marginal likelihood is as follows:

(8)

where, is a covariance matrix for all possible observed data pairs , calculated from a pre-defined kernel function, and is a hyperparameter of the covariance function. Based on the that maximizes this marginal likelihood, the distribution of the function value for the unobserved data is . That is, the posterior distribution has mean function and variance function , where and arecalculated through the following equations:

(9)

where, denotes the covariance values between all observed data and the new data based on the hyperparameter value , and is the variance value at . Fig. 3 shows the distribution of the function obtained by the Gaussian process regression as mean function and variance function. Consequently, the value of the function for the unobserved data can be predicted through the mean function, and the variance function implies the uncertainty of the function.

Fig. 3. Distribution of functions calculated by Gaussian process regression. The shaded area represents the 95 % confidence interval [31].

Since Gaussian process regression defines prior and predicts posterior in consideration of covariance between data, it is possible to obtain more accurate regression results with only asmall number of data for the data that follow the Gaussian process, than by a general regression method. Our proposed detection method works based on Gaussian process regression, so that it can achieve high detection accuracy with only a small number of advers arial examples.

3. Gaussian Process Regression-based Detector

We define the characteristics of the adversarial detection method as follows to effectively prevent the adversarial attack. First, the adversarial detection method should operate with high detection accuracy with only a small number of adversarial examples. This is because the number of adversarial examples held by the system is equal to the number of attacks received. It is also directly associated with the shortcomings of existing deep learning-based detectors. Second, the adversarial detection method should be non-differentiable, or it must be difficult for the attacker to obtain the gradient of the detector. This is to prevent secondary attacks that

use the gradient of the detector to generate an adversarial example that can fool not only the target model, but also the detector.

In order to satisfy these properties, we propose a method for detecting advers arialexamples based on the Gaussian process regression. First, we extract the intermediate features generated by the pre-trained classification model for natural or adversarial images. The intermediate feature is the output vector of the model's last hidden layer, whose dimension is the class number of the image set. If the natural image set and the adversarial image generated by applying adversarial attack to each xi is , the extracted intermediate featureset is as follows:

(10)

Second, we use the extracted intermediate feature set as the observed data of the Gaussian process regression-based detector. The observed dataset for fitting the Gaussian process regression-based detector is as follows:

(11)

In the application process for the real model, the intermediate feature is extracted through a pre-learned model for a given input image, and the result of the Gaussian process regression is obtained when the extracted value is input.

Fig. 4 shows the structure of our proposed adversarial image detector. The output value of the model’s last hidden layer is the classification probability value for the image. In the case of advers arial images far from the two centers of the image classification boundary, classification probability values for the two classes tend to be similar to each other. The detector would train this information to perform detection.

Fig. 4. Gaussian process regression-based adversarial image detector.

In the Gaussian process regression, the influence between two similar data is defined as covariance. If the dimension of data is high, it is difficult to grasp the pattern of covariance between data. Therefore, inputting low-dimensional high-level features extracted through convolution and pooling layers, rather than a high-dimensional raw image, might perform the Gaussian process regression more efficiently.

4. Experiments

To verify the performance of our proposed Gaussian process regression-based detector, the datasets used in the experiments are MNIST and CIFAR10. Sections 4.1 and 4.2 describe the classification models for datasets that are deep convolutional neural network-structured models, while Section 2 described the attack methods used in the experiments, which are the FGSM, BIM, JSMA, DeepFool, and C&W attacks.

Table 1 shows the accuracy of the classification model for adversarial images, and the average L2 distance of perturbations generated by each attack. According to the definition of the adversarial attack, the smaller the perturbation generated by the attack and the lower the classification accuracy for the target model, the more powerful the attack. As a result of applying the attack against datasets and classification models used in the experiments, the C&W attack produced the least perturbation, and generally showed low classificationaccuracy. Therefore, the C&W attack is the most powerful attack among the five advers arialattacks used in the experiments.

For the Gaussian process regression-based detector, 300 natural images and 300 advers arial examples are used for training. The covariance function used in the proposed detector is the squared exponential function [31], which is as follows:

(12)

The baseline model compared with our detector is the deep convolutional neural network-structured binary classification model proposed by Gong et al. [29], and the training data of the baseline model is set to 300 natural images and 300 adversarial examples for the same experimental conditions.

Table 1. Classification model accuracy for adversarial images and average L2 distance of perturbations.

Table 2. Detection accuracy for the MNIST dataset.

Table 3. Detection accuracy for the CIFAR10 dataset.

4.1 MNIST dataset

The model for classifying the MNIST dataset is a simple 5-layer convolution neural network consisting of two convolution layers, one max pooling layer, and two dense layers. For training, 60,000 of 28×28×1 MNIST images are used, while for validation, 10,000 images are used. The optimizer used for model training is Adadelta [38], the training epoch is 20, learningrate is 0.001, and batch size is 128. As a result of the training, the accuracy of the classification model for the MNIST data is 99.3 %. The hyperparameters of the five attacks are set as follows; for FGSM and BIM, ε is 0.4. For the C&W attack, we set the maximum iterations to 1,000, the initial constant to 0.001, and the learning rate to 0.005.

Table 2 shows the experimental results for the MNIST dataset. For the FGSM and BIMattacks, the detection accuracies were relatively lower than the baseline detection model, but for the DeepFool, JSMA, and C & W attacks, which have higher attack success rates, ourmodel is far superior to the baseline model. Since the detection accuracies of the baselinemodel for the DeepFool and C&W attacks are quite low, we can observe that the baselinemodel, which is a deep neural network, cannot train at all with just a few training images.

4.2 CIFAR10 dataset

The model for classifying the CIFAR10 dataset is the 32-layer ResNet model [8], and 60,000of CIFAR10 images are used for training, while 10,000 images are used for validation. The optimizer used for model training is Adam [39], the training epoch is 120, learning rate is 0.001, and batch size is 128. As a result of training, the accuracy of the CIFAR10 datasetclassification model is 91.41 %. For FGSM and BIM, ε is set to 9/255. Hyperparameters for the other attacks are the same as for the previous MNIST experiment.

Table 3 shows the adversarial detection performances of the baseline model and ourproposed model in experiments using CIFAR dataset. Experimental results show that the Gaussian process regression-based detector shows better detection performance than the baseline, except for the JSMA attack. Due to the small training dataset, the baseline model cannot train at all to detect the BIM, DeepFool, and C&W attacks. Also, for the C & W attack, which is considered the most powerful attack, the proposed method shows higher detectionaccuracy than the baseline model in both MNIST and CIFAR10 datasets.

4.3 Extremely small adversarial examples

Due to the nature of the Gaussian process regression, the proposed detection method canachieve high detection performance with only a small number of adversarial examples. Todemonstrate this, we performed experiments to measure the performance of the detector by reducing the number of adversarial examples used in the detector training. The combinations of the dataset and adversarial attack used in the experiments are the MNIST dataset-FGSMattack and CIFAR10 dataset-JSMA, where the performance of the proposed detector is lowerthan the baseline model in the experiments using 300 adversarial examples in the training dataset. Without changing the model structure or other hyperparameters, we changed the number of adversarial examples used in model training to (300, 200, 100, 50, 30, and 10), and set natural images to the same number as the adversarial examples.

Fig. 5 shows the performance of the proposed detection method and the baseline method according to the number of adversarial examples in the training dataset. When 300 advers arialexamples were used for training, the baseline model performed better than the Gaussian process regression-based detector, but when training with fewer adversarial examples, the performance of our detector was higher than that of the baseline model. In particular, compared to the baseline model, our detector showed less variability in accuracy as the number of adversarial examples in the training dataset decreased. Thus, the Gaussian process regression-based detector can operate at high performance in environments with a small number of adversarial examples.

Fig. 5. Gaussian process regression-based adversarial image detector.

5. Conclusion

In this paper, we proposed the Gaussian process regression-based adversarial detection method. The proposed method first extracts the intermediate feature for a given input image from a pre-trained classification model, and then performs adversarial detection by the Gaussian process regression-based detector that has been trained with the extracted low-dimensional information of images. Since Gaussian process regression expresses the correlation information between data by covariance matrix and performs regression based on this information, it can show high performance with only a small number of observed data.

The experimental result shows that our model demonstrates higher performance than the deep learning-based detection model for a small number of adversarial images. In particular, the proposed detector shows less accuracy variation on the number of adversarial examples in the training dataset than do deep learning-based detection models. Therefore, the Gaussian process regression-based detector can perform detection with high performance in the case of having a small number of adversarial examples, that is, when there is little information about the attack performed by the attacker. In future work, we plan to improve the performance ofour detector by reflecting the characteristics of the adversarial image generated by the FGSMand BIM attacks.