Smoke Image Recognition Method Based on the optimization of SVM parameters with Improved Fruit Fly Algorithm

Abstract

The traditional method of smoke image recognition has low accuracy. For this reason, we proposed an algorithm based on the good group of IMFOA which is GMFOA to optimize the parameters of SVM. Firstly, we divide the motion region by combining the three-frame difference algorithm and the ViBe algorithm. Then, we divide it into several parts and extract the histogram of oriented gradient and volume local binary patterns of each part. Finally, we use the GMFOA to optimize the parameters of SVM and multiple kernel learning algorithms to Classify smoke images. The experimental results show that the classification ability of our method is better than other methods, and it can better adapt to the complex environmental conditions.

1. Introduction

Fire is a great threat to people's property safety and life safety. At present, various types of fire detectors on the market mainly include thermal sensing detector, light sensing detector, smoke sensing detector. These traditional detectors are cheap and accurate, but some deficiencies are difficult to solve. For space above a certain height or outdoor places where fires frequently occur, such detectors are no longer suitable. Smoke sensing detectors are usually used in some office buildings or factories. As time goes by, some dust and other particles will enter the air to cause corrosion to those detectors. At this time when a real fire occurs, those detectors can't alarm at all. To prevent this from happening, those detectors need to be regularly checked and repaired which will also bring extra manpower and material consumption. Therefore, most researchers attempt to combine some coding [1][2][3], classification [4], recognition [5], feature fusion model[6][7], and some other image methods[8][9][10] to design efficient smoke image detection methods.

Russo uses the approximate median filtering algorithm to subtracts the background region from the input frame and the shape-based filtering algorithm to find the motion region[11]. This method has a high smoke image recognition rate. Murat TOPTAŞ uses the YUV color space to detect smoke areas. After that, the gray level co-occurrence matrix (GLCM) is used to extract the features of the smoke image and the support vector machine (SVM) method is used to classify it [12]. However, the YUV color space of the smoke image is not obvious, so the recognition ability of this method is not good. Mengtao Huang takes the main direction of smoke movement as the dynamic features of indoor low-light smoke images and selects the texture features of the smoke images as the static features of indoor low-light smoke images to recognize the smoke images. It has good anti-interference ability and has good performance for smoke image recognition under low-light indoor conditions [13]. Lin Wang extracts the color features of the suspicious region from the RGB color space and the HSI color space [14] of the early smoke images. Then Lin Wang uses a two-dimensional discrete wavelet transform to extract background blur features and calculates a ratio of the number of pixels in the suspected smoke region to the number of pixels in the corresponding minimum enclosing rectangle to extract the contour irregularity features. Lin Wang uses the optical flow method to extract the features of the main direction of the smoke movement. This method can effectively improve the smoke image recognition rate, but the use of the optical flow method for the main direction of the smoke movement will result in low algorithm efficiency and cannot achieve real-time monitoring on smoke images detecting[15]. At present, the way of how to extract smoke images features, and the way of how to select SVM parameters based on the data model becomes the key issue. This paper intends to use the good group of improved fruit fly optimization algorithm to find the optimal kernel parameters and penalty terms of SVM, and combine the smoke images features to train SVM to recognize the smoke images under complex conditions.

2. Related Work

2.1 Three-frame difference algorithm

The three-frame difference algorithm is an extension of the inter-frame difference algorithm. It is an algorithm for obtaining motion regions by performing two-two different operations on successive three frames of video image sequences. When abnormal target motion occurs in the monitoring camera, there will be a more obvious difference between the adjacent two frames. Then we subtract the two frames to get the absolute value of the difference in pixel values at the corresponding position and determine whether it is greater than a certain threshold, to extract the motion region. Fig. 1 shows the steps.

Fig. 1. Three-frame difference algorithm

2.2 ViBe algorithm

The ViBe algorithm is a motion region recognition algorithm based on background updates. The principle is to create a set of K sample pixel values for each pixel in the current area. If the number of the Euclidean distance between the new pixel and all K pixel values in the sample set is less than a threshold T₁ is greater than another threshold T₂, then the new pixel point is considered to be a background point, otherwise, it is a pixel in the motion region. Fig. 2 shows the steps, where D is the radius and the current pixel V(x, y) is the center of the circle [16].

Fig. 2. ViBe algorithm

2.3 Gaussian mixture model for background modeling

The motion region detection method based on Gaussian mixture model is robust to the dynamic change of the scene. The basic idea is to define the distribution model of each pixel as a set consisting of multiple single Gaussian models. Then update the model parameters based on each new pixel value, according to some criteria, the pixel is determined as the background point or the pixel in the motion region [17].

2.4 IMFOA

IMFOA is proposed by Xuzhi Miao. The algorithm is a global optimization algorithm based on the process of fruit flies looking for food, which has the advantages of a few parameters and higher optimization accuracy [18].

Initialize the fruit fly population. The number of fruit fly groups is N, The number of iterations is G_max, randomly given group location:

X_-axis, Y_-axis#       (1)

Randomly give the direction and distance of the fruit fly's optimization, Rand() is defined as the optimization distance:

X_i = X_-axis + Rand()#       (2)

Y_i = Y_-axis + Rand()#       (3)

The source of the taste of the food is unknown. Dist_i is defined as the distance from fruit fly individuals to the origin. S_i is defined as the judgment value of the taste concentration of fruit flies.

Dist_i = #(4)

S_i = \(\frac{1}{\text { Dist }_{i}}\)#(5)

The taste concentration judgment value S_i is substituted into the taste concentration judgment function, and the taste concentration of the fruit fly individuals is defined as Smell_i·(Function() is the taste concentration judgment function, also called the fitness functions)

Smell_i = Function(S_i)#       (6)

Find the fruit fly with the best taste concentration in the fruit fly population.

[bestSmell bestindex] = max(Smell_i)#       (7)

Based on the obtained Smell_i and bestSmell and the parameter a, where a is a random number with value (0,1). The fruit fly population is divided into two groups: good group and bad group. The formula is as follows:

a_i = \(\frac{\text { Smell }_{i}}{\text { best Smell }}\) ≥ a#       (8)

Fruit fly individuals that meet the above formula are assigned to the good group, and the remaining individuals are assigned to the bad group.

Record and preserve the best taste concentration bestSmell and its X, Y coordinates, and other fruit fly individuals in the population use visual flight to this position.

Smellbest = bestSmell#       (9)

X_-axis = X(bestindex)#       (10)

Y_-axis = Y(bestindex)#       (11)

Finally, start iterative optimization and repeat steps 2) to 6), and when the taste concentration is no longer better than the previous taste concentration or when the number of repetitions reaches the maximum number of iterations, the cycle is stopped.

2.5 Histogram of Gradients features

The Histogram of Gradients features is also called HOG, it is based on the gradient direction histogram of the local region to form the feature, which was first proposed by Navneet Dalal for human body posture recognition [19].

2.6 Volume Local Binary Patterns

The Volume Local Binary Patterns is an extension of the local binary mode operator that is widely used in static texture analysis. The texture features of the image are analyzed by combining the motion and shape of the image. It was proposed by G. Zhao. and they also proposed a rotation-invariant Volume Local Binary Patterns [20].

3. Smoke recognition algorithm based on GMFOA optimized SVM

3.1 System framework

The system framework is shown in Fig. 3 below. The smoke video is obtained from the cameras, and then we use the mixed motion region extraction algorithm to extract the motion region of smoke video. After that, we extend the boundary of the motion region to an external rectangular region and divide the rectangular region into several regions of the same size, then we extract the HOG features and VLBP features of the several regions. Finally, we use GMFOA to optimize the parameters of the SVM and the synthetic kernel method of multiple kernel learning to classify the fused features.

Fig. 3. The system framework

3.2 Mixed motion region extraction algorithm

Because the speed of smoke's movement is slow, it is difficult to extract a complete motion region of smoke video by using a single motion region extraction algorithm. Therefore, we compare the three-frame difference algorithm, ViBe algorithm, and Gaussian mixture model for background modeling to find the best combination.

The three-frame difference algorithm is an extension of the interframe difference method, and it is a method for obtaining a motion region by performing a pairwise difference operation on successive three frames of a video image sequence. Set the grayscale image of the current frame to be r_k, the grayscale image of the previous frame adjacent to it is r_k−1, the gray image of the next frame is r_k+1, then we make a difference between the pixel value of r_k−1 and the corresponding pixel value of r_k to get a motion image S_k and a difference between the pixel value of r_k and the corresponding pixel value of r_k+1 to get the motion region image S_k+1, we define the final motion region image as F_k = S_k ∩ S_k+1. Finally, we form an external rectangular region based on the boundary extension of the motion region and divide this region into multiple motion regions of the same size.

F_k = {F_k1, F_k2, F_k3, ⋯ }#       (12)

The ViBe algorithm is a motion region extraction algorithm based on background updates. We define a region centered on pixel (x, y) and D as the radius, v(x, y) is the pixel's value. We make a sample set L(x, y) = {v₁(x, y), v₂(x, y), … , v_N(x, y)} of size N = 20 to store the background point (x, y) of the previous frame and a new set M(v(x, y)) = {v₁(x, y), v₂(x, y), … , v_N(x, y)} of size N = 20 to store the background point (x, y) of the next frame. Then we calculate the distance between each pixel in the new set and each pixel in the sample set, if the distance between those two pixels is greater than the threshold T₃, the pixel (x, y) which belongs to the new set is considered to be the motion pixel, otherwise the background pixel. We can get the motion region image V_k by this algorithm. Finally, we form an external rectangular region based on the boundary extension of the motion region and divide this region into multiple motion regions of the same size.

V_k = {V_k1, V_k2, V_k3, ⋯ }#       (13)

The Gaussian mixture model for background modeling is used to extract the motion region of smoke video. We define the RGB image of the current frame of smoke video is I_k, and its motion region which is extracted by the Gaussian mixture model for background modeling is G_k. Then we form an external rectangular region based on the boundary extension of the motion region and divide this region into multiple motion regions of the same size.

G_k = {G_k1, G_k2, G_k3, ⋯ }#       (14)

To select the best-mixed motion region extraction algorithm, we use four combinations of FG₁ = F_k ∩ G_k, FG₂ = F_k ∪ G_k, VG₃ = V_k ∩ G_k, and VG₄ = V_k ∪ G_k to extract the motion region of the same smoke video. The source of the smoke video is (http://staff.ustc.edu.cn/~yfn/vsd.html), and the experimental results are shown below.

It can be seen from Fig. 4 that FG₁ and VG₃ can’t completely extract the overall shape of the smoke, and the motion region extracted by FG₂ has more background noises. The extracted motion region obtained by VG₄ is more conducive to feature extraction of smoke images. Therefore, this paper uses VG₄ methods to extract the motion region of the smoke video.

Fig. 4. Mixed motion region extraction algorithm

3.3 Feature extraction

3.3.1 HOG feature extraction

The edges in the image contain abundant information. The HOG can effectively distinguish the difference between objects by counting the edge directions of different objects. Sobel operator is one of the most important operators in pixel image edge recognition, it combines Gaussian smoothing and differential derivation to calculate the approximate gradient of image gray function. The corresponding gradient vector or normal vector can be obtained by using the Sobel operator for each pixel in the image. The kernel of the Sobel operator is:

S_x = \(\left[\begin{array}{lll} -1 & 0 & 1 \\ -2 & 0 & 2 \\ -1 & 0 & 1 \end{array}\right]\), S_y = \(\left[\begin{array}{ccc} 1 & 2 & 1 \\ 0 & 0 & 0 \\ -1 & -2 & -1 \end{array}\right]\)

If I is the original image, G_x = S_{x *} I is the image detected by the lateral edge, and  is Gy = Sy _* I the image detected by the longitudinal edge. It is easy to know that the gradient of the image is [G_x, G_y]^T. Therefore, the gradient vector of each pixel is [G_x(x, y), G_y(x, y)]^T and so we can infer its polar form is [m(x, y), θ(x, y)]^T. Then we can easily get the amplitude 𝑚𝑚 and the angle θ of the gradient. The amplitude m is defined as follows:

m(x, y) = \(\sqrt{G_{x}(x, y)^{2}+G_{y}(x, y)^{2}}\)#       (15)

The angle θ is defined as follows:

θ(x, y) = arctan \(\left(\frac{G_{x}(x, y)}{G_{y}(x, y)}\right)\)#       (16)

Divide 360 degrees into 12 bins, each bin contains 30 degrees, and the entire histogram contains 12 dimensions which are also the 12 bins. Then according to the gradient direction of each pixel, add its amplitude to the histogram by using bilinear interpolation.

3.3.2 VLBP feature extraction

The basic LBP operator proposed by Ojala [21] for texture analysis can be defined as follows:

LBP_P,R = \(\sum \begin{array}{l} P-1 \\ P=0 \end{array}\)s(g_p - g_c)2^p, s(x) = \(\left\{\begin{array}{l} 1, x \geq 0 \\ 0, x<0 \end{array}\right.\)#       (17)

g_c is the gray value of the center pixel, g_p is the gray value of the nearby pixel, R is the radius, and P is the total number of sampling points in the circular neighborhood. Fig. 5 is the basic calculation process for an LBP with P = 8, R = 1. For any other value (P, R), if a point on the circle is not in the image coordinates, we use his interpolation point to estimate the gray value of the neighbor at the center of the pixel by interpolation.

Fig. 5. The calculation process of LBP

The definition of VLBP is as follows:

VLBP_{L, P, R} = \(\sum_{P=0}^{3 P+2}\)s(g_p - g_c)2^p#       (18)

g_c corresponds to the gray value of the center pixel of the local neighborhood, g_p represents the gray value of 3P+2 pixels in the three frames having the interval L, and R is the radius of the circle in each frame image. Fig. 6 shows the calculation process of VLBP_1.4.1.

Fig. 6. The calculation process of VLBP_1.4.1

We divide the recognition window into 16×16 small cells and compare the gray value of 14 pixels in adjacent consecutive 3 frames with each pixel in each cell. This will produce a 14-bit binary number, which is the LBP value of the center pixel of the window. Then we calculate the histogram of each cell and normalize it. Then we connect the statistical histogram of each cell into a feature vector which is the VLBP texture feature vector of the entire image.

4. Optimization of SVM parameters based on GMVOA

4.1 SVM decision function and kernel function

When we use the RBF kernel as the kernel of the SVM, the kernel parameter σ and the penalty term c are the main influencing factors affecting the classification accuracy of the SVM. The change of the parameter σ can affect the mapping function and change the distribution of the sample in the high-dimensional feature space, meanwhile, when the parameter σ constructs the optimal classification hyperplane in the high-dimensional feature space will directly affect the performance of the SVM. Therefore, the parameter σ has a great influence on the classification accuracy of SVM. The error penalty factor c is defined as the tolerance of the SVM for errors in the classification. With the increase of c, the classification accuracy of the training sample data set may be too high, which will lead to the low classification accuracy of the SVM on the test sample data set, and the generalization ability of the SVM becomes correspondingly low. On the contrary, with the decrease of c, the classification accuracy of the test sample data set may not reach the expectation, which will lead to the reduction of SVM classification ability. We can conclude that the prerequisite for ensuring the optimal classification result of SVM is to calculate the optimal SVM parameters [22][23].

4.2 SVM parameter optimization method based on GMFOA

In IMFOA, iterative step Q_g of the good group in step (8) is:

Q_g =\(\frac{\left(Q_{\max }+Q_{\min }\right)-\frac{\left(Q_{\max }-Q_{\min }\right)(0-1)}{O_{\max }-1}}{2}\)#       (19)

Q_max is the maximum iteration step size, Q_min is the minimum iteration step size, Q_max is the maximum number of iterations, and O is the current number of iterations. The iterative step Qb of the bad group is:

Qb = \(\frac{\left(Q_{\max }+Q_{\min }\right)+\frac{\left(Q_{\max }-Q_{\min }\right)(O-1)}{O_{\max }-1}}{2}\)#       (20)

At the beginning of the algorithm, the moving step is (Q_max + Q_min) /2. After the first iteration is completed, all flies are divided into two groups based on the clustering judgment formula (8). Then the two groups iterated until the number of iterations reached the maximum. After each iteration, all flies are regrouped. It can be seen from the formula (19) and the formula (20) that the search step size of the good group changes from large to small as the number of iterations increases, the search space becomes smaller, and the global optimization ability of the algorithm becomes weaker, the local optimization ability is enhanced to avoid missing the local optimal solution. On the contrary, the search step size of the bad group changes from small to large as the number of iterations increases, and the global optimization ability of the algorithm is enhanced, which can quickly search for the vicinity of the global optimal solution and avoid the algorithm falling into the local optimal solution. After each iteration, all flies are rearranged and then the two groups are optimized in each iteration step.

In the parameter optimization experiment of SVM in this paper, we use the IMFOA and discarding the iterative steps of the bad group to obtain a global or local optimal solution. In IMFOA, the advantages of the good group's global optimization ability and the advantage of the bad group's local optimization ability are weakened at the same time, which leads to the mediocrity of the good group and the bad group. And if the difference between the search step and the number of iterations increases from small to large, the search step can be simplified, and the overall optimization ability of the algorithm can be ensured.

Algorithm 1 SVM classifier based on GMFOA

N = Number of fruit fly population, X_{_axis}: The X-axis coordinate of the fruit fly, Y_{_axis} :

Y-axis coordinate of the fruit fly, Dist_i : The length between fruit fly and origin,  S_i : 

Reciprocal of Dist_i

for gen = 1 : maxgen 

for i = 1 : sizepop 

if R(i) >= a

X(i) = X_axis +2Q_g rand = Q_g

Y(i) = Y_axis + 2Q_g rand = Q_g 

 end 

S(i) = 1 / D(i); 

 smell(i) = Function(Si)

R(i) = smell(i)/bestsmell; 

[bestsmell bestindex] = max(smell);

if smell(i) < bestsmell

bestsmell = smell(i)

 Smellbest = bestsmell

X_axis = X(bestindex)

Y_axis = Y(bestindex)

end

4.3 GMFOA performance test

To verify the classification ability of GMFOA. We use three optimization algorithms, GMFOA, IMFOA, and FOA to test three fitness functions. Table 1 is the specific formula of the fitness functions given, and Table 2 is the mean and standard deviation of the fitness functions obtained under the optimization of the three algorithms.

Table 1. Fitness functions

Table 2. Experimental comparison of different fitness functions

It can be seen from Table 2 that among the three fitness functions, the accuracy of the mean and standard deviation of GMFOA is significantly better than the other two algorithms, and GMFOA has better convergence speed and optimization accuracy than the IMFOA algorithm.

4.4 Multiple kernel learning method based on GMVOA optimized SV

MIt is not ideal to use the traditional SVM method for classifying smoke images, because the features of samples from multiple sources have different characteristics. The studies show that using multiple kernel learning can improve the interpretability and performance of decision functions. When we use multiple kernel learning methods to fuse features, we are essentially combining several basic kernel matrices corresponding to a single feature. We can implement heterogeneous data fusion on the multiple kernel matrix synthesized based on multiple basic kernel matrices and use it to train the classifier.

The multiple kernel learning method [24][25] has different classification methods based on different classification criteria. According to the construction methods and characteristics of multiple kernel functions, multiple kernel learning methods can be roughly divided into three categories: synthetic kernel learning method, multiscale kernel learning method, and infinite kernel learning method. The synthetic kernel learning method is to combine multiple kernel functions with different characteristics, so that we can get a multiple kernel function containing the characteristics of each kernel function. The multiple kernel learning method is more accurate and Stronger mapping capabilities than the traditional SVM method, in practical applications, the advantages of multiple kernel learning will be very obvious for the classification and regression of sample data with complex distribution structures. The synthetic multiple kernel learning method is shown in Fig. 7.

Fig. 7. Synthetic multiple kernel learning method

In this paper, we compared the recognition rate of the four methods of Method₁ = VLBP/RBF, HOG/RBF, Method₂ = VLBP/RBF, HOG/Sigmoid, Method₃ = VLBP/Sigmoid, HOG/Sigmoid, Method₄ = VLBP/RBF to the smoke video of two different scenesof forest and town. Table 3 is the recognition rate of the different methods.

Table 3. the recognition rate of the different methods

We can see from Table 3, the recognition rate of the Method₁ is higher than other methods, and the recognition rate of Method₁, Method2, Method3 is higher than the Method4. This shows that GMFOA for SVM parameter optimization is helpful to improve the recognition rate for the smoke image, and the recognition rate of multiple kernel learning method is higher than the traditional SVM method.

5. Analysis of results

To verify the effectiveness of the algorithm we proposed in this paper, we pick two interference videos and four smoke videos, as shown in Fig. 8 and Fig. 9 The video frame rate used in the experiment was 25 fps and the video size was 320×240. The algorithm is based on the Windows 10 operating system. The computer is configured as Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz, 8.00 GB memory, and the graphics card is NVIDIA GeForce GTX 1050 Ti.

The experimental results are evaluated by true positive (TP), false positive (FP), true negative (TN), and false negative (FN) [26]. The calculation formula of the recognition rate is TP/(TP+FN), which means in all smoke videos, the proportion of frames where smoke appears among all the frames recognized as smoke. The recognition rate represents the probability that the system correctly recognizes smoke images. The calculation formula of the false alert rate is FN/(TN+FN), which means in all interference videos, the number of frames where non-smoke appears is mistakenly recognized as the proportion of the number of frames where smoke appears in the total number of non-smoke frames. The false alert rate represents the probability of system failure.

Fig. 8. Two interference videos

Fig. 9. Four smoke videos

In these 6 videos, vid3 and vid4 are interference videos, and vid5 is a smoke video in the forest which the ignition point is far away from cameras, vid6 is a smoke video in the forest which the ignition point is closer to the cameras, vid7 is a smoke video in the forest with interference factors (smoke and clouds mixed), vid8 is a smoke video in town.

We use the three methods of GMFOA-SVM (Method₁), IMFOA-SVM and TsPSO-SVM for the experiment [27], Table 4 is the false alert rate of those three methods for interference videos, Table 5 is the recognition rate of those three methods for smoke videos.

Table 4. Interference videos false alert rate of three methods

Table 5. Smoke videos recognition rate of three methods

It can be seen from Table 4 that the false alert rate of the GMFOA-SVM method is slightly lower than the IMFOA-SVM method and the TsPSO-SVM method. Because the smoke near the camera has obvious HOG features and VLBP features. It can be seen from Table 5 that the recognition rate of the GMFOA-SVM method for Vid 6 is higher than Vid 5. When smoke and clouds are mixed, the recognition rate of forest smoke by those three methods is generally low, which indicates that the HOG features and VLBP features of smoke are not ideal for classification. We can conclude the experiment that the GMFOA algorithm has fewer parameters which is better than the TsPSO algorithm when optimizing SVM parameters. The GMFOA-SVM method has the highest recognition rate of smoke images in those three methods.

6. Conclusion

In this paper, we use the good group of improved fruit fly optimization algorithm to optimize the parameter of SVM, then, we detect the smoke image based on the obtained smoke image features. When we optimized the parameters of SVM, by improving the fruit fly optimization algorithm, the recognition accuracy of smoke images was improved. The algorithm proposed in this paper has important practical significance for the research and application of fire smoke monitoring based on video and pictures. However, the algorithm proposed in this paper is slightly inadequate in real-time monitoring, and there will be a high rate of misjudgment for the places where the interference factors such as smoke and non-smoke exist at the same time. When under the interference of strong light and strong wind, the recognition rate of the algorithm we proposed in this paper will be further reduced, which still needs further improvement.