An Intelligent Automatic Early Detection System of Forest Fire Smoke Signatures using Gaussian Mixture Model

Abstract

The most important things for a forest fire detection system are the exact extraction of the smoke from image and being able to clearly distinguish the smoke from those with similar qualities, such as clouds and fog. This research presents an intelligent forest fire detection algorithm via image processing by using the Gaussian Mixture model (GMM), which can be applied to detect smoke at the earliest time possible in a forest. GMMs are usually addressed by making the model adaptive so that its parameters can track changing illuminations and by making the model more complex so that it can represent multimodal backgrounds more accurately for smoke plume segmentation in the forest. Also, in this paper, we suggest a way to classify the smoke plumes via a feature extraction using HSL(Hue, Saturation and Lightness or Luminanace) color space analysis.

1. INTRODUCTION

Recently, environmental problems have been a pressing concern all around the world. As one of them, forest fires deserve special attention due to their effects on forest preservation, their economical and ecological damages, and the fact that they create human suffering [1]. In Korea, the total number of forest fires over the past decade from 2000-2009, is 5,226, and almost 38,000 ha of forests have been destroyed. The estimated damage to forest resources amounts to 25.7 million USD [2].

Prior to popular methods of forest fire detection, data mining (DM), which is also known as Knowledge Discovery in Database (KDD), was used due to the advances of information technology. This led to an exponential growth of business, and to the size of the scientific and engineering areas. Indeed, several DM techniques have been applied to the fire detection domain, such as the neural network (NN) to predict man-made wildfires and the combination of infrared and NN to reduce false forest fire alarms. Recently, an efficient forest fire detection system that is related to the DM method was proposed. It uses fuzzy logic to extract valuable information from spatial data and employs the information with the aid of image processing and artificial intelli- gence techniques [3]. On the other hand, the key factor in disaster surveillance is to detect it at the very beginning to decrease damages and to prevent disasters caused by forest fires. However, traditional human surveillance cannot monitor a wide forest area, so there has been an emphasis on developing automatic detection solutions. A rather new, advanced technical approach to human surveillance is the installation of a remotely controlled video or CCTV cameras at a monitoring spot. These are relatively more efficient and cost-friendly methods [4].

When a forest fire occurs, the most important measure is early detection for preventing the fire from spreading and for decreasing the heavy damages of forest ecosystems. Generally, the smoke signature appears at the beginning, and the fire is seen as time passes. Therefore, the effective approach to minimizing the damage caused by a forest fire is the early detection of the smoke signature or plume.

This paper presents an intelligent forest fire detection algorithm via image processing by using the Gaussian Mixture model (GMM), which can be applied to detect the smoke as early as possible in the forest. The most common solutions involve the changing illumination levels and temporal background clutter, as is often found in outdoor scenes. Simple background subtraction for smoke plume detection has its limits in uncontrolled environments due to there being a lot of noise. GMMs are usually addressed by making the background model adaptive so that its parameters can track changing illumination levels and by making the model more complex so that it can more accurately represent multimodal backgrounds for smoke plume segmentations in the forest. The experimental images in this paper were acquired in real-time from video cameras.

 

2. PRIOR RESEARCHES AND THE STATE OF THE ARTS

Traditionally, there have been three kinds of methods for detecting forest with remote sensing. There is the spectral method, the spatial method, and the temporal method [5]. The spectral method uses simple thresholds for single- or multi-band data to identify fire pixels [5][6]. The spatial method computes the statistical characteristics of the local area, and calculates the average and the standard deviation of pixel values surrounding a pixel. The temporal method uses temperature differences between remotely sensed imageries that have been acquired at different times [5].

On the other hand, Y. Byun et al .[5] proposed a graph-based algorithm for spatial outlier detection, by using the ordinary scatter plot and Moran’s scatter plot. However, the surveillance of the satellite images that are used in the large and homogeneous regions in this approach cannot be used for local fire detection.

Another approach by A. Ollero et al.[7] presents a multiple-sensorial system, in which several information and data sources have been used via integration systems, which include infraredimages, visual images, data from sensors, maps, and models.

Recently, the most popular algorithm used in earlier researches is using the artificial intelligent techniques with spatial data.

Angayarkkani et al. [8] [9] presented an efficient forest fire detection method using fuzzy logic, which is used to extract valuable information from spatial data. They employed the method for locating the regions that are vulnerable to forest fires with the aid of an image processing technique. The image in the forest spatial data with the presence of fires was utilized in training the neural network. However, this approach cannot distinguish between smoke signatures. It can only detect the presence of fires. Therefore, it is difficult to incipiently detect a forest fire.

Armando et al. [10] have studied the automatic recognition of smoke signatures in lidar (light detection and ranging) signals sent from very small-scale experimental forest fires by using neural network algorithms. A lidar system consists of a radiation emitter, receiver optics (usually a Newtonian or Cassegrainian telescope), a photo-detector, and data processing hardware and software. In this research, a neural network is used to identify a smoke plume, which causes the alarm, and the pre-processing algorithm provides the distance to the smoke plume. However, this lidar system can only be installed in special locations because this system consists of complex devices and is very expensive.

For the exact detection of a forest fire, B. Arrue et al. [11] applied new infrared image processing techniques and artificial neural networks (ANNs) using additional information from meteorological sensors and from a geographical information database. They took advantage of the information redundancy from visual and infrared cameras through a marching process, and designed a fuzzy expert rule base to develop a decision function. Because this system aims for combining data from various available multiple sensors together with expert knowledge, the devices are costly for integrating the system.

Grubisic et al. [12] proposed a smoke detection algorithm that combines motion and edge detection, spectrum analyzing, and moving shape analyzing algorithms. All of which are matched together through the image processing from a surveillance camera that is placed at a high position in the forest. This research showed a 100% rate of smoke plume detection, no false alarms, and 15 seconds of maximal detection time. However, the experimental environment of this research cannot be guaranteed because of the limits of fire image in the real world.

 

3. PRINCIPLES OF FOREST FIRE DETECTION USING GAUSSIAN MIXTURE MODEL

This paper is based on the Gaussian Mixture Model (GMM) to detect smoke signatures through image processing. In this study, the algorithm for GMM [13] [14] is applied to successfully find the background image under uncontrolled outdoor conditions.

3.1 Detection of the Smoke Signatures using the Gaussian Mixture Model

3.1.1 Saving the Background Image using a Gaussian Mixture Model

The detection algorithm proposed in this study applies the Gaussian Mixture Model, which has the highest reliability for a background image and object extraction. It also records the background images for a certain period of time.

The automatic detection of some objects, such as a smoke plume, uses a method that saves a background images. Background subtraction is the most commonly used method to extract foreground objects of segmentation in a sequence of frames. The image pixel in the frame sequence, through previous frame follows the Gaussian distribution. Pixel value changes variously in accordance with the frame sequence. This method estimates a background that is renewed by the most approximate distribution value of the background. At this time, the pixel state in the frame sequence is defined as k ∈ {1,2,… ,n}. Some of the values in this case can be the background and the rest can be considered as the foreground. Typically, k has a value of 3 to 7 [13] and this is called multi-modal distribution. The value of the X pixel changes depending on the state of the sequence of image frames. This is known as the pixel process. This pixel process assumes that the probability density function of f has a Gaussian distribution of equation (1),

where, X is the random variable of pixel changes, we have used the symbols πk, Σk which are the mean and the variance of pixel state of k respectively.

An example of the variable distribution of a certain pixel passing the time sequence of frame applying forest fire image is shown in Fig. 1. In Fig. 1, k is a state variable where its pixels are shown in a frame sequence. For example, the probabilities of pixel states changing within 100 frames are expressed as p(k=1), p(k=2), p(k=3). The probability of pixel intensity being in a normal state p(k=1), and the probability of pixel intensity being in a light shadow is p(k=2), and the probability of pixel intensity being in a dark shadow is p(k=3). The sum of the probability is 1 (i.e., p(k=1) + p(k=2) + p(k=3) = 1).

If the pixel process changes are stable (stationary process), it is possible to get the maximum likelihood function (MLF) by using the Expectation Maximum (EM) algorithm. Stauffer and Grimson [14] estimated the parameters via the K-mean approximation.

The k value determines the current state and initiates the updating process based on the corresponding pixel of background distribution when a new image frame is entered. It is computed as Equation (2):

Fig. 1.The variable distributions of a certain pixel passing the time sequence of frame

where, ωk,t=p(k), the probability of the k state at frame t, and , the sum of the probability of the k state at time t is 1. α is the learning rate. Mk,t+1 has the value 0 or 1, if it is matched it is, Mk,t+1 is 1 and 0 if it is not matched.

Also the, μ, Σ, mean and variance are recalculated by Equations (3) and (4) respectively, when a new frame is processed. When a match, ωk,t+1 equals to ωk,t, the probability of the current state of the new frame (t+1) is same as the prior state at time t. On the other hand, if it does not match, the current state of the new frame has been updated as much as the learning rate, α.

where

Fig. 2.shows the process of background estimation with GMM.

3.1.2 Detection of the Smoke Plumes from Background Sequences

This idea stems from the automatic incident detection system [15]. At the scene of a minor or major accident, the accident vehicle is also saved as the background image when using the Gaussian Mixture Model because all of the accident vehicles are stopped. To reduce the false alarms, it should be recognized that there will be cases of stopped objects.

For the early detection of a smoke signature using the Gaussian Background Model, the background of images with and without smoke plumes are saved as the background image sequences, as represented in Equation (5). The sequence of the background image (X, Y) is made at the n second interval and is saved into the buffer, and then it can detect an accident in the area where the higher value is greater than the critical value.

where, Bk(0≤k≤N) is the background image sequences stored in buffer, I is the intensity of pixels, and (X,Y) is the image size.

In this research, a 60 second interval was set to store the background image. In this equation, (k)−(k−1) = n (second), B1 is the background image that was stored in the 60 second buffer later and B2 is the background image 120 seconds later and so on. The stored background sequences can be represented as {B0, B1, B2,…BN}. Non-fixed objects, such as sudden, smallscale changes in illumination due to shifting clouds, or a temporal background clutter caused by local fog, were removed from fixed objects, such as rocks and trees. They were then stored into the background a certain period of time later when a forest fire has taken place. For the exact detection of a smoke plume, the difference of each background image was obtained at regular time intervals. This approach is superior to simple background subtraction because there are being insensitive image noise under various outdoor environments. Fig. 3. depicts background image sequences, including the normal smoke plume image, and the subtracted image for extracting the smoke plumes in this research.

Fig. 3.Background image sequences, including the normal smoke plume image, and the subtracted image for extracting the smoke plumes

3.1.3 Feature Extraction of a Smoke Plume using HSL Color Space Analysis

A color space is defined as a means by which the specification, creation, and visualization of colors is performed. The majority of image processing with computer color produces colors that are based on the varied combinations of RGB phosphor emissions, which are required to form a color [8]. However, the processing of RGB color space cannot always discriminate between smoke, fog, and clouds, which results in a false alarm. Therefore, we have carried out the color conversion into Hue, Saturation, and Lightness or Luminance (HSL). HSL takes three values. The H (Hue) component describes the color itself in the form of an angle between [0,360] degrees. 0 degrees means red, 120 means green, and 240 means blue. 60 degrees is yellow and 300 degrees is magenta. These Numbers reflect different shades. Saturation is a percentage value; 100% is the full color. Lightness is also a percentage; 0% is dark (black), 100% is light (white), and 50% is the average.

Fig. 4.Classifying smoke, fog, and clouds according to the degree of saturation and lightness (S and L mean Saturation and Luminance, respectively)

In this research, we suggest a way to classify smoke plumes by using the S and L channel of HSL color space. The range of the S and L channel, 0-100%, was proportionally rescaled to 0-255 degree in order to distinguish between each feature. As depicted in Fig. 4., the saturation and lightness show the distribution of 0-50, and 40-140 degrees, respectively. On the other hand, fog has a saturation range of 0-80, which is similar to smoke, but it has a lightness range of 140-200, which shows a difference from smoke. Most fluffy clouds relatively show 120-250 degrees of saturation and a high value of 180-250 degrees of lightness.

 

4. EXPERIMENTAL RESULTS AND ANALYSIS

We have conducted the experiments with five test images that were collected from the hills near Wonju City, South Korea.

Due to the difficulty of obtaining images of an early smoke signature or plume of a forest fire, the test images used in this paper are not of real forest fires. Rather, they are recordings of smoke in areas that are adjacent to the hills. These 10-15-minute video recordings were then applied to the GMM algorithm in order to roughly detect the smoke plumes within image sequences and to verify real forest fire plumes with HSL color space.

The program was developed using LabVIEW 8.5 (graphic language) and the VISION module library. In this research, 30 fps images at a resolution of 640*480 were acquired from video cameras. These images were then converted into digital images using the frame-grabber board for image processing. An algorithm was implemented from the hardware system with Quad Core CPU 2.4 GHz, 2GB RAM, and our system measured smoke signatures with detailed detection in real time for 18–20 fps.

Fig. 5. contains five cases of test video that show the extraction of smoke plumes through the GMM and the distribution of saturation and luminance. In the case of the smoke plumes, all of the test images show the saturation and luminance at a distribution of 0-50, 40-140 degrees, respectively

In this research, the background image, which is obtained by the GMM algorithm, was stored at 60-second intervals. These background sequences were stored at one frame per minute for 30 minutes.

In the saturation and lightness of HSL color space, test image (1) had the distributions of the saturation as 0-60 degrees and the lightness as 40-150 degrees. Test image (2) had the distributions of the saturation as 0-75 degrees and lightness as 40-120 degrees. Test image (3) had the distributions of the saturation and lightness as approximately 0-75, 20-140 degrees, respectively. Test image (4) had the distributions of the saturation and lightness as approximately 10-65, 50- 110 degrees, respectively. Whereas, test image (5) could almost detect the thin plume of smoke. It had the narrow distribution of 10-25, 90-120 degrees, respectively.

For all of these cases, the distributions of saturation included the range of 0-60 degrees over the 90% of the plots, and lightness included the range of 40-140 degrees over the 95% of the plots. Also, the system detected all of the test videos and revealed detection alarm times of 95 seconds and 110 seconds. The detection alarm time was estimated by counting how long it took for the system to recognize the smoke plumes. It includes the initial prediction by GMM and the second judgment that was based on HSL analysis processing displayed a warning message. The best system had the shortest possible time and had a minimal amount of false alarms.

Fig. 5.The experimental results of the saturation and lightness channel. Detecting smoke plumes and verifying the real forest fire plumes with HSL color space.

 

5. CONCLUSION

This paper focuses on the early detection of forest fires to prevent their spread and to decrease the heavy amount of damages inflicted on forest ecosystems. For detecting forest fires, especially for detecting smoke plumes as early as possible, the most important things are as follows: the exact extraction of the smoke from an image, and distinguishing the smoke from those with similar qualities, such as clouds and fog, and to be able to do without confusion, in order to minimize there being any false alarms.

This paper presented a scheme for a single-sensorial system, which consists only of video images, for the more efficient, cost-friendly monitoring of a small-scale area.

The first step that we proposed for our smoke detection algorithm applies the Gaussian Mixture Model. The GMM has the highest reliability of background image and object extraction as it records the background images for a certain period of time.

Secondly, we suggested a way to classify smoke plumes by using the distribution of the saturation and lightness of the HSL color space. Fog has a lightness range that is higher than smoke, and the saturation and lightness value of a fluffy cloud is relatively higher than for smoke.

Although it is unfortunate that the experiment could not be conducted with a real forest fire, the close proximity of the smoke signature in the test videos on real forest fires assures its validity. Furthermore, the detection of thin smoke, which occurs early on in a forest fire, demonstrates the significance of this paper. However, the disadvantage of this approach is that for detecting smoke plumes it can only be applied during the daytime. In future bodies of work, we must improve on this detection system via infrared imagery so that it can be applied to various environments, such as at night or during hostile weather.