Evaluating the Contribution of Spectral Features to Image Classification Using Class Separability

Abstract

Image classification needs the spectral similarity comparison between spectral features of each pixel and the representative spectral features of each class. The spectral similarity is obtained by computing the spectral feature vector distance between the pixel and the class. Each spectral feature contributes differently in the image classification depending on the class separability of the spectral feature, which is computed using a suitable vector distance measure such as the Bhattacharyya distance. We propose a method to determine the weight value of each spectral feature in the computation of feature vector distance for the similarity measurement. The weight value is determined by the ratio between each feature separability value to the total separability values of all the spectral features. We created ten spectral features consisting of seven bands of Landsat-8 OLI image and three indices, NDVI, NDWI and NDBI. For three experimental test sites, we obtained the overall accuracies between 95.0% and 97.5% and the kappa coefficients between 90.43% and 94.47%.

1. Introduction

Imaging sensors provides various type of images depending on the sensor characteristic. The various type of images are often used together to improve the reliability of image fusion and recognition techniques such as the fusion of CCD image and Infrared image for improvement of pedestrian detection performance (Son et al., 2017), the fusion of electro-optical image and infrared image for dual-mode seeker missile (Jung et al., 2017) and the fusion of electro-optical image and Synthetic Aperture Radar image for urban area classification (Ye, 2012). Imaging sensors of satellites such as Landsat or KOMPSAT (KOrean MultiPurpose SATellite) provide several types of images including RGB and near-infrared (NIR) bands. Image classification is one of the most important technique using the various types of images simultaneously to obtain the information of the scene. Many research studies on image classification have carried out in various application area (Magpantay et al., 2019; Jeon and Kim, 2018; Yoo et al., 2017). Image classification is a process to classify pixels into several groups with specific attributes. The specific attributes of the earth surface are often detected by optical or microwave sensors. The multispectral bands of optical sensor such as Landsat-8 operational land imager (OLI) are basic sources to distinguish the attributes like vegetation, water, buildings and so on. Each multispectral band has its own spectral property depending on the specific attribute of the earth surface. For example, water has low reflectance property which results in the pixels of dark intensity in the NIR band. Index image such as normalized difference vegetation index (NDVI) (Rouse et al., 1974) has the property that pixels of vigorous vegetation have bright intensity in the NDVI image. Normalized difference water index (NDWI) (Mcfeeters, 1996) is also a well-known index image used to detect water body, which has relatively bright intensity in the NDWI image.

The index images, however, shows sometimes similar values for other different attributes. For example, grasslands and built-up areas in NDVI and NDWI images were not discriminated from each other (Szabo, 2016). The attributes of the earth surface which have similar spectral property can be distinct from each other by using an additional spectral band or index image. The accuracy of image classification were improved by simply inserting three index images into the Landsat-8 band images (Park et al., 2018). It is common that all features are used in the conventional image classification methods. However, if many features are created by computing spectral or texture property and also are available together in image classification, we need a feature selection method to overcome the problem of efficiency by selecting some relevant features among the features. Nussbaum and Menz (2008) proposed the separability of two classes in the feature to select the most relevant feature to discriminate two classes. The effect of the selected feature on other classes, however, is not considered in their study. In this paper, we analyze the feature selection based on the separability of two classes and then develop a feature selection method using the separability of all class pairs. We propose a method to select weight values of spectral feature images for multiple classes, which is based on our previous research on the feature weight analysis (Ye, 2019).

2. Methodology

It is common that multiple features are used in image classification. As a feature for the image classification, multi-spectral information such as red, blue and green bands is used to classify the image into multiple classes. The multi-spectral bands produce index images such as NDVI, NDWI and normalized difference built-up index (NDBI), which are given as follows:

\(N D V I=\frac{\left(B_{N I R}-B_{R E D}\right)}{\left(B_{N I R}+B_{R E D}\right)}\)       (1)

\(N D V I=\frac{\left(B_{N I R}-B_{G R E E N}\right)}{\left(B_{N I R}+B_{G R E E N}\right)}\)       (2)

\(N D V I=\frac{\left(B_{S W I R 1}-B_{N I R}\right)}{\left(B_{S W I R 1}+B_{N I R}\right)}\)       (3)

where B represents normally the reflectance of each band. In this study, we use pixel values of each band instead of reflectance because we do not need consider cosine effect due to image acquisition difference and also each index image is rescaled to the range (0, 255) by the min-max normalization.

The NDVI, NDWI and NDBI are useful features to detect vegetation, water and built-up area, respectively. These index images were used as features together with the original Landsat-8 OLI image to classify Landsat-8 OLI images (Park et al., 2018). We can create spectral features from original band images and also need sometimes to select a part of features or reduce the dimensionality of the features. Fig. 1 shows the overall procedure for the proposed method. After creating index images from Landsat-8 OLI images, we compute the separability of each class pair using the index images together with the seven Landsat-8 band images. The weight value of each feature for image classification is determined by the separability of each class pair. Finally, we classify pixels based on the feature vector distance. In the following sessions, feature selection by the Separability of Two Classes (STC method) and feature selection by feature Weighting based on Separability (FWS) will be explained in detail.

Fig. 1. The flow chart of proposed image classification method.

1) Feature Selection by the Separability of Two Classes (STC)

The feature selection is a process to find a subset of features in order to reduce the dimensionality of the features for various purpose. The relevant features which have good separability of two classes are determined by measuring the separability using a suitable distance measure such as the Bhattacharyya distance given as follows:

\(B=\frac{1}{8}\left(m_{i}-m_{j}\right)^{2} \frac{2}{\sigma_{i}^{2}+\sigma_{j}^{2}}+\frac{1}{2} \ln \left(\frac{\sigma_{i}^{2}+\sigma_{j}^{2}}{2 \sigma_{i} \sigma_{j}}\right)\)       (4)

where (m_i, m_j) and (σ_i, σ_j) are the mean and the variance, respectively, for two classes C_i and C_j. Given a feature image v, v=1,…,V, a measure \(S^\nu_{ij}\) for the separability of the two classes in the feature v is defined using the Bhattacharyya distance as follows:

\(S_{i j}^{\nu}=2\left(1-e^{-B}\right)\)       (5)

The separabiltiy \(S^\nu_{ij}\) is the Jeffries-Matusita distance and is close to zero if the two means and variances are similar to each other, respectively. Nussbaun and Menz (2008) proposed a method to select the most relevant feature among the features by finding a feature with maximum separability of class pair using the Jeffries-Matusita distance. For example, given three classes, C_i,i=1, 2, 3, the v -th feature value x_v and the mean u_v of v-th feature, we classify pixels using the distance measure D_k, which is a vector distance between the feature vector x and the mean vector u^k of the class C_k, as follows:

\(\begin{aligned} D_{k}=& \sum_{\nu=1}^{V} l_{\nu} \cdot\left|x_{\nu}-u_{\nu}^{k}\right| \\ & l_{\nu}=\left\{\begin{array}{ll} 1 & \text { if } \nu \in\left\{\nu_{12}^{\max }, \nu_{23}^{\max }, \nu_{31}^{\max }\right. \\ 0 & \text { otherwise } \end{array}\right. \end{aligned} \)       (6)

where \(\text { where } \nu_{i j}^{\max }=\underset{\nu}{\operatorname{argmax}} S_{i j}^{\nu}, \nu=1, \ldots, V\).

The drawbacks of this method is that the separability of the other remaining class pairs. For example, if \(S^1_{12}\) is maximum, then \(S^1_{23}\) and \(S^1_{31}\) is not considered in finding the feature \(\nu^{max}_{12}\). The selection of the feature \(\nu ^{max}_{ij}\) may cause positive or negative effect on the separability of \(S^1_{23}\) and \(S^1_{31}\) depending on the characteristic of the class C₃. The second drawback is that the number of selected features is basically limited to the number of the class pairs, i.e., one feature is selected for each class pair. This means that some features are not used in image classification. This reduces the possibility of accuracy improvement which may be obtained by the insertion of the features into image classification.

2) Feature Selection by Feature Weighting based on Separability (FWS)

The proposed method for feature selection considers all the separability of the class pairs in each feature v, while the method for feature selection by the separability of two classes uses only one feature with maximum separability for each class pair. We compute the sum Jv of all separability of the class pairs in the featureas follows:

\(J_{\nu}=\sum_{i}^{N-1} \sum_{j=i+1}^{N} S_{i j}^{v}\)(7)

The sum J_v represents the effect of the feature v on the separability of all class pairs. If the sum J_v has a high value, the feature v has a good characteristic in the sense of separability of all class pairs. This means that the feature v is a useful feature in image classification. Therefore, the feature with high J_v needs to contribute much more to the computation of vector distance between the feature vector x and the mean vector u^k of the class C_k. The contribution of the feature v to the computation of vector distance in image classification is made by inserting the sum J_v of separability of all class pairs as weights into the distance measurement D_k between the feature vector x and the mean vector uk of the class C_k as follows:

\(D_{k}=\left(\sum_{\nu=1}^{V} w_{\nu} \cdot\left|x_{\nu}-u_{\nu}^{k}\right|^{p}\right)^{1 / p}, p=2 \)       (8)

\(w_{\nu}=J_{\nu} / \sum_{i=1}^{V} J_{i}\)       (9)

The weight value w_v of distance measurement D_k is proportional to the sum J_v of separability of all class pairs. The sum J_v of separability of all class pairs in the feature v represents the relative importance of the feature v in image classification. Table 1 shows the examples of feature selection based on feature weighting. The sum J_v of separability contains all the separabiltiy, i.e., S₁₂, S₂₃ and S₃₁, for each class pair. The relative importance of the feature v, i.e., the weight value wi is proportional to the sum of separability for class pair.

Table 1. Examples of feature selection based on feature weighting

3. Experimental Results

We selected two test areas of Landsat-8 OLI image over Metro Manila in Philippines and one test area over Cairo in Egypt as shown in Fig. 2. Built-up area and water are dominant in the Site A, while the area of vegetation spreads wide in the Site B. Site C contains the area of sands besides built-up area, water and the area of vegetation. We selected the seven bands from band 1 to band 7 of Landsat-8 OLI image and created the three indices, NDVI, NDWI and NDBI. The seven bands of Landsat-8 OLI image and the three index images were rescaled to the range (0, 255) by the minmax normalization. Three classes of vegetation, water and built-up area were chosen from both the Site A and the Site B, while four classes including sands were chosen from Site C. The means and variances of the training sites for the three classes were computed for the seven bands of Landsat-8 OLI image and the three index images, respectively.

Fig. 2. Two test areas (Site A and Site B) of Landsat-8 OLI image over Metro Manila in Philippines (Feb. 13, 2016) and one test area over Cairo in Egypt (Feb. 12, 2020) (a) Site A (b) Site B (c) Site C.

We selected the three features from ten spectral features consisting of seven bands of Landsat-8 OLI image and the three index images by using the STC method. For each class pair (BV: Built-up area-Vegetation, VW: Vegetation-Water, WB: Water-Builtup area), we obtained the three features, NDVI, NDWI and NIR, respectively as shown in Table 2. We classified the pixels by the minimum distance method using NDVI, NDWI and NIR images. Fig. 3(a) and (b) show the result of image classification using the STC method and the proposed FWS method, respectively. Many pixels located along the shore on the center in the Fig. 3(a) were classified into built-up area, while those are classified into vegetation in Fig. 3(b). Many pixels classified into built-up area on the left in Fig. 3(a) were also classified into vegetation in Fig. 3(b). The proposed FWS method produced more pixels classified into water along the shore and in the built-up area than the STC method.

Table 2. Result of feature selection based on the STC method

Fig. 3. Result of image classification for Site A using (a) STC method (b) proposed FWS method.

We selected 200 reference points from the Site A and the Site B, respectively, by the simple random sampling to assess the accuracy of image classification. Table 3 shows the accuracy assessment of the STC method for the Site A. The overall accuracy and kappa coefficient of the STC method were 90% and 83.95%, respectively. The low producer’s accuracy 77.08% of vegetation class was due to misclassification of vegetation into built-up area, in which the ratio of pixels misclassified into built-up area to the total number of reference vegetation points was 10/48 (20.83%). In contrast, the proposed FWS method produced higher overall accuracy of 95% and Kappa coefficient of 92.11% than the STC method as shown in Table 4. The producer’s accuracy of vegetation class was also increased from 77.08% to 91.67%. The ratio of pixels misclassified into built-up area to the total number of reference vegetation points was reduced to 3/48 (6.25%).

Table 3. Accuracy assessment of the STC method for Site A

Table 4. Accuracy assessment of the proposed FWS method for Site A

Fig. 4 shows the result of image classification for Site B using the STC method and the proposed FWS method, respectively. We can see the tendency for the misclassification of vegetation into built-up area in Fig. 4(a), while the proposed FWS method did not produce the misclassification of vegetation near the water boundary and on the left side of Fig. 4(b). Fig. 5(a) shows the image of RGB composite obtained by assigning NIR, NDVI and NDWI images to red, green and blue channels, respectively, to analyze visually the selected three features. The spectral difference between built-up area and vegetation is ambiguous in Fig. 5, while water is noticeable in the image of RGB composite and also in NDVI, NIR and NDWI images. The area classified into built-up area especially on the bottom left in Fig. 4(a) shows slightly darker than vegetation area in Fig. 5(b) and (c), and slightly brighter than vegetation area in Fig. 5(d). The small spectral difference between built-up area and vegetation area, which may result from the difference of terrain aspect, caused misclassification near water boundary and in the area with pixels whose intensities are relatively dark in NDVI and NIR images, and relatively bright in NDWI image as shown in Fig. 6.

Fig. 4. Result of image classification for Site B using (a) STC method (b) proposed FWS method.

Fig. 5. Images of (a) RGB composite obtained by assigning NIR, NDVI and NDWI images to red, green and blue channels, respectively (b) NDVI (c) NIR (d) NDWI.

Fig. 6. Enlarged images of (a) classification result by the STC method (b) NDVI (c) NIR (d) NDWI.

Table 5 shows the accuracy assessment of the STC method for Site B. The overall accuracy (O.A.) and kappa coefficient (K.C.) of the STC method were 92% and 72.92%, respectively. The user’s accuracy of Built-up area was 55.17%, which was mainly due to the misclassification of vegetation into built-up area. The proposed FWS method produced higher overall accuracy of 97.5% and kappa coefficient of 90.43% than the STC method as shown in Table 6. The user’s accuracy of Built-up area was dramatically increased from 55.17% to 85%.

Table 5. Accuracy assessment of the STC method for Site B

Table 6. Accuracy assessment of the proposed FWS method for Site B

Fig. 7 and Fig. 8 show the comparison of the accuracy of STC method and proposed FWS method for Site A and Site B, respectively, according to the number of features used in image classification including the case of ten spectral features in Table 2. The overall accuracy and kappa coefficient of the STC method for Site A and Site B increased until the number of features reached up to eight and then decreased when the number of features was ten. In contrast, the overall accuracy and kappa coefficient of the proposed FWS method for Site A and Site B increased gradually when the number of features increased up to ten. We can see that the insertion of each feature into image classification has contributed to the improvement of overall accuracy and kappa coefficient in the proposed FWS method.

Fig. 7. Comparison of the accuracy of proposed FWS method and STC method for Site A according to the number of features used in image classification.

Fig. 8. Comparison of the accuracy of proposed FWS method and STC method for Site B according to the number of features used in image classification.

Fig. 9 shows the result of image classification for Site C, where four classes of built-up area vegetation, water and sands were chosen. Some pixels in built-up area were classified into vegetation in STC method. More pixels on the bottom in Fig. 9(b) were classified into built-up than the STC method. Fig. 10 shows comparison of the accuracy of proposed FWS method and STC method. The proposed method showed high accuracies regardless of the number of features, while STC method showed relatively low accuracies when the number of features selected for classification is less than eight.

Fig. 9. Result of image classification for Site C using (a) STC method (b) proposed FWS method.

Fig. 10. Comparison of the accuracy of proposed FWS method and STC method for Site C according to the number of features used in image classification.

4. Conclusions

We described a method to compute the contribution of each spectral feature to image classification. The contribution of each spectral feature was estimated based on the class separability of each spectral feature. The ratio between the sum of class separability of each spectral feature and the total sum of class separability of all the spectral features was assigned to the weight value of the spectral feature in vector distance measure. The proposed method showed more improved accuracies in overall accuracy and kappa coefficient than the method of feature selection by the Separability of Two Classes (STC), which selects the most relevant spectral feature among spectral features by finding a spectral feature with maximum separability of class pair. The proposed method also showed good performance compared to STC method when the number of spectral features are increased. The main advantage of the proposed method is that the weight values of the spectral features are adaptively determined depending on the relative class separability of each spectral feature compared to other spectral features. The proposed method can also be used as a kind of dimension reduction method by selecting only some spectral features, which have class separability higher than a predefined threshold value. Further study on the case of more various features including texture features of high resolution satellite images is needed to test the feasibility of the proposed method.