A Comparative Assessment of the Efficacy of Frequency Ratio, Statistical Index, Weight of Evidence, Certainty Factor, and Index of Entropy in Landslide Susceptibility Mapping

Abstract

The rapid climatic changes being caused by global warming are resulting in abnormal weather conditions worldwide, which in some regions have increased the frequency of landslides. This study was aimed to analyze and compare the landslide susceptibility using the Frequency Ratio (FR), Statistical Index, Weight of Evidence, Certainty Factor, and Index of Entropy (IoE) at Woomyeon Mountain in South Korea. Through the construction of a landslide inventory map, 164 landslide locations in total were found, of which 50 (30%) were reserved to validate the model after 114 (70%) had been chosen at random for model training. The sixteen landslide conditioning factors related to topography, hydrology, pedology, and forestry factors were considered. The results were evaluated and compared using relative operating characteristic curve and the statistical indexes. From the analysis, it was shown that the FR and IoE models were better than the other models. The FR model, with a prediction rate of 0.805, performed slightly better than the IoE model with a prediction rate of 0.798. These models had the same sensitivity values of 0.940. The IoE model gave a specific value of 0.329 and an accuracy value of 0.710, which outperforms the FR model which gave 0.276 and 0.680, respectively, to predict the spatial landslide in the study area. The generated landslide susceptibility maps can be useful for disaster and land use planning.

1. Introduction

Landslides are common phenomena in mountainous regions that is defined as mass movement of rock, earth and debris, from the top to bottom of slope (Gruden, 1991). South Korea has a mountainous area, making up approximately 70% of the Korean peninsula. These areas consist of granite or gneiss lithology that is vulnerable to landslides. Especially, landslide risk increases during the summer because of heavy precipitation and typhoons. Since landslides cause loss of life and property, people have paid more attention to mitigate and manage such hazard.

One of the main approaches for mitigating the impacts of landslides is creating the landslide susceptibility map (LSM). The LSM can provide spatial distribution of potential slope failures, thus has a significant role to support and enhance spatial planning decisions focused on reducing landslides hazards (Goetz et al., 2015; Nohani et al., 2019). In recent decades, a number of different techniques with feasible and effective to use remote sensing (RS) and geographic information system (GIS) have been applied to produce LSM, including heuristic, deterministic and statistical methods.

The statistical methods are the most widely used to landslide susceptibility assessments, including certainty factor (CF), frequency ratio (FR), index of entropy (IoE), logistic regression (LR), statistical index (SI), and weight of evidence (WoE) (Akgun et al., 2008; Bui et al., 2011; Devkota et al., 2013; Jaafari et al., 2014; Kanungo et al., 2011; Mohammady et al., 2012; Pourghasemi et al., 2013; Zhang et al., 2016).

More recently, machine learning methods, which is the main source of techniques for the data-driven modeling problem (Jebur et al., 2013), have been popularly applied (Bui et al., 2016; Goetz et al., 2015; Pham et al., 2016a; Pradhan et al., 2013). Some researchers have developed ensemble model combining statistical and machine learning methods to obtain more accurate results and overcome drawbacks of individual methods(Abedini et al., 2019; Arabameri et al., 2019).

Since each method has different advantages and disadvantages, the prior step is to have a comprehensive understanding on the application of statistical methods. It can be helped to understand about application and combination with each method. In the literature, FR, SI, WoE, and CF can be seen most frequently on a landslide susceptibility assessment. Despite the popularity of these approaches, there has been little research comparing these approaches. Therefore, the main objectives of this study was to analyze and compare landslide susceptibility using different statistical methods, namely, the FR, SI, WoE, CF, and IoE models. In addition, the results of each model were compared using the relative operating characteristic (ROC) curve and statistical indexes to evaluate a more robust model.

2. Study area

The study area was Woomyeon Mountain, located in the Seocho district of Seoul City, South Korea. This area lies between longitudes 126°59′02″E and 127°01′41″E, and latitudes 37°27′00″N and 37°28′ 55″N (Fig. 1). The elevation is 293 m above sea level and the slope is approximately 30-35°. The bedrock is pre-Cambrian banded biotite gneiss, which is believed to be highly susceptible to landslides because of severe weathering and abundant faults(KGS, 2011). This area experienced a debris flow landslide due to heavy precipitation concentrated from 26-29 July, 2011. The maximum precipitation, which occurred for two hours one morning, was 164 mm. This exceeded the 156 mm 100-year return period. This resulted in serious damages to human life and property, with 68 casualties, 30 buried home, and 116 inundated homes(KGS, 2011; Park et al., 2019).

Fig. 1. Location map related to study area and landslide: (a) Orth-photo mosaic and landslide detection and (b) Landslide inventory map with hill shaded map of the study area.

3. Data and methodology

1) Landslide inventory

Landslide locations were identified using 32 aerial photographs of the study area taken after the occurrence of the landslides. These aerial photographs were taken by a digital mapping camera with a spatial resolution of 10 cm (Fig. 1(a)). A total of 164 landslide occurrence locations and landslide inventory map was prepared using these locations. The landslide inventory map was partitioned into 70% and 30% to be used for calibration and validation respectively (Fig. 1(b)). Additionally, non-landslide pixels were selected randomly from the non-landslide area.

2) Landslide conditioning factors

Landslide conditioning factors were produced in raster format with a cell size of 10 × 10 m, considering the scale of the input data. ArcGIS 10.2 and ERDAS Imagine 2011 were used (Fig. 2). The digital elevation model (DEM) was produced from airborne LiDAR data acquired using an ALTM Germini System (Optech, Inc.) owned by Saehan GeotechCo., Ltd. The topography factors including elevation, slope, aspect, profile curvature, and plan curvature were derived automatically from the DEM in ArcGIS.

Fig. 2. Landslide conditioning factors used to analyze landslide susceptibility: (a) Elevation, (b) Slope, (c) Aspect, (d) Profile curvature, (e) Plan curvature, (f) Distance to drainage, (g) Topographic wetness index, (h) Stream power index, (i) Sediment transport index, (j) Soil material, (k) Soil texture, (l) Soil topography, (m) Tree type, (n) Tree diameter, (o) Tree age, and (p) Crown density.

The hydrology factors were distance to drainage, topographic wetness index (TWI), stream power index (SPI), and sediment transport index (STI). The distance to drainage was calculated using the distance function in ArcGIS. TWI, SPI, and STI were calculated with their base in specific catchment areas (As) and slope maps as in the following (Beven and Kirby, 1979; Moore and Burch, 1986; Moore et al., 1991) :

\(T W I=\ln \left(\frac{A_{s}}{\tan \beta}\right)\)       (1)

SPI = A_s × tanβ       (2)

\(S T I=\left(\frac{A_{s}}{22.13}\right)^{0.6}\left(\frac{\sin \beta}{0.0896}\right)^{1.3}\)       (3)

where A_s represents the specific catchment area (m² / m), and β represents the local slope gradient, where degrees is the measurement.

The pedology factors included soil material, soil texture, and soil topography. These values were obtained from a 1:5000-scale soil map produced by the Korea NationalAcademy of Agricultural Science. The forestry factors were originated by the use of a 1:25,000-scale forest map from the Korea Forest Service.

3) Landslide susceptibility assessment

(1) Statistical Index Method

The SI method is considered as one of more simple and most quantitatively suitable methods in landslide susceptibility mapping and therefore it has been adopted by numerous other researchers (Chen et al., 2016). For this method, the natural logarithm of landslide density in the parameter class is divided by the landslide density in the study area defines the parameter class’s weight value. This method is based upon the following equation (van Western et al., 1997):

\(W_{i j}=\ln \left(\frac{A_{i j}}{A}\right)=\ln \left[\left(\frac{B_{i j}}{C_{i j}} / \frac{B}{C}\right)\right]\)       (4)

Where A_ij is the landslide density within the i-th parameter class j, A is the landslide density total, B_ij is the amount of landslides in the i-th parameter class j, C_ij is the amount of pixels in the i-th parameter class j, B is the total amount of landslides, and C is the total number of pixels.

(2) Weights of Evidence model

WoE is a data-driven method, which fundamentally is the Bayesian approach in a log-linear form using posterior (conditional) probability and prior (unconditional) probability (Bonham-Carter, 1994; Spiegelhalter, 1986). To perform WoE, positive weight (W⁺) and negative weight (W^–) need to be calculated as the essential parameters. These weights are defined as follows:

\(W_{i}^{+}=\ln \frac{P\{B \mid A\}}{P\{B \mid \bar{A}\}}\)       (5)

\(W_{i}^{-}=\ln \frac{P\{\bar{B} \mid A\}}{P\{\bar{B} \mid \bar{A}\}}\)       (6)

where P is the probability, B and \(\overline B\) are the presence and absence of landslide-related factors, respectively. A and \(\overline A\) are the presence and absence of a landslide (Bonham-Carter, 1994).

Weights contrast, C(C=W⁺– W^–), isthe name given to the difference between W⁺ and W⁺ weights. When there is no difference (C = 0), there is no significance for the analysis in that conditioning factor class. If the value ofCis positive, then so isspatial correlation, and the same applies in reverse to a negative C value (Corsini et al., 2009)

(3) Certainty Factors model

A possible approach to dealing with the question: how to combine different data layers, and also to the input data’s heterogeneity and uncertainty, is the CF model (Devkota et al., 2013). This model is defined as a function of probability as follows:

\(\mathrm{CF}=\left\{\begin{array}{ll} \frac{p p_{c}-p p_{p}}{p p_{c}\left(1-p p_{p}\right)} & \text { if, } p p_{c} \geq p p_{p} \\ \frac{p p_{c}-p p_{p}}{p p_{c}\left(1-p p_{p}\right)} & \text { if, } p p_{c}<p p_{p} \end{array}\right.\)       (7)

Where, pp_c is the conditional probability of landslide events in the i-th parameter class j and pp_p is the prior probability of the total amount of landslide event.

Each class has a value assigned of the CF that ranges between -1 and +1. If the value is positive, a landslide is more certain; if negative, a landslide is less so. Where the value is at or close to zero, there is little or no difference between prior likelihood and conditional likelihood. Thus, it is hard to provide any information regarding the certainty of the occurrence of landslides (Pourghasemi et al., 2012).

(4) Index of Entropy

Entropy, when applied to landslides, describes the degree of influence on a landslide’s development of a variety offactors.The equations used in the calculation of the information coefficient (W_j) representing the weight value for each landslide conditioning factor (Bednarik et al., 2010; Constantin et al., 2011) are given in the following:

\(P_{i j}=\frac{b}{a}\)       (8)

\(\left(P_{i j}\right)=\frac{P_{i j}}{\sum_{j=1}^{S_{j}} P_{i j}}\)       (9)

\(H_{j}=-\sum_{i=1}^{S_{i}}\left(P_{i j}\right) \log _{2}\left(P_{i j}\right), \quad j=1,2, \ldots, n\)       (10)

\(H_{j \max }=\log _{2} S_{j}\)       (11)

\(I_{j}=\frac{H_{j m a x}-H_{j}}{H_{j m a x}}, I=(0,1), J=1,2, \ldots, n\)       (12)

W_j = I_j P_ij         (13)

where a and b are, respectively, the domain and landslide percentages,(P_ij)isthe probability density. H_j and H_jmax are entropy values and S_j is the number of classes. I_j is the information coefficient and W_j is the resulting weight value for the factor as a whole.

4. Results and Discussion

1) Landslide susceptibility assessment and mapping

The results of the spatial relationship between a landslide and the landslide conditioning factors using CF, FR, IoE, SI, and WoE models are displayed in Table 1. The landslide susceptibility index was calculated about five different models over the study area and the LSMs were generated. Using the geometrical internals method the LSMs were reclassified into five susceptibility classes (Fig. 3). Overall, the very high susceptibility class covered about 20% of the total area. The CF model had the highest value (22.94%) and the IoE model had the lowest value (15.70%). The valuesfor the FR, SI, and WoE models were 22.04%, 21.49%, and 19.62%, respectively.

Table 1. Spatial relationship between each factor and landslide by the certainty frequency ratio, statistical index, Weight of evidence, certainty factor, and index of entropy models

^a Number of pixels in domain

^b Number of landslide

Fig. 3. Landslide susceptibility map produced by (a) frequency ratio, (b) statistical index, (c) weight of evidence, (d) certainty factor, and (e) index of entropy models.

LSMs produced from five different models were validated based on the Landslide Density (LD) of each susceptibility class on the maps. LD is the ratio of the percentage of landslide pixels and also the percentage of all pixels of each susceptibility class shown on the map (Pham et al., 2016b). LD was calculated by overlaying the five LSMs and the landslide inventory map. Generally, the value of LD increased gradually from very low susceptibility to very high susceptibility for the study area (Table 2). Based on the value of LD at the very high class, the IoE model had the highest value (2.292), followed by FR (2.324), CF (2.286), SI (2.100), and WoE (1.927). In addition, the high class also had the highest value for the results of some models. Overall, it can be shown that the models used in this study are suitable for LSM.

Table 2. Landslide density on landslide susceptibility maps produced from different models

2) Evaluation of Landslide Susceptibility Maps

The statistical indexes and the ROC curve was used in the evaluation and comparison of the performance of the five LSMs. The LSMs were reclassified as landslide (1) and non-landslide (0) to calculate the SIs. Based on the results of the LD analysis, the high and very high susceptibility classes were assigned as landslide and the very low susceptibility class was assigned as non-landslide. The SIs were calculated using IBM® SPSS® Statistics Version 21 (SPSS Inc., Chicago, IL, USA).

(1) Assessing predictive performance

The ROC curve is plotted by false positive rate (sensitivity) on the x-axis and 100 – false negative rate (100 – specificity) on the y-axis. The ROC curve can be classified into successrate curve and prediction date curve, depending on the used dataset. The successrate curve calculated using the training dataset represents how well the LSMs fit over the data. The prediction rate curve calculation made using the validation dataset is representative of how well the model and landslide conditioning factors predict the landslide (Bui et al., 2011). The area under the ROC curve (AUC) can have a value ranging from 0.5 to 1.0. When the AUC value is closer to 1, the model is more accurate.

The analysis of the success rate curve is shown in Fig. 4. Overall, theAUC values of the five LSMs were more than 0.7. In addition, there were no significant differences among the five LSMs. These results show that the LSMs produced in this study display a reasonable level of good accuracy in the spatial prediction of landslide susceptibility. The FR and IoE models were found to have the highest predictive performance when compared with the other LSMs.

Fig. 4. Analysis of the ROC curve on five landslide susceptibility maps: (a) Success rate curve using the training dataset and (b) Prediction rate curve using the validation dataset.

(2) Statistical Index-based evaluations

SIswere calculated froma confusionmatrix produced by observed data and LSMs reclassified with two classes. In this study, the sensitivity, specificity, and accuracy were used to validate the performance of the LSMs.What percentage oflandslide and non-landslide pixels are classified correctly into those two categories enables calculation ofsensitivity and specificity, while the overall percentage classified correctly (in both categoriestogether)is the accuracy ofthe LSMs(Pham et al., 2016a).

The performance analysis of the five LSMs using calibration and validation datasets has been performed (Table 3). In the case of using a calibration dataset, the FR and IoE models had the highest sensitivity value (0.930). The specificity value was the highest for the CF model (0.540). In the case of the accuracy value on each LSM, the SI and IoE models had the highest value (0.710), followed by CF (0.700), FR(0.680), and WoE (0.660) models.

Table 3. Model performance of each landslide susceptibility map using the training and validation dataset

In the case of using the validation data, the sensitivity values of FR and IoE models were the highest (0.940), which wasthe same when using the calibration dataset. The specificity value of the CF model was the highest (0.534). Overall, the accuracy values of IoE and SI models were the highest (0.710), followed by CF (0.700), FR (0.680), and WoE (0.660). Based on these results, it can be concluded that the IoE and SI models outperform the other models for spatial prediction of landslide susceptibility.

(3) Significance test of differences

McNemar’s test (a nonparametric test) was used to analyze the statistical significance of differences in modeling performances of the LSMs that were generated. McNemar’s Chi-squared statistic can be calculated as in the following (Japkowicz and Shag, 2011; Kavzoglu et al., 2015)

\(\chi^{2}=\frac{\left(\left|n_{i j}-n_{j i}\right|-1\right)^{2}}{n_{i j}+n_{j i}}\)       (14)

where n_ij represents the number of pixels that are misclassified by method i, but classified correctly by method j, and n_ji represents the number of pixels that are misclassified by method j, but classified correctly by method i.

McNemar’s test was applied to five LSMs and the calculated symmetric matrix is presented in Table 4. When the calculated significance probability is greater than 0.05 with a 95% confidence level, the null hypothesis can be rejected. Because there is a difference in performance between models i and j, there is a statistical significance to the difference in accuracy (Kavzoglu et al., 2015). The FR model showed a statistically similar performance to the SI and CF models. The CF model showed a statistically similar performance to the WoE model. In addition, the IoE model showed a statistically similar performance to the SI and CF models. These results show that there is not any significant difference among the aforementioned models.

Table 4. McNemar’s statistic test results for the frequency ratio, statistical index, weight of evidence, certainty factor, and index of entropy

^a The landslide susceptibility map produced by frequency ratio

^b The landslide susceptibility map produced by statistical index

^c The landslide susceptibility map produced by weight of evidence

^d The landslide susceptibility map produced by certainty factor

^e The landslide susceptibility map produced by index of entropy

5. Conclusions

The study aimed to compare and analyze landslide susceptibility using different models at Woomyeon Mountain. The study was comprised of three steps: collection and preparation of landslide-related spatial data; landslide susceptibility assessment; and results validation. Firstly, the landslide inventory map was built using aerial photographs. Additionally, the 16 landslide conditioning factors were constructed from government organizational collected spatial data, such as elevation, aspect, slope, profile curvature, plan curvature, distance to drainage, TWI, SPI, STI, soil texture, soil material, soil topography, tree type, tree age, tree diameter, and crown density. Secondly, landslide susceptibility indexes were calculated using the FR, SI, WoE, CF, and IoE models to analyze the spatial relationship between landslides and landslide condition factors. Thirdly, based on the results of the landslide susceptibility index, the LSMs were produced. These LSMs were evaluated and compared using the ROC curve and the statistical indexes.

The results showed that, overall, theAUC values of the LSMs used in this study were about 80%, with reasonable accuracy. The FR and IoE models had higher AUC values for the success rate and prediction rate curves compared with the other models. It can be shown that these models have the highest predictive performance. In the case of the prediction rate curve, the AUC value (0.805) of the FR model was higher than the IoE (0.798). There did not appear to be much difference between the models, so the results were observed to be similar.

The sensitivity values of the FR and IoE models were 0.940, which showed high accuracy for classifying landslide areas. Nevertheless, the specific value of the IoE model(0.329) was higher than that of the FRmodel (0.276). From these results, it can be demonstrated that the IoE model outperforms the FR model in the classification of non-landslide pixels. As a result, the accuracy value of the IoE model (0.710) was higher than that of the FR model (0.680). It can be shown that the IoE model outperformed the FR model to predict the landslide spatially in the study area. In addition, based on the result of McNemar’s test, McNemar’s Chi-squared statistic between the FR and IoE models was 0.375. This demonstrated that there is a significant difference between the FR and IoE models.

The LSMs produced in this study may prove useful for decision makers, planners, and engineers in disaster planning to minimize economic losses and casualties. In a future study, the LSM could be developed by combining with machine learning methods. Such a study can be improved the results of a landslide susceptibility assessment and helped to produce more accurate LSMs.