Assessing Stream Vegetation Dynamics and Revetment Impact Using Time-Series RGB UAV Images and ResNeXt101 CNNs

Abstract

Small streams, despite their rich ecosystems, face challenges in vegetation assessment due to the limitations of traditional, time-consuming methods. This study presents a groundbreaking approach, combining unmanned aerial vehicles(UAVs), convolutional neural networks(CNNs), and the vegetation differential vegetation index (VDVI), to revolutionize both assessment and management of stream vegetation. Focusing on Idong Stream in South Korea (2.7 km long, 2.34 km² basin area)with eight diverse revetment methods, we leveraged high-resolution RGB images captured by UAVs across five dates (July-December). These images trained a ResNeXt101 CNN model, achieving an impressive 89% accuracy in classifying vegetation cover(soil,water, and vegetation). This enabled detailed spatial and temporal analysis of vegetation distribution. Further, VDVI calculations on classified vegetation areas allowed assessment of vegetation vitality. Our key findings showcase the power of this approach:(a) TheCNN model generated highly accurate cover maps, facilitating precise monitoring of vegetation changes overtime and space. (b) August displayed the highest average VDVI(0.24), indicating peak vegetation growth crucial for stabilizing streambanks and resisting flow. (c) Different revetment methods impacted vegetation vitality. Fieldstone sections exhibited initial high vitality followed by decline due to leaf browning. Block-type sections and the control group showed a gradual decline after peak growth. Interestingly, the "H environment block" exhibited minimal change, suggesting potential benefits for specific ecological functions.(d) Despite initial differences, all sections converged in vegetation distribution trends after 15 years due to the influence of surrounding vegetation. This study demonstrates the immense potential of UAV-based remote sensing and CNNs for revolutionizing small-stream vegetation assessment and management. By providing high-resolution, temporally detailed data, this approach offers distinct advantages over traditional methods, ultimately benefiting both the environment and surrounding communities through informed decision-making for improved stream health and ecological conservation.

1. Introduction

Streams, fundamental to our planet’s health, face unique management challenges due to their size and dynamic nature. Despite their critical role in ecosystem health and flood control, smaller streams (less than 30 meters wide) often fall through the cracks due to a lack of dedicated funding and management strategies. Unlike larger rivers, managing small streams requires sustained effort and a shift in perspective toward long-term health (Kelly-Quinn et al., 2023). These ecosystems are dynamic, constantly changing with seasons and human influences, demanding adaptable management strategies (Verdonschot and Verdonschot, 2023). Droughts and floods pose constant risks, requiring proactive monitoring and mitigation (Ferreira et al., 2023; Tigkas et al., 2012).

Additionally, fragmented jurisdictional landscapes create further hurdles; Lack of clear oversight strains personnel responsible for these streams, hindering effective management. The absence ofregulations and guidelines creates a management vacuum, hindering efficient strategies (Scoggins et al., 2022). Traditional methods like visual inspections are costly and raise concerns about worker safety and accuracy. Research efforts should focus on developing effective and efficient management strategies for small streams. Exploring the use of technology for improved monitoring and data collection. Addressing the challenges offragmented governance and promoting collaborative approaches(Langhammer et al., 2023; Svane et al., 2022). Raising public awareness about the importance of these often-overlooked ecosystems.

Beyond management challenges, artificial modifications for irrigation and flood control have significantly degraded the environmental health of many waterways. To address this, the Ministry of Environment haslaunched nature-friendly restoration projects that promote vegetation growth through methods like vegetation mats, natural stone stacking, and vegetation blocks (Koo et al., 2014;Kim et al., 2011).These methods are particularly suitable forsmallstreams due to their unique characteristics and intricate environmental interactions. Restoring and conserving these vital arteriesis crucial forthe ecological health of our planet.

Understanding stream health and function requires detailed vegetation surveys, as vegetation plays a critical role in stream flow, light penetration, and overall ecosystem health (Hahm and Kim, 2010; Kwon and Sim, 2010). Traditionally, such surveys relied on time-consuming and labor-intensive field visits, meticulously mapping vegetation distribution (Lee et al., 2011). While satellite imagery offers a broader perspective, its limited resolution (typically meters) makes it unsuitable for most small streams. These narrow corridors remain largely unseen, hindering effective management (Watanabe and Kawahara, 2016).

Unmanned aerial vehicles (UAVs) equipped with remote sensing (RS) technology are transforming the way we monitor small streams. Their ability to capture high-resolution images at the centimeter level, coupled with their affordability, offers a game-changing alternative to traditional methods. This technological leap provides detailed and accurate views of these often-overlooked ecosystems, paving the way for improved monitoring, management, and ultimately, protection.

As climate change disrupts precipitation patterns, efficient and accurate stream management becomes even more critical. By adopting UAVs and leveraging information communication technology (ICT) and artificial intelligence (AI),we can automate tasks, reduce reliance on manual labor, and ultimately improve the efficiency and effectiveness of small-stream management. This technological shift is crucial for ensuring the ecological health and safety of these vital waterways in the face of an uncertain future.

Traditionally, vegetation cover classification in RS relied on analyzing multi-wavelength optical images. These images enable detailed vegetation characterization and vitality assessment through well-established vegetation indices like the normalized difference vegetation index (NDVI) (Pace et al., 2022). While valuable, this approach requires specialized knowledge and can be expensive. RGB images offer a compelling alternative to cheaper sensors and higher resolution. vegetation differential vegetation index (VDVI) is designed to be more sensitive to changes in vegetation cover compared to traditional vegetation indices like NDVI.

Recent advancements in AI, particularly deep learning, have yielded powerful toolsfor analyzing these images.Convolutional neural networks (CNNs), like ResNeXt101 (Xie et al., 2017), excel at processing two-dimensional data, making them ideal for building image-based classification models.Compared to simpler models, ResNeXt101 can capture complex relationships and patterns within the image data, leading to more accurate and detailed results, especially for subtle variations in vegetation characteristics.

VDVI andResNeXt101 offer distinct advantagesfor vegetation assessment compared to traditional methods. VDVI’ssensitivity to changes in vegetation cover and its ability to account for soil variations make it ideal for detecting early signs of stress. ResNeXt101’s ability to extract deeperfeaturesfrom RGB images allows for more accurate and detailed classification. Their combined use provides a powerful and efficient approach for obtaining high-resolution, temporally detailed data on vegetation cover and health, especially when combined with UAV-based image acquisition.

This study aimed to advance small stream vegetation surveying by (1) acquiring high-resolution image data of small streams using RGB UAV RS and (2) developing a CNN-based classification model forsmallstream vegetation cover, analyzing vegetation vitality.

2. Materials and Methods

2.1. Study Site and Field Survey Section

The studywas conducted atIdong Stream, located inCheongan-myeon, Goesan-gun, Chungcheongbuk-do, South Korea. Idong Stream has a total length of 2.7 km and a basin area of 2.34 km² (Fig. 1).

Fig. 1. The study area, depicted in this figure, was mapped using RGB UAV remote sensing. Eleven ground control points (GCPs) were established for precise georeferencing, and natural streambank protection methods (A–I) were identified within the area.

A 1.8 km section of the stream, stretching from the Munbang river confluence to a tributary,wasselected forfield survey (area: 0.25 km²). This section is significant for two reasons: (1) it was designated as a national pilot project area for small stream maintenance, and (2) it served as a testing ground for various natural revetment methods in 2008 (Lee et al., 2010).

A total of eight revetment construction methods were used within the surveyed section (Fig. 1). These methods are denoted A-I in Fig. 1. The characteristics of each method are shown in Table 1. The primary objective of this study was to compare and analyze the vegetation vitality associatedwith each implemented revetment method.

Table 1. The characteristics and corresponding images of the fieldstone and block type revetment method

2.2. RGB UAV Remote Sensing

Thisstudy utilized a fixed-wingUAVand sensor,the specifications ofwhich are detailed in Table 2.Compared to rotary-wing UAVs, fixed-wing models offer advantagessuch aslighterweight, better battery efficiency, and the ability to film larger areas.

Table 2. The specifications of the UAV and sensors used in this study

However, they require a larger takeoff and landing space. Choosing the appropriate UAV and sensor settings is crucial for vegetation monitoring using aerial imagery. Based on previous research experience, we prioritized image resolution, quality, and flight time while considering the set flight path, height, and image overlap. The acquired images had a resolution of 0.02 m/pixel, chosen to enable the development of a 0.5 m resolution classification model with improved accuracy, suitable for future integration with Korea’s next-generation medium-sized satellite.

Vertical and horizontal image overlap percentages were set to 60% and 70%, respectively. Each UAV flight spanned 30–40 minutes, capturing the entire target area in a single session. Data collection occurred five times at approximately 30-day intervals, from July 20, 2023, to December 4, 2023. Each shooting timewas scheduled from 10 AM to 2 PM, including the noon hour.

2.3. CNN Classification Model Selection

While traditional vegetation monitoring methodsface limitations, CNNs offer a powerful and versatile alternative. Their ability to automatically extract features (Janiesch et al., 2021), achieve high classification accuracy (Zhang et al., 2021), minimize human error, and readily adapt to diverse datasets makes them valuable tools for ecological studies.

Among various CNN architectures, this study employed the ResNeXt101 model due to its specific advantages for vegetation monitoring: (a) ResNeXt’s aggregated identity paths enable the model to learn diverse and complementary information from image data, leading to a more nuanced understanding of vegetation characteristics compared to standard ResNet architectures(Xie et al., 2017). Thisis crucial for accurate classification ofsubtle differences between plantspecies, particularly in complex ecosystems.(b) ResNeXt utilizes bottleneck connections to efficiently build deeper models without compromising computational resources.

This allows the model to learn complex relationships within the data, leading to improved vegetation classification accuracy, especially when dealing with large and diverse datasets (Xie et al., 2017). Therefore, the ResNeXt101 model’s superior feature representation and enhanced model capacity, combinedwith the general advantages of CNNs, make it a well-suited choice for vegetation monitoring in this study.

2.4. Vegetation Vitality

The relationship between vegetation and stream dynamics is complex. While vegetation thrives in well-lit areas, it plays a dual role in stream ecosystems. On the one hand, it protects streambeds and banks from erosion. On the other hand, dense vegetation can hinder stream flow during floods. Vegetation vitality, measured by indicators like growth values, affects its impact on flow resistance, with higher vitality leading to greater resistance (Ryu et al., 2019). This makes effective assessment of vegetation vitality crucial for understanding stream behavior.

UAV remote sensing offers a valuable tool for studying vegetation vitality through analysis of optical image reflectivity-based indices (Xue and Su, 2017). The VDVI stands out as a particularly suitable index for visible wavelength images, due to its ability to normalize vegetation signals and track changes over time (Zhou et al., 2021).Asshown in Eq.(1), VDVIis calculated using the reflectance values of red, green, and blue bands (R_i, where i = b, g, r).

\(\begin{align}V D V I=\frac{2 \times R_{g}-\left(R_{r}+R_{b}\right)}{2 \times R_{g}+\left(R_{r}+R_{b}\right)}\end{align}\)       (1)

2.5. Methods and Study’s Workflow

The study’s workflow, as illustrated in Fig. 2, proceeded in the following order: (a) UAV-based RGB image acquisition, (b) image preprocessing, (c) dataset production, (d) model learning and evaluation, and (e) vegetation vitality analysis.

Fig. 2. The entire research processisillustrated in thisflowchart,visually depicting the key steps and their sequence.

Image preprocessing involved generating an orthoimage with a resolution of 0.02 m/pixel using Pix4Dmapper (Pix4D, Prilly, Switzerland). The geometric correction was performed through reference correction with GCPs applied using V30 (Hi-target, Guangzhou, China) VRS-GNSS (Martínez Carricondo et al., 2018).

To create the image dataset, the processed orthoimage was used to mask the revetment and stream bed, and the masked image was then cut into 25 × 25-pixel segments. The resulting images were divided into Train and Test zones, as depicted in Fig. 1. Streams are mainly composed of soil and water on the streambed and vegetation on the banks, so they were classified into three categories. Each image was classified into Vegetation (0), Water (1), and Soil (2) categories and labeled with numeric codes. The dataset comprised 10,030 learning data and 10,018 evaluation data for each date, totaling 50,150 learning data and 50,090 evaluation data, amounting to 100,240 pieces.

For model learning and evaluation, the ResNeXt101_64 × 4D model in PyTorch ver. 2.0.1 (Meta AI, Menlo Park, CA, USA) was employed. ResNeXt101_64 × 4D leverages its deeper architecture, increased cardinality, and wider bottlenecks to achieve higher accuracy and generalization compared to its ResNet counterpart. To prevent overfitting and optimize the model, we performed 30 repetitions with a batch size of 1,000, comparing the training and evaluation accuracy for each repetition.

Vegetationvitalitywasstatistically analyzed using a classification model and VDVI. VDVI was extracted from vegetation images using the classification model and calculated using Eq. (1).

3. Results and Discussion

3.1. Dataset and Model Training

The dataset was organized into folders and applied within the PyTorch framework (Fig. 3). During model learning, the class distribution was as follows: 2.21% Soil, 85.19% Vegetation, and 12.59% Water. During the model evaluation, the distribution was 1.89% Soil, 86.51% Vegetation, and 11.60% Water. These percentages represent data from all dates: July 20, August 18, October 5, October 27, and December 4. The high proportion of vegetation in the images is attributable to the active growth of stream vegetation between July and October, which visually dominates the lakeshore and streambank. To address this imbalance, weights were assigned to each class during model training.

Fig. 3. Configuring training and testing dataset folders.

Fig. 4 illustrates model accuracy and loss rates for training and testing sets over 30 iterations. Training accuracy steadily increased, reaching a maximum of 0.9383. However, testing accuracy plateaued between 0.79 and 0.85 after 10 iterations, indicating a divergence from training accuracy. To circumvent potential overfitting, the modelwith the highest testing accuracy before the 10th iterationwasselected asthe optimalstream cover classification model. This model, derived from the second iteration, achieved an accuracy of 0.86 on the training set and 0.89 on the testing set.

Fig. 4. Model training and testing accuracy and loss rate results.

Harnessing RGB sensors, the trained model offers exceptional efficiency in data acquisition. Its high accuracy of 0.89 in testing demonstrates its potential to generate cover classification maps with exceptional temporal and spatial resolution.

3.2. Stream Cover Classification

The classification results using the trained model are shown in Table 3. Overall accuracywas 0.81, but the kappa coefficientwas low at 0.36. This difference indicates a variation in classification performance between each class. It was confirmed that the trained model showed significant results for vegetation and water classification, but the classification performance for soil was low. Since the purpose of this study is to analyze the vitality of vegetation in streams, the resultswere significant forthe model used for vegetation classification.

Table 3. Error matrix for stream cover classification

Mapping stream cover classificationwas achieved by combining location informationwith classification resultsfor each acquisition date, using the previously trained model. Fig. 5 depicts a portion ofthe test areawithin the full cover classification map,showcasing a distribution of Vegetation, Water, and Soil classes. This area served as a basis for interpreting cover classification outcomes.

Fig. 5. Time-series analysis ofvegetation,water, and soil distribution for testsection:(a) UAV images,(b) truth images, and (c)model prediction outputs.

Three challenges emerged during model-based classification: (1) Distinguishing water from soil in low water areas: Shallow water in RGB images is transparent, posing a challenge even for human visual interpretation. In Fig. 5, the July 20 image shows flow due to precipitation, suggesting less soil than the model predicted (approximately 50% classified as soil). (2) Shadow-induced classification errors: Images from October 5 to December 4 (Fig. 5) were captured between 10 and 11 a.m. Despite similar capture times, variations in sunrise, sunset, and solar altitude cast shadows across the streambed. Consequently, shaded vegetation areas were often misclassified as water, as seen in the December 4 image. (3) Image brightness-related classification errors: The model was trained on a mix of five time-series images for temporal versatility. However, input data brightness during training varied based on capture dates.Images with lowbrightness values,such asthe December 4 image in Fig. 5, exhibited lower classification accuracy because the training data lacked images with similar low brightness levels.

Despite these challenges, the model proved valuable for understanding vegetation distribution in small streams across most images. The results corroborate the potential of vegetation distribution status and vegetation indices for stream roughness analysis.

3.3. Vegetation Vitality Analysis

Vegetation vitality was assessed using VDVI calculated for vegetation classified by the ResNeXt101 model. Given the varied naturalrevetment methods applied throughout the study stream, vegetation vitality was further analyzed for each section and its corresponding method.

Fig. 6 depicts the spatiotemporal changes in VDVI within the study stream, categorized by the implemented revetment methods. Fig. 6(a) presents a boxplot highlighting the overall change in VDVI within classified vegetation areas across the entire study area. Dates were analyzed using the stream cover classification model. August 18th exhibited the highest average VDVI of 0.24 among the five periods, with values gradually decreasing thereafter. By December 4, most vegetation displayed a vitality close to 0. This pattern suggests a peak in vegetation growth betweenAugust and September, indicating this period as crucial for flow resistance within small streams. Notably, this timeframe coincides with significant rainfall, emphasizing the importance of vegetation management for stream maintenance and flood prevention in surrounding areas.

Fig. 6. Changes in VDVI distribution across revetment methods. (a) Overall VDVI distribution dynamics. The central box represents the interquartile range (IQR), with the median marked as a horizontal line. (b) Fieldstone and (c) block type revetment method specific VDVI trends (Figs. 1A–I).

Figs. 6(b, c) delves deeper, illustrating the variation in average vegetation vitality per survey period and the corresponding revetment method. Thisline graph depicts average VDVI values for each survey period, differentiated by the implemented method in each stream section. Distinct line styles and colors facilitate easy comparison. Section G (three flights of fieldstone) displayed the highest average vitality in August. This section’s vegetation mix ofwoody and herbaceous plants explainsitsrapid decline in vitality after October due to leaf browning. Section F (vegetation mat and a flight of fieldstone), harboring both herbaceous and high vegetation, mirrored this trend. Therefore, areas utilizing the fieldstone method, including sectionsA, F, and G, exhibited a characteristic pattern of high vitality in July and August, followed by a sharp decrease from October. In contrast, sections B, C, D, and E, employing the block-type construction method, displayed a change pattern like the control section I, where no revetment method was applied. Section I showcased the natural dynamics of vegetationwith rapid growth in July and August followed by a gradual decline. Interestingly, unlike other block-type methods,section H (H environment block) exhibited minimal change in vitality between July and August.

VDVI’s strength lies in its sensitivity to chlorophyll content, making it an excellent tool for monitoring vegetation vigor. However, its limited ability to differentiate between various soil types and confounding factors from other elements hinders its accuracy in soil classification. Utilizing alternative indices or combining VDVI with other data sources can address these limitationsformore comprehensive assessments of both vegetation and soil conditions. Despite the differing construction methods, the influence of vegetation introduced from the surrounding area led to remarkably similar distribution trends after 15 years. These results demonstrate the combined power of UAVs, VDVI, and ResNeXt101 models as a valuable tool for identifying vegetation distribution in streams and informing effective vegetation management strategies.

4. Conclusions

This study demonstrates the effectiveness of UAVs and CNN models for high-resolution, temporally detailed vegetation mapping in small streams. The trained ResNeXt101 model achieved 89% accuracy in classifying vegetation cover(soil,water, and vegetation), enabling precise monitoring of vegetation changes over time. This technology offers valuable insights for understanding the impact of different revetment methods on vegetation dynamics and flow resistance.

VDVI proved effective in assessing vegetation vitality, with August exhibiting peak values crucial forstream flow resistance. Fieldstone revetment sections showed high initial vitality followed by a sharp decline due to leaf browning, while block-type sections displayed a gradual decline like the controlsection. Interestingly, the environment block maintained relatively stable vitality throughout the year, suggesting potential for revetment design optimization.

The study highlightsthe importance of vegetation management for stream health and flood mitigation. VDVI is well-suited for vegetation assessment, alternative indices like soil-adjusted vegetation index or land surface water index are specifically designed to address soil and moisture variations, offering improved soil classification accuracy. Integrating VDVI with other spectral bands or indices sensitive to soil properties can also enhance soil estimation accuracy. By monitoring vegetation dynamics and vitality, informed decisions can be made regarding revetment design, vegetation planting, and maintenance practices. Understanding the impact of revetment methods on vegetation diversity and habitat suitability can inform more sustainable practicesforstreamrestorationandconservation.The combination of UAVs, CNNs, and VDVI provides a powerful platform for collecting and analyzing detailed stream vegetation data. This information can be used to develop data-driven models for predicting vegetation dynamics, assessing stream health, and optimizing river management strategies.

The study identified challenges like distinguishing water from the soil in low-water areas, shadow-induced errors, and brightness variations. Addressing these limitations through advanced image processing techniques and sensor selection will furtherimprove the accuracy and applicability of the technology. The study focused on a single stream with specific revetment methods. Further research on a wider range of streams and revetment types is needed to validate the findings and develop broader recommendations for river management.

Overall, this study presents a promising approach for revolutionizing small-stream vegetation assessment and management. By leveraging the power of UAVs,CNNs, and VDVI,we can gain valuable insights into vegetation dynamics and inform more effective strategies forstream and river health, flood prevention, and ecological conservation.