A Web-GIS Based Monitoring Module for Illegal Dumping in Smart Cities

Abstract

This study was conducted to develop a Web-GIS based monitoring module of smart city that can effectively respond, manage and improve situation in all stages of illegal dumping management on a city scale. First, five technologies were set for the core technical elements of the module configuration. Five core technical elements are as follows; video screening technology based on motion vector analysis, human behavior detection based on intelligent video analytics technology, mobile app for receiving civil complaints about illegal dumping, illegal dumping risk model and street cleanliness map, Web-GIS based situation monitoring technology. The development contents and results for each set of core technical elements were evaluated. Finally, a Web-GIS based 'illegal dumping monitoring module' was proposed. It is possible to collect and analyze city data at the local government level through operating the proposed module. Based on this, it is able to effectively detect illegal dumpers at relatively low cost and identify the tendency of illegal dumping by systematically managing habitual occurrence areas. In the future, it is expected to be developed in the form of an add-on module of the smart city integration platform operated by local governments to ensure interoperability and scalability.

1. Introduction

Urban solid waste is a crucial problem of cities in the 21st century [1]. According to the World Bank, solid waste generation in cities is expected to increase to 3.4 billion tones by 2050 [2]. Meanwhile, along with the generation of urban waste, unauthorized dumping waste that hinders the movement, aesthetics, and odor generation of citizens is discharged from city regardless of time and place. Illegal dumping cause considerable inconveniences of citizens' daily lives and degenerate quality of life. As such, illegal dumping is an urgent issue to solve. Globally governments and related institutions are struggling with illegal dumping of unsightly and unhealthy urban waste, and putting much effort and funds to solve it.

Despite South Korea has the highest waste separation collection rate in the world, illegal dumping remains as an important issue to remedy. According to the metropolitan city Seoul, the number of fines imposed for illegal dumping in 2017 was 118,200 an increase of 8,000 from previous year. And 62% of 730 complaints received through the response office during the first quarter of 2018 was related to illegal dumping.

Korean local governments have implemented various policies to manage areas where illegal dumping occurs. Most are focused on the traditional way of deploying enforcement personnel on a large scale, and some are spending a lot of money, such as installing

enforcement CCTVs and movable surveillance cameras. However it is valid only at the time and place of implementation, and is generally unsuccessful. In particular, it was found that the installation of CCTV for crackdowns did not have a significant effect on the reduction of illegal dumping. In case of Jung district in Daegu, CCTV crackdowns accounted for only 0.75% of all illegal dumping, and remaining 99% were arrested by temporary workers. Various equipment is being introduced for crackdowns, but the introduction has not been actively expanded because the equipment is too expensive.

Meanwhile, various studies are being conducted to solve urban waste problem by applying IoT [3]. Most of studies focus on removing unsightly waste through timely collection [4], introduction of smart bins [5], introduction of trucks to fit the demand [6], and garbage pick-up route optimization [7]. Recently, research based on machine learning algorithms has been actively executed. This includes studies such as predict generation of municipal solid waste [8], classify types of solid waste [9], and predict generation of household waste by applying LSTM neural networks [10]. Up to date, no studies have been carried out to develop solutions for illegal dumping management on a city scale.

To effectively prevent illegal dumping and maintain streets clean, it is necessary to create an environment that prevents illegal dumping during the high-probability times. Also, system for prevention and response system is needed to prevent recurrence by providing evidence on perpetrators to the crackdown teams.

In this study, the author propose a Web-GIS based monitoring module that can collect, store, convert, and analyze various types of data generated in smart cities. First, the author describe the core technical elements necessary to configure the module, and then develop the technical elements, execute performance evaluate of each element, successively.

2. Establishment of Core Technical Elements

The author defined following five core technological elements which constitute illegal dumping monitoring module in smart cities. Fig. 1 shows how the core elements are related and configured in Web-GIS based monitoring module.

Fig. 1 Five core technologies that constitute Web-GIS based monitoring module for illegal dumping in smart cities

2.1 Video screening technology

The existing CCTV integrated control system decodes the data received from CCTV to create real-time streaming videos. Based on this, objects and events are recognized through simple-eye analysis by monitoring personnel. This type of system require significant resources to decode all received CCTV image frames. In addition, there is a quantitative limit to the maximum number of channels that an employee can handle directly where one person can monitor up to 32 CCTV channels. A report [11] found that monitoring personnel missed up to 95% of the footage after 22 minutes of viewing.

Local governments in South Korea hope to actively introduce deep learning-based intelligent video analytics (IVA) system to overcome these limitations, but IVA system require lots of server resources and generally only up to 32 channels of analysis per server. Therefore it is impossible for local governments that operate thousands of CCTVs to apply the IVA system to all CCTVs due to budget limitations. In order to obtain a similar effect without adding a server, it is necessary to develop a video screening technology (VST) to be applied to the front of the IVA system. The principle of applying VST is to filter CCTV images without object movement which can be achieved by selectively applying VST to areas with low population density.

According to a 24-hour weekday survey in Ilsan District 3 in Korea, 553 videos were displayed to monitoring personnel in VMS (video management system), of which only 20% (107 out of 553) were found to have actual object movements. This means that 80% of videos that do not require additional IVA can be filtered first through VST. Since IVA is performed intensively only on filtered meaningful phases, it can speed up processing, reduce system-wide load, and monitor up to 100 to 150 channels with a single server. Accordingly, multi-channel video analysis is possible with only a small amount of resource input, and the monitoring range may be dramatically widened.

2.2 Intelligent video analytics technology

Currently, 2D digital camera videos used in CCTV integrated control centers of various local governments, road traffic monitors, and industrial complexes have very limited field applications since object detection and behavior type recognition accuracy is low. Most CCTV video analytics technologies are based on background difference method (BDM). It is too unstable to apply to outdoor environments because of significant influence of illuminance and wind. BDM has very good computational speed performance, but the disadvantage of continuously accumulating image frames and have to correcting unstable results exist [12].

Therefore it is necessary to develop algorithms with deep learning techniques that overcome the shortcomings of BDM and search for objects directly from images in line with the latest research trends. YOLO- based deep learning algorithms are much faster in processing than conventional image analysis techniques and can be used for accurate object recognition and object behavior type classification. Taking these advantages, it is necessary to develop IVA techniques that are robust to environmental noise, such as overlapping between objects, lighting changes, light-induced diffuse reflection, and (always) shaded areas. If DB manages it as an index using only key information such as the location and time of illegal dumping obtained through IVA, it can quickly collect waste on-site and support the post-response (arrest of perpetrators).

2.3 Mobile app for receiving civil complaints about illegal dumping

Since the first CCTV installation for crime prevention in Nonhyeon-dong in Gangnam district of Seoul in 2002, CCTV have been installed in each municipality across the country. However, blind spots are inevitably occurred in some areas due to budget limitations or urban facilities and topography. When waste was dumped in a blind spot, it is difficult to specify the time when it occurred. Therefore the range of video that must be searched by monitoring personnel to collect evidences is increased. However this problem can be addressed with a mobile app for civil complaints on illegal dumping.

The information of illegal dumping area is collected in real-time based on mobile global positioning system (GPS), description about the location typed by the user at the time of filing and photos of the dumping site. Another advantage is that monitoring personnel only need to watch CCTV footage before the time of filing a complaint for illegal dumping. At least, there is no need to watch the video between the time of the complaint and the present time. Reducing search time translates into lower operating costs. Some data of civil complaint can be used to update street cleanliness map (SCM) described in Section 2.4

2.4 Illegal dumping risk model and street cleanliness map

Illegal dumping risk model can be developed by combining information of habitual waste dumping areas, illegal dumping events information obtained through CCTV IVA, civil complaint information received through mobile app, post-response tracking and analysis results. Information on hot spots and hot times of illegal dumping can be obtained through the risk model. The model can create a heat map type SCM, which can be used as a layer in Web-GIS based monitoring page. Therefore, SCM is necessary because it is possible to preemptively and intensively respond to habitual dumping areas and to prevent inconvenience to citizens by quickly responding without a separate report.

2.5 Web-GIS based monitoring technology

In smart cities, spatial information technology is especially important because it can provide a standard to measure and monitor the city’s operational status by connecting data with real time-locations. In order for practitioners to effectively recognize various information related to the occurrence of illegal dumping of garbage collected using the aforementioned technology, data must be integrated and displayed based on spatial on Web-GIS based monitoring page. Researches [13-14] have extensively used remote sensing and GIS to address waste management challenges worldwide. With the advent of new geo-spatial systems such as GIS, large-scale waste management surveys have proven simpler in recent decades. These systems for waste management allow critical data to be collected, processed and transmitted in a simple and appropriate manner [15].

3. Development Contents for Each Core Technology Element

3.1 Video screening technology

Comparing the front and back frame of a video where an object moved, most of the background does not change, and only the location of the moved object changes. When the movement of an object is estimated through the difference between frames, the 2D element of the direction and size component of the object's movement is called a motion vector. The motion vector may be extracted and obtained from each frame by parsing the compressed image packet. Using motion vectors, it is possible to determine whether there is movement of an object within a specific video without decoding a video, which is a task with large data throughput.

Since unauthorized dumping of garbage is caused by a person, the object to be detected in the VST is limited to a person and a vehicle. Fig. 2(a) is a conceptual diagram for extracting a motion vector of a vehicle. Since most motion vector groups are partially generated, it is difficult to determine the overall outline of an object only by the geometry of motion vectors obtained from two image frames. Therefore, in order to emphasize the characteristics of the motion vector for each object, it is necessary to accumulate the motion vector generated in the frame for a predetermined period.

Fig. 2 Concept diagram of motion vector extraction of single frame (a) results of motion vector group visualization (b) car, (c) human

Normalization of the accumulated motion vector group allows the blanks generated in the motion vector group to be filled, making it easy to draw boundary box of the motion vector group. Meanwhile, since the motion vector does not have color information (RGB) such as an image, it is difficult to visualize direction and size information. Therefore, there is a need for a visualization method so that practitioners can effectively recognize motion vector groups. Motion vector’s direction component can be expressed as color according to the angle, and magnitude component can be expressed in a way that conveys the saturation within the same color series. Fig. 2(b) and 2(c) show the visualization results of the accumulated motion vector groups of car and humans, respectively.

I propose to place the developed motion vector-based VST in front of the IVA. Through VST, more CCTV channels can be analyzed using the same server resources by intensively performing IVA only on primary filtered images in which movement of objects is detected.

3.2 Intelligent video analytics technology based on Tiny YOLO-LITE

There are limitations in learning and image processing speed when using deep learning algorithm such as convolutional neural network (CNN) to implement object recognition. In order to overcome the limitations and improve detection accuracy, it is essential to upgrade the processing of the object region in the image. In addition, it is necessary to select and apply a machine learning algorithm that requires relatively little resource input to perform IVA in consideration of available resources when monitoring in CCTV integrated control center.

In this study, since detection targets were limited to people, behavioral types, cars, bicycles, etc., that means, only a few objects need to be distinguished, so the size of network is decreased. Therefore, the author developed an IVA technology based on Tiny YOLO-Lite network structure suitable for actual field application. Tiny YOLO-Lite has excellent recognition rate of performance and much faster detection speed. YOLO divides the image into grids, calculates probability values for the existence, determines type of objects, and detects objects by calculating a confidence value. Processing speed is 40-90 fps when using GPU suitable for real-time object detection.

For the development and application of IVA algorithms, CCTV video DB were built by filming videos in parking lots, alleys, and playgrounds with the cooperation of CCTV integrated control centers of local governments such as A city, G city, and U city. In addition, in order to establish CCTV video DB filmed at various angles and locations, additional filming was conducted through CCTV indoors, vacant lots, and offices of K university in Seoul.

The videos were filmed by establishing scenarios for ‘abandonment’ which is the actual detection target behavior, and ‘loitering’ and ‘invasion’ which can be called pre-abnormal behavior (see definitions in Table 1). Types of objects were learned and a technique was developed to detect the behavioral type of the object based on image frames captured in the CCTV videos. Using YOLO, it was possible to determine the location, size of person(s), movement path in the image by continuously detecting and tracking the person(s) in successive frames (see Fig. 3).

Fig. 3 Detection of abandonment event through IVA; (a) Person appeared (b) Person abandons an object (c) Separation of person and object (d) Event occurs

Table 1. Evaluation methods and standards for each detection target behavior

YOLO was implemented using Darkflow framework, and experiments were conducted with GTX1080 8G and GPU versions. Videos for each scenario were randomly selected for each filming location and classified for training/testing. Images were extracted from each video and used as input images for training/testing (5,967 training images and 964 testing images). Since there are not many behavior types to be distinguished, 750 epochs were performed with a batch size of 40 based on the Tiny YOLO-Lite network structure when execute training. As a result of testing randomly selected scenarios foreach place by setting the object detection standard value to 0.2, classification accuracy was 92.38%. The accuracy was calculated as the ratio of the number of normal detections to the total number of detections.

Loitering and intrusion behaviors, which occur prior to waste abandonment, were generally able to be distinguished successfully, even when there were several people. And they were not significantly affected by changes in external environmental conditions such as weather. In the case of abandonment, the detection of events can be successful only when objects (separated from humans) distinguished accurately by objects other than humans, so it was difficult to detect them at the beginning of the study, but finally showed an accuracy of more than 92%. Since the detection accuracy of untrained objects is very low, abandoned object images in the Internet and YouTube were used for training to improve the overall detection accuracy.

Through IVA, GPS and time information of the area where ‘abandoned’ is detected are collected and transmitted to the web server.

3.3 Mobile app for receiving civil complaints about illegal dumping

Mobile app was designed so that any citizen can easily report any illegal dumping site at anytime, anywhere. Important information to be collected through civil complaints are as follows; descriptive information and mobile GPS coordinates of the place, the time when it reported by citizens, and actual photoshoots at the site. Table 2 shows how DB was designed in consideration for post-data analysis. The information will be used as input values of illegal dumping risk model to generate the SCM and for subsequent updates. GPS accuracy problem is easily solved by describing the places of the violation typed by citizens. Also, in order to prevent false reports and help at-hand workers to quickly understand on-site situation, a photo of the site should be attached to the complaint. To ensure the reliability of the attached photo, the camera within the app leave a time stamp in upper left corner of the photo.

Table 2. DB design of the citizen’s mobile app

Meanwhile, the information collected through the mobile app should be transmit to Web-GIS based monitoring page. In prototype version, the mobile app was simply implemented with a report reception page and a processing status page. Through testing the author confirmed that when the app receives a civil complaint, the relevant information was displayed on the monitoring page along with an alarm of the occurrence of the event. Through 50 data transmission experiments, the information was transmitted to the monitoring page within 10 seconds without any problems.

3.4 Illegal dumping risk model and street cleanliness map (SCM)

A risk model for illegal dumping is established to calculate the risk by time of each location, and to generate it as an heat-map type SCM. It can help practitioners systematically manage habitual occurrence areas and help make decisions in future policies.

First, in order to establish illegal dumping risk model, the area was divided into a grid of 50m × 50m, and then grid numbers were given. As the author define the risk of illegal dumping at time zone j in the \(i^{th}\) grid as \(Y_{ij}\) (as in the ‘life safety map’ of the Ministry of Public Administration and Security, It can be divided into six time zones: 00:00-04:00, 04:00-08:00, 08:00-12:00, 12:00-16:00, 16:00-20:00, 20:00-24:00), it can be expressed as Eq. (1) and sum of weights is as in Eq. (2):

\(Y _ { i j } = w _ { 0 i } + w _ { 1 } x _ { 1 i j } + w _ { 2 } x _ { 2 i j } + w _ { 3 } x _ { 3 i j } j\)       (1)

\(w _ { 1 } + w _ { 2 } + w _ { 3 } = 1\)       (2)

where \(x_{1ij}\), \(x_{2ij}\) and \(x_{3ij}\) correspond to the number of past detections, civil complaints, and detected abandoned events through IVA for the time zone j in the \(i^{th}\) grid, respectively. If the number of past detections with no time zone information exist, it can be handled by dividing the value of the \(i^{th}\) grid by the number of time zone groups or by inputting the same value. \(w_{0i}\) is the initial value of illegal dumping risk of the \(i^{th}\) grid, and is assigned through an initial survey. Based on ‘street cleanliness evaluation’ criteria of Seoul, practitioners assign five grades and give them points for \(w_{0i}\). The state ‘Very good (1 pts)’ means that there are ‘0-1 places’ where waste is not collected in the inspection area. The other states are as follows; Good (0.4 pts, 2-3 places), Normal (0.3 pts, 4-5 places), Inadequate (0.2 pts, 6-7 places), Very poor (0.1 pts, over 8 places).

Since illegal dumping only occurs on the road, 0 for \(x_{kij}\) is entered (i.e. \(Y_{ij}\) = 0) as an invariant characteristic value for a grid that does not include roads. The weight also can be earned from numerical methods such as AHP [16], ANP [17], and BWM [18], widely used in multi criteria decision making methodologies targeting decision maker groups.

After the weights were derived, the author performed weighted linear combination to calculate the risk of illegal dumping for each grid. In this study, the risk analysis was performed using QGIS 3.16, an open source software. It was difficult to obtain real-time data as input data for the aforementioned risk model. the weights were \(w_{1}=0.5\), \(w_{2}=0.3\), \(w_{3}=0.2\) and grid values were generated as arbitary values. High-risk areas were visualized in red and low-risk areas in green using GIS software. The results showing the risk of illegal dumping on the monitoring page are shown in Fig. 4 From the estimation results of the occurrence risk model, it can be seen that the higher occurrence risk of illegal dumping means, the higher probability of occurrence in each region by period. Even if the situation of the habitual dumping area improves and changes, can get an SCM close to the current state by update analysis result.

Fig. 4 Displaying the ‘street cleanliness map’ in Web-GIS monitoring page

3.5 Web-GIS based monitoring technology

Implementing various information collected by the aforementioned technology on a Web-GIS based monitoring page based on space allows users to effectively recognize and respond to related information in a timely manner. Therefore, the author designed the layout of the monitoring page through bench-marking analysis of other successful similar applications with same objectives (Naver Map, Kakao Map, Clean Streets LA).

The monitoring page enables the user to quickly grasp event-related information, including location, when an illegal dumping event occurs. Since user needs to quickly scan the page, the author configured the user's information recognition flow in an F-pattern.

The SCM is implemented as a layer on the map as shown in Fig. 4 After selecting the risk level of illegal dumping, hot spots by time-zones are displayed on the map so that users can easily identify high-probability areas by time zone. The practitioner can watch real-time video streaming by pressing the CCTV viewing button and also can play recorded CCTV videos if monitoring module is linked with local VMS. If the video of scene and suspect's movement route information are provided to enforcement team dispatched to the site, it can help quickly to understand the situation and support response. Based on the CCTV video evidence stored in the system, it is expected that it will be able to replace the current process of imposing fine only when there is evidence that can identify the suspect, tediously produced by disassembling the waste piece by piece.

4. Conclusions

In this study, five key technologies needed to configure a Web-GIS based monitoring module that show the functionality and feasibility of effective management and improvement in all stages of illegal dumping management on a city scale was defined, and implemented a prototype module through the development of core technologies.

Video screening technology based on motion vector analysis shows the possibility of obtaining up to 5 times analysis coverage in the same server environment by performing IVA only on filtered videos, excluding 80% of the motionless videos of object. The intelligent video analytics technology developed based on the YOLO-Lite algorithm detected intrusion, wandering, and abandonment events in CCTV images with an accuracy of over 92%, which can help overcome the physical control limitations of monitoring personnel. The real-time civil complaint reporting mobile app to compensate IVA and CCTV blind spot problems has the advantage of notifying the occurrence of illegal dumping and reducing the amount of work required for backtracking by limiting the time of occurrence. When illegal dumping incidents are detected in real time through IVA and mobile apps, they are displayed on the monitoring page of the Web-GIS environment so that monitoring agents can respond in real time. Monitoring agents can increase the success rate of crackdowns by inquiring data that can identify the identity or whereabouts of the perpetrator, such as on-site video and video recorded on CCTV. In addition, by instructing the timely collection of illegally dumped garbage, additional garbage dumping according to ‘social adaptation’ can be prevented. In the module, illegal dumping occurrence data (time/spatial information) collected through various sources can be systematically managed and analyzed to predict event occurrence areas, and SCM can be created to systematically manage illegal dumping risk areas by time zone. The SCM is displayed on the monitoring page as a map layer in the form of a heat map. The module can generate various types of statistical analysis with actual event occurrence data, so it can be used to support the following decision making to solve the problem of illegal dumping; optimal CCTV installation location evaluation, enforcement manpower placement, movable CCTV installation preventive environmental design support.

The prototype of monitoring module was developed as a standalone. However, the final form will be an add-on module that can increase scalability and interoperability with the smart city integrated platform package already operated by many local governments in Korea. It is expected to help solve the problem of illegal dumping and improve the quality of life of citizens.