A CNN Image Classification Analysis for 'Clean-Coast Detector' as Tourism Service Distribution

Abstract

Purpose: This study is to analyze the image classification using Convolution Neural Network and Transfer Learning for Jeju Island and to suggest related implications. As the biggest tourist destination in Korea, Jeju Island encounters environmental issues frequently caused by marine debris along the seaside. The ever-increasing volume of plastic waste requires multidirectional management and protection. Research design, data and methodology: In this study, the deep learning CNN algorithm was used to train a number of images from Jeju clean and polluted beaches. In the process of validating and testing pre-processed images, we attempted to explore their applicability to coastal tourism applications through probabilities of classifying images and predicting clean shores. Results: We transformed and augmented 194 small image dataset into 3,880 image data. The results of the pre-trained test set were 85%, 70% and 86%, and then its accuracy has increased through the process. We finally obtained a rapid convergence of 97.73% and 100% (20/20) in the actual training and validation sets. Conclusions: The tested algorithms are expected to implement in applications for tourism service distribution aimed at reducing coastal waste or in CCTVs as a detector or indicator for residents and tourists to protect clean beaches on Jeju Island.

1. Introduction

Marine Debris (Marine Litter) is a man-made waste that is intentionally or accidentally released into a lake, sea, or waterway, and can be defined as a permanent product or processed solid that has been abandoned or disposed of in marine and coastal environments (Lee et al., 2016; Terzi &Seyhan, 2017; Research Institute of Ships & Engineering,2019). According to the United Nations Environment Program (UNEP), about 8 million tons of marine debris is produced annually worldwide, of which 80% are land-produced municipal waste (Baker et al., 2019). Wastes that are not completely disposed of are not only visually problematic, but which also pose a threat are addressed in a timely manner without changing human consciousness about the processes and responsibilities of production,consumption and disposal as a threat to the entire environmental ecosystem humans, animals and plants (Gall& Thompson, 2015).

In particular, the amount of marine debris in Jeju has surpassed 10,000 tons annually for the past four to five years and is emerging as a new local issue (Kim, 2019). According to the results announced by the Korean National Coastal Garbage Monitoring in 2018, plastics account for 59% of the marine debris, and Jeju's coastal wastes are easily attributed to external sources (Lee et al., 2016), from foreign countries (especially China). However, the amount of the waste originated from other countries estimated is actually less than 10 % (Kim, 2019). This means that most of the garbage produced inside Jeju Island is closely related to the increase of tourists and the entire population of Jeju Island, which has expanded rapidly in recent years (Lee et al, 2016, Chang et al, 2019; Choi et al, 2019).

As we are aware of these matters, various studies have been conducted in academia on the seriousness of the marine debris problem that threatens humans and the ecosystem of the earth. Among them, Alkalay et al. (2007) presented the concept of the Clean Coast Index (CCI),which indexed the cleanliness of the coastal areas to five grades, measured the amount of plastic parts per square meter out of costal wastes. Since then, CCI index was often cited in marine environmental studies (Laglbauer et al,2014; Fernandino et al, 2015; Campbel et al., 2016; Terzi,Y., & Seyhan, K. 2017). In addition to the attempts to monitor and measure or predict the severity of plastic waste,especially in marine debris, recent researches using artificial intelligence, in particular CNN algorithms have been noticeable (Abbasi & El Hanandeh, 2016; Munari etal., 2016; Azarmi et al, 2018). Here, the Convolutional Neural Network(CNN) has been known as a model that can apply filtering techniques to artificial neural networks to process images or voices more effectively (LeCun et al.,1989) and it is evolving into a number of advanced model algorithms used in Deep Learning (Chen et al., 2018;Raghavendra et al., 2018; Xie et al., 2019).

However, most of the existing researches on marine waste measurement or indexing methods have been conducted in the field of environmental engineering, and it is very difficult to find the previous studies that approach this problem from the perspective of tourism. Furthermore,the mechanism of the Artificial Intelligence, which is divided into the gradual stages of Deep Learning, Machine Learning, and Artificial Intelligence, requires background knowledge of computer or data science. It is considered to be far convoluted for researchers in the social science field to access.

Therefore, this study intends to find an experimental approach that can be applied to the tourism industry. To do this, we label and train the image dataset consists of clean and beautiful views well as dirty ones with plastic debris of Jeju beaches using the CNN method. Through this, we explored the possibility of developing applications or sensors that enable to inform us the cleanliness of the coast. Once the successful deep learning algorithm is developed, it may be useful for the residents and tourists who would like to visit the clean coast of Jeju, and are helpful for the department-in-charge at the provincial government.

2. Literature Review

2.1. Marine Debris and Clean-Coast Index

Marine debris is recognized as a global environmental problem as it flows into the fluid environment of the sea, not only directly affecting neighboring countries, but also spreading throughout streams, rivers and lakes (Paler et al.,2019). Recently, as the seriousness of the effects of micro plastics on marine animals is known, it is urgent to clarify plastic marine garbage such as size, color and material(Baker et al., 2019; Ebere. et al., 2019; Fallati et al., 2019;Fulton et al., 2019; Hartmann et al., 2019). According to local reports in Korea, plastics account for 80% of all marine debris, which takes about 500 years to disintegrate at sea (Ministry of Oceans and Fisheries, 2019) while implementing waste reduction policies(Jian, 2012). If it is not collected and processed, it will adversely affect the environment (Choi et al., 2018; Hartman et al., 2019). This is why many countries such as the United Kingdom, the United States, Japan, and many other European countries,along with China and Australia are fighting against plasticsand establishing new regulations/codes for plastic waste matters (Choi et al., 2018; Lam et al., 2018).

In particular, marine scientists continue to raise the question of the high proportion of plastics in marine debris that threatens the survival of humans as well as marine life(Schulyer et al., 2014; Gall & Thompson, 2015; Paler et al.,2019). Plastic waste flowing to the sea not only threatens the lives of marine animals, but also impairs and damages fishing businesses for humans, and causes accidents for those who visit the sea for recreational purposes. Moreover,the spread of various pathogens to humans is caused by plastic waste (Kim et al., 2010; Campbell et al., 2016;Baker et al., 2019). Indeed, the UNEP has stated in a 2019 report that fishery damage from marine debris would cost about $ 72 million a year in Europe alone and $ 735 million for waste disposal (Baker et al., 2019).

As a part of problem solving, researchers suggest that the flow of plastics from production to distribution,consumption, and recycling through classification should be monitored by labeling according to their material or size(Choi et al., 2018). Furthermore, it is necessary to establish a standard for classification according to size, shape, color,etc. by ranging them from the size, color, and shape including the physical and chemical properties of plastic,and suggest that a basic design that requires consensus by community (Hartmann et al., 2019; Ebere et al., 2019; Paleret al., 2019). In this context, Alkalay et al. (2007) proposed the Clean-Cost Index (CCI) for Israel's Clean Coast Campaign Program, which uses a 10-metre coast as an indicator of cleanliness. Divided horizontally by the meter interval, the index of the plastic bottle cap size (more than 2cm) found was measured as the number of wastes and the index was presented as follows (See Table 1).

Table 1: Clean-Cost Index

Source: Alkalay et al. (2007)

The index derivation formula of the Clean-Cost Index(CCI) is as follows (Paler et al., 2019):

CCI = Total number of plastic items on transect/Total area of transect × K

Since then, the Clean-Cost Index (CCI) has been applied to many studies of observing and classifying plastic marine debris. Munari et al. (2016) studied marine waste components, Terzi and Seyhan (2017) measured seasonal plastic waste density fluctuations, and Ebere et al. (2019) found relatively. It was used as a measuring tool to determine the relationship between the large sizes of macro plastic waste. In the above studies, the more tourists, the more marine debris increased, and plastic waste was the highest among the beach wastes, and there was no significant difference between seasons. The results show that there is a correlation

In addition, in order to cope with the marine debris issues,the Ministry of Maritime Affairs and Fisheries of Korea provides incentives to fishermen for re-collection of marine debris (Research Institute of Ships & Engineering, 2019). However, multi-level measurement activities to identify the present situation should be continually prioritized to inform the hazards of marine debris, especially plastic waste.

Moreover, Jeju Island and its provincial government are facing more burdensome critiques than any other areas in Korea about the marine debris problem as the largest tourist attraction in Korea and its natural characteristic circumstance surrounded by the sea. In the next section, we look into the status and problems of marine debris on Jeju Island.

2.2. Marine Debris of Jeju Island

According to the Jeju tourism statistics, the number of tourists in Jeju, which has shown a sharp rise since 2009,did not slow down drastically even after its peaking in 2015(Chang et al., 2019). Even though the number of Chinese tourists have decreased since then, the number of local tourists who visit Jeju every year increases, reaching over14 million, more than 20 times the number of residents. As of the end of September 2019, the cumulative number of visitors exceeds 11 million so far (Jeju Tourism Association,2019), which is expected to remain at the similar level as the previous year. In addition, the population of Jeju Island,which was about 570,000 in 2010, has increased steadily due to the popularity of the „Living in Jeju Island for one month‟ trend (Chang et al., 2019) which may have been causing the garbage problem.

Korean islands including Jeju often face the problem of waste landfill reaching its limit due to the geographicalcharacteristics. This is due to the lack of land space and the difficulty of finding new landfills due to conflicts of interest between local egoism and municipalities, as well as strict regulations and capacity issues (Koo et al., 2019). Often,the ports, harbors, and coastal areas are unclear subjects of marine garbage disposal, and cases of evading responsibilities often occur. In addition, each local government takes a long time to identify the causes and fundamental solution of the marine garbage problems. There is an urgent need to promptly dispose of marine debris and it has been serious issues on the island. It was reported that around 20,000 tons of garbage werecompressed and shipped out of Jeju Island in 2018 and they revealed that part of the waste had flowed into thePhilippines; embarrassingly more than 1,800 tons of the garbage from Jeju Island still remain in the Philippines(Chon & Choi, 2019). The number of waste has doubled in the past eight years due to the massive influx of tourists and migrants, making it the nation's No. 1 in terms of household waste emissions per capita in 2017, and the amount of compressed waste in the province exceeded 90,000 tons. The JEMA (Jeju Environmental Movement Association)reported it would fill 2.231,800 square meters in February 2019, and it will no longer be landfilled on the island (Park,2019).

In response to the Jeju marine debris problem, which is becoming a major issue, the JEMA criticized that there was practical research has been done (Park, 2019). On the other hand, Kang & Cha (2017) cited the example that the Japanese government intends to promote effective policies by enacting legislation to control the occurrence of marine debris by treating coastal deposits. Therefore, government,local communities and business corporates have redirected the waste management system globally (Hannon et al.,2019).

In this study, we intend to find a way to use Deep Learning technology among various methods to grasp scientific and accurate status before planning/suggesting effective marine debris policies or applications for reducing marine wastes on Jeju Island. In other words, the purpose of this study is to explore the feasibility of Deep Learning algorithm for the clean beach classification based on the volume of plastic debris (wastes) possibly like CCI (Clean Cost Index) one day. The following section introduces the background of the development of Deep Learning technology and the contents of literature reviews on the development of CNN (Convolutional Neural Network) for image classification.

2.3. Deep Learning and Image Classification Using CNN

Today, development of the advanced technology using Deep Learning plays a wide range of roles as the background of Machine Learning, which is one of the main basic operating principles of Artificial Intelligent.Neurophysiologist McCulloch W. and computational neuroscientist Pitts W. originally developed the concept of deep learning using an artificial formula based on the Neurological Threshold Theory Neuron (McCulloch & Pitts,1943). McCarthy J. then began to use the term Artificial Intelligent officially in a workshop at Dartmouth College in the United States in 1956 (Cantu-Ortiz, 2014). It is based on mathematical calculations according to McCarthy(1990).

Since then, the theory of the perceptron hypothetical nervous system, which can predict the learning curve through repetitive learning according to neurological variables, was highlighted (Rosenblatt, 1958). However,this theory based on binary method did not solve the problem of non-classification in the linear form, but only suggested the possibility of repeating several layers ofperceptron, yet it was not definite theory (Minsky & Papert,1969). Later then, Rumelhart, Hinton, and Williams (1986) used the Backpropagation algorithm to train multiple layers of perceptron by adjusting the weights of nodes in multiple layers hidden between the input and output layers of data in the multilayer perceptron. In his later paper, Hinton G. first introduced the concept of Deep Learning called „Learn Deep‟ (Hinton, Osindero, & Teh, 2006).

Since then, several algorithms have been developed to implement Deep Learning, and synthetic neural networks or convolutional neural networks (CNN) have been spotlighted as artificial neural networks recognized for their superiority by being used for visual image and voice analysis (Krizhevsky, Sutskever, & Hinton, 2012; Karpathy et al., 2014). As shown in Figure 1, the process of learning using the CNN model is extracted, the reduced image is repeated to generate a convolution layer and a subsample layer after extracting part of the input data. In this case, the performance of the neural network can be improved by reducing overfitting caused by excessive learning based on the application of the theory (LeCun et al., 1990; Srivastava et al., 2014; Lee, 2018; Park & Bae, 2019).

Figure 1: Architecture of Convolutional Neural Network for Digit Recognition

Source: LeCun et al. (1998)

Meanwhile, the CNN model requires a large amount of data input and has the difficulty of manually adjustinghyperparameters. Therefore, when the data set is large and the layer of learning is deepened, the sigmoid based on logistic regression is used as the activation function. This problem has been raised because the learning disappears due to the slope disappearance of the error backpropagation algorithm. An improvement of this phenomenon is the Rectified Linear Unit (ReLU) function, which is effective in large-scale deep neural networks (Arora, Mianjy, &Mukherjee, 2016; Agarap, 2018; Lee, 2018). On the other hand, using the Support Vector Machine (SVM) algorithm,high classification performance can be obtained even when the training set does not have a common class or the target data set is small (Lee, 2018). The main advantage of the CNN architecture is that it is possible for humans to train through a variety of existing images without directly producing/designing the data needed for image classification training. In addition, the learning time can be cut in half compared to the 'Bag of features', which is a method of extracting and clustering image features (Kyliliet al., 2019). In the next chapter, we show the step-by-step procedure to execute the image classification using„Transfer Learning‟ and pre-trained Xception Model.

3. Convolutional Neural Network (CNN) Composition and Image Classification Prediction

3.1. Method for Dataset & Preprocessing

In this study, we performed a predictive experiment to classify the images of “Clean Beach” and “Polluted Beach”of Jeju Island before measuring the level of pollution on the beach. When normalizing specific datasets, the process of designing each layer and creating a model requires expertise and skills, consuming plenty of time and input data (big data) for training process, therefore it can be very difficult when the data is small. Thus, using a readydesigned model such as Inception-v3, if retrained, an effective normalization result can be achieved in a relatively small amount of input data and in a short time. This retraining of the trained model in other cases is called„Transfer Learning‟ (Lee et al., 2019). The Inception module was located in the middle of the spectrum of regular and depth-wise separable convolutions of transfer learning. To make hypothesis stronger (decoupling between spatial correlation and cross-channel correlation), the Xception network, created by applying depth-wise separable convolution instead of the traditional inception module, has better performance when the number of parameters is similar to that of the existing Inception V3(Chollet, 2017). In this study, a pre-trained CNN model using Xception was used as the text. A user-designated head was created to decode the final function. The Xception V1model that pre-learned weights for ImageNet is as follows(Chollet, 2017).

keras.applications.xception.Xception(include_top=True, eights='imagenet', input_tensor=None, input_shape=None, pooling=None, classes=1000)

Figure 2: ImageNet Validation Accuracy Using Xception

In this experiment, we tried to build a small data set consisting of 194 images of “Clean and Polluted Beaches”to utilize the model. All of these images were crawled by searching for 'Jeju Beach' in 'Google', and searched for clean beach and polluted beach respectively. The key words were 'Jeju Beach, Jeju Clean Beach, Jeju Polluted Beach,and Plastic Waste of Jeju Beach' etc. Google Chrome's extension provides an image batch download program called “Fatkun”. In this study, Fatkun Batch Download Image 5.3.10 was used to download desired images all at once quickly. Overlapping images and images from nonJeju islands were filtered and excluded, and labeled 97 images from 0 to 96 and stored in each „train‟ folder. The validation set (folder) contained a total of 20 images (10 per class) and the test set (folder) contained 5 per class, a total of 10 shuffled images. The image set for train and validation goes through image preprocessing and augmentation, and images in the test set were used only at the final stage to evaluate the normalization of the model. The pre-trained model used a 299x299x3 image input and outputs a 2048x1-feature vector after global full average pooling. Then we added 10 units of fully connected layersand 1 unit of output layers for binary classification. L2 normalization was used at both layers to prevent overfitting. Retraining was done using Tensorflow with Keras as backend.

Figure 3: Diagrams from Clean and Polluted Beach Test

Image preprocessing allowed each image to be 299x299square by using OpenCV-Python. The processed images are stored in a numpy array in `polluted_beach_dataset.py` and then normalized to N (0, 1) for each channel. We then zoom in on a set of 194 training data images (97 clean shore and trash-contaminated data each), applying 20 epochs by randomly rotating, moving, skewing, zooming or flipping the images to be trained (194x20= 3,880 trainable image data. This pre-trained model can be used to extract feature vectors from augmented data (aug_and_feature_extract.py). In this way, we did not need to carry out real-time augmentation, iteratively calculate the entire model,computing for the two layer heads later. The source of features to perform every dataset tasked in Python in this paper using Jupyter Notebook, an interactive data visualization tool under online environment.

3.2. Training and Prediction of Datasets

Through the above practice, we have x_train “array expansion and feature extraction (194 x 20 = 3,880)”followed by an array of 3,880x2,048, y_train is a vector of3,880x1 and 194 datasets with 3,880 in order to expand them as observable datasets. When fitting a model to avoid the possibility of selecting more than one image with the same origin in the same batch, you can create a random image that is different from the original 194 images without mixing the training set. Adam is an active function used for optimizations with a basic learning rate of 1e-3, and dichotomous cross-entropy is used as a loss function. In addition, using 0.01 as the weight loss for L2 normalization,user-defined heads are stored in the model directory. As shown in Figure 4, the results of the pre-trained test set were 85% (0.85), 70% (0.70) and 86% (0.86). It can be seen that the accuracy has increased through the process. We can then see a rapid convergence of 97.73% and 100%(20/20) accuracy in the actual training and validation sets,respectively.

Figure 4: Training and Test Accuracy from Clean and Polluted Beach Test

Subsequently, the confusion matrix (see Figure 5) shows the accuracy of the experiment at a glance. In addition, the classification result of training and prediction (see Figure 6& 6) shows that error and loss are reduced in both training and verification set. Finally, the results of classifying 10 randomly given images are shown in Figure 8.

Figure 5: Confusion Matrix for Training and Test Set

Figure 6: Training Result (Validation Accuracies) of Augmented Dataset

Figure 7: Training Result of Polluted Beach Classifier

Figure 8: Final Predictions on Test Images

The 10 images in the test folder were used as a set of data(images) that were never presented during training and validation to assess the model's normalization probability(accuracy), and matched the probability of Polluted Beach almost correct as shown in Figure 8. In other words, the sizes of debris on the image #3 with Polluted Prob 0.39(mistakenly recognized as a clean beach) is contrasted with the sizes of the garbage in the image # 6 (Polluted Prob 0.74). Based on such experience, we found that there are new questions are to be solved in the future whether to sort marine waste according to their sizes or by quantity.

4. Conclusion and Implication

Since the WTO began discussing the concept of sustainable tourism in 1993, the desire to find alternatives to mass tourism around the world had initiated (Carlo & Ko,2018). Since then, there has been a growing voice of concern about overtourism related to the environmental issues such as global warming as the number of tourists has increased due to the economic development and the advancement of the aviation industry. The extent of the political, economic, social, cultural and environmental impacts of the tourism business in different countries will vary, and interests in each industry are intertwined, making it difficult to state the importance of tourism in each country. However, it is hard to deny that Jeju is the largest tourist‟s destinations in Korea that needs universal discussion with environmental concern (Chang et al., 2019;Choi et al., 2019). Recently, moreover, the damage caused by marine plastic waste has been frequently reported in the press and media in Jeju province (Chon & Choi, 2019; Park,2019). This is the time when national awareness such as tourism subjects (local residents) and objects (tourists), as well as tourist companies and organizations is needed before it gets too late.

The purpose of this study is to find out the solution of marine debris caused by the increase of resident population including tourists on Jeju Island from an environmental and tourism perspective. Deep learning, a kind of Artificial Intelligence-based technology, is a field of computer Machine Learning at the computer science department. An automated technology identifies the object of interest in the image by using the principle of probability to distinguish it from its background view. In other words, we needed only a small number of images in this study as an experiment,using the Convolutional Neural Network algorithm most commonly used for image classification techniques (LeCunet al., 1990; Srivastava et al., 2014; Lee, 2018; Park & Bae,2019) and continuous upgraded active functions to identify images of dirty Jeju coast with marine debris. We tried to test the accuracy performance, then, we got the accuracy of test and verification data was 80% and 86% by logistic regression respectively. After analyzing the image data set consisting of 194 training data-20 verification data-10 test data.

As we discussed and performed the test, Deep Learning technology of reading and classifying images of photographs or videos will find a variety of applications for the Jeju tourism industry in the near future. Jeju Island is an environment where all slopes are close to the beautiful seaside. In particular, over 20 beaches along 258km throughout the island are adjacent to the walking trail called'Olle', making it a popular spot for tourists regardless of the season. We expect that recent photos posted on social media by visitors (or nearby merchants) to a particular beach or sea side can be categorized by Deep Learning algorithm and used in an indicator app that shows the cleanliness of the Jeju coast. For businesses or management offices near the coast, they will try to remove waste(especially plastic marine debris) to get rid of the stigma of a less clean coast. In addition, it will provide tourists with real-time guidance on high-purity beaches, and it can be an application that invites tourists to participate in reducing marine debris. On the other hand, if this function is combined with a closed-circuit camera (CCTV) on the coast, it will be possible that the real-time image transmission and classification model can be used to create a meaningful task of 'maintaining and managing the clean coast on Jeju Island'.

Although the CCI (Clean Cost Index) has classified and scaled coastal cleanliness in five grades (Alkalay et al.,2007), measuring plastic waste on a wide range of shores in real time is a time-and labor-intensive task. On the other hand, image classification using Deep Learning technology will be able to develop into an app that anyone can participate by taking a picture/video and uploading it in real time as well as utilizing devices such as smartphones andCCTVs. Until recently, most of the studies that applied CNN algorithms to marine debris detection were carried by researchers from earth or environmental science and engineering department, and using the UAV(Unmanned Aerial Vehicle), known as drones (Tran, 2018; Bak et al.,2019; Fallati et al. , 2019; Fulton et al., 2019).

It is worthwhile that this study may be the very first door opener to propose a new way to study Jeju tourism by suggesting feasible ideas to reduce marine debris through an experiment using Deep Learning algorithms that can anticipate accuracy even with small image datasets using Transfer Learning (Lee et al., 2019) and Xception Model(Chollet, 2017). If upgraded functions with stronger accuracy is developed in the future, it is expected that the advancement of Deep Learning-based CCI (Clean Cost Index) will enable to utilize in the market. In addition, we would like to reiterate that we need endless efforts to understand and develop Deep Learning algorithms, since there are no existing studies comparable to this study in practical tourism dimension.