A Two-Stage Learning Method of CNN and K-means RGB Cluster for Sentiment Classification of Images (이미지 감성분류를 위한 CNN과 K-means RGB Cluster 이-단계 학습 방안)
-
- Journal of Intelligence and Information Systems
- /
- v.27 no.3
- /
- pp.139-156
- /
- 2021
The biggest reason for using a deep learning model in image classification is that it is possible to consider the relationship between each region by extracting each region's features from the overall information of the image. However, the CNN model may not be suitable for emotional image data without the image's regional features. To solve the difficulty of classifying emotion images, many researchers each year propose a CNN-based architecture suitable for emotion images. Studies on the relationship between color and human emotion were also conducted, and results were derived that different emotions are induced according to color. In studies using deep learning, there have been studies that apply color information to image subtraction classification. The case where the image's color information is additionally used than the case where the classification model is trained with only the image improves the accuracy of classifying image emotions. This study proposes two ways to increase the accuracy by incorporating the result value after the model classifies an image's emotion. Both methods improve accuracy by modifying the result value based on statistics using the color of the picture. When performing the test by finding the two-color combinations most distributed for all training data, the two-color combinations most distributed for each test data image were found. The result values were corrected according to the color combination distribution. This method weights the result value obtained after the model classifies an image's emotion by creating an expression based on the log function and the exponential function. Emotion6, classified into six emotions, and Artphoto classified into eight categories were used for the image data. Densenet169, Mnasnet, Resnet101, Resnet152, and Vgg19 architectures were used for the CNN model, and the performance evaluation was compared before and after applying the two-stage learning to the CNN model. Inspired by color psychology, which deals with the relationship between colors and emotions, when creating a model that classifies an image's sentiment, we studied how to improve accuracy by modifying the result values based on color. Sixteen colors were used: red, orange, yellow, green, blue, indigo, purple, turquoise, pink, magenta, brown, gray, silver, gold, white, and black. It has meaning. Using Scikit-learn's Clustering, the seven colors that are primarily distributed in the image are checked. Then, the RGB coordinate values of the colors from the image are compared with the RGB coordinate values of the 16 colors presented in the above data. That is, it was converted to the closest color. Suppose three or more color combinations are selected. In that case, too many color combinations occur, resulting in a problem in which the distribution is scattered, so a situation fewer influences the result value. Therefore, to solve this problem, two-color combinations were found and weighted to the model. Before training, the most distributed color combinations were found for all training data images. The distribution of color combinations for each class was stored in a Python dictionary format to be used during testing. During the test, the two-color combinations that are most distributed for each test data image are found. After that, we checked how the color combinations were distributed in the training data and corrected the result. We devised several equations to weight the result value from the model based on the extracted color as described above. The data set was randomly divided by 80:20, and the model was verified using 20% of the data as a test set. After splitting the remaining 80% of the data into five divisions to perform 5-fold cross-validation, the model was trained five times using different verification datasets. Finally, the performance was checked using the test dataset that was previously separated. Adam was used as the activation function, and the learning rate was set to 0.01. The training was performed as much as 20 epochs, and if the validation loss value did not decrease during five epochs of learning, the experiment was stopped. Early tapping was set to load the model with the best validation loss value. The classification accuracy was better when the extracted information using color properties was used together than the case using only the CNN architecture.
This paper focuses on 'clean(cheong: 淸)' kinds of style terminologies among various style terminologies appearing in Heo Gyun's Seongsushihwa("惺?詩話") and tries to analyze the distinctive points which 'clean(cheong: 淸)' kinds of style terminologies include. In Heo Gyun's Seongsushihwa, 11 of 'clean' kinds of style terminologies, such as "cheonggyeong(淸勁), cheonghryang(淸亮), cheongryeo(淸麗), cheongseom(淸贍), cheongso(淸?), cheongweol(淸越), cheongjang(淸壯), cheongjeol(淸絶), cheongjeol(淸切), cheongchang(淸?), cheongcho(淸楚)," were used. This paper focuses and analyzes 'cheonggyeong(淸勁)', 'cheongjeol(淸切)', 'cheongcho(淸楚)', and 'cheongweol(淸越)' that he suggested through applying to real literary pieces. The result of analysis indicates that 'clean' kinds of style terminologies 'cheonggyeong', 'cheongjeol', 'cheongcho', and 'cheongweol' share the same 1st character 'clean(淸)', yet have distinctive qualities by the 2nd characters. These 4 style terminologies all share 'cheong(淸)' image which means clear and clean, yet each one has the attribute of the 2nd character that indicates each one's individual characteristic. It is apparent that 'Cheonggyeong(淸勁)' reflects the 'gyeong(勁)' image meaning upright and solid and implies poems of poets' steadfast spirit within clear boundary; 'cheongjeol(淸切)' reflects the 'jeol(切)' image meaning either desperation and imminence or pitifulness and sorrow and implies poems of poets' urgent and pitiful emotions within clear and clean boundary; 'cheongcho(淸楚)' reflects the 'cho(楚)' image meaning either delicacy and fineness or slenderness and tenderness and implies poems of poets' beautiful but not luxurious, delicate and tender emotions within clear and clean boundary; and 'cheongweol(淸越)' reflects the image of 'weol(越)' meaning unworldliness and excellency and implies poems, within clear and clean boundary, of excellent appearance and mentality surpassing mundane world. Compared with the 1st character's attributes of the style terminologies which Heo Gyun used, the 2nd characters's attributes do not appear that vivid. Especially, in the case that the 2nd characters have similar meanings, it is not easy to clarify the categories. Indeed, in order to grasp clear and distinctive qualities of style terminologies, the kinds of them need to be initially categorized by the 1st characters, and then sorted by the 2nd characters. In this case, the contents which the 2nd characters of style terminologies indicate should be considered. It is because style terminologies explain both literary pieces' aesthetic qualities and writers' personalities, and because explanations about literary pieces' aesthetic qualities includes not only the conclusive poetic or semantic boundaries which literary pieces' created but also literary pieces' creation processes and expression techniques. Through the style terminologies with Heo Gyun used in Seongsushihwa, it can be aware that he evaluated poems focussing more on the conclusive semantic boundaries that poets' spirits and poems created than expression techniques or creation methods. The overall aspects Heo Gyun's such style criticism has will be checked out in more detail through further studies by examining more materials.
From January 2020 to October 2021, more than 500,000 academic studies related to COVID-19 (Coronavirus-2, a fatal respiratory syndrome) have been published. The rapid increase in the number of papers related to COVID-19 is putting time and technical constraints on healthcare professionals and policy makers to quickly find important research. Therefore, in this study, we propose a method of extracting useful information from text data of extensive literature using LDA and Word2vec algorithm. Papers related to keywords to be searched were extracted from papers related to COVID-19, and detailed topics were identified. The data used the CORD-19 data set on Kaggle, a free academic resource prepared by major research groups and the White House to respond to the COVID-19 pandemic, updated weekly. The research methods are divided into two main categories. First, 41,062 articles were collected through data filtering and pre-processing of the abstracts of 47,110 academic papers including full text. For this purpose, the number of publications related to COVID-19 by year was analyzed through exploratory data analysis using a Python program, and the top 10 journals under active research were identified. LDA and Word2vec algorithm were used to derive research topics related to COVID-19, and after analyzing related words, similarity was measured. Second, papers containing 'vaccine' and 'treatment' were extracted from among the topics derived from all papers, and a total of 4,555 papers related to 'vaccine' and 5,971 papers related to 'treatment' were extracted. did For each collected paper, detailed topics were analyzed using LDA and Word2vec algorithms, and a clustering method through PCA dimension reduction was applied to visualize groups of papers with similar themes using the t-SNE algorithm. A noteworthy point from the results of this study is that the topics that were not derived from the topics derived for all papers being researched in relation to COVID-19 (