• The Journal of the Korea institute of electronic communication sciences
• /
• v.18 no.4
• /
• pp.693-700
• /
• 2023

Sentiment analysis, a branch of natural language processing that classifies and identifies subjective opinions and emotions in text documents as positive or negative, can be used for various promotions and services through customer preference analysis. To this end, recent research has been conducted utilizing various techniques in machine learning and deep learning. In this study, we propose an optimal language model by comparing the accuracy of sentiment analysis for movie, product, and game reviews using existing RNN-based models and recent Transformer-based language models. In our experiments, LMKorBERT and GPT3 showed relatively good accuracy among the models pre-trained on the Korean corpus.

• Electronics and Telecommunications Trends
• /
• v.35 no.3
• /
• pp.9-19
• /
• 2020

Recently, a technique for applying a deep learning language model pretrained from a large corpus to fine-tuning for each application task has been widely used as a language processing technology. The pretrained language model shows higher performance and satisfactory generalization performance than existing methods. This paper introduces the major research trends related to deep learning pretrained language models in the field of language processing. We describe in detail the motivations, models, learning methods, and results of the BERT language model that had significant influence on subsequent studies. Subsequently, we introduce the results of language model studies after BERT, focusing on SpanBERT, RoBERTa, ALBERT, BART, and ELECTRA. Finally, we introduce the KorBERT pretrained language model, which shows satisfactory performance in Korean language. In addition, we introduce techniques on how to apply the pretrained language model to Korean (agglutinative) language, which consists of a combination of content and functional morphemes, unlike English (refractive) language whose endings change depending on the application.

• The Bulletin of The Korean Astronomical Society
• /
• v.44 no.1
• /
• pp.82.3-82.3
• /
• 2019

In this study, we generate future full disk magnetograms in 12, 24, 36 and 48 hours advance from SDO/HMI images using deep learning. To perform this generation, we apply the convolutional generative adversarial network (cGAN) algorithm to a series of SDO/HMI magnetograms. We use SDO/HMI data from 2011 to 2016 for training four models. The models make AI-generated images for 2017 HMI data and compare them with the actual HMI magnetograms for evaluation. The AI-generated images by each model are very similar to the actual images. The average correlation coefficient between the two images for about 600 data sets are about 0.85 for four models. We are examining hundreds of active regions for more detail comparison. In the future we will use pix2pix HD and video2video translation networks for image prediction.

• Journal of Korea Water Resources Association
• /
• v.54 no.5
• /
• pp.301-309
• /
• 2021

In this study, we tried to improve the performance of the existing U-net-based deep learning rainfall prediction model, which can weaken the meaning of time series order. For this, ConvLSTM2D U-Net structure model considering temporal consistency of data was applied, and we evaluated accuracy of the ConvLSTM2D U-Net model using a RainNet model and an extrapolation-based advection model. In addition, we tried to improve the uncertainty in the model training process by performing learning not only with a single model but also with 10 ensemble models. The trained neural network rainfall prediction model was optimized to generate 10-minute advance prediction data using four consecutive data of the past 30 minutes from the present. The results of deep learning rainfall prediction models are difficult to identify schematically distinct differences, but with ConvLSTM2D U-Net, the magnitude of the prediction error is the smallest and the location of rainfall is relatively accurate. In particular, the ensemble ConvLSTM2D U-Net showed high CSI, low MAE, and a narrow error range, and predicted rainfall more accurately and stable prediction performance than other models. However, the prediction performance for a specific point was very low compared to the prediction performance for the entire area, and the deep learning rainfall prediction model also had limitations. Through this study, it was confirmed that the ConvLSTM2D U-Net neural network structure to account for the change of time could increase the prediction accuracy, but there is still a limitation of the convolution deep neural network model due to spatial smoothing in the strong rainfall region or detailed rainfall prediction.

• The Transactions of The Korean Institute of Electrical Engineers
• /
• v.67 no.5
• /
• pp.669-674
• /
• 2018

Data augmentation is an efficient way to reduce overfitting on models and to improve a performance supplementing extra data for training. It is more important in deep learning based industrial machine vision. Because deep learning requires huge scale of learning data to learn a model, but acquisition of data can be limited in most of industrial applications. A very generic method for augmenting image data is to perform geometric transformations, such as cropping, rotating, translating and adjusting brightness of the image. The effectiveness of data augmentation in image classification has been reported, but it is rare in defect inspections. We explore and compare various basic augmenting operations for the metal surface defects. The experiments were executed for various types of defects and different CNN networks and analysed for performance improvements by the data augmentations.

• Journal of Information Processing Systems
• /
• v.17 no.2
• /
• pp.385-398
• /
• 2021

The complexity of deep learning models affects the real-time performance of gesture recognition, thereby limiting the application of gesture recognition algorithms in actual scenarios. Hence, a residual learning neural network based on a deep convolutional neural network is proposed. First, small convolution kernels are used to extract the local details of gesture images. Subsequently, a shallow residual structure is built to share weights, thereby avoiding gradient disappearance or gradient explosion as the network layer deepens; consequently, the difficulty of model optimisation is simplified. Additional convolutional neural networks are used to accelerate the refinement of deep abstract features based on the spatial importance of the gesture feature distribution. Finally, a fully connected cascade softmax classifier is used to complete the gesture recognition. Compared with the dense connection multiplexing feature information network, the proposed algorithm is optimised in feature multiplexing to avoid performance fluctuations caused by feature redundancy. Experimental results from the ISOGD gesture dataset and Gesture dataset prove that the proposed algorithm affords a fast convergence speed and high accuracy.

• Proceedings of the Korean Institute of Building Construction Conference
• /
• 2022.04a
• /
• pp.73-74
• /
• 2022

Concrete structures occupy the largest proportion of modern infrastructure, and concrete structures often have cracking problems. Existing concrete crack diagnosis methods have limitations in crack evaluation because they rely on expert visual inspection. Therefore, in this study, we design a deep learning model that detects, visualizes, and outputs cracks on the surface of RC structures based on image data by using a CNN (Convolution Neural Networks) model that can process two- and three-dimensional data such as video and image data. do. An experimental study was conducted on an algorithm to automatically detect concrete cracks and visualize them using a CNN model. For the three deep learning models used for algorithm learning in this study, the concrete crack prediction accuracy satisfies 90%, and in particular, the 'InceptionV3'-based CNN model showed the highest accuracy. In the case of the crack detection visualization model, it showed high crack detection prediction accuracy of more than 95% on average for data with crack width of 0.2 mm or more.

• Journal of Korean Society of Water and Wastewater
• /
• v.35 no.1
• /
• pp.83-91
• /
• 2021

The increased turbidity in rivers during flood events has various effects on water environmental management, including drinking water supply systems. Thus, prediction of turbid water is essential for water environmental management. Recently, various advanced machine learning algorithms have been increasingly used in water environmental management. Ensemble machine learning algorithms such as random forest (RF) and gradient boosting decision tree (GBDT) are some of the most popular machine learning algorithms used for water environmental management, along with deep learning algorithms such as recurrent neural networks. In this study GBDT, an ensemble machine learning algorithm, and gated recurrent unit (GRU), a recurrent neural networks algorithm, are used for model development to predict turbidity in a river. The observation frequencies of input data used for the model were 2, 4, 8, 24, 48, 120 and 168 h. The root-mean-square error-observations standard deviation ratio (RSR) of GRU and GBDT ranges between 0.182~0.766 and 0.400~0.683, respectively. Both models show similar prediction accuracy with RSR of 0.682 for GRU and 0.683 for GBDT. The GRU shows better prediction accuracy when the observation frequency is relatively short (i.e., 2, 4, and 8 h) where GBDT shows better prediction accuracy when the observation frequency is relatively long (i.e. 48, 120, 160 h). The results suggest that the characteristics of input data should be considered to develop an appropriate model to predict turbidity.

• The Journal of Bigdata
• /
• v.7 no.2
• /
• pp.11-29
• /
• 2022

Pre-trained language models are the most important and widely used tools in natural language processing tasks. Since those have been pre-trained for a large amount of corpus, high performance can be expected even with fine-tuning learning using a small number of data. Since the elements necessary for implementation, such as a pre-trained tokenizer and a deep learning model including pre-trained weights, are distributed together, the cost and period of natural language processing has been greatly reduced. Transformer variants are the most representative pre-trained language models that provide these advantages. Those are being actively used in other fields such as computer vision and audio applications. In order to make it easier for researchers to understand the pre-trained language model and apply it to natural language processing tasks, this paper describes the definition of the language model and the pre-learning language model, and discusses the development process of the pre-trained language model and especially representative Transformer variants.

• Journal of Korean Society of Coastal and Ocean Engineers
• /
• v.34 no.3
• /
• pp.72-81
• /
• 2022

Positions of five drifting buoys deployed on August 2020 near southwestern area of Jeju Island and numerically predicted velocities were used to develop five Artificial Intelligence-based models (AI models) for the prediction of particle tracks. Five AI models consisted of three machine learning models (Extra Trees, LightGBM, and Support Vector Machine) and two deep learning models (DNN and RBFN). To evaluate the prediction accuracy for six models, the predicted positions from five AI models and one numerical model were compared with the observed positions from five drifting buoys. Three skills (MAE, RMSE, and NCLS) for the five buoys and their averaged values were calculated. DNN model showed the best prediction accuracy in MAE, RMSE, and NCLS.