• Journal of The Korean Association of Information Education
• /
• v.26 no.3
• /
• pp.197-207
• /
• 2022

In this paper, we develop a machine learning based block code generation and recommendation model for the purpose of reducing cognitive load of learners during coding education that learns the learner's block that has been made in the block programming environment using natural processing model and fine-tuning and then generates and recommends the selectable blocks for the next step. To develop the model, the training dataset was produced by pre-processing 50 block codes that were on the popular block programming language web site 'Entry'. Also, after dividing the pre-processed blocks into training dataset, verification dataset and test dataset, we developed a model that generates block codes based on LSTM, Seq2Seq, and GPT-2 model. In the results of the performance evaluation of the developed model, GPT-2 showed a higher performance than the LSTM and Seq2Seq model in the BLEU and ROUGE scores which measure sentence similarity. The data results generated through the GPT-2 model, show that the performance was relatively similar in the BLEU and ROUGE scores except for the case where the number of blocks was 1 or 17.

• Journal of Intelligence and Information Systems
• /
• v.28 no.4
• /
• pp.309-328
• /
• 2022

This study empirically analyzed a Korean pre-trained language models (PLMs) designed for natural language generation. The performance of two PLMs - BART and GPT - at the task of abstractive text summarization was compared. To investigate how performance depends on the characteristics of the inference data, ten different document types, containing six types of informational content and creation content, were considered. It was found that BART (which can both generate and understand natural language) performed better than GPT (which can only generate). Upon more detailed examination of the effect of inference data characteristics, the performance of GPT was found to be proportional to the length of the input text. However, even for the longest documents (with optimal GPT performance), BART still out-performed GPT, suggesting that the greatest influence on downstream performance is not the size of the training data or PLMs parameters but the structural suitability of the PLMs for the applied downstream task. The performance of different PLMs was also compared through analyzing parts of speech (POS) shares. BART's performance was inversely related to the proportion of prefixes, adjectives, adverbs and verbs but positively related to that of nouns. This result emphasizes the importance of taking the inference data's characteristics into account when fine-tuning a PLMs for its intended downstream task.

• Journal of the Korea Convergence Society
• /
• v.12 no.11
• /
• pp.35-43
• /
• 2021

Recently, there has been a growing interest in zero-shot cross-lingual transfer, which leverages cross-lingual language models (CLLMs) to perform downstream tasks that are not trained in a specific language. In this paper, we point out the limitations of the data-centric aspect of quality estimation (QE), and perform zero-shot cross-lingual transfer even in environments where it is difficult to construct QE data. Few studies have dealt with zero-shots in QE, and after fine-tuning the English-German QE dataset, we perform zero-shot transfer leveraging CLLMs. We conduct comparative analysis between various CLLMs. We also perform zero-shot transfer on language pairs with different sized resources and analyze results based on the linguistic characteristics of each language. Experimental results showed the highest performance in multilingual BART and multillingual BERT, and we induced QE to be performed even when QE learning for a specific language pair was not performed at all.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2022.05a
• /
• pp.399-401
• /
• 2022

A definitive diagnosis to classify the breast malignancy status may be achieved by microscopic analysis using surgical open biopsy. However, this procedure requires experts in the specializing of histopathological image analysis directing to time-consuming and high cost. To overcome these issues, deep learning is considered practically efficient to categorize breast cancer into benign and malignant from histopathological images in order to assist pathologists. This study presents a pre-trained convolutional neural network model architecture with a 100% fine-tuning scheme and Adagrad optimizer to classify the breast cancer histopathological images into benign and malignant using a 40× magnification BreaKHis dataset. The pre-trained architecture was constructed using the InceptionResNetV2 model to generate a modified InceptionResNetV2 by substituting the last layer with dense and dropout layers. The results by demonstrating training loss of 0.25%, training accuracy of 99.96%, validation loss of 3.10%, validation accuracy of 99.41%, test loss of 8.46%, and test accuracy of 98.75% indicated that the modified InceptionResNetV2 model is reliable to predict the breast malignancy type from histopathological images. Future works are necessary to focus on k-fold cross-validation, optimizer, model, hyperparameter optimization, and classification on 100×, 200×, and 400× magnification.

• Journal of the Korean Society of Marine Environment & Safety
• /
• v.29 no.6
• /
• pp.543-551
• /
• 2023

As climate change continues to prompt an increasing demand for advancements in disaster and safety management technologies to address abnormal high water temperatures, typhoons, floods, and droughts, sea surface temperature has emerged as a pivotal factor for swiftly assessing the impacts of summer harmful algal blooms in the seas surrounding Korean Peninsula and the formation and dissipation of cold water along the East Coast of Korea. Therefore, this study sought to gauge predictive performance by leveraging statistical methods and deep learning algorithms to harness sea surface temperature data effectively for marine anomaly research. The sea surface temperature data employed in the predictions spans from 2018 to 2022 and originates from the Heuksando Tidal Observatory. Both traditional statistical ARIMA methods and advanced deep learning models, including long short-term memory (LSTM) and gated recurrent unit (GRU), were employed. Furthermore, prediction performance was evaluated using the attention LSTM technique. The technique integrated an attention mechanism into the sequence-to-sequence (s2s), further augmenting the performance of LSTM. The results showed that the attention LSTM model outperformed the other models, signifying its superior predictive performance. Additionally, fine-tuning hyperparameters can improve sea surface temperature performance.

• Journal of Intelligence and Information Systems
• /
• v.29 no.4
• /
• pp.129-164
• /
• 2023

In a situation where the use and introduction of Large Language Models (LLMs) is expanding due to recent developments in generative AI technology, it is difficult to find actual application cases or implementation methods for the use of internal company data in existing studies. Accordingly, this study presents a method of implementing generative AI services using the LLM application architecture using the most widely used LangChain framework. To this end, we reviewed various ways to overcome the problem of lack of information, focusing on the use of LLM, and presented specific solutions. To this end, we analyze methods of fine-tuning or direct use of document information and look in detail at the main steps of information storage and retrieval methods using the retrieval augmented generation (RAG) model to solve these problems. In particular, similar context recommendation and Question-Answering (QA) systems were utilized as a method to store and search information in a vector store using the RAG model. In addition, the specific operation method, major implementation steps and cases, including implementation source and user interface were presented to enhance understanding of generative AI technology. This has meaning and value in enabling LLM to be actively utilized in implementing services within companies.

• Informatization Policy
• /
• v.31 no.2
• /
• pp.3-38
• /
• 2024

Hallucination is a significant barrier to the utilization of large-scale language models or multimodal models. In this study, we collected 654 computer science papers with "hallucination" in the abstract from arXiv from December 2022 to January 2024 following the advent of Chat GPT and conducted frequency analysis, knowledge network analysis, and literature review to explore the latest trends in hallucination research. The results showed that research in the fields of "Computation and Language," "Artificial Intelligence," "Computer Vision and Pattern Recognition," and "Machine Learning" were active. We then analyzed the research trends in the four major fields by focusing on the main authors and dividing them into data, hallucination detection, and hallucination mitigation. The main research trends included hallucination mitigation through supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF), inference enhancement via "chain of thought" (CoT), and growing interest in hallucination mitigation within the domain of multimodal AI. This study provides insights into the latest developments in hallucination research through a technology-oriented literature review. This study is expected to help subsequent research in both engineering and humanities and social sciences fields by understanding the latest trends in hallucination research.

• The Journal of Bigdata
• /
• v.9 no.1
• /
• pp.75-88
• /
• 2024

Recently, there have been cases reported in the news of individuals experiencing symptoms of food poisoning after consuming raw beef purchased from online platforms, or reviews claiming that cherry tomatoes tasted bitter. This suggests the potential for analyzing food reviews on online platforms to detect food hazards, enabling government agencies, food manufacturers, and distributors to manage consumer food safety risks. This study proposes a classification model that uses sentiment analysis and large language models to analyze food reviews and detect negative ones, multi-labeling key food safety hazards (food poisoning, spoilage, chemical odors, foreign objects). The sentiment analysis model effectively minimized the misclassification of negative reviews with a low False Positive rate using a 'funnel' model. The multi-labeling model for food safety hazards showed high performance with both recall and accuracy over 96% when using GPT-4 Turbo compared to GPT-3.5. Government agencies, food manufacturers, and distributors can use the proposed model to monitor consumer reviews in real-time, detect potential food safety issues early, and manage risks. Such a system can protect corporate brand reputation, enhance consumer protection, and ultimately improve consumer health and safety.

• Journal of the Korean Geographical Society
• /
• v.36 no.4
• /
• pp.474-482
• /
• 2001

The purpose of this study is to investigate the formation and development processes of beach resort by fabric construction in a island environment. The results are as follows. (1) The research area(Tong-ri beach, Bokil-myon, Chollanam-do)has been transformed to belch by sedimentary environmental change since latter half of 1800's. (2) The mean slope of beach face is 0.96°, and the difference of attitude between beach and mud flat face is 75cm. (3) The mean particle size of beach surface sediment is 3.53$\Phi$. This value is very finer than that of any other beach in Korea peninsula. But its value is coarser than that of mud flat surface sediment. (4) The particle size distribution of core sediment is become changed to fine particle in 70cm depth. This value is corresponded to difference of altitude between beach face and mud flat face. (5) The analysis of aerial photographs after 1970 indicates that sedimentation process was not brisked since 1970's. Consequently, the research ares has been developed by sedimentary environmental change for sea-level rise effect and wave height energy rise effect.

• Journal of Korea Water Resources Association
• /
• v.57 no.7
• /
• pp.437-449
• /
• 2024

Rainfall-runoff prediction studies using deep learning while considering catchment attributes have been gaining attention. In this study, we selected two models: the Transformer model, which is suitable for large-scale data training through the self-attention mechanism, and the LSTM-based multi-state-vector sequence-to-sequence (LSTM-MSV-S2S) model with an encoder-decoder structure. These models were constructed to incorporate catchment attributes and predict the inflow of 10 multi-purpose dam watersheds in South Korea. The experimental design consisted of three training methods: Single-basin Training (ST), Pretraining (PT), and Pretraining-Finetuning (PT-FT). The input data for the models included 10 selected watershed attributes along with meteorological data. The inflow prediction performance was compared based on the training methods. The results showed that the Transformer model outperformed the LSTM-MSV-S2S model when using the PT and PT-FT methods, with the PT-FT method yielding the highest performance. The LSTM-MSV-S2S model showed better performance than the Transformer when using the ST method; however, it showed lower performance when using the PT and PT-FT methods. Additionally, the embedding layer activation vectors and raw catchment attributes were used to cluster watersheds and analyze whether the models learned the similarities between them. The Transformer model demonstrated improved performance among watersheds with similar activation vectors, proving that utilizing information from other pre-trained watersheds enhances the prediction performance. This study compared the suitable models and training methods for each multi-purpose dam and highlighted the necessity of constructing deep learning models using PT and PT-FT methods for domestic watersheds. Furthermore, the results confirmed that the Transformer model outperforms the LSTM-MSV-S2S model when applying PT and PT-FT methods.