• Proceedings of the Korean Institute of Building Construction Conference
• /
• 2023.11a
• /
• pp.237-238
• /
• 2023

Urbanization and the increase in building scale have amplified the complexity of M.E.P design. Traditional design methods face limitations when considering intricate pathways and variables, leading to an emergent need for research in automated design. Initial algorithmic approaches encountered challenges in addressing complex architectural structures and the diversity of M.E.P types. However, with the launch of OpenAI's ChatGPT-3.5 beta version in 2022, new opportunities in the automated design sector were unlocked. ChatGPT, based on the Large Language Model (LLM), has the capability to deeply comprehend the logical structures and meanings within training data. This study analyzed the potential application and latent value of LLMs in M.E.P design. Ultimately, the implementation of LLM in M.E.P design will make genuine automated design feasible, which is anticipated to drive advancements across designs in the construction sector.

• Journal of Internet Computing and Services
• /
• v.25 no.1
• /
• pp.57-67
• /
• 2024

This research investigates training methods for large language models to accurately identify sentiments and comprehend information about increasing and decreasing fluctuations in the financial domain. The main goal is to identify suitable datasets that enable these models to effectively understand expressions related to financial increases and decreases. For this purpose, we selected sentences from Wall Street Journal that included relevant financial terms and sentences generated by GPT-3.5-turbo-1106 for post-training. We assessed the impact of these datasets on language model performance using Financial PhraseBank, a benchmark dataset for financial sentiment analysis. Our findings demonstrate that post-training FinBERT, a model specialized in finance, outperformed the similarly post-trained BERT, a general domain model. Moreover, post-training with actual financial news proved to be more effective than using generated sentences, though in scenarios requiring higher generalization, models trained on generated sentences performed better. This suggests that aligning the model's domain with the domain of the area intended for improvement and choosing the right dataset are crucial for enhancing a language model's understanding and sentiment prediction accuracy. These results offer a methodology for optimizing language model performance in financial sentiment analysis tasks and suggest future research directions for more nuanced language understanding and sentiment analysis in finance. This research provides valuable insights not only for the financial sector but also for language model training across various domains.

• Journal of the Korea Society of Computer and Information
• /
• v.27 no.5
• /
• pp.109-116
• /
• 2022

Recently, a model-based development method centered on highly readable and standardized UML (Unified Modeling Language) models has been applied to solve unclear communications in large-scale software development. However, it is difficult to apply consistent UML models depending on software developers' proficiency, understanding of models and modeling tools. In this paper, we propose a method for developing a Model Verification System to apply an consistent UML model to software development. Then, the developed Model Verification System is partially applied to the Naval Combat System Software development to prove its function. The Model Verification System provides automatic verification of models created by developers according to domain characteristics. If the Model Verification System proposed in this paper is used, It has the advantage of being able to apply the consistent UML model more easily to Naval Combat System Software Development.

• Journal of Intelligence and Information Systems
• /
• v.28 no.3
• /
• pp.45-55
• /
• 2022

Recently, large-scale language models (LPLM) have been shown state-of-the-art performances in various tasks of natural language processing including intent classification. However, fine-tuning LPLM requires much computational cost for training and inference which is not appropriate for dialog system. In this paper, we propose compressed intent classification model for multi-agent in low-resource like CPU. Our method consists of two stages. First, we trained sentence encoder from LPLM then compressed it through knowledge distillation. Second, we trained agent-specific adapter for intent classification. The results of three intent classification datasets show that our method achieved 98% of the accuracy of LPLM with only 21% size of it.

• IEMEK Journal of Embedded Systems and Applications
• /
• v.19 no.2
• /
• pp.73-81
• /
• 2024

Recently, mobile manipulators have been utilized in agriculture industry for weed removal and harvest automation. This paper proposes a domain adaptive fruit detection method for harvest automation, by utilizing OWL-ViT model which is an open-vocabulary object detection model. The vision-language model can detect objects based on text prompt, and therefore, it can be extended to detect objects of undefined categories. In the development of deep learning models for real-world problems, constructing a large-scale labeled dataset is a time-consuming task and heavily relies on human effort. To reduce the labor-intensive workload, we utilized a large-scale public dataset as a source domain data and employed a domain adaptation method. Adversarial learning was conducted between a domain discriminator and feature extractor to reduce the gap between the distribution of feature vectors from the source domain and our target domain data. We collected a target domain dataset in a real-like environment and conducted experiments to demonstrate the effectiveness of the proposed method. In experiments, the domain adaptation method improved the AP50 metric from 38.88% to 78.59% for detecting objects within the range of 2m, and we achieved 81.7% of manipulation success rate.

• Journal of Korea Society of Industrial Information Systems
• /
• v.28 no.5
• /
• pp.31-39
• /
• 2023

The evolution of Natural Language Processing (NLP) and the rise of large language models (LLM) like ChatGPT have paved the way for specialized question-answering (QA) systems tailored to specific domains. This study outlines a system harnessing the power of LLM in conjunction with document search algorithms to interpret and address user inquiries using documents from the Korea Atomic Energy Research Institute (KAERI). Initially, the system refines multiple documents for optimized search and analysis, breaking the content into managable paragraphs suitable for the language model's processing. Each paragraph's content is converted into a vector via an embedding model and archived in a database. Upon receiving a user query, the system matches the extracted vectors from the question with the stored vectors, pinpointing the most pertinent content. The chosen paragraphs, combined with the user's query, are then processed by the language generation model to formulate a response. Tests encompassing a spectrum of questions verified the system's proficiency in discerning question intent, understanding diverse documents, and delivering rapid and precise answers.

• The Journal of the Acoustical Society of Korea
• /
• v.20 no.2E
• /
• pp.3-8
• /
• 2001

In this paper, we will examine a Korean large vocabulary continuous speech recognition (LVCSR) system for broadcast news speech. The combined vowel and implosive unit is included in a phone set together with other short phone units in order to obtain a longer unit acoustic model. The effect of this unit is compared with conventional phone units. The dictionary units for language processing are automatically extracted from eojeols appearing in transcriptions. Triphone models are used for acoustic modeling and a trigram model is used for language modeling. Among three major speaker groups in news broadcasts-anchors, journalists and people (those other than anchors or journalists, who are being interviewed), the speech of anchors and journalists, which has a lot of noise, was used for testing and recognition.

• Annual Conference on Human and Language Technology
• /
• 2023.10a
• /
• pp.155-161
• /
• 2023

최근 대규모 언어 모델의 활용 방법에 대한 많은 관심이 지속되고 있다. LLM이 생성한 정보의 가장 중요한 도전 과제는 출력 문장이 사실인지 판단하기 어렵다는 점이다. 본 논문은 하나 또는 복수의 LLM를 구조화하여, 생산되는 다양한 결과를 활용하는 방안을 탐구한다. LLM이 생성한 출력 정보를 고려하여 사실 관계 확인 과제를 수행함에 있어서, 성능 향상 가능성을 평가하기 위한 실험을 진행했다. 대규모 언어모델의 구조화를 통해 입력 정보의 제어를 할 경우, 기존 최고 성능보다 4.75의 정확도 향상을 관찰할 수 있었다.

• Annual Conference on Human and Language Technology
• /
• 2023.10a
• /
• pp.425-431
• /
• 2023

음성 어시스턴트 시스템에서 발화의 의도를 분류하고 새로운 의도를 탐지하는 것은 매우 중요한 작업이다. 끊임없이 인입되는 새로운 발화로 인해 기존에 학습된 모델의 의도 분류 성능은 시간이 지남에 따라 점차 낮아진다. 기존 연구들에서 새로운 의도 발견을 위해 제안되었던 클러스터링 방법은 최적의 클러스터 수 결정과 명명에 어려움이 있다. 이러한 제한 사항을 보완하기 위해, 본 연구에서는 대규모 언어 모델 기반의 효과적인 의도 발견 방법을 제안한다. 이 방법은 기존 의도 분류기로 판단하기 어려운 발화에 새로운 의도 레이블을 할당하는 방법이다. 새롭게 인입되는 OOD(Out-of-Domain) 발화 내에서 오분류를 찾아 기존에 정의된 의도를 탐지하고, 새로운 의도를 발견하는 효율적인 프롬프팅 방법도 분석한다. 이를 액티브 러닝 전략과 결합할 경우, 분류 가능한 의도의 개수를 지속 증가시면서도 모델의 성능 하락을 방지할 수 있고, 동시에 새로운 의도 발견을 자동화 할 수 있다.

• Annual Conference on Human and Language Technology
• /
• 2023.10a
• /
• pp.171-176
• /
• 2023

최근 등장한 대규모 데이터로 사전학습된 자연어 생성 모델들은 대화 능력 및 코드 생성 태스크등에서 인상적인 성능을 보여주고 있어, 본 논문에서는 대형 언어 모델 (LLM)의 한국어 질문을 SQL 쿼리 (Text-to-SQL) 변환하는 성능을 평가하고자 한다. 먼저, 영어 Text-to-SQL 벤치마크 데이터셋을 활용하여 영어 질의문을 한국어 질의문으로 번역하여 한국어 Text-to-SQL 데이터셋으로 만들었다. 대형 생성형 모델 (GPT-3 davinci, GPT-3 turbo) 의 few-shot 세팅에서 성능 평가를 진행하며, fine-tuning 없이도 대형 언어 모델들의 경쟁력있는 한국어 Text-to-SQL 변환 성능을 확인한다. 또한, 에러 분석을 수행하여 한국어 문장을 데이터베이스 쿼리문으로 변환하는 과정에서 발생하는 다양한 문제와 프롬프트 기법을 활용한 가능한 해결책을 제시한다.