• 전자통신동향분석
• /
• 제38권6호
• /
• pp.1-11
• /
• 2023

Large language models seem promising for handling reasoning problems, but their underlying solving mechanisms remain unclear. Large language models will establish a new paradigm in artificial intelligence and the society as a whole. However, a major challenge of large language models is the massive resources required for training and operation. To address this issue, researchers are actively exploring compact large language models that retain the capabilities of large language models while notably reducing the model size. These research efforts are mainly focused on improving pretraining, instruction tuning, and alignment. On the other hand, chain-of-thought prompting is a technique aimed at enhancing the reasoning ability of large language models. It provides an answer through a series of intermediate reasoning steps when given a problem. By guiding the model through a multistep problem-solving process, chain-of-thought prompting may improve the model reasoning skills. Mathematical reasoning, which is a fundamental aspect of human intelligence, has played a crucial role in advancing large language models toward human-level performance. As a result, mathematical reasoning is being widely explored in the context of large language models. This type of research extends to various domains such as geometry problem solving, tabular mathematical reasoning, visual question answering, and other areas.

• ETRI Journal
• /
• 제46권1호
• /
• pp.11-21
• /
• 2024

We focus on open-domain question-answering tasks that involve a chain-of-reasoning, which are primarily implemented using large language models. With an emphasis on cost-effectiveness, we designed EffiChainQA, an architecture centered on the use of small language models. We employed a retrieval-based language model to address the limitations of large language models, such as the hallucination issue and the lack of updated knowledge. To enhance reasoning capabilities, we introduced a question decomposer that leverages a generative language model and serves as a key component in the chain-of-reasoning process. To generate training data for our question decomposer, we leveraged ChatGPT, which is known for its data augmentation ability. Comprehensive experiments were conducted using the HotpotQA dataset. Our method outperformed several established approaches, including the Chain-of-Thoughts approach, which is based on large language models. Moreover, our results are on par with those of state-of-the-art Retrieve-then-Read methods that utilize large language models.

• 동의생리병리학회지
• /
• 제38권2호
• /
• pp.49-58
• /
• 2024

The rapid advancement of generative large language models has revolutionized various real-life domains, emphasizing the importance of exploring their applications in healthcare. This study aims to examine how generative large language models are implemented in the medical domain, with the specific objective of searching for the possibility and potential of integration between generative large language models and East Asian medicine. Through a comprehensive current state analysis, we identified limitations in the deployment of generative large language models within East Asian medicine and proposed directions for future research. Our findings highlight the essential need for accumulating and generating structured data to improve the capabilities of generative large language models in East Asian medicine. Additionally, we tackle the issue of hallucination and the necessity for a robust model evaluation framework. Despite these challenges, the application of generative large language models in East Asian medicine has demonstrated promising results. Techniques such as model augmentation, multimodal structures, and knowledge distillation have the potential to significantly enhance accuracy, efficiency, and accessibility. In conclusion, we expect generative large language models to play a pivotal role in facilitating precise diagnostics, personalized treatment in clinical fields, and fostering innovation in education and research within East Asian medicine.

• 전자통신동향분석
• /
• 제39권1호
• /
• pp.95-105
• /
• 2024

In natural language processing, large language models such as GPT-4 have recently been in the spotlight. The performance of natural language processing has advanced dramatically driven by an increase in the number of model parameters related to the number of acceptable input tokens and model size. Research on multimodal models that can simultaneously process natural language and image data is being actively conducted. Moreover, natural-language and image-based reasoning capabilities of large language models is being explored in robot artificial intelligence technology. We discuss research and related patent trends in robot task planning and code generation for robot control using large language models.

• 한국컴퓨터정보학회:학술대회논문집
• /
• 한국컴퓨터정보학회 2024년도 제69차 동계학술대회논문집 32권1호
• /
• pp.479-480
• /
• 2024

본 논문에서는 원자력 안전조치 용어를 미세조정(fine tuning) 알고리즘을 활용해 추가 학습한 공개 거대 언어모델(Large Language Model, LLM)이 안전조치 관련 질문에 대해 답변한 결과를 정성적으로 평가하였다. 평가 결과, 학습 데이터 범위 내 질문에 대해 학습 모델은 기반 모델 답변에 추가 학습 데이터를 활용한 낮은 수준의 추론을 수행한 답변을 출력하였다. 평가 결과를 통해 추가 학습 개선 방향을 도출하였으며 저비용 전문 분야 언어 모델 구축에 활용할 수 있을 것으로 보인다.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제16권3호
• /
• pp.771-791
• /
• 2022

Protein-protein interaction (PPI) extraction from original text is important for revealing the molecular mechanism of biological processes. With the rapid growth of biomedical literature, manually extracting PPI has become more time-consuming and laborious. Therefore, the automatic PPI extraction from the raw literature through natural language processing technology has attracted the attention of the majority of researchers. We propose a PPI extraction model based on the large pre-trained language model and adversarial training. It enhances the learning of semantic and syntactic features using BioBERT pre-trained weights, which are built on large-scale domain corpora, and adversarial perturbations are applied to the embedding layer to improve the robustness of the model. Experimental results showed that the proposed model achieved the highest F1 scores (83.93% and 90.31%) on two corpora with large sample sizes, namely, AIMed and BioInfer, respectively, compared with the previous method. It also achieved comparable performance on three corpora with small sample sizes, namely, HPRD50, IEPA, and LLL.

• 한국전자통신학회논문지
• /
• 제19권1호
• /
• pp.119-126
• /
• 2024

건물 정보 모델(BIM: Building Information Model)은 관련 기업의 개별화된 프로젝트와 학습 데이터양 부족으로 인해 인공지능(AI: Artificial Intelligence) 기반 BIM 애플리케이션 개발이 쉽지 않다. 본 연구에서는 데이터가 제한적인 상황에서 BIM의 강구조물 부재 중량비를 예측하기 위해 사전 학습이 된 대형 언어 모델을 기반으로 신경망을 학습하는 방법을 제시하고 실험하였다. 제안된 모델은 대형 언어 모델을 활용하여 BIM에 내재하는 데이터 부족 문제를 극복할 수 있어 데이터의 양이 부족한 상황에서도 성공적인 학습이 가능하며 대형 언어 모델과 연계된 신경망을 활용하여 자연어와 더불어 숫자 데이터까지 처리할 수 있다. 실험 결과는 제안된 대형 언어 모델 기반 신경망이 기존 소형 언어 모델 기반보다 높은 정확도를 보였다. 이를 통해, 대형 언어 모델이 BIM에 효과적으로 적용될 수 있음이 확인되었으며, 향후 건물 사고 예방 및 건설 비용의 효율적인 관리가 기대된다.

• 한국정보과학회 언어공학연구회:학술대회논문집(한글 및 한국어 정보처리)
• /
• 한국정보과학회언어공학연구회 2023년도 제35회 한글 및 한국어 정보처리 학술대회
• /
• pp.295-300
• /
• 2023

대화 데이터와 같은 데이터는 사람이 수작업으로 작성해야 하기 때문에 데이터셋 구축에 시간과 비용이 크게 발생한다. 현재 대두되고 있는 Large Language Model은 이러한 대화 생성에서 보다 자연스러운 대화 생성이 가능하다는 이점이 존재한다. 이번 연구에서는 LLM을 통해 사람이 만든 적은 양의 데이터셋을 Fine-tuning 하여 위키백과 문서로부터 데이터셋을 만들어내고, 이를 통해 문서 검색 모델의 성능을 향상시켰다. 그 결과 학습 데이터와 같은 문서집합에서 MRR 3.7%p, 위키백과 전체에서 MRR 4.5%p의 성능 향상을 확인했다.

• 한국정보과학회 언어공학연구회:학술대회논문집(한글 및 한국어 정보처리)
• /
• 한국정보과학회언어공학연구회 2023년도 제35회 한글 및 한국어 정보처리 학술대회
• /
• pp.19-24
• /
• 2023

자연어 처리 분야에서 데이터의 중요성이 더욱 강조되고 있으며, 특히 리소스가 부족한 도메인에서 데이터 부족 문제를 극복하는 방법으로 데이터 증강이 큰 주목을 받고 있다. 이 연구는 대규모 언어 모델(Large Language Model, LLM)을 활용한 키워드 기반 데이터 증강 방법을 제안하고자 한다. 구체적으로 한국어에 특화된 LLM을 활용하여 주어진 키워드를 기반으로 특정 주제에 관한 대화 내용을 생성하고, 이를 통해 대화 주제를 분류하는 분류 모델의 성능 향상을 입증했다. 이 연구 결과는 LLM을 활용한 데이터 증강의 유의미성을 입증하며, 리소스가 부족한 상황에서도 이를 활용할 수 있는 방법을 제시한다.

• ETRI Journal
• /
• 제46권1호
• /
• pp.96-105
• /
• 2024

We propose a method to automatically predict Alzheimer's disease from speech data using the ChatGPT large language model. Alzheimer's disease patients often exhibit distinctive characteristics when describing images, such as difficulties in recalling words, grammar errors, repetitive language, and incoherent narratives. For prediction, we initially employ a speech recognition system to transcribe participants' speech into text. We then gather opinions by inputting the transcribed text into ChatGPT as well as a prompt designed to solicit fluency evaluations. Subsequently, we extract embeddings from the speech, text, and opinions by the pretrained models. Finally, we use a classifier consisting of transformer blocks and linear layers to identify participants with this type of dementia. Experiments are conducted using the extensively used ADReSSo dataset. The results yield a maximum accuracy of 87.3% when speech, text, and opinions are used in conjunction. This finding suggests the potential of leveraging evaluation feedback from language models to address challenges in Alzheimer's disease recognition.