• The Journal of Bigdata
• /
• v.7 no.2
• /
• pp.11-29
• /
• 2022

Pre-trained language models are the most important and widely used tools in natural language processing tasks. Since those have been pre-trained for a large amount of corpus, high performance can be expected even with fine-tuning learning using a small number of data. Since the elements necessary for implementation, such as a pre-trained tokenizer and a deep learning model including pre-trained weights, are distributed together, the cost and period of natural language processing has been greatly reduced. Transformer variants are the most representative pre-trained language models that provide these advantages. Those are being actively used in other fields such as computer vision and audio applications. In order to make it easier for researchers to understand the pre-trained language model and apply it to natural language processing tasks, this paper describes the definition of the language model and the pre-learning language model, and discusses the development process of the pre-trained language model and especially representative Transformer variants.

• Journal of Korean Society of Industrial and Systems Engineering
• /
• v.46 no.3
• /
• pp.123-138
• /
• 2023

The Fourth Industrial Revolution and sensor technology have led to increased utilization of sensor data. In our modern society, data complexity is rising, and the extraction of valuable information has become crucial with the rapid changes in information technology (IT). Recurrent neural networks (RNN) and long short-term memory (LSTM) models have shown remarkable performance in natural language processing (NLP) and time series prediction. Consequently, there is a strong expectation that models excelling in NLP will also excel in time series prediction. However, current research on Transformer models for time series prediction remains limited. Traditional RNN and LSTM models have demonstrated superior performance compared to Transformers in big data analysis. Nevertheless, with continuous advancements in Transformer models, such as GPT-2 (Generative Pre-trained Transformer 2) and ProphetNet, they have gained attention in the field of time series prediction. This study aims to evaluate the classification performance and interval prediction of remaining useful life (RUL) using an advanced Transformer model. The performance of each model will be utilized to establish a health index (HI) for cutting blades, enabling real-time monitoring of machine health. The results are expected to provide valuable insights for machine monitoring, evaluation, and management, confirming the effectiveness of advanced Transformer models in time series analysis when applied in industrial settings.

• Annual Conference on Human and Language Technology
• /
• 2023.10a
• /
• pp.448-453
• /
• 2023

대규모 언어 모델 (Large Language Model, LLM)을 인간의 선호도 관점에서 평가하는 것은 기존의 벤치마크 평가와는 다른 도전적인 과제이다. 이를 위해, 기존 연구들은 강력한 LLM을 평가자로 사용하여 접근하였지만, 높은 비용 문제가 부각되었다. 또한, 평가자로서 LLM이 사용하는 주관적인 점수 기준은 모호하여 평가 결과의 신뢰성을 저해하며, 단일 모델에 의한 평가 결과는 편향될 가능성이 있다. 본 논문에서는 엄격한 기준을 활용하여 편향되지 않은 평가를 수행할 수 있는 평가 프레임워크 및 평가자 모델 'FubaoLM'을 제안한다. 우리의 평가 프레임워크는 심층적인 평가 기준을 통해 다수의 강력한 한국어 LLM을 활용하여 연쇄적 사고(Chain-of-Thought) 기반 평가를 수행한다. 이러한 평가 결과를 다수결로 통합하여 편향되지 않은 평가 결과를 도출하며, 지시 조정 (instruction tuning)을 통해 FubaoLM은 다수의 LLM으로 부터 평가 지식을 증류받는다. 더 나아가 본 논문에서는 전문가 기반 평가 데이터셋을 구축하여 FubaoLM 효과성을 입증한다. 우리의 실험에서 앙상블된 FubaoLM은 GPT-3.5 대비 16% 에서 23% 향상된 절대 평가 성능을 가지며, 이항 평가에서 인간과 유사한 선호도 평가 결과를 도출한다. 이를 통해 FubaoLM은 비교적 적은 비용으로도 높은 신뢰성을 유지하며, 편향되지 않은 평가를 수행할 수 있음을 보인다.

• Journal of the Korea Institute of Information Security & Cryptology
• /
• v.34 no.3
• /
• pp.451-461
• /
• 2024

Recently, various AI chatbots based on large language models have been released. Chatbots have the advantage of providing users with quick and easy information through interactive prompts, making them useful in various fields such as question answering, writing, and programming. However, a vulnerability in chatbots called "prompt injection attacks" has been proposed. This attack involves injecting instructions into the chatbot to violate predefined guidelines. Such attacks can be critical as they may lead to the leakage of confidential information within large language models or trigger other malicious activities. However, the vulnerability of Korean prompts has not been adequately validated. Therefore, in this paper, we aim to generate malicious Korean prompts and perform attacks on the popular chatbot to analyze their feasibility. To achieve this, we propose a system that automatically generates malicious Korean prompts by analyzing existing prompt injection attacks. Specifically, we focus on generating malicious prompts that induce harmful expressions from large language models and validate their effectiveness in practice.

• Journal of The Korean Association of Information Education
• /
• v.25 no.2
• /
• pp.423-436
• /
• 2021

In the future, artificial intelligence (AI) technology is expected to become a general-purpose technology (GPT), and it is predicted that AI competency will become an essential competency. Several nations around the world are fostering experts in the field of AI to achieve technological proficiency while working to develop the necessary infrastructure and educational environment. In this study, we investigated the status of software education at the liberal arts level at 31 universities in Seoul, along with precedents from domestic and foreign AI education research. Based on this, we concluded that an AI literacy education model is needed to link software education at the liberal arts level with professional AI education. And we classified 20 AI-related lectures released in the KOCW according to the AI literacy competencies required; based on the results of this classification, we propose a model for AI literacy education in the liberal arts for undergraduate students. The proposed AI literacy education model may be considered as AI·SW convergence to experience AI along with literacy in the humanities, deviating from the existing theoretical and computer-science-based approach. We expect that our proposed AI literacy education model can contribute to the proliferation of AI.

• Journal of the Korean Society of Food Science and Nutrition
• /
• v.39 no.11
• /
• pp.1595-1603
• /
• 2010

The present study was conducted to investigate the effect of dietary grape pomace on lipid metabolism and hepatic morphology of rats fed a high fat diet. The high fat diet contained additional 15% lard to AIN 93-based diet. Male Sprague-Dawley rats were fed experimental diets containing 5% grape pomace for 4 weeks. Serum activities of glutamic-oxaloacetic transaminase (GOT) and glutamic-pyruvic transaminase (GPT) were not changed by high fat and grape pomace feeding. Serum concentration of triglyceride in rats fed a high fat diet was decreased significantly by dietary grape pomace. Hepatic concentrations of total lipid, total cholesterol and triglyceride were reduced in grape pomace groups with a high fat diet. Fecal concentrations of total cholesterol and triglyceride were increased in grape pomace groups with a high fat diet. The fecal content of coprostanol was not different among the groups. Dietary grape pomace increased the fecal excretion of cholesterol and coprostanone in rats fed a high fat diet. The fecal excretion of bile acid was not affected by feeding grape pomace in rats fed a high fat diet. Light micrographs of liver tissue revealed lipid droplets were increased by a high fat diet, but dietary supplementation of grape pomace tended to alleviate such changes.

• Journal of the Korean Society of Food Science and Nutrition
• /
• v.41 no.1
• /
• pp.87-94
• /
• 2012

We evaluated the bioavailability of a novel organic Ca supplement chelated with milk protein (CaMP) in growing rats and compared it with those of Ca carbonate and Ca citrate. Four-week-old male rats were divided into six groups (n=6/group) and fed AIN-93G-based experimental diets containing three Ca sources, Ca carbonate, Ca citrate, and CaMP at two levels, normal (0.5%, w/w) and high (1.5%, w/w), for 6 weeks. Growth, mineral contents of serum, Ca content and breaking force of femur, and Ca absorption were measured. There were no significant differences in weight gain or food intake, but food efficiency ratio (FER) of CaMP in the high Ca group was higher than those of other groups. Ca and P concentrations in serum were within normal range in all experimental rats. There was no difference in Ca content of the femur among all of the groups. Although there was no statistical significance in bone breaking force of the femur among the groups, the CaMP groups had a higher breaking force compared to other groups. Further, Ca absorption rate significantly increased in the CaMP groups (p<0.05). These results demonstrate that the CaMP supplement did not exhibit any negative effect on growth or Ca metabolism of the rats. Therefore, CaMP can be recommended as a good Ca supplement with regard to bone metabolism and Ca bioavailability.

• Herbal Formula Science
• /
• v.13 no.2
• /
• pp.1-16
• /
• 2005

Objectives: To find out the effects GGTl, an antiobestic drug widely used in clinics, has on the blood-antiobestic index and the toxicity index using the data from the hGHTg obese male rats. We looked closely into both of the two indices because GGTl antiobestic effect can happen not only by pharmacological action, but also by its toxicity. Also, we verified the difference in effect between GGTl and reductil (sibutramine), which has been approved by the FDA of the United States. Methods: After performing the experiments for 8 weeks on the hGHTg obese male rats divided into three groups: the control group, the GGTl group, and the reductil (RD) group, we anesthetized the rats with Diethyl ether and took a 3ml blood sample from the heart. Then, after coagulating the blood in room temperature by using the plasma separator, we centrifuged it for 25 minutes in 3,000rpm using the high-speed refrigerated centrifuge. We kept the separated plasma in a deep freezer at $-80^{\circ}C$, and repeatedly measured the antiobestic index and the toxicity index twice using the hematology biochemistry analyzer. Also, in order to judge the indirect toxicity index, we separated liver from kidney and observed them. Results: When we looked at the results of the analysis of covariance on the measuring elements related to the antiobestic index (TC, HDL, LDL, TG, and GLU), there was no significant difference among the groups in all measuring elements. Also, the results of the analysis of covariance on the two roups (RD group and GGTl group) showed that the p-values had no significant difference under the level of significance 0.05. When we looked at the result of the analysis of covariance on the measuring elements related to the toxicity index (GOT, GPT, GGT, CREA, UA, ALB, and TP), we could see that the p-values in GPT, ALB, and TP have a significant difference among the groups. Also, the results of the analysis of covariance about the measuring elements related to the toxicity index on both groups, RD group and GGTl group, showed no significant difference in the p-values of all of the measuring elements in the two groups, RD and GGTl group. Conclusions: In conclusion, through this experiment, the safety of GGTl has been approved, and although the verification on its medical effect has not been clearly approved, when we consider the fact that it belongs to the same group as reductil, an antiobestic drug approved by the FDA of the United States, we could indirectly verify that GGTl has an antiobestic effect. We believe that when doing a sample design for a future experiment, it needs to be performed on a greater sample size based on the power analysis that needs to be performed primarily in experiments, and a more accurate verification is needed through more systematic experiment plans.

• Journal of Animal Science and Technology
• /
• v.52 no.5
• /
• pp.389-398
• /
• 2010

The objective of this study was to investigate the effects of three strains of Bacillus subtilis (B. subtilis) supplemented to diets on egg production, egg quality, egg yolk cholesterol levels, the profile of cecal microflora, and tibia characteristics in laying hens. One hundred sixty 76-week-old Hy-Line Brown layers were randomly divided into 4 groups with 4 replicates per group (10 birds per replicate). Birds in the control group were fed a corn-soybean meal based diet. The remaining three treated groups were fed the control diet containing either 0.05% B. subtilis Ch3 (T1), 0.05% B. subtilis Ch3 + B. subtilis W1 (T2) or 0.05% B. subtilis commercial product (T3) for 6 weeks, respectively. There were no differences in feed intake, egg weight, egg production and egg mass among the groups. The dietary supplementation of B. subtilis improved eggshell strength and Haugh units compared to those of control (P<0.05). The activities of GOT and GPT in serum were not also affected by the dietary treatments. The population of total microbes and lactic acid bacteria in cecum were significantly increased by the dietary B. subtilis (P<0.05), but not the coliforms. The cholesterol concentration in egg yolk and serum in the treated groups were significantly decreased compared to those of control (P<0.05). Also, The levels of phospholipids in serum were significantly decreased compared to those of control (P<0.05). The supplementation of three strains of B. subtilis to diets significantly increased the contents of tibia ash compared to that of control (P<0.05). Thus, this study showed significant improvements in egg quality, such as eggshell strength and Haugh unit, by dietary B. subtilis strains. The B. subtilis strains added to the diets modulated the profiles of cecal microflora, reflecting beneficial effects in laying hens.

• Korean Journal of Community Nutrition
• /
• v.17 no.4
• /
• pp.450-462
• /
• 2012

We analyzed data from the combined 2008~2009 Korean National Health and Nutrition Examination Survey (KNHANES) to compare the health factors related to menopausal stages in 45~60 year old Korean women. In this study, we classified the subjects into a premenopausal group (n = 439) and a postmenopausal group (n = 683). In the postmenopausal group, age was higher (p < 0.001), monthly income (p < 0.01) and education levels (p < 0.001) were significantly lower than in the premenopausal group. Body fat % and waist circumferences were also higher in the postmenopausal group than in the premenopausal group. The serum glucose (p < 0.05), total cholesterol (p < 0.001), LDL-cholesterol (p < 0.001), triglyceride (p < 0.001), GOT (p < 0.001), GPT (p < 0.001) in the postmenopausal group were higher than in the premenopausal group. The postmenopausal group showed a significantly lower quality of life compared to the premenopausal group (p < 0.01). With regard to dietary quality, nutrient adequacy ratio (NAR) of vitamin A, vitamin $B_1$, vitamin $B_2$ and niacin in the postmenopausal group were significantly lower than in the premenopausal group. The levels of glucose, total cholesterol, LDL-cholesterol, and triglyceride showed a significantly positive correlation with age, waist circumferences, body fat % and BMI. The 45~60 year old Korean women in this study showed high levels of obesity and serum lipids. Also, intakes of the vitamins and minerals of the women did not meet the level of Dietary reference intakes for Koreans. Therefore, nutritional risk may be high in the women, especially in postmenopausal women. In order to prevent the health risk, women's health care including the quality of the meal should be considered.