• Proceedings of the Korean Society of Computer Information Conference
• /
• 2022.07a
• /
• pp.63-66
• /
• 2022

본 논문에서는 분자의 특성을 정확하게 예측하기 위해 효과적인 사전학습(pretraining) 전략과 트랜스포머(Transformer) 모델을 활용한 방법을 제시한다. 딥러닝을 활용한 분자의 성능을 예측하는 연구는 그동안 레이블이 부족한 분자데이터의 특성에 의해 학습 때 사용된 데이터이외의 분자데이터에 대해 일반화 능력이 떨어지는 어려움을 겪었다. 이 논문에서 제시한 모델은 사전학습(pretraining)을 수행할 때 자기지도학습(self-supervised training)을 사용하여 부족한 레이블에 의한 문제점을 피할 수 있다. 대규모 분자 데이터셋으로부터 학습된 이 모델은 4가지 다운스트림 데이터셋에 대해 모두 우수한 성능을 보여주어 일반화 성능이 뛰어나며 효과적인 분자표현을 얻을 수 있음을 보인다.

• Nuclear Engineering and Technology
• /
• v.55 no.10
• /
• pp.3515-3524
• /
• 2023

A reinforcement learning framework is proposed for the control problem of outlet steam pressure of the once-through steam generator(OTSG) in this paper. The double-layer controller using Proximal Policy Optimization(PPO) algorithm is applied in the control structure of the OTSG. The PPO algorithm can train the neural networks continuously according to the process of interaction with the environment and then the trained controller can realize better control for the OTSG. Meanwhile, reinforcement learning has the characteristic of difficult application in real-world objects, this paper proposes an innovative pretraining method to solve this problem. The difficulty in the application of reinforcement learning lies in training. The optimal strategy of each step is summed up through trial and error, and the training cost is very high. In this paper, the LSTM model is adopted as the training environment for pretraining, which saves training time and improves efficiency. The experimental results show that this method can realize the self-adjustment of control parameters under various working conditions, and the control effect has the advantages of small overshoot, fast stabilization speed, and strong adaptive ability.

• Journal of Korean Academy of Nursing
• /
• v.18 no.2
• /
• pp.105-117
• /
• 1988

To evaluate training effect, aerobic dance was performed by eight female collegestudents for 8 weeks. Body composition, cardiopulmonary function at rest and during maximal exercise, blood cholesterol concentration at rest were determined before and after 8weeks of aerobic dance training. Maximal exercise was performed on the treadmill according of Bruce protocol. Pre to post training differences were evaluated. The results obtained were as follows : 1. After the training, skinfold thickness and total body fat decreased significantly(p＜0.1) while lean body mass increased with significance (p＜0.1). 2. Heart rate and arterial blood pressure at rest decreased without sinificance after the training. 3. As a result of training, forced vital capacity and forced expiratory volume for a second increased significantly (p＜0.01, p＜0.1). 4. After the training period, heart rate at 3, 6, and 9 min. during treadmill exercise was significantly lower than those of pretraining (p＜0.05). 5. After the training, systolic and diastolic blood pressure at 6 and 9 min during the exercise was significantly lower than those of pretraining (p＜0.025, p＜0.1). 6. After the training, oxygen uptake at 3 and 6 min during the exercise was significantly greater than those of pretraining (p＜0.05). 7. As a result of training, the maximal oxygen uptake increased significantly during the exercise (p＜0.1). 8. After the training, expired air volume for a minute at 3 and 6 min during the exercise was signigicantly grerter than those of pretraining (p＜0.1). 9. After the training, repiratory quotient during the exercise was lower than pretaining without significance. 10. After the training, blood HDL -cholesterol concentration incereased with significance, (p＜0.1) blood total cholesterol and triglycerids concentration decreasedsignificantly (p＜0.1). From these results, it may be concluded that 8 week aerobic dance training reduces skinfold thickness and body fat contents, improves the cardiopulmonary function and tissue oxygen utilization, reduces blood cholesterol and triglyceride concentration and brings about the increase of blood HDL-cholesterol concentriation.

• ETRI Journal
• /
• v.43 no.6
• /
• pp.1049-1057
• /
• 2021

Automatic spacing in Korean is used to correct spacing units in a given input sentence. The demand for automatic spacing has been increasing owing to frequent incorrect spacing in recent media, such as the Internet and mobile networks. Therefore, herein, we propose a transformer encoder that reads a sentence bidirectionally and can be pretrained using an out-of-task corpus. Notably, our model exhibited the highest character accuracy (98.42%) among the existing automatic spacing models for Korean. We experimentally validated the effectiveness of bidirectional encoding and pretraining for automatic spacing in Korean. Moreover, we conclude that pretraining is more important than fine-tuning and data size.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.14 no.8
• /
• pp.3168-3186
• /
• 2020

Automatic topic essay generation (TEG) is a controllable text generation task that aims to generate informative, diverse, and topic-consistent essays based on multiple topics. To make the generated essays of high quality, a reasonable method should consider both diversity and topic-consistency. Another essential issue is the intrinsic link of the topics, which contributes to making the essays closely surround the semantics of provided topics. However, it remains challenging for TEG to fill the semantic gap between source topic words and target output, and a more powerful model is needed to capture the semantics of given topics. To this end, we propose a pretraining-based contextual self-attention (PC-SAN) model that is built upon the seq2seq framework. For the encoder of our model, we employ a dynamic weight sum of layers from BERT to fully utilize the semantics of topics, which is of great help to fill the gap and improve the quality of the generated essays. In the decoding phase, we also transform the target-side contextual history information into the query layers to alleviate the lack of context in typical self-attention networks (SANs). Experimental results on large-scale paragraph-level Chinese corpora verify that our model is capable of generating diverse, topic-consistent text and essentially makes improvements as compare to strong baselines. Furthermore, extensive analysis validates the effectiveness of contextual embeddings from BERT and contextual history information in SANs.

• The Journal of Korean Physical Therapy
• /
• v.11 no.1
• /
• pp.111-119
• /
• 1999

The object of study who healthufl thirty persons have been enforced Isokinetic exercise of non-dominant muscular strength. The next same that each dominant muscular strength and non-dominant strength, peak torque and total work have been comparative analysis 1, Shoulder muscles comparion increase peak torque at low speed from pretraining Isokinetic exercise of non-dominant strength side to ten week of post-training.. Flexor and extensor come out high and statistically significant 6, 8, 10 week than pretraining. Adductor and abductor come out high and statistically significant 4, 6, 8. 10 week then pre-training, Internal rotator and external rotator come out statistically significant 2, 4, 6, 8, 10 week then pre-ttraining. 2. Shoulder muscles comparion increase peak torque at high speed from pre-training Isokinetic execise of non-dominant strenght side to ten week of post-training. Flexor and extensor come out high and statistically significant 4, 6, 8, 10 week then pie-training. Adductor and abductor come out high and statistically significant 2, 4, 6, 8, 10 week than pre-training. Futernal rotator and external rotator come out statistically significant 4. 6, 8, 10 week than pre-training.

• Annual Conference on Human and Language Technology
• /
• 2021.10a
• /
• pp.320-325
• /
• 2021

BERT는 사전 학습 단계에서 다음 문장 예측 문제와 마스킹된 단어에 대한 예측 문제를 학습하여 여러 자연어 다운스트림 태스크에서 높은 성능을 보였다. 본 연구에서는 BERT의 사전 학습 문제 중 다음 문장 예측 문제에 대해 주목했다. 다음 문장 예측 문제는 자연어 추론 문제와 질의 응답 문제와 같이 임의의 두 문장 사이의 관계를 모델링하는 문제들에 성능 향상을 위해 사용되었다. 하지만 BERT의 다음 문장 예측 문제는 두 문장을 특수 토큰으로 분리하여 단일 문자열 형태로 모델에 입력으로 주어지는 cross-encoding 방식만을 학습하기 때문에 문장을 각각 인코딩하는 bi-encoding 방식의 다운스트림 태스크를 고려하지 않은 점에서 아쉬움이 있다. 본 논문에서는 기존 BERT의 다음 문장 예측 문제를 확장하여 bi-encoding 방식의 다음 문장 예측 문제를 추가적으로 사전 학습하여 단일 문장 분류 문제와 문장 임베딩을 활용하는 문제에서 성능을 향상 시키는 Bi-Cross 사전 학습 기법을 소개한다. Bi-Cross 학습 기법은 영화 리뷰 감성 분류 데이터 셋인 NSMC 데이터 셋에 대해 학습 데이터의 0.1%만 사용하는 학습 환경에서 Bi-Cross 사전 학습 기법 적용 전 모델 대비 5점 가량의 성능 향상이 있었다. 또한 KorSTS의 bi-encoding 방식의 문장 임베딩 성능 평가에서 Bi-Cross 사전 학습 기법 적용 전 모델 대비 1.5점의 성능 향상을 보였다.

• Journal of Korean Library and Information Science Society
• /
• v.42 no.1
• /
• pp.183-203
• /
• 2011

The purpose of this study is to propose an effective development of teacher librarians' practice teaching and to identify how much teacher librarians' practice teaching affects their expertise in Korea. For this, two research models are constructed. The first model is to analyze the relationship of variables between the effect of practice teaching and the following variables: pretraining, practice program, advisory teacher, training condition and organizational atmosphere. The second model is to analyze how much the practice teaching affects their role perception and expertise compared with training courses and field experiences. The result is as follow: It is the training program that greatly affects teacher librarians' practice teaching among those variables. And the practice teaching has a positive effect on the ability of role performance. According to the above result, the practice teaching is effective in developing teacher librarians' expertise. And more systematic practice teaching program, enough pretraining and the efforts of advisory teacher are necessary to enhance the effect of teacher librarians' practice teaching.

• Annual Conference on Human and Language Technology
• /
• 2021.10a
• /
• pp.588-591
• /
• 2021

다중추론 질의응답 태스크는 하나의 문서만 필요한 기존의 단일추론 질의응답(Single-hop QA)을 넘어서 복잡한 추론을 요구하는 질문에 응답하는 것이 목표이다. IRQA에서는 검색 모델의 역할이 중요한 반면, 주목받고 있는 Dense Retrieval 모델 기반의 다중추론 질의응답 검색 모델은 찾기 어렵다. 본 논문에서는 검색분야에서 좋은 성능 보이고 있는 Dense Retrieval 모델의 다중추론을 위한 사전학습 방법을 제안하고 관련 한국어 데이터 셋에서 이전 방법과의 성능을 비교 측정하여 학습 방법의 유효성을 검증하고 있다. 이를 통해 지식 베이스, 엔터티 링킹, 개체명 인식모듈을 비롯한 다른 서브모듈을 사용하지 않고도 다중추론 Dense Retrieval 모델을 학습시킬 수 있음을 보였다.

• Annual Conference on Human and Language Technology
• /
• 2020.10a
• /
• pp.572-577
• /
• 2020

일관된 발화를 생성함에 있어 인격데이터(persona)의 도입을 이용한 연구가 활발히 진행되고 있지만, 한국어 데이터셋의 부재와 데이터셋 생성의 어려움이 문제점으로 지적된다. 본 연구에서는 인격데이터를 포함하지 않고 일관된 발화를 생성할 수 있는 방법으로 다중 대화 시스템에서 사전 학습된 자연어 추론(NLI) 모델을 도입하는 방법을 제안한다. 자연어 추론 모델을 이용한 관계 분석을 통해 과거 대화 내용 중 발화 생성에 이용할 대화를 선택하고, 자가 참조 모델(self-attention)과 다중 어텐션(multi-head attention) 모델을 활용하여 과거 대화 내용을 반영한 발화를 생성한다. 일관성 있는 발화 생성을 위해 기존 NLI데이터셋으로 수행할 수 있는 새로운 학습모델 nMLM을 제안하고, 이 방법이 일관성 있는 발화를 만드는데 기여할 수 있는 방법에 대해 연구한다.