• Title/Summary/Keyword: Further Pre-Training

Search Result 89, Processing Time 0.023 seconds

도메인 특수성이 도메인 특화 사전학습 언어모델의 성능에 미치는 영향 (The Effect of Domain Specificity on the Performance of Domain-Specific Pre-Trained Language Models)

  • 한민아;김윤하;김남규
    • 지능정보연구
    • /
    • 제28권4호
    • /
    • pp.251-273
    • /
    • 2022
  • 최근 텍스트 분석을 딥러닝에 적용한 연구가 꾸준히 이어지고 있으며, 특히 대용량의 데이터 셋을 학습한 사전학습 언어모델을 통해 단어의 의미를 파악하여 요약, 감정 분류 등의 태스크를 수행하려는 연구가 활발히 이루어지고 있다. 하지만 기존 사전학습 언어모델이 특정 도메인을 잘 이해하지 못한다는 한계를 나타냄에 따라, 최근 특정 도메인에 특화된 언어모델을 만들고자 하는 방향으로 연구의 흐름이 옮겨가고 있는 추세이다. 도메인 특화 추가 사전학습 언어모델은 특정 도메인의 지식을 모델이 더 잘 이해할 수 있게 하여, 해당 분야의 다양한 태스크에서 성능 향상을 가져왔다. 하지만 도메인 특화 추가 사전학습은 해당 도메인의 말뭉치 데이터를 확보하기 위해 많은 비용이 소요될 뿐 아니라, 고성능 컴퓨팅 자원과 개발 인력 등의 측면에서도 많은 비용과 시간이 투입되어야 한다는 부담이 있다. 아울러 일부 도메인에서 추가 사전학습 후의 성능 개선이 미미하다는 사례가 보고됨에 따라, 성능 개선 여부가 확실하지 않은 상태에서 도메인 특화 추가 사전학습 모델의 개발에 막대한 비용을 투입해야 하는지 여부에 대해 판단이 어려운 상황이다. 이러한 상황에도 불구하고 최근 각 도메인의 성능 개선 자체에 초점을 둔 추가 사전학습 연구는 다양한 분야에서 수행되고 있지만, 추가 사전학습을 통한 성능 개선에 영향을 미치는 도메인의 특성을 규명하기 위한 연구는 거의 이루어지지 않고 있다. 본 논문에서는 이러한 한계를 극복하기 위해, 실제로 추가 사전학습을 수행하기 전에 추가 사전학습을 통한 해당 도메인의 성능 개선 정도를 선제적으로 확인할 수 있는 방안을 제시한다. 구체적으로 3개의 도메인을 분석 대상 도메인으로 선정한 후, 각 도메인에서의 추가 사전학습을 통한 분류 정확도 상승 폭을 측정한다. 또한 각 도메인에서 사용된 주요 단어들의 정규화된 빈도를 기반으로 해당 도메인의 특수성을 측정하는 지표를 새롭게 개발하여 제시한다. 사전학습 언어모델과 3개 도메인의 도메인 특화 사전학습 언어모델을 사용한 분류 태스크 실험을 통해, 도메인 특수성 지표가 높을수록 추가 사전학습을 통한 성능 개선 폭이 높음을 확인하였다.

한국인과 중국인 예비 한국어 교사 대상 채점자 교육 사례 (A Case Study on Rater Training for Pre-service Korean Language Teacher of Native Speakers and Chinese Speakers)

  • 이두용
    • 한국어교육
    • /
    • 제29권1호
    • /
    • pp.85-108
    • /
    • 2018
  • This study pointed out the reality that many novice Korean language teachers who lack rater training are scoring the learners' writing skill. The study performed and analyzed a case where pre-service teachers were educated in order to explore the possibility of promoting rater training in a Korean language teacher training course. The pre-service teachers majoring in Korean language education at the graduate school scored TOPIK compositions and were provided feedback by the FACETS program, which were further discussed at the rater meeting. In three scoring processes, the raters scored with conscious of own rating patterns and showed positive change or over correction due to excessive consciousness. Consequentially, ongoing training can improve rating ability, and considering the fact that professional rater training is hard to progress, the method composed of FACETS analysis and rater training revealed positive effects. On the other hand, the rater training including native Korean and non-native(Chinese) speakers together showed no significant difference by mother tongue but by individual difference. This can be interpreted as a positive implication to the rating reliability of non-native speakers possessing advanced Korean language abilities. However, this must be supplemented through extended research.

20대 정상성인에게 6주간 플라이오메트릭 훈련이 동적 균형능력에 미치는 영향 (The Effects of Plyometric Training on Dynamic Balance Ability with Twenty Normal Adults Six Weeks)

  • 조현래;이강성
    • PNF and Movement
    • /
    • 제8권1호
    • /
    • pp.59-65
    • /
    • 2010
  • Purpose : The purposes of this study was to determine the effect of plyometric training and agility training on SEBT and dynamic balance of health young. Methods : Thirty healthy subjects in their 20s were randomly assigned to a plyometric exercise group, an agility training group, and a control group; each group had 10 subjects. The training starts first 2set after more 1set 2 weeks. SEBT is measured every two weeks. Results : The results of this research were as followings: (1) After treatment, there were significant SEBT scores differences in both plyometric and agility group compared with pre-treatment(p<0.05). (2) After treatment, there were significant SEBT scores differences in both agility and control group compared with pre-treatment (p<0.05). Conclusion : It was concluded that ployometric training was effective for improving balance than agility and control group. Therefore, further studies are required to investigate the effect of plyometric training for improving balance with sports injury patient.

  • PDF

추가 사전학습 기반 지식 전이를 통한 국가 R&D 전문 언어모델 구축 (Building Specialized Language Model for National R&D through Knowledge Transfer Based on Further Pre-training)

  • 유은지;서수민;김남규
    • 지식경영연구
    • /
    • 제22권3호
    • /
    • pp.91-106
    • /
    • 2021
  • 최근 딥러닝 기술이 빠르게 발전함에 따라 국가 R&D 분야의 방대한 텍스트 문서를 다양한 관점에서 분석하기 위한 수요가 급증하고 있다. 특히 대용량의 말뭉치에 대해 사전학습을 수행한 BERT(Bidirectional Encoder Representations from Transformers) 언어모델의 활용에 대한 관심이 높아지고 있다. 하지만 국가 R&D와 같이 고도로 전문화된 분야에서 높은 빈도로 사용되는 전문어는 기본 BERT에서 충분히 학습이 이루어지지 않은 경우가 많으며, 이는 BERT를 통한 전문 분야 문서 이해의 한계로 지적되고 있다. 따라서 본 연구에서는 최근 활발하게 연구되고 있는 추가 사전학습을 활용하여, 기본 BERT에 국가 R&D 분야 지식을 전이한 R&D KoBERT 언어모델을 구축하는 방안을 제시한다. 또한 제안 모델의 성능 평가를 위해 보건의료, 정보통신 분야의 과제 약 116,000건을 대상으로 분류 분석을 수행한 결과, 제안 모델이 순수한 KoBERT 모델에 비해 정확도 측면에서 더 높은 성능을 나타내는 것을 확인하였다.

Voting and Ensemble Schemes Based on CNN Models for Photo-Based Gender Prediction

  • Jhang, Kyoungson
    • Journal of Information Processing Systems
    • /
    • 제16권4호
    • /
    • pp.809-819
    • /
    • 2020
  • Gender prediction accuracy increases as convolutional neural network (CNN) architecture evolves. This paper compares voting and ensemble schemes to utilize the already trained five CNN models to further improve gender prediction accuracy. The majority voting usually requires odd-numbered models while the proposed softmax-based voting can utilize any number of models to improve accuracy. The ensemble of CNN models combined with one more fully-connected layer requires further tuning or training of the models combined. With experiments, it is observed that the voting or ensemble of CNN models leads to further improvement of gender prediction accuracy and that especially softmax-based voters always show better gender prediction accuracy than majority voters. Also, compared with softmax-based voters, ensemble models show a slightly better or similar accuracy with added training of the combined CNN models. Softmax-based voting can be a fast and efficient way to get better accuracy without further training since the selection of the top accuracy models among available CNN pre-trained models usually leads to similar accuracy to that of the corresponding ensemble models.

노인의 낙상방지를 위한 바이오피드백 훈련의 효과 (Effects of Biofeedback Training for Prevention of Falling in Elderly Persons)

  • 강권영;이상빈
    • 대한물리치료과학회지
    • /
    • 제16권2호
    • /
    • pp.19-26
    • /
    • 2009
  • Background: The purpose of this study was to investigate effects of six-week biofeedback training for prevention of falling in elderly persons. Biofeedback training for using the TETRAX system. Methods: Thirty healthy elderly persons(men=17, women=13) who were 79 years of mean age participated in sensory regulation training. They were trained for thirty minutes a day, three times per week. We measured subjects sensory regulated function by TETRAX system, and analyzed mean difference of observed variables by paired t-test between the pre and post test. Results: The first experimental group were significantly difference between pre and post test. The second control group were not significantly between the pre and post test. The third there are significant between group. Conclusion: The results of this study reveal that biofeedback training exercise will improve sensory balance function, and further studies needs to identify which specific factors are related to fall in the elderly population, and it is expected this study may contribute in reducing fall and therapeutic exercise in falling.

  • PDF

Assessment of Village Health Worker Training Program in Tuguegarao, Philippine

  • Kim, Jung-Min;Koh, Kwang-Wook;Oak, Chul-Ho;Jung, Woo-Hyuk;Kim, Sung-Hyun;Park, Dae-Hee
    • Journal of Preventive Medicine and Public Health
    • /
    • 제42권6호
    • /
    • pp.377-385
    • /
    • 2009
  • Objectives : This study was performed to evaluate the effectiveness of 'village health worker training program' which aimed to build community participatory health promotion capacity of community leaders in villages of low developed country and to develop methods for further development of the program. Methods : The intervention group were 134 community leaders from 25 barangays (village). Control group were 149 form 4 barangays. Intervention group participated 3-day training program. Questionnaire was developed based on 'Health Promotion Capacity Checklist' which assessed capacity in 4 feathers; 'knowledge', 'skill', 'commitment', and 'resource'. Each feather was assessed in 4 point rating scale. Capacity scores between intervention group and control group were examined to identify changes between the pre- and post-intervention periods. A qualitative evaluation of the program was conducted to assess the appropriateness of the program. The program was conducted in Tuguegarao city, Philippine in January, 2009. Results : The result showed significant increases in the total health promotion capacity and each feather of health promotion capacities between pre and post assessment of intervention group. But there was no significant change in that of control group. Participants marked high level of satisfaction for preparedness, selection of main subjects and education method. Qualitative evaluation revealed that training program facilitated community participatory health promotion capacity of participants. Conclusions : This study suggested that the Village health worker training program is effective for building health promotion capacity of community leaders and it can be a main method for helping low developed countries with further development.

편마비 환자에서 트레드밀 보행훈련이 보행에 미치는 효과 - 지면 보행훈련과의 비교 - (Effects of Treadmill Gait Training on Gait Patterns in Hemiplegic Patients comparison with conventional gait training)

  • 김현희;허진강;양영애
    • 대한물리치료과학회지
    • /
    • 제10권2호
    • /
    • pp.17-28
    • /
    • 2003
  • The aim of this study was to investigate the effects of treadmill gait training on the functional characteristics and the temporal-distance parameters of gait in hemiplegic patients, as compared with conventional gait training. The subjects of this study were 32 hemiplegic patients who had been admitted or were visited out-patients at Kangdong Sacred Heart Hospital, Hallym University, from March 3 through April 25, 2003. These subjects were randomly divided into treadmill gait training group or conventional gait training group. We evaluated the gait ability, motor functions, muscle strength, spasticity, physiological cost index, and temporal-distance parameters. We analyzed the changes between pre and post training in each groups, and the difference between two groups. Temporal-distance parameters were obtained using the ink footprint method and then energy consumption using physiological cost index. The results were as follows: 1. After a six-week training, treadmill gait training group significantly improved, as. compared to pre-training, in gait ability, motor functions for the leg and trunk and gross function, muscle strength of the lower limb, gait speed, cadence, step length both on the affected and on the unaffected side, step length symmetry, and energy consumption(p<0.05). 2. After a six-week training, conventional gait training group significantly improved, as compared to pretraining, in gait ability, motor functions for the leg and trunk, muscle strength of the lower limb, spasticity the upper limb, gait speed, cadence, step length both on the affected and on the unaffected side, and energy consumption(p<0.05). 3. After a six-week training, the treadmill gait training group significantly improved, as compared to the conventional gait, training, in gait speed and step length on the unaffected side. These results show that treadmill gait training was improved gait speed and step length on the unaffected side of hemiplegic patients, as compared with conventional gait training. Further research is needed to confirm the generalization of these findings and to identify which hemiplegic patients might benefit from treadmill gait training.

  • PDF

사용자 리뷰 분석을 통한 제품 요구품질 도출 방법론 (Methodology for Deriving Required Quality of Product Using Analysis of Customer Reviews)

  • 유예린;변정은;배국진;서수민;김윤하;김남규
    • Journal of Information Technology Applications and Management
    • /
    • 제30권2호
    • /
    • pp.1-18
    • /
    • 2023
  • Recently, as technology development has accelerated and product life cycles have been shortened, it is necessary to derive key product features from customers in the R&D planning and evaluation stage. More companies want differentiated competitiveness by providing consumer-tailored products based on big data and artificial intelligence technology. To achieve this, the need to correctly grasp the required quality, which is a requirement of consumers, is increasing. However, the existing methods are centered on suppliers or domain experts, so there is a gap from the actual perspective of consumers. In other words, product attributes were defined by suppliers or field experts, but this may not consider consumers' actual perspective. Accordingly, the demand for deriving the product's main attributes through reviews containing consumers' perspectives has recently increased. Therefore, we propose a review data analysis-based required quality methodology containing customer requirements. Specifically, a pre-training language model with a good understanding of Korean reviews was established, consumer intent was correctly identified, and key contents were extracted from the review through a combination of KeyBERT and topic modeling to derive the required quality for each product. RevBERT, a Korean review domain-specific pre-training language model, was established through further pre-training. By comparing the existing pre-training language model KcBERT, we confirmed that RevBERT had a deeper understanding of customer reviews. In addition, all processes other than that of selecting the required quality were linked to the automation process, resulting in the automation of deriving the required quality based on data.

Deep Learning-based Target Masking Scheme for Understanding Meaning of Newly Coined Words

  • Nam, Gun-Min;Kim, Namgyu
    • 한국컴퓨터정보학회논문지
    • /
    • 제26권10호
    • /
    • pp.157-165
    • /
    • 2021
  • 최근 대량의 텍스트 분석을 위해 딥 러닝(Deep Learning)을 활용하는 연구들이 활발히 수행되고 있으며, 특히 대량의 텍스트에 대한 학습 결과를 특정 도메인 텍스트의 분석에 적용하는 사전 학습 언어 모델(Pre-trained Language Model)이 주목받고 있다. 다양한 사전 학습 언어 모델 중 BERT(Bidirectional Encoder Representations from Transformers) 기반 모델이 가장 널리 활용되고 있으며, 최근에는 BERT의 MLM(Masked Language Model)을 활용한 추가 사전 학습(Further Pre-training)을 통해 분석 성능을 향상시키기 위한 방안이 모색되고 있다. 하지만 전통적인 MLM 방식은 신조어와 같이 새로운 단어가 포함된 문장의 의미를 충분히 명확하게 파악하기 어렵다는 한계를 갖는다. 이에 본 연구에서는 기존의 MLM을 보완하여 신조어에 대해서만 집중적으로 마스킹을 수행하는 신조어 표적 마스킹(NTM: Newly Coined Words Target Masking)을 새롭게 제안한다. 제안 방법론을 적용하여 포털 'N'사의 영화 리뷰 약 70만 건을 분석한 결과, 제안하는 신조어 표적 마스킹이 기존의 무작위 마스킹에 비해 감성 분석의 정확도 측면에서 우수한 성능을 보였다.