• Title/Summary/Keyword: language models

Search Result 885, Processing Time 0.024 seconds

Assessment of maximum liquefaction distance using soft computing approaches

  • Kishan Kumar;Pijush Samui;Shiva S. Choudhary
    • Geomechanics and Engineering
    • /
    • v.37 no.4
    • /
    • pp.395-418
    • /
    • 2024
  • The epicentral region of earthquakes is typically where liquefaction-related damage takes place. To determine the maximum distance, such as maximum epicentral distance (Re), maximum fault distance (Rf), or maximum hypocentral distance (Rh), at which an earthquake can inflict damage, given its magnitude, this study, using a recently updated global liquefaction database, multiple ML models are built to predict the limiting distances (Re, Rf, or Rh) required for an earthquake of a given magnitude to cause damage. Four machine learning models LSTM (Long Short-Term Memory), BiLSTM (Bidirectional Long Short-Term Memory), CNN (Convolutional Neural Network), and XGB (Extreme Gradient Boosting) are developed using the Python programming language. All four proposed ML models performed better than empirical models for limiting distance assessment. Among these models, the XGB model outperformed all the models. In order to determine how well the suggested models can predict limiting distances, a number of statistical parameters have been studied. To compare the accuracy of the proposed models, rank analysis, error matrix, and Taylor diagram have been developed. The ML models proposed in this paper are more robust than other current models and may be used to assess the minimal energy of a liquefaction disaster caused by an earthquake or to estimate the maximum distance of a liquefied site provided an earthquake in rapid disaster mapping.

KorPatELECTRA : A Pre-trained Language Model for Korean Patent Literature to improve performance in the field of natural language processing(Korean Patent ELECTRA)

  • Jang, Ji-Mo;Min, Jae-Ok;Noh, Han-Sung
    • Journal of the Korea Society of Computer and Information
    • /
    • v.27 no.2
    • /
    • pp.15-23
    • /
    • 2022
  • In the field of patents, as NLP(Natural Language Processing) is a challenging task due to the linguistic specificity of patent literature, there is an urgent need to research a language model optimized for Korean patent literature. Recently, in the field of NLP, there have been continuous attempts to establish a pre-trained language model for specific domains to improve performance in various tasks of related fields. Among them, ELECTRA is a pre-trained language model by Google using a new method called RTD(Replaced Token Detection), after BERT, for increasing training efficiency. The purpose of this paper is to propose KorPatELECTRA pre-trained on a large amount of Korean patent literature data. In addition, optimal pre-training was conducted by preprocessing the training corpus according to the characteristics of the patent literature and applying patent vocabulary and tokenizer. In order to confirm the performance, KorPatELECTRA was tested for NER(Named Entity Recognition), MRC(Machine Reading Comprehension), and patent classification tasks using actual patent data, and the most excellent performance was verified in all the three tasks compared to comparative general-purpose language models.

The Effects of Early Cumulative Risk Factors on Children's Development at Age 3 - The Mediation of Home Learning Environment - (유아기 발달에 대한 생애 초기 가족 누적위험요인의 영향 - 가정학습환경을 매개로 -)

  • Chang, Young Eun
    • Journal of the Korean Society of Child Welfare
    • /
    • no.54
    • /
    • pp.79-111
    • /
    • 2016
  • The purpose of this study was to examine the structural models in which early cumulative risk factors affect children's language(indicated by expressive vocabularies) and social development(indicated by peer competence) at age 3 thorough their effects on the home learning environment. To examine the hypothesized models, the data of 1,725 families from the second and the fourth waves of the Panel Study of Korean Children was used. Correlation analysis and structural equation modeling were conducted to test the models. First, the cumulative risk factors at age 1 and 3 were highly correlated, implying the stability of the risk factors over time. The more cumulative risk factors at age 1 predicted the lower level of the home learning environment at age 3, which, in turn, was significantly related to both language and social development at age 3. However, the early cumulative risk factors did not directly influence later developmental outcomes. Moreover, the cumulative risk factors at age 3 were directly related to the child's language development, but neither social development northe home learning environment. In addition, the mediational role of the home learning environment (i.e., cumulative risk factors at age 1${\rightarrow}$home learning environment${\rightarrow}$language and social development) was statistically supported. In conclusion, the early cumulative risk factors in infancy indirectly predicted children's development at age 3 through the home learning environment. The practical implications for the early intervention and support for the families with infants who are experiencing multiple risk factors were discussed.

Implementation of GPM Core Model Using OWL DL (OWL DL을 사용한 GPM 핵심 모델의 구현)

  • Choi, Ji-Woong;Park, Ho-Byung;Kim, Hyung-Jean;Kim, Myung-Ho
    • Journal of the Korea Society of Computer and Information
    • /
    • v.15 no.1
    • /
    • pp.31-42
    • /
    • 2010
  • GPM(Generic Product Model) developed by Hitachi in Japan is a common data model to integrate and share life cycle data of nuclear power plants. GPM consists of GPM core model, an abstract model, implementation language for the model and reference library written in the language. GPM core model has a feature that it can construct a semantic network model consisting of relationships among objects. Initial GPM developed and provided GPML as an implementation language to support the feature of the core model, but afterwards the GPML was replaced by GPM-XML based on XML to achieve data interoperability with heterogeneous applications accessing a GPM data model. However, data models written in GPM-XML are insufficient to be used as a semantic network model for lack of studies which support GPM-XML and enable the models to be used as a semantic network model. This paper proposes OWL as the implementation language for GPM core model because OWL can describe ontologies similar to semantic network models and has an abundant supply of technical standards and supporting tools. Also, OWL which can be expressed in terms of RDF/XML based on XML guarantees data interoperability. This paper uses OWL DL, one of three sublanguages of OWL, because it can guarantee complete reasoning and the maximum expressiveness at the same time. The contents of this paper introduce the way how to overcome the difference between GPM and OWL DL, and, base on this way, describe how to convert the reference library written in GPML into ontologies based on OWL DL written in RDF/XML.

Self-Supervised Document Representation Method

  • Yun, Yeoil;Kim, Namgyu
    • Journal of the Korea Society of Computer and Information
    • /
    • v.25 no.5
    • /
    • pp.187-197
    • /
    • 2020
  • Recently, various methods of text embedding using deep learning algorithms have been proposed. Especially, the way of using pre-trained language model which uses tremendous amount of text data in training is mainly applied for embedding new text data. However, traditional pre-trained language model has some limitations that it is hard to understand unique context of new text data when the text has too many tokens. In this paper, we propose self-supervised learning-based fine tuning method for pre-trained language model to infer vectors of long-text. Also, we applied our method to news articles and classified them into categories and compared classification accuracy with traditional models. As a result, it was confirmed that the vector generated by the proposed model more accurately expresses the inherent characteristics of the document than the vectors generated by the traditional models.

The Verification of the Transfer Learning-based Automatic Post Editing Model (전이학습 기반 기계번역 사후교정 모델 검증)

  • Moon, Hyeonseok;Park, Chanjun;Eo, Sugyeong;Seo, Jaehyung;Lim, Heuiseok
    • Journal of the Korea Convergence Society
    • /
    • v.12 no.10
    • /
    • pp.27-35
    • /
    • 2021
  • Automatic post editing is a research field that aims to automatically correct errors in machine translation results. This research is mainly being focus on high resource language pairs, such as English-German. Recent APE studies are mainly adopting transfer learning based research, where pre-training language models, or translation models generated through self-supervised learning methodologies are utilized. While translation based APE model shows superior performance in recent researches, as such researches are conducted on the high resource languages, the same perspective cannot be directly applied to the low resource languages. In this work, we apply two transfer learning strategies to Korean-English APE studies and show that transfer learning with translation model can significantly improves APE performance.

Contextual Modeling in Context-Aware Conversation Systems

  • Quoc-Dai Luong Tran;Dinh-Hong Vu;Anh-Cuong Le;Ashwin Ittoo
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.17 no.5
    • /
    • pp.1396-1412
    • /
    • 2023
  • Conversation modeling is an important and challenging task in the field of natural language processing because it is a key component promoting the development of automated humanmachine conversation. Most recent research concerning conversation modeling focuses only on the current utterance (considered as the current question) to generate a response, and thus fails to capture the conversation's logic from its beginning. Some studies concatenate the current question with previous conversation sentences and use it as input for response generation. Another approach is to use an encoder to store all previous utterances. Each time a new question is encountered, the encoder is updated and used to generate the response. Our approach in this paper differs from previous studies in that we explicitly separate the encoding of the question from the encoding of its context. This results in different encoding models for the question and the context, capturing the specificity of each. In this way, we have access to the entire context when generating the response. To this end, we propose a deep neural network-based model, called the Context Model, to encode previous utterances' information and combine it with the current question. This approach satisfies the need for context information while keeping the different roles of the current question and its context separate while generating a response. We investigate two approaches for representing the context: Long short-term memory and Convolutional neural network. Experiments show that our Context Model outperforms a baseline model on both ConvAI2 Dataset and a collected dataset of conversational English.

Context-Based Prompt Selection Methodology to Enhance Performance in Prompt-Based Learning

  • Lib Kim;Namgyu Kim
    • Journal of the Korea Society of Computer and Information
    • /
    • v.29 no.4
    • /
    • pp.9-21
    • /
    • 2024
  • Deep learning has been developing rapidly in recent years, with many researchers working to utilize large language models in various domains. However, there are practical difficulties that developing and utilizing language models require massive data and high-performance computing resources. Therefore, in-context learning, which utilizes prompts to learn efficiently, has been introduced, but there needs to be clear criteria for effective prompts for learning. In this study, we propose a methodology for enhancing prompt-based learning performance by improving the PET technique, which is one of the contextual learning methods, to select PVPs that are similar to the context of existing data. To evaluate the performance of the proposed methodology, we conducted experiments with 30,100 restaurant review datasets collected from Yelp, an online business review platform. We found that the proposed methodology outperforms traditional PET in all aspects of accuracy, stability, and learning efficiency.

Performance Comparison and Error Analysis of Korean Bio-medical Named Entity Recognition (한국어 생의학 개체명 인식 성능 비교와 오류 분석)

  • Jae-Hong Lee
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.19 no.4
    • /
    • pp.701-708
    • /
    • 2024
  • The advent of transformer architectures in deep learning has been a major breakthrough in natural language processing research. Object name recognition is a branch of natural language processing and is an important research area for tasks such as information retrieval. It is also important in the biomedical field, but the lack of Korean biomedical corpora for training has limited the development of Korean clinical research using AI. In this study, we built a new biomedical corpus for Korean biomedical entity name recognition and selected language models pre-trained on a large Korean corpus for transfer learning. We compared the name recognition performance of the selected language models by F1-score and the recognition rate by tag, and analyzed the errors. In terms of recognition performance, KlueRoBERTa showed relatively good performance. The error analysis of the tagging process shows that the recognition performance of Disease is excellent, but Body and Treatment are relatively low. This is due to over-segmentation and under-segmentation that fails to properly categorize entity names based on context, and it will be necessary to build a more precise morphological analyzer and a rich lexicon to compensate for the incorrect tagging.

A study on the optimal task-based instructional model: Focused on Korean EFL classroom practice (효율적인 과업중심 교수.학습모형 연구: EFL 교실 상황을 중심으로)

  • Jeon, In-Jae
    • English Language & Literature Teaching
    • /
    • v.11 no.4
    • /
    • pp.365-389
    • /
    • 2005
  • The purpose of this study is to present the task model that is the most effective in English language methodology based on the investigation of task-based performance in Korean EFL classroom practice. The subjects were 538 high school students and 126 high school teachers, each of whom had common experiences using the materials of task-based activities for more than one year. To analyze the data, the program SPSS WIN 11.0 including frequency distribution and chi-square analysis was used. The results of the questionnaire analysis showed that both teachers and students had a comparatively high level of satisfaction in task rationale, but that they had some mixed responses in the fields of input data, settings, and activity types. To conclude, a few suggestions are made to provide some meaningful considerations for the EFL teachers and material developers: a) task goals and rationale that encourage the learner's positive motivation; b) authenticity of input data based on the real-world context; c) collaborative learning environment that enhances communicative interaction; d) proportional representation of the creative problem-solving activities related to discussions and decision-making processes; e) systematic introduction of integrated language skills. It also suggests that the multi-lateral task model, which has some positive assets compared to previous task models, be newly introduced and applied to the second language learning classrooms.

  • PDF