• Title/Summary/Keyword: bidirectional encoder representations from transformers (BERT)

Search Result 38, Processing Time 0.021 seconds

Hot Keyword Extraction of Sci-tech Periodicals Based on the Improved BERT Model

  • Liu, Bing;Lv, Zhijun;Zhu, Nan;Chang, Dongyu;Lu, Mengxin
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.16 no.6
    • /
    • pp.1800-1817
    • /
    • 2022
  • With the development of the economy and the improvement of living standards, the hot issues in the subject area have become the main research direction, and the mining of the hot issues in the subject currently has problems such as a large amount of data and a complex algorithm structure. Therefore, in response to this problem, this study proposes a method for extracting hot keywords in scientific journals based on the improved BERT model.It can also provide reference for researchers,and the research method improves the overall similarity measure of the ensemble,introducing compound keyword word density, combining word segmentation, word sense set distance, and density clustering to construct an improved BERT framework, establish a composite keyword heat analysis model based on I-BERT framework.Taking the 14420 articles published in 21 kinds of social science management periodicals collected by CNKI(China National Knowledge Infrastructure) in 2017-2019 as the experimental data, the superiority of the proposed method is verified by the data of word spacing, class spacing, extraction accuracy and recall of hot keywords. In the experimental process of this research, it can be found that the method proposed in this paper has a higher accuracy than other methods in extracting hot keywords, which can ensure the timeliness and accuracy of scientific journals in capturing hot topics in the discipline, and finally pass Use information technology to master popular key words.

Aspect-based Sentiment Analysis of Product Reviews using Multi-agent Deep Reinforcement Learning

  • M. Sivakumar;Srinivasulu Reddy Uyyala
    • Asia pacific journal of information systems
    • /
    • v.32 no.2
    • /
    • pp.226-248
    • /
    • 2022
  • The existing model for sentiment analysis of product reviews learned from past data and new data was labeled based on training. But new data was never used by the existing system for making a decision. The proposed Aspect-based multi-agent Deep Reinforcement learning Sentiment Analysis (ADRSA) model learned from its very first data without the help of any training dataset and labeled a sentence with aspect category and sentiment polarity. It keeps on learning from the new data and updates its knowledge for improving its intelligence. The decision of the proposed system changed over time based on the new data. So, the accuracy of the sentiment analysis using deep reinforcement learning was improved over supervised learning and unsupervised learning methods. Hence, the sentiments of premium customers on a particular site can be explored to other customers effectively. A dynamic environment with a strong knowledge base can help the system to remember the sentences and usage State Action Reward State Action (SARSA) algorithm with Bidirectional Encoder Representations from Transformers (BERT) model improved the performance of the proposed system in terms of accuracy when compared to the state of art methods.

Modern Methods of Text Analysis as an Effective Way to Combat Plagiarism

  • Myronenko, Serhii;Myronenko, Yelyzaveta
    • International Journal of Computer Science & Network Security
    • /
    • v.22 no.8
    • /
    • pp.242-248
    • /
    • 2022
  • The article presents the analysis of modern methods of automatic comparison of original and unoriginal text to detect textual plagiarism. The study covers two types of plagiarism - literal, when plagiarists directly make exact copying of the text without changing anything, and intelligent, using more sophisticated techniques, which are harder to detect due to the text manipulation, like words and signs replacement. Standard techniques related to extrinsic detection are string-based, vector space and semantic-based. The first, most common and most successful target models for detecting literal plagiarism - N-gram and Vector Space are analyzed, and their advantages and disadvantages are evaluated. The most effective target models that allow detecting intelligent plagiarism, particularly identifying paraphrases by measuring the semantic similarity of short components of the text, are investigated. Models using neural network architecture and based on natural language sentence matching approaches such as Densely Interactive Inference Network (DIIN), Bilateral Multi-Perspective Matching (BiMPM) and Bidirectional Encoder Representations from Transformers (BERT) and its family of models are considered. The progress in improving plagiarism detection systems, techniques and related models is summarized. Relevant and urgent problems that remain unresolved in detecting intelligent plagiarism - effective recognition of unoriginal ideas and qualitatively paraphrased text - are outlined.

Analysis of International Research Trends on Metaverse

  • Mina, Shim
    • International Journal of Advanced Culture Technology
    • /
    • v.10 no.4
    • /
    • pp.453-459
    • /
    • 2022
  • This study attempted to explore the realization and research direction of a successful metaverse environment in the future by analyzing international research trends of the metaverse using topic modeling. A total of 208 papers among WoS and ScienceDirect papers using metaverse as keywords were selected, and quantitative frequency analysis and topic modeling were performed. As a result, it was confirmed that research has rapidly increased after 2022. The main keywords of the research topics were 'second', 'life', 'learning', 'reality', 'metaverse', 'virtual', 'blockchain', 'nft', 'medical', 'avatar', etc. The topic keywords 'Second life & Education' and 'Virtual Reality & Medical' accounted for a large proportion of 57%, followed by 'Blockchain & Cryptocurrency', 'Avatar & Interaction', and 'Sensing and Device'. As a result of semantic analysis, current metaverse research is focused on application and utilization, and research on underlying technologies and devices is also active. Therefore, it is necessary to identify the commonalities and differences between domestic and foreign studies, and to study the application method considering the domestic environment. In addition, new jurisprudence research is more necessary along with predicting new problems. It is expected that the results of study will provide the right research direction for domestic researchers in the era of digital transformation and contribute to the realization of a digital society.

Proposal of Git's Commit Message Complex Classification Model for Efficient S/W Maintenance (효율적인 S/W 유지관리를 위한 Git의 커밋메시지 복합 분류모델 제안)

  • Choi, Ji-Hoon;Kim, Jae-Woong;Lee, Youn-Yeoul;Chae, Yi-Geun;Kim, Joon-Yong
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2022.07a
    • /
    • pp.123-125
    • /
    • 2022
  • Git의 커밋 메시지는 프로젝트가 진행되면서 발생하는 각종 이슈 및 코드의 변경이력을 저장하고 관리하고 있기 때문에 소프트웨어 유지관리와 프로젝트의 생명주기와 밀접한 연관성을 갖고 있다. 이러한 Git의 커밋 메시지에 대한 정확한 분석 결과는 소프트웨어 개발 및 유지관리 활동 시, 시간과 비용의 효율적인 관리에 많은 영향을 끼치고 있다. 이에 대한 기존 연구로 Git에서 발생하는 커밋 메시지를 소프트웨어 유지관리의 세 가지 형태로 분류하고 매핑하여 정확한 분석을 시도하려는 연구가 진행되었으나, 최대 87%의 정확도를 제시한 연구 결과가 있었다. 이러한 연구들은 정확도가 낮아 실제 프로젝트의 개발 및 유지관리에 적용하기에는 위험성과 어려움이 있는 현실이다. 본 논문에서는 커밋 메시지 분류에 대한 선행 연구 조사를 통해 각 연구들의 프로세스와 특징을 추출하였고, 이를 이용한 분류 정확도를 높일 수 있는 커밋 복합 분류 모델에 대해 제안한다.

  • PDF

Implementation of Git's Commit Message Complex Classification Model for Software Maintenance

  • Choi, Ji-Hoon;Kim, Joon-Yong;Park, Seong-Hyun
    • Journal of the Korea Society of Computer and Information
    • /
    • v.27 no.11
    • /
    • pp.131-138
    • /
    • 2022
  • Git's commit message is closely related to the project life cycle, and by this characteristic, it can greatly contribute to cost reduction and improvement of work efficiency by identifying risk factors and project status of project operation activities. Among these related fields, there are many studies that classify commit messages as types of software maintenance, and the maximum accuracy among the studies is 87%. In this paper, the purpose of using a solution using the commit classification model is to design and implement a complex classification model that combines several models to increase the accuracy of the previously published models and increase the reliability of the model. In this paper, a dataset was constructed by extracting automated labeling and source changes and trained using the DistillBERT model. As a result of verification, reliability was secured by obtaining an F1 score of 95%, which is 8% higher than the maximum of 87% reported in previous studies. Using the results of this study, it is expected that the reliability of the model will be increased and it will be possible to apply it to solutions such as software and project management.

A method for metadata extraction from a collection of records using Named Entity Recognition in Natural Language Processing (자연어 처리의 개체명 인식을 통한 기록집합체의 메타데이터 추출 방안)

  • Chiho Song
    • Journal of Korean Society of Archives and Records Management
    • /
    • v.24 no.2
    • /
    • pp.65-88
    • /
    • 2024
  • This pilot study explores a method of extracting metadata values and descriptions from records using named entity recognition (NER), a technique in natural language processing (NLP), a subfield of artificial intelligence. The study focuses on handwritten records from the Guro Industrial Complex, produced during the 1960s and 1970s, comprising approximately 1,200 pages and 80,000 words. After the preprocessing process of the records, which included digitization, the study employed a publicly available language API based on Google's Bidirectional Encoder Representations from Transformers (BERT) language model to recognize entity names within the text. As a result, 173 names of people and 314 of organizations and institutions were extracted from the Guro Industrial Complex's past records. These extracted entities are expected to serve as direct search terms for accessing the contents of the records. Furthermore, the study identified challenges that arose when applying the theoretical methodology of NLP to real-world records consisting of semistructured text. It also presents potential solutions and implications to consider when addressing these issues.

Content-based Korean journal recommendation system using Sentence BERT (Sentence BERT를 이용한 내용 기반 국문 저널추천 시스템)

  • Yongwoo Kim;Daeyoung Kim;Hyunhee Seo;Young-Min Kim
    • Journal of Intelligence and Information Systems
    • /
    • v.29 no.3
    • /
    • pp.37-55
    • /
    • 2023
  • With the development of electronic journals and the emergence of various interdisciplinary studies, the selection of journals for publication has become a new challenge for researchers. Even if a paper is of high quality, it may face rejection due to a mismatch between the paper's topic and the scope of the journal. While research on assisting researchers in journal selection has been actively conducted in English, the same cannot be said for Korean journals. In this study, we propose a system that recommends Korean journals for submission. Firstly, we utilize SBERT (Sentence BERT) to embed abstracts of previously published papers at the document level, compare the similarity between new documents and published papers, and recommend journals accordingly. Next, the order of recommended journals is determined by considering the similarity of abstracts, keywords, and title. Subsequently, journals that are similar to the top recommended journal from previous stage are added by using a dictionary of words constructed for each journal, thereby enhancing recommendation diversity. The recommendation system, built using this approach, achieved a Top-10 accuracy level of 76.6%, and the validity of the recommendation results was confirmed through user feedback. Furthermore, it was found that each step of the proposed framework contributes to improving recommendation accuracy. This study provides a new approach to recommending academic journals in the Korean language, which has not been actively studied before, and it has also practical implications as the proposed framework can be easily applied to services.