• Title/Summary/Keyword: Korean text classification

Search Result 413, Processing Time 0.021 seconds

A Study on the Classification Model of Overseas Infringing Websites based on Web Hierarchy Similarity Analysis using GNN (GNN을 이용한 웹사이트 Hierarchy 유사도 분석 기반 해외 침해 사이트 분류 모델 연구)

  • Ju-hyeon Seo;Sun-mo Yoo;Jong-hwa Park;Jin-joo Park;Tae-jin Lee
    • Convergence Security Journal
    • /
    • v.23 no.2
    • /
    • pp.47-54
    • /
    • 2023
  • The global popularity of K-content(Korean Wave) has led to a continuous increase in copyright infringement cases involving domestic works, not only within the country but also overseas. In response to this trend, there is active research on technologies for detecting illegal distribution sites of domestic copyrighted materials, with recent studies utilizing the characteristics of domestic illegal distribution sites that often include a significant number of advertising banners. However, the application of detection techniques similar to those used domestically is limited for overseas illegal distribution sites. These sites may not include advertising banners or may have significantly fewer ads compared to domestic sites, making the application of detection technologies used domestically challenging. In this study, we propose a detection technique based on the similarity comparison of links and text trees, leveraging the characteristic of including illegal sharing posts and images of copyrighted materials in a similar hierarchical structure. Additionally, to accurately compare the similarity of large-scale trees composed of a massive number of links, we utilize Graph Neural Network (GNN). The experiments conducted in this study demonstrated a high accuracy rate of over 95% in classifying regular sites and sites involved in the illegal distribution of copyrighted materials. Applying this algorithm to automate the detection of illegal distribution sites is expected to enable swift responses to copyright infringements.

Rule-based Speech Recognition Error Correction for Mobile Environment (모바일 환경을 고려한 규칙기반 음성인식 오류교정)

  • Kim, Jin-Hyung;Park, So-Young
    • Journal of the Korea Society of Computer and Information
    • /
    • v.17 no.10
    • /
    • pp.25-33
    • /
    • 2012
  • In this paper, we propose a rule-based model to correct errors in a speech recognition result in the mobile device environment. The proposed model considers the mobile device environment with limited resources such as processing time and memory, as follows. In order to minimize the error correction processing time, the proposed model removes some processing steps such as morphological analysis and the composition and decomposition of syllable. Also, the proposed model utilizes the longest match rule selection method to generate one error correction candidate per point, assumed that an error occurs. For the purpose of deploying memory resource, the proposed model uses neither the Eojeol dictionary nor the morphological analyzer, and stores a combined rule list without any classification. Considering the modification and maintenance of the proposed model, the error correction rules are automatically extracted from a training corpus. Experimental results show that the proposed model improves 5.27% on the precision and 5.60% on the recall based on Eojoel unit for the speech recognition result.

Surgical prevention of terminal neuroma and phantom limb pain: a literature review

  • Bogdasarian, Ronald N.;Cai, Steven B.;Tran, Bao Ngoc N.;Ignatiuk, Ashley;Lee, Edward S.
    • Archives of Plastic Surgery
    • /
    • v.48 no.3
    • /
    • pp.310-322
    • /
    • 2021
  • The incidence of extremity amputation is estimated at about 200,000 cases annually. Over 25% of patients suffer from terminal neuroma or phantom limb pain (TNPLP), resulting in pain, inability to wear a prosthetic device, and lost work. Once TNPLP develops, there is no definitive cure. Therefore, there has been an emerging focus on TNPLP prevention. We examined the current literature on TNPLP prevention in patients undergoing extremity amputation. A literature review was performed using Ovid Medline, Cochrane Collaboration Library, and Google Scholar to identify all original studies that addressed surgical prophylaxis against TNPLP. The search was conducted using both Medical Subject Headings and free-text using the terms "phantom limb pain," "amputation neuroma," and "surgical prevention of amputation neuroma." Fifteen studies met the inclusion criteria, including six prospective trials, two comprehensive literature reviews, four retrospective chart reviews, and three case series/technique reviews. Five techniques were identified, and each was incorporated into a targetbased classification system. A small but growing body of literature exists regarding the surgical prevention of TNPLP. Targeted muscle reinnervation (TMR), a form of physiologic target reassignment, has the greatest momentum in the academic surgical community, with multiple recent prospective studies demonstrating superior prevention of TNPLP. Neurorrhaphy and transposition with implantation are supported by less robust evidence, but merit future study as alternatives to TMR.

Analysis of Plant Species in Elementary School Textbooks in South Korea

  • Kwon, Min Hyeong
    • Journal of People, Plants, and Environment
    • /
    • v.24 no.5
    • /
    • pp.485-498
    • /
    • 2021
  • Background and objective: This study was conducted to find out the status of plant utilization in the current textbooks by analyzing the plants by grade and subject in the national textbooks for all elementary school grades in the 2015 revised curriculum in Korea. Methods: The data collected was analyzed using Microsoft Office Excel to obtain the frequency and ratio of collected plant data and SPSS for Windows 26.0 to determine learning content areas by grade and the R program was used to visualize the learning content areas. Results: A total of 232 species of plants were presented 1,047 times in the national textbooks. Based on an analysis of the plants presented by grade, the species that continued to increase in the lower grades tended to decrease in the fifth and sixth grades, the upper grades of elementary school. As for the number and frequency of plant species by subject, Korean Language had the highest number and frequency of plant species. The types of presentation of plants in textbooks were mainly text, followed by illustrations and photos of plants, which were largely used in first grade textbooks. In addition, as for the area of learning contents in which plants are used, in the lower grades, plants were used in the linguistic domain, and in the upper grades, in the botanical and environmental domains of the natural sciences. Herbaceous plants were presented more than woody plants, and according to an analysis of the plants based on the classification of crops, horticultural crops were presented the most, followed by food crops. Out of horticultural crops, flowering plants were found the most diversity with 63 species, but the plants that appeared most frequently were fruit trees that are commonly encountered in real life. Conclusion: As a result of this study, various plant species were included in elementary school textbooks, but most of them were horticultural crops encountered in real life depending on their use. Nevertheless, plant species with high frequency have continued a similar trend of frequency from the previous curriculums. Therefore, in the next curriculum, plant learning materials should be reflected according to social changes and students' preference for plants.

An Accurate Log Object Recognition Technique

  • Jiho, Ju;Byungchul, Tak
    • Journal of the Korea Society of Computer and Information
    • /
    • v.28 no.2
    • /
    • pp.89-97
    • /
    • 2023
  • In this paper, we propose factors that make log analysis difficult and design technique for detecting various objects embedded in the logs which helps in the subsequent analysis. In today's IT systems, logs have become a critical source data for many advanced AI analysis techniques. Although logs contain wealth of useful information, it is difficult to directly apply techniques since logs are semi-structured by nature. The factors that interfere with log analysis are various objects such as file path, identifiers, JSON documents, etc. We have designed a BERT-based object pattern recognition algorithm for these objects and performed object identification. Object pattern recognition algorithms are based on object definition, GROK pattern, and regular expression. We find that simple pattern matchings based on known patterns and regular expressions are ineffective. The results show significantly better accuracy than using only the patterns and regular expressions. In addition, in the case of the BERT model, the accuracy of classifying objects reached as high as 99%.

Classification of Unstructured Customer Complaint Text Data for Potential Vehicle Defect Detection (잠재적 차량 결함 탐지를 위한 비정형 고객불만 텍스트 데이터 분류)

  • Ju Hyun Jo;Chang Su Ok;Jae Il Park
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • v.46 no.2
    • /
    • pp.72-81
    • /
    • 2023
  • This research proposes a novel approach to tackle the challenge of categorizing unstructured customer complaints in the automotive industry. The goal is to identify potential vehicle defects based on the findings of our algorithm, which can assist automakers in mitigating significant losses and reputational damage caused by mass claims. To achieve this goal, our model uses the Word2Vec method to analyze large volumes of unstructured customer complaint data from the National Highway Traffic Safety Administration (NHTSA). By developing a score dictionary for eight pre-selected criteria, our algorithm can efficiently categorize complaints and detect potential vehicle defects. By calculating the score of each complaint, our algorithm can identify patterns and correlations that can indicate potential defects in the vehicle. One of the key benefits of this approach is its ability to handle a large volume of unstructured data, which can be challenging for traditional methods. By using machine learning techniques, we can extract meaningful insights from customer complaints, which can help automakers prioritize and address potential defects before they become widespread issues. In conclusion, this research provides a promising approach to categorize unstructured customer complaints in the automotive industry and identify potential vehicle defects. By leveraging the power of machine learning, we can help automakers improve the quality of their products and enhance customer satisfaction. Further studies can build upon this approach to explore other potential applications and expand its scope to other industries.

Stock Market Prediction Using Sentiment on YouTube Channels (유튜브 주식채널의 감성을 활용한 코스피 수익률 등락 예측)

  • Su-Ji, Cho;Cheol-Won Yang;Ki-Kwang Lee
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • v.46 no.2
    • /
    • pp.102-108
    • /
    • 2023
  • Recently in Korea, YouTube stock channels increased rapidly due to the high social interest in the stock market during the COVID-19 period. Accordingly, the role of new media channels such as YouTube is attracting attention in the process of generating and disseminating market information. Nevertheless, prior studies on the market forecasting power of YouTube stock channels remain insignificant. In this study, the market forecasting power of the information from the YouTube stock channel was examined and compared with traditional news media. To measure information from each YouTube stock channel and news media, positive and negative opinions were extracted. As a result of the analysis, opinion in channels operated by media outlets were found to be leading indicators of KOSPI market returns among YouTube stock channels. The prediction accuracy by using logistic regression model show 74%. On the other hand, Sampro TV, a popular YouTube stock channel, and the traditional news media simply reported the market situation of the day or instead showed a tendency to lag behind the market. This study is differentiated from previous studies in that it verified the market predictive power of the information provided by the YouTube stock channel, which has recently shown a growing trend in Korea. In the future, the results of advanced analysis can be confirmed by expanding the research results for individual stocks.

A Study on the Construction of Financial-Specific Language Model Applicable to the Financial Institutions (금융권에 적용 가능한 금융특화언어모델 구축방안에 관한 연구)

  • Jae Kwon Bae
    • Journal of Korea Society of Industrial Information Systems
    • /
    • v.29 no.3
    • /
    • pp.79-87
    • /
    • 2024
  • Recently, the importance of pre-trained language models (PLM) has been emphasized for natural language processing (NLP) such as text classification, sentiment analysis, and question answering. Korean PLM shows high performance in NLP in general-purpose domains, but is weak in domains such as finance, medicine, and law. The main goal of this study is to propose a language model learning process and method to build a financial-specific language model that shows good performance not only in the financial domain but also in general-purpose domains. The five steps of the financial-specific language model are (1) financial data collection and preprocessing, (2) selection of model architecture such as PLM or foundation model, (3) domain data learning and instruction tuning, (4) model verification and evaluation, and (5) model deployment and utilization. Through this, a method for constructing pre-learning data that takes advantage of the characteristics of the financial domain and an efficient LLM training method, adaptive learning and instruction tuning techniques, were presented.

Enhancing Search Functionality for Website Posts and Product Reviews: Improving BM25 Ranking Algorithm Performance Using the ResNet-Transformer Model

  • Hong-Ju Yang;In-Yeop Choi
    • Journal of the Korea Society of Computer and Information
    • /
    • v.29 no.11
    • /
    • pp.67-77
    • /
    • 2024
  • This paper proposes a method to improve the search functionality for website posts and product reviews by using a ResNet-Transformer model in conjunction with the BM25 ranking algorithm. BM25 is a widely used algorithm in text-based search that ranks documents by evaluating their relevance to user queries. However, it has limitations in capturing local features of words and understanding the context of a sentences. To address these issues, this study applies a classification approach that combines the ResNet model, which excels at extracting local features, with the Transformer model, known for its strong contextual understanding, as weights for BM25. Experimental results demonstrate that the proposed method improves the nDCG metric by 9.38% and the aP@5 metric by 11.82% compared to BM25 alone. This suggests that implementing this method in search engines across various websites can provide more accurate results for post and review searches.

Investigating Dynamic Mutation Process of Issues Using Unstructured Text Analysis (부도예측을 위한 KNN 앙상블 모형의 동시 최적화)

  • Min, Sung-Hwan
    • Journal of Intelligence and Information Systems
    • /
    • v.22 no.1
    • /
    • pp.139-157
    • /
    • 2016
  • Bankruptcy involves considerable costs, so it can have significant effects on a country's economy. Thus, bankruptcy prediction is an important issue. Over the past several decades, many researchers have addressed topics associated with bankruptcy prediction. Early research on bankruptcy prediction employed conventional statistical methods such as univariate analysis, discriminant analysis, multiple regression, and logistic regression. Later on, many studies began utilizing artificial intelligence techniques such as inductive learning, neural networks, and case-based reasoning. Currently, ensemble models are being utilized to enhance the accuracy of bankruptcy prediction. Ensemble classification involves combining multiple classifiers to obtain more accurate predictions than those obtained using individual models. Ensemble learning techniques are known to be very useful for improving the generalization ability of the classifier. Base classifiers in the ensemble must be as accurate and diverse as possible in order to enhance the generalization ability of an ensemble model. Commonly used methods for constructing ensemble classifiers include bagging, boosting, and random subspace. The random subspace method selects a random feature subset for each classifier from the original feature space to diversify the base classifiers of an ensemble. Each ensemble member is trained by a randomly chosen feature subspace from the original feature set, and predictions from each ensemble member are combined by an aggregation method. The k-nearest neighbors (KNN) classifier is robust with respect to variations in the dataset but is very sensitive to changes in the feature space. For this reason, KNN is a good classifier for the random subspace method. The KNN random subspace ensemble model has been shown to be very effective for improving an individual KNN model. The k parameter of KNN base classifiers and selected feature subsets for base classifiers play an important role in determining the performance of the KNN ensemble model. However, few studies have focused on optimizing the k parameter and feature subsets of base classifiers in the ensemble. This study proposed a new ensemble method that improves upon the performance KNN ensemble model by optimizing both k parameters and feature subsets of base classifiers. A genetic algorithm was used to optimize the KNN ensemble model and improve the prediction accuracy of the ensemble model. The proposed model was applied to a bankruptcy prediction problem by using a real dataset from Korean companies. The research data included 1800 externally non-audited firms that filed for bankruptcy (900 cases) or non-bankruptcy (900 cases). Initially, the dataset consisted of 134 financial ratios. Prior to the experiments, 75 financial ratios were selected based on an independent sample t-test of each financial ratio as an input variable and bankruptcy or non-bankruptcy as an output variable. Of these, 24 financial ratios were selected by using a logistic regression backward feature selection method. The complete dataset was separated into two parts: training and validation. The training dataset was further divided into two portions: one for the training model and the other to avoid overfitting. The prediction accuracy against this dataset was used to determine the fitness value in order to avoid overfitting. The validation dataset was used to evaluate the effectiveness of the final model. A 10-fold cross-validation was implemented to compare the performances of the proposed model and other models. To evaluate the effectiveness of the proposed model, the classification accuracy of the proposed model was compared with that of other models. The Q-statistic values and average classification accuracies of base classifiers were investigated. The experimental results showed that the proposed model outperformed other models, such as the single model and random subspace ensemble model.