• Title/Summary/Keyword: 의미 연관성 기반 추출

Search Result 50, Processing Time 0.027 seconds

Multi-document Summarization Based on Cluster using Term Co-occurrence (단어의 공기정보를 이용한 클러스터 기반 다중문서 요약)

  • Lee, Il-Joo;Kim, Min-Koo
    • Journal of KIISE:Software and Applications
    • /
    • v.33 no.2
    • /
    • pp.243-251
    • /
    • 2006
  • In multi-document summarization by means of salient sentence extraction, it is important to remove redundant information. In the removal process, the similarities and differences of sentences are considered. In this paper, we propose a method for multi-document summarization which extracts salient sentences without having redundant sentences by way of cohesive term clustering method that utilizes co-occurrence Information. In the cohesive term clustering method, we assume that each term does not exist independently, but rather it is related to each other in meanings. To find the relations between terms, we cluster sentences according to topics and use the co-occurrence information oi terms in the same topic. We conduct experimental tests with the DUC(Document Understanding Conferences) data. In the tests, our method shows better performance of summarization than other summarization methods which use term co-occurrence information based on term cohesion of document or sentence unit, and simple statistical information.

WV-BTM: A Technique on Improving Accuracy of Topic Model for Short Texts in SNS (WV-BTM: SNS 단문의 주제 분석을 위한 토픽 모델 정확도 개선 기법)

  • Song, Ae-Rin;Park, Young-Ho
    • Journal of Digital Contents Society
    • /
    • v.19 no.1
    • /
    • pp.51-58
    • /
    • 2018
  • As the amount of users and data of NS explosively increased, research based on SNS Big data became active. In social mining, Latent Dirichlet Allocation(LDA), which is a typical topic model technique, is used to identify the similarity of each text from non-classified large-volume SNS text big data and to extract trends therefrom. However, LDA has the limitation that it is difficult to deduce a high-level topic due to the semantic sparsity of non-frequent word occurrence in the short sentence data. The BTM study improved the limitations of this LDA through a combination of two words. However, BTM also has a limitation that it is impossible to calculate the weight considering the relation with each subject because it is influenced more by the high frequency word among the combined words. In this paper, we propose a technique to improve the accuracy of existing BTM by reflecting semantic relation between words.

A Text Mining-based Intrusion Log Recommendation in Digital Forensics (디지털 포렌식에서 텍스트 마이닝 기반 침입 흔적 로그 추천)

  • Ko, Sujeong
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.2 no.6
    • /
    • pp.279-290
    • /
    • 2013
  • In digital forensics log files have been stored as a form of large data for the purpose of tracing users' past behaviors. It is difficult for investigators to manually analysis the large log data without clues. In this paper, we propose a text mining technique for extracting intrusion logs from a large log set to recommend reliable evidences to investigators. In the training stage, the proposed method extracts intrusion association words from a training log set by using Apriori algorithm after preprocessing and the probability of intrusion for association words are computed by combining support and confidence. Robinson's method of computing confidences for filtering spam mails is applied to extracting intrusion logs in the proposed method. As the results, the association word knowledge base is constructed by including the weights of the probability of intrusion for association words to improve the accuracy. In the test stage, the probability of intrusion logs and the probability of normal logs in a test log set are computed by Fisher's inverse chi-square classification algorithm based on the association word knowledge base respectively and intrusion logs are extracted from combining the results. Then, the intrusion logs are recommended to investigators. The proposed method uses a training method of clearly analyzing the meaning of data from an unstructured large log data. As the results, it complements the problem of reduction in accuracy caused by data ambiguity. In addition, the proposed method recommends intrusion logs by using Fisher's inverse chi-square classification algorithm. So, it reduces the rate of false positive(FP) and decreases in laborious effort to extract evidences manually.

Inferring Undiscovered Public Knowledge by Using Text Mining-driven Graph Model (텍스트 마이닝 기반의 그래프 모델을 이용한 미발견 공공 지식 추론)

  • Heo, Go Eun;Song, Min
    • Journal of the Korean Society for information Management
    • /
    • v.31 no.1
    • /
    • pp.231-250
    • /
    • 2014
  • Due to the recent development of Information and Communication Technologies (ICT), the amount of research publications has increased exponentially. In response to this rapid growth, the demand of automated text processing methods has risen to deal with massive amount of text data. Biomedical text mining discovering hidden biological meanings and treatments from biomedical literatures becomes a pivotal methodology and it helps medical disciplines reduce the time and cost. Many researchers have conducted literature-based discovery studies to generate new hypotheses. However, existing approaches either require intensive manual process of during the procedures or a semi-automatic procedure to find and select biomedical entities. In addition, they had limitations of showing one dimension that is, the cause-and-effect relationship between two concepts. Thus;this study proposed a novel approach to discover various relationships among source and target concepts and their intermediate concepts by expanding intermediate concepts to multi-levels. This study provided distinct perspectives for literature-based discovery by not only discovering the meaningful relationship among concepts in biomedical literature through graph-based path interference but also being able to generate feasible new hypotheses.

Attention-based word correlation analysis system for big data analysis (빅데이터 분석을 위한 어텐션 기반의 단어 연관관계 분석 시스템)

  • Chi-Gon, Hwang;Chang-Pyo, Yoon;Soo-Wook, Lee
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.27 no.1
    • /
    • pp.41-46
    • /
    • 2023
  • Recently, big data analysis can use various techniques according to the development of machine learning. Big data collected in reality lacks an automated refining technique for the same or similar terms based on semantic analysis of the relationship between words. Since most of the big data is described in general sentences, it is difficult to understand the meaning and terms of the sentences. To solve these problems, it is necessary to understand the morphological analysis and meaning of sentences. Accordingly, NLP, a technique for analyzing natural language, can understand the word's relationship and sentences. Among the NLP techniques, the transformer has been proposed as a way to solve the disadvantages of RNN by using self-attention composed of an encoder-decoder structure of seq2seq. In this paper, transformers are used as a way to form associations between words in order to understand the words and phrases of sentences extracted from big data.

Pattern Analysis-Based Query Expansion for Enhancing Search Convenience (검색 편의성 향상을 위한 패턴 분석 기반 질의어 확장)

  • Jeon, Seo-In;Park, Gun-Woo;Nam, Kwang-Woo;Ryu, Keun-Ho
    • Journal of Korea Society of Industrial Information Systems
    • /
    • v.17 no.2
    • /
    • pp.65-72
    • /
    • 2012
  • In the 21st century of information systems, the amount of information resources are ever increasing and the role of information searching system is becoming criticalto easily acquire required information from the web. Generally, it requires the user to have enough pre-knowledge and superior capabilities to identify keywords of information to effectively search the web. However, most of the users undertake searching of the information without holding enough pre-knowledge and spend a lot of time associating key words which are related to their required information. Furthermore, many search engines support the keywords searching system but this only provides collection of similar words, and do not provide the user with exact relational search information with the keywords. Therefore this research report proposes a method of offering expanded user relationship search keywords by analyzing user query patterns to provide the user a system, which conveniently support their searching of the information.

A Study on Ontology Generation by Machine Learning in Big Data (빅 데이터에서 기계학습을 통한 온톨로지 생성에 관한 연구)

  • Hwang, Chi-Gon;Yoon, Chang-Pyo
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2018.10a
    • /
    • pp.645-646
    • /
    • 2018
  • Recently, the concept of machine learning has been introduced as a decision making method through data processing. Machine learning uses the results of running based on existing data as a means of decision making. The data generated by the development of technology is vast. This data is called big data. It is important to extract the necessary data from these data. In this paper, we propose a method for extracting related data for constructing an ontology through machine learning. The results of machine learning can be given a relationship from a semantic perspective. it can be added to the ontology to support relationships depending on the needs of the application.

  • PDF

User's Emotional Preference on PC OS GUI - Though Semantic Differential Method (PC OS GUI 의 사용자 감성에 관한 연구 - 의미분별 척도법을 활용한 사용자 감성 선호도 분석)

  • Moon, Hyun-Jung;Lee, Jung-Yeun
    • 한국HCI학회:학술대회논문집
    • /
    • 2008.02b
    • /
    • pp.30-35
    • /
    • 2008
  • The purpose of this study is to analyze and define user's emotional satisfaction factors to the PC OS GUI image. The study is to investigate the relationship between PC OS GUI Image and Sensitive Vocabula교 based on user's emotional preference. 47 user preferred sensitive words are collected by the initial survey. Through the similarity test, 47 words are narrowed down to 20 comprehend words. The semantic differential methods is used in the final survey with 5 step questionnaire. From this process, user preferred the GUI design that is vocabularized as Clear, Easy, Safety, Stability. Additionally, the result shows that the image of Clear is related to Safety and the image of Easy is related to Stability. The result of the study could be used in design PC OS GUI as base data.

  • PDF

Genetic Polymorphisms of SLC8A1 Are Associated with Hypertension and Left Ventricular Hypertrophy in the Korean Population (한국인에서 SLC8A1의 유전적 다형성과 고혈압 및 좌심실 비대와 연관 연구)

  • Park, Hye-Jeong;Kim, Sung-Soo;Jin, Hyun-Seok
    • Korean Journal of Clinical Laboratory Science
    • /
    • v.51 no.3
    • /
    • pp.286-293
    • /
    • 2019
  • Hypertension (HTN) is one of the major chronic diseases, and HTN is defined as being in a state of continuous high blood pressure. Left ventricular hypertrophy (LVH) is a condition in which the mass of the left ventricle has increased, and HTN is a leading cause of LVH. HTN and LVH are known to be caused by the interaction of environmental factors and genetic factors. It has been reported that the polymorphisms of SLC8A1, among the genetic factors that affect high blood pressure, are related to salt sensitivity hypertension. In this study, the genetic polymorphisms of SLC8A1 were chosen based on the Korean Genome and Epidemiology data. Logistic regression analysis was then performed for HTN and LVH. Linear regression analysis was also performed for systolic blood pressure (SBP) and diastolic blood pressure (DBP). As a result, 5 SNPs showed statistically significant associations (P<0.05) with HTN, and 10 SNPs showed statistically significant associations with LVH. rs1002671 and rs9789739 showed significant correlation at the same time with HTN and LVH. These results suggest that the polymorphisms of the SLC8A1 gene are linked to the development of HTN and LVH in Koreans. We expect these results to help us understand the pathogenic mechanisms for HTN and LVH.

Relation Extraction based on Composite Kernel combining Pattern Similarity of Predicate-Argument Structure (술어-논항 구조의 패턴 유사도를 결합한 혼합 커널 기반관계 추출)

  • Jeong, Chang-Hoo;Choi, Sung-Pil;Choi, Yun-Soo;Song, Sa-Kwang;Chun, Hong-Woo
    • Journal of Internet Computing and Services
    • /
    • v.12 no.5
    • /
    • pp.73-85
    • /
    • 2011
  • Lots of valuable textual information is used to extract relations between named entities from literature. Composite kernel approach is proposed in this paper. The composite kernel approach calculates similarities based on the following information:(1) Phrase structure in convolution parse tree kernel that has shown encouraging results. (2) Predicate-argument structure patterns. In other words, the approach deals with syntactic structure as well as semantic structure using a reciprocal method. The proposed approach was evaluated using various types of test collections and it showed the better performance compared with those of previous approach using only information from syntactic structures. In addition, it showed the better performance than those of the state of the art approach.