• 제목/요약/키워드: TFIDF

검색결과 32건 처리시간 0.016초

TF-IDF와 소설 텍스트의 구조를 이용한 주제어 추출 연구 (Study on Extraction of Keywords Using TF-IDF and Text Structure of Novels)

  • 유은순;최건희;김승훈
    • 한국컴퓨터정보학회논문지
    • /
    • 제20권2호
    • /
    • pp.121-129
    • /
    • 2015
  • 도서 상품에 대한 정보량이 폭증하면서 고객이 도서 선택에 어려움을 겪는 상황이 발생하고 있다. 이에 따라 고객에게 적합한 도서 정보를 제공하여 구매를 유도하는 도서 추천시스템의 중요성이 커지고 있다. 하지만 도서의 서지정보나 사용자 정보 등을 이용한 기존의 추천시스템은 추천 결과의 신뢰도에 문제를 드러내고 있기 때문에 도서 본문 텍스트의 의미적 정보를 추천시스템에 반영하는 것이 필요하다. 따라서 본 논문은 이에 대한 선행연구로 TF-IDF기법과 소설의 외형적 구조를 이용한 소설 텍스트의 주제어 추출 방법을 제안하였다. 이를 위해 100권의 소설텍스트를 수집하고 각각의 소설을 머리말, 대화문, 비대화문, 맺음말의 4개의 구조로 분리한 후 TF-IDF 가중치를 계산하였다. 실험결과 본문 텍스트만을 이용했을 때 보다 머리말과 맺음말을 포함하고 대화문에 가중치를 높게 부여하였을 때 주제어의 추출 정확도가 42.1%의 성능 향상을 보였다.

유사도 알고리즘을 활용한 시맨틱 프로세스 검색방안 (Semantic Process Retrieval with Similarity Algorithms)

  • 이홍주
    • Asia pacific journal of information systems
    • /
    • 제18권1호
    • /
    • pp.79-96
    • /
    • 2008
  • One of the roles of the Semantic Web services is to execute dynamic intra-organizational services including the integration and interoperation of business processes. Since different organizations design their processes differently, the retrieval of similar semantic business processes is necessary in order to support inter-organizational collaborations. Most approaches for finding services that have certain features and support certain business processes have relied on some type of logical reasoning and exact matching. This paper presents our approach of using imprecise matching for expanding results from an exact matching engine to query the OWL(Web Ontology Language) MIT Process Handbook. MIT Process Handbook is an electronic repository of best-practice business processes. The Handbook is intended to help people: (1) redesigning organizational processes, (2) inventing new processes, and (3) sharing ideas about organizational practices. In order to use the MIT Process Handbook for process retrieval experiments, we had to export it into an OWL-based format. We model the Process Handbook meta-model in OWL and export the processes in the Handbook as instances of the meta-model. Next, we need to find a sizable number of queries and their corresponding correct answers in the Process Handbook. Many previous studies devised artificial dataset composed of randomly generated numbers without real meaning and used subjective ratings for correct answers and similarity values between processes. To generate a semantic-preserving test data set, we create 20 variants for each target process that are syntactically different but semantically equivalent using mutation operators. These variants represent the correct answers of the target process. We devise diverse similarity algorithms based on values of process attributes and structures of business processes. We use simple similarity algorithms for text retrieval such as TF-IDF and Levenshtein edit distance to devise our approaches, and utilize tree edit distance measure because semantic processes are appeared to have a graph structure. Also, we design similarity algorithms considering similarity of process structure such as part process, goal, and exception. Since we can identify relationships between semantic process and its subcomponents, this information can be utilized for calculating similarities between processes. Dice's coefficient and Jaccard similarity measures are utilized to calculate portion of overlaps between processes in diverse ways. We perform retrieval experiments to compare the performance of the devised similarity algorithms. We measure the retrieval performance in terms of precision, recall and F measure? the harmonic mean of precision and recall. The tree edit distance shows the poorest performance in terms of all measures. TF-IDF and the method incorporating TF-IDF measure and Levenshtein edit distance show better performances than other devised methods. These two measures are focused on similarity between name and descriptions of process. In addition, we calculate rank correlation coefficient, Kendall's tau b, between the number of process mutations and ranking of similarity values among the mutation sets. In this experiment, similarity measures based on process structure, such as Dice's, Jaccard, and derivatives of these measures, show greater coefficient than measures based on values of process attributes. However, the Lev-TFIDF-JaccardAll measure considering process structure and attributes' values together shows reasonably better performances in these two experiments. For retrieving semantic process, we can think that it's better to consider diverse aspects of process similarity such as process structure and values of process attributes. We generate semantic process data and its dataset for retrieval experiment from MIT Process Handbook repository. We suggest imprecise query algorithms that expand retrieval results from exact matching engine such as SPARQL, and compare the retrieval performances of the similarity algorithms. For the limitations and future work, we need to perform experiments with other dataset from other domain. And, since there are many similarity values from diverse measures, we may find better ways to identify relevant processes by applying these values simultaneously.