• Journal of Information Processing Systems
• /
• 제19권5호
• /
• pp.631-640
• /
• 2023

Short-text similarity calculation is one of the hot issues in natural language processing research. The conventional keyword-overlap similarity algorithms merely consider the lexical item information and neglect the effect of the word order. And some of its optimized algorithms combine the word order, but the weights are hard to be determined. In the paper, viewing the keyword-overlap similarity algorithm, the short English text similarity algorithm based on lexical chunk theory (LC-SETSA) is proposed, which introduces the lexical chunk theory existing in cognitive psychology category into the short English text similarity calculation for the first time. The lexical chunks are applied to segment short English texts, and the segmentation results demonstrate the semantic connotation and the fixed word order of the lexical chunks, and then the overlap similarity of the lexical chunks is calculated accordingly. Finally, the comparative experiments are carried out, and the experimental results prove that the proposed algorithm of the paper is feasible, stable, and effective to a large extent.

• Journal of Information Processing Systems
• /
• 제13권4호
• /
• pp.863-875
• /
• 2017

The traditional text similarity measurement methods based on word frequency vector ignore the semantic relationships between words, which has become the obstacle to text similarity calculation, together with the high-dimensionality and sparsity of document vector. To address the problems, the improved singular value decomposition is used to reduce dimensionality and remove noises of the text representation model. The optimal number of singular values is analyzed and the semantic relevance between words can be calculated in constructed semantic space. An inverted index construction algorithm and the similarity definitions between vectors are proposed to calculate the similarity between two documents on the semantic level. The experimental results on benchmark corpus demonstrate that the proposed method promotes the evaluation metrics of F-measure.

• International Journal of Contents
• /
• 제13권4호
• /
• pp.70-79
• /
• 2017

Images are an important element in patents and many experts use images to analyze a patent or to check differences between patents. However, there is little research on image analysis for patents partly because image processing is an advanced technology and typically patent images consist of visual parts as well as of text and numbers. This study suggests two methods for using image processing; the Scale Invariant Feature Transform(SIFT) algorithm and Optical Character Recognition(OCR). The first method which works with SIFT uses image feature points. Through feature matching, it can be applied to calculate the similarity between documents containing these images. And in the second method, OCR is used to extract text from the images. By using numbers which are extracted from an image, it is possible to extract the corresponding related text within the text passages. Subsequently, document similarity can be calculated based on the extracted text. Through comparing the suggested methods and an existing method based only on text for calculating the similarity, the feasibility is achieved. Additionally, the correlation between both the similarity measures is low which shows that they capture different aspects of the patent content.

• 한국정보통신학회:학술대회논문집
• /
• 한국정보통신학회 2021년도 춘계학술대회
• /
• pp.441-443
• /
• 2021

문장 또는 텍스트 유사도란 두 가지 문장의 유사한 정도를 나타내는 척도이다. 텍스트의 유사도를 측정하는 기법으로 자카드 유사도, 코사인 유사도, 유클리디언 유사도, 맨하탄 유사도 등과 같이 있다. 현재 코사인 유사도 기법을 가장 많이 사용하고 있으나 이는 문장에서 단어의 출현 여부와 빈도수에 따른 분석이기 때문에, 의미적 관계에 대한 분석이 부족하다. 이에 우리는 온톨로지를 이용하여 단어 간의 관계를 부여하고, 두 문장에서 공통으로 포함된 단어를 추출할 때 의미적 유사성을 포함함으로써 문장의 유사도에 분석의 효율을 향상하고자 한다.

• 한국컴퓨터정보학회논문지
• /
• 제21권11호
• /
• pp.85-91
• /
• 2016

This paper proposes a collective approach to cited text recognition by exploiting a set of citing text from different articles citing the same article. First, the proposed method gathers highly-ranked cited sentences from the cited article using a group of citing text to create a collective information of probable cited sentences. Then, such collective information is used to determine final cited sentences among highly-ranked sentences from similarity-based cited text recognition. Experiments have been conducted on the data set which consists of research articles from a computational linguistics domain. Evaluation results showed that the proposed method could improve the performance of similarity-based baseline approaches.

• 한국정보과학회논문지:소프트웨어및응용
• /
• 제26권1호
• /
• pp.167-167
• /
• 1999

The majority of text categorization systems use the term-based classification method. However, because of too many terms, this method is not effective to classify the documents in areal-time environment. This paper presents a real-time concept-based text categorization system,which classifies texts using thesaurus. The system consists of a Korean morphological analyzer, athesaurus tool, and a probability-vector similarity measurer. The thesaurus tool acquires the meaningsof input terms and represents the text with not the term-vector but the concept-vector. Because theconcept-vector consists of semantic units with the small size, it makes the system enable to analyzethe text with real-time. As representing the meanings of the text, the vector supports theconcept-based classification. The probability-vector similarity measurer decides the subject of the textby calculating the vector similarity between the input text and each subject. In the experimentalresults, we show that the proposed system can effectively analyze texts with real-time and do aconcept-based classification. Moreover, the experiment informs that we must expand the thesaurustool for the better system.

• 대한수학교육학회지:수학교육학연구
• /
• 제19권1호
• /
• pp.1-24
• /
• 2009

비와 닮음의 활용은 고대로부터 일상생활에서 필요한 것들이었고, 유클리드 원론에서도 제 5권에서는 비를 제6권에서는 닮음을 다루고 있다. 본 연구에서는 우리나라 교과서에서 취급하고 있는 비와 닮음의 내용을 유클리드 원론, 일본, 미국의 교과서에서 취급하고 있는 내용들과 비교 분석하였는데, 도입 방법과 내용 전개 방법에 서로 차이가 있음을 알 수 있다. 우리나라의 교과서에서는 비를 도입하면서 미국 일본과 달리 비에 대한 정의 없이 보기 문제를 통해 비를 나타냈으며, 닮음에서는 우리나라와 일본의 교과서가 미국의 교과서와 달리 삼각형의 닮음조건과 삼각형의 변과 한 변에 평행인 선분에 의한 비의 관계를 다루는 순서가 다르며 삼각형의 닮음조건을 직관적으로 증명 없이 공준처럼 사용하고 있다. 이와 같은 도입 방법과 내용 전개 그리고 내용 전개 순서의 차이에 따른 학습지도는 학생들의 수준에 의해 학습내용 이해와 활용에 많은 영향을 줄 수 있다. 보다 바람직한 수학 교육을 위해 현행 모든 교과서와 같이 학습 내용을 일률적인 방법으로 취급하는 것 보다 학생들의 수준을 생각한 다양한 방법으로 취급한 교과서를 제공하는 것이 필요하다.

• International Journal of Computer Science & Network Security
• /
• 제22권4호
• /
• pp.111-118
• /
• 2022

This research article presents the work that is related to the application of different machine learning based similarity techniques on religious text for identifying similarities and differences among its various translations. The dataset includes 10 different English translations of verses (Arabic: Ayah) of two Surahs (chapters) namely, Al-Humazah and An-Nasr. The quantitative similarity values for different translations for the same verse were calculated by using the cosine similarity and semantic similarity. The corpus went through two series of experiments: before pre-processing and after pre-processing. In order to determine the performance of machine learning based similarities, human annotated similarities between translations of two Surahs (chapters) namely Al-Humazah and An-Nasr were recorded to construct the ground truth. The average difference between the human annotated similarity and the cosine similarity for Surah (chapter) Al-Humazah was found to be 1.38 per verse (ayah) per pair of translation. After pre-processing, the average difference increased to 2.24. Moreover, the average difference between human annotated similarity and semantic similarity for Surah (chapter) Al-Humazah was found to be 0.09 per verse (Ayah) per pair of translation. After pre-processing, it increased to 0.78. For the Surah (chapter) An-Nasr, before preprocessing, the average difference between human annotated similarity and cosine similarity was found to be 1.93 per verse (Ayah), per pair of translation. And. After pre-processing, the average difference further increased to 2.47. The average difference between the human annotated similarity and the semantic similarity for Surah An-Nasr before preprocessing was found to be 0.93 and after pre-processing, it was reduced to 0.87 per verse (ayah) per pair of translation. The results showed that as expected, the semantic similarity was proven to be better measurement indicator for calculation of the word meaning.

• 융합신호처리학회논문지
• /
• 제20권4호
• /
• pp.193-198
• /
• 2019

인터넷상에서의 진짜 및 가짜 정보의 범람이 수많은 텍스트 분석에 대한 연구를 이끌었다. 문헌 표기 없이 타인의 저작물을 무단 복제 및 관련 없는 연구결과 조작 등이 한동안 세간의 주목을 이끌었다. 연구 분야에서 표절과 이의 대항 및 감소를 위해 다양한 도구들이 개발되었다. Pearson Spearman 본 연구에서는 코사인 유사성과 및 상관관계를 이용하는 파라미터 및 비 파라미터 방법을 이용하여 문장 유사성을 측정한다. Pearson 코사인 유사성과 상관관계는 가장 높은 유사성 계수를 얻었으나 Spearman 상관관계는 낮은 유사성 계수를 보여주었다. 본 논문에서는 정상성 가정과 편향성에 의존하는 파라미터 방법들에 반하도록 비정상성 가정으로 인한 문장 유사도를 측정하는 데 있어 비 파라미터 방법들을 사용하는 것을 제안한다.

• 한국언어정보학회지:언어와정보
• /
• 제14권2호
• /
• pp.131-157
• /
• 2010

This article reports an attempt to perform a similarity analysis on a collection of 25 culinary manuscripts in Chosun periods using a set of quantitative text analysis methods. Historical culinary texts are valuable resources for linguistic, historic, and cultural studies. We consider the similarity of two texts as the distributional similarities of the functional components of the texts. In the case of culinary texts, text elements such as food names, cooking methods, and ingredients are regarded as functional components. We derive the similarity information from the distributional characteristics of the two key functional components, cooking methods and ingredients. The results are also quantified and visualized to achieve a better understanding of the properties of the individual texts and the collection of the texts as a whole.