• Title/Summary/Keyword: similarity calculation

Search Result 208, Processing Time 0.031 seconds

Word Similarity Calculation by Using the Edit Distance Metrics with Consonant Normalization

  • Kang, Seung-Shik
    • Journal of Information Processing Systems
    • /
    • v.11 no.4
    • /
    • pp.573-582
    • /
    • 2015
  • Edit distance metrics are widely used for many applications such as string comparison and spelling error corrections. Hamming distance is a metric for two equal length strings and Damerau-Levenshtein distance is a well-known metrics for making spelling corrections through string-to-string comparison. Previous distance metrics seems to be appropriate for alphabetic languages like English and European languages. However, the conventional edit distance criterion is not the best method for agglutinative languages like Korean. The reason is that two or more letter units make a Korean character, which is called as a syllable. This mechanism of syllable-based word construction in the Korean language causes an edit distance calculation to be inefficient. As such, we have explored a new edit distance method by using consonant normalization and the normalization factor.

A Collaborative Filtering System Combined with Users' Review Mining : Application to the Recommendation of Smartphone Apps (사용자 리뷰 마이닝을 결합한 협업 필터링 시스템: 스마트폰 앱 추천에의 응용)

  • Jeon, ByeoungKug;Ahn, Hyunchul
    • Journal of Intelligence and Information Systems
    • /
    • v.21 no.2
    • /
    • pp.1-18
    • /
    • 2015
  • Collaborative filtering(CF) algorithm has been popularly used for recommender systems in both academic and practical applications. A general CF system compares users based on how similar they are, and creates recommendation results with the items favored by other people with similar tastes. Thus, it is very important for CF to measure the similarities between users because the recommendation quality depends on it. In most cases, users' explicit numeric ratings of items(i.e. quantitative information) have only been used to calculate the similarities between users in CF. However, several studies indicated that qualitative information such as user's reviews on the items may contribute to measure these similarities more accurately. Considering that a lot of people are likely to share their honest opinion on the items they purchased recently due to the advent of the Web 2.0, user's reviews can be regarded as the informative source for identifying user's preference with accuracy. Under this background, this study proposes a new hybrid recommender system that combines with users' review mining. Our proposed system is based on conventional memory-based CF, but it is designed to use both user's numeric ratings and his/her text reviews on the items when calculating similarities between users. In specific, our system creates not only user-item rating matrix, but also user-item review term matrix. Then, it calculates rating similarity and review similarity from each matrix, and calculates the final user-to-user similarity based on these two similarities(i.e. rating and review similarities). As the methods for calculating review similarity between users, we proposed two alternatives - one is to use the frequency of the commonly used terms, and the other one is to use the sum of the importance weights of the commonly used terms in users' review. In the case of the importance weights of terms, we proposed the use of average TF-IDF(Term Frequency - Inverse Document Frequency) weights. To validate the applicability of the proposed system, we applied it to the implementation of a recommender system for smartphone applications (hereafter, app). At present, over a million apps are offered in each app stores operated by Google and Apple. Due to this information overload, users have difficulty in selecting proper apps that they really want. Furthermore, app store operators like Google and Apple have cumulated huge amount of users' reviews on apps until now. Thus, we chose smartphone app stores as the application domain of our system. In order to collect the experimental data set, we built and operated a Web-based data collection system for about two weeks. As a result, we could obtain 1,246 valid responses(ratings and reviews) from 78 users. The experimental system was implemented using Microsoft Visual Basic for Applications(VBA) and SAS Text Miner. And, to avoid distortion due to human intervention, we did not adopt any refining works by human during the user's review mining process. To examine the effectiveness of the proposed system, we compared its performance to the performance of conventional CF system. The performances of recommender systems were evaluated by using average MAE(mean absolute error). The experimental results showed that our proposed system(MAE = 0.7867 ~ 0.7881) slightly outperformed a conventional CF system(MAE = 0.7939). Also, they showed that the calculation of review similarity between users based on the TF-IDF weights(MAE = 0.7867) leaded to better recommendation accuracy than the calculation based on the frequency of the commonly used terms in reviews(MAE = 0.7881). The results from paired samples t-test presented that our proposed system with review similarity calculation using the frequency of the commonly used terms outperformed conventional CF system with 10% statistical significance level. Our study sheds a light on the application of users' review information for facilitating electronic commerce by recommending proper items to users.

Purchase Transaction Similarity Measure Considering Product Taxonomy (상품 분류 체계를 고려한 구매이력 유사도 측정 기법)

  • Yang, Yu-Jeong;Lee, Ki Yong
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.8 no.9
    • /
    • pp.363-372
    • /
    • 2019
  • A sequence refers to data in which the order exists on the two items, and purchase transaction data in which the products purchased by one customer are listed is one of the representative sequence data. In general, all goods have a product taxonomy, such as category/ sub-category/ sub-sub category, and if they are similar to each other, they are classified into the same category according to their characteristics. Therefore, in this paper, we not only consider the purchase order of products to compare two purchase transaction sequences, but also calculate their similarity by giving a higher score if they are in the same category in spite of their difference. Especially, in order to choose the best similarity measure that directly affects the calculation performance of the purchase transaction sequences, we have compared the performance of three representative similarity measures, the Levenshtein distance, dynamic time warping distance, and the Needleman-Wunsch similarity. We have extended the existing methods to take into account the product taxonomy. For conventional similarity measures, the comparison of goods in two sequences is calculated by simply assigning a value of 0 or 1 according to whether or not the product is matched. However, the proposed method is subdivided to have a value between 0 and 1 using the product taxonomy tree to give a different degree of relevance between the two products, even if they are different products. Through experiments, we have confirmed that the proposed method was measured the similarity more accurately than the previous method. Furthermore, we have confirmed that dynamic time warping distance was the most suitable measure because it considered the degree of association of the product in the sequence and showed good performance for two sequences with different lengths.

Construction of Local Document Management System based on Associative Search

  • Kasagi, Yoshimasa;Yamaguchi, Toru;Takama, Yasufumi
    • Proceedings of the Korean Institute of Intelligent Systems Conference
    • /
    • 2003.09a
    • /
    • pp.146-149
    • /
    • 2003
  • As the information that can collect from the web to local database is increasing, we propose a system that can suggest related local documents when new document arrives. We also propose for constructing an association dictionary using web search engines for similarity calculation. The prototype system is also developed, which is described in detail.

  • PDF

A Partial Pole Placement Method. (부분 극배치 기법)

  • 김성열;김진용;이정국;이금원
    • Proceedings of the Korea Institute of Convergence Signal Processing
    • /
    • 2003.06a
    • /
    • pp.188-192
    • /
    • 2003
  • Pole placement method is widely used in controller design. For the stability of the closed loop system, user-specified desired locations including extra pole locations in the s-plane is chosen and by some procedure, feedback gain is obtained. In this paper, only desired pole location is used, and the calculation process is done for attaining linear quadratic stability. Similarly transformation is used for this. By computer simulations using MATLAB, the effectiveness is shown.

  • PDF

Fractal Analysis of Urban Morphology Considering Distributed Situation of Buildings (건물분포를 고려한 도시형태의 프랙털(Fractal) 해석)

  • Moon, Tae-Heon
    • Journal of the Korean Association of Geographic Information Studies
    • /
    • v.8 no.3
    • /
    • pp.1-10
    • /
    • 2005
  • The purpose of this paper is to conduct an experimental measurement and analysis of cities' morphology. Fractal theory that is an effective tool for evaluating self-similarity and complexity of objects was applied. For the comparative analysis of fractailities and computational verification, two totally different cities in Japan were selected. They are Kitakyushu City, which is a big and fully developed city, and Jinguu Machi of which almost all the area is covered with agricultural land use. After converting vector data to raster data within GIS, fractal dimensions of two cases in Kitakyushu City and one case in Jinguu Machi were calculated. The calculation showed that two parts of Kitakyushu City were already fractal. Jinguu Machi, however, was difficult to find fractality. As a conclusion, fractal was proved to be an useful tool to estimate the shape of cities reflecting their internal spatial structure, that is self-similarity and complexity.

  • PDF

Semantic Image Retrieval Using Color Distribution and Similarity Measurement in WordNet (컬러 분포와 WordNet상의 유사도 측정을 이용한 의미적 이미지 검색)

  • Choi, Jun-Ho;Cho, Mi-Young;Kim, Pan-Koo
    • The KIPS Transactions:PartB
    • /
    • v.11B no.4
    • /
    • pp.509-516
    • /
    • 2004
  • Semantic interpretation of image is incomplete without some mechanism for understanding semantic content that is not directly visible. For this reason, human assisted content-annotation through natural language is an attachment of textual description to image. However, keyword-based retrieval is in the level of syntactic pattern matching. In other words, dissimilarity computation among terms is usually done by using string matching not concept matching. In this paper, we propose a method for computerized semantic similarity calculation In WordNet space. We consider the edge, depth, link type and density as well as existence of common ancestors. Also, we have introduced method that applied similarity measurement on semantic image retrieval. To combine wi#h the low level features, we use the spatial color distribution model. When tested on a image set of Microsoft's 'Design Gallery Line', proposed method outperforms other approach.

Efficient Hardware Architecture for Fast Image Similarity Calculation (고속 영상 유사도 분석을 위한 효율적 하드웨어 구조)

  • Kwon, Soon;Lee, Chung-Hee;Lee, Jong-Hun;Moon, Byung-In;Lee, Yong-Hwan
    • Journal of the Institute of Electronics Engineers of Korea SD
    • /
    • v.48 no.4
    • /
    • pp.6-13
    • /
    • 2011
  • Due to its robustness to illumination change, normalized cross-correlation based similarity measurement is widely used in many machine vision applications. However, its inefficient computation structure is not adequate for real-time embedded vision system. In this paper, we present an efficient hardware architecture based on a normalized cross correlation (NCC) for fast image similarity measure. The proposed architecture simplifies window-sum process of the NCC using the integral-image. Relieving the overhead to constructing integral image, we make it possible to process integral image construction at the same time that pixel sequences are inputted. Also the proposed segmented integral image method can reduce the buffer size for storing integral image data.

An Index-Based Search Method for Performance Improvement of Set-Based Similar Sequence Matching (집합 유사 시퀀스 매칭의 성능 향상을 위한 인덱스 기반 검색 방법)

  • Lee, Juwon;Lim, Hyo-Sang
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.6 no.11
    • /
    • pp.507-520
    • /
    • 2017
  • The set-based similar sequence matching method measures similarity not for an individual data item but for a set grouping multiple data items. In the method, the similarity of two sets is represented as the size of intersection between them. However, there is a critical performances issue for the method in twofold: 1) calculating intersection size is a time consuming process, and 2) the number of set pairs that should be calculated the intersection size is quite large. In this paper, we propose an index-based search method for improving performance of set-based similar sequence matching in order to solve these performance issues. Our method consists of two parts. In the first part, we convert the set similarity problem into the intersection size comparison problem, and then, provide an index structure that accelerates the intersection size calculation. Second, we propose an efficient set-based similar sequence matching method which exploits the proposed index structure. Through experiments, we show that the proposed method reduces the execution time by 30 to 50 times then the existing methods. We also show that the proposed method has scalability since the performance gap becomes larger as the number of data sequences increases.

A Semantic Distance Measurement Model using Weights on the LOD Graph in an LOD-based Recommender System (LOD-기반 추천 시스템에서 LOD 그래프에 가중치를 사용한 의미 거리 측정 모델)

  • Huh, Wonwhoi
    • Journal of the Korea Convergence Society
    • /
    • v.12 no.7
    • /
    • pp.53-60
    • /
    • 2021
  • LOD-based recommender systems usually leverage the data available within LOD datasets, such as DBpedia, in order to recommend items(movies, books, music) to the end users. These systems use a semantic similarity algorithm that calculates the degree of matching between pairs of Linked Data resources. In this paper, we proposed a new approach to measuring semantic distance in an LOD-based recommender system by assigning weights converted from user ratings to links in the LOD graph. The semantic distance measurement model proposed in this paper is based on a processing step in which a graph is personalized to a user through weight calculation and a method of applying these weights to LDSD. The Experimental results showed that the proposed method showed higher accuracy compared to other similar methods, and it contributed to the improvement of similarity by expanding the range of semantic distance measurement of the recommender system. As future work, we aim to analyze the impact on the model using different methods of LOD-based similarity measurement.