• Title/Summary/Keyword: Data sparsity

Search Result 174, Processing Time 0.022 seconds

Improving the MAE by Removing Lower Rated Items in Recommender System

  • Kim, Sun-Ok;Lee, Seok-Jun;Park, Young-Seo
    • Journal of the Korean Data and Information Science Society
    • /
    • v.19 no.3
    • /
    • pp.819-830
    • /
    • 2008
  • Web recommender system was suggested in order to solve the problem which is cause by overflow of information. Collaborative filtering is the technique which predicts and recommends the suitable goods to the user with collection of preference information based on the history which user was interested in. However, there is a difficulty of recommendation by lack of information of goods which have less popularity. In this paper, it has been researched the way to select the sparsity of goods and the preference in order to solve the problem of recommender system's sparsity which is occurred by lack of information, as well as it has been described the solution which develops the quality of recommender system by selection of customers who were interested in.

  • PDF

Auxiliary Stacked Denoising Autoencoder based Collaborative Filtering Recommendation

  • Mu, Ruihui;Zeng, Xiaoqin
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.14 no.6
    • /
    • pp.2310-2332
    • /
    • 2020
  • In recent years, deep learning techniques have achieved tremendous successes in natural language processing, speech recognition and image processing. Collaborative filtering(CF) recommendation is one of widely used methods and has significant effects in implementing the new recommendation function, but it also has limitations in dealing with the problem of poor scalability, cold start and data sparsity, etc. Combining the traditional recommendation algorithm with the deep learning model has brought great opportunity for the construction of a new recommender system. In this paper, we propose a novel collaborative recommendation model based on auxiliary stacked denoising autoencoder(ASDAE), the model learns effective the preferences of users from auxiliary information. Firstly, we integrate auxiliary information with rating information. Then, we design a stacked denoising autoencoder based collaborative recommendation model to learn the preferences of users from auxiliary information and rating information. Finally, we conduct comprehensive experiments on three real datasets to compare our proposed model with state-of-the-art methods. Experimental results demonstrate that our proposed model is superior to other recommendation methods.

Collaborative Filtering Algorithm Based on User-Item Attribute Preference

  • Ji, JiaQi;Chung, Yeongjee
    • Journal of information and communication convergence engineering
    • /
    • v.17 no.2
    • /
    • pp.135-141
    • /
    • 2019
  • Collaborative filtering algorithms often encounter data sparsity issues. To overcome this issue, auxiliary information of relevant items is analyzed and an item attribute matrix is derived. In this study, we combine the user-item attribute preference with the traditional similarity calculation method to develop an improved similarity calculation approach and use weights to control the importance of these two elements. A collaborative filtering algorithm based on user-item attribute preference is proposed. The experimental results show that the performance of the recommender system is the most optimal when the weight of traditional similarity is equal to that of user-item attribute preference similarity. Although the rating-matrix is sparse, better recommendation results can be obtained by adding a suitable proportion of user-item attribute preference similarity. Moreover, the mean absolute error of the proposed approach is less than that of two traditional collaborative filtering algorithms.

Intelligent recommendation method of intelligent tourism scenic spot route based on collaborative filtering

  • Liu Hui
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.18 no.5
    • /
    • pp.1260-1272
    • /
    • 2024
  • This paper tackles the prevalent challenges faced by existing tourism route recommendation methods, including data sparsity, cold start, and low accuracy. To address these issues, a novel intelligent tourism route recommendation method based on collaborative filtering is introduced. The proposed method incorporates a series of key steps. Firstly, it calculates the interest level of users by analyzing the item attribute rating values. By leveraging this information, the method can effectively capture the preferences and interests of users. Additionally, a user attribute rating matrix is constructed by extracting implicit user behavior preferences, providing a comprehensive understanding of user preferences. Recognizing that user interests can evolve over time, a weight function is introduced to account for the possibility of interest shifting during product use. This weight function enhances the accuracy of recommendations by adapting to the changing preferences of users, improving the overall quality of the suggested tourism routes. The results demonstrate the significant advantages of the approach. Specifically, the proposed method successfully alleviates the problem of data sparsity, enhances neighbor selection, and generates tourism route recommendations that exhibit higher accuracy compared to existing methods.

Analysis of Data Imputation in Recommender Systems (추천 시스템에서의 데이터 임퓨테이션 분석)

  • Lee, Youngnam;Kim, Sang-Wook
    • Journal of KIISE
    • /
    • v.44 no.12
    • /
    • pp.1333-1337
    • /
    • 2017
  • Recommender systems (RS) that predict a set of items a target user is likely to prefer have been extensively studied in academia and have been aggressively implemented by many companies such as Google, Netflix, eBay, and Amazon. Data imputation alleviates the data sparsity problem occurring in recommender systems by inferring missing ratings and adding them to the original data. In this paper, we point out the drawbacks of existing approaches and make suggestions for data imputation techniques. We also justify our suggestions through extensive experiments.

A Music Recommendation System based on Context-awareness using Association Rules (연관규칙을 이용한 상황인식 음악 추천 시스템)

  • Oh, Jae-Taek;Lee, Sang-Yong
    • Journal of Digital Convergence
    • /
    • v.17 no.9
    • /
    • pp.375-381
    • /
    • 2019
  • Recently, the recommendation system has attracted the attention of users as customized recommendation services have been provided focusing on fashion, video and music. But these services are difficult to provide users with proper service according to many different contexts because they do not use contextual information emerging in real time. When applied contextual information expands dimensions, it also increases data sparsity and makes it impossible to recommend proper music for users. Trying to solve these problems, our study proposed a music recommendation system to recommend proper music in real time by applying association rules and using relationships and rules about the current location and time information of users. The accuracy of the recommendation system was measured according to location and time information through 5-fold cross validation. As a result, it was found that the accuracy of the recommendation system was improved as contextual information accumulated.

Similarity Measure based on Utilization of Rating Distributions for Data Sparsity Problem in Collaborative Filtering

  • Lee, Soojung
    • Journal of the Korea Society of Computer and Information
    • /
    • v.25 no.12
    • /
    • pp.203-210
    • /
    • 2020
  • Memory-based collaborative filtering is one of the representative types of the recommender system, but it suffers from the inherent problem of data sparsity. Although many works have been devoted to solving this problem, there is still a request for more systematic approaches to the problem. This study exploits distribution of user ratings given to items for computing similarity. All user ratings are utilized in the proposed method, compared to previous ones which use ratings for only common items between users. Moreover, for similarity computation, it takes a global view of ratings for items by reflecting other users' ratings for that item. Performance is evaluated through experiments and compared to that of other relevant methods. The results reveal that the proposed demonstrates superior performance in prediction and rank accuracies. This improvement in prediction accuracy is as high as 2.6 times more than that achieved by the state-of-the-art method over the traditional similarity measures.

Conditional Generative Adversarial Network based Collaborative Filtering Recommendation System (Conditional Generative Adversarial Network(CGAN) 기반 협업 필터링 추천 시스템)

  • Kang, Soyi;Shin, Kyung-shik
    • Journal of Intelligence and Information Systems
    • /
    • v.27 no.3
    • /
    • pp.157-173
    • /
    • 2021
  • With the development of information technology, the amount of available information increases daily. However, having access to so much information makes it difficult for users to easily find the information they seek. Users want a visualized system that reduces information retrieval and learning time, saving them from personally reading and judging all available information. As a result, recommendation systems are an increasingly important technologies that are essential to the business. Collaborative filtering is used in various fields with excellent performance because recommendations are made based on similar user interests and preferences. However, limitations do exist. Sparsity occurs when user-item preference information is insufficient, and is the main limitation of collaborative filtering. The evaluation value of the user item matrix may be distorted by the data depending on the popularity of the product, or there may be new users who have not yet evaluated the value. The lack of historical data to identify consumer preferences is referred to as data sparsity, and various methods have been studied to address these problems. However, most attempts to solve the sparsity problem are not optimal because they can only be applied when additional data such as users' personal information, social networks, or characteristics of items are included. Another problem is that real-world score data are mostly biased to high scores, resulting in severe imbalances. One cause of this imbalance distribution is the purchasing bias, in which only users with high product ratings purchase products, so those with low ratings are less likely to purchase products and thus do not leave negative product reviews. Due to these characteristics, unlike most users' actual preferences, reviews by users who purchase products are more likely to be positive. Therefore, the actual rating data is over-learned in many classes with high incidence due to its biased characteristics, distorting the market. Applying collaborative filtering to these imbalanced data leads to poor recommendation performance due to excessive learning of biased classes. Traditional oversampling techniques to address this problem are likely to cause overfitting because they repeat the same data, which acts as noise in learning, reducing recommendation performance. In addition, pre-processing methods for most existing data imbalance problems are designed and used for binary classes. Binary class imbalance techniques are difficult to apply to multi-class problems because they cannot model multi-class problems, such as objects at cross-class boundaries or objects overlapping multiple classes. To solve this problem, research has been conducted to convert and apply multi-class problems to binary class problems. However, simplification of multi-class problems can cause potential classification errors when combined with the results of classifiers learned from other sub-problems, resulting in loss of important information about relationships beyond the selected items. Therefore, it is necessary to develop more effective methods to address multi-class imbalance problems. We propose a collaborative filtering model using CGAN to generate realistic virtual data to populate the empty user-item matrix. Conditional vector y identify distributions for minority classes and generate data reflecting their characteristics. Collaborative filtering then maximizes the performance of the recommendation system via hyperparameter tuning. This process should improve the accuracy of the model by addressing the sparsity problem of collaborative filtering implementations while mitigating data imbalances arising from real data. Our model has superior recommendation performance over existing oversampling techniques and existing real-world data with data sparsity. SMOTE, Borderline SMOTE, SVM-SMOTE, ADASYN, and GAN were used as comparative models and we demonstrate the highest prediction accuracy on the RMSE and MAE evaluation scales. Through this study, oversampling based on deep learning will be able to further refine the performance of recommendation systems using actual data and be used to build business recommendation systems.

A Movie Recommendation Method Using Rating Difference Between Items (항목 간 선호도 차이를 이용한 영화 추천 방법)

  • Oh, Se-Chang;Choi, Min
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.17 no.11
    • /
    • pp.2602-2608
    • /
    • 2013
  • User-based and item-based method have been developed as the solutions of the movie recommendation problem. However, these methods are faced with the sparsity problem and the problem of not reflecting user's rating respectively. In order to solve these problems, there is a research on the combination of the two methods using the concept of similarity. In reality, it is not free from the problem of sparsity, since it has a lot of parameters to be calculated. In this study, we propose a recommendation method using rating difference between items in order to complement this problem. This method is relatively free from the problem of sparsity, since it has less parameters to be calculated. And it can get more accurate results by reflecting the users rating to calculate the parameters. In experiments for the proposed method, the initial error is large, but the performance has been quickly stabilized after. In addition, it showed a 0.0538 lower average error compared to the existing method using similarity.

An Agent-based Approach for Distributed Collaborative Filtering (분산 협력 필터링에 대한 에이전트 기반 접근 방법)

  • Kim, Byeong-Man;Li, Qing;Howe Adele E.;Yeo, Dong-Gyu
    • Journal of KIISE:Software and Applications
    • /
    • v.33 no.11
    • /
    • pp.953-964
    • /
    • 2006
  • Due to the usefulness of the collaborative filtering, it has been widely used in both the research and commercial field. However, there are still some challenges for it to be more efficient, especially the scalability problem, the sparsity problem and the cold start problem. In this paper. we address these problems and provide a novel distributed approach based on agents collaboration for the problems. We have tried to solve the scalability problem by making each agent save its users ratings and broadcast them to the users friends so that only friends ratings and his own ratings are kept in an agents local database. To reduce quality degradation of recommendation caused by the lack of rating data, we introduce a method using friends opinions instead of real rating data when they are not available. We also suggest a collaborative filtering algorithm based on user profile to provide new users with recommendation service. Experiments show that our suggested approach is helpful to the new user problem as well as is more scalable than traditional centralized CF filtering systems and alleviate the sparsity problem.