• Title/Summary/Keyword: Data sparsity

Search Result 176, Processing Time 0.025 seconds

Improvement on Similarity Calculation in Collaborative Filtering Recommendation using Demographic Information (인구 통계 정보를 이용한 협업 여과 추천의 유사도 개선 기법)

  • 이용준;이세훈;왕창종
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.9 no.5
    • /
    • pp.521-529
    • /
    • 2003
  • In this paper we present an improved method by using demographic information for overcoming the similarity miss-calculation from the sparsity problem in collaborative filtering recommendation systems. The similarity between a pair of users is only determined by the ratings given to co-rated items, so items that have not been rated by both users are ignored. To solve this problem, we add virtual neighbor's rating using demographic information of neighbors for improving prediction accuracy. It is one kind of extentions of traditional collaborative filtering methods using the peason correlation coefficient. We used the Grouplens movie rating data in experiment and we have compared the proposed method with the collaborative filtering methods by the mean absolute error and receive operating characteristic values. The results show that the proposed method is more efficient than the collaborative filtering methods using the pearson correlation coefficient about 9% in MAE and 13% in sensitivity of ROC.

Recommender Systems using SVD with Social Network Information (사회연결망정보를 고려하는 SVD 기반 추천시스템)

  • Kim, Min-Gun;Kim, Kyoung-jae
    • Journal of Intelligence and Information Systems
    • /
    • v.22 no.4
    • /
    • pp.1-18
    • /
    • 2016
  • Collaborative Filtering (CF) predicts the focal user's preference for particular item based on user's preference rating data and recommends items for the similar users by using them. It is a popular technique for the personalization in e-commerce to reduce information overload. However, it has some limitations including sparsity and scalability problems. In this paper, we use a method to integrate social network information into collaborative filtering in order to mitigate the sparsity and scalability problems which are major limitations of typical collaborative filtering and reflect the user's qualitative and emotional information in recommendation process. In this paper, we use a novel recommendation algorithm which is integrated with collaborative filtering by using Social SVD++ algorithm which considers social network information in SVD++, an extension algorithm that can reflect implicit information in singular value decomposition (SVD). In particular, this study will evaluate the performance of the model by reflecting the real-world user's social network information in the recommendation process.

Pulse Sequence based MR Images for Compressed Sensing Algorithm Applications (펄스열을 이용한 MR 영상의 Compressed Sensing 알고리즘 적용)

  • Gho, Sung-Mi;Choi, Na-Rae;Kim, Dong-Hyun
    • Journal of the Institute of Electronics Engineers of Korea SP
    • /
    • v.46 no.5
    • /
    • pp.1-7
    • /
    • 2009
  • In recent years, compressed sensing (CS) algorithm has been studied in various research areas including medical imaging. To use the CS algorithm, the signal that is to be reconstructed needs to have the property of sparsity But, most medical images generally don't have this property. One method to overcome this problem is by using sparsifying transform. However, MR imaging, compared to other medical imaging modality, has the unique property that by using appropriate image acquisition pulse sequences, the image contrast can be modified. In this paper, we propose the possibility of applying the CS algorithm with non-sparsifying transform to the pulse sequence modified MR images and improve the reconstruction performance of the CS algorithm by using an appropriate sparsifying transform. We verified the proposed contents by computer simulation using Shepp-Logan phantom and in vivo data.

A Prediction System of User Preferences for Newly Released Items Based on Words (새로 출시되는 품목들을 위한 단어 기반의 사용자 선호도 예측 기법)

  • Choi, Yoon-Seok;Moon, Byung-Ro
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • v.16 no.2
    • /
    • pp.156-163
    • /
    • 2006
  • CF systems are widely used in recommendation due to the easy implementation and the outstanding performance. They have several problems such as the sparsity problem, the first-rater problem, and recommending explanation. Many studies are suggested to resolve these problems. While the influence of the sparsity problem lessens as the users' data are accumulated, but the first-rater problem is originated from the CF systems and there are a number of researches to overcome the disadvantages of CF systems based on the content-based methods. Also CF systems are black boxes, providing no explanation of working of the recommendation. In this paper we present a content-based prediction system based on the preference words, which exposes the reasoning behind a recommendation. Our system predicts user's rating of a new movie and we suggest a semiotic network-based method to solve the mismatching problem between the items. For experimental comparison, we used EachMovie and IMDb dataset.

Ultra-WideBand Channel Measurement with Compressive Sampling for Indoor Localization (실내 위치추정을 위한 Compressive Sampling적용 Ultra-WideBand 채널 측정기법)

  • Kim, Sujin;Myung, Jungho;Kang, Joonhyuk;Sung, Tae-Kyung;Lee, Kwang-Eog
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.40 no.2
    • /
    • pp.285-297
    • /
    • 2015
  • In this paper, Ulta-WideBand (UWB) channel measurement and modeling based on compressive sampling (CS) are proposed. The sparsity of the channel impulse response (CIR) of the UWB signal in frequency domain enables the proposed channel measurement to have a low-complexity and to provide a comparable performance compared with the existing approaches especially used for the indoor geo-localization purpose. Furthermore, to improve the performance under noisy situation, the soft thresholding method is also investigated in solving the optimization problem for signal recovery of CS. Via numerical results, the proposed channel measurement and modeling are evaluated with the real measured data in terms of location estimation error, bandwidth, and compression ratio for indoor geo-localization using UWB system.

Sparse reconstruction of guided wavefield from limited measurements using compressed sensing

  • Qiao, Baijie;Mao, Zhu;Sun, Hao;Chen, Songmao;Chen, Xuefeng
    • Smart Structures and Systems
    • /
    • v.25 no.3
    • /
    • pp.369-384
    • /
    • 2020
  • A wavefield sparse reconstruction technique based on compressed sensing is developed in this work to dramatically reduce the number of measurements. Firstly, a severely underdetermined representation of guided wavefield at a snapshot is established in the spatial domain. Secondly, an optimal compressed sensing model of guided wavefield sparse reconstruction is established based on l1-norm penalty, where a suite of discrete cosine functions is selected as the dictionary to promote the sparsity. The regular, random and jittered undersampling schemes are compared and selected as the undersampling matrix of compressed sensing. Thirdly, a gradient projection method is employed to solve the compressed sensing model of wavefield sparse reconstruction from highly incomplete measurements. Finally, experiments with different excitation frequencies are conducted on an aluminum plate to verify the effectiveness of the proposed sparse reconstruction method, where a scanning laser Doppler vibrometer as the true benchmark is used to measure the original wavefield in a given inspection region. Experiments demonstrate that the missing wavefield data can be accurately reconstructed from less than 12% of the original measurements; The reconstruction accuracy of the jittered undersampling scheme is slightly higher than that of the random undersampling scheme in high probability, but the regular undersampling scheme fails to reconstruct the wavefield image; A quantified mapping relationship between the sparsity ratio and the recovery error over a special interval is established with respect to statistical modeling and analysis.

Sparse and low-rank feature selection for multi-label learning

  • Lim, Hyunki
    • Journal of the Korea Society of Computer and Information
    • /
    • v.26 no.7
    • /
    • pp.1-7
    • /
    • 2021
  • In this paper, we propose a feature selection technique for multi-label classification. Many existing feature selection techniques have selected features by calculating the relation between features and labels such as a mutual information scale. However, since the mutual information measure requires a joint probability, it is difficult to calculate the joint probability from an actual premise feature set. Therefore, it has the disadvantage that only a few features can be calculated and only local optimization is possible. Away from this regional optimization problem, we propose a feature selection technique that constructs a low-rank space in the entire given feature space and selects features with sparsity. To this end, we designed a regression-based objective function using Nuclear norm, and proposed an algorithm of gradient descent method to solve the optimization problem of this objective function. Based on the results of multi-label classification experiments on four data and three multi-label classification performance, the proposed methodology showed better performance than the existing feature selection technique. In addition, it was showed by experimental results that the performance change is insensitive even to the parameter value change of the proposed objective function.

Hot Topic Discovery across Social Networks Based on Improved LDA Model

  • Liu, Chang;Hu, RuiLin
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.15 no.11
    • /
    • pp.3935-3949
    • /
    • 2021
  • With the rapid development of Internet and big data technology, various online social network platforms have been established, producing massive information every day. Hot topic discovery aims to dig out meaningful content that users commonly concern about from the massive information on the Internet. Most of the existing hot topic discovery methods focus on a single network data source, and can hardly grasp hot spots as a whole, nor meet the challenges of text sparsity and topic hotness evaluation in cross-network scenarios. This paper proposes a novel hot topic discovery method across social network based on an im-proved LDA model, which first integrates the text information from multiple social network platforms into a unified data set, then obtains the potential topic distribution in the text through the improved LDA model. Finally, it adopts a heat evaluation method based on the word frequency of topic label words to take the latent topic with the highest heat value as a hot topic. This paper obtains data from the online social networks and constructs a cross-network topic discovery data set. The experimental results demonstrate the superiority of the proposed method compared to baseline methods.

Recommender Systems using Structural Hole and Collaborative Filtering (구조적 공백과 협업필터링을 이용한 추천시스템)

  • Kim, Mingun;Kim, Kyoung-Jae
    • Journal of Intelligence and Information Systems
    • /
    • v.20 no.4
    • /
    • pp.107-120
    • /
    • 2014
  • This study proposes a novel recommender system using the structural hole analysis to reflect qualitative and emotional information in recommendation process. Although collaborative filtering (CF) is known as the most popular recommendation algorithm, it has some limitations including scalability and sparsity problems. The scalability problem arises when the volume of users and items become quite large. It means that CF cannot scale up due to large computation time for finding neighbors from the user-item matrix as the number of users and items increases in real-world e-commerce sites. Sparsity is a common problem of most recommender systems due to the fact that users generally evaluate only a small portion of the whole items. In addition, the cold-start problem is the special case of the sparsity problem when users or items newly added to the system with no ratings at all. When the user's preference evaluation data is sparse, two users or items are unlikely to have common ratings, and finally, CF will predict ratings using a very limited number of similar users. Moreover, it may produces biased recommendations because similarity weights may be estimated using only a small portion of rating data. In this study, we suggest a novel limitation of the conventional CF. The limitation is that CF does not consider qualitative and emotional information about users in the recommendation process because it only utilizes user's preference scores of the user-item matrix. To address this novel limitation, this study proposes cluster-indexing CF model with the structural hole analysis for recommendations. In general, the structural hole means a location which connects two separate actors without any redundant connections in the network. The actor who occupies the structural hole can easily access to non-redundant, various and fresh information. Therefore, the actor who occupies the structural hole may be a important person in the focal network and he or she may be the representative person in the focal subgroup in the network. Thus, his or her characteristics may represent the general characteristics of the users in the focal subgroup. In this sense, we can distinguish friends and strangers of the focal user utilizing the structural hole analysis. This study uses the structural hole analysis to select structural holes in subgroups as an initial seeds for a cluster analysis. First, we gather data about users' preference ratings for items and their social network information. For gathering research data, we develop a data collection system. Then, we perform structural hole analysis and find structural holes of social network. Next, we use these structural holes as cluster centroids for the clustering algorithm. Finally, this study makes recommendations using CF within user's cluster, and compare the recommendation performances of comparative models. For implementing experiments of the proposed model, we composite the experimental results from two experiments. The first experiment is the structural hole analysis. For the first one, this study employs a software package for the analysis of social network data - UCINET version 6. The second one is for performing modified clustering, and CF using the result of the cluster analysis. We develop an experimental system using VBA (Visual Basic for Application) of Microsoft Excel 2007 for the second one. This study designs to analyzing clustering based on a novel similarity measure - Pearson correlation between user preference rating vectors for the modified clustering experiment. In addition, this study uses 'all-but-one' approach for the CF experiment. In order to validate the effectiveness of our proposed model, we apply three comparative types of CF models to the same dataset. The experimental results show that the proposed model outperforms the other comparative models. In especial, the proposed model significantly performs better than two comparative modes with the cluster analysis from the statistical significance test. However, the difference between the proposed model and the naive model does not have statistical significance.

Weighted Window Assisted User History Based Recommendation System (가중 윈도우를 통한 사용자 이력 기반 추천 시스템)

  • Hwang, Sungmin;Sokasane, Rajashree;Tri, Hiep Tuan Nguyen;Kim, Kyungbaek
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.4 no.6
    • /
    • pp.253-260
    • /
    • 2015
  • When we buy items in online stores, it is common to face recommended items that meet our interest. These recommendation system help users not only to find out related items, but also find new things that may interest users. Recommendation system has been widely studied and various models has been suggested such as, collaborative filtering and content-based filtering. Though collaborative filtering shows good performance for predicting users preference, there are some conditions where collaborative filtering cannot be applied. Sparsity in user data causes problems in comparing users. Systems which are newly starting or companies having small number of users are also hard to apply collaborative filtering. Content-based filtering should be used to support this conditions, but content-based filtering has some drawbacks and weakness which are tendency of recommending similar items, and keeping history of a user makes recommendation simple and not able to follow up users preference changes. To overcome this drawbacks and limitations, we suggest weighted window assisted user history based recommendation system, which captures user's purchase patterns and applies them to window weight adjustment. The system is capable of following current preference of a user, removing useless recommendation and suggesting items which cannot be simply found by users. To examine the performance under user and data sparsity environment, we applied data from start-up trading company. Through the experiments, we evaluate the operation of the proposed recommendation system.