• Title/Summary/Keyword: Data Scalability Problem

Search Result 116, Processing Time 0.027 seconds

NoSQL-based User Behavior Detection System in Cloud Computing Environment (NoSQL 기반 클라우드 사용자 행동 탐지 시스템 설계)

  • Ahn, Kwang-Min;Lee, Bong-Hwan
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2012.10a
    • /
    • pp.804-807
    • /
    • 2012
  • Cloud service provider has to protect client's information securely since all the resources are offered by the service provider, and a large number of users share the resources. In this paper, a NoSQL-based anomaly detection system is proposed in order to enhance the security of mobile cloud services. The existing integrated security management system that uses a relational database can not be used for real-time processing of data since security log from a variety of security equipment and data from cloud node have different data format with unstructured features. The proposed system can resolve the emerging security problem because it provides real time processing and scalability in distributed processing environment.

  • PDF

Finding Frequent Route of Taxi Trip Events Based on MapReduce and MongoDB (택시 데이터에 대한 효율적인 Top-K 빈도 검색)

  • Putri, Fadhilah Kurnia;An, Seonga;Purnaningtyas, Magdalena Trie;Jeong, Han-You;Kwon, Joonho
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.4 no.9
    • /
    • pp.347-356
    • /
    • 2015
  • Due to the rapid development of IoT(Internet of Things) technology, traditional taxis are connected through dispatchers and location systems. Typically, modern taxis have embedded with GPS(Global Positioning System), which aims for obtaining the route information. By analyzing the frequency of taxi trip events, we can find the frequent route for a given query time. However, a scalability problem would occur when we convert the raw location data of taxi trip events into the analyzed frequency information due to the volume of location data. For this problem, we propose a NoSQL based top-K query system for taxi trip events. First, we analyze raw taxi trip events and extract frequencies of all routes. Then, we store the frequency information into hash-based index structure of MongoDB which is a document-oriented NoSQL database. Efficient top-K query processing for frequent route is done with the top of the MongoDB. We validate the efficiency of our algorithms by using real taxi trip events of New York City.

An Index-Based Search Method for Performance Improvement of Set-Based Similar Sequence Matching (집합 유사 시퀀스 매칭의 성능 향상을 위한 인덱스 기반 검색 방법)

  • Lee, Juwon;Lim, Hyo-Sang
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.6 no.11
    • /
    • pp.507-520
    • /
    • 2017
  • The set-based similar sequence matching method measures similarity not for an individual data item but for a set grouping multiple data items. In the method, the similarity of two sets is represented as the size of intersection between them. However, there is a critical performances issue for the method in twofold: 1) calculating intersection size is a time consuming process, and 2) the number of set pairs that should be calculated the intersection size is quite large. In this paper, we propose an index-based search method for improving performance of set-based similar sequence matching in order to solve these performance issues. Our method consists of two parts. In the first part, we convert the set similarity problem into the intersection size comparison problem, and then, provide an index structure that accelerates the intersection size calculation. Second, we propose an efficient set-based similar sequence matching method which exploits the proposed index structure. Through experiments, we show that the proposed method reduces the execution time by 30 to 50 times then the existing methods. We also show that the proposed method has scalability since the performance gap becomes larger as the number of data sequences increases.

Density Scalability of Video Based Point Cloud Compression by Using SHVC Codec (SHVC 비디오 기반 포인트 클라우드 밀도 스케일러빌리티 방안)

  • Hwang, Yonghae;Kim, Junsik;Kim, Kyuheon
    • Journal of Broadcast Engineering
    • /
    • v.25 no.5
    • /
    • pp.709-722
    • /
    • 2020
  • Point Cloud which is a cluster of numerous points can express 3D object beyond the 2D plane. Each point contains 3D coordinate and color data basically, reflectance or etc. additionally. Point Cloud demand research and development much higher effective compression technology. Video-based Point Cloud Compression (V-PCC) technology in development and standardization based on the established video codec. Despite its high effective compression technology, point cloud service will be limited by terminal spec and network conditions. 2D video had the same problems. To remedy this kind of problem, 2D video is using Scalable High efficiency Video Coding (SHVC), Dynamic Adaptive Streaming over HTTP (DASH) or diverse technology. This paper proposed a density scalability method using SHVC codec in V-PCC.

Separated Address/Data Network Design for Bus Protocol compatible Network-on-Chip (버스 프로토콜 호환 가능한 네트워크-온-칩에서의 분리된 주소/데이터 네트워크 설계)

  • Chung, Seungh Ah;Lee, Jae Hoon;Kim, Sang Heon;Lee, Jae Sung;Han, Tae Hee
    • Journal of the Institute of Electronics and Information Engineers
    • /
    • v.53 no.4
    • /
    • pp.68-75
    • /
    • 2016
  • As the number of cores and IPs increase in multiprocessor system-on-chip (MPSoC), network-on-chip (NoC) has emerged as a promising novel interconnection architecture for its parallelism and scalability. However, minimization of the latency in NoC with legacy bus IPs must be addressed. In this paper, we focus on the latency minimization problem in NoC which accommodates legacy bus protocol based IPs considering the trade-offs between hop counts and path collisions. To resolve this problem, we propose separated address/data network for independent address and data phases of bus protocol. Compared to Mesh and irregular topologies generated by TopGen, experimental results show that average latency and execution time are reduced by 19.46% and 10.55%, respectively.

A proposal for the unified architecture of switches and severs for the efficient processing of multimedia communications and services (멀티미디어 통신과 서비스의 효율적 처리를 위한 교환기-서버 통합구조의 제안)

  • 함진호;최병욱
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.21 no.11
    • /
    • pp.2869-2885
    • /
    • 1996
  • There werer many researches on the switch system for high speed netowrk such as B-ISDN and the server system for the multimedia services respectively. But, in case of simple connection of these two systems, we have to suffer the bottlenck problem of data transmission, and pay the wasteful expenditure because of duplicated architecture of both systems such as interconnecting network at the switch and the server. Therefore, we propse the unified architecture of switehces and servers, which can be used as switches and servers simultaneously. This is based on the hybercube structure. The links are used iterconnection network of switch system, and each node has the subscriber subsystem and the server subsystem. The proposed architecure has the benifits as follows; the easy espansion of capacity due to the scalability,the simple system development and maintenance because of the equivalance of each nodes, the high reliability against the fault of nodes and links due to the existence of the many alternative links between nodes, the easy flow and QoS managment due to the non-blocking data transmission between any two nodes, the flexible adaptation for additional new services owing to simple insertion server board to node. In this paper, we present overal configuration and node component of proposed architecture, and the procesing flow for the various services.

  • PDF

A Study of Recommendation System Using Association Rule and Weighted Preference (연관규칙과 가중 선호도를 이용한 추천시스템 연구)

  • Moon, Song Chul;Cho, Young-Sung
    • Journal of Information Technology Services
    • /
    • v.13 no.3
    • /
    • pp.309-321
    • /
    • 2014
  • Recently, due to the advent of ubiquitous computing and the spread of intelligent portable device such as smart phone, iPad and PDA has been amplified, a variety of services and the amount of information has also increased fastly. It is becoming a part of our common life style that the demands for enjoying the wireless internet are increasing anytime or anyplace without any restriction of time and place. And also, the demands for e-commerce and many different items on e-commerce and interesting of associated items are increasing. Existing collaborative filtering (CF), explicit method, can not only reflect exact attributes of item, but also still has the problem of sparsity and scalability, though it has been practically used to improve these defects. In this paper, using a implicit method without onerous question and answer to the users, not used user's profile for rating to reduce customers' searching effort to find out the items with high purchasability, it is necessary for us to analyse the segmentation of customer and item based on customer data and purchase history data, which is able to reflect the attributes of the item in order to improve the accuracy of recommendation. We propose the method of recommendation system using association rule and weighted preference so as to consider many different items on e-commerce and to refect the profit/weight/importance of attributed of a item. To verify improved performance of proposing system, we make experiments with dataset collected in a cosmetic internet shopping mall.

An Energy Efficient Multi-hop Cluster-Head Election Strategy for Wireless Sensor Networks

  • Zhao, Liquan;Guo, Shuaichao
    • Journal of Information Processing Systems
    • /
    • v.17 no.1
    • /
    • pp.63-74
    • /
    • 2021
  • According to the double-phase cluster-head election method (DCE), the final cluster heads (CHs) sometimes are located at the edge of cluster. They have a long distance from the base station (BS). Sensor data is directly transmitted to BS by CHs. This makes some nodes consume much energy for transmitting data and die earlier. To address this problem, energy efficient multi-hop cluster-head election strategy (EEMCE) is proposed in this paper. To avoid taking these nodes far from BS as CH, this strategy first introduces the distance from the sensor nodes to the BS into the tentative CH election. Subsequently, in the same cluster, the energy of tentative CH is compared with those of other nodes, and then the node that has more energy than the tentative CH and being nearest the tentative CH are taken as the final CH. Lastly, if the CH is located at the periphery of the network, the multi-hop method will be employed to reduce the energy that is consumed by CHs. The simulation results suggest that the proposed method exhibits higher energy efficiency, longer stability period and better scalability than other protocols.

Enhancement of Semantic Interoper ability in Healthcare Systems Using IFCIoT Architecture

  • Sony P;Siva Shanmugam G;Sureshkumar Nagarajan
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.18 no.4
    • /
    • pp.881-902
    • /
    • 2024
  • Fast decision support systems and accurate diagnosis have become significant in the rapidly growing healthcare sector. As the number of disparate medical IoT devices connected to the human body rises, fast and interrelated healthcare data retrieval gets harder and harder. One of the most important requirements for the Healthcare Internet of Things (HIoT) is semantic interoperability. The state-of-the-art HIoT systems have problems with bandwidth and latency. An extension of cloud computing called fog computing not only solves the latency problem but also provides other benefits including resource mobility and on-demand scalability. The recommended approach helps to lower latency and network bandwidth consumption in a system that provides semantic interoperability in healthcare organizations. To evaluate the system's language processing performance, we simulated it in three different contexts. 1. Polysemy resolution system 2. System for hyponymy-hypernymy resolution with polysemy 3. System for resolving polysemy, hypernymy, hyponymy, meronymy, and holonymy. In comparison to the other two systems, the third system has lower latency and network usage. The proposed framework can reduce the computation overhead of heterogeneous healthcare data. The simulation results show that fog computing can reduce delay, network usage, and energy consumption.

Recommender Systems using Structural Hole and Collaborative Filtering (구조적 공백과 협업필터링을 이용한 추천시스템)

  • Kim, Mingun;Kim, Kyoung-Jae
    • Journal of Intelligence and Information Systems
    • /
    • v.20 no.4
    • /
    • pp.107-120
    • /
    • 2014
  • This study proposes a novel recommender system using the structural hole analysis to reflect qualitative and emotional information in recommendation process. Although collaborative filtering (CF) is known as the most popular recommendation algorithm, it has some limitations including scalability and sparsity problems. The scalability problem arises when the volume of users and items become quite large. It means that CF cannot scale up due to large computation time for finding neighbors from the user-item matrix as the number of users and items increases in real-world e-commerce sites. Sparsity is a common problem of most recommender systems due to the fact that users generally evaluate only a small portion of the whole items. In addition, the cold-start problem is the special case of the sparsity problem when users or items newly added to the system with no ratings at all. When the user's preference evaluation data is sparse, two users or items are unlikely to have common ratings, and finally, CF will predict ratings using a very limited number of similar users. Moreover, it may produces biased recommendations because similarity weights may be estimated using only a small portion of rating data. In this study, we suggest a novel limitation of the conventional CF. The limitation is that CF does not consider qualitative and emotional information about users in the recommendation process because it only utilizes user's preference scores of the user-item matrix. To address this novel limitation, this study proposes cluster-indexing CF model with the structural hole analysis for recommendations. In general, the structural hole means a location which connects two separate actors without any redundant connections in the network. The actor who occupies the structural hole can easily access to non-redundant, various and fresh information. Therefore, the actor who occupies the structural hole may be a important person in the focal network and he or she may be the representative person in the focal subgroup in the network. Thus, his or her characteristics may represent the general characteristics of the users in the focal subgroup. In this sense, we can distinguish friends and strangers of the focal user utilizing the structural hole analysis. This study uses the structural hole analysis to select structural holes in subgroups as an initial seeds for a cluster analysis. First, we gather data about users' preference ratings for items and their social network information. For gathering research data, we develop a data collection system. Then, we perform structural hole analysis and find structural holes of social network. Next, we use these structural holes as cluster centroids for the clustering algorithm. Finally, this study makes recommendations using CF within user's cluster, and compare the recommendation performances of comparative models. For implementing experiments of the proposed model, we composite the experimental results from two experiments. The first experiment is the structural hole analysis. For the first one, this study employs a software package for the analysis of social network data - UCINET version 6. The second one is for performing modified clustering, and CF using the result of the cluster analysis. We develop an experimental system using VBA (Visual Basic for Application) of Microsoft Excel 2007 for the second one. This study designs to analyzing clustering based on a novel similarity measure - Pearson correlation between user preference rating vectors for the modified clustering experiment. In addition, this study uses 'all-but-one' approach for the CF experiment. In order to validate the effectiveness of our proposed model, we apply three comparative types of CF models to the same dataset. The experimental results show that the proposed model outperforms the other comparative models. In especial, the proposed model significantly performs better than two comparative modes with the cluster analysis from the statistical significance test. However, the difference between the proposed model and the naive model does not have statistical significance.