• Title/Summary/Keyword: Write Replication

Search Result 11, Processing Time 0.023 seconds

Dynamic Load Balancing and Network Adaptive Virtual Storage Service for Mobile Appliances

  • Ong, Ivy;Lim, Hyo-Taek
    • Journal of Information Processing Systems
    • /
    • v.7 no.1
    • /
    • pp.53-62
    • /
    • 2011
  • With the steady growth of mobile technology and applications, demand for more storage in mobile devices has also increased. A lightweight block-level protocol, Internet Advanced Technology Attachment (iATA), has been developed to deliver a cost-effective storage network solution for mobile devices to obtain more storage. This paper seeks to contribute to designing and implementing Load Balancing (LB), Network Monitoring (NM) and Write Replication (WR) modules to improve the protocol's scalability and data availability. LB and NM modules are invoked to collect system resources states and current network status at each associate node (server machine). A dynamic weight factor is calculated based on the collected information and sent to a referral server. The referral server is responsible to analyze and allocate the most ideal node with the least weight to serve the client. With this approach, the client can avoid connecting to a heavily loaded node that may cause delays in subsequent in-band I/O operations. Write replication is applied to the remaining nodes through a WR module by utilizing the Unison file synchronization program. A client initially connected to node IP A for write operations will have no hindrances in executing the relevant read operations at node IP B in new connections. In the worst case scenario of a node crashing, data remain recoverable from other functioning nodes. We have conducted several benchmark tests and our results are evaluated and verified in a later section.

Performance Enhancement and Evaluation of Distributed File System for Cloud (클라우드 분산 파일 시스템 성능 개선 및 평가)

  • Lee, Jong Hyuk
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.7 no.11
    • /
    • pp.275-280
    • /
    • 2018
  • The choice of a suitable distributed file system is required for loading large data and high-speed processing through subsequent applications in a cloud environment. In this paper, we propose a write performance improvement method based on GlusterFS and evaluate the performance of MapRFS, CephFS and GlusterFS among existing distributed file systems in cloud environment. The write performance improvement method proposed in this paper enhances the response time by changing the synchronization level used by the synchronous replication method from disk to memory. Experimental results show that the distributed file system to which the proposed method is applied is superior to other distributed file systems in the case of sequential write, random write and random read.

A Data-Consistency Scheme for the Distributed-Cache Storage of the Memcached System

  • Liao, Jianwei;Peng, Xiaoning
    • Journal of Computing Science and Engineering
    • /
    • v.11 no.3
    • /
    • pp.92-99
    • /
    • 2017
  • Memcached, commonly used to speed up the data access in big-data and Internet-web applications, is a system software of the distributed-cache mechanism. But it is subject to the severe challenge of the loss of recently uncommitted updates in the case where the Memcached servers crash due to some reason. Although the replica scheme and the disk-log-based replay mechanism have been proposed to overcome this problem, they generate either the overhead of the replica synchronization or the persistent-storage overhead that is caused by flushing related logs. This paper proposes a scheme of backing up the write requests (i.e., set and add) on the Memcached client side, to reduce the overhead resulting from the making of disk-log records or performing the replica consistency. If the Memcached server fails, a timestamp-based recovery mechanism is then introduced to replay the write requests (buffered by relevant clients), for regaining the lost-data updates on the rebooted Memcached server, thereby meeting the data-consistency requirement. More importantly, compared with the mechanism of logging the write requests to the persistent storage of the master server and the server-replication scheme, the newly proposed approach of backing up the logs on the client side can greatly decrease the time overhead by up to 116.8% when processing the write workloads.

Performance Evaluation of WAN Storage Migration Scheme for Cloud Computing Environment (클라우드 컴퓨팅 환경을 위한 WAN 스토리지 이주 기법 성능평가)

  • Chang, Jun-Hyub;Lee, Won-Joo;Jeon, Chang-Ho
    • Journal of the Korea Society of Computer and Information
    • /
    • v.17 no.5
    • /
    • pp.1-7
    • /
    • 2012
  • In this paper, we design and implement the simulator for WAN storage replication model performance evaluation in cloud computing environment. Each cloud of simulator is composed of virtual machine emulator and storage emulator. The virtual machine emulator is composed of read/write ratio module, the read/write sequence combination module, and the read/write request module. The storage emulator is composed of storage management module, data transfer module, read/write operations module, and overhead processing module. Using the simulator, we evaluate performance of migration scheme, pre-copy and the post-copy, considering about read/write ratio, network delay, and network bandwidth. Through simulation, we have confirmed that the average migration time of pre-copy was decreased proportional to the read operation. However, average migration time of post-copy was on the increase. Also, the average migration time of post-copy was increased proportional to the network delay. However, average migration time of pre-copy was shown uniformly. Therefore, we show that pre-copy model more effective to reduce the average migration time than the post-copy model. The average migration time of pre-copy and post-copy were not affected by the change of network bandwidth. Therefore, these results show that selects the storage replication model to be, the network bandwidth know not being the important element.

Cross Compressed Replication Scheme for Large-Volume Column Storages (대용량 컬럼 저장소를 위한 교차 압축 이중화 기법)

  • Byun, Siwoo
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.14 no.5
    • /
    • pp.2449-2456
    • /
    • 2013
  • The column-oriented database storage is a very advanced model for large-volume data analysis systems because of its superior I/O performance. Traditional data storages exploit row-oriented storage where the attributes of a record are placed contiguously in hard disk for fast write operations. However, for search-mostly datawarehouse systems, column-oriented storage has become a more proper model because of its superior read performance. Recently, solid state drive using MLC flash memory is largely recognized as the preferred storage media for high-speed data analysis systems. In this paper, we introduce fast column-oriented data storage model and then propose a new storage management scheme using a cross compressed replication for the high-speed column-oriented datawarehouse system. Our storage management scheme which is based on two MLC SSD achieves superior performance and reliability by the cross replication of the uncompressed segment and the compressed segment under high workloads of CPU and I/O. Based on the results of the performance evaluation, we conclude that our storage management scheme outperforms the traditional scheme in the respect of update throughput and response time of the column segments.

Data Access Frequency based Data Replication Method using Erasure Codes in Cloud Storage System (클라우드 스토리지 시스템에서 데이터 접근빈도와 Erasure Codes를 이용한 데이터 복제 기법)

  • Kim, Ju-Kyeong;Kim, Deok-Hwan
    • Journal of the Institute of Electronics and Information Engineers
    • /
    • v.51 no.2
    • /
    • pp.85-91
    • /
    • 2014
  • Cloud storage system uses a distributed file system for storing and managing data. Traditional distributed file system makes a triplication of data in order to restore data loss in disk failure. However, enforcing data replication method increases storage utilization and causes extra I/O operations during replication process. In this paper, we propose a data replication method using erasure codes in cloud storage system to improve storage space efficiency and I/O performance. In particular, according to data access frequency, the proposed method can reduce the number of data replications but using erasure codes can keep the same data recovery performance. Experimental results show that proposed method improves performance in storage efficiency 40%, read throughput 11%, write throughput 10% better than HDFS does.

An Efficient Hybrid Replication Protocol for High Available Distributed System (고 가용성 분산 시스템을 위한 효율적인 하이브리드 복제 프로토콜)

  • Youn Hee Yong;Choi Sung Chune
    • The KIPS Transactions:PartA
    • /
    • v.12A no.2 s.92
    • /
    • pp.171-180
    • /
    • 2005
  • In distributed systems data are replicated and stored at several nodes to increase the availability and overall performance. Here Quorum protocol doffing a certain set of replicas required for read/write operation exists for global concurrency control. One of the representative replication Protocols - the Tree Quorum protocol - has a drawback of rapidly growing number of replicas as the level increases, while the Grid protocol requires the same operation cost even without any failure. In this paper, thus, we propose a new replication protocol called hybrid protocol which capitalizes the merits of the existing protocols and solves the problems of them at the same time. The proposed hybrid protocol has very low operation cost in the absence of failure like the tree quorum protocol, and has relatively lower operation cost and higher availability than existing protocols when failure occurs by employing tree architecture as the overall organization while each level of the tree is organized as a row of a grid architecture. It is thus effective to be applied to survival storage system. We conduct cost and availability analysis of the proposed protocol through mathematical modeling, and response time and throughput are compared with those of the Tree Quorum protocol through computer simulation.

Characteristic of Data Distribution and Data Replication based Model of LDAP System in High Performance Grid Environments (고성능 Grid 환경에서의 LDAP 시스템의 분산모델과 복제모델의 특성)

  • 권성호;김희철
    • Journal of Korea Society of Industrial Information Systems
    • /
    • v.9 no.1
    • /
    • pp.77-84
    • /
    • 2004
  • Recently, as the number of entities participating in the Grid system increased, the response time of LDAP system became inadequate. Consequently, we have to design new LDAP that suitable for high performance Grid environments. For this, researches about analysis of performance LDAP system are needed firstly. However, because researches are focused mostly on read operation optimized environments, so these result of researches are not directly applied to high performance Grid environments that write operation occupies most. In this paper, we provide overall results of analysis of performance of LDAP system with respect to number of node, query arrival rate, probability of read and so on. The analysis is based on in analytic performance model by applying the M/M/1 queuing model. Finally, based on the results, we suggest the direction for the design of high performance LDAP system and this research results can be applied as basic materials to design of GIS in high performance Grid environments as well as.

  • PDF

Design and Evaluation of an Efficient Flushing Scheme for key-value Store (키-값 저장소를 위한 효율적인 로그 처리 기법 설계 및 평가)

  • Han, Hyuck
    • The Journal of the Korea Contents Association
    • /
    • v.19 no.5
    • /
    • pp.187-193
    • /
    • 2019
  • Key-value storage engines are an essential component of growing demand in many computing environments, including social networks, online e-commerce, and cloud services. Recent key-value storage engines offer many features such as transaction, versioning, and replication. In a key-value storage engine, transaction processing provides atomicity through Write-Ahead-Logging (WAL), and a synchronous commit method for transaction processing flushes log data before the transaction completes. According to our observation, flushing log data to persistent storage is a performance bottleneck for key-value storage engines due to the significant overhead of fsync() calls despite the various optimizations of existing systems. In this article, we propose a group synchronization method to improve the performance of the key-value storage engine. We also design and implement a transaction scheduling method to perform other transactions while the system processes fsync() calls. The proposed method is an efficient way to reduce the number of frequent fsync() calls in the synchronous commit while supporting the same level of transaction provided by the existing system. We implement our scheme on the WiredTiger storage engine and our experimental results show that the proposed system improves the performance of key-value workloads over existing systems.

A Distributed Real-Time Concurrency Control Scheme using Transaction the Rise of Priority (트랜잭션 우선 순위 상승을 이용한 분산 실시간 병행수행제어 기법)

  • Lee, Jong-Sul;Shin, Jae-Ryong;Cho, Ki-Hyung;Yoo, Jae-Soo
    • Journal of KIISE:Databases
    • /
    • v.28 no.3
    • /
    • pp.484-493
    • /
    • 2001
  • As real-time database systems are extended to the distributed computing environment, the need to apply the existing real-time concurrency control schemes to the distributed computing environment has been made. In this paper we propose an efficient concurrency control scheme for distributed real-time database system. Our proposed scheme guarantees a transaction to commit at its maximum, reduces the restart of a transaction that is on the prepared commit phase, and minimizes the time of the lock holding. This is because it raises the priority of the transaction that is on the prepared commit phase in the distributed real-time computing environment. In addition, it reduces the waiting time of a transaction that owns borrowed data and improves the performance of the system, as a result of lending the data that the transaction with the raised priority holds. We compare the proposed scheme with DO2PL_PA(Distributed Optimistic Two-Phase Locking) and MIRROR(Managing Isolation in Replicated Real-time Object Repositories) protocol in terms of the arrival rate of transactions, the size of transactions, the write probability of transactions, and the replication degree of data in a firm-deadline real-time database system based on two-phase commit protocol. It is shown through the performance evaluation that our scheme outperforms the existing schemes.

  • PDF