• Title/Summary/Keyword: Data replication management

Search Result 60, Processing Time 0.032 seconds

Dynamic Replication Based on Availability and Popularity in the Presence of Failures

  • Meroufel, Bakhta;Belalem, Ghalem
    • Journal of Information Processing Systems
    • /
    • v.8 no.2
    • /
    • pp.263-278
    • /
    • 2012
  • The data grid provides geographically distributed resources for large-scale applications. It generates a large set of data. The replication of this data in several sites of the grid is an effective solution for achieving good performance. In this paper we propose an approach of dynamic replication in a hierarchical grid that takes into account crash failures in the system. The replication decision is taken based on two parameters: the availability and popularity of the data. The administrator requires a minimum rate of availability for each piece of data according to its access history in previous periods, but this availability may increase if the demand is high on this data. We also proposed a strategy to keep the desired availability respected even in case of a failure or rarity (no-popularity) of the data. The simulation results show the effectiveness of our replication strategy in terms of response time, the unavailability of requests, and availability.

Grid Information Service-Based Large Data Replication Management in OGSA (OGSA 에서의 그리드 정보 서비스를 기반으로한 대용량 Data Replication 관리)

  • Kim, Mi-Ok;Ramakrishna, R.S.
    • Annual Conference of KIPS
    • /
    • 2003.11a
    • /
    • pp.193-196
    • /
    • 2003
  • 그리드 환경에서 OGSA(Open Grid Service Architecture)는 분산된 서비스의 이용 편의를 위한 시스템 독립적인 인터페이스를 제공한다. 하지만 OGSA 에서 사용자가 작업 수행시 필요로 하는 QoS 와 서비스의 신뢰성을 보장하기 위해, 동일한 그리드 정보 제공자를 이용하는 여러 서비스간의 공유 자원에 대한 경쟁 문제를 해결해야 한다. 본 논문에서는 OCSA 에서 여러 서비스의 효율적인 자원 할당을 보장하는 다이나믹 Data Replication 관리를 위한 의사결정 알고리즘을 제안한다.

  • PDF

E2GSM: Energy Effective Gear-Shifting Mechanism in Cloud Storage System

  • You, Xindong;Han, GuangJie;Zhu, Chuan;Dong, Chi;Shen, Jian
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.10 no.10
    • /
    • pp.4681-4702
    • /
    • 2016
  • Recently, Massive energy consumption in Cloud Storage System has attracted great attention both in industry and research community. However, most of the solutions utilize single method to reduce the energy consumption only in one aspect. This paper proposed an energy effective gear-shifting mechanism (E2GSM) in Cloud Storage System to save energy consumption from multi-aspects. E2GSM is established on data classification mechanism and data replication management strategy. Data is classified according to its properties and then be placed into the corresponding zones through the data classification mechanism. Data replication management strategies determine the minimum replica number through a mathematical model and make decision on replica placement. Based on the above data classification mechanism and replica management strategies, the energy effective gear-shifting mechanism (E2GSM) can automatically gear-shifting among the nodes. Mathematical analytical model certificates our proposed E2GSM is energy effective. Simulation experiments based on Gridsim show that the proposed gear-shifting mechanism is cost effective. Compared to the other energy-saved mechanism, our E2GSM can save energy consumption substantially at the slight expense of performance loss while meeting the QoS of user.

Cross Compressed Replication Scheme for Large-Volume Column Storages (대용량 컬럼 저장소를 위한 교차 압축 이중화 기법)

  • Byun, Siwoo
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.14 no.5
    • /
    • pp.2449-2456
    • /
    • 2013
  • The column-oriented database storage is a very advanced model for large-volume data analysis systems because of its superior I/O performance. Traditional data storages exploit row-oriented storage where the attributes of a record are placed contiguously in hard disk for fast write operations. However, for search-mostly datawarehouse systems, column-oriented storage has become a more proper model because of its superior read performance. Recently, solid state drive using MLC flash memory is largely recognized as the preferred storage media for high-speed data analysis systems. In this paper, we introduce fast column-oriented data storage model and then propose a new storage management scheme using a cross compressed replication for the high-speed column-oriented datawarehouse system. Our storage management scheme which is based on two MLC SSD achieves superior performance and reliability by the cross replication of the uncompressed segment and the compressed segment under high workloads of CPU and I/O. Based on the results of the performance evaluation, we conclude that our storage management scheme outperforms the traditional scheme in the respect of update throughput and response time of the column segments.

A Study on the Distribution of Overload in Academic Affairs Management System Using Replication Server (데이터 복제 서버를 이용한 학사 관리 시스템의 부하 분산에 관한 연구)

  • Han, Gwang-Rok;Lee, Seung-Won
    • The KIPS Transactions:PartD
    • /
    • v.8D no.5
    • /
    • pp.605-612
    • /
    • 2001
  • In order to solve the overload of academic affairs management system, we propose a method builds a distributed Replication server and uses this server with the present centralized system. Normal query transactions which are not required for data modification are composed of almost all DML sentences. So we construct the distributed replication servers according to the data characteristics and make them perform the query transaction without modification. In this way, we can simultaneously distribute users and data, and cut down processing time for every transaction. Also Replication server has the advantages of implemental efficiency and economical because it uses resources of present centralized system without and additional configurations. Usually, to distribute the overload of server, they can use way, Client-side overload distribution that user program get present overload status then can choose a less overloaded server, and the other way, Server-side overload distribution that make use of Application Layer Scheduling Technique and IP Layer Scheduling Technique. Our Replication server can reduce the overload of centralized system by eliminating or complementing those defects of overload distribution, referred to in the forehead.

  • PDF

Efficient Data-replication between Cluster-heads for Solar-powered Wireless Sensor Networks with Mobile Sinks

  • Jun Min Yi;Hong Sub Lee;Ikjune Yoon;Dong Kun Noh
    • Journal of Internet Technology
    • /
    • v.19 no.6
    • /
    • pp.1801-1810
    • /
    • 2018
  • In this study, an energy-aware data-replication is proposed to effectively support a mobile sink in a solar-powered wireless sensor network (WSN). By utilizing the redundant energy efficiently, the proposed scheme shares the gathered data among the cluster heads using a backbone network, in order to increase data-reliability. It also maintains a backup cluster head in each cluster to enhance topological resilience. The simulation result showed that, compared to conventional clustering techniques, the proposed scheme decreases the total amount of data loss from the mobile sink as well as saving its energy (by reducing its moving distance), while minimizing the unexpected blackout time of the sensor node.

Delayed Block Replication Scheme of Hadoop Distributed File System for Flexible Management of Distributed Nodes (하둡 분산 파일시스템에서의 유연한 노드 관리를 위한 지연된 블록 복제 기법)

  • Ryu, Woo-Seok
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.12 no.2
    • /
    • pp.367-374
    • /
    • 2017
  • This paper discusses management problems of Hadoop distributed node, which is a platform for big data processing, and proposes a novel technique for enabling flexible node management of Hadoop Distributed File System. Hadoop cannot configure Hadoop cluster dynamically because it judges temporarily unavailable nodes as a failure. Delayed block replication scheme proposed in this paper delays the removal of unavailable node as much as possible so as to be easily rejoined. Experimental results show that the proposed scheme increases flexibility of node management with little impact on distributed processing performance when the cluster size changes.

Design and Implementation of Data Replication Web Agent between Heterogeneous DBMSs based on XML (XML 기반의 이기종 DBMS간 데이터 복제 웹 에이전트 설계 및 구현)

  • Yu, Sun-Young;Yim, Jae-Hong
    • Journal of Navigation and Port Research
    • /
    • v.26 no.4
    • /
    • pp.427-433
    • /
    • 2002
  • Since current HTML used on the internet is to use restricted tag, it is not easy to store information and extract data from information of document. XML defined newly tag and is easy to store information and extract data from information. So XML is easier to transact information rather than HTML. XML is suitable for enterprise's requirement needs data exchange between heterogeneous databases. This paper proposes web agent for data replication between heterogeneous DBMSs(Database Management System). Web agent system maneges database on the web and exchange data in heterogeneous database using XML. Then we designed and implemented for web agent of data replication between heterogeneous DBMSs.

PosCFS+: A Self-Managed File Service in Personal Area Network

  • Lee, Woo-Joong;Kim, Shi-Ne;Park, Chan-Ik
    • ETRI Journal
    • /
    • v.29 no.3
    • /
    • pp.281-291
    • /
    • 2007
  • Wearable computers consisting of various small devices such as smart phones, digital cameras, MP3 players and specialized I/O devices in personal area networks will play an important role in future ubiquitous computing. In this environment, accessing user data is quite complex due to the dynamic and heterogeneous characteristics of the underlying networks. Moreover, since the amount of user data increases rapidly, automatic data backup management is also critical. To overcome these challenges, several studies have been conducted including our previously proposed file service system, PosCFS, which could be adapted to the requirements with a virtualization technique allowing per-user global namespace for managing and accessing data stored on physical storage spaces detected in PAN. In this paper, we present a smart file service framework, PosCFS+ which is an improved and extended version of our previous work. Performance improvement is made possible by redesigning the metadata management scheme based on database and keywords rather than ontology. In addition, the automatic data replication management is newly designed based on the OSD protocol.

  • PDF