• Title/Summary/Keyword: Erasure codes

Search Result 30, Processing Time 0.03 seconds

Enhanced Upper Bound for Erasure Recovery in SPC Product Codes

  • Muqaibel, Ali
    • ETRI Journal
    • /
    • v.31 no.5
    • /
    • pp.518-524
    • /
    • 2009
  • Single parity check (SPC) product codes are simple yet powerful codes that are used to correct errors and/or recover erasures. The focus of this paper is to evaluate the performance of such codes under erasure scenarios and to develop a closed-form tight upper bound for the post-decoding erasure rate. Closed-form exact expressions are derived for up to seven erasures. Previously published closed-form bounds assumed that all unrecoverable patterns should contain four erasures in a square. Additional non-square patterns are accounted for in the proposed expressions. The derived expressions are verified using exhaustive search. Eight or more erasures are accounted for by using a bound. The developed expressions improve the evaluation of the recoverability of SPC product codes without the need for simulation or search algorithms, whether exhaustive or novel.

The Design of Regenerating Codes with a Varying Number of Helper Nodes (다양한 도움 노드의 수를 가지는 재생 부호의 설계)

  • Lee, Hyuk;Lee, Jungwoo
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.41 no.12
    • /
    • pp.1684-1691
    • /
    • 2016
  • Erasure codes have recently been applied to distributed storage systems due to their high storage efficiency. Regenerating codes are a kind of erasure codes, which are optimal in terms of minimum repair bandwidth. An (n,k,d)-regenerating code consists of n storage nodes where a failed node can be recovered with the help of the exactly d numbers of surviving nodes. However, if node failures occur frequently or network connection is unstable, the number of helper nodes that a failed node can contact may be smaller than d. In such cases, regenerating codes cannot repair the failed nodes efficiently since the node repair process of the codes does not work when the number of helper nodes is less than d. In this paper, we propose an operating method of regenerating codes where a failed node can be repaired from ${\bar{d}}$ helper nodes where $$k{\leq_-}{\bar{d}}{\leq_-}d$$.

An adaptive fault tolerance strategy for cloud storage

  • Xiai, Yan;Dafang, Zhang;Jinmin, Yang
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.10 no.11
    • /
    • pp.5290-5304
    • /
    • 2016
  • With the growth of the massive amount of data, the failure probability of the cloud storage node is becoming more and more big. A single fault tolerance strategy, such as replication and erasure codes, has some unavoidable disadvantages, which can not meet the needs of the today's fault tolerance. Therefore, according to the file access frequency and size, an adaptive hybrid redundant fault tolerance strategy is proposed, which can dynamically change between the replication scheme and erasure codes scheme throughout the lifecycle. The experimental results show that the proposed scheme can not only save the storage space(reduced by 32% compared with replication), but also ensure the fast recovery of the node failures(increased by 42% compared with erasure codes).

Data Access Frequency based Data Replication Method using Erasure Codes in Cloud Storage System (클라우드 스토리지 시스템에서 데이터 접근빈도와 Erasure Codes를 이용한 데이터 복제 기법)

  • Kim, Ju-Kyeong;Kim, Deok-Hwan
    • Journal of the Institute of Electronics and Information Engineers
    • /
    • v.51 no.2
    • /
    • pp.85-91
    • /
    • 2014
  • Cloud storage system uses a distributed file system for storing and managing data. Traditional distributed file system makes a triplication of data in order to restore data loss in disk failure. However, enforcing data replication method increases storage utilization and causes extra I/O operations during replication process. In this paper, we propose a data replication method using erasure codes in cloud storage system to improve storage space efficiency and I/O performance. In particular, according to data access frequency, the proposed method can reduce the number of data replications but using erasure codes can keep the same data recovery performance. Experimental results show that proposed method improves performance in storage efficiency 40%, read throughput 11%, write throughput 10% better than HDFS does.

A Family of Concatenated Network Codes for Improved Performance With Generations

  • Thibault, Jean-Pierre;Chan, Wai-Yip;Yousefi, Shahram
    • Journal of Communications and Networks
    • /
    • v.10 no.4
    • /
    • pp.384-395
    • /
    • 2008
  • Random network coding can be viewed as a single block code applied to all source packets. To manage the concomitant high coding complexity, source packets can be partitioned into generations; block coding is then performed on each set. To reach a better performance-complexity tradeoff, we propose a novel concatenated network code which mixes generations while retaining the desirable properties of generation-based coding. Focusing on the code's erasure performance, we show that the probability of successfully decoding a generation on erasure channels can increase substantially for any erasure rate. Using both analysis (for small networks) and simulations (for larger networks), we show how the code's parameters can be tuned to extract best performance. As a result, the probability of failing to decode a generation is reduced by nearly one order of magnitude.

Decoding of LT-Like Codes in the Absence of Degree-One Code Symbols

  • Abdulkhaleq, Nadhir I.;Gazi, Orhan
    • ETRI Journal
    • /
    • v.38 no.5
    • /
    • pp.896-902
    • /
    • 2016
  • Luby transform (LT) codes were the first practical rateless erasure codes proposed in the literature. The performances of these codes, which are iteratively decoded using belief propagation algorithms, depend on the degree distribution used to generate the coded symbols. The existence of degree-one coded symbols is essential for the starting and continuation of the decoding process. The absence of a degree-one coded symbol at any instant of an iterative decoding operation results in decoding failure. To alleviate this problem, we proposed a method used in the absence of a degree-one code symbol to overcome a stuck decoding operation and its continuation. The simulation results show that the proposed approach provides a better performance than a conventional LT code and memory-based robust soliton distributed LT code, as well as that of a Gaussian elimination assisted LT code, particularly for short data lengths.

On Estimation of Redundancy Information Transmission based on Systematic Erasure code for Realtime Packet Transmission in Bursty Packet Loss Environments. (연속 패킷 손실 환경에서 실시간 패킷 전송을 위한 systematic erasure code의 부가 전송량 추정 방법)

  • 육성원;강민규;김두현;신병철;조동호
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.24 no.10B
    • /
    • pp.1824-1831
    • /
    • 1999
  • In this paper, the data recovery performance of systematic erasure codes in burst loss environments is analyzed and the estimation method of redundant data according to loss characteristics is suggested. The burstness of packet loss is modeled by Gilbert model, and the performance of proposed packet loss recovery method in the case of using systematic erasure code is analyzed based on previous study on the loss recovery in the case of using erasure code. The required redundancy data fitting method for systematic erasure code in the condition of given loss property is suggested in the consideration of packet loss characteristics such as average packet loss rate and average loss length.

  • PDF

HTSC and FH HTSC: XOR-based Codes to Reduce Access Latency in Distributed Storage Systems

  • Shuai, Qiqi;Li, Victor O.K.
    • Journal of Communications and Networks
    • /
    • v.17 no.6
    • /
    • pp.582-591
    • /
    • 2015
  • A massive distributed storage system is the foundation for big data operations. Access latency performance is a key metric in distributed storage systems since it greatly impacts user experience while existing codes mainly focus on improving performance such as storage overhead and repair cost. By generating parity nodes from parity nodes, in this paper we design new XOR-based erasure codes hierarchical tree structure code (HTSC) and high failure tolerant HTSC (FH HTSC) to reduce access latency in distributed storage systems. By comparing with other popular and representative codes, we show that, under the same repair cost, HTSC and FH HTSC codes can reduce access latency while maintaining favorable performance in other metrics. In particular, under the same repair cost, FH HTSC can achieve lower access latency, higher or equal failure tolerance and lower computation cost compared with the representative codes while enjoying similar storage overhead. Accordingly, FH HTSC is a superior choice for applications requiring low access latency and outstanding failure tolerance capability at the same time.

A Packet Loss Control Scheme based on Network Conditions and Data Priority (네트워크 상태와 데이타 중요도에 기반한 패킷 손실 제어 기법)

  • Park, Tae-Uk;Chung, Ki-Dong
    • Journal of KIISE:Information Networking
    • /
    • v.31 no.1
    • /
    • pp.1-10
    • /
    • 2004
  • This study discusses Application-layer FEC using erasure codes. Because of the simple decoding process, erasure codes are used effectively in Application-layer FEC to deal with Packet-level errors. The large number of parity packets makes the loss rate to be small, but causes the network congestion to be worse. Thus, a redundancy control algorithm that can adjust the number of parity packets depending on network conditions is necessary. In addition, it is natural that high-priority frames such as I frames should produce more parity packets than low-priority frames such as P and B frames. In this paper, we propose a redundancy control algorithm that can adjust the amount of redundancy depending on the network conditions and depending on data priority, and test the performance in simple links and congestion links.

Erasure decoding strategies for RS product code reducing undetected error rate (검출 불능 오류율을 향상기키는 Reed-Solomon 적부호의 이레이져 복호방법)

  • 김정헌;염창열;송홍엽;강구호;김순태;백세현
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.26 no.4B
    • /
    • pp.427-436
    • /
    • 2001
  • RS product codes are widely used in digital storage systems. There are lots of decoding strategies for product code for short-length RS codes. Unfortunately many of them cannot be applied to long-length RS product codes because of the complexity of decoder. This paper proposes new decoding strategies which can be used in long length RS product codes.

  • PDF