• Title/Summary/Keyword: distributed storage systems

Search Result 207, Processing Time 0.025 seconds

Optimal File Migration Policies in Distributed Database Systems (분산 데이터베이스 시스템에서의 최적 파일 이동 정책)

  • 이기태;김재련
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • v.18 no.33
    • /
    • pp.1-10
    • /
    • 1995
  • The allocation of files is essential to the efficiency and effectiveness of a distributed system that must meet geographically dispersed data processing demands. In this paper, we address an optimization model that generates optimal file migration policies in distributed database systems. The proposed model is a more generalized model that includes system's capacity constraints - computing sites' storage capacity and communication networks' capacity - which have not taken into consideration in previous researches. Using this model, we can establish initial file allocation, file reallocation and file migration polices that minimize a system operating cost under system's capacity constraints at an initial system design or reorganization point The proposed model not only can be adopted by small-sized systems but also provides a foundation for effective and simple heuristics for adaptive file migration in large systems.

  • PDF

Intermedia Synchronization Protocol for Continuous Media Using MPEG-4 in Mobile Distributed Systems

  • Dominguez, Eduardo Lopez;Hernandez, Saul Eduardo Pomares;Gil, Pilar Gomez;Calleja, Jorge De La;Benitez, Antonio;Marin-Hernandez, Antonio
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.6 no.6
    • /
    • pp.1689-1706
    • /
    • 2012
  • The preservation of temporal dependencies among a group of processes that exchange continuous media at runtime is a key issue for emerging mobile distributed systems (MDS), such as monitoring of biosignals and interactive multiuser games. Although several works are oriented to satisfy temporal dependencies, most of them are not suitable for MDSs. In general, an MDS is characterized by the absence of global references (e.g. shared memory and wall clock), host mobility, limited processing and storage capabilities in mobile hosts, and limited bandwidth on wireless communication channels. This paper proposes an asymmetric synchronization protocol to be used at runtime in an MDS without using a common reference. One main aspect of our synchronization protocol is that it translates temporal constraints to causal dependencies of the continuous media data as seen by the mobile hosts. We simulate the protocol by considering a cellular network environment and by using MPEG-4 encoders. The simulation results show that our protocol is effective in reducing the synchronization error. In addition, the protocol is efficient in terms of processing and storage costs at the mobile devices, as well as in the overhead attached per message across the wired and wireless channels.

A Study on Data Storage and Recovery in Hadoop Environment (하둡 환경에 적합한 데이터 저장 및 복원 기법에 관한 연구)

  • Kim, Su-Hyun;Lee, Im-Yeong
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.2 no.12
    • /
    • pp.569-576
    • /
    • 2013
  • Cloud computing has been receiving increasing attention recently. Despite this attention, security is the main problem that still needs to be addressed for cloud computing. In general, a cloud computing environment protects data by using distributed servers for data storage. When the amount of data is too high, however, different pieces of a secret key (if used) may be divided among hundreds of distributed servers. Thus, the management of a distributed server may be very difficult simply in terms of its authentication, encryption, and decryption processes, which incur vast overheads. In this paper, we proposed a efficiently data storage and recovery scheme using XOR and RAID in Hadoop environment.

RDP: A storage-tier-aware Robust Data Placement strategy for Hadoop in a Cloud-based Heterogeneous Environment

  • Muhammad Faseeh Qureshi, Nawab;Shin, Dong Ryeol
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.10 no.9
    • /
    • pp.4063-4086
    • /
    • 2016
  • Cloud computing is a robust technology, which facilitate to resolve many parallel distributed computing issues in the modern Big Data environment. Hadoop is an ecosystem, which process large data-sets in distributed computing environment. The HDFS is a filesystem of Hadoop, which process data blocks to the cluster nodes. The data block placement has become a bottleneck to overall performance in a Hadoop cluster. The current placement policy assumes that, all Datanodes have equal computing capacity to process data blocks. This computing capacity includes availability of same storage media and same processing performances of a node. As a result, Hadoop cluster performance gets effected with unbalanced workloads, inefficient storage-tier, network traffic congestion and HDFS integrity issues. This paper proposes a storage-tier-aware Robust Data Placement (RDP) scheme, which systematically resolves unbalanced workloads, reduces network congestion to an optimal state, utilizes storage-tier in a useful manner and minimizes the HDFS integrity issues. The experimental results show that the proposed approach reduced unbalanced workload issue to 72%. Moreover, the presented approach resolve storage-tier compatibility problem to 81% by predicting storage for block jobs and improved overall data block placement by 78% through pre-calculated computing capacity allocations and execution of map files over respective Namenode and Datanodes.

Scalable Blockchain Storage Model Based on DHT and IPFS

  • Chen, Lu;Zhang, Xin;Sun, Zhixin
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.16 no.7
    • /
    • pp.2286-2304
    • /
    • 2022
  • Blockchain is a distributed ledger that combines technologies such as cryptography, consensus mechanism, peer-to-peer transmission, and time stamping. The rapid development of blockchain has attracted attention from all walks of life, but storage scalability issues have hindered the application of blockchain. In this paper, a scalable blockchain storage model based on Distributed Hash Table (DHT) and the InterPlanetary File System (IPFS) was proposed. This paper introduces the current research status of the scalable blockchain storage model, as well as the basic principles of DHT and the InterPlanetary File System. The model construction and workflow are explained in detail. At the same time, the DHT network construction mechanism, block heat identification mechanism, new node initialization mechanism, and block data read and write mechanism in the model are described in detail. Experimental results show that this model can reduce the storage burden of nodes, and at the same time, the blockchain network can accommodate more local blocks under the same block height.

Modeling of a storage subsystem in multimedia information system

  • Lim, Cheol-Su
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.22 no.11
    • /
    • pp.2521-2530
    • /
    • 1997
  • In this paepr, we present a video-on-demand (VOD) system design model that address and integrates a number of inter-related issues. Then with analysis and performance evaluation, we investigate various aspects of disk and buffer managements in the given model. Based on the analysis results, we suggest that a distributed buffering scheme with intermediate buffers may te useful to transform bursty disk accesses into a continuous stream for for glitch-free performance of VOD systems. Also, through simulation, we illustrate that massive multimedia information storage design techniques such as prefetching, clustered striping, and real-time disk scheduling integrated with the distributed buffering mechanism may enhance end-to-end real-time performance of VOD systems under wide-area networks.

  • PDF

Agents for Autonomous Distributed Secret Sharing Storage System

  • Hayashi, Daisuke;Miyamoto, Toshiyuki;Doi, Shinji;Kumagai, Sadatoshi
    • Proceedings of the IEEK Conference
    • /
    • 2002.07a
    • /
    • pp.482-485
    • /
    • 2002
  • For mission-critical and safe-critical operations of medical information, financial, or administrative systems, a reliable and robust storage system is indispensable. The main purpose of our research is to develop a high-confidential, reliable, and survivable storage system.

  • PDF

Binary Locally Repairable Codes from Complete Multipartite Graphs (완전다분할그래프 기반 이진 부분접속복구 부호)

  • Kim, Jung-Hyun;Nam, Mi-Young;Song, Hong-Yeop
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.40 no.9
    • /
    • pp.1734-1740
    • /
    • 2015
  • This paper introduces a generalized notion, referred to as joint locality, of the usual locality in distributed storage systems and proposes a code construction of binary locally repairable codes with joint locality ($r_1$=2, $r_2$=3 or 4). Joint locality is a set of numbers of nodes for repairing various failure patterns of nodes. The proposed scheme simplifies the code design problem utilizing complete multipartite graphs. Moreover, our construction can generate binary locally repairable codes achieving (2,t)-availability for any positive integer t. It means that each node can be repaired by t disjoint repair sets of cardinality 2. This property is useful for distributed storage systems since it permits parallel access to hot data.

Review of Virtual Power Plant Applications for Power System Management and Vehicle-to-Grid Market Development (전력시스템 관리 및 Vehicle to Grid 전력시장 개발을 위한 가상발전소의 활용방안)

  • Jin, Tae-Hwan;Park, Herie;Chung, Mo;Shin, Ki-Yeol;Foley, Aoife;Cipcigan, Liana
    • The Transactions of The Korean Institute of Electrical Engineers
    • /
    • v.65 no.12
    • /
    • pp.2251-2261
    • /
    • 2016
  • The use of renewable energy sources and energy storage systems is increasing due to new policies in the energy industries. However, the increase in distributed generation hinders the reliability of power systems. In order to stabilize power systems, a virtual power plant has been proposed as a novel power grid management system. The virtual power plant plays includes different distributed energy resources and energy storage systems. We define a core virtual power plant technology related to demand response and ancillary service for the cases of Korea, America, and Europe. We also suggest applications of the proposed virtual power plant to the vehicle-to-grid market for restructuring national power industries in Korea.

A COMPARATIVE STUDY ON BLOCKCHAIN DATA MANAGEMENT SYSTEMS: BIGCHAINDB VS FALCONDB

  • Abrar Alotaibi;Sarah Alissa;Salahadin Mohammed
    • International Journal of Computer Science & Network Security
    • /
    • v.23 no.5
    • /
    • pp.128-134
    • /
    • 2023
  • The widespread usage of blockchain technology in cryptocurrencies has led to the adoption of the blockchain concept in data storage management systems for secure and effective data storage and management. Several innovative studies have proposed solutions that integrate blockchain with distributed databases. In this article, we review current blockchain databases, then focus on two well-known blockchain databases-BigchainDB and FalconDB-to illustrate their architecture and design aspects in more detail. BigchainDB is a distributed database that integrates blockchain properties to enhance immutability and decentralization as well as a high transaction rate, low latency, and accurate queries. Its architecture consists of three layers: the transaction layer, consensus layer, and data model layer. FalconDB, on the other hand, is a shared database that allows multiple clients to collaborate on the database securely and efficiently, even if they have limited resources. It has two layers: the authentication layer and the consensus layer, which are used with client requests and results. Finally, a comparison is made between the two blockchain databases, revealing that they share some characteristics such as immutability, low latency, permission, horizontal scalability, decentralization, and the same consensus protocol. However, they vary in terms of database type, concurrency mechanism, replication model, cost, and the usage of smart contracts.