• Title/Summary/Keyword: 분산 스토리지

Search Result 86, Processing Time 0.029 seconds

User-Centric Disaster Recovery System Based on Proxy Re-Encryption Using Blockchain and Distributed Storage (블록체인과 분산 스토리지를 활용한 프록시 재암호화 기반의 사용자 중심 재해 복구 시스템)

  • Park, Junhoo;Kim, Geunyoung;Kim, Junseok;Ryou, Jaecheol
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.31 no.6
    • /
    • pp.1157-1169
    • /
    • 2021
  • The disaster recovery refers to policies and procedures to ensure continuity of services and minimize loss of resources and finances in case of emergency situations such as natural disasters. In particular, the disaster recovery method by the cloud service provider has advantages such as management flexibility, high availability, and cost effectiveness. However, this method has a dependency on a service provider and has a structural limitation in which a user cannot be involved in personal data. In this paper, we propose a protocol using proxy re-encryption for data confidentiality by removing dependency on service providers by backing up user data using blockchain and distributed storage. The proposed method is implemented in Ethereum and IPFS environments, and presents the performance and cost required for backup and recovery operations.

Performance Analysis of Lustre File System using High Performance Storage Devices (고성능 스토리지를 이용한 Lustre 파일 시스템의 성능 분석 연구)

  • Lee, Jaehwan;Koo, Donghun;Park, Kyungmin;Kim, Jiksoo;Hwang, Soonwook
    • KIISE Transactions on Computing Practices
    • /
    • v.22 no.4
    • /
    • pp.163-169
    • /
    • 2016
  • Lustre is a scalable, distributed file system, which is popular in the field of high-performance computing. Recently, the advent of SSD has enabled high-performance storage hardware, but software development requires further improvement. In this paper, we analyzed performance of the Lustre system using SSD via extensive experimentation. We compared performance of Lustre on SSDs and HDDs in terms of file read/write throughputs and metadata access latencies. Our experimental results showed that 1) SSDs improve metadata access performance due to fast random read/write access of SSD characteristics, and 2) SSD are benefited to a greater extent under multiple threads and large numbers of small sized files.

Optimization and Stabilization of Satellite Data Distributed Processing System (위성 데이터 분산처리 시스템 최적화 및 안정화)

  • Choi, Yun-Soo;Lee, Won-Goo;Lee, Min-Ho;Kim, Sun-Tae;Lee, Sang-Hwan
    • Journal of the Korea Society of Computer and Information
    • /
    • v.18 no.11
    • /
    • pp.13-21
    • /
    • 2013
  • The goal of this paper is to provide performance improvement and stability for satellite data correction of some distortions due to cloud or radiance through distributed processing on cluster. To do this, we proposed and implemented SGE(Sun Grid Engine) based distributed processing methods using local storages and a status table. In the verification, the experiment result revealed that the proposed system on seven nodes improved the processing speed by 138.81% as compare to the existing system and provided good stability as well. This result showed that the proposed distributed processing work is more appropriate to process CPU bound jobs than I/O bound jobs. We expect that the proposed system will give scientists improved analysis performance in various fields and near-real time analysis services.

Design and performance evaluation of a storage cloud service model over KREONET (KREONET 기반의 스토리지 클라우드 서비스 모델 설계 및 성능평가)

  • Hong, Wontaek;Chung, Jinwook
    • Journal of the Korea Convergence Society
    • /
    • v.8 no.7
    • /
    • pp.29-37
    • /
    • 2017
  • Compared to the commercial networks, R&E networks have the strength such as flexible network engineering and design. Based on those features of R&E networks, we propose our storage cloud service model which supports general-purpose network users in a central region and experimental network users in distributed regions simultaneously. We prototype our service model utilizing multiple proxy controllers of OpenStack Swift service in order to deploy several regions via experimental backbone networks. Our experiments on the influence of the network latency and the size of data to be transmitted show that the bigger size of data is preferable to the smaller size of data in an experimental backbone network where the network latency increases within 10ms because the rate of throughput decline in the bigger object is comparatively small. It means that our service model is appropriate for experimental network users who directly access the service in order to move intermittently high volume of data as well as normal users in the central region who access the service frequently.

Design and Implementation of a Metadata Structure for Large-Scale Shared-Disk File System (대용량 공유디스크 파일 시스템에 적합한 메타 데이타 구조의 설계 및 구현)

  • 이용주;김경배;신범주
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.30 no.1
    • /
    • pp.33-49
    • /
    • 2003
  • Recently, there have been large storage demands for manipulating multimedia data. To solve the tremendous storage demands, one of the major researches is the SAN(Storage Area Network) that provides the local file requests directly from shared-disk storage and also eliminates the server bottlenecks to performance and availability. SAN also improve the network latency and bandwidth through new channel interface like FC(Fibre Channel). But to manipulate the efficient storage network like SAN, traditional local file system and distributed file system are not adaptable and also are lack of researches in terms of a metadata structure for large-scale inode object such as file and directory. In this paper, we describe the architecture and design issues of our shared-disk file system and provide the efficient bitmap for providing the well-formed block allocation in each host, extent-based semi flat structure for storing large-scale file data, and two-phase directory structure of using Extendible Hashing. Also we describe a detailed algorithm for implementing the file system's device driver in Linux Kernel and compare our file system with the general file system like EXT2 and shard disk file system like GFS in terms of file creation, directory creation and I/O rate.

Distributed In-Memory Caching Method for ML Workload in Kubernetes (쿠버네티스에서 ML 워크로드를 위한 분산 인-메모리 캐싱 방법)

  • Dong-Hyeon Youn;Seokil Song
    • Journal of Platform Technology
    • /
    • v.11 no.4
    • /
    • pp.71-79
    • /
    • 2023
  • In this paper, we analyze the characteristics of machine learning workloads and, based on them, propose a distributed in-memory caching technique to improve the performance of machine learning workloads. The core of machine learning workload is model training, and model training is a computationally intensive task. Performing machine learning workloads in a Kubernetes-based cloud environment in which the computing framework and storage are separated can effectively allocate resources, but delays can occur because IO must be performed through network communication. In this paper, we propose a distributed in-memory caching technique to improve the performance of machine learning workloads performed in such an environment. In particular, we propose a new method of precaching data required for machine learning workloads into the distributed in-memory cache by considering Kubflow pipelines, a Kubernetes-based machine learning pipeline management tool.

  • PDF

Non-Disruptive Server Management for Sustainable Resource Service Based on On-Premise (온-프레미스 기반 지속적인 자원 서비스를 위한 서버 무중단 기법)

  • Kim, Hyun-Woo
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.7 no.12
    • /
    • pp.295-300
    • /
    • 2018
  • The rapid development of IT, many conventional passive jobs have been automated. This automation increases the leisure time of many people and various services are being developed for them. In addition, with the advent of smart devices that are compact and portable, it is possible to use various internet services without any time and place discretion. Various studies based on virtualization are under way to efficiently store and process large data generated by many devices and services. Desktop Storage Virtualization (DSV), which integrates and provides users with on-premise-based distributed desktop resources during these studies, uses virtualization to consolidate unused resources within distributed, legacy desktops. This DSV is very important for providing high reliability to users. In addition, research on hierarchical structure and resource integration for efficient data distribution storage processing in a distributed desktop-based resource integration environment is underway. However, there is a lack of research on efficient operation in case of server failure in on-premise resource integration environment. In this paper, we propose Non-disruptive Server Management (NSM) which can actively cope with the failure of desktop server in distributed desktop storage environment based on on-premise. NSM is easy to add and remove desktops in a desktop-based integrated environment. In addition, an alternative server is actively performed in response to a failure occurrence.

Implementation of Data processing of the High Availability for Software Architecture of the Cloud Computing (클라우드 서비스를 위한 고가용성 대용량 데이터 처리 아키텍쳐)

  • Lee, Byoung-Yup;Park, Junho;Yoo, Jaesoo
    • The Journal of the Korea Contents Association
    • /
    • v.13 no.2
    • /
    • pp.32-43
    • /
    • 2013
  • These days, there are more and more IT research institutions which foresee cloud services as the predominant IT service in the near future and there, in fact, are actual cloud services provided by some IT leading vendors. Regardless of physical location of the service and environment of the system, cloud service can provide users with storage services, usage of data and software. On the other hand, cloud service has challenges as well. Even though cloud service has its edge in terms of the extent to which the IT resource can be freely utilized regardless of the confinement of hardware, the availability is another problem to be solved. Hence, this paper is dedicated to tackle the aforementioned issues; prerequisites of cloud computing for distributed file system, open source based Hadoop distributed file system, in-memory database technology and high availability database system. Also the author tries to body out the high availability mass distributed data management architecture in cloud service's perspective using currently used distributed file system in cloud computing market.

Safe Data Sharing Scheme in Cloud Computing (클라우드 컴퓨팅 환경에서의 안전한 데이터 공유 기법)

  • Kim, Su-Hyun;Lee, Im-Yeong
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2015.10a
    • /
    • pp.722-725
    • /
    • 2015
  • 분산 컴퓨팅 환경에서 다양한 데이터 서비스가 가능해지면서 대용량 데이터의 분산관리가 주요 이슈로 떠오르고 있다. 한편, 대용량 데이터의 다양한 이용 형태로부터 악의적인 공격자나 내부 사용자에 의한 보안 취약성 및 프라이버시 침해가 발생할 수 있다. 민감한 데이터들이 클라우드 서버 내에 저장되어 사용될 때, 외부 공격자나 내부 사용자의 미흡한 관리로 인한 데이터 유출 문제가 발생할 수 있다. 이를 해결하기 위해 데이터에 대한 암호화를 통해 관리가 가능하다. 하지만 기존의 단순한 암호화 방식은 클라우드 환경에 저장된 데이터의 접근 관리에 따른 문제점이 존재한다. 또한, 기존의 데이터 암호 기술들은 클라우드 스토리지 상에서 여러 사용자 간의 데이터 공유 서비스에 적용하기 힘든 단점을 가지고 있다. 따라서 본 논문에서는 속성기반 암호로 암호화된 키를 재암호화하여 다른 사용자와 안전하고 효율적으로 공유할 수 있는 데이터 공유기법을 제안한다.

Improvement of Reliability for Hadoop Distributed File System using Snapshot and Access Control (스냅샷과 접근제한 기법을 이용한 하둡 분산 파일 시스템의 신뢰성 향상)

  • Shin, Dong Hoon;Youn, Hee Yong
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2009.11a
    • /
    • pp.137-138
    • /
    • 2009
  • 다양한 스토리지와 파일 시스템이 시스템의 신뢰도를 증가시키기 위해 스냅샷을 이용하고 있다.[1] 또한, 최근에는 정보 보호의 중요성에 관심이 많아지면서 많은 시스템이 자료 보안에 신경을 쓰고 있다. 하지만, 대표적인 분산 컴퓨터 시스템 중 하나인 하둡은 관련 기능을 제공하지 않는데, 이는 나중에 문제가 될 만한 여지가 농후하다. 본 논문에서는 현재 하둡 시스템의 신뢰도에 영향을 끼치는 결점에 대하여 언급하고, 그에 대한 보완의 일부로 스냅샷과 접근 제어 기능을 제안한다.