• Title/Summary/Keyword: cluster file system

Search Result 91, Processing Time 0.028 seconds

A Study of designing Parallel File System for Massive Information Processing (대규모 정보처리를 위한 병렬 화일시스템 설계에 관한 연구)

  • Jang, Si-Ung;Jeong, Gi-Dong
    • The Transactions of the Korea Information Processing Society
    • /
    • v.4 no.5
    • /
    • pp.1221-1230
    • /
    • 1997
  • In this study, the performance of a parallel file system(N-PFS), which is inplemented using conventional disks as disk arrays on a Workstation Cluster, is analyzed by using analytical method and adtual values in experiments.N-PFS can be used as high-performance file sever in small-scale server systems and effciently pro-cess massive data I/Os such as multimedia and scientifid data. In this paper, an analytical model was suggested and the correctness of the suggested was verified by analyzing the experimental values on a system.The result of the appropriate stping unit for processing massive data of the Workstation Cluster with 8 disks is 64-128Kbytes and the maximum throughput on it is 15.8 Mbytes/ses.In addition, the performance of parallel file system on massive data is bounded by the time required to copy data between buffers.

  • PDF

Fips : Dynamic File Prefetching Scheme based on File Access Patterns (Fips : 파일 접근 유형을 고려한 동적 파일 선반입 기법)

  • Lee, Yoon-Young;Kim, Chei-Yol;Seo, Dae-Wha
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.29 no.7
    • /
    • pp.384-393
    • /
    • 2002
  • A Parallel file system is normally used to support excessive file requests from parallel applications in a cluster system, whereas prefetching is useful for improving the file system performance. This paper proposes a new prefetching method, Fips(dynamic File Prefetching Scheme based on file access patterms), that is particularly suitable for parallel scientific applications and multimedia web services in a parallel file system. The proposed prefetching method introduces a dynamic prefetching scheme to predict data blocks precisely in run-time although the file access patterns are irregular. In addition, it includes an algorithm to determine whether and when the prefetching is performed using the current available I/O bandwidth. Experimental results confirmed that the use of the proposed prefetching policy in a parallel file system produced a higher file system performance.

A Method of Data Hiding in a File System by Modifying Directory Information

  • Cho, Gyu-Sang
    • Journal of the Korea Society of Computer and Information
    • /
    • v.23 no.8
    • /
    • pp.85-93
    • /
    • 2018
  • In this research, it is proposed that a method to hide data by modifying directory index entry information. It consists of two methods: a directory list hiding and a file contents hiding. The directory list hiding method is to avoid the list of files from appearing in the file explorer window or the command prompt window. By modifying the file names of several index entries to make them duplicated, if the duplicated files are deleted, then the only the original file is deleted, but the modified files are retained in the MFT entry intact. So, the fact that these files are hidden is not exposed. The file contents hiding is to allocate data to be hidden on an empty index record page that is not used. If many files are made in the directory, several 4KB index records are allocated. NTFS leaves the empty index records unchanged after deleting the files. By modifying the run-list of the index record with the cluster number of the file-to-hide, the contents of the file-to-hide are hidden in the index record. By applying the proposed method to the case of hiding two files, the file lists are not exposed in the file explorer and the command prompt window, and the contents of the file-to-hide are hidden in the empty index record. It is proved that the proposed method has effectiveness and validity.

A Global Buffer Manager for a Shared Disk File System in SAN Clusters (SAN 환경에서 공유 디스크 파일 시스템을 위한 전역 버퍼 관리자)

  • 박선영;손덕주;신범주;김학영;김명준
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.10 no.2
    • /
    • pp.134-145
    • /
    • 2004
  • With rapid growth in the amount of data transferred on the Internet, traditional storage systems have reached the limits of their capacity and performance. SAN (Storage Area Network), which connects hosts to disk with the Fibre Channel switches, provides one of the powerful solutions to scale the data storage and servers. In this environment, the maintenance of data consistency among hosts is an important issue because multiple hosts share the files on disks attached to the SAN. To preserve data consistency, each host can execute the disk I/O whenever disk read and write operations are requested. However, frequent disk I/O requests cause the deterioration of the overall performance of a SAN cluster. In this paper, we introduce a SANtopia global buffer manager to improve the performance of a SAN cluster reducing the number of disk I/Os. We describe the design and algorithms of the SANtopia global buffer manager, which provides a buffer cache sharing mechanism among the hosts in the SAN cluster. Micro-benchmark results to measure the performance of block I/O operations show that the global buffer manager achieves speed-up by the factor of 1.8-12.8 compared with the existing method using disk I/O operations. Also, File system micro-benchmark results show that SANtopia file system with the global buffer manager improves performance by the factor of 1.06 in case of directories and 1.14 in case of files compared with the file system without a global buffer manager.

Design and Implementation of APFS Object Identification Tool for Digital Forensics

  • Cho, Gyu-Sang
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.14 no.1
    • /
    • pp.10-18
    • /
    • 2022
  • Since High Sierra, APFS has been used as the main file system. It is a well-established file system that has been used stably thus far. From the perspective of digital forensics, there are still many areas to be investigated. Apple File System Reference is provided to the apple developer site, but it is not satisfactory to fully analyze APFS. Researchers know more about the structure of APFS than before, but they have not yet fully analyzed its structure to a perfect level about it. In this paper, we develop APFS object identification tool for digital forensics. The most basic and essential object identification and analysis of the APFS filesystem will be conducted with the tool. The analysis in this study serves as the background for an analysis of the checkpoint operation principle and structure, including the more complex B-tree structure of APFS. There are several options for the developed tool, but the results of two use cases will be shown here. Based on the implemented tool, it is hoped that more functions will be added to make APFS a useful tool for faster and more accurate analyses.

A Clustering File Backup Server Using Multi-level De-duplication (다단계 중복 제거 기법을 이용한 클러스터 기반 파일 백업 서버)

  • Ko, Young-Woong;Jung, Ho-Min;Kim, Jin
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.14 no.7
    • /
    • pp.657-668
    • /
    • 2008
  • Traditional off-the-shelf file server has several potential drawbacks to store data blocks. A first drawback is a lack of practical de-duplication consideration for storing data blocks, which leads to worse storage capacity waste. Second drawback is the requirement for high performance computer system for processing large data blocks. To address these problems, this paper proposes a clustering backup system that exploits file fingerprinting mechanism for block-level de-duplication. Our approach differs from the traditional file server systems in two ways. First, we avoid the data redundancy by multi-level file fingerprints technology which enables us to use storage capacity efficiently. Second, we applied a cluster technology to I/O subsystem, which effectively reduces data I/O time and network bandwidth usage. Experimental results show that the requirement for storage capacity and the I/O performance is noticeably improved.

A Linux Cluster File System for Efficient Multimedia Services (효율적인 멀티미디어 서비스를 위한 리눅스 클러스터 파일 시스템)

  • 강미연;홍재연;김형식
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2002.04a
    • /
    • pp.652-654
    • /
    • 2002
  • 최근 리눅스 클러스터 시스템의 활용 범위가 커지면서 멀티미디어 서비스를 제공하려는 시도가 나타났다. 리눅스 클러스터 시스템 상에서 효과적으로 멀티미디어 서비스를 제공하려면 적합한 파일 시스템의 도움이 필수적이다. 즉, 클러스터 파일 시스템을 통하여 응용 프로그램에 대한 단일 입출력공간을 제공하고 효율적인 파일/디렉토리 연산을 제공하는 기술이 중요하다. 본 논문에서는 리눅스 클러스터 시스템을 위한 클러스터 파일 시스템(Cluster File System, 이하 CFS로 표기)을 설계 및 구현한다. CFS는 리눅스 파일 시스템 위에 사용자 수준에서 구현된 시스템으로 사용자에게는 단일 시스템 이미지를 제공한다. 내부적으로는 대용량의 파일이 분산되어 저장되며 이를 위해 파일/디렉토리 정보도 각각의 노드에서 분산 관리된다. 사용자에게는 응용 프로그램의 개발이 용이하도록 API가 제공되며, 또한 CFS를 관리하기 위한 도구들이 제공된다.

  • PDF

Optimization of LDPC Code Replication Scheme for Cluster File System (클러스터 파일 시스템을 위한 LDPC 코드 복제 기법 최적화)

  • Kim, Se-Hoe;Lee, Won-Joo;Jeon, Chang-Ho
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2010.07a
    • /
    • pp.15-16
    • /
    • 2010
  • 최근 이슈가 되고 있는 클라우드 컴퓨팅은 대용량의 데이터를 분산 저장하고 제공할 수 있는 클러스터 파일 시스템을 필요로 한다. 이러한 클러스터 파일 시스템은 높은 신뢰성과 고가용성을 보장하기 위해서 파일 복제 기법을 사용하고 있다. 가장 많이 쓰이고 있는 복제 기법은 전체-파일 복제 기법으로 높은 파일 가용성을 제공하지만 그만큼 스토리지 오버헤드가 크다는 단점이 있다. 또 다른 복제 기법으로는 LDPC 코드를 이용한 것으로 비교적 적은 스토리지 오버헤드를 가지면서 동시에 비슷한 수준의 파일 가용성을 제공한다. 따라서 본 논문에서는 클러스터 파일 시스템을 위한 LDPC 코드 복제 기법의 최적화 방법을 제안한다.

  • PDF

Development and Performance Study of a Zero-Copy File Transfer Mechanism for Ink-based PC Cluster Systems (VIA 기반 PC 클러스터 시스템을 위한 무복사 파일 전송 메커니즘의 개발 및 성능분석)

  • Park Sejin;Chung Sang-Hwa;Choi Bong-Sik;Kim Sang-Moon
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.32 no.11_12
    • /
    • pp.557-565
    • /
    • 2005
  • This paper presents the development and implementation of a zero-copy file transfer mechanism that improves the efficiency of file transfers for PC cluster systems using hardware-based VIA(Virtual Interface Architecture) network adapters. VIA is one of the representative user-level communication interfaces, but because there is no library for file transfer, one copy occurs between kernel buffer and user boilers. Our mechanism presents a file transfer primitive that does not require the file system to be modified and allows the NIC to transfer data from the kernel buffer to the remote node directly without copying. To do this, we have developed a hardware-based VIA network adapter, which supports the PCI 64bit/66MHz bus and Gigabit Ethernet, as a NIC, and implemented a zero-copy file transfer mechanism. The experimental results show that the overhead of data coy and context switching in the sender is greatly reduced and the CPU utilization of the sender is reduced to $30\%\~40\%$ of the VIA send/receive mechanism. We demonstrate the performance of the zero-copy file transfer mechanism experimentally. and compare the results with those from existing file transfer mechanisms.

Detection and Recovery of Failure Node in SAN-based Cluster Shared File System $SANique^{TM}$ (SAN 기반 클러스터 공유 파일 시스템 $SANique^{TM}$의 오류 노드 탐지 및 회복 기법)

  • Lee, Kyu-Woong
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.13 no.12
    • /
    • pp.2609-2617
    • /
    • 2009
  • This paper describes the design overview of shared file system $SANique^{TM}$ and proposes the method for detection of failure node and recovery management algorithm. We also illustrate the characteristics and system architecture of shared file system based on SAN. In order to provide uninterrupted service, the detection and recovery methods are proposed under the all possible system failures and natural disasters. The various kinds of system failures and disasters are characterized and then the detection and recovery method are proposed in each disconnected computing node group.