• Title/Summary/Keyword: file I/O

Search Result 165, Processing Time 0.024 seconds

Design and Implementation of I/O Tracer for PVFS (PVFS를 위한 I/O Tracer 설계 및 구현)

  • Hyeyoung Cho;Kwangho Cha;Sungho Kim;SangDong Lee
    • Annual Conference of KIPS
    • /
    • 2008.11a
    • /
    • pp.966-969
    • /
    • 2008
  • 사용자 프로그램의 I/O 패턴을 분석하거나 파일 시스템의 워크로드를 보다 정확하게 분석하기 위해서 실제 가동중인 파일 시스템의 동적 I/O 로그를 확보하기 위한 연구들이 많이 진행되어 왔다. 그러나 대량의 I/O 트렌젝션(transcation)이 처리되는 파일 시스템에서 동적 I/O 로그를 확보하는 일은 시스템의 부하와 막대한 데이터량 때문에 한계가 많다. 특히 다수의 이용자가 사용하는 대용량 분산/병렬 파일 시스템에서의 I/O Tracing은 로컬 파일 시스템에서 I/O Tracing에 비해 더욱 복잡하고 오버헤드가 크다. 본 논문에서는 기존의 파일 시스템 로깅 방법들을 알아보고, 클러스터 시스템에서 널리 이용되고 있는 분산 파일 시스템인 PVFS(Parallel Virtual File System)에서 동적 I/O 연산들의 로그를 생성할 수 있는 로깅 시스템을 제안하고 설계하였다.

Performance Evaluation of a RAM based Storage System NGS

  • Kang, Yun-Hee;Kung, Jae-Ha;Cheong, Seung-Kook
    • International Journal of Contents
    • /
    • v.5 no.4
    • /
    • pp.75-80
    • /
    • 2009
  • Recently high-speed memory array based on RAM, which is a type of solid-state drive (SSD), has been introduced to handle the input/output (I/O) bottleneck. But there are only a few performance studies on RAM based SSD storage with regard to diverse workloads. In this paper, we focus on the file system for RAM based memory array based NGS (Next Generation Storage) system which is running on Linux operating system. Then we perform benchmark tests on practical file systems including Ext3, ReiserFS, XFS. The result shows XFS significantly outperforms other file systems in tests that represent the storage and data requests typically made by enterprise applications in many aspects. The experiment is used to design the dedicated file system for NGS system. The results presented here can help enterprises improve their performance significantly.

Performance Analysis of Cluster File System $SANique^{TM}$ based on Storage Area Network (SAN 기반 클러스터 파일 시스템 $SANique^{TM}$의 성능평가 및 분석)

  • Lee, Kyu-Woong
    • Journal of Information Technology Services
    • /
    • v.7 no.1
    • /
    • pp.195-204
    • /
    • 2008
  • As the dependency to network system and demands of efficient storage systems rapidly grows in every networking filed, the current trends initiated by explosive networked data grow due to the wide-spread of internet multimedia data and internet requires a paradigm shift from computing-centric to data-centric in storage systems. Furthermore, the new environment of file systems such as SAN(Storage Area Network) is adopted to the existing storage paradigm for providing high availability and efficient data access. We describe the design issues and system components of $SANique^{TM}$, which is the cluster file system based on SAN environment. We, especially, present the comparative results of performance analysis for the intensive I/O test by using the DBMSs that are operated at the top of cluster file system $SANique^{TM}$, EXT3 and NFS respectively.

Optimizing Garbage Collection Overhead of Host-level Flash Translation Layer for Journaling Filesystems

  • Son, Sehee;Ahn, Sungyong
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.13 no.2
    • /
    • pp.27-35
    • /
    • 2021
  • NAND flash memory-based SSD needs an internal software, Flash Translation Layer(FTL) to provide traditional block device interface to the host because of its physical constraints, such as erase-before-write and large erase block. However, because useful host-side information cannot be delivered to FTL through the narrow block device interface, SSDs suffer from a variety of problems such as increasing garbage collection overhead, large tail-latency, and unpredictable I/O latency. Otherwise, the new type of SSD, open-channel SSD exposes the internal structure of SSD to the host so that underlying NAND flash memory can be managed directly by the host-level FTL. Especially, I/O data classification by using host-side information can achieve the reduction of garbage collection overhead. In this paper, we propose a new scheme to reduce garbage collection overhead of open-channel SSD by separating the journal from other file data for the journaling filesystem. Because journal has different lifespan with other file data, the Write Amplification Factor (WAF) caused by garbage collection can be reduced. The proposed scheme is implemented by modifying the host-level FTL of Linux and evaluated with both Fio and Filebench. According to the experiment results, the proposed scheme improves I/O performance by 46%~50% while reducing the WAF of open-channel SSDs by more than 33% compared to the previous one.

Design and Evaluation of a High-performance Journaling Scheme for Non-volatile Memory (비휘발성 메모리를 고려한 고성능 저널링 기법 설계 및 평가)

  • Han, Hyuck
    • The Journal of the Korea Contents Association
    • /
    • v.20 no.8
    • /
    • pp.368-374
    • /
    • 2020
  • Journaling file systems (JFS) manage changes of file systems not yet committed in a data structure known as a journal to restore the file system in the event of an unexpected failure. Extra write operations required for journaling negatively affect the performance of JFS. The high-performance and byte-addressable non-volatile memory (NVM) was expected to easily mitigate these performance problems by providing NVM space as journal storage. However, even with such non-volatile memory technologies, performance problems still arise due to scalability problems inherent in processing transactions of JFS. To solve this problem, we proposes a technique for processing file system transactions for scalable performance. To this end, lock-free data structures are used and multiple I/O requests are allowed to simultaneously be processed on high-performance storage devices with multiple I/O channels. We evaluate the file system with the proposed technique by comparing the original ext4 file system and the recent proposed NVM-based journaling file system on a multi-core server, and experimental results show that our file system has better performance (up-to 2.9/2.3 times) than the original ext4 file system and the recent NVM-based journaling file system, respectively.

Ubiquitous Workspace Synchronization in a Cloud-based Framework (클라우드 기반 프레임워크에서 유비쿼터스 워크스페이스 동기화)

  • Elijorde, Frank I.;Yang, Hyunho;Lee, Jaewan
    • Journal of Internet Computing and Services
    • /
    • v.14 no.1
    • /
    • pp.53-62
    • /
    • 2013
  • It is common among users to have multiple computing devices as well as to access their files or do work at different locations. To achieve file consistency as well as mobility in this scenario, an efficient approach for workspace synchronization should be used. However, file synchronization alone cannot guarantee the mobility of work environment which allows activities to be resumed at any place and time. This paper proposes a ubiquitous synchronization approach which provides cloud-based access to a user's workspace. Efficient synchronization is achieved by combining session monitoring with file system management. Experimental results show that the proposed mechanism outperforms Cloud Master-replica Synchronization in terms of number of I/O operations, CPU utilization, as well as the average and maximum latencies in responding to client requests.

Analysis for File Access Characteristics of Mobile Artificial Intelligence Workloads (모바일 인공지능 워크로드의 파일 접근 특성 분석)

  • Jeongha Lee;Soojung Lim;Hyokyung Bahn
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.24 no.4
    • /
    • pp.77-82
    • /
    • 2024
  • Recent advancements in artificial intelligence (AI) technology have led to an increase in the implementation of AI applications in mobile environments. However, due to the limited resources in mobile devices compared to desktops and servers, there is growing interest in research aimed at efficiently executing AI workloads on mobile platforms. While most studies focus on offloading to edge or cloud solutions to mitigate computing resource constraints, research on the characteristics of file I/O related to storage access in mobile settings remains underexplored. This paper analyzes file I/O traces generated during the execution of deep learning applications in mobile environments and investigates how they differ from traditional mobile workloads. We anticipate that the findings of this study will be utilized to design future smartphone system software more efficiently, considering the file access characteristics of deep learning.

Optimizing I/O Stack for Fast Storage Devices (고속 저장 장치를 위한 입출력 스택 최적화)

  • Han, Hyuck
    • The Journal of the Korea Contents Association
    • /
    • v.16 no.5
    • /
    • pp.251-258
    • /
    • 2016
  • Recently, the demand for fast storage devices is rapidly increasing in cloud platforms, social network services, etc. Despite the development of fast storage devices, the traditional Linux I/O stack is not able to exploit the full extent of the performance improvement since it has been optimized for disk-based storage devices. In this paper, we propose an optimized I/O stack which can fully utilize the I/O bandwidth and latency of fast storage devices. To this end, we design a new I/O interface to replace the current block I/O interface and optimize our I/O interface. Our optimized I/O interface bypasses operations/layers in block I/O subsystems of the current Linux I/O stack to fully exploit fast storage devices. We also optimize the Linux file systems such as ext2 and ext4 to run on our I/O interface. We evaluate our I/O stack with multiple benchmarks and the experimental results show that our I/O stack achieves 1.7 times better throughput compared to traditional Linux I/O stack.

A Study of Storage Device Control Method for File Outflow Protecting (파일유출 방지를 위한 저장장치 제어기법에 대한 연구)

  • Choi, Joo-Ho;Rhew, Sung-Yul
    • Convergence Security Journal
    • /
    • v.6 no.2
    • /
    • pp.1-11
    • /
    • 2006
  • The files of intellectual property on computer systems have increasingly been exposed to such threats that they can be flowed out by internal users or outer attacks through the network. The File Outflow Protection System detects file outflow when users not only copy files on client computers into storage devices, but also print them. This Protection system has been designed to Win32 API hooking by I/O Manager in kernel level if files are flowed out by copying. As a result, the monitoring system has exactly detected file outflows, which is proved through testing.

  • PDF

Genome Analysis Pipeline I/O Workload Analysis (유전체 분석 파이프라인의 I/O 워크로드 분석)

  • Lim, Kyeongyeol;Kim, Dongoh;Kim, Hongyeon;Park, Geehan;Choi, Minseok;Won, Youjip
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.2 no.2
    • /
    • pp.123-130
    • /
    • 2013
  • As size of genomic data is increasing rapidly, the needs for high-performance computing system to process and store genomic data is also increasing. In this paper, we captured I/O trace of a system which analyzed 500 million sequence reads data in Genome analysis pipeline for 86 hours. The workload created 630 file with size of 1031.7 Gbyte and deleted 535 file with size of 91.4 GByte. What is interesting in this workload is that 80% of all accesses are from only two files among 654 files in the system. Size of read and write request in the workload was larger than 512 KByte and 1 Mbyte, respectively. Majority of read write operations show random and sequential patterns, respectively. Throughput and bandwidth observed in each processing phase was different from each other.