• Title/Summary/Keyword: file I/O

Search Result 165, Processing Time 0.024 seconds

An Integrated File System for Guaranteeing the Quality of Service of Multimedia Stream (멀티미디어 스트림의 QoS를 보장하는 통합형 파일시스템)

  • 김태석;박경민;최정완;김두한;원유집;고건;박승민;김정기
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.31 no.9
    • /
    • pp.527-535
    • /
    • 2004
  • Handling mixed workload in digital set-top box or streaming server becomes an important issue as integrated file system gets momentum as the choice for the next generation file system. The next generation file system is required to handle real-time audio/video playback while being able to handle text requests such as web page, image file, etc. Legacy file system provides only best effort I/O service and thus cannot properly support the QoS of soft real-time I/O. In this paper, we would like to present our experience in developing the file system which fan guarantee the QoS of multimedia stream. We classify all application I/O requests into two category: periodic I/O and sporadic I/O. The QoS requirement of multimedia stream could be guaranteed by giving a higher priority to periodic requests than sporadic requests. The proto-type file system(Qosfs) is developed on Linux Operating System.

AIOPro: A Fully-Integrated Storage I/O Profiler for Android Smartphones (AIOPro: 안드로이드 스마트폰을 위한 통합된 스토리지 I/O 분석도구)

  • Hahn, Sangwook Shane;Yee, Inhyuk;Ryu, Donguk;Kim, Jihong
    • Journal of KIISE
    • /
    • v.44 no.3
    • /
    • pp.232-238
    • /
    • 2017
  • Application response time is critical to end-user response time in Android smartphones. Due to the plentiful resources of recent smartphones, storage I/O response time becomes a major key factor in application response time. However, existing storage I/O trace tools for Android and Linux give limited information only for a specific I/O layer which makes it difficult to combine I/O information from different I/O layers, because not helpful for application developer and researchers. In this paper, we propose a novel storage I/O trace tool for Android, called AIOPro (Android I/O profiler). It traces storage I/O from application - Android platform - system call - virtual file system - native file system - page cache - block layer - SCSI layer and device driver. It then combines the storage I/O information from I/O layers by linking them with file information and physical address. Our evaluations of real smartphone usage scenarios and benchmarks show that AIOPro can track storage I/O information from all I/O layers without any data loss under 0.1% system overheads.

Performance Analysis of Multimedia File System

  • Park, Jinyoun;Youjip Won;Jaideep Srivastava
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2001.04a
    • /
    • pp.100-102
    • /
    • 2001
  • Intensive I/O bandwidth demand of the multimedia streaming service puts significant burden on file system. Different from the legacy text based or image data, the semantics of the data in multimedia format can be significantly affected if the data block is not delivered by the predefined deadline. The legacy file system used in Unix or Unix like environment is designed to efficiently handle the files who sizes range from few hundreds of byte to several tens of gigabytes. This fundamental design philosophy results in the file system based on multi level skewed tree structure. Multi level i-node structure has significant drawback when the application performs sequential read operation. In this article, we present the result of the performance study of the file system which is specifically designed for handling multimedia streams. We implemented the file system on Linux Operating System environment and examines the performance behavior of the file system under streaming I/O workload. The result of the study shows that the proposed file system performs much more efficiently than the ext2 file system of Linux does.

A Method of Selecting Layered File System Based on Learning Block I/O History for Service-Customized Container (서비스 맞춤형 컨테이너를 위한 블록 입출력 히스토리 학습 기반 컨테이너 레이어 파일 시스템 선정 기법)

  • Yong, Chanho;Na, Sang-Ho;Lee, Pill-Woo;Huh, Eui-Nam
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.6 no.10
    • /
    • pp.415-420
    • /
    • 2017
  • Virtualization technique of OS-level is a new paradigm for deploying applications, and is attracting attention as a technology to replace traditional virtualization technique, VM (Virtual Machine). Especially, docker containers are capable of distributing application images faster and more efficient than before by applying layered image structures and union mount point to existing linux container. These characteristics of containers can only be used in layered file systems that support snapshot functionality, so it is required to select appropriate layered file systems according to the characteristics of the containerized application. We examine the characteristics of representative layered file systems and conduct write performance evaluations of each layered file systems according to the operating principles of the layered file system, Allocate-on-Demand and Copy-up. We also suggest the method of determining a appropriate layered file system principle for unknown containerized application by learning block I/O usage history of each layered file system principles in artificial neural network. Finally we validate effectiveness of artificial neural network created from block I/O history of each layered file system principles.

Monitoring System of File Outflow through Storage Devices and Printers (저장매체와 프린터를 통한 파일유출 모니터링시스템)

  • Choi Joo-ho;Rhew Sung-yul
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.15 no.4
    • /
    • pp.51-60
    • /
    • 2005
  • The riles or intellectual property on computer systems have increasingly been exposed to such threats that they can be flowed out by internal users or outer attacks through the network. The File Outflow Monitoring System monitors file outflows at server by making the toe when users copy files on client computers into storage devices or print them, The monitoring system filters I/O Request packet by I/O Manager in kernel level if files are flowed out by copying, while it uses Win32 API hooking if printed. As a result, it has exactly made the log and monitored file outflows, which is proved through testing in Windows 2000 and XP.

Design and Implementation of the Parallel Multimedia File System on Fast Ethernet (Fast Ethernet 환경에서 병렬 멀티미디어 파일 시스템의 설계와 구현)

  • Park, Seong-Ho;Kim, Gwang-Mun;Jeong, Gi-Dong
    • The KIPS Transactions:PartB
    • /
    • v.8B no.1
    • /
    • pp.89-97
    • /
    • 2001
  • 대용량 멀티미디어 미디어 서버를 구성함에 있어 I/O 병목현상을 극복하기 위하여 저장 서버들과 제어 서버로 구성되어진 2계층 분산 클러스터 서버구조가 많이 사용된다. 2 계층 분산 클러스터 서버는 부하 균등, 대역폭 관리 및 저장 서버의 관리 측면에서 유리한 반면, 저장 서버와 제어 서버간의 통신 오버헤드를 발생시킨다. 이러한 오버헤드를 줄이기 위해서는 저장 서버에서 읽은 미디어 데이터를 제어 서버를 거치지 않고 직접 클라이언트에 전송할 수 있어야 한다. 그리고, 저장 용량을 확장하거나 손상된 디스크를 교체하는 경우를 대비하여 분산 클러스터 서버는 다양한 성능의 이기종 디스크를 지원하여야 한다. 또한, I/O 장치와 운영체제가 빠르게 발전됨에 따라 미디어 서버는 새로운 I/O 장치 및 운영체제 등에 쉽게 이식될 수 있어야 하고, 응용 소프트웨어 개발자가 시스템의 환경에 따라 블록크기, 데이터 배치정책, 사본 정책 등을 유연하게 조절할 수 있어야 한다. 본 논문에서 위에서 언급한 멀티미디어 서버의 요구를 고려하여 Fast Ethernet 환경에서 병렬 멀티미디어 파일 시스템(PMFS : Parallel Multimedia File System)을 설계 및 구현하고 실험을 통해 PVFS(Parallel Virtual File System)와 성능을 비교 분석하였다. 이 실험의 결과에 따르면 PMFS는 멀티미디어 데이터에 대하여 PVFS보다 3%∼15%의 향상된 성능을 보였다.

  • PDF

Sim-Hadoop : Leveraging Hadoop Distributed File System and Parallel I/O for Reliable and Efficient N-body Simulations (Sim-Hadoop : 신뢰성 있고 효율적인 N-body 시뮬레이션을 위한 Hadoop 분산 파일 시스템과 병렬 I / O)

  • Awan, Ammar Ahmad;Lee, Sungyoung;Chung, Tae Choong
    • Annual Conference of KIPS
    • /
    • 2013.05a
    • /
    • pp.476-477
    • /
    • 2013
  • Gadget-2 is a scientific simulation code has been used for many different types of simulations like, Colliding Galaxies, Cluster Formation and the popular Millennium Simulation. The code is parallelized with Message Passing Interface (MPI) and is written in C language. There is also a Java adaptation of the original code written using MPJ Express called Java Gadget. Java Gadget writes a lot of checkpoint data which may or may not use the HDF-5 file format. Since, HDF-5 is MPI-IO compliant, we can use our MPJ-IO library to perform parallel reading and writing of the checkpoint files and improve I/O performance. Additionally, to add reliability to the code execution, we propose the usage of Hadoop Distributed File System (HDFS) for writing the intermediate (checkpoint files) and final data (output files). The current code writes and reads the input, output and checkpoint files sequentially which can easily become bottleneck for large scale simulations. In this paper, we propose Sim-Hadoop, a framework to leverage HDFS and MPJ-IO for improving the I/O performance of Java Gadget code.

A Disk Allocation Scheme for High-Performance Parallel File System (고성능 병렬화일 시스템을 위한 디스크 할당 방법)

  • Park, Kee-Hyun
    • The Transactions of the Korea Information Processing Society
    • /
    • v.7 no.9
    • /
    • pp.2827-2835
    • /
    • 2000
  • In recent years, much attention has been focused on improving I/O devices' processing speed which is essential in such large data processing areas as multimedia data processing. And studies on high-performance parallel file systems are considered to be one of such efforts. In this paper, an efficient disk allocation scheme is proposed for high-performance parallel file systems. In other words, the concept of a parallel disk file's parallelism is defined using data declustering characteristic of a given parallel file. With the concept, an efficient disk allocation scheme is proposed which calculates the appropriate degree of data declustering on disks for each parallel file in order to obtain the maximum throughput when more than one parallel file is used at the same time. Since, calculation for obtaining the maximum throughput is too complex as the number of parallel files increases, an approximate disk allocation algorithm is also proposed in this paper. The approximate algorithm is very simple and especially provides very good results when I/O workload is high. In addition, it has shown that the approximate algorithm provides the optimal disk allocation for the maximum throughput when the arrival rate of I/O requests is infinite.

  • PDF

Provisioning Scheme of Large Volume File for Efficient Job Execution in Grid Environment (그리드 환경에서 효율적인 작업 처리를 위한 대용량 파일 프로비저닝 방안)

  • Kim, Eun-Sung;Yeom, Beon-Y.
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.15 no.8
    • /
    • pp.525-533
    • /
    • 2009
  • Staging technique is used to provide files for a job in the Grid. If a staged file has large volume, the start time of the job is delayed and the throughput of job in the Grid may decrease. Therefore, removal of staging overhead helps the Grid operate more efficiently. In this paper, we present two methods for efficient file provisioning to clear the overhead. First, we propose RA-RFT, which extends RFT of Globus Toolkit and enables it to utilize RLS with replica information. RA-RFT can reduce file transfer time by doing partial transfer for each replica in parallel. Second, we suggest Remote Link that uses remote I/O instead of file transfer. Remote link is able to save storage of computational nodes and enables fast file provisioning via prefetching. Through various experiments, we argue that our two methods have an advantage over existing staging techniques.

I/O Performance Analysis about Memory Allocation of the UBIFS (UBIFS 메모리 할당에 관한 I/O 성능 분석)

  • Lee, Jaekang;Oh, Sejin;Chung, Kyungho;Yun, Taejin;Ahn, Kwangseon
    • Journal of Korea Society of Industrial Information Systems
    • /
    • v.18 no.4
    • /
    • pp.9-18
    • /
    • 2013
  • Flash memory is mostly used on smart devices and embedded systems because of its nonvolatile memory, low power, quick I/O, resistant shock, and other benefits. Generally the typical file systems base on the NAND flash memory are YAFFS2, JFFS2, UBIFS, and etc. In this paper, we had variously made an experiment regarding I/O performance using our schemes and the UBIFS of the latest Linux Kernel. The proposed I/O performance analyses were classified as a sequential access and a random access. Our experiment consists of 6 cases using kmalloc(), vmalloc(), and kmem_cache(). As a result of our experiment analyses, the sequential reading and the sequential rewriting increased by 12%, 11% when the Case 2 has applied vmalloc() and kmalloc() to the UBI subsystem and the UBIFS. Also, the performance improved more by 7.82%, 6.90% than the Case 1 at the random read and the random write.