• Title/Summary/Keyword: parallel file system

Search Result 72, Processing Time 0.027 seconds

Implementation of High Speed Transfer System for HD Video Files using Parallel TCP (Parallel TCP 를 이용한 고속 HD 영상파일 전송시스템의 구현)

  • Park, Hyoungyill;Song, Byungjun;Lee, Junggyu;Shin, Yongtae
    • Proceedings of the Korean Society of Broadcast Engineers Conference
    • /
    • 2013.11a
    • /
    • pp.20-23
    • /
    • 2013
  • Recently for the transfer of large size video file between sites for high-definition craft editing used by broadcasting company the Public Network is used a lot. In the IP Public Network with QoS(Quality-of-Service) not guaranteed, degradation of performance has several causes. In this paper, we have analyzed the causes of low performance to transfer a large size HD video file in long distance, in order to maximize the effectiveness, the high efficiency network could be implemented enabling the highspeed of HD video file using the connection with networks between hosts through packet creation and multi-session of Parallel TCP. We survey the result of high speed transfer system and verify the efficient transfer method using Public IP Network for large HD video file transfer in broadcasting cooperation.

  • PDF

A Study of designing Parallel File System for Massive Information Processing (대규모 정보처리를 위한 병렬 화일시스템 설계에 관한 연구)

  • Jang, Si-Ung;Jeong, Gi-Dong
    • The Transactions of the Korea Information Processing Society
    • /
    • v.4 no.5
    • /
    • pp.1221-1230
    • /
    • 1997
  • In this study, the performance of a parallel file system(N-PFS), which is inplemented using conventional disks as disk arrays on a Workstation Cluster, is analyzed by using analytical method and adtual values in experiments.N-PFS can be used as high-performance file sever in small-scale server systems and effciently pro-cess massive data I/Os such as multimedia and scientifid data. In this paper, an analytical model was suggested and the correctness of the suggested was verified by analyzing the experimental values on a system.The result of the appropriate stping unit for processing massive data of the Workstation Cluster with 8 disks is 64-128Kbytes and the maximum throughput on it is 15.8 Mbytes/ses.In addition, the performance of parallel file system on massive data is bounded by the time required to copy data between buffers.

  • PDF

A Parallel Processing Technique for Large Spatial Data (대용량 공간 데이터를 위한 병렬 처리 기법)

  • Park, Seunghyun;Oh, Byoung-Woo
    • Spatial Information Research
    • /
    • v.23 no.2
    • /
    • pp.1-9
    • /
    • 2015
  • Graphical processing unit (GPU) contains many arithmetic logic units (ALUs). Because many ALUs can be exploited to process parallel processing, GPU provides efficient data processing. The spatial data require many geographic coordinates to represent the shape of them in a map. The coordinates are usually stored as geodetic longitude and latitude. To display a map in 2-dimensional Cartesian coordinate system, the geodetic longitude and latitude should be converted to the Universal Transverse Mercator (UTM) coordinate system. The conversion to the other coordinate system and the rendering process to represent the converted coordinates to screen use complex floating-point computations. In this paper, we propose a parallel processing technique that processes the conversion and the rendering using the GPU to improve the performance. Large spatial data is stored in the disk on files. To process the large amount of spatial data efficiently, we propose a technique that merges the spatial data files to a large file and access the file with the method of memory mapped file. We implement the proposed technique and perform the experiment with the 747,302,971 points of the TIGER/Line spatial data. The result of the experiment is that the conversion time for the coordinate systems with the GPU is 30.16 times faster than the CPU only method and the rendering time is 80.40 times faster than the CPU.

Sim-Hadoop : Leveraging Hadoop Distributed File System and Parallel I/O for Reliable and Efficient N-body Simulations (Sim-Hadoop : 신뢰성 있고 효율적인 N-body 시뮬레이션을 위한 Hadoop 분산 파일 시스템과 병렬 I / O)

  • Awan, Ammar Ahmad;Lee, Sungyoung;Chung, Tae Choong
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2013.05a
    • /
    • pp.476-477
    • /
    • 2013
  • Gadget-2 is a scientific simulation code has been used for many different types of simulations like, Colliding Galaxies, Cluster Formation and the popular Millennium Simulation. The code is parallelized with Message Passing Interface (MPI) and is written in C language. There is also a Java adaptation of the original code written using MPJ Express called Java Gadget. Java Gadget writes a lot of checkpoint data which may or may not use the HDF-5 file format. Since, HDF-5 is MPI-IO compliant, we can use our MPJ-IO library to perform parallel reading and writing of the checkpoint files and improve I/O performance. Additionally, to add reliability to the code execution, we propose the usage of Hadoop Distributed File System (HDFS) for writing the intermediate (checkpoint files) and final data (output files). The current code writes and reads the input, output and checkpoint files sequentially which can easily become bottleneck for large scale simulations. In this paper, we propose Sim-Hadoop, a framework to leverage HDFS and MPJ-IO for improving the I/O performance of Java Gadget code.

Research about Cluster file system construction through performance analysis of Parallel file system (병렬 가상 파일 시스템의 성능 분석을 통한 Cluster 파일 시스템 구축에 대한 연구)

  • 박준완;윤천균
    • Proceedings of the Korea Multimedia Society Conference
    • /
    • 2003.11b
    • /
    • pp.1002-1005
    • /
    • 2003
  • 최근 리눅스의 보급이 활발하게 이루어지면서 저비용으로 슈퍼컴퓨터의 성능에 버금가는 클러스터 시스템이 활발히 구현되고 있다. 이에 맞추어 클러스터 전체 노드 디스크의 저장 공간을 효율적으로 사용할 수 있는 네트워크 파일 시스템이 필요로 하게 되었다. 클러스터 파일 시스템 구축시 가장 보편적으로 사용되는 파일 시스템인 Network File System(NFS)은 보안의 위험성과 데이터 access시 성능이 떨어지는 단점 때문에 NFS을 대체할 수 있는 파일시스템을 필요로 하게 되었다. 본 논문에서는 리눅스에서 사용하는 Ext3 파일 시스템의 성능을 기준으로 최근 활발한 연구가 진행중인 파일 시스템중 하나인 가상 병렬 파일 시스템과 NFS와의 성능 비교 및 시험을 통하여 최적화된 Cluster 파일 시스템에 대해서 연구하고자 한다.

  • PDF

High Performance Computing: Infrastructure, Application, and Operation

  • Park, Byung-Hoon;Kim, Youngjae;Kim, Byoung-Do;Hong, Taeyoung;Kim, Sungjun;Lee, John K.
    • Journal of Computing Science and Engineering
    • /
    • v.6 no.4
    • /
    • pp.280-286
    • /
    • 2012
  • The last decades have witnessed an increasingly indispensible role of high performance computing (HPC) in science, business and financial sectors, as well as military and national security areas. To introduce key aspects of HPC to a broader community, an HPC session was organized for the first time ever for the United States and Korea Conference (UKC) during 2012. This paper summarizes four invited talks that each covers scientific HPC applications, large-scale parallel file systems, administration/maintenance of supercomputers, and green technology towards building power efficient supercomputers of the next generation.

Design and Implementation of parallel Media server in current system environment (기존 시스템 환경에서의 병렬 미디어 서버의 설계 및 구현)

  • 김경훈;류재상;김서균;남지승
    • Proceedings of the IEEK Conference
    • /
    • 2000.06c
    • /
    • pp.97-100
    • /
    • 2000
  • As network resources have become faster and demands for multimedia service through network have increased, the demand for Media server system has increased. These kinds of media server solve their bottle neck problem of internal storage device by using parallel system which takes advantage of fast network resource. Many vendors have suggested each of their media server system to solve these problem radically, but most of them require major modification of infra component and additional drawback has added. For example, storage mechanism for specific media requires new file system which is totally different from traditional one, and algorithm for enhancing performance may not suit for traditional operating system environment. In this paper, we designed a parallel media server based on web interface of traditional system and implemented a program for media server. Implemented server system performs parallel processing through web interface without any modification of traditional system, and controls which is related to merging load by distributed data is charged only to client and control server and consequently load of storage server can be minimized. And also, data transfer protocol for streaming media includes Retransfer algorithm and client Admission control policy relevant to performance of whole system.

  • PDF

A Parallel HDFS and MapReduce Functions for Emotion Analysis (감성분석을 위한 병렬적 HDFS와 맵리듀스 함수)

  • Back, BongHyun;Ryoo, Yun-Kyoo
    • Journal of the Korea society of information convergence
    • /
    • v.7 no.2
    • /
    • pp.49-57
    • /
    • 2014
  • Recently, opinion mining is introduced to extract useful information from SNS data and to evaluate the true intention of users. Opinion mining are required several efficient techniques to collect and analyze a large amount of SNS data and extract meaningful data from them. Therefore in this paper, we propose a parallel HDFS(Hadoop Distributed File System) and emotion functions based on Mapreduce to extract some emotional information of users from various unstructured big data on social networks. The experiment results have verified that the proposed system and functions perform faster than O(n) for data gathering time and loading time, and maintain stable load balancing for memory and CPU resources.

  • PDF

A Study on Phon Call Big Data Analytics (전화통화 빅데이터 분석에 관한 연구)

  • Kim, Jeongrae;Jeong, Chanki
    • Journal of Information Technology and Architecture
    • /
    • v.10 no.3
    • /
    • pp.387-397
    • /
    • 2013
  • This paper proposes an approach to big data analytics for phon call data. The analytical models for phon call data is composed of the PVPF (Parallel Variable-length Phrase Finding) algorithm for identifying verbal phrases of natural language and the word count algorithm for measuring the usage frequency of keywords. In the proposed model, we identify words using the PVPF algorithm, and measure the usage frequency of the identified words using word count algorithm in MapReduce. The results can be interpreted from various viewpoints. We design and implement the model based HDFS (Hadoop Distributed File System), verify the proposed approach through a case study of phon call data. So we extract useful results through analysis of keyword correlation and usage frequency.

Performance Study of the Index-based Parallel Join

  • Jeong, Byeong-Soo;Edward Omiecinski
    • The Journal of Information Technology and Database
    • /
    • v.2 no.2
    • /
    • pp.87-109
    • /
    • 1995
  • The index file has been used a access database records effectively. The join operation in a relational database system requires a large execution time, especially in the case of handling large size tables. If the indexes are available on the joining attributes for both relations involved in the join and the join selectivity is relatively small, we can improve the execution time of the join operation. In this paper. we investigate the performance trade-offs of parallel index-based join algorithms where different indexing schemes are used. We also present a comparison of our index-based parallel join algorithms with the hash-based parallel join algorithm.

  • PDF