• Title/Summary/Keyword: Parallel data processing

Search Result 751, Processing Time 0.047 seconds

A Study of designing Parallel File System for Massive Information Processing (대규모 정보처리를 위한 병렬 화일시스템 설계에 관한 연구)

  • Jang, Si-Ung;Jeong, Gi-Dong
    • The Transactions of the Korea Information Processing Society
    • /
    • v.4 no.5
    • /
    • pp.1221-1230
    • /
    • 1997
  • In this study, the performance of a parallel file system(N-PFS), which is inplemented using conventional disks as disk arrays on a Workstation Cluster, is analyzed by using analytical method and adtual values in experiments.N-PFS can be used as high-performance file sever in small-scale server systems and effciently pro-cess massive data I/Os such as multimedia and scientifid data. In this paper, an analytical model was suggested and the correctness of the suggested was verified by analyzing the experimental values on a system.The result of the appropriate stping unit for processing massive data of the Workstation Cluster with 8 disks is 64-128Kbytes and the maximum throughput on it is 15.8 Mbytes/ses.In addition, the performance of parallel file system on massive data is bounded by the time required to copy data between buffers.

  • PDF

A New Prediction-Based Parallel Event-Driven Logic Simulation (새로운 예측기반 병렬 이벤트구동 로직 시뮬레이션)

  • Yang, Seiyang
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.4 no.3
    • /
    • pp.85-90
    • /
    • 2015
  • In this paper, anew parallel event-driven logic simulation is proposed. As the proposed prediction-based parallel event-driven simulation method uses both prediction data and actual data for the input and output values of local simulations executed in parallel, the synchronization overhead and the communication overhead, the major bottleneck of the performance improvement, are greatly reduced. Through the experimentation with multiple designs, we have observed the effectiveness of the proposed approach.

Real-Time 3D Volume Deformation and Visualization by Integrating NeRF, PBD, and Parallel Resampling (NeRF, PBD 및 병렬 리샘플링을 결합한 실시간 3D 볼륨 변형체 시각화)

  • Sangmin Kwon;Sojin Jeon;Juni Park;Dasol Kim;Heewon Kye
    • Journal of the Korea Computer Graphics Society
    • /
    • v.30 no.3
    • /
    • pp.189-198
    • /
    • 2024
  • Research combining deep learning-based models and physical simulations is making important advances in the medical field. This extracts the necessary information from medical image data and enables fast and accurate prediction of deformation of the skeleton and soft tissue based on physical laws. This study proposes a system that integrates Neural Radiance Fields (NeRF), Position-Based Dynamics (PBD), and Parallel Resampling to generate 3D volume data, and deform and visualize them in real-time. NeRF uses 2D images and camera coordinates to produce high-resolution 3D volume data, while PBD enables real-time deformation and interaction through physics-based simulation. Parallel Resampling improves rendering efficiency by dividing the volume into tetrahedral meshes and utilizing GPU parallel processing. This system renders the deformed volume data using ray casting, leveraging GPU parallel processing for fast real-time visualization. Experimental results show that this system can generate and deform 3D data without expensive equipment, demonstrating potential applications in engineering, education, and medicine.

Design and Implementation of a TMN Agent Platform based on a Multi-thread Parallel Processing Architecture (멀티쓰레드 기반 병렬처리 구조를 이용한 TMN 에이젼트 플랫폼 설계 및 구현)

  • Kim, Seong-U;Kim, Yeong-Tak
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.5 no.6
    • /
    • pp.793-800
    • /
    • 1999
  • TMN Agent Platform은 망 요소의 운영상태와 자원들을 GDMO에 따라 관리객체(Managed Object : MO)로 모델링 하고, 자원들의 현재 상태를 유지하며, 관리자(Manager)로부터의 망 관리 기능 요구에 따라 조작된다. 그러므로, 에이전트의 성능향상은 전체적인 통신망 관리의 성능향상에 직접적인 영향을 미친다.본 논문에서는 TMN 에이전트의 기능요구 사항을 분석하고, 이를 토대로 성능향상을 위해 멀티스레드 기법을 사용하는 병렬 처리 구조의 TMN Agent Platform의 기능구조를 제시한다. 또한 에이전트와 다양한 자원들간의 효율적인 메시지전달을 위한 체계를 제시하며, 구현된 TMN Agent Platform의 성능을 분석한다.Abstract TMN Agent manages the operational status and real-resources of network elements, such as switching nodes and transmission systems. It performs the requested management functions from manager and maintains consistent status data of real-resource. The performance of agent system affects directly the performance of network management operation. If the agent is implemented by sequential processing scheme with single process, the agent processing can be delayed or blocked according to the status of real-resources. This problem can be solved by parallel and distributed processing scheme.To improve the processing performance of TMN Agent, we propose a TMN Agent Platform's functional architecture that is based on parallel processing with multi-tread and effective message transferring scheme between agent and various real-resource. We analyze the performance of the implemented TMN Agent Platform.

Architecture design for speeding up Multi-Access Memory System(MAMS) (Multi-Access Memory System(MAMS)의 속도 향상을 위한 아키텍처 설계)

  • Ko, Kyung-sik;Kim, Jae Hee;Lee, S-Ra-El;Park, Jong Won
    • Journal of the Institute of Electronics and Information Engineers
    • /
    • v.54 no.6
    • /
    • pp.55-64
    • /
    • 2017
  • High-capacity, high-definition image applications need to process considerable amounts of data at high speed. Accordingly, users of these applications demand a high-speed parallel execution system. To increase the speed of a parallel execution system, Park (2004) proposed a technique, called MAMS (Multi-Access Memory System), to access data in several execution units without the conflict of parallel processing memories. Since then, many studies on MAMS have been conducted, furthering the technique to MAMS-PP16 and MAMS-PP64, among others. As a memory architecture for parallel processing, MAMS must be constructed in one chip; therefore, a method to achieve the identical functionality as the existing MAMS while minimizing the architecture needs to be studied. This study proposes a method of miniaturizing the MAMS architecture in which the architectures of the ACR (Address Calculation and Routing) circuit and MMS (Memory Module Selection) circuit, which deliver data in memories to parallel execution units (PEs), do not use the MMS circuit, but are constructed as one shift and conditional statements whose number is the same as that of memory modules inside the ACR circuit. To verify the performance of the realized architecture, the study conducted the processing time of the proposed MAMS-PP64 through an image correlation test, the results of which demonstrated that the ratio of the image correlation from the proposed architecture was improved by 1.05 on average.

Sort-Based Distributed Parallel Data Cube Computation Algorithm using MapReduce (맵리듀스를 이용한 정렬 기반의 데이터 큐브 분산 병렬 계산 알고리즘)

  • Lee, Suan;Kim, Jinho
    • Journal of the Institute of Electronics and Information Engineers
    • /
    • v.49 no.9
    • /
    • pp.196-204
    • /
    • 2012
  • Recently, many applications perform OLAP(On-Line Analytical Processing) over a very large volume of data. Multidimensional data cube is regarded as a core tool in OLAP analysis. This paper focuses on the method how to efficiently compute data cubes in parallel by using a popular parallel processing tool, MapReduce. We investigate efficient ways to implement PipeSort algorithm, a well-known data cube computation method, on the MapReduce framework. The PipeSort executes several (descendant) cuboids at the same time as a pipeline by scanning one (ancestor) cuboid once, which have the same sorting order. This paper proposed four ways implementing the pipeline of the PipeSort on the MapReduce framework which runs across 20 servers. Our experiments show that PipeMap-NoReduce algorithm outperforms the rest algorithms for high-dimensional data. On the contrary, Post-Pipe stands out above the others for low-dimensional data.

Weather Radar Image Gener ation Method Using Inter polation based on CUDA

  • Yang, Liu;Jang, Bong-Joo;Lim, Sanghun;Kwon, Ki-Chang;Lee, Suk-Hwan;Kwon, Ki-Ryong
    • Journal of Korea Multimedia Society
    • /
    • v.18 no.4
    • /
    • pp.473-482
    • /
    • 2015
  • Doppler weather radar is an important tool for meteorological research. Through several decades of development, Doppler weather radar has enormous progress in understanding, detection and warning of meso and micro scale weather system. It makes a significant contribution to weather forecast and weather disaster warning. But the large amount of data process limits the application of Doppler weather radar. This paper proposed for fast weather radar data processing based on CUDA. CDUA is a powerful platform for highly parallel programming developed by NVIDIA. Through running plenty of threads, radar data can be calculated at same time. In experiment, CUDA parallel program can significantly improve weather data processing time.

Design and Performance Analysis of a Parallel Cell-Based Filtering Scheme using Horizontally-Partitioned Technique (수평 분할 방식을 이용한 병렬 셀-기반 필터링 기법의 설계 및 성능 평가)

  • Chang, Jae-Woo;Kim, Young-Chang
    • The KIPS Transactions:PartD
    • /
    • v.10D no.3
    • /
    • pp.459-470
    • /
    • 2003
  • It is required to research on high-dimensional index structures for efficiently retrieving high-dimensional data because an attribute vector in data warehousing and a feature vector in multimedia database have a characteristic of high-dimensional data. For this, many high-dimensional index structures have been proposed, but they have so called ‘dimensional curse’ problem that retrieval performance is extremely decreased as the dimensionality is increased. To solve the problem, the cell-based filtering (CBF) scheme has been proposed. But the CBF scheme show a linear decreasing on performance as the dimensionality. To cope with the problem, it is necessary to make use of parallel processing techniques. In this paper, we propose a parallel CBF scheme which uses a horizontally-partitioned technique as declustering. In order to maximize the retrieval performance of the proposed parallel CBF scheme, we construct our parallel CBF scheme under a SN (Shared Nothing) cluster architecture. In addition, we present a data insertion algorithm, a rage query processing one, and a k-NN query processing one which are suitable for the SN cluster architecture. Finally, we show that our parallel CBF scheme achieves better retrieval performance in proportion to the number of servers in the SN cluster architecture, compared with the conventional CBF scheme.

A Design of Discrete Wavelet Transform Encoder for Multimedia Image Signal Processing (멀티미디어 영상신호 처리를 위한 DWT 부호화기 설계)

  • 이강현
    • Proceedings of the IEEK Conference
    • /
    • 2003.07d
    • /
    • pp.1685-1688
    • /
    • 2003
  • The modem multimedia applications which are video Processor, video conference or video phone and so forth require real time processing. Because of a large amount of image data, those require high compression performance. In this paper, the proposed image processing encoder was designed by using wavelet transform encoding. The proposed filter block can process image data on tile high speed because of composing individual function blocks by parallel and compute both highpass and lowpass coefficient in the same clock cycle. When image data is decomposed into multiresolution, the proposed scheme needs external memory and controller to save intermediate results and it can operate within 33㎒.

  • PDF

Real-time and Parallel Semantic Translation Technique for Large-Scale Streaming Sensor Data in an IoT Environment (사물인터넷 환경에서 대용량 스트리밍 센서데이터의 실시간·병렬 시맨틱 변환 기법)

  • Kwon, SoonHyun;Park, Dongwan;Bang, Hyochan;Park, Youngtack
    • Journal of KIISE
    • /
    • v.42 no.1
    • /
    • pp.54-67
    • /
    • 2015
  • Nowadays, studies on the fusion of Semantic Web technologies are being carried out to promote the interoperability and value of sensor data in an IoT environment. To accomplish this, the semantic translation of sensor data is essential for convergence with service domain knowledge. The existing semantic translation technique, however, involves translating from static metadata into semantic data(RDF), and cannot properly process real-time and large-scale features in an IoT environment. Therefore, in this paper, we propose a technique for translating large-scale streaming sensor data generated in an IoT environment into semantic data, using real-time and parallel processing. In this technique, we define rules for semantic translation and store them in the semantic repository. The sensor data is translated in real-time with parallel processing using these pre-defined rules and an ontology-based semantic model. To improve the performance, we use the Apache Storm, a real-time big data analysis framework for parallel processing. The proposed technique was subjected to performance testing with the AWS observation data of the Meteorological Administration, which are large-scale streaming sensor data for demonstration purposes.