• Title/Summary/Keyword: Parallel data processing

Search Result 751, Processing Time 0.043 seconds

Design of an Expandable VLSI Rebound Sorter (확장형 VLSI 리바운드 정렬기의 설계)

  • Yun, Ji-Heon;Ahn, Byoung-Chul
    • The Transactions of the Korea Information Processing Society
    • /
    • v.2 no.3
    • /
    • pp.433-442
    • /
    • 1995
  • This paper presents an improved VLSI implementation of a parallel sorter to achieve O(Ν) time complexity. Many fast VLSI sort algorithms have been proposed for sorting N elements in O(log Ν) time. However, most such algorithms proposed have complex network structure without considering data input and output time. They are also very difficult to expand or to use in real applications. After analyzing the chip area and time complexity of several parallel sort algorithms with overlapping data input and output time, the most effective algorithm, the rebound sort algorithm, is implemented in VLSI with some improvements. To achieve O(Ν) time complexity, an improved rebound sorter is able to sort 8 16-bits records on a chip. And it is possible to sort more than 8 records by connecting chips in a chain vertically.

  • PDF

Sim-Hadoop : Leveraging Hadoop Distributed File System and Parallel I/O for Reliable and Efficient N-body Simulations (Sim-Hadoop : 신뢰성 있고 효율적인 N-body 시뮬레이션을 위한 Hadoop 분산 파일 시스템과 병렬 I / O)

  • Awan, Ammar Ahmad;Lee, Sungyoung;Chung, Tae Choong
    • Annual Conference of KIPS
    • /
    • 2013.05a
    • /
    • pp.476-477
    • /
    • 2013
  • Gadget-2 is a scientific simulation code has been used for many different types of simulations like, Colliding Galaxies, Cluster Formation and the popular Millennium Simulation. The code is parallelized with Message Passing Interface (MPI) and is written in C language. There is also a Java adaptation of the original code written using MPJ Express called Java Gadget. Java Gadget writes a lot of checkpoint data which may or may not use the HDF-5 file format. Since, HDF-5 is MPI-IO compliant, we can use our MPJ-IO library to perform parallel reading and writing of the checkpoint files and improve I/O performance. Additionally, to add reliability to the code execution, we propose the usage of Hadoop Distributed File System (HDFS) for writing the intermediate (checkpoint files) and final data (output files). The current code writes and reads the input, output and checkpoint files sequentially which can easily become bottleneck for large scale simulations. In this paper, we propose Sim-Hadoop, a framework to leverage HDFS and MPJ-IO for improving the I/O performance of Java Gadget code.

Radar Signal Processor Design Using FPGA (FPGA를 이용한 레이더 신호처리 설계)

  • Ha, Changhun;Kwon, Bojun;Lee, Mangyu
    • Journal of the Korea Institute of Military Science and Technology
    • /
    • v.20 no.4
    • /
    • pp.482-490
    • /
    • 2017
  • The radar signal processing procedure is divided into the pre-processing such as frequency down converting, down sampling, pulse compression, and etc, and the post-processing such as doppler filtering, extracting target information, detecting, tracking, and etc. The former is generally designed using FPGA because the procedure is relatively simple even though there are large amounts of ADC data to organize very quickly. On the other hand, in general, the latter is parallel processed by multiple DSPs because of complexity, flexibility and real-time processing. This paper presents the radar signal processor design using FPGA which includes not only the pre-processing but also the post-processing such as doppler filtering, bore-sight error, NCI(Non-Coherent Integration), CFAR(Constant False Alarm Rate) and etc.

Interoperability between NoSQL and RDBMS via Auto-mapping Scheme in Distributed Parallel Processing Environment (분산병렬처리 환경에서 오토매핑 기법을 통한 NoSQL과 RDBMS와의 연동)

  • Kim, Hee Sung;Lee, Bong Hwan
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.21 no.11
    • /
    • pp.2067-2075
    • /
    • 2017
  • Lately big data processing is considered as an emerging issue. As a huge amount of data is generated, data processing capability is getting important. In processing big data, both Hadoop distributed file system and unstructured date processing-based NoSQL data store are getting a lot of attention. However, there still exists problems and inconvenience to use NoSQL. In case of low volume data, MapReduce of NoSQL normally consumes unnecessary processing time and requires relatively much more data retrieval time than RDBMS. In order to address the NoSQL problem, in this paper, an interworking scheme between NoSQL and the conventional RDBMS is proposed. The developed auto-mapping scheme enables to choose an appropriate database (NoSQL or RDBMS) depending on the amount of data, which results in fast search time. The experimental results for a specific data set shows that the database interworking scheme reduces data searching time by 35% at the maximum.

Seismic Data Processing For Gas Hydrate using Geobit (Geobit을 이용한 가스 하이드레이트 탐사자료 처리)

  • Jang Seong-Hyung;Suh Sang-Yong;Chung Bu-Heung;Ryu Byung-Jae
    • Geophysics and Geophysical Exploration
    • /
    • v.2 no.4
    • /
    • pp.184-190
    • /
    • 1999
  • A study of gas hydrate is a worldwide popular interesting subject as a potential energy source. A seismic survey for gas hydrate have performed over the East sea by the KIGAM since 1997. General indicators of natural submarine gas hydrates in seismic data is commonly inferred from the BSR (Bottom Simulating Reflection) that occurred parallel to the see floor, amplitude decrease at the top of the BSR, amplitude Blanking at the bottom of the BSR, decrease of the interval velocity, and the reflection phase reversal at the BSR. So the seismic data processing for detecting gas hydrates indicators is required the true amplitude recovery processing, a accurate velocity analysis and the AVO (Amplitude Variation with Offset) analysis. In this paper, we had processed the field data to detect the gas hydrate indicators, which had been acquired over the East sea in 1998. Applied processing modules are spherical divergence, band pass filtering, CDP sorting and accurate velocity analysis. The AVO analysis was excluded, since this field data had too short offset to apply the AVO analysis. The accurate velocity analysis was performed by XVA (X-window based Velocity Analysis). This is the method which calculate the velocity spectrum by iterative and interactive. With XVA, we could determine accurate stacking velocity. Geobit 2.9.5 developed by the KIGAM was used for processing data. Processing results say that the BSR occurred parallel to the sea floor were shown at $367\~477m$ depths (two way travel time about 1800 ms) from the sea floor through shot point 1650-1900, the interval velocity decrease around BSR and the reflection phase reversal corresponding to the reflection at the sea floor.

  • PDF

Implementation of Multicore-Aware Load Balancing on Clusters through Data Distribution in Chapel (클러스터 상에서 다중 코어 인지 부하 균등화를 위한 Chapel 데이터 분산 구현)

  • Gu, Bon-Gen;Carpenter, Patrick;Yu, Weikuan
    • The KIPS Transactions:PartA
    • /
    • v.19A no.3
    • /
    • pp.129-138
    • /
    • 2012
  • In distributed memory architectures like clusters, each node stores a portion of data. How data is distributed across nodes influences the performance of such systems. The data distribution scheme is the strategy to distribute data across nodes and realize parallel data processing. Due to various reasons such as maintenance, scale up, upgrade, etc., the performance of nodes in a cluster can often become non-identical. In such clusters, data distribution without considering performance cannot efficiently distribute data on nodes. In this paper, we propose a new data distribution scheme based on the number of cores in nodes. We use the number of cores as the performance factor. In our data distribution scheme, each node is allocated an amount of data proportional to the number of cores in it. We implement our data distribution scheme using the Chapel language. To show our data distribution is effective in reducing the execution time of parallel applications, we implement Mandelbrot Set and ${\pi}$-Calculation programs with our data distribution scheme, and compare the execution times on a cluster. Based on experimental results on clusters of 8-core and 16-core nodes, we demonstrate that data distribution based on the number of cores can contribute to a reduction in the execution times of parallel programs on clusters.

Thermal Imager Implementation Using Infrared Sensor (적외선 센서를 이용한 열상장비의 구현)

  • Yu, W.K.;Yoon, E.S.;Kim, C.W.;Song, I.S.;Hong, S.M.
    • Proceedings of the KIEE Conference
    • /
    • 1992.07b
    • /
    • pp.1250-1254
    • /
    • 1992
  • This paper describes the designed and fabricated thermal imaging system with the SPRITE(Signal PRocessing in The Element) detector, operating in the 3-12 micron band. This system consists of an afocal telescope, a scan unit containing the SPRITE detector, an electronic processor unit and a cooler. The optical scan system utilizing rotating polygon and oscillating mirror, is 2-dimensional serial/parallel scan type using five elements of the detector. And the electronic processor unit performs digital scan conversion to reform the parallel data stream into serial analog data compatable with conventional RS-170 video. The scan field of view is 40 ${\times}$ 26.7 and the MRTD(Minium Resolvable Temperature Difference) is 0.6 K at 7.5 cycles/mm. The acquired thermal image indicates that this system has a satisfactory performance.

  • PDF

Design and Implementation of High-Resolution Integral Imaging Display System using Expanded Depth Image

  • Song, Min-Ho;Lim, Byung-Muk;Ryu, Ga-A;Ha, Jong-Sung;Yoo, Kwan-Hee
    • International Journal of Contents
    • /
    • v.14 no.3
    • /
    • pp.1-6
    • /
    • 2018
  • For 3D display applications, auto-stereoscopic display methods that can provide 3D images without glasses have been actively developed. This paper is concerned with developing a display system for elemental images of real space using integral imaging. Unlike the conventional method, which reduces a color image to the level as much as a generated depth image does, we have minimized original color image data loss by generating an enlarged depth image with interpolation methods. Our method was efficiently implemented by applying a GPU parallel processing technique with OpenCL to rapidly generate a large amount of elemental image data. We also obtained experimental results for displaying higher quality integral imaging rather than one generated by previous methods.

Development of 64-Channel 12-bit 1ks/s Hardware for MCG Signal Acquisition (심자도 신호 획득을 위한 실시간 64-Ch 12-bit 1ks/s 하드웨어 개발)

  • Lee, Dong-Ha;Yoo, Jae-Tack
    • Proceedings of the Korean Institute of Electrical and Electronic Material Engineers Conference
    • /
    • 2004.07b
    • /
    • pp.902-905
    • /
    • 2004
  • A heart diagnosis system adopts Superconducting Quantum Interface Device(SQUID) sensors for precision MCG signal acquisitions. Such system is composed of hundreds of sensors, requiring fast signal sampling and precise analog-digital conversions(ADC). Our development of hardware board, processing 64-channel 12-bit 1ks/s, is built by using 8-channel ADC chips, 8-bit microprocessors, SPI interfaces, and parallel data transfers between microprocessors to meet the 1ks/s, i.e. 1 ms speed. The test result shows that the signal acquisition is done in 168 usuc which is much shorter than the required 1 ms period. This hardware will be extended to 256 channel data acquisition to be used for the diagnosis system.

  • PDF

Efficiency Enhancement of CFDS Code (CFDS 코드의 효율성 개선)

  • Kim J. G.;Lee J.;Kim C.;Hong S. K.;Lee K. S.;Ahn C. S.
    • 한국전산유체공학회:학술대회논문집
    • /
    • 2005.04a
    • /
    • pp.123-127
    • /
    • 2005
  • The numerical analyses of the complicated flows are widely attempted in these days. Because of the enormous demanding memory and calculation time, parallel processing is used for these problems. In order to obtain calculation efficiency, it is important to choose proper domain decomposition technique and numerical algorithm. In this research we enhanced the efficiency of the CFDS code developed by ADD, using parallel computation and newly developed numerical algorithms. For the huge amount of data transfer between blocks non-blocking method is used, and newly developed data transfer algorithm is used for non-aligned block interface. Recently developed RoeM scheme is adpoted as a spatial difference method, and AF-ADI and LU-SGS methods are used as a time integration method to enhance the convergence of the code. Analyses of the flows around the ONERA M6 wing and the high angle of attack missile configuration are performed to show the efficiency improvement.

  • PDF