• Title/Summary/Keyword: Parallel data processing

Search Result 751, Processing Time 0.028 seconds

A Pipelined Parallel Optimized Design for Convolution-based Non-Cascaded Architecture of JPEG2000 DWT (JPEG2000 이산웨이블릿변환의 컨볼루션기반 non-cascaded 아키텍처를 위한 pipelined parallel 최적화 설계)

  • Lee, Seung-Kwon;Kong, Jin-Hyeung
    • Journal of the Institute of Electronics Engineers of Korea SD
    • /
    • v.46 no.7
    • /
    • pp.29-38
    • /
    • 2009
  • In this paper, a high performance pipelined computing design of parallel multiplier-temporal buffer-parallel accumulator is present for the convolution-based non-cascaded architecture aiming at the real time Discrete Wavelet Transform(DWT) processing. The convolved multiplication of DWT would be reduced upto 1/4 by utilizing the filter coefficients symmetry and the up/down sampling; and it could be dealt with 3-5 times faster computation by LUT-based DA multiplication of multiple filter coefficients parallelized for product terms with an image data. Further, the reutilization of computed product terms could be achieved by storing in the temporal buffer, which yields the saving of computation as well as dynamic power by 50%. The convolved product terms of image data and filter coefficients are realigned and stored in the temporal buffer for the accumulated addition. Then, the buffer management of parallel aligned storage is carried out for the high speed sequential retrieval of parallel accumulations. The convolved computation is pipelined with parallel multiplier-temporal buffer-parallel accumulation in which the parallelization of temporal buffer and accumulator is optimize, with respect to the performance of parallel DA multiplier, to improve the pipelining performance. The proposed architecture is back-end designed with 0.18um library, which verifies the 30fps throughput of SVGA(800$\times$600) images at 90MHz.

Fast Multi-View Synthesis Using Duplex Foward Mapping and Parallel Processing (순차적 이중 전방 사상의 병렬 처리를 통한 다중 시점 고속 영상 합성)

  • Choi, Ji-Youn;Ryu, Sae-Woon;Shin, Hong-Chang;Park, Jong-Il
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.34 no.11B
    • /
    • pp.1303-1310
    • /
    • 2009
  • Glassless 3D display requires multiple images taken from different viewpoints to show a scene. The simplest way to get multi-view image is using multiple camera that as number of views are requires. To do that, synchronize between cameras or compute and transmit lots of data comes critical problem. Thus, generating such a large number of viewpoint images effectively is emerging as a key technique in 3D video technology. Image-based view synthesis is an algorithm for generating various virtual viewpoint images using a limited number of views and depth maps. In this paper, because the virtual view image can be express as a transformed image from real view with some depth condition, we propose an algorithm to compute multi-view synthesis from two reference view images and their own depth-map by stepwise duplex forward mapping. And also, because the geometrical relationship between real view and virtual view is repetitively, we apply our algorithm into OpenGL Shading Language which is a programmable Graphic Process Unit that allow parallel processing to improve computation time. We demonstrate the effectiveness of our algorithm for fast view synthesis through a variety of experiments with real data.

Developing an Intrusion Detection Framework for High-Speed Big Data Networks: A Comprehensive Approach

  • Siddique, Kamran;Akhtar, Zahid;Khan, Muhammad Ashfaq;Jung, Yong-Hwan;Kim, Yangwoo
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.12 no.8
    • /
    • pp.4021-4037
    • /
    • 2018
  • In network intrusion detection research, two characteristics are generally considered vital to building efficient intrusion detection systems (IDSs): an optimal feature selection technique and robust classification schemes. However, the emergence of sophisticated network attacks and the advent of big data concepts in intrusion detection domains require two more significant aspects to be addressed: employing an appropriate big data computing framework and utilizing a contemporary dataset to deal with ongoing advancements. As such, we present a comprehensive approach to building an efficient IDS with the aim of strengthening academic anomaly detection research in real-world operational environments. The proposed system has the following four characteristics: (i) it performs optimal feature selection using information gain and branch-and-bound algorithms; (ii) it employs machine learning techniques for classification, namely, Logistic Regression, Naïve Bayes, and Random Forest; (iii) it introduces bulk synchronous parallel processing to handle the computational requirements of large-scale networks; and (iv) it utilizes a real-time contemporary dataset generated by the Information Security Centre of Excellence at the University of Brunswick (ISCX-UNB) to validate its efficacy. Experimental analysis shows the effectiveness of the proposed framework, which is able to achieve high accuracy, low computational cost, and reduced false alarms.

An Efficient Distributed Shared Memory System for Parallel GIS (병렬 GIS를 위한 효율적인 분산공유메모리 시스템)

  • Jeong, Sang-Hwa;Ryu, Gwang-Yeol;Go, Yun-Yeong;Gwak, Min-Seok
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.5 no.6
    • /
    • pp.700-707
    • /
    • 1999
  • 본 논문에서는 GIS 관련 연산을 실시간에 효율적으로 처리하기 위한 분산공유메모리 기반 병렬처리 시스템을 제안한다. 본 논문의 분산공유메모리 시스템은 메시지전달 방식의 분산메모리 MIMD 컴퓨터 상에 소프트웨어 기반 분산공유메모리 모듈을 탑재함으로써 구현되었다. 또한 GIS 연산의 기본이 되는 공간 객체를 공유의 기본 단위로 설정하고, GIS 데이타의 특성을 반영하여 읽기전용 공유데이타 타입을 추가하였으며, 네트워크 오버헤드를 줄이기 위하여 복수의 객체를 한번에 읽어오는 bulk access가 가능하도록 하였다. 본 시스템에서는 GIS 데이타의 효율적인 분배를 위하여 부하균등화 기법으로 guided self scheduling을 사용하였다. 실험결과 본 시스템은 네트워크 캐쉬의 효율적인 활용을 통하여 소프트웨어 기반 분산메모리 시스템의 오버헤드에도 불구하고 MPI 기반 메시지전달 방식에 비하여 향상된 성능을 얻을 수 있었다.Abstract In this paper, we propose a distributed shared memory(DSM) based parallel processing system to process GIS related computations efficiently in real time. The system is based on a software DSM module implemented on top of a distributed MIMD computer. In the DSM system, spatial object, which is a fundamental structure to represent GIS data, is used as a basic unit for sharing, and a read-only shared data type is added to reflect the characteristics of GIS data. In addition, a bulk access to multiple shared data is made possible to reduce the network overhead. A guided self scheduling method is devised for efficient load balancing in distributing GIS data to parallel processors. The experimental results show that the DSM system performs better than an MPI based message-passing system through the efficient utilization of network cache in spite of the system's software overhead.

A Real-Time Image Processing Algorithms for An Automatic Assembly System of Electronic Components (전자부품 조립공정의 자동화를 \ulcorner나 실시간 영상처리 알고리즘에 관한 연구)

  • ;;;Zeungnam Bien
    • The Transactions of the Korean Institute of Electrical Engineers
    • /
    • v.37 no.11
    • /
    • pp.804-815
    • /
    • 1988
  • Real-time image processing algorithms to detect position and orientation of rectangular type electronic components are developed. The position detection algorithm is implemented with the use of projection method which is insensitive to noise. Also dynamic thresholding method of projection is employed in order to distinguish between the boundary of a component and any marking on the component. The orientation is determined by Hough transform of boundary candidates of a component, which is obtained a priori by a simple edge detection method. For real-time processing of both position and orientation for a component which is not aligned well, parallel processing method of image data is proposed and tested in real-time.

  • PDF

The Parallel Processing Method of the Continuous Data Stream Processing Tasks (데이터 스트림 연속 처리 태스크의 병렬 처리 방법)

  • Yang, Kyungah;Lee, Daewoo;Kim, Kiheon
    • Annual Conference of KIPS
    • /
    • 2015.10a
    • /
    • pp.214-215
    • /
    • 2015
  • 데이터 스트림 환경에서 연속 질의를 처리하기 위한 데이터 스트림 처리 시스템이 개발되었다. 데이터 스트림 처리 시스템에서 질의를 처리하는 태스크에 과도한 데이터가 발생할 경우 일반적으로 데이터 스트림을 선별적으로 버리는 load shedding 방법을 이용하지만 이러한 방법은 처리 결과의 정확도가 저하될 수 있다. 따라서, 본 논문은 이를 해결하는 방법으로 분산 데이터 스트림 처리 시스템에서 데이터 스트림 분할을 통한 데이터 스트림 연속 처리 태스크의 병렬 처리 방법을 제시한다. 이를 위해 분산 데이터 스트림을 처리하기 위한 기준을 제시 및 데이터 분할 방법에 대해서 언급한다.

Enhancing the performance of taxi application based on in-memory data grid technology (In-memory data grid 기술을 활용한 택시 애플리케이션 성능 향상 기법 연구)

  • Choi, Chi-Hwan;Kim, Jin-Hyuk;Park, Min-Kyu;Kwon, Kaaen;Jung, Seung-Hyun;Nazareno, Franco;Cho, Wan-Sup
    • Journal of the Korean Data and Information Science Society
    • /
    • v.26 no.5
    • /
    • pp.1035-1045
    • /
    • 2015
  • Recent studies in Big Data Analysis are showing promising results, utilizing the main memory for rapid data processing. In-memory computing technology can be highly advantageous when used with high-performing servers having tens of gigabytes of RAM with multi-core processors. The constraint in network in these infrastructure can be lessen by combining in-memory technology with distributed parallel processing. This paper discusses the research in the aforementioned concept applying to a test taxi hailing application without disregard to its underlying RDBMS structure. The application of IMDG technology in the application's backend API without restructuring the database schema yields 6 to 9 times increase in performance in data processing and throughput. Specifically, the change in throughput is very small even with increase in data load processing.

Semantic Depth Data Transmission Reduction Techniques using Frame-to-Frame Masking Method for Light-weighted LiDAR Signal Processing Platform (LiDAR 신호처리 플랫폼을 위한 프레임 간 마스킹 기법 기반 유효 데이터 전송량 경량화 기법)

  • Chong, Taewon;Park, Daejin
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.25 no.12
    • /
    • pp.1859-1867
    • /
    • 2021
  • Multi LiDAR sensors are being mounted on autonomous vehicles, and a system to multi LiDAR sensors data is required. When sensors data is transmitted or processed to the main processor, a huge amount of data causes a load on the transport network or data processing. In order to minimize the number of load overhead into LiDAR sensor processors, only semantic data is transmitted through data comparison between frames in LiDAR data. When data from 4 LiDAR sensors are processed in a static environment without moving objects and a dynamic environment in which a person moves within sensor's field of view, in a static experiment environment, the transmitted data reduced by 89.5% from 232,104 to 26,110 bytes. In dynamic environment, it was possible to reduce the transmitted data by 88.1% to 29,179 bytes.

Design of Lightweight Artificial Intelligence System for Multimodal Signal Processing (멀티모달 신호처리를 위한 경량 인공지능 시스템 설계)

  • Kim, Byung-Soo;Lee, Jea-Hack;Hwang, Tae-Ho;Kim, Dong-Sun
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.13 no.5
    • /
    • pp.1037-1042
    • /
    • 2018
  • The neuromorphic technology has been researched for decades, which learns and processes the information by imitating the human brain. The hardware implementations of neuromorphic systems are configured with highly parallel processing structures and a number of simple computational units. It can achieve high processing speed, low power consumption, and low hardware complexity. Recently, the interests of the neuromorphic technology for low power and small embedded systems have been increasing rapidly. To implement low-complexity hardware, it is necessary to reduce input data dimension without accuracy loss. This paper proposed a low-complexity artificial intelligent engine which consists of parallel neuron engines and a feature extractor. A artificial intelligent engine has a number of neuron engines and its controller to process multimodal sensor data. We verified the performance of the proposed neuron engine including the designed artificial intelligent engines, the feature extractor, and a Micro Controller Unit(MCU).

Performance Evaluation of the GPU Architecture Executing Parallel Applications (병렬 응용프로그램 실행 시 GPU 구조에 따른 성능 분석)

  • Choi, Hong-Jun;Kim, Cheol-Hong
    • The Journal of the Korea Contents Association
    • /
    • v.12 no.5
    • /
    • pp.10-21
    • /
    • 2012
  • The role of GPU has evolved from graphics-specific processing to general-purpose processing with the development of unified shader core architecture. Especially, execution methods for general-purpose parallel applications using GPU have been researched intensively, since the parallel hardware architecture can be utilized efficiently when the parallel applications are executed. However, current GPU architecture has limitations in executing general-purpose parallel applications, since the GPU is not specialized for general-purpose computing yet. To improve the GPU performance when general-purpose parallel applications are executed, the GPU architecture should be evolved. In this work, we analyze the GPU performance according to the architecture varying the number of cores and clock frequency. Our simulation results show that the GPU performance improves by up to 125.8% and 16.2% as the number of cores increases and the clock frequency increases, respectively. However, note that the improvement of the GPU performance is saturated even though the number of cores increases and the clock frequency increases continuously, since the data cannot be provided to the GPU due to the limit of memory bandwidth. Consequently, to accomplish high performance effectiveness on GPU, computational resources must be more suitably considered.