• Title/Summary/Keyword: Parallel data processing

Search Result 751, Processing Time 0.025 seconds

A Parallel Match Method for Path-oriented Query Processing in iW- Databases (XML 데이타베이스에서 경로-지향 질의처리를 위한 병렬 매치 방법)

  • Park Hee-Sook;Cho Woo-Hyun
    • Journal of KIISE:Databases
    • /
    • v.32 no.5
    • /
    • pp.558-566
    • /
    • 2005
  • The XML is the new standard fir data representation and exchange on the Internet. In this paper, we describe a new approach for evaluating a path-oriented query against XML document. In our approach, we propose the Parallel Match Indexing Fabric to speed up evaluation of path-oriented query using path signature and design the parallel match algorithm to perform a match process between a path signature of input query and path signatures of elements stored in the database. To construct a structure of the parallel match indexing, we first make the binary tie for all path signatures on an XML document and then which trie is transformed to the Parallel Match Indexing Fabric. Also we use the Parallel Match Indexing Fabric and a parallel match algorithm for executing a search operation of a path-oriented query. In our proposed approach, Time complexity of the algorithm is proportional to the logarithm of the number of path signatures in the XML document.

GPGPU Acceleration of SAT Algorithm with Propagation Routine Parallelization (전달 루틴의 병렬화를 통한 SAT 알고리즘의 GPGPU 가속화)

  • Kang, Hyeong-Ju
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.20 no.10
    • /
    • pp.1919-1926
    • /
    • 2016
  • Because of the enormous processing ability, General-Purpose Graphics Processing Unit(GPGPU) has been applied to many fields including electronics design automation. The SAT algorithm is one of the core algorithm in many electronics design automation tools. There has been some efforts to apply GPGPU to the SAT algorithm, but it is difficult to parallelize the SAT algorithm because of its characteristics. In this paper, I applied GPGPU to the SAT algorithm by parallelizing the propagation routine that is relatively suitable to parallel processing. On the basis of the similarity of the propagation routine to the sparse matrix multiplication, the data structure for the SAT problem is constituted, and the parallel propagation routine is described. To prevent data loss between paralllel threads, atomic operations are exploited. The experimental results for some benchmark SAT problems show that the proposed algorithm is superior to the previous GPGPU-based SAT solver.

Correction for SPECT image distortion by non-circular detection orbits (비원형 궤도에서의 검출에 의한 SPECT 영상 왜곡 보정)

  • Lee, Nam-Yong
    • Journal of the Institute of Convergence Signal Processing
    • /
    • v.8 no.3
    • /
    • pp.156-162
    • /
    • 2007
  • The parallel beam SPECT system acquires projection data by using collimators in conjunction with photon detectors. The projection data of the parallel beam SPECT system is, however, blurred by the point response function of the collimator that is used to define the range of directions where photons can be detected. By increasing the number of parallel holes per unit area in collimator, one can reduce such blurring effect. This approach also, however, has the blurring problem if the distance between the object and the collimator becomes large. In this paper we consider correction methods for artifacts caused by non-circular orbit of parallel beam SPECT with many parallel holes per detector cell. To do so, we model the relationship between the object and its projection data as a linear system, and propose an iterative reconstruction method including artifacts correction. We compute the projector and the backprojector, which are required in iterative method, as a sum of convolutions with distance-dependent point response functions instead of matrix form, where those functions are analytically computed from a single function. By doing so, we dramatically reduce the computation time and memory required for the generation of the projector and the backprojector. We conducted several simulation studies to compare the performance of the proposed method with that of conventional Fourier method. The result shows that the proposed method outperforms Fourier methods objectively and subjectively.

  • PDF

Unmanned Aircraft Platform Based Real-time LiDAR Data Processing Architecture for Real-time Detection Information (실시간 탐지정보 제공을 위한 무인기 플랫폼 기반 실시간 LiDAR 데이터 처리구조)

  • Eum, Junho;Berhanu, Eyassu;Oh, Sangyoon
    • KIISE Transactions on Computing Practices
    • /
    • v.21 no.12
    • /
    • pp.745-750
    • /
    • 2015
  • LiDAR(Light Detection and Ranging) technology provides realistic 3-dimension image information, and it has been widely utilized in various fields. However, the utilization of this technology in the military domain requires prompt responses to dynamically changing tactical environment and is therefore limited since LiDAR technology requires complex processing in order for extensive amounts of LiDAR data to be utilized. In this paper, we introduce an Unmanned Aircraft Platform Based Real-time LiDAR Data Processing Architecture that can provide real-time detection information by parallel processing and off-loading between the UAV processing and high-performance data processing areas. We also conducted experiments to verify the feasibility of our proposed architecture. Processing with ARM cluster similar to the UAV platform processing area results in similar or better performance when compared to the current method. We determined that our proposed architecture can be utilized in the military domain for tactical and combat purposes such as unmanned monitoring system.

Design of Spark SQL Based Framework for Advanced Analytics (Spark SQL 기반 고도 분석 지원 프레임워크 설계)

  • Chung, Jaehwa
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.5 no.10
    • /
    • pp.477-482
    • /
    • 2016
  • As being the advanced analytics indispensable on big data for agile decision-making and tactical planning in enterprises, distributed processing platforms, such as Hadoop and Spark which distribute and handle the large volume of data on multiple nodes, receive great attention in the field. In Spark platform stack, Spark SQL unveiled recently to make Spark able to support distributed processing framework based on SQL. However, Spark SQL cannot effectively handle advanced analytics that involves machine learning and graph processing in terms of iterative tasks and task allocations. Motivated by these issues, this paper proposes the design of SQL-based big data optimal processing engine and processing framework to support advanced analytics in Spark environments. Big data optimal processing engines copes with complex SQL queries that involves multiple parameters and join, aggregation and sorting operations in distributed/parallel manner and the proposing framework optimizes machine learning process in terms of relational operations.

Web-Based Distributed Visualization System for Large Scale Geographic Data (대용량 지형 데이터를 위한 웹 기반 분산 가시화 시스템)

  • Hwang, Gyu-Hyun;Yun, Seong-Min;Park, Sang-Hun
    • Journal of Korea Multimedia Society
    • /
    • v.14 no.6
    • /
    • pp.835-848
    • /
    • 2011
  • In this paper, we propose a client server based distributed/parallel system to effectively visualize huge geographic data. The system consists of a web-based client GUI program and a distributed/parallel server program which runs on multiple PC clusters. To make the client program run on mobile devices as well as PCs, the graphical user interface has been designed by using JOGL, the java-based OpenGL graphics library, and sending the information about current available memory space and maximum display resolution the server can minimize the amount of tasks. PC clusters used to play the role of the server access requested geographic data from distributed disks, and properly re-sample them, then send the results back to the client. To minimize the latency happened in repeatedly access the distributed stored geography data, cache data structures have been maintained in both every nodes of the server and the client.

An Analytical Model for Performance Prediction of AES on GPU Architecture (GPU 아키텍처의 AES 암호화 성능 예측 분석 모델)

  • Kim, Kyuwoon;Kim, Hyunwoo;Kim, Huijeong;Huh, Taeyoung;Jung, Sanghyuk;Song, Yong Ho
    • Journal of the Institute of Electronics and Information Engineers
    • /
    • v.50 no.4
    • /
    • pp.89-96
    • /
    • 2013
  • The graphic processor unit (GPU) has been developed to process not only graphic data but also general system data. It shows a better performance than CPU in algorithm for 3D graphics and parallel program. In order to execute algorithm for CPU on GPU, we should understand about GPU architectures and rewrite program considering parallel processing capability and new memory model of GPU. For this reasons, a performance prediction model for the algorithm and its predicted performance through GPU system are required. These can predict problems in GPU application development or construct a performance evaluation standard for GPU. In this paper, we applied the AES encryption algorithms on our performance model and accomplished performance prediction with high accuracy under a heavy workload.

VIA-Based PC Cluster System for Efficient Information Retrieval (효율적인 정보 검색을 위한 VIA 기반 PC 클러스터 시스템)

  • Kang, Na-Young;Chung, Sang-Hwa;Jang, Han-Kook
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.29 no.10
    • /
    • pp.539-549
    • /
    • 2002
  • PC cluster-based Information Retrieval (IR) systems improve their performances by parallel processing of query terms using cluster nodes. However TCP/IP based communication used to exchange data between cluster nodes prevents the performance from being improved further. The user-level communication mechanisms solve the problem by eliminating the time-consuming kernel access in exchanging data between cluster nodes. The Virtual Interface Architecture (VIA) is one of the representative user-level communication mechanisms which provide low latency and high bandwidth. In this paper, we propose a VIA-based parallel IR system on a PC cluster. The IR system is implemented using the following three communication methods: Sealable Coherent Interface (SCI) based VIA, MPI on SCI based VIA, MPI on Fast Ethernet based VIA. Through experiments, the performances of the three methods are analyzed in various aspects.

FDANT-PCSV: Fast Detection of Abnormal Network Traffic Using Parallel Coordinates and Sankey Visualization (FDANT-PCSV: Parallel Coordinates 및 Sankey 시각화를 이용한 신속한 이상 트래픽 탐지)

  • Han, Ki hun;Kim, Huy Kang
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.30 no.4
    • /
    • pp.693-704
    • /
    • 2020
  • As a company's network structure is getting bigger and the number of security system is increasing, it is not easy to quickly detect abnormal traffic from huge amounts of security system events. In this paper, We propose traffic visualization analysis system(FDANT-PCSV) that can detect and analyze security events of information security systems such as firewalls in real time. FDANT-PCSV consists of Parallel Coordinates visualization using five factors(source IP, destination IP, destination port, packet length, processing status) and Sankey visualization using four factors(source IP, destination IP, number of events, data size) among security events. In addition, the use of big data-based SIEM enables real-time detection of network attacks and network failure traffic from the internet and intranet. FDANT-PCSV enables cyber security officers and network administrators to quickly and easily detect network abnormal traffic and respond quickly to network threats.

Join Operation of Parallel Database System with Large Main Memory (대용량 메모리를 가진 병렬 데이터베이스 시스템의 조인 연산)

  • Park, Young-Kyu
    • Journal of the Korea Society of Computer and Information
    • /
    • v.12 no.3
    • /
    • pp.51-58
    • /
    • 2007
  • The shared-nothing multiprocessor architecture has advantages in scalability, this architecture has been adopted in many multiprocessor database system. But, if the data are not uniformly distributed across the processors, load will be unbalanced. Therefore, the whole system performance will deteriorate. This is the data skew problem, which usually occurs in processing parallel hash join. Balancing the load before performing join will resolve this problem efficiently and the whole system performance can be improved. In this paper, we will present an algorithm using merit of very large memory to reduce disk access overhead in performing load balancing and to efficiently solve the data skew problem. Also, we will present analytical model of our new algorithm and present the result of some performance study we made comparing our algorithm with the other algorithms in handling data skew.

  • PDF