• Title/Summary/Keyword: Parallel data processing

Search Result 751, Processing Time 0.034 seconds

Real-Time IoT Big-data Processing for Stream Reasoning (스트림-리즈닝을 위한 실시간 사물인터넷 빅-데이터 처리)

  • Yun, Chang Ho;Park, Jong Won;Jung, Hae Sun;Lee, Yong Woo
    • Journal of Internet Computing and Services
    • /
    • v.18 no.3
    • /
    • pp.1-9
    • /
    • 2017
  • Smart Cities intelligently manage numerous infrastructures, including Smart-City IoT devices, and provide a variety of smart-city applications to citizen. In order to provide various information needed for smart-city applications, Smart Cities require a function to intelligently process large-scale streamed big data that are constantly generated from a large number of IoT devices. To provide smart services in Smart-City, the Smart-City Consortium uses stream reasoning. Our stream reasoning requires real-time processing of big data. However, there are limitations associated with real-time processing of large-scale streamed big data in Smart Cities. In this paper, we introduce one of our researches on cloud computing based real-time distributed-parallel-processing to be used in stream-reasoning of IoT big data in Smart Cities. The Smart-City Consortium introduced its previously developed smart-city middleware. In the research for this paper, we made cloud computing based real-time distributed-parallel-processing available in the cloud computing platform of the smart-city middleware developed in the previous research, so that we can perform real-time distributed-parallel-processing with them. This paper introduces a real-time distributed-parallel-processing method and system for stream reasoning with IoT big data transmitted from various sensors of Smart Cities and evaluate the performance of real-time distributed-parallel-processing of the system where the method is implemented.

Distributed Collision-Resolvable Medium Access Control for Wireless LANs with Interference Cancellation Support

  • Shen, Hu;Lv, Shaohe;Wang, Xiaodong;Zhou, Xingming
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.8 no.8
    • /
    • pp.2691-2707
    • /
    • 2014
  • Medium access control is critical in wireless networks for efficient spectrum utilization. In this paper, we introduce a novel collision resolution method based on the technique of known interference cancellation, and propose a new MAC protocol named as CR-MAC, in which AP tries to decode all the collided data packets by combining partial retransmissions and known interference cancellation. As the collided transmissions are fully utilized, less retransmission is required, especially in a crowded network. The NS-2simulation and MATLAB numerical results show that, under various network settings, CR-MAC performs much better than the IEEE 802.11 DCF in terms of the aggregation throughput and the expected packet delay.

A Parallel Kalman Filter for Discrete Linear Time-invariant System (이산 선형 시불변시스템에 대한 병렬칼만필터)

  • Kim, Yong Joon;Lee, Jang Gyu;Kim, Hyoung Joong
    • Journal of Industrial Technology
    • /
    • v.10
    • /
    • pp.15-20
    • /
    • 1990
  • A parallel processing algorithm for discrete Kalman filter, which is one of the most commonly used filtering technique in modern control, signal processing, and communication, is proposed. Previously proposed parallel algorithms to decrease the number of computations needed in the Kalman filter are the hierachical structures by distributed processing of measurements, or the systolic structures to disperse the computational burden. In this paper, a new parallel Kalman filter employing a structure similar to recursive doubling is proposed. Estimated values of state variables by the new algorithm converge with two times faster data processing speed than that of the conventional Kalman filter. Moreover it maintains the optimality of the conventional Kalman filter.

  • PDF

A Parallel Processing Structure for the Discrete Kalman Filter (이산 칼만 필터의 병렬처리 구조)

  • 김용준;이장규;김병중
    • The Transactions of the Korean Institute of Electrical Engineers
    • /
    • v.39 no.10
    • /
    • pp.1057-1065
    • /
    • 1990
  • A parallel processing algorithm for the discrete Kalman filter, which is one of the most commonly used filtering techniques in modern control, signal processing, and communication, is proposed. To decrease the number of computations critical in the Kalman filter, previously proposed parallel algorithms are of the hierarchical structure by distributed processing of measurements, or of the systolic structure to disperse the computational burden. In this paper, a new parallel Kalman filter employing a structure similar to recursive doubling is proposed. Estimated valuse of state variables by the new algorithm converge faster to the true values because the new algorithm can process data twice faster than the conventional Kalman filter. Moreover, it maintains the optimality of the conventional Kalman filter.

  • PDF

A Parallel Kalman Filter for Discrete Linear Time-invariant System (이산 선형 시불변시스템에 대한 병렬칼만필터)

  • Lee, Jang-Gyu;Kim, Yong-Joon;Kim, Hyoung-Joong
    • Proceedings of the KIEE Conference
    • /
    • 1990.07a
    • /
    • pp.64-67
    • /
    • 1990
  • A parallel processing algorithm for discrete Kalman filter, which is one of the most commonly used filtering technique in modern control, signal processing, and communication. is proposed. Previously proposed parallel algorithms to decrease the number of computations needed in the Kalman filter are the hierachical structures by distributed processing of measurements, or the systolic structures to disperse the computational burden. In this paper, a new parallel Kalman filter employing a structure similar to recursive doubling is proposed. Estimated values of state variables by the new algorithm converge with two times faster data processing speed than that of the conventional Kalman filter. Moreover it maintains the optimality of the conventional Kalman filter.

  • PDF

An Implementation of Pipelined Prallel Processing System for Multi-Access Memory System

  • Lee, Hyung;Cho, Hyeon-Koo;You, Dae-Sang;Park, Jong-Won
    • Proceedings of the IEEK Conference
    • /
    • 2002.07a
    • /
    • pp.149-151
    • /
    • 2002
  • We had been developing the variety of parallel processing systems in order to improve the processing speed of visual media applications. These systems were using multi-access memory system(MAMS) as a parallel memory system, which provides the capability of the simultaneous accesses of image points in a line-segment with an arbitrary degree, which is required in many low-level image processing operations such as edge or line detection in a particular direction, and so on. But, the performance of these systems did not give a faithful speed because of asynchronous feature between MAMS and processing elements. To improve the processing speed of these systems, we have been investigated a pipelined parallel processing system using MAMS. Although the system is considered as being the single instruction multiple data(SIMD) type like the early developed systems, the performance of the system yielded about 2.5 times faster speed.

  • PDF

Modeling and Verification of Workflows with Various Parallel Dependencies (다양한 병행 종속성을 포함한 워크플로우 모델링 및 검증)

  • 정희택;이도헌
    • The Journal of Information Technology and Database
    • /
    • v.6 no.1
    • /
    • pp.59-72
    • /
    • 1999
  • A study on workflow system as an automated business processing system is done recently. However, it did not consider various dependencies between parallel tasks. Therefore, this paper proposes modeling and verification of workflows with various parallel dependencies. For this, firstly, we propose four dependencies to specify various parallel dependencies between tasks. They contain sequential starts, parallel starts, sequential commits, and parallel commits. Secondly, we suggest a method to specify various parallel dependencies on workflow graph. Thirdly, we propose a verification method to detect contradictions on workflow specifications.

  • PDF

Parallelism point selection in nested parallelism situations with focus on the bandwidth selection problem (평활량 선택문제 측면에서 본 중첩병렬화 상황에서 병렬처리 포인트선택)

  • Cho, Gayoung;Noh, Hohsuk
    • The Korean Journal of Applied Statistics
    • /
    • v.31 no.3
    • /
    • pp.383-396
    • /
    • 2018
  • Various parallel processing R packages are used for fast processing and the analysis of big data. Parallel processing is used when the work can be decomposed into tasks that are non-interdependent. In some cases, each task decomposed for parallel processing can also be decomposed into non-interdependent subtasks. We have to choose whether to parallelize the decomposed tasks in the first step or to parallelize the subtasks in the second step when facing nested parallelism situations. This choice has a significant impact on the speed of computation; consequently, it is important to understand the nature of the work and decide where to do the parallel processing. In this paper, we provide an idea of how to apply parallel computing effectively to problems by illustrating how to select a parallelism point for the bandwidth selection of nonparametric regression.

A Fast Transmission of Mobile Agents Using Binomial Trees (바이노미얼 트리를 이용한 이동 에이전트의 빠른 전송)

  • Cho, Soo-Hyun;Kim, Young-Hak
    • The KIPS Transactions:PartA
    • /
    • v.9A no.3
    • /
    • pp.341-350
    • /
    • 2002
  • As network environments have been improved and the use of internet has been increased, mobile agent technologies are widely used in the fields of information retrieval, network management, electronic commerce, and parallel/distributed processing. Recently, a lot of researchers have studied the concepts of parallel/distributed processing based on mobile agents. SPMD is the parallel processing method which transmits a program to all the computers participated in parallel environment, and performs a work with different data. Therefore, to transmit fast a program to all the computers is one of important factors to reduce total execution time. In this paper, we consider the parallel environment consisting of mobile agents system, and propose a new method which transmits fast a mobile agent code to all the computers using binomial trees in order to efficiently perform the SPMD parallel processing. The proposed method is compared with another ones through experimental evaluation on the IBM's Aglets, and gets greatly better performance. Also this paper deals with fault tolerances which can be occurred in transmitting a mobile agent using binomial trees.

Design and Implementation of Big Data Platform for Image Processing in Agriculture (농업 이미지 처리를 위한 빅테이터 플랫폼 설계 및 구현)

  • Nguyen, Van-Quyet;Nguyen, Sinh Ngoc;Vu, Duc Tiep;Kim, Kyungbaek
    • Annual Conference of KIPS
    • /
    • 2016.10a
    • /
    • pp.50-53
    • /
    • 2016
  • Image processing techniques play an increasingly important role in many aspects of our daily life. For example, it has been shown to improve agricultural productivity in a number of ways such as plant pest detecting or fruit grading. However, massive quantities of images generated in real-time through multi-devices such as remote sensors during monitoring plant growth lead to the challenges of big data. Meanwhile, most current image processing systems are designed for small-scale and local computation, and they do not scale well to handle big data problems with their large requirements for computational resources and storage. In this paper, we have proposed an IPABigData (Image Processing Algorithm BigData) platform which provides algorithms to support large-scale image processing in agriculture based on Hadoop framework. Hadoop provides a parallel computation model MapReduce and Hadoop distributed file system (HDFS) module. It can also handle parallel pipelines, which are frequently used in image processing. In our experiment, we show that our platform outperforms traditional system in a scenario of image segmentation.