• Title/Summary/Keyword: Real-time Data Processing

Search Result 2,092, Processing Time 0.03 seconds

Squall: A Real-time Big Data Processing Framework based on TMO Model for Real-time Events and Micro-batch Processing (Squall: 실시간 이벤트와 마이크로-배치의 동시 처리 지원을 위한 TMO 모델 기반의 실시간 빅데이터 처리 프레임워크)

  • Son, Jae Gi;Kim, Jung Guk
    • Journal of KIISE
    • /
    • v.44 no.1
    • /
    • pp.84-94
    • /
    • 2017
  • Recently, the importance of velocity, one of the characteristics of big data (5V: Volume, Variety, Velocity, Veracity, and Value), has been emphasized in the data processing, which has led to several studies on the real-time stream processing, a technology for quick and accurate processing and analyses of big data. In this paper, we propose a Squall framework using Time-triggered Message-triggered Object (TMO) technology, a model that is widely used for processing real-time big data. Moreover, we provide a description of Squall framework and its operations under a single node. TMO is an object model that supports the non-regular real-time processing method for certain conditions as well as regular periodic processing for certain amount of time. A Squall framework can support the real-time event stream of big data and micro-batch processing with outstanding performances, as compared to Apache storm and Spark Streaming. However, additional development for processing real-time stream under multiple nodes that is common under most frameworks is needed. In conclusion, the advantages of a TMO model can overcome the drawbacks of Apache storm or Spark Streaming in the processing of real-time big data. The TMO model has potential as a useful model in real-time big data processing.

A Study on Development of Middleware for Real Time Data Processing with RFID (RFID 시스템 도입을 위한 실시간 정보처리용 미들웨어의 개발에 관한 연구)

  • Li Guang Zhu;Jho Yong-Chul;Lee Chang-Ho
    • Journal of the Korea Safety Management & Science
    • /
    • v.8 no.4
    • /
    • pp.109-117
    • /
    • 2006
  • This paper deals with the middleware development for real time data processing using ubiquitous RFID(Radio Frequency Identification). This middleware system supports the real time data gathering, filtering, and collaborative operations within SCM. We expect to decrease the data processing cost and to upgrade the status of RTE(Real Time Enterprise) and competitive power of company.

Data Transmission Processing System Design for Real-Time Distributed Simulation by Using Software Design Patterns (소프트웨어 디자인 패턴을 적용한 실시간 분산 시뮬레이션을 위한 데이터 전달처리 시스템 설계)

  • Suk, Jin-Weon;Ryoo, In-Tae
    • Journal of Digital Contents Society
    • /
    • v.10 no.4
    • /
    • pp.649-657
    • /
    • 2009
  • Usually, The data transmission processing efficiency of the distributed system running on high speed networks depends on the system architecture and the data transmission processing system. In order to secure the real-time rate and the system reliability, the real-time distributed simulation system on the distributed environment has tried to satisfy the performance required by the data transmission processing system. However, the client/server-based data transmission processing system in the real-time simulation system has been difficult to satisfy the system stability, extensibility and maintenability, especially when system changes. So, it is natural to study another improved data transmission processing system to solve the problems at the existing real-time simulation system. After analyzing the existing real-time simulation system, this paper will propose the improved real-time data transmission system by using Software Design Pattern, which enhances extensibility, interoperability, reusability and maintenability of the system.

  • PDF

Real-Time Sink Node Architecture for a Service Robot Based on Active Healthcare/Living-support USN (능동 건강/생활지원 USN 기반 서비스 로봇 시스템의 실시간 싱크 노드 구조)

  • Shin, Dong-Gwan;Yi, Soo-Yeong;Choi, Byoung-Wook
    • Journal of Institute of Control, Robotics and Systems
    • /
    • v.14 no.7
    • /
    • pp.720-725
    • /
    • 2008
  • This paper proposes a system architecture for USN with a service robot to provide more active assisted living services for elderly persons by monitoring their mental and physical well-being with USN environments at home, hospital, or silver town. Sensors embedded in USN are used to detect preventive measures for chronic disease. Logged data are transferred to main controller of a service robot via wireless channel in which the analysis of data is performed. For the purpose of handling emergency situations, it needs real-time processing on gathering variety sensor data, routing algorithms for sensor nodes to a moving sink node and processing of logged data. This paper realized multi-hop sensor network to detect user movements with biometric data transmission and performed algorithms on Xenomai, a real-time embedded Linux. To leverage active sensing, a mobile robot is used of which task was implemented with a priority to process urgent data came from the sink-node. This software architecture is anticipated to integrate sensing, communication and computing with real-time manner. In order to verify the usefulness of a proposed system, the performance of data transferring and processing on a real-time OS with non real-time OS is also evaluated.

Design of Image Processing Unit for Real Time Processing (Real Time Processing을 위한 Image Processing Unit의 설계)

  • 김진욱;김석태
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 1998.11a
    • /
    • pp.194-197
    • /
    • 1998
  • Image Processing은 Image Data가 대량이고 내재된 정보가 병렬로 연관성을 가진다는 측면에서 실시간 처리가 용이하지 알다. 본 연구에서는 High Speed Real Time Image Processing을 위한 IPU(Image Processing Unit)와 이를 구동하기 위한 High Speed Real Time image Processing Language인 IPASM(Image Processing Assembly)을 제안한다. 우선 IPU의 기본개념을 설명하고 IPU의 구현을 위한 IPLU(Image Processing Logic Unit)를 설계한다. 그 후 Window98환경에서 구동 가능한 IPASM Interpreter를 실제로 제작하여 IPU의 동작방식을 간접적으로 진단한다.

  • PDF

Data Processing Method for Real-time Safety Supervision System in Railway (실시간 철도안전 관제를 위한 데이터 처리 방안 연구)

  • Shin, Kwang-Ho;Jung, Hye-Ran;Ahn, Jin
    • Journal of the Korean Society for Railway
    • /
    • v.19 no.4
    • /
    • pp.445-455
    • /
    • 2016
  • A goal of the Real-time railway safety supervision system is to improve the safety oversight efficiency and to prevent accidents by integrating existing distributed monitoring systems, train, signal, power and facilities. So, the system require better performance regarding real-time processing based on big data. The disk-based database that is used in existing railway control systems has a problem with real-time processing; memory-based databases haves a limitation in terms of big-data processing; and time series databases haves a limitation in terms of real-time processing. So, we need a new database architecture for simultaneous real-time processing based on big data. In this study, we review the existing railway monitoring systems and propose a new database architecture for a real-time railway safety supervision system.

Real-time PCM Data Processing System Development for Flight Test Control (비행시험통제용 실시간 PCM 자료처리시스템 개발)

  • Park, In Hee
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.25 no.6
    • /
    • pp.825-833
    • /
    • 2021
  • In flight tests, aircraft moves in real time, so it is important that data from instrumentation/measurement equipment used to determine aircraft status are processed in necessary form and transmitted to flight control systems in real time. Therefore, through telemetry data processing time reduction and processing cycle improvement in flight test control computer data processing system, in order to provide faster slave-data and safety judgment information to radar/telemetry slave-data processing, flight safety analysis system, emergency destruction transmission system, etc., we developed a PCM processing system that can be operated independently by installing data processing software that can receive and process PCM data in current telemetry data processing system and radar information at the same time. In this paper, we explain classified software functions in detail, starting with overall structure of PCM data processing systems developed by supplementing existing systems. Additionally, PCM data processing system will be supplemented through system stabilization and test operation.

Interactive chinese Character Distance Learning System on the WWW (WWW에서 대화형 원격 한자학습 시스템)

  • Gang, Jong-Gyu;Park, Sang-U;Kim, Hyeon-Suk;Kim, Gye-Hwan;Jin, Seong-Il
    • The Transactions of the Korea Information Processing Society
    • /
    • v.4 no.3
    • /
    • pp.698-708
    • /
    • 1997
  • To construct distance learing servers and provide their service using the WWW(World Wide Web), it is necessary that we use a real-time processing mehtod rather than the processing after downloading method for multimedia data transmission and their processing.To fulfill such requirements, we developed a real-time processing muduloe for distance education which can process multimedia data in AVI and WAV formats in distrbuted eviroments.We in turn developede a real-time WWW server that can provide real-time services of hypertxt and motion poctures data in temsw of adding the real-time porcessing modute to the MuX framework and intergarting them with WWW. We frnally developed as distance lerming system for real-time interactive chinese character learming, bassed on the results from the pre-vious steps.

  • PDF

Application Of Open Data Framework For Real-Time Data Processing (실시간 데이터 처리를 위한 개방형 데이터 프레임워크 적용 방안)

  • Park, Sun-ho;Kim, Young-kil
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.23 no.10
    • /
    • pp.1179-1187
    • /
    • 2019
  • In today's technology environment, most big data-based applications and solutions are based on real-time processing of streaming data. Real-time processing and analysis of big data streams plays an important role in the development of big data-based applications and solutions. In particular, in the maritime data processing environment, the necessity of developing a technology capable of rapidly processing and analyzing a large amount of real-time data due to the explosion of data is accelerating. Therefore, this paper analyzes the characteristics of NiFi, Kafka, and Druid as suitable open source among various open data technologies for processing big data, and provides the latest information on external linkage necessary for maritime service analysis in Korean e-Navigation service. To this end, we will lay the foundation for applying open data framework technology for real-time data processing.

Real-time system control for the 6-DOF simulation (6-DOF 시뮬레이터의 real-time 시스템 제어에 관한 연구)

  • 김영대;김충영;백인철;민성기
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 1989.10a
    • /
    • pp.17-21
    • /
    • 1989
  • 6-DOE simulator system is designed to real-time processing for motion control, data acquisition, image generation and image processing etc.. In this paper, we introduce hardware and software design technologies for distributed processing, event-trapping, system monitoring and time scheduling procedure in 6-DOF simulator system design.

  • PDF