• Title/Summary/Keyword: Real-time data analysis

Search Result 2,781, Processing Time 0.091 seconds

Real-time data analysis technique using large data compression based spark (스파크 기반의 대용량 데이터 압축을 이용한 실시간 데이터 분석 기법)

  • Park, Soo-Yong;Shin, Yong-Tae
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2020.07a
    • /
    • pp.545-546
    • /
    • 2020
  • 스파크는 데이터 분석을 위한 오픈소스 툴이다. 스파크에서는 실시간 데이터 분석을 위하여 스파크 스트리밍이라는 기술을 제공한다. 스파크 스트리밍은 데이터 소스가 분석서버로 데이터 스트림을 전송한다. 이때 전송하는 데이터의 크기가 커질 경우 전송과정에서 지연이 발생할 수 있다. 제안하는 기법은 전송하고자 하는 데이터의 크기가 클 때 허프만 인코딩을 이용하여 데이터를 압축하여 전송시키므로 지연시간을 줄일 수 있다.

  • PDF

The Effect of Real-time Traffic Information System Relieving Traffic Congestion

  • Kang, Ho Jun;Moon, Tae Nam;Lee, Kang Hyeok;Song, Young Do;Shin, Do Hyoung
    • International conference on construction engineering and project management
    • /
    • 2015.10a
    • /
    • pp.652-653
    • /
    • 2015
  • This study investigates the effect of real-time traffic information on the traffic flows in Korea. Recently, the development of smartphones has made it easier to use the route guidance service based on real-time traffic information. By the Big Data analysis in the study, it was found that the number of postings on the web community sites increased sharply in 2010 and 2011 when the smartphones spread widely. In the analysis of the traffic speeds by time, the average traffic speeds for morning and evening rush hours on weekdays from 2009 to 2014 of the 142 sections in the 6 national highways in Gyeonggi-do, Korea were used. From the results of the analysis, it was found that the percentage of the number of sections with the improved traffic flows increased greatly in 2012 compared to 2011. The findings of the study indicate the effect of the real-time traffic information on improving traffic flows.

  • PDF

A study on the prediction method of the real fault distance using probability to the relay data of transmission line fault location (송전선로 거리표정치에 대한 실 고장거리의 확률적 예측방안)

  • Lee, Y.H.;Back, D.H.;Jang, S.H.
    • Proceedings of the KIEE Conference
    • /
    • 2006.07a
    • /
    • pp.10-11
    • /
    • 2006
  • The fault location is obtained from the distance relay that detects the fault of the transmission line. In this time, transmission line crews track down the fault location and the reasons. However, because of having error at the fault location of the distance relay, there is a discordance between real and obtained fault location. As this reason, the inspection time for finding fault location can be longer. In this paper, we proposed the statistical (regression) analysis method based on each type of relay's the historical fault location data and the real fault distance data to improve the problems. With finding the regression equation based on the regression analysis, and putting the relay fault location into that equation, the real fault distance is calculated. As a result of the Prediction fault location, the inspection time of transmission line can be reduced.

  • PDF

A Study on the Security Technology of Real-time Biometric Data in IoT Environment

  • Shin, Yoon-Hwan
    • Journal of the Korea Society of Computer and Information
    • /
    • v.21 no.1
    • /
    • pp.85-90
    • /
    • 2016
  • In this paper, the biometric data is transmitted in real time from the IoT environment is runoff, forgery, alteration, prevention of the factors that can be generated from a denial-of-service in advance, and the security strategy for the biometric data to protect the biometric data secure from security threats offer. The convenience of living in our surroundings to life with the development of ubiquitous computing and smart devices are available in real-time. And is also increasing interest in the IOT. IOT environment is giving the convenience of life. However, security threats to privacy also are exposed for 24 hours. This paper examines the security threats to biological data to be transmitted in real time from IOT environment. The technology for such security requirements and security technology according to the analysis of the threat. And with respect to the biometric data transmitted in real time on the IoT environment proposes a security strategy to ensure the stability against security threats and described with respect to its efficiency.

PhysioCover: Recovering the Missing Values in Physiological Data of Intensive Care Units

  • Kim, Sun-Hee;Yang, Hyung-Jeong;Kim, Soo-Hyung;Lee, Guee-Sang
    • International Journal of Contents
    • /
    • v.10 no.2
    • /
    • pp.47-58
    • /
    • 2014
  • Physiological signals provide important clues in the diagnosis and prediction of disease. Analyzing these signals is important in health and medicine. In particular, data preprocessing for physiological signal analysis is a vital issue because missing values, noise, and outliers may degrade the analysis performance. In this paper, we propose PhysioCover, a system that can recover missing values of physiological signals that were monitored in real time. PhysioCover integrates a gradual method and EM-based Principle Component Analysis (PCA). This approach can (1) more readily recover long- and short-term missing data than existing methods, such as traditional EM-based PCA, linear interpolation, 5-average and Missing Value Singular Value Decomposition (MSVD), (2) more effectively detect hidden variables than PCA and Independent component analysis (ICA), and (3) offer fast computation time through real-time processing. Experimental results with the physiological data of an intensive care unit show that the proposed method assigns more accurate missing values than previous methods.

A study on real-time communication of remote station in the distributed control system (분산 제어 시스템에서 원격 제어국의 실시간 통신에 관한 연구)

  • 김내진;김진태;박인갑
    • Journal of the Korean Institute of Telematics and Electronics A
    • /
    • v.31A no.10
    • /
    • pp.21-30
    • /
    • 1994
  • We discussed the Distributed Control System's design on preface and analyzed time of the real-time communication by using designed system. The DCS proposed in this thesis was implemented to network file system having recovery advantage and shared memory method to access file system of a Remote Station with ease. Also, this system minimized the network delay-time by using the real-time VME147 board. In implemented DCS, the performance analysis of real-time process of a Remote Station was done to get the total time for reak-tune communication from a Remote Station to the Central Station after terminating of process. For the analysis of system performance, we experiented by three steps. Firstly, we measuredthe processing the of LOOP function that real-time CPU convertes to-2,500~10.000 values from the input data of the Analog Interface Card. Secondly, we measured the processing time of the LOGIC function and the LOOP function. Lastly, we measured total processing time for communication from a Remote Station to the Centrol Station.

  • PDF

A Study on Defect Prediction through Real-time Monitoring of Die-Casting Process Equipment (주조공정 설비에 대한 실시간 모니터링을 통한 불량예측에 대한 연구)

  • Chulsoon Park;Heungseob Kim
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • v.45 no.4
    • /
    • pp.157-166
    • /
    • 2022
  • In the case of a die-casting process, defects that are difficult to confirm by visual inspection, such as shrinkage bubbles, may occur due to an error in maintaining a vacuum state. Since these casting defects are discovered during post-processing operations such as heat treatment or finishing work, they cannot be taken in advance at the casting time, which can cause a large number of defects. In this study, we propose an approach that can predict the occurrence of casting defects by defect type using machine learning technology based on casting parameter data collected from equipment in the die casting process in real time. Die-casting parameter data can basically be collected through the casting equipment controller. In order to perform classification analysis for predicting defects by defect type, labeling of casting parameters must be performed. In this study, first, the defective data set is separated by performing the primary clustering based on the total defect rate obtained during the post-processing. Second, the secondary cluster analysis is performed using the defect rate by type for the separated defect data set, and the labeling task is performed by defect type using the cluster analysis result. Finally, a classification learning model is created by collecting the entire labeled data set, and a real-time monitoring system for defect prediction using LabView and Python was implemented. When a defect is predicted, notification is performed so that the operator can cope with it, such as displaying on the monitoring screen and alarm notification.

User Identification Using Real Environmental Human Computer Interaction Behavior

  • Wu, Tong;Zheng, Kangfeng;Wu, Chunhua;Wang, Xiujuan
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.13 no.6
    • /
    • pp.3055-3073
    • /
    • 2019
  • In this paper, a new user identification method is presented using real environmental human-computer-interaction (HCI) behavior data to improve method usability. User behavior data in this paper are collected continuously without setting experimental scenes such as text length, action number, etc. To illustrate the characteristics of real environmental HCI data, probability density distribution and performance of keyboard and mouse data are analyzed through the random sampling method and Support Vector Machine(SVM) algorithm. Based on the analysis of HCI behavior data in a real environment, the Multiple Kernel Learning (MKL) method is first used for user HCI behavior identification due to the heterogeneity of keyboard and mouse data. All possible kernel methods are compared to determine the MKL algorithm's parameters to ensure the robustness of the algorithm. Data analysis results show that keyboard data have a narrower range of probability density distribution than mouse data. Keyboard data have better performance with a 1-min time window, while that of mouse data is achieved with a 10-min time window. Finally, experiments using the MKL algorithm with three global polynomial kernels and ten local Gaussian kernels achieve a user identification accuracy of 83.03% in a real environmental HCI dataset, which demonstrates that the proposed method achieves an encouraging performance.

Establishment of Early Warning System of Steep Slope Failure Using Real-time Rainfall Data Analysis (실시간 강우자료분석을 활용한 산사태 경보시스템 연구)

  • Kim, Sung-Wook;Choi, Eun-Kyoung;Park, Dug-Keun;Park, Jung-Hoon;Son, Sung-Gon
    • Proceedings of the Korean Geotechical Society Conference
    • /
    • 2010.09a
    • /
    • pp.253-262
    • /
    • 2010
  • In this study, localized heavy rainfall occurred during the collapse of steep slopes adjacent to the construction site and to ensure the safety of residents to build an early warning system was performed. Forecast/Alert range was estimated based on vulnerability landslide map and past disaster history. And established a critical line in consideration of the characteristics of local rainfall and operating a snake line, the study calculated causing and non-causing points. Also, be measured in real-time analysis of rainfall data in conjunction with the system before the steep slope failure occurred forecast/Alert System is presented.

  • PDF

Design and Implementation of Smart Gardening System Using Real-Time Visualization Algorithm Based on IoT (IoT 기반 실시간 시각화 알고리즘을 이용한 스마트가드닝 시스템 설계 및 구현)

  • Son, Soo-A;Park, Seok-Cheon
    • Journal of Internet Computing and Services
    • /
    • v.16 no.6
    • /
    • pp.31-37
    • /
    • 2015
  • Data generated from sensors are exploding with recent development of IoT. This paradigm shift requires various industry fields that demand instant actions to analyze the arising data on a real-time basis, along with the real-time visualization analysis. As the existing visualization systems, however, perform visualization after storing data, the response time of the server cannot guarantee the ms-level processing that is close to real-time. They also have a problem of destroying data that can be major resources as they do not possess the process resources. Therefore, a smart gardening system that applies a real-time visualization algorithm using IoT sensing data under a gardening environment was designed and implement in this study. The response time of the server was measured to evaluate the performance of the suggested system. As a result, the response speed of the suggested real-time visualization algorithm was guaranteeing the ms-level processing close to real-time.