• Title/Summary/Keyword: Real data

Search Result 15,678, Processing Time 0.04 seconds

Real-time and Parallel Semantic Translation Technique for Large-Scale Streaming Sensor Data in an IoT Environment (사물인터넷 환경에서 대용량 스트리밍 센서데이터의 실시간·병렬 시맨틱 변환 기법)

  • Kwon, SoonHyun;Park, Dongwan;Bang, Hyochan;Park, Youngtack
    • Journal of KIISE
    • /
    • v.42 no.1
    • /
    • pp.54-67
    • /
    • 2015
  • Nowadays, studies on the fusion of Semantic Web technologies are being carried out to promote the interoperability and value of sensor data in an IoT environment. To accomplish this, the semantic translation of sensor data is essential for convergence with service domain knowledge. The existing semantic translation technique, however, involves translating from static metadata into semantic data(RDF), and cannot properly process real-time and large-scale features in an IoT environment. Therefore, in this paper, we propose a technique for translating large-scale streaming sensor data generated in an IoT environment into semantic data, using real-time and parallel processing. In this technique, we define rules for semantic translation and store them in the semantic repository. The sensor data is translated in real-time with parallel processing using these pre-defined rules and an ontology-based semantic model. To improve the performance, we use the Apache Storm, a real-time big data analysis framework for parallel processing. The proposed technique was subjected to performance testing with the AWS observation data of the Meteorological Administration, which are large-scale streaming sensor data for demonstration purposes.

DEVELOPMENT OF REAL-TIME DATA REDUCTION PIPELINE FOR KMTNet (KMTNet 실시간 자료처리 파이프라인 개발)

  • Kim, D.J.;Lee, C.U.;Kim, S.L.;Park, B.G.
    • Publications of The Korean Astronomical Society
    • /
    • v.28 no.1
    • /
    • pp.1-6
    • /
    • 2013
  • Real-time data reduction pipeline for the Korea Microlensing Telescope Network (KMTNet) was developed by Korea Astronomy and Space Science Institute (KASI). The main goal of the data reduction pipeline is to find variable objects and to record their light variation from the large amount of observation data of about 200 GB per night per site. To achieve the goal we adopt three strategic implementations: precision pointing of telescope using the cross correlation correction for target fields, realtime data transferring using kernel-level file handling and high speed network, and segment data processing architecture using the Sun-Grid engine. We tested performance of the pipeline using simulated data which represent the similar circumstance to CTIO (Cerro Tololo Inter-American Observatory), and we have found that it takes about eight hours for whole processing of one-night data. Therefore we conclude that the pipeline works without problem in real-time if the network speed is high enough, e.g., as high as in CTIO.

Interactive Locomotion Controller using Inverted Pendulum Model with Low-Dimensional Data (역진자 모델-저차원 모션 캡처 데이터를 이용한 보행 모션 제어기)

  • Han, KuHyun;Kim, YoungBeom;Park, Byung-Ha;Jung, Kwang-Mo;Han, JungHyun
    • Journal of Korea Multimedia Society
    • /
    • v.19 no.8
    • /
    • pp.1587-1596
    • /
    • 2016
  • This paper presents an interactive locomotion controller using motion capture data and inverted pendulum model. Most of the data-driven character controller using motion capture data have two kinds of limitation. First, it needs many example motion capture data to generate realistic motion. Second, it is difficult to make natural-looking motion when characters navigate dynamic terrain. In this paper, we present a technique that uses dimension reduction technique to motion capture data together with the Gaussian process dynamical model (GPDM), and interpolates the low-dimensional data to make final motion. With the low-dimensional data, we can make realistic walking motion with few example motion capture data. In addition, we apply the inverted pendulum model (IPM) to calculate the root trajectory considering the real-time user input upon the dynamic terrain. Our method can be used in game, virtual training, and many real-time applications.

Development of Big-data Management Platform Considering Docker Based Real Time Data Connecting and Processing Environments (도커 기반의 실시간 데이터 연계 및 처리 환경을 고려한 빅데이터 관리 플랫폼 개발)

  • Kim, Dong Gil;Park, Yong-Soon;Chung, Tae-Yun
    • IEMEK Journal of Embedded Systems and Applications
    • /
    • v.16 no.4
    • /
    • pp.153-161
    • /
    • 2021
  • Real-time access is required to handle continuous and unstructured data and should be flexible in management under dynamic state. Platform can be built to allow data collection, storage, and processing from local-server or multi-server. Although the former centralize method is easy to control, it creates an overload problem because it proceeds all the processing in one unit, and the latter distributed method performs parallel processing, so it is fast to respond and can easily scale system capacity, but the design is complex. This paper provides data collection and processing on one platform to derive significant insights from various data held by an enterprise or agency in the latter manner, which is intuitively available on dashboards and utilizes Spark to improve distributed processing performance. All service utilize dockers to distribute and management. The data used in this study was 100% collected from Kafka, showing that when the file size is 4.4 gigabytes, the data processing speed in spark cluster mode is 2 minute 15 seconds, about 3 minutes 19 seconds faster than the local mode.

Real-world multimodal lifelog dataset for human behavior study

  • Chung, Seungeun;Jeong, Chi Yoon;Lim, Jeong Mook;Lim, Jiyoun;Noh, Kyoung Ju;Kim, Gague;Jeong, Hyuntae
    • ETRI Journal
    • /
    • v.44 no.3
    • /
    • pp.426-437
    • /
    • 2022
  • To understand the multilateral characteristics of human behavior and physiological markers related to physical, emotional, and environmental states, extensive lifelog data collection in a real-world environment is essential. Here, we propose a data collection method using multimodal mobile sensing and present a long-term dataset from 22 subjects and 616 days of experimental sessions. The dataset contains over 10 000 hours of data, including physiological, data such as photoplethysmography, electrodermal activity, and skin temperature in addition to the multivariate behavioral data. Furthermore, it consists of 10 372 user labels with emotional states and 590 days of sleep quality data. To demonstrate feasibility, human activity recognition was applied on the sensor data using a convolutional neural network-based deep learning model with 92.78% recognition accuracy. From the activity recognition result, we extracted the daily behavior pattern and discovered five representative models by applying spectral clustering. This demonstrates that the dataset contributed toward understanding human behavior using multimodal data accumulated throughout daily lives under natural conditions.

Field Test of Automated Activity Classification Using Acceleration Signals from a Wristband

  • Gong, Yue;Seo, JoonOh
    • International conference on construction engineering and project management
    • /
    • 2020.12a
    • /
    • pp.443-452
    • /
    • 2020
  • Worker's awkward postures and unreasonable physical load can be corrected by monitoring construction activities, thereby increasing the safety and productivity of construction workers and projects. However, manual identification is time-consuming and contains high human variance. In this regard, an automated activity recognition system based on inertial measurement unit can help in rapidly and precisely collecting motion data. With the acceleration data, the machine learning algorithm will be used to train classifiers for automatically categorizing activities. However, input acceleration data are extracted either from designed experiments or simple construction work in previous studies. Thus, collected data series are discontinuous and activity categories are insufficient for real construction circumstances. This study aims to collect acceleration data during long-term continuous work in a construction project and validate the feasibility of activity recognition algorithm with the continuous motion data. The data collection covers two different workers performing formwork at the same site. An accelerator, as well as portable camera, is attached to the worker during the entire working session for simultaneously recording motion data and working activity. The supervised machine learning-based models are trained to classify activity in hierarchical levels, which reaches a 96.9% testing accuracy of recognizing rest and work and 85.6% testing accuracy of identifying stationary, traveling, and rebar installation actions.

  • PDF

A Study on the Real-time Data Interface Technology based on SCM for Shipbuilding and Marine Equipment Production (조선해양기자재 제작을 위한 SCM 기반 실시간 데이터 인터페이스 기술에 관한 연구)

  • Myeong-Ki Han;Young-Hun Kim;Jun-Su Park;Won-Ho Lee
    • Journal of the Korean Society of Industry Convergence
    • /
    • v.27 no.1
    • /
    • pp.143-149
    • /
    • 2024
  • The production and procurement of shipbuilding and offshore equipment is an important competitive factor in the shipbuilding and offshore industry. Recently, ICT-based digital technology has been rapidly applied to the manufacturing industry following the Fourth Industrial Revolution. Under the digital transformation, real-time data interface technology based on SCM (Supply Chain Management) is emerging as an important tool to improve the efficiency of the equipment manufacturing process. In this study, the characteristics and advantages and disadvantages of interface technologies of web-based data interface technologies were compared and analyzed. The performance was compared between theoretical evaluation based on technical features and practical application cases. As a result, it was confirmed that GraphQL is useful for selective data processing, but there is a problem with optimization, and REST API has a problem with receiving data due to a fixed data structure. Therefore, this study aims to suggest ways to utilize and optimize these data interface technologies.

Target Classification Algorithm Using Complex-valued Support Vector Machine (복소수 SVM을 이용한 목표물 식별 알고리즘)

  • Kang, Youn Joung;Lee, Jaeil;Bae, Jinho;Lee, Chong Hyun
    • Journal of the Institute of Electronics and Information Engineers
    • /
    • v.50 no.4
    • /
    • pp.182-188
    • /
    • 2013
  • In this paper, we propose a complex-valued support vector machine (SVM) classifier which process the complex valued signal measured by pulse doppler radar (PDR) to identify moving targets from the background. SVM is widely applied in the field of pattern recognition, but features which used to classify are almost real valued data. Proposed complex-valued SVM can classify the moving target using real valued data, imaginary valued data, and cross-information data. To design complex-valued SVM, we consider slack variables of real and complex axis, and use the KKT (Karush-Kuhn-Tucker) conditions for complex data. Also we apply radial basis function (RBF) as a kernel function which use a distance of complex values. To evaluate the performance of the complex-valued SVM, complex valued data from PDR were classified using real-valued SVM and complex-valued SVM. The proposed complex-valued SVM classification was improved compared to real-valued SVM for dog and human, respectively 8%, 10%, have been improved.

A Real-time Context Integration System for Multimodal Sensor Networks using XML (XML을 활용한 멀티모달 센서기반 실시간 컨텍스트 통합 시스템)

  • Yang, Sung-Ihk;Hong, Jin-Hyuk;Cho, Sung-Bae
    • 한국HCI학회:학술대회논문집
    • /
    • 2008.02a
    • /
    • pp.141-146
    • /
    • 2008
  • As the interest about ubiquitous environment is increasing, there are many researches about the services in this environment. These services have important issues in interpreting the users' context, using many kinds of sensors, like PDA, GPS and accelerometers. Low level raw data, which sensors like accelerometers calibrates, are hard to use, and to provide real-time services preprocessing and interpreting the data into context, in real-time, is important. This paper describes a context integrate system which can integrate these sensors and also sensors which has raw data, like accelerometers and physiological sensors, and define the context interpret rule with XML. The proposing system reduces programming operations when adding a sensor to the sensor network or modifying the context interpreting rule by using XML. By using this system, we implemented a real-time data monitoring system which can describe the numeric data into graphs, and assist the user to validate the data and results of the preprocess phase, and also support the external services and applications to use the context of the user.

  • PDF

Path Finding with Maximum Speed Dynamic Heuristic (최고 속력 동적 휴리스틱을 이용한 경로탐색)

  • Kim, Ji-Soo;Lee, Ji-Wan;Cho, Dae-Soo
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.13 no.8
    • /
    • pp.1615-1622
    • /
    • 2009
  • Generally, the Terminal Based Navigation System(TBNS) used embedded road data searches a path that has less qualitative than The Center Based Navigation System(CBNS). TBNS has not used real time road data but it is recently able to use it with technique such as TPEG. However, it causes to increase a cost of exploring by using real time road data for improvement quality of a path, because of limited performance. In this paper, we propose a Dynamic Heuristic to improve quality of path in the TBNS. Dynamic Heuristic(DH) is not fixed data and is dynamically modified using transferred real time road data from server. In this paper, we propose path-lading algorithm with Maximum Speed Dynamic Heuristic (DH-MAX) and do an experiment. The DH-MAX is to be used the highest speed as DH, in real map divided by same size. And proposed algorithm searches path using the priority searching only of the fixed data, but also the highest speed with real time information. In the performance test, the quality of path is enhanced but the cost of searching is increased than A* algorithm.