• 제목/요약/키워드: Cloud Data Preprocessing

검색결과 24건 처리시간 0.032초

다단계 정육면체 격자 기반의 가상점 생성을 통한 대용량 3D point cloud 가시화 (Massive 3D Point Cloud Visualization by Generating Artificial Center Points from Multi-Resolution Cube Grid Structure)

  • 양승찬;한수희;허준
    • 한국측량학회지
    • /
    • 제30권4호
    • /
    • pp.335-342
    • /
    • 2012
  • 건축, 토목, 의료, 컴퓨터 그래픽스 분야 등 다양한 분야에서 이용되는 3D point cloud는 최근 레이저 스캐너의 발달로 인해 그 용량이 점점 커지게 되었다. 컴퓨터 메모리의 용량을 넘어서서 모든 데이터를 한 번에 처리할 수 없는 대용량 3D point cloud를 가시화하고 편집하기 위해 여러 전처리 및 가시화 방법들이 소개되었고 본 논문에서 비교한 QSplat의 경우 3D 모델의 형상 확인과 용량 감소를 목적으로 원본 좌표를 손실 압축하여 저장하였다. 본 논문에서 제시하는 방법은 3D point cloud를 정육면체 격자로 분할하고 center sampling을 통해 가상점 집합을 생성하며 가시화 과정에서 격자에 저장된 point 집합 취득을 통한 빠른 렌더링이 가능하다. 홍익대학교 인근 지역을 측정한 약 1억 2천만 개 point의 대용량 3D point cloud를 QSplat과 다단계 정육면체 격자 기반 방법으로 비교한 결과 전처리 과정에서는 QSplat이, 가시화 과정에서는 다단계 정육면체 격자 기반 방법이 빠른 속도를 보여주었다. 또한 다단계 정육면체 격자 기반 방법은 point의 원본 좌표를 저장하기에 추후 가시화 외에 편집, segmentation 등의 작업을 고려하여 고안되었다.

무인수상선의 디지털 트윈 공간 재구성을 위한 이미지 보정 및 점군데이터 간의 매핑 프레임워크 설계 (Design of a Mapping Framework on Image Correction and Point Cloud Data for Spatial Reconstruction of Digital Twin with an Autonomous Surface Vehicle)

  • 허수현;강민주;최진우;박정홍
    • 대한조선학회논문집
    • /
    • 제61권3호
    • /
    • pp.143-151
    • /
    • 2024
  • In this study, we present a mapping framework for 3D spatial reconstruction of digital twin model using navigation and perception sensors mounted on an Autonomous Surface Vehicle (ASV). For improving the level of realism of digital twin models, 3D spatial information should be reconstructed as a digitalized spatial model and integrated with the components and system models of the ASV. In particular, for the 3D spatial reconstruction, color and 3D point cloud data which acquired from a camera and a LiDAR sensors corresponding to the navigation information at the specific time are required to map without minimizing the noise. To ensure clear and accurate reconstruction of the acquired data in the proposed mapping framework, a image preprocessing was designed to enhance the brightness of low-light images, and a preprocessing for 3D point cloud data was included to filter out unnecessary data. Subsequently, a point matching process between consecutive 3D point cloud data was conducted using the Generalized Iterative Closest Point (G-ICP) approach, and the color information was mapped with the matched 3D point cloud data. The feasibility of the proposed mapping framework was validated through a field data set acquired from field experiments in a inland water environment, and its results were described.

A Study on the Effective Preprocessing Methods for Accelerating Point Cloud Registration

  • Chungsu, Jang;Yongmin, Kim;Taehyun, Kim;Sunyong, Choi;Jinwoo, Koh;Seungkeun, Lee
    • 대한원격탐사학회지
    • /
    • 제39권1호
    • /
    • pp.111-127
    • /
    • 2023
  • In visual slam and 3D data modeling, the Iterative Closest Point method is a primary fundamental algorithm, and many technical fields have used this method. However, it relies on search methods that take a high search time. This paper solves this problem by applying an effective point cloud refinement method. And this paper also accelerates the point cloud registration process with an indexing scheme using the spatial decomposition method. Through some experiments, the results of this paper show that the proposed point cloud refinement method helped to produce better performance.

Resource Metric Refining Module for AIOps Learning Data in Kubernetes Microservice

  • Jonghwan Park;Jaegi Son;Dongmin Kim
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제17권6호
    • /
    • pp.1545-1559
    • /
    • 2023
  • In the cloud environment, microservices are implemented through Kubernetes, and these services can be expanded or reduced through the autoscaling function under Kubernetes, depending on the service request or resource usage. However, the increase in the number of nodes or distributed microservices in Kubernetes and the unpredictable autoscaling function make it very difficult for system administrators to conduct operations. Artificial Intelligence for IT Operations (AIOps) supports resource management for cloud services through AI and has attracted attention as a solution to these problems. For example, after the AI model learns the metric or log data collected in the microservice units, failures can be inferred by predicting the resources in future data. However, it is difficult to construct data sets for generating learning models because many microservices used for autoscaling generate different metrics or logs in the same timestamp. In this study, we propose a cloud data refining module and structure that collects metric or log data in a microservice environment implemented by Kubernetes; and arranges it into computing resources corresponding to each service so that AI models can learn and analogize service-specific failures. We obtained Kubernetes-based AIOps learning data through this module, and after learning the built dataset through the AI model, we verified the prediction result through the differences between the obtained and actual data.

태양 에너지 수집형 IoT 엣지 컴퓨팅 환경에서 효율적인 오디오 딥러닝을 위한 에너지 적응형 데이터 전처리 기법 (Energy-Aware Data-Preprocessing Scheme for Efficient Audio Deep Learning in Solar-Powered IoT Edge Computing Environments)

  • 유연태;노동건
    • 대한임베디드공학회논문지
    • /
    • 제18권4호
    • /
    • pp.159-164
    • /
    • 2023
  • Solar energy harvesting IoT devices prioritize maximizing the utilization of collected energy due to the periodic recharging nature of solar energy, rather than minimizing energy consumption. Meanwhile, research on edge AI, which performs machine learning near the data source instead of the cloud, is actively conducted for reasons such as data confidentiality and privacy, response time, and cost. One such research area involves performing various audio AI applications using audio data collected from multiple IoT devices in an IoT edge computing environment. However, in most studies, IoT devices only perform sensing data transmission to the edge server, and all processes, including data preprocessing, are performed on the edge server. In this case, it not only leads to overload issues on the edge server but also causes network congestion by transmitting unnecessary data for learning. On the other way, if data preprocessing is delegated to each IoT device to address this issue, it leads to another problem of increased blackout time due to energy shortages in the devices. In this paper, we aim to alleviate the problem of increased blackout time in devices while mitigating issues in server-centric edge AI environments by determining where the data preprocessed based on the energy state of each IoT device. In the proposed method, IoT devices only perform the preprocessing process, which includes sound discrimination and noise removal, and transmit to the server if there is more energy available than the energy threshold required for the basic operation of the device.

Analysis of Laughter Therapy Trend Using Text Network Analysis and Topic Modeling

  • LEE, Do-Young
    • 웰빙융합연구
    • /
    • 제5권4호
    • /
    • pp.33-37
    • /
    • 2022
  • Purpose: This study aims to understand the trend and central concept of domestic researches on laughter therapy. For the analysis, this study used total 72 theses verified by inputting the keyword 'laughter therapy' from 2007 to 2021. Research design, data and methodology: This study performed the development and analysis of keyword co-occurrence network, analyzed the types of researches through topic modeling, and verified the visualized word cloud and sociogram. The keyword data that was cleaned through preprocessing, was analyzed in the method of centrality analysis and topic modeling through the 1-mode matrix conversion process by using the NetMiner (version 4.4) Program. Results: The keywords that most appeared for last 14 years were laughter therapy, depression, the elderly, and stress. The five topics analyzed in thesis data from 2007 to 2021 were therapy, cognitive behavior, quality of life, stress, and the elderly. Conclusions: This study understood the flow and trend of research topics of domestic laughter therapy for last 14 years, and there should be continuous researches on laughter therapy, which reflects the flow of time in the future.

Cloud monitoring system for assembled beam bridge based on index of dynamic strain correlation coefficient

  • Zhao, Yiming;Dan, Danhui;Yan, Xingfei;Zhang, Kailong
    • Smart Structures and Systems
    • /
    • 제26권1호
    • /
    • pp.11-21
    • /
    • 2020
  • The hinge joint is the key to the overall cooperative working performance of the assembled beam bridge, and it is also the weakest part during the service period. This paper proposes a method for monitoring and evaluating the lateral cooperative working performance of fabricated beam bridges based on dynamic strain correlation coefficient indicator. This method is suitable for monitoring and evaluation of hinge joints status between prefabricated girders and overall cooperative working performance of bridge, without interruption of traffic and easy implementation. The remote cloud monitoring and diagnosis system was designed and implemented on a real assembled beam bridge. The algorithms of data preprocessing, online indicator extraction and status diagnosis were given, and the corresponding software platform and scientific computing environment for cloud operation were developed. Through the analysis of real bridge monitoring data, the effectiveness and accuracy of the method are proved and it can be used in the health monitoring system of such bridges.

딥러닝 기반 네트워크 침입탐지를 위한 데이터 전처리 방안 연구 (A Study of Data Preprocessing for Network Intrusion Detection based on Deep Learning)

  • 정기문
    • 한국컴퓨터정보학회:학술대회논문집
    • /
    • 한국컴퓨터정보학회 2018년도 제58차 하계학술대회논문집 26권2호
    • /
    • pp.165-166
    • /
    • 2018
  • 최근 딥러닝 기술이 발전함에 따라 이를 네트워크 침입탐지 분야에 적용하려는 연구가 활발히 이루어지고 있으며 이에 따라 대용량 네트워크 데이터에 대한 처리 방법이 주목받고 있다. 본 논문에서는 네트워크 데이터를 이미지화하는 전처리 방법을 제안한다. 네트워크 데이터를 세션단위로 처리하여 손실율을 줄이면서 딥러닝 알고리즘에 바로 적용할 수 있도록 정규화된 이미지로 변환하는 방법이다. 이를 통해 딥러닝 기술을 적용한 네트워크 정보보안 분야의 연구 활성화를 기대할 수 있다.

  • PDF

국지예보모델에서 고해상도 마이크로파 위성자료(MHS) 동화에 관한 연구 (A Study on the Assimilation of High-Resolution Microwave Humidity Sounder Data for Convective Scale Model at KMA)

  • 김혜영;이은희;이승우;이용희
    • 대기
    • /
    • 제28권2호
    • /
    • pp.163-174
    • /
    • 2018
  • In order to assimilate MHS satellite data into the convective scale model at KMA, ATOVS data are reprocessed to utilize the original high-resolution data. And then to improve the preprocessing experiments for cloud detection were performed and optimized to convective-scale model. The experiment which is land scattering index technique added to Observational Processing System to remove contaminated data showed the best result. The analysis fields with assimilation of MHS are verified against with ECMWF analysis fields and fit to other observations including Sonde, which shows improved results on relative humidity fields at sensitive level (850-300 hPa). As the relative humidity of upper troposphere increases, the bias and RMSE of geopotential height are decreased. This improved initial field has a very positive effect on the forecast performance of the model. According to improvement of model field, the Equitable Threat Score (ETS) of precipitation prediction of $1{\sim}20mm\;hr^{-1}$ was increased and this impact was maintained for 27 hours during experiment periods.

Cell Transmission Model 시뮬레이션을 기반으로 한 클라우드 환경 아래에서의 고속도로 교통 예측 및 최적 제어 시스템 개발 (Development of Traffic Prediction and Optimal Traffic Control System for Highway based on Cell Transmission Model in Cloud Environment)

  • 탁세현;여화수
    • 한국ITS학회 논문지
    • /
    • 제15권4호
    • /
    • pp.68-80
    • /
    • 2016
  • 자율주행 차량은 다양한 센서를 활용하여 사람과 유사한 수준으로 실시간 도로환경 변화를 인지, 환경 변화에 대한 적절한 판단 및 제어를 수행하여야 한다. 특히 영상센서는 차선인식 기능을 통해 주행방향 결정 및 차로이탈 방지 등 조향제어 수행을 위한 인지에 활용된다. 하지만 관련 성능기준은 ADAS(Advanced Driver Assistance System)와 연계된 '운전자 보조' 역할에 초점이 맞춰져, 자율주행시 요구되는 '주체적 상황 인지'를 위한 성능조건과 다를 것으로 판단된다. 본 연구에서는 자율주행시 차선인식 기능이 정상적으로 작동되지 않는 상황이 지속될 때 차량 진행방향과 도로 선형방향의 불일치에 따라 발생되는 횡방향 차로이탈을 차량의 이동 궤적을 기반하여 추정하고, 안전성 확보를 위한 차로이탈 허용 수준 및 영상센서 성능수준을 제시하였다. 분석 결과 승용차 조건에서 차선인식 기능이 1초 이상 연속적인 오작동을 일으킨다면 차로이탈에 의한 위험한 상황에 놓일 수 있는 것으로 나타났다. 따라서 자율주행 차량을 위한 차선인식 기능 평가 시 현재 기준보다 큰 횡방향 차로이탈상황에 대한 검토가 필요할 것으로 판단된다.