• Title/Summary/Keyword: Data collection and analysis system

Search Result 812, Processing Time 0.027 seconds

A Study on the Development of Advanced LOSA Method (진보된 LOSA 방법론 개발에 관한 연구 )

  • Jihun Choi
    • Journal of Advanced Navigation Technology
    • /
    • v.27 no.4
    • /
    • pp.351-355
    • /
    • 2023
  • The need for Advanced LOSA arises from the limitations and drawbacks of traditional LOSA. Amended LOSA aims to address some of the shortcomings of the original methodology and make it more effective and relevant to current aviation safety needs. Some of the key reasons for developing Advanced LOSA include Enhancing the scope, Improving data collection and analysis, Providing more targeted safety recommendations. First, Traditional LOSA mainly focuses on flight deck operations, but Advanced LOSA expands the scope to include other operational areas such as cabin operations, ground handling, and maintenance. Second, Advanced LOSA can build a Forecasting System that can predict the future through data collection and data analysis. Third, Advanced LOSA aims to provide more specific and targeted safety recommendations based on the Aviation data collection and Aviation data analysis. Overall, Advanced LOSA seeks to improve aviation safety by addressing the limitations of traditional LOSA and providing a more comprehensive and effective methodology for identifying and mitigating safety risks in aviation operations.

Designing Cost Effective Open Source System for Bigdata Analysis (빅데이터 분석을 위한 비용효과적 오픈 소스 시스템 설계)

  • Lee, Jong-Hwa;Lee, Hyun-Kyu
    • Knowledge Management Research
    • /
    • v.19 no.1
    • /
    • pp.119-132
    • /
    • 2018
  • Many advanced products and services are emerging in the market thanks to data-based technologies such as Internet (IoT), Big Data, and AI. The construction of a system for data processing under the IoT network environment is not simple in configuration, and has a lot of restrictions due to a high cost for constructing a high performance server environment. Therefore, in this paper, we will design a development environment for large data analysis computing platform using open source with low cost and practicality. Therefore, this study intends to implement a big data processing system using Raspberry Pi, an ultra-small PC environment, and open source API. This big data processing system includes building a portable server system, building a web server for web mining, developing Python IDE classes for crawling, and developing R Libraries for NLP and visualization. Through this research, we will develop a web environment that can control real-time data collection and analysis of web media in a mobile environment and present it as a curriculum for non-IT specialists.

Implementation of AIoT Edge Cluster System via Distributed Deep Learning Pipeline

  • Jeon, Sung-Ho;Lee, Cheol-Gyu;Lee, Jae-Deok;Kim, Bo-Seok;Kim, Joo-Man
    • International journal of advanced smart convergence
    • /
    • v.10 no.4
    • /
    • pp.278-288
    • /
    • 2021
  • Recently, IoT systems are cloud-based, so that continuous and large amounts of data collected from sensor nodes are processed in the data server through the cloud. However, in the centralized configuration of large-scale cloud computing, computational processing must be performed at a physical location where data collection and processing take place, and the need for edge computers to reduce the network load of the cloud system is gradually expanding. In this paper, a cluster system consisting of 6 inexpensive Raspberry Pi boards was constructed to perform fast data processing. And we propose "Kubernetes cluster system(KCS)" for processing large data collection and analysis by model distribution and data pipeline method. To compare the performance of this study, an ensemble model of deep learning was built, and the accuracy, processing performance, and processing time through the proposed KCS system and model distribution were compared and analyzed. As a result, the ensemble model was excellent in accuracy, but the KCS implemented as a data pipeline proved to be superior in processing speed..

SCADA System for Semiconductor Equipment Condition Monitoring (반도체 장비상태 모니터링을 위한 SCADA 시스템 구현)

  • Lee, Youn Ji;Yun, Hak Jae;Park, Hyoeun;Hong, Sang Jeen
    • Journal of the Semiconductor & Display Technology
    • /
    • v.18 no.4
    • /
    • pp.92-95
    • /
    • 2019
  • Automation control and the data for control of industrial equipment for the diagnosis and prediction is a key to success in the 4th industrial revolution. It increases process efficiency and productivity through data collection, realtime monitoring, and the data analysis. However, university and research environment are still suffering from logging the data in manual way, and we occasionally loss the equipment data logging due to the lack of automatic data logging system. State variable presents the current condition of the equipment operation which is closely related to process result, and it is valuable to monitor and analyze the data for the equipment health monitoring. In this paper, we demonstrate the collection of equipment state variable data via programmable logic controller (PLC) and the visualization of the collected data over the Web access supervisory control and data acquisition (SCADA). Test vehicle for the implementation of the suggested SCADA system is a relay switched physical vapor deposition system in the university environment.

Development of Cloud based Data Collection and Analysis for Manufacturing (클라우드 기반의 생산설비 데이터 수집 및 분석 시스템 개발)

  • Young-Dong Lee
    • Journal of the Institute of Convergence Signal Processing
    • /
    • v.23 no.4
    • /
    • pp.216-221
    • /
    • 2022
  • The 4th industrial revolution is accelerating the transition to digital innovation in various aspects of our daily lives, and efforts for manufacturing innovation are continuing in the manufacturing industry, such as smart factories. The 4th industrial revolution technology in manufacturing can be used based on AI, big data, IoT, cloud, and robots. Through this, it is required to develop a technology to establish a production facility data collection and analysis system that has evolved from the existing automation and to find the cause of defects and minimize the defect rate. In this paper, we implemented a system that collects power, environment, and status data from production facility sites through IoT devices, quantifies them in real-time in a cloud computing environment, and displays them in the form of MQTT-based real-time infographics using widgets. The real-time sensor data transmitted from the IoT device is stored to the cloud server through a Rest API method. In addition, the administrator could remotely monitor the data on the dashboard and analyze it hourly and daily.

Distributed System Architecture Modeling of a Performance Monitoring and Reporting Tool (분산 시스템의 성능 모니터링과 레포팅 툴의 아키텍처 모델링)

  • Kim, Ki;Choi, Eun-Mi
    • Journal of the Korea Society for Simulation
    • /
    • v.12 no.3
    • /
    • pp.69-81
    • /
    • 2003
  • To manage a cluster of distributed server systems, a number of management aspects should be considered in terms of configuration management, fault management, performance management, and user management. System performance monitoring and reporting take an important role for performance and fault management. In this paper, we present distributed system architecture modeling of a performance monitoring and reporting tool. Modeling architecture of four subsystems are introduced: node agent, data collection, performance management & report, and DB schema. The performance-related information collected from distributed servers are categorized into performance counters, event data for system status changes, service quality, and system configuration data. In order to analyze those performance information, we use a number of ways to evaluate data corelation. By using some results from a real site of a company and from simulation of artificial workload, we show the example of performance collection and analysis. Since our report tool detects system fault or node component failure and analyzes performances through resource usage and service quality, we are able to provide information for server load balancing, in short term view, and the cause of system faults and decision for system scale-out and scale-up, in long term view.

  • PDF

A Case Study on Big Data Analysis Systems for Policy Proposals of Engineering Education (공학교육 정책제안을 위한 빅데이터 분석 시스템 사례 분석 연구)

  • Kim, JaeHee;Yoo, Mina
    • Journal of Engineering Education Research
    • /
    • v.22 no.5
    • /
    • pp.37-48
    • /
    • 2019
  • The government has tried to develop a platform for systematically collecting and managing engineering education data for policy proposals. However, there have been few cases of big data analysis platform for policy proposals in engineering education, and it is difficult to determine the major function of the platform, the purpose of using big data, and the method of data collection. This study aims to collect the cases of big data analysis systems for the development of a big data system for educational policy proposals, and to conduct a study to analyze cases using the analysis frame of key elements to consider in developing a big data analysis platform. In order to analyze the case of big data system for engineering education policy proposals, 24 systems collecting and managing big data were selected. The analysis framework was developed based on literature reviews and the results of the case analysis were presented. The results of this study are expected to provide from macro-level such as what functions the platform should perform in developing a big data system and how to collect data, what analysis techniques should be adopted, and how to visualize the data analysis results.

A Study on the Real-Time Monitoring System of Wind Power in Jeju (제주지역 풍력발전량 실시간 감시 시스템 구축에 관한 연구)

  • Kim, Kyoung-Bo;Yang, Kyung-Bu;Park, Yun-Ho;Mun, Chang-Eun;Park, Jeong-Keun;Huh, Jong-Chul
    • Journal of the Korean Solar Energy Society
    • /
    • v.30 no.3
    • /
    • pp.25-32
    • /
    • 2010
  • A real-time monitoring system was developed for transfer, receive, backup and analysis of wind power data at three wind farm(Hang won, Hankyung and Sung san) in Jeju. For this monitoring system a communication system analysis, a collection of data and transmission module development, data base construction and data analysis and management module was developed, respectively. These modules deal with mechanical, electrical and environmental problem. Especially, time series graphic is supported by the data analysis and management module automatically. The time series graphic make easier to raw data analysis. Also, the real-time monitoring system is connected with wind power forecasting system through internet web for data transfer to wind power forecasting system's data base.

A Study on Big Data Processing Technology Based on Open Source for Expansion of LIMS (실험실정보관리시스템의 확장을 위한 오픈 소스 기반의 빅데이터 처리 기술에 관한 연구)

  • Kim, Soon-Gohn
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.14 no.2
    • /
    • pp.161-167
    • /
    • 2021
  • Laboratory Information Management System(LIMS) is a centralized database for storing, processing, retrieving, and analyzing laboratory data, and refers to a computer system or system specially designed for laboratories performing inspection, analysis, and testing tasks. In particular, LIMS is equipped with a function to support the operation of the laboratory, and it requires workflow management or data tracking support. In this paper, we collect data on websites and various channels using crawling technology, one of the automated big data collection technologies for the operation of the laboratory. Among the collected test methods and contents, useful test methods and contents useful that the tester can utilize are recommended. In addition, we implement a complementary LIMS platform capable of verifying the collection channel by managing the feedback.

Changes in Measuring Methods of Walking Behavior and the Potentials of Mobile Big Data in Recent Walkability Researches (보행행태조사방법론의 변화와 모바일 빅데이터의 가능성 진단 연구 - 보행환경 분석연구 최근 사례를 중심으로 -)

  • Kim, Hyunju;Park, So-Hyun;Lee, Sunjae
    • Journal of the Architectural Institute of Korea Planning & Design
    • /
    • v.35 no.1
    • /
    • pp.19-28
    • /
    • 2019
  • The purpose of this study is to evaluate the walking behavior analysis methodology used in the previous studies, paying attention to the demand for empirical data collecting for urban and neighborhood planning. The preceding researches are divided into (1)Recording, (2) Surveys, (3)Statistical data, (4)Global positioning system (GPS) devices, and (5)Mobile Big Data analysis. Next, we analyze the precedent research and identify the changes of the walkability research. (1)being required empirical data on the actual walking and moving patterns of people, (2)beginning to be measured micro-walking behaviors such as actual route, walking facilities, detour, walking area. In addition, according to the trend of research, it is analyzed that the use of GPS device and the mobile big data are newly emerged. Finally, we analyze pedestrian data based on mobile big data in terms of 'application' and distinguishing it from existing survey methodology. We present the possibility of mobile big data. (1)Improvement of human, temporal and spatial constraints of data collection, (2)Improvement of inaccuracy of collected data, (3)Improvement of subjective intervention in data collection and preprocessing, (4)Expandability of walking environment research.