• Title/Summary/Keyword: Big data analysis system

Search Result 1,040, Processing Time 0.026 seconds

Development of Data Visualization Tools for Land-Based Fish Farm Big Data Analysis System (육상 양식장 빅데이터 분석 시스템 개발을 위한 데이터 시각화 도구 개발)

  • Seoung-Bin Ye;Jeong-Seon Park;Hyi-Thaek Ceong;Soon-Hee Han
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.19 no.4
    • /
    • pp.763-770
    • /
    • 2024
  • Currently, land-based fish farms utilizing seawater have introduced and are utilizing various equipment such as real-time water quality monitoring systems, facility automation systems, and automated dissolved oxygen supply devices. Furthermore, data collected from various equipment in these fish farms produce structured and unstructured big data related to water quality environment, facility operations, and workplace visual information. The big data generated in the operational environment of fish farms aims to improve operational and production efficiency through the development and application of various methods. This study aims to develop a system for effectively analyzing and visualizing big data produced from land-based fish farms. It proposes a data visualization process suitable for use in a fish farm big data analysis system, develops big data visualization tools, and compares the results. Additionally, it presents intuitive visualization models for exploring and comparing big data with time-series characteristics.

A Container Orchestration System for Process Workloads

  • Jong-Sub Lee;Seok-Jae Moon
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.15 no.4
    • /
    • pp.270-278
    • /
    • 2023
  • We propose a container orchestration system for process workloads that combines the potential of big data and machine learning technologies to integrate enterprise process-centric workloads. This proposed system analyzes big data generated from industrial automation to identify hidden patterns and build a machine learning prediction model. For each machine learning case, training data is loaded into a data store and preprocessed for model training. In the next step, you can use the training data to select and apply an appropriate model. Then evaluate the model using the following test data: This step is called model construction and can be performed in a deployment framework. Additionally, a visual hierarchy is constructed to display prediction results and facilitate big data analysis. In order to implement parallel computing of PCA in the proposed system, several virtual systems were implemented to build the cluster required for the big data cluster. The implementation for evaluation and analysis built the necessary clusters by creating multiple virtual machines in a big data cluster to implement parallel computation of PCA. The proposed system is modeled as layers of individual components that can be connected together. The advantage of a system is that components can be added, replaced, or reused without affecting the rest of the system.

Comparison of Sentiment Analysis from Large Twitter Datasets by Naïve Bayes and Natural Language Processing Methods

  • Back, Bong-Hyun;Ha, Il-Kyu
    • Journal of information and communication convergence engineering
    • /
    • v.17 no.4
    • /
    • pp.239-245
    • /
    • 2019
  • Recently, effort to obtain various information from the vast amount of social network services (SNS) big data generated in daily life has expanded. SNS big data comprise sentences classified as unstructured data, which complicates data processing. As the amount of processing increases, a rapid processing technique is required to extract valuable information from SNS big data. We herein propose a system that can extract human sentiment information from vast amounts of SNS unstructured big data using the naïve Bayes algorithm and natural language processing (NLP). Furthermore, we analyze the effectiveness of the proposed method through various experiments. Based on sentiment accuracy analysis, experimental results showed that the machine learning method using the naïve Bayes algorithm afforded a 63.5% accuracy, which was lower than that yielded by the NLP method. However, based on data processing speed analysis, the machine learning method by the naïve Bayes algorithm demonstrated a processing performance that was approximately 5.4 times higher than that by the NLP method.

A Study on Efficient Building Energy Management System Based on Big Data

  • Chang, Young-Hyun;Ko, Chang-Bae
    • International journal of advanced smart convergence
    • /
    • v.8 no.1
    • /
    • pp.82-86
    • /
    • 2019
  • We aim to use public data different from the remote BEMS energy diagnostics technology and already established and then switch the conventional operation environment to a big-data-based integrated management environment to operate and build a building energy management environment of maximized efficiency. In Step 1, various network management environments of the system integrated with a big data platform and the BEMS management system are used to collect logs created in various types of data by means of the big data platform. In Step 2, the collected data are stored in the HDFS (Hadoop Distributed File System) to manage the data in real time about internal and external changes on the basis of integration analysis, for example, relations and interrelation for automatic efficient management.

Design and Implementation of Big Data Cluster for Indoor Environment Monitering (실내 환경 모니터링을 위한 빅데이터 클러스터 설계 및 구현)

  • Jeon, Byoungchan;Go, Mingu
    • Journal of Korea Society of Digital Industry and Information Management
    • /
    • v.13 no.2
    • /
    • pp.77-85
    • /
    • 2017
  • Due to the expansion of accommodation space caused by increase of population along with lifestyle changes, most of people spend their time indoor except for the travel time. Because of this, environmental change of indoor is very important, and it affects people's health and economy in resources. But, most of people don't acknowledge the importance of indoor environment. Thus, monitoring system for sustaining and managing indoor environment systematically is needed, and big data clusters should be used in order to save and manage numerous sensor data collected from many spaces. In this paper, we design a big data cluster for the indoor environment monitoring in order to store the sensor data and monitor unit of the huge building Implementation design big data cluster-based system for the analysis, and a distributed file system and building a Hadoop, HBase for big data processing. Also, various sensor data is saved for collection, and effective indoor environment management and health enhancement through monitoring is expected.

Big data platform for health monitoring systems of multiple bridges

  • Wang, Manya;Ding, Youliang;Wan, Chunfeng;Zhao, Hanwei
    • Structural Monitoring and Maintenance
    • /
    • v.7 no.4
    • /
    • pp.345-365
    • /
    • 2020
  • At present, many machine leaning and data mining methods are used for analyzing and predicting structural response characteristics. However, the platform that combines big data analysis methods with online and offline analysis modules has not been used in actual projects. This work is dedicated to developing a multifunctional Hadoop-Spark big data platform for bridges to monitor and evaluate the serviceability based on structural health monitoring system. It realizes rapid processing, analysis and storage of collected health monitoring data. The platform contains offline computing and online analysis modules, using Hadoop-Spark environment. Hadoop provides the overall framework and storage subsystem for big data platform, while Spark is used for online computing. Finally, the big data Hadoop-Spark platform computational performance is verified through several actual analysis tasks. Experiments show the Hadoop-Spark big data platform has good fault tolerance, scalability and online analysis performance. It can meet the daily analysis requirements of 5s/time for one bridge and 40s/time for 100 bridges.

A Study on Construction of Crime Prevention System using Big Data in Korea (한국에서 빅데이터를 활용한 범죄예방시스템 구축을 위한 연구)

  • Kim, SungJun
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.17 no.5
    • /
    • pp.217-221
    • /
    • 2017
  • Proactive prevention is important for crime. Past crimes have focused on coping after death and punishing them. But with Big Data technology, crime can be prevented spontaneously. Big data can predict the behavior of criminals or potential criminals. This article discusses how to build a big data system for crime prevention. Specifically, it deals with the way to combine unstructured data of big data with basic form data, and as a result, designs crime prevention system. Through this study, it is expected that the possibility of using big data for crime prevention is described through fingerprints, and it is expected to help crime prevention program and research in future.

Big Data Analysis for Public Libraries Utilizing Big Data Platform: A Case Study of Daejeon Hanbat Library (도서관 빅데이터 플랫폼을 활용한 공공도서관 빅데이터 분석 연구: 대전한밭도서관을 중심으로)

  • On, Jeongmee;Park, Sung Hee
    • Journal of the Korean Society for information Management
    • /
    • v.37 no.3
    • /
    • pp.25-50
    • /
    • 2020
  • Since big data platform services for the public library began January 1, 2016, libraries have used big data to improve their work performance. This paper aims to examine the use cases of library big data and attempts to draw improvement plan to improve the effectiveness of library big data. For this purpose, first, we examine big data used while utilizing the library big data platform, the usage pattern of big data and services/policies drawn by big data analysis. Next, the limitations and advantages of the library big data platform are examined by comparing the data analysis of the integrated library management system (ILUS) currently used in public libraries and data analysis through the library big data platform. As a result of case analysis, big data usage patterns were found program planning and execution, collection, collection, and other types, and services/policies were summarized as customizing bookshelf themes for the book curation and reading promotion program, increasing collection utilization, and building a collection based on special topics. and disclosure of loan status data. As a result of the comparative analysis, ILUS is specialized in statistical analysis of library collection unit, and the big data platform enables selective and flexible analysis according to various attributes (age, gender, region, time of loan, etc.) reducing analysis time. Finally, the limitations revealed in case analysis and comparative analysis are summarized and suggestions for improvement are presented.

Design and Implementation of a Food Price Information Analysis System Based on Public Big Data (공공 빅데이터 기반의 식품 가격 정보 분석 시스템의 설계 및 구현)

  • Lim, Jongtae;Lee, Hyeonbyeong;Bok, Kyoungsoo;Yoo, Jaesoo
    • The Journal of the Korea Contents Association
    • /
    • v.22 no.7
    • /
    • pp.10-17
    • /
    • 2022
  • Recently, with the issue of the 4th Industrial Revolution, many services using big data have been developed. Accordingly, studies have been conducting to utilize public data, which is considered as the most valuable data among big data. In this paper, we design and implement a food price information analysis system based on public big data. The proposed system analyzes the collected food price-related data in various forms from various sources and classifies them according to characteristics. In addition, the proposed system analyzes the factors affecting the price of food through big data analysis techniques and uses them as data to predict the price of food in the near future. Finally, the proposed system provides the user with the analyzed results through data visualization.

A Business Application of the Business Intelligence and the Big Data Analytics (비즈니스 인텔리전스와 빅데이터 분석의 비즈니스 응용)

  • Lee, Ki-Kwang;Kim, Tae-Hwan
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • v.42 no.4
    • /
    • pp.84-90
    • /
    • 2019
  • Lately, there have been tremendous shifts in the business technology landscape. Advances in cloud technology and mobile applications have enabled businesses and IT users to interact in entirely new ways. One of the most rapidly growing technologies in this sphere is business intelligence, and associated concepts such as big data and data mining. BI is the collection of systems and products that have been implemented in various business practices, but not the information derived from the systems and products. On the other hand, big data has come to mean various things to different people. When comparing big data vs business intelligence, some people use the term big data when referring to the size of data, while others use the term in reference to specific approaches to analytics. As the volume of data grows, businesses will also ask more questions to better understand the data analytics process. As a result, the analysis team will have to keep up with the rising demands on the infrastructure that supports analytics applications brought by these additional requirements. It's also a good way to ascertain if we have built a valuable analysis system. Thus, Business Intelligence and Big Data technology can be adapted to the business' changing requirements, if they prove to be highly valuable to business environment.