• Title/Summary/Keyword: Big Data Platform

Search Result 516, Processing Time 0.024 seconds

Building a Big Data Platform Using Real-time Wearable Devices and Cases of Safety Accidents in KOREA

  • LEE, Ki Seok;CHOI, Youngjin;LEE, Kyung-cheun;SHIN, Yoonseok;YOO, Wi Sung
    • International conference on construction engineering and project management
    • /
    • 2022.06a
    • /
    • pp.375-381
    • /
    • 2022
  • Safety accidents are of concern during construction projects, even given the recent innovations in digital technologies. These projects remain focused on overcoming specific and limited applications on construction sites. For this reason, the development of an inclusive safety management system has become crucial. This study aims to build a Big Data platform to inform decisions on how to proactively eliminate worker hazards on construction sites. The platform consists of about 100,000 real records and a real-time monitored database featuring various safety indices, such as workers' altitudes, heart rates, and fatigability during construction, which are determined through various wearable devices. The data types are customized and integrated by a research team in accordance with the characteristics of a specific project using hypertext transfer protocol (HTTP). The results can be helpful as efficient tools to ensure successful safety management in complex construction situations. This study is expected to provide three significant contributions to the field, including real-time fatigability analysis and tracking of workers on-site; providing early GPS-based warnings to workers who might be accessing dangerous spaces or places; and monitoring the workers' health indices, based on details from 100,000 cases.

  • PDF

Big Data Astronomy : Let's "PySpark" the Universe (빅데이터 천문학 : PySpark를 이용한 천문자료 분석)

  • Hong, Sungryong
    • The Bulletin of The Korean Astronomical Society
    • /
    • v.43 no.1
    • /
    • pp.63.1-63.1
    • /
    • 2018
  • The modern large-scale surveys and state-of-the-art cosmological simulations produce various kinds of big data composed of millions and billions of galaxies. Inevitably, we need to adopt modern Big Data platforms to properly handle such large-scale data sets. In my talk, I will briefly introduce the de facto standard of modern Big Data platform, Apache Spark, and present some examples to demonstrate how Apache Spark can be utilized for solving data-driven astronomical problems.

  • PDF

Big Data Platform for Utilizing and Analyzing Real-Time Sensing Information in Industrial Sites (산업현장 실시간 센싱정보 활용/분석을 위한 빅데이터 플랫폼)

  • Lee, Yonghwan;Suh, Jinhyung
    • Journal of Creative Information Culture
    • /
    • v.6 no.1
    • /
    • pp.15-21
    • /
    • 2020
  • In order to utilize big data in general industrial sites, the structured big data collected from facilities, processes, and environments of industrial sites must first be processed and stored, and in the case of unstructured data, it must be stored as unstructured data or converted into structured data and stored in a database. In this paper, we study a method of collecting big data based on open IoT standards that can converge and utilize measurement information, environmental information of industrial sites to collect big data. The platform for collecting big data proposed in this paper is capable of collecting, processing, and storing big data at industrial sites to process real-time sensing information. For processing and analyzing data according to the purpose of the stored industrial, various big data technologies also can be applied.

Design and Implementation of a Big Data Analytics Framework based on Cargo DTG Data for Crackdown on Overloaded Trucks

  • Kim, Bum-Soo
    • Journal of the Korea Society of Computer and Information
    • /
    • v.24 no.12
    • /
    • pp.67-74
    • /
    • 2019
  • In this paper, we design and implement an analytics platform based on bulk cargo DTG data for crackdown on overloaded trucks. DTG(digital tachograph) is a device that stores the driving record in real time; that is, it is a device that records the vehicle driving related data such as GPS, speed, RPM, braking, and moving distance of the vehicle in one second unit. The fast processing of DTG data is essential for finding vehicle driving patterns and analytics. In particular, a big data analytics platform is required for preprocessing and converting large amounts of DTG data. In this paper, we implement a big data analytics framework based on cargo DTG data using Spark, which is an open source-based big data framework for crackdown on overloaded trucks. As the result of implementation, our proposed platform converts real large cargo DTG data sets into GIS data, and these are visualized by a map. It also recommends crackdown points.

The Method of Failure Management through Big Data Flow Management in Platform Service Operation Environment (플랫폼 서비스 운용환경에서 빅데이터 플로우 관리를 통한 장애 상황 관리 방법)

  • Baik, Song-Ki;Lim, Jae-Hyun
    • Journal of Convergence for Information Technology
    • /
    • v.11 no.5
    • /
    • pp.23-29
    • /
    • 2021
  • Recently, a situation in which a specific content service is impossible worldwide has occurred due to a failure of the platform service and a significant social and economic problem has been caused in the global service market. In order to secure the stability of platform services, intelligent platform operation management is required. In this study, big data flow management(BDFM) and implementation method were proposed to quickly detect to abnormal service status in the platform operation environment. As a result of analyzing, BDFM technique improved the characteristics of abnormal failure detection by more than 30% compared to the traditional NMS. The big data flow management method has the advantage of being able to quickly detect platform system failures and abnormal service conditions, and it is expected that when connected with AI-based technology, platform management is performed intelligently and the ability to prevent and preserve failures can be greatly improved.

Semantic-based Mashup Platform for Contents Convergence

  • Yongju Lee;Hongzhou Duan;Yuxiang Sun
    • International journal of advanced smart convergence
    • /
    • v.12 no.2
    • /
    • pp.34-46
    • /
    • 2023
  • A growing number of large scale knowledge graphs raises several issues how knowledge graph data can be organized, discovered, and integrated efficiently. We present a novel semantic-based mashup platform for contents convergence which consists of acquisition, RDF storage, ontology learning, and mashup subsystems. This platform servers a basis for developing other more sophisticated applications required in the area of knowledge big data. Moreover, this paper proposes an entity matching method using graph convolutional network techniques as a preliminary work for automatic classification and discovery on knowledge big data. Using real DBP15K and SRPRS datasets, the performance of our method is compared with some existing entity matching methods. The experimental results show that the proposed method outperforms existing methods due to its ability to increase accuracy and reduce training time.

Analyzing XR(eXtended Reality) Trends in South Korea: Opportunities and Challenges

  • Sukchang Lee
    • International Journal of Advanced Culture Technology
    • /
    • v.12 no.2
    • /
    • pp.221-226
    • /
    • 2024
  • This study used text mining, a big data analysis technique, to explore XR trends in South Korea. For this research, I utilized a big data platform called BigKinds. I collected data focusing on the keyword 'XR', spanning approximately 14 years from 2010 to 2024. The gathered data underwent a cleansing process and was analyzed in three ways: keyword trend analysis, relational analysis, and word cloud. The analysis identified the emergence and most active discussion periods of XR, with XR devices and manufacturers emerging as key keywords.

Knowledge Extractions, Visualizations, and Inference from the big Data in Healthcare and Medical

  • Kim, Jin Sung
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • v.23 no.5
    • /
    • pp.400-405
    • /
    • 2013
  • The purpose of this study is to develop a composite platform for knowledge extractions, visualizations, and inference. Generally, the big data sets were frequently used in the healthcare and medical area. To help the knowledge managers/users working in the field, this study is focused on knowledge management (KM) based on Data Mining (DM), Knowledge Distribution Map (KDM), Decision Tree (DT), RDBMS, and SQL-inference. The proposed mechanism is composed of five key processes. Firstly, in Knowledge Parsing, it extracts logical rules from a big data set by using DM technology. Then it transforms the rules into RDB tables. Secondly, through Knowledge Maintenance, it refines and manages the knowledge to be ready for the computing of knowledge distributions. Thirdly, in Knowledge Distribution process, we can see the knowledge distributions by using the DT mechanism.Fourthly, in Knowledge Hierarchy, the platform shows the hierarchy of the knowledge. Finally, in Inference, it deduce the conclusions by using the given facts and data.This approach presents the advantages of diversity in knowledge representations and inference to improve the quality of computer-based medical diagnosis.

Implementation of Cloud-Based Artificial Intelligence Education Platform (클라우드 기반 인공지능 교육 플랫폼 구현)

  • Wi, Woo-Jin;Moon, Hyung-Jin;Ryu, Gab-Sang
    • Journal of Internet of Things and Convergence
    • /
    • v.8 no.6
    • /
    • pp.85-92
    • /
    • 2022
  • Demand for big data analysis and AI developers is increasing, but there is a lack of an education base to supply them. In this paper, by developing a cloud-based artificial intelligence education platform, the goal was to establish an environment in which practical practical training can be efficiently learned at low cost at educational institutions and IT companies. The development of the education platform was carried out by planning scenarios for each user, architecture design, screen design, implementation of development functions, and hardware construction. This training platform consists of a containerized workload, service management platform, lecture and development platform for instructors and students, and secured cloud stability through real-time alarm system and age test, CI/CD development environment, and reliability through docker image distribution. The development of this education platform is expected to expand opportunities to enter new businesses in the education field and contribute to fostering working-level human resources in the AI and big data fields.

A Study on the Perception of Fashion Platforms and Fashion Smart Factories using Big Data Analysis (빅데이터 분석을 이용한 패션 플랫폼과 패션 스마트 팩토리에 대한 인식 연구)

  • Song, Eun-young
    • Fashion & Textile Research Journal
    • /
    • v.23 no.6
    • /
    • pp.799-809
    • /
    • 2021
  • This study aimed to grasp the perceptions and trends in fashion platforms and fashion smart factories using big data analysis. As a research method, big data analysis, fashion platform, and smart factory were identified through literature and prior studies, and text mining analysis and network analysis were performed after collecting text from the web environment between April 2019 and April 2021. After data purification with Textom, the words of fashion platform (1,0591 pieces) and fashion smart factory (9750 pieces) were used for analysis. Key words were derived, the frequency of appearance was calculated, and the results were visualized in word cloud and N-gram. The top 70 words by frequency of appearance were used to generate a matrix, structural equivalence analysis was performed, and the results were displayed using network visualization and dendrograms. The collected data revealed that smart factory had high social issues, but consumer interest and academic research were insufficient, and the amount and frequency of related words on the fashion platform were both high. As a result of structural equalization analysis, it was found that fashion platforms with strong connectivity between clusters are creating new competitiveness with service platforms that add sharing, manufacturing, and curation functions, and fashion smart factories can expect future value to grow together, according to digital technology innovation and platforms. This study can serve as a foundation for future research topics related to fashion platforms and smart factories.