• Title/Summary/Keyword: Big data Processing

Search Result 1,063, Processing Time 0.026 seconds

Study on the Big Data Platform Construction of Fisheries (수산업 빅데이터 플랫폼 구축 방안에 대한 연구)

  • Choi, Joowon;Jung, Jaewook;Kim, Youngae;Shin, Yongtae
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.9 no.8
    • /
    • pp.181-188
    • /
    • 2020
  • The fisheries industry is rapidly shifting from a traditional fishery to aquaculture paradigm and it faces various problems such as depletion of fishery resources and aging of fishing villages. We need the establishment of a fisheries big data platform that includes both the data of the central and surrounding industries of the fisheries industry for enhancement of establishment of a fisheries, 6th industrialization of fishing villages, establishment of related technical standards, and discovery of the new industries to overcome this. Data center agencies should collect, link, and pre-processing, and the platform organizer should create a water industry data virtuous circle through the establishment, operation, and data market of big data platforms to help overcome the current crisis, secure smart fisheries hegemony, and use it as a key to value transfer. Through this study, I would like to propose a policy and technical big data platform construction plan to successfully promote it.

Automatic Generation of Issue Analysis Report Based on Social Big Data Mining (소셜 빅데이터 마이닝 기반 이슈 분석보고서 자동 생성)

  • Heo, Jeong;Lee, Chung Hee;Oh, Hyo Jung;Yoon, Yeo Chan;Kim, Hyun Ki;Jo, Yo Han;Ock, Cheol Young
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.3 no.12
    • /
    • pp.553-564
    • /
    • 2014
  • In this paper, we propose the system for automatic generation of issue analysis report based on social big data mining, with the purpose of resolving three problems of the previous technologies in a social media analysis and analytic report generation. Three problems are the isolation of analysis, the subjectivity of experts and the closure of information attributable to a high price. The system is comprised of the natural language query analysis, the issue analysis, the social big data analysis, the social big data correlation analysis and the automatic report generation. For the evaluation of report usefulness, we used a Likert scale and made two experts of big data analysis evaluate. The result shows that the quality of report is comparatively useful and reliable. Because of a low price of the report generation, the correlation analysis of social big data and the objectivity of social big data analysis, the proposed system will lead us to the popularization of social big data analysis.

Aircraft Recognition from Remote Sensing Images Based on Machine Vision

  • Chen, Lu;Zhou, Liming;Liu, Jinming
    • Journal of Information Processing Systems
    • /
    • v.16 no.4
    • /
    • pp.795-808
    • /
    • 2020
  • Due to the poor evaluation indexes such as detection accuracy and recall rate when Yolov3 network detects aircraft in remote sensing images, in this paper, we propose a remote sensing image aircraft detection method based on machine vision. In order to improve the target detection effect, the Inception module was introduced into the Yolov3 network structure, and then the data set was cluster analyzed using the k-means algorithm. In order to obtain the best aircraft detection model, on the basis of our proposed method, we adjusted the network parameters in the pre-training model and improved the resolution of the input image. Finally, our method adopted multi-scale training model. In this paper, we used remote sensing aircraft dataset of RSOD-Dataset to do experiments, and finally proved that our method improved some evaluation indicators. The experiment of this paper proves that our method also has good detection and recognition ability in other ground objects.

A Stochastic Model for Virtual Data Generation of Crack Patterns in the Ceramics Manufacturing Process

  • Park, Youngho;Hyun, Sangil;Hong, Youn-Woo
    • Journal of the Korean Ceramic Society
    • /
    • v.56 no.6
    • /
    • pp.596-600
    • /
    • 2019
  • Artificial intelligence with a sufficient amount of realistic big data in certain applications has been demonstrated to play an important role in designing new materials or in manufacturing high-quality products. To reduce cracks in ceramic products using machine learning, it is desirable to utilize big data in recently developed data-driven optimization schemes. However, there is insufficient big data for ceramic processes. Therefore, we developed a numerical algorithm to make "virtual" manufacturing data sets using indirect methods such as computer simulations and image processing. In this study, a numerical algorithm based on the random walk was demonstrated to generate images of cracks by adjusting the conditions of the random walk process such as the number of steps, changes in direction, and the number of cracks.

The Creation and Placement of VMs and Tasks in Virtualized Hadoop Cluster Environments

  • Kim, Tae-Won;Chung, Hae-jin;Kim, Joon-Mo
    • Journal of Korea Multimedia Society
    • /
    • v.15 no.12
    • /
    • pp.1499-1505
    • /
    • 2012
  • Recently, the distributed processing system for big data has been actively investigated owing to the development of high speed network and storage technologies. In addition, virtual system that can provide efficient use of system resources through the consolidation of servers has been increasingly recognized. But, when we configure distributed processing system for big data in virtual machine environments, many problems occur. In this paper, we did an experiment on the optimization of I/O bandwidth according to the creation and placement of VMs and tasks with composing Hadoop cluster in virtual environments and evaluated the results of an experiment. These results conducted by this paper will be used in the study on the development of Hadoop Scheduler supporting I/O bandwidth balancing in virtual environments.

Development of Solar Power Output Prediction Method using Big Data Processing Technic (태양광 발전량 예측을 위한 빅데이터 처리 방법 개발)

  • Jung, Jae Cheon;Song, Chi Sung
    • Journal of the Korean Society of Systems Engineering
    • /
    • v.16 no.1
    • /
    • pp.58-67
    • /
    • 2020
  • A big data processing method to predict solar power generation using systems engineering approach is developed in this work. For developing analytical method, linear model (LM), support vector machine (SVN), and artificial neural network (ANN) technique are chosen. As evaluation indices, the cross-correlation and the mean square root of prediction error (RMSEP) are used. From multi-variable comparison test, it was found that ANN methodology provides the highest correlation and the lowest RMSEP.

Efficient Complex Event Processing Scheme through Similar Operation Processing in Duplicate Events (중복 이벤트 유사 연산 처리를 통한 효율적인 복합 이벤트 처리 기법)

  • Kim, Daeyun;Kim, Byounghoon;Ko, Geonsik;Noh, Yeonwoo;Choi, Dojin;Lim, Jongtae;Bok, Kyoungsoo;Yoo, Jaesoo
    • Proceedings of the Korea Contents Association Conference
    • /
    • 2016.05a
    • /
    • pp.59-60
    • /
    • 2016
  • 사물통신 기기의 발달로 다양한 응용에서 대용량의 스트림 데이터의 실시간 복합 이벤트 처리 기법에 대한 중요성이 증가되고 있다. 본 논문에서는 유사 연산 처리 비용을 감소시키기 위한 다수의 복합 이벤트 처리 기법을 제안한다. 제안하는 기법은 다수의 복합 이벤트를 처리하기 위한 연산자를 그래프로 표현하고 중복적인 연산을 감소시킨다.

  • PDF

A Study on the Analysis System for Determination of Separation of Liberal Arts Evaluation between Majors and Nonmajors (전공자와 비전공자 간의 교양과목 평가 분리 여부 결정을 위한 분석 시스템에 관한 연구)

  • Oh, DaSom;Choi, BoAh;Kim, Joo-Eun;Lee, JongHyuk
    • Annual Conference of KIPS
    • /
    • 2019.10a
    • /
    • pp.129-132
    • /
    • 2019
  • 대학기관연구는 대학에서 이루어지는 다양한 의사결정을 체계적으로 지원하는 활동이다. 본 대학에서는 최근에 전공자와 비전공자 간의 교양과목 평가 분리에 대한 제도를 마련하였으나 이 결정을 교수자의 재량에 맡겨 놓아 근거 데이터 없이 교수자가 평가 분리 여부를 실제로 결정하기에는 어려운 실정이다. 이에 본 논문은 이전 학기의 성적 데이터를 기반으로 통계 분석한 결과와 이에 대한 시각화를 제공하여 교수자가 조금 더 쉽게 평가 분리 여부를 결정할 수 있도록 분석 시스템을 제안하고 실제 교양과목에 대한 분석 결과를 예시를 통해 보여준다.

A Study on High-speed Synchronization of the PON-based Blockchain (PON 기반 블록체인의 고속 동기화 연구)

  • Kim, Dong-Oh;Oh, Jin-Tae;Kim, Ki-Young
    • Annual Conference of KIPS
    • /
    • 2021.11a
    • /
    • pp.320-321
    • /
    • 2021
  • 블록체인은 모든 참여자가 동일한 원장을 유지하는 분산 원장 기술로써, 신규로 참여하는 블록체인 노드는 원장을 동일하게 유지하기 위한 동기화 절차를 거쳐야 한다. 일반적으로, 동기화는 블록체인 상의 모든 블록을 순차적으로 적용하는 과정을 거처야 함으로 많은 시간이 걸리게 된다. 본 논문에서는 ETRI에서 자체 개발한 PON 기반 블록체인에서 동기화 성능을 개선하기 위해 비잔틴 환경에서 병렬적으로 동기화 요청하는 고속 병렬 동기화 모드와 최신 상태만 동기화하는 최신 상태동기화 모드를 개발하였다. 성능 평가 결과 100,000 개 블록 동기화시 고속 병렬 동기화 모드가 기본 동기화 대비 5 배, 최신 상태 동기화 모드가 기본 동기화 대비 880 배 빠른 것을 확인하였다.