• Title/Summary/Keyword: Big data Processing

Search Result 1,063, Processing Time 0.025 seconds

Design of Log Management System based on Document Database for Big Data Management (빅데이터 관리를 위한 문서형 DB 기반 로그관리 시스템 설계)

  • Ryu, Chang-ju;Han, Myeong-ho;Han, Seung-jo
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.19 no.11
    • /
    • pp.2629-2636
    • /
    • 2015
  • Recently Big Data management have a rapid increases interest in IT field, much research conducting to solve a problem of real-time processing to Big Data. Lots of resources are required for the ability to store data in real-time over the network but there is the problem of introducing an analyzing system due to aspect of high cost. Need of redesign of the system for low cost and high efficiency had been increasing to solve the problem. In this paper, the document type of database, MongoDB, is used for design a log management system based a document type of database, that is good at big data managing. The suggested log management system is more efficient than other method on log collection and processing, and it is strong on data forgery through the performance evaluation.

An Efficient Cloud Service Quality Performance Management Method Using a Time Series Framework (시계열 프레임워크를 이용한 효율적인 클라우드서비스 품질·성능 관리 방법)

  • Jung, Hyun Chul;Seo, Kwang-Kyu
    • Journal of the Semiconductor & Display Technology
    • /
    • v.20 no.2
    • /
    • pp.121-125
    • /
    • 2021
  • Cloud service has the characteristic that it must be always available and that it must be able to respond immediately to user requests. This study suggests a method for constructing a proactive and autonomous quality and performance management system to meet these characteristics of cloud services. To this end, we identify quantitative measurement factors for cloud service quality and performance management, define a structure for applying a time series framework to cloud service application quality and performance management for proactive management, and then use big data and artificial intelligence for autonomous management. The flow of data processing and the configuration and flow of big data and artificial intelligence platforms were defined to combine intelligent technologies. In addition, the effectiveness was confirmed by applying it to the cloud service quality and performance management system through a case study. Using the methodology presented in this study, it is possible to improve the service management system that has been managed artificially and retrospectively through various convergence. However, since it requires the collection, processing, and processing of various types of data, it also has limitations in that data standardization must be prioritized in each technology and industry.

A Study on Distributed Processing of Big Data and User Authentication for Human-friendly Robot Service on Smartphone (인간 친화적 로봇 서비스를 위한 대용량 분산 처리 기술 및 사용자 인증에 관한 연구)

  • Choi, Okkyung;Jung, Wooyeol;Lee, Bong Gyou;Moon, Seungbin
    • Journal of Internet Computing and Services
    • /
    • v.15 no.1
    • /
    • pp.55-61
    • /
    • 2014
  • Various human-friendly robot services have been developed and mobile cloud computing is a real time computing service that allows users to rent IT resources what they want over the internet and has become the new-generation computing paradigm of information society. The enterprises and nations are actively underway of the business process using mobile cloud computing and they are aware of need for implementing mobile cloud computing to their business practice, but it has some week points such as authentication services and distributed processing technologies of big data. Sometimes it is difficult to clarify the objective of cloud computing service. In this study, the vulnerability of authentication services on mobile cloud computing is analyzed and mobile cloud computing model is constructed for efficient and safe business process. We will also be able to study how to process and analyze unstructured data in parallel to this model, so that in the future, providing customized information for individuals may be possible using unstructured data.

Operating Simulation of RPS using DEVS W/S in Web Service Environment

  • Cho, Kyu-Cheol
    • Journal of the Korea Society of Computer and Information
    • /
    • v.21 no.12
    • /
    • pp.107-114
    • /
    • 2016
  • Web system helps high-performance processing for big-data analysis and practical use to make various information using IT resources. The government have started the RPS system in 2012. The system invigorates the electricity production as using renewable energy equipment. The government operates system gathered big-data with various related information system data and the system users are distributed geographically. The companies have to fulfill the system, are available to purchase the REC to other electricity generation company sellers to procure REC for their duty volumes. The REC market operates single auction methods with users a competitive price. But the price have the large variation with various user trading strategy and sellers situations. This papler proposed RPS system modeling and simulation in web environment that is modeled in geographically distributed computing environment for web user with DEVS W/S. Web simulation system base on web service helps to analysis correlation and variables that act on trading price and volume within RPS big-data and the analysis can be forecast REC price.

Constructing a Standard Clinical Big Database for Kidney Cancer and Development of Machine Learning Based Treatment Decision Support Systems (신장암 표준임상빅데이터 구축 및 머신러닝 기반 치료결정지원시스템 개발)

  • Song, Won Hoon;Park, Meeyoung
    • Journal of the Korean Society of Industry Convergence
    • /
    • v.25 no.6_2
    • /
    • pp.1083-1090
    • /
    • 2022
  • Since renal cell carcinoma(RCC) has various examination and treatment methods according to clinical stage and histopathological characteristics, it is required to determine accurate and efficient treatment methods in the clinical field. However, the process of collecting and processing RCC medical data is difficult and complex, so there is currently no AI-based clinical decision support system for RCC treatments worldwide. In this study, we propose a clinical decision support system that helps clinicians decide on a precision treatment to each patient. RCC standard big database is built by collecting structured and unstructured data from the standard common data model and electronic medical information system. Based on this, various machine learning classification algorithms are applied to support a better clinical decision making.

A Model of Vital Signs Analysis based on Big Data using OCL (OCL을 이용한 빅데이터 기반의 생체신호 분석 모델)

  • Kim, Tae-Woong
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.23 no.12
    • /
    • pp.1485-1491
    • /
    • 2019
  • As the type and size of vital signs become extensive at the moment lately, a research is actively progressing to define vital signs as big data and analyze it. We generally use a similar method of processing big data on social network as a way to treat vital signs as big data. Vital Sign Big Data should be extracted as feature data, stored separately, and analyzed with various analytical instruments. In other words, it should ensure interoperability and compatability of data, and the index expression in analytical tools should be concise. For this end, I defined the vital sign as the standard meta-model base of HL7 in this dissertation, and I propose a model for analyzing vital signs using OCL, the OMG's standard mathematical specification language. In addition, the proposed model can be confirmed the applicability by figuring out the consumption of calories using ECG data.

A Study of Bigdata Platform for Supporting Engineering Services (엔지니어링 서비스 지원을 위한 클라우드 기반 빅데이터 플랫폼 개발 연구)

  • Seo, Dongwoo;Kim, Myungil;Park, Sangjin;Kim, Jaesung;Jeong, Seok Chan
    • The Journal of Bigdata
    • /
    • v.4 no.1
    • /
    • pp.119-127
    • /
    • 2019
  • This study explains how to solve engineering problems easily and efficiently by using cloud based big data platform. To do this, we propose a cloud based big data analysis platform. The application helps users easily create models for data analysis using cloud based big data analysis platform. Analytical models modeled using components are analyzed through an analysis engine. Our platform include pre-processing, analysis, and visualization algorithms required for data analysis. Finally, we show an application of effluent concentration in a sewage treatment process.

  • PDF

An Efficient Algorithm for Big Data Prediction of Pipelining, Concurrency (PCP) and Parallelism based on TSK Fuzzy Model (TSK 퍼지 모델 이용한 효율적인 빅 데이터 PCP 예측 알고리즘)

  • Kim, Jang-Young
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.19 no.10
    • /
    • pp.2301-2306
    • /
    • 2015
  • The time to address the exabytes of data has come as the information age accelerates. Big data transfer technology is essential for processing large amounts of data. This paper posits to transfer big data in the optimal conditions by the proposed algorithm for predicting the optimal combination of Pipelining, Concurrency, and Parallelism (PCP), which are major functions of GridFTP. In addition, the author introduced a simple design process of Takagi-Sugeno-Kang (TSK) fuzzy model and designed a model for predicting transfer throughput with optimal combination of Pipelining, Concurrency and Parallelism. Hence, the author evaluated the model of the proposed algorithm and the TSK model to prove the superiority.

Real-time Abnormal Behavior Detection by Online Data Collection (온라인 데이터 수집 기반 실시간 비정상 행위 탐지)

  • Lee, Myungcheol;Kim, ChangSoo;Kim, Ikkyun
    • Annual Conference of KIPS
    • /
    • 2016.10a
    • /
    • pp.208-209
    • /
    • 2016
  • APT (Advanced Persistent Threat) 공격 사례가 증가하면서, 이러한 APT 공격을 해결하고자 이상 행위 탐지 기술 관련 연구가 활발히 진행되고 있다. 최근에는 APT 공격의 탐지율을 높이기 위해서 빅데이터 기술을 활용하여 다양한 소스로부터 대규모 데이터를 수집하여 실시간 분석하는 연구들이 시도되고 있다. 본 논문은 빅데이터 기술을 활용하여 기존 시스템들의 실시간 처리 및 분석 한계를 극복하기 위한 실시간 비정상 행위 탐지 시스템에서, 파일 시스템에 수집된 오프라인 데이터 기반이 아닌 온라인 수집 데이터 기반으로 실시간 비정상 행위를 탐지하여 실시간성을 제고하고 입출력 병목 문제로 인한 처리 성능 확장성 문제를 해결하는 방법 및 시스템에 대해서 제안한다.

Compression-Friendly Low Power Test Application Based on Scan Slices Reusing

  • Wang, Weizheng;Wang, JinCheng;Cai, Shuo;Su, Wei;Xiang, Lingyun
    • JSTS:Journal of Semiconductor Technology and Science
    • /
    • v.16 no.4
    • /
    • pp.463-469
    • /
    • 2016
  • This paper presents a compression-friendly low power test scheme in EDT environment. The proposed approach exploits scan slices reusing to reduce the switching activity during shifting for test scheme based on linear decompressor. To avoid the impact on encoding efficiency from resulting control data, a counter is utilized to generate control signals. Experimental results obtained for some larger ISCAS'89 and ITC'99 benchmark circuits illustrate that the proposed test application scheme can improve significantly the encoding efficiency of linear decompressor.