• Title/Summary/Keyword: batch processing

Search Result 297, Processing Time 0.026 seconds

The efficiency analysis and improvement plan of real time batch processing in web environment (웹 환경의 실시간 일괄처리 성능 분석 및 개선 방안)

  • Lee, Sun-Hyun
    • Journal of the Korea Computer Industry Society
    • /
    • v.5 no.5
    • /
    • pp.771-780
    • /
    • 2004
  • The demands of users have become variety and diversification by the conversion of application to Web environment from C/S(Client/Server) enviroment as well as by the development of application which puts into practie an information technology with the advance of internet basic technology. Therefore if it attempt to satisfy their all demands, the services like a real time batch processing are faced with a difficult obstacle that the limited existing system cannot manage. Finally it is come to a conculsion that the extension of system or the induction of new system is necessary in order to solve this problem according to a ragular tuning and an examination about infra of web environment. This thesis intends to propose about inter- relation and improvement plan as well as considerable facts in case of the introduction of system through the efficiency analysis process of the system.

  • PDF

Anomaly Detection System of IoT Platform using Machine Learning (기계학습을 활용한 IoT 플랫폼의 이상감지 시스템)

  • Im, SeonYeol;Choi, HyoKeun;Yi, KyuYull;Lee, TeaHun;Yu, HeonChang
    • Annual Conference of KIPS
    • /
    • 2018.10a
    • /
    • pp.1001-1004
    • /
    • 2018
  • As the industry generates a lot of data, it is increasingly dependent on the IoT platform. For this reason, the performance and anomaly detection of IoT platform is becoming an important factor. In this paper, we propose a system model of IoT platform that detects device anomaly without performance issue. The proposed system uses Micro Batch which calculates the data transmission cycle to provide Soft Real-time service. In the industry, it was difficult to collect abnormal data, so the Hotelling's $T^2$ model was applied to the data analysis experiment. And the Hotelling's $T^2$ model successfully detected anomalies.

A Study on the Skirt Size Selection of a Composite Pressure Vessel using Optimum Analysis Technique (최적화 해석 기법을 이용한 복합재 압력용기의 스커트 치수 선정에 관한 연구)

  • Kim, Jun-Hwan;Jeon, Kwang-Woo;Shin, Kwang-Bok;Hwang, Tae-Kyung
    • Proceedings of the Korean Society of Propulsion Engineers Conference
    • /
    • 2012.05a
    • /
    • pp.403-407
    • /
    • 2012
  • The purpose of this study is to find the optimum skirt size for a composite pressure vessel using optimum analysis technique. The size optimization for skirt shape of a composite pressure vessel was conducted using sub-problem approximation method and batch processing codes programmed by APDL(ANSYS Parametric Design Language). The thickness and length of skirt part were selected as design variables for the optimum analysis. The objective function and constraints were chosen as weight and displacement of skirt part, respectively. The numerical results showed that the weight of skirt of a composite pressure vessel would be saved by maximum 4.38% through the size optimization analysis for the skirt shape.

  • PDF

Squall: A Real-time Big Data Processing Framework based on TMO Model for Real-time Events and Micro-batch Processing (Squall: 실시간 이벤트와 마이크로-배치의 동시 처리 지원을 위한 TMO 모델 기반의 실시간 빅데이터 처리 프레임워크)

  • Son, Jae Gi;Kim, Jung Guk
    • Journal of KIISE
    • /
    • v.44 no.1
    • /
    • pp.84-94
    • /
    • 2017
  • Recently, the importance of velocity, one of the characteristics of big data (5V: Volume, Variety, Velocity, Veracity, and Value), has been emphasized in the data processing, which has led to several studies on the real-time stream processing, a technology for quick and accurate processing and analyses of big data. In this paper, we propose a Squall framework using Time-triggered Message-triggered Object (TMO) technology, a model that is widely used for processing real-time big data. Moreover, we provide a description of Squall framework and its operations under a single node. TMO is an object model that supports the non-regular real-time processing method for certain conditions as well as regular periodic processing for certain amount of time. A Squall framework can support the real-time event stream of big data and micro-batch processing with outstanding performances, as compared to Apache storm and Spark Streaming. However, additional development for processing real-time stream under multiple nodes that is common under most frameworks is needed. In conclusion, the advantages of a TMO model can overcome the drawbacks of Apache storm or Spark Streaming in the processing of real-time big data. The TMO model has potential as a useful model in real-time big data processing.

Optimal Design of Silo System for Drying and Storage of Grains (I)-Simulation Modeling with SLAMSYSTEM

  • Chung, Jong-Hoon
    • Proceedings of the Korean Society for Agricultural Machinery Conference
    • /
    • 1993.10a
    • /
    • pp.952-965
    • /
    • 1993
  • A simulation modeling is necessary for the optimal design of a rice processing plant, which consists of a facility (a silo system) of rice drying and storage and a rice mill plant. In a rice processing plant, the production scheduling and the decision on capcity of each unit based on a queuing theory is very important and difficult. In this study a process-oriented simulation model was developed for the design of a rice drying and storage system with SLAMSYSTEM. The simulation model is capable of simulating virtually all the processing activities and provides work schedules which minimize total processing time , mean flow time and bottleneck of the plant system and estimate drying time for a batch in a drying silo. Model results were used for determination the size and capacity of each processing unit and for analyzing the performance of the plant . The developed model was actually applied to construct a grain silo system for rice drying and storage.

  • PDF

Convolutional Neural Network Based Image Processing System

  • Kim, Hankil;Kim, Jinyoung;Jung, Hoekyung
    • Journal of information and communication convergence engineering
    • /
    • v.16 no.3
    • /
    • pp.160-165
    • /
    • 2018
  • This paper designed and developed the image processing system of integrating feature extraction and matching by using convolutional neural network (CNN), rather than relying on the simple method of processing feature extraction and matching separately in the image processing of conventional image recognition system. To implement it, the proposed system enables CNN to operate and analyze the performance of conventional image processing system. This system extracts the features of an image using CNN and then learns them by the neural network. The proposed system showed 84% accuracy of recognition. The proposed system is a model of recognizing learned images by deep learning. Therefore, it can run in batch and work easily under any platform (including embedded platform) that can read all kinds of files anytime. Also, it does not require the implementing of feature extraction algorithm and matching algorithm therefore it can save time and it is efficient. As a result, it can be widely used as an image recognition program.

Design of Extended Real-time Data Pipeline System Architecture (확장형 실시간 데이터 파이프라인 시스템 아키텍처 설계)

  • Shin, Hoseung;Kang, Sungwon;Lee, Jihyun
    • Journal of KIISE
    • /
    • v.42 no.8
    • /
    • pp.1010-1021
    • /
    • 2015
  • Big data systems are widely used to collect large-scale log data, so it is very important for these systems to operate with a high level of performance. However, the current Hadoop-based big data system architecture has a problem in that its performance is low as a result of redundant processing. This paper solves this problem by improving the design of the Hadoop system architecture. The proposed architecture uses the batch-based data collection of the existing architecture in combination with a single processing method. A high level of performance can be achieved by analyzing the collected data directly in memory to avoid redundant processing. The proposed architecture guarantees system expandability, which is an advantage of using the Hadoop architecture. This paper confirms that the proposed architecture is approximately 30% to 35% faster in analyzing and processing data than existing architectures and that it is also extendable.

Transfer Batch Scheduling for a Two-machine Flow Shop with Attached and Separated Setup Times (기계가 2대인 흐름생산시스템에서 생산준비시간 분리와 이송배치를 고려한 일정계획)

  • 김중순
    • Journal of the Korean Operations Research and Management Science Society
    • /
    • v.26 no.1
    • /
    • pp.87-96
    • /
    • 2001
  • This paper considers the transfer batch scheduling problem for a two-machine flow shop with setup times consisting of attached setup time and separate setup time. The attached setup is attached to the processing and can be scheduled in anticipation of arriving work. Two cases of the separate setup are addressed. One is the case that the separate setup can be done only when a machine is idle. The other is the case that the separate setup can be done even though the machine is idle. the other is the case that the separate setup can be done even though the machine is operating. A scheduling rule similar to Johnson\`s rule is suggested to minimize makespan. the scheduling rule developed can be applied to most of the two-machine flow shop scheduling problems if some parameters are adjusted. A numerical example is presented to illustrate the scheduling rule.

  • PDF

Rescheduling algorithms considering unit failure on the batch process management (회분공정의 장치 고장을 고려한 동적생산계획 기법)

  • Ko, Dae-Ho;Moon, Il
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 1996.10b
    • /
    • pp.1028-1031
    • /
    • 1996
  • Dynamic scheduling is very important in constructing CIM and improving productivity of chemical processing systems. Computation at the scheduling level requires mostly a long time to generate an optimal schedule, so it is difficult to immediately respond to actual process events in real-time. To solve these problems, we developed dynamic scheduling algorithms such as DSMM(Dynamic Shift Modification Method), PUOM(Parallel Unit Operation Method) and UVVM(Unit Validity Verification Method). Their main functions are to minimize the effects of unexpected disturbances such as process time variations and unit failure, to predict a makespan of the updated dynamic schedule and to modify schedule desirably in real-time responding to process time variations. As a result, the algorithms generate a new pertinent schedule in real-time which is close to the original schedule but provides an efficient way of responding to the variation of process environment. Examples in a shampoo production batch process illustrate the efficiency of the algorithms.

  • PDF

Simultanceous Saccharification and Fermentation of Cellulose for Lactic Acid Production

  • Yoon, Hyon-Hee
    • Biotechnology and Bioprocess Engineering:BBE
    • /
    • v.2 no.2
    • /
    • pp.101-104
    • /
    • 1997
  • Lactic acid production from ${\alpha}$-cellulose by simultaneous saccharification and fermentation (SSF) was studied. The cellulose was converted in a batch SSF using cellulase enzyme Cytolase CL to produce glucose sugar and Lactobacillus delbrueckii to ferment the glucose to lactic acid. The effects of temperature, PH, yeast extract loading, and lactic acid inhibition were studied to determine the optimum conditions for the batch processing. Cellulose was converted efficiently to lactic acid, and enzymatic hydrolysis was the rate controlling step in the SSF. The highest conversion rate was obtained at 46$^{\circ}C$ and pH 5.0. The observed yield of lactic acid from ${\alpha}$-cellulose was 0.90 at 72 hours. The optimum pH of the SSF was coincident with that of enzymatic hydrolysis. The optimum temperature of the SSF was chosen as the highest temperature the microoraganism could withstand. The optimum yeast extract loading was found to be 2.5g/L. Lactic acid was observed to be inhibitory to the microorganisms' activity.

  • PDF