• Title/Summary/Keyword: Incremental Processing

Search Result 210, Processing Time 0.04 seconds

Incremental Processing Scheme for Graph Streams Considering Data Reuse (데이터 재사용을 고려한 그래프 스트림의 점진적 처리 기법)

  • Cho, Jungkweon;Han, Jinsu;Kim, Minsoo;Choi, Dojin;Bok, Kyoungsoo;Yoo, Jaesoo
    • The Journal of the Korea Contents Association
    • /
    • v.18 no.1
    • /
    • pp.465-475
    • /
    • 2018
  • Recently, as the use of social media and IoT has increased, large graph streams has been generating and studies on real-time processing for them have been actively carrying out. In this paper we propose a incremental graph stream processing scheme that reuses previous result data when the graph changes continuously. We also propose a cost model to selectively perform incremental processing and static processing. The proposed cost model computes the predicted value of the detection cost and the processing cost of the recalculation area based on the actually processed history and performs the incremental processing when the incremental processing is more profit than the static processing. The proposed incremental processing increases the efficiency by processing only the part that changes when the graph update occurs. Also, by collecting only the previous result data of the changed part and performing the incremental processing, the disk I/O costs are reduced. It is shown through various performance evaluations that the proposed scheme outperforms the existing schemes.

Fast Incremental Checkpoint Based on Page-Level Rewrite Interval Prediction

  • Huang, Yulei
    • Journal of Information Processing Systems
    • /
    • v.16 no.4
    • /
    • pp.859-869
    • /
    • 2020
  • This paper introduces page-level rewrite interval prediction (PRWIP). By recording and analyzing the memory access history at page-level, we are able to predict the future memory accesses to any pages. Leveraging this information, this paper proposes a faster incremental checkpoint design by overlapping checkpoint phase with computing phase and thus achieves higher performance. Experimental results show that our new incremental checkpoint design can achieve averagely 22% speedup over traditional incremental checkpoint and 14% over the previous state-of-the-art work.

Incremental MapReduce of atypical Big Data Processing in Mobile Game (모바일게임에 적용 가능한 비정형 Big Data 처리를 위한 Incremental MapReduce)

  • Park, Sung-Joon;Kim, Jung-Woong
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2014.04a
    • /
    • pp.301-304
    • /
    • 2014
  • 비정형 게임 Big Data에서 고효율 정보를 추출하고, 신뢰 할 수 있는 클러스터 게임서버 환경을 위한 병렬 처리를 위해 MapReduce를 사용한다. 본 논문에서는 빈번하게 입력되는 신규 게임데이터 처리를 위해 함수 Demap을 사용하는 Incremental MapReduce를 적용하여 불필요한 중간 값 저장과 재계산 없이 점차적으로 MapReduce 함수를 실행한다.

The Study for Development of Damper Case Production Technique using Incremental Forming (Incremental Forming 기술을 적용한 Damper Case 생산 기술 개발에 관한 연구)

  • Park, Jeong-Ho;Lee, Tae-Won;Jeong, Young-Duk
    • Journal of the Korean Society of Manufacturing Process Engineers
    • /
    • v.10 no.5
    • /
    • pp.72-78
    • /
    • 2011
  • Currently, for the damper case, the material produced by cast/forge welding is mechanically processed and then the final product is mass-produced. By cutting the cast/forge welded material, the issues of excessive cutting time, multiple process production, and a large amount of chips (40% loss from the original material) arise, causing increased production cost and reduced profitability. Thus, in this study, the incremental forming technology which generates no chips was applied in production. Analysis was excuted for 1st and 2nd works by change of tool diameter and working tool. For this, 3D molding and analysis were executed, which was applied to the processing the result, successful processing could be achieved through a few trials of molding processing according to tool forming and rotation counts.

A Concurrent Incremental Evaluation Technique Using Multitasking (멀티태스킹에 의한 병행 점진 평가 방법)

  • Han, Jung-Lan
    • The KIPS Transactions:PartA
    • /
    • v.17A no.2
    • /
    • pp.73-80
    • /
    • 2010
  • As the power of hardware has improved, there have been numerous researches in processing concurrently using multitasking method. The incremental evaluation is the evaluation method of reevaluating only affected parts instead of reevaluating overall program when the program has been changed. It is necessary to do more studies that improve the efficiency of concurrent incremental evaluation to do multitasking using multi-threading of Java not to do in parallel using multiprocessor. In this paper, the dependency in the dependency chart is based on the attribute that describes the real value of the variable that directly affects the semantics, thereby doing efficient evaluation. So using the dependency, this paper presents the concurrent incremental evaluation algorithm for Java Languages and proves its correctness, analyzing the efficiency of concurrent incremental evaluation by the simulation.

IMTAR: Incremental Mining of General Temporal Association Rules

  • Dafa-Alla, Anour F.A.;Shon, Ho-Sun;Saeed, Khalid E.K.;Piao, Minghao;Yun, Un-Il;Cheoi, Kyung-Joo;Ryu, Keun-Ho
    • Journal of Information Processing Systems
    • /
    • v.6 no.2
    • /
    • pp.163-176
    • /
    • 2010
  • Nowadays due to the rapid advances in the field of information systems, transactional databases are being updated regularly and/or periodically. The knowledge discovered from these databases has to be maintained, and an incremental updating technique needs to be developed for maintaining the discovered association rules from these databases. The concept of Temporal Association Rules has been introduced to solve the problem of handling time series by including time expressions into association rules. In this paper we introduce a novel algorithm for Incremental Mining of General Temporal Association Rules (IMTAR) using an extended TFP-tree. The main benefits introduced by our algorithm are that it offers significant advantages in terms of storage and running time and it can handle the problem of mining general temporal association rules in incremental databases by building TFP-trees incrementally. It can be utilized and applied to real life application domains. We demonstrate our algorithm and its advantages in this paper.

Design of Incremental K-means Clustering-based Radial Basis Function Neural Networks Model (증분형 K-means 클러스터링 기반 방사형 기저함수 신경회로망 모델 설계)

  • Park, Sang-Beom;Lee, Seung-Cheol;Oh, Sung-Kwun
    • The Transactions of The Korean Institute of Electrical Engineers
    • /
    • v.66 no.5
    • /
    • pp.833-842
    • /
    • 2017
  • In this study, the design methodology of radial basis function neural networks based on incremental K-means clustering is introduced for learning and processing the big data. If there is a lot of dataset to be trained, general clustering may not learn dataset due to the lack of memory capacity. However, the on-line processing of big data could be effectively realized through the parameters operation of recursive least square estimation as well as the sequential operation of incremental clustering algorithm. Radial basis function neural networks consist of condition part, conclusion part and aggregation part. In the condition part, incremental K-means clustering algorithms is used tweights of the conclusion part are given as linear function and parameters are calculated using recursive least squareo get the center points of data and find the fitness using gaussian function as the activation function. Connection s estimation. In the aggregation part, a final output is obtained by center of gravity method. Using machine learning data, performance index are shown and compared with other models. Also, the performance of the incremental K-means clustering based-RBFNNs is carried out by using PSO. This study demonstrates that the proposed model shows the superiority of algorithmic design from the viewpoint of on-line processing for big data.

Design of Incremental FCM-based Recursive RBF Neural Networks Pattern Classifier for Big Data Processing (빅 데이터 처리를 위한 증분형 FCM 기반 순환 RBF Neural Networks 패턴 분류기 설계)

  • Lee, Seung-Cheol;Oh, Sung-Kwun
    • The Transactions of The Korean Institute of Electrical Engineers
    • /
    • v.65 no.6
    • /
    • pp.1070-1079
    • /
    • 2016
  • In this paper, the design of recursive radial basis function neural networks based on incremental fuzzy c-means is introduced for processing the big data. Radial basis function neural networks consist of condition, conclusion and inference phase. Gaussian function is generally used as the activation function of the condition phase, but in this study, incremental fuzzy clustering is considered for the activation function of radial basis function neural networks, which could effectively do big data processing. In the conclusion phase, the connection weights of networks are given as the linear function. And then the connection weights are calculated by recursive least square estimation. In the inference phase, a final output is obtained by fuzzy inference method. Machine Learning datasets are employed to demonstrate the superiority of the proposed classifier, and their results are described from the viewpoint of the algorithm complexity and performance index.

A Cumulative Incremental Effort Based Software Growth Model Considering System Size and Complexity (시스템 크기와 복잡도를 고려한 누적 노력 기반의 소프트웨어 성장 모델)

  • Park, Jung-Yang;Kim, Seong-Hui;Park, Jae-Heung
    • The Transactions of the Korea Information Processing Society
    • /
    • v.6 no.1
    • /
    • pp.90-95
    • /
    • 1999
  • A software growth model, a mathematical model describing the growth behavior of a software system during the evolution process, enables us to predict the future system size and incremental erfort required to meet the planned system size. This paper first introduces a software growth model defined with respect to the cumulative incremental effort. It was assumed that the incremental growth of a software system is proportional to the incremental effort and function of the system size is suggested as a system complexity and then applied to real data for its validation. such a system complexity additionally provides us with a measure for complexity comparison. since the measure is independent of the system size, it is useful for comparing complexities of software systems of different size.

  • PDF

Incremental Fuzzy Clustering Based on a Fuzzy Scatter Matrix

  • Liu, Yongli;Wang, Hengda;Duan, Tianyi;Chen, Jingli;Chao, Hao
    • Journal of Information Processing Systems
    • /
    • v.15 no.2
    • /
    • pp.359-373
    • /
    • 2019
  • For clustering large-scale data, which cannot be loaded into memory entirely, incremental clustering algorithms are very popular. Usually, these algorithms only concern the within-cluster compactness and ignore the between-cluster separation. In this paper, we propose two incremental fuzzy compactness and separation (FCS) clustering algorithms, Single-Pass FCS (SPFCS) and Online FCS (OFCS), based on a fuzzy scatter matrix. Firstly, we introduce two incremental clustering methods called single-pass and online fuzzy C-means algorithms. Then, we combine these two methods separately with the weighted fuzzy C-means algorithm, so that they can be applied to the FCS algorithm. Afterwards, we optimize the within-cluster matrix and betweencluster matrix simultaneously to obtain the minimum within-cluster distance and maximum between-cluster distance. Finally, large-scale datasets can be well clustered within limited memory. We implemented experiments on some artificial datasets and real datasets separately. And experimental results show that, compared with SPFCM and OFCM, our SPFCS and OFCS are more robust to the value of fuzzy index m and noise.