• 한국시뮬레이션학회논문지
• /
• 제4권1호
• /
• pp.1-11
• /
• 1995

The model of a FMS (Flexible Manufacturing System) admits to a natural hierarchical decomposition of highly decoupled units with similar structure and control. The FMS fractal architecture model represents a hierarchical structure built from elements of a single basic design. A SES (System Entity Structure) is a structural knowledge representation scheme that contains knowledge of decomposition, taxonomy, and coupling relationships of a system necessary to direct model synthesis. A substructure of a SES is extracted for use as the skeleton for a model. This substructure is called pruned SES and the extraction operation of a pruned SES from a SES is called pruning (or pruning operation). This paper presents a pruning operation called recursive pruning. It is applied to SES for generating a model structure whose sub-structure contains copies if itself as in FMS fractal architecture. Another pruning operation called delay pruning is also presented. Combined with recursive pruning the delay pruningis a useful tool for representing and constructing complex systems.

• International Journal of Computer Science & Network Security
• /
• 제21권6호
• /
• pp.54-60
• /
• 2021

Monte-Carlo GO is a computer GO program that is sufficiently competent without using knowledge expressions of IGO. Although it is computationally intensive, the computational complexity can be reduced by properly pruning the IGO game tree. Here, I achieve this by using a potential model based on the knowledge expressions of IGO. The potential model treats GO stones as potentials. A specific potential distribution on the GO board results from a unique arrangement of stones on the board. Pruning using the potential model categorizes legal moves into effective and ineffective moves in accordance with the potential threshold. Here, certain pruning strategies based on potentials and potential gradients are experimentally evaluated. For different-sized boards, including an official-sized board, the effects of pruning strategies are evaluated in terms of their robustness. I successfully demonstrate pruning using a potential model to reduce the computational complexity of GO as well as the robustness of this effect across different-sized boards.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제16권11호
• /
• pp.3507-3522
• /
• 2022

Model pruning methods have attracted huge attention owing to the increasing demand of deploying models on low-resource devices recently. Most existing methods use the weight norm of filters to represent their importance, and discard the ones with small value directly to achieve the pruning target, which ignores the contribution of the small norm filters. This is not only results in filter contribution waste, but also gives comparable performance to training with the random initialized weights [1]. In this paper, we point out that the small norm filters can harm the performance of the pruned model greatly, if they are discarded directly. Therefore, we propose a novel filter contribution recycle (FCR) method for structured model pruning to resolve the fore-mentioned problem. FCR collects and reassembles contribution from the small norm filters to obtain a mixed contribution collector, and then assigns the reassembled contribution to other filters with higher probability to be preserved. To achieve the target FLOPs, FCR also adopts a weight decay strategy for the small norm filters. To explore the effectiveness of our approach, extensive experiments are conducted on ImageNet2012 and CIFAR-10 datasets, and superior results are reported when comparing with other methods under the same or even more FLOPs reduction. In addition, our method is flexible to be combined with other different pruning criterions.

• 반도체공학회 논문지
• /
• 제1권1호
• /
• pp.23-30
• /
• 2023

최근 거대 IT 기업들의 Generative AI 기술 개발로 Transformer 모델의 규모가 조 단위를 넘어가며 기하급수적으로 증가하고 있다. 이러한 AI 서비스를 지속적으로 가능케 하기 위해선 모델 경량화가 필수적이다. 본 논문에서는 하드웨어 친화적으로 구조화된(structured) 프루닝 패턴을 찾아 Transformer 모델의 경량화 방법을 제안한다. 이는 모델 알고리즘의 특성을 살려 압축을 진행하기 때문에 모델의 크기는 줄어들면서 성능은 최대한 유지할 수 있다. 실험에 따르면 GPT2 와 BERT 언어 모델을 프루닝할 때 제안하는 구조화된 프루닝 기법은 희소성이 높은 영역에서도 미세 조정된(fine-grained) 프루닝과 거의 흡사한 성능을 보여준다. 이 접근 방식은 미세 조정된 프루닝 대비 0.003%의 정확도 손실로 모델매개 변수를 80% 줄이고 구조화된 형태로 하드웨어 가속화를 진행할 수 있다.

• 한국군사과학기술학회지
• /
• 제27권3호
• /
• pp.346-354
• /
• 2024

Deep learning models require high computing power due to a substantial amount of computation. It is difficult to use them in devices with limited computing environments, such as coastal surveillance equipments. In this study, a lightweight model is constructed by analyzing the weight changes of the convolutional layers during the training process based on MobileNet and then pruning the layers that affects the model less. The performance comparison results show that the lightweight model maintains performance while reducing computational load, parameters, model size, and data processing speed. As a result of this study, an effective pruning method for constructing lightweight deep learning models and the possibility of using equipment resources efficiently through lightweight models in limited computing environments such as coastal surveillance equipments are presented.

• 대한임베디드공학회논문지
• /
• 제10권2호
• /
• pp.65-71
• /
• 2015

It has been known that GMM (Gaussian Mixture Model) based speaker identification systems using ML (Maximum Likelihood) and WMR (Weighting Model Rank) demonstrate very high performances. However, such systems are not so effective under practical environments, in terms of real time processing, because of their high calculation costs. In this paper, we propose a new speaker-pruning algorithm that effectively reduces the calculation cost. In this algorithm, we select 20% of speaker models having higher likelihood with a part of input speech and apply MWMR (Modified Weighted Model Rank) to these selected speaker models to find out identified speaker. To verify the effectiveness of the proposed algorithm, we performed speaker identification experiments using TIMIT database. The proposed method shows more than 60% improvement of reduced processing time than the conventional GMM based system with no pruning, while maintaining the recognition accuracy.

• 융합정보논문지
• /
• 제10권8호
• /
• pp.23-34
• /
• 2020

최근 인공지능 딥러닝 분야는 컴퓨팅 자원의 높은 연산량과 가격문제로 인해 상용화에 어려움이 존재했다. 본 논문은 더블 프루닝 기법을 적용하여 심층신경망 모델들과 다수의 데이터셋에서의 성능을 평가하고자 한다. 더블 프루닝은 기본의 네트워크 간소화(Network-Slimming)과 파라미터 프루닝(Parameter-Pruning)을 결합한다. 이는 기존의 학습에 중요하지 않는 매개변수를 절감하여 학습 정확도를 저해하지 않고 속도를 향상시킬 수 있다는 장점이 있다. 다양한 데이터셋 학습 이후에 프루닝 비율을 증가시켜, 모델의 사이즈를 감소시켰다. NetScore 성능 분석 결과 MobileNet-V3가 가장 성능이 높게 나타났다. 프루닝 이후의 성능은 Cifar 10 데이터셋에서 깊이 우선 합성곱 신경망으로 구성된 MobileNet-V3이 가장 성능이 높았고, 전통적인 합성곱 신경망으로 이루어진 VGGNet, ResNet또한 높은 폭으로 성능이 증가함을 확인하였다.

• 한국정보과학회 언어공학연구회:학술대회논문집(한글 및 한국어 정보처리)
• /
• 한국정보과학회언어공학연구회 2023년도 제35회 한글 및 한국어 정보처리 학술대회
• /
• pp.437-442
• /
• 2023

최근에 등장한 대규모 언어 모델은 다양한 언어 처리 작업에서 놀라운 성능을 발휘하고 있다. 그러나 이러한 모델의 크기와 복잡성 때문에 모델 경량화의 필요성이 대두되고 있다. Pruning은 이러한 경량화 전략 중 하나로, 모델의 가중치나 연결의 일부를 제거하여 크기를 줄이면서도 동시에 성능을 최적화하는 방법을 제시한다. 본 논문에서는 한국어 언어 모델인 Polyglot-Ko에 Wanda[1] 기법을 적용하여 Pruning 작업을 수행하였다. 그리고 이를 통해 가중치가 제거된 모델의 Perplexity, Zero-shot 성능, 그리고 Fine-tuning 후의 성능을 분석하였다. 실험 결과, Wanda-50%, 4:8 Sparsity 패턴, 2:4 Sparsity 패턴의 순서로 높은 성능을 나타냈으며, 특히 일부 조건에서는 기존의 Dense 모델보다 더 뛰어난 성능을 보였다. 이러한 결과는 오늘날 대규모 언어 모델 중심의 연구에서 Pruning 기법의 효과와 그 중요성을 재확인하는 계기가 되었다.

• Computers and Concrete
• /
• 제6권3호
• /
• pp.203-223
• /
• 2009

Regression analysis (RA) can establish an explicit formula to predict the strength of High-Performance Concrete (HPC); however, the accuracy of the formula is poor. Back-Propagation Networks (BPNs) can establish a highly accurate model to predict the strength of HPC, but cannot generate an explicit formula. Genetic Operation Trees (GOTs) can establish an explicit formula to predict the strength of HPC that achieves a level of accuracy in between the two aforementioned approaches. Although GOT can produce an explicit formula but the formula is often too complicated so that unable to explain the substantial meaning of the formula. This study developed a Backward Pruning Technique (BPT) to simplify the complexity of GOT formula by replacing each variable of the tip node of operation tree with the median of the variable in the training dataset belonging to the node, and then pruning the node with the most accurate test dataset. Such pruning reduces formula complexity while maintaining the accuracy. 404 experimental datasets were used to compare accuracy and complexity of three model building techniques, RA, BPN and GOT. Results show that the pruned GOT can generate simple and accurate formula for predicting the strength of HPC.

• 로봇학회논문지
• /
• 제17권2호
• /
• pp.133-141
• /
• 2022

This paper introduces model compression algorithms which make a deep neural network smaller and faster for embedded systems. The model compression algorithms can be largely categorized into pruning, quantization and knowledge distillation. In this study, gradual pruning, quantization aware training, and knowledge distillation which learns the activation boundary in the hidden layer of the teacher neural network are integrated. As a large deep neural network is compressed and accelerated by these algorithms, embedded computing boards can run the deep neural network much faster with less memory usage while preserving the reasonable accuracy. To evaluate the performance of the compressed neural networks, we evaluate the size, latency and accuracy of the deep neural network, DenseNet201, for image classification with CIFAR-10 dataset on the NVIDIA Jetson Xavier.