Journal of the Institute of Convergence Signal Processing (융합신호처리학회논문지)

The Korea Institute of Convergence Signal Processing (한국융합신호처리학회)
권호 표지
DOI QR코드

DOI QR Code

HSE Block : Automatic Optimization of the Number of Convolutional Layer Filters using SE Block

HSE Block : SE Block을 활용한 합성곱 신경망 필터 수 자동 최적화

  • Tae-Wook Kim (Division of Software, Yonsei University) ;
  • Hyeon-Jin Jung (Department of Computer & Telecommunications Engineering, Yonsei University) ;
  • Ellen J. Hong (Division of Software, Yonsei University)
  • 김태욱 (연세대학교 소프트웨어학부) ;
  • 정현진 (연세대학교 컴퓨터정보통신공학부) ;
  • 홍정희 (연세대학교 소프트웨어학부)
  • Received : 2022.09.13
  • Accepted : 2022.09.29
  • Published : 2022.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we are going to study how we can automatically determine the number of convolutional filters for the optimal model without a search algorithm. This paper proposes HSE Block by connecting SE Block proposed in SENet to a convolutional neural network and connecting a convolutional neural network not learned at the bottom. An experiment was conducted to increase the number of filters by one per 3 epoch using two datasets for the HSEBlock model and to increase the number of filters by the value in the filter. Based on this experiment, the model was constructed with multi-layer HSE Block instead of layer HSE Block, and the experiment was carried out using a dataset that was more difficult to learn than the one used in the previous experiment. The effect of HSE Block was verified by conducting an experiment with the number of HSE Blocks set to 2, 3, 4, and 5 on a dataset that is more difficult to learn than before.

본 논문은 탐색 알고리즘 없이 자동으로 모델의 합성곱 필터의 개수를 최적으로 결정할 방법에 대해 연구하고자 한다. 본 논문은 SENet에서 제안한 SE Block을 합성곱 신경망에 연결하고 하단의 학습하지 않는 합성곱 신경망을 연결한 HSE Block을 제안한다. HSE Block 모델에 두 개의 데이터셋을 이용하여 필터의 개수를 3 epoch 당 1개씩 증가시키는 실험과 필터 내의값에 따라 필터의 개수를 증가시키는 실험을 수행하였다. 이 실험을 바탕으로 한 층의 HSE Block이 아닌 다층의 HSE Block으로 모델을 구성하고, 기존의 실험할 때 사용한 데이터셋에 비해 더욱 학습하기 어려운 데이터셋을 사용하여 실험을 진행하였다. 기존보다 학습하기 어려운 데이터셋에 대해 HSE Block의 개수를 2개, 3개, 4개, 5개로 두고 실험을 수행함으로써 HSE Block의 효과를 검증하였다.

Keywords

Acknowledgement

이 성과는 정부(과학기술정보통신부)의 재원으로 한국연구재단의 지원을 받아 수행된 연구임 (No, 2022R1F1A1074273).

References

  1. Bergstra, J., & Bengio, Y. (2012). Random search for hyper-parameter optimization. Journal of machine learning research, 13(2).
  2. Liashchynskyi, P., & Liashchynskyi, P. (2019). Grid search, random search, genetic algorithm: A big comparison for NAS. arXiv preprint arXiv:1912.06059.
  3. Hu, J., Shen, L., & Sun, G. (2018). Squeezeand-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7132-7141).
  4. He, K., Zhang, X., Ren, S., & Sun, J. (2015). Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision (pp. 1026-1034).
  5. Muresan, H., & Oltean, M. (2017). Fruit recognition from images using deep learning. arXiv preprint arXiv:1712.00580.
  6. Li, Z., Li, F., Zhu, L., & Yue, J. (2020). Vegetable recognition and classification based on improved VGG deep learning network model. International Journal of Computational Intelligence Systems, 13(1), 559-564. https://doi.org/10.2991/ijcis.d.200425.001
  7. Roy, P., Ghosh, S., Bhattacharya, S., & Pal, U. (2018). Effects of degradations on deep neural network architectures. arXiv preprint arXiv:1807.10108.