IEMEK Journal of Embedded Systems and Applications (대한임베디드공학회논문지)

Institute of Embedded Engineering of Korea (대한임베디드공학회)
권호 표지
DOI QR코드

DOI QR Code

CNN-based Weighted Ensemble Technique for ImageNet Classification

대용량 이미지넷 인식을 위한 CNN 기반 Weighted 앙상블 기법

  • Received : 2020.07.10
  • Accepted : 2020.08.12
  • Published : 2020.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

The ImageNet dataset is a large scale dataset and contains various natural scene images. In this paper, we propose a convolutional neural network (CNN)-based weighted ensemble technique for the ImageNet classification task. First, in order to fuse several models, our technique uses weights for each model, unlike the existing average-based ensemble technique. Then we propose an algorithm that automatically finds the coefficients used in later ensemble process. Our algorithm sequentially selects the model with the best performance of the validation set, and then obtains a weight that improves performance when combined with existing selected models. We applied the proposed algorithm to a total of 13 heterogeneous models, and as a result, 5 models were selected. These selected models were combined with weights, and we achieved 3.297% Top-5 error rate on the ImageNet test dataset.

Keywords

References

  1. M. Bojarski, D. Del Testa, D. Dworakowski, B. Firner, B. Flepp, P. Goyal, L. D. Jackel, M. Monfort, U. Muller, J. Zhang, X. Zhang, J. Zhao, K. Zieba, "End to end learning for self-driving cars.", arXiv preprint arXiv:1604.07316, 2016.
  2. Y. Sun, D. Liang, X. Wang, X. Tang, "Deepid3: Face Recognition with Very Deep Neural Networks.", arXiv preprint arXiv:1502.00873, 2015.
  3. M. A. Wani, F. A. Bhat, S. Afzal, A. I. Khan, "Supervised Deep Learning in Fingerprint Recognition. Advances in Deep Learning.", Springer, Singapore, pp. 111-132, 2020.
  4. H. Jung, M. K. Choi, J. Jung, J. H. Lee, S. Kwon, W. Young Jung, "ResNet-based Vehicle Classification and Localization in Traffic Surveillance Systems.", Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 61-67, 2017.
  5. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A.C. Berg, L. Fei-Fei, "ImageNet Large Scale Visual Recognition Challenge," International Journal of Computer Vision, Vol. 115, pp. 211-252, 2015. https://doi.org/10.1007/s11263-015-0816-y
  6. Krizhevsky, Alex, Ilya Sutskever, Geoffrey E. Hinton. "Imagenet Classification with Deep Convolutional Neural Networks." Advances in neural information processing systems, pp. 1097-1105, 2012.
  7. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, A. Rabinovich, "Going Deeper with Convolutions.", In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1-9, 2015.
  8. SIMONYAN, Karen ZISSERMAN, Andrew. "Very Deep Convolutional Networks for Large-scale Image Recognition.", arXiv preprint arXiv:1409.1556, 2014.
  9. K. He, X. Zhang, S. Ren, J. Sun, "Deep Residual Learning for Image Recognition." Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
  10. M. Tan, Q. V. Le. "Efficientnet: Rethinking Model Scaling for Convolutional Neural Networks." arXiv preprint arXiv:1905.11946 (2019).
  11. B. Zoph, Q. V. Le, "Neural Architecture Search with Reinforcement Learning." arXiv preprint arXiv:1611.01578 (2016).
  12. HU, Jie SHEN, Li SUN, Gang. "Squeeze-and-excitation Networks.", In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7132-7141, 2018.
  13. F. Iandola, M. Moskewicz, S. Karayev, R. Girshick, T. Darrell, K. Keutzer, "Densenet: Implementing Efficient Convnet Descriptor Pyramids." arXiv preprint arXiv:1404.1869 (2014).
  14. S. Xie, R. Girshick, P. Dollár, Z. Tu, K. He, "Aggregated Residual Transformations for Deep Neural Networks." Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1492-1500, 2017.
  15. D. MATTHEW, R. FERGUS, "Visualizing and Understanding Convolutional Neural Networks.", In Proceedings of the 13th European Conference Computer Vision and Pattern Recognition, Zurich, Switzerland, pp. 6-12, 2014.
  16. C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, Z. Wojna, "Rethinking the Inception Architecture for Computer Vision." Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2818-2826, 2016.
  17. C. Szegedy, S. Ioffe, V. Vanhoucke, A. A. Alemi, "Inception-v4, Inception-resnet and the Impact of Residual Connections on Learning." Thirty-first AAAI conference on artificial intelligence. 2017.