Journal of the Institute of Convergence Signal Processing (융합신호처리학회논문지)

The Korea Institute of Convergence Signal Processing (한국융합신호처리학회)
권호 표지
DOI QR코드

DOI QR Code

Super-Resolution Using NLSA Mechanism

비지역 희소 어텐션 메커니즘을 활용한 초해상화

  • Kim, Sowon (Department of Electronic Engineering, Pukyong National University) ;
  • Park, Hanhoon (Department of Electronic Engineering, Pukyong National University)
  • Received : 2022.03.16
  • Accepted : 2022.03.28
  • Published : 2022.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

With the development of deep learning, super-resolution (SR) methods have tried to use deep learning mechanism, instead of using simple interpolation. SR methods using deep learning is generally based on convolutional neural networks (CNN), but recently, SR researches using attention mechanism have been actively conducted. In this paper, we propose an approach of improving SR performance using one of the attention mechanisms, non-local sparse attention (NLSA). Through experiments, we confirmed that the performance of the existing SR models, IMDN, CARN, and OISR-LF-s can be improved by using NLSA.

딥러닝이 발전하면서 초해상화 기술은 단순 보간법(Interpolation)에서 벗어나 딥러닝을 활용해 발전하고 있다. 딥러닝을 사용한 초해상화 기술은 합성곱 신경망(Convolutional Neural Network, CNN) 기반의 연구가 일반적이지만, 최근에는 어텐션(Attention) 메커니즘을 활용한 초해상화 연구가 활발히 진행되고 있다. 본 논문에서는 어텐션 메커니즘 중 하나인 비지역 희소 어텐션(Non-Local Sparse Attention, NLSA)을 활용한 초해상화 성능 향상 방법을 제안한다. 실험을 통해 NLSA를 함께 활용하면 기존 초해상화 신경망 모델인 IMDN, CARN, OISR-LF-s의 성능이 향상되는 것을 확인할 수 있었다.

Keywords

Acknowledgement

본 연구는 산업통상자원부와 한국산업기술진흥원의 "지역혁신클러스터육성사업(R&D, P0004797)"으로 수행된 연구결과 입니다.

References

  1. Z. Wang, J. Chen, and S. C. H. Hoi, "Deep learning for image super-resolution: a survey," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, pp. 3365-3387, 2021. https://doi.org/10.1109/TPAMI.2020.2982166
  2. S. Kim and H. Park, "CG/VR image super-resolution using balanced attention mechanism," Journal of Korea Institute of Convergence Signal Processing, vol. 22, no. 4, pp. 156-163, 2021. https://doi.org/10.23087/JKICSP.2021.22.4.002
  3. Y. Zhang, K. Li, K. Li, L. Wang, B. Zhong, and Y. Fu, "Image super-resolution using very deep residual channel attention networks," Proc. ECCV, 2018.
  4. T. Dai, J. Cai, Y. Zhang, S.-T. Xia, and L. Zhang, "Second-order attention network for single image super-resolution," Proc. CVPR, pp. 11065-11074, 2019.
  5. N. Ahn, B. Kang, and K.-A. Sohn, "Fast, accurate, and lightweight super-resolution with cascading residual network," Proc. ECCV, 2018.
  6. Z. Hui, X. Wang, and X. Gao, "Fast and accurate single image super-resolution via information distillation network," Proc. CVPR, pp. 723-731, 2018.
  7. Z. Hui et al., "Lightweight image super-resolution with information multi-distillation network," Proc. ACM MM, 2019.
  8. X. He et al., "ODE-inspired network design for single image super-resolution," Proc. CVPR, 2019.
  9. Y. Mei, Y. Fan, and Y. Zhou, "Image super-resolution with non-local sparse attention," Proc. CVPR, 2021.
  10. F. Wang, H. Hu, and C. Shen, "BAM: a lightweight and efficient balanced attention mechanism for single image super resolution," arXiv preprint arXiv:2104.07566, 2021.
  11. E. Agustsson and R. Timofte, "NTIRE 2017 challenge on single image super-resolution: dataset and study," Proc. CVPRW, 2017.
  12. M. Bevilacqua, A. Roumy, C. Guillemot, and A. Morel, "Low complexity single image super-resolution based on nonnegative neighbor embedding," Proc. BMVC, 2012.
  13. R. Zeyde, M. Elad, and M. Protter, "On single image scale-up using sparse-representations," Proc. International Conference on Curves and Surfaces, pp. 711-730, 2010.
  14. D. Martin, C. Fowlkes, D. Tal, and J. Malik, "A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics," Proc. ICCV, vol. 2, pp. 416-423, 2001.
  15. R. Narita, K. Tsubota, T. Yamasaki, and K. Aizawa, "Sketch-based manga retrieval using deep features," Proc. ICDAR, pp. 49-53, 2017.
  16. https://github.com/dandingbudanding/BAM