실천공학교육논문지 (Journal of Practical Engineering Education)

한국실천공학교육학회 (Korean Institute for Pratical Engineering Education)
권호 표지
DOI QR코드

DOI QR Code

이미지 초해상도 향상을 위한 피드백 네트워크 연구

Study on Feedback Networks for Enhanced Image Super-Resolution

  • 정헌석 (극동대학교 에너지IT공학과) ;
  • 허재혁 (극동대학교 에너지IT공학과) ;
  • 양수미 (극동대학교 에너지IT공학과) ;
  • 곽성범 (주식회사 위즈윙)
  • Hunsuk Chung (Far East University, Department of Energy IT Engineering) ;
  • Jaehyeok Hur (Far East University, Department of Energy IT Engineering) ;
  • Sumi Yang (Far East University, Department of Energy IT Engineering) ;
  • Seongbeom Kwak (Wizwing Co., Ltd.)
  • 투고 : 2024.08.31
  • 심사 : 2024.09.24
  • 발행 : 2024.10.31
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

딥러닝 기술의 급속한 발전은 단일 이미지 초해상도(SR, super-resolution) 성능 향상에 큰 기여를 하였다. 그러나 대부분의 기존 딥러닝 기반 이미지 SR 네트워크는 정보 흐름이 순방향으로만 이루어져 성능에 한계를 보인다. 본 연구에서는 정확한 이미지 SR을 위한 피드백 네트워크를 제안한다. 이 피드백 네트워크는 여러 상위 수준의 특징을 재라우팅하여 하위 수준 특징 표현을 효과적으로 강화한다. 우리는 여러 잔차 밀도 모듈을 연속적으로 구성하고, 이를 시간에 따라 반복적으로 적용한다. 인접한 두 시간 단계 사이의 다중 피드백 연결은 충분한 문맥 정보를 가지지 못한 하위 수준 특징을 개선하기 위해, 큰 수용 필드에서 캡처된 여러 상위 수준 특징을 활용한다. 정교하게 설계된 피드백 모듈은 재라우팅된 상위 수준 특징에서 유용한 정보를 효율적으로 선택하고 이를 강화하여, 향상된 상위 수준 정보를 바탕으로 하위 수준 특징을 개선한다. 다양한 실험을 통해 제안된 방법이 객관적 및 주관적 평가에서 우수함을 입증하였다.

The rapid advancement of deep learning has significantly enhanced the performance of single image super-resolution (SR). However, most existing deep learning-based image SR networks only facilitate information flow in the forward direction, which limits their performance. In this study, we investigate a feedback network for precise image SR. This feedback network effectively enhances lower-level feature representation by rerouting multiple higher-level features. We sequentially construct several Residual Density Modules and deploy them repeatedly over time. Multiple feedback connections between two adjacent time steps leverage high-level features captured within a large receptive field to refine low-level features lacking sufficient contextual information. A carefully designed feedback module efficiently selects and enhances valuable information from the rerouted high-level features, thereby improving low-level features with enriched high-level information. Extensive experiments demonstrate that the proposed method outperforms existing approaches in both objective and subjective evaluations.

키워드

과제정보

본 논문은 2024년도 행정안정부 및 산업기술기획평가원(KEIT) 연구비 지원에 의한 연구(20025104)와, 2024년도 산업통상자원부 및 한국에너지기술평가 원(KETEP) 연구비 지원에 의한 연구임(20224000000070).

참고문헌

  1. A. Niu, K. Zhang, T. X. Pham, J. Sun, Y. Zhu, I. S. Kweon, and Y. Zhang, "CDPMSR: Conditional diffusion probabilistic models for single image super-resolution," 2023 IEEE International Conference on Image Processing (ICIP).
  2. Q. Ding and J. Yang, "Sparse-aware transformer for single image super-resolution, 2023 2nd international conference on cloud computing," Big Data Application and Software Engineering (CBASE), 2023.
  3. M. Mikaeili and H. S. Bilge, "Evaluating deep neural network models on ultrasound single image super resolution," 2023 Medical Technologies Congress (TIPTEKNO).
  4. C. Dong, C. C. Loy, K. He, and X. Tang, "Learning a deep convolutional network for image super-resolution," In ECCV, 2014.
  5. Y. Zhang, K. Li, K. Li, L. Wang, B. Zhong, and Y. Fu, "Image super-resolution using very deep residual channel attention networks," In ECCV, 2018.
  6. C. Dong, C. C. Loy, and X. Tang, "Accelerating the superresolution convolutional neural network," In ECCV, 2016.
  7. M. Haris, G. Shakhnarovich, and N. Ukita, "Deep backprojection networks for super-resolution," In CVPR, 2018.
  8. W. Han, S. Chang, D. Liu, M. Yu, M. Witbrock, and T. S. Huang, "Image super-resolution via dual-state recurrent networks," In CVPR, 2018.
  9. J. Kim, J. K. Lee, and K. M. Lee, "Deeply-recursive convolutional network for image super-resolution," In CVPR, 2016.
  10. J. Carreira, P. Agrawal, K. Fragkiadaki, and J. Malik, "Human pose estimation with iterative error feedback," In CVPR, 2016.
  11. X. Jin, Y. Chen, Z. Jie, J. Feng, and S. Yan, "Multi-path feedback recurrent neural networks for scene parsing," In AAAI, 2017.
  12. Z. Li, J. Yang, Z. Liu, X. Yang, G. Jeon, and W. Wu, "Feedback network for image super-resolution." In CVPR, 2019.
  13. A. R. Zamir, T. L. Wu, L. Sun, W. B. Shen, B. E. Shi, J. Malik, and S. Savarese, "Feedback networks," In CVPR, 2017.
  14. M. Liang, X. Hu, and B. Zhang, "Convolutional neural networks with intralayer recurrent connections for scene labeling," In NeurIPS, 2015.
  15. J. Kim, J. K. Lee, and K. M. Lee, "Accurate image superresolution using very deep convolutional networks," In CVPR, 2016.
  16. B. Lim, S. Son, H. Kim, S. Nah, and K. M. Lee, "Enhanced deep residual networks for single image super-resolution," In CVPRW, 2017.
  17. K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," In CVPR, 2016.
  18. T. Tong, G. Li, X. Liu, and Q. Gao, "Image super-resolution using dense skip connections," In ICCV, 2017.
  19. G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, "Densely connected convolutional networks," In CVPR, 2017.
  20. Y. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu, "Residual dense network for image super-resolution," In CVPR, 2018.
  21. X. Wang, K. Yu, S. Wu, J. Gu, Y. Liu, C. Dong, Y. Qiao, and C. Change Loy, "Esrgan: Enhanced super-resolution generative adversarial networks," In ECCV, 2018.
  22. K. He, X. Zhang, S. Ren, and J. Sun, "Delving Deepinto Rectifiers: Surpassing human-level performance on imagenet classification," In ICCV, 2015.
  23. M. Bevilacqua, A. Roumy, C. Guillemot, and M. L. Alberi-Morel, "Low-complexity single image superresolution based on nonnegative neighbor embedding," In BMVC, 2012.
  24. R. Zeyde, M. Elad, and M. Protter, "On single image scale-up using sparse-representations," In Curves and Surfaces, 2010.
  25. D. Martin, C. Fowlkes, D. Tal, and J. Malik, "A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics," In ICCV, 2001.
  26. J. B. Huang, A. Singh, and N. Ahuja, "Single image super-resolution from transformed self-exemplars," In CVPR, 2015.
  27. Y. Matsui, K. Ito, Y. Aramaki, A. Fujimoto, T. Ogawa, T. Yamasaki, and K. Aizawa, "Sketch-based manga retrieval using manga109 dataset," Multimedia Tools and Applications, vol. 76, pp. 21811-2183, 2017.
  28. Y. Tai, J. Yang, and X. Liu, "Image super-resolution via deep recursive residual network," In CVPR, 2017.
  29. D. Liu, B. Wen, Y. Fan, C. C. Loy, and T. S. Huang, "Non-local recurrent network for image restoration," In NeurIPS, 2018.