Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

Deep Learning-based SISR (Single Image Super Resolution) Method using RDB (Residual Dense Block) and Wavelet Prediction Network

RDB 및 웨이블릿 예측 네트워크 기반 단일 영상을 위한 심층 학습기반 초해상도 기법

  • NGUYEN, HUU DUNG (Department of Electronics Engineering, Korea Polytechnic University) ;
  • Kim, Eung-Tae (Department of Electronics Engineering, Korea Polytechnic University)
  • Received : 2019.07.22
  • Accepted : 2019.09.04
  • Published : 2019.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

Single image Super-Resolution (SISR) aims to generate a visually pleasing high-resolution image from its degraded low-resolution measurement. In recent years, deep learning - based super - resolution methods have been actively researched and have shown more reliable and high performance. A typical method is WaveletSRNet, which restores high-resolution images through wavelet coefficient learning based on feature maps of images. However, there are two disadvantages in WaveletSRNet. One is a big processing time due to the complexity of the algorithm. The other is not to utilize feature maps efficiently when extracting input image's features. To improve this problems, we propose an efficient single image super resolution method, named RDB-WaveletSRNet. The proposed method uses the residual dense block to effectively extract low-resolution feature maps to improve single image super-resolution performance. We also adjust appropriated growth rates to solve complex computational problems. In addition, wavelet packet decomposition is used to obtain the wavelet coefficients according to the possibility of large scale ratio. In the experimental result on various images, we have proven that the proposed method has faster processing time and better image quality than the conventional methods. Experimental results have shown that the proposed method has better image quality by increasing 0.1813dB of PSNR and 1.17 times faster than the conventional method.

단일 영상 초해상도 (Single Image Super-Resolution - SISR)기법은 카메라로 획득된 저해상도 영상에 필터 기반의 연산을 적용하여 좋은 화질의 고해상도 영상을 복원하는 과정이다. 최근에 심층 합성곱 신경망 학습의 발전에 따라 단일 영상 초해상도에 적용되는 심층 학습 기법들은 좋은 성과를 보여 주고 있다. 그 대표적인 방법으로 영상의 특징 맵 기반 웨이블릿 계수 학습을 통해 고해상도 영상을 복원하는 WaveletSRNet이 있다. 하지만 복잡한 알고리즘으로 인해 계산량이 증대되어 처리 속도가 늦고 특징 추출할 때 특징 맵을 효율적으로 활용하지 못 한다는 단점을 가지고 있다. 이를 개선하기 위해 본 논문에서는 단일 영상 초해상도 RDB-WaveletSRNet 기법을 제안한다. 제안된 기법은 잔여밀집블록(Residual Dense Block)을 사용하여 저해상도의 특징 맵을 효과적으로 추출하여 초해상도의 성능을 향상시키고 적절한 성장률을 설정하여 복잡한 계산량 문제까지 해결하였다. 또한 웨이블릿 패킷 분해를 사용하여 확대율에 맞게 웨이블릿 계수를 획득하므로 높은 확대율의 단일 영상 초해상도를 얻게 하였다. 다양한 영상에 대한 실험을 통하여, 제안하는 기법이 기존 기법보다 수행시간이 빠르며 영상 품질도 우수함을 입증하였다. 제안하는 방법은 기존 방법보다 화질은 PSNR 0.1813dB만큼 우수하며 속도는 1.17배 빠른 것을 실험을 통해 확인하였다.

Keywords

References

  1. X. Zhang and X. Wu, "Image Interpolation by Adaptive 2-D Autoregressive Modeling and Soft-Decision Estimation," in IEEE Trans. Image Process., 2008, https://doi.org/10.1109/tip.2008.924279
  2. Y. Romano, M. Protter and M. Elad, "Single Image Interpolation Via Adaptive Nonlocal Sparsity-Based Modeling," in IEEE Trans. Image Process., 2014, https://doi.org/10.1109/tip.2014.2325774
  3. W. Ye and K. Ma, "Convolutional Edge Diffusion for Fast Contrast-guided Image Interpolation," in IEEE Signal Process. Lett., 2016, https://doi.org/10.1109/lsp.2016.2571738
  4. C. Ji, X. He, Y. Fu, Z. Liang, and L. Qing, ''An edge directed interpolation algorithm based on regularization,'' J. Electron. Inf. Technol., vol. 36, no. 2, pp. 293-297, 2014.
  5. X. Gao and H. Xiong, "A hybrid wavelet convolution network with sparse-coding for image super-resolution," in ICIP, 2016, https://doi.org/10.1109/icip.2016.7532596
  6. K. Zhang, X. Gao, D. Tao and X. Li, "Single Image Super-Resolution With Non-Local Means and Steering Kernel Regression," in IEEE Trans. Image Process., 2012, https://doi.org/10.1109/tip.2012.2208977
  7. V. Papyan and M. Elad, "Multi-Scale Patch-Based Image Restoration," in IEEE Trans. Image Process., 2016, https://doi.org/10.1109/tip.2015.2499698
  8. C. Ren, X. He and T. Q. Nguyen, "Single Image Super-Resolution via Adaptive High-Dimensional Non-Local Total Variation and Adaptive Geometric Feature," in IEEE Trans. Image Process., 2017, https://doi.org/10.1109/tip.2016.2619265
  9. S. Mallat, "Wavelets for a vision," in Proc. IEEE, 1996, https://doi.org/10.1109/5.488702
  10. H. Huang, R. He, Z. Sun and T. Tan, "Wavelet-SRNet: A Wavelet-Based CNN for Multi-Scale Face Super Resolution," in ICCV, 2017, https://doi.org/10.1109/iccv.2017.187
  11. Y. Matsuo and S. Sakaida, "Super-resolution for 2K/8K television using wavelet-based image registration," in IEEE GlobalSIP, 2017, https://doi.org/10.1109/globalsip.2017.8308668
  12. Y. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu, "Residual dense network for image super-resolution," in CVPR, 2018, https://doi.org/10.1109/cvpr.2018.00262
  13. C. Dong, C. C. Loy, K. He, and X. Tang, "Learning a Deep Convolutional Network for Image Super-Resolution," in ECCV, 2014, https://doi.org/10.1007/978-3-319-10593-2_13
  14. J. Bruna, P. Sprechmann, and Y. LeCun, "Super-Resolution with Deep Convolutional Sufficient Statistics," in ICLR, 2016, https://arxiv.org/abs/1511.05666
  15. S. Hitawala, Y. Li, X. Wang, D. Yang, "Image Super-Resolution Using VDSR-ResNeXt and SRCGAN," in CVPR, 2018, https://arxiv.org/abs/1810.05731
  16. S. G. Mallat, "A theory for multiresolution signal decomposition: the wavelet representation," in TPAMI, 1989, https://doi.org/10.1109/34.192463
  17. H. Chang, D.-Y. Yeung and Y. Xiong, "Super-resolution through neighbor embedding," in CVPR, 2004, https://doi.org/10.1109/CVPR.2004.1315043
  18. W. Zhang, Y. Liu, C. Dong, Y. Qiao, "RankSRGAN: Generative Adversarial Networks with Ranker for Image Super-Resolution," in CVPR, 2019, https://arxiv.org/abs/1908.06382
  19. K. He, X. Zhang, S. Ren and J. Sun, "Deep Residual Learning for Image Recognition," in CVPR, 2015, https://doi.org/10.1109/cvpr.2016.90
  20. T. Tong, G. Li, X. Liu and Q. Gao, "Image Super-Resolution Using Dense Skip Connections," in ICCV, 2017, https://doi.org/10.1109/iccv.2017.514
  21. Z. Liu, P. Luo, X. Wang and X. Tang, "Deep Learning Face Attributes in the Wild," in ICCV, 2015, https://doi.org/10.1109/iccv.2015.425
  22. G. Huang, Z. Liu, L. v. d. Maaten and K. Q. Weinberger, "Densely Connected Convolutional Networks,"in Computer Vision and Pattern Recognition, 2017, https://arxiv.org/abs/1608.06993