Journal of the Korea Computer Graphics Society (한국컴퓨터그래픽스학회논문지)

Korea Computer Graphics Society (한국컴퓨터그래픽스학회)
권호 표지
DOI QR코드

DOI QR Code

Image Restoration Network with Adaptive Channel Attention Modules for Combined Distortions

적응형 채널 어텐션 모듈을 활용한 복합 열화 복원 네트워크

  • Lee, Haeyun (Department of Information and Communication Engineering, DGIST) ;
  • Cho, Sunghyun (Department of Information and Communication Engineering, DGIST)
  • 이해윤 (대구경북과학기술원 정보통신융합전공) ;
  • 조성현 (대구경북과학기술원 정보통신융합전공)
  • Received : 2019.05.29
  • Accepted : 2019.06.22
  • Published : 2019.07.14
Download PDF
⟨ Previous Next ⟩

Abstract

The image obtained from systems such as autonomous driving cars or fire-fighting robots often suffer from several degradation such as noise, motion blur, and compression artifact due to multiple factor. It is difficult to apply image recognition to these degraded images, then the image restoration is essential. However, these systems cannot recognize what kind of degradation and thus there are difficulty restoring the images. In this paper, we propose the deep neural network, which restore natural images from images degraded in several ways such as noise, blur and JPEG compression in situations where the distortion applied to images is not recognized. We adopt the channel attention modules and skip connections in the proposed method, which makes the network focus on valuable information to image restoration. The proposed method is simpler to train than other methods, and experimental results show that the proposed method outperforms existing state-of-the-art methods.

자율 주행 자동차나 소방 로봇과 같은 시스템에서 영상을 얻을 때 다양한 요인들로 인해 잡음, 블러와 같은 열화가 발생한다. 이런 열화된 영상에 직접 영상 분류와 같은 기술을 적용하기 어려워 열화 제거가 불가피하나 이러한 시스템들은 영상의 열화를 인식할 수 없어서 열화된 영상을 복원하는데 어려움이 있다. 본 논문에서는 영상에 적용된 열화를 인지하지 못하는 상황에서 여러 방법들로 열화된 영상으로부터 자연스럽고 선명한 영상을 복원하는 방법을 제안한다. 우리가 제안한 방법은 딥러닝 모델에 채널 어텐션 모듈과 스킵 커넥션을 사용하여 영상에 적용된 열화에 따라 복원에 필요한 채널에 높은 가중치를 적용해 복합 열화 영상의 복원을 진행한다. 이 방법은 다른 복합 열화 복원 방법에 비해 학습이 간단하고 기존의 다른 방법들에 비해 높은 복합 열화 복원 성능을 낸다.

Keywords

References

  1. C. Tomasi and R. Manduchi, "Bilateral filtering for gray and color images," in Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271), Jan 1998, pp. 839-846.
  2. L. I. Rudin, S. Osher, and E. Fatemi, "Nonlinear total variation based noise removal algorithms," Physica D: nonlinear phenomena, vol. 60, no. 1-4, pp. 259-268, 1992. https://doi.org/10.1016/0167-2789(92)90242-F
  3. S. Osher, M. Burger, D. Goldfarb, J. Xu, and W. Yin, "An iterative regularization method for total variation-based image restoration," Multiscale Modeling & Simulation, vol. 4, no. 2, pp. 460-489, 2005. https://doi.org/10.1137/040605412
  4. A. Buades, B. Coll, and J.-M. Morel, "A non-local algorithm for image denoising," in Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, vol. 2. IEEE, 2005, pp. 60-65.
  5. K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, "Image denoising by sparse 3-d transform-domain collaborative filtering," IEEE Transactions on image processing, vol. 16, no. 8, pp. 2080-2095, 2007. https://doi.org/10.1109/TIP.2007.901238
  6. T. Michaeli and M. Irani, "Nonparametric blind super-resolution," in Proceedings of the IEEE International Conference on Computer Vision, 2013, pp. 945-952.
  7. W. T. Freeman, T. R. Jones, and E. C. Pasztor, "Example-based super-resolution," IEEE Computer graphics and Applications, vol. 22, no. 2, pp. 56-65, 2002. https://doi.org/10.1109/38.988747
  8. M. Elad and M. Aharon, "Image denoising via sparse and redundant representations over learned dictionaries," IEEE Transactions on Image processing, vol. 15, no. 12, pp. 3736-3745, 2006. https://doi.org/10.1109/TIP.2006.881969
  9. K. Zhang,W. Zuo, Y. Chen, D. Meng, and L. Zhang, "Beyond a Gaussian denoiser: Residual learning of deep cnn for image denoising," IEEE Transactions on Image Processing, vol. 26, no. 7, pp. 3142-3155, 2017. https://doi.org/10.1109/TIP.2017.2662206
  10. T. Remez, O. Litany, R. Giryes, and A. M. Bronstein, "Deep class-aware image denoising," in Sampling Theory and Applications (SampTA), 2017 International Conference on. IEEE, 2017, pp. 138-142.
  11. K. Yu, C. Dong, L. Lin, and C. Change Loy, "Crafting a toolchain for image restoration by deep reinforcement learning," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 2443-2452.
  12. M. Suganuma, X. Liu, and T. Okatani, "Attention-based adaptive selection of operations for image restoration in the presence of unknown combined distortions," in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
  13. D. G. S. B. M. Irani, "Super-resolution from a single image," in Proceedings of the IEEE International Conference on Computer Vision, Kyoto, Japan, 2009, pp. 349-356.
  14. S. Cho, J. Wang, and S. Lee, "Handling outliers in non-blind image deconvolution," in 2011 International Conference on Computer Vision. IEEE, 2011, pp. 495-502.
  15. C. Dong, C. C. Loy, K. He, and X. Tang, "Image super-resolution using deep convolutional networks," IEEE transactions on pattern analysis and machine intelligence, vol. 38, no. 2, pp. 295-307, 2016. https://doi.org/10.1109/TPAMI.2015.2439281
  16. J. Kim, J. Kwon Lee, and K. Mu Lee, "Accurate image super-resolution using very deep convolutional networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 1646-1654.
  17. L. Xu, J. S. Ren, C. Liu, and J. Jia, "Deep convolutional neural network for image deconvolution," in Advances in Neural Information Processing Systems, 2014, pp. 1790-1798.
  18. Y. Tai, J. Yang, X. Liu, and C. Xu, "Memnet: A persistent memory network for image restoration," in Proceedings of the IEEE international conference on computer vision, 2017, pp. 4539-4547.
  19. D. Liu, B. Wen, Y. Fan, C. C. Loy, and T. S. Huang, "Non-local recurrent network for image restoration," arXiv preprint arXiv:1806.02919, 2018.
  20. J. Hu, L. Shen, and G. Sun, "Squeeze-and-excitation networks," in IEEE Conference on Computer Vision and Pattern Recognition, 2018.
  21. J. Park, S.Woo, J.-Y. Lee, and I. S. Kweon, "Bam: Bottleneck attention module," arXiv preprint arXiv:1807.06514, 2018.
  22. S. Woo, J. Park, J.-Y. Lee, and I. So Kweon, "Cbam: Convolutional block attention module," in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 3-19.
  23. Y. Zhang, K. Li, K. Li, L. Wang, B. Zhong, and Y. Fu, "Image super-resolution using very deep residual channel attention networks," in Computer Vision - ECCV 2018, V. Ferrari, M. Hebert, C. Sminchisescu, and Y. Weiss, Eds. Cham: Springer International Publishing, 2018, pp. 294-310.
  24. X. Cheng, X. Li, J. Yang, and Y. Tai, "Sesr: Single image super resolution with recursive squeeze and excitation networks," in 2018 24th International Conference on Pattern Recognition (ICPR), Aug 2018, pp. 147-152.
  25. D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," arXiv preprint arXiv:1412.6980, 2014.
  26. A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. De-Vito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer, "Automatic differentiation in pytorch," 2017.

Cited by

  1. A Systematic Review on Cognitive Tasks Based on Virtual Reality for Assessing Episodic Memory of Older Adults vol.28, pp.1, 2020, https://doi.org/10.14519/kjot.2020.28.1.07