Journal of Information Processing Systems

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Evaluation of Artificial Intelligence-Based Denoising Methods for Global Illumination

  • Received : 2020.09.17
  • Accepted : 2020.12.29
  • Published : 2021.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

As the demand for high-quality rendering for mixed reality, videogame, and simulation has increased, global illumination has been actively researched. Monte Carlo path tracing can realize global illumination and produce photorealistic scenes that include critical effects such as color bleeding, caustics, multiple light, and shadows. If the sampling rate is insufficient, however, the rendered results have a large amount of noise. The most successful approach to eliminating or reducing Monte Carlo noise uses a feature-based filter. It exploits the scene characteristics such as a position within a world coordinate and a shading normal. In general, the techniques are based on the denoised pixel or sample and are computationally expensive. However, the main challenge for all of them is to find the appropriate weights for every feature while preserving the details of the scene. In this paper, we compare the recent algorithms for removing Monte Carlo noise in terms of their performance and quality. We also describe their advantages and disadvantages. As far as we know, this study is the first in the world to compare the artificial intelligence-based denoising methods for Monte Carlo rendering.

Keywords

References

  1. J. T. Kajiya, "The rendering equation," in Proceedings of the 13th Annual Conference on Computer Graphics And Interactive Techniques, Dallas, TX, 1986, pp. 143-150.
  2. E. Veach, "Robust Monte Carlo methods for light transport simulation," Ph.D. dissertation, Stanford University, Stanford, CA, 1997.
  3. H. W. Jensen, "Global illumination using photon maps," in Rendering Techniques '96. Vienna, Austria: Springer, 1996, pp. 21-30.
  4. Z. Zeng, L. Wang, B. B. Wang, C. M. Kang, and Y. N. Xu, "Denoising stochastic progressive photon mapping renderings using a multi-residual network," Journal of Computer Science and Technology, vol. 35, pp. 506-521, 2020. https://doi.org/10.1007/s11390-020-0264-1
  5. K. Vardis, "Efficient illumination algorithms for global illumination in interactive and real-time rendering," Ph.D. dissertation, Athens University of Economics & Business, Greece, 2016.
  6. T. Chen, J. Shi, J. Yang, and G. Li, "Enhancing network cluster synchronization capability based on artificial immune algorithm," Human-centric Computing and Information Sciences, vol. 9, article no. 3, 2019. https://doi.org/10.1186/s13673-019-0164-y
  7. P. Dutre, K. Bala, and P. Bekaert, Advanced Global Illumination. Boca Raton, FL: AK Peters/CRC Press, 2006.
  8. M. Mara, M. McGuire, B. Bitterli, and W. Jarosz, "An efficient denoising algorithm for global illumination," in Proceedings of High Performance Graphics, Los Angeles, CA, 2017.
  9. Y. Zhang, H. Wang, and X. Fan, "Algorithm for detection of fire smoke in a video based on wavelet energy slope fitting," Journal of Information Processing Systems, vol. 16, no. 3, pp. 557-571, 2020. https://doi.org/10.3745/JIPS.01.0054
  10. R. Wan, L. Ding, N. Xiong, W. Shu, and L. Yang, "Dynamic dual threshold cooperative spectrum sensing for cognitive radio under noise power uncertainty," Human-centric Computing and Information Sciences, vol. 9, article no. 22, 2019. https://doi.org/10.1186/s13673-019-0181-x
  11. A. Buades, B. Coll, and J. M. Morel, "A non-local algorithm for image denoising," in Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), San Diego, CA, 2005, pp. 60-65.
  12. J. Lv and X. Luo, "Image denoising via fast and fuzzy non-local means algorithm," Journal of Information Processing Systems, vol. 15, no. 5, pp. 1108-1118, 2019. https://doi.org/10.3745/jips.02.0122
  13. K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, "Image denoising by sparse 3-D transform-domain collaborative filtering," IEEE Transactions on Image Processing, vol. 16, no. 8, pp. 2080-2095, 2007. https://doi.org/10.1109/TIP.2007.901238
  14. L. Fan, F. Zhang, H. Fan, and C. Zhang, "Brief review of image denoising techniques," Visual Computing for Industry, Biomedicine, and Art, vol. 2, article no. 7, 2019. https://doi.org/10.1186/s42492-019-0016-7
  15. N. K. Kalantari, S. Bako, and P. Sen, "A machine learning approach for filtering Monte Carlo noise," ACM Transactions on Graphics, vo. 34, no. 4, article no. 122, 2015. https://doi.org/10.1145/2766977
  16. N. K. Kalantari, "Utilizing machine learning for filtering general Monte Carlo noise," Ph.D. dissertation, University of California, Santa Barbara, CA, 2015.
  17. S. Bako, T. Vogels, B. McWilliams, M. Meyer, J. Novak, A. Harvill, P. Sen, T. Derose, and F. Rousselle, "Kernel-predicting convolutional networks for denoising Monte Carlo renderings," ACM Transactions on Graphics, vol. 36, no. 4, article no. 97, 2017. https://doi.org/10.1145/3072959.3073708
  18. C. Schied, A. Kaplanyan, C. Wyman, A. Patney, C. R. A. Chaitanya, J. Burgess, S. Liu, C. Dachsbacher, A. Lefohn, and M. Salvi, "Spatiotemporal variance-guided filtering: real-time reconstruction for path-traced global illumination," in Proceedings of High Performance Graphics, Los Angeles, CA, 2017, pp. 1-12.
  19. C. Schied, C. Peters, and C. Dachsbacher, "Gradient estimation for real-time adaptive temporal filtering," Proceedings of the ACM on Computer Graphics and Interactive Techniques, vol. 1, no. 2, article no. 24, 2018. https://doi.org/10.1145/3233301
  20. Y. Liu, C. Zheng, Q. Zheng, and H. Yuan, "Removing Monte Carlo noise using a Sobel operator and a guided image filter," The Visual Computer, vol. 34, no. 4, pp. 589-601, 2018. https://doi.org/10.1007/s00371-017-1363-z
  21. M. Gharbi, T. M. Li, M. Aittala, J. Lehtinen, and F. Durand, "Sample-based Monte Carlo denoising using a kernel-splatting network," ACM Transactions on Graphics, vol. 38, no. 4, article no. 125, 2019. https://doi.org/10.1145/3306346.3322954
  22. B. Bitterli, F. Rousselle, B. Moon, J. A. Iglesias-Guitian, D. Adler, K. Mitchell, W. Jarosz, and J. Novak, "Nonlinearly weighted first-order regression for denoising Monte Carlo renderings," Computer Graphics Forum, vol. 35, no. 4, pp. 107-117, 2016.
  23. D. Vicini, D. Adler, J. Novak, F. Rousselle, and B. Burley, "Denoising deep Monte Carlo renderings," Computer Graphics Forum, vol. 38, no. 1, pp. 316-327, 2019. https://doi.org/10.1111/cgf.13533
  24. H. Park, B. Moon, S. Kim, and S., E. Yoon, "P-RPF: pixel-based random parameter filtering for Monte Carlo rendering," in Proceedings of 2013 International Conference on Computer-Aided Design and Computer Graphics, Guangzhou, China, 2013, pp. 123-130.
  25. Z. Wang, X. Huang, and F. Huang, "A new image enhancement algorithm based on bidirectional diffusion," Journal of Information Processing Systems, vol. 16, no. 1, pp. 49-60, 2020. https://doi.org/10.3745/JIPS.04.0155
  26. J. Xie, L. Xu, and E. Chen, "Image denoising and inpainting with deep neural networks," Advances in Neural Information Processing Systems, vol. 25, pp. 341-349, 2012.
  27. B. Bayar and M. C. Stamm, "A deep learning approach to universal image manipulation detection using a new convolutional layer," in Proceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security, Vigo Galicia, Spain, 2016, pp. 5-10.
  28. K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, "Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising," IEEE Transactions on Image Processing, vol. 26, no. 7, pp. 3142-3155, 2017. https://doi.org/10.1109/TIP.2017.2662206
  29. K. M. Wong and T. T. Wong, "Deep residual learning for denoising Monte Carlo renderings," Computational Visual Media, vol. 5, no. 3, pp. 239-255, 2019. https://doi.org/10.1007/s41095-019-0142-3
  30. X. Yang, D. Wang, W. Hu, L. J. Zhao, B. C. Yin, Q. Zhang, X. P. Wei, and H. Fu, "DEMC: a deep dual-encoder network for denoising Monte Carlo rendering," Journal of Computer Science and Technology, vol. 34, no. 5, pp. 1123-1135, 2019. https://doi.org/10.1007/s11390-019-1964-2
  31. M. Kettunen, E. Harkonen, and J. Lehtinen, "Deep convolutional reconstruction for gradient-domain rendering," ACM Transactions on Graphics, vol. 38, no. 4, pp. 126, 2019. https://doi.org/10.1145/3306346.3323038
  32. J. Talbot, D. Cline, and P. K. Egbert, "Importance resampling for global illumination," in Proceedings of the Eurographics Symposium on Rendering Techniques, Konstanz, Germany, 2005, pp. 139-146.
  33. B. Bitterli, C. Wyman, M. Pharr, P. Shirley, A. Lefohn, and W. Jarosz, "Spatiotemporal reservoir resampling for real-time ray tracing with dynamic direct lighting," ACM Transactions on Graphics, vol. 39, no. 4, article no. 148, 2020. https://doi.org/10.1145/3386569.3392481
  34. M. Pharr, W. Jakob, and G. Humphreys, Physically Based Rendering: From Theory to Implementation, 3rd ed. Cambridge, MA: Morgan Kaufmann, 2014.
  35. P. Gallinari, Y. Lecun, S. Thiria, and F. F. Soulie, "Distributed associative memories: a comparison," in Proceedings of COGNITIVA, Paris, La Villette, 1987.
  36. D. G. Mixon and S. Villar, "Sunlayer: stable denoising with generative networks," 2018 [Online]. Available: https://arxiv.org/abs/1803.09319.
  37. N. K. Kalantari and P. Sen, "Removing the noise in Monte Carlo rendering with general image denoising algorithms," Computer Graphics Forum, vol. 32, no. 2pt1, pp. 93-102, 2013.
  38. T. Vogels, F. Rousselle, B. McWilliams, G. Rothlin, A. Harvill, D. Adler, M. Meyer, and J. Novak, "Denoising with kernel prediction and asymmetric loss functions," ACM Transactions on Graphics, vol. 37, no. 4, article no. 124, 2018. https://doi.org/10.1145/3197517.3201388
  39. H. Dahlberg, D. Adler, and J. Newlin, "Machine-learning denoising in feature film production," in Proceedings of ACM SIGGRAPH 2019 Talks, Los Angeles, CA, 2019.
  40. S. Laine, T. Karras, J. Lehtinen, and T. Aila, "High-quality self-supervised deep image denoising," Advances in Neural Information Processing Systems, vol. 32, pp. 6970-6980, 2019.
  41. C. R. A. Chaitanya, A. S. Kaplanyan, C. Schied, M. Salvi, A. Lefohn, D. Nowrouzezahrai, and T. Aila, "Interactive reconstruction of Monte Carlo image sequences using a recurrent denoising autoencoder," ACM Transactions on Graphics, vol. 36, no. 4, article no. 98, 2017. https://doi.org/10.1145/3072959.3073601
  42. A. Keller, L. Fascione, M. Fajardo, I. Georgiev, P. Christensen, J. Hanika, C. Eisenacher, and G. Nichols, "The path tracing revolution in the movie industry," in Proceedings of ACM SIGGRAPH 2015 Courses, Los Angeles, CA, 2015.
  43. A. Alsaiari, R. Rustagi, M. M. Thomas, and A. G. Forbes, "Image denoising using a generative adversarial network," in Proceedings of 2019 IEEE 2nd International Conference on Information and Computer Technologies (ICICT), Kahului, HI, 2019, pp. 126-132.
  44. S. Agarwal, A. Agarwal, and M. Deshmukh, "Denoising images with varying noises using autoencoders," in Computer Vision and Image Processing. Singapore: Springer, 2019, pp. 3-14.
  45. H. Dai and L. Shao, "Pointae: point auto-encoder for 3d statistical shape and texture modelling," in Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea, 2019, pp. 5410-5419.
  46. K. P. Kumar, "Fuzzy-based machine learning algorithm for intelligent systems," in Data Management, Analytics and Innovation. Singapore: Springer, 2019, pp. 321-339
  47. N. Chauhan and B. J. Choi, "Denoising approaches using fuzzy logic and convolutional autoencoders for human brain MRI image," International Journal of Fuzzy Logic and Intelligent Systems, vol. 19, no. 3, pp. 135-139, 2019. https://doi.org/10.5391/ijfis.2019.19.3.135
  48. B. Costa and J. Jain, "Fuzzy deep stack of autoencoders for dealing with data uncertainty," in Proceedings of 2019 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), New Orleans, LA, 2019, pp. 1-6.
  49. J. Lehtinen, T. Aila, J. Chen, S. Laine, and F. Durand, "Temporal light field reconstruction for rendering distribution effects," in Proceedings of ACM SIGGRAPH 2011 papers, Vancouver, Canada, 2011, pp. 1-12.
  50. J. Lehtinen, T. Aila, S. Laine, and F. Durand, "Reconstructing the indirect light field for global illumination," ACM Transactions on Graphics, vol. 31, no. 4, article no. 51, 2012. https://doi.org/10.1145/2185520.2185547
  51. T, Hachisuka, W. Jarosz, R. P. Weistroffer, K. Dale, G. Humphreys, M. Zwicker, and H. W. Jensen, "Multidimensional adaptive sampling and reconstruction for ray tracing," in Proceedings of ACM SIGGRAPH 2008 papers, Los Angeles, CA, 2008, pp. 1-10.
  52. P. Sen and S. Darabi, "On filtering the noise from the random parameters in Monte Carlo rendering," ACM Transactions on Graphics, vol. 31, no. 3, article no. 18, 2012. https://doi.org/10.1145/2167076.2167083
  53. P. Bauszat, M. Eisemann, S. John, and M. Magnor, "Sample-based manifold filtering for interactive global illumination and depth of field," Computer Graphics Forum, vol. 34, no. 1, pp. 265-276, 2015.
  54. Q. Zhang, Y. Li, F. Al-Turjman, X. Zhou, and X. Yang, "Transient ischemic attack analysis through noncontact approaches," Human-centric Computing and Information Sciences, vol. 10, article no. 16, 2020. https://doi.org/10.1186/s13673-020-00223-z
  55. B. Cantrell and N. Yates, Modeling the Environment: Techniques and Tools for the 3D Illustration of Dynamic Landscapes. Hoboken, NJ: John Wiley & Sons, 2012.
  56. X. Meng, Q. Zheng, V. Varshney, G. Singh, and M. Zwicker, "Real-time Monte Carlo denoising with the neural bilateral grid," in Proceedings of the 31st Eurographics Symposium on Rendering (EGSR): Digital Library Only Track, London, UK, 2020, pp. 13-24.
  57. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: from error visibility to structural similarity," IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, 2004. https://doi.org/10.1109/TIP.2003.819861
  58. F. Rousselle, C. Knaus, and M. Zwicker, "Adaptive sampling and reconstruction using greedy error minimization," ACM Transactions on Graphics, vol. 30, no. 6, pp. 1-12, 2011.