Journal of the Korea Institute of Military Science and Technology (한국군사과학기술학회지)

The Korea Institute of Military Science and Technology (한국군사과학기술학회)
권호 표지
DOI QR코드

DOI QR Code

Improving Confidence in Synthetic Infrared Image Refinement using Grad-CAM-based Explainable AI Techniques

Grad-CAM 기반의 설명가능한 인공지능을 사용한 합성 이미지 개선 방법

  • Taeho Kim (Mechanical Engineering Research Institute, KAIST) ;
  • Kangsan Kim (Department of Aerospace Engineering, KAIST) ;
  • Hyochoong Bang (Department of Aerospace Engineering, KAIST)
  • 김태호 (한국과학기술원 기계기술연구소) ;
  • 김강산 (한국과학기술원 항공우주공학과) ;
  • 방효충 (한국과학기술원 항공우주공학과)
  • Received : 2024.07.10
  • Accepted : 2024.09.13
  • Published : 2024.12.05
Download PDF
⟨ Previous Next ⟩

Abstract

Infrared imaging is a powerful non-destructive and non-invasive technique to detect infrared radiations and capture valuable insights inaccessible through the visible spectrum. It has been widely used in the military for reconnaissance, hazard detection, night vision, guidance systems, and countermeasures. There is a huge potential for machine learning models to improve trivial infrared imaging tasks in military applications. One major roadblock is the scarcity and control over infrared imaging datasets related to military applications. One possible solution is to use synthetic infrared images to train machine learning networks. However, synthetic IR images present a domain gap and produce weak learning models that do not generalize well. We investigate adversarial networks and Explainable AI(XAI) techniques to refine synthetic infrared imaging data, enhance their realism, and synthesize refiner networks with XAI. We use a U-Net-based refiner network to refine synthetic infrared data and a PatchGAN discriminator to distinguish between the refined and real IR images. Grad-CAM XAI technique is used for network synthesis. We also analyzed the frequency domain patterns and power spectra of real and synthetic infrared images to find key attributes to distinguish real from synthetic. We tested our refined images on the realism benchmarks using frequency domain analysis.

Keywords

Acknowledgement

본 논문은 국방과학연구소를 통해 인공지능 비행제어 특화연구실 산하 인공지능 기반 합성 센서 영상 분석 및 개선기법 연구(IC-05) 관련 지원을 받아 수행되었음(UD230014SD).

References

  1. Y. Wang, C. Peng, H. Zhang, and H. Q. Le, "Wavelength modulation imaging with tunable mid-infrared semiconductor laser: spectroscopic and geometrical effects," Optics Express, Vol. 12, No. 21, pp. 5243-5257, 2004. 
  2. I. L. Rasskazov, N. Spegazzini, P. S. Carney, and R. Bhargava, "Dielectric sphere clusters as a model to understand infrared spectroscopic imaging data recorded from complex samples," Analytical chemistry, Vol. 89, No. 20, pp. 10813-10818, 2017. 
  3. J. Huang, L. He, J. Wang, J. Xu, and L. Yuan, "Near-infrared hemicyanine fluorophores with optically tunable groups: A 'leap forward' for in vivo sensing and imaging," Synlett, 2023. 
  4. A. Mase, Y. Kogi, D. Kuwahara, Y. Nagayama, N. Ito, T. Maruyama, H. Ikezi, X. Wang, M. Inutake, T. Tokuzawa, et al., "Development and application of radar reflectometer using micro to infrared waves," Advances in physics: X, Vol. 3, No. 1, p. 1472529, 2018. 
  5. A. N. Sexton, "Regulation and Recruitment of Human Telomerase," Ph. D. Dissertation, UC Berkeley, 2014. 
  6. Thermo analytics, "EO/IR signature simulation software," www.thermoanalytics.com, 2024. 
  7. Pregagis, "Vega prime: Comprehensive visualization toolkit," www.presagis.com/en/product/vega-prime/, 2024. 
  8. Oktal, "OKTAL-SE synthetic enviroment simulator," https://www.oktal-se.fr/, 2024. 
  9. I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative adversarial networks. arxiv 2014," arXiv preprint arXiv:1406.2661, Vol. 10, 2014. 
  10. J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, "Unpaired image-to-image translation using cycle-consistent adversarial networks," Proceedings of the IEEE international conference on computer vision, pp. 2223-2232, 2017. 
  11. T. Kim and H. Bang, "Fractal texture enhancement of simulated infrared images using a cnn-based neural style transfer algorithm with a histogram matching technique," Sensors, Vol. 23, No. 1, p. 422, 2022. 
  12. R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, "Grad-cam: Visual explanations from deep networks via gradient-based localization," Proceedings of the IEEE international conference on computer vision, pp. 618-626, 2017. 
  13. P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, "Image-to-image translation with conditional adversarial networks," Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1125-1134, 2017. 
  14. A. Chattopadhay, A. Sarkar, P. Howlader, and V. N. Bala- subramanian, "Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks," 2018 IEEE winter conference on applications of computer vision (WACV), pp. 839-847, IEEE, 2018. 
  15. M. G. Core, H. C. Lane, M. Van Lent, D. Gomboc, S. Solomon, M. Rosenberg, "Building Explainable Artificial Intelligence systems," AAAI, pp. 1766-1773, 2006. 
  16. M. Van Lent, W. Fisher, M. Mancuso, "An Explainable Artificial Intelligence system for small-unit tactical behavior," Proceedings of the National Conference on Artificial Intelligence, pp. 900-907, 2004. 
  17. G. Alicioglu, B. Sun, "A survey of visual analytics for Explainable Artificial Intelligence methods," Computers & Graphics, Vol. 102, pp. 502-520, 2022. 
  18. S. Mohseni, N. Zarei, E. D. Ragan, "A multidisciplinary survey and framework for design and evaluation of explainable AI systems," ACM Trans. Interact. Intell. Syst.(TiiS), 11(3-4), pp. 1-45, 2021. 
  19. S. Ali, T. Abuhmed, S. El-Sappagh, K. Muhammad, J. M. Alonso-Moral, R. Confalonieri, R., Guidotti, J. Del Ser, N. Diaz-Rodriguez, and F. Herrera, "Explainable Artificial Intelligence(XAI): What we know and what is left to attain Trustworthy Artificial Intelligence," Information Fusion, Vol. 99, p. 101805, 2023.