Korean Journal of Materials Research (한국재료학회지)

Materials Research Society of Korea (한국재료학회)
권호 표지
DOI QR코드

DOI QR Code

Super-Resolution Transmission Electron Microscope Image of Nanomaterials Using Deep Learning

딥러닝을 이용한 나노소재 투과전자 현미경의 초해상 이미지 획득

  • Nam, Chunghee (Department of Electrical and Electronic Engineering, Hannam University)
  • 남충희 (한남대학교 전기전자공학과)
  • Received : 2022.07.26
  • Accepted : 2022.08.22
  • Published : 2022.08.27
Download PDF
⟨ Previous Next ⟩

Abstract

In this study, using deep learning, super-resolution images of transmission electron microscope (TEM) images were generated for nanomaterial analysis. 1169 paired images with 256 × 256 pixels (high resolution: HR) from TEM measurements and 32 × 32 pixels (low resolution: LR) produced using the python module openCV were trained with deep learning models. The TEM images were related to DyVO4 nanomaterials synthesized by hydrothermal methods. Mean-absolute-error (MAE), peak-signal-to-noise-ratio (PSNR), and structural similarity (SSIM) were used as metrics to evaluate the performance of the models. First, a super-resolution image (SR) was obtained using the traditional interpolation method used in computer vision. In the SR image at low magnification, the shape of the nanomaterial improved. However, the SR images at medium and high magnification failed to show the characteristics of the lattice of the nanomaterials. Second, to obtain a SR image, the deep learning model includes a residual network which reduces the loss of spatial information in the convolutional process of obtaining a feature map. In the process of optimizing the deep learning model, it was confirmed that the performance of the model improved as the number of data increased. In addition, by optimizing the deep learning model using the loss function, including MAE and SSIM at the same time, improved results of the nanomaterial lattice in SR images were achieved at medium and high magnifications. The final proposed deep learning model used four residual blocks to obtain the characteristic map of the low-resolution image, and the super-resolution image was completed using Upsampling2D and the residual block three times.

Keywords

Acknowledgement

This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science and ICT (NRF-2021R1F1A1052971).

References

  1. J.-W. Lee, S.-J. Kim and M.-S. Oh, Korean J. Met. Mater., 58, 169 (2020).
  2. S.-W. Shon, B.-A. Kim, H.-W. Kwon and J.-G. Lee, J. Korean Magn. Soc., 31, 249 (2021).
  3. Y. Lin, M. Zhou, X. Tai, H. Li, Xi. Han and J. Yu, Matter, 4, 2309 (2021).
  4. S. Chen and P. Gao, J. Appl. Phys., 128, 010901 (2020).
  5. T. Susi, J. C. Meyer and J. Kotakoski, Nat. Rev. Phys., 1, 397 (2019).
  6. K. de Haan, Z. S. Ballard, Y. Rivenson, Y. Wu and A. Ozcan, Sci. Rep., 9, 12050 (2019).
  7. L. Fan, Z. Wang, Y. Lu and J. Zhou, Nanomaterials, 11, 3305 (2021).
  8. J. Jung, J. Na, H. K. Park, J. M. Park, G. Kim, S. Lee and H. S. Kim, Npj Comput. Mater., 7, 96 (2021).
  9. J. Na, G. Kim, S.-H. Kang, S.-J. Kim and S. Lee, Acta Mater., 214, 116987 (2021).
  10. K. Hagita, T. Higuchi and H. Jinnai, Sci. Rep., 8, 5877 (2018).
  11. E. Giannatou, G. Papavieros, V. Constantoudis, H. Papageorgiou and E. Gogolides, Microelectron. Eng., 216, 111051 (2019).
  12. J. P. Horwath, D. N. Zakharov, R. Megret and E. A. Stach, Npj Comput. Mater., 6, 108 (2020).
  13. H. Zhao, O. Gallo, I. Frosio and J. Kautz, Loss Functions for Neural Networks for Image Processing, Retrieved April 20, 2018 from https://arxiv.org/pdf/1511.08861
  14. A. Tanchenko, J. Vis. Commun. Image Represent., 25, 874 (2014).
  15. Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, IEEE Trans. Image Process., 13, 600 (2004).
  16. S.-M. Ryu and C. Nam, J. Magn. Magn. Mater., 537, 168161 (2021).
  17. Geometric Image Transformation, Retrieved July 1, 2022 from https://docs.opencv.org/3.4/da/d54/group__imgproc__transform.html
  18. L. Alzubaidi, J. Zhang, A. J. Humaidi, A. Al-Dujaili, Y. Duan, O. Al-Shamma, J. Santamaria, M. A. Fadhel, M. Al-Amidie and L. Farhan, J. Big Data, 8, 53 (2021).
  19. H.-J. Lee, I.-K. Hwang, S.-J. Jeong, I.-S. Cho and H.-S. Kim, Korean J. Met. Mater., 59, 430 (2021).
  20. ImageJ is a public domain Java image processing program inspired by NIH Image, Retrieved July 1, 2022 from https://imagej.nih.gov/ij/index.html
  21. TensorFlow is an end-to-end open source platform for machine learning, Retrieved July 1, 2022 from https://www.tensorflow.org/