Journal of Cadastre & Land InformatiX (지적과 국토정보)

LX Spatial Information Research Institute (한국국토정보공사 공간정보연구원)
권호 표지
DOI QR코드

DOI QR Code

Quantitative Evaluation of Super-resolution Drone Images Generated Using Deep Learning

딥러닝을 이용하여 생성한 초해상화 드론 영상의 정량적 평가

  • Seo, Hong-Deok (Department of Spatial Information Engineering, Namseoul University) ;
  • So, Hyeong-Yoon (Disaster Information Research Division, National Disaster Management Research Institute) ;
  • Kim, Eui-Myoung (Department of Drone & GIS Engineering, Namseoul University)
  • 서홍덕 (남서울대학교 공간정보공학과) ;
  • 소형윤 (국립재난안전연구원 재난정보연구실) ;
  • 김의명 (남서울대학교 드론공간정보공학과)
  • Received : 2023.08.29
  • Accepted : 2023.11.29
  • Published : 2023.12.10
Download PDF
⟨ Previous Next ⟩

Abstract

As the development of drones and sensors accelerates, new services and values are created by fusing data acquired from various sensors mounted on drone. However, the construction of spatial information through data fusion is mainly constructed depending on the image, and the quality of data is determined according to the specification and performance of the hardware. In addition, it is difficult to utilize it in the actual field because expensive equipment is required to construct spatial information of high-quality. In this study, super-resolution was performed by applying deep learning to low-resolution images acquired through RGB and THM cameras mounted on a drone, and quantitative evaluation and feature point extraction were performed on the generated high-resolution images. As a result of the experiment, the high-resolution image generated by super-resolution was maintained the characteristics of the original image, and as the resolution was improved, more features could be extracted compared to the original image. Therefore, when generating a high-resolution image by applying a low-resolution image to an super-resolution deep learning model, it is judged to be a new method to construct spatial information of high-quality without being restricted by hardware.

드론과 센서의 발달이 가속화됨에 따라 드론에 장착된 다양한 센서로 취득된 데이터를 융합하여 새로운 서비스 및 가치를 창출하고 있다. 그러나 데이터 융합을 통한 공간정보 구축은 주로 영상에 의존하여 구축하며, 하드웨어의 사양 및 성능에 따라 데이터 품질이 결정된다. 또한, 고품질 공간정보를 구축하기 위해 고가 장비가 요구되므로 실제 현장에서 사용하기에는 어려운 실정이다. 본 연구에서는 드론에 장착된 RGB 및 THM 카메라를 통해 취득된 저해상도 영상을 딥러닝에 적용하여 초해상화를 수행하고, 이를 통해 생성된 고해상도 영상의 정량적 평가 및 특징점 추출에 대한 평가를 수행하였다. 실험 결과 초해상화를 수행하여 생성된 고해상도 영상은 원본 영상의 특징을 유지하고 있었으며, 해상도가 개선됨에 따라 원본 영상 대비 많은 특징을 추출할 수 있었다. 따라서, 저해상도 영상을 초해상화 딥러닝 모델에 적용하여 고해상도 영상을 생성할 경우 하드웨어에 제약을 받지 않고 고품질의 공간정보를 구축하기 위한 새로운 방법일 것으로 판단하였다.

Keywords

References

  1. Kim J, Jang HY, Shin YH, Kim KS. 2021. New changes in Smart City, Digital Twin. Urban Information Service. 468:5-15.
  2. Nho HY, Shin DY, Sohn HG, Kim SS. 2020. Fast Geocoding of UAV Images for Disaster Site Monitoring. Korean Journal of Remote Sensing. 36(5):1221-1229.
  3. Park SW, Kim YH, Kim MS. 2022. Impact Analysis of Deep Learning Super-resolution Technology for Improving the Accuracy of Ship Detection Based on Optical Satellite Imagery. Korean Journal of Remote Sensing. 38(5):559-570.
  4. Bang CH. A Study on the Management Plan of Fire Protection System Utilizing Urban Spatial Information. Korean Institute of Fire Science & Engineering. 33(3):98-104.
  5. Seo HD, Kim EM. 2023. Conversion of Low-resolution Image to High-resolution Image Using Deep Learning. The Journal of Korean Society for Geospatial Information Science Conference. 113-114.
  6. Ahn SH, Kang SJ. 2021. Deep Learning-based Real-Time Super-Resolution Architecture Design. JOURNAL OF BROADCAST ENGINEERING. 26(2):167-174. https://doi.org/10.5909/JBE.2021.26.2.167
  7. Ministry of Foreign Affairs Republic of Korea. 2021. The current and future development direction of drone[Interent]. [https://overseas.mofa.go.kr/]. Last accessed 25 August 2023.
  8. Yoon SY, Lee YW. 2022. Cloud based Distributed Parallel Processing for Deep Learning based Real-time Person Detection in an AI Middleware for Smart City. Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography. 20(5):9-19.
  9. Lee JG, Suh YC. 2022. Indoor and Outdoor Building Modeling Based on Point Cloud Data Convergence using Drones and Terrestrial LiDAR. Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography. 40(6):613-620. https://doi.org/10.7848/ksgpc.2022.40.6.613
  10. Lee CH, Kim EM. 2022. Performance Comparison and Analysis between Keypoints Extraction Algorithms using Drone Images. Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography. 40(2):79-89.
  11. Lim JS, You SB, Kim YS. 2020. Monitoring System based on Bigdata Platform for Safety Management of Road Facilities. Journal of Korean Institute of Information Technology. 18(11):139-151.
  12. Chang SH, Yun HC. 2021. Evaluation of Geospatial Information Construction Characteristics and Usability According to Type and Sensor of Unmanned Aerial Vehicle. Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography. 39(6):555-562.
  13. Hong JS, Shin JS, Shin DM. 2021. Development of Multi-Camera based Mobile Mapping System for HD Map Production. Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography. 39(6):587-598.
  14. Alcantarilla, P.F., Bartoli, A., & Davison, A. J., 2012. KAZE features. In Computer Vision-ECCV 2012: 12th European Conference on Computer Vision, 7-13 October 2012, Florence, Italy, 214-227.
  15. Alcantarilla, P.F., Nuevo, J., Bartoli, A.. 2013. Fast explicit diffusion for accelerated features in nonlinear scale spaces. British Machine Vision Conference 2013. 1-13.
  16. Batchuluun, G., Lee, Y. W., Nguyen, D. T., Pham, T. D., Park, K. R. 2020. Thermal image reconstruction using deep learning. RIEEE Access. 8:126839-126858. https://doi.org/10.1109/ACCESS.2020.3007896
  17. Banerjee, N., Karakonstantis, G., Roy, K. 2007. Process variation tolerant low power DCT architecture. In 2007 Design, Automation & Test in Europe Conference & Exhibition, 16-20 April 2007, Nice, France. 1-6.
  18. Dong, C., Loy, C. C., He, K., Tang, X. 2015. Image super-resolution using deep convolutional networks. IEEE transactions on pattern analysis and machine intelligence, 38(2):295-307. https://doi.org/10.1109/TPAMI.2015.2439281
  19. Dong, C., Loy, C. C., Tang, X. 2016. Accelerating the super-resolution convolutional neural network. In Computer Vision-ECCV 2016: 14th European Conference, 11-14 October 2016, Amsterdam, Netherlands. 391-407.
  20. Kruse, F. A., Lefkoff, A. B., Boardman, J. W., Heidebrecht, K. B., Shapiro, A. T., Barloon, P. J., Goetz, A. F. H. 1993. The spectral image processing system (SIPS)-interactive visualization and analysis of imaging spectrometer data, Remote sensing of environment.. 44(2-3):145-163. https://doi.org/10.1063/1.44433
  21. Sacks, R., Girolami, M., Brilakis, I. 2020. Building information modelling, artificial intelligence and construction tech. Developments in the Built Environment, 4, 100011.
  22. Setiadi, D. 2021. PSNR vs SSIM: imperceptibility quality assessment for image steganography, Multimedia Tools and Applications. 80(6):8423-8444. https://doi.org/10.1007/s11042-020-10035-z
  23. Wang, P., Bayram, B., Sertel, E.. 2022. A comprehensive review on deep learning based remote sensing image super-resolution methods. Earth-Science Reviews. 104110.
  24. Wang, Z., Bovik, A. C.. 2002. A universal image quality index. EIEEE signal processing letters. 9(3):81-84. https://doi.org/10.1109/97.995823
  25. Xie, X., Li, L., An, Z., Lu, G., Zhou, Z. 2022. Small Ship Detection Based on Hybrid Anchor Structure and Feature Super-Resolution. Remote Sensing. 14(15):3530.
  26. Xue, W., Zhang, L., Mou, X., Bovik, A. C. 2013. Gradient magnitude similarity deviation: A highly efficient perceptual image quality index, IEEE transactions on image processing. 23(2):684-695.
  27. Zhang, L., Zhang, L., Mou, X., Zhang, D. 2011. FSIM: A feature similarity index for image quality assessment, IEEE transactions on Image Processing. 20(8):2378-2386. https://doi.org/10.1109/TIP.2011.2109730