The Journal of Korea Robotics Society (로봇학회논문지)

Korea Robotics Society (한국로봇학회)
권호 표지
DOI QR코드

DOI QR Code

CNN-based Opti-Acoustic Transformation for Underwater Feature Matching

수중에서의 특징점 매칭을 위한 CNN기반 Opti-Acoustic변환

  • Jang, Hyesu (Dept. of Civil and Environmental Engineering, KAIST) ;
  • Lee, Yeongjun (Korea Research Institute Ship and Ocean engineering (KRISO)) ;
  • Kim, Giseop (Dept. of Civil and Environmental Engineering, KAIST) ;
  • Kim, Ayoung (Dept. of Civil and Environmental Engineering, KAIST)
  • Received : 2019.12.10
  • Accepted : 2020.01.07
  • Published : 2020.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we introduce the methodology that utilizes deep learning-based front-end to enhance underwater feature matching. Both optical camera and sonar are widely applicable sensors in underwater research, however, each sensor has its own weaknesses, such as light condition and turbidity for the optic camera, and noise for sonar. To overcome the problems, we proposed the opti-acoustic transformation method. Since feature detection in sonar image is challenging, we converted the sonar image to an optic style image. Maintaining the main contents in the sonar image, CNN-based style transfer method changed the style of the image that facilitates feature detection. Finally, we verified our result using cosine similarity comparison and feature matching against the original optic image.

Keywords

References

  1. Y. Cho, Y.-S. Shin, and A. Kim, "Online depth estimation and application to underwater image dehazing," OCEANS 2016 MTS/IEEE Monterey, Monterey, CA, USA, pp. 1-7, 2016.
  2. Y. Cho and A. Kim, "Visibility enhancement for underwater visual slam based on underwater light scattering model," 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore, pp. 710-717, 2017.
  3. J. Li, K. A. Skinner, R. M. Eustice, and M. Johnson- Roberson, "Watergan: unsupervised generative network to enable realtime color correction of monocular underwater images," IEEE Robotics and Automation Letters, vol. 3, no. 1, pp. 387-394, Jan., 2018. https://doi.org/10.1109/LRA.2017.2730363
  4. N. Hurtós, X. Cuf', Y. Petillot, and J. Salvi, "Fourier-based registrations for two-dimensional forward-looking sonar image mosaicing," 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, Vilamoura, Portugal, pp. 5298-5305, 2012.
  5. J. Li, M. Kaess, R. M. Eustice, and M. Johnson- Roberson, "Pose-graph slam using forward-looking sonar," IEEE Robotics and Automation Letters, vol. 3, no. 3, pp. 2330-2337, Jul., 2018. https://doi.org/10.1109/LRA.2018.2809510
  6. S. Negahdaripour, H. Sekkati, and H. Pirsiavash, "Optiacoustic stereo imaging: On system calibration and 3-d target reconstruction," IEEE Transactions on Image Processing, vol. 18, no. 6, pp. 1203-1214, Jun., 2009. https://doi.org/10.1109/TIP.2009.2013081
  7. D.-H. Gwon, Y.-S. Shin, Y. Kim, A. Kim, Y. Lee, and H.-T. Choi, "Nontemporal relative pose estimation for opti-acoustic bundle adjustment," OCEANS 2016 MTS/IEEE Monterey, Monterey, CA, USA, pp. 1-5, 2016.
  8. L. A. Gatys, A. S. Ecker, and M. Bethge, "Texture synthesis using convolutional neural networks," Computer Vision and Pattern Recognition, pp. 262-270, 2015.
  9. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv:1409.1556, 2014.
  10. L. Gatys, A. Ecker, and M. Bethge, "A neural algorithm of artistic style," Journal of Vision, 2015.
  11. P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, "Imageto- image translation with conditional adversarial networks," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, pp. 1125-1134, 2017.
  12. J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, "Unpaired image-to-image translation using cycle-consistent adversarial networks," 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, pp. 2223-2232, 2017.
  13. L. A. Gatys, A. S. Ecker, and M. Bethge, "Image style transfer using convolutional neural networks," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016.
  14. Y.-S. Shin, Y. Cho, Y. Lee, H.-T. Choi, and A. Kim, "Comparative Study of Sonar Image Processing for Underwater Navigation," Journal of Ocean Engineering and Technology, vol. 30, no. 3, pp. 214-220, 2016. https://doi.org/10.5574/KSOE.2016.30.3.214
  15. M. H. Jeon, Y. Lee, Y.-S. Shin, H. Jang, T. Yeu, and A. Kim "Synthesizing Image and Automated Annotation Tool for CNN based Under Water Object Detection," Journal of Korea Robotics Society, vol. 14, no. 2, pp. 139-149, Jun., 2019. https://doi.org/10.7746/jkros.2019.14.2.139
  16. Y.-S. Shin, Y.-J. Lee, H-T. Choi, and A. Kim, "Bundle Adjustment and 3D Reconstruction Method for Underwater Sonar Image," Journal of Korea Robotics Society, vol. 11, no. 2, pp. 051-059, Jun., 2016. https://doi.org/10.7746/jkros.2016.11.2.051

Cited by

  1. Matching Underwater Sonar Images by the Learned Descriptor Based on Style Transfer Method vol.2029, pp.1, 2020, https://doi.org/10.1088/1742-6596/2029/1/012118