Journal of the Society of Naval Architects of Korea (대한조선학회논문집)

The Society of Naval Architects of Korea (대한조선학회)
권호 표지
DOI QR코드

DOI QR Code

Design of a Mapping Framework on Image Correction and Point Cloud Data for Spatial Reconstruction of Digital Twin with an Autonomous Surface Vehicle

무인수상선의 디지털 트윈 공간 재구성을 위한 이미지 보정 및 점군데이터 간의 매핑 프레임워크 설계

  • Suhyeon Heo (Advanced-Intelligent Ship Research Division, Korea Research Institute of Ships and Ocean Engineering (KRISO)) ;
  • Minju Kang (Advanced-Intelligent Ship Research Division, Korea Research Institute of Ships and Ocean Engineering (KRISO)) ;
  • Jinwoo Choi (Advanced-Intelligent Ship Research Division, Korea Research Institute of Ships and Ocean Engineering (KRISO)) ;
  • Jeonghong Park (Advanced-Intelligent Ship Research Division, Korea Research Institute of Ships and Ocean Engineering (KRISO))
  • 허수현 (한국해양과학기술원 부설 선박해양플랜트연구소 지능형선박연구본부) ;
  • 강민주 (한국해양과학기술원 부설 선박해양플랜트연구소 지능형선박연구본부) ;
  • 최진우 (한국해양과학기술원 부설 선박해양플랜트연구소 지능형선박연구본부) ;
  • 박정홍 (한국해양과학기술원 부설 선박해양플랜트연구소 지능형선박연구본부)
  • Received : 2024.03.26
  • Accepted : 2024.04.18
  • Published : 2024.06.20
Download PDF
⟨ Previous Next ⟩

Abstract

In this study, we present a mapping framework for 3D spatial reconstruction of digital twin model using navigation and perception sensors mounted on an Autonomous Surface Vehicle (ASV). For improving the level of realism of digital twin models, 3D spatial information should be reconstructed as a digitalized spatial model and integrated with the components and system models of the ASV. In particular, for the 3D spatial reconstruction, color and 3D point cloud data which acquired from a camera and a LiDAR sensors corresponding to the navigation information at the specific time are required to map without minimizing the noise. To ensure clear and accurate reconstruction of the acquired data in the proposed mapping framework, a image preprocessing was designed to enhance the brightness of low-light images, and a preprocessing for 3D point cloud data was included to filter out unnecessary data. Subsequently, a point matching process between consecutive 3D point cloud data was conducted using the Generalized Iterative Closest Point (G-ICP) approach, and the color information was mapped with the matched 3D point cloud data. The feasibility of the proposed mapping framework was validated through a field data set acquired from field experiments in a inland water environment, and its results were described.

Keywords

Acknowledgement

본 논문은 선박해양플랜트연구소의 주요 사업인 '디지털트윈쉽모델링/시뮬레이션 및 상태 인식/추론 핵심기술 개발(PES5210)'의 지원으로 수행되었습니다.

References

  1. Campos, D.F., Matos, A., and Pinto, A.M., 2020. Multi-domain mapping for offshore asset inspection using an autonomous surface vehicle. In 2020 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC) (pp. 221-226). IEEE.
  2. Dhall, A., Chelani, K., Radhakrishnan, V. and Krishna, K. M., 2017. LiDAR-camera calibration using 3D-3D point correspondences. arXiv preprint arXiv:1705.09785.
  3. Feixiang, Z.H.U. and Jiandao, L.I.U., 2019. Research on Marine Buoy Surface Reconstruction Algorithm Based on Point Cloud Data. In 2019 6th International Conference on Information Science and Control Engineering (ICISCE) (pp. 480-484). IEEE.
  4. Heo, S.H., Kang, M.J., Choi, J.W., Yun, S.W. and Park, J.H, 2023. A Preliminary Study for Development of Digital Twin Ship Technology Using Autonomous Surface Vehicle. In 2023 20th International Conference on Ubiquitous Robots (UR) (pp. 255-260). IEEE.
  5. Joung, J.H., An, K.H., Kang, J.W., Chung, M.J. and Yu, W.P., 2009. 3D environment reconstruction using modified color ICP algorithm by fusion of a camera and a 3D laser range finder. IEEE/RSJ International Conference on Intelligent Robots and Systems, pp.3082-3088.
  6. Jung, J.D., Park, J.H., Lee, Y.J., Yeu, T.K., Choi, J.W. and Han, J.B., 2023. Consistent mapping of marine structures with an autonomous surface vehicle using motion compensation and submap-based filtering. Ocean Engineering, 290, 116418.
  7. Kaur, H. and Sohi, N., 2017. A study for applications of histogram in image enhancement. International Journal of Engineering and Science (IJES) 6(6), 59-63
  8. Kim, H.G., Song, S.W. and Myung, H., 2019. GP-ICP: Ground plane ICP for mobile robots. IEEE Access, 7, 76599-76610.
  9. Kusuma, B.N. and Santoso, D.B., 2022. Improvement of the automatic gamma correction method in cloud image detection. Matrix: Journal Manajemen Teknologi dan Informatika, 12(3), 137-146.
  10. Leedekerken, J.C., Fallon, M.F. and Leonard, J.J., 2010. Mapping complex marine environments with autonomous surface craft. In Experimental Robotics: The 12th International Symposium on Experimental Robotics, pp.525-539.
  11. Madusanka, N.S., Fan, Y., Yang, S. and Xiang, X., 2023. Digital twin in the maritime domain: A review and emerging trends. Journal of Marine Science and Engineering, 11(5), 1021.
  12. Park, H.S., Son, N.S. and Pyo. C.S., 2023. A Study on Playback of Ship Collision Accident Using Free Running Model Test. Journal of the Society of Naval Architects of Korea, 60(6),450-459.
  13. Sattler, F., Barnes, S., Carrillo Perez, B.J., Stebner, K., Stephan, M. and Lux, G., 2022. Real-time embedded reconstruction of dynamic objects for a 3D maritime situational awareness picture. In European Workshop on Maritime Systems Resilience and Security.
  14. Segal, A., Haehnel, D. and Thrun, S., 2009. Generalized-icp. In Robotics: science and systems, 2(4), pp.435
  15. Servos, J. and Waslander, S.L., 2017. Multi-Channel Generalized-ICP: A robust framework for multi-channel scan registration. Robotics and Autonomous systems, 87, 247-257.
  16. Wang, P., 2021. Research on comparison of LiDAR and camera in autonomous driving. Journal of Physics: Conference Series, 2093(1), pp.012032.
  17. Zhang, T., Liu, C., Li, J., Pang, M. and Wang, M., 2022. A new visual inertial simultaneous localization and mapping (SLAM) algorithm based on point and line features. Drones, 6(1), 23.
  18. Zhao, X., Sun, P., Xu, Z., Min, H. and Yu, H., 2020. Fusion of 3D LIDAR and camera data for object detection in autonomous vehicle applications. IEEE Sensors Journal, 20(9), pp.4901-4913.