Journal of Internet of Things and Convergence (사물인터넷융복합논문지)

The Korea Internet of Things Society (한국사물인터넷학회)
권호 표지
DOI QR코드

DOI QR Code

Designing a Reinforcement Learning-Based 3D Object Reconstruction Data Acquisition Simulation

강화학습 기반 3D 객체복원 데이터 획득 시뮬레이션 설계

  • Received : 2023.10.18
  • Accepted : 2023.11.23
  • Published : 2023.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

The technology of 3D reconstruction, primarily relying on point cloud data, is essential for digitizing objects or spaces. This paper aims to utilize reinforcement learning to achieve the acquisition of point clouds in a given environment. To accomplish this, a simulation environment is constructed using Unity, and reinforcement learning is implemented using the Unity package known as ML-Agents. The process of point cloud acquisition involves initially setting a goal and calculating a traversable path around the goal. The traversal path is segmented at regular intervals, with rewards assigned at each step. To prevent the agent from deviating from the path, rewards are increased. Additionally, rewards are granted each time the agent fixates on the goal during traversal, facilitating the learning of optimal points for point cloud acquisition at each traversal step. Experimental results demonstrate that despite the variability in traversal paths, the approach enables the acquisition of relatively accurate point clouds.

물체나 공간을 디지털화하는 기술인 3D 복원은 주로 포인트 클라우드 데이터를 활용한다. 본 논문은 강화학습을 활용하여 주어진 환경에서 포인트 클라우드의 획득을 목표로 한다. 이를 위해 시뮬레이션 환경은 유니티를 이용하여 구성하고, 강화학습은 유니티 패키지인 ML-Agents를 활용한다. 포인트 클라우드 획득 과정은 먼저 목표를 설정하고, 목표 주변을 순회할 수 있는 경로를 계산한다. 순회 경로는 일정 비율로 분할하여 각 스텝마다 보상한다. 이때 에이전트의 경로 이탈을 방지하기 위해 보상을 증가시킨다. 에이전트가 순회하는 동안 목표를 응시할 때마다 보상을 부여하여 각 순회 스텝에서 포인트 클라우드의 획득 시점을 학습하도록 한다. 실험결과, 순회 경로가 가변적이지만 상대적으로 정확한 포인트 클라우드를 획득할 수 있었다.

Keywords

Acknowledgement

이 논문은 정부(과학기술정보통신부)의 재원으로 한국연구재단의 지원을 받아 수행된 연구임(No. 2022R1G1A1012974).

References

  1. To, Alex, et al. "Drone-based AI and 3D reconstruction for digital twin augmentation." International Conference on Human-Computer Interaction. Cham: Springer International Publishing, pp.511-529, 2021.
  2. Navarro, Francisco, et al. "Integrating 3D reconstruction and virtual reality: A new approach for immersive teleoperation." ROBOT 2017: Third Iberian Robotics Conference: Springer International Publishing, Vol.2, pp.606-616, 2018.
  3. Ham, Hanry, Julian Wesley, and Hendra Hendra. "Computer vision based 3D reconstruction: A review." International Journal of Electrical and Computer Engineering Vol.9, No.4, pp.2394-2402, 2019.
  4. Han, Xian-Feng, Hamid Laga, and Mohammed Bennamoun. "Image-based 3D object reconstruction: State-of-the-art and trends in the deep learning era." IEEE transactions on pattern analysis and machine intelligence Vol.43, No.5, pp.1578-1604, 2019.
  5. Guo, Yulan, et al. "Deep learning for 3d point clouds: A survey." IEEE transactions on pattern analysis and machine intelligence Vol.43, No.12, pp.4338-4364, 2020.
  6. Samavati, Taha, and Mohsen Soryani. "Deep learning-based 3D reconstruction: A survey." Artificial Intelligence Review pp.1-45, 2023.
  7. Gu, Shixiang, et al. "Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates." 2017 IEEE international conference on robotics and automation (ICRA). IEEE, pp.3389-3396, 2017.
  8. Lillicrap, Timothy P., et al. "Continuous control with deep reinforcement learning." arXiv preprint arXiv:1509.02971, 2015.
  9. Mirowski, Piotr, et al. "Learning to navigate in complex environments." arXiv preprint arXiv:1611.03673, 2016.
  10. Tatarchenko, Maxim, et al. "What do single-view 3d reconstruction networks learn?." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp.3405-3414, 2019.
  11. Riou, Kevin, Kevin Subrin, and Patrick Le Callet. "Reinforcement Learning Based Point-Cloud Acquisition and Recognition Using Exploration-Classification Reward Combination." 2022 IEEE International Conference on Multimedia and Expo (ICME). IEEE, pp.1-6, 2022.
  12. Ali, Waleed, et al. "Yolo3d: End-to-end real-time 3d oriented object bounding box detection from lidar point cloud." Proceedings of the European conference on computer vision (ECCV) workshops. pp.716-728, 2018.
  13. Placed, Julio A., and Jose A. Castellanos. "A deep reinforcement learning approach for active SLAM." Applied Sciences Vol.10, No.23, p.8386, 2020.
  14. Swazinna, Phillip, et al. "Comparing model-free and model-based algorithms for offline reinforcement learning." IFAC-PapersOnLine Vol.55, No.15, pp.19-26, 2022. https://doi.org/10.1016/j.ifacol.2022.07.602
  15. The Operation of ML-Agents Graphics [Internet], https://unity-technologies.github.io/ml-agents/ML-Agents-Overview/.
  16. Bianco, Simone, Gianluigi Ciocca, and Davide Marelli. "Evaluating the performance of structure from motion pipelines." Journal of Imaging Vol.4, No.8, p.98, 2018.
  17. Jin, Young-Hoon, Kwang-Woo Ko, and Won-Hyung Lee. "An indoor location-based positioning system using stereo vision with the drone camera." Mobile Information Systems, Vol.2018, 2018.