The Journal of Korea Robotics Society (로봇학회논문지)

Korea Robotics Society (한국로봇학회)
권호 표지
DOI QR코드

DOI QR Code

Safety and Efficiency Learning for Multi-Robot Manufacturing Logistics Tasks

다중 로봇 제조 물류 작업을 위한 안전성과 효율성 학습

  • Received : 2023.02.07
  • Accepted : 2023.03.23
  • Published : 2023.05.31
Download PDF
⟨ Previous Next ⟩

Abstract

With the recent increase of multiple robots cooperating in smart manufacturing logistics environments, it has become very important how to predict the safety and efficiency of the individual tasks and dynamically assign them to the best one of available robots. In this paper, we propose a novel task policy learner based on deep relational reinforcement learning for predicting the safety and efficiency of tasks in a multi-robot manufacturing logistics environment. To reduce learning complexity, the proposed system divides the entire safety/efficiency prediction process into two distinct steps: the policy parameter estimation and the rule-based policy inference. It also makes full use of domain-specific knowledge for policy rule learning. Through experiments conducted with virtual dynamic manufacturing logistics environments using NVIDIA's Isaac simulator, we show the effectiveness and superiority of the proposed system.

Keywords

Acknowledgement

This work was partly supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No.2020-0-00096, Robot task planning for single and multiple robots connected to a cloud system)

References

  1. E. Tunstel, A. Howard, and H. Seraji, "Rule-based reasoning and neural network perception for safe off-road robot mobility," Expert Systems, vol. 19, no. 4, pp. 191-200, 2002, DOI: 10.1111/1468-0394.00204.
  2. A. Kreutzmann, D. Wolter, F. Dylla, and J. H. Lee, "Towards Safe Navigation by Formalizing Navigation Rules," TransNav: International Journal on Marine Navigation and Safety of Sea Transportation, vol. 7, no. 2, pp. 161-168, Jun, 2013, DOI: 10.12716/1001.07.02.01.
  3. Z. Jiang and S. Luo, "Neural Logic Reinforcement Learning," Machine Learning, 2019, DOI: 10.48550/arXiv.1904.10729.
  4. A. Payani and F. Fekri, "Incorporating Relational Background Knowledge into Reinforcement Learning via Differentiable Inductive Logic Programming," Machine Learning, 2020, DOI: 10.48550/arXiv.2003.10386.
  5. M. Rojas, G. Hermosilla, D. Yunge, and G. Faris, "An Easy to Use Deep Reinforcement Learning Library for AI Mobile Robots in Isaac Sim," Applied Sciences, vol. 12, no. 17, 2022, DOI: 10.3390/app12178429.
  6. V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riediller, "Playing atari with deep reinforcement learning," Machine Learning, 2013, DOI: 10.48550/arXiv.1312.5602.
  7. J. Schulman, F. Wolski, P. Dhariwal, A. Radford and O. Klimov, "Proximal Policy Optimization Algorithms," Machine Learning, 2017, DOI: 10.48550/arXiv.1707.06347.
  8. R. Evans and E. Grefenstette, "Learning Explanatory Rules from Noisy Data," Journal of Artificial Intelligence Research, vol. 61, Jan, 2018, DOI: 10.1613/jair.5714.
  9. A. Payani and F. Fekri, "Inductive Logic Programming via Differentiable Deep Neural Logic Networks," Artificial Intelligence, 2019, DOI: 10.48550/arXiv.1906.03523.
  10. O. Rivlin, T. Hazan, and E. Karpas, "Generalized Planning With Deep Reinforcement Learning," Artificial Intelligence, 2020, DOI: 10.48550/arXiv.2005.02305.
  11. V. Zambaldi, D. Raposo, A. Santoro, V. Bapst, Y. Li, I. Babuschkin, K. Tuyls, D. Reichert, T. Lillicrap, E. Lockhart, M. Shanahan, V. Langston, R. Pascanu, M. Botvinick, O. Vinyals, and P. Battaglia, "Relational Deep Reinforcement Learning," Machine Learning, 2018, DOI: 10.48550/arXiv.1806.01830.
  12. J. Janisch, T. Pevny and V. Lisy, "Symbolic Relational Deep Reinforcement Learning based on Graph Neural Networks," Machine Learning, 2021, DOI: 10.48550/arXiv.2009.12462.
  13. S. Garg and A. Bajpai, "Symbolic Network: Generalized Neural Policies for Relational MDPs," the 37th International conference on machine learning, 2020, [Online] https://proceedings.mlr.press/v119/garg20a.html.
  14. D. Adjodah, T. Klinger, and J. Joseph, "Symbolic Relation Networks for Reinforcement Learning," 32nd Conference on Neural Information Processing Systems (NIPS 2018), Montreal, Canada, 2018, [Online] https://r2learning.github.io/assets/papers/CameraReadySubmission%203.pdf.
  15. S. Das, S. Natarajan, K. Roy, R. Parr, and K. Kersting, "Fitted Q-Learning for Relational Domains," Machine Learning, 2020, DOI: 10.48550/arXiv.2006.05595.
  16. T. Gokhale, S. Sampat, Z. Fang, Y. Yang, and C. Baral, "Blocksworld Revisited: Learning and Reasoning to Generate Event-Sequences from Image Pairs," Computer Vision and Pattern Recognition, 2019, DOI : 10.48550/arXiv.1905.12042.