A HARMS-based heterogeneous human-robot team for gathering and collecting

  • Kim, Miae (M2M Lab and Computer and Information Technology, Purdue University) ;
  • Koh, Inseok (Department of Computer Engineering, Pohang University of Science and Technology) ;
  • Jeon, Hyewon (M2M Lab and Computer and Information Technology, Purdue University) ;
  • Choi, Jiyeong (Department of Computer Science, Kyung Hee University) ;
  • Min, Byung Cheol (SMART Lab and Computer and Information Technology, Purdue University) ;
  • Matson, Eric T. (M2M Lab and Computer and Information Technology, Purdue University) ;
  • Gallagher, John (Department of Computer Science and Engineering, Wright State University)
  • Received : 2018.10.03
  • Accepted : 2018.10.26
  • Published : 2018.09.25


Agriculture production is a critical human intensive task, which takes place in all regions of the world. The process to grow and harvest crops is labor intensive in many countries due to the lack of automation and advanced technology. Much of the difficult, dangerous and dirty labor of crop production can be automated with intelligent and robotic platforms. We propose an intelligent, agent-oriented robotic team, which can enable the process of harvesting, gathering and collecting crops and fruits, of many types, from agricultural fields. This paper describes a novel robotic organization enabling humans, robots and agents to work together for automation of gathering and collection functions. The focus of the research is a model, called HARMS, which can enable Humans, software Agents, Robots, Machines and Sensors to work together indistinguishably. With this model, any capability-based human-like organization can be conceived and modeled, such as in manufacturing or agriculture. In this research, we model, design and implement a technology application of knowledge-based robot-to-robot and human-to-robot collaboration for an agricultural gathering and collection function. The gathering and collection functions were chosen as they are some of the most labor intensive and least automated processes in the process acquisition of agricultural products. The use of robotic organizations can reduce human labor and increase efficiency allowing people to focus on higher level tasks and minimizing the backbreaking tasks of agricultural production in the future. In this work, the HARMS model was applied to three different robotic instances and an integrated test was completed with satisfactory results that show the basic promise of this research.



Supported by : Institute for Information and Communications Technology Promotion (IITP)


  1. Altaf, M.M., Elbagoury, B.M., Alraddady, F. and Roushdy, M. (2016), "Extended case-based behavior control for multi-humanoid robots", Int. J. Human. Robotics, 13(2), 1550035.
  2. Bauer, A., Wollherr, D. and Buss, M. (2008), "Human-robot collaboration: A survey", Int. J. Human. Robotics, 5(1), 47-66.
  3. DeLoach, S., Oyenan, W. and Matson, E.T. (2008), "A capabilities-based model for adaptive organizations", Autonom. Agents Multi-Agent Syst., 16(1), 13-56.
  4. Erickson, D., DeWees, M., Lewis, J. and Matson, E.T. (2012), "Communication for task completion with heterogeneous robots", Proceedings of the 1st International Conference on Robot Intelligence Technology and Applications, Gwangju, South Korea, December.
  5. Esmaeili, A., Mozayani, N., Motlagh, M.R.J. and Matson, E.T. (2017), "A socially-based distributed selforganizing algorithm for holonic multi-agent systems: Case study in a task environment", Cogn. Syst. Res., 43, 21-44.
  6. Grizard, A.,Vercouter, L., Stratulat, T. and Muller, G. (2007), A Peer-to-Peer Normative System to Achieve Social Order, Proceedings of the COIN II, Coordination, Organizations, Institutions and Norms in Multiagent Systems, AAMAS '06/ECAI '06, in Lecture Notes in Artificial Intelligence(LNAI), 4386, Springer-Verlag, Berlin, Germany.
  7. Hayashia, S., Shigematsu, K., Yamamoto, S., Kobayashi, K., Kohno, Y., Kamata, J. and Kurita, M. (2010), "Evaluation of a strawberry-harvesting robot in a field test", Biosyst. Eng., 105(2), 160-171.
  8. Hidayat, S.S., Kim, B.K. and Ohba, K. (2008), "Learning affordance for semantic robots using ontology approach", Proceedings of the 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, Nice, France, October.
  9. Hsu, H.C., and Liu, A. (2007), "Applying a taxonomy of formation control in developing a robotic system", Int. J. Artif. Intell. Tools, 16(4), 565-582.
  10. IEEE RAS (2014), Agricultural Robotics, .
  11. iRobot, Inc. (2014), iRobot Create, .
  12. Kim, J.H., Kim, Y.D. and Lee, K.H. (2004), "The third generation of robotics: Ubiquitous robot", Proceedings of the 2nd International Conference on Autonomous Robots and Agents, Palmerston North, New Zealand, December.
  13. Kim, M., Matson, E.T., Min, B.C., Koh, I., Jeon, H., Choi, J., Gallagher, J. and Cho, Y. (2015), "A HARMS-based heterogeneous human-robot team for a gathering and collection function", Proceedings of the 16th International Symposium on Advanced Intelligent Systems (ISIS2015), Mokpo, South Korea, November.
  14. Lewis, J., Matson, E.T. and Wei, S. (2012), "Using indistinguishability in ubiquitous robot organizations", Proceedings of the ACM Conference on Ubiquitous Computing, Pittsburgh, Pennsylvania, U.S.A., September.
  15. Lim, H., Kang, Y., Lee, J., Kim, J. and You, B. (2009), "Software architecture and task definition of a multiple humanoid cooperative control system", Int. J. Human. Robotics, 6(2), 173-203.
  16. Lorini, E. and Verdicchio, M. (2010), Coordination, Organizations, Institutions and Norms in Agent Systems V, Proceedings of the AAMAS '09, IJCAI '09, MALLOW '09, Springer-Verlag Lecture Notes in AI (LNAI), 6069, Springer, Heidelberg, Berlin, Germany.
  17. Matson, E.T. and Min, B.C. (2011), "M2M infrastructure to integrate humans, agents and robots into collectives", Proceedings of the IEEE International Instrumentation and Measurement Technology Conference (I2MTC 2011), Hangzhou, China, May.
  18. Matson, E.T., Taylor, J., Raskin, V., Min, B.C. and Wilson, E.C. (2011), "A natural language exchange model for enabling human, agent, robot and machine interaction", Proceedings of the ICARA 2011 : The 5th IEEE International Conference on Automation, Robotics and Applications, Wellington, New Zealand, December.
  19. Rani, P., Liu, C. and Sarkar, N. (2008), "Interaction between human and robot-an affect-inspired approach", Interaction Stud., 9(1), 230-257.
  20. Raskin, V., Taylor, J. and Matson, E.T. (2012), Towards an ontological modeling of something very much like consciousness: The HARMS way", Proceedings of the SDPS 2012 Conference, Berlin, Germany, June.
  21. Robotis, Inc. (2015), DarwIn humanoid robot, .
  22. Ruckelshausen, A., Biber, P., Dorna, M., Gremmes, H., Klose, R., Linz, A., Rahe, F., Resch, R., Thiel, M., Trautz, D. and Weiss, U. (2009), "BoniRob: An autonomous field robot platform for individual plant phenotyping", Proceedings of the Joint International Agricultural Conference, Wageningen, The Netherlands, July.
  23. Savarimuthu, B., Cranefield, S., Purvis, M. and Purvis, M. (2008), Coordination, Organizations, Institutions, and Norms in Agent Systems III, in Lecture Notes in Artificial Intelligence(LNAI), 4870, Springer, Heidelberg , Berlin.
  24. Tanigaki, K., Fujiura, T., Akase, A. and Imagawa, J. (2008), "Cherry-harvesting robot", Comput. Electron. Agr., 63(1), 65-72.
  25. Vujic, J. and Marincic, A. and Ercegovac, M. and Milovanovic, B. (2001), "Telecommunications in modern satellite, cable and broadcasting service (TELSIKS 2001)", Proceedings of the 5th International Conference on Nikola Tesla: 145 Years of Visionary Ideas, Nis, Yugoslavia, September.
  26. Weiss, G. (1999), Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence, MIT Press, Cambridge, Massachusetts, U.S.A.
  27. Yachir, A., Tari, K., Amirat, Y., Chibani, A. and Badache, N. (2009), "QoS based framework for ubiquitous robotic services composition", Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2009), St. Louis, Missouri, U.S.A., October.