Advances in robotics research

Techno-Press (테크노프레스)
권호 표지
DOI QR코드

DOI QR Code

Projection mapping onto multiple objects using a projector robot

  • Yamazoe, Hirotake (College of Information Science and Engineering, Ritsumeikan University) ;
  • Kasetani, Misaki (Graduate School of Information Science and Engineering, Ritsumeikan University) ;
  • Noguchi, Tomonobu (Graduate School of Information Science and Engineering, Ritsumeikan University) ;
  • Lee, Joo-Ho (College of Information Science and Engineering, Ritsumeikan University)
  • Received : 2018.01.01
  • Accepted : 2018.02.23
  • Published : 2018.03.25
⟨ Previous Next ⟩

Abstract

Even though the popularity of projection mapping continues to increase and it is being implemented in more and more settings, most current projection mapping systems are limited to special purposes, such as outdoor events, live theater and musical performances. This lack of versatility arises from the large number of projectors needed and their proper calibration. Furthermore, we cannot change the positions and poses of projectors, or their projection targets, after the projectors have been calibrated. To overcome these problems, we propose a projection mapping method using a projector robot that can perform projection mapping in more general or ubiquitous situations, such as shopping malls. We can estimate a projector's position and pose with the robot's self-localization sensors, but the accuracy of this approach remains inadequate for projection mapping. Consequently, the proposed method solves this problem by combining self-localization by robot sensors with position and pose estimation of projection targets based on a 3D model. We first obtain the projection target's 3D model and then use it to accurately estimate the target's position and pose and thus achieve accurate projection mapping with a projector robot. In addition, our proposed method performs accurate projection mapping even after a projection target has been moved, which often occur in shopping malls. In this paper, we employ Ubiquitous Display (UD), which we are researching as a projector robot, to experimentally evaluate the effectiveness of the proposed method.

Keywords

Acknowledgement

Supported by : JSPS

References

  1. Akiyama, R., Yamamoto, G., Amano, T., Taketomi, T., Sandor, C. and Kato, H. (2016), "Appearance control in dynamic light environments with a projector-camera system", Proceedings of IEEE VR Workshop on Perceptual and Cognitive Issues in AR (PERCAR), Greenville, South Carolina, U.S.A., March.
  2. Bermano, A.H., Billeter, M., Iwai, D. and Grundhofer, A. (2017), "Makeup lamps: Live augmentation of human faces via projection", Comput. Graph. Forum, 36(2), 311-323. https://doi.org/10.1111/cgf.13128
  3. Bimber, O. and Raskar, R. (2005), Spatial Augmented Reality: Merging Real and Virtual Worlds, CRC Press Inc., Boca Raton, Florida, U.S.A.
  4. Box (2013), .
  5. Brenneke, C., Wulf, O. and Wagner, B. (2003), "Using 3D laser range data for slam in outdoor environments", Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003), Orlando, Florida, U.S.A., May.
  6. Cerevo Inc. (2016), Tipron, .
  7. Chen, J., Yamamoto, T., Aoyama, T., Takaki, T. and Ishii, I. (2015), "Real-time projection mapping using high-frame-rate structured light 3D vision", SICE J. Control Measure. Syst. Integr., 8(4), 265-272. https://doi.org/10.9746/jcmsi.8.265
  8. Choi, S.W., Kim, W.J. and Lee, C.H. (2013), "Interactive display robot: Projector robot with natural user interface", Proceedings of the 8th ACM/IEEE International Conference on Human-Robot Interaction, Tokyo, Japan, March.
  9. Cignoni, P., Callieri, M., Corsini, M., Dellepiane, M., Ganovelli, F. and Ranzuglia, G. (2008), "MeshLab: An open-source mesh processing tool", Proceedings of the 6th Eurographics Italian Chapter Conference, Salerno, Italy, July.
  10. Cole, D.M. and Newman, P.M. (2006), "Using laser range data for 3D SLAM in outdoor environments", Proceedings of IEEE International Conference on Robotics and Automation (ICRA2006), Orlando, Florida, U.S.A., May.
  11. Doering, N., Poeschl, S., Gross, H.M., Bley, A., Martin, C. and Boehme, H.J. (2015), "User-centered design and evaluation of a mobile shopping robot", J. Soc. Robot., 7(2), 203-225. https://doi.org/10.1007/s12369-014-0257-8
  12. Fukiage, T., Kawabe, T., Sawayama, M. and Nishida, S. (2017), "Animating static objects by illusion-based projection mapping", J. Soc. Inform. Display, 25(7), 434-443. https://doi.org/10.1002/jsid.572
  13. Glas, D.F., Satake, S., Ferreri, F., Kanda, T., Hagita, N. and Ishiguro, H. (2012), "The network robot system: Enabling social human-robot interaction in public spaces", J. Hum. Robot Interact., 1(2), 5-32.
  14. Kasetani, M., Noguchi, T., Yamazoe, H. and Lee, J.H. (2015), "Projection mapping by mobile projector robot", Proceedings of the 12th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI2015), Goyang, Korea, October.
  15. Keecker (2017), Keecker, .
  16. Lee, J., Kim, Y., Heo, M.H., Kim, D. and Shin, B.S. (2015), "Real-time projection-based augmented reality system for dynamic objects in the performing arts", Symmetry, 7(1), 182-192. https://doi.org/10.3390/sym7010182
  17. Lee, J.H. (2007), "Human centered ubiquitous display in intelligent space", Proceedings of the 33rd Annual Conference of the IEEE Industrial Electronics Society (IECON2007), Taipei, Taiwan, November.
  18. Leutert, F., Herrmann, C. and Schilling, K. (2013), "A spatial augmented reality system for intuitive display of robotic data", Proceedings of the 8th ACM/IEEE International Conference on Human-Robot Interaction, Tokyo, Japan, March.
  19. Machino, T., Iwaki, S., Kawata, H., Yanagihara, Y., Nanjo, Y. and Shimokura, K. (2006), "Remotecollaboration system using mobile robot with camera and projector", Proceedings of IEEE International Conference on Robotics and Automation (ICRA2006), Orlando, Florida, U.S.A., May.
  20. Maegawa, K., Shiotani, T., Iwamoto, K., Noguchi, T., Kasetani, M. and Lee, J.H. (2013), "Ubiquitous display 2.0: Development of new prototype and software modules for improvement", Proceedings of the 10th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI2013), Jeju, Korea, November.
  21. Mine, M.R., van Baar, J., Grundhofer, A., Rose, D. and Yang, B. (2012), "Projection-based augmented reality in Disney theme parks", Comput., 45(7), 32-40. https://doi.org/10.1109/MC.2012.154
  22. MRPT (n.d.), Mobile Robot Programming Toolkit (MRPT), .
  23. Narita, G., Watanabe, Y. and Ishikawa, M. (2017), "Dynamic projection mapping onto deforming non-rigid surface using deformable dot cluster marker", IEEE Trans. Visual. Comput. Graph., 23(3), 1235-1248. https://doi.org/10.1109/TVCG.2016.2592910
  24. Newcombe, R.A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A.J., Kohi, P., Shotton, J., Hodges, S. and Fitzgibbon, A. (2011), "KinectFusion: Real-time dense surface mapping and tracking", Proceedings of the 10th IEEE International Symposium on Mixed and Augmented Reality (ISMAR2011), Basel, Switzerland, October.
  25. Noguchi, T., Kasetani, M., Yamazoe, H. and Lee, J.H. (2015), "Proposal of a behavior model for information support with ubiquitous display", Proceedings of the 12th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI2015), Goyang, Korea, October.
  26. Park, J. and Kim, G.J. (2009), "Robots with projectors: An alternative to anthropomorphic HRI", Proceedings of the 4th ACM/IEEE International Conference on Human Robot Interaction, San Diego, California, U.S.A., March.
  27. Rok, H. and Woong, D. (2013), "A study of digital media art utilizing the contents of the architecture cultural property", J. Asia Digital Art Des., 17(2), 77-84.
  28. Rusu, R.B. and Cousins, S. (2011), "3D is here: Point cloud library (PCL)", Proceedings of the IEEE International Conference on Robotics and Automation (ICRA2011), Shanghai, China, May.
  29. Schoning, J., Rohs, M., Kratz, S., Lochtefeld, M. and Kruger, A. (2009), "Map torchlight: A mobile augmented reality camera projector unit", Proceedings of the CHI'09 Extended Abstracts on Human Factors in Computing Systems, Boston, Massachusetts, U.S.A., April.
  30. Siegl, C., Lange, V., Stamminger, M., Bauer, F. and Thies, J. (2017), "FaceForge: Markerless non-rigid face multi-projection mapping", IEEE Trans. Visual. Comput. Graph., 23(11), 2440-2446. https://doi.org/10.1109/TVCG.2017.2734428
  31. Stricker, R., Muller, S. and Gross, H.M. (2015), "Universal usage of a video projector on a mobile guide robot", Proceedings of the International Conference on Intelligent Robotics and Applications, Portsmouth, England, August.
  32. Sueishi, T., Oku, H. and Ishikawa, M. (2017), "Lumipen 2: Dynamic projection mapping with mirror-based robust high-speed tracking against illumination changes", PRESENCE Teleoper. Virtual Environ., 24(4), 299-321.
  33. Tatsumoto, K., Iwaki, S. and Ikeda, T. (2017), "Tracking projection method for 3D space by a mobile robot with camera and projector based on a structured-environment approach", Artif. Life Robot., 22(1), 90-101. https://doi.org/10.1007/s10015-016-0312-7