Journal of Korea Multimedia Society (한국멀티미디어학회논문지)

Korea Multimedia Society (한국멀티미디어학회)
권호 표지
DOI QR코드

DOI QR Code

A Monocular Vision Based Technique for Estimating Direction of 3D Parallel Lines and Its Application to Measurement of Pallets

모노 비전 기반 3차원 평행직선의 방향 추정 기법 및 파렛트 측정 응용

  • Kim, Minhwan (School of Electrical and Computer Engineering, College of Engineering, Pusan National University) ;
  • Byun, Sungmin (School of Electrical and Computer Engineering, College of Engineering, Pusan National University) ;
  • Kim, Jin (Dept. of Computer Engineering, Jungwon University)
  • Received : 2018.09.20
  • Accepted : 2018.10.22
  • Published : 2018.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

Many parallel lines may be shown in our real life and they are useful for analyzing structure of objects or buildings. In this paper, a vision based technique for estimating three-dimensional direction of parallel lines is suggested, which uses a calibrated camera and is applicable to an image being captured from the camera. Correctness of the technique is theoretically described and discussed in this paper. The technique is well applicable to measurement of orientation of a pallet in a warehouse, because a pair of parallel lines is well detected in the front plane of the pallet. Thereby the technique enables a forklift with a well-calibrated camera to engage the pallet automatically. Such a forklift in a warehouse can engage a pallet on a storing rack as well as one on the ground. Usefulness of the suggested technique for other applications is also discussed. We conducted an experiment of measuring a real commercial pallet with various orientation and distance and found for the technique to work correctly and accurately.

Keywords

MTMDCW_2018_v21n11_1254_f0001.png 이미지

Fig. 1. An example image with a few support lines in order to discuss usefulness of parallel lines in our life.

MTMDCW_2018_v21n11_1254_f0002.png 이미지

Fig. 2. Various pallet images with two or three edge lines that are useful for estimating orientation of each pallet.

MTMDCW_2018_v21n11_1254_f0003.png 이미지

Fig. 3. Perspective projection in a XYZ-camera coor-dinate system onto a xy-image coordinate sys-tem.

MTMDCW_2018_v21n11_1254_f0004.png 이미지

Fig. 4. Core idea of estimating three-dimensional di-rection of parallel lines. Cross product of two normal vectors for the upper plane and the low-er plane indicates their 3D direction.

MTMDCW_2018_v21n11_1254_f0005.png 이미지

Fig. 5. Diagram to show an estimation method of three-dimensional parallel line direction based on four selected points.

MTMDCW_2018_v21n11_1254_f0006.png 이미지

Fig. 6. An approach to determination of pallet orientation in [6]. (a) relation diagram between the (yellow) front plane of a pallet and its back-projection onto a virtual plane with various orientation, (b) a case of aligning the virtual plane parallel to the front plane of pallet.

MTMDCW_2018_v21n11_1254_f0007.png 이미지

Fig. 7. Sample images of pallet with various orientation and distance to test usefulness of the proposed method.

Table 1. Experimental results for four pallets with various distance and orientation

MTMDCW_2018_v21n11_1254_t0001.png 이미지

Table 2. Execution times for measuring orientation and location of pallets

MTMDCW_2018_v21n11_1254_t0002.png 이미지

References

  1. G. Garibotto, S. Masciangelo, M. Ilic, and P. Bassino, "ROBOLIFT: A Vision Guided Autonomous Fork-lift for Pallet Handling," Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 656-663, 1996.
  2. J. Pages, X. Armangue, J. Salvi, J. Freixenet, and J. Marti, "A Computer Vision System for Autonomous Forklift Vehicles in Industrial Environments," Proceedings of 9th Mediterranean Conference on Control and Automation, pp. 1-6, 2001.
  3. D. Lecking, O. Wulf, and B. Wagner, "Variable Pallet Pick-Up for Automatic Guided Vehicles in Industrial Environments," Proceedings of IEEE Conference on Emerging Technologies and Factory Automation, pp. 1169-1174, 2006.
  4. M. Seelinger and J.D. Yoder, "Automatic Visual Guidance of a Forklift Engaging a Pallet," Robotics and Autonomous Systems, Vol. 54, Issue 12, pp. 1026-1038, 2006. https://doi.org/10.1016/j.robot.2005.10.009
  5. S. Byun and M. Kim, "Real-Time Positioning and Orienting of Pallets Based on Monocular Vision," Proceedings of IEEE International Conference on Tools with Artificial Intelligence (ICTAI), pp. 505-508, 2008.
  6. S. Byun and M. Kim, "Pallet Measurement Method for Automatic Pallet Engaging in Real-Time," Journal of Korea Multimedia Society, Vol. 14, No. 2, pp. 171-181, 2011. https://doi.org/10.9717/kmms.2011.14.2.171
  7. M.R. Walter, S. Karaman, E. Frazzoli, and S. Teller, "Closed-Loop Pallet Engagement in an Unstructured Environment," Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5119-5126, 2010.
  8. D. Haanpaa, G. Beach, and C.J. Cohen, "Machine Vision Algorithms for Robust Pallet Engagement and Stacking," Proceedings of IEEE Applied Imagery Pattern Recognition Workshop (AIPR), pp. 1-8, 2016.
  9. J. Xiao, H. Lu, L. Zhang, and J. Zhang, "Pallet Recognition and Localization Using an RGB-D Camera," International Journal of Advanced Robotic Systems, Vol. 14, Issue 6, 2017.
  10. L. Baglivo, N. Biasi, F. Biral, N. Bellomo, E. Bertolazzi, M.D. Lio, et al., "Autonomous Pallet Localization and Picking for Industrial Forklifts: A Robust Range and Look Method," Measurement Science and Technology, Vol. 22, No. 8, 2011.
  11. R. Jain, R. Kasturi, and B.G. Schunck, Machine Vision, McGraw-Hill, New York, 1995.
  12. Z. Zhang, A Flexible New Technique for Camera Calibration, Technical Report MSR-TR-98-71, 1998.
  13. R. Hartley and A. Zisserman, Multiview Geometry in Computer Vision, Cambridge University Press, Cambridge, 2003.
  14. Flat Pallets for Through Transit (KS T 1372), http://standard.go.kr/KSCI/standardIntro/getStandardSearchView.do (accessed Sep., 13, 2018).
  15. A.J. Davison, I.D. Reid, N.D. Molton, and O. Stasse, “MonoSLAM: Real-Time Single Camera SLAM,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 29, No. 6, pp. 1052-1067, 2007. https://doi.org/10.1109/TPAMI.2007.1049
  16. W. Hu, T. Tan, L. Wang, and S. Maybank, “A Survey on Visual Surveillance of Object Motion and Behaviors,” IEEE Transaction on System, Man and Cybernetics, Part C: Applications and Reviews, Vol. 34, No. 3, pp. 334-352, 2004. https://doi.org/10.1109/TSMCC.2004.829274