A User Interface for Vision Sensor based Indirect Teaching of a Robotic Manipulator

시각 센서 기반의 다 관절 매니퓰레이터 간접교시를 위한 유저 인터페이스 설계

  • Kim, Tae-Woo (Computer Software Engineering, University of Science and Technology) ;
  • Lee, Hoo-Man (Intelligence and Robot Control Research Section, Eletronics and Telecommunications research institute) ;
  • Kim, Joong-Bae (Intelligence and Robot Control Research Section, Eletronics and Telecommunications research institute)
  • 김태우 (과학기술연합대학원대학교(UST) 컴퓨터 소프트웨어 및 공학) ;
  • 이후만 (한국전자통신연구원(ETRI) 지능로봇제어연구실) ;
  • 김중배 (한국전자통신연구원(ETRI) 지능로봇제어연구실)
  • Received : 2013.04.12
  • Accepted : 2013.09.02
  • Published : 2013.10.01


This paper presents a user interface for vision based indirect teaching of a robotic manipulator with Kinect and IMU (Inertial Measurement Unit) sensors. The user interface system is designed to control the manipulator more easily in joint space, Cartesian space and tool frame. We use the skeleton data of the user from Kinect and Wrist-mounted IMU sensors to calculate the user's joint angles and wrist movement for robot control. The interface system proposed in this paper allows the user to teach the manipulator without a pre-programming process. This will improve the teaching time of the robot and eventually enable increased productivity. Simulation and experimental results are presented to verify the performance of the robot control and interface system.


Grant : 양팔 작업을 위한 센서융합 인지 기반 제어기술 개발 및 다중로봇 협업 생산공정 적용 기술 개발

Supported by : 한국산업기술평가관리원


  1. H. M. Lee, J. B. Kim, and S. H. Kim, "A survey on robot teaching: categorization and brief review," ICROS (in Korean), vol. 19, no. 1, pp. 49-59, May 2013.
  2. Z. Pan, J. Polden, N. Larkin, S. V. Duin, and J. Norrish, "Recent progress on programming methods for industrial robots," in Robotics and Computer-Integrated Manufacturing, vol. 28, no. 2, pp. 87-94, Apr. 2012.
  3. C. H. Park and K. T. Park "Design and control of industrial dual arm robot," KSPE (in Korean), vol. 25, no. 11, pp. 58-65, Nov. 2008.
  4. S. Y. Lee, J. W. Lee, W. J. Chung, M. S. Kim, C. W. Lee, and M. Park, "A new exoskeleton-type masterarm with force reflection: controller and integration," Proc. of 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems, Kyongju, Korea, vol. 3, pp. 1438-1443, Oct. 1999.
  5. H. C. Kim and C. Y. Lee, "Analysis on Kinematics and dynamics of human arm movement toward upper limb exoskeleton robot control Part 1: System model and kinematic constraint," Journal of Institute of Control, Robotics and Systems), vol. 18, no. 12, pp. 1106-1114, 2012.
  6. N. E. Sian, K. Yokoi, S. Kajita, F. Kanehiro, and K. Tanie, "Whole body teleoperation of a humanoid robot-development of a simple master device using joysticks," Proc. of 2002 IEEE/RSJ International Conference on Intelligent Robots and Systems, EPFL, Lausanne, Switzerland, vol. 3. pp. 2569-2574, Oct. 2002.
  7. V. V. Nguyen and J. H. Lee, "Full-body imitation of human motions with kinect and heterogeneous kinematic structure of humanoid robot," 2012 IEEE/SICE International Symposium on System Integration, Kyushu University, Fukuoka, Japan, pp. 93- 98, Dec. 2012.
  8. T. W. Kim, J. B. Kim, and H. M. Lee, "A user interface for indirect teaching of a robotic manipulator," Proc. of 2013 28th ICROS Annual Conference (in Korean), pp. 554-555, 2013.
  9. B. Jayasekara, K. Watanabe, and K. Izumi, "Controlling a robot manipulator with fuzzy voice commands guided by visual motor coordination learning," SICE Annual Conference, 2008, The University Electro-Communications, Japan, pp. 2540-2544, Aug. 2008.
  12. K. H. Ang, G. Chong, and Y. Li, "PID control system analysis, design, and technology," IEEE Transactions on Control Systems Technology, vol. 13, no. 4, pp. 559-576 Jul. 2005.
  13. ReferenceManual_D28065204B.pdf?version=1&modificatioDa te=1262834800000