KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Study on 3 DoF Image and Video Stitching Using Sensed Data

  • Kim, Minwoo (Department of Computer Engineering, Myongji University) ;
  • Chun, Jonghoon (Department of Computer Engineering, Myongji University) ;
  • Kim, Sang-Kyun (Department of Computer Engineering, Myongji University)
  • Received : 2016.11.22
  • Accepted : 2017.05.25
  • Published : 2017.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

This paper proposes a method to generate panoramic images by combining conventional feature extraction algorithms (e.g., SIFT, SURF, MPEG-7 CDVS) with sensed data from inertia sensors to enhance the stitching results. The challenge of image stitching increases when the images are taken from two different mobile phones with no posture calibration. Using inertia sensor data obtained by the mobile phone, images with different yaw, pitch, and roll angles are preprocessed and adjusted before performing stitching process. Performance of stitching (e.g., feature extraction time, inlier point numbers, stitching accuracy) between conventional feature extraction algorithms is reported along with the stitching performance with/without using the inertia sensor data. In addition, the stitching accuracy of video data was improved using the same sensed data, with discrete calculation of homograph matrix. The experimental results for stitching accuracies and speed using sensed data are presented in this paper.

Keywords

References

  1. Panorama VR, from http://www.uok3d.com/bussiness/panorama.php
  2. K. M. Park, W. H. Seok, and K. H. Lee, "Sensor Industry and Major Promising Sensor Market and Technology Trend," Electronics and Telecommunications Research Institute, February, 2015.
  3. S. H. Lee, "State and the Issues of IOT," Institute for Information & communications Technology Promotion, April, 2014.
  4. D. G. Lowe, "Distinctive Image Features from Scale-invariant Keypoints," International Journal of Computer Vision, vol. 60, no. 2, pp. 91-110, May, 2004. https://doi.org/10.1023/B:VISI.0000029664.99615.94
  5. H. Bay, A. Ess, T. Tuytelaars, and L. V. Gool, "Speeded-Up Robust Features (SURF)," Similarity Matching in Computer Vision and Multimedia, vol. 110, no. 3, pp. 346-359, June, 2008.
  6. H. Bay, T. Tuytelaars, and L. V. Gool, "SURF: Speeded Up Robust Features," in Proc. of 9th European Conference on Computer Vision, 404-417, May, 2006.
  7. P. M. Panchal, S. R. Panchal, and S. K. Shah, "A Comparison of SIFT and SURF," International Journal of Innovative Research in Computer and Communication Engineering, vol. 1, no. 2, pp. 323-327, April, 2013.
  8. L. Juan and O. Gwun, "A Comparison of SIFT, PCA-SIFT and SURF," International Journal of Image Processing (IJIP), vol. 3, no. 4, pp. 143-152, 2009.
  9. CDVS1, "Call for Proposals for Compact Descriptors for Visual Search," N12201, Turin, Italy, ISO/IEC JTC1/SC29/WG11, 2011.
  10. L. Y. Duan, F. Gao, J. Chen, J. Lin, and T. Huang, "Compact Descriptors for Mobile Visual Search and MPEG CDVS Standardization," in Proc. of IEEE International Symposium on Circuits and Systems, pp. 885-888, May, 2013.
  11. H. K. Kim, "MPEG-7 CDVS Standardization Media Searching under Mobile Environment," Telecommunications Technology Association, vol. 144, pp. 35-38, 2012.
  12. M. A. Fischler and R. C. Bolles, "Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography," Communications of the ACM, vol. 24, no. 6, pp. 381-395, June, 1981. https://doi.org/10.1145/358669.358692
  13. E. Dubrofsky, "Homography Estimation," University of British Columbia, 2009.
  14. Y. H. Park and O. S. Kwon, "Multiple Homographies Estimation using a Guided Sequential RANSAC," The Korea Contents Association, vol. 10, no. 7, pp. 10-22, July, 2010.
  15. Kyoungro Yoon, Sang-Kyun Kim, Jae Joon Han, Seungju Han, Marius Preda, "MPEG-V: Bridging the virtual and real world," Elsevier, ISBN: 978-0-12-420140-8.
  16. Yong-Soo Joo, Sang-Kyun Kim, "Sensory Effect Authoring Tool for Sensible Media," Journal of Broadcast Engineering, Vol. 16, No. 5, pp. 693-893, Sept. 2011 (in Korean). https://doi.org/10.5909/JEB.2011.16.5.693
  17. Sang-Kyun Kim, "Authoring Multisensorial Content," Signal Processing: Image Communication, vol. 28, Issue 2, pp. 162-167, Feb. 2013. https://doi.org/10.1016/j.image.2012.10.011
  18. Sang-Kyun Kim, Yong-Soo Joo, YongMi Lee, "Sensible Media Simulation in an Automobile Application and Human Responses to Sensory Effects," ETRI Journal, Vol. 35, No. 6, pp. 1001-1010, Dec. 2013. https://doi.org/10.4218/etrij.13.2013.0038
  19. Sang-Kyun Kim, Seung-Jun Yang, Chung Hyun AHN, Yong Soo Joo, "Sensorial Information Extraction and Mapping to Generate Temperature Sensory Effects," ETRI Journal, Vol. 36, No. 2, pp. 224-231, Apr. 2014. https://doi.org/10.4218/etrij.14.2113.0065
  20. Sang-Kyun Kim, Jae Joon Han, Seungju Han, Yong Soo Joo, "Virtual world control system using sensed information and adaptation engine," Signal Processing: Image Communication, vol. 28, Issue 2, pp. 87-96, Feb. 2013. https://doi.org/10.1016/j.image.2012.10.006
  21. M. Brown and D. G. Lowe, "Automatic Panoramic Image Stitching using Invariant Features," International Journal of Computer Vision, vol. 74, no. 1, pp. 59-73, December, 2007. https://doi.org/10.1007/s11263-006-0002-3
  22. J. Heikkila and O. Silven, "A Four-step Camera Calibration Procedure with Implicit Image Correction," in Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1106-1112, Jun. 1997.