Journal of Korea Multimedia Society (한국멀티미디어학회논문지)

Korea Multimedia Society (한국멀티미디어학회)
권호 표지
DOI QR코드

DOI QR Code

Automatic Jitter Evaluation Method from Video using Optical Flow

Optical Flow를 사용한 동영상의 흔들림 자동 평가 방법

  • Baek, Sang Hyune (Dept. of Computer Eng., Graduate School, Ajou University) ;
  • Hwang, WonJun (Dept. of Computer Eng., Graduate School, Ajou University)
  • Received : 2017.07.04
  • Accepted : 2017.07.28
  • Published : 2017.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose a method for evaluating the uncomfortable shaking in the video. When you shoot a video using a handheld device, such as a smartphone, most of the video contains unwanted shake. Most of these fluctuations are caused by hand tremors that occurred during shooting, and many methods for correcting them automatically have been proposed. It is necessary to evaluate the shake correction performance in order to compare the proposed shake correction methods. However, since there is no standardized performance evaluation method, a correction performance evaluation method is proposed for each shake correction method. Therefore, it is difficult to make objective comparison of shake correction method. In this paper, we propose a method for objectively evaluating video shake. Automatically analyze the video to find out how much tremors are included in the video and how much the tremors are concentrated at a specific time. In order to measure the shaking index, we proposed jitter modeling. We applied the algorithm implemented by Optical Flow to the real video to automatically measure shaking frequency. Finally, we analyzed how the shaking indices appeared after applying three different image stabilization methods to nine sample videos.

Keywords

References

  1. Image Stabilization, https://en.wikipedia.org/wiki/Image_stabilization (accessed Jun., 29, 2017).
  2. N. Ejaz, W.I. Kim, S.I. Kwon, and S.W. Baik, "Video Stabilization by Detecting Intentional and Unintentional Camera Motions," Proceeding of International Conference on Intelligent Systems, Modeling and Simulation, pp. 312-316, 2012.
  3. L. Mercenaro, G. Vernazza, and C. Regazzoni, "Image Stabilization Algorithms for Video-Survellance Application," Proceeding of International Conference on Image Processing, pp. 349-352, 2001.
  4. Y.G. Ryu and M.J. Chung, "Robust Online Digital Image Stabilization Based on Point-Feature Trajectory Without Accumulative Global Motion Estimation," Journal of IEEE Signal Processing Letters, Vol. 19, No. 4. pp. 223-226, 2012. https://doi.org/10.1109/LSP.2012.2188286
  5. J.H. Kim, Y.M. Baek, J.H. Yun, and W.Y. Kim, "In-Car Video Stabilization using Focus of Expansion," J. of Korea Multimedia Society Vol. 14, No. 12, pp. 1536-1542, 2011. https://doi.org/10.9717/kmms.2011.14.12.1536
  6. S.H. Yang and F.M. Jheng, "An Adaptive Image Stabilization Technique," Proceeding of IEEE International Conference on Systems, Man and Cybernetics, pp. 1968-1973, 2006.
  7. STMicroelectronics, Optical Image Stabilization, http://www.st.com/content/ccc/resource/technical/document/white_paper/c9/a6/fd/e4/e6/4e/48/60/ois_white_paper.pdf/files/ois_white_paper.pdf/jcr:content/translations/en.ois_white_paper.pdf (accessed Jun., 29, 2017).
  8. Interntional Telecommunication Union, Methods for Subjective Determination of Transmission Quality, ITU-T Recommendation, http://www.itu.int/rec/T-REC-P.800-199608-I (accessed Jun., 29, 2017).
  9. M. Niskanen, O. Silven, and M. Tico, "Video Stabilization Performance Assessment," Proceeding of IEEE International Conference on Multimedia and Expo, pp. 405-408, 2006.
  10. D. Halliday, R. Resnick, and J. Walker, Fundamentals of Physics Extended 10th Edition, Wiley, Hoboken, New Jersey, 2014.
  11. H.W. Baek, Y.J. Hur, M.G. Song, N.C. Park, Y.P. Park, K.S. Park, S.C. Lim, et al., “Development for OIS Actuator for Mobile Phone Camera,” Journal of The Society Information Storage System, Vol. 5, No. 1, pp. 8-13, 2009.
  12. B.D. Lucas and T. Kanade, "An Iterative Image Registration Technique with and Application to Stereo Vision," Proceeding of International Joint Conference on Artificial Intelligence, pp. 674-679, 1981.
  13. J.Y. Bouguet, Pyramidal Imaplementation of the Affine Lucas Kanade Feature Tracker Description of the Algorithm, Intel Corporation, Technical Report, 1999.
  14. Google Drive, Sample Video, https://drive.google.com/drive/folders/0B-Ws5b8zpBH-OVJZS2JwSkk3NDA?usp=sharing, (accessed Jun., 29, 2017).
  15. iMovie, https://itunes.apple.com/us/app/imovie/id408981434?mt=12, (accessed Jun., 29, 2017).
  16. ffmpeg, https://www.ffmpeg.org/, (accessed Jun., 29, 2017).
  17. Elasty (Video Toolbox), http://www.creaceed.com/elasty (accessed Jun., 29, 2017).