대한전자공학회논문지SP (Journal of the Institute of Electronics Engineers of Korea SP)

대한전자공학회 (The Institute of Electronics and Information Engineers)
권호 표지

내용기반 동영상 기하학적 변환을 위한 실시간 처리 기법

A Real Time Processing Technique for Content-Aware Video Scaling

  • 이강희 (강원대학교 컴퓨터정보통신공학과) ;
  • 유재욱 (강원대학교 컴퓨터정보통신공학과) ;
  • 박대현 (강원대학교 컴퓨터정보통신공학과) ;
  • 김윤 (강원대학교 컴퓨터정보통신공학과)
  • Lee, Kang-Hee (Dept. of Computer and Communications Engineering, Kangwon National University) ;
  • Yoo, Jae-Wook (Dept. of Computer and Communications Engineering, Kangwon National University) ;
  • Park, Dae-Hyun (Dept. of Computer and Communications Engineering, Kangwon National University) ;
  • Kim, Yoon (Dept. of Computer and Communications Engineering, Kangwon National University)
  • 투고 : 2010.04.16
  • 심사 : 2010.12.20
  • 발행 : 2011.01.25
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

본 논문에서는 동영상이 가지고 있는 컨텐츠를 보존하면서 영상의 크기를 조절할 수 있는 실시간 동영상 표가 변환 기법을 제안한다. 제안하는 방법은 동영상 내의 연속하는 두 프레임 사이에 존재하는 상관성(correlation)을 이용하여, 이전 프레임의 seam 정보로부터 현재 프레임의 seam을 결정한다. 따라서, 전체 프레임들을 분석하지 않으면서도 컨텐츠의 떨림 현상을 발생시키지 않는다. 먼저, 전체 동영상 내에서 특정이 서로 비슷한 프레임들을 scene으로 구분하고, 각 scene 내의 첫번째 프레임은 정지영상의 seam carving을 사용하여 최대한 컨텐츠를 보존할 수 있도록 크기를 변환한다. 이 때, 영상의 크기를 변환하기 위해 추출한 seam에 대한 정보를 저장하고 그 이후의 프레임들은 이전 프레임에서 저장된 seam 정보를 참조하여 프레임 단위로 영상의 크기를 조절한다. 실험 결과는 제안하는 방법이 처리 속도와 메모리 사용량 면에서 실시간 처리에 적합하고, 영상이 가지고 있는 컨텐츠를 보전하면서 영상의 크기를 조절할 수 있음을 보여준다.

In this paper, a new real time video scaling technique which preserved the contents of a movie was proposed. Because in a movie a correlation exists between consecutive frames, in this paper by determining the seam of the current frame considering the seam of the previous frame, it was proposed the real time video scaling technique without the shaking phenomenon of the contents even though the entire video is not analyzed. For this purpose, frames which have similar features in a movie are classified into a scene, and the first frame of a scene is resized by the seam carving at the static images so that it can preserve the contents of the image to the utmost. At this time, the information about the seam extracted to convert the image size is saved, and the sizes of the next frames are controlled with reference to the seam information stored in the previous frame by each frame. The proposed algorithm has the fast processing speed of the extent of being similar to a bilinear method and preserves the main content of an image to the utmost at the same time. Also because the memory usage is remarkably small compared with the existing seam carving method, the proposed algorithm is usable in the mobile terminal in which there are many memory restrictions. Computer simulation results indicate that the proposed technique provides better objective performance and subjective image quality about the real time processing and shaking phenomenon removal and contents conservation than conventional algorithms.

키워드

참고문헌

  1. A. Santella and M. Agrawala and D. Decarlo and D. Salesin and M. Cohen, "Gaze-based interaction for semi-automatic photo cropping," ACM Human Factors in Computing Systems, pp. 771-780, 2006.
  2. F. Liu and M. Gleicher, "Automatic Image retargeting with fisheye-view warping," ACM Multimedia, pp. 153-164, Oct. 2005.
  3. S. Avidan and A. Shamir, "Seam Carving for Content-Aware Image Resizing," ACM Trans. on Graphics, Vol. 26, Issue 3, Jul. 2007.
  4. V. Setlur and S. Takagi and R. Raskar and M. Gleicher and B. Gooch, "Automatic Image retargeting," Proc. ACM, Vol. 154, pp. 59-68, 2005.
  5. C. Tao and J. Jia and H. Sun, "Active window oriented dynamic video retargeting," ICCV Proc. Workshop on Dynamical Vision, 2007.
  6. S. Cho and Y. Matsushita and S. Lee, "Image Retargeting with Importance Diffusion," KIISE, vol. 35, pp. 236-239, Jun. 2008.
  7. V. Setlur and T. Lechner and M. Nienhaus and B. Gooch, "Retargeting Images and Video for Preserving Information Saliency," IEEE Computer Graphics and Applications, vol. 27, no. 5, pp. 80-88, Oct. 1998.
  8. F. Liu and M. Gleichar, "Video Retargeting: Automating Pan and Scan," ACM Multimedia, pp. 241-250, Oct. 2006.
  9. M. Rubinstein and A. Shamir and S. Avidan, "Improved seam carving for video retargeting," ACM Transactions on Graphics, Vol. 27, Issue 3, Aug. 2008.
  10. B. Chen and P. Sem, "Video Carving," Proc. Eurographics, Short Papers, Apr. 2008.
  11. U. Gargi and R. Kasturi, Sh. Strayer, "Performance Characterization of Video-Shot- Change Detection Methods," IEEE Trans. On Circuits and Systems for Video Technology, vol. 10, no. 1, pp. 1-13, 2000. https://doi.org/10.1109/76.825852
  12. Y. Gong, "An accurate and robust method for detecting video shot boundaries," IEEE Multimedia Computing and Systems, vol. 1, pp. 850-854, 1999.
  13. L. Chen, X. Xie ,X. Fan, W. Ma, H. Zhang, and H. Zhou, "A visual attention model for adapting images on small displays," Multimedia Systems 9, 4, pp. 353-364, 2003. https://doi.org/10.1007/s00530-003-0105-4
  14. H. Liu, X. Xie, W. Ma, and H.Zhang "Automatic browsing of large pictures on mobile devices," Proceedings of the eleventh ACM international conference on Multimedia, pp. 148-155, 2003.
  15. B. Suh, H. Ling, B. B. Bederson, and D. W. Jacobs, "Automatic thumbnail cropping and its effectiveness," In UIST '03: Proceedings of the 16th annual ACM symposium on User interface software and technology, ACM Press, New York, NY, USA, pp. 95-104, 2003.
  16. X. Fan and X. Xie and H. Q. Zhou and W. Y. Ma, "Looking into video frames on small displays," ACM on Multimedia, pp. 247-250, 2003.
  17. Richard Bellman, "Some Problems in The Theory of Dynamic Programming," Econometrica, Vol. 22, No. 1, pp. 37-48, 1954. https://doi.org/10.2307/1909830
  18. D. P. Bertsekas, "Dynamic Programming and Optimal Control," Athena Scientific, 2000.
  19. P. Kohli and P. H. S. Torr, "Dynamic graph cuts for efficient inference in markov random fields," IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI) 29, 12, pp.2079-2088, 2007. https://doi.org/10.1109/TPAMI.2007.1128
  20. V. Kwatra, A. Sch'odl, I. Essa, G. Turk and A. Bobick, "Graphcut textures: image and video synthesis using graph cuts," ACM Trans. Graph. 22, 3, pp. 277-286, 2003. https://doi.org/10.1145/882262.882264