Journal of the Korea Computer Graphics Society (한국컴퓨터그래픽스학회논문지)

Korea Computer Graphics Society (한국컴퓨터그래픽스학회)
권호 표지
DOI QR코드

DOI QR Code

Composition of Foreground and Background Images using Optical Flow and Weighted Border Blending

옵티컬 플로우와 가중치 경계 블렌딩을 이용한 전경 및 배경 이미지의 합성

  • Received : 2014.06.19
  • Accepted : 2014.08.14
  • Published : 2014.09.01
Download PDF
⟨ Previous Next ⟩

Abstract

We propose a method to compose a foreground object into a background image, where the foreground object is a part (or a region) of an image taken by a front-facing camera and the background image is a whole image taken by a back-facing camera in a smart phone at the same time. Recent high-end cell-phones have two cameras and provide users with preview video before taking photos. We extract the foreground object that is moving along with the front-facing camera using the optical flow during the preview. We compose the extracted foreground object into a background image using a simple image composition technique. For better-looking result in the composed image, we apply a border smoothing technique using a weighted-border mask to blend transparency from background to foreground. Since constructing and grouping pixel-level dense optical flow are quite slow even in high-end cell-phones, we compute a mask to extract the foreground object in low-resolution image, which reduces the computational cost greatly. Experimental result shows the effectiveness of our extraction and composition techniques, with much less computational time in extracting the foreground object and better composition quality compared with Poisson image editing technique which is widely used in image composition. The proposed method can improve limitedly the color bleeding artifacts observed in Poisson image editing using weighted-border blending.

스마트폰의 전면 및 후면 카메라를 이용하여 동시에 획득한 전경 이미지와 배경 이미지에서, 전경 이미지의 일부분인 전경 물체를 추출하여 배경 이미지에 합성하는 방법을 제시한다. 최근의 고사양 스마트폰은 대개 두 개의 카메라를 가지고 있고, 사진을 촬영하는 과정에서 미리보기 화면을 제공한다. 전면 카메라로부터 전경 이미지를 획득하는 과정에서 미리보기 화면의 비디오에 대한 옵티컬 플로우를 이용하여 전경 물체를 추출한다. 추출된 전경 물체와 배경 화면을 단순히 합성한 후, 전경 물체와 배경화면의 경계에서 가중치 경계 블렌딩을 이용하여 시각적으로 부드러운 경계를 갖는 합성을 수행한다. 화소 수준의 조밀한 옵티컬 플로우의 계산은 고사양의 스마트폰에서도 상당히 느리기 때문에, 전경 물체 추출을 위한 마스크의 계산을 저해상도에서 수행하여 계산시간을 크게 절약할 수 있다. 실험적 결과에 의하면 제안하는 방법은 더 적은 계산 시간을 사용하며, 널리 사용되는 Poisson 이미지 합성 방법에 비하여 시각적으로 더 우수한 결과를 얻을 수 있다. 제안하는 방법은 Poisson 이미지 합성 방법에서 자주 관찰되는 색 번짐 결점을 가중치 경계 블렌딩을 이용하여 제한적인 수준에서 극복할 수 있다.

Keywords

References

  1. M. Piccardi, "Background subtraction techniques: a review," in IEEE International Conference on Systems, Man and Cybernetics, vol. 4, Oct 2004, pp. 3099-3104 vol.4.
  2. I. Kartika and S. Mohamed, "Frame differencing with postprocessing techniques for moving object detection in outdoor environment," in IEEE 7th International Colloquium on Signal Processing and its Applications (CSPA), March 2011, pp. 172-176.
  3. J.Weber and J. Malik, "Robust computation of optical flow in a multi-scale differential framework," in Fourth International Conference on Computer Vision, May 1993, pp. 12-20.
  4. M. Grundland, R. Vohra, G. P. Williams, and N. A. Dodgson, "Nonlinear multiresolution image blending," International Journal of Machine Graphics & Vision, vol. 15, no. 3, pp. 381-390, Jan. 2006. [Online]. Available: http://dl.acm.org/citation.cfm?id=1375858.1375874
  5. P. P´erez, M. Gangnet, and A. Blake, "Poisson image editing," ACM Trans. Graph., vol. 22, no. 3, pp. 313-318, July 2003. [Online]. Available: http://doi.acm.org/10.1145/882262.882269
  6. M. Kazhdan and H. Hoppe, "Streaming multigrid for gradient-domain operations on large images," ACM Trans. Graph., vol. 27, no. 3, pp. 21:1-21:10, Aug. 2008. [Online]. Available: http://doi.acm.org/10.1145/1360612.1360620
  7. A. Agarwala, "Efficient gradient-domain compositing using quadtrees," ACM Trans. Graph., vol. 26, no. 3, July 2007. [Online]. Available: http://doi.acm.org/10.1145/1276377.1276495
  8. Z. Farbman, G. Hoffer, Y. Lipman, D. Cohen- Or, and D. Lischinski, "Coordinates for instant image cloning," ACM Trans. Graph., vol. 28, no. 3, pp. 67:1-67:9, July 2009. [Online]. Available: http://doi.acm.org/10.1145/1531326.1531373
  9. S. Jeschke, D. Cline, and P.Wonka, "A GPU Laplacian solver for diffusion curves and Poisson image editing," ACM Trans. Graph., vol. 28, no. 5, pp. 116:1-116:8, Dec. 2009. [Online]. Available: http://doi.acm.org/10.1145/1618452.1618462
  10. J. Jia, J. Sun, C.-K. Tang, and H.-Y. Shum, "Dragand- drop pasting," ACM Trans. Graph., vol. 25, no. 3, pp. 631-637, July 2006. [Online]. Available: http://doi.acm.org/10.1145/1141911.1141934
  11. J. Sun, J. Jia, C.-K. Tang, and H.-Y. Shum, "Poisson matting," ACM Trans. Graph., vol. 23, no. 3, pp. 315-321, Aug. 2004. [Online]. Available: http://doi.acm.org/10.1145/1015706.1015721
  12. T. Georgiev, "Photoshop healing brush: a tool for seamless cloning." [Online]. Available: http://www.tgeorgiev.net/Photoshop Healing.pdf
  13. Samsumg galaxy s4: Dual shot. [Online]. Available: http://www.samsung.com/global/microsite/galaxys4/fun.html #page=dualshot
  14. C. Zach, T. Pock, and H. Bischof, "A duality based approach for realtime TV-L1 optical flow," in Proceedings of the 29th DAGM Conference on Pattern Recognition. Berlin, Heidelberg: Springer-Verlag, 2007, pp. 214-223. [Online]. Available: http://dl.acm.org/citation.cfm?id=1771530.1771554
  15. Y. Tanaka, M. Hasegawa, and S. Kato, "Improved image concentration for artifact-free image dilution and its application to image coding," in 17th IEEE International Conference on Image Processing (ICIP), Sept 2010, pp. 1225-1228.