Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography (한국측량학회지)

Korean Society of Surveying, Geodesy, Photogrammetry and Cartography (한국측량학회)
권호 표지
DOI QR코드

DOI QR Code

High Resolution Photo Matting for Construction of Photo-realistic Model

실감모형 제작을 위한 고해상도 유물 이미지 매팅

  • Received : 2022.01.13
  • Accepted : 2022.02.04
  • Published : 2022.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, there are various studies underway on the deep learning-used image matting methods. Even in the field of photogrammetry, a process of extracting information about relics from images photographed is essential to produce a high-quality realistic model. Such a process requires a great deal of time and manpower, so chroma-key has been used for extraction so far. This method is low in accuracy of sub-classification, however, it is difficult to apply the existing method to high-quality realistic models. Thus, this study attempted to remove background information from high-resolution relic images by using prior background information and trained learning data and evaluate both qualitative and quantitative results of the relic images extracted. As a result, this proposed method with FBA(manual trimap) showed quantitatively better results, and even in the qualitative evaluation, it was high in accuracy of classification around relics. Accordingly, this study confirmed the applicability of the proposed method in the indoor relic photography since it showed high accuracy and fast processing speed by acquiring prior background information when classifying high-resolution relic images.

최근 딥러닝을 이용한 이미지 매팅 방법에 관한 다양한 연구가 진행되고 있다. 특히, 사진측량 분야에서도 고품질의 실감모형을 제작하기 위해서는 촬영된 이미지에서 유물 정보를 추출하는 과정이 필요하며, 이와 같은 과정은 많은 시간과 인력이 들어 기존에는 크로마키를 이용하여 추출하는 방법이 많이 활용되고 있다. 그러나, 기존의 방법은 세부 분류에 대한 정확도가 떨어져 고품질 실감모형에 적용하기에는 어려움이 있었다. 본 연구에서는 사전배경정보와 훈련된 학습데이터를 이용하여 고해상도 유물 이미지에서 배경정보를 제거하고 추출된 유물 이미지에 대하여 정성적, 정량적 결과를 평가하였다. 그 결과 제안된 방법과 FBA(매뉴얼 트라이맵)이 정량적으로 높은 결과를 나타냈으며, 정성적 평가에서도 유물 주변부의 분류도가 높은 정확도를 보였다. 따라서 제안된 방법은 고해상도 유물 이미지 분류에 있어 사전배경정보 취득을 통하여 높은 정확도와 빠른 처리 속도를 나타냈으며, 실내 유물 촬영에서 그 활용 가능성을 확인하였다.

Keywords

Acknowledgement

본 연구는 2019학년도 충북대학교 학술연구지원사업에 의하여 연구되었고, 한국연구재단 (교육부) 기초과학연구프로그램 지원사업에 의해 수행된 연구임(NRF2018R1D1A1B07048841). 이 논문은 2020학년도 충북대학교 연구년제 사업의 연구비 지원에 의하여 연구되었음.

References

  1. Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F. and Adam, H.(2018), Encoder-decoder with atrous separable convolution for semantic image segmentation, In Proceedings of the European conference on computer vision, pp. 801-818.
  2. Chuang, Y.Y., Curless, B., Salesin, D.H. and Szeliski, R.(2001, December), A bayesian approach to digital matting, In Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, pp. II-II.
  3. Forte, M. and Pitie, F.(2020), F, B Alpha Matting, arXiv preprint arXiv:2003.07711.
  4. Gastal, E.S. and Oliveira, M.M.(2010, May), Shared sampling for real-time alpha matting, In Computer Graphics Forum, Vol. 29, No. 2, pp. 575-584. https://doi.org/10.1111/j.1467-8659.2009.01627.x
  5. He, K., Gkioxari, G., Dollar, P. and Girshick, R.(2017), Mask r-cnn, In Proceedings of the IEEE international conference on computer vision, pp. 2961-2969.
  6. Levin, A., Lischinski, D. and Weiss, Y.(2007), A closed-form solution to natural image matting, IEEE transactions on pattern analysis and machine intelligence, Vol. 30, No. 2, pp. 228-242. https://doi.org/10.1109/TPAMI.2007.1177
  7. Lin, S., Ryabtsev, A., Sengupta, S., Curless, B.L., Seitz, S.M. and Kemelmacher-Shlizerman, I.(2021), Real-time high-resolution background matting, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8762-8771.
  8. Liu, J., Yao, Y., Hou, W., Cui, M., Xie, X., Zhang, C. and Hua, X.S.(2020), Boosting semantic human matting with coarse annotations, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8563-8572.
  9. Ni, J., Zhang, F., Ma, F., Yin, Q. and Xiang, D.(2021), Random region matting for the high-resolution PolSAR image semantic segmentation, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, Vol. 14, pp 3040-3051. https://doi.org/10.1109/JSTARS.2021.3062447
  10. Qiao, Y., Liu, Y., Yang, X., Zhou, D., Xu, M., Zhang, Q. and Wei, X.(2020), Attention-guided hierarchical structure aggregation for image matting, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13676-13685.
  11. Rhemann, C., Rother, C., Wang, J., Gelautz, M., Kohli, P. and Rott, P.(2009, June), A perceptually motivated online benchmark for image matting, In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1826-1833.
  12. Sengupta, S., Jayaram, V., Curless, B., Seitz, S.M. and Kemelmacher-Shlizerman, I.(2020), Background matting: The world is your green screen, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2291-2300.
  13. Shen, X., Tao, X., Gao, H., Zhou, C. and Jia, J.(2016, October), Deep automatic portrait matting, In European conference on computer vision, pp. 92-107.
  14. Sun, J., Jia, J., Tang, C.K. and Shum, H.Y.(2004), Poisson matting, In ACM SIGGRAPH 2004 Papers, pp. 315-321.
  15. Sun, Y., Tang, C.K. and Tai, Y.W.(2021), Semantic image matting, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11120-11129.
  16. Xu, N., Price, B., Cohen, S. and Huang, T.(2017), Deep image matting, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2970-2979.
  17. Zhu, B., Chen, Y., Wang, J., Liu, S., Zhang, B. and Tang, M.(2017, October), Fast deep matting for portrait animation on mobile phone, In Proceedings of the 25th ACM international conference on Multimedia, pp. 297-305.