Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

Change Attention-based Vehicle Scratch Detection System

변화 주목 기반 차량 흠집 탐지 시스템

  • Lee, EunSeong (Department of Computer Engineering, Kwangwoon Uninversity) ;
  • Lee, DongJun (Department of Computer Engineering, Kwangwoon Uninversity) ;
  • Park, GunHee (Department of Electronic Engineering, Kwangwoon University) ;
  • Lee, Woo-Ju (Department of Computer Engineering, Kwangwoon Uninversity) ;
  • Sim, Donggyu (Department of Computer Engineering, Kwangwoon Uninversity) ;
  • Oh, Seoung-Jun (Department of Electronic Engineering, Kwangwoon University)
  • 이은성 (광운대학교 컴퓨터공학과) ;
  • 이동준 (광운대학교 컴퓨터공학과) ;
  • 박건희 (광운대학교 전자공학과) ;
  • 이우주 (광운대학교 컴퓨터공학과) ;
  • 심동규 (광운대학교 컴퓨터공학과) ;
  • 오승준 (광운대학교 전자공학과)
  • Received : 2022.02.08
  • Accepted : 2022.03.21
  • Published : 2022.03.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose an unmanned vehicle scratch detection deep learning model for car sharing services. Conventional scratch detection models consist of two steps: 1) a deep learning module for scratch detection of images before and after rental, 2) a manual matching process for finding newly generated scratches. In order to build a fully automatic scratch detection model, we propose a one-step unmanned scratch detection deep learning model. The proposed model is implemented by applying transfer learning and fine-tuning to the deep learning model that detects changes in satellite images. In the proposed car sharing service, specular reflection greatly affects the scratch detection performance since the brightness of the gloss-treated automobile surface is anisotropic and a non-expert user takes a picture with a general camera. In order to reduce detection errors caused by specular reflected light, we propose a preprocessing process for removing specular reflection components. For data taken by mobile phone cameras, the proposed system can provide high matching performance subjectively and objectively. The scores for change detection metrics such as precision, recall, F1, and kappa are 67.90%, 74.56%, 71.08%, and 70.18%, respectively.

본 논문에서는 카셰어링 서비스(car sharing service)에서 차량 상태 무인 검수를 위한 흠집 탐지 딥 러닝 모델을 제안한다. 기존의 차량 상태 검수 시스템은 대여 전, 후 사진에서 각각 흠집을 탐지하는 딥 러닝 모델과 탐지된 두 흠집 영상을 수작업으로 대조하여 새롭게 발생한 흠집을 탐색하는 두 단계로 구성되어 있다. 따라서 수동작업이 필요한 두 단계 모델을 한 단계로 줄이는 무인 흠집 탐지 모델을 위성영상에서 변화를 탐지하는 딥 러닝 모델에 전이 학습을 적용하여 구축한다. 그리고 광택 처리된 자동차 표면의 휘도가 비등방성이고 비전문가인 이용자가 일반 카메라로 촬영하기 때문에 정반사(specular reflection)가 흠집 탐지 성능에 크게 영향을 미친다. 따라서 정반사광으로 발생하는 오탐지를 감소시키기 위하여 정반사광 성분을 제거하는 전처리 과정을 적용한다. 이용자가 휴대폰 카메라로 촬영한 데이터에 대해 제안하는 시스템은 주관적인 측면과 정밀도(precision), 재현율(recall), F1, Kappa 척도면에서 각각 67.90%, 74.56%, 71.08%, 70.18%로서 높은 일치도를 보인다.

Keywords

Acknowledgement

이 논문은 2021년도 광운대학교 우수연구자 지원 사업에 의해 연구되었음.

References

  1. Gyeonggi Research Institute(GRI), Growing Pains of Car Sharing Services, Beyond Conflicts and Regulations to the Future Gyeonggi Research Institute(GRI) No.440, 2020 Nov. 15. https://www.dbpia.co.kr/journal/articleDetail?nodeId=NODE10541172
  2. Korea Consumer Agency, Survey Car sharing Terms of Use and Advertisements, 1-79, 2017 May. https://www.dbpia.co.kr/journal/articleDetail?nodeId=NODE07456356
  3. Developing Car Damage Detection Model using Semantic Segmentation. (accessed 2020. Feb. 13) doi: https://tech.socarcorp.kr/data/2020/02/13/car-damage-segmentation-model.html
  4. Q. Zhang, X. Chang and S. Bian Bian, "Vehicle-Damage-Detection Segmentation Algorithm Based on Improved Mask RCNN," IEEE Access, Vol.8, pp.6997-7004, Jan. 06, 2020. doi: https://doi.org/10.1109/ACCESS.2020.2964055
  5. U. Waqas, N. Akram, S. Kim, D. Lee and J. Jeo, "Vehicle Damage Classification and Fraudulent Image Detection Including Moire Effect Using Deep Learning", 2020 IEEE Canadian Conference on Electrical and Computer Engineering, Aug. 30 - Sep. 2 2020. (accessed Nov. 19, 2020) doi: https://doi.org/10.1109/ CCECE47787.2020.9255806
  6. X. Zhu, S. Liu, P. Zhang and Y. Duan, "A Unified Framework of Intelligent Vehicle Damage Assessment based on Computer Vision Technology," 2019 IEEE 2nd International Conference on Automation, Nov. 22-24 2019. doi: https://doi.org/10.1109/AUTEEE48671.2019.9033150
  7. H. S. Malik, M. Dwivedi, S. N. Omakar, S. R. Samal, A. Rathi, E. B. Monis, B. Khanna and A. Tiwari, "Deep Learning Based Car Damage Classification and Detection," Advances in Artificial Intelligence and Data Engineering, Vol.1133, pp 207-221, Aug 2020. https://easychair.org/publications/preprint/RlQ2
  8. A. Artusi, F. Banterle, and D. Chetverikov, "A Survey of Specularity Removal Methods," COMPUTER GRAPHICS forum, Vol.30, No.8, pp. 2208-2230, 2011. doi: https://doi.org/10.1111/j.1467-8659.2011.01971.x
  9. C. G. P. Suescun, J. O. P. Arenas and R. J. Moreno, "Detection of Scratches on Cars by Means of CNN and R-CNN," International Journal on Advanced Science, Engineering and Information Technology, Vol.9, No.3, pp.745-752, 2019. doi: http://dx.doi.org/10.18517/ijaseit.9.3.6470
  10. R. C. Daudt, R. L. Saux, and A. Boulch, "Fully convolutional siamese networks for change detection," Proceedings of the 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 4063-4067, 2018. (accessed Sep. 6 2018) doi: 10.1109/ICIP.2018.8451652
  11. M. E. A. Larabi, S. Chaib, K. Bakhtj, and M. S. Karoui, "Transfer Learning for Changes Detection in Optical Remote Sensing Imagery," Proceedings of the IGARSS 2019-2019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan, pp.1582-1585, 2019. doi: https://doi.org/10.1109/IGARSS.2019.8900296
  12. H. Chen, and S. Zhenwei, "A Spatial-Temporal Attention-Based Method and a New Dataset for Remote Sensing Image Change Detection," Remote Sensing, Vol.12, No.10, pp.1-23, May 2020. doi: https://doi.org/10.3390/rs12101662
  13. G. Hwang, W.J. Lee, and S.J. Oh, "Change Attention based Dense Siamese Network for Remote Sensing Change Detection," Journal of Broadcast Engineering, Vol.26, No.1, pp.14-25, Jan 2021. doi: https://doi.org/10.5909/JBE.2021.26.1.14
  14. R. T. Collins, A. J. Lipton and T. Kanade, "Introduction to the special section on video surveillance," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, No. 8, pp. 745-746, Aug 2000. doi: https://doi.org/10.1109/TPAMI.2000.868676
  15. C. Stauffer and W. E. L. Grimson, "Learning patterns of activity using real-time tracking," in IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, No. 8, pp. 747-757, Aug 2000. doi: https://doi.org/10.1109/34.868677
  16. M. Bosc, F. Heitz, J. P. Armspach, I. Namer, D. Gounot and L. Rumbach, "Automatic change detection in multimodal serial MRI: Application to multiple sclerosis lesion evolution," Neuroimage, Vol. 20, pp. 643-656, 2003. doi: https://doi.org/10.1016/s1053-8119(03)00406-3
  17. D. Rey, G. Subsol, H. Delingette and N. Ayache, "Automatic detection and segmentation of evolving processes in 3D medical images: Application to multiple sclerosis," Med. Image Anal, Vol.6, No.2, pp.163-179, Jun 2002. doi: https://doi.org/10.1016/s1361-8415(02)00056-7
  18. D. R. Edgington, K. A. Salamy, M. Risi, R. E. Sherlock, D. Walther and C. Koch, "Automated event detection in underwater video," Oceans 2003. Celebrating the Past ... Teaming Toward the Future (IEEE Cat. No.03CH37492), Vol.5, pp. 2749-2753, 2003. doi: https://doi.org/10.1109/OCEANS.2003.178344
  19. K. Lebart, E. Trucco and D. M. Lane, "Real-time automatic sea-floor change detection from video," Oceans Conference Record, Vol. 2, No. 7, pp. 1337-1343, 2000. doi: https://doi.org/10.1109/OCEANS.2000.881789
  20. L. Khelifi and M. Mignotte, "Deep Learning for Change Detection in Remote Sensing Images: Comprehensive Review and Meta-Analysis," IEEE Access, Vol. 8, pp. 126385 -126400. (accessed Jul. 8 2020) doi: https://doi.org/10.1109/ACCESS.2020.3008036
  21. Y. Liu, C. Pang, Z. Zhan, X. Zhang, and X. Yang, "Building Change Detection for Remote Sensing Images Using a Dual Task Constrained Deep Siamese Convolutional Network Model," arXiv preprint, arXiv:1909.07726, 2019. doi: https://doi.org/10.48550/arXiv.1909.07726
  22. Y. Zhan, K. Fu, M. Yan, X. Sun, H. Wang and X. Qiu, "Change Detection Based on Deep Siamese Convolutional Network for Optical Aerial Images," IEEE Geoscience and Remote Sensing Letters, Vol.14, Issue: 10, pp.1845-1849, Oct. 2017. doi: https://doi.org/10.1109/LGRS.2017.2738149
  23. S. A. Shafer, "Using color to separate reflection components", Color Res. Appl. 10, 210-218, 1985. doi: https://doi.org/10.1002/col.5080100409
  24. K. Yoon, Y. Choi and I.S. Kweon,"Fast Separation of Reflection Components using a Specularity-Invariant Image Representation," 2006 International Conference on Image Processing, Atlanta, Georgia, USA, pp. 973-97, Oct 2006. doi: https://doi.org/10.1109/ICIP.2006.312650
  25. S. K. Nayar, X. S. Fang, and T. Boult, "Separation of reflection components using color and polarization," International Journal of Computer Vision, Vol. 21, pp.163-186, Feb 1997. doi: https://doi.org/10.1023/A:1007937815113
  26. R. T. Tan, K. Nishino, and K. Ikeuchi, "Separating reflection components based on chromaticity and noise analysis," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 26, pp.1373-1379, Oct 2004. doi: https://doi.org/10.1109/TPAMI.2004.90
  27. R. T. Tan and K. Ikeuchi, "Separating reflection components of textured surfaces using a single image," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, Iss.2, pp. 178 -193, Feb 2005. doi: https://doi.org/10.1109/TPAMI.2005.36
  28. H.L. Shen and Q.Y. Cai, "Simple and efficient method for specularity removal in an image," Virtual Journal for Biomedical Optics, Vol. 4, Iss.7, May. 6 2009. doi: https://doi.org/10.1364/AO.48.002711
  29. J. Fu, J. Liu, H. Tian, Y. Li, Y. Bao, Z. Fang, and H. Lu, "Dual attention network for scene segmentation." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, California, USA, pp.3146-3154, 2019. doi: https://doi.org/10.48550/arXiv.1809.02983
  30. C. Tan, F. Sun, T. Kong, W. Zhang, C. Yang and C. Liu, "A Survey on Deep Transfer Learning," Artificial Neural Networks and Machine Learning, pp. 270-279. (accessed Oct 2018) doi:10.1007/978-3-030-01424-7_27
  31. S. J. Pan and Q. Yang, "A Survey on Transfer Learning," IEEE Transactions on Knowledge and Data Engineering, Vol. 22, Iss.10, pp. 1345-1359, Oct. 2010. doi: https://doi.org/10.1109/TKDE.2009.191
  32. X. Xu, J. Lin, Y. Tao and X. Wang, "An Improved DenseNet Method Based on Transfer Learning for Fundus Medical Images," 2018 7th International Conference on Digital Home (ICDH), pp. 137-140, Nov. 30 -Dec. 1 2018. doi: https://doi.org/10.1109/ICDH.2018.00033
  33. J. Yosinski, J. Clune, Y. Bengio and H. Lipson, "How transferable are features in deep neural networks?" Advances in Neural Information Processing Systems, Vol.27, pp.3320-3328, Dec. 2014. doi: https://doi.org/10.48550/arXiv.1411.1792
  34. A. Asokan, and J. Anitha, "Change detection techniques for remote sensing applications: a survey." Earth Science Informatics, Vol.12, No.2, pp.143-160, March 2019. doi: https://doi.org/10.1007/s12145-019-00380-5
  35. L. Richard, and G. G. Koch. "The Measurement of Observer Agreement for Categorical Data." Wiley, International Biometric Society, Vol. 33, No. 1, pp. 159-74, 1977. doi: https://doi.org/10.2307/2529310
  36. S. Wang, D. Quan, X. Liang, M. Ning, Y. Guo and L. Jiao, "A deep learning framework for remote sensing image registration". ISPRS Journal of Photogrammetry and Remote Sensing, Vol.145, pp.148-164, 2018. doi: http://dx.doi.org/10.1016/j.isprsjprs.2017.12.012
  37. F. Wang and D.M.J. Tax. "Survey on the attention based RNN model and its applications in computer vision." arXiv preprint arXiv: 1601.06823, 2016. doi: https://doi.org/10.48550/arXiv.1601.06823