Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

Deep Learning based Domain Adaptation: A Survey

딥러닝 기반의 도메인 적응 기술: 서베이

  • Na, Jaemin (Dept. of Artificial Intelligence, Ajou University) ;
  • Hwang, Wonjun (Dept. of Artificial Intelligence, Ajou University)
  • Received : 2022.05.17
  • Accepted : 2022.07.26
  • Published : 2022.07.30
Download PDF
⟨ Previous Next ⟩

Abstract

Supervised learning based on deep learning has made a leap forward in various application fields. However, many supervised learning methods work under the common assumption that training and test data are extracted from the same distribution. If it deviates from this constraint, the deep learning network trained in the training domain is highly likely to deteriorate rapidly in the test domain due to the distribution difference between domains. Domain adaptation is a methodology of transfer learning that trains a deep learning network to make successful inferences in a label-poor test domain (i.e., target domain) based on learned knowledge of a labeled-rich training domain (i.e., source domain). In particular, the unsupervised domain adaptation technique deals with the domain adaptation problem by assuming that only image data without labels in the target domain can be accessed. In this paper, we explore the unsupervised domain adaptation techniques.

딥러닝 기반의 지도학습은 다양한 응용 분야에서 비약적인 발전을 이루었다. 그러나 많은 지도 학습 방법들은 학습 및 테스트 데이터가 동일한 분포에서 추출된다는 공통된 가정 하에 이루어진다. 이 제약 조건에서 벗어나는 경우, 학습 도메인에서 훈련된 딥러닝 네트워크는 도메인 간의 분포 차이로 인하여 테스트 도메인에서의 성능이 급격하게 저하될 가능성이 높다. 도메인 적응 기술은 레이블이 풍부한 학습 도메인 (소스 도메인)의 학습된 지식을 기반으로 레이블이 불충분한 테스트 도메인 (타겟 도메인) 에서 성공적인 추론을 할 수 있도록 딥러닝 네트워크를 훈련하는 전이 학습의 한 방법론이다. 특히 비지도 도메인 적응 기술은 타겟 도메인에 레이블이 전혀 없는 이미지 데이터에만 접근할 수 있는 상황을 가정하여 도메인 적응 문제를 다룬다. 본 논문에서는 이러한 비지도 학습 기반의 도메인 적응 기술들에 대해 탐구한다.

Keywords

Acknowledgement

본 연구는 2022년도 과학기술정보통신부 및 정보통신기획평가원의 대학ICT연구센터육성지원사업 (IITP-2022-2018-0-01431), 2021년도 과학기술정보통신부의 재원으로 정보통신기획평가원 (No.2021-0-00951, (세부2)클라우드 기반 자율주행 AI 학습 SW 개발)과 교육부 및 한국연구재단의 4단계 두뇌한국21 사업(4단계 BK21 사업)으로 지원된 연구임.

References

  1. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative adversarial nets" Advances in neural information processing systems, Vol.27, 2014.
  2. Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and V. Lempitsky, "Domain-Adversarial Training of Neural Networks" The journal of machine learning research, Vol.17, 2016.
  3. E. Tzeng, J. Hoffman, J. Saenko, and C. Chen, "Adversarial discriminative domain adaptation" Proceedings of the IEEE conference on computer vision and pattern recognition, 2017. doi: https://doi.org/10.1109/cvpr.2017.316
  4. S. Xie, Z. Zheng, L. Chen, and C. Chen, "Learning semantic representations for unsupervised domain adaptation" International conference on machine learning. PMLR, 2018.
  5. J. Na, H. Jung, H. Chang, and W. Hwang, "FixBi: Bridging Domain Spaces for Unsupervised Domain Adaptation" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021. doi: https://doi.org/10.1109/cvpr46437.2021.00115
  6. J. Na, D. Han, H. Chang, and W. Hwang, "Contrastive Vicinal Space for Unsupervised Domain Adaptation" arXiv preprint arXiv:2111.13353, 2021.
  7. X. Gu, J. Sun, and Z. Xu. "Spherical space domain adaptation with robust pseudo-label loss" In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9101-9110, 2020. doi: https://doi.org/10.1109/cvpr42600.2020.00912
  8. K. Saito, Y. Ushiku, and T. Harada. Asymmetric tri-training for unsupervised domain adaptation. International Conference on Machine Learning, pages 2988-2997. PMLR, 2017.
  9. G. Kang, L. Jiang, Y. Yang, and A. G. Hauptmann. Contrastive adaptation network for unsupervised domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4893-4902, 944 2019. doi: https://doi.org/10.1109/cvpr.2019.00503
  10. Mengxue Li, Yi Ming Zhai, You Wei Luo, Peng Fei Ge, and Chuan Xian Ren. Enhanced transport distance for unsupervised domain adaptation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13936-13944, 2020. doi: https://doi.org/10.1109/cvpr42600.2020.01395
  11. Shimodaira Hidetoshi. Improving predictive inference under covariate shift by weightingthe log-likelihood function. Journal of statistical planning and inference 90(2),227-244 (2000). https://doi.org/10.1016/S0378-3758(00)00115-4
  12. Xiao, N., Zhang, L.: Dynamic weighted learning for unsupervised domain adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision andPattern Recognition. pp. 15242-15251 (2021) doi:https://doi.org/10.1109/cvpr46437.2021.01499
  13. Yue, Z., Sun, Q., Hua, X.S., Zhang, H.: Transporting causal mechanisms for un-supervised domain adaptation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 8599-8608 (2021) doi: https://doi.org/10.1109/iccv48922.2021.00848
  14. Zhang, H., Cisse, M., Dauphin, Y.N., Lopez-Paz, D.: mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412 (2017)
  15. Saenko, K., Kulis, B., Fritz, M., Darrell, T.: Adapting visual category models to new domains. In: European conference on computer vision. pp. 213-226. Springer(2010) doi: https://doi.org/10.1007/978-3-642-15561-1_16
  16. Venkateswara, H., Eusebio, J., Chakraborty, S., Panchanathan, S.: Deep hashing network for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 5018-5027 (2017) doi: https://doi.org/10.1109/cvpr.2017.572
  17. Lee, D.H., et al.: Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In: Workshop on challenges in representation learning, ICML. p. 896 (2013)
  18. H. Pham, Z. Dai, Q. Xie, and Q. V. Le. Meta pseudo labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11557-11568, 2021. doi: https://doi.org/10.1109/cvpr46437.2021.01139
  19. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 770-778 (2016) doi: https://doi.org/10.1109/cvpr.2016.90
  20. He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: European conference on computer vision. pp. 630-645. Springer (2016) doi: https://doi.org/10.1007/978-3-319-46493-0_38