Smart Media Journal (스마트미디어저널)

THE KOREAN INSTITUTE OF SMART MEDIA (한국스마트미디어학회)
권호 표지
DOI QR코드

DOI QR Code

Few-shot learning using the median prototype of the support set

Support set의 중앙값 prototype을 활용한 few-shot 학습

  • 백으뜸 (호남대학교 AI빅데이터학과)
  • Received : 2022.10.26
  • Accepted : 2023.02.24
  • Published : 2023.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

Meta-learning is metacognition that instantly distinguishes between knowing and unknown. It is a learning method that adapts and solves new problems by self-learning with a small amount of data.A few-shot learning method is a type of meta-learning method that accurately predicts query data even with a very small support set. In this study, we propose a method to solve the limitations of the prototype created with the mean-point vector of each class. For this purpose, we use the few-shot learning method that created the prototype used in the few-shot learning method as the median prototype. For quantitative evaluation, a handwriting recognition dataset and mini-Imagenet dataset were used and compared with the existing method. Through the experimental results, it was confirmed that the performance was improved compared to the existing method.

메타 학습(meta learning)이란 즉각적으로 아는 것과 모르는 것을 구별하는 메타 인지로 적은 양의 데이터로 스스로 학습하고, 학습한 정보와 알고리즘으로 새로운 문제에 적응하며 해결하는 학습 방식이다. 그 중, few-shot 학습 방법은 메타 학습 방법의 한 종류로 매우 적은 학습 데이터 (support set)으로도 질의 데이터(query set)를 올바르게 예측하도록 하는 학습 방법이다. 본 연구에서는 각 클래스의 mean-point vector로 생성한 프로토타입의 한계점인 높은 밀도값을 낮추면서 이상치(outlier)값을 극복하는 방법을 제안한다. 제안한 방법은 기존의 방법을 해결하기 위해, 딥러닝 모델에서 feature를 추출하고, 획득한 feature사이의 요소별로 중앙값 계산하여 프로토타입을 생성하는 방법을 사용한다. 그 후, 앞서 생성한 중앙값 프로토타입을 기반으로 few-shot 학습 방법에 사용한다. 제안한 방법의 정량적인 평가를 위해 필체 인식 데이터셋을 사용하여 기존의 방법과 비교하였다. 실험 결과를 통해 기존의 방법보다 향상된 성능을 내는 것을 확인할 수 있었다.

Keywords

Acknowledgement

이 논문은 2022년도 호남대학교 학술연구비 지원을 받아 연구되었음

References

  1. F. Li, et al. "One-shot learning of object categories," IEEE transactions on pattern analysis and machine intelligence, Vol. 28, No. 4, pp. 594-611, 2006.  https://doi.org/10.1109/TPAMI.2006.79
  2. J. Schmidhuber, "Evolutionary principles in self-referential learning, or on learning how to learn: the meta-meta-... hook," Diploma Thesis, Tech. Univ. Munich, 1987. 
  3. W. Lilian, "Meta-Learning: Learning to Learn Fast," OpenAI. 2019. 
  4. S. Flood, et al. "Learning to compare: Re lation network for few-shot learning," IEEE Conference on Computer Vision and Pattern Recognition, pp. 1199-1208, 2018. 
  5. S. Qianru et. al. "Meta-transfer learning for few-shot learning," IEEE Conference on Computer Vision and Pattern Recognition, pp. 403-412, 2019. 
  6. S. Gidaris, et. al. "Dynamic few-shot visual learning without forgetting," IEEE Conference on Computer Vision and Pattern Recognition, pp. 4367-4375, 2018. 
  7. X. Wang, et. al. "Tafe-net: Task-aware feature embed dings for low shot learning," IEEE Conference on Computer Vision and Pattern Recognition, pp. 1831-1840, 2019. 
  8. C. Finn, et. al. "Model agnostic meta-learning for fast adaptation of deep networks," International Conference on Machine Learning-, Vol. 70, pp. 1126-1135. 2017. 
  9. D. Park, et. al "Host-Based Intrusion Detection Model Using Few-Shot Learning," KIPS Transactions on Software and Data Engineering, Vol. 10, No. 7, pp. 271-278, 2021.  https://doi.org/10.3745/KTSDE.2021.10.7.271
  10. S. Ravi et al. "Optimization as a model for few-shot learning," Learning Representations (ICLR), 2017 
  11. T. Munkhdalai, et. al. "Rapid adaptation with conditionally shifted neurons," arXiv preprint arXiv:1712.09926, 2017 
  12. E. Grant, et. al. "Recasting gradient-based meta-learning as hierarchical bayes," arXiv preprint arXiv:1801.08930, 2018. 
  13. A. Andrei. "Meta-learning with latent embedding optimization," Learning Representations, 2019. 
  14. F. Chelsea "Model agnostic meta-learning for fast adaptation of deep networks," International Conference on Machine Learning-, Vol. 70, pp. 1126-1135. 2017. 
  15. O. Vinyals, et al. "Matching networks for one shot learning," Advances in neural information processing systems, pp. 3630-3638, 2016. 
  16. J. Snell, et. al. "Prototypical networks for few-shot learning," 2017. 
  17. C. Davide, "Siamese neural networks: an overview," Artificial Neural Networks, Methods in Molecular Biology, Vol. 2190 (3rd ed.), Springer Protocols, Humana Press, pp. 73-94, New York, USA, 2020. 
  18. B. Jane, et. al."Signature verification using a "Siamese" time delay neural network," Advances in Neural Information Processing Systems 6, pp. 737-744. 1994 
  19. G. Chechik, et. al. "Large Scale Online Learning of Image Similarity Through Ranking," Journal of Machine Learning Research. Vol. 11, pp. 1109-1135. 2010.  https://doi.org/10.1007/978-3-642-02172-5_2
  20. S. Hsiao, "Malware image classification using one-shot learning with Siamese networks," Procedia Computer Science, Vol. 159, pp. 1863-1871, 2019.  https://doi.org/10.1016/j.procs.2019.09.358
  21. Y. Taigman, et al. "Deep face: Closing the gap to human-level performance in face verification," IEEE Conference on Computer Vision and Pattern Recognition, 2014. 
  22. S. Ravi et al. "Optimization as a model for few-shot learning," Learning Representations (ICLR), 2017 
  23. W. Jeremy, "Spring Research Presentation: A Theoretical Foundation for Inductive Transfer," Brigham Young University, College of Physical and Mathematical Sciences. 2007. 
  24. H. Kaiming, et. al. "Rethinking imagenet pre-training," IEEE/CVF International Conference on Computer Vision. 2019. 
  25. H. Kaiming, et al. "Deep residual learning for image recognition," IEEE conference on computer vision and pattern recognition. 2016. 
  26. V. Oriol, et al. "Matching networks for one shot learning," Advances in neural information processing systems, Vol. 29, 2016. 
  27. B. Lake, "Human-level concept learning through probabilistic program induction," Science, 350(6266), pp.1332-1338. 2015.  https://doi.org/10.1126/science.aab3050
  28. B. Lake, "Human-level concept learning through probabilistic program induction," Science, 350(6266), pp. 1332-1338, 2015.  https://doi.org/10.1126/science.aab3050
  29. I. J. Good, "Some Terminology and Notation in Information Theory," Proceedings of the IEEE - Part C: Monographs, Vol. 103, No. 3, pp. 200-204, Mar. 1956.  https://doi.org/10.1049/pi-c.1956.0024
  30. D. Kingma, et. al "Adam: A method for stochastic optimization," arXiv preprint arXiv:1412.6980, 2014.