산업경영시스템학회지 (Journal of Korean Society of Industrial and Systems Engineering)

  • 제46권1호
  • /
  • Pages.55-67
  • /
  • 2023
  • /
  • 2005-0461(pISSN)
  • /
  • 2287-7975(eISSN)
한국산업경영시스템학회 (Society of Korea Industrial and System Engineering)
권호 표지
DOI QR코드

DOI QR Code

얼굴 인식을 위한 경량 인공 신경망 연구 조사

A Comprehensive Survey of Lightweight Neural Networks for Face Recognition

  • 장영립 (전북대학교 산업정보시스템공학과) ;
  • 양재경 (전북대학교 산업정보시스템공학과)
  • Yongli Zhang (Department of Industrial and Information Systems Engineering, Jeonbuk National University) ;
  • Jaekyung Yang (Department of Industrial and Information Systems Engineering, Jeonbuk National University)
  • 투고 : 2023.02.15
  • 심사 : 2023.03.09
  • 발행 : 2023.03.31
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

Lightweight face recognition models, as one of the most popular and long-standing topics in the field of computer vision, has achieved vigorous development and has been widely used in many real-world applications due to fewer number of parameters, lower floating-point operations, and smaller model size. However, few surveys reviewed lightweight models and reimplemented these lightweight models by using the same calculating resource and training dataset. In this survey article, we present a comprehensive review about the recent research advances on the end-to-end efficient lightweight face recognition models and reimplement several of the most popular models. To start with, we introduce the overview of face recognition with lightweight models. Then, based on the construction of models, we categorize the lightweight models into: (1) artificially designing lightweight FR models, (2) pruned models to face recognition, (3) efficient automatic neural network architecture design based on neural architecture searching, (4) Knowledge distillation and (5) low-rank decomposition. As an example, we also introduce the SqueezeFaceNet and EfficientFaceNet by pruning SqueezeNet and EfficientNet. Additionally, we reimplement and present a detailed performance comparison of different lightweight models on the nine different test benchmarks. At last, the challenges and future works are provided. There are three main contributions in our survey: firstly, the categorized lightweight models can be conveniently identified so that we can explore new lightweight models for face recognition; secondly, the comprehensive performance comparisons are carried out so that ones can choose models when a state-of-the-art end-to-end face recognition system is deployed on mobile devices; thirdly, the challenges and future trends are stated to inspire our future works.

키워드

참고문헌

  1. Abate, A.F., Nappi, M., Riccio, D., and Sabatino, G., 2D and 3D face recognition: A survey, Pattern Recognition Letters, 2007,Vol.28,No.14,pp.1885-1906. https://doi.org/10.1016/j.patrec.2006.12.018
  2. Adjabi, I., Ouahabi, A., Benzaoui, A., and Taleb-Ahmed, A., Past, present, and future of face recognition: A review, Electronics, 2020, Vol. 9, No. 8, pp. 1188.
  3. Alonso-Fernandez, F., Barrachina, J., Hernandez-Diaz, K., and Bigun, J., SqueezeFacePoseNet: Lightweight Face Verification Across Different Poses for Mobile Platforms, International Conference on Pattern Recognition, 2021, pp. 139-153.
  4. Belhumeur, P.N., Hespanha, J.P., and Kriegman, D.J., Eigenfaces vs. fisherfaces: Recognition using class specific linear projection, IEEE Transactions on Pattern Analysis and Machine Intelligence, 1997, Vol, 19, No. 7, pp. 711-720. https://doi.org/10.1109/34.598228
  5. Boutros F., Damer, N., and Kuijper, A., QuantFace: Towards lightweight face recognition by synthetic data low-bit quantization, arXiv preprint arXiv:2206.10526, 2022, pp. 855-862.
  6. Boutros, F., Damer, N., Fang, M., Kirchbuchner, F., and Kuijper, A., Mixfacenets: Extremely efficient face recognition networks, 2021 IEEE International Joint Conference on Biometrics (IJCB), 2021, pp. 1-8.
  7. Boutros, F., Siebke, P., Klemt, M., Damer, N., Kirchbuchner, F., and Kuijper, A., PocketNet: extreme lightweight face recognition network using neural architecture search and multistep knowledge distillation, IEEEAccess, 2022, pp. 46823-46833.
  8. Cai, H., Zhu, L., and Han, S., Proxylessnas: Direct neural architecture search on target task and hardware, arXiv preprint arXiv:1812.00332, 2018.
  9. Cao, Q., Shen, L., Xie, W., Parkhi, O. M., and Zisserman, A., Vggface2: A dataset for recognising faces across pose and age, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), 2018.
  10. Cao, Z., Yin, Q., Tang, X., and Sun, J., Face recognition with learning-based descriptor, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2010, pp. 2707-2714.
  11. Chen S., Liu, Y., Gao, X., and Han, Z., Mobilefacenets: Efficient cnns for accurate real-time face verification on mobile devices, Chinese Conference on Biometric Recognition, 2018, pp. 428-438.
  12. Chen, K., Taihe, Y., and Qi, L., LightQNet: Lightweight Deep Face Quality Assessment for Risk-Controlled Face Recognition, IEEE Signal Processing Letters, 2021, Vol. 28, pp. 1878-1882. https://doi.org/10.1109/LSP.2021.3109781
  13. Deng, J., Dong, W., Socher, R., Li, L. J., Li, K., and Fei-Fei, L., Imagenet: A large-scale hierarchical image database, 2009 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2009, pp. 248-255.
  14. Deng, J., Guo, J., Xue, N., and Zafeiriou, S., Arcface: Additive angular margin loss for deep face recognition, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, 4690-4699.
  15. Deng, J., Guo, J., Zhang, D., Deng, Y., Lu, X., and Shi, S., Lightweight face recognition challenge, Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops. 2019.
  16. Deng, W., Hu, J., and Guo, J., Extended SRC: Undersampled face recognition via intraclass variant dictionary, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, Vol. 34, No. 9, pp. 1864-1870. https://doi.org/10.1109/TPAMI.2012.30
  17. Du, H., Shi, H., Zeng, D., Zhang, X.P., and Mei, T., The elements of end-to-end deep face recognition: A survey of recent advances, ACM Computing Surveys (CSUR), 2022, Vol.54, No.10s, pp. 1-42. https://doi.org/10.1145/3507902
  18. Duong, C.N., Luu, K., Quach, K.G., and Le, N., Shrinkteanet: Million-scale lightweight face recognition via shrinking teacher-student networks, arXiv preprint arXiv:1905.10620, 2019.
  19. Duong, C.N., Quach, K.G., Jalata, I., Le, N., and Luu, K., Mobiface: A lightweight deep learning face recognition on mobile devices, 2019 IEEE 10th International Conference on Biometrics Theory, Applications and Systems (BTAS), 2019, pp. 1-6.
  20. Feng, Y., Wang, H., Hu, H.R., Yu, L., Wang, W., and Wang, S., Triplet distillation for deep face recognition, 2020 IEEE International Conference on Image Processing (ICIP), 2020, pp. 808-812.
  21. Fu, C., Zhou, X., He, W., and He, R., Towards Lightweight Pixel-Wise Hallucination for Heterogeneous Face Recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
  22. Ge, S., Zhao, S., Li, C., and Li, J., Low-resolution face recognition in the wild via selective knowledge distillation, IEEE Transactions on Image Processing, 2018, Vol.28, No. 4, pp. 2051-2062. https://doi.org/10.1109/TIP.2018.2883743
  23. Ge, S., Zhao, S., Li, C., Zhang, Y., and Li, J., Efficient low-resolution face recognition via bridge distillation, IEEE Transactions on Image Processing, 2020, Vol.29, pp. 6898-6908. https://doi.org/10.1109/tip.2020.2995049
  24. Guo, L., Bai, H., and Zhao, Y., A lightweight and robust face recognition network on noisy condition, 2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), 2019, pp. 1964-1969.
  25. Guo, Y., Zhang, L., Hu, Y., He, X., and Gao, J., Ms-celeb- 1m: A dataset and benchmark for large-scale face recognition, European Conference on Computer Vision, 2016, pp. 87-102.
  26. He, K., Zhang, X., Ren, S., and Sun, J., Deep residual learning for image recognition, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770-778.
  27. He, X., Yan, S., Hu, Y., Niyogi, P., and Zhang, H.J., Face recognition using laplacianfaces, IEEE transactions on Pattern Analysis and Machine Intelligence, 2005, Vol. 27, No. 3, pp. 328-340. https://doi.org/10.1109/TPAMI.2005.55
  28. Hinton, G., Vinyals, O., and Dean, J., Distilling the knowledge in a neural network, arXiv preprint arXiv:1503.02531 2.7, 2015.
  29. Hoo, S.C., Ibrahim, H., and Suandi, S.A., ConvFaceNeXt: Lightweight Networks for Face Recognition, Mathematics, 2022, Vol. 10, No. 19.
  30. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H., Mobilenets: Efficient convolutional neural networks for mobile vision applications, arXiv preprint arXiv:1704.04861, 2017.
  31. Huang, G., Liu, Z., VanDer Maaten, L., and Weinberger, K.Q., Densely connected convolutional networks, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017, pp. 4700-4708.
  32. Huang, G.B., Mattar, M., Berg, T., and Learned-Miller, E., Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. Workshop on faces in'Real-Life'Images: detection, alignment, and recognition, 2008.
  33. Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., and Keutzer, K., SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size, arXiv preprint arXiv:1602.07360, 2016
  34. Jafri, R. and Arabnia, H.R., A survey of face recognition techniques, Journal of Information Processing Systems, 2009, Vol.5, No.2, pp. 41-68. https://doi.org/10.3745/JIPS.2009.5.2.041
  35. Kim, M., Jain, A.K., and Liu, X., AdaFace: Quality Adaptive Margin for Face Recognition, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 18750-18759.
  36. Krizhevsky, A., Sutskever, I., and Hinton, G. E., Imagenet classification with deep convolutional neural networks, Communications of the ACM, 2017, Vol. 60, No. 6, pp. 84-90. https://doi.org/10.1145/3065386
  37. LeCun, Y., Boser, B., Denker, J., Henderson, D., Howard, R., Hubbard, W., and Jackel, L., Handwritten digit recognition with a back-propagation network, Advances in Neural Information Processing Systems, 1989, Vol. 2,pp. 396-404.
  38. Lei, Z., Pietikainen, M., and Li, S.Z., Learning discriminant face descriptor, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, Vol. 36, No. 2, pp. 289-302. https://doi.org/10.1109/TPAMI.2013.112
  39. Li, X., Wang, F., Hu, Q., and Leng, C., Airface: Lightweight and efficient model for face recognition, Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, 2019.
  40. Li, Z., Xi, T., Deng, J., Zhang, G., Wen, S., and He, R., Gp-nas: Gaussian process based neural architecture search, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 11933-11942.
  41. Lin, M., Chen, Q., and Yan, S., Network in network, arXiv preprint arXiv:1312.4400, 2013.
  42. Liu, C. and Wechsler, H., Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition, IEEE Transactions on Image processing, 2002, Vol. 11, No. 4, pp. 467-476. https://doi.org/10.1109/TIP.2002.999679
  43. Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., and Song, L., Sphereface: Deep hypersphere embedding for face recognition, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 212-220.
  44. Liu, W., Zhou, L., and Chen, J., Face recognition based on lightweight convolutional neural networks, Information 2021, Vol. 12, No. 5, pp. 191.
  45. Lyu, Y., Jiang, J., and Zhang, K., Hua, Y., Cheng, M., Factorizing and reconstituting large-kernel MBConv for lightweight face recognition, Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, 2019.
  46. Ma, N., Zhang, X., Zheng, H.T., and Sun, J., Shufflenet v2: Practical guidelines for efficient cnn architecture design, Proceedings of the European conference on computer vision (ECCV), 2018, pp. 116-131.
  47. Ma, Y., Effective methods for lightweight image-based and video-based face recognition, Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, 2019.
  48. Martindez-Diaz, Y., Luevano, L.S., Mendez-Vazquez, H., Nicolas-Diaz, M., Chang, L., and Gonzalez-Mendoza, M., Shufflefacenet: A lightweight face architecture for efficient and highly-accurate face recognition, Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, 2019.
  49. Martinez-Diaz, Y., Nicolas-Diaz, M., Mendez-Vazquez, H., Luevano, L.S., Chang, L., Gonzalez-Mendoza, M., and Sucar, L.E., Benchmarking lightweight face architectures on specific face recognition scenarios, Artificial Intelligence Review, 2021, pp. 1-44.
  50. Masi, I., Wu, Y., Hassner, T., and Natarajan, P., Deep face recognition: A survey, 2018 31st SIBGRAPI Conference on Graphics, Patterns and Images, 2018, pp. 471-478.
  51. Maze, B., Adams, J., Duncan, J.A., Kalka, N., Miller, T., Otto, C., Jain, A.K., Niggel, W.T., Anderson, J., Cheney, J., and Grother, P., Iarpa janus benchmark-c: Face dataset and protocol, 2018 International Conference on Biometrics, 2018, pp 158-165.
  52. Micikevicius P., Narang, S., Alben, J., Diamos, G., Elsen, E., Garcia, D., Ginsburg, B., Houston, M., Kuchaiev, O., Venkatesh, G., and Wu, H., Mixed precision training, arXiv preprint arXiv:1710.03740, 2017.
  53. Moghaddam, B., Jebara, T., and Pentland, A., Bayesian face recognition, Pattern recognition, 2000, Vol. 33, No. 11, pp. 1771-1782. https://doi.org/10.1016/S0031-3203(99)00179-X
  54. Moschoglou, S., Papaioannou, A., Sagonas, C., Deng, J., Kotsia, I., and Zafeiriou, S., Agedb: the first manually collected, in-thewild age database, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops, Honolulu, HI, USA, 2017, pp. 51-59.
  55. Parkhi, O. M., Vedaldi, A., and Zisserman, A., Deep face recognition, 2015.
  56. Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., and Lerer, A., Automatic differentiation in pytorch, NeurIPS workshop 2017.
  57. Romero, A., Ballas, N., Kahou, S.E., Chassang, A., Gatta, C., and Bengio, Y., Fitnets: Hints for thin deep nets, arXiv preprint arXiv:1412.6550, 2014.
  58. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., KArpathy, A., Khosla, A., Bernstein, M., and Fei-Fei, L., Imagenet large scale visual recognition challenge, International Journal of Computer Vision, 2015, Vol. 115, pp. 211-252. https://doi.org/10.1007/s11263-015-0816-y
  59. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.C., Mobilenetv2:Inverted residuals and linear bottlenecks, Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 4510-4520.
  60. Schroff, F., Kalenichenko, D., and Philbin, J., Facenet: A unified embedding for face recognition and clustering, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015.
  61. Sengupta, S., Chen, J.C., Castillo, C., Patel, V.M., Chellappa, R., and Jacobs, D.W., Frontal to profile face verification in the wild, 2016 IEEE Winter Conference on Applications of Computer Vision(WACV), Lake Placid, NY, USA, 2016, pp.1-9.
  62. Simonyan, K. and Zisserman, A., Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv:1409.1556, 2014.
  63. Sun, , Y., Wang, X., and Tang, X., Deeply learned face representations are sparse, selective, and robust, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 2892-2900.
  64. Sun, Y., Chen, Y., Wang, X., and Tang, X., Deep learning face representation by joint identification-verification, Advances in Neural Information Processing Systems, Vol. 27, 2014.
  65. Sun, Y., Liang, D., Wang, X., and Tang, X., Deepid3: Face recognition with very deep neural networks, arXiv preprint arXiv:1502.00873, 2015.
  66. Sun, Y., Wang, X., and Tang, X., Deep learning face representation from predicting10,000 classes, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp 1891-1898.
  67. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A., Going deeper with convolutions, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp.1-9.
  68. Taigman, Y., Yang, M., Ranzato, M. A., and Wolf, L., Deepface: Closing the gap to human-level performance in face verification, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 1701-1708.
  69. Tan, M. and Le, V., Mixconv: Mixed depthwise convolutional kernels, arXiv preprint arXiv:1907.09595, 2019.
  70. Tan, M. and Quoc, L., Efficientnet: Rethinking model scaling for convolutional neural networks, International Conference on Machine Learning, 2019, pp. 6105-6114.
  71. Turk, M. and Pentland, A., Eigenfaces for recognition, Journal of cognitive neuroscience, 1991, Vol. 3, No. 1, pp. 71-86. https://doi.org/10.1162/jocn.1991.3.1.71
  72. Wang, M. and Deng, W., Deep face recognition: A survey, Neurocomputing, 2021, Vol. 429, pp. 215-244. https://doi.org/10.1016/j.neucom.2020.10.081
  73. Wang, H., Wang, Y., Zhou, Z., Ji, X., Gong, D., Zhou, J., Li, Z., and Liu, W., Cosface: Large margin cosine loss for deep face recognition, Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2018, pp. 5265-5274.
  74. Wang, X., Fu, T., Liao, S., Wang, S., Lei, Z., and Mei, T., Exclusivity-consistency regularized knowledge distillation for face recognition, European Conference on Computer Vision, 2020, pp. 325-342.
  75. Wang, X., Teacher guided neural architecture search for face recognition, Proceedings of the AAAI Conference on Artificial Intelligence, 2021, Vol. 35. No. 4, pp. 2817-2825.
  76. Whitelam, C., Taborsky, E., Blanton, A., Maze, B., Adams, J., Miller, T., Kalka, N., Jain, A.K., Duncan, J.A., Allen, K., and Cheney, J., Iarpa janus benchmark-b face dataset, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017, pp 90-98
  77. Wright, J., Yang, A.Y., Ganesh, A., Sastry, S.S., and Ma, Y., Robust face recognition via sparse representation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, Vol. 31, No. 2, pp. 210-227. https://doi.org/10.1109/TPAMI.2008.79
  78. Wu, X., He, R., Sun, Z., and Tan, T., A light CNN for deep face representation with noisy labels, IEEE Transactions on Information Forensics and Security, 2018, Vol. 13, No. 11, pp. 2884-2896. https://doi.org/10.1109/TIFS.2018.2833032
  79. Xiao, J., Jiang, G., and Liu, H., A Lightweight Face Recognition Model based on MobileFaceNet for Limited Computation Environment, EAI Endorsed Transactions on Internet of Things, 2021, Vol. 7, No. 27, pp. 1-9.
  80. Yan, M., Zhao, M., Xu, Z., Zhang, Q., Wang, G., and Su, Z., Vargfacenet: An efficient variable group convolutional neural network for lightweight face recognition, Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, 2019.
  81. Yang, S., Wen, Y., He, L., Zhou, M., and Abusorrah, A., Sparse Individual Low-Rank Component Representation for Face Recognition in the IoT-Based System, IEEE Internet of Things Journal, 2021, Vol. 8, No. 24, 17320-17332. https://doi.org/10.1109/JIOT.2021.3080084
  82. Zagoruyko, S. and Nikos, K., Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer, arXiv preprint arXiv:1612.03928, 2016.
  83. Zhang, L., Yang, M., and Feng, X., Sparse representation or collaborative representation: Which helps face recognition?, 2011 International Conference on Computer Vision, 2011, pp. 471-478.
  84. Zhang, P., Zhao, F., Liu, P., and Li, M., Efficient LightweightAttention Network for Face Recognition, IEEEAccess, 2022, Vol. 10, pp. 31740-31750. https://doi.org/10.1109/ACCESS.2022.3150862
  85. Zhang, Q., Li, J., Yao, M., Song, L., Zhou, H., Li, Z., Meng, W., Zhang, X., and Wang, G., Vargnet: Variable group convolutional neural network for efficient embedded computing, arXiv preprint arXiv:1907.05653, 2019.
  86. Zhao, J., Lv, Y., Zhou, Z., and Cao, F., A novel deep learning algorithm for incomplete face recognition: Low-rank-recovery network, Neural Networks, 2017, Vol. 94, pp. 115-124. https://doi.org/10.1016/j.neunet.2017.06.013
  87. Zhao, W., Chellappa, R., Phillips, P.J., and Rosenfeld, A., Face recognition: A literature survey, ACM Computing Surveys (CSUR), 2003, Vol. 35, No. 4, pp. 399-458. https://doi.org/10.1145/954339.954342
  88. Zheng, T., Deng, W., and Hu, J., Cross-age LFW: A database for studying cross-age face recognition in unconstrained environments. arXiv preprint arXiv:1708.08197, 2017.
  89. Zhu, N., Yu, Z., and Kou, C., A new deep neural architecture search pipeline for face recognition, IEEE Access, 2020, Vol. 8, pp. 91303-91310. https://doi.org/10.1109/access.2020.2994207
  90. Zoph, B. and Le, Q.V., Neural architecture search with reinforcement learning, arXiv preprint arXiv:1611.01578, 2016.