International Journal of Computer Science & Network Security

International Journal of Computer Science & Network Security (국제컴퓨터통신보호논문지학회)
권호 표지
DOI QR코드

DOI QR Code

Transfer Learning for Face Emotions Recognition in Different Crowd Density Situations

  • Received : 2024.04.05
  • Published : 2024.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

Most human emotions are conveyed through facial expressions, which represent the predominant source of emotional data. This research investigates the impact of crowds on human emotions by analysing facial expressions. It examines how crowd behaviour, face recognition technology, and deep learning algorithms contribute to understanding the emotional change according to different level of crowd. The study identifies common emotions expressed during congestion, differences between crowded and less crowded areas, changes in facial expressions over time. The findings can inform urban planning and crowd event management by providing insights for developing coping mechanisms for affected individuals. However, limitations and challenges in using reliable facial expression analysis are also discussed, including age and context-related differences.

Keywords

Acknowledgement

The author would like to express her gratitude to Umm Alqura University for supporting this research (ID:4401095348).

References

  1. Mehrabian, A.; Ferris, S.R. Inference of attitudes from nonverbal communication in two channels. Journal of consulting psychology 1967, 31, 248.
  2. Quach, K.G.; Le, N.; Duong, C.N.; Jalata, I.; Roy, K.; Luu, K. Non-volume preserving-based fusion to group-level emotion recognition on crowd videos. Pattern Recognition 2022, 128, 108646.
  3. Veltmeijer, E.A.; Gerritsen, C.; Hindriks, K.V. Automatic emotion recognition for groups: a review. IEEE Transactions on Affective Computing 2021, 14, 89-107. https://doi.org/10.1109/TAFFC.2021.3065726
  4. Holder, R.P.; Tapamo, J.R. Using facial expression recognition for crowd monitoring. In Proceedings of the Image and Video Technology: 8th Pacific-Rim Symposium, PSIVT 2017, Wuhan, China, November 20-24, 2017, Revised Selected Papers 8. Springer, 2018, pp. 463-476.
  5. Dildine, T.C.; Atlas, L.Y. The need for diversity in research on facial expressions of pain. Pain 2019, 160, 1901.
  6. Ghosh, R.; Sinha, D. Human emotion recognition by analyzing facial expressions, heart rate and blogs using deep learning method. Innovations in Systems and Software Engineering 2022, pp. 1-9.
  7. Rosenberg, E.L.; Ekman, P. What the face reveals: Basic and applied studies of spontaneous expression using the Facial Action Coding System (FACS); Oxford University Press, 2020.
  8. Li, S.; Deng, W. Deep Facial Expression Recognition: A Survey. IEEE Transactions on Affective Computing 2022, 13, 1195-1215. https://doi.org/10.1109/TAFFC.2020.2981446
  9. Turan, B.; Algedik Demirayak, P.; Yildirim Demirdogen, E.; Gulsen, M.; Cubukcu, H.C.; Guler, M.; Alarslan, H.; Yilmaz, A.E.; Dursun, O.B. Toward the detection of reduced emotion expression intensity: an autism sibling study. Journal of Clinical and Experimental Neuropsychology 2023, pp. 1-11.
  10. Dollinger, L.; Laukka, P.; Hogman, L.B.; Banziger, T.; Makower, I.; Fischer, H.; Hau, S. Training emotion recognition accuracy: Results for multimodal expressions and facial micro expressions. Frontiers in Psychology 2021, 12, 708867.
  11. Rokhsaritalemi, S.; Sadeghi-Niaraki, A.; Choi, S.M. Exploring Emotion Analysis using Artificial Intelligence, Geospatial Information Systems, and Extended Reality for Urban Services. IEEE Access 2023.
  12. Gjoreski, M.; Kiprijanovska, I.; Stankoski, S.; Mavridou, I.; Broulidakis, M.J.; Gjoreski, H.; Nduka, C. Facial EMG sensing for monitoring affect using a wearable device. Scientific Reports 2022, 12, 16876.
  13. Engelniederhammer, A.; Papastefanou, G.; Xiang, L. Crowding density in urban environment and its effects on emotional responding of pedestrians: Using wearable device technology with sensors capturing proximity and psychophysiological emotion responses while walking in the street. Journal of Human Behavior in the Social Environment 2019, 29, 630-646. https://doi.org/10.1080/10911359.2019.1579149
  14. Abbas, A.; Chalup, S.K. Group emotion recognition in the wild by combining deep neural networks for facial expression classification and scene-context analysis. In Proceedings of the Proceedings of the 19th ACM international conference on multimodal interaction, 2017, pp. 561-568.
  15. Griffiths, S.; Rhodes, G.; Jeffery, L.; Palermo, R.; Neumann, M.F. The average facial expression of a crowd influences impressions of individual expressions. Journal of Experimental Psychology: Human Perception and Performance 2018, 44, 311.
  16. Bucher, A.; Voss, A. Judging the mood of the crowd: Attention is focused on happy faces. Emotion 2019, 19, 1044.
  17. Halamova, J.; Strnadelova, B.; Kanovsky, M.; Moro, R.; Bielikova, M. Anger or happiness ' superiority effect: A face in the crowd study involving nine emotions expressed by nine people. Current Psychology 2023, 42, 15381-15387.
  18. Tersek, M.; Kljun, M.; Peer, P.; Emersic, Z. Reevaluation of the CNN-based state-of-the-art crowdcounting methods with enhancements. Computer Science and Information Systems 2022, 19, 1177-1198.
  19. Shami, M.B.; Maqbool, S.; Sajid, H.; Ayaz, Y.; Cheung, S.C.S. People counting in dense crowd images using sparse head detections. IEEE Transactions on Circuits and Systems for Video Technology 2018, 29, 2627-2636. https://doi.org/10.1109/TCSVT.2018.2803115
  20. Fu, M.; Xu, P.; Li, X.; Liu, Q.; Ye, M.; Zhu, C. Fast crowd density estimation with convolutional neural networks. Engineering Applications of Artificial Intelligence 2015, 43, 81-88.
  21. Still, G.K. Applied crowd science; CRC Press, 2021.
  22. Fan, Z.; Zhang, H.; Zhang, Z.; Lu, G.; Zhang, Y.; Wang, Y. A survey of crowd counting and density estimation based on convolutional neural network. Neurocomputing 2022, 472, 224-251. https://doi.org/10.1016/j.neucom.2021.02.103
  23. Deng, J.; Guo, J.; Ververas, E.; Kotsia, I.; Zafeiriou, S. Retinaface: Single-shot multi-level face localisation in the wild. In Proceedings of the Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 5203-5212.
  24. International journal of computer vision 2023.
  25. Akhand, M.; Roy, S.; Siddique, N.; Kamal, M.A.S.; Shimamura, T. Facial emotion recognition using transfer learning in the deep CNN. Electronics 2021, 10, 1036.
  26. Rescigno, M.; Spezialetti, M.; Rossi, S. Personalized models for facial emotion recognition through transfer learning. Multimedia Tools and Applications 2020, 79, 35811-35828. https://doi.org/10.1007/s11042-020-09405-4
  27. Kong, Y.S.; Suresh, V.; Soh, J.; Ong, D.C. A systematic evaluation of domain adaptation in facial expression recognition. arXiv preprint arXiv:2106.15453 2023. 2106.15453
  28. Atanassov, A.; Pilev, D. Pre-trained Deep Learning Models for Facial Emotions Recognition. In Proceedings of the 2020 International Conference Automatics and Informatics (ICAI). IEEE, 2020, pp. 6.
  29. Li, S.; Deng, W. A deeper look at facial expression dataset bias. IEEE Transactions on Affective Computing 2020, 13, 881-893.
  30. Russakovsky, O.; Deng, J.; Su, H.; Krause, J.; Satheesh, S.; Ma, S.; Huang, Z.; Karpathy, A.; Khosla, A.; Bernstein, M.; et al. Imagenet large scale visual recognition challenge. International journal of computer vision 2015, 115, 211-252.
  31. Summa, M. Are emotions "recollected in tranquility"? Phenomenological reflections on emotions, memory, and the temporal dynamics of experience. In Feeling and value, willing and action: Essays in the context of a phenomenological psychology; Springer, 2014; pp. 163-181.
  32. Reicher, S. The psychology of crowd dynamics. Blackwell handbook of social psychology: Group processes 2001, pp. 182-208.
  33. Qureshi, A. W., Monk, R. L., Quinn, S., Gannon, B., McNally, K., & Heim, D. Catching a smile from individuals and crowds: Evidence for distinct emotional contagion processes. Journal of Personality and Social Psychology 2023. Advance online publication. https://doi.org/10.1037/pspi0000445
  34. Halamova, J., Strnadelova, B., Kanovsky, M., Moro, R. and Bielikova, M. Anger or happiness superiority effect: A face in the crowd study involving nine emotions expressed by nine people. Current Psychology 2023, 42(18), pp.15381-15387. https://doi.org/10.1007/s12144-022-02762-3
  35. Siddiqui MF, Javaid AY. A multimodal facial emotion recognition framework through the fusion of speech with visible and infrared images. Multimodal Technologies and Interaction 2020, 4(3):46.
  36. Ezzameli K, Mahersia H. Emotion recognition from unimodal to multimodal analysis: A review. Information Fusion 2023, 101847.