Journal of information and communication convergence engineering

The Korea Institute of Information and Commucation Engineering (한국정보통신학회)
권호 표지
DOI QR코드

DOI QR Code

Comparison of Fall Detection Systems Based on YOLOPose and Long Short-Term Memory

  • Seung Su Jeong (ICT & Robotics Engineering and IITC, Hankyong National University) ;
  • Nam Ho Kim (Bundang Convergence Technology Campus of Korea Polytechnic) ;
  • Yun Seop Yu (ICT & Robotics Engineering and IITC, Hankyong National University)
  • Received : 2024.02.19
  • Accepted : 2024.04.13
  • Published : 2024.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this study, four types of fall detection systems - designed with YOLOPose, principal component analysis (PCA), convolutional neural network (CNN), and long short-term memory (LSTM) architectures - were developed and compared in the detection of everyday falls. The experimental dataset encompassed seven types of activities: walking, lying, jumping, jumping in activities of daily living, falling backward, falling forward, and falling sideways. Keypoints extracted from YOLOPose were entered into the following architectures: RAW-LSTM, PCA-LSTM, RAW-PCA-LSTM, and PCA-CNN-LSTM. For the PCA architectures, the reduced input size stemming from a dimensionality reduction enhanced the operational efficiency in terms of computational time and memory at the cost of decreased accuracy. In contrast, the addition of a CNN resulted in higher complexity and lower accuracy. The RAW-LSTM architecture, which did not include either PCA or CNN, had the least number of parameters, which resulted in the best computational time and memory while also achieving the highest accuracy.

Keywords

Acknowledgement

This research was supported by the Basic Science Research Program through the NRF of Korea, funded by the Ministry of Education (NRF-2019R1F1A1060383).

References

  1. World Health Organization. Available online: https://www.who.int/news-room/fact-sheets/detail/falls (accessed on 26 April 2021).
  2. National Health Administration, Ministry of Health and Welfare. Available online: https://www.hpa.gov.tw/Pages/Detail.aspx?nodeid=807&pid=4326 (accessed on 9 March 2020).
  3. Y. Ren, Y. Chen, M. C. Chuah, and J. Yang, "User verification leveraging gait recognition for smartphone enabled mobile healthcare systems," IEEE Trans. Mobile Comput. vol. 14, pp. 1961-1974, 2014. DOI: 10.1109/TMC.2014.2365185.
  4. A. Ramachandran and A. Karuppiah, "A Survey on Recent Advances in Wearable Fall Detection Systems," BioMed Research International, vol. 2020, pp. 1-17, Jan. 2020. DOI: 10.1155/2020/2167160.
  5. E. Casilari, R. Lora-Rivera, and F. Garcia-Lagos, "A Study on the Application of Convolutional Neural Networks to Fall Detection Evaluated with Multiple Public Datasets," Sensors, vol. 20, no. 5, pp. 1466, Mar. 2020. DOI:10.3390/s20051466.
  6. S. S. Jeong, N. H. Kim, and Y. S. Yu, "Fall Detection System Based on Simple Threshold Method and Long Short-Term Memory: Comparison with Hidden Markov Model and Extraction of Optimal Parameters," Applied Sciences, vol. 12, no. 21, pp. 11031, Oct. 31, 2022. DOI: 10.3390/app122111031.
  7. D. Lim, C. Park, N. H. Kim, S.-H. Kim, and Y. S. Yu, "Fall- Detection Algorithm Using 3-Axis Acceleration: Combination with Simple Threshold and Hidden Markov Model," Journal of Applied Mathematics, vol. 2014, pp. 1-8, 2014. DOI:10.1155/2014/896030.
  8. X. Wang, J. Ellul, and G. Azzopardi, "Elderly Fall Detection Systems: A Literature Survey," Frontiers in Robotics and AI, vol. 7, Jun. 23, 2020. DOI:10.3389/frobt.2020.00071.
  9. M. Salimi, J. J. M. Machado, and J. M. R. S. Tavares, "Using Deep Neural Networks for Human Fall Detection Based on Pose Estimation," Sensors, vol. 22, no. 12, pp. 4544, Jun. 16, 2022. DOI: 10.3390/s22124544.
  10. C.-B. Lin, Z. Dong, W.-K. Kuan, and Y.-F. Huang, "A Framework for Fall Detection Based on OpenPose Skeleton and LSTM/GRU Models," Applied Sciences, vol. 11, no. 1, pp. 329, Dec. 2020. DOI:10.3390/app11010329.
  11. W. Chen, Z. Jiang, H. Guo, and X. Ni, "Fall Detection Based on Key Points of Human-Skeleton Using OpenPose," Symmetry, vol. 12, no. 5, pp. 744, May 2020. DOI:10.3390/sym12050744.
  12. N. Lu, Y. Wu, L. Feng, and J. Song, "Deep Learning for Fall Detection: Three-Dimensional CNN Combined with LSTM on Video Kinematic Data," IEEE Journal of Biomedical and Health Informatics, vol. 23, no. 1, pp. 314-323, Jan. 2019. DOI: 10.1109/JBHI.2018.2808281.
  13. F. Demrozi, G. Pravadelli, A. Bihorac, and P. Rashidi, "Human Activity Recognition Using Inertial, Physiological and Environmental Sensors: A Comprehensive Survey," IEEE Access, vol. 8, pp. 210816-210836, 2020. DOI: 10.1109/ACCESS.2020.3037715.
  14. E. Kanjo, E. M. G. Younis, and C. S. Ang, "Deep learning analysis of mobile physiological, environmental and location sensor data for emotion detection," Information Fusion, vol. 49, pp. 46-56, Sep. 2019. DOI: 10.1016/j.inffus.2018.09.001.
  15. J. Maitre, K. Bouchard, and S. Gaboury, "Fall Detection With UWB Radars and CNN-LSTM Architecture," IEEE Journal of Biomedical and Health Informatics, vol. 25, no. 4, pp. 1273-1283, Apr. 2021. DOI:10.1109/JBHI.2020.3027967.
  16. H. Fu and J. Gao, "Human Fall Detection Based on Posture Estimation and Infrared Thermography," IEEE Sensors Journal, vol. 23, no. 20, pp. 24744-24751, Oct. 2023. DOI: 10.1109/JSEN.2023.3307160.
  17. C.-F. Wu and S.-K. Lin, "Fall detection for multiple pedestrians using a PCA approach to 3-D inclination," International Journal of Engineering Business Management, vol. 11, pp. 184797901987897, Jan. 2019. DOI:10.1177/1847979019878971.
  18. C. Park and Y. S. Yu, "Video-based fall detection algorithm combining simple threshold method and Hidden Markov Model," Journal of the Korea Institute of Information and Communication Engineering, vol. 18, no. 9, pp. 2101-2108, Sep. 2014. DOI: jkiice.2014.18.9.2101. 18.9.2101
  19. Maji, D.; Nagori, S.; Mathew, M.; Poddar, D. YOLO-Pose: Enhancing YOLO for multi person pose estimation using object keypoint similarity loss. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, New Orleans, LA, USA, 19-21 June 2022; pp. 2637-2646, [online] Available: https://arxiv.org/abs/2204.06806 (accessed on 20 June 2022).
  20. H.-C. Nguyen, T.-H. Nguyen, R. Scherer, and V.-H. Le, "Unified End-to-End YOLOv5-HR-TCM Framework for Automatic 2D/3D Human Pose Estimation for Real-Time Applications," Sensors, vol. 22, no. 14, pp. 5419, Jul. 20, 2022. DOI: 10.3390/s22145419.
  21. M. M. Hasan, M. S. Islam, and S. Abdullah, "Robust Pose-Based Human Fall Detection Using Recurrent Neural Network," In Proceedings of 2019 IEEE International Conference on Robotics, Automation, Artificial-intelligence and Internet-of-Things (RAAICON). IEEE, Nov. 2019. DOI: 10.1109/RAAICON48939.2019.23.
  22. K. Adhikari, H. Bouchachia, and H. Nait-Charif, "Long short-term memory networks based fall detection using unified pose estimation," In Proceedings of Twelfth International Conference on Machine Vision (ICMV 2019). SPIE, Jan. 2020. DOI: 10.1117/12.2556540.
  23. J. Xu, Z. He, and Y. Zhang, "CNN-LSTM Combined Network for IoT Enabled Fall Detection Applications," Journal of Physics: Conference Series, vol. 1267, no. 1, pp. 012044, Jul. 2019. DOI: 10.1088/1742-6596/1267/1/012044.
  24. M. Li and D. Kim, "Classification in Different Genera by Cytochrome Oxidase Subunit I Gene Using CNN-LSTM Hybrid Model," Journal of information and communication convergence engineering, vol. 21, no. 2, pp. 159-166, Jun. 2023. DOI: 10.56977/jicce.2023.21.2.159.
  25. B. D. Romaissa, O. Mourad, N. Brahim, and B. Yazid, "Vision-Based Fall Detection Using Body Geometry," In Proceedings of Pattern Recognition. ICPR International Workshops and Challenges, pp. 170-185, 2021. DOI:10.1007/978-3-030-68799-1_13.
  26. F. Burden and D. Winkler, "Bayesian Regularization of Neural Networks," Methods in Molecular BiologyTM. Humana Press, pp. 23-42, 2008. DOI:10.1007/978-1-60327-101-1_3.
  27. A. Borji, "Enhancing sensor resolution improves CNN accuracy given the same number of parameters or FLOPS." arXiv, 2021. DOI: 10.48550/arXiv.2103.05251.
  28. W. Song, C. Gao, Y. Zhao, and Y. Zhao, "A Time Series Data Filling Method Based on LSTM-Taking the Stem Moisture as an Example," Sensors, vol. 20, no. 18, pp. 5045, Sep. 2020. DOI: 10.3390/s20185045.