ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

Danger detection technology based on multimodal and multilog data for public safety services

  • Park, Hyunho (Defense & Safety ICT Research Department, Electronics and Telecommunications Research Institute) ;
  • Kwon, Eunjung (Defense & Safety ICT Research Department, Electronics and Telecommunications Research Institute) ;
  • Byon, Sungwon (Defense & Safety ICT Research Department, Electronics and Telecommunications Research Institute) ;
  • Shin, Won-Jae (Defense & Safety ICT Research Department, Electronics and Telecommunications Research Institute) ;
  • Jung, Eui-Suk (Defense & Safety ICT Research Department, Electronics and Telecommunications Research Institute) ;
  • Lee, Yong-Tae (Defense & Safety ICT Research Department, Electronics and Telecommunications Research Institute)
  • Received : 2020.09.25
  • Accepted : 2021.10.29
  • Published : 2022.04.10
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, public safety services have attracted significant attention for their ability to protect people from crimes. Rapid detection of dangerous situations (that is, abnormal situations where someone may be harmed or killed) is required in public safety services to reduce the time required to respond to such situations. This study proposes a novel danger detection technology based on multimodal data, which includes data from multiple sensors (for example, accelerometer, gyroscope, heart rate, air pressure, and global positioning system sensors), and multilog data, which includes contextual logs of humans and places (for example, contextual logs of human activities and crime-ridden districts) over time. To recognize human activity (for example, walk, sit, and punch), the proposed technology uses multimodal data analysis with an attitude heading reference system and long short-term memory. The proposed technology also includes multilog data analysis for detecting whether recognized activities of humans are dangerous. The proposed danger detection technology will benefit public safety services by improving danger detection capabilities.

Keywords

Acknowledgement

This research was supported and funded by the Korean National Police Agency (Project Name: 112 Emergency Dispatch Decision Support System/Project Number: PR08-03-000-21).

References

  1. J. Bang et al., AR/VR based smart policing for fast response to crimes in safe city, in Proc. IEEE Int. Symp. Mixed Augment. Reality Adjunct (ISMAR-Adjunct), (Beijing, China), Oct. 2019, pp. 470-475.
  2. W. C. Choi and J. Y. Na, Study on real-time crime response service using multi-CCTV collaboration technology, in Proc. Int. Conf. Cloud Big Data (ICCBC), (Barcelona, Spain), Aug. 2018, pp. 78-81.
  3. A. G. Ferguson, Predictive policing theory, American University, WCL Research Paper No. 2020-10, 2020, pp. 491-510.
  4. J. Bang et al., Trends of intelligent public safety service technologies, Electron. Telecommun. Trends 34 (2019), 111-122.
  5. S. Lim, Bridge damage identification and its severity estimation using artificial intelligence, Ph.D. dissertation, Deptment of Civil & Environmental Engineering, Seoul National University, Seoul, South Korea, 2020.
  6. G. Chevalier, LSTMs for human activity recognition, 2017, Retrieved from: https://github.com/guillaume-chevalier/LSTM-Human-Activity-Recognition
  7. A. Hakim et al., Smartphone based data mining for fall detection: Analysis and design, Procedia Comput. Sci. 105 (2017), 46-51. https://doi.org/10.1016/j.procs.2017.01.188
  8. T. R. Mauldin et al., SmartFall: A smartwatch-based fall detection system using deep learning, Sensors, 18 (2018), no. 10, 3-11.
  9. X. Wu et al., Using deep learning and smartphone for automatic detection of fall and daily activities, in Smart Health, vol. 11924, Springer, Cham, Switzerland, 2019, pp. 61-74.
  10. H. Keeffe, Wrist-based device accelerometer analysis for establishing thresholds of various violent attacks, M.S. thesis, The School of Veterinary and Life Sciences, Murdoch University, Perth, Australia, 2018.
  11. F. Tchuente, Recognition and classification of aggressive motion using smartwatches, M.S. thesis, Mechanical Engineering, University of Otawa, Otawa, Canada, 2018.
  12. M. Hwang, Punch-buddy-fight-club, 2019, Retrieved from: https://github.com/mdhwang/Punch-Buddy-Fight-Club
  13. R. Mahony, T. Hamel, and J.-M. Pflimlin, Nonlinear complementary filters on the special orthogonal group, IEEE Trans. Autom. Control 53 (2008), no. 5, 1203-1218. https://doi.org/10.1109/TAC.2008.923738
  14. B. Li, B. Harvey, and T. Gallagher, Using barometers to determine the height for indoor positioning, in Proc. Int. Conf. Indoor Positioning Indoor Navig. (Montbeliard, France), Oct. 2013.
  15. Y. Mae et al., Extraction of mental stress scene in driving car by wearable heart rate sensor, in Proc. IEEE Int. Conf. Proc. Intell. Saf. Robot. (Shenyang, China), Aug. 2018, 480-485.
  16. H. Park et al., Multi-log analysis platform for supporting public safety service, in Proc. Int. Conf. Inf. Commun. Technol. Converg. (ICTC), (Jeju, South Korea), Oct. 2017, 1137-1139.
  17. S. A. Ludwig, A. R. Jimenez, and P. A. Touma, Comparison of attitude and heading reference systems using foot mounted MIMU sensor data: Basic, Madgwick, and Mahony, in Proc. Sens. Smart Struct. Technol. Civ. Mech. Aerosp. Syst. (Denver, CO, USA), Mar. 2018, pp. 105982L-1-105982L-7.
  18. E. H. Bahadur et al., LSTM based approach for diabetic symptomatic activity recognition using smartphone sensors, in Proc. Int. Conf. Comput. Inf. Technol. (ICCIT), (Dhaka, Bangladesh), Dec. 2019, pp. 1-6.
  19. J. L. Reyes. Human activity recognition using smartphones data set, 2012, Retrieved from: https://archive.ics.uci.edu/ml/datasets/human+activity+recognition+using+smartphones
  20. xioTechnologies, gait-tracking-With-x-IMU, 2017, Retrieved from: https://github.com/xioTechnologies/Gait-Tracking-With-x-IMU
  21. J. O. Stone, Air pressure and cosmogenic isotope production, J. Geophys. Res. 105 (2000), 23753-23759. https://doi.org/10.1029/2000JB900181
  22. A. Jimenez-Meza, J. Aramburo-Lizarraga, and E. de la Fuente, Framework for estimating travel time, distance, speed, and street segment level of service (LOS), based on GPS data, Procedia Technol. 7 (2013), 61-70. https://doi.org/10.1016/j.protcy.2013.04.008
  23. H. Park et al., Danger detection based on fusion analysis of multimodal data from a smartphone and smartwatch, in Proc. Int. Conf. Inf. Commun. Technol. Converg. (ICTC), (Jeju, South Korea), Oct. 2017, pp. 1143-1145.