Journal of the Korea Society for Simulation (한국시뮬레이션학회논문지)

The Korea Society for Simulation (한국시뮬레이션학회)
권호 표지
DOI QR코드

DOI QR Code

An Approach Using LSTM Model to Forecasting Customer Congestion Based on Indoor Human Tracking

실내 사람 위치 추적 기반 LSTM 모델을 이용한 고객 혼잡 예측 연구

  • Received : 2023.07.20
  • Accepted : 2023.09.13
  • Published : 2023.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this detailed and comprehensive study, our primary focus has been placed on accurately gauging the number of visitors and their real-time locations in commercial spaces. Particularly, in a real cafe, using security cameras, we have developed a system that can offer live updates on available seating and predict future congestion levels. By employing YOLO, a real-time object detection and tracking algorithm, the number of visitors and their respective locations in real-time are also monitored. This information is then used to update a cafe's indoor map, thereby enabling users to easily identify available seating. Moreover, we developed a model that predicts the congestion of a cafe in real time. The sophisticated model, designed to learn visitor count and movement patterns over diverse time intervals, is based on Long Short Term Memory (LSTM) to address the vanishing gradient problem and Sequence-to-Sequence (Seq2Seq) for processing data with temporal relationships. This innovative system has the potential to significantly improve cafe management efficiency and customer satisfaction by delivering reliable predictions of cafe congestion to all users. Our groundbreaking research not only demonstrates the effectiveness and utility of indoor location tracking technology implemented through security cameras but also proposes potential applications in other commercial spaces.

본 연구는 실내 상업적 공간, 특히 카페에서 보안 카메라를 이용해 방문자 수와 위치를 실시간으로 파악하고, 이를 통해 사용 가능한 좌석 정보와 혼잡도 예측을 제공하는 시스템의 개발을 목표로 한다. 우리는 실시간 객체 탐지 및 추적 알고리즘인 YOLO를 활용하여 방문자 수와 위치를 실시간으로 파악하며, 이 정보를 카페 실내 지도에 업데이트하여 카페 방문자가 사용 가능한 좌석을 확인할 수 있도록 한다. 또한, 우리는 vanishing gradient문제를 해결한 장단기 메모리(Long Short Term Memory, LSTM)와 시간적인 관계를 가지는 데이터를 처리하는데 유용한 시퀀스-투-시퀀스(Sequence-to-Sequence, Seq2Seq)기법을 활용해 다양한 시간 간격에 따른 방문자 수와 움직임 패턴을 학습하고, 이를 바탕으로 카페의 혼잡도를 실시간으로 예측하는 시스템을 개발하였다. 이 시스템은 카페의 관리자와 이용자 모두에게 예상 혼잡도를 제공함으로써, 카페의 운영 효율성을 향상시키고, 고객 만족도를 높일 수 있다. 본 연구에서는 보안 카메라를 활용한 실내 위치 추적 기술의 효용성을 입증하며, 상업적 공간에서의 활용 가능성과 더불어 미래 연구 방향을 제시한다.

Keywords

Acknowledgement

본 연구는 2023년 과학기술정보통신부 및 정보통신기획평가원의 SW중심대학사업의 연구결과로 수행되었음(2022-0-01068)

References

  1. Boukary, N. A. (2016) "A comparison of time series forecasting learning algorithms on the task of predicting event timing", Master of Applied Science thesis, Royal Military College of Canada.
  2. Brostrom, M. "YOLO Tracking", Available at https://github.com/mikel-brostrom/yolo_tracking (Accessed July 14. 2023).
  3. Chae, H., Kwak, K., Lee, D., and Kim, E. (2023) "An paproach using LSTM model to forecasting customer congestion based on indoor human tracking in video streaming", KSS Spring Conference, June 2023.
  4. Choi, G.W., Ahn, W.S., Yang, J.Y., and Kim, D.H. (2021) "Congestion measurement and visualization system for Han River Park", Proceedings of the Korean Information Technology Society Conference, 559-564.
  5. Dey, R. and Salem, F. M. (2017) "Gate-variants of gated recurrent unit (GRU) neural networks", 2017 IEEE 60th International Midwest Symposium on Circuits and Systems (MWSCAS), Boston, MA, USA, 1597-1600.
  6. Du, Y., Zhao, Z., Song, Y., Zhao, Y., Su, F., Gong, T., and Meng, H. (2023) "StrongSORT: make DeepSORT great again" IEEE Transactions on Multimedia, Early Access. doi: 10.1109/TMM.2023.3240881 (Accessed July 18. 2023).
  7. Iqbal, M., Al-Obeidat, F., Razzaq, S., Anwar, S., Tubaishat, A., Khan, M. S. and Shah, B. (2021) "COVID-19 patient count prediction using LSTM", IEEE Trans. on Computational Social Systems, 8 (4), 974-981. https://doi.org/10.1109/TCSS.2021.3056769
  8. Ito, K. and Xiong, K. (2000) "Gaussian filters for nonlinear filtering problems", IEEE Trans. on Automatic Control, 45(5), 910-927. https://doi.org/10.1109/9.855552
  9. Kim, B.S. (2020) "Optimization study of intelligent CCTV system for visitor detection", Master's thesis, Kangwon National University Graduate School, Gangneung.
  10. Kim, D.H., Hwang, K.Y., and Yoon, Y. (2019) "Prediction of traffic congestion by Deep Neural Networks (DNN) and multidimensional context information for Seoul city road links", Journal of the Korean ITS Society, 18(4), 44-57. https://doi.org/10.12815/kits.2019.18.4.44
  11. Kim, G.W., No, G.S., Kim, D.W. and Lee, J.Y. (2015) "Exploratory study on improving subway congestion through big data fusion", Journal of Digital Convergence, 13(2), 35-42. https://doi.org/10.14400/JDC.2015.13.2.35
  12. Kim, J.S. (2016) "prediction and recommendation system for subway congestion using big data analysis", Journal of Digital Convergence, 14(11), 289-295. https://doi.org/10.14400/JDC.2016.14.11.289
  13. Kim, J.W., Lee, T.W., Kim, D.Y., and Lee, J.H. (2019) "Implementation of real-time people counting and congestion detection system based on ultrasonic sensor", Proceedings of the Korean Institute of Communications and Information Sciences Conference, 308-309.
  14. Kim, M.J., Go, S.Y., and Jeong, N.H. (2021) "Case study of Jeju tourism organization's real-time tourism site congestion analysis service", Journal of Information Technology Services, 20(5), 29-41.
  15. Kim, S.H., Park, H.J., Oh, J.E., and Lee, K.Y. (2021) "Development of cafe congestion information application using deep Learning-based object detection technology", Proceedings of the Korean Information Science Society Conference, 1342-1344.
  16. Koutnik, J., Greff, K., Gomez, and F., Schmidhuber, J. (2014) "A Clockwork RNN", arXiv preprint arXiv:1402.3511. Available at https://doi.org/10.48550/arXiv.1402.3511 (Accessed July 18. 2023).
  17. Kwon, S.H., Lee, S.C., and Kim, H.S. (2021) "Development of congestion estimation program in spatial environment using IEEE 802.11 proberequest", Proceedings of the Korean Computer Information Society Academic Conference, 29(2), 257-260.
  18. Lee, H.S., Nam, B.C., and Seon, C.N. (2020) "Deep learning LSTM framework for urban traffic flow and fine dust prediction", Journal of Korean Information Science Society, 47(3), 292-297.
  19. Lee, T.W., Kim, J.W., Kim, D.Y., and Lee, J.H. (2018) "Personnel counting algorithm for emergency exits based on multiple ultrasonic sensors" Proceedings of the Korean Communications Society Conference, 299-300.
  20. Park, Y. (2023) "Concise logarithmic loss function for robust training of anomaly detection model", arXiv preprint arXiv: 2201.05748v2. Available at https://doi.org/10.48550/arXiv.2201.05748 (Accessed July 17. 2023).
  21. poiuyreq0 (2023) "KOKO", Available at https://github.com/poiuyreq0/KOKO(Accessed July 14. 2023).
  22. Qolomany, B., Al-Fuqaha, A., Benhaddou, D. and Gupta, A. (2017) "Role of deep LSTM neural networks and Wi-Fi networks in support of occupancy prediction in Smart Buildings", 2017 IEEE 19th International Conference on High Performance Computing and Communications; IEEE 15th International Conference on Smart City; IEEE 3d Systems (HPCC/SmartCity/DSS), 50-57, doi: 10.1109/HPCC-SmartCity-DSS.2017.7 (Accessed July 16, 2023).
  23. Sojasingarayar, A. (2020) "Seq2Seq AI Chatbot with Attention Mechanism", arXiv preprint arXiv: 2006.02767. Available at https://doi.org/10.48550/arXiv.2006.02767 (Accessed July 16. 2023).
  24. Sultana, F., Sufian, A., Dutta, P. (2019) "A review of object detection models based on convolutional neural network", arXiv preprint arXiv: 1905.01614. Available at https://doi.org/10.48550/arXiv.1905.01614 (Accessed July 18. 2023).
  25. Ultralytics "YOLO" Available at https://github.com/ultralytics/ultralytics/tree/15b3b0365ab2f12993a58985f3cb7f2137409a0c (Accessed July 14. 2023).