DOI QR코드

DOI QR Code

A Study on People Counting in Public Metro Service using Hybrid CNN-LSTM Algorithm

Hybrid CNN-LSTM 알고리즘을 활용한 도시철도 내 피플 카운팅 연구

  • Choi, Ji-Hye (Department of Computer & Telecommunications Engineering, Yonsei University) ;
  • Kim, Min-Seung (Department of Computer & Telecommunications Engineering, Yonsei University) ;
  • Lee, Chan-Ho (Department of Computer & Telecommunications Engineering, Yonsei University) ;
  • Choi, Jung-Hwan (Department of Computer & Telecommunications Engineering, Yonsei University) ;
  • Lee, Jeong-Hee (Department of Computer Science, Graduate School, Yonsei University) ;
  • Sung, Tae-Eung (Department of Computer & Telecommunications Engineering, Yonsei University)
  • 최지혜 (연세대학교 컴퓨터정보통신공학부) ;
  • 김민승 (연세대학교 컴퓨터정보통신공학부) ;
  • 이찬호 (연세대학교 컴퓨터정보통신공학부) ;
  • 최정환 (연세대학교 컴퓨터정보통신공학부) ;
  • 이정희 (연세대학교 대학원 전산학과) ;
  • 성태응 (연세대학교 컴퓨터정보통신공학부)
  • Received : 2020.05.12
  • Accepted : 2020.06.29
  • Published : 2020.06.30

Abstract

In line with the trend of industrial innovation, IoT technology utilized in a variety of fields is emerging as a key element in creation of new business models and the provision of user-friendly services through the combination of big data. The accumulated data from devices with the Internet-of-Things (IoT) is being used in many ways to build a convenience-based smart system as it can provide customized intelligent systems through user environment and pattern analysis. Recently, it has been applied to innovation in the public domain and has been using it for smart city and smart transportation, such as solving traffic and crime problems using CCTV. In particular, it is necessary to comprehensively consider the easiness of securing real-time service data and the stability of security when planning underground services or establishing movement amount control information system to enhance citizens' or commuters' convenience in circumstances with the congestion of public transportation such as subways, urban railways, etc. However, previous studies that utilize image data have limitations in reducing the performance of object detection under private issue and abnormal conditions. The IoT device-based sensor data used in this study is free from private issue because it does not require identification for individuals, and can be effectively utilized to build intelligent public services for unspecified people. Especially, sensor data stored by the IoT device need not be identified to an individual, and can be effectively utilized for constructing intelligent public services for many and unspecified people as data free form private issue. We utilize the IoT-based infrared sensor devices for an intelligent pedestrian tracking system in metro service which many people use on a daily basis and temperature data measured by sensors are therein transmitted in real time. The experimental environment for collecting data detected in real time from sensors was established for the equally-spaced midpoints of 4×4 upper parts in the ceiling of subway entrances where the actual movement amount of passengers is high, and it measured the temperature change for objects entering and leaving the detection spots. The measured data have gone through a preprocessing in which the reference values for 16 different areas are set and the difference values between the temperatures in 16 distinct areas and their reference values per unit of time are calculated. This corresponds to the methodology that maximizes movement within the detection area. In addition, the size of the data was increased by 10 times in order to more sensitively reflect the difference in temperature by area. For example, if the temperature data collected from the sensor at a given time were 28.5℃, the data analysis was conducted by changing the value to 285. As above, the data collected from sensors have the characteristics of time series data and image data with 4×4 resolution. Reflecting the characteristics of the measured, preprocessed data, we finally propose a hybrid algorithm that combines CNN in superior performance for image classification and LSTM, especially suitable for analyzing time series data, as referred to CNN-LSTM (Convolutional Neural Network-Long Short Term Memory). In the study, the CNN-LSTM algorithm is used to predict the number of passing persons in one of 4×4 detection areas. We verified the validation of the proposed model by taking performance comparison with other artificial intelligence algorithms such as Multi-Layer Perceptron (MLP), Long Short Term Memory (LSTM) and RNN-LSTM (Recurrent Neural Network-Long Short Term Memory). As a result of the experiment, proposed CNN-LSTM hybrid model compared to MLP, LSTM and RNN-LSTM has the best predictive performance. By utilizing the proposed devices and models, it is expected various metro services will be provided with no illegal issue about the personal information such as real-time monitoring of public transport facilities and emergency situation response services on the basis of congestion. However, the data have been collected by selecting one side of the entrances as the subject of analysis, and the data collected for a short period of time have been applied to the prediction. There exists the limitation that the verification of application in other environments needs to be carried out. In the future, it is expected that more reliability will be provided for the proposed model if experimental data is sufficiently collected in various environments or if learning data is further configured by measuring data in other sensors.

산업혁신의 흐름에 발맞추어 다양한 분야에서 활용되고 있는 IoT 기술은 빅데이터의 접목을 통한 새로운 비즈니스 모델의 창출 및 사용자 친화적 서비스 제공의 핵심적인 요소로 부각되고 있다. 사물인터넷이 적용된 디바이스에서 누적된 데이터는 사용자 환경 및 패턴 분석을 통해 맞춤형 지능 시스템을 제공해줄 수 있어 편의 기반 스마트 시스템 구축에 다방면으로 활용되고 있다. 최근에는 이를 공공영역 혁신에 확대 적용하여 CCTV를 활용한 교통 범죄 문제 해결 등 스마트시티, 스마트 교통 등에 활용하고 있다. 그러나 이미지 데이터를 활용하는 기존 연구에서는 개인에 대한 사생활 침해 문제 및 비(非)일반적 상황에서 객체 감지 성능이 저하되는 한계가 있다. 본 연구에 활용된 IoT 디바이스 기반의 센서 데이터는 개인에 대한 식별이 불필요해 사생활 이슈로부터 자유로운 데이터로, 불특정 다수를 위한 지능형 공공서비스 구축에 효과적으로 활용될 수 있다. 대다수의 국민들이 일상적으로 활용하는 도시철도에서의 지능형 보행자 트래킹 시스템에 IoT 기반의 적외선 센서 디바이스를 활용하고자 하였으며 센서로부터 측정된 온도 데이터를 실시간 송출하고, CNN-LSTM(Convolutional Neural Network-Long Short Term Memory) 알고리즘을 활용하여 구간 내 보행 인원의 수를 예측하고자 하였다. 실험 결과 MLP(Multi-Layer Perceptron) 및 LSTM(Long Short-Term Memory), RNN-LSTM(Recurrent Neural Network-Long Short Term Memory)에 비해 제안한 CNN-LSTM 하이브리드 모형이 가장 우수한 예측성능을 보임을 확인하였다. 본 논문에서 제안한 디바이스 및 모델을 활용하여 그간 개인정보와 관련된 법적 문제로 인해 서비스 제공이 미흡했던 대중교통 내 실시간 모니터링 및 혼잡도 기반의 위기상황 대응 서비스 등 종합적 메트로 서비스를 제공할 수 있을 것으로 기대된다.

Keywords

References

  1. Chan, A. B., Z. J. Liang and N. V. "Privacy preserving crowd monitoring: Counting people without people models or tracking", IEEE Conference on Computer Vision and Pattern Recognition, 1-7, 2008.
  2. Depatla, S., A. Muralidharan and Y. Mostofi, "Occupancy estimation using only WiFi power measurements", IEEE Journal on Selected Areas in Communications, Vol.33, No.7(2015), 1381-1393. https://doi.org/10.1109/JSAC.2015.2430272
  3. Fu, H., H. Ma and H. Xiao, "Real-time accurate crowd counting based on RGB-D information", 2012 19th IEEE International Conference on Image Processing, 2685-2688, 2012.
  4. Jacques Jr, J. C. S. and S. R. Musse, "Crowd Analysis Using Computer Vision", IEEE Signal Processing Magazine, Vol.27, No.5(2010), 66-77. https://doi.org/10.1109/MSP.2010.937394
  5. Jin, H. and A. Anish "A regression-based radar-mote system for people counting", IEEE International Conference On Pervasive Computing and Communications, 95-102, 2014.
  6. Jung, W. S., S. H. Kim and K. S. Min, "An Analysis of the Economic Effect for the IoT Industry", Journal of Korean Society for Internet Information, Vol.22, No.1(2013), 7-13.
  7. Kim, G. J., K. Y. Eom, J. Y. Jung and M. H. Kim, "A Study on the Analysis of Congestion Degree in Platform Image for Urban Traffic", Proceedings of the KISS conference, Vol.36, No.2(2009), 422-426.
  8. Kim, H. W. and D. K. Kim, "IoT technology and security", Review of KIISC, Vol.22, No.1 (2012), 7-13.
  9. Kim, T. H., J. M. Choi, H. Lee, J. H. So and M. J. Kim, "A Study on the Implementation Strategies of Smart City Mobility", Korea Transport Institute, 2018.
  10. Lee, H. and J. K. Choi, "Development of an Algorithm for Estimating Subway Platform Congestion Using Public Transportation Card Data", Journal of the Korean Society for Railway, Vol.18, No.3(2015), 270-277. https://doi.org/10.7782/JKSR.2015.18.3.270
  11. Mehmood, Y., F. Ahmad, I. Yaqoob, A. Adnane, M. Imran and S. Guizani, "Internetof-Things-Based Smart Cities: Recent Advances and Challenges", IEE Communications Magazine, Vol.55, No.9(2017), 16-24.
  12. Park, K. C., D. H. Kim, W. S. Ji and M. Y. Bin, "Making safe public transportation", Issue & Diagnostics, Vol.20(2011), 1-25.
  13. Park, J. W. and K. S. Lee, "Analysis of Passenger Flows in the Subway Transportation Network of the Metropolitan Seoul", Journal of KIISE:Computing Practices and Letters, Vol.16, No.3(2010), 316-323.
  14. Pyo, C. S., "Trend of Internet of Things", Journal of Korean Institute of Electromagnetic Engineering and Science, Vol.25, No.4(2014), 49-58.
  15. Xu, H. and L. Meng, "A people counting system based on head-shoulder detection and tracking in surveillance video", IEEE International Conference On Computer Design and Applications, 394-398, 2010.
  16. Yuan, Y., J. Zhao, C. Qiu and W. Xi, "Estimating Crowd Density in an RF-Based Dynamic Environment", IEEE Sensors Journal, Vol.13, No.10(2013), 3837-3845. https://doi.org/10.1109/JSEN.2013.2259692
  17. Yang, H. J., H. Jang, J. H. Jeong, B. W. Lee and D. S. Jeong, "A Real-time People Counting Algorithm Using Background Modeling and CNN," Journal of the Institute of Electronics and Information Engineers, Vol.54, No.3(2017), 70-77. https://doi.org/10.5573/ieie.2017.54.3.70
  18. Zhang, E. and F. Chen, "A Fast Robust People Counting Method in Video Surveillance", IEEE International Conference on Computational Intelligence and Security, 339-343, 2017.
  19. http://news.seoul.go.kr/traffic/archives/31616