DOI QR코드

DOI QR Code

Abnormal Behavior Recognition Based on Spatio-temporal Context

  • Yang, Yuanfeng (Jiangsu Province Support Software Engineering R&D Center for Modern Information Technology Application in Enterprise) ;
  • Li, Lin (School of Computer and Information Engineering, Xiamen University of Technology) ;
  • Liu, Zhaobin (School of Computer Engineering, Suzhou Vocational College) ;
  • Liu, Gang (School of Computer Engineering, Suzhou Vocational College)
  • Received : 2018.12.11
  • Accepted : 2019.05.08
  • Published : 2020.06.30

Abstract

This paper presents a new approach for detecting abnormal behaviors in complex surveillance scenes where anomalies are subtle and difficult to distinguish due to the intricate correlations among multiple objects' behaviors. Specifically, a cascaded probabilistic topic model was put forward for learning the spatial context of local behavior and the temporal context of global behavior in two different stages. In the first stage of topic modeling, unlike the existing approaches using either optical flows or complete trajectories, spatio-temporal correlations between the trajectory fragments in video clips were modeled by the latent Dirichlet allocation (LDA) topic model based on Markov random fields to obtain the spatial context of local behavior in each video clip. The local behavior topic categories were then obtained by exploiting the spectral clustering algorithm. Based on the construction of a dictionary through the process of local behavior topic clustering, the second phase of the LDA topic model learns the correlations of global behaviors and temporal context. In particular, an abnormal behavior recognition method was developed based on the learned spatio-temporal context of behaviors. The specific identification method adopts a top-down strategy and consists of two stages: anomaly recognition of video clip and anomalous behavior recognition within each video clip. Evaluation was performed using the validity of spatio-temporal context learning for local behavior topics and abnormal behavior recognition. Furthermore, the performance of the proposed approach in abnormal behavior recognition improved effectively and significantly in complex surveillance scenes.

Keywords

References

  1. C. Piciarelli and G. L. Foresti, "On-line trajectory clustering for anomalous events detection," Pattern Recognition Letters, vol. 27, no. 15, pp. 1835-1842, 2006. https://doi.org/10.1016/j.patrec.2006.02.004
  2. S. Kamijo, Y. Matsushita, K. Ikeuchi, and M. Sakauchi, "Traffic monitoring and accident detection at intersections," IEEE Transactions on Intelligent Transportation Systems, vol. 1, no. 2, pp. 108-118, 2000. https://doi.org/10.1109/6979.880968
  3. Y. Sun, L. Sun, H. Zhu, and X. Zhou, "Activity anomaly detection based on vehicle trajectory of automatic number plate recognition system," Journal of Computer Research and Development, vol. 52, no. 8, pp. 1921-1929, 2015.
  4. C. Micheloni, L. Snidaro, and G. L. Foresti, "Exploiting temporal statistics for events analysis and understanding," Image and Vision Computing, vol. 27, no. 10, pp. 1459-1469, 2009. https://doi.org/10.1016/j.imavis.2008.07.005
  5. C. Piciarelli, C. Micheloni, G. L. Foresti, "Trajectory-based anomalous event detection," IEEE Transactions on Circuits and Systems for Video Technology, vol. 18, no. 11, pp. 1544-1554, 2008. https://doi.org/10.1109/TCSVT.2008.2005599
  6. H. Y. Hu, Q. N. Wang, Z. W. Qu, and Z. H. Li, "Spatial pattern recognition and abnormal traffic behavior detection of moving object," Journal of Jilin University (Engineering and Technology Edition), vol. 41, no. 6, pp. 1598-1602, 2011.
  7. L. Zelnik-Manor and M. Irani, "Event-based analysis of video," in Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Kauai, HI, 2001, pp. 123-130.
  8. H. Zhong, J. Shi, and M. Visontai, "Detecting unusual activity in video," in Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Washington, DC, 2004, pp. 819-826.
  9. T. Xiang and S. Gong, "Video behaviour profiling and abnormality detection without manual labelling," in Proceedings of the 10th IEEE International Conference on Computer Vision (ICCV), Beijing, China, 2005, pp. 1238-1245.
  10. Y. Wang, H. Jiang, M. S. Drew, Z. N. Li, and G. Mori, "Unsupervised discovery of action classes," in Proceedings of 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), New York, NY, 2006, pp. 1654-1661.
  11. T. Xiang and S. Gong, "Beyond tracking: modelling activity and understanding behavior," International Journal of Computer Vision, vol. 67, no. 1, pp. 21-51, 2006. https://doi.org/10.1007/s11263-006-4329-6
  12. T. Xiang and S. Gong, "Video behavior profiling for anomaly detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 5, pp. 893-908, 2008. https://doi.org/10.1109/TPAMI.2007.70731
  13. M. Brand and V. Kettnaker, "Discovery and segmentation of activities in video," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 844-851, 2000. https://doi.org/10.1109/34.868685
  14. X. Wang, X. Ma, and E. Grimson, "Unsupervised activity perception by hierarchical Bayesian models," in Proceedings of 2007 IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, 2007, pp. 1-8.
  15. M. Y. Yang, W. Liao, Y. Cao, and B. Rosenhahn, "Video event recognition and anomaly detection by combining Gaussian process and hierarchical Dirichlet process models," Photogrammetric Engineering & Remote Sensing, vol. 84, no. 4, pp. 203-214, 2018. https://doi.org/10.14358/pers.84.4.203
  16. X. Wang, K. T. Ma, G. W. Ng, and W. E. Grimson, "Trajectory analysis and semantic region modeling using a nonparametric Bayesian model," in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVRP), Anchorage, AK, 2008, pp. 1-8.
  17. X. Wang, K. T. Ma, G. W. Ng, and W. E. L. Grimson, "Trajectory analysis and semantic region modeling using nonparametric hierarchical Bayesian models," International Journal of Computer Vision, vol. 95, no. 3, pp. 287-312, 2011. https://doi.org/10.1007/s11263-011-0459-6
  18. B. Zhou, X. Wang, and X. Tang, "Random field topic model for semantic region analysis in crowded scenes from tracklets," in Proceedings of 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI 2011, pp. 3441-3448.
  19. R. Kaviani, P. Ahmadi, and I. Gholampour, "Incorporating fully sparse topic models for abnormality detection in traffic videos," in Proceedings of 2014 4th International Conference on Computer and Knowledge Engineering (ICCKE), Mashhad, Iran, 2014, pp. 586-591.
  20. L. Song, F. Jiang, Z. Shi, and A. K. Katsaggelos, "Understanding dynamic scenes by hierarchical motion pattern mining," in Proceedings of 2011 IEEE International Conference on Multimedia and Expo, Barcelona, Spain, 2011, pp. 1-6.
  21. J. Li, S. Gong, and T. Xiang, "Scene segmentation for behaviour correlation," in Computer Vision - ECCV 2008. Heidelberg: Springer, 2008, pp. 383-395.
  22. J. Li, S. Gong, and T. Xiang, "Global behaviour inference using probabilistic latent semantic analysis," in Proceedings of the British Machine Vision Conference, Leeds, UK, 2008.
  23. C. C. Loy, T. Xiang, and S. Gong, "Detecting and discriminating behavioural anomalies," Pattern Recognition, vol. 44, no. 1, pp. 117-132, 2011. https://doi.org/10.1016/j.patcog.2010.07.023
  24. D. M. Blei and J. D. Lafferty, "Dynamic topic models," in Proceedings of the 23rd International Conference on Machine Learning, Pittsburgh, PA, 2006, pp. 113-120.
  25. D. M. Blei, A. Y. Ng, and M. I. Jordan, "Latent Dirichlet allocation," Journal of Machine Learning Research, vol. 3, pp. 993-1022, 2003.
  26. T. Hospedales, S. Gong, and T. Xiang, "Video behaviour mining using a dynamic topic model," International Journal of Computer Vision, vol. 98, no. 3, pp. 303-323, 2012. https://doi.org/10.1007/s11263-011-0510-7
  27. J. Li, S. Gong, and T. Xiang, "Learning behavioural context," International Journal of Computer Vision, vol. 97, no. 3, pp. 276-304, 2012. https://doi.org/10.1007/s11263-011-0487-2