KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Social Pedestrian Group Detection Based on Spatiotemporal-oriented Energy for Crowd Video Understanding

  • Huang, Shaonian (Key Laboratory of Hunan Province for New Retail Virtual Reality Technology, School of Computer and Information Engineering, Hunan University of Commerce) ;
  • Huang, Dongjun (School of Information Science and Engineering, Central South University) ;
  • Khuhroa, Mansoor Ahmed (School of Information Science and Engineering, Central South University)
  • Received : 2017.02.25
  • Accepted : 2018.04.06
  • Published : 2018.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

Social pedestrian groups are the basic elements that constitute a crowd; therefore, detection of such groups is scientifically important for modeling social behavior, as well as practically useful for crowd video understanding. A social group refers to a cluster of members who tend to keep similar motion state for a sustained period of time. One of the main challenges of social group detection arises from the complex dynamic variations of crowd patterns. Therefore, most works model dynamic groups to analysis the crowd behavior, ignoring the existence of stationary groups in crowd scene. However, in this paper, we propose a novel unified framework for detecting social pedestrian groups in crowd videos, including dynamic and stationary pedestrian groups, based on spatiotemporal-oriented energy measurements. Dynamic pedestrian groups are hierarchically clustered based on energy flow similarities and trajectory motion correlations between the atomic groups extracted from principal spatiotemporal-oriented energies. Furthermore, the probability distribution of static spatiotemporal-oriented energies is modeled to detect stationary pedestrian groups. Extensive experiments on challenging datasets demonstrate that our method can achieve superior results for social pedestrian group detection and crowd video classification.

Keywords

References

  1. T. Li, H. Chang, M.Wang, B. Ni, R. Hong and S. Yan, "Crowded Scene Analysis: A Survey," IEEE Transactions on Circuits and Systems for Video Technology, vol. 25, no. 3, pp. 367-386, 2015. https://doi.org/10.1109/TCSVT.2014.2358029
  2. Hoogs, Anthony and AG Amitha Perera, "Video Activity Recognition in the Real World," AAAI, pp.1551-1554, 2008.
  3. S.Ali and M. Shah,"A Lagrangian Particle Dynamics Approach for Crowd Flow Segmentation and Stability Analysis," in Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp.1-6, 2007.
  4. M. Moussaïd, N. Perozo, S. Garnier and D. Helbing, "The walking behaviour of pedestrian social groups and its impact on crowd dynamics," PloS ONE, vol. 5, no. 4, pp. 10047-10057, 2010. https://doi.org/10.1371/journal.pone.0010047
  5. J. Sochman and D.C. Hogg, "Who knows who - Inverting the Social Force Model for finding groups," in Proc. of IEEE Int. Conf. on Computer Vision, pp. 830-837, 2011.
  6. B. Zhou, X. Tang and X. Wang, "Coherent Filtering: Detecting Coherent Motions from Crowd Clutters," in Proc. of European Conf. on Computer Vision, pp. 857-871, 2012.
  7. M.C. Chang, N. Krahnstoever and W. Ge, "Probabilistic group-level motion analysis and scenario recognition," in Proc. of IEEE Int. Conf. on Computer Vision, pp.747-754 2011.
  8. S. Yi and X. Wang, "Profiling stationary crowd groups," in Proc. of IEEE Int. Conf. on Multimedia and Expo (ICME), pp.1-6, 2014.
  9. E.H. Adelson and J.R. Bergen,"Spatiotemporal energy models for the perception of motion," JOSA A, vol. 2, no. 2, pp. 284-299, 1985. https://doi.org/10.1364/JOSAA.2.000284
  10. D. Helbing, I. J. Farkas, P. Molnar, and T. Vicsek, "Simulation of pedestrian crowds in normal and evacuation situations," Pedestrian and evacuation dynamics, vol. 21, pp. 21-58, 2002.
  11. R. Mehran, A. Oyama and M. Shah, "Abnormal crowd behavior detection using social force model," in Proc. of IEEE Int. Conf. on Computer Vision and Pattern Recognition, pp. 935-942, 2009.
  12. Luber.M , Stork.J. A and Tipaldi.G.D, "People Tracking with Social Force-Based Motion Prediction," in Proc. of Int. Conf. on Cognitive Systems, 2010.
  13. V. J. Blue and J. L. Adler, "Cellular automata microsimulation for modeling bi-directional pedestrian walkways," Transportation Research Part B: Methodological, vol. 35, pp. 293-312, 2001. https://doi.org/10.1016/S0191-2615(99)00052-1
  14. N. Fridman and G. A. Kaminka, "Towards a cognitive model of crowd behavior based on social comparison theory," AAAI, pp. 731-737, 2007.
  15. R. Mehran, B. E. Moore, and M. Shah, "A streakline representation of flow in crowded scenes," in Proc. of European conf. on computer vision, pp. 439-452, 2010.
  16. H. Y. Hang Su, Shibao Zheng, Yawen Fan, and Sha We, "The Large-Scale Crowd Behavior Perception Based on Spatio-Temporal Viscous Fluid Field," IEEE Transaction on Information forensics and Security, vol. 8, no. 10, pp. 1575-1590, 2013. https://doi.org/10.1109/TIFS.2013.2277773
  17. B. Zhou, X. Tang, H. Zhang, and X. Wang, "Measuring Crowd Collectiveness," IEEE Transactions on Pattern Analysis and Machine Intelligence , vol.36, pp. 1586-1599, 2014. https://doi.org/10.1109/TPAMI.2014.2300484
  18. A. Bera, S. Kim, and D. Manocha, "Realtime anomaly detection using trajectory-level crowd behavior learning," in Proc. of IEEE Int. Conf. on Computer Vision and Pattern Recognition Workshops, pp. 50-57, 2016.
  19. J. Shao, C. C. Loy, and X. Wang, "Learning Scene-Independent Group Descriptors for Crowd Understanding," IEEE Transactions on Circuits and Systems for Video Technology, vol. 27, no.6, pp. 1290-1303, 2017. https://doi.org/10.1109/TCSVT.2016.2539878
  20. C.W. Reynolds, "Flocks, herds and schools: A distributed behavioral model," ACM SIGGRAPH Computer Graphics, vol. 21, no. 4, pp. 25-34, 1987. https://doi.org/10.1145/37402.37406
  21. N. Pelechano and N.I. Badler, "Modeling crowd and trained leader behavior during building evacuation," IEEE Computer Graphics and Applications, vol. 26, no. 6, pp. 80-86, 2006. https://doi.org/10.1109/MCG.2006.133
  22. A. Kendon, "Conducting Interaction: Patterns of Behavior in Focused Encounters," Studies in Interactional Sociolinguistics, 1990.
  23. M. Cristani, R. Raghavendra, A. Del Bue, and V. Murino, "Human behavior analysis in video surveillance: A social signal processing perspective," Neurocomputing, vol. 100, pp. 86-97, 2013. https://doi.org/10.1016/j.neucom.2011.12.038
  24. M. Rehm, E. Andre and M. Nischt, "Let's come together - Social navigation behaviors of virtual and real humans," in Proc. of Intelligent Technologies for Interactive Entertainment, pp. 124-133, 2005.
  25. F.S. Qiu and X.L. Hu, "Modeling group structures in pedestrian crowd simulation," Simulation Modelling Practice and Theory, vol. 18, no. 2, pp. 190-205, 2010. https://doi.org/10.1016/j.simpat.2009.10.005
  26. M. Hu, S. Ali and M. Shah, "Learning motion patterns in crowded scenes using motion flow field," in Proc. of Int. Conf. on Pattern Recognition, pp.1-5, 2008.
  27. M. Zanotto,L. Bazzan, M. Marco, "Online bayesian nonparametrics for group detection," in Proc. of British Machine Vision Conference, 2012.
  28. B. Zhou, X. Wang and X. Tang, "Random field topic model for semantic region analysis in crowded scenes from tracklets," in Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp. 3441-3448, 2011.
  29. W. Ge, R.T. Collins and R.B. Ruback, "Vision-Based Analysis of Small Groups in Pedestrian Crowds," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 5, pp. 1003-1016, 2012. https://doi.org/10.1109/TPAMI.2011.176
  30. F. Solera, S. Calderara and R. Cucchiara, "Socially Constrained Structural Learning for Groups Detection in Crowd," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 5, pp. 995-1008, 2016. https://doi.org/10.1109/TPAMI.2015.2470658
  31. K.G. Derpanis and J.M. Gryn,"Three-dimensional nth derivative of Gaussian separable steerable filters," in Proc. of IEEE Int. Conf. on Image Processing, 2005.
  32. K.G. Derpanis and R.P. Wildes,"Early spatiotemporal grouping with a distributed oriented energy representation," in Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp. 232-239, 2009.
  33. S. Jianbo and C. Tomasi,"Good features to track," in Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp. 593-600, 1994.
  34. M.J. Black and P. Anandan, "The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields," Computer Vision and Image Understanding, vol. 63, no. 1, pp. 75-104, 1996. https://doi.org/10.1006/cviu.1996.0006
  35. Deqing Sun, S.R. and M.J. Black, "Secrets of Optical Flow Estimation and Their Principles," in Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp. 2432-2439, 2010.
  36. Z. Zivkovic, "Improved adaptive Gaussian mixture model for background subtraction," in Proc. of Int. Conf. on Pattern Recognition , pp. 28-31, 2004.
  37. R. Cucchiara, C. Grana, M. Piccardi and A. Prati, "Detecting moving objects, ghosts, and shadows in video streams," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 10, pp. 1337-1342, 2003. https://doi.org/10.1109/TPAMI.2003.1233909
  38. V. Mahadevan, W. Li, V. Bhalodia and N. Vasconcelos, "Anomaly detection in crowded scenes," in Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp. 1975-1981, 2010.
  39. S. Pellegrini, A. Ess, K. Schindle and L. van Gool, "You'll never walk alone: Modeling social behavior for multi-target tracking," in proc. of IEEE Int. Conf. on Computer Vision , pp. 261-268, 2009.
  40. Ramin Meran, B.E.M. and MubarakShah, "A Streakline Representation of Flow in Crowded Scenes," in Proc. of European Conf. on Computer Vision , pp. 439-452, 2010.
  41. S. Yi and X. Wang, "Profiling stationary crowd groups," in Proc. of IEEE Int. Conf. on Multimedia and Expo, pp. 1-6, 2014.
  42. K. Kim, T. Chalidabhongse, D. Harwood and L. Davis, "Real-time foreground-background segmentation using codebook model," Real-time Imaging, vol. 11, no. 3, pp. 172-185, 2005. https://doi.org/10.1016/j.rti.2004.12.004