KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Human Action Recognition via Depth Maps Body Parts of Action

  • Farooq, Adnan (Department of Biomedical Engineering, Kyung Hee University) ;
  • Farooq, Faisal (Department of Biomedical Engineering, Kyung Hee University) ;
  • Le, Anh Vu (Optoelectronics Research Group, Faculty of Electrical and Electronics Engineering, Ton Duc Thang University)
  • Received : 2017.04.07
  • Accepted : 2017.12.26
  • Published : 2018.05.31
Download PDF
⟨ Previous Next ⟩

Abstract

Human actions can be recognized from depth sequences. In the proposed algorithm, we initially construct depth, motion maps (DMM) by projecting each depth frame onto three orthogonal Cartesian planes and add the motion energy for each view. The body part of the action (BPoA) is calculated by using bounding box with an optimal window size based on maximum spatial and temporal changes for each DMM. Furthermore, feature vector is constructed by using BPoA for each human action view. In this paper, we employed an ensemble based learning approach called Rotation Forest to recognize different actions Experimental results show that proposed method has significantly outperforms the state-of-the-art methods on Microsoft Research (MSR) Action 3D and MSR DailyActivity3D dataset.

Keywords

References

  1. O. Masoud, N.Papanikolopoulos, "A method for human action recognition," Image Vis. Comput., vol. 12, no. 8, pp. 729-743, 2003.
  2. Z. Gao, J.-M. Song, H. Zhang, A.-A.Liu, Y.-B.Xue, G.-P. Xu, "Human action recognition via Mmulti-modality Information," J. Electr. Eng. Technol., vol 9 no. 2, pp. 739-748, 2014. https://doi.org/10.5370/JEET.2014.9.2.739
  3. M. Ye, Q. Zhang, L. Wang, J. Zhu, R. Yang, J. Gall, "A Survey on human motion analysis from depth data," in Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications'(Springer Berlin Heidelberg), pp. 149-187, 2013.
  4. J.W. Davis, A.F.Bobick, "The representation and recognition of human movement using temporal templates," in Proc. of Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 928-934, 1997.
  5. P. Dollar, V. Rabaud, G. Cottrell, S. Belongie, "Behavior recognition via sparse spatio-temporal features," in Proc. of IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, pp. 65-72, 2005.
  6. R. Gupta, A. Y.-S. Chia, R. Rajan, "Human activities recognition using depth images," in Proceedings of the 21st ACM International Conference on Multimedia' , pp. 283-292, 2013.
  7. W. Li, Z. Zhang, Z. Liu "Action recognition based on a bag of 3D points," in Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops, pp. 9-14, 2010.
  8. X. Yang, Y.L. Tian, "EigenJoints-based action recognition using Na #x00EF;ve-Bayes-Nearest-Neighbor," in Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 14-19, 2012.
  9. J. Sung, C. Ponce, B. Selman, A. Saxena, "Unstructured human activity detection from RGBD image," in Proc. of IEEE International Conference on Robotics and Automation, pp. 842-849, 2012.
  10. A. W. Vieira, E.R. Nascimento, G.L. Oliveira, Z. Liu, M.F.M. Campos, "STOP: Space-Time Occupancy Patterns for 3D Action Recognition from Depth Map Sequences," in Proc. of Iberoamerican Congress on Pattern Recognition Springer Berlin Heidelberg, pp. 252-259, 2012.
  11. X. Yang, C. Zhang, Y. Tian, "Recognizing actions using depth Motion maps-based histograms of oriented gradients," in Proc. of 'Proceedings of the 20th ACM International Conference on Multimedi, pp. 1057-1060, 2012.
  12. N. Dalal, B. Triggs, "Histograms of oriented gradients for human detection," in Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), vol. 1, pp. 886-893, 2005.
  13. C. Chen, K. Liu, N. Kehtarnavaz, "Real-time human action recognition based on depth motion maps," J. Real-Time Image Process., vol 12, no. 1, pp. 155-163, 2016. https://doi.org/10.1007/s11554-013-0370-1
  14. C. Chen, R. Jafari, N. Kehtarnavaz, "Action recognition from depth sequences using depth motion maps-based local binary patterns," in Proc. of IEEE Winter Conference on Applications of Computer Vision, pp. 1092-1099, 2015.
  15. T. Ojala, M. Pietikainen, T. Maenpaa, "Multiresolution gray-scale and rotation invariant texture classification with local binary patterns," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 7, pp. 971-987, 2002. https://doi.org/10.1109/TPAMI.2002.1017623
  16. P. Turaga, R. Chellappa, V.S. Subrahmanian, O. Udrea, "Machine recognition of human activities a survey," IEEE Trans. Circuits Syst. Video Technol., vol. 18, no. 11, pp. 1473-1488, 2008. https://doi.org/10.1109/TCSVT.2008.2005594
  17. A.V. Le, S.-W. Jung, C.S. Won, "Nonuniform video size reduction for moving objects," Sci. World J., p. e832871, 2014.
  18. J.C. Niebles, H. Wang, L. Fei-Fei, "Unsupervised learning of human action categories using spatial-temporal words," Int. J. Comput. Vis, vo. 79, no. 3, pp. 299-318, 2008. https://doi.org/10.1007/s11263-007-0122-4
  19. J. Sun, X. Wu, S. Yan, L.F. Cheong, T.S. Chua, J. Li, "Hierarchical spatio-temporal context modeling for action recognition," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2004-2011,2009.
  20. M. Raptis, S. Soatto, "Tracklet descriptors for action modeling and video analysis," in Proc. of European Conference on Computer Vision, (Springer Berlin Heidelber), pp. 577-590, 2010.
  21. I. Laptev, M. Marszalek, C. Schmid, B. Rozenfeld, "Learning realistic human actions from movies,", in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8, 2008.
  22. O. Oreifej, Z. Liu, "HON4D: Histogram of oriented 4D normals for activity recognition from depth sequences," in Proc. of Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 716-723, 2013.
  23. H. Rahmani, A. Mahmood, A, D. Q. Huynh, A. Mian, "Real time action recognition using histograms of depth gradients and random decision forests," in Proc. of IEEE Winter Conference on Applications of Computer Vision, pp. 626-633, 2014.
  24. Q.D. Tran, N.Q. Ly, "Sparse spatio-temporal representation of joint shape-motion cues for human action recognition in depth sequences," in Proc. of 'The 2013 RIVF International Conference on Computing Communication Technologies - Research, Innovation, and Vision for Future (RIVF), pp. 253-258, 2013.
  25. L. Xia, C.C. Chen, J.K. Aggarwal, "View invariant human action recognition using histograms of 3D joints, " in Proc. of '2 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 20-27, 2012.
  26. Kamal, Shaharyar, Ahmad Jalal, and Daijin Kim. "Depth images-based human detection, tracking and activity recognition using spatiotemporal features and modified HMM," J. Electr. Eng. Technol, vol. 11, pp. 1857-1862, 2016. https://doi.org/10.5370/JEET.2016.11.6.1857
  27. Jalal, Ahmad, Yeonho Kim, Shaharyar Kamal, Adnan Farooq, and Daijin Kim. "Human daily activity recognition with joints plus body features representation using Kinect sensor," in Proc. of Informatics, Electronics & Vision (ICIEV), 2015 International Conference on, pp. 1-6, 2015.
  28. Wang, J., Liu, Z., Wu, Y. and Yuan, J. "Mining actionlet ensemble for action recognition with depth cameras," in Proc. of Computer Vision and Pattern Recognition (CVPR) IEEE Conference on pp. 1290-1297, 2012.
  29. Vemulapalli, R., Arrate, F. and Chellappa, R., "Human action recognition by representing 3d skeletons as points in a lie group," in Proc. of Proceedings of the IEEE conference on computer vision and pattern recognition pp. 588-595, 2014.
  30. Y. Yang et al., "Latent max-margin multitask learning with skelets for 3D action recognition," IEEE Trans. Cybern.,vol. 47, no2, pp.439-448, 2017. https://doi.org/10.1109/TCYB.2016.2519448
  31. C. Chen R. Jafari, N. Kehtarnavaz,, "Improving human action recognition using fusion of depth camera and inertial sensors," IEEE Trans. Hum.-Mach. Syst., vo. 45, no. 1. , pp. 51-61, 2015. https://doi.org/10.1109/THMS.2014.2362520
  32. G. Stiglic, P. Kokol, "Effectiveness of rotation forest in meta-learning based gene Expression classification," in Proc. of Twentieth IEEE International Symposium on Computer-Based Medical Systems, pp. 243-250, 2007.
  33. K.-H. Liu, D.-S. Huang, "Cancer classification using rotation forest," Comput. Biol. Med., vo. 38, no. 5, pp. 601-610, 2008. https://doi.org/10.1016/j.compbiomed.2008.02.007
  34. J.J Rodriguez,L.I. Kuncheva, C.J Alonso, "Rotation Forest: A new classifier ensemble method," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 10, pp. 1619-1630, 2006. https://doi.org/10.1109/TPAMI.2006.211
  35. S. Wold, K. Esbensen, P. Geladi, "Principal component analysis," Chemom. Intell. Lab. Syst., vo. 2, no. 1, pp. 37-52, 1987. https://doi.org/10.1016/0169-7439(87)80084-9
  36. L.I. Kuncheva, J.J. Rodriguez, "An experimental study on rotation forest ensembles," in Proc. of International Workshop on Multiple Classifier Systems, (Springer Berlin Heidelberg), pp. 459-468, 2007.
  37. C.-X. Zhang, J.-S. Zhang, "RotBoost: A technique for combining rotation forest and AdaBoost," Pattern Recognit. Lett., vol. 29, no. 10, pp. 1524-1536, 2008. https://doi.org/10.1016/j.patrec.2008.03.006
  38. J.-F. Xia, K. Han, D.-S. Huang, "Sequence-based prediction of protein-protein interactions by means of rotation forest and autocorrelation descriptor," Protein Pept. Lett., vo. 17, no. 1, pp. 137-145, 2010. https://doi.org/10.2174/092986610789909403
  39. S.B. Kotsiantis, P.E. Pintelas, "Local rotation forest of decision stumps for regression problems," in Proc. of 2nd IEEE International Conference on Computer Science and Information Technology, pp. 170-174, 2009.
  40. M. Shaheryar, M. Khalid, A.M. Qamar, "Rot-SiLA: A novel ensemble classification approach based on rotation forest and similarity learning using nearest neighbor algorithm," in Proc. of 12th International Conference on Machine Learning and Applications, pp. 46-51, 2013.
  41. J. Xia, P. Du., X. He, J. Chanussot, "Hyperspectral remote sensing image classification based on rotation forest," IEEE Geosci. Remote Sens. Lett., vol. 11, no. 1, pp. 239-243, 2014. https://doi.org/10.1109/LGRS.2013.2254108
  42. M. Zanfir, M. Leordeanu, and C. Sminchisescu, "The moving pose: An efficient 3d kinematics descriptor for low-latency action recognition and detection," in Proc. of Proceedings of the IEEE International Conference on Computer Vision, pp. 2752-2759, 2013.