Journal of Korea Multimedia Society (한국멀티미디어학회논문지)

Korea Multimedia Society (한국멀티미디어학회)
권호 표지
DOI QR코드

DOI QR Code

Simple Online Multiple Human Tracking based on LK Feature Tracker and Detection for Embedded Surveillance

  • Received : 2017.02.15
  • Published : 2017.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose a simple online multiple object (human) tracking method, LKDeep (Lucas-Kanade feature and Detection based Simple Online Multiple Object Tracker), which can run in fast online enough on CPU core only with acceptable tracking performance for embedded surveillance purpose. The proposed LKDeep is a pragmatic hybrid approach which tracks multiple objects (humans) mainly based on LK features but is compensated by detection on periodic times or on necessity times. Compared to other state-of-the-art multiple object tracking methods based on 'Tracking-By-Detection (TBD)' approach, the proposed LKDeep is faster since it does not have to detect object on every frame and it utilizes simple association rule, but it shows a good object tracking performance. Through experiments in comparison with other multiple object tracking (MOT) methods using the public DPM detector among online state-of-the-art MOT methods reported in MOT challenge [1], it is shown that the proposed simple online MOT method, LKDeep runs faster but with good tracking performance for surveillance purpose. It is further observed through single object tracking (SOT) visual tracker benchmark experiment [2] that LKDeep with an optimized deep learning detector can run in online fast with comparable tracking performance to other state-of-the-art SOT methods.

Keywords

References

  1. Multiple Object Tracking Benchmark, https://motchallenge.net/ (accessed May, 20, 2017).
  2. Visual Tracker Benchmark Results, https://github.com/foolwood/benchmark_results (accessed May, 20, 2017).
  3. H. Yanga, L. Shaoa, F. Zhenga, L. Wangd, and Z. Songa, "Recent Advances and Trends in Visual Tracking: A Review," Journal of Neurocomputing, Vol. 74, No. 18, pp. 3823-3831, 2011. https://doi.org/10.1016/j.neucom.2011.07.024
  4. A. Smeulders, D. Chu, R. Cucchiara, S. Calderena, A. Dehghan, and M. Shah, "Visual Tracking: An Experimental Survey," Journal of IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 36, No. 7, pp. 1442-1468, 2014. https://doi.org/10.1109/TPAMI.2013.230
  5. W. Luo, X. Zhao, and T. Kim, "Multiple Object Tracking: A Literature Review," arXiv Preprint 1409.7618, 2014.
  6. Y. Wu, J. Lim, and M. Yang, "Object Tracking Benchmark," Journal of IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 37, No. 9, pp. 1834-1848, 2015. https://doi.org/10.1109/TPAMI.2014.2388226
  7. A. Milan, L. Leal-Taixe, I. Reid, S. Roth, and K. Schindler, "Multiple Object Tracking Benchmark16: A Benchmark for Multi-Object Tracking," arXiv preprint 1603.00831, 2016.
  8. Visual Tracker Benchmark, http://www.visual-tracking.net (accessed May, 20, 2017).
  9. H.T. Nguyen and A.W.M. Smeulders, "Fast Occluded Object Tracking by a Robust Appearance Filter," Journal of IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 26, No. 8, pp. 1099-1104, 2004. https://doi.org/10.1109/TPAMI.2004.45
  10. D.Y. Kim, J.W. Park, and C.W. Lee, "Object-Tracking System Using Combination of CAMshift and Kalman Filter Algorithm," Journal of Korea Multimedia Society, Vol. 6, No. 5, pp. 619-628, 2013.
  11. B. Lucas and T. Kanade, "An Iterative Image Registration Technique with an Application to Stereo Vision," Proceeding of International Joint Conference on Artificial Intelligence, pp. 674-679, 1981.
  12. B. Ristic, S. Arulampalam, and N. Gordon, Beyond the Kalman Filter: Particle Filters for Tracking Applications, Artech House, Boston, Massachusetts, USA, 2003.
  13. D. Comaniciu, V. Ramesh, and P. Meer, "Real-Time Tracking of Non-Rigid Objects Using Mean Shift," Proceeding of IEEE Conference on Computer Vision and Pattern Recognition, pp. 142-149, 2000.
  14. Z. Chen, Z. Hong, and D. Tao, "An Experimental Survey on Correlation Filter-Based Tracking," arXiv Preprint 1509.05520, 2015.
  15. Yang and G. Bilodeau, "Multi-Kernel Correlation Filter for Visual Tracking," arXiv Preprint 1611.02364, 2016.
  16. L. Zhang, Y. Li, and R. Nevatia, "Global Data Association for Multi-Object Tracking Using Network Flows," Proceeding of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8, 2008.
  17. A. Milan, S. Roth, and K. Schindler, "Continuous Energy Minimization for Multitarget Tracking," Journal of IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 36, No. 1, pp. 58-72, 2014. https://doi.org/10.1109/TPAMI.2013.103
  18. B. Yang and R. Nevatia, "Multi-Target Tracking by Online Learning of Non-Linear Motion Patterns and Robust Appearance models," Proceeding of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1918-1925, 2012.
  19. D.B. Reid, "An Algorithm for Tracking Multiple Targets," Journal of IEEE On Automatic Control, Vol. 24, No. 6, pp. 843-854, 1979. https://doi.org/10.1109/TAC.1979.1102177
  20. T.E. Fortmann, Y. Bar-Shalom, and M. Scheffe, "Sonar Tracking of Multiple Targets Using Joint Probabilistic Fata Association," Journal of IEEE Oceanic Engineering Society, Vol. 8, No. 3, pp. 173-184, 1983. https://doi.org/10.1109/JOE.1983.1145560
  21. C. Kim, F. Li, A. Ciptadi, and J.M. Rehg, "Multiple Hypothesis Tracking Revisited," Proceeding of IEEE International Conference on Computer Vision, pp. 4696-4704, 2015.
  22. S.H. Rezatofighi, A. Milan, Z. Zhang, Q. Shi, A. Dick, and I. Reid, "Joint Probabilistic Data Association Revisited," Proceeding of IEEE International Conference on Computer Vision, pp. 3047-3055, 2015.
  23. H.K. Galoogahi, A. Fagg, C. Huang, D. Ramanan, and S. Lucey, "Need for Speed: A Benchmark for Higher Frame Rate Object Tracking," arXiv Preprint 1703.05884, 2017.
  24. Z. Kalal, K. Mikolajczyk, and J. Matas, "Forward-Backward Error: Automatic Detection of Tracking Failures," Proceeding of International Conference on Pattern Recognition, pp. 23-26, 2010.
  25. J.Y. Bouguet, "Pyramidal Implementation of the Affine Lucas-Kanade Feature Tracker Description of the Algorithm," Journal of Intel Corporation, Vol. 1, No. 2, pp. 1-9, 2001.
  26. E. Rosten and T. Drummond, "Fusing Points and Lines for High Performance Tracking," Proceeding of IEEE International Conference on Computer Vision, pp. 1508-1515, 2015.
  27. E. Rosten and T. Drummond, "Machine Learning for High-Speed Corner Detection," Proceeding of European Conference on Computer Vision, pp. 430-443, 2006.
  28. J. Shi, and C. Tomasi, "Good Feature To Track," Proceeding of IEEE Conference on Computer Vision and Pattern Recognition, pp. 593-600, 1994.
  29. P.F. Felzenszwalb, R.B. Girshick, D. McAllester, and D. Ramanan, "Object Detection with Discriminatively Trained Part Based Models," Journal of IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 32, No. 9, pp. 1627-1645, 2010. https://doi.org/10.1109/TPAMI.2009.167
  30. C. Zhang, J. Xu, A. Beaugendre, and S. Goto, "A KLT-Based Approach for Occlusion Handling in Human Tracking," Proceeding of Picture Coding Symposium, pp. 337-340, 2012.
  31. J. Redmon and A. Farhadi, "YOLO9000: Better, Faster, Stronger," arXiv Preprint 1612.08242, 2016.
  32. H. Li, Y. Li, and F. Porikli, "DeepTrack: Learning Discriminative Feature Representations Online for Robust Visual Tracking," arXiv preprint 1503.00072, 2015.
  33. A. Bewley, G. Zongyuan, F. Ramos, and B. Upcroft, "Simple Online and Real-Time Tracking," Proceeding of IEEE International Conference on Image Processing, pp. 3464-3468, 2016.
  34. R. Kalman, "A New Approach to Linear Filtering and Prediction Problems," Journal of Basic Engineering, Vol. 82, No. Series D, pp. 35-45, 1960. https://doi.org/10.1115/1.3662552
  35. S. Ren, K. He, R. Girshick, and J. Sun, "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks," Proceeding of Neural Information Processing Systems, pp. 91-99, 2015.
  36. N. Wojke, A. Bewley, and D. Paulus, "Simple Online and Real-Time Tracking with a Deep Association Metric," arXiv Preprint 1703.07402, 2017.
  37. F. Yu, W. Li, Q. Li, Y. Liu, X. Shi, and J. Yan, "POI: Multiple Object Tracking with High Performance Detection and Appearance Feature," arXiv: 1610.06136, 2016.
  38. H.W. Kuhn, "The Hungarian Method for the Assignment Problem," Journal of Naval Research Logistics Quarterly, Vol. 2, No. 1, pp. 83-97, 1955. https://doi.org/10.1002/nav.3800020109
  39. Y. Liu, J. Yan, and W. Ouyang, "Quality Aware Network for Set to Set Recognition," arXiv preprint 1704.03373, 2017.
  40. R. Sanchez-Matilla, F. Poiesi, and A. Cavallaro, "Online Multi-Target Tracking with Strong and Weak Detections," Proceeding of Benchmarking Multi-target Tracking, European Conference on Computer Vision, pp. 84-99, 2016.
  41. A. Sadeghian, A. Alahi, and S. Savarese, "Tracking The Untrackable: Learning To Track Multiple Cues with Long-Term Dependencies," arXiv Preprint 1701.01909, 2017.
  42. S. Bae and K. Yoon, "Confidence-Based Data Association and Discriminative Deep Appearance Learning for Robust Online Multi-Object Tracking," Journal of IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 99, 2017.
  43. H. Kieritz, sS. Becker, W. Hubner, and M. Arens, "Online Multi-Person Tracking Using Integral Channel Features," Proceeding of IEEE Conference on Advanced Video and Signal-based Surveillance, pp. 122-130, 2016.
  44. Y. Ban, S. Ba, X. Alameda-Pineda, and R. Horaud, "Tracking Multiple Persons Based on a Variational Bayesian Model," Proceeding of Benchmarking Multi-Target Tracking, pp. 52-67, 2016.
  45. Y. Song and M. Jeon, "Online Multiple Object Tracking with the Hierarchically Adopted GM-PHD Filter Using Motion and Appearance," Proceeding of IEEE International Conference on Consumer Electronics-Asia, pp. 256-259, 2016.
  46. P. Dollar, C. Wojek, B. Schiele, and P. Perona, "Pedestrian Detection: An Evaluation of the State of the Art," Journal of IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 34, No. 4, pp. 743-761, 2011.
  47. R. Stiefelhagen, K. Bernardin, R. Bowers, J. S. Garofolo, D. Mostefa, and P. Soundararajan, "The Clear 2006 Evaluation," Proceeding of Classification of Events, Activities and Relationships, pp. 1-44, 2006.
  48. MOT16 Result, https://motchallenge.net/results/MOT16/?det=All (accessed May, 20, 2017).