KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Background Prior-based Salient Object Detection via Adaptive Figure-Ground Classification

  • Zhou, Jingbo (School of Computer Engineering, Nanjing Institute of Technology) ;
  • Zhai, Jiyou (School of Computer Engineering, Nanjing Institute of Technology) ;
  • Ren, Yongfeng (School of Computer Engineering, Nanjing Institute of Technology) ;
  • Lu, Ali (School of Computer Engineering, Nanjing Institute of Technology)
  • Received : 2016.11.21
  • Accepted : 2017.11.23
  • Published : 2018.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, a novel background prior-based salient object detection framework is proposed to deal with images those are more complicated. We take the superpixels located in four borders into consideration and exploit a mechanism based on image boundary information to remove the foreground noises, which are used to form the background prior. Afterward, an initial foreground prior is obtained by selecting superpixels that are the most dissimilar to the background prior. To determine the regions of foreground and background based on the prior of them, a threshold is needed in this process. According to a fixed threshold, the remaining superpixels are iteratively assigned based on their proximity to the foreground or background prior. As the threshold changes, different foreground priors generate multiple different partitions that are assigned a likelihood of being foreground. Last, all segments are combined into a saliency map based on the idea of similarity voting. Experiments on five benchmark databases demonstrate the proposed method performs well when it compares with the state-of-the-art methods in terms of accuracy and robustness.

Keywords

References

  1. L. Itti, C. Koch and E. Niebur, "A model of saliency-based visual attention for rapid scene analysis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol.20, no.11, pp.1254-1259, 1998. https://doi.org/10.1109/34.730558
  2. L. Itti and C. Koch, "Computational modelling of visual attention," Nature reviews neuroscience, vol. 2, pp.194-203, 2001. https://doi.org/10.1038/35058500
  3. A. Borji, D. N. Sihite, and L. Itti, "Salient object detection: A benchmark," In Proc. of ECCV, pp. 414-429, 2012.
  4. M. Cheng, G. Zhang, N. Mitra, X. Huang, and S. Hu, "Global contrast based salient region detection," in Proc. of IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp.409-416, 2011.
  5. V. Mahadevan and N. Vasconcelos, "Saliency-based discriminant tracking," in Proc. of IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp.1007-1013, 2009.
  6. Y. Ding, J. Xiao, and J. Yu, "Importance filtering for image retargeting," in Proc. of IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp.89-96, 2011.
  7. J. Sun and H. Ling, "Scale and object aware image retargeting for thumbnail browsing," in Proc. of 13th IEEE International Conference on Computer Vision (ICCV), pp.1511-1518, 2011.
  8. C. Siagian and L. Itti, "Rapid biologically-inspired scene classification using features shared with visual attention," IEEE Trans. Pattern Analysis and Machine Intelligence, vol.29, no.2, pp.300-312, 2007. https://doi.org/10.1109/TPAMI.2007.40
  9. C. Rother, V. Kolmogorov, and A. Blake, "Grabcut: Interactive foreground extraction using iterated graph cuts," In ACM Transactions on Graphics (TOG), vo.23, pp.309-314, 2004. https://doi.org/10.1145/1015706.1015720
  10. N. Bruce and J. Tsotsos, "Saliency based on information maximization," Advances in Neural Information Processing Systems, pp.155-162, 2006.
  11. H. Jiang, J. Wang, Z. Yuan, T. Liu, N. Zheng, and S. Li, "Automatic salient object segmentation based on context and shape prior," in Proc. of BMVC, 2011.
  12. X. Hou and L. Zhang, "Saliency detection: A spectral residual approach," in Proc. of IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp.1-8, 2007.
  13. C. Guo, Q. Ma, and L. Zhang, "Spatio-temporal saliency detection using phase spectrum of quaternion fourier transform," in Proc. of IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 1-8, 2008.
  14. F. Perazzi, P. Krahenbuhl, Y. Pritch, and A. Hornung, "Saliency filters: Contrast based filtering for salient region detection," in Proc. of IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 733-740, 2012.
  15. H. Jiang, J. Wang, Z. Yuan, Y. Wu, N. Zheng, and S. Li, "Salient object detection: A discriminative regional feature integration approach," in Proc. of IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 2083-2090, 2013.
  16. Y. Wei, F.Wen,W. Zhu, and J. Sun, "Geodesic saliency using background priors," in Proc. of ECCV, 2012.
  17. C. Yang, L. Zhang, H. Lu, X. Ruan and M. Yang, "Saliency detection via graph-based manifold ranking," in Proc. of IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp.3166-3173, 2013.
  18. Y.Chen, A. Chan, "Enhanced figure-ground classification with background prior propagation," IEEE Transactions on Image Processing, vol.24, no.3, pp.873-885, 2015. https://doi.org/10.1109/TIP.2015.2389612
  19. R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Susstrunk, "SLIC superpixels compared to state-of-the-art superpixel methods," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 11, pp. 2274-2282, Nov., 2012. https://doi.org/10.1109/TPAMI.2012.120
  20. Y. Lin, Y. Y. Tang, B. Fang, Z. Shang, Y. Huang, and S. Wang, "A visual-attention model using earth mover's distance-based saliency measurement and nonlinear feature combination," IEEE Trans. Pattern Analysis and Machine Intelligence, vol.35, no.2, pp.314-328, 2013. https://doi.org/10.1109/TPAMI.2012.119
  21. A. Borji and L. Itti, "State-of-the-art in visual attention modeling," IEEE Trans. Pattern Analysis and Machine Intelligence, vol.35, no.1, pp.185-207, 2013. https://doi.org/10.1109/TPAMI.2012.89
  22. D. Walther and C. Koch, "Modeling attention to salient proto-objects," Neural Networks, vol.19, no.9, pp.1395-1407, 2006. https://doi.org/10.1016/j.neunet.2006.10.001
  23. W. Wang, Y. Wang, Q. Huang, and W. Gao, "Measuring visual saliency by site entropy rate," in Proc. of IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 2368-2375, 2010.
  24. B. Jiang, L. Zhang, H. Lu, C. Yang, and M. Yang, "Saliency detection via absorbing markov chain," in Proc. of 14th IEEE International Conference on Computer Vision (ICCV), pp.1665-1672, 2013.
  25. X. Li, Y. Li, C. Shen, A. Dick, and A. Hengel, "Contextual hypergraph modeling for salient object detection," in Proc. of 13th IEEE International Conference on Computer Vision (ICCV), pp.3328-3335, 2013.
  26. M. Cheng, J.Warrell,W. Lin, et al, "Efficient salient region detection with soft image abstraction," in Proc. of 13th IEEE International Conference on Computer Vision (ICCV), pp.1529-1536, 2013.
  27. C. Scharfenberger, A. Wong, K. Fergani, J. Zelek, and D. Clausi, "Statistical textural distinctiveness for salient region detection in natural images," in Proc. of IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp.979-986, 2013.
  28. Y. Xie, H. Lu, and M. Yang, "Bayesian saliency via low and mid-level cues," IEEE Transactions on Image Processing, vol.34, no.11, pp.1689-1698, 2013.
  29. X. Shen and Y. Wu, "A unified approach to salient object detection via low rank matrix recovery," in Proc. of IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp.853-860, 2012.
  30. W. Zou, K. Kpalma, Z. Liu, and J. Ronsin, "Segmentation driven low-rank matrix recovery for saliency detection," in Proc. of BMVC, 2013.
  31. R. Liu, J. Cao, Z. Lin, and S. Shan, "Adaptive partial differential equation learning for visual saliency detection," in Proc. of IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp.3866-3873, 2014.
  32. J. Kim, D. Han, Y.-W. Tai, and J. Kim, "Salient region detection via high-dimensional color transform," in Proc. of IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp.883-890, 2014.
  33. T. Liu, Z. Yuan, J. Sun, et.al, "Learning to detect a salient object," IEEE Trans. Pattern Analysis and Machine Intelligence, vol.33, no.2, pp.353-367, 2011. https://doi.org/10.1109/TPAMI.2010.70
  34. S. Lu, V. Mahadevan, and N. Vasconcelos, "Learning optimal seeds for diffusion-based salient object detection," in Proc. of IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp.2790-2797, 2014.
  35. Z. Jiang and L. Davis, "Submodular salient region detection," in Proc. of IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp.2043-2050, 2013.
  36. W. Zhu, S. Liang, Y. Wei, and J. Sun, "Saliency optimization from robust background detection," in Proc. of IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp.2814-2821, 2014.
  37. P. Jiang, H. Ling, J. Yu, and J. Peng, "Salient region detection by UFO: Uniqueness, Focusness and Objectness," in Proc. of 13th IEEE International Conference on Computer Vision (ICCV), pp.1976-1983, 2013.
  38. N. Li, J. Ye, Y. Ji, H. Ling, and J. Yu, "Saliency detection on light field," in Proc. of IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp.1605-1616, 2014.
  39. K. Chang, T. Liu, H. Chen and S. Lai, "Fusing generic objectness and visual saliency for salient object detection," in Proc. of 12th IEEE International Conference on Computer Vision (ICCV), pp.914-921, 2011.
  40. X. Li, H. Lu, L. Zhang, X. Ruan, and M.Yang, "Saliency detection via dense and sparse reconstruction," in Proc. of 13th IEEE International Conference on Computer Vision (ICCV), pp.2976-2983, 2013.
  41. Q. Yan, L. Xu, J. Shi, and J. Jia, "Hierarchical saliency detection," in Proc. of IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp.1155-1162, 2013.
  42. B. Alexe, T. Deselaers, V.Ferrari, "Measuring the objectness of image windows," IEEE Trans. Pattern Analysis and Machine Intelligence, vol.34, no.11, pp.2189-2202, 2012. https://doi.org/10.1109/TPAMI.2012.28
  43. Z. Zhang, J. Warrell, P. Torr, "Proposal generation for object detection using cascaded ranking SVMs," in Proc. of IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp.1497-1504, 2011.
  44. M. Cheng, Z. Zhang, W. Lin, et al, "BING: Binarized normed gradients for objectness estimation at 300fps," in Proc. of IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp.3286-3293, 2014.
  45. V. Gopalakrishnan, Y. Hu, and D. Rajan, "Random walks on graphs for salient object detection in images," IEEE Transactions on Image Processing, vol. 19, no. 12, pp. 3232-3242, 2010. https://doi.org/10.1109/TIP.2010.2053940
  46. N. Tong, H. Lu, L. Zhang, X. Ruan., M. Yang, "Salient object detection via bootstrap learning," in Proc. of IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 1884-1892, 2015.
  47. D.Gao, V.Mahadevan, and N.Vasconcelos, "On the plausibility of the discriminant center-surround hypothesis for visual saliency," Journal of Vision, vol.8, no.7, pp.1-18, 2008.
  48. J. Harel, C. Koch, P. Perona, "Graph-based visual saliency," Advances in Neural Information Processing Systems, pp.545-552, 2006.
  49. D. Martin, C. Fowlkes, J. Malik, "Learning to detect natural image boundaries using local brightness, color, and texture cues," IEEE Trans. Pattern Analysis and Machine Intelligence, vol.26, no.5, pp.530-549, 2004. https://doi.org/10.1109/TPAMI.2004.1273918
  50. N. OTSU, "A threshold selection method from gray level histograms," IEEE Transactions on Systems Man & Cybernetics, vol.9, no.1, pp.62-66, 1979. https://doi.org/10.1109/TSMC.1979.4310076
  51. S. Rao, H. Mobahi, A.Yang, S. Sastry, and Y. Ma, "Natural image segmentation with adaptive texture and boundary encoding," In Proc. ACCV, pp.135-146, 2009.
  52. T. M. Cover and J. A. Thomas, "Elements of Information Theory (Telecommunications)," New York, NY, USA: Wiley, 1991.
  53. J. Shi and J. Malik, "Normalized cuts and image segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp.888-905, Aug. 2000. https://doi.org/10.1109/34.868688
  54. R. Horn and C. Johnson, "Matrix Analysis," Cambridge, U.K.: Cambridge Univ. Press, 1985.
  55. H. Li, H. Lu, Z. Lin, et al., "Inner and inter label propagation: salient object detection in the wild," IEEE Transactions on Image Processing, vol. 24, no.10, pp.3176-3186, 2015. https://doi.org/10.1109/TIP.2015.2440174
  56. N. Tong, H. Lu, L. Zhang, X. Ruan, "Saliency detection with multi-scale superpixels," IEEE Signal Processing Letters, vol. 21, no. 9, pp.1035-1039, 2014. https://doi.org/10.1109/LSP.2014.2323407
  57. Qin Y, Lu H, Xu Y, et al, "Saliency detection via cellular automata," in Proc. of IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp.110-119, 2015.
  58. P. Jiang, N. Vasconcelos, J. Peng, "Generic promotion of diffusion-based salient object detection," in Proc. of 14th IEEE International Conference on Computer Vision (ICCV), pp.217-225, 2015.
  59. N. Tong, H. Lu, Y. Zhang, et al, "Salient object detection via global and local cues," Pattern Recognition, vol.48, no.10, pp.3258-3267, 2015. https://doi.org/10.1016/j.patcog.2014.12.005
  60. S. Alpert, M. Galun, R. Basri, and A. Brandt, "Image segmentation by probabilistic bottom-up aggregation and cue integration," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 2, pp.315-327, 2011. https://doi.org/10.1109/TPAMI.2011.130
  61. D. Martin, C. Fowlkes, D. Tal, et al, "A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics," in Proc. of 7th IEEE International Conference on Computer Vision (ICCV), pp.416-423, 2001.
  62. M. Cheng, N. Mitra, X. Huang, P. Torr, et al, "Global contrast based salient region detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, no. 3, pp.569-582, 2015. https://doi.org/10.1109/TPAMI.2014.2345401

Cited by

  1. Multi-scale Diffusion-based Salient Object Detection with Background and Objectness Seeds vol.12, pp.10, 2018, https://doi.org/10.3837/tiis.2018.10.019