KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Multi-scale Diffusion-based Salient Object Detection with Background and Objectness Seeds

  • Yang, Sai (School of Electrical Engineering, Nantong University) ;
  • Liu, Fan (College of Computer and Information, Hohai University) ;
  • Chen, Juan (School of Electrical Engineering, Nantong University) ;
  • Xiao, Dibo (School of Electrical Engineering, Nantong University) ;
  • Zhu, Hairong (School of Electrical Engineering, Nantong University)
  • Received : 2017.03.31
  • Accepted : 2018.05.25
  • Published : 2018.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

The diffusion-based salient object detection methods have shown excellent detection results and more efficient computation in recent years. However, the current diffusion-based salient object detection methods still have disadvantage of detecting the object appearing at the image boundaries and different scales. To address the above mentioned issues, this paper proposes a multi-scale diffusion-based salient object detection algorithm with background and objectness seeds. In specific, the image is firstly over-segmented at several scales. Secondly, the background and objectness saliency of each superpixel is then calculated and fused in each scale. Thirdly, manifold ranking method is chosen to propagate the Bayessian fusion of background and objectness saliency to the whole image. Finally, the pixel-level saliency map is constructed by weighted summation of saliency values under different scales. We evaluate our salient object detection algorithm with other 24 state-of-the-art methods on four public benchmark datasets, i.e., ASD, SED1, SED2 and SOD. The results show that the proposed method performs favorably against 24 state-of-the-art salient object detection approaches in term of popular measures of PR curve and F-measure. And the visual comparison results also show that our method highlights the salient objects more effectively.

Keywords

References

  1. Z. Guo, L. L. Gao, J. K. Song, X. Xu, J. Shao, H. T. Shen, "Attention-based LSTM with semantic consistency for videos captioning," in Proc. of the 24th ACM Conference on Multimedia, pp.357-361, Oct.15-19, 2016.
  2. M. S. Gide, and L.J. Karam, "A locally weighted fixation density-based metric for assessing the quality of visual saliency predictions," IEEE Transactions on Image Processing, vol.25, no.8, pp.3852-3861, Aug., 2016. https://doi.org/10.1109/TIP.2016.2577498
  3. J.W.Wang, A. Borji, C. C. Jay Kuo, and L. Itti, "Learning a combined model of visual saliency for fixation prediction," IEEE Transactions on Image Processing, vol.25, no.4, pp.1566 - 1579, Apr., 2016. https://doi.org/10.1109/TIP.2016.2522380
  4. J. J. Lai, B. R. Wang, Y. M. Fang, W. S. Lin, P. L. Callet, N.Ling, and C. P. Hou, "A universal framework for salient object detection," IEEE Transactions on Multimedia, vol.18, no.9, pp.1783 - 1795, Sep., 2016. https://doi.org/10.1109/TMM.2016.2592325
  5. H. W. Peng, B. Li, H. B. Ling, W. M. Hu, W. H. Xiong, and S. J. Maybank, "Salient object detection via structured matrix decomposition," IEEE Transactions Pattern Analysis and Machine Intelligence, vol.39, no.4, pp.818-832, Apr., 2017. https://doi.org/10.1109/TPAMI.2016.2562626
  6. L. Itti, C. Koch, and E. Niebur, "A model of saliency-based visual attention for rapid scene analysis," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 11, pp. 1254-1259, Nov., 1998. https://doi.org/10.1109/34.730558
  7. Y. F. Ma, and H. J. Zhang, "Contrast-based image attention analysis by using fuzzy growing," in Proc. of the 11th ACM Conference on Multimedia, pp.374-381, Nov.2-8, 2003.
  8. R. Achanta, F. Estrada, P. Wils, and S. Susstrunk, "Salient region detection and segmentation," in Proc. of the 6th International Conference on Computer Vision Systems, pp.66-75, May.12-15, 2008.
  9. S. Goferman, L. Zelnik-Mamor, and A. Tal, "Context-aware saliency detection," in Proc. of the 23rd International Conference on Computer Vision and Pattern Recognition, pp.2376-2383, Jun.13-18, 2010.
  10. Y. Zhai, and M. Shah, "Visual attention detection in video sequences using spatiotemporal cues," in Proc. of the 14th ACM Conference on Multimedia, pp.815-824, Oct.23-27, 2006.
  11. M. M. Cheng, G. X. Zhang, N. J. Mitra, X. Huang, and S. M. Hu, "Global contrast based salient region detection," in Proc. of the 24th International Conference on Computer Vision and Pattern Recognition, pp.409-416, Jun.20-25, 2011.
  12. R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, "Frequency-tuned salient region detection," in Proc. of the 22nd Conference on Computer Vision and Pattern Recognition, pp.1597-1604, Jun.20-25, 2009.
  13. J. K. Song, L. L. Gao, M. M. Puscas, F. P. Nie, F. M. Shen, N. C. Sebe, "Joint graph learning and video segmentation via multiple cues and topology calibration," in Proc. of the 24th ACM Conference on Multimedia, pp.831-840, Oct.15-19, 2016.
  14. M. M. Cheng, J. Warrell, and W. Y. Lin, "Efficient salient region detection with soft image abstraction," in Proc. of the 14th International Conference on Computer Vision, pp.1529-1536, Dec.1-8, 2013.
  15. R. Margolin, A. Tal, and L. Zelnik-Manor, "What makes a patch distinct?," in Proc. of the 26th International Conference on Computer Vision and Pattern Recognition, pp.1139-1146, Jun.23-28,2013.
  16. H. Z. Jiang, J. D. Wang, Z. J. Yuan, and T. Liu. "Automatic salient object segmentation based on context and shape prior," in Proc. of the 14th British Machine Vision Conference, pp.111-122, Aug.29-Sep.2, 2011.
  17. Q. Yan, L. Xu, J. P. Shi, and J. Y. Jia. "Hierarchical Saliency Detection," in Proc. of the 26th International Conference on Computer Vision and Pattern Recognition, pp.1155 - 1162, Jun.23-28, 2013.
  18. N. Tong, H. C. Lu, L. H. Zhang, and X. Ruan, "Saliency detection with multi-scale superpixels," IEEE Signal Processing Letters, vol. 21, no. 9, pp.1035-1039, Sep, 2014. https://doi.org/10.1109/LSP.2014.2323407
  19. L. H. Zhang, S. F. Zhao, W. Liu, and H. C. Lu, "Saliency detection via sparse reconstruction and joint label inference in multiple features," Neurocomputing, vol. 55, no. C, pp.1-11, May., 2015.
  20. Y. L. Xie, H. C. Lu, and M. H. Yang, "Bayesian saliency via low and mid level cues," IEEE Transactions on Image Processing, vol. 22, no. 5, pp.1689-1698, May, 2013. https://doi.org/10.1109/TIP.2012.2216276
  21. P. Jiang, H. B. Ling, J. Yu, and J. L. Peng, "Salient region detection by UFO: uniqueness, focusness and objectness," in Proc. of the 14th International Conference on Computer Vision, pp.1976 - 1983, Dec.1-8, 2013.
  22. Y.Wei, F.Wen,W. Zhu, and J. Sun, " Geodesic saliency using background priors," in Proc. of the 12th European Conference on Computer Vision, pp.29-42, Oct.7-13, 2012.
  23. H. C. Lu, X. H. Li, L. H. Zhang, X. Ruan, and M. H. Yang, "Dense and sparse reconstruction error based saliency descriptor," IEEE Transaction on Image Processing, vol. 25, no. 4, pp.1592-1603, Apr., 2016. https://doi.org/10.1109/TIP.2016.2524198
  24. H. C. Lu, N. Tong, X. N. Zhang, J. Q. Qi, X. Ruan, and M. H. Yang, "Co-bootstrapping saliency,"IEEE Transaction on Image Processing, vol. 26, no. 1, pp.414-425, Jan., 2017. https://doi.org/10.1109/TIP.2016.2627804
  25. C. Jia, J. Q. Qi, X. H. Li, and H. C. Lu, "Saliency detection via a unified generative and discriminative model," Neurocomputing, vol. 173, no. P2, pp.406-417, Jan., 2016. https://doi.org/10.1016/j.neucom.2015.03.122
  26. N. Tong, H. C. Lu, L. H. Zhang, and X. Ruan, "Salient object detection via global and local cues," Pattern Recognition, vol. 48, no. 10, pp.3258-3267, Oct., 2015. https://doi.org/10.1016/j.patcog.2014.12.005
  27. W. J. Zhu, S. Liang, Y. C. Wei, and J. Sun, "Saliency optimization from robust background detection," in Proc. of the 27th International Conference on Computer Vision and Pattern Recognition, pp.2814 - 2821, Jun.24-27, 2014.
  28. J. B. Zhou, J.Y. Zhai, Y. F. Ren, A. L. Lu, "Background prior-based salient object detection via adaptive figure-ground classification," KSII Transactions on Internet and Information Systems, vol. 12, no. 3, pp. 1264-1286, Mar., 2018. https://doi.org/10.3837/tiis.2018.03.016
  29. J. P. Wang, H. C. Lu, X. H. Li, N. Tong, and W. Liu, "Saliency detection via background and foreground seed selection," Neurocomputing, vol. 152, no. C, pp. 359-368, Mar., 2015. https://doi.org/10.1016/j.neucom.2014.10.056
  30. J. G. Sun, H. C. Lu, and X. P. Liu, "Saliency region detection based on Markov absorption probabilities," IEEE Transactions on Image Processing, vol. 24, no. 5, pp. 1639-1649, May, 2015. https://doi.org/10.1109/TIP.2015.2403241
  31. B. W. Jiang, L. H. Zhang, H. C. Lu, C. Yang, and M. H. Yang, "Saliency detection via absorbing Markov chain," in Proc. of the 14th International Conference on Computer Vision, pp.1665 - 1672, Dec.1-8, 2013.
  32. J. Y. Zhai, J. B. Zhou, Y. F. Ren, Z. J. Wang, "Salient object detection via multiple random walks," KSII Transactions on Internet and Information Systems, vol. 10, no. 4, pp. 1712-1731, Apr., 2016. https://doi.org/10.3837/tiis.2016.04.014
  33. L. H. Zhang, C. Yang, H. C. Lu, X. Ruan, and M. H. Yang, " Ranking saliency," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 9, pp. 1-15, Sep., 2016. https://doi.org/10.1109/TPAMI.2016.2592468
  34. B. Alexe, T. Deselaers, and F. Vittorio, "Measuring the objectness of image windows," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 11, pp. 2189-2202, Nov., 2012. https://doi.org/10.1109/TPAMI.2012.28
  35. J. K. Song, L. L. Gao, F. P. Nie, H. T. Shen, Y. Yan, N. Sebe, "Optimized graph learning using partial tags and multiple features for image and video annotation," IEEE Transactions on Image Processing, vol. 25, no. 11, pp.4999-5011, Nov., 2016. https://doi.org/10.1109/TIP.2016.2601260
  36. L. L. Gao, J. K. Song, F. P. Nie, F. H. Zhou, N. C. Sebe, H. T. Shen, "Graph-without-cut:an ideal graph learning for image segmentation," in Proc. of the 30th AAAI Conference on Artificial Intelligence, pp.1188 - 1194, Feb.12-17, 2016.