KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Salient Region Extraction based on Global Contrast Enhancement and Saliency Cut for Image Information Recognition of the Visually Impaired

  • Received : 2017.06.21
  • Accepted : 2018.01.18
  • Published : 2018.05.31
Download PDF
⟨ Previous Next ⟩

Abstract

Extracting key visual information from images containing natural scene is a challenging task and an important step for the visually impaired to recognize information based on tactile graphics. In this study, a novel method is proposed for extracting salient regions based on global contrast enhancement and saliency cuts in order to improve the process of recognizing images for the visually impaired. To accomplish this, an image enhancement technique is applied to natural scene images, and a saliency map is acquired to measure the color contrast of homogeneous regions against other areas of the image. The saliency maps also help automatic salient region extraction, referred to as saliency cuts, and assist in obtaining a binary mask of high quality. Finally, outer boundaries and inner edges are detected in images with natural scene to identify edges that are visually significant. Experimental results indicate that the method we propose in this paper extracts salient objects effectively and achieves remarkable performance compared to conventional methods. Our method offers benefits in extracting salient objects and generating simple but important edges from images containing natural scene and for providing information to the visually impaired.

Keywords

References

  1. Z. Liu, O. Le Meur and Sh. Luo, "Superpixel-based saliency detection," in Proc. of 14th Int. Workshop on Image and Audio Analysis for Multimedia Interactive Services, pp. 1-4, July 3-5, 2013.
  2. M. M. Cheng, G. X. Zhang, N. J. Mitra, X. Huang and Sh. M. Hu, "Global contrast based salient region detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, no. 3, pp. 569-582, March, 2015. https://doi.org/10.1109/TPAMI.2014.2345401
  3. Y. F. Ma and H. J. Zhang, "Contrast-based Image Attention Analysis by Using Fuzzy Growing," in Proc. of the 11th ACM Int. Conf. on Multimedia, pp. 374-381, November 2-8, 2003.
  4. T. Zhao, L. Li, X. Ding, Y. Huang and D. Zeng, "Saliency Detection with Spaces of Background-Based Distribution," IEEE Signal Processing Letters, vol. 23, no. 5, pp. 683-687, May, 2016. https://doi.org/10.1109/LSP.2016.2544781
  5. A. Borji and L. Itti, "State-of-the-art in visual attention modeling," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 1, pp. 185-207, January, 2012. https://doi.org/10.1109/TPAMI.2012.89
  6. M. Donoser, M. Urschler, M. Hirzer and H. Bischof, "Saliency Driven Total Variation Segmentation," in Proc. of IEEE 12th Int. Conf. on Computer Vision, pp. 814-824, September 29-October 2, 2009.
  7. G. Li, Y. Xie, L. Lin and Y. Yu, "Instance-Level salient object segmentation," arXiv preprint, April 12, 2017.
  8. Y. Gao, M. Shi, D. Tao and Ch. Xu, "Database Saliency for Fast Image Retrieval," IEEE Transactions on Multimedia, vol. 17, no. 3, pp. 359-369, March, 2015. https://doi.org/10.1109/TMM.2015.2389616
  9. X. Wei, Z. Tao, Ch. Zhang and X. Cao, "Structured Saliency Fusion Based on Dempster-Shafer Theory," IEEE Signal Processing Letters, vol. 22, no. 9, pp. 1345-1349, September, 2015. https://doi.org/10.1109/LSP.2015.2399621
  10. M. Wang, R. Hong, X. T. Yuan, Sh. Yan and T. S. Chua, "Movie2Comics: Towards a Lively Video Content Presentation," IEEE Transactions on Multimedia, vol. 14, no. 3, pp. 858-870, June, 2012. https://doi.org/10.1109/TMM.2012.2187181
  11. M. Rabbani and R. Joshi, "An overview of the JPEG2000 still image compression standard," Signal Processing: Image Communication, vol. 17, no. 1, pp. 3-48, January, 2002. https://doi.org/10.1016/S0923-5965(01)00024-8
  12. R. Gallea, E. Ardizzone and R. Pirrone, "Physical Metaphor for Streaming Media Retargeting," IEEE Transactions on Multimedia, vol. 16, no. 4, pp. 971-979, June, 2014. https://doi.org/10.1109/TMM.2014.2305917
  13. U. Rutishauser, D. Walther, Ch. Koch and P. Perona, "Is bottom-up attention useful for object recognition?," in Proc. of IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, vol. 2, pp. II-37-II-44, June 27-July 2, 2004.
  14. J. Lei, C. Zhang, Y. Fang, Z. Gu, N. Ling and Ch. Hou, "Depth Sensation Enhancement for Multiple Virtual View Rendering," IEEE Transactions on Multimedia, vol. 17, no. 4, pp. 457-469, April, 2015. https://doi.org/10.1109/TMM.2015.2400823
  15. G. X. Zhang, M. M. Cheng, Sh. M. Hu and R. Martin, "A Shape-Preserving Approach to Image Resizing," Journal compilation:The Eurographics Association and Blackwell Publishing Ltd, vol. 28, no. 7, pp. 1897-1906, Octobor,2009.
  16. M. M. Cheng, F. L. Zhang, N. J. Mitra, X. Huang and Sh.-M. Hu, "RepFinder: Finding Approximately Repeated Scene Elements for Image Editing," ACM Transactions on Graphics, vol. 29, no. 4, pp. 83, July, 2010. https://doi.org/10.1145/1778765.1778820
  17. R. Hong, M. Wang, G. Li and X. T. Yuan, "iComics. Automatic conversion of movie into comics," in Proc. of 18th ACM Int. Conf. on Multimedia, pp. 1599-1602, October 25-29, 2010.
  18. R. Hong, J. He, H. Zhang and T. S. Chua, "Mental visual indexing: Towards Fast Video Browsing," in Proc. of the 2016 ACM on Multimedia Conf., pp. 621-625, October 15-19, 2016.
  19. L. Nie, R. Hong, L. Zhang, Y. Xia, D. Tao and N. Sebe, "Perceptual attributes optimization for multivideo summarization," IEEE Transactions on Cybernetics, vol. 46, no. 12, pp. 2991-3003, December, 2016. https://doi.org/10.1109/TCYB.2015.2493558
  20. Ch. Cheng, Sh. Li, Y. Wang, H. Qin and A. Hao, "Video saliency detection via spatial-temporal fusion and low-rank coherency diffusion," IEEE Transactions on Image Processing, February, 2017.
  21. Q. Yan, L. Xu, J. Shi and J. Jia, "Hierarchical Saliency Detection," in Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp. 1155-1162, June 23-28, 2013.
  22. J. Jungil, Y. Hongchan, L. Hyelim and C. Jinsoo, "Graphic haptic electronic board-based education assistive technology system for blind people," in Proc. of IEEE Int. Conf. on Consumer Electronics, pp. 364-365, January 9-12, 2015.
  23. R. S. Srivatsa and R. V. Babu, "Salient object detection via objectness measure," in Proc. of IEEE Int. Conf. on Image Processing, pp. 4481-4485, September 27-30, 2015.
  24. J. Shi, Q. Yan, L. Xu and J. Jia, "Hierarchical Image Saliency Detection on Extended CSSD," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 4, pp. 717-729, April, 2016. https://doi.org/10.1109/TPAMI.2015.2465960
  25. C. Koch and S. Ullman, "Shifts in selective visual attention: towards the underlying neural circuitry," Human Neurbiol, vol. 4, no. 4, pp. 219-227, 1985.
  26. L. Itti, C. Koch and E. Niebur, "A model of saliency-based visual attention for rapid scene analysis," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 11, pp. 1254-1259, November, 1998. https://doi.org/10.1109/34.730558
  27. J. Harel, C. Koch and P. Perona, "Graph-based visual saliency," in Proc. of 19th Int. Conf. on Neural Information Processing Systems, pp. 545-552, December 4-7, 2006.
  28. R. Achanta, S. Hemami, F. Estrada and S. Susstrunk, "Frequency-tuned salient region detection," in Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp. 1597-1604, June 20-25, 2009.
  29. M. Du, X. Wu, W. Chen and J. Wang, "Fusing Region Contrast and Graph Regularization for Saliency Detection," in Proc. of 2015 27th Chinese Control and Decision Conf., pp. 5789-5794, May 23-25, 2015.
  30. T. Liu, J. Sun, N. N. Zheng, X. Tang and H. Y. Shum, "Learning to Detect A Salient Object," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 2, pp. 353-367, February, 2011. https://doi.org/10.1109/TPAMI.2010.70
  31. P. Felzenszwalb and D. Huttenlocher, "Efficient graph-based image segmentation," Int. Journal of Computer Vision, vol. 59, no. 2, pp. 167- 181, September, 2004. https://doi.org/10.1023/B:VISI.0000022288.19776.77
  32. D. Comaniciu and P. Meer, "Mean shift: A robust approach toward feature space analysis," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 603-619, May, 2002. https://doi.org/10.1109/34.1000236
  33. R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua and S. Susstrunk, "SLIC Superpixels," EPFL, no. Technical Report 149300, June, 2010.
  34. J. Lou, M. Ren and H. Wang, "Regional principal color based saliency detection," PLoS ONE, vol. 9, no. 11, pp. 1-13, November, 2014.
  35. F. Perazzi, P. Krahenbuhl, Y. Pritch and A. Hornung, "Saliency filters: Contrast based filtering for salient region detection," in Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp. 733-740, June 16-21, 2012.
  36. M. Hua, X. Bie and W. Wang, "Contrast based saliency detection using representative background priors," in Proc. IEEE Int. Conf. on Image Processing, pp. 3422-3425, September 15-18, 2013.
  37. J. Han, D. Zhang, X. Hu, L. Guo, J. Ren and F. Wu, "Background Prior-Based Salient Object Detection via Deep Reconstruction Residual," IEEE Transactions on Circuits and Systems for Video Technology, vol. 25, no. 8, August, 2015.
  38. Q. Jiang, F. Shao and G. Jiang, "Efficient saliency detection using color contrast and similarity distribution information," in Proc. of Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, December 9-12, 2014.
  39. Q. Zhou, J. Chen, Sh. Ren, Y. Zhou, J. Chen and W. Liu, "On Contrast Combinations for Visual Saliency Detection," in Proc. of IEEE Int. Conf. on Image Processing, pp. 2665-2669, September 15-18, 2013.
  40. J. Zhou, J. Zhai and Y. Ren, "Salient Object Detection via Adaptive Region Merging," KSII Transactions on Internet and Information Systems, vol. 10, no. 9, pp. 4386-4404, September, 2016. https://doi.org/10.3837/tiis.2016.09.020
  41. X. Qian, J. Han, G. Cheng and L. Guo, "Optimal contrast based saliency detection," Pattern Recognition Letters, vol. 34, no. 11, pp. 1270-1278, August 1, 2013. https://doi.org/10.1016/j.patrec.2013.04.009
  42. X. Hou and L. Zhang, "Saliency detection: A spectral residual approach," in Proc. of IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, June 17-22, 2007.
  43. C. Guo and L. Zhang, "A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression," IEEE Transactions on Image Processing, vol. 19, no. 1, pp. 185-198, January, 2010. https://doi.org/10.1109/TIP.2009.2030969
  44. C. Rother, V. Kolmogorov, and A. Blake, "GrabCut: Interactive foreground extraction using iterated graph cuts," Association for Computing Machinery Transactions on Graphics, vol. 23, no. 3, pp. 309-314, August, 2004.
  45. S. Li, R. Ju, T. Ren and G. Wu, "Saliency cuts based on adaptive triple thresholding," in Proc. of IEEE Int. Conf. on Image Processing, pp. 4609-4613, September 27-30, 2015.
  46. P. Mehrani and O. Veksler, "Saliency Segmentation based on Learning and Graph Cut Refinement," in Proc. of British Machine Vision Conf., pp. 1-12. August 31-September 3, 2010.
  47. Y. Fu, J. Cheng, Z. Li, and H. Lu, "Saliency Cuts: An automatic approach to object segmentation," in Proc. of 19th Int. Conf. Pattern Recognition, pp. 1-4, December 8-11, 2008.
  48. C. Aytekin, E. C. Ozan, S. Kiranyaz and M. Gabbouj, "Visual saliency by extended quantum cuts," in Proc. of Int. Conf. on Image Processing, pp. 1692-1696, September 27-30, 2015.
  49. J. Peng, J. Shen, Y. Jia and X. Li, "Saliency Cut in Stereo Images," in Proc. of IEEE Int. Conf. on Computer Vision Workshops, pp. 22-28, December 2-8, 2013.
  50. L. Chen, B. Guo and W. Sun, "Obstacle Detection System for Visually Impaired People Based on Stereo Vision," in Proc. of IEEE 4th Int. Conf. on Genetic and Evolutionary Computing, pp. 723-726, December 13-15, 2010.
  51. L. Muthulakshmi and A. B. Ganesh, "Bimodal Based Environmental Awareness System for Visually Impaired People," in Proc. of Int. Conf. on modeling, optimization and computing, vol. 38, pp. 1132-1137, 2012.
  52. S. Wang, X. Yang and Y. Tian, "Detecting signage and doors for blind navigation and wayfinding," Network Modeling Analysis in Health Informatics and Bioinformatics, vol. 2, no. 2, pp. 81-93, July, 2013. https://doi.org/10.1007/s13721-013-0027-9
  53. N. Tagaki and J. Chen, "Character string extraction from scene images by eliminating non-character elements," in Proc. of IEEE Int. Conf. on Systems, pp. 3685-3690, October 5-8, 2014.
  54. N. Takagi, J. Chen, "A Broken Line Classification Method of Mathematical Graphs for Automating Translation into Scalable Vector Graphic," in Proc. of 43rd IEEE Int. Symposium on Multiple-Valued Logic, pp. 71-76, May 22-24, 2013.
  55. J. Chen and N. Takagi, "A Pattern Recognition Method for Automating Tactile Graphics Translation from Hand-Drawn Maps," in Proc. of IEEE Int. Conf. on Systems, pp. 4173-4178, October 13-16, 2013.
  56. S. S. Pathak, P. Dahiwale and G. Padole, "A combined effect of local and global method for contrast image enhancement," in Proc. of IEEE Int. Conf. on Engineering and Technology, pp. 1-5, March 20-25, 2015.
  57. N. Otsu, "A threshold selection method from gray-level histograms," IEEE Transactions on Systems, Man, and Cybernetics, vol. 9, no. 1, pp. 62-66, January, 1979. https://doi.org/10.1109/TSMC.1979.4310076
  58. S. Paris, P. Kornprobst, J. Tumblin and F. Durand, "Bilateral Filtering: Theory and Applications," Foundations and Trends in Computer Graphics and Vision, vol. 4, no. 1, pp. 1-73, August, 2009. https://doi.org/10.1561/0600000020
  59. G. Jie and L. Ning, "An improved adaptive threshold canny edge detection algorithm," in Proc. of Int. Conf. on Computer Science and Electronics Engineerin, pp. 164-168, March 23-25, 2012.
  60. S. Goferman, L. Z. Manor and A. Tal, "Context-aware saliency detection," IEEE Transactions on Pattern Analysis Machine Intelligence, vol. 34, no. 10, pp. 1915-1926, October, 2012. https://doi.org/10.1109/TPAMI.2011.272
  61. M. M. Cheng, J. Warrell, W. Y. Lin, S. Zheng, V. Vineet and N. Crook, "Efficient salient region detection with soft image abstraction" in Proc. of IEEE Int. Conf. on Computer Vision, pp. 1529-1536, December 1-8, 2013.
  62. X. Li, Y. Li, C. Shen, A. R. Dick and A. V. D. Hengel, "Contextual hypergraph modeling for salient object detection" in Proc. of IEEE Int. Conf. on Computer Vision, pp.3328-3335, December 1-8, 2013.
  63. B. Jiang, L. Zhang, H. Lu, C. Yang and M. H. Yang, "Saliency detection via absorbing Markov chain," in Proc. of IEEE Int. Conf. on Computer Vision, pp. 1665-1672, December 1-8, 2013.
  64. L. Duan, C. Wu, J. Miao, L. Qing and Y. Fu, "Visual saliency detection by spatially weighted dissimilarity," in Proc. of IEEE Conf. Computer Vision and Pattern Recognition, pp. 473-480, June 20-25, 2011.
  65. R. Margolin, L. Z. Manor and A. Tal, "Saliency for image manipulation," The Visual Computer, vol. 29. no. 5, pp. 381-392, May 2013. https://doi.org/10.1007/s00371-012-0740-x
  66. R. Margolin, A. Tal and L. Z. Manor, "What makes a patch distinct?," in Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp. 1139-1146, June 23-28, 2013.
  67. E. Erdem and A. Erdem, "Visual saliency estimation by nonlinearly integrating features using region covariances," Journal Vision, vol. 13, no. 4, pp. 1-20, March, 2013. https://doi.org/10.1167/13.4.1
  68. X. Hou, J. Harel and C. Koch, "Image signature: Highlighting sparse salient regions," IEEE transactions on pattern analysis and machine intelligence, vol. 34, no. 1, pp. 194-201, January, 2012. https://doi.org/10.1109/TPAMI.2011.146
  69. P. Siva, C. Russell, T. Xiang and L. Agapito, "Looking beyond the image: Unsupervised learning for object saliency and detection," in Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp. 3238-3245, June 23-28, 2013.