ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

Content-based image retrieval using a fusion of global and local features

  • Hee Hyung Bu (School of Computer Science and Engineering, Kyungpook National University) ;
  • Nam Chul Kim (School of Electronics Engineering, Kyungpook National University) ;
  • Sung Ho Kim (School of Computer Science and Engineering, Kyungpook National University)
  • Received : 2022.02.21
  • Accepted : 2022.10.11
  • Published : 2023.06.20
Download PDF
⟨ Previous Next ⟩

Abstract

Color, texture, and shape act as important information for images in human recognition. For content-based image retrieval, many studies have combined color, texture, and shape features to improve the retrieval performance. However, there have not been many powerful methods for combining all color, texture, and shape features. This study proposes a content-based image retrieval method that uses the combined local and global features of color, texture, and shape. The color features are extracted from the color autocorrelogram; the texture features are extracted from the magnitude of a complete local binary pattern and the Gabor local correlation revealing local image characteristics; and the shape features are extracted from singular value decomposition that reflects global image characteristics. In this work, an experiment is performed to compare the proposed method with those that use our partial features and some existing techniques. The results show an average precision that is 19.60% higher than those of existing methods and 9.09% higher than those of recent ones. In conclusion, our proposed method is superior over other methods in terms of retrieval performance.

Keywords

References

  1. M. J. Swain and D. H. Ballard, Color indexing, Int. J. Comput. Vis. 7 (1991), no. 1, 11-32. https://doi.org/10.1007/BF00130487
  2. J. Huang, S. R. Kumar, M. Mitra, W. J. Zhu, and R. Zabih, Image indexing using color correlograms, (Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Juan, Puerto Rico), 1997, 762-768.
  3. ISO/IEC 15938-3/FDIS Information Technology, Multimedia content description interface-Part 3: Visual, ISO/IEC/JTC1/SC 29/WG 11, Doc. N4358, 2001.
  4. X. Duanmu, Image retrieval using color moment invariant, (Seventh International Conference on Information Technology: New Generations, Las Vegas, NV, USA), 2010, pp.200-203.
  5. R. M. Haralick, K. Shanmugam, and I. Dinstein, Texture features for image classification, IEEE Trans. Syst. Man Cybern. 3 (1973), no. 6, 610-621.
  6. J. Han and K. K. Ma, Rotation-invariant and scale-invariant Gabor features for texture image retrieval, J. Image Vis. Comput. 25 (2007), no. 9, 1474-1481. https://doi.org/10.1016/j.imavis.2006.12.015
  7. C. Anibou, M. N. Saidi, and D. Aboutajdine, Classification of textured images based on discrete wavelet transform and information fusion, J. Inf. Process. Syst. 11 (2015), no. 3, 421-437.
  8. H. Bu, N. C. Kim, C. Moon, and J. Kim, Content-based image retrieval using combined color and texture features extracted by multi-resolution multi-direction filtering, J. Inf. Process. Syst. 13 (2017), no. 3, 464-475.
  9. T. Ojala, M. Pietikainen, and T. Maenpaa, Multiresolution gray-scale and rotation invariant texture classification with local binary patterns, IEEE Trans. Pattern Anal. Mach. Intell. 24 (2002), no. 7, 971-987. https://doi.org/10.1109/TPAMI.2002.1017623
  10. Z. Guo, L. Zhang, and D. Zhang, A completed modeling of local binary pattern operator for texture classification, IEEE Trans. Image Process. 19 (2010), no. 6, 1657-1663. https://doi.org/10.1109/TIP.2010.2044957
  11. H. Mahi, H. Isabaten, and C. Serief, Zernike moments and SVM for shape classification in very high resolution satellite images, Int. Arab J. Inf. Technol. 11 (2014), no. 1, 43-51.
  12. M. R. Teague, Image analysis via the general theory of moments, J. Opt. Soc. Am. 70 (1980), no. 8, 920-930. https://doi.org/10.1364/JOSA.70.000920
  13. M. K. Hu, Visual pattern recognition by moment invariants, IRE Trans. Inf. Theory 8 (1962), no. 2, 179-187. https://doi.org/10.1109/TIT.1962.1057692
  14. C. S. Lin and C. L. Hwang, New forms of shape invariants from elliptic Fourier descriptors, Pattern Recognit. 20 (1987), no. 5, 535-545. https://doi.org/10.1016/0031-3203(87)90080-X
  15. H. Lynn Beus and S. H. Tiu, An improved corner detection algorithm based on chain coded plane curve, Pattern Recognit. 20 (1987), no. 3, 291-296. https://doi.org/10.1016/0031-3203(87)90004-5
  16. S. Selvan and S. Ramakrishnan, SVD-based modeling for image texture classification using wavelet transformation, IEEE Trans. Image Process. 16 (2007), no. 11, 2688-2696. https://doi.org/10.1109/TIP.2007.908082
  17. S. R. Dubey, S. K. Singh, and R. K. Singh, Multichannel decoded local binary patterns for content-based image retrieval, IEEE Trans. Image Process. 25 (2016), no. 9, 4018-4032. https://doi.org/10.1109/TIP.2016.2577887
  18. S. Murala, R. P. Maheshwari, and R. Balasubramanian, Directional local extrema patterns: a new descriptor for content based image retrieval, Int. J. Multimed. Inf. Retr. 1 (2012), 191-203. https://doi.org/10.1007/s13735-012-0008-2
  19. A. K. Bhunia, A. Bhattacharyya, P. Banerjee, P. P. Roy, and S. Murala, A novel feature descriptor for image retrieval by combining modified color histogram and diagonally symmetric co-occurrence texture pattern, Pattern Anal. Appl. 23 (2020), 703-723. https://doi.org/10.1007/s10044-019-00827-x
  20. A. Ahmed and S. Mohamed, Implementation of early and late fusion methods for content-based image retrieval, Int. J. Adv. Appl. Sci. 8 (2022), no. 7, 97-105. https://doi.org/10.21833/ijaas.2021.07.012
  21. S. R. Dubey, S. K. Singh, and R. K. Singh, Boosting local binary pattern with bag-of-filters for content based image retrieval, (IEEE UP Section Conference on Electrical Computer and Electronics, Allahabad, India), (2015), pp 1-6. https://doi.org/ 10.1109/UPCON.2015.7456703
  22. I. M. Hameed, S. H. Abdulhussain, and B. M. Mahmmod, Content-based image retrieval: A review of recent trends, Cogent Eng. 8 (2021), no. 1, 1927469.
  23. N. K. Rout, M. Atulkar, and M. K. Ahirwal, A review on contentbased image retrieval system: present trends and future challenges, Int. J. Comput. Vis. Robot. 11 (2021), no. 5, 461-485. https://doi.org/10.1504/IJCVR.2021.117578
  24. M. M. Kumar and M. Kumar, XGBoost: 2D-object recognition using shape descriptors and extreme gradient boosting classifier, In Computational methods and data engineering. Advances in Intelligent Systems and Computing, Vol. 1227, Springer, 2020, 207-222.
  25. P. Chhabra, N. K. Garg, and M. Kumar, Content-based image retrieval system using ORB and SIFT features, Neural Comput. Applic. 32 (2020), 2725-2733. https://doi.org/10.1007/s00521-018-3677-9
  26. I. M. Hameed, S. H. Abdulhussain, B. M. Mahmmod, A. Hussain, Content based image retrieval based on feature fusion and support vector machine, (14th International Conference on Developments in eSystems Engineering, Sharjah, United Arab Emirates), 2021, pp. 552-558. https://doi.org/10.1109/DeSE54285.2021.9719539
  27. N. C. Kim and H. J. So, Comments on SVD-based modeling for image texture classification using wavelet transform, IEEE Trans. Image Process. 22 (2013), no. 12, 5408.
  28. en.wikipedia.org/wiki/Singular_value_decomposition
  29. info.hiroshima-cu.ac.jp/~miyazaki/knowledge/teche0104.html
  30. codeproject.com/Articles/741559/Uniform-LBP-Features-andspatial-Histogram-Computa
  31. mathworks.com/matlabcentral/fileexchange/23253-gaborfilter
  32. H. H. Bu, N. C. Kim, and S.-H. Kim, Content-based image retrieval using a combination of texture and color features, Hum.-Centric Comput. Inf. Sci. 11 (2021), no. 23, 1-13.
  33. Y. D. Chun, N. C. Kim, and I. H. Jang, Content-based image retrieval using multiresolution color and texture features, IEEE Trans. Multimed. 10 (2008), no. 6, 1073-1084.
  34. R. Picard, C. Graczyk, S. Mann, J. Wachman, L. Picard, and L. Campbell, Vision texture, Massachusetts Institute of Technology, Cambridge, MA, (1995). Available: vismod.media.mit.edu/vismod/imagery/VisionTexture/vistex.html
  35. J. Z. Wang, J. Li, and G. Wiederhold, SIMPLIcity: Semanticssensitive integrated matching for picture libraries, IEEE Trans. Pattern Anal. Mach. Intell. 23 (2001), no. 9, 947-963. https://doi.org/10.1109/34.955109
  36. W. Y. Ma and B. S. Manjunath, A comparison of wavelet transform features for texture image annotation, (Proc. International Conference on Image Processing, Washington, DC, USA), 1995, pp. 256-259.
  37. D. Comaniciu, P. Meer, K. Xu, and D. Tyler, Retrieval performance improvement through low rank corrections, (Proc. IEEE Workshop on Content-Based Access of Image and Video Libraries [CBAIVL], Fort Collins, CO, USA), 1999, pp. 50-54.