KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Deep Hashing for Semi-supervised Content Based Image Retrieval

  • Bashir, Muhammad Khawar (Department of Computer Science & Engineering, University of Engineering and Technology) ;
  • Saleem, Yasir (Department of Computer Science & Engineering, University of Engineering and Technology)
  • Received : 2017.10.23
  • Accepted : 2018.04.02
  • Published : 2018.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

Content-based image retrieval is an approach used to query images based on their semantics. Semantic based retrieval has its application in all fields including medicine, space, computing etc. Semantically generated binary hash codes can improve content-based image retrieval. These semantic labels / binary hash codes can be generated from unlabeled data using convolutional autoencoders. Proposed approach uses semi-supervised deep hashing with semantic learning and binary code generation by minimizing the objective function. Convolutional autoencoders are basis to extract semantic features due to its property of image generation from low level semantic representations. These representations of images are more effective than simple feature extraction and can preserve better semantic information. Proposed activation and loss functions helped to minimize classification error and produce better hash codes. Most widely used datasets have been used for verification of this approach that outperforms the existing methods.

Keywords

References

  1. Y. Gong, S. Lazebnik, A. Gordo, and F. Perronnin, "Iterative Quantization: A Procrustean Approach to Learning Binary Codes for Large-Scale Image Retrieval," IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 12, pp. 2916-2929, Dec. 2013. https://doi.org/10.1109/TPAMI.2012.193
  2. G. Irie, Z. Li, X. M. Wu, and S. F. Chang, "Locally Linear Hashing for Extracting Non-linear Manifolds," in Proc. of 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2123-2130, 2014.
  3. M. Kan, D. Xu, S. Shan, and X. Chen, "Semisupervised Hashing via Kernel Hyperplane Learning for Scalable Image Search," IEEE Trans. Circuits Syst. Video Technol., vol. 24, no. 4, pp. 704-713, Apr. 2014. https://doi.org/10.1109/TCSVT.2013.2276713
  4. J. He, W. Liu, and S.-F. Chang, "Scalable Similarity Search with Optimized Kernel Hashing," in Proc. of Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, NY, USA, pp. 1129-1138, 2010.
  5. B. Kulis and K. Grauman, "Kernelized Locality-Sensitive Hashing," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 6, pp. 1092-1104, Jun. 2012. https://doi.org/10.1109/TPAMI.2011.219
  6. Y. Weiss, A. Torralba, and R. Fergus, "Spectral Hashing," in Advances in Neural Information Processing Systems 21, D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, Eds. Curran Associates, Inc., pp. 1753-1760, 2009.
  7. H. Lai, Y. Pan, Y. Liu, and S. Yan, "Simultaneous Feature Learning and Hash Coding with Deep Neural Networks," CoRR, vol. abs/1504.03410, 2015.
  8. R. Xia, Y. Pan, H. Lai, C. Liu, and S. Yan, "Supervised Hashing for Image Retrieval via Image Representation Learning," in Proc. of Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, Qubec City, Qubec, Canada, pp. 2156-2162, 2014.
  9. F. Zhao, Y. Huang, L. Wang, and T. Tan, "Deep Semantic Ranking Based Hashing for Multi-Label Image Retrieval," CoRR, vol. abs/1501.06272, 2015.
  10. H. Zhu, M. Long, J. Wang, and Y. Cao, "Deep Hashing Network for Efficient Similarity Retrieval," in Proc. of Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, Arizona, pp. 2415-2421, 2016.
  11. A. Gionis, P. Indyk, and R. Motwani, "Similarity Search in High Dimensions via Hashing," in Proc. of Proceedings of the 25th International Conference on Very Large Data Bases, San Francisco, CA, USA, pp. 518-529, 1999.
  12. J. Wang and S. Chang, "Sequential Projection Learning for Hashing with Compact Codes," 2010.
  13. R. Zhang, L. Lin, R. Zhang, W. Zuo, and L. Zhang, "Bit-Scalable Deep Hashing with Regularized Similarity Learning for Image Retrieval and Person Re-identification," CoRR, vol. abs/1508.04535, 2015.
  14. H. Liu, R. Wang, S. Shan, and X. Chen, "Deep Supervised Hashing for Fast Image Retrieval," in Proc. of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2064-2072, 2016.
  15. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet Classification with Deep Convolutional Neural Networks," Adv. Neural Inf. Process. Syst. 25, pp. 1106-1114, 2012.
  16. K. Simonyan and A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition," CoRR, vol. abs/1409.1556, 2014.
  17. C. Szegedy et al., "Going deeper with convolutions," in Proc. of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1-9, 2015.
  18. R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation," in Proc. of Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Washington, DC, USA, pp. 580-587, 2014.
  19. P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun, "OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks," CoRR, vol. abs/1312.6229, 2013.
  20. E. Shelhamer, J. Long, and T. Darrell, "Fully Convolutional Networks for Semantic Segmentation," CoRR, vol. abs/1605.06211, 2016.
  21. A. Oliva and A. Torralba, "Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope," Int J Comput Vis., vol. 42, no. 3, pp. 145-175, May 2001. https://doi.org/10.1023/A:1011139631724
  22. N. Dalal and B. Triggs, "Histograms of Oriented Gradients for Human Detection," in Proc. of Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 1 - Volume 01, Washington, DC, USA, pp. 886-893, 2005.
  23. H. F. Yang, K. Lin, and C. S. Chen, "Supervised Learning of Semantics-Preserving Hash via Deep Convolutional Neural Networks," IEEE Trans. Pattern Anal. Mach. Intell., vol. PP, no. 99, pp. 1-1, 2017.
  24. Y. Zhang, K. Lee, and H. Lee, "Augmenting Supervised Neural Networks with Unsupervised Objectives for Large-scale Image Classification," ArXiv E-Prints, Jun. 2016.
  25. Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, pp. 436-444, May 2015. https://doi.org/10.1038/nature14539
  26. J. Zhao, M. Mathieu, R. Goroshin, and Y. LeCun, "Stacked What-Where Auto-encoders," ArXiv E-Prints, Jun. 2015.
  27. A. Makhzani and B. Frey, "Winner-Take-All Autoencoders," ArXiv E-Prints, Sep. 2014.
  28. B. Kulis and T. Darrell, "Learning to Hash with Binary Reconstructive Embeddings," in Proc. of Proceedings of the 22Nd International Conference on Neural Information Processing Systems, USA, pp. 1042-1050, 2009.
  29. M. Norouzi and D. J. Fleet, "Minimal Loss Hashing for Compact Binary Codes," in Proc. of Proceedings of the 28th International Conference on International Conference on Machine Learning, USA, pp. 353-360, 2011.
  30. F. Shen, C. Shen, W. Liu, and H. T. Shen, "Supervised Discrete Hashing," in Proc. of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 37-45, 2015.
  31. V. E. Liong, J. Lu, G. Wang, P. Moulin, and J. Zhou, "Deep hashing for compact binary codes learning," in Proc. of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2475-2483, 2015.
  32. Y. Kang, S. Kim, and S. Choi, "Deep Learning to Hash with Multiple Representations," in Proc. of 2012 IEEE 12th International Conference on Data Mining, pp. 930-935, 2012.
  33. Z. Xia, X. Feng, J. Peng, and A. Hadid, "Unsupervised Deep Hashing for Large-scale Visual Search," ArXiv E-Prints, Jan. 2016.
  34. J. Wang, S. Kumar, and S. F. Chang, "Semi-Supervised Hashing for Large-Scale Search," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 12, pp. 2393-2406, Dec. 2012. https://doi.org/10.1109/TPAMI.2012.48
  35. Q. Wang, L. Si, and D. Zhang, "Learning to Hash with Partial Tags: Exploring Correlation between Tags and Hashing Bits for Large Scale Image Retrieval," in Proc. of Computer Vision - ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part III, D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, Eds. Cham: Springer International Publishing, pp. 378-392, 2014.
  36. G. E. Hinton, A. Krizhevsky, and S. D. Wang, "Transforming Auto-Encoders," in Proc. of Artificial Neural Networks and Machine Learning - ICANN 2011: 21st International Conference on Artificial Neural Networks, Espoo, Finland, June 14-17, 2011, Proceedings, Part I, T. Honkela, W. Duch, M. Girolami, and S. Kaski, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, pp. 44-51, 2011.
  37. M. Ranzato, F. J. Huang, Y. L. Boureau, and Y. LeCun, "Unsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition," in Proc. of 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8, 2007.
  38. J. Zhang, Y. Peng, and J. Zhang, "SSDH: Semi-supervised Deep Hashing for Large Scale Image Retrieval," CoRR, vol. abs/1607.08477, 2016.