KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Medical Image Retrieval with Relevance Feedback via Pairwise Constraint Propagation

  • Wu, Menglin (School of Computer Science and Engineering, Nanjing University of Science and Technology) ;
  • Chen, Qiang (School of Computer Science and Engineering, Nanjing University of Science and Technology) ;
  • Sun, Quansen (School of Computer Science and Engineering, Nanjing University of Science and Technology)
  • Received : 2013.11.08
  • Accepted : 2014.01.04
  • Published : 2014.01.30
Download PDF
⟨ Previous Next ⟩

Abstract

Relevance feedback is an effective tool to bridge the gap between superficial image contents and medically-relevant sense in content-based medical image retrieval. In this paper, we propose an interactive medical image search framework based on pairwise constraint propagation. The basic idea is to obtain pairwise constraints from user feedback and propagate them to the entire image set to reconstruct the similarity matrix, and then rank medical images on this new manifold. In contrast to most of the algorithms that only concern manifold structure, the proposed method integrates pairwise constraint information in a feedback procedure and resolves the small sample size and the asymmetrical training typically in relevance feedback. We also introduce a long-term feedback strategy for our retrieval tasks. Experiments on two medical image datasets indicate the proposed approach can significantly improve the performance of medical image retrieval. The experiments also indicate that the proposed approach outperforms previous relevance feedback models.

Keywords

References

  1. H. Muller, N. Michoux, D. Bandon, and A. Geissbuhler, "A review of content-based image retrieval systems in medical applications-clinical benefits and future directions," International Journal of Medical Informatics, vol. 73, pp. 1-23, February, 2004. https://doi.org/10.1016/j.ijmedinf.2003.11.024
  2. L. Yang, R. Jin, L. Mummert, R. Sukthankar, A. Goode, B. Zheng S. Hoi and M. Satya-narayanan, "A boosting framework for visuality-preserving distance metric learning and its application to medical image retrieval," IEEE Pattern Analysis and Machine Intelligence, vol. 32, no. 1, pp. 30-44, January, 2010. https://doi.org/10.1109/TPAMI.2008.273
  3. U. Avni, H. Greenspan, E. Konen, M. Sharon and J. Goldberger, "X-ray categorization and retrieval on the organ and pathology level, using patch-based visual words," IEEE Medical Imaging, vol. 30, no. 3, pp. 733-746, March, 2011. https://doi.org/10.1109/TMI.2010.2095026
  4. G. Quellec, M. Lamard, G. Cazuguel, "Case retrieval in medical databases by fusing heterogeneous information," IEEE Medical Imaging, vol. 30, no. 1, pp. 108-118, January, 2011. https://doi.org/10.1109/TMI.2010.2063711
  5. M.M. Rahman, S.K. Antani, and G.R. Thoma, "A learning-based similarity fusion and filtering approach for biomedical Image retrieval using SVM classification and relevance feedback," IEEE Information Technology in Biomedicine, vol. 15, no. 4, pp. 640-646, July, 2011. https://doi.org/10.1109/TITB.2011.2151258
  6. H.W. Chia, Y Li, C. Li. "Effective extraction of Gabor features for adaptive mammogram retrieval," in Proc. of IEEE Conf. on Multimedia & Expo 2007, pp. 1053-1056, July 2-5, 2007.
  7. Y. Rui , T.S. Huang, M. Orgega and S. Mehrotra, "Relevance feedback: a power tool for interactive content-based image retrieval," IEEE Circuits and Systems for Video Technology, vol. 8, no. 5, pp. 644-655, September, 1998. https://doi.org/10.1109/76.718510
  8. S.C.H Hoi, M.R. Lyu and R Jin, "A unified log-based relevance feedback scheme for image retrieval," IEEE Knowledge and Data Engineering, vol. 18, no. 4, pp. 509-524, April, 2006. https://doi.org/10.1109/TKDE.2006.1599389
  9. P. Yin, B. Bhanu, K. Chang and A. Dong, "Long-term cross-session relevance feedback using virtual features," IEEE Knowledge and Data Engineering, vol. 20, no. 3, pp. 352-368, March, 2008. https://doi.org/10.1109/TKDE.2007.190697
  10. Z. Li, J. Liu and X. Tang, "Pairwise constraint propagation by semidefinite programming for semi-supervised classification," in Proc. of 25th Int. Conf. on Machine learning, pp. 576-583, July 5-9, 2008.
  11. Z. Lu and Y. Peng, "Exhaustive and Efficient Constraint Propagation: A Semi-Supervised Learning Perspective and Its Applications," International Journal of Computer Vision, vol. 103, no. 3, pp. 306-325, July, 2013. https://doi.org/10.1007/s11263-012-0602-z
  12. Z. Fu, H. Lp, H. Lu and Z. Lu, "Multi-modal constraint propagation for heterogeneous image clustering," in Proc. of 19th ACM Int. Conf. on Multimedia, pp. 143-152, November 28 - December 1, 2011.
  13. B. Thomee and M.S. Lew, "Interactive search in image retrieval: a survey," International Journal of Multimedia Information Retrieval, vol. 1, no. 2, pp. 71-86, July, 2012. https://doi.org/10.1007/s13735-012-0014-4
  14. H. Muller, W. Muller, S. Marchand-Maillet, T. Pun and D.M. Squire, "Strategies for positive and negative relevance feedback in image retrieval," in Proc. of 15th Int. Conf. on Pattern Recognition, pp. 1043-1046, September 3-7, 2000.
  15. G. Das, S. Ray and C. Wilson, "Feature re-weighting in content-based image retrieval," in Proc. of 5th Int. Conf. on Image and Video Retrieval, pp. 193-200, July 13-15, 2006.
  16. J. Laaksonen, M. Koskela and E. Oja, "PicSOM: self-organizing image retrieval with MPEG-7 content descriptors," IEEE Neural Networks, vol. 13, no. 4, pp. 841-853, July, 2002. https://doi.org/10.1109/TNN.2002.1021885
  17. S. Tong and E. Chang, "Support vector machine active learning for image retrieval," in Proc. of 9th ACM Int. Conf. on Multimedia, pp. 107-118, September 30 - October 5, 2001.
  18. C.H. Hoi, C.H. Chan, K.Z. Huang, M.R. Lyu and I. King. "Biased support vector machine for relevance feedback in image retrieval," in Proc. of IEEE Int. Conf. on Neural Networks, pp. 3189-3194, July 25-29, 2004.
  19. D. Tao, X. Tang, X. Li and X. Wu, "Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval," IEEE Pattern Analysis and Machine Intelligence, vol. 28, no. 7, pp. 1088-1099, July, 2006. https://doi.org/10.1109/TPAMI.2006.134
  20. J. Ye, R. Janardan and Q. Li, "Two-dimensional linear discriminant analysis," in Proc. of the Advances in Neural Information Processing Systems, pp. 1569-1576, December 13-18, 2004.
  21. D. Xu, S. Yan, D. Tao, S. Lin and H.J. Zhang, "Marginal Fisher analysis and its variants for human gait recognition and content- based image retrieval," IEEE Image Processing, vol. 16, no. 11, pp. 2811-2821, November, 2007. https://doi.org/10.1109/TIP.2007.906769
  22. X. He, D. Cai and J. Han. "Learning a maximum margin subspace for image retrieval," IEEE Knowledge and Data Engineering, vol. 20, no. 2, pp. 189-201, February, 2008. https://doi.org/10.1109/TKDE.2007.190692
  23. E.D. Vesa, J. Domingob, G. Ayalac and P. Zuccarello, "A novel Bayesian framework for relevance feedback in image content-based retrieval systems," Pattern Recognition, vol. 39, no. 9, pp. 1622-1632, September, 2006. https://doi.org/10.1016/j.patcog.2006.01.006
  24. G. Giacinto and F. Roli. "Instance-based relevance feedback in image retrieval using dissimilarity spaces," Case-Based Reasoning on Images and Signals, vol. 73, no. 1, pp. 419-436, January, 2008. https://doi.org/10.1007/978-3-540-73180-1_14
  25. M.A. Herraez and F.J. Ferri, "An improved distance-based relevance feedback strategy for image retrieval," Image and Vision Computing, vol. 31, no. 10, pp. 704-713, October, 2013. https://doi.org/10.1016/j.imavis.2013.07.004
  26. T. Amin, M. Zeytinoglu and L. Guan, "Application of Laplacian mixture model to image and video retrieval," IEEE Multimedia, vol. 9, no. 7, pp. 1416-1429, November, 2007. https://doi.org/10.1109/TMM.2007.906587
  27. M. Arevalillo-Herraeza, M. Zacaresb, X. Benaventc and E.D. Vesa, "A relevance feedback CBIR algorithm based on fuzzy sets," Signal Processing: Image Communication, vol. 23, no. 7, pp. 490-504, August, 2008. https://doi.org/10.1016/j.image.2008.04.016
  28. H. Chang, D.Y. Yeung, "Kernel-based distance metric learning for content-based image retrieval," Image and Vision Computing, vol. 25, no. 5, pp. 695-703, May, 2007. https://doi.org/10.1016/j.imavis.2006.05.013
  29. Z.H. Zhou, K.J Chen and H.B. Dai. "Enhancing relevance feedback in image retrieval using unlabeled data," ACM Information Systems, vol. 24, no. 2, pp. 219-244, April, 2006. https://doi.org/10.1145/1148020.1148023
  30. L. Zhang, L. Wang and W. Lin, "Semi-supervised biased maximum margin analysis for interactive image retrieval," IEEE Image Processing, vol. 21, no. 4, pp. 2294-2308, April, 2012. https://doi.org/10.1109/TIP.2011.2177846
  31. J. He, M. Li, H. Zhang and H. Tong, "Generalized manifold-ranking-based image retrieval," IEEE Image Processing, vol. 15, no. 10, pp. 3170-3177, October, 2006. https://doi.org/10.1109/TIP.2006.877491
  32. S.R. Bulo, M. Rabbi and M. Pelillo, "Content-based image retrieval with relevance feedback using random walks," Pattern Recognition, vol. 44, no. 9, pp. 2109-2122, September, 2011. https://doi.org/10.1016/j.patcog.2011.03.016
  33. D. Zhou, O. Bousquet, T.N. Lal, J. Weston and B. Scholkopf. "Learning with local and global consistency," in Proc. of the Advances in Neural Information Processing Systems, pp. 321-328, December 8-13, 2003.
  34. X. Zhu, Z. Ghahramani and J. Lafferty. "Semi-supervised learning using gaussian fields and harmonic functions," in Proc. of 20th Int. Conf. on Machine Learning, pp. 912-919, August 21-24, 2003.
  35. T. Tommasi, B. Caputo, P. Welter, M.O. Guld and T.M. Deserno, "Overview of the CLEF 2009 medical image annotation track," in Proc. of 10th Int. Conf. on Cross-language evaluation forum, pp. 85-93, September 30- October 2, 2009.
  36. J. Oliveira, A. Machado, G. Chavez, A. Lopes, T.M. Deserno and A. Araujo, "MammoSys: a content-based image retrieval system using breast density patterns," Computer Methods and Programs in Biomedicine, vol. 99, no. 3, pp. 289-297, September, 2010. https://doi.org/10.1016/j.cmpb.2010.01.005