Smart Media Journal (스마트미디어저널)

THE KOREAN INSTITUTE OF SMART MEDIA (한국스마트미디어학회)
권호 표지
DOI QR코드

DOI QR Code

Super-resolution in Music Score Images by Instance Normalization

  • Received : 2019.07.16
  • Accepted : 2019.11.19
  • Published : 2019.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

The performance of an OMR (Optical Music Recognition) system is usually determined by the characterizing features of the input music score images. Low resolution is one of the main factors leading to degraded image quality. In this paper, we handle the low-resolution problem using the super-resolution technique. We propose the use of a deep neural network with instance normalization to improve the quality of music score images. We apply instance normalization which has proven to be beneficial in single image enhancement. It works better than batch normalization, which shows the effectiveness of shifting the mean and variance of deep features at the instance level. The proposed method provides an end-to-end mapping technique between the high and low-resolution images respectively. New images are then created, in which the resolution is four times higher than the resolution of the original images. Our model has been evaluated with the dataset "DeepScores" and shows that it outperforms other existing methods.

Keywords

References

  1. Nasrollahi; Moeslund; "Super-resolution: A comprehensive survey," Machine Vision and Applications, vol.25, pp.1423-1468, 2014 https://doi.org/10.1007/s00138-014-0623-4
  2. Yang; Ma; Yang; "Single-image super-resolution: A benchmark," European Conference on Computer Vision (ECCV), vol.4, pp.372-386, 2014
  3. Borman; Stevenson; "Super-Resolution from Image Sequences - A Review," Midwest Symposium on Circuits and Systems, pp.374-378, 1998
  4. Farsiu; Robinson; Elad; Milanfar; "Fast and robust multiframe super-resolution," IEEE Transactions on Image Processing, vol.13, pp.1327-1344, 2004 https://doi.org/10.1109/TIP.2004.834669
  5. Ankit Lat; C. V. Jawahar; "Enhancing OCR Accuracy With Super Resolution," International Conference on Pattern Recognition (ICPR), pp.3162-3167, 2018
  6. S. Baker; T. Kanade; "Limits on super-resolution and how to break them," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.24, pp.1167-1183, 2002 https://doi.org/10.1109/TPAMI.2002.1033210
  7. Lim; Bee; Son; Sanghyun; Kim; Heewon; Nah; Seungjun; Lee; Kyoung Mu; "Enhanced Deep Residual Networks for Single Image Super-Resolution," IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp.1132-1140, 2017
  8. Lukas Tuggener; Ismail Elezi; Jurgen Schmidhuber; Marcello Pelillo; Thilo Stadelmann; "DeepScores - a dataset for segmentation, detection and classification of tiny objects," International Conference on Pattern Recognition, pp.3704-3709, 2018
  9. Y.-W. Tai; S. Liu; M. S. Brown; S. Lin; "Super-resolution using Edge Prior and Single Image Detail Synthesis," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.2400-2407, 2010
  10. J. Sun; Z. Xu; H.-Y. Shum; "Image super-resolution using gradient profile prior," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2008
  11. K. Zhang; X. Gao; D. Tao; X. Li; "Multi-scale dictionary for single image super-resolution," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.1114-1121, 2012
  12. H. Yue; X. Sun; J. Yang; F. Wu; "Landmark image super-resolution by retrieving web images," IEEE Transactions on Image Processing, vol.22, pp.4865-4878, 2013 https://doi.org/10.1109/TIP.2013.2279315
  13. K. Gregor; Y. LeCun; "Learning fast approximations of sparse coding," In Proceedings of the 27th International Conference on Machine Learning, pp.399-406, 2010
  14. C. Dong; C. C. Loy; K. He; X. Tang; "Learning a deep convolutional network for image super-resolution," European Conference on Computer Vision, pp.184-199 vol.8692, 2014
  15. C. Dong; C. C. Loy; K. He; X. Tang; "Image super-resolution using deep convolutional networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.38, pp.295-307, 2016 https://doi.org/10.1109/TPAMI.2015.2439281
  16. C. E. Duchon; Lanczos; "Filtering in One and Two Dimensions," Journal of Applied Meteorology, vol.18, pp.1016-1022, 1979 https://doi.org/10.1175/1520-0450(1979)018<1016:LFIOAT>2.0.CO;2
  17. Christian Ledig; Lucas Theis; Ferenc Huszar; Jose Caballero; Andrew Cunningham; Alejandro Acosta; Andrew Aitken; Alykhan Tejani; annes Totz; Zehan Wang; Wenzhe Shi Twitter; "Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network," Conference on Computer Vision and Pattern Recognition (CVPR), pp.105-114, 2017
  18. Zheng Xu; Xitong Yang; Xue Li; Xiaoshuai Sun; "Strong Baseline for Single Image Dehazing with Deep Features and Instance Normalization," 29th British Machine Vision Conference, 2018
  19. Zhou Wang; A.C. Bovik; H.R. Sheikh; E.P. Simoncelli; "Image Quality Assessment: From Error Visibility To Structural Similarity," IEEE Transactions on Image Processing, vol.13, pp.600-612, 2004 https://doi.org/10.1109/TIP.2003.819861
  20. D. Kingma; J. Ba; "Adam: A method for stochastic optimization," International Conference on Learning Representations (ICLR), vol.abs/1412.6980, 2014
  21. J. Kim; J. Kwon Lee; and K. M. Lee; "Accurate image superresolution using very deep convolutional networks," Conference on Computer Vision and Pattern Recognition (CVPR), pp.1646-1654, 2016
  22. J. Kim; J. Kwon Lee; K. M. Lee; "Deeply-recursive convolutional network for image super-resolution," Conference on Computer Vision and Pattern Recognition (CVPR), pp.1637-1645, 2016
  23. Dmitry Ulyanov; Andrea Vedaldi; Victor Lempitsky; "Instance Normalization: The Missing Ingredient for Fast Stylization," arXiv:1607.08022v3, 2017 (accessed Dec., 24, 2019).
  24. Sergey Ioffe; Christian Szegedy; "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift," Proc. of the 32nd International Conference on Machine Learning, vol.37, pp.448-456, 2015
  25. Yuxin Wu; Kaiming He; "Group Normalization," International Journal of Computer Vision, pp.1-14, 2018
  26. Wenzhe Shi; Jose Caballero; Ferenc Huszar; Johannes Totz; Andrew P. Aitken; "Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network," Conference on Computer Vision and Pattern Recognition (CVPR), 2016
  27. QuangNhat Vo; GueeSang Lee; SooHyung Kim; HyungJeong Yang; "Recognition of Music Scores with Non-Linear Distortions in Mobile Devices," Multimedia Tools and Applications, vol.77, pp.15951-15969, 2018 https://doi.org/10.1007/s11042-017-5169-9
  28. Jorge Calvo-Zaragoza; Jose J. Valero-Mas; Antonio Pertusa; "End-to-End Optical Music Recognition Using Neural Networks," Proc, of the 18th International Society for Music Information Retrieval Conference (ISMIR), pp.472-477, 2017
  29. Luu-Ngoc Do; Hyung-Jeong Yang; Soo-Hyung Kim; Guee-Sang Lee; Cong Minh Dinh; "A Covariance-matching-based Model for Musical Symbol Recognition," Smart Media Journal, vol.7, no.2, pp.23-33, 2018 https://doi.org/10.30693/SMJ.2018.7.2.23
  30. Son Tung Trieu; Guee-Sang Lee; "Machine Printed and Handwritten Text Discrimination in Korean Document Images," Smart Media Journal, vol.5, no.3, pp.30-34, 2016
  31. Van Khien Pham; Soo-Hyung Kim; Hyung-Jeong Yang; Guee-Sang Lee; "Text Detection based on Edge Enhanced Contrast Extremal Region and Tensor Voting in Natural Scene Images," Smart Media Journal, vol.6, no. 4, pp.32-40, 2017