International journal of advanced smart convergence

The Institute of Internet, Broadcasting and Communication (한국인터넷방송통신학회)
권호 표지
DOI QR코드

DOI QR Code

Object detection in financial reporting documents for subsequent recognition

  • Sokerin, Petr (Research laboratory Monetary policy research and financial market analysis, Plekhanov Russian University of Economics) ;
  • Volkova, Alla (KPMG Taxes and Consulting, LLC) ;
  • Kushnarev, Kirill (Research laboratory Monetary policy research and financial market analysis, Plekhanov Russian University of Economics)
  • Received : 2020.12.10
  • Accepted : 2020.12.19
  • Published : 2021.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

Document page segmentation is an important step in building a quality optical character recognition module. The study examined already existing work on the topic of page segmentation and focused on the development of a segmentation model that has greater functional significance for application in an organization, as well as broad capabilities for managing the quality of the model. The main problems of document segmentation were highlighted, which include a complex background of intersecting objects. As classes for detection, not only classic text, table and figure were selected, but also additional types, such as signature, logo and table without borders (or with partially missing borders). This made it possible to pose a non-trivial task of detecting non-standard document elements. The authors compared existing neural network architectures for object detection based on published research data. The most suitable architecture was RetinaNet. To ensure the possibility of quality control of the model, a method based on neural network modeling using the RetinaNet architecture is proposed. During the study, several models were built, the quality of which was assessed on the test sample using the Mean average Precision metric. The best result among the constructed algorithms was shown by a model that includes four neural networks: the focus of the first neural network on detecting tables and tables without borders, the second - seals and signatures, the third - pictures and logos, and the fourth - text. As a result of the analysis, it was revealed that the approach based on four neural networks showed the best results in accordance with the objectives of the study on the test sample in the context of most classes of detection. The method proposed in the article can be used to recognize other objects. A promising direction in which the analysis can be continued is the segmentation of tables; the areas of the table that differ in function will act as classes: heading, cell with a name, cell with data, empty cell.

Keywords

References

  1. Agne S., Rogger M., Rohrschneider J. Benchmarking of document page segmentation //Document Recognition and Retrieval VII. - International Society for Optics and Photonics, 1999. - Т. 3967. - С. 165-171.
  2. Ale L., Zhang N., Li L. Road damage detection using RetinaNet //2018 IEEE International Conference on Big Data (Big Data). - IEEE, 2018. - С. 5197-5200.
  3. Antonacopoulos A., Ritchings R. T. Flexible page segmentation using the background //Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3-Conference C: Signal Processing (Cat. No. 94CH3440-5). - IEEE, 1994. - Т. 2. - С. 339-344.
  4. Bertelli L. et al. Kernelized structural SVM learning for supervised object segmentation //CVPR 2011. - IEEE, 2011. - С. 2153-2160.
  5. Bradski G., Kaehler A. Learning OpenCV: Computer vision with the OpenCV library. - " O'Reilly Media, Inc.", 2008.
  6. Brahmbhatt S. Practical OpenCV. - Apress, 2013.
  7. Cai Z. et al. A unified multi-scale deep convolutional neural network for fast object detection //European conference on computer vision. - Springer, Cham, 2016. - С. 354-370.
  8. Can Y. S., Kabadayi M. E. CNN-Based Page Segmentation and Object Classification for Counting Population in Ottoman Archival Documentation //Journal of Imaging. - 2020. - Т. 6. - No. 5. - С. 32. https://doi.org/10.3390/jimaging6050032
  9. Cattoni R. et al. Geometric layout analysistechniquesfor document image understanding: a review //ITC-irst Technical Report. - 1998. - Т. 9703. - No. 09.
  10. Dong J. et al. Low-level cursive word representation based on geometric decomposition //International Workshop on Machine Learning and Data Mining in Pattern Recognition. - Springer, Berlin, Heidelberg, 2005. - С. 590-599.
  11. Feng Z. et al. Deep retinal image segmentation: a FCN-based architecture with short and long skip connections for retinal image segmentation. //International Conference on Neural Information Processing. - Springer, Cham, 2017. - С. 713-722.
  12. He D. et al. Multi-scale multi-task fcn for semantic page segmentation and table detection //2017 14th IAPR International Conference on Do cument Analysis and Recognition (ICDAR). - IEEE, 2017. - Т. 1. - С. 254-261.
  13. Henderson P., Ferrari V. End-to-end training of object class detectors for mean average precision //Asian Conference on Computer Vision. - Springer, Cham, 2016. - С. 198-213.
  14. Hones F., Lichter J. Layout extraction of mixed mode documents //Machine vision and applications. - 1994. - Т. 7. - No. 4. - С. 237-246. https://doi.org/10.1007/BF01213414
  15. Hu H. et al. Relation networks for object detection //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. - 2018. - С. 3588-3597.
  16. International Conference on Neural Information Processing. - Springer, Cham, 2017. - С. 713-722.
  17. Jain A. K., Zhong Y. Page segmentation using texture analysis //Pattern recognition. - 1996. - Т. 29. - No. 5. - С. 743-770. https://doi.org/10.1016/0031-3203(95)00131-X
  18. Kisantal M. et al. Augmentation for small object detection //arXiv preprint arXiv:1902.07296. - 2019.
  19. Laganiere R. OpenCV Computer Vision Application Programming Cookbook Second Edition. - Packt Publishing Ltd, 2014.
  20. Li K. et al. A maximal figure-of-merit learning approach to maximizing mean average precision with deep neural network based classifiers //2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). - IEEE, 2014. - С. 4503-4507.
  21. Lin T. Y. et al. Focal loss for dense object detection //Proceedings of the IEEE international conference on computer vision. - 2017. - С. 2980-2988.
  22. Nagy G. Twenty years of document image analysis in PAMI //IEEE Transactions on Pattern Analysis and Machine Intelligence. - 2000. - Т. 22. - No. 1. - С. 38-62. https://doi.org/10.1109/34.824820
  23. Perner P., Imiya A. (ed.). Machine Learning and Data Mining in Pattern Recognition: 4th International Conference, MLDM 2005, Leipzig, Germany, July 9-11, 2005, Proceedings. - Springer Science & Business Media, 2005. - Т. 3587.
  24. Randriamasy S., Vincent L. A region-based system for the automatic evaluation of page segmentation algorithms //Proceedings of the International Association for Pattern Recognition Workshop on Document Analysis Systems DAS94. - 1994. - С. 29-41.
  25. Review: RetinaNet - Focal Loss (Object Detection). Towards Data Science. URL: https://towardsdatascience.com/review-retinanet-focal-loss-object-detection-38fba6afabe4 (режим доступа: 17.01.2021)
  26. Setitra I. et al. Text Line Segmentation in Handwritten Documents Based on Connected Components Trajectory Generation //International Conference on Pattern Recognition Applications and Methods. - Springer, Cham, 2017. - С. 222-234.
  27. Szegedy C., Toshev A., Erhan D. Deep neural networks for object detection //Advances in neural information processing systems. - 2013. - С. 2553-2561.
  28. Taheri S. et al. OpenCV. js: Computer Vision processing for the open Web platform //Proceedings of the 9th ACM Multimedia Systems Conference. - 2018. - С. 478-483.
  29. Wang Y. et al. Automatic ship detection based on RetinaNet using multi-resolution Gaofen-3 imagery //Remote Sensing. - 2019. - Т. 11. - No. 5. - С. 531. https://doi.org/10.3390/rs11050531
  30. Wood S. L., Marks J. P., Pearlman J. A segmentation algorithm for ocr application to low resolution images //Conference Record of the Fourteenth Asilomar Conference on Circuits, Systems and Computers. - 1980. - С. 411-415.
  31. Wei H. et al. Evaluation of SVM, MLP and GMM classifiers for layout analysis of historical documents //2013 12th International Conference on Document Analysis and Recognition. - IEEE, 2013. - С. 1220-1224.
  32. Yi X. et al. CNN based page object detection in document images //2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR). - IEEE, 2017. - Т. 1. - С. 230-235.
  33. Zeng N. RetinaNet Explained and Demystified [Электронный ресурс]. 2018 URL: blog.zenggyu.com/en/post/2018-12-05/retinanet-explained-and-demystified
  34. Zhang H. et al. Cascade retinanet: Maintaining consistency for single-stage object detection //arXiv preprint arXiv:1907.06881. - 2019.
  35. Zou Z. et al. Object detection in 20 years: A survey //arXiv preprint arXiv:1905.05055. - 2019.