ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

Oriented object detection in satellite images using convolutional neural network based on ResNeXt

  • Asep Haryono (Faculty of Computer Science, Universitas Indonesia) ;
  • Grafika Jati (Faculty of Computer Science, Universitas Indonesia) ;
  • Wisnu Jatmiko (Faculty of Computer Science, Universitas Indonesia)
  • Received : 2022.12.03
  • Accepted : 2023.05.03
  • Published : 2024.04.20
Download PDF
⟨ Previous Next ⟩

Abstract

Most object detection methods use a horizontal bounding box that causes problems between adjacent objects with arbitrary directions, resulting in misaligned detection. Hence, the horizontal anchor should be replaced by a rotating anchor to determine oriented bounding boxes. A two-stage process of delineating a horizontal bounding box and then converting it into an oriented bounding box is inefficient. To improve detection, a box-boundary-aware vector can be estimated based on a convolutional neural network. Specifically, we propose a ResNeXt101 encoder to overcome the weaknesses of the conventional ResNet, which is less effective as the network depth and complexity increase. Owing to the cardinality of using a homogeneous design and multi-branch architecture with few hyperparameters, ResNeXt captures better information than ResNet. Experimental results demonstrate more accurate and faster oriented object detection of our proposal compared with a baseline, achieving a mean average precision of 89.41% and inference rate of 23.67 fps.

Keywords

Acknowledgement

This work was supported by Tokopedia-UI AI Center of Excellence, National Research and Innovation Agency, and PUTI Q2 from Universitas Indonesia for Research Project "High-Level Data Fusion of RGB and Thermal Data for Robust Search and Rescue Applications" (number NKB-572/UN2.RST/HKP.05.00/2022).

References

  1. S. Jain, A. Jatain, and S. Bhaskar, Smart city management system using IoT with deep learning, (Proc. 4th Int. Conf. Commun. Electron. Syst., Coimbatore, India), 2019, pp. 1214-1222. 
  2. S. Sasi Priya, S. Rajarajeshwari, K. Sowmiya, and P. Vinesha, Road traffic condition monitoring using deep learning, (Proc. 5th Int. Conf. Invent. Comput. Technol., Coimbatore, India), 2020, pp. 330-335. 
  3. S. Javadi, M. Dahl, and M. I. Pettersson, Vehicle detection in aerial images based on 3D depth maps and deep neural networks, IEEE Access 9 (2021), 8381-8391.  https://doi.org/10.1109/ACCESS.2021.3049741
  4. J. Wang, J. Ding, H. Guo, W. Cheng, T. Pan, and W. Yang, Mask OBB: a semantic attention-based mask oriented bounding box representation for multi-category object detection in aerial images, Remote Sens. (Basel) 11 (2019), doi:10.3390/rs11242930 
  5. L. Liu, W. Ouyang, X. Wang, P. Fieguth, J. Chen, X. Liu, and M. Pietikainen, Deep learning for generic object detection: a survey, International Journal of Computer Vision 128 (2020), 261-318.  https://doi.org/10.1007/s11263-019-01247-4
  6. X. Han, J. Chang, and K. Wang, Real-time object detection based on YOLO-v2 for tiny vehicle object, Proc. Comp. Sci. 183 (2021), 61-72.  https://doi.org/10.1016/j.procs.2021.02.031
  7. Q. Shuai and X. Wu, Object detection system based on SSD algorithm, (Proc. 2020 Int. Conf. Cult.-Oriented Sci. Technol., Beijing, China), 2020, pp. 141-144. 
  8. M. Ahmad, M. Abdullah, and D. Han, Small object detection in aerial imagery using RetinaNet with anchor optimization, (2020 Int. Conf. Electron. Inf. Commun., Barcelona, Spain), 2020, pp. 13-15. 
  9. W. Jie and Y. Hong, Detection and location technology of substation personnel based on EfficientDet, (China Int. Conf. Electr. Distrib., Shanghai, China), 2021, pp. 280-284. 
  10. V. Murugan, V. R. Vijaykumar, and A. Nidhila, Vehicle logo recognition using RCNN for intelligent transportation systems, (2019 Int. Conf. Wirel. Commun. Signal Process. Netw., Chennai, India), 2019, pp. 107-111. 
  11. K. S. Htet and M. M. Sein, Event analysis for vehicle classification using fast RCNN, (2020 IEEE 9th Glob. Conf. Consum. Electron., Kobe, Japan), 2020, pp. 403-404. 
  12. A. Laishram and K. Thongam, Detection and classification of dental pathologies using faster-RCNN in orthopantomogram radiography image, (2020 7th Int. Conf. Signal Process. Integr. Netw., Noida, India), 2020, pp. 423-428. 
  13. Y. Zhang, J. H. Han, Y. W. Kwon, and Y. S. Moon, A new architecture of feature pyramid network for object detection, (2020 IEEE 6th Int. Conf. Comput. Commun., Chengdu, China), 2020, pp. 1224-1228. 
  14. J. Wang, W. Yang, H. Guo, R. Zhang, and G. S. Xia, Tiny object detection in aerial images, (Proc. Int. Conf. Pattern Recognit., Milan, Italy), 2020, pp. 3791-3798. 
  15. Q. Ming, L. Miao, Z. Zhou, J. Song, and X. Yang, Sparse label assignment for oriented object detection in aerial images, Remote Sens. (Basel) 13 (2021), doi:10.3390/rs13142664 
  16. J. Yi, P. Wu, B. Liu, Q. Huang, H. Qu, and D. Metaxas, Oriented object detection in aerial images with box boundary-aware vectors, (Proc. 2021 IEEE Winter Conf. Appl. Comput. Vis.), 2021, pp. 2149-2158. 
  17. T. Ronneberger, O. Ronneberger, P. Fischer, and T. Brox, UNet: convolutional networks for biomedical image segmentation, IEEE Access 9 (2015), 16591-16603. 
  18. S. S. A. Zaidi, M. S. Ansari, A. Aslam, N. Kanwal, M. Asghar, and B. Lee, A survey of modern deep learning based object detection models, Digit. Signal Process. 126 (2021), 1-18. 
  19. Y. Wang, Y. Zhang, Y. Zhang, L. Zhao, X. Sun, and Z. Guo, SARD: towards scale-aware rotated object detection in aerial imagery, IEEE Access 7 (2019), 173855-173865.  https://doi.org/10.1109/ACCESS.2019.2956569
  20. B. Wang and Y. Gu, An improved FBPN-based detection network for vehicles in aerial images, Sensors 20 (2020), doi:10.3390/s20174709 
  21. S. B. Yoo and M. Han, Temporal matching prior network for vehicle license plate detection and recognition in videos, ETRI J 42 (2020), 411-419.  https://doi.org/10.4218/etrij.2019-0245
  22. T. S. Siadari, M. Han, and H. Yoon, Three-stream network with context convolution module for human-object interaction detection, ETRI J 42 (2020), 230-238.  https://doi.org/10.4218/etrij.2019-0230
  23. Z. Fang, J. Ren, H. Sun, S. Marshall, J. Han, and H. Zhao, SAFDet: A semi-anchor-free detector for effective detection of oriented objects in aerial images, Remote Sens. (Basel) 12 (2020), doi:10.3390/rs12193225 
  24. K. He and J. Sun, Deep residual learning for image recognition, (Proc. IEEE Conf. Comput. Vis. Pattern Recognit), 2016, pp. 770-778. 
  25. S. S. Saini and P. Rawat, Deep residual network for image recognition, (IEEE Int. Conf. Distrib. Comput. Electr. Circuits Electron., Ballari, India), 2022, pp. 6-9. 
  26. K. Simonyan and A. Zisserman, Very deep convolutional networks for large-scale image recognition, (3rd Int. Conf. Learn. Represent.), 2015, pp. 1-14. 
  27. S. Xie, R. Girshick, P. Dollar, Z. Tu, and K. He, Aggregated residual transformations for deep neural networks, (Proc. 30th IEEE Conf. Comput. Vis. Pattern Recognit.), 2017, pp. 1492-1500. 
  28. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich, Going deeper with convolutions, (Proc. IEEE Conf. Comput. Vis. Pattern Recognit), 2015, pp. 1-9. 
  29. U. Khusni, A. M. Arymurthy, and H. Susanto, Small object detection based on SSD-ResNeXt101 small object detection based on SSD-ResNeXt101, (Proc. 11th Int. Conf. Robot. Vis. Signal Process. Power Appl. Enhanc. Res. Innovat. Fourth Indus. Rev), 2022, pp. 1058-1064. 
  30. K. He, X. Zhang, S. Ren, and J. Sun, Identity mappings in deep residual networks, (14th Eur. Conf. Comput. Vis), Mar. 2016, pp. 630-645. 
  31. J. Ding, N. Xue, Y. Long, G. Xia, and Q. Lu, Learning RoI transformer for oriented object detection in aerial images, (Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit), 2019, pp. 2844-2853. 
  32. Y. Li, H. Mao, R. Liu, X. Pei, L. Jiao, and R. Shang, A lightweight keypoint-based oriented object detection of remote sensing images, Remote Sens. (Basel) 13 (2021), doi:10.3390/rs13132459 
  33. Z. Xiao, L. Qian, W. Shao, X. Tan, and K. Wang, Axis learning for orientated objects detection in aerial images, Remote Sens. (Basel) 12 (2020), doi:10.3390/rs12060908 
  34. X. Yang, J. Yan, Z. Feng, and T. He, R3Det: refined singlestage detector with feature refinement for rotating object, (Proc. 35th AAAI Conf. Artif. Intell.), 2021, pp. 3163-3171. 
  35. J. Yi, P. Wu, and D. N. Metaxas, ASSD: attentive single shot multibox detector, Comput. Vis. Image Underst 189 (2019), doi: 10.1016/j.cviu.2019.102827 
  36. X. Zhou, D. Wang, and P. Krahenbuhl, Objects as points, arXiv Preprint, 2019, arXiv:1904.07850. 
  37. Y. Xu, M. Fu, Q. Wang, Y. Wang, K. Chen, G. S. Xia, and X. Bai, Gliding vertex on the horizontal bounding box for multi-oriented object detection, IEEE Tran. Pattern Anal. Mach. Intell 43 (2021), 1452-1459.  https://doi.org/10.1109/TPAMI.2020.2974745
  38. W. Qian, X. Yang, S. Peng, J. Y. Member, and X. Zhang, RSDet++: point-based modulated loss for more accurate rotated object detection, IEEE Trans. Circuits Syst. Video Technol 32 (2022), 7869-7879.  https://doi.org/10.1109/TCSVT.2022.3186070
  39. L. Zhou, H. Wei, H. Li, W. Zhao, Y. Zhang, and Y. Zhang, Objects detection for remote sensing images based on polar coordinates, arXiv Preprint, 2020, arXiv:2001.02988. 
  40. L. Liao, X. Chen, J. Yang, S. Roth, M. Goesele, M. Y. Yang, and B. Rosenhahn, LR-CNN: local-aware region CNN for vehicle detection in aerial imagery, ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci 5 (2020), 381-388.  https://doi.org/10.5194/isprs-annals-V-2-2020-381-2020
  41. M. Mostofa, S. N. Ferdous, B. S. Riggan, and N. M. Nasrabadi, Joint-SRVDNet: joint super resolution and vehicle detection network, IEEE Access 8 (2020), 82306-82319.  https://doi.org/10.1109/ACCESS.2020.2990870
  42. J. Lu, T. Li, J. Ma, Z. Li, and H. Jia, SAR: single-stage anchorfree rotating object detection, IEEE Access 8 (2020), 205902-205912.  https://doi.org/10.1109/ACCESS.2020.3037350
  43. X. Yang, J. Yan, W. Liao, X. Yang, J. Tang, and T. He, SCRDet++: detecting small, cluttered and rotated objects via instancelevel feature denoising and rotation loss smoothing, IEEE Trans. Pattern Anal. Mach. Intell. 45 (2022), 2384-2399.  https://doi.org/10.1109/TPAMI.2022.3166956
  44. Q. Song, F. Yang, L. Yang, C. Liu, M. Hu, and L. Xia, Learning point-guided localization for detection in remote sensing images, IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens 14 (2021), 1084-1094.  https://doi.org/10.1109/JSTARS.2020.3036685
  45. Z. Liu, L. Yuan, L. Weng, and Y. Yang, A high resolution optical satellite image dataset for ship recognition and some new baselines, (Proc. 6th Int. Conf. Pattern Recognit. Appl. Methods, Porto, Portugal), 2017, pp. 324-331.